adambom / dictionary Goto Github PK

View Code? Open in Web Editor NEW

668.0 28.0 173.0 32.08 MB

A JSON representation of Webster's Unabridged Dictionary

License: Other

Julia 100.00%

dictionary's Introduction

English Language Dictionary

This repository houses the contents of Webster's Unabridged English Dictionary.

The dictionary can be found in plain text form here

You'll also find some julia files that were used to parse the text and organize it into the nice json you see here.

dictionary.json: This is the raw data scraped from the dictionary. Unsurprisingly, it's in the format of a dictionary, i.e. { "Word": "Definition" }
graph.json: This is a graph representation of the dictionary. Each word is paired with a list of the words that define it
dictionary.txt: This is the plain text file (I converted it from ISO-8859-1 to UTF-8)
main.jl: The julia script that parses the data
_.jl: My in-progress implementation of underscore in julia

How to run

If you want to run the code yourself, I would recommend downloading Julia Studio, which is and IDE for Julia. It comes with the binaries pre-installed. You can also get the binaries or build from source from julialang.org.

License

The works in this repository are licensed under the MIT License, with the exception of the contents of dictionary.txt, which are licensed under the terms of the Project Gutenberg License:

From Project Gutenberg:

This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.net

dictionary's People

Contributors

Stargazers

Watchers

Forkers

awatson1978 isaacwhite moscova89 pwest kelwill james1912 evogler fireprogrammer35 sahilgupta jackphelps kazazes amrocha imclab ajwarnick asifaly swesh mnjstwins githubber vqzx mattcho aglorei suyashkumar2409 learn-usertoken ariel1segal syrianadus outboundexplorer usmanm77 achang0406 ernst-fanfan ddd-malay arccoxx nonbeing akashgutha panfayang wasv jk128 ferreiro wypul treble37 sevastos adster93 dazzlingdevelopment skytan4 jamesrp13 ermead patch-benjamin leedaniel84 mikeandike petestreet iamogunyinka markojak ddmng redreamality rizkywellyanto plr108-zz jparkerrogers kyungbaekim einarolafs redroma spencercurtis wktdev patrickridd commoncode openbiblesearch carvjust stiege calendee pengjoshua fredibach natehopkins repo-archives soldierforus getgimphed aworkingapp sayamkanwar pralgomathic dratcliff stormasm pi19404 wesleylhandy cat-of-devil sprax owsas excitoon-favorites chaichou chamuth donganhhanoi1110 ivan-parapanov iameaszy termermc tedcart joepfortunato jacksonrmc johnking douglasnaphas gunnar-o shermin antonioribeiro armatures vishalseshagiri

dictionary's Issues

Missing the word Tiger

Could not find the word tiger, although "tigerish" is present.

Line breaks not parsed correctly

The parser does not deal with newlines in the dictionary file correctly. If a line ends in a word, the white space seems to have been trimmed and words will be merged. See below.

ABATOR
A*ba"tor, n. (Law)

Defn: (a) One who abates a nuisance. (b) A person who, without right,
enters into a freehold on the death of the last possessor, before the
heir or devisee. Blackstone.

becomes

...the death of the last possessor, before theheir or devisee...

Dictionary.json is cut off

the dictionary file gets cut off at ...,"COVARIANT":"A function involving the coefficients and the variables of aquantic, and such that when the quantic is lineally transformed thes which obv. isn't valid JSON

Licensing

This project's license is CC-BY-NC, while the original Webster1913 project is in the public domain / under the Project Gutenberg license. Is the intent here to license the JSON data derived from Webster1913, or to license the tools associated?

And if it is the former, who should this work be attributed to? You, the contributors to the Webster1913 project on Project Gutenberg, the original authors, all of the above?

Some spaces are missing

Try searching in the json, txt or graph file for anythinginjurious as an example. Many spaces fails like that.

wordnet is important and missing

your json should provide the defintions and what type of word it is, adverb, adjective, noun, pronoun, verb... etc.

Maybe JSON should be something like this?

{
  "word": {
    "wordnet": "noun",
    "definitions": ["defo 1", "defo 2"]
  }
}

Weird characters and extra quotes in graph.json

Several entries in graph.json contain odd entries at the end like "[OBS" or "[R"

For example, near the beginning of the file:

"DEFIGURE":[
"TO",
"DELINEATE",
"[OBS",
"]THESE",
"TWO",
"STONES",
"AS",
"THEY",
"ARE",
"HERE",
"DEFIGURED",
"WEEVER"
]

Only "To delineate" makes up the definition of "Defigure", "[OBS" and "]THESE" contain weird characters, and "These two stones as they are here defigured --Weever" is a quotation, not a definition.

Getting ValueError while json.loads

This is the error I get.

ValueError: Invalid \uXXXX escape: line 1 column 11520 (char 11520)

Removing those I get these type of errors -

Expecting , delimiter: line 1 column 3998 (char 3998)