The filtering process contains three steps:
- Preprocessing
- Summarization
- Dependency parsing
Description:
scripts
: Code for building the knowledge graph- Crawl from Reddit
- Filter the submissions and comments of Reddit
- Filter ED, and then merge with Reddit
- Cluster
data
: All the data for building the knowledge graphed
: First and second turns of ED after filteringreddit
: Reddit submissions and commentsraw
: Raw data crawled using PushShift APIsfiltered
: Submissions and comments after filteringmatched
: Submissions and comments matched into singlecsv
files
merged
: Reddit and ED merged
plot
: Some visualizations
Due to the file size limitation of GitHub, some files are stored on Google Drive:
data/merged/all_events_embed.npy
data/merged/all_replies_embed.npy
data/reddit/raw/*