valaydave / arxiv-miner Goto Github PK

Integrate data about Comments, likes, etc. from social media (Twitter, Reddit) as a part of the data model.

Create scrapers and parsers that can help translate that to the data model so It can be index in Elasticsearch.

After running scraping , when i try to mine the papers , getting following error
python scripts/mine_papers.py --with-config default_config.ini start-miner

2021-07-07 11:24:42,091 - MiningProcess__pocgaxce - INFO - Mining Paper 2107.02381
2021-07-07 11:24:42,093 - MiningProcess__pocgaxce - ERROR - Failed Mining Paper : 2107.02381

Exception Reaching Arxiv API. PaperId 2107.02381 Not Found on FS

module 'arxiv' has no attribute 'query'
Computer Science Ontology and cached model loaded.
2021-07-07 11:24:44,682 - MiningProcess__pocgaxce - ERROR - Couldnt Mine Paper : 2107.02381

• Create Docs Website using Spinx : https://www.sphinx-doc.org/en/master/usage/quickstart.html

Figure methodology to parse equations and add the representations as a part of the data model.

Hi

I am trying to set up and explore this code. I found the pip install -r git+https://github.com/valayDave/arxiv-miner doesn't work . Also
sh cso_setup.sh giving the error
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'nltk'
Traceback (most recent call last):
File "", line 2, in
AttributeError: module 'classifier.classifier' has no attribute 'setup'

• Add an attribute to the TextFilterSearch object regarding the fuzzy search.
• Update DSL Configuration to add fuzzy search

Hi
Based on this reference . I am trying to Mine the epub books in the CS domain. Can you give some inputs , Have you come across similar thing ?

Some Very nice ideas on PDF parsing from Reddit commentor:

https://www.reddit.com/r/MachineLearning/comments/nn9q8s/p_arxivminer_a_toolkit_for_scraping_parsing_and/gzv089h?utm_source=share&utm_medium=web2x&context=3