gregorut / vgchartzscrape Goto Github PK

Hi,

I found your Kaggle dataset very interesting. I've worked on this crawler fork to keep it up to date and operate with the latest version of the data. I also want to made a Pull Request when it will be finished.

I wanted to know if you thought about licensing the code, e.g. MIT? Since I'd like to apply it and respect your wishes about it.

Is it possible to apply MIT through generating a LICENSE.md file?

Greetings and thanks in advace for your reply.
Manuel

Good Afternoon! I recently tried rerunning the script for a class project but have been unsuccessful in rerunning the script. I have tried running in on Linux (multiple python versions and windows). I have no experience coding python and was able to figure out how to fix a couple of other errors before this one completely stumped me.

Traceback (most recent call last):
File "vgchartsfull2.py", line 29, in
for row in chart.find_all('tr')[1:]:
AttributeError: 'NoneType' object has no attribute 'find_all'

Any help will be greatly appreciated :)

Hi there,

I am very new to python, but I want to use this code to get a more current dataset than the one posted in Kaggle.

After the code has finished scraping, it gives this message:

Page: 1
Page: 2
Page: 3
Page: 4
Page: 5
Page: 6
Page: 7
Page: 8
Page: 9
Page: 10
Page: 11
Page: 12
Page: 13
Page: 14
Page: 15
Page: 16
Page: 17
Page: 18
0
Index(['Rank', 'Name', 'Platform', 'Year', 'Genre', 'Critic_Score',
'User_Score', 'Publisher', 'Developer', 'NA_Sales', 'PAL_Sales',
'JP_Sales', 'Other_Sales', 'Global_Sales'],
dtype='object')

Then when I open the csv it has written, all it has is the column headers but no data.

Any help would be much appreciated!!

Jules

Hi,

I saw the dataset over on Kaggle (https://www.kaggle.com/gregorut/videogamesales) and was hoping to use it for a college project. I wanted to get the most recent copy of the data, so I came here. I would appreciate if you could add a license to this repository (or the dataset on Kaggle if you would prefer) so that I can make use of it. https://choosealicense.com/no-permission/

Thank,
Philip

vgchartzfull.py:33: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 33 of the file vgchartzfull.py. To get rid of this warning, pass the additional argument 'features="html.parser"' to the BeautifulSoup constructor.

soup = BeautifulSoup(r)
Page: 1
1 Fetch data for game Wii Sports
2 Fetch data for game Super Mario Bros.
Traceback (most recent call last):
File "vgchartzfull.py", line 92, in
site_raw = urllib.request.urlopen(url_to_game).read()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 429: Rate

Hi,

I tried running the script (in Python 3) however I encountered an issue at line 39 (lambda x: x.attrs['href'].startswith('http://www.vgchartz.com/game/') ) which resulted in a KeyError 'href'.

Kindly advice what is needed to solve this?

Many thanks!