Hi All,

I am new to python if anyone can support !
First on running this code i faced the issue of 'pwd' since i am running on windows not Linux and i followed the solution proposed in another ticket by replacing 'pwd with 'cd'

Then another error came of "list index out of range" as below:

in save_specification_to_file(self)
119 # This function save the devices specification to csv file.
120 def save_specification_to_file(self):
--> 121 phone_brand = self.crawl_phone_brands()
122 self.create_folder()
123 files_list = self.check_file_exists()

in crawl_phone_brands(self)
43 phones_brands = []
44 soup = self.crawl_html_page('makers.php3')
---> 45 table = soup.find_all('table')[0]
46 table_a = table.find_all('a')
47 for a in table_a:

IndexError: list index out of range

Is that because too many request? any idea? can anyone help solve that?

The script crashes when it encounters Unicode data.
Changing line 134 to with open(self.absolute_path + '/' + brand[0].title() + ".csv", "w",encoding='utf-8-sig',newline='') as file: will add Unicode support.

Please change the following line:
line 20: self.absolute_path = os.popen('pwd').read().strip() + '/' + self.new_folder_name # It create the absolute path of the GSMArenaDataset folder.

new code:
self.absolute_path = os.path.dirname(os.path.realpath(__file__)) + '/' + self.new_folder_name # It create the absolute path of the GSMArenaDataset folder.

I had to use self.absolute_path = os.getcwd() + '/' + self.new_folder_name at line 18 to get it working

Hey, I was trying to use the library and encountered the following error when I did a python gsmarena_scraping.py

  File "gsmarena_scraping.py", line 154, in <module>
    obj.save_specification_to_file()
  File "gsmarena_scraping.py", line 120, in save_specification_to_file
    phone_brand = self.crawl_phone_brands()
  File "gsmarena_scraping.py", line 44, in crawl_phone_brands
    table = soup.find_all('table')[0]
IndexError: list index out of range

I'm new to web scraping (and python in general), any pointers would be super helpful, thanks!

IP is blocked by the website after few hits. How to spoof the ip or solve this issue?

Hello,
This script scraping brands with ASC
for complete files need meny time

How we can scrap brand individual?

Just I need scrap Samsung but it starting with acer

Thank you

Traceback (most recent call last):
File "scrape.py", line 155, in
obj.save_specification_to_file()
File "scrape.py", line 131, in save_specification_to_file
datum = self.crawl_phones_models_specification(value, brand[0])
File "scrape.py", line 77, in crawl_phones_models_specification
model_name = soup.find(class_='specs-phone-name-title').text
AttributeError: 'NoneType' object has no attribute 'text'

GSMArenaDataset directory already exists
Traceback (most recent call last):
File "gsmarena_scraping.py", line 149, in
obj.save_specification_to_file()
File "gsmarena_scraping.py", line 121, in save_specification_to_file
files_list = self.check_file_exists()
File "gsmarena_scraping.py", line 115, in check_file_exists
return os.listdir(self.absolute_path)
FileNotFoundError: [WinError 3] Sistem belirtilen yolu bulamıyor: '/GSMArenaDataset'

We've launched an API that provides the easiest way for you to get the official technical specifications of every consumer electronics product in the world, just in a few seconds. Our database is updated on a daily basis so you always have access to the latest specs.

Read the docs https://techspecs.readme.io

Get early access https://developer.techspecs.io/