Scraper for Solivia Solar Inverters web site.
Clone the project.
git clone https://github.com/kinow/solivia-scraper.git
Create a dotEnv file
cd solivia-scraper
touch .env
Here's an example dotEnv file.
[email protected]
SOLIVIA_PASS=MyPassWord
SOLIVIA_INVERTERS=comma,separated,list,of,inverters
SOLIVIA_PLANTGUID=my-plant-gui-id
You also need to install the project dependencies
pip install -r requirements.txt
Finally, you can run the scrap.
python scraper.py --date YYYY-MM-DD
The output will be logged to your console, as in the follow example.
user@host$ python scraper.py --date 2016-08-09
2016-08-09 13:58:55,759 Starting Solivia scraper, for date 09/08/2016
2016-08-09 13:58:55,763 Starting new HTTPS connection (1): login.solar-inverter.com
2016-08-09 13:58:58,213 Starting new HTTPS connection (1): monitoring.solar-inverter.com
2016-08-09 13:59:07,054 Time,RPI M10A[1],,,
, AC Power (P1), AC Power (P2), AC Power (P3), Total Power
07:53,"0","0","0","0"
07:58,"0","0","0","0"
08:03,"0.015","0.009","0.022","0.046"
08:08,"0.033","0.023","0.037","0.093"
08:13,"0.042","0.033","0.045","0.12"
08:18,"0.052","0.042","0.052","0.146"
08:23,"0.059","0.05","0.061","0.17"
08:28,"0.065","0.056","0.066","0.187"
...
You can simply call the script with python scraper.py, or you can specify the date
you would like to download the data.
python scraper.py --date 2016-08-04
Where the format is YYYY-MM-DD (i.e. year with four digits, month with two digits, and finally day with two digits too).
The complete usage:
usage: scraper.py [-h] [--date DATE] [--from FROM_] [--to TO] --types TYPES
[--interval INTERVAL]
Solivia Monitoring scraper
optional arguments:
-h, --help show this help message and exit
--date DATE Date (YYYY-mm-dd)
--from FROM_ Date (YYYY-mm-dd)
--to TO Date (YYYY-mm-dd)
--types TYPES Comma separated types e.g. Power,Energy,AcParam,DcParam
--interval INTERVAL Being nice to servers, and waiting for an interval in
milliseconds before firing more requests (defaults to
300)
the --date parameter is exclusive to --to and --from. If --date is used, then
the others will be ignored
For every time you execute the script, you should see two files in your script directory. One CSV and one JSON file, named with the timestamp for the date used to retrieve data.
Examples:
- 20160809000000.csv
- 20160809000000.json
The names will match.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent