jchao01 / tradingview-data-scraper Goto Github PK

View Code? Open in Web Editor NEW

458.0 21.0 111.0 22 KB

Extract price and indicator data from TradingView charts to create ML datasets

License: Apache License 2.0

Python 100.00%

algorithmic-trading tradingview data-mining webscraping json

tradingview-data-scraper's Introduction

TradingView Chart Data Extractor

Video Tutorial

How-to screen recording: https://d.pr/v/VGCDNf

The resulting file from the tutorial above: https://d.pr/f/bnQ75v

Ensure that you zoom/pan such that the oldest date you desire is visible on TradingView before publishing the chart. Too many indicators or too low a time resolution will increase the data points and potentially overload the free server. Avoid this by hosting/running the script on your local machine or scraping multiple times with fewer indicators and manually combine the CSV afterwards.

Usage

Simply append the URL of a chart/idea published on TradingView to the link below. This is not the URL of a security's chart, but the URL for a user-published chart: https://tradingview-data.herokuapp.com/quotes?url=

i.e. for this chart: https://www.tradingview.com/chart/SPY/vjYfwgMu-SPY-Export-Test/

You'd use: https://tradingview-data.herokuapp.com/quotes?url=https://www.tradingview.com/chart/SPY/vjYfwgMu-SPY-Export-Test/

Install

pip3 install virtualenv
python3 -m venv .
source bin/activate
pip3 install -r requirements.txt
git init
heroku create
heroku git:remote -a projectname
heroku stack:set heroku-16
heroku buildpacks:add https://github.com/jontewks/puppeteer-heroku-buildpack.git
heroku buildpacks:add heroku/python
git add .
git commit -am 'fix'
git push heroku master

tradingview-data-scraper's People

Contributors

Stargazers

Watchers

Forkers

cruddyshad0w layzhi kenter1 tmorgan4 klovanone paulseperformance vitaizs slovke ramakrishnamekala129 krsoy karagul sjquant sahanduiuc jac0x sudarsan-sridharan philoteos afcarl erikhayton fagan2888 jpisfil10 rahulksom add1ct3d verb55nta auserj bugyip hdaoff wukuan405 maruu987654321 distroooy peduallex johnjdailey webclinic017 gregory-d84 mn3711698 dream-s-follower trumanhung datafields-team ilymak7 rahulmr ozgurtt richlysakowski vbartle chmohit25 rmfkdehd benwaldner chugh-ak5 betoxx1 gbeck-917 gokitau alenny811 quickpanda derinus salikin midnightfs gigabytegoob mqhub oiasam ahmedsakrr atlonxp 5l1v3r1 suvodeep-pyne albertrida bt ranpo rdutra7 lt-tl rahulsingh50 muharremokutan thanhen mohitnagarkotibca dg92 wolfholdingsinc lakithasahan ggardiakos pranay144 turnupdigital phungvankhanh metinogtem mikpim01 niloysaha keshavrdudhe szz1233 crypto-currency-projects python-repository-hub imsathiya17 noquee zoraezm arianeskandarii pawanmodi93 tamhankar55 munifico dst1213 cleancoindev alaatekleh daodewang ctksmethod ayushme001 sarangapa x48github01 liserom

tradingview-data-scraper's Issues

How do I install it on a local machine?

I'm running Windows, I'd like to run the code locally, how does it work?

I basically want to still be able to get the CSV using the Python code you listed but I don't know how to program.

Thank you in advance. Great tool!

Scrapper not longer works

Hi bro!
I've tried your links, and try to make my published chart% https://www.tradingview.com/chart/UKOIL/kiqWXRR6-teet-ukoil/
but have no success. Both times i got response from https://tradingview-data.herokuapp.com/quotes?url=https://www.tradingview.com/chart/UKOIL/kiqWXRR6-teet-ukoil/

{"error":"Protocol error Runtime.callFunctionOn: Target closed."}

Where is a problem? its heroku's issue or tradingview gus has changed something?

How to find a TradingView chart's address?

Hello and thanks for your great work!
I like to know is it possible to read data of this chart for example: http://www.tgju.org/chart/price_dollar_soleymani/trading with your library? How?

{"error":"Protocol error Runtime.callFunctionOn: Target closed."}

Dear Mr,

I should have installed everything correctly.

Unfortunately I get an error message when I use e.g. your link: https://tradingview-data.herokuapp.com/quotes?url=https://www.tradingview.com/chart/SPY/vjYfwgMu-SPY-Export-Test/

Same error appeared when I used your own link before I installed everything.

Thank you very much if you find the time to help me. If not, no problem. All the best!

Hi

Hey buddy,
Is it possible to extract a pine script logic from private indicators ?
it will be really apretiated if yo could or give me some pointers .
Thanks,
Ak

Date conversion problem

Hi,

I have found a problem in the following conversion:
moment.unix(item['value'][0] * 1000, utc=True)

The first parameter should be in sec instead of millisec.

Test url:
https://www.tradingview.com/chart/ADABTC/thKUZImc-test/

requirements file error

Hi,
I am taking this error,

Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'

Could you help me?
Thanks

More history?

I'm getting about ~800 data points for history.
Any possibly way to grab all points from a chart?
A way to use this on a chart rather then an idea?

{"error":"'NoneType' object has no attribute 'get_text'"} . error

Hello, I try to use you tool but I get this error every time
I'm using this URL : https://tradingview-data.herokuapp.com/quotes?url=https://www.tradingview.com/chart/BTCEUR/bEKjwimr-btceur/

any idea on what's goin on ?
thanks

Feature Request

I think most traders would love to have this feature.

As a user when I provide you a chart link, you should give me the most up to date data.

Manually, a user will click "Load new bars" button on the chart to achieve this

--single-process problem

Hi,

In case of using the '--single-process' argument I get an error:
--single-process is not supported in chrome multiple dll browser

Removing this argument the app works well.
Is it really necessary to use this arg?

Browser: Chrome
Version: 76.0.3809.100

"list index out of range" error

Great app! I've successfully downloaded your sample data but have been receiving a "list index out of range" error when trying do the same on an idea I've published myself. Can you help me on this? Many thanks

There's nothing here, yet.

everything is working through CLI as in the code is working, but when i add the url https://tradingview-data.herokuapp.com/quotes?url=https://in.tradingview.com/chart/BTCUSDT.P/xayXb8s2-6-months-OI-chart/
I get this error, it's so confusing as Heroku is completely up to date and up and running. Please help

(screen of the page after entering the complete URL)

and heres the output after running git push heroku master

`
remote: Procfile declares types -> web
remote:
remote: -----> Compressing...
remote: Done: 55.8M
remote: -----> Launching...
remote: Released v4
remote: https://murmuring-spire-93487-378ce2afb6cf.herokuapp.com/ deployed to Heroku
remote:
remote: Verifying deploy... done.
To https://git.heroku.com/murmuring-spire-93487.git

[new branch] master -> master
`

{"error":"File is not a zip file"}

Hi,

When I try to use this demo link, or my own private charts:
https://tradingview-data.herokuapp.com/quotes?url=https://www.tradingview.com/chart/SPY/vjYfwgMu-SPY-Export-Test/

I get the error:
{"error":"File is not a zip file"}

I have used this tool in the past without issues :/

Please help!

TradingView now allows all chart data to be exported in CSV format.

Just thought I'd let you know :) Custom indicators, plots, everything. Worked on strategies and studies for me (but only indicator, price data, etc. No strategy data unfortunately). Perhaps it'd be a better direction for this project to automate the process of getting the CSV from them instead of scraping it and running into issues.