cristianpjensen / stock-market-prediction-via-google-trends Goto Github PK
View Code? Open in Web Editor NEWAttempt to predict future stock prices based on Google Trends data.
License: MIT License
Attempt to predict future stock prices based on Google Trends data.
License: MIT License
self-explanatory.
Credits, links to profiles, conclusion...
Happens sometimes when a suggestion has been clicked. The search term doesn't work anymore after that and the webpage has to be refreshed for the search box to be used again.
Determine which features are worth keeping and which aren't.
When a model has been made - with features - which can outperform the stock market using a buy-and-hold strategy.
This is an essential part of the machine learning workflow.
Self-explanatory.
The stock price data has to be pulled, so it can be fed to the neural network. However the weekends aren't incorporated in the stock price data, so it has to be manipulated so that they fit. Either the weekends have to be skipped, the weekends have to be indicated as NaN, or it has to be the same data has the Friday before (or work day before the NaN). Research has to be done on this subject.
When a viable solution to the weekend/holiday problem has been found, and incorporated.
The neural network has to be given an outcome of all the Google Trends data.
Use a cloud computing service (AWS, Google Cloud, Microsoft Azure ...) to find the best hyperparameters for the model.
When a model with > 0.65 accuracy has been found.
Hyperparameter tuning is a big part of machine learning and can make or break the algorithm. Good hyperparameters mean better accuracy, better accuracy means more money in this case.
Change data_adjuster and data_collector, so that they are one script, and not all files from Google Trends will be downloaded, only the adjusted daily data is outputted.
When one script does the job of both of these scripts.
This saves space on hard drives, it declutters, and it looks more clean.
My name is Luis, I'm a big-data machine-learning developer, I'm a fan of your work, and I usually check your updates.
I was afraid that my savings would be eaten by inflation. I have created a powerful tool that based on past technical patterns (volatility, moving averages, statistics, trends, candlesticks, support and resistance, stock index indicators).
All the ones you know (RSI, MACD, STOCH, Bolinger Bands, SMA, DEMARK, Japanese candlesticks, ichimoku, fibonacci, williansR, balance of power, murrey math, etc) and more than 200 others.
The tool creates prediction models of correct trading points (buy signal and sell signal, every stock is good traded in time and direction).
For this I have used big data tools like pandas python, stock market libraries like: tablib, TAcharts ,pandas_ta... For data collection and calculation.
And powerful machine-learning libraries such as: Sklearn.RandomForest , Sklearn.GradientBoosting, XGBoost, Google TensorFlow and Google TensorFlow LSTM.
With the models trained with the selection of the best technical indicators, the tool is able to predict trading points (where to buy, where to sell) and send real-time alerts to Telegram or Mail. The points are calculated based on the learning of the correct trading points of the last 2 years (including the change to bear market after the rate hike).
I think it could be useful to you, to improve, I would like to share it with you, and if you are interested in improving and collaborating I am also willing, and if not file it in the box.
On Google Chrome, initial-letter
is not an option (it is in safari). Thus a way of making sure the initial letter also works on Google Chrome will have to be figured out. This could be the fix.
When the drop caps works on all web browsers.
Because accessibility is important, and accessibility means all browsers.
Use only one keyword (stock market
) and create many types of features with that (Bollinger Bands, EMA, MA, etc.). Use stock market
, because what of what I can find out, it is the best search term that correlates with the stock market.
When good features for this problem have been found, implement, and visualised.
This will make for some great visualisations and I might be able to find out which feature(s) is the best for multiple keywords
Hello, thank you for this repository, i've been doing some research and all of your methodology was very helpful. But, I have a question, how can I set the API to get daily data without the geo parameter? I was trying and every time i get some error. Thanks for your time!
There has to be a method to how the data will be fed to a neural network. So that is what this issue has to solve. This issue has to answer some questions, like "How many weeks back will the model be fed?".
This issue will be considered done when a viable method of feeding data to the neural network has been made.
This is required, because the data the neural network gets fed, will also determine the accuracy of it.
There has to be a text under the graphs to explain what is being visualised.
There has to be the ability to compare the search terms to the stock price of ^DJI.
When the line for the stock price of ^DJI is also in the "Explore" graph.
The interesting part of the project is to compare the search terms to the stock price, since that is the point of the project.
Make it possible to download Google Trends data of keywords with more than 1 word.
Automatically delete the first two lines of the csv files on download.
To make it better integrate with a pandas Dataframe.
The main README is quite outdated, a lot more progress has been made since its creation. Thus an updated README should be made. Things to cover:
When a good-looking README has been made. It should be up to par with the Google documentation style.
This feature is required to get more people interested in the project. It helps others with understanding the project and the decisions made.
The data pulled from Google Trends has to be adjusted after the data has been pulled. The adjustments have to be made according to Method in the README.md.
This issue will be considered done when after the data from Google Trends has been pulled, it will be automatically adjusted, and exported to a .csv-file, as described in Method
This feature is required, because in order to feed data to the neural network, the structure of the data has to stay consistent.
Create a README.md for the data
-folder with multiple graphs containing data from the data in this folder. The plots should be made using seaborn. I will be making all plots for this project with seaborn so that all READMEs are consistent with each other.
When a README has been made. The graphs should be clear. There should be multiple plots (look in plot galleries for inspiration.
This is required because it shows people what the data looks like. They can of course also look at the deployment, but graphs in a README are more accessible.
Make sure that all information that should be in the documentation (readme), is present. Also write following the Google developer documentation style guide.
When all documentation, which is needed, is written according to the Google developer documentation style guide, and that anyone who is not necessarily an expert (but is interested), understands everything written in the documentation.
This feature is required, because the documentation is the first thing that people look at, and good-looking documentation inclines people to leave a star, or get engaged in the project.
The search box needs to only change the graph upon selecting a search term.
When after hitting either "enter" or pressing on a search term suggestion, the graph changes, and not in another way.
Because else the graph will be empty half the time, and it causes lag, because the graph keeps updating.
The documentation for the scripts is still documenting previous code. Thus it has to be updated to document the current code.
The deployment needs to be shown via GIFs. No one will open the index.html
in a live server. This is the easiest/best way to show it.
Utilise the powers of k-fold cross-validation in the machine learning model.
When K-fold cross-validation has been implemented.
All machine learning models that perform well are very overfitted (training accuracy = 1.0). This may help against that.
Combine features and figure out which features are the best for making predictions.
When the best possible - with the best possible features - machine learning algorithm has been found.
Better features mean a better algorithm. This could enforce the machine learning model and better its accuracy.
The last and the first of the daily data are the same days, however only the daily data files shouldn't end on the first, but on the last day of a particular month.
When - after export - the data doesn't overlap anymore.
Makes it easier to manipulate them as panda dataframes.
This makes it easier to import a big amount of keywords.
The scripts need documentation.
Make a neural network, which is configurable to the user's likings.
When a neural network model, which is configurable is made. This will most likely be made using TensorFlow.
Because one size doesn't fit all in neural networks. The model has to be configurable to account for this.
Links should be changing color to blue when hovered over, but they are remaining white in google chrome.
Various graphs could be added to this issue.
The text in the introduction is not selectable, probably because the graph (with opacity 0) is overlapping it.
Consistency, consistency, consistency...
A lot of " in make_dataset.py
in particular.
A feature, which is essentially just how many days the last peak was. Peaks can be found using the scipy.signals
library. Only major peaks should be used, not all the small little peaks (where there are hundreds of). For example, only the peaks preceding a stock crash. Define the characteristics of these peaks, so that the algorithm can start looking for the same kind of peaks in the future. This could definitely help with prediction.
Also, this feature could be one-hot encoded. For example:
Days since last peak
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | > 15 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Three criteria:
All features, which could help with better accuracy, is a feature that should be explored.
The suggestions made by the search box while typing need to 1. look nice, and 2. be clickable.
When above criteria are fulfilled.
This makes it easier and look nicer.
The pulling of the data from Google Trends has to be quick and automatic. Google Trends doesn't have an API, so it will have to be done from scratch.
This issue will be considered closed when a script is able to pull data from Google Trends; an unofficial API.
This feature is necessary, because if a user wants to use another search term for their instance, they would have to spend hours collecting all data. However, with this feature it would only require seconds/minutes. This would also provide a foundation to being able to collect new data from Google Trends when actively using this program.
Make all docstrings in make_dataset.py
comply with the Google Docstrings Style.
This is the convention that has been chosen for this project. It has already been implemented in build_features.py
.
Update the README.md with information on how the data_collector.py script works.
When a technically written paragraph has been added to the README.md about the data_collector.py script.
To improve the documentation.
Instead of using absolute values, relative values would be more valuable. Would also be easier to compute in the future to make predictions when deployed.
Because it is easier to retrieve after the fact. Don't need to normalize the future data points with the existing data points (google trends is quite annoying in this regard). But percentage changes are the same always. That the Google Trends data is relative doesn't affect percentage changes.
The opacity doesn't go to 1 when the page is being scrolled quickly.
Structure the project according to the cookiecutter data science template.
Because it makes it more structured and helps people navigate through the project.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.