Giter VIP home page Giter VIP logo

finmind / finmind Goto Github PK

View Code? Open in Web Editor NEW
2.0K 109.0 331.0 27.92 MB

Open Data, more than 50 financial data. 提供超過 50 個金融資料(台股為主),每天更新 https://finmind.github.io/

Home Page: http://finmindtrade.com/

License: Apache License 2.0

Python 23.97% Makefile 0.09% HTML 0.09% Jupyter Notebook 75.85%
python financial-data stock-prices gold-price api exchange-rates interest-rates crude-oil r hacktoberfest

finmind's Introduction

Build Status license Documentation Status Gitter PyPI version

Donate

Quickstart

pip install FinMind

這是什麼?

FinMind 是超過 50 種金融開源數據 50 datasets。 包含

  • 技術面 : 台股股價 daily、即時報價、歷史 tick、PER、PBR、每5秒委託成交統計、加權指數、當日沖銷交易標的及成交量值。
  • 基本面 : 綜合損益表、現金流量表、資產負債表、股利政策表、除權除息結果表、月營收。
  • 籌碼面 : 外資持股、股權分散表、融資融券、三大法人買賣、借券成交明細。
  • 消息面 : 台股相關新聞。
  • 衍生性商品 : 期貨、選擇權 daily data、即時報價、交易明細,選擇權、期貨三大法人買賣,期貨各卷商每日交易、選擇權各卷商每日交易。
  • 國際市場 : 美股股價 daily、minute、美國債券殖利率、貨幣發行量(美國)、黃金價格、原油價格、G8 央行利率、G8 匯率、

資料每天更新。你不需收集資料,就可進行分析。

What is this?

FinMind is open source of more than 50 datasets, including

Taiwan stock trade data daily, Taiwan stock trade data (5 seconds) (2019-05-29 ~ now, more than 30 million data in total), Financial Statements, Balance Sheet, Cash Flows Statement, Month Revenue, Holding Shares Per, Institutional Investors Buy Sell. Taiwan Futures Trade Detail, Taiwan Option Trade Detail.

US stock price daily, minute (2019-06-01 ~ now, more than 80 million data in total), oil price, gold price, G7 exchange rate, interest rate.

US Government Bonds Yield.

The datasets are automatically updated daily. You can analyze financial data without having to collect the data by yourself.


License

  • License Detail

  • 本專案提供的所有內容均用於教育、非商業用途。資料僅供參考,使用者依本資料交易發生交易損失需自行負責,本專案不對資料內容錯誤、更新延誤或傳輸中斷負任何責任。

Project of Contents

Plotting

# 取得股價
from FinMind.data import DataLoader

dl = DataLoader()
# 下載台股股價資料
stock_data = dl.taiwan_stock_daily(
    stock_id='2609', start_date='2018-01-01', end_date='2021-06-26'
)
# 下載三大法人資料
stock_data = dl.feature.add_kline_institutional_investors(
    stock_data
) 
# 下載融資券資料
stock_data = dl.feature.add_kline_margin_purchase_short_sale(
    stock_data
)

# 繪製k線圖
from FinMind import plotting

plotting.kline(stock_data)

gif

dashboard

png

Contact

Email: [email protected]

每週日早上零點至早上七點為維護時間,不提供服務。

Note

  • 未來預計新增更多功能,包含個人化回測分析、chatbot 監控策略,
  • API Request 上限:300 / 小時。
  • Limit amount of request, 300 / hour.
  • FinMind官網註冊並驗證信箱後,API 的 Request 加上 token 參數可以提高使用上限到 600/hr。
  • token 獲取方法,可在官網登入後獲取。

finmind's People

Contributors

featasy avatar hms5232 avatar jcs090218 avatar linsamtw avatar machinecyc avatar penut85420 avatar sakkyoi avatar stitetsati avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

finmind's Issues

[請益] 爬蟲相關

你好,

我看程式的部分,資料是從 DB 倒出來處理
因此,想請教一下,爬蟲代碼的部分是否也有開源?

另外,DB 是自己 own 的嗎?
如果是,要如何評估,若有大量用戶 access 你們的資料,而 DB 依然保持穩健?

PS 因為我自己也有寫類似的東西,但是開發得很卡,爬蟲經常被阻擋。所以想找其他人學習 XD

Type Error

-- coding: utf-8 --

@author: boyac

@Date: 2019-01-18 14:17:56

@last Modified by: boyac

@last Modified time: 2019-02-21 17:38:57

from FinMind.Data import Load
import numpy as np
from keras.layers.core import Dense, Dropout
from keras.layers import LSTM , GRU
from keras.models import Sequential
from sklearn.preprocessing import MinMaxScaler
import matplotlib.pyplot as plt
import math
from keras.optimizers import Adam

print('input data')
data = Load.FinData(
dataset = 'TaiwanStockPrice',
select = ['2330.TW'],
date = '2010-01-01')

#colname = ['date', 'open', 'high', 'low', 'close', 'volume']
#data = data[colname]
print('select close price')
stock_price = data['Close'].values.astype('float32')
stock_price = stock_price.reshape(len(stock_price), 1)
print(' 畫圖 ')
plt.plot(stock_price)
plt.show()

print(' 歸一化 ')
scaler = MinMaxScaler(feature_range=(0, 1))
stock_price = scaler.fit_transform(stock_price)
print(' 取 80% data 當作 training data, 20% data 當作 testing data 做模型驗證 ')
train_size = int(len(stock_price) * 0.8)
test_size = len(stock_price) - train_size
train, test = stock_price[0:train_size,:], stock_price[train_size:len(stock_price),:]

print('切 data,拿前五天的股價,預測未來1天的股價,先做個 demo')
def process_data(data , n_features,future_days):
dataX, dataY = [], []
for i in range(len(data)-n_features-future_days):
a = data[i:(i+n_features), 0]
dataX.append(a)
dataY.append(data[i + n_features-1+future_days, 0])
return np.array(dataX), np.array(dataY)

reshape into X=t and Y=t+5

n_features = 5
future_days = 1

trainX, trainY = process_data(train, n_features,future_days)
testX, testY = process_data(test, n_features,future_days)
print(trainX.shape , trainY.shape , testX.shape , testY.shape)

lstm need input to have 3 dimensions

print('轉換成 LSTM 建模所需 data 的型態')
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

print('設定參數')
filepath="stock_weights.hdf5"
from keras.callbacks import ReduceLROnPlateau , ModelCheckpoint
lr_reduce = ReduceLROnPlateau(monitor='val_loss', factor=0.1, epsilon=0.0001, patience=1, verbose=1)
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='max')

print('建立 DL 模型,使用 RNN 常見的 GRU and LSTM')
model = Sequential()
model.add(GRU(256 , input_shape = (1 , n_features), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(2048))
model.add(Dropout(0.2))
model.add(Dense(64 , activation = 'relu'))
model.add(Dense(1))

#model.load_weights('/home/linsam/job/stock_weight.h5')
model.compile(loss='mean_squared_error', optimizer=Adam(lr = 0.0005) , metrics = ['mean_squared_error'])
print('training')
history = model.fit(trainX, trainY, epochs=5 , batch_size = 128 ,
callbacks = [checkpoint , lr_reduce] , validation_data = (testX,testY))
print(' save weight ')
model.save_weights('stock_weight.h5')

print('計算 error')
def model_score(model, X_train, y_train, X_test, y_test):
trainScore = model.evaluate(X_train, y_train, verbose=0)
print('Train Score: %.5f MSE (%.2f RMSE)' % (trainScore[0], math.sqrt(trainScore[0])))
testScore = model.evaluate(X_test, y_test, verbose=0)
print('Test Score: %.5f MSE (%.2f RMSE)' % (testScore[0], math.sqrt(testScore[0])))
return trainScore[0], testScore[0]

tem = model_score(model, trainX, trainY , testX, testY)

pred = model.predict(testX)
pred = scaler.inverse_transform(pred)
print('\n Actual Stock Prices')
print( pred[:10] )

testY2 = testY.reshape(testY.shape[0] , 1)
testY2 = scaler.inverse_transform(testY2)
print('\n Predicted Stock Prices ')
print( testY2[:10] )

print('畫圖')
print("\nRed - Predicted Stock Prices, Blue - Actual Stock Prices")
plt.rcParams["figure.figsize"] = (15,7)
plt.plot(testY2 , 'b')
plt.plot(pred , 'r')
plt.xlabel('Time')
plt.ylabel('Stock Prices')
plt.title('Red - Predicted Stock Prices, Blue - Actual Stock Prices')
plt.grid(True)
plt.show()

2012-01-07 TaiwanStockPrice股價過高

您好,這天的資料似乎有誤,不只2330,0050,0056都有類似的情況。
查了一下日曆那天是禮拜六,可能是多出來的資料?


library(httr)
library(data.table)
url = 'http://api.finmindtrade.com/api/v2/data'
response = httr::GET(url = url,
                     query = list(
                             dataset="TaiwanStockPrice",
                             stock_id= "2330",
                             end_date= "2020-04-08"
                     )
)
data = content(response)
df = data.table(matrix(unlist(data$data), 
                       nrow=length(unlist(data$data[1]))
))
colnames(df) = names(data$data)
df[date>= '2012-01-01' & date<='2012-01-10' ]

pic

Can't fetch complete TPEX data.

Here is a simple code to load TPEX data.

from FinMind.Data import Load
Load.FinData(dataset = 'TaiwanStockPrice', select = '1240', date = '2020-1-1')

The return does not contain 3/4, 3/5, 3/10, 3/12 data.
By the way, I do not check all the dates.

Finally, TWSE stock works pretty well.

from FinMind.Data import Load
Load.FinData(dataset = 'TaiwanStockPrice', select = '1101', date = '2020-1-1')

新增 FinancialStatements 資料欄位

金融股 2891, 5880

  • 所得稅費用(利益)[營業外收益(損失)]
  • 繼續營業單位稅前損益 [營業利益, 稅前淨利]
  • 其他利息以外淨損益 [其他利益及損失]

請問API可以提供POST的嗎

很久以前API是POST的,後來以為壞了,最近才發現是換URL了,但只能用GET,我是寫JAVA的,大部的library的GET的interface沒辦法帶request body,滿困擾的,不然就只能用URL?dataset=TaiwanStockPrice&stock_id=2317&date=2018-06-01&end_date=2020-04-12

TX futures all day data?

經過測試stock id 打 TX
可以拉到大台的每秒資料,但是是不是沒辦法得到台指期夜盤的資料呢

取得股價時 出現InternalError

不好意思,
我在爬股價時,出現InternalError: (1049, "Unknown database 'TaiwanStockPrice'")
想請問是否為在伺服器上出現問題呢~?

Trading_Volume and Trading_money are confusing.

我想這並不是一個新的問題,正如同 #28所提,FinMind所提供的資料偶爾會出現"張數" & "股數"混亂出現的狀況。而且去年八月就存在至今,我會這麼說是因為我追朔部分過往從FinMind下載的資料。

這是一個有點嚴重的問題,因為若不是這次download的第一筆資料就出現異常與現在有API Request 上限,我甚至不會特別驗證資料的正確性。如果我當初存data是用append,甚至不可能發現異常。

而且看起來FinMind的修正似乎是重新將爬蟲跑一遍後直接覆蓋掉舊資料?
資料庫應該必須、有維護驗證資料的必要

至於股票的代號先提供一個5284
我粗略地翻了約200隻股票,大約找到5~7隻有這個問題的股票。
肯定存在更多,因此我希望持續開著這個issue,直到做出完善的驗證機制為止。

台灣股票清單不同index 內容是同樣的stock id

idx=17736 id=6552 type=twse industry_category=半導體類
Grab BS table: id=6552 type=twse

idx=17737 id=6552 type=twse industry_category=電子工業
Grab BS table: id=6552 type=twse

idx=17738 id=6558 type=twse industry_category=其他電子業
Grab BS table: id=6558 type=twse

idx=17739 id=6558 type=twse industry_category=其他電子類
Grab BS table: id=6558 type=twse

idx=17740 id=6558 type=twse industry_category=電子工業
Grab BS table: id=6558 type=twse

取到的表可以看到不同的index 卻是同樣的id

database TaiwanStockPrice disappear

Does anyone have this error?

pymysql.err.InternalError: (1049, "Unknown database 'TaiwanStockPrice'")

When I run below code

from FinMind.Data import Load
TaiwanStockInfo = Load.FinData(dataset = 'TaiwanStockInfo')

請問TaiwanStockPER特定日期取不到資料

library(httr)
library(data.table)
library(dplyr)
url = 'https://api.finmindtrade.com/api/v3/data'
response = httr::GET(url = url,
                     query = list(
                             dataset="TaiwanStockPER",
                             stock_id= "2834",
                             date= "2010-01-01"
                     )
)

image

日期改到2011-01-01便可拿到資料,有辦法知道各股的最早日期嗎?
image

謝謝!

買賣日報

看了一下好像沒有上市個股買賣日報的資料
請問會開發嗎?

謝謝
不好意思打擾

[API 異常] 台股 籌碼面 TaiwanStockHoldingSharesPer

您好

近期抓取台股 "籌碼面" 相關資料時,經常出現 500 status code
以 TaiwanStockHoldingSharesPer 為例:
1256, 1592, 2546, 5284 都有這樣的狀況
(目前只發現這幾個,前陣子測試 TaiwanStockMarginPurchaseShortSale 也有類似狀況)

個人測試時發現,若時間區間拉短一點,則會正常返回資料
全抓的話則會出狀況
image

想請教一下,
是 API 服務有異常狀況,還是 API 服務政策有更新之類的
抑或是我忽略了甚麼 API 呼叫的基本規範?

煩請提供協助
感謝您

Trading_Volume單位不一致

您好
4171的成交量單位不一致
image
前陣子是張 後來突然改用股(後面多3個0) 連帶影響Trading_money

StockDividend dataset have wrong data type

部分欄位型態錯誤,reproduce code 如下

import requests
import pandas as pd
url = "http://api.finmindtrade.com/api/v2/data"
parameter = {
    "dataset": "StockDividend",
    "stock_id": "2330",
    "date": "2019-01-01",
}
data = requests.get(url, params=parameter)
data = data.json()
data = pd.DataFrame(data['data'])

print(data['StockEarningsDistribution'].unique())

下列欄位可能都會有此問題

StockEarningsDistribution                6 non-null object
StockStatutorySurplus                    6 non-null object
TotalEmployeeStockDividend               6 non-null object
TotalEmployeeStockDividendAmount         6 non-null object
RatioOfEmployeeStockDividendOfTotal      6 non-null object
RatioOfEmployeeStockDividend             6 non-null object
CashStatutorySurplus                     6 non-null object
TotalEmployeeCashDividend                6 non-null object
TotalNumberOfCashCapitalIncrease         6 non-null object
CashIncreaseSubscriptionRate             6 non-null object
CashIncreaseSubscriptionpRrice           6 non-null object
RemunerationOfDirectorsAndSupervisors    6 non-null object

dataset/TaiwanStockInfo 資料都有兩筆?

想請問一下為何股票的資料都會有兩筆? 以下是其中一小段

        {
            "industry_category": "電腦及週邊設備業",
            "stock_id": "2341",
            "stock_name": "英群",
            "type": "twse"
        },
        {
            "industry_category": "半導體業",
            "stock_id": "2342",
            "stock_name": "茂矽",
            "type": "twse"
        },
        {
            "industry_category": "電子工業",
            "stock_id": "2342",
            "stock_name": "茂矽",
            "type": "twse"
        },
        {
            "industry_category": "半導體業",
            "stock_id": "2344",
            "stock_name": "華邦電",
            "type": "twse"
        },
        {
            "industry_category": "電子工業",
            "stock_id": "2344",
            "stock_name": "華邦電",
            "type": "twse"
        },

TaiwanStockPrice column "spread" looks weird

2020-06-06 133231

原先我以為TaiwanStockPrice的spread欄位是指與前日收盤的差,但是沒想到現在重新download絕大多數的spread都被改為-1.0,請問spread到底代表什麼意義? 更令我困惑的是,有部分的spread依舊維持原有的data。

TypeError - Ticker, stock_id

Hi, there's a TypeError. However, it's not quite clear where should I input the stock_id. Thanks

super(ClassTaiwanStockPrice, self).init(TABLE,'stock_id')
TypeError: super() argument 1 must be type, not classobj

個股單日成交量資訊

您好,

想請教一下,是否有提供單日成交量資訊?
還是這個值可以從其他支 API 回傳結果計算得出?

煩請提供指教

感謝

Internal Server Error

I try to use postman to test 2 samples in your readme.
It showing Internal Server Error for both URI samples.

cors problem

跨網域請求會失敗,因為 options method 回傳 405

似乎沒有當日的即時資料?

感謝大大開放這麼方便的API讓大家使用

在看到文件教學時試著操作以下部分取得今日5/8的股票資訊

form_data = {'dataset':'TaiwanStockPrice','stock_id':'2330','date':'2020-05-08'}
res = requests.post(url,verify = True,data = form_data)

temp = res.json()
data = pd.DataFrame(temp['data'])
data.head()

但是卻沒有跑出今日5/8的資料? 請問是通常會延遲一天嗎

謝謝

Taiwan Futures中的TX資料不見了

你好
今天在使用TaiwanFutures抓取stock id為TX時發現抓去不到資料
使用translate_url檢查後發現似乎只剩下個股期貨的代號,而TX跟MTX的部分不見了
請問這部分是有更動嗎?

大盤指數

請問是不是沒有各國大盤指數的資料呢?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.