Hi @edtechre,

Since pyb support talib indicators, I tried Bollinger Bands like:

bb = pybroker.indicator('bb', lambda data: talib.BBANDS(data.close))
bb(df)

I got this error:

ValueError: Length of values (3) does not match length of index (505)

I think the reason is BBANDS return 3 outputs.

help(talib.BBANDS)
Help on function BBANDS in module talib._ta_lib:
BBANDS(...)
    BBANDS(real[, timeperiod=?, nbdevup=?, nbdevdn=?, matype=?])
    
    Bollinger Bands (Overlap Studies)
    
    Inputs:
        real: (any ndarray)
    Parameters:
        timeperiod: 5
        nbdevup: 2
        nbdevdn: 2
        matype: 0 (Simple Moving Average)
    Outputs:
        upperband
        middleband
        lowerband

I figured i will share some thoughts i have after trying PyBroker a bit.

Writing indicators could be more convenient i think. For example, in freqtrade it works like this:
Bot calls populate_indicators() that we implement and passes entire dataframe to it. There we can do things like this:

df['spread'] = df['high'] - df['low']
df['spread_sma20'] = ta.SMA(df['spread'], 20)
df['spread_sma40'] = ta.SMA(df['spread'], 40)

This looks trivial on surface and of course is nothing PyBroker can not do, but actually this is very powerful.

To achieve something like this in PyBroker we would have to create a custom indicator functions for spread_sma20 and spread_sma40. But here we waste calculation of the spread column as it is done twice now.

It also is rather cumbersome to use indicator libraries like lib-ta or pandas_ta. These libraries already provide one-func-call indicators that we now must wrap in another function to acquaint them with PyBroker.

Normally i would just say "whatever, ill do it on symbol dataframe", however datasource.query() merges all symbols into one dataframe and thats the only place where it seems to make sense to insert custom indicators for backtesting.

What would be convenient

First of all it seems to me it would make more sense if data_source.query() returned a list of dataframes instead of one dataframe with all symbols. This dataframe need to be split anyway, besides merging dataframes of different symbols puts a burden on the user to make sure that dataframes of all queried symbols are of equal length and user must properly merge them in case there are missing candles. If everyone has to do it - might as well do it in the library.

Then, if dataframes were separate, we could also have a user-implemented indicators_fn(df) in the same spirit as exec_fn, which would allow massaging dataframe in any way we see necessary and utilizing all power of pandas.

This approach should be future-proof as well as adding support for multiple timeframes could be implemented as specifying indicator_fn for each timeframe. It should play well into live trading as well, since indicator_fn could be called once every new bar comes in.

Multi-symbol indicators

There is one special case where my proposed approach is not good enough: pairs trading. We need price data of two symbols in order to calculate necessary metrics. Maybe a way to get raw (just OHLC data, no indicators) symbol dataframe in indicator_fn could be an option. On same accord order entry for pairs trading is also bit unintuitive as entire process is split over two execute_fn iterations, but thats another topic..

Anyhow, by no means a request, just some food for thought and discussions. My proposition may have shortcomings that are unobvious to me.

does pybroker support dynamic universe. for example, sp500 universe varies in different years, can this be support in pybroker?

Hi edteche! I have a question as the issue title. hope for solve it.

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

alpaca-py 0.12.0 requires pandas<3.0.0,>=2.0.0, but you'll have pandas 1.5.3 which is incompatible.

alpaca-py v0.12.0 depends on pandas = "^2.0.0"
while pybroker v1.1.29 depends on pandas>=1.5.1

When using result = strategy.backtest(calc_bootstrap=True, train_size=0.5) I often see the result.metrics.profit_factor being lower than result.bootstrap.profit_factor.low_2p5. I see it more often than one would expect from a 2.5% chance.

I have two questions:

1. Can you confirm this (without a minimal reproducing example)?
2. Is boostrapping in combination with the train_size parameter even supported?

Hi edtechre,

I've been using backtesting.py for a while now and I think one of its nice features is its mapping feature, which automatically opens the browser after each backtesting and shows an interactive chart, so it's easy to see where the current strategy needs further optimization. You can see such interactive charts on their website (https://kernc.github.io/backtesting.py), is pybroker planning to make such charts as well?

Hello,

I currently have some questions about the warmup property in the Strategy.backtest(...) method. My goal is to set the propper warmup period that is required to initialize all my indicators and to start trading at a specific date.

So let's calculate a ROC Indicator with the lenght 5 and start trading at the first of January.

    warmup: int = 5
    roc = pybroker.indicator('roc', lambda data: pandas_ta.roc(Series(data.close), length=warmup))

    start_date: datetime = datetime(2023, 1, 1, tzinfo=pytz.timezone('America/New_York'))
    end_date: datetime = datetime(2023, 2, 1, tzinfo=pytz.timezone('America/New_York'))

    strategy: Strategy = Strategy(
        Alpaca(os.getenv('ALPACA_KEY_ID'), os.getenv('ALPACA_SECRET')),
        start_date, end_date,
        StrategyConfig(initial_cash=10000, exit_on_last_bar=True)
    )
    strategy.set_before_exec(before_exec)
    strategy.add_execution(exec_fn, ['IYY', 'IWM', 'IVV'], indicators=[roc])

    result: TestResult = strategy.backtest(timeframe='1d', warmup=warmup)

So we need 5 periods to get the ROC(5) indicator calculated. This takes until the 2023-01-10 and the first trade is opened on the 2023-01-11:

date                              cash           equity           margin        market_value   pnl     unrealized_pnl  fees                                                                                         
2023-01-03 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-04 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-05 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-06 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-09 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-10 05:00:00  10000.00  10000.00      0.00                     10000.00    0.00   0.0                       0.0
2023-01-11 05:00:00        30.70  10082.50      0.00                     10082.50    82.50  0.0                      0.0

My original intention was to use 'Strategy.backtest(start_date, end_date, ...)' to start trading at the 2023-01-01 and to use Pyhtons 'timedelta(days=warmup) to shift the data fetching so the warmup period is substracted from the start_date.
In this way we could start trading exactly at the 2023-01-01 (where stock exchanges closed so it was in reality the 2023-01-03) .

    warmup: int = 5
    roc = pybroker.indicator('roc', lambda data: pandas_ta.roc(Series(data.close), length=warmup))

    start_date: datetime = datetime(2023, 1, 1, tzinfo=pytz.timezone('America/New_York'))
    end_date: datetime = datetime(2023, 2, 1, tzinfo=pytz.timezone('America/New_York'))

    strategy: Strategy = Strategy(
        Alpaca(os.getenv('ALPACA_KEY_ID'), os.getenv('ALPACA_SECRET')),
        start_date - timedelta(days=warmup), end_date,
        StrategyConfig(initial_cash=10000, exit_on_last_bar=True)
    )
    strategy.set_before_exec(before_exec)
    strategy.add_execution(exec_fn, ['IYY', 'IWM', 'IVV'], indicators=[roc])

    result: TestResult = strategy.backtest(start_date, end_date, timeframe='1d', warmup=warmup)

But doing so this exception is thown:

Traceback (most recent call last):
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 582, in _validate_comparison_value
    self._check_compatible_with(other)
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 461, in _check_compatible_with
    self._assert_tzawareness_compat(other)
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 694, in _assert_tzawareness_compat
    raise TypeError(
TypeError: Cannot compare tz-naive and tz-aware datetime-like objects.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 1054, in _cmp_method
    other = self._validate_comparison_value(other)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 585, in _validate_comparison_value
    raise InvalidComparison(other) from err
pandas.core.arrays.datetimelike.InvalidComparison: 2023-01-01 00:00:00-04:56

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pirat/PycharmProjects/pybroker-experiments/main.py", line 120, in <module>
    main()
  File "/home/pirat/PycharmProjects/pybroker-experiments/main.py", line 110, in main
    result: TestResult = strategy.backtest(start_date, end_date, timeframe='1d', warmup=warmup)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1060, in backtest
    return self.walkforward(
           ^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1179, in walkforward
    df = self._filter_dates(
         ^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1340, in _filter_dates
    df = _between(df, start_date, end_date).reset_index(drop=True)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 79, in _between
    (df[DataCol.DATE.value].dt.tz_localize(None) >= start_date)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/common.py", line 72, in new_method
    return method(self, other)
           ^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arraylike.py", line 62, in __ge__
    return self._cmp_method(other, operator.ge)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/series.py", line 6243, in _cmp_method
    res_values = ops.comparison_op(lvalues, rvalues, op)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/array_ops.py", line 273, in comparison_op
    res_values = op(lvalues, rvalues)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/common.py", line 72, in new_method
    return method(self, other)
           ^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arraylike.py", line 62, in __ge__
    return self._cmp_method(other, operator.ge)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 1056, in _cmp_method
    return invalid_comparison(self, other, op)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pandas/core/ops/invalid.py", line 36, in invalid_comparison
    raise TypeError(f"Invalid comparison between dtype={left.dtype} and {typ}")
TypeError: Invalid comparison between dtype=datetime64[ns] and datetime

So I came across an other idea to move the backtest start_date into future remove the warmup parameter, but this resulted into the same exception:

    result: TestResult = strategy.backtest(start_date + timedelta(days=5), end_date, timeframe='1d')

So I came to the conclusion that Strategy start_date and end_date should not be different then Strategy.backtest(start_date, end_date) and therefore I dont understand why this redundancy is kept?

So idealy I would like to simply fetch i.e 1 month more data then required from the start_date from Alpaca and then start backtesting at exaclty the start_date (or the next candle). Is it possible to achive this?

Hi there! I tried to run a sample code and got this error: "RuntimeError: Native library not available at ... /Users/usr/opt/anaconda3/envs/pybroker39/lib/python3.9/site-packages/py_mini_racer/libmini_racer.dylib

Anyone had the same issue before?

Add a date check where it validates start_date <= end_date always

Hi @edtechre,

I have currently some issues with timezones. Especially between_time has unexpected behavior. So normally I integrate all my software systems in UTC timezone, since it is everywhere the same and there are no issues with daylight saving time and so on.

When integrating trading systems with other systems it can be also an option to use the US /Eastern timezone. So I have taken a look at pybroker.

Here is what happens (in my opinion):
I can assrue that start_date and end_date are set up to use the US/Eastern timezone when I create them.

• When creating Alpacas StockBarsRequest https://github.com/edtechre/pybroker/blob/master/src/pybroker/data.py#L357 both dates are converted to UTC which is fine
  

• After the data is fetched from Alpaca the DataFrame is in UTC which is fine:
  

• https://github.com/edtechre/pybroker/blob/master/src/pybroker/data.py#L386 converts the DataFrame to US/Estern time which is not ideal for me but it works good enough.
  

• Then the data is fiterded: https://github.com/edtechre/pybroker/blob/master/src/pybroker/strategy.py#L1179

• After executing: https://github.com/edtechre/pybroker/blob/master/src/pybroker/strategy.py#L1346 the timezone is None and the between_time filtering is applied so I wanted to filter it by (9:30,16:00) US/Estern.
  

• But since the date column in the DataFrame does not have any timezone any more my DataFrame is broken after this method:
  

So I will try to filter it with time in UTC timezone or None (which is supprisingly my local timezone)

I would strongly encourage using time with timezone instate of string tuples in between_time. Maybe this thing can be done easier if between_time is a tuple of time objects?

Hi, there. It's my first time to try pybroker. And when I test a script, I just met the error "ValueError: DataSource is empty." Here are my test script:
import pybroker
from pybroker import Strategy, StrategyConfig, ExecContext
from pybroker.ext.data import AKShare

config = StrategyConfig(initial_cash=500_000)

strategy = Strategy(data_source=AKShare(), start_date='4/6/2022', end_date='11/24/2023', config=config)

def buy_low(ctx):
# If shares were already purchased and are currently being held, then return.
if ctx.long_pos():
return
# If the latest close price is less than the previous day's low price,
# then place a buy order.
if ctx.bars >= 2 and ctx.close[-1] < ctx.low[-2]:
# Buy a number of shares that is equal to 25% the portfolio.
ctx.buy_shares = ctx.calc_target_shares(0.25)
# Set the limit price of the order.
ctx.buy_limit_price = ctx.close[-1] - 0.01
# Hold the position for 3 bars before liquidating (in this case, 3 days).
ctx.hold_bars = 3

strategy.add_execution(buy_low, symbols=['sh113055'])

def short_high(ctx):
# If shares were already shorted then return.
if ctx.short_pos():
return
# If the latest close price is more than the previous day's high price,
# then place a sell order.
if ctx.bars >= 2 and ctx.close[-1] > ctx.high[-2]:
# Short 100 shares.
ctx.sell_shares = 100
# Cover the shares after 2 bars (in this case, 2 days).
ctx.hold_bars = 2

strategy.add_execution(short_high, symbols=['sh110079'])

result = strategy.backtest()"

Hi @edtechre,

I noticed a weirdness about some number in result.trades.

Steps to reproduce:

1. I made the following settings:

config = StrategyConfig(
    initial_cash=100000000,
    fee_mode=FeeMode.ORDER_PERCENT,
    fee_amount=0.0,
    bars_per_year=252
)

2. During trading:

# to buy
ctx.buy_shares = ctx.calc_target_shares(1)
ctx.buy_fill_price = PriceType.OPEN

# to sell
ctx.sell_all_shares()
ctx.sell_fill_price = PriceType.OPEN

3. backtesting...

The first two rows of "result.trades" look like this:

The first shares value is 112694 rather than 100000000 / 887.064(entry price) = 112731.

Personally, I think 112731 is right.

And even shares value of 112694 were correct, shouldn't the pnl be 112694 * 1012.578(exit price) - 100000000 = 14111465.13?

Ideally, I think pnl should be 112731 * 1012.578 - 100000000 = 14148930.52

I did some digging in source code, and I found this:

    def calc_target_shares(
        self,
        target_size: float,
        price: Optional[float] = None,
        cash: Optional[float] = None,
    ) -> int:
        price = self.close[-1] if price is None else price
        return super().calc_target_shares(target_size, price, cash)

It seems the execution price when buying is the price of last bar, even "ctx.buy_fill_price = PriceType.OPEN" has been set. Is this intentional?

I apologize for not having enough time to find the code that calculating pnl.

Hi, @edtechre,

I have several intraday related scenarios but don't know how to implement in pybroker. It's appreciated if you could give some suggestions.

Scenario 1:

1. Day1, got the signal to long BTC; buy order scheduled to be filled on Day2
2. Day2, buy order filled; and got the signal to short BTC in Day3
3. Day3, expect to close the long position then open a short position. How to achieve this in a right way?

Scenario 2:

1. Day1, got the signal to long BTC; and set the stop loss pct as 2
2. Day2, buy order filled with OPEN price. BTC price dropped 5%. Expect to have the stop order filled in Day2 rather than Day3. (because in Day3, we may get bigger loss than 5%) How to achieve this in a right way?

Thanks

In chinese stock market, when a stock reach a limit-up price, traders cannot buy it on that day, as well as selling at limit-down price.
Is there any parameter to avoid this type of operation in backtest, to make the result more convincing?

Can pybroker realize the multi time frame backtest? For example, for 5 minutes data transaction, the signal uses 10 minutes of data.

Hello,

I have red https://www.pybroker.com/en/latest/notebooks/5.%20Writing%20Indicators.html#Computing-Multiple-Indicators and https://www.pybroker.com/en/latest/_modules/pybroker/indicator.html#IndicatorsMixin but I can not figure out how to use the IndicatorSet with pybroker's Strategy?

I am able to add and remove the Indicators to / from the IndicatorSet but I don't know where the 'DataFramementioned in the documentation cames from and how to use it inside aStrategy`?

Also I would like to calculate Indicators on dedicated symbols not on all. Is this possible?

Could you please give me a hint? Thank you very much.

Hello @edtechre,

I have studied #48 a little bit more intensive. So from what I have understood is that there are 2 different start_date / end_date combinatoins.

• Strategy(start_date, end_date) -> Is used to define the inteval of the data that is fetched
• Strategy#backtest(start_date, end_date) -> Is used to defines backtest inteval

The Strategy#backtest(..., warmup) parameter is added to the start_date and then the Strategy starts trading from start_date + warump until end_date.

Here an example with the warmup period of 5 daily bars: https://github.com/Pirat83/pybroker-experiments/blob/master/main.py

                         cash    equity    margin  market_value     pnl  unrealized_pnl  fees
date                                                                                         
2023-01-03 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-04 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-05 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-06 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-09 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-10 05:00:00  10000.00  10000.00      0.00      10000.00    0.00             0.0   0.0
2023-01-11 05:00:00     30.70  10082.50      0.00      10082.50   82.50             0.0   0.0
2023-01-12 05:00:00     30.70  10251.35      0.00      10251.35  251.35             0.0   0.0
2023-01-13 05:00:00     30.70  10318.45      0.00      10318.45  318.45             0.0   0.0
2023-01-17 05:00:00     30.70  10305.80      0.00      10305.80  305.80             0.0   0.0
2023-01-18 05:00:00     30.70  10139.70      0.00      10139.70  139.70             0.0   0.0
2023-01-19 05:00:00     30.70  10042.90      0.00      10042.90   42.90             0.0   0.0
2023-01-20 05:00:00  10107.25  10107.25   9942.50       9961.75  107.25          -145.5   0.0

My chalenge is to make multiple strategies comparable. So they should all start trading on the 2023-01-01 regardless of there warmup period.

So it is a little bit complicated for me to calculate the concrete start_date - warmup so that the first trades are done exaclty on the 2023-01-01. On the daily timeframe we just have vacation days, weekend and days where the stock exchanges simply are closed. But thing start to get very messy if I want to multiply the indicator values by an timeframe multiplier to apply the daily logic on a lower timframe (i.e 390 when trading 1 min bars). In this scenario things get very complicated.

So is there an option to solve my issue without requiring a business day calendar and a list of days when the stock exchanges openend or not?

I think once the data is read the warmup period can be substracted and all indicators can be calculated. Doing so everything would be waruped before the Strategy.backtest(start_date, ....) and the first trades could happen on this date (if there is a day where the stock exchanges have opened - otherwise the next candle could be taken to start). In the example above the 2022-01-03 would be the date to start trading.

Ideally we could add an additional param to StrategyConfig to keep the APIs and the behavior backward compatible (if desiered). I hope that calculating the warumup then would be much easier (for me) and backtesting results would be more comparable.

What do you think about this idea? Or maybe there is an easy sollution for this, which I did not find until now?

Thank you for your time and your effort. I rally like your work.

I am using macOS 10.14 system, and I encountered a failure while installing lib-pybroker in a Conda virtual environment. I have successfully installed all the packages listed in the requirements.txt file of the source code.

The specific error message is as follows:

(quantitative) MacBook-Pro:pybroker Xxx$ pip install pybroker
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
ERROR: Could not find a version that satisfies the requirement pybroker (from versions: none)
ERROR: No matching distribution found for pybroker
(quantitative) MacBook-Pro:pybroker Xxx$ pip install lib-pybroker -i https://pypi.org/simple
Collecting lib-pybroker
Downloading lib-pybroker-1.1.29.tar.gz (99 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.3/99.3 kB 2.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: akshare<2,>=1.10.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (1.11.79)
Requirement already satisfied: alpaca-py<1,>=0.7.2 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (0.13.3)
Requirement already satisfied: diskcache<6,>=5.4.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (5.6.3)
Requirement already satisfied: joblib<2,>=1.2.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (1.3.2)
Requirement already satisfied: numba<1,>=0.56.3 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (0.58.1)
Requirement already satisfied: numpy<2,>=1.23.4 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (1.26.2)
Collecting pandas<2,>=1.5.1 (from lib-pybroker)
Downloading pandas-1.5.3-cp310-cp310-macosx_10_9_x86_64.whl (12.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.0/12.0 MB 8.2 MB/s eta 0:00:00
Requirement already satisfied: progressbar2<5,>=4.1.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from lib-pybroker) (4.2.0)
Collecting yfinance<1,>=0.1.84 (from lib-pybroker)
Downloading yfinance-0.2.32-py2.py3-none-any.whl.metadata (11 kB)
Requirement already satisfied: beautifulsoup4>=4.9.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (4.12.2)
Requirement already satisfied: lxml>=4.2.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (4.7.0)
Requirement already satisfied: requests>=2.22.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (2.31.0)
Requirement already satisfied: pypinyin>=0.35.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (0.49.0)
Requirement already satisfied: html5lib>=1.0.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (1.1)
Requirement already satisfied: xlrd>=1.2.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (2.0.1)
Requirement already satisfied: urllib3>=1.25.8 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (2.1.0)
Requirement already satisfied: tqdm>=4.43.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (4.66.1)
Requirement already satisfied: openpyxl>=3.0.3 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (3.1.2)
Requirement already satisfied: jsonpath>=0.82 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (0.82.2)
Requirement already satisfied: tabulate>=0.8.6 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (0.9.0)
Requirement already satisfied: decorator>=4.4.2 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (5.1.1)
Requirement already satisfied: py-mini-racer>=0.6.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (0.6.0)
Requirement already satisfied: akracer>=0.0.11 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from akshare<2,>=1.10.1->lib-pybroker) (0.0.13)
Requirement already satisfied: msgpack<2.0.0,>=1.0.3 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from alpaca-py<1,>=0.7.2->lib-pybroker) (1.0.7)
Requirement already satisfied: pydantic<3.0.0,>=2.0.3 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from alpaca-py<1,>=0.7.2->lib-pybroker) (2.5.2)
Requirement already satisfied: sseclient-py<2.0.0,>=1.7.2 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from alpaca-py<1,>=0.7.2->lib-pybroker) (1.8.0)
Requirement already satisfied: websockets<12.0.0,>=11.0.3 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from alpaca-py<1,>=0.7.2->lib-pybroker) (11.0.3)
Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from numba<1,>=0.56.3->lib-pybroker) (0.41.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from pandas<2,>=1.5.1->lib-pybroker) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from pandas<2,>=1.5.1->lib-pybroker) (2023.3.post1)
Requirement already satisfied: python-utils>=3.0.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from progressbar2<5,>=4.1.1->lib-pybroker) (3.8.1)
Requirement already satisfied: multitasking>=0.0.7 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from yfinance<1,>=0.1.84->lib-pybroker) (0.0.11)
Collecting lxml>=4.2.1 (from akshare<2,>=1.10.1->lib-pybroker)
Downloading lxml-4.9.3.tar.gz (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 13.8 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting appdirs>=1.4.4 (from yfinance<1,>=0.1.84->lib-pybroker)
Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting frozendict>=2.3.4 (from yfinance<1,>=0.1.84->lib-pybroker)
Downloading frozendict-2.3.9-cp310-cp310-macosx_10_9_x86_64.whl.metadata (20 kB)
Collecting peewee>=3.16.2 (from yfinance<1,>=0.1.84->lib-pybroker)
Downloading peewee-3.17.0.tar.gz (2.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 12.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: soupsieve>1.2 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from beautifulsoup4>=4.9.1->akshare<2,>=1.10.1->lib-pybroker) (2.5)
Requirement already satisfied: six>=1.9 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from html5lib>=1.0.1->akshare<2,>=1.10.1->lib-pybroker) (1.16.0)
Requirement already satisfied: webencodings in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from html5lib>=1.0.1->akshare<2,>=1.10.1->lib-pybroker) (0.5.1)
Requirement already satisfied: et-xmlfile in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from openpyxl>=3.0.3->akshare<2,>=1.10.1->lib-pybroker) (1.1.0)
Requirement already satisfied: annotated-types>=0.4.0 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from pydantic<3.0.0,>=2.0.3->alpaca-py<1,>=0.7.2->lib-pybroker) (0.6.0)
Requirement already satisfied: pydantic-core==2.14.5 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from pydantic<3.0.0,>=2.0.3->alpaca-py<1,>=0.7.2->lib-pybroker) (2.14.5)
Requirement already satisfied: typing-extensions>=4.6.1 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from pydantic<3.0.0,>=2.0.3->alpaca-py<1,>=0.7.2->lib-pybroker) (4.8.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from requests>=2.22.0->akshare<2,>=1.10.1->lib-pybroker) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from requests>=2.22.0->akshare<2,>=1.10.1->lib-pybroker) (3.5)
Requirement already satisfied: certifi>=2017.4.17 in /Users/Xxx/.conda/envs/quantitative/lib/python3.10/site-packages (from requests>=2.22.0->akshare<2,>=1.10.1->lib-pybroker) (2023.11.17)
Downloading yfinance-0.2.32-py2.py3-none-any.whl (68 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 69.0/69.0 kB 2.3 MB/s eta 0:00:00
Downloading frozendict-2.3.9-cp310-cp310-macosx_10_9_x86_64.whl (36 kB)
Building wheels for collected packages: lib-pybroker, lxml, peewee
Building wheel for lib-pybroker (pyproject.toml) ... done
Created wheel for lib-pybroker: filename=lib_pybroker-1.1.29-py3-none-any.whl size=71825 sha256=b9db78455d0672c6bf18171f4ff00ebcc8df11a5ab7655df3cfb9a40686e1888
Stored in directory: /Users/Xxx/Library/Caches/pip/wheels/e1/bc/eb/f36148fc0144901bd21f9a6572b6fb72769ec8e0541d372c1e
Building wheel for lxml (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [128 lines of output]
Building lxml version 4.9.3.
/private/var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn/T/pip-install-59lluing/lxml_e9c9fa4358b64bbc91e6a3f26a68f09a/setup.py:67: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
Building without Cython.
Building against libxml2 2.9.2 and libxslt 1.1.28
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-cpython-310
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/_elementpath.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/sax.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/pyclasslookup.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/init.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/builder.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/doctestcompare.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/usedoctest.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/cssselect.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/ElementInclude.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/init.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/defs.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/clean.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/init.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/formfill.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/builder.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
copying src/lxml/html/diff.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/html
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron
copying src/lxml/isoschematron/init.py -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron
copying src/lxml/etree.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/etree_api.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/lxml.etree.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/objectify.pyx -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/etree.pyx -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/public-api.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xmlid.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/cleanup.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xslt.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xpath.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/debug.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/serializer.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/classlookup.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/saxparser.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/objectpath.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/relaxng.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xinclude.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/iterparse.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/parser.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/dtd.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/apihelpers.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xmlerror.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/readonlytree.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/extensions.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xmlschema.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/schematron.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/docloader.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/nsclasses.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/parsertarget.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/xsltext.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/proxy.pxi -> build/lib.macosx-10.9-x86_64-cpython-310/lxml
copying src/lxml/includes/xmlerror.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/c14n.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/init.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/dtdvalid.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/relaxng.pxd -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/includes
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/rng
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl
creating build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.macosx-10.9-x86_64-cpython-310/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.macosx-10.9-x86_64-cpython-310
creating build/temp.macosx-10.9-x86_64-cpython-310/src
creating build/temp.macosx-10.9-x86_64-cpython-310/src/lxml
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/Xxx/.conda/envs/quantitative/include -fPIC -O2 -isystem /Users/Xxx/.conda/envs/quantitative/include -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/Users/Xxx/.conda/envs/quantitative/include/python3.10 -c src/lxml/etree.c -o build/temp.macosx-10.9-x86_64-cpython-310/src/lxml/etree.o -w -flat_namespace
In file included from src/lxml/etree.c:864:
src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
#include "libxml/xmlversion.h"
^
1 error generated.
Compile failed: command '/usr/bin/clang' failed with exit code 1
creating var
creating var/folders
creating var/folders/t3
creating var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn
creating var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn/T
cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn/T/xmlXPathInitainbuq9j.c -o var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn/T/xmlXPathInitainbuq9j.o
/var/folders/t3/58ns5ktj705bstp2bzfbpkqr0000gn/T/xmlXPathInitainbuq9j.c:1:10: fatal error: 'libxml/xpath.h' file not found
#include "libxml/xpath.h"
^
1 error generated.
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
Perhaps try: xcode-select --install
*********************************************************************************
error: command '/usr/bin/clang' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lxml
Running setup.py clean for lxml
Building wheel for peewee (pyproject.toml) ... done
Created wheel for peewee: filename=peewee-3.17.0-cp310-cp310-macosx_10_9_x86_64.whl size=255501 sha256=0a1f570d6ec6f16531907c3e21175faf301076fdb05f470460daf76c4584b7a8
Stored in directory: /Users/Xxx/Library/Caches/pip/wheels/c7/70/ad/212867e96e7004265a69c4aa5dcff00a95f547a67ba26e7e76
Successfully built lib-pybroker peewee
Failed to build lxml
ERROR: Could not build wheels for lxml, which is required to install pyproject.toml-based projects

I guess the PriceType is MIDDLE, based on

When liquidated, the shares are sold at market price equal to ExecContext.sell_fill_price, which is configurable and defaults to the midpoint between the bar’s open and close price.

But the definition of MIDDLE is:

Midpoint between low price and high price of the current bar.

There is a discrepancy between these two. So the question is, what is the PriceType of tailing stop trades?

Is it possible to set a particular PriceType for trailing stop? Currently, it gives following error:

ValueError: sell_shares or hold_bars must be set when sell_fill_price is set.

Hello I am trying to experiment with pybroker.

It looks very promissing. However I am facing some problems and require help.
My strategy gets into serious trouble when I add shorting to it. Taking only long positions works quite well. Taking only short positions also crashes the numbers. I have published the code here: https://github.com/Pirat83/pybroker-experiments/tree/master and will explain the issues later.

Traceback (most recent call last):
  File "/pybroker-experiments/main.py", line 116, in <module>
    main()
  File "/pybroker-experiments/main.py", line 107, in main
    result: TestResult = strategy.backtest(timeframe='15m', warmup=54)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1069, in backtest
    return self.walkforward(
           ^^^^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1220, in walkforward
    self._run_walkforward(
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 1323, in _run_walkforward
    self.backtest_executions(
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 295, in backtest_executions
    self._place_buy_orders(
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/strategy.py", line 505, in _place_buy_orders
    order = portfolio.buy(
            ^^^^^^^^^^^^^^
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/portfolio.py", line 534, in buy
    self._verify_input(shares, fill_price, limit_price)
  File "/.conda/envs/pybroker-experiments/lib/python3.11/site-packages/pybroker/portfolio.py", line 375, in _verify_input
    raise ValueError(f"Shares cannot be negative: {shares}")
ValueError: Shares cannot be negative: -4159135348598

Hi,
I found that if i set strategy like below, the winning rate is alway 100%, as losers have alway been held even the backtesting ended, and the losses were not included in the statistics.
Is there any parameter of backtest to avoid this condition, or where i could execute sell_all_shares() at the end_date of backtesting?

if not pos and some conditions:
        ctx.buy_shares = ctx.calc_target_shares(0.2)
        ctx.stop_profit_pct = 3
elif pos:
        do nothing

Which means,

Limit up: maximum amount a price is permitted to increase during one trading day.

Limit down: maximum permitted decline in one trading day.

Hello

Thank you for developping this fantastic library. I'd love to use it, but running Python 3.11.2. Will it be available for that version in the near future?

However, this seems to be an issue with numba:

Collecting numba<1,>=0.56.3 (from lib-pybroker)
  Using cached numba-0.56.4.tar.gz (2.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\center\AppData\Local\Temp\pip-install-36wtc8qn\numba_4157527a6be444afb3ba3015b4b34b93\setup.py", line 51, in <module>
          _guard_py_ver()
        File "C:\Users\center\AppData\Local\Temp\pip-install-36wtc8qn\numba_4157527a6be444afb3ba3015b4b34b93\setup.py", line 48, in _guard_py_ver
          raise RuntimeError(msg.format(cur_py, min_py, max_py))
      RuntimeError: Cannot install on Python version 3.11.2; only versions >=3.7,<3.11 are supported.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Thank you

Thank you for your wonderful tool.

(1) May I ask how to buy and sell on the same day? when the sell strategy is satisfied it will sell the stocks, then moved to the next day. It just ignore if there exist a buy strategy on the same day. How to make it possible to buy and sell on the same day?

(2) how to get the price of today? I want to buy the stock on the same day it satisfied the strategy, but it seems the system only allow the transactions on the next day

(3) is there any ways to get the current profit or loss percent directly? I use the code below to get the current profit

    initial_cash = ctx.config.initial_cash
    current_cash = ctx.total_market_value + ctx.cash
    profit_pct = current_cash/initial_cash - 1

(4) how can I control the buy or sell action manually? just set the sell_shares/sell_fill_price? it will sell immediately when I set them?

Hi,

The project is very promising, thanks for that.

I couldn't find in the documentation how PyBroker deals with tickers that start missing data at a specific date, due to de-listing or bankruptcy.

For example, if I am using a custom DataSource based on pd.DataFrame, and backtesting from 2000-01 to 2020-01, rebalancing every year. Say if a given company goes bankrupt in the middle of 2015, what will happen when I try to sell it at the beginning of 2016, when rebalancing?

I understand this logic can be built on the algorithm side, but was wondering if PyBroker provides any built-in mechanisms. The simplest example - sell at the last provided price (which is mid-2015).
This leads to a more general question of how PyBroker deals with edge cases and if there are any safe-guards for custom data sources.

Thank you,
Denis

I tried installing lib-pybroker in a conda environment with Python 3.10 and I'm getting an error:

$ conda create -n test_pyb python==3.10
$ conda activate test_pyb
$ pip install -U pip setuptools wheel
$ pip install lib-pybroker
...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
nbconvert 7.2.9 requires jinja2>=3.0, which is not installed.
nbconvert 7.2.9 requires markupsafe>=2.0, which is not installed.
nbclassic 0.5.1 requires jinja2, which is not installed.
jupyter-server 2.1.0 requires jinja2, which is not installed.
jupyter-events 0.6.3 requires jsonschema[format-nongpl]>=3.2.0, which is not installed.

Here's my pip freeze

aiohttp==3.8.1
aiosignal==1.3.1
alpaca-trade-api==2.3.0
anyio==3.6.2
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.2.1
async-timeout==4.0.2
attrs==22.2.0
backcall==0.2.0
beautifulsoup4==4.11.2
bleach==6.0.0
certifi @ file:///croot/certifi_1671487769961/work/certifi
cffi==1.15.1
charset-normalizer==2.1.1
comm==0.1.2
cryptography==39.0.2
debugpy==1.6.6
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
diskcache==5.4.0
docker==6.0.1
dockerpty==0.4.1
docopt==0.6.2
executing==1.2.0
fastjsonschema==2.16.2
frozendict==2.3.5
frozenlist==1.3.3
html5lib==1.1
idna==3.4
ipykernel==6.20.2
ipython==8.9.0
ipython-genutils==0.2.0
ipywidgets==8.0.4
jedi==0.18.2
joblib==1.2.0
jupyter==1.0.0
jupyter-console==6.4.4
jupyter-events==0.6.3
jupyter_client==8.0.1
jupyter_core==5.1.5
jupyter_server==2.1.0
jupyter_server_terminals==0.4.4
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.5
lib-pybroker==1.0.17
llvmlite==0.39.1
lxml==4.9.2
matplotlib-inline==0.1.6
mistune==2.0.4
msgpack==1.0.3
multidict==6.0.4
multitasking==0.0.11
nbclassic==0.5.1
nbclient==0.7.2
nbconvert==7.2.9
nbformat==5.7.3
nest-asyncio==1.5.6
notebook==6.5.2
notebook_shim==0.2.2
numba==0.56.4
numpy==1.23.5
packaging==23.0
pandas==1.5.3
pandocfilters==1.5.0
parso==0.8.3
pickleshare==0.7.5
platformdirs==2.6.2
progressbar2==4.2.0
prometheus-client==0.16.0
prompt-toolkit==3.0.36
psutil==5.9.4
pure-eval==0.2.2
pycparser==2.21
Pygments==2.14.0
pyrsistent==0.19.3
python-dateutil==2.8.2
python-dotenv==0.21.1
python-json-logger==2.0.4
python-utils==3.5.2
pytz==2022.7.1
PyYAML==6.0
pyzmq==25.0.0
qtconsole==5.4.0
QtPy==2.3.0
requests==2.28.2
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
Send2Trash==1.8.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.4
stack-data==0.6.2
terminado==0.17.1
texttable==1.6.7
tinycss2==1.2.1
tornado==6.2
traitlets==5.8.1
urllib3==1.26.14
wcwidth==0.2.6
webencodings==0.5.1
websocket-client==1.5.1
websockets==10.4
widgetsnbextension==4.0.5
yarl==1.8.2
yfinance==0.2.12

This is non blocking, manually installing jinja2 solves the issue. Also, for some reason it seems this happens only on conda environments.

Hi, @edtechre,

I noticed that the "calc_target_shares" function consistently returns an integer value. It seems not to factor in the "enable_fractional_shares" option. Was this intentional?
For context, higher-priced cryptocurrencies like BTC permit trading in fractional shares. When backtesting BTC with a limited initial cash amount, I always get a result of 0 shares from "calc_target_shares". While I can create my own function to determine shares, I'm wondering if I've correctly understood the purpose of the "enable_fractional_shares" option.

def calc_target_shares(
        self, target_size: float, price: float, cash: Optional[float] = None
    ) -> int:
        r"""Calculates the number of shares given a ``target_size`` allocation
        and share ``price``.

        Args:
            target_size: Proportion of cash used to calculate the number of
                shares, where the max ``target_size`` is ``1``. For example, a
                ``target_size`` of ``0.1`` would represent 10% of cash.
            price: Share price used to calculate the number of shares.
            cash: Cash used to calculate the number of number of shares. If
                ``None``, then the :class:`pybroker.portfolio.Portfolio` equity
                is used to calculate the number of shares.

        Returns:
            Number of shares given ``target_size`` and share ``price``.
        """
        shares = int(
            (to_decimal(cash) if cash is not None else self._portfolio.equity)
            * to_decimal(target_size)
            / to_decimal(price)
        )
        return max(shares, 0)

Hi, do you mind adding issues? I would love to contribute to your project

Hello,

Thanks for building this framework!

We at AlgoTrading101 are planning to create a few videos on using a backtesting library and my idea was to use this one as it has a nice pythonic way of building out strategies. But, I do have some difficulties and questions.

For example, is there a way that I can execute an order for a specific ticker (e.g. TSLA) if the ctx that I'm currently on is another stock? For example, if I was doing a pairs trade strategy with assets A and B. Can I long asset A and short asset B at the same time or do I need to wait for the ctx symbols to switch out?

If I was to rebalance the portfolio is there a built-in method or do I need to loop through each ctx and rebalance based on that?

Great work on this library!

My team maintains a few open-source projects, and we've seen that having interactive examples greatly helps newcomers! We're maintaining a cluster that provides ephemeral JupyterLab sessions (example). It essentially spins up a JupyterLab session with pre-made examples.

We're happy to sponsor GeneticAlgorithmPython with some compute power. If you're interested, please get in touch with me! (my contact info is in my profile)

hi edtechre! i want to use one model for all my symbols,but i want still use the walkforward=3 or more , How to use pretrained model with walkforward windows larger than 1, can i use the fn load different models when walkforward to next windows?
ps: do you have the plan to support one model for all symbols?

Hi @edtechre,

I am currently adding live trading capabilties to pybroker. So far it works quite well. One thing that would make my life easier, would be the capability to externalize the Portfolio. Currenlty it is created in the walkforward method and there is no clean way to provide it from a SQL database or an object storage / file system, etc...

It would be a benefit if we could pickle the Portfolio and on the next trading iteration run a backtest / trade method with the previosly unpickled Portfolio. In this way we would also gain incremental backtest capability, which is also one of my goals. So if someone runs a complicated backtest or runs many of them, this persion could simply save the Portfolio and then reuse it on the next trading interation without executing all the past trades again and again.

For now I have extended the Strategy class and override the methods where necessary. This works but is not a very clean way to do it. The benefit of this approach is, that it will be easier to integrate code changes from upcomming pybroker releases.

So currently the Portfolio is created here:

Also this place

If you are interested I can share my progress and we can togeather add life trading capabilities to pybroker.

Hi @edtechre,

I'm reviewing examples in "10. Rotational Trading.ipynb", I modified nothing but adding a line of trailing stop setting code to function "rotate" like this:

def rotate(ctx: ExecContext):
    if ctx.long_pos():
        if ctx.symbol not in pyb.param('top_symbols'):
            ctx.sell_all_shares()
    else:
        target_size = pyb.param('target_size')
        ctx.buy_shares = ctx.calc_target_shares(target_size)
        ctx.score = ctx.indicator('roc_20')[-1]
        ctx.stop_trailing_pct = 5 # <------- adding trailing stop setting here

The backtesting result is interesting. There are some weird short trades in this purely long scenario. Is this normal?

I went across the docs and I only see that bar means day.
In a lot of cases that's not enough.

1. There is case when algorithm is running on more granular data.
2. There is case when bar is forming (before its bar window is closed) on live trading. Would be great that that backtesting algorithm support this scenario as it may give very different results when switching to live.

I just came across this project recently. I really like what saw so far!

I am wondering if you currently (... / are planing to) support trading individual pairs? As in multiple currencies (forex / crypto). From what I saw you only model positions with an opening and closing trade.

你好，
https://pypi.org/project/lib-pybroker/ 页面的例子
换用pandas的dataframe作源
strategy = Strategy(df, start_date='1/1/2022', end_date='7/1/2022')
strategy.add_execution(
exec_fn, ['000001', '600000'], indicators=highest('high_10d', 'close', period=10))

Run the backtest after 20 days have passed.

result = strategy.backtest(warmup=20)

报标题错误。这是什么问题，如何解决？

docs里的notebooks 2. backtesting a strategy.ipynb 同样换df源正常执行

Hi edtechre:

I'm just playing the notebook: https://github.com/edtechre/pybroker/blob/master/docs/notebooks/10.%20Rotational%20Trading.ipynb

I can reproduce the same result when modifying nothing to the code, however, if I add following line to the first cell of notebook, the result is different from first run nor the result in online doc.

pyb.enable_caches('rotational_trading')

the first 10 row result using cached data:
1,buy,META,2018-01-03,275,NaN,183.06,0.0
2,buy,NFLX,2018-01-03,243,NaN,203.86,0.0
3,sell,META,2018-02-01,275,NaN,191.61,0.0
4,buy,AMD,2018-02-01,3903,NaN,13.53,0.0
5,sell,AMD,2018-02-05,3903,NaN,11.56,0.0
6,buy,AMZN,2018-02-05,649,NaN,69.49,0.0
7,sell,NFLX,2018-04-03,243,NaN,284.63,0.0
8,sell,AMZN,2018-04-03,649,NaN,69.23,0.0
9,buy,INTC,2018-04-03,1151,NaN,49.18,0.0
10,buy,MSFT,2018-04-03,636,NaN,88.97,0.0

the first 10 row result using live data:
1,buy,AAPL,2018-01-03,1161,NaN,43.31,0.0
2,buy,AMD,2018-01-03,4231,NaN,11.75,0.0
3,sell,AAPL,2018-02-01,1161,NaN,41.92,0.0
4,buy,NFLX,2018-02-01,181,NaN,267.67,0.0
5,sell,AMD,2018-02-05,4231,NaN,11.56,0.0
6,buy,AMZN,2018-02-05,707,NaN,69.49,0.0
7,sell,AMZN,2018-04-03,707,NaN,69.23,0.0
8,sell,NFLX,2018-04-03,181,NaN,284.63,0.0
9,buy,INTC,2018-04-03,1014,NaN,49.18,0.0
10,buy,MSFT,2018-04-03,560,NaN,88.97,0.0

Is this normal?

Hi,
I am trying to register one model for backtesting all symbols. Is there an intended way to do that?
If I understood the documentation right, while backtesting a symbol the respective model gets loaded. How to load a "general" model?

https://www.pybroker.com/en/latest/notebooks/6.%20Training%20a%20Model.html

Best regards

Hi Ed,
I have some suggstions about PyBroker and described in "title", you can refer: https://gist.github.com/danieldogeanu/739f88ea5312aaa23180e162e3ae89ab

Hi @edtechre,

In the backtest, I set ctx.buy_fill_price = PriceType.OPEN and also used stop trailing. From the trade records in result, it appears that a lot of stop losses occurred on the same day as the buy, which can't be done in T+1 market, it has to be achieved at next trading day. How do I simulate such market rules inside pybroker?

Hi edtechre,

Is it possible to add more metrics? Like following:

Buy & Hold Return [%] 293.941538
Return (Ann.) [%] 20.080276
Volatility (Ann.) [%] 18.147456
Sortino Ratio 2.091836
Calmar Ratio 1.127568

Hi,

Trying to reconcile how final metrics are calculated - how end_market_value is calculated and what is its relationship to PnL and unrealized PnL?

For example in the screenshot below, the final market value of the long-only portfolio is much smaller than realized + unrealized PnL:

Hi, edtechre
I was deeply attracted by pyb recently.
I want to train and backtest model using "Walkforward", but the framework of pyb is based on one symbol one model.
Mining global rule is a native need for ML, e.g. technology stocks(goog, msft) may have same latent law.
So, In pybroker.model(...,fn(symbol, train_data, test_data),...), "symbol" may can be a group of symbols, as well as train_data and test_data?

Hi Ed,

After trying the Trailing Stop function in example (https://www.pybroker.com/en/latest/notebooks/8.%20Applying%20Stops.html), it was natural for me to want to implement an ATR Trailing Stop.

I saw an attribute "stop_trailing" in the ExecContext, and the documentation describes it as "Sets a trailing stop loss on a new pybroker.portfolio.Entry, where value is measured in points from entry price.", I'm a little confused here, in the case of long, will it stop when it is below the "highest market price" by a certain margin (percent/amount), like "stop_trailing_pct", or when it is below the "entry price/buy price" by a certain margin (percent/amount)?

I tried stop_trailing in following code, however, it seems it doesn't work, there is no trade been created, am I missing anything here?

def buy_with_trailing_stop_loss_and_profit(ctx):
    if not ctx.long_pos():
        ctx.buy_shares = ctx.calc_target_shares(1)
    elif ctx.long_pos():
        ctx.cancel_stops(ctx.symbol)
        # I guess stop_trailing setting a point where below the highest market price, so I put a 2 times of ATR here.
        # BTW, I used talib caculated ATR
        ctx.stop_trailing = ctx.indicator('atr_100')[-1] * 2

the result.metrics_df:
name value
0 trade_count 0.00
1 initial_market_value 100000000.00
2 end_market_value 385076252.89
3 total_pnl 0.00
4 unrealized_pnl 285076252.89
5 total_return_pct 0.00
6 total_profit 0.00
7 total_loss 0.00
8 total_fees 0.00
9 max_drawdown -429843907.78
10 max_drawdown_pct -72.30
11 win_rate 0.00
12 loss_rate 0.00
13 winning_trades 0.00
14 losing_trades 0.00
15 avg_pnl 0.00
16 avg_return_pct 0.00
17 avg_trade_bars 0.00
18 avg_profit 0.00
19 avg_profit_pct 0.00
20 avg_winning_trade_bars 0.00
21 avg_loss 0.00
22 avg_loss_pct 0.00
23 avg_losing_trade_bars 0.00
24 largest_win 0.00
25 largest_win_pct 0.00
26 largest_win_bars 0.00
27 largest_loss 0.00
28 largest_loss_pct 0.00
29 largest_loss_bars 0.00
30 max_wins 0.00
31 max_losses 0.00
32 sharpe 0.01
33 profit_factor 1.03
34 ulcer_index 4.87
35 upi 0.01
36 equity_r2 0.44
37 std_error 112812652.60

It would be great if we could have various order types supported:

• Stop-loss
• Take-profit
• Stop-buy/sell
• Limit-buy/sell

Hope there is no fundamental problem with supporting these things?