If you check out the website at:

https://www.quantconnect.com/forum/discussion/369/auto-updating-indicators#

You'll see that there's some code at the top. On first glance it looks like we're not properly escaping something.

This was reproduced in Firefox and Chrome, seems to be an HTML issue, not a browser issue.

Make an indicator consolidator for when the data is custom, but extends a TradeBar (so still has OHLC).

Update the ResolveConsolidator in QCAlgorithm to accept/search for this Custom : TradeBar data

When leveraged asset value falls below 25% of capital, or $25k (whichever is greater), call the liquidate function on the algorithm. https://www.interactivebrokers.com/en/?f=margin

Currently this isn't done and its possible to for portfolio to drop below zero when highly leveraged and asset is volatile.

Hi Jared,
In QC.Lean.Engine, some references such as "Fasterflect" and "RestSharp" are missing. Couldn't find them in the directory. Please help.
Cub

I have created custom data sets for Lean based on the same forex data format, but using data from an external source. Data is 1min and each file has been broken into 1 day from 00:00 through 23:59, and zipped. File formatting and naming is the same (other than the inclusion of volume in my data).

Sample data files are available here.

I have found that UTC+2 works best for this data (FXCM sourced via IQFeed) as I don't get any Sat/Sun data files.

There are sometimes data files that don't a complete 24 hour period, and certain days that don't exist (ie. Sat/Sun). There will be other times when the first trading day of the week doesn't kick off right at 00:00. My assumption is that using Fill Forward sorts out any errors relating to these issues, and this seems to be correct based on the limited understanding of QC.

My issue is when I run a backtest against my data, there are times when the algorithm never finishes, eg: running the MovingAverageCross algorithm with default settings other than:

private const string Symbol = "AUDUSD";
AddSecurity(SecurityType.Forex, Symbol, Resolution.Minute, true);

In this example, the End Date is 1 Jan 2015. When running this against my data (which has not file for this date), the following output is received:

What looks to be occuring is that the algorithm is waiting for data from the FileSystemDataFeed that never arrives (and never will since it doesn't exist). The Isolator.ExecuteWithTimeLimit() message occurs regularly (every 60 seconds).

If I change the End Data to a day when there is a data file available, the algorithm seems to execute (and end) correctly.

Am I missing something here? Is there a way to Fill Forward across days to resolve this issue?

I don't really want to add a check for file existence into the algo as this means nothing other than in backtesting.

Regards

Adam

OnEndOfDay uses the backtesting realtime event handler to fire callbacks. It takes in the current time from the data feed and when its less than 10 minutes from close fires the event.

Because its using the time-stream with daily data the event never gets called, and the next day just resets the events so it gets forgotten.

To fix; before resetting with the next day data; check if any un-triggered events from day before - and fire them all.

Used to make price action (time eliminated) bars to make more better trends.

Create the Williams % R Indicator (http://en.wikipedia.org/wiki/Williams_%25R), and the associated "WILR" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the WILR values match up with the data provided in Tests/TestData/spy_with_williamsR14.txt file.

Create an "AverageTrueRange" indicator class and the associated "ATR" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the Average True Range values match up with the data provided in Tests/TestData/spy_atr.txt file.

Trying to build on fresh Win8 Visual Stduio 2013 Commuity, exception at QuantConnect.Extensions/Extensions.cs

System.ArgumentException was unhandled
HResult=-2147024809
Message=Unable to locate any exports matching the requested typeName: QuantConnect.Messaging.Messaging
Parameter name: typeName
Source=QuantConnect.Common
ParamName=typeName
StackTrace:
at QuantConnect.Extensions.GetExportedValueByTypeName[T](CompositionContainer container, String typeName) in c:\Users\Xenu2\Desktop\Lean-master\Common\Extensions.cs:line 330
at QuantConnect.Lean.Engine.Engine.Main(String[] args) in c:\Users\Xenu2\Desktop\Lean-master\Engine\Engine.cs:line 188
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:

I found open source C# API for Metatrader 4 at Codeplex, didn't tested yet if it works, but this is good way to get free live data from various brokers on FX, CFDs, some major stocks (all minute level, ticks need to be collected live), and also have implementation of all brokers who support MT4.

Here: https://tradeplatform.codeplex.com/

ex.,
Bought at 07/29/2014 09:30:00 with Quand time 07/29/2014 00:00:00
when actually it's Close/Settle value is end of day.

Create an "MoneyFlowIndex" indicator class and the associated "MFI" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the Money Flow Index values match up with the data provided in Tests/TestData/spy_mfi.txt file.

Create test helper which runs through all example algorithms included in Lean. As we add new features write an example algorithm for each feature. The test helper will make sure the full system continues to work going forward.

Several indicators don't seem to properly implement the Reset() function. Review each indicator and verify a test exists for the Reset() feature.

Create an "OnDelisting" event in the QCAlgorithm class which is called when a symbol is to be delisted.

This should be triggered in the Algorithm Manager(?) from a feed.DataEvents queue which has generic status events on the data. The data event will include the subscription-id, symbol and relevant event types.

Before the OnData event, the DataEvents queue would be cleared (synchronously processed) and passed into the algorithm event handlers.

This same mechanism could later be used for circuit breakers, and unexpected market closes.

For now this would only be supported in backtesting using the map_files. If the map files final date is today, pass through the delisting data event. For live trading we would need to organize a data source.

I am trying to get what amounts to a brokerage confirmation to be reported in my algo so I can compute the actual cost of the transaction/order. Right now I am doing it through the OnOrderEvent method overriding the virtual QCAlgorithm event as follows.

    public override void OnOrderEvent(OrderEvent orderEvent)
    {
        base.OnOrderEvent(orderEvent);
        var direction = orderEvent.Direction;
        var qty = orderEvent.FillQuantity;
        var price = orderEvent.FillPrice;
        var tradevalue = qty*price;
        var security = Securities[orderEvent.Symbol];
        var projectedFees = security.Model.GetOrderFee(qty, price);
        var orderNet = tradevalue + projectedFees;
    }

This method is not very satisfactory because things like exchange fees may not be included in the GetOrderFee method on the parent. The response from the broker should have those fees.

Dredging through the IBClient code on line 4498 of https://github.com/krs43/ib-csharp/blob/master/Krs.Ats.IBNet/IBClient.cs are the lines:

orderState.Commission = ReadDoubleMax();
orderState.MinCommission = ReadDoubleMax();
orderState.MaxCommission = ReadDoubleMax();
orderState.CommissionCurrency = ReadStr();

So it appears that IB's OrderState object (https://github.com/krs43/ib-csharp/blob/master/Krs.Ats.IBNet/OrderState.cs) has the commission and fees which can be obtained thorough the IncomingMessage.OpenOrder in ProcessMessage(IncomingMessage msgId) starting on line 3835

Also I have found a CommissionReport object at https://www.interactivebrokers.com/en/software/api/api.htm

Nick Stein

I'm trying to explore the examples and have found some problem exists with the MovingAverageCross. I'm a little new to C$#, so the solution is not obvious to me. Hopefully, someone can work it out. Here is the error:

ERROR:: ConsoleSetupHandler().Setup(): Type mismatch found between consolidator and indicator for symbol: SPY.Consolidator outputs type TradeBar but indicator expects input type indicatorDataPoint

Create an "CommodityChannelIndex" indicator class and the associated "ATR" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the Commodity Channel Index values match up with the data provided in /Tests/TestData/spy_with_cci.txt file.

Add market on open and market on close order type.

1. Specific Order Classes in the /Common/Orders folder, and added OrderTypes to the enum

2. Add the default fill models for each order type (one method each). Update the Interface and implement the default fill models in SecurityTransactionModels.cs. Ensure to check the validity of the order in the fill model.

3. Map the OrderType to its fill method in BacktestingBrokerage

4. Add helper methods in QCAlgorithm.Trading.cs which return the order id for the new order. Add an

Market On Open - According to (http://www.investopedia.com/terms/m/marketonopen-order-moo.asp) the orders can only be submitted/or cancelled between 7am - 9.28am.

Market On Close - According to (http://www.investopedia.com/terms/m/marketonclose.asp) the orders must be submitted before 3.45pm and after which can't be modified.

The order must always be rejected if its a forex type in the helper method.

We don't have a good way of updating orders yet.. currently order objects must be updated directly and resubmitted for processing. This puts [too much] responsibility in the algorithm to not break things. We're weighing up whether to make the orders immutable. This will break some existing community algorithms but would ensure they can't modify order state, and avoid threading issues.

Feedback welcome -

Holdings price in Live is rounded to 2dp in view, but not in data.

oResults: {
"Charts": {},
Holdings: {
EURUSD: {
Symbol: "EURUSD",
AveragePrice: 1.1349713496,
Quantity: 1101421,
MarketPrice: 1.13538
}
},
"Orders": {},

Create an "OnBalanceVolume" indicator class and the associated "OBV" helper method in the QCAlgorithm base class. See introduction to indicators https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the OBV values match up with the data provided in Tests/TestData/spy_with_obv.txt file.

Create an "ParabolicSAR" indicator class and the associated "SAR" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the Parabolic SAR values match up with the data provided in Tests/TestData/spy_parabolic_SAR.txt file.

If ResolveConsolidator has no matching consolidator found which better suits the data type provided, use a BaseData consolidator which used only Time-Value which will be provided by the custom data itself.

This way indicators can be universally applied to any data types which implement BaseData.

If you use the engine behind a proxy server you could get into trouble (if you work with custom data and during statistics generation). To solve that I recommend to set the Proxy property of WebClient instances always to WebRequest.GetSystemWebProxy().

Example (SubscriptionDataReader.cs):

using (var client = new WebClient())
{
client.Proxy = WebRequest.GetSystemWebProxy();
client.DownloadFile(source, location);
}

Create Donchian Price Channels Indicator configurable for n-periods, http://en.wikipedia.org/wiki/Donchian_channel

Add the associated tests to Tests project under /Indicators to confirm the DON values match up with the data provided in Tests/TestData/spy_with_don50.txt file.

Could use the max-min sub indicators for the given window.

I didn't manage to run the project after a git clone. It complained that the format was invalid. The problem seems to be that the data files are formatted with a dot as number separator, but my culture settings (Swedish) and hence Convert.ToDecimal expects a comma.

If I change the Convert.ToDecimal's that handle csv-split arrays, into:
Convert.ToDecimal(csv[0], new CultureInfo("en-US"))
everything works perfectly. Thought you should know.

Cheers and thanks for all your great work!

Refactor the Statistics class to be an instance driven, with a constructor which takes the required information and creates an instance of Statistics.

Statistics object should define all the currently generated statistics as public properties and include description parameters for the English text.

Benchmark should be separated out and implement an IBenchmark; be generated & cached in a local file once per day.

It would require two new algorithm helper methods:

1. void SetBenchmark(IStatisticsBenchmark) - in the initialize set the IBenchmark to use for generating the statistics.
2. Statistics GetStatistics(TimeSpan period) - In the QCAlgorithm generate an instance of the statistics object for a specified timespan (default to all the algorithm run time).

Replace the statistics generation in Engine with a call to algorithm.GetStatistics, (and update the Result handlers to take a Statistics Object instead of a Dictionary<string,string>.

Create a consolidator which can pipe multiple data symbols into an indicator. It would send a IDictionary<string, BaseData> into the indicator which would support custom and tradebar data.

(TradeBars object is a IDictionary<string, TradeBar> which could extend from the generic base data version)

Add the ability to use a custom benchmark, or at least any symbol from yahoo. Steps:

1. Add helper method SetBenchmark(string symbol) which takes the Yahoo symbol,
2. Save the symbol to a public "string Benchmark" property,
3. Update IAlgorithm to include new Benchmark property.
4. Use this symbol when downloading the benchmark data from Yahoo in the Statistics class.

I was able to fork Lean into my Github account. I then cloned it into VS2013 Pro and cleaned up the XML missing comments. I commited the changes to my local repo. When I tried to push or sync the changes, I get...
An error occurred. Detailed message: Response status code does not indicate success: 403 (Forbidden).

When I look at pull requests, I can do a new PR and compare files, but they are the same between my fork on GitHub and the QuantConnect/Lean on GitHub.

What do I need to do?

Sorry about the newbie question. I just wanted to make sure I could do a pull and commit before I started really adding my indicators. Good thing I did too.

Nick

Leverage is currently free in LEAN. Implement a system for deducting interest charges from cash when holding leveraged stock overnight. Apply a default interest rate of 1.5% on the leveraged fraction of holdings. Potentially best place is to apply cash charge on day change in AlgorithmManager data loop.

OANDA has a REST API with reference implementation in C#
https://github.com/oanda/rest-csharp-refimpl/
I would like to implement this feature. I will study it over the next few days.

Add Stop Limit order type to LEAN

1. Extend the order type enum to add the new type.
2. Add new default fill methods to SecurityTransactionModel, EquityTransactionModel and ForexTransactionModel. High chance only need to implement the base type (SecurityTransactionModel) and the others can just use that.
3. Create helper methods in QCAlgorithm.Transactions.cs which return the order id of the newly created and submitted order.

int LimitOrder(symbol, quantity, limitPrice); 
int StopLimitOrder(symbol, quantity, stopPrice, limitPrice);
void CancelOrder(int orderId);

Because limit orders are filled at a later moment in time they can bypass the equity checks(?) and be submitted directly to the Orders queue (they'll need to be be checked at fill time dynamically by the fill methods**). If an invalid order is requested (e.g. stop price is zero or negative) return a negative value for order id (See Order method in QCAlgorithm Transactions file).

** Should research/get references this is possible from a brokerage or other source - maybe in reality the brokerage limits the size of limit orders when they are placed to the account buying power.

Allow converting adjusted prices back to the raw price by adding a method and value for the conversion factor. Thanks @AlexCatarino

Check out the discussion on the LWMA issue about weighted indicators and how it could be composed. This would allow things like VWAP to be easily defined.

Security class is fundamental entity representing an asset object in Lean. For the long term future of Lean we need solid tests which cover this object to ensure future changes don't break current behavior.

Following test-driven-design we must break testing units into their smallest components. For Security we need to test each property and method, and that changes to its internals, then make sure its still behaving.

To test Security we propose the following steps, first create a security test file in the tests project.

1. Init Security object with configuration object and update its holdings to a known state.
2. Test simple properties: e.g. Test security.Symbol is same as config.Symbol which constructed it. (Symbol, Type, Resolution, IsFillForward, IsExtendedMarketHours, SubscriptionDataConfig, MarginModel.Leverage == Leverage initializer,
3. Test holdings properties behave as expected: HoldStock,Invested true when long or short,
4. Test Methods: Updating the security with price data causes the price property to change.
5. SetLeverage() command updates the security.Leverage value.
6. Test constructor - Verify the state of the internal initialization is as expected (e.g. Cache == typeof SecurityCache() class for each of the objects initialized in the constructor)

Unable to start Lean.Engine.exe as: see image

Create an "AverageDirectionalIndex" indicator class and the associated "ADX" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the ADX values match up with the data provided in Tests/TestData/spy_with_adx.txt file.

I need a WMA for a Algo I am writing, so I have written a WMA. This issue is so that I can make a branch and pull request to add the WMA to the project while tesing the contributor workflow.

When the WMA indicator is created, the user provides a period (an int) and optionally a name (a string). The WeightedMovingAverage is derived from WindowIndicator as are most of the indicators. The constructor calls the base class constructor which creates the RollingWindow of period length.

This version of the WMA computes the average as a linear weighted moving average.

Quandl data currently is fixed to look for the closed column. This could be made configurable to support a broader range of Quandl dynamic data.

Create a Rate of Change indicator and associated ROC and associated helper method QCAlgorithm base class. See introduction to indicators https://lean.quantconnect.com/docs#topic9.html

More information on the ROC indicator.

• http://www.investopedia.com/terms/p/pricerateofchange.asp
• http://research.scottrade.com/knowledgecenter/Public/education/Article?docid=8813a3b19ac04f2fbcad244836a5e43c

The Momentum.cs indicator could be a good template to start from. ROCP is almost identical; just like MOM%. It makes sense to create both these indicators at the same time.

Add the associated tests to Tests project under /Indicators to confirm the ROC and ROCP values match up with the data provided in Tests/TestData/spy_with_roc50.txt file, spy_with_rocp50.txt file.

Create an "BollingerBand" indicator class and the associated "BB" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the Bollinger Bands match up with the data provided in Tests/TestData/spy_bollinger_bands.txt file.

Rolling alpha/ beta indicators. Probably along with covariance and correlation. As part of flexible portfolio management ranks and relationships.

Alpha and beta are the most overlooked professional indicators in the so called popular "technical analysis".

Implement a system for retrieving historical data as an array. The QCAlgorithm helper methods should match the interface below:

Get all data from start to end; for specified symbol:
IEnumerable<BaseData> History(string symbol, DateTime start, DateTime end);

Overload: Get all data from now back a timspan period for symbol and return it as an enumerable list:
IEnumerable<BaseData> History(string symbol, TimeSpan period);

QCAlgorithm should have an IHistoryManager plugin point which is set by Lean Engine according to the DataFeed. The IDataFeed would provide a GetHistoryManager method which returns a IHistoryManager, and that is set in the algorithm.

IHistoryManager implementation for the FileSystemDataFeed would implement this with a physical pull from the file system for the associated dates.

When an indicator requires the use of other indicators in order to do its calculation, it is often useful to expose these 'sub' indicators as public get/private set properties. This allows them to be used in composition if desired.

Update the AverageTrueRange indicator to expose it's internal _smoother (rename?) so consumers can bind to it. Also consider defining an indicator to expose the true range calculation as an indicator.

Create a Stochastic indicator and associated STOCH helper method QCAlgorithm base class. See introduction to indicators https://lean.quantconnect.com/docs#topic9.html

Add the associated tests to Tests project under /Indicators to confirm the STOCH values match up with the data provided in Tests/TestData/spy_with_stoch12k3.txt file

Create class for notifying the algorithm owner of significant events via email, sms or web hooks.

Build as property in base class QCAlgorithm and store the objects in same pattern as Logs and Debug.

Usage would fit the following API inside the algorithm:

algorithm.Notify.Email(string toAddress, string message, string ccAddress = "", string data = "");
algorithm.Notify.Sms(string toNumber, string message);
algorithm.Notify.Web(string websiteEndpoint, object data);

In the Lean Engine IResultHandler process the objects via IMessagingHandler plugin to send the messages to their endpoints.

Before sending to Messaging Handler

• Ensure only send during live trading.
• Use per per day caps to generically limit message sending

The Sell methods all call the Order method, which in turn call the MarketOrder method. The MarketOrder method creates a new MarketOrder type which derives from the Order type. If the Quantity property is > 0 the OrderDirection is set to Buy and if the Quantity < 0, the OrderDirection is set to Sell.

So in an algo, when I called Sell(_symbol, 100m), QuantConnect executed a Market buy order. This method name seems counter intuitive to me. When an algorithm writer calls the Sell method, (s)he would expect the algo to try and sell the security.

I believe the 4 Sell methods should check the sign of the quantity and convert it to negative, so that the method will always try to execute a Sell. The logic should be

        public int Sell(string symbol, decimal quantity)
        {
            var qty = quantity;
            if (quantity > 0)
            { 
                 qty = qty * -1;
            }
            return Order(symbol, qty);
        }

If all agree, I would be happy to make this change and will do a fork, pull for the issue.