Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: QuantConnect/lean-cli
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.0.212
Choose a base ref
...
head repository: QuantConnect/lean-cli
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
  • 10 commits
  • 17 files changed
  • 6 contributors

Commits on Dec 30, 2024

  1. Fix: missed project-id in command lean cloud live deploy ... (#531)

    * fix: add project in lean config when run cloud project
    
    * test:fix: use CharlesSchwab
    
    * test:fix: mock response for CharlesSchwab in test_cloud_live_deploy_with_live_holdings
    Romazes authored Dec 30, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    7f2fb6c View commit details

Commits on Jan 7, 2025

  1. Feature: streaming delay optional config [TradeStation, CharlesSchwab] (

    #536)
    
    * feat: update Readme with new input params
    
    * feat: new param of CS in Readme
    
    * remove: extra configs to several brokerages
    
    * remove: extra configs of TS brokerage
    
    * remove: extra configs of TS brokerage
    
    * feat: delay config for CharlesSchwab brokerage
    Romazes authored Jan 7, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    52437d0 View commit details

Commits on Jan 10, 2025

  1. Remove: IEX data provider (#537)

    * remove: IEX from README.md
    
    * remove: all IEX mention in test_live.py
    Romazes authored Jan 10, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    71f231b View commit details
  2. Fix: typo in description data download docs (#538)

    * fix: typo in description data download docs
    
    * fix: typo in description data download docs and README
    
    * fix: typo in description data download docs and README
    Romazes authored Jan 10, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    17556ea View commit details
  3. Fix: backtest is aborted in the middle with half of report (#539)

    * feat: compare backtest result when run it in cloud
    
    * feat: use delay between is_backtest_done validation
    
    * refactor: more performance in is_backtest_done
    
    * refactor: more readable and reset counter in is_backtest_done
    
    * Update cloud_runner.py
    
    ---------
    
    Co-authored-by: Martin-Molinero <[email protected]>
    Romazes and Martin-Molinero authored Jan 10, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    ff59030 View commit details

Commits on Jan 30, 2025

  1. First attempt to solve the bug

    Marinovsky committed Jan 30, 2025
    Copy the full SHA
    8115985 View commit details
  2. Merge pull request #542 from Marinovsky/bug-TryFixMaskPassBugWithMacOS

    Switch from maskpass to getpass
    Marinovsky authored Jan 30, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    878d5f2 View commit details

Commits on Feb 3, 2025

  1. Fix mount points conflict causing empty project DLL in research envir…

    …onment (#541)
    
    * chore: exclude .python-version for git tracking
    
    * fix: mount points conflict by LeanCLI and NoteBooks directory
    
    * test: check both mount points available
    
    * chore: add explanation for the test issue
    
    * fix: use Assembly.LoadFrom(<csproj_name>) in Initialize.csx for research to keep consistent with docs
    
    * Minor unit test fix
    
    ---------
    
    Co-authored-by: Jhonathan Abreu <[email protected]>
    efJerryYang and jhonabreul authored Feb 3, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    6c6fa92 View commit details

Commits on Feb 5, 2025

  1. Add config option to define database update frequency (#540)

    * First attempt to solve the bug
    
    * Address suggestions
    
    * Fix failing unit test
    
    * Few tweaks
    
    * Nit changes
    
    * Address requested changes
    
    * Address requested changes
    
    * Nit change
    
    * Nit change
    
    * Address review
    
    ---------
    
    Co-authored-by: Martin Molinero <[email protected]>
    Marinovsky and Martin-Molinero authored Feb 5, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    ba8da2f View commit details

Commits on Feb 13, 2025

  1. Minor fix for new holdings mapping (#549)

    Martin-Molinero authored Feb 13, 2025

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    f1de75d View commit details
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -88,7 +88,7 @@ ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
44 changes: 16 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -150,7 +150,7 @@ Options:
-d, --detach Run the backtest in a detached Docker container and return immediately
--debug [pycharm|ptvsd|debugpy|vsdbg|rider|local-platform]
Enable a certain debugging method (see --help for more information)
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
Update the Lean configuration file to retrieve data from the given historical provider
--ib-user-name TEXT Your Interactive Brokers username
--ib-account TEXT Your Interactive Brokers account id
@@ -188,9 +188,6 @@ Options:
--polygon-api-key TEXT Your Polygon.io API Key
--factset-auth-config-file FILE
The path to the FactSet authentication configuration file
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--alpha-vantage-api-key TEXT Your Alpha Vantage Api Key
--alpha-vantage-price-plan [Free|Plan30|Plan75|Plan150|Plan300|Plan600|Plan1200]
Your Alpha Vantage Premium API Key plan
@@ -353,7 +350,7 @@ Usage: lean cloud live deploy [OPTIONS] PROJECT
Options:
--brokerage [Paper Trading|Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|Bybit|TradeStation|Alpaca]
The brokerage to use
--data-provider-live [QuantConnect|Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|Polygon|IEX|CoinApi|Bybit|TradeStation|Alpaca]
--data-provider-live [QuantConnect|Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|Polygon|CoinApi|Bybit|TradeStation|Alpaca]
The live data provider to use
--ib-user-name TEXT Your Interactive Brokers username
--ib-account TEXT Your Interactive Brokers account id
@@ -442,9 +439,6 @@ Options:
--alpaca-environment [live|paper]
Whether Live or Paper environment should be used
--polygon-api-key TEXT Your Polygon.io API Key
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--coinapi-api-key TEXT Your coinapi.io Api Key
--coinapi-product [Free|Startup|Streamer|Professional|Enterprise]
CoinApi pricing plan (https://www.coinapi.io/market-data-api/pricing)
@@ -819,16 +813,16 @@ _See code: [lean/commands/create_project.py](lean/commands/create_project.py)_

### `lean data download`

Purchase and download data directly from QuantConnect or download from Support Data Providers
Purchase and download data directly from QuantConnect or download from supported data providers

```
Usage: lean data download [OPTIONS]
Purchase and download data directly from QuantConnect or download from Support Data Providers
Purchase and download data directly from QuantConnect or download from supported data providers
1. Acquire Data from QuantConnect Datasets: Purchase and seamlessly download data directly from QuantConnect.
2. Streamlined Access from Support Data Providers:
2. Streamlined Access from supported data providers:
- Choose your preferred historical data provider.
@@ -851,7 +845,7 @@ Usage: lean data download [OPTIONS]
https://www.quantconnect.com/datasets
Options:
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
The name of the downloader data provider.
--ib-user-name TEXT Your Interactive Brokers username
--ib-account TEXT Your Interactive Brokers account id
@@ -889,9 +883,6 @@ Options:
--polygon-api-key TEXT Your Polygon.io API Key
--factset-auth-config-file FILE
The path to the FactSet authentication configuration file
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--alpha-vantage-api-key TEXT Your Alpha Vantage Api Key
--alpha-vantage-price-plan [Free|Plan30|Plan75|Plan150|Plan300|Plan600|Plan1200]
Your Alpha Vantage Premium API Key plan
@@ -1292,9 +1283,9 @@ Options:
-d, --detach Run the live deployment in a detached Docker container and return immediately
--brokerage [Paper Trading|Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|Bybit|TradeStation|Alpaca]
The brokerage to use
--data-provider-live [Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|IQFeed|Polygon|IEX|CoinApi|ThetaData|Custom data only|Bybit|TradeStation|Alpaca]
--data-provider-live [Interactive Brokers|Tradier|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Zerodha|Samco|Terminal Link|Trading Technologies|Kraken|CharlesSchwab|IQFeed|Polygon|CoinApi|ThetaData|Custom data only|Bybit|TradeStation|Alpaca]
The live data provider to use
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Bybit|TradeStation|Alpaca]
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Bybit|TradeStation|Alpaca]
Update the Lean configuration file to retrieve data from the given historical provider
--ib-user-name TEXT Your Interactive Brokers username
--ib-account TEXT Your Interactive Brokers account id
@@ -1400,22 +1391,25 @@ Options:
--ib-enable-delayed-streaming-data BOOLEAN
Whether delayed data may be used when your algorithm subscribes to a security you
don't have a market data subscription for (Optional).
--charles-schwab-enable-delayed-streaming-data BOOLEAN
Whether delayed data may be used when your algorithm subscribes to a security you
don't have a market data subscription for (Optional).
--iqfeed-iqconnect TEXT The path to the IQConnect binary
--iqfeed-username TEXT Your IQFeed username
--iqfeed-password TEXT Your IQFeed password
--iqfeed-version TEXT The product version of your IQFeed developer account
--iqfeed-host TEXT The IQFeed host address (Optional).
--polygon-api-key TEXT Your Polygon.io API Key
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--coinapi-api-key TEXT Your coinapi.io Api Key
--coinapi-product [Free|Startup|Streamer|Professional|Enterprise]
CoinApi pricing plan (https://www.coinapi.io/market-data-api/pricing)
--thetadata-ws-url TEXT The ThetaData host address (Optional).
--thetadata-rest-url TEXT The ThetaData host address (Optional).
--thetadata-subscription-plan [Free|Value|Standard|Pro]
Your ThetaData subscription price plan
--trade-station-enable-delayed-streaming-data BOOLEAN
Whether delayed data may be used when your algorithm subscribes to a security you
don't have a market data subscription for (Optional).
--alpaca-api-key TEXT Your Alpaca Api Key
--alpaca-api-secret TEXT Your Alpaca Api Secret
--factset-auth-config-file FILE
@@ -1728,7 +1722,7 @@ Options:
--parameter <TEXT FLOAT FLOAT FLOAT>...
The 'parameter min max step' pairs configuring the parameters to optimize
--constraint TEXT The 'statistic operator value' pairs configuring the constraints of the optimization
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
Update the Lean configuration file to retrieve data from the given historical provider
--download-data Update the Lean configuration file to download data from the QuantConnect API, alias
for --data-provider-historical QuantConnect
@@ -1779,9 +1773,6 @@ Options:
--polygon-api-key TEXT Your Polygon.io API Key
--factset-auth-config-file FILE
The path to the FactSet authentication configuration file
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--alpha-vantage-api-key TEXT Your Alpha Vantage Api Key
--alpha-vantage-price-plan [Free|Plan30|Plan75|Plan150|Plan300|Plan600|Plan1200]
Your Alpha Vantage Premium API Key plan
@@ -1989,7 +1980,7 @@ Usage: lean research [OPTIONS] PROJECT
Options:
--port INTEGER The port to run Jupyter Lab on (defaults to 8888)
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|IEX|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
--data-provider-historical [Interactive Brokers|Oanda|Bitfinex|Coinbase Advanced Trade|Binance|Kraken|CharlesSchwab|IQFeed|Polygon|FactSet|AlphaVantage|CoinApi|ThetaData|QuantConnect|Local|Terminal Link|Bybit|TradeStation|Alpaca]
Update the Lean configuration file to retrieve data from the given historical provider
--ib-user-name TEXT Your Interactive Brokers username
--ib-account TEXT Your Interactive Brokers account id
@@ -2027,9 +2018,6 @@ Options:
--polygon-api-key TEXT Your Polygon.io API Key
--factset-auth-config-file FILE
The path to the FactSet authentication configuration file
--iex-cloud-api-key TEXT Your iexcloud.io API token publishable key
--iex-price-plan [Launch|Grow|Enterprise]
Your IEX Cloud Price plan
--alpha-vantage-api-key TEXT Your Alpha Vantage Api Key
--alpha-vantage-price-plan [Free|Plan30|Plan75|Plan150|Plan300|Plan600|Plan1200]
Your Alpha Vantage Premium API Key plan
1 change: 1 addition & 0 deletions lean/commands/cloud/live/deploy.py
Original file line number Diff line number Diff line change
@@ -240,6 +240,7 @@ def deploy(project: str,

live_data_provider_settings = {}
lean_config = container.lean_config_manager.get_lean_config()
lean_config["project-id"] = cloud_project.projectId

if brokerage is not None:
ensure_options(["brokerage", "node", "auto_restart", "notify_order_events", "notify_insights"])
4 changes: 2 additions & 2 deletions lean/commands/data/download.py
Original file line number Diff line number Diff line change
@@ -576,10 +576,10 @@ def download(ctx: Context,
update: bool,
no_update: bool,
**kwargs) -> None:
"""Purchase and download data directly from QuantConnect or download from Support Data Providers
"""Purchase and download data directly from QuantConnect or download from supported data providers
1. Acquire Data from QuantConnect Datasets: Purchase and seamlessly download data directly from QuantConnect.\n
2. Streamlined Access from Support Data Providers:\n
2. Streamlined Access from supported data providers:\n
- Choose your preferred historical data provider.\n
- Initiate hassle-free downloads from our supported providers.
14 changes: 8 additions & 6 deletions lean/commands/research.py
Original file line number Diff line number Diff line change
@@ -149,6 +149,14 @@ def research(project: Path,
research_image,
paths_to_mount)

# Mount project dir to the Notebooks directory first, avoid using volumes to prevent overwriting mounting logic for /LeanCLI
run_options["mounts"].append(Mount(
target=f"{LEAN_ROOT_PATH}/Notebooks",
source=str(project),
type="bind",
read_only=False
))

# Mount the config in the notebooks directory as well
local_config_path = next(m["Source"] for m in run_options["mounts"] if m["Target"].endswith("config.json"))
run_options["mounts"].append(Mount(target=f"{LEAN_ROOT_PATH}/Notebooks/config.json",
@@ -166,12 +174,6 @@ def research(project: Path,
# Make Ctrl+C stop Jupyter Lab immediately
run_options["stop_signal"] = "SIGKILL"

# Mount the project to the notebooks directory
run_options["volumes"][str(project)] = {
"bind": f"{LEAN_ROOT_PATH}/Notebooks",
"mode": "rw"
}

# Allow notebooks to be embedded in iframes
run_options["commands"].append("mkdir -p ~/.jupyter")
run_options["commands"].append(
40 changes: 39 additions & 1 deletion lean/components/cloud/cloud_runner.py
Original file line number Diff line number Diff line change
@@ -36,6 +36,44 @@ def __init__(self, logger: Logger, api_client: APIClient, task_manager: TaskMana
self._logger = logger
self._api_client = api_client
self._task_manager = task_manager
self._mismatch_counter = 0

def is_backtest_done(self, backtest_data: QCBacktest, delay: float = 10.0):
"""Checks if the backtest is complete.
:param backtest_data: The current state of the backtest.
:param delay: The delay in seconds between consecutive checks. Default is 60 seconds (1 minute).
:return: True if the backtest is complete and the state has changed, False otherwise.
"""
try:
if backtest_data.error or backtest_data.stacktrace:
self._mismatch_counter = 0
return True

is_complete = backtest_data.is_complete()
self._logger.debug(f"[Backtest ID: {backtest_data.backtestId}] Completion status: {is_complete}")

if is_complete:
if backtest_data.totalPerformance:
self._mismatch_counter = 0
return True

if self._mismatch_counter >= 6:
self._logger.error(f"[Backtest ID: {backtest_data.backtestId}] We could not retrieve "
f"the complete backtest results, please try again later.")
self._mismatch_counter = 0
return True

self._mismatch_counter += 1
self._logger.debug(f"[Backtest ID: {backtest_data.backtestId}] Incremented mismatch counter to "
f"{self._mismatch_counter}. Will re-check after {delay} seconds.")
import time
time.sleep(delay)

return False
except Exception as e:
self._logger.error(f"Error checking backtest completion status for ID {backtest_data.backtestId}: {e}")
raise

def run_backtest(self, project: QCProject, name: str) -> QCBacktest:
"""Runs a backtest in the cloud.
@@ -53,7 +91,7 @@ def run_backtest(self, project: QCProject, name: str) -> QCBacktest:
try:
return self._task_manager.poll(
make_request=lambda: self._api_client.backtests.get(project.projectId, created_backtest.backtestId),
is_done=lambda data: data.is_complete(),
is_done=self.is_backtest_done,
get_progress=lambda data: data.progress
)
except KeyboardInterrupt as e:
Loading