kenneth/Auto-Trader
1
0 Fork 0
Code Issues Pull Requests Actions 16 Packages Projects Releases Wiki Activity

fix: complete October 11-12 maintenance and research updates

Browse Source 
Fix live option guards, grounding operand evidence, source-bound benchmarks, trade artifacts, attribution, drawdowns, valuation tolerances, and SEC point-in-time factor inputs.

Add Brazil research data and market support, 13 fundamental and valuation factors, channel progress forwarding, and opt-in ZimaOS templates. Update seven README News sections with three visible dates.

Validation: 17,635 backend tests passed; 1,084 final targeted tests and 103 frontend tests passed. Frontend build, package build, documentation checks, and CI gates passed.
This commit is contained in:
Haozhe Wu committed 2026-10-12 02:29:46 +08:00
1 parent 8e430079fd
commit d3c76819c3
143 files changed
+6746 -436
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.md
+42
View File
@@ -5,6 +5,48 @@ This project adheres to [Semantic Versioning](https://semver.org/).
## [Unreleased] ## [Unreleased]
### Maintenance — 2026-10-12
- Preserve the original opening fill after scale-ins (#1780), use the book
immediately before terminal liquidation for Brinson attribution (#1782),
and start drawdown duration at the latest visit to the peak (#1784).
Three-statement projections honor the caller's balance tolerance throughout
the model (#1786); existing balance checks remain enforced.
- Add ten SEC fundamental factors and three A-share valuation factors
(`val:pb`, `val:ps_ttm`, `val:dv_ttm`) via Tushare daily_basic (#1787, #1788,
#1789). The registry now has 475 factors, 17 in the fundamental family.
Date-only SEC filings become usable one bar later on the IC path (#1790,
#1794). Prior IC measurements using same-day filings need to be rerun.
XOM keeps its current SEC identity while pre-reorganization filings extend
its history. Basic shares never substitute for missing diluted shares.
Price-times-diluted-shares ratios are valuation proxies, not reported market
capitalization. Missing input data remains missing.
- Add Brazil `.SA`/BRL routing, an equity engine, local B3/CVM/BCB/Tesouro
ingestion and the `get_br_market_data` tool, plus market selection and
labels in all nine UI languages (#1783). COTAHIST bars remain raw;
CVM statements contain latest revisions, not point-in-time evidence.
ZimaOS Compose and a fork publishing template require explicit setup;
the upstream repository gains no automatic publishing schedule.
- Forward active attempt status, tool hints and opt-in reasoning to IM
channels, including Feishu CardKit (#1792). Final answers still pass
grounding before delivery; draft text and tool-result payloads stay private.
- Complete write-target helper extraction with actual loop delegation (#1791,
#1624), preserving runtime model provenance. Clarify that connector profiles,
not the legacy config.readonly field, govern order access (#1793).
### Maintenance — 2026-10-11
- Reject unsupported live options before broker dispatch (#1773). This is a
guard fix; live options execution is not enabled.
- Require evidence for every data operand in derived figures (#1622).
Chosen multipliers/divisors need an explicit parameter declaration and
matching visible label; arithmetic checks remain enforced.
- Keep explicit offline/source-bound benchmarks on their selected loader and
report unresolved benchmarks (#1781). Distinguish entry/exit events so
break-even closes remain in trade artifacts and downstream analysis (#1779).
- Preserve optional Codex tool arguments with explicit non-strict schemas
(#1785).
### Maintenance — 2026-10-10 ### Maintenance — 2026-10-10
- Slack and Discord gain localized connection forms and read-only credential - Slack and Discord gain localized connection forms and read-only credential
README.md
+29 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **Security warning:** The X account `VibeTrading_HKU`, Virtuals project `101845`, and token contract `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` are not official Vibe-Trading assets. We have never launched or endorsed any token or memecoin. Do not buy, connect a wallet, or sign anything. [Details](SECURITY.md#official-channels--impersonation). > ⚠️ **Security warning:** The X account `VibeTrading_HKU`, Virtuals project `101845`, and token contract `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` are not official Vibe-Trading assets. We have never launched or endorsed any token or memecoin. Do not buy, connect a wallet, or sign anything. [Details](SECURITY.md#official-channels--impersonation).
- **2026-10-12** 🧮 **Research and backtest updates**: Correct opening fills, liquidation-time attribution, drawdown duration, valuation tolerances and SEC filing timing in factor IC ([#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)). Add 13 fundamental/valuation factors, Brazil data and backtests, opt-in ZimaOS templates, and Feishu run progress. See [CHANGELOG](CHANGELOG.md) for scope and data limitations.
- **2026-10-11** 🛠️ **Five priority fixes**: Derived figures require evidence for every data operand; unsupported live options stay blocked. Offline benchmarks retain their selected source, trade artifacts distinguish break-even exits, and Codex tools preserve optional arguments ([#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622), [#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779), [#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781), [#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)).
- **2026-10-10** 🛠️ **Slack/Discord setup and reproducible research**: Localized channel forms verify bot credentials without saving changes ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). Portfolio research can reuse an immutable snapshot, and Web exports its sanitized JSON ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Indicators report their date boundary and exact input fingerprint; corrupted session records recover without rewriting files. Memory recall, credential redaction and trade-journal detection are corrected. See [CHANGELOG](CHANGELOG.md). - **2026-10-10** 🛠️ **Slack/Discord setup and reproducible research**: Localized channel forms verify bot credentials without saving changes ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). Portfolio research can reuse an immutable snapshot, and Web exports its sanitized JSON ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Indicators report their date boundary and exact input fingerprint; corrupted session records recover without rewriting files. Memory recall, credential redaction and trade-journal detection are corrected. See [CHANGELOG](CHANGELOG.md).
<details>
<summary>Earlier news</summary>
- **2026-10-09** 🛠️ **Channel updates and backtest correctness**: Channel settings preserve embedded URL credentials and authenticated WeChat endpoints during in-place updates, with save/reset feedback in all nine languages ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). Cross-market option volatility uses only timestamps known at each bar; drawdown includes starting capital. Crypto/precious-metal routing, HK duplicate detection, cache recovery, scientific-notation amounts and missing-equity validation are corrected. mootdx leaves the default A-share chain. See [CHANGELOG](CHANGELOG.md) for all thirteen PRs. - **2026-10-09** 🛠️ **Channel updates and backtest correctness**: Channel settings preserve embedded URL credentials and authenticated WeChat endpoints during in-place updates, with save/reset feedback in all nine languages ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). Cross-market option volatility uses only timestamps known at each bar; drawdown includes starting capital. Crypto/precious-metal routing, HK duplicate detection, cache recovery, scientific-notation amounts and missing-equity validation are corrected. mootdx leaves the default A-share chain. See [CHANGELOG](CHANGELOG.md) for all thirteen PRs.
- **2026-10-08** 🔐 **Protected PDF reports and clearer broker reads**: Scheduled Email PDFs support optional AES-256 password protection across Web, CLI playbooks and agent confirmations; the password stays in private channel configuration ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS queries all three venue buckets, refreshes an explicitly expired token once, and reports incomplete reads as errors ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Derived formulas keep their arithmetic after descriptive labels while evidence checks remain enforced ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)). - **2026-10-08** 🔐 **Protected PDF reports and clearer broker reads**: Scheduled Email PDFs support optional AES-256 password protection across Web, CLI playbooks and agent confirmations; the password stays in private channel configuration ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS queries all three venue buckets, refreshes an explicitly expired token once, and reports incomplete reads as errors ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Derived formulas keep their arithmetic after descriptive labels while evidence checks remain enforced ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)).
<details>
<summary>Earlier news</summary>
- **2026-10-07** 🛠️ **Research continuity and reliable risk metrics**: Volatile workspace state now travels outside the system prompt, preserving its stable prefix ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Cross-market dates align correctly, missing prices no longer bridge return observations, and options Sortino uses full-sample downside deviation ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Loader fixtures share one OHLCV contract; credit valuation, style exposures, walk-forward folds and audit diagnostics are corrected. - **2026-10-07** 🛠️ **Research continuity and reliable risk metrics**: Volatile workspace state now travels outside the system prompt, preserving its stable prefix ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Cross-market dates align correctly, missing prices no longer bridge return observations, and options Sortino uses full-sample downside deviation ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Loader fixtures share one OHLCV contract; credit valuation, style exposures, walk-forward folds and audit diagnostics are corrected.
- **2026-10-06** 🛠️ **Live controls and report corrections**: Stopping a live runner cancels its current analysis and waits for scheduler cleanup, including startup cancellation and API shutdown; status timestamps use the correct units ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). Emergency cancel/flatten scans skip malformed records without abandoning the remaining book ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); broker schedules are stored separately and writes handle concurrency and partial writes. US equity half-days respect the early closing bell ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Report corrections preserve figures that passed validation, and bounded feedback no longer implies that unlisted figures passed ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)). - **2026-10-06** 🛠️ **Live controls and report corrections**: Stopping a live runner cancels its current analysis and waits for scheduler cleanup, including startup cancellation and API shutdown; status timestamps use the correct units ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). Emergency cancel/flatten scans skip malformed records without abandoning the remaining book ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); broker schedules are stored separately and writes handle concurrency and partial writes. US equity half-days respect the early closing bell ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Report corrections preserve figures that passed validation, and bounded feedback no longer implies that unlisted figures passed ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)).
@@ -376,7 +379,7 @@ It is designed for research, simulation, and backtesting — and, when you choos
| **Run analyst teams** | Multi-agent research reviews for investment, quant, crypto, macro, and risk workflows. | | **Run analyst teams** | Multi-agent research reviews for investment, quant, crypto, macro, and risk workflows. |
| **Put research into IM channels** | Run the same session runtime through WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, and Mochat with CLI, REST, and Web UI controls. | | **Put research into IM channels** | Run the same session runtime through WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, and Mochat with CLI, REST, and Web UI controls. |
| **Ship usable artifacts** | Reports, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, and later research sessions. | | **Ship usable artifacts** | Reports, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, and later research sessions. |
| **Bench a pre-built alpha zoo** | One-line IC + alive/reversed/dead categorisation across 462 alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental) on your universe. | | **Bench a pre-built alpha zoo** | One-line IC + alive/reversed/dead categorisation across 475 alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental) on your universe. |
| **Spot correlation regimes** | An edge-density + hysteresis timeline on the `/correlation` surface showing when markets fuse into one bloc — descriptive risk context, not a signal. Undefined pair correlations keep edge density unknown, so constant-price histories cannot claim a measured zero density. | | **Spot correlation regimes** | An edge-density + hysteresis timeline on the `/correlation` surface showing when markets fuse into one bloc — descriptive risk context, not a signal. Undefined pair correlations keep edge density unknown, so constant-price histories cannot claim a measured zero density. |
Daily correlation and regime requests align each source by its local calendar date, including mixed timezone-aware and timezone-naive histories. Daily correlation and regime requests align each source by its local calendar date, including mixed timezone-aware and timezone-naive histories.
@@ -580,7 +583,7 @@ the wall-clock activity watchdog.
## 📡 Data Sources & Smart Fallback ## 📡 Data Sources & Smart Fallback
One `get_market_data` call, **28 market-data sources**, one of them the optional **QVeris** premium marketplace. Set `source: "auto"` — the loader picks by symbol, then walks a per-market chain ordered by **IP-ban risk**: never-banned public sources first, throttled / key-gated ones last. Zero config, no single point of failure. One `get_market_data` call, **29 market-data sources**, one of them the optional **QVeris** premium marketplace. Set `source: "auto"` — the loader picks by symbol, then walks a per-market chain ordered by **IP-ban risk**: never-banned public sources first, throttled / key-gated ones last. Zero config, no single point of failure.
| Source | Markets | Auth | Role | | Source | Markets | Auth | Role |
|--------|---------|------|------| |--------|---------|------|------|
@@ -603,6 +606,7 @@ One `get_market_data` call, **28 market-data sources**, one of them the optional
| `tickerall` | forex / metals | key + account (read-only) | same broker MT5 feed, **hosted** — no local terminal, any OS (explicit-only, never in auto fallback) | | `tickerall` | forex / metals | key + account (read-only) | same broker MT5 feed, **hosted** — no local terminal, any OS (explicit-only, never in auto fallback) |
| `pykrx` | Korea (KRX: KOSPI/KOSDAQ) | none | daily KOSPI / KOSDAQ bars for `.KS` / `.KQ` (optional `krx` extra) | | `pykrx` | Korea (KRX: KOSPI/KOSDAQ) | none | daily KOSPI / KOSDAQ bars for `.KS` / `.KQ` (optional `krx` extra) |
| `india_broker` | India (NSE/BSE) | broker login | read-only Zerodha / Shoonya / Dhan bars for `.NS` / `.BO` (fallback-chain tail) | | `india_broker` | India (NSE/BSE) | broker login | read-only Zerodha / Shoonya / Dhan bars for `.NS` / `.BO` (fallback-chain tail) |
| `b3_cotahist` | Brazil (B3) | none | official daily B3 cash/odd-lot bars (raw, BRL) from the local store populated by `python -m src.br_data sync`; offline fallback for `.SA` |
| `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix | | `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix |
**Fallback chains (by IP-ban risk):** **Fallback chains (by IP-ban risk):**
@@ -611,6 +615,7 @@ One `get_market_data` call, **28 market-data sources**, one of them the optional
- **US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local` - **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **Korea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local` - **Korea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local`
- **UK (LSE)** → `yahoo` · `yfinance` · `local` *(declared GBP/GBp quotes only)* - **UK (LSE)** → `yahoo` · `yfinance` · `local` *(declared GBP/GBp quotes only)*
- **Crypto** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **Crypto** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -793,9 +798,9 @@ alongside the bundled roster (same-name files override it, like user skills) and
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>462 pre-built quant alphas across 5 families</sub></summary> <summary><b>Alpha Zoo</b> <sub>475 pre-built quant alphas across 5 families</sub></summary>
- 🧬 462 cross-sectional alphas, lookahead-banned at the operator layer - 🧬 475 cross-sectional alphas, lookahead-banned at the operator layer
- 📈 IC + IR + alive/reversed/dead categorisation in one CLI command - 📈 IC + IR + alive/reversed/dead categorisation in one CLI command
- 🔬 AST purity gate + 300-row lookahead sentinel test + `pytest-socket` network kill-switch - 🔬 AST purity gate + 300-row lookahead sentinel test + `pytest-socket` network kill-switch
- 📦 Apache-2 attribution for Qlib; per-zoo `LICENSE.md` declaring formulas as mathematical content - 📦 Apache-2 attribution for Qlib; per-zoo `LICENSE.md` declaring formulas as mathematical content
@@ -814,7 +819,7 @@ Run `vibe-trading alpha list` to browse, `vibe-trading alpha show <id>` for form
</details> </details>
<details> <details>
<summary><b>Backtest Engines</b> <sub>10 engines + options portfolio, cross-market composite</sub></summary> <summary><b>Backtest Engines</b> <sub>11 engines + options portfolio, cross-market composite</sub></summary>
| Engine | Market | Notes | | Engine | Market | Notes |
|--------|--------|-------| |--------|--------|-------|
@@ -823,6 +828,7 @@ Run `vibe-trading alpha list` to browse, `vibe-trading alpha show <id>` for form
| **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack | | **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack |
| **KoreaEquity** | Korea (KRX: KOSPI/KOSDAQ) | long-only, ±30% band judged at execution time on the unified tick grid, 2026 0.20% transaction tax | | **KoreaEquity** | Korea (KRX: KOSPI/KOSDAQ) | long-only, ±30% band judged at execution time on the unified tick grid, 2026 0.20% transaction tax |
| **VietnamEquity** | Vietnam (HOSE) | long-only, T+2 settlement hold, ±7% band on the 10/50/100-VND tick grid, 100-share lots, 0.1% sell-side tax | | **VietnamEquity** | Vietnam (HOSE) | long-only, T+2 settlement hold, ±7% band on the 10/50/100-VND tick grid, 100-share lots, 0.1% sell-side tax |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split | | **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split |
| **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers | | **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers |
| **Forex** | FX / metals | via the `mt5` loader (local terminal) or the hosted `tickerall` loader (no terminal, any OS) | | **Forex** | FX / metals | via the `mt5` loader (local terminal) or the hosted `tickerall` loader (no terminal, any OS) |
@@ -1195,7 +1201,7 @@ The interactive TUI (`vibe-trading`) now uses a terminal-native transcript: a st
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # browse 462 pre-built alphas; show / bench / compare / export-manifest sub-commands available vibe-trading alpha list # browse 475 pre-built alphas; show / bench / compare / export-manifest sub-commands available
vibe-trading playbook list # five scheduled-research templates; show / create sub-commands available vibe-trading playbook list # five scheduled-research templates; show / create sub-commands available
vibe-trading channels status --local # inspect IM channel config and install hints vibe-trading channels status --local # inspect IM channel config and install hints
vibe-trading provider doctor # print redacted provider/proxy/package diagnostics vibe-trading provider doctor # print redacted provider/proxy/package diagnostics
@@ -1339,6 +1345,12 @@ Saves write the `channels.<name>` section of `~/.vibe-trading/agent.json` atomic
### Strategy & Backtesting ### Strategy & Backtesting
New BaseEngine `artifacts/trades.csv` files include an `event` column:
`entry` marks an opening row and `exit` marks a completed trade, including
zero-PnL exits. Validation and strategy-discovery readers use this marker.
Files without it retain legacy loading; their zero-PnL exits remain ambiguous.
The separate options-engine trade format is unchanged.
```bash ```bash
# Moving average crossover on US equities # Moving average crossover on US equities
vibe-trading run -p "Backtest a 20/50-day moving average crossover on AAPL for the past year, show Sharpe ratio and max drawdown" vibe-trading run -p "Backtest a 20/50-day moving average crossover on AAPL for the past year, show Sharpe ratio and max drawdown"
@@ -1701,6 +1713,8 @@ with `--host` / `--port`.
`quantlib_call` attribution requires finite sector weights and returns, finite Carino return inputs, and a finite non-negative weight-sum tolerance. Invalid data is rejected explicitly rather than returned as an attribution result containing `NaN`. Missing returns for a zero-weight sector may still be inferred from the other side. `quantlib_call` attribution requires finite sector weights and returns, finite Carino return inputs, and a finite non-negative weight-sum tolerance. Invalid data is rejected explicitly rather than returned as an attribution result containing `NaN`. Missing returns for a zero-weight sector may still be inferred from the other side.
`quantlib_call` maximum-drawdown analysis measures underwater duration from the latest visit to the peak before the worst trough. An earlier drawdown that already recovered is excluded from that duration.
`quantlib_call` VaR backtests count finite return/forecast pairs for coverage. Missing pairs break the consecutive-observation chain for independence testing; they never create a transition between the observations on either side of a gap. `quantlib_call` VaR backtests count finite return/forecast pairs for coverage. Missing pairs break the consecutive-observation chain for independence testing; they never create a transition between the observations on either side of a gap.
`quantlib_call` option analytics use an absolute **price** tolerance for implied-volatility inversion. A finite result must reprice the quote within that tolerance; the underlying analytics function returns `NaN` for an unconverged or volatility-unidentifiable quote rather than an asserted volatility. `quantlib_call` option analytics use an absolute **price** tolerance for implied-volatility inversion. A finite result must reprice the quote within that tolerance; the underlying analytics function returns `NaN` for an unconverged or volatility-unidentifiable quote rather than an asserted volatility.
@@ -2103,7 +2117,7 @@ Vibe-Trading/
│ │ ├── memory/ # Cross-session persistent memory │ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/) │ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 auto-discovered agent tools │ │ ├── tools/ # 110 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests │ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget) │ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
@@ -2112,7 +2126,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo web search │ │ │ ├── web_search_tool.py # DuckDuckGo web search
│ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc. │ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 462 alphas across 5 families │ │ ├── factors/ # Alpha Zoo — 475 alphas across 5 families
│ │ │ ├── base.py # 19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # AST-only metadata load + lazy compute + sanity gates │ │ │ ├── registry.py # AST-only metadata load + lazy compute + sanity gates
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead categorisation │ │ │ ├── bench_runner.py # IC + alive/reversed/dead categorisation
@@ -2128,8 +2142,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM provider abstraction │ │ └── providers/ # LLM provider abstraction
│ │ │ │
│ └── backtest/ # Backtest engines │ └── backtest/ # Backtest engines
│ ├── engines/ # 9 engines + composite cross-market engine + options_portfolio │ ├── engines/ # 10 engines + composite cross-market engine + options_portfolio
│ ├── loaders/ # 28 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata │ ├── loaders/ # 29 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + auto-fallback chains │ │ └── registry.py # Registry + auto-fallback chains
│ └── optimizers/ # MVO, equal vol, max div, risk parity │ └── optimizers/ # MVO, equal vol, max div, risk parity
@@ -2195,7 +2209,7 @@ Vibe-Trading is part of the **[HKUDS](https://github.com/HKUDS)** agent ecosyste
| **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | Local loader Shipped | | **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | Local loader Shipped |
| **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Analytic payoff/scenario tool **Shipped**; surface/dashboard Planned | | **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Analytic payoff/scenario tool **Shipped**; surface/dashboard Planned |
| **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover-aware optimizer **Shipped 0.1.11**; rest Planned | | **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover-aware optimizer **Shipped 0.1.11**; rest Planned |
| **Alpha Zoo** | 462 pre-built alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental) with one-line bench, agent integration, and Web UI | **Shipped 0.1.8**, extended through 0.1.12 | | **Alpha Zoo** | 475 pre-built alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental) with one-line bench, agent integration, and Web UI | **Shipped 0.1.8**, extended through 0.1.12 |
| **Strategy Development Manager** | Register papers / broker research as factors & strategies with a persistent store + automated IC/Sharpe decay lifecycle | **Shipped 0.1.11** | | **Strategy Development Manager** | Register papers / broker research as factors & strategies with a persistent store + automated IC/Sharpe decay lifecycle | **Shipped 0.1.11** |
| **Correlation Regime** | Edge-density + hysteresis regime timeline layered on `/correlation` — spot when markets fuse into one bloc | **Shipped 0.1.12** | | **Correlation Regime** | Edge-density + hysteresis regime timeline layered on `/correlation` — spot when markets fuse into one bloc | **Shipped 0.1.12** |
| **Research Delivery** | Scheduled briefs and live research sessions through Slack / Telegram / email-style IM channels | Scheduler + IM Runtime Shipped | | **Research Delivery** | Scheduled briefs and live research sessions through Slack / Telegram / email-style IM channels | Scheduler + IM Runtime Shipped |
README_ar.md
+21 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **تحذير أمني:** حساب X باسم `VibeTrading_HKU`، ومشروع Virtuals رقم `101845`، وعقد التوكن `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` كلّها غير رسمية ولا تتبع Vibe-Trading. لم نُطلق أو نؤيد مطلقًا أي توكن أو عملة ميم. لا تشترِ هذا التوكن، ولا تربط محفظتك، ولا توقّع أي شيء. [التفاصيل](SECURITY.md#official-channels--impersonation). > ⚠️ **تحذير أمني:** حساب X باسم `VibeTrading_HKU`، ومشروع Virtuals رقم `101845`، وعقد التوكن `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` كلّها غير رسمية ولا تتبع Vibe-Trading. لم نُطلق أو نؤيد مطلقًا أي توكن أو عملة ميم. لا تشترِ هذا التوكن، ولا تربط محفظتك، ولا توقّع أي شيء. [التفاصيل](SECURITY.md#official-channels--impersonation).
- **2026-10-12** 🧮 **تحديثات البحث والاختبار التاريخي**: تصحيح تنفيذ الافتتاح وإسناد الأداء عند التصفية ومدة التراجع وتسامح التقييم وتوقيت إفصاحات SEC في IC للعوامل ([#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)). إضافة 13 عاملًا أساسيًا وللتقييم، وبيانات البرازيل واختباراتها، وقوالب ZimaOS بتفعيل اختياري، وتقدم التشغيل في Feishu. النطاق والقيود في [CHANGELOG](CHANGELOG.md).
- **2026-10-11** 🛠️ **خمسة إصلاحات ذات أولوية**: تتطلب الأرقام المشتقة دليلًا لكل قيمة مستخدمة، وتبقى أوامر الخيارات الحية غير المدعومة محظورة. تحافظ المعايير دون اتصال على المصدر المحدد، وتميز سجلات التداول الإغلاق بلا ربح أو خسارة، وتحفظ أدوات Codex المعاملات الاختيارية ([#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622)، [#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779)، [#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781)، [#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)).
- **2026-10-10** 🛠️ **إعداد Slack وDiscord وبحث قابل لإعادة الإنتاج**: تتحقق النماذج المترجمة من بيانات اعتماد البوت دون حفظ التغييرات ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)، [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). يعيد بحث المحفظة استخدام لقطات ثابتة، وتصدّر واجهة الويب JSON دون بيانات اعتماد أو أرقام حسابات ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). تذكر المؤشرات حد التاريخ وبصمة المدخلات؛ تُستعاد سجلات الجلسة التالفة دون إعادة كتابة الملفات الأصلية. أُصلح استرجاع الذاكرة وإخفاء بيانات الاعتماد والتعرف على سجل التداول. راجع [CHANGELOG](CHANGELOG.md). - **2026-10-10** 🛠️ **إعداد Slack وDiscord وبحث قابل لإعادة الإنتاج**: تتحقق النماذج المترجمة من بيانات اعتماد البوت دون حفظ التغييرات ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)، [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). يعيد بحث المحفظة استخدام لقطات ثابتة، وتصدّر واجهة الويب JSON دون بيانات اعتماد أو أرقام حسابات ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). تذكر المؤشرات حد التاريخ وبصمة المدخلات؛ تُستعاد سجلات الجلسة التالفة دون إعادة كتابة الملفات الأصلية. أُصلح استرجاع الذاكرة وإخفاء بيانات الاعتماد والتعرف على سجل التداول. راجع [CHANGELOG](CHANGELOG.md).
<details>
<summary>أخبار سابقة</summary>
- **2026-10-09** 🛠️ **إعدادات القنوات ودقة الاختبارات التاريخية**: تحافظ تحديثات الإعدادات دون إعادة الاتصال على بيانات اعتماد الروابط وعنوان خادم WeChat الناتج عن المصادقة، وتعرض الواجهة نتائج الحفظ وإعادة التشغيل بتسع لغات ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)، [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)، [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). يعتمد تقلب الخيارات عبر الأسواق على التوقيتات المعروفة عند كل شمعة فقط، ويشمل التراجع رأس المال الأولي. صُححت مسارات بيانات العملات الرقمية والمعادن الثمينة، وكشف تكرار أسهم هونغ كونغ، واستعادة الذاكرة المؤقتة، والمبالغ بالصيغة العلمية، والتحقق من قيم حقوق الملكية المفقودة. أُزيل mootdx من سلسلة أسهم A الافتراضية. تفاصيل طلبات الدمج الثلاثة عشر في [CHANGELOG](CHANGELOG.md). - **2026-10-09** 🛠️ **إعدادات القنوات ودقة الاختبارات التاريخية**: تحافظ تحديثات الإعدادات دون إعادة الاتصال على بيانات اعتماد الروابط وعنوان خادم WeChat الناتج عن المصادقة، وتعرض الواجهة نتائج الحفظ وإعادة التشغيل بتسع لغات ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)، [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)، [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). يعتمد تقلب الخيارات عبر الأسواق على التوقيتات المعروفة عند كل شمعة فقط، ويشمل التراجع رأس المال الأولي. صُححت مسارات بيانات العملات الرقمية والمعادن الثمينة، وكشف تكرار أسهم هونغ كونغ، واستعادة الذاكرة المؤقتة، والمبالغ بالصيغة العلمية، والتحقق من قيم حقوق الملكية المفقودة. أُزيل mootdx من سلسلة أسهم A الافتراضية. تفاصيل طلبات الدمج الثلاثة عشر في [CHANGELOG](CHANGELOG.md).
- **2026-10-08** 🔐 **حماية تقارير PDF وتصحيح قراءات الوسيط**: تدعم تقارير البريد المجدولة حماية اختيارية بكلمة مرور عبر AES-256 في الويب وCLI وتأكيدات الوكيل؛ تُحفظ كلمة المرور في إعدادات القناة الخاصة فقط ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). يستعلم KIS عن مسارات التداول الثلاثة ويجدد الرمز المنتهي صراحةً مرة واحدة ويرفض القراءة غير المكتملة ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). تُحفظ الصيغة الحسابية بعد الوصف مع استمرار التحقق من الأدلة ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)). - **2026-10-08** 🔐 **حماية تقارير PDF وتصحيح قراءات الوسيط**: تدعم تقارير البريد المجدولة حماية اختيارية بكلمة مرور عبر AES-256 في الويب وCLI وتأكيدات الوكيل؛ تُحفظ كلمة المرور في إعدادات القناة الخاصة فقط ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). يستعلم KIS عن مسارات التداول الثلاثة ويجدد الرمز المنتهي صراحةً مرة واحدة ويرفض القراءة غير المكتملة ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). تُحفظ الصيغة الحسابية بعد الوصف مع استمرار التحقق من الأدلة ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)).
<details>
<summary>أخبار سابقة</summary>
- **2026-10-07** 🛠️ **استمرارية البحث وتصحيح مقاييس المخاطر**: نُقلت حالة التشغيل خارج تعليمات النظام للحفاظ على بادئة ثابتة ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). صُححت محاذاة التواريخ بين الأسواق وعوائد الأسعار المفقودة وحساب Sortino للخيارات باستخدام العينة كاملةً ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)، [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)، [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). تشترك اختبارات مصادر البيانات في عقد OHLCV واحد، مع إصلاحات لتقييم الائتمان والتعرضات والتحقق الزمني وتشخيص التدقيق. - **2026-10-07** 🛠️ **استمرارية البحث وتصحيح مقاييس المخاطر**: نُقلت حالة التشغيل خارج تعليمات النظام للحفاظ على بادئة ثابتة ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). صُححت محاذاة التواريخ بين الأسواق وعوائد الأسعار المفقودة وحساب Sortino للخيارات باستخدام العينة كاملةً ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)، [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)، [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). تشترك اختبارات مصادر البيانات في عقد OHLCV واحد، مع إصلاحات لتقييم الائتمان والتعرضات والتحقق الزمني وتشخيص التدقيق.
- **2026-10-06** 🛠️ **إصلاح التحكم بالتداول وتصحيح التقارير**: يلغي إيقاف مشغّل التداول الفعلي التحليل الجاري وينتظر انتهاء المجدول، بما يشمل الإلغاء أثناء بدء التشغيل وإغلاق API، مع تصحيح وحدات الوقت في الحالة ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). يتجاوز إلغاء الأوامر وإغلاق المراكز في الطوارئ السجلات غير الصالحة ويواصل معالجة الباقي ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)). تُحفظ جداول الوسطاء بصورة منفصلة مع معالجة الكتابات المتزامنة والجزئية. تُراعى ساعات الإغلاق المبكر للأسهم الأمريكية ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). تحافظ تصحيحات التقارير على الأرقام التي اجتازت التحقق، ولا توحي الملاحظات المحدودة بأن الأرقام غير المذكورة قد اجتازته ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)). - **2026-10-06** 🛠️ **إصلاح التحكم بالتداول وتصحيح التقارير**: يلغي إيقاف مشغّل التداول الفعلي التحليل الجاري وينتظر انتهاء المجدول، بما يشمل الإلغاء أثناء بدء التشغيل وإغلاق API، مع تصحيح وحدات الوقت في الحالة ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). يتجاوز إلغاء الأوامر وإغلاق المراكز في الطوارئ السجلات غير الصالحة ويواصل معالجة الباقي ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)). تُحفظ جداول الوسطاء بصورة منفصلة مع معالجة الكتابات المتزامنة والجزئية. تُراعى ساعات الإغلاق المبكر للأسهم الأمريكية ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). تحافظ تصحيحات التقارير على الأرقام التي اجتازت التحقق، ولا توحي الملاحظات المحدودة بأن الأرقام غير المذكورة قد اجتازته ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)).
@@ -377,7 +380,7 @@ Vibe-Trading مساحة عمل بحثية مفتوحة المصدر تحول ا
| **تشغيل فرق محللين** | مراجعات بحث متعددة الوكلاء لتدفقات الاستثمار والكم والكريبتو والماكرو والمخاطر. | | **تشغيل فرق محللين** | مراجعات بحث متعددة الوكلاء لتدفقات الاستثمار والكم والكريبتو والماكرو والمخاطر. |
| **وصل الأبحاث بقنوات IM** | إدارة بيئة جلسة واحدة عبر WebSocket وTelegram وSlack وDiscord وMatrix وWhatsApp وSignal وQQ/NapCat وWeChat/WeCom وFeishu/Lark وDingTalk وTeams وemail وMochat من CLI وREST وWeb UI. | | **وصل الأبحاث بقنوات IM** | إدارة بيئة جلسة واحدة عبر WebSocket وTelegram وSlack وDiscord وMatrix وWhatsApp وSignal وQQ/NapCat وWeChat/WeCom وFeishu/Lark وDingTalk وTeams وemail وMochat من CLI وREST وWeb UI. |
| **إنتاج مخرجات قابلة للاستخدام** | تقارير، وTradingView Pine Script، وTDX، وMetaTrader 5، وأدوات MCP، وجلسات بحث لاحقة. | | **إنتاج مخرجات قابلة للاستخدام** | تقارير، وTradingView Pine Script، وTDX، وMetaTrader 5، وأدوات MCP، وجلسات بحث لاحقة. |
| **bench ألفا zoo جاهزة** | تشغيل IC + IR + تصنيف alive/reversed/dead عبر 462 ألفا (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental) بسطر أوامر واحد على universe الخاص بك. | | **bench ألفا zoo جاهزة** | تشغيل IC + IR + تصنيف alive/reversed/dead عبر 475 ألفا (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental) بسطر أوامر واحد على universe الخاص بك. |
| **رصد أنظمة الارتباط** | جدول زمني قائم على كثافة الحواف + التباطؤ (hysteresis) على واجهة `/correlation` يُظهر متى تندمج الأسواق في كتلة واحدة — سياق مخاطر وصفي، لا إشارة تداول. | | **رصد أنظمة الارتباط** | جدول زمني قائم على كثافة الحواف + التباطؤ (hysteresis) على واجهة `/correlation` يُظهر متى تندمج الأسواق في كتلة واحدة — سياق مخاطر وصفي، لا إشارة تداول. |
--- ---
@@ -461,7 +464,7 @@ vibe-trading connector install /tmp/my-broker
## 📡 مصادر البيانات والتراجع الذكي ## 📡 مصادر البيانات والتراجع الذكي
استدعاء واحد لـ `get_market_data`، **28 مصدر بيانات سوقية**، أحدها سوق **QVeris** المدفوع الاختياري (إضافة إلى سوق مدفوع اختياري **QVeris**). اضبط `source: "auto"` — يختار المُحمّل حسب الرمز، ثم يسير عبر سلسلة لكل سوق مرتبة بحسب **خطر حظر عنوان IP**: المصادر العامة التي لا تُحظر أبداً أولاً، والمصادر المُقيّدة أو المحمية بمفتاح أخيراً. بلا أي إعداد، ولا نقطة فشل واحدة. استدعاء واحد لـ `get_market_data`، **29 مصدر بيانات سوقية**، أحدها سوق **QVeris** المدفوع الاختياري (إضافة إلى سوق مدفوع اختياري **QVeris**). اضبط `source: "auto"` — يختار المُحمّل حسب الرمز، ثم يسير عبر سلسلة لكل سوق مرتبة بحسب **خطر حظر عنوان IP**: المصادر العامة التي لا تُحظر أبداً أولاً، والمصادر المُقيّدة أو المحمية بمفتاح أخيراً. بلا أي إعداد، ولا نقطة فشل واحدة.
| Source | Markets | Auth | Role | | Source | Markets | Auth | Role |
|--------|---------|------|------| |--------|---------|------|------|
@@ -484,6 +487,7 @@ vibe-trading connector install /tmp/my-broker
| `tickerall` | الفوركس / المعادن | مفتاح + حساب (للقراءة فقط) | تغذية MT5 لنفس الوسيط لكن **مستضافة** — دون طرفية محلية وعلى أي نظام تشغيل (بالاختيار الصريح فقط، ولا تدخل سلسلة auto أبدًا) | | `tickerall` | الفوركس / المعادن | مفتاح + حساب (للقراءة فقط) | تغذية MT5 لنفس الوسيط لكن **مستضافة** — دون طرفية محلية وعلى أي نظام تشغيل (بالاختيار الصريح فقط، ولا تدخل سلسلة auto أبدًا) |
| `pykrx` | كوريا (KRX: KOSPI/KOSDAQ) | لا شيء | أشرطة يومية لـ KOSPI / KOSDAQ لرموز `.KS` / `.KQ` (إضافة `krx` اختيارية) | | `pykrx` | كوريا (KRX: KOSPI/KOSDAQ) | لا شيء | أشرطة يومية لـ KOSPI / KOSDAQ لرموز `.KS` / `.KQ` (إضافة `krx` اختيارية) |
| `india_broker` | الهند (NSE/BSE) | تسجيل دخول الوسيط | قراءة فقط لأشرطة Zerodha / Shoonya / Dhan لرموز `.NS` / `.BO` (ذيل سلسلة التراجع) | | `india_broker` | الهند (NSE/BSE) | تسجيل دخول الوسيط | قراءة فقط لأشرطة Zerodha / Shoonya / Dhan لرموز `.NS` / `.BO` (ذيل سلسلة التراجع) |
| `b3_cotahist` | Brazil (B3) | none | شموع B3 اليومية المحلية للسوق النقدي والكسور (خام، BRL)، عبر `python -m src.br_data sync`؛ مصدر بديل دون اتصال لـ `.SA` |
| `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix | | `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix |
**سلاسل التراجع (بحسب خطر حظر عنوان IP):** **سلاسل التراجع (بحسب خطر حظر عنوان IP):**
@@ -492,6 +496,7 @@ vibe-trading connector install /tmp/my-broker
- **أسهم US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **أسهم US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **أسهم HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **أسهم HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **أسهم الهند (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local` - **أسهم الهند (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **كوريا (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local` - **كوريا (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local`
- **المملكة المتحدة (LSE)** → `yahoo` · `yfinance` · `local` *(الأسعار المعلنة بـ GBP/GBp فقط)* - **المملكة المتحدة (LSE)** → `yahoo` · `yfinance` · `local` *(الأسعار المعلنة بـ GBP/GBp فقط)*
- **الكريبتو** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **الكريبتو** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -662,9 +667,9 @@ LONGBRIDGE_ACCESS_TOKEN=...
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>462 ألفا كمّي جاهز عبر 5 families</sub></summary> <summary><b>Alpha Zoo</b> <sub>475 ألفا كمّي جاهز عبر 5 families</sub></summary>
- 🧬 462 ألفا cross-sectional، مع منع lookahead على طبقة العوامل (operators) - 🧬 475 ألفا cross-sectional، مع منع lookahead على طبقة العوامل (operators)
- 📈 IC + IR + تصنيف alive/reversed/dead بأمر CLI واحد - 📈 IC + IR + تصنيف alive/reversed/dead بأمر CLI واحد
- 🔬 بوابة نقاء AST + اختبار حماية lookahead بـ 300 صف + قاطع شبكة عبر `pytest-socket` - 🔬 بوابة نقاء AST + اختبار حماية lookahead بـ 300 صف + قاطع شبكة عبر `pytest-socket`
- 📦 إسناد Apache-2 لـ Qlib؛ ملف `LICENSE.md` لكل zoo يصرّح بأن الصيغ محتوى رياضي - 📦 إسناد Apache-2 لـ Qlib؛ ملف `LICENSE.md` لكل zoo يصرّح بأن الصيغ محتوى رياضي
@@ -683,7 +688,7 @@ LONGBRIDGE_ACCESS_TOKEN=...
</details> </details>
<details> <details>
<summary><b>محرّكات الاختبار الرجعي</b> <sub>10 محرّكات + محفظة خيارات، ومركّب عبر الأسواق</sub></summary> <summary><b>محرّكات الاختبار الرجعي</b> <sub>11 محرّكات + محفظة خيارات، ومركّب عبر الأسواق</sub></summary>
| المحرّك | السوق | ملاحظات | | المحرّك | السوق | ملاحظات |
|--------|-------|---------| |--------|-------|---------|
@@ -692,6 +697,7 @@ LONGBRIDGE_ACCESS_TOKEN=...
| **IndiaEquity** | الهند (NSE/BSE) | T+1، ونطاقات القاطع (circuit)، وحزمة تكاليف STT / الدمغة / SEBI / GST قابلة للتهيئة | | **IndiaEquity** | الهند (NSE/BSE) | T+1، ونطاقات القاطع (circuit)، وحزمة تكاليف STT / الدمغة / SEBI / GST قابلة للتهيئة |
| **KoreaEquity** | كوريا (KRX: KOSPI/KOSDAQ) | شراء فقط، ونطاق ±30% يُحكم عليه لحظة التنفيذ على شبكة الخطوة السعرية الموحّدة، وضريبة تداول 0.20% لعام 2026 | | **KoreaEquity** | كوريا (KRX: KOSPI/KOSDAQ) | شراء فقط، ونطاق ±30% يُحكم عليه لحظة التنفيذ على شبكة الخطوة السعرية الموحّدة، وضريبة تداول 0.20% لعام 2026 |
| **VietnamEquity** | فيتنام (HOSE) | شراء فقط، واحتجاز تسوية T+2، ونطاق ±7% على شبكة الخطوة السعرية 10/50/100 دونغ، ولوت 100 سهم، وضريبة 0.1% على البيع | | **VietnamEquity** | فيتنام (HOSE) | شراء فقط، واحتجاز تسوية T+2، ونطاق ±7% على شبكة الخطوة السعرية 10/50/100 دونغ، ولوت 100 سهم، وضريبة 0.1% على البيع |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | crypto فوري / عقود USD-M الدائمة | تسويات التمويل، وفصل سعر التنفيذ عن سعر العلامة | | **Crypto** | crypto فوري / عقود USD-M الدائمة | تسويات التمويل، وفصل سعر التنفيذ عن سعر العلامة |
| **ChinaFutures** · **GlobalFutures** | العقود الآجلة | الهامش، ومضاعِفات العقد | | **ChinaFutures** · **GlobalFutures** | العقود الآجلة | الهامش، ومضاعِفات العقد |
| **Forex** | FX / المعادن | عبر مُحمّل `mt5` | | **Forex** | FX / المعادن | عبر مُحمّل `mt5` |
@@ -971,7 +977,7 @@ Vibe-Trading وكيل كثيف الأدوات؛ المهارات والاختب
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # استعرض 462 ألفا جاهز؛ متاح show / bench / compare / export-manifest vibe-trading alpha list # استعرض 475 ألفا جاهز؛ متاح show / bench / compare / export-manifest
vibe-trading playbook list # خمسة قوالب بحث مجدولة؛ متاح show / create vibe-trading playbook list # خمسة قوالب بحث مجدولة؛ متاح show / create
vibe-trading channels status --local # فحص إعدادات قنوات IM وتلميحات التثبيت vibe-trading channels status --local # فحص إعدادات قنوات IM وتلميحات التثبيت
vibe-trading provider doctor # طباعة تشخيصات المزود/الوكيل/الحزم بعد إخفاء الأسرار vibe-trading provider doctor # طباعة تشخيصات المزود/الوكيل/الحزم بعد إخفاء الأسرار
@@ -1716,7 +1722,7 @@ Vibe-Trading/
│ │ ├── memory/ # Cross-session persistent memory │ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/) │ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 auto-discovered agent tools │ │ ├── tools/ # 110 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests │ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget) │ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
@@ -1725,7 +1731,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo web search │ │ │ ├── web_search_tool.py # DuckDuckGo web search
│ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc. │ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 462 ألفا عبر 5 families │ │ ├── factors/ # Alpha Zoo — 475 ألفا عبر 5 families
│ │ │ ├── base.py # 19 عاملاً (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19 عاملاً (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # تحميل بيانات وصفية AST فقط + حساب كسول + بوابات سلامة │ │ │ ├── registry.py # تحميل بيانات وصفية AST فقط + حساب كسول + بوابات سلامة
│ │ │ ├── bench_runner.py # IC + تصنيف alive/reversed/dead │ │ │ ├── bench_runner.py # IC + تصنيف alive/reversed/dead
@@ -1741,8 +1747,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM provider abstraction │ │ └── providers/ # LLM provider abstraction
│ │ │ │
│ └── backtest/ # Backtest engines │ └── backtest/ # Backtest engines
│ ├── engines/ # 9 engines + composite cross-market engine + options_portfolio │ ├── engines/ # 10 engines + composite cross-market engine + options_portfolio
│ ├── loaders/ # 28 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata │ ├── loaders/ # 29 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + auto-fallback chains │ │ └── registry.py # Registry + auto-fallback chains
│ └── optimizers/ # MVO, equal vol, max div, risk parity │ └── optimizers/ # MVO, equal vol, max div, risk parity
@@ -1808,7 +1814,7 @@ Vibe-Trading جزء من نظام وكلاء **[HKUDS](https://github.com/HKUDS)
| **Data Bridge** | أحضر بياناتك: موصلات CSV/Parquet/SQL محلية مع schema mapping | المُحمِّل المحلي شُحن | | **Data Bridge** | أحضر بياناتك: موصلات CSV/Parquet/SQL محلية مع schema mapping | المُحمِّل المحلي شُحن |
| **Options Lab** | سطح تقلب، ولوحة Greeks، ومستكشف payoff/scenario | مخطط | | **Options Lab** | سطح تقلب، ولوحة Greeks، ومستكشف payoff/scenario | مخطط |
| **Portfolio Studio** | أشعة مخاطر، وقيود، ومحسن يراعي الدوران، وملاحظات إعادة توازن | محسن يراعي الدوران **تم الإطلاق 0.1.11**؛ الباقي مخطط | | **Portfolio Studio** | أشعة مخاطر، وقيود، ومحسن يراعي الدوران، وملاحظات إعادة توازن | محسن يراعي الدوران **تم الإطلاق 0.1.11**؛ الباقي مخطط |
| **Alpha Zoo** | 462 ألفا كمّي جاهز (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental)، سطر أوامر واحد للـ bench، تكامل agent، وواجهة Web | **تم الإطلاق 0.1.8**، موسّع حتى 0.1.12 | | **Alpha Zoo** | 475 ألفا كمّي جاهز (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental)، سطر أوامر واحد للـ bench، تكامل agent، وواجهة Web | **تم الإطلاق 0.1.8**، موسّع حتى 0.1.12 |
| **Strategy Development Manager** | تسجيل الأوراق البحثية / أبحاث الوسطاء كعوامل واستراتيجيات مع مخزن دائم + دورة حياة آلية لاضمحلال IC/Sharpe | **تم الإطلاق 0.1.11** | | **Strategy Development Manager** | تسجيل الأوراق البحثية / أبحاث الوسطاء كعوامل واستراتيجيات مع مخزن دائم + دورة حياة آلية لاضمحلال IC/Sharpe | **تم الإطلاق 0.1.11** |
| **Correlation Regime** | جدول زمني لنظام الارتباط قائم على كثافة الحواف + التباطؤ (hysteresis) فوق `/correlation` — رصد متى تندمج الأسواق في كتلة واحدة | **تم الإطلاق 0.1.12** | | **Correlation Regime** | جدول زمني لنظام الارتباط قائم على كثافة الحواف + التباطؤ (hysteresis) فوق `/correlation` — رصد متى تندمج الأسواق في كتلة واحدة | **تم الإطلاق 0.1.12** |
| **Research Delivery** | موجزات مجدولة وجلسات بحث حي عبر Slack / Telegram / قنوات IM شبيهة بالبريد | المُجدوِل + IM Runtime شُحنا | | **Research Delivery** | موجزات مجدولة وجلسات بحث حي عبر Slack / Telegram / قنوات IM شبيهة بالبريد | المُجدوِل + IM Runtime شُحنا |
README_es.md
+21 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **Advertencia de seguridad:** la cuenta de X `VibeTrading_HKU`, el proyecto de Virtuals `101845` y el contrato de token `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` no son activos oficiales de Vibe-Trading. Nunca hemos lanzado ni respaldado ningún token o memecoin. No compres, conectes una wallet ni firmes nada. [Detalles](SECURITY.md#official-channels--impersonation). > ⚠️ **Advertencia de seguridad:** la cuenta de X `VibeTrading_HKU`, el proyecto de Virtuals `101845` y el contrato de token `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` no son activos oficiales de Vibe-Trading. Nunca hemos lanzado ni respaldado ningún token o memecoin. No compres, conectes una wallet ni firmes nada. [Detalles](SECURITY.md#official-channels--impersonation).
- **2026-10-12** 🧮 **Actualizaciones de investigación y backtesting**: corregidos la ejecución inicial, la atribución al liquidar, la duración de caídas, la tolerancia de valoración y el momento de los informes SEC en el IC ([#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)). Se añaden 13 factores fundamentales y de valoración, datos y backtests de Brasil, plantillas ZimaOS de activación explícita y progreso en Feishu. Alcance y límites en [CHANGELOG](CHANGELOG.md).
- **2026-10-11** 🛠️ **Cinco correcciones prioritarias**: cada dato de un cálculo necesita evidencia; se bloquean las opciones reales aún no soportadas. Los índices de referencia sin conexión conservan su fuente, los registros distinguen cierres sin ganancias ni pérdidas y Codex conserva argumentos opcionales ([#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622), [#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779), [#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781), [#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)).
- **2026-10-10** 🛠️ **Configuración de Slack/Discord y análisis reproducible**: Los formularios traducidos verifican credenciales del bot sin guardar cambios ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). El análisis de cartera reutiliza instantáneas inmutables y la Web exporta JSON sin datos sensibles ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Los indicadores informan la fecha límite y la huella de los datos. Los registros de sesión dañados se recuperan sin reescribir los originales. También se corrigen búsqueda de memoria, filtrado de credenciales e importación de operaciones. Consulta [CHANGELOG](CHANGELOG.md). - **2026-10-10** 🛠️ **Configuración de Slack/Discord y análisis reproducible**: Los formularios traducidos verifican credenciales del bot sin guardar cambios ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). El análisis de cartera reutiliza instantáneas inmutables y la Web exporta JSON sin datos sensibles ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Los indicadores informan la fecha límite y la huella de los datos. Los registros de sesión dañados se recuperan sin reescribir los originales. También se corrigen búsqueda de memoria, filtrado de credenciales e importación de operaciones. Consulta [CHANGELOG](CHANGELOG.md).
<details>
<summary>Noticias anteriores</summary>
- **2026-10-09** 🛠️ **Configuración de canales y correcciones de backtests**: Las actualizaciones sin reconexión conservan las credenciales de las URL y la dirección del servidor autenticado de WeChat; la web explica guardados y reinicios en nueve idiomas ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). La volatilidad de opciones entre mercados usa solo marcas temporales conocidas en cada barra; la caída máxima incluye el capital inicial. Corregimos las rutas de criptoactivos y metales preciosos, los duplicados de Hong Kong, la recuperación de caché, los importes en notación científica y la validación de valores ausentes. mootdx sale de la cadena predeterminada de acciones A. Los trece PR están en [CHANGELOG](CHANGELOG.md). - **2026-10-09** 🛠️ **Configuración de canales y correcciones de backtests**: Las actualizaciones sin reconexión conservan las credenciales de las URL y la dirección del servidor autenticado de WeChat; la web explica guardados y reinicios en nueve idiomas ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). La volatilidad de opciones entre mercados usa solo marcas temporales conocidas en cada barra; la caída máxima incluye el capital inicial. Corregimos las rutas de criptoactivos y metales preciosos, los duplicados de Hong Kong, la recuperación de caché, los importes en notación científica y la validación de valores ausentes. mootdx sale de la cadena predeterminada de acciones A. Los trece PR están en [CHANGELOG](CHANGELOG.md).
- **2026-10-08** 🔐 **Informes PDF protegidos y lecturas del bróker corregidas**: Los PDF de correo programado admiten protección AES-256 opcional en Web, CLI y confirmaciones del agente; la contraseña permanece en la configuración privada del canal ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS consulta las tres rutas, renueva una sola vez un token explícitamente caducado y rechaza lecturas incompletas ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Las fórmulas conservan la expresión tras las etiquetas descriptivas y mantienen la verificación de evidencias ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)). - **2026-10-08** 🔐 **Informes PDF protegidos y lecturas del bróker corregidas**: Los PDF de correo programado admiten protección AES-256 opcional en Web, CLI y confirmaciones del agente; la contraseña permanece en la configuración privada del canal ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS consulta las tres rutas, renueva una sola vez un token explícitamente caducado y rechaza lecturas incompletas ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Las fórmulas conservan la expresión tras las etiquetas descriptivas y mantienen la verificación de evidencias ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)).
<details>
<summary>Noticias anteriores</summary>
- **2026-10-07** 🛠️ **Continuidad de investigación y métricas de riesgo corregidas**: El estado de ejecución se separa del mensaje del sistema para preservar su prefijo estable ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Se corrigen las fechas entre mercados, los retornos alrededor de precios ausentes y el Sortino de opciones con toda la muestra ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Los cargadores comparten un contrato OHLCV; también se corrigen valoración crediticia, exposiciones, validación temporal y diagnóstico de auditoría. - **2026-10-07** 🛠️ **Continuidad de investigación y métricas de riesgo corregidas**: El estado de ejecución se separa del mensaje del sistema para preservar su prefijo estable ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Se corrigen las fechas entre mercados, los retornos alrededor de precios ausentes y el Sortino de opciones con toda la muestra ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Los cargadores comparten un contrato OHLCV; también se corrigen valoración crediticia, exposiciones, validación temporal y diagnóstico de auditoría.
- **2026-10-06** 🛠️ **Control de operaciones reales y corrección de informes**: Detener el agente de operaciones reales cancela el análisis en curso y espera a que termine el planificador, incluso al cancelar durante el inicio o cerrar la API; se corrigen también las unidades de tiempo del estado ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). La cancelación y el cierre de posiciones de emergencia omiten registros inválidos y continúan con los restantes ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); cada bróker conserva su propia programación y las escrituras manejan concurrencia y escrituras parciales. Las sesiones reducidas de acciones estadounidenses respetan el cierre anticipado ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Las correcciones conservan las cifras verificadas y no presentan las cifras omitidas de los comentarios limitados como aprobadas ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)). - **2026-10-06** 🛠️ **Control de operaciones reales y corrección de informes**: Detener el agente de operaciones reales cancela el análisis en curso y espera a que termine el planificador, incluso al cancelar durante el inicio o cerrar la API; se corrigen también las unidades de tiempo del estado ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). La cancelación y el cierre de posiciones de emergencia omiten registros inválidos y continúan con los restantes ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); cada bróker conserva su propia programación y las escrituras manejan concurrencia y escrituras parciales. Las sesiones reducidas de acciones estadounidenses respetan el cierre anticipado ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Las correcciones conservan las cifras verificadas y no presentan las cifras omitidas de los comentarios limitados como aprobadas ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)).
@@ -376,7 +379,7 @@ Está diseñado para investigación, simulación y backtesting, y, cuando tú lo
| **Ejecutar equipos de analistas** | Revisiones de investigación multiagente para flujos de trabajo de inversión, cuantitativo, cripto, macro y riesgo. | | **Ejecutar equipos de analistas** | Revisiones de investigación multiagente para flujos de trabajo de inversión, cuantitativo, cripto, macro y riesgo. |
| **Llevar la investigación a canales de mensajería** | Ejecuta el mismo runtime de sesión a través de WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, correo electrónico y Mochat, con controles de CLI, REST y Web UI. | | **Llevar la investigación a canales de mensajería** | Ejecuta el mismo runtime de sesión a través de WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, correo electrónico y Mochat, con controles de CLI, REST y Web UI. |
| **Entregar artefactos utilizables** | Informes, TradingView Pine Script, TDX, MetaTrader 5, herramientas MCP y sesiones de investigación posteriores. | | **Entregar artefactos utilizables** | Informes, TradingView Pine Script, TDX, MetaTrader 5, herramientas MCP y sesiones de investigación posteriores. |
| **Evaluar un zoológico de alfas preconstruido** | Clasificación IC + viva/invertida/muerta en una línea, en 462 alfas (Qlib 158 + Kakushadze 101 + GTJA 191 + académicas + fundamentales PIT-safe) sobre tu universo. | | **Evaluar un zoológico de alfas preconstruido** | Clasificación IC + viva/invertida/muerta en una línea, en 475 alfas (Qlib 158 + Kakushadze 101 + GTJA 191 + académicas + fundamentales PIT-safe) sobre tu universo. |
| **Detectar regímenes de correlación** | Una línea de tiempo de densidad de aristas + histéresis en la superficie `/correlation` que muestra cuándo los mercados se fusionan en un solo bloque: contexto de riesgo descriptivo, no una señal. | | **Detectar regímenes de correlación** | Una línea de tiempo de densidad de aristas + histéresis en la superficie `/correlation` que muestra cuándo los mercados se fusionan en un solo bloque: contexto de riesgo descriptivo, no una señal. |
--- ---
@@ -460,7 +463,7 @@ La mayoría de las ejecuciones siguen la misma ruta de evidencia: enrutar la sol
## 📡 Fuentes de Datos y Fallback Inteligente ## 📡 Fuentes de Datos y Fallback Inteligente
Una sola llamada `get_market_data`, **28 fuentes de datos de mercado**, una de ellas el mercado premium opcional **QVeris** (además del mercado premium opcional **QVeris**). Establece `source: "auto"`: el cargador elige según el símbolo y luego recorre una cadena por mercado ordenada por **riesgo de bloqueo de IP**: primero las fuentes públicas que nunca se bloquean, al final las limitadas o que requieren clave. Cero configuración, sin punto único de fallo. Una sola llamada `get_market_data`, **29 fuentes de datos de mercado**, una de ellas el mercado premium opcional **QVeris** (además del mercado premium opcional **QVeris**). Establece `source: "auto"`: el cargador elige según el símbolo y luego recorre una cadena por mercado ordenada por **riesgo de bloqueo de IP**: primero las fuentes públicas que nunca se bloquean, al final las limitadas o que requieren clave. Cero configuración, sin punto único de fallo.
| Fuente | Mercados | Autenticación | Rol | | Fuente | Mercados | Autenticación | Rol |
|--------|---------|------|------| |--------|---------|------|------|
@@ -483,6 +486,7 @@ Una sola llamada `get_market_data`, **28 fuentes de datos de mercado**, una de e
| `tickerall` | forex / metales | clave + cuenta (solo lectura) | el mismo feed MT5 del bróker, **alojado**: sin terminal local y en cualquier sistema operativo (solo por selección explícita, nunca en el respaldo automático) | | `tickerall` | forex / metales | clave + cuenta (solo lectura) | el mismo feed MT5 del bróker, **alojado**: sin terminal local y en cualquier sistema operativo (solo por selección explícita, nunca en el respaldo automático) |
| `pykrx` | Corea (KRX: KOSPI/KOSDAQ) | ninguna | barras diarias de KOSPI / KOSDAQ para `.KS` / `.KQ` (extra opcional `krx`) | | `pykrx` | Corea (KRX: KOSPI/KOSDAQ) | ninguna | barras diarias de KOSPI / KOSDAQ para `.KS` / `.KQ` (extra opcional `krx`) |
| `india_broker` | India (NSE/BSE) | login de broker | barras de solo lectura de Zerodha / Shoonya / Dhan para `.NS` / `.BO` (al final de la cadena de fallback) | | `india_broker` | India (NSE/BSE) | login de broker | barras de solo lectura de Zerodha / Shoonya / Dhan para `.NS` / `.BO` (al final de la cadena de fallback) |
| `b3_cotahist` | Brazil (B3) | none | barras diarias oficiales del mercado al contado y de lotes fraccionarios (sin ajustar, BRL), importadas con `python -m src.br_data sync`; alternativa local para `.SA` |
| `local` | cualquiera | ninguna | tu propio CSV / Parquet / DuckDB mediante el prefijo `local:` | | `local` | cualquiera | ninguna | tu propio CSV / Parquet / DuckDB mediante el prefijo `local:` |
**Cadenas de fallback (por riesgo de bloqueo de IP):** **Cadenas de fallback (por riesgo de bloqueo de IP):**
@@ -491,6 +495,7 @@ Una sola llamada `get_market_data`, **28 fuentes de datos de mercado**, una de e
- **EE. UU.** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **EE. UU.** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local` - **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **Corea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local` - **Corea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local`
- **Reino Unido (LSE)** → `yahoo` · `yfinance` · `local` *(solo cotizaciones declaradas en GBP/GBp)* - **Reino Unido (LSE)** → `yahoo` · `yfinance` · `local` *(solo cotizaciones declaradas en GBP/GBp)*
- **Cripto** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **Cripto** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -668,9 +673,9 @@ junto al catálogo incluido (los archivos con el mismo nombre lo sobrescriben, i
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>462 alphas cuantitativos preconstruidos en 5 familias</sub></summary> <summary><b>Alpha Zoo</b> <sub>475 alphas cuantitativos preconstruidos en 5 familias</sub></summary>
- 🧬 462 alphas cross-sectional, con prohibición de lookahead a nivel de la capa de operadores - 🧬 475 alphas cross-sectional, con prohibición de lookahead a nivel de la capa de operadores
- 📈 Categorización de IC + IR + vivo/invertido/muerto en un solo comando de la CLI - 📈 Categorización de IC + IR + vivo/invertido/muerto en un solo comando de la CLI
- 🔬 Puerta de pureza AST + test centinela de lookahead de 300 filas + interruptor de apagado de red `pytest-socket` - 🔬 Puerta de pureza AST + test centinela de lookahead de 300 filas + interruptor de apagado de red `pytest-socket`
- 📦 Atribución Apache-2 para Qlib; `LICENSE.md` por zoo que declara las fórmulas como contenido matemático - 📦 Atribución Apache-2 para Qlib; `LICENSE.md` por zoo que declara las fórmulas como contenido matemático
@@ -689,7 +694,7 @@ Ejecuta `vibe-trading alpha list` para explorar, `vibe-trading alpha show <id>`
</details> </details>
<details> <details>
<summary><b>Motores de Backtest</b> <sub>10 motores + cartera de opciones, composite cross-market</sub></summary> <summary><b>Motores de Backtest</b> <sub>11 motores + cartera de opciones, composite cross-market</sub></summary>
| Motor | Mercado | Notas | | Motor | Mercado | Notas |
|--------|--------|-------| |--------|--------|-------|
@@ -697,6 +702,7 @@ Ejecuta `vibe-trading alpha list` para explorar, `vibe-trading alpha show <id>`
| **GlobalEquity** | EE. UU. / HK / Canadá / Reino Unido | trading en la misma sesión; lotes, ticks, divisas de liquidación y costos específicos de cada mercado | | **GlobalEquity** | EE. UU. / HK / Canadá / Reino Unido | trading en la misma sesión; lotes, ticks, divisas de liquidación y costos específicos de cada mercado |
| **IndiaEquity** | India (NSE/BSE) | T+1, bandas de circuito, pila de costos STT / stamp / SEBI / GST basada en configuración | | **IndiaEquity** | India (NSE/BSE) | T+1, bandas de circuito, pila de costos STT / stamp / SEBI / GST basada en configuración |
| **KoreaEquity** | Corea (KRX: KOSPI/KOSDAQ) | solo largo, banda de ±30% evaluada en el momento de ejecución sobre la malla de ticks unificada, impuesto de transacción del 0.20% en 2026 | | **KoreaEquity** | Corea (KRX: KOSPI/KOSDAQ) | solo largo, banda de ±30% evaluada en el momento de ejecución sobre la malla de ticks unificada, impuesto de transacción del 0.20% en 2026 |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | spot cripto / perps USD-M | liquidaciones de funding, división ejecución/mark | | **Crypto** | spot cripto / perps USD-M | liquidaciones de funding, división ejecución/mark |
| **ChinaFutures** · **GlobalFutures** | futuros | margen, multiplicadores de contrato | | **ChinaFutures** · **GlobalFutures** | futuros | margen, multiplicadores de contrato |
| **Forex** | FX / metales | vía el loader `mt5` | | **Forex** | FX / metales | vía el loader `mt5` |
@@ -1009,7 +1015,7 @@ La TUI interactiva (`vibe-trading`) ahora usa una transcripción nativa de termi
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # explora 462 alphas preconstruidos; subcomandos show / bench / compare / export-manifest disponibles vibe-trading alpha list # explora 475 alphas preconstruidos; subcomandos show / bench / compare / export-manifest disponibles
vibe-trading playbook list # cinco plantillas de investigación programada; subcomandos show / create disponibles vibe-trading playbook list # cinco plantillas de investigación programada; subcomandos show / create disponibles
vibe-trading channels status --local # inspecciona la configuración de canales IM y sugerencias de instalación vibe-trading channels status --local # inspecciona la configuración de canales IM y sugerencias de instalación
vibe-trading provider doctor # imprime diagnósticos redactados de provider/proxy/paquetes vibe-trading provider doctor # imprime diagnósticos redactados de provider/proxy/paquetes
@@ -1804,7 +1810,7 @@ Vibe-Trading/
│ │ ├── memory/ # Memoria persistente entre sesiones │ │ ├── memory/ # Memoria persistente entre sesiones
│ │ │ └── persistent.py # memoria basada en archivos (~/.vibe-trading/memory/) │ │ │ └── persistent.py # memoria basada en archivos (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 herramientas de agente autodescubiertas │ │ ├── tools/ # 110 herramientas de agente autodescubiertas
│ │ │ ├── backtest_tool.py # ejecuta backtests │ │ │ ├── backtest_tool.py # ejecuta backtests
│ │ │ ├── remember_tool.py # memoria entre sesiones (save/recall/forget) │ │ │ ├── remember_tool.py # memoria entre sesiones (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # CRUD de skills (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # CRUD de skills (save/patch/delete/file)
@@ -1813,7 +1819,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # búsqueda web con DuckDuckGo │ │ │ ├── web_search_tool.py # búsqueda web con DuckDuckGo
│ │ │ └── ... # bash, E/S de archivos, análisis de factores, opciones, navegador de alpha + bench, etc. │ │ │ └── ... # bash, E/S de archivos, análisis de factores, opciones, navegador de alpha + bench, etc.
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 462 alphas en 5 familias │ │ ├── factors/ # Alpha Zoo — 475 alphas en 5 familias
│ │ │ ├── base.py # 19 operadores (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19 operadores (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # carga de metadatos solo por AST + cómputo diferido + puertas de sanidad │ │ │ ├── registry.py # carga de metadatos solo por AST + cómputo diferido + puertas de sanidad
│ │ │ ├── bench_runner.py # IC + categorización alive/reversed/dead │ │ │ ├── bench_runner.py # IC + categorización alive/reversed/dead
@@ -1829,8 +1835,8 @@ Vibe-Trading/
│ │ └── providers/ # Abstracción de proveedores LLM │ │ └── providers/ # Abstracción de proveedores LLM
│ │ │ │
│ └── backtest/ # Motores de backtest │ └── backtest/ # Motores de backtest
│ ├── engines/ # 9 motores + motor compuesto multi-mercado + options_portfolio │ ├── engines/ # 10 motores + motor compuesto multi-mercado + options_portfolio
│ ├── loaders/ # 28 fuentes: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata │ ├── loaders/ # 29 fuentes: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata, b3_cotahist
│ │ ├── base.py # Protocolo DataLoader │ │ ├── base.py # Protocolo DataLoader
│ │ └── registry.py # Registro + cadenas de fallback automáticas │ │ └── registry.py # Registro + cadenas de fallback automáticas
│ └── optimizers/ # MVO, equal vol, max div, risk parity │ └── optimizers/ # MVO, equal vol, max div, risk parity
@@ -1896,7 +1902,7 @@ Vibe-Trading forma parte del ecosistema de agentes de **[HKUDS](https://github.c
| **Data Bridge** | Trae tus propios datos: conectores locales CSV/Parquet/SQL con mapeo de esquema | Cargador local publicado | | **Data Bridge** | Trae tus propios datos: conectores locales CSV/Parquet/SQL con mapeo de esquema | Cargador local publicado |
| **Options Lab** | Superficie de volatilidad, panel de Greeks, explorador de payoff/escenarios | Herramienta analítica de payoff/escenarios **publicada**; superficie/panel planificados | | **Options Lab** | Superficie de volatilidad, panel de Greeks, explorador de payoff/escenarios | Herramienta analítica de payoff/escenarios **publicada**; superficie/panel planificados |
| **Portfolio Studio** | Risk x-ray, restricciones, optimizador con reducción de turnover, notas de rebalanceo | Optimizador con reducción de turnover **publicado en 0.1.11**; el resto planificado | | **Portfolio Studio** | Risk x-ray, restricciones, optimizador con reducción de turnover, notas de rebalanceo | Optimizador con reducción de turnover **publicado en 0.1.11**; el resto planificado |
| **Alpha Zoo** | 462 alphas preconstruidas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental) con benchmark en una línea, integración con el agente y Web UI | **Publicado en 0.1.8**, ampliado hasta 0.1.12 | | **Alpha Zoo** | 475 alphas preconstruidas (Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental) con benchmark en una línea, integración con el agente y Web UI | **Publicado en 0.1.8**, ampliado hasta 0.1.12 |
| **Strategy Development Manager** | Registra papers / research de brokers como factores y estrategias con un almacén persistente + lifecycle automatizado de decaimiento de IC/Sharpe | **Publicado en 0.1.11** | | **Strategy Development Manager** | Registra papers / research de brokers como factores y estrategias con un almacén persistente + lifecycle automatizado de decaimiento de IC/Sharpe | **Publicado en 0.1.11** |
| **Correlation Regime** | Línea temporal de régimen por edge-density + histéresis, superpuesta en `/correlation` — detecta cuándo los mercados se fusionan en un solo bloque | **Publicado en 0.1.12** | | **Correlation Regime** | Línea temporal de régimen por edge-density + histéresis, superpuesta en `/correlation` — detecta cuándo los mercados se fusionan en un solo bloque | **Publicado en 0.1.12** |
| **Research Delivery** | Briefs programados y sesiones de investigación en vivo a través de canales IM tipo Slack / Telegram / correo | Scheduler + runtime de IM publicados | | **Research Delivery** | Briefs programados y sesiones de investigación en vivo a través de canales IM tipo Slack / Telegram / correo | Scheduler + runtime de IM publicados |
README_id.md
+21 -15
View File
@@ -53,16 +53,19 @@
> ⚠️ **Peringatan keamanan:** Akun X `VibeTrading_HKU`, proyek Virtuals `101845`, dan kontrak token `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` bukan aset resmi Vibe-Trading. Kami tidak pernah meluncurkan atau mendukung token maupun memecoin apa pun. Jangan membeli, menghubungkan wallet, atau menandatangani apa pun. [Detail](SECURITY.md#official-channels--impersonation). > ⚠️ **Peringatan keamanan:** Akun X `VibeTrading_HKU`, proyek Virtuals `101845`, dan kontrak token `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` bukan aset resmi Vibe-Trading. Kami tidak pernah meluncurkan atau mendukung token maupun memecoin apa pun. Jangan membeli, menghubungkan wallet, atau menandatangani apa pun. [Detail](SECURITY.md#official-channels--impersonation).
- **2026-10-12** 🧮 **Pembaruan riset dan backtest**: memperbaiki eksekusi pembukaan, atribusi saat likuidasi, durasi drawdown, toleransi valuasi, dan waktu laporan SEC dalam IC faktor ([#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)). Menambahkan 13 faktor fundamental/valuasi, data dan backtest Brasil, templat ZimaOS dengan aktivasi eksplisit, serta progres Feishu. Cakupan dan batasan ada di [CHANGELOG](CHANGELOG.md).
- **2026-10-11** 🛠️ **Lima perbaikan prioritas**: setiap nilai data dalam perhitungan turunan memerlukan bukti; opsi perdagangan riil yang belum didukung tetap diblokir. Benchmark luring mempertahankan sumber pilihan, catatan membedakan penutupan impas, dan alat Codex mempertahankan argumen opsional ([#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622), [#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779), [#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781), [#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)).
- **2026-10-10** 🛠️ **Pengaturan Slack/Discord dan riset yang dapat direproduksi**: Formulir terjemahan memverifikasi kredensial bot tanpa menyimpan perubahan ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). Riset portofolio dapat memakai ulang snapshot yang tidak berubah; Web mengekspor JSON tanpa data sensitif ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Indikator mencatat batas tanggal dan sidik data masukan. Catatan sesi rusak dipulihkan tanpa menulis ulang berkas asli. Pencarian memori, penyamaran kredensial, dan impor transaksi juga diperbaiki. Lihat [CHANGELOG](CHANGELOG.md). - **2026-10-10** 🛠️ **Pengaturan Slack/Discord dan riset yang dapat direproduksi**: Formulir terjemahan memverifikasi kredensial bot tanpa menyimpan perubahan ([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). Riset portofolio dapat memakai ulang snapshot yang tidak berubah; Web mengekspor JSON tanpa data sensitif ([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). Indikator mencatat batas tanggal dan sidik data masukan. Catatan sesi rusak dipulihkan tanpa menulis ulang berkas asli. Pencarian memori, penyamaran kredensial, dan impor transaksi juga diperbaiki. Lihat [CHANGELOG](CHANGELOG.md).
<details>
<summary>Berita sebelumnya</summary>
- **2026-10-09** 🛠️ **Pengaturan kanal dan perbaikan backtest**: Pembaruan tanpa menyambungkan ulang mempertahankan kredensial URL dan alamat server WeChat hasil autentikasi; web menjelaskan hasil penyimpanan dan mulai ulang dalam sembilan bahasa ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). Volatilitas opsi lintas pasar hanya memakai waktu yang sudah diketahui pada setiap bar; drawdown mencakup modal awal. Perbaikan meliputi rute data kripto/logam mulia, duplikat saham Hong Kong, pemulihan cache, nominal notasi ilmiah, dan validasi nilai ekuitas yang hilang. mootdx dikeluarkan dari rantai saham A bawaan. Ketiga belas PR tercantum dalam [CHANGELOG](CHANGELOG.md). - **2026-10-09** 🛠️ **Pengaturan kanal dan perbaikan backtest**: Pembaruan tanpa menyambungkan ulang mempertahankan kredensial URL dan alamat server WeChat hasil autentikasi; web menjelaskan hasil penyimpanan dan mulai ulang dalam sembilan bahasa ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). Volatilitas opsi lintas pasar hanya memakai waktu yang sudah diketahui pada setiap bar; drawdown mencakup modal awal. Perbaikan meliputi rute data kripto/logam mulia, duplikat saham Hong Kong, pemulihan cache, nominal notasi ilmiah, dan validasi nilai ekuitas yang hilang. mootdx dikeluarkan dari rantai saham A bawaan. Ketiga belas PR tercantum dalam [CHANGELOG](CHANGELOG.md).
- **2026-10-08** 🔐 **Laporan PDF terlindungi dan pembacaan broker diperbaiki**: PDF Email terjadwal mendukung perlindungan AES-256 opsional melalui Web, CLI, dan konfirmasi agen; kata sandi hanya disimpan dalam konfigurasi privat kanal ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS menanyakan ketiga jalur perdagangan, memperbarui token yang secara eksplisit kedaluwarsa satu kali, dan menolak pembacaan yang tidak lengkap ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Rumus turunan tetap utuh setelah label deskriptif dengan pemeriksaan bukti tetap berlaku ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)). - **2026-10-08** 🔐 **Laporan PDF terlindungi dan pembacaan broker diperbaiki**: PDF Email terjadwal mendukung perlindungan AES-256 opsional melalui Web, CLI, dan konfirmasi agen; kata sandi hanya disimpan dalam konfigurasi privat kanal ([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS menanyakan ketiga jalur perdagangan, memperbarui token yang secara eksplisit kedaluwarsa satu kali, dan menolak pembacaan yang tidak lengkap ([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). Rumus turunan tetap utuh setelah label deskriptif dengan pemeriksaan bukti tetap berlaku ([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)).
<details>
<summary>Berita sebelumnya</summary>
- **2026-10-07** 🛠️ **Kesinambungan riset dan koreksi metrik risiko**: Status eksekusi dipindahkan dari prompt sistem agar prefiksnya tetap stabil ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Tanggal lintas pasar, imbal hasil di sekitar harga yang hilang, dan Sortino opsi berdasarkan seluruh sampel diperbaiki ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Pengujian loader memakai satu kontrak OHLCV, disertai koreksi valuasi kredit, eksposur, validasi berjalan, dan diagnosis audit. - **2026-10-07** 🛠️ **Kesinambungan riset dan koreksi metrik risiko**: Status eksekusi dipindahkan dari prompt sistem agar prefiksnya tetap stabil ([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). Tanggal lintas pasar, imbal hasil di sekitar harga yang hilang, dan Sortino opsi berdasarkan seluruh sampel diperbaiki ([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). Pengujian loader memakai satu kontrak OHLCV, disertai koreksi valuasi kredit, eksposur, validasi berjalan, dan diagnosis audit.
- **2026-10-06** 🛠️ **Kontrol trading langsung dan koreksi laporan**: Menghentikan runner trading langsung membatalkan analisis aktif dan menunggu penjadwal selesai, termasuk pembatalan saat mulai dan penutupan API; satuan waktu pada status juga diperbaiki ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). Pembatalan order dan penutupan posisi darurat melewati catatan tidak valid lalu melanjutkan catatan lainnya ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); jadwal setiap broker disimpan terpisah dengan penanganan penulisan bersamaan dan parsial. Sesi saham AS yang dipersingkat mengikuti waktu tutup lebih awal ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Koreksi laporan mempertahankan angka yang lolos verifikasi, dan umpan balik terbatas tidak menyiratkan bahwa angka yang tidak tercantum sudah lolos ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)). - **2026-10-06** 🛠️ **Kontrol trading langsung dan koreksi laporan**: Menghentikan runner trading langsung membatalkan analisis aktif dan menunggu penjadwal selesai, termasuk pembatalan saat mulai dan penutupan API; satuan waktu pada status juga diperbaiki ([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). Pembatalan order dan penutupan posisi darurat melewati catatan tidak valid lalu melanjutkan catatan lainnya ([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)); jadwal setiap broker disimpan terpisah dengan penanganan penulisan bersamaan dan parsial. Sesi saham AS yang dipersingkat mengikuti waktu tutup lebih awal ([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). Koreksi laporan mempertahankan angka yang lolos verifikasi, dan umpan balik terbatas tidak menyiratkan bahwa angka yang tidak tercantum sudah lolos ([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)).
@@ -499,7 +502,7 @@ Vibe-Trading dirancang untuk riset, simulasi, dan backtest — serta, jika Anda
| **Jalankan tim analis** | Review riset multi-agent untuk workflow investasi, quant, crypto, makro, dan risiko. | | **Jalankan tim analis** | Review riset multi-agent untuk workflow investasi, quant, crypto, makro, dan risiko. |
| **Bawa riset ke channel IM** | Jalankan runtime sesi yang sama melalui WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, dan Mochat dengan kontrol CLI, REST, dan Web UI. | | **Bawa riset ke channel IM** | Jalankan runtime sesi yang sama melalui WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, dan Mochat dengan kontrol CLI, REST, dan Web UI. |
| **Hasilkan artefak yang siap digunakan** | Laporan, TradingView Pine Script, TDX, MetaTrader 5, tool MCP, dan sesi riset lanjutan. | | **Hasilkan artefak yang siap digunakan** | Laporan, TradingView Pine Script, TDX, MetaTrader 5, tool MCP, dan sesi riset lanjutan. |
| **Benchmark alpha zoo siap pakai** | Satu baris untuk IC + kategorisasi alive/reversed/dead pada 462 alpha (Qlib 158 + Kakushadze 101 + GTJA 191 + akademik + fundamental PIT-safe) di universe Anda. | | **Benchmark alpha zoo siap pakai** | Satu baris untuk IC + kategorisasi alive/reversed/dead pada 475 alpha (Qlib 158 + Kakushadze 101 + GTJA 191 + akademik + fundamental PIT-safe) di universe Anda. |
| **Deteksi regime korelasi** | Timeline edge-density + hysteresis pada surface `/correlation` yang menunjukkan kapan market menyatu menjadi satu blok — konteks risiko deskriptif, bukan sinyal. | | **Deteksi regime korelasi** | Timeline edge-density + hysteresis pada surface `/correlation` yang menunjukkan kapan market menyatu menjadi satu blok — konteks risiko deskriptif, bukan sinyal. |
--- ---
@@ -622,7 +625,7 @@ the wall-clock activity watchdog.
## 📡 Sumber Data & Smart Fallback ## 📡 Sumber Data & Smart Fallback
Satu call `get_market_data`, **28 sumber data market**, salah satunya marketplace premium opsional **QVeris**. Atur `source: "auto"` — loader memilih berdasarkan simbol lalu mengikuti chain per market yang diurutkan berdasarkan **risiko IP-ban**: sumber publik yang tidak pernah diblokir lebih dulu, sumber throttled / membutuhkan key belakangan. Zero-config, tanpa single point of failure. Satu call `get_market_data`, **29 sumber data market**, salah satunya marketplace premium opsional **QVeris**. Atur `source: "auto"` — loader memilih berdasarkan simbol lalu mengikuti chain per market yang diurutkan berdasarkan **risiko IP-ban**: sumber publik yang tidak pernah diblokir lebih dulu, sumber throttled / membutuhkan key belakangan. Zero-config, tanpa single point of failure.
| Sumber | Market | Auth | Peran | | Sumber | Market | Auth | Peran |
|--------|---------|------|------| |--------|---------|------|------|
@@ -645,6 +648,7 @@ Satu call `get_market_data`, **28 sumber data market**, salah satunya marketplac
| `tickerall` | forex / metals | key + akun (read-only) | feed broker MT5 yang sama, **hosted** — tanpa terminal lokal, OS apa pun (hanya eksplisit, tidak pernah auto fallback) | | `tickerall` | forex / metals | key + akun (read-only) | feed broker MT5 yang sama, **hosted** — tanpa terminal lokal, OS apa pun (hanya eksplisit, tidak pernah auto fallback) |
| `pykrx` | Korea (KRX: KOSPI/KOSDAQ) | tidak ada | bar harian KOSPI / KOSDAQ untuk `.KS` / `.KQ` (extra `krx` opsional) | | `pykrx` | Korea (KRX: KOSPI/KOSDAQ) | tidak ada | bar harian KOSPI / KOSDAQ untuk `.KS` / `.KQ` (extra `krx` opsional) |
| `india_broker` | India (NSE/BSE) | login broker | bar Zerodha / Shoonya / Dhan read-only untuk `.NS` / `.BO` (ujung fallback chain) | | `india_broker` | India (NSE/BSE) | login broker | bar Zerodha / Shoonya / Dhan read-only untuk `.NS` / `.BO` (ujung fallback chain) |
| `b3_cotahist` | Brazil (B3) | none | bar harian resmi B3 untuk pasar tunai/odd-lot (mentah, BRL), diimpor dengan `python -m src.br_data sync`; cadangan luring untuk `.SA` |
| `local` | apa pun | tidak ada | CSV / Parquet / DuckDB Anda sendiri melalui prefix `local:` | | `local` | apa pun | tidak ada | CSV / Parquet / DuckDB Anda sendiri melalui prefix `local:` |
**Fallback chain (berdasarkan risiko IP-ban):** **Fallback chain (berdasarkan risiko IP-ban):**
@@ -653,6 +657,7 @@ Satu call `get_market_data`, **28 sumber data market**, salah satunya marketplac
- **US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **US** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **HK** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local` - **India (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **Korea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local` - **Korea (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local`
- **UK (LSE)** → `yahoo` · `yfinance` · `local` *(declared GBP/GBp quotes only)* - **UK (LSE)** → `yahoo` · `yfinance` · `local` *(declared GBP/GBp quotes only)*
- **Crypto** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **Crypto** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -828,9 +833,9 @@ bersama roster bawaan (file dengan nama sama akan override, seperti user skill)
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>462 alpha quant siap pakai dalam 5 keluarga</sub></summary> <summary><b>Alpha Zoo</b> <sub>475 alpha quant siap pakai dalam 5 keluarga</sub></summary>
- 🧬 462 alpha cross-sectional, lookahead dilarang pada layer operator - 🧬 475 alpha cross-sectional, lookahead dilarang pada layer operator
- 📈 IC + IR + kategorisasi alive/reversed/dead dalam satu perintah CLI - 📈 IC + IR + kategorisasi alive/reversed/dead dalam satu perintah CLI
- 🔬 AST purity gate + test sentinel lookahead 300 baris + network kill-switch `pytest-socket` - 🔬 AST purity gate + test sentinel lookahead 300 baris + network kill-switch `pytest-socket`
- 📦 Atribusi Apache-2 untuk Qlib; `LICENSE.md` per-zoo menyatakan formula sebagai konten matematis - 📦 Atribusi Apache-2 untuk Qlib; `LICENSE.md` per-zoo menyatakan formula sebagai konten matematis
@@ -849,7 +854,7 @@ Jalankan `vibe-trading alpha list` untuk menjelajah, `vibe-trading alpha show <i
</details> </details>
<details> <details>
<summary><b>Engine Backtest</b> <sub>10 engine + portofolio opsi, komposit lintas market</sub></summary> <summary><b>Engine Backtest</b> <sub>11 engine + portofolio opsi, komposit lintas market</sub></summary>
| Engine | Market | Catatan | | Engine | Market | Catatan |
|--------|--------|-------| |--------|--------|-------|
@@ -858,6 +863,7 @@ Jalankan `vibe-trading alpha list` untuk menjelajah, `vibe-trading alpha show <i
| **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack | | **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack |
| **KoreaEquity** | Korea (KRX: KOSPI/KOSDAQ) | long-only, ±30% band judged at execution time on the unified tick grid, 2026 0.20% transaction tax | | **KoreaEquity** | Korea (KRX: KOSPI/KOSDAQ) | long-only, ±30% band judged at execution time on the unified tick grid, 2026 0.20% transaction tax |
| **VietnamEquity** | Vietnam (HOSE) | long-only, T+2 settlement hold, ±7% band on the 10/50/100-VND tick grid, 100-share lots, 0.1% sell-side tax | | **VietnamEquity** | Vietnam (HOSE) | long-only, T+2 settlement hold, ±7% band on the 10/50/100-VND tick grid, 100-share lots, 0.1% sell-side tax |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split | | **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split |
| **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers | | **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers |
| **Forex** | FX / metals | via the `mt5` loader (local terminal) or the hosted `tickerall` loader (no terminal, any OS) | | **Forex** | FX / metals | via the `mt5` loader (local terminal) or the hosted `tickerall` loader (no terminal, any OS) |
@@ -1196,7 +1202,7 @@ TUI interaktif (`vibe-trading`) kini memakai transcript native terminal: startup
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # browse 462 pre-built alphas; show / bench / compare / export-manifest sub-commands available vibe-trading alpha list # browse 475 pre-built alphas; show / bench / compare / export-manifest sub-commands available
vibe-trading playbook list # five scheduled-research templates; show / create sub-commands available vibe-trading playbook list # five scheduled-research templates; show / create sub-commands available
vibe-trading channels status --local # inspect IM channel config and install hints vibe-trading channels status --local # inspect IM channel config and install hints
vibe-trading provider doctor # print redacted provider/proxy/package diagnostics vibe-trading provider doctor # print redacted provider/proxy/package diagnostics
@@ -2049,7 +2055,7 @@ Vibe-Trading/
│ │ ├── memory/ # Cross-session persistent memory │ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/) │ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 auto-discovered agent tools │ │ ├── tools/ # 110 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests │ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget) │ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
@@ -2058,7 +2064,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo web search │ │ │ ├── web_search_tool.py # DuckDuckGo web search
│ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc. │ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 462 alphas across 5 families │ │ ├── factors/ # Alpha Zoo — 475 alphas across 5 families
│ │ │ ├── base.py # 19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # AST-only metadata load + lazy compute + sanity gates │ │ │ ├── registry.py # AST-only metadata load + lazy compute + sanity gates
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead categorisation │ │ │ ├── bench_runner.py # IC + alive/reversed/dead categorisation
@@ -2074,8 +2080,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM provider abstraction │ │ └── providers/ # LLM provider abstraction
│ │ │ │
│ └── backtest/ # Backtest engines │ └── backtest/ # Backtest engines
│ ├── engines/ # 9 engines + composite cross-market engine + options_portfolio │ ├── engines/ # 10 engines + composite cross-market engine + options_portfolio
│ ├── loaders/ # 28 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata │ ├── loaders/ # 29 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + auto-fallback chains │ │ └── registry.py # Registry + auto-fallback chains
│ └── optimizers/ # MVO, equal vol, max div, risk parity │ └── optimizers/ # MVO, equal vol, max div, risk parity
@@ -2142,7 +2148,7 @@ Vibe-Trading adalah bagian dari ekosistem agent **[HKUDS](https://github.com/HKU
| **Data Bridge** | Bawa data sendiri: connector CSV/Parquet/SQL lokal dengan schema mapping | Loader lokal Dirilis | | **Data Bridge** | Bawa data sendiri: connector CSV/Parquet/SQL lokal dengan schema mapping | Loader lokal Dirilis |
| **Options Lab** | Vol surface, dashboard Greeks, explorer payoff/scenario | Tool analitik payoff/scenario **Dirilis**; surface/dashboard Direncanakan | | **Options Lab** | Vol surface, dashboard Greeks, explorer payoff/scenario | Tool analitik payoff/scenario **Dirilis**; surface/dashboard Direncanakan |
| **Portfolio Studio** | Risk x-ray, constraint, optimizer turnover-aware, catatan rebalance | Optimizer turnover-aware **Dirilis 0.1.11**; sisanya Direncanakan | | **Portfolio Studio** | Risk x-ray, constraint, optimizer turnover-aware, catatan rebalance | Optimizer turnover-aware **Dirilis 0.1.11**; sisanya Direncanakan |
| **Alpha Zoo** | 462 alpha siap pakai (Qlib 158 + Kakushadze 101 + GTJA 191 + akademik + fundamental) dengan bench satu baris, integrasi agent, dan Web UI | **Dirilis 0.1.8**, diperluas hingga 0.1.12 | | **Alpha Zoo** | 475 alpha siap pakai (Qlib 158 + Kakushadze 101 + GTJA 191 + akademik + fundamental) dengan bench satu baris, integrasi agent, dan Web UI | **Dirilis 0.1.8**, diperluas hingga 0.1.12 |
| **Strategy Development Manager** | Daftarkan paper / riset broker sebagai faktor & strategi dengan store persisten + lifecycle decay IC/Sharpe otomatis | **Dirilis 0.1.11** | | **Strategy Development Manager** | Daftarkan paper / riset broker sebagai faktor & strategi dengan store persisten + lifecycle decay IC/Sharpe otomatis | **Dirilis 0.1.11** |
| **Correlation Regime** | Timeline regime edge-density + hysteresis di `/correlation` — deteksi saat market menyatu menjadi satu blok | **Dirilis 0.1.12** | | **Correlation Regime** | Timeline regime edge-density + hysteresis di `/correlation` — deteksi saat market menyatu menjadi satu blok | **Dirilis 0.1.12** |
| **Research Delivery** | Brief terjadwal dan sesi riset live melalui Slack / Telegram / channel IM bergaya email | Scheduler + IM Runtime Dirilis | | **Research Delivery** | Brief terjadwal dan sesi riset live melalui Slack / Telegram / channel IM bergaya email | Scheduler + IM Runtime Dirilis |
README_ja.md
+21 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **セキュリティ警告：** Xアカウント `VibeTrading_HKU`、Virtualsプロジェクト `101845`、およびトークンコントラクト `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` は、いずれもVibe-Trading公式のものではありません。Vibe-Tradingはこれまで、いかなるトークンやミームコインも発行・公認していません。購入、ウォレットの接続、署名は行わないでください。[詳細](SECURITY.md#official-channels--impersonation) > ⚠️ **セキュリティ警告：** Xアカウント `VibeTrading_HKU`、Virtualsプロジェクト `101845`、およびトークンコントラクト `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` は、いずれもVibe-Trading公式のものではありません。Vibe-Tradingはこれまで、いかなるトークンやミームコインも発行・公認していません。購入、ウォレットの接続、署名は行わないでください。[詳細](SECURITY.md#official-channels--impersonation)
- **2026-10-12** 🧮 **調査とバックテストの更新**：初回約定、清算時の寄与分析、ドローダウン期間、評価許容誤差、因子 IC の SEC 開示タイミングを修正（[#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)）。基本面・評価因子 13 種、ブラジル市場データとバックテスト、明示的に有効化する ZimaOS テンプレート、Feishu の実行進捗を追加。範囲と制約は [CHANGELOG](CHANGELOG.md) を参照。
- **2026-10-11** 🛠️ **優先度の高い 5 件を修正**：導出値の各データ項に根拠を要求し、未対応の実取引オプション注文を拒否。オフライン基準は指定ソースを維持し、取引記録は損益ゼロの決済を識別、Codex ツールは任意引数を保持します（[#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622)、[#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779)、[#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781)、[#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)）。
- **2026-10-10** 🛠️ **Slack/Discord 設定と再現可能な調査**：翻訳済みフォームでボット認証情報を検証できます。テストは変更を保存しません（[#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)、[#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)）。ポートフォリオ調査は不変のスナップショットを再利用でき、Web から認証情報や口座番号を除いた JSON を出力できます（[#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)）。指標は取得日と入力の指紋を報告し、破損したセッション記録は元のファイルを書き換えずに復旧します。メモリ検索、認証情報の秘匿、取引履歴の識別も修正しました。[CHANGELOG](CHANGELOG.md) を参照。 - **2026-10-10** 🛠️ **Slack/Discord 設定と再現可能な調査**：翻訳済みフォームでボット認証情報を検証できます。テストは変更を保存しません（[#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)、[#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)）。ポートフォリオ調査は不変のスナップショットを再利用でき、Web から認証情報や口座番号を除いた JSON を出力できます（[#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)）。指標は取得日と入力の指紋を報告し、破損したセッション記録は元のファイルを書き換えずに復旧します。メモリ検索、認証情報の秘匿、取引履歴の識別も修正しました。[CHANGELOG](CHANGELOG.md) を参照。
<details>
<summary>過去のニュース</summary>
- **2026-10-09** 🛠️ **チャンネル設定とバックテストの修正**：再接続なしの設定更新で URL 内の認証情報と WeChat 認証後のサーバーアドレスを保持し、保存・再起動の結果を Web で9言語表示します（[#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)、[#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)、[#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)）。複数市場のオプション価格に使うボラティリティは各時点までのタイムスタンプだけで計算し、ドローダウンには初期資金を反映。暗号資産・貴金属のデータ取得経路、香港株の重複判定、キャッシュ復旧、科学表記の金額、欠損資産額の検証を修正しました。mootdx は標準の A 株取得経路から外れます。全13件は [CHANGELOG](CHANGELOG.md)。 - **2026-10-09** 🛠️ **チャンネル設定とバックテストの修正**：再接続なしの設定更新で URL 内の認証情報と WeChat 認証後のサーバーアドレスを保持し、保存・再起動の結果を Web で9言語表示します（[#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)、[#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)、[#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)）。複数市場のオプション価格に使うボラティリティは各時点までのタイムスタンプだけで計算し、ドローダウンには初期資金を反映。暗号資産・貴金属のデータ取得経路、香港株の重複判定、キャッシュ復旧、科学表記の金額、欠損資産額の検証を修正しました。mootdx は標準の A 株取得経路から外れます。全13件は [CHANGELOG](CHANGELOG.md)。
- **2026-10-08** 🔐 **PDF レポートのパスワード保護とブローカー読み取り修正**：定期メール PDF に AES-256 保護を選択でき、Web、CLI、エージェント確認画面に反映されます。パスワードは非公開のチャネル設定だけに保存します（[#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)）。KIS は三つの取引経路を照会し、明示的なトークン失効を一度だけ再取得、不完全な読み取りはエラーにします（[#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)）。説明ラベルの後の導出式を保持し、証拠検証を維持します（[#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)）。 - **2026-10-08** 🔐 **PDF レポートのパスワード保護とブローカー読み取り修正**：定期メール PDF に AES-256 保護を選択でき、Web、CLI、エージェント確認画面に反映されます。パスワードは非公開のチャネル設定だけに保存します（[#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)）。KIS は三つの取引経路を照会し、明示的なトークン失効を一度だけ再取得、不完全な読み取りはエラーにします（[#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)）。説明ラベルの後の導出式を保持し、証拠検証を維持します（[#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)）。
<details>
<summary>過去のニュース</summary>
- **2026-10-07** 🛠️ **リサーチの継続性とリスク指標の修正**：実行状態をシステムプロンプトから分離し、安定したプレフィックスを保ちます（[#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)）。市場間の日付整列、欠損価格前後のリターン、全標本によるオプションのソルティノ比率を修正しました（[#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)、[#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)、[#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)）。データ取得テストは共通の OHLCV 契約を使い、信用評価、スタイル・エクスポージャー、時系列検証、監査診断も改善しました。 - **2026-10-07** 🛠️ **リサーチの継続性とリスク指標の修正**：実行状態をシステムプロンプトから分離し、安定したプレフィックスを保ちます（[#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)）。市場間の日付整列、欠損価格前後のリターン、全標本によるオプションのソルティノ比率を修正しました（[#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)、[#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)、[#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)）。データ取得テストは共通の OHLCV 契約を使い、信用評価、スタイル・エクスポージャー、時系列検証、監査診断も改善しました。
- **2026-10-06** 🛠️ **ライブ運用制御とレポート修正**：ライブランナーの停止は実行中の分析をキャンセルしてスケジューラーの終了を待ち、起動中のキャンセルと API 終了にも対応します。状態の時刻単位も修正しました（[#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)）。緊急キャンセル・決済処理は不正なレコードを除外して残りの注文・保有を処理します（[#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)）。ブローカーごとにスケジュールを保存し、並行書き込みと部分書き込みに対応。米国株の短縮取引日は早い閉場時刻を守ります（[#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)）。レポート修正は検証済みの数値を保持し、省略された指摘を検証済みと誤認させません（[#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)）。 - **2026-10-06** 🛠️ **ライブ運用制御とレポート修正**：ライブランナーの停止は実行中の分析をキャンセルしてスケジューラーの終了を待ち、起動中のキャンセルと API 終了にも対応します。状態の時刻単位も修正しました（[#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)）。緊急キャンセル・決済処理は不正なレコードを除外して残りの注文・保有を処理します（[#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)）。ブローカーごとにスケジュールを保存し、並行書き込みと部分書き込みに対応。米国株の短縮取引日は早い閉場時刻を守ります（[#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)）。レポート修正は検証済みの数値を保持し、省略された指摘を検証済みと誤認させません（[#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)）。
@@ -377,7 +380,7 @@ Vibe-Trading は、金融に関する問いを実行可能な分析へ変換す
| **アナリストチームを走らせる** | 投資、クオンツ、暗号資産、マクロ、リスクのワークフロー向けマルチエージェント・リサーチレビュー。 | | **アナリストチームを走らせる** | 投資、クオンツ、暗号資産、マクロ、リスクのワークフロー向けマルチエージェント・リサーチレビュー。 |
| **リサーチを IM チャンネルへ接続する** | WebSocket、Telegram、Slack、Discord、Matrix、WhatsApp、Signal、QQ/NapCat、WeChat/WeCom、Feishu/Lark、DingTalk、Teams、email、Mochat から同じ session runtime を CLI、REST、Web UI で管理。 | | **リサーチを IM チャンネルへ接続する** | WebSocket、Telegram、Slack、Discord、Matrix、WhatsApp、Signal、QQ/NapCat、WeChat/WeCom、Feishu/Lark、DingTalk、Teams、email、Mochat から同じ session runtime を CLI、REST、Web UI で管理。 |
| **使える artifacts を出力する** | レポート、TradingView Pine Script、TDX、MetaTrader 5、MCP tools、後続リサーチセッション。 | | **使える artifacts を出力する** | レポート、TradingView Pine Script、TDX、MetaTrader 5、MCP tools、後続リサーチセッション。 |
| **事前構築 alpha zoo をベンチ** | 462 個の alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental）に対し、1 行 CLI で IC + IR + alive/reversed/dead 分類を実行 | | **事前構築 alpha zoo をベンチ** | 475 個の alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental）に対し、1 行 CLI で IC + IR + alive/reversed/dead 分類を実行 |
| **相関レジームを見抜く** | `/correlation` 面上のエッジ密度 + ヒステリシスのタイムライン。市場が 1 つのブロックに融合するタイミングを示す —— シグナルではなく記述的なリスクコンテキスト。 | | **相関レジームを見抜く** | `/correlation` 面上のエッジ密度 + ヒステリシスのタイムライン。市場が 1 つのブロックに融合するタイミングを示す —— シグナルではなく記述的なリスクコンテキスト。 |
--- ---
@@ -461,7 +464,7 @@ vibe-trading connector install /tmp/my-broker
## 📡 データソースとスマートフォールバック ## 📡 データソースとスマートフォールバック
1 回の `get_market_data` 呼び出しで **28 のマーケットデータソース**（うち **QVeris** はオプションの有料マーケットプレイス）にアクセスできます。`source: "auto"` を指定すれば、loader が銘柄に応じてソースを選び、**IP 規制リスク**の順に並んだ市場別チェーンをたどります。規制を受けない公開ソースを先に、スロットリングや key を要するソースを最後に試します。設定不要、単一障害点なし。 1 回の `get_market_data` 呼び出しで **29 のマーケットデータソース**（うち **QVeris** はオプションの有料マーケットプレイス）にアクセスできます。`source: "auto"` を指定すれば、loader が銘柄に応じてソースを選び、**IP 規制リスク**の順に並んだ市場別チェーンをたどります。規制を受けない公開ソースを先に、スロットリングや key を要するソースを最後に試します。設定不要、単一障害点なし。
| Source | Markets | Auth | Role | | Source | Markets | Auth | Role |
|--------|---------|------|------| |--------|---------|------|------|
@@ -484,6 +487,7 @@ vibe-trading connector install /tmp/my-broker
| `tickerall` | forex / metals | key + アカウント（読み取り専用） | 同じブローカーの MT5 フィードを**ホスト型**で — ローカル端末不要、OS を問わない（明示指定のみ、auto フォールバックには決して入らない） | | `tickerall` | forex / metals | key + アカウント（読み取り専用） | 同じブローカーの MT5 フィードを**ホスト型**で — ローカル端末不要、OS を問わない（明示指定のみ、auto フォールバックには決して入らない） |
| `pykrx` | 韓国（KRX：KOSPI/KOSDAQ） | 不要 | `.KS` / `.KQ` の KOSPI / KOSDAQ 日足（任意の `krx` extra） | | `pykrx` | 韓国（KRX：KOSPI/KOSDAQ） | 不要 | `.KS` / `.KQ` の KOSPI / KOSDAQ 日足（任意の `krx` extra） |
| `india_broker` | インド（NSE/BSE） | ブローカーログイン | `.NS` / `.BO` 向けの読み取り専用 Zerodha / Shoonya / Dhan bars（フォールバックチェーン末尾） | | `india_broker` | インド（NSE/BSE） | ブローカーログイン | `.NS` / `.BO` 向けの読み取り専用 Zerodha / Shoonya / Dhan bars（フォールバックチェーン末尾） |
| `b3_cotahist` | Brazil (B3) | none | ローカル B3 公式現物・単元未満株の日足（未調整、BRL）。`python -m src.br_data sync` で取得する `.SA` のオフライン代替ソース |
| `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix | | `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix |
**フォールバックチェーン（IP 規制リスク順）：** **フォールバックチェーン（IP 規制リスク順）：**
@@ -492,6 +496,7 @@ vibe-trading connector install /tmp/my-broker
- **米国株** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **米国株** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **香港株** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **香港株** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **インド株（NSE/BSE）** → `yahoo` · `yfinance` · `india_broker` · `local` - **インド株（NSE/BSE）** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **韓国（KOSPI/KOSDAQ）** → `pykrx` · `yahoo` · `yfinance` · `local` - **韓国（KOSPI/KOSDAQ）** → `pykrx` · `yahoo` · `yfinance` · `local`
- **英国（LSE）** → `yahoo` · `yfinance` · `local` *（GBP/GBp と明示されたクォートのみ）* - **英国（LSE）** → `yahoo` · `yfinance` · `local` *（GBP/GBp と明示されたクォートのみ）*
- **暗号資産** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **暗号資産** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -662,9 +667,9 @@ Paper-vs-live is a **structural per-broker runtime guard** (account-id format, h
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>5 つのファミリーに渡る 462 個の事前構築 quant alpha</sub></summary> <summary><b>Alpha Zoo</b> <sub>5 つのファミリーに渡る 475 個の事前構築 quant alpha</sub></summary>
- 🧬 462 個のクロスセクショナル alpha、オペレーター層でルックアヘッドを禁止 - 🧬 475 個のクロスセクショナル alpha、オペレーター層でルックアヘッドを禁止
- 📈 IC + IR + alive/reversed/dead 分類を 1 つの CLI コマンドで - 📈 IC + IR + alive/reversed/dead 分類を 1 つの CLI コマンドで
- 🔬 AST 純関数ゲート + 300 行のルックアヘッド sentinel テスト + `pytest-socket` によるネットワーク遮断 - 🔬 AST 純関数ゲート + 300 行のルックアヘッド sentinel テスト + `pytest-socket` によるネットワーク遮断
- 📦 Qlib には Apache-2 帰属表示、各 zoo ごとに `LICENSE.md` で formula を数学的内容として宣言 - 📦 Qlib には Apache-2 帰属表示、各 zoo ごとに `LICENSE.md` で formula を数学的内容として宣言
@@ -683,7 +688,7 @@ Paper-vs-live is a **structural per-broker runtime guard** (account-id format, h
</details> </details>
<details> <details>
<summary><b>Backtest Engines</b> <sub>10 engines + options portfolio, cross-market composite</sub></summary> <summary><b>Backtest Engines</b> <sub>11 engines + options portfolio, cross-market composite</sub></summary>
| Engine | Market | Notes | | Engine | Market | Notes |
|--------|--------|-------| |--------|--------|-------|
@@ -692,6 +697,7 @@ Paper-vs-live is a **structural per-broker runtime guard** (account-id format, h
| **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack | | **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack |
| **KoreaEquity** | 韓国（KRX：KOSPI/KOSDAQ） | ロングオンリー、統一呼値グリッド上で ±30% 制限値幅を約定時点に判定、2026 年 0.20% の証券取引税 | | **KoreaEquity** | 韓国（KRX：KOSPI/KOSDAQ） | ロングオンリー、統一呼値グリッド上で ±30% 制限値幅を約定時点に判定、2026 年 0.20% の証券取引税 |
| **VietnamEquity** | ベトナム（HOSE） | ロングオンリー、T+2 決済ホールド、10/50/100 ドンの呼値グリッド上で ±7% 制限値幅、100 株単元、売却側 0.1% 課税 | | **VietnamEquity** | ベトナム（HOSE） | ロングオンリー、T+2 決済ホールド、10/50/100 ドンの呼値グリッド上で ±7% 制限値幅、100 株単元、売却側 0.1% 課税 |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split | | **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split |
| **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers | | **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers |
| **Forex** | FX / metals | via the `mt5` loader | | **Forex** | FX / metals | via the `mt5` loader |
@@ -976,7 +982,7 @@ Vibe-Trading は tool-heavy agent です。skills、backtests、memory、swarms
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # 462 個の事前構築 alpha を閲覧；show / bench / compare / export-manifest サブコマンド利用可 vibe-trading alpha list # 475 個の事前構築 alpha を閲覧；show / bench / compare / export-manifest サブコマンド利用可
vibe-trading playbook list # 定期リサーチのテンプレート 5 本；show / create サブコマンド利用可 vibe-trading playbook list # 定期リサーチのテンプレート 5 本；show / create サブコマンド利用可
vibe-trading channels status --local # IM チャンネル設定と install hints を確認 vibe-trading channels status --local # IM チャンネル設定と install hints を確認
vibe-trading provider doctor # 秘匿処理済みの provider/proxy/package 診断を出力 vibe-trading provider doctor # 秘匿処理済みの provider/proxy/package 診断を出力
@@ -1724,7 +1730,7 @@ Vibe-Trading/
│ │ ├── memory/ # クロスセッション永続メモリ │ │ ├── memory/ # クロスセッション永続メモリ
│ │ │ └── persistent.py # ファイルベースメモリ (~/.vibe-trading/memory/) │ │ │ └── persistent.py # ファイルベースメモリ (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 個の自動検出エージェントツール │ │ ├── tools/ # 110 個の自動検出エージェントツール
│ │ │ ├── backtest_tool.py # バックテスト実行 │ │ │ ├── backtest_tool.py # バックテスト実行
│ │ │ ├── remember_tool.py # クロスセッションメモリ (save/recall/forget) │ │ │ ├── remember_tool.py # クロスセッションメモリ (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
@@ -1733,7 +1739,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo Web 検索 │ │ │ ├── web_search_tool.py # DuckDuckGo Web 検索
│ │ │ └── ... # bash、file I/O、factor analysis、options、alpha browser + bench など │ │ │ └── ... # bash、file I/O、factor analysis、options、alpha browser + bench など
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 5 つのファミリーにまたがる 462 個の alpha │ │ ├── factors/ # Alpha Zoo — 5 つのファミリーにまたがる 475 個の alpha
│ │ │ ├── base.py # 19 個のオペレーター (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19 個のオペレーター (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # AST 限定のメタデータ読み込み + 遅延計算 + sanity gate │ │ │ ├── registry.py # AST 限定のメタデータ読み込み + 遅延計算 + sanity gate
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead 分類 │ │ │ ├── bench_runner.py # IC + alive/reversed/dead 分類
@@ -1749,8 +1755,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM プロバイダー抽象化 │ │ └── providers/ # LLM プロバイダー抽象化
│ │ │ │
│ └── backtest/ # バックテストエンジン │ └── backtest/ # バックテストエンジン
│ ├── engines/ # 9 エンジン + クロスマーケット composite engine + options_portfolio │ ├── engines/ # 10 エンジン + クロスマーケット composite engine + options_portfolio
│ ├── loaders/ # 28 ソース: tushare、okx、nobitex、wallex、binance、yfinance、akshare、baostock、tencent、mootdx、ccxt、futu、pykrx、local、eastmoney、sina、stooq、yahoo、finnhub、alphavantage、tiingo、fmp、longbridge、mt5、qveris、india_broker、tickerall、gildata │ ├── loaders/ # 29 ソース: tushare、okx、nobitex、wallex、binance、yfinance、akshare、baostock、tencent、mootdx、ccxt、futu、pykrx、local、eastmoney、sina、stooq、yahoo、finnhub、alphavantage、tiingo、fmp、longbridge、mt5、qveris、india_broker、tickerall、gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + 自動フォールバックチェーン │ │ └── registry.py # Registry + 自動フォールバックチェーン
│ └── optimizers/ # MVO、equal vol、max div、risk parity │ └── optimizers/ # MVO、equal vol、max div、risk parity
@@ -1816,7 +1822,7 @@ Vibe-Trading は **[HKUDS](https://github.com/HKUDS)** agent ecosystem の一部
| **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | ローカルローダー出荷済み | | **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | ローカルローダー出荷済み |
| **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Planned | | **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Planned |
| **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover を考慮したオプティマイザは **0.1.11 でリリース済み**；残りは Planned | | **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover を考慮したオプティマイザは **0.1.11 でリリース済み**；残りは Planned |
| **Alpha Zoo** | 462 個の事前構築 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental）、1 行 CLI でベンチ、agent 統合、Web UI | **0.1.8 でリリース済み**、0.1.12 まで拡張 | | **Alpha Zoo** | 475 個の事前構築 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental）、1 行 CLI でベンチ、agent 統合、Web UI | **0.1.8 でリリース済み**、0.1.12 まで拡張 |
| **Strategy Development Manager** | Register papers / broker research as factors & strategies with a persistent store + automated IC/Sharpe decay lifecycle | **0.1.11 でリリース済み** | | **Strategy Development Manager** | Register papers / broker research as factors & strategies with a persistent store + automated IC/Sharpe decay lifecycle | **0.1.11 でリリース済み** |
| **Correlation Regime** | Edge-density + hysteresis regime timeline layered on `/correlation` — spot when markets fuse into one bloc | **0.1.12 でリリース済み** | | **Correlation Regime** | Edge-density + hysteresis regime timeline layered on `/correlation` — spot when markets fuse into one bloc | **0.1.12 でリリース済み** |
| **Research Delivery** | Slack / Telegram / email-style IM channels 経由の scheduled briefs と live research sessions | スケジューラ + IM Runtime 出荷済み | | **Research Delivery** | Slack / Telegram / email-style IM channels 経由の scheduled briefs と live research sessions | スケジューラ + IM Runtime 出荷済み |
README_ko.md
+21 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **보안 경고:** X 계정 `VibeTrading_HKU`, Virtuals 프로젝트 `101845`, 토큰 컨트랙트 `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4`는 모두 Vibe-Trading 공식과 무관합니다. Vibe-Trading은 어떠한 토큰이나 밈코인도 발행하거나 공식적으로 지지한 적이 없습니다. 해당 토큰을 구매하거나 지갑을 연결하거나 어떠한 서명도 하지 마세요. [자세히 보기](SECURITY.md#official-channels--impersonation). > ⚠️ **보안 경고:** X 계정 `VibeTrading_HKU`, Virtuals 프로젝트 `101845`, 토큰 컨트랙트 `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4`는 모두 Vibe-Trading 공식과 무관합니다. Vibe-Trading은 어떠한 토큰이나 밈코인도 발행하거나 공식적으로 지지한 적이 없습니다. 해당 토큰을 구매하거나 지갑을 연결하거나 어떠한 서명도 하지 마세요. [자세히 보기](SECURITY.md#official-channels--impersonation).
- **2026-10-12** 🧮 **리서치와 백테스트 업데이트**: 최초 체결, 청산 시 성과 귀속, 낙폭 지속 기간, 가치평가 허용 오차, 팩터 IC의 SEC 공시 반영 시점을 수정했습니다([#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)). 기본적·가치평가 팩터 13개, 브라질 데이터와 백테스트, 선택적으로 활성화하는 ZimaOS 템플릿, Feishu 실행 진행 표시를 추가했습니다. 범위와 제약은 [CHANGELOG](CHANGELOG.md)를 참고하세요.
- **2026-10-11** 🛠️ **우선순위 수정 5건**: 계산에 쓰인 각 데이터 값에 근거를 요구하고 미지원 실거래 옵션 주문을 차단합니다. 오프라인 벤치마크는 지정 소스를 유지하며, 거래 기록은 손익 0 청산을 구분하고 Codex 도구는 선택 인수를 보존합니다([#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622), [#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779), [#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781), [#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)).
- **2026-10-10** 🛠️ **Slack/Discord 설정과 재현 가능한 연구**: 번역된 양식에서 봇 인증 정보를 검증하며 테스트는 변경 사항을 저장하지 않습니다([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). 포트폴리오 연구는 불변 스냅샷을 재사용하고 Web에서 인증 정보와 계좌 번호를 제거한 JSON을 내보냅니다([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). 지표는 조회 날짜와 입력 지문을 보고하며 손상된 세션 기록은 원본을 다시 쓰지 않고 복구합니다. 메모리 검색, 인증 정보 마스킹, 거래 내역 식별도 수정했습니다. [CHANGELOG](CHANGELOG.md)를 참조하세요. - **2026-10-10** 🛠️ **Slack/Discord 설정과 재현 가능한 연구**: 번역된 양식에서 봇 인증 정보를 검증하며 테스트는 변경 사항을 저장하지 않습니다([#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764), [#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)). 포트폴리오 연구는 불변 스냅샷을 재사용하고 Web에서 인증 정보와 계좌 번호를 제거한 JSON을 내보냅니다([#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)). 지표는 조회 날짜와 입력 지문을 보고하며 손상된 세션 기록은 원본을 다시 쓰지 않고 복구합니다. 메모리 검색, 인증 정보 마스킹, 거래 내역 식별도 수정했습니다. [CHANGELOG](CHANGELOG.md)를 참조하세요.
<details>
<summary>이전 뉴스</summary>
- **2026-10-09** 🛠️ **채널 설정 및 백테스트 수정**: 재연결 없는 설정 업데이트에서 URL 인증 정보와 WeChat 인증 후 서버 주소를 보존하며, 저장·재시작 결과를 웹에서 9개 언어로 표시합니다 ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). 여러 시장의 옵션 변동성은 각 시점까지 알려진 타임스탬프만 사용하고, 낙폭은 초기 자금을 반영합니다. 암호화폐·귀금속 데이터 경로, 홍콩 주식 중복 판별, 캐시 복구, 과학적 표기 금액 및 누락된 자산 값 검증을 수정했습니다. 기본 A주 경로에서 mootdx를 제외했습니다. 13개 PR은 [CHANGELOG](CHANGELOG.md)를 참조하세요. - **2026-10-09** 🛠️ **채널 설정 및 백테스트 수정**: 재연결 없는 설정 업데이트에서 URL 인증 정보와 WeChat 인증 후 서버 주소를 보존하며, 저장·재시작 결과를 웹에서 9개 언어로 표시합니다 ([#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737), [#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742), [#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)). 여러 시장의 옵션 변동성은 각 시점까지 알려진 타임스탬프만 사용하고, 낙폭은 초기 자금을 반영합니다. 암호화폐·귀금속 데이터 경로, 홍콩 주식 중복 판별, 캐시 복구, 과학적 표기 금액 및 누락된 자산 값 검증을 수정했습니다. 기본 A주 경로에서 mootdx를 제외했습니다. 13개 PR은 [CHANGELOG](CHANGELOG.md)를 참조하세요.
- **2026-10-08** 🔐 **PDF 보고서 비밀번호 보호와 증권사 조회 수정**: 예약 이메일 PDF에 AES-256 보호를 선택할 수 있으며 Web, CLI 및 에이전트 확인에 반영됩니다. 비밀번호는 비공개 채널 설정에만 저장합니다([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS는 세 거래 경로를 조회하고 명시적으로 만료된 토큰을 한 번만 갱신하며 불완전한 조회는 오류로 처리합니다([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). 설명 뒤의 전체 파생 수식을 보존하고 증거 검증을 유지합니다([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)). - **2026-10-08** 🔐 **PDF 보고서 비밀번호 보호와 증권사 조회 수정**: 예약 이메일 PDF에 AES-256 보호를 선택할 수 있으며 Web, CLI 및 에이전트 확인에 반영됩니다. 비밀번호는 비공개 채널 설정에만 저장합니다([#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)). KIS는 세 거래 경로를 조회하고 명시적으로 만료된 토큰을 한 번만 갱신하며 불완전한 조회는 오류로 처리합니다([#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)). 설명 뒤의 전체 파생 수식을 보존하고 증거 검증을 유지합니다([#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)).
<details>
<summary>이전 뉴스</summary>
- **2026-10-07** 🛠️ **리서치 연속성과 위험 지표 수정**: 실행 상태를 시스템 프롬프트 밖으로 옮겨 안정적인 접두부를 유지합니다([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). 시장 간 날짜 정렬, 누락 가격 주변의 수익률, 전체 표본을 사용하는 옵션 Sortino 계산을 수정했습니다([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). 데이터 로더 테스트는 공통 OHLCV 계약을 사용하며 신용 평가, 스타일 노출, 순차 검증 및 감사 진단도 개선했습니다. - **2026-10-07** 🛠️ **리서치 연속성과 위험 지표 수정**: 실행 상태를 시스템 프롬프트 밖으로 옮겨 안정적인 접두부를 유지합니다([#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)). 시장 간 날짜 정렬, 누락 가격 주변의 수익률, 전체 표본을 사용하는 옵션 Sortino 계산을 수정했습니다([#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710), [#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717), [#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)). 데이터 로더 테스트는 공통 OHLCV 계약을 사용하며 신용 평가, 스타일 노출, 순차 검증 및 감사 진단도 개선했습니다.
- **2026-10-06** 🛠️ **실거래 제어와 보고서 수정**: 실거래 러너 중지는 현재 분석을 취소하고 스케줄러 종료를 기다리며 시작 중 취소와 API 종료도 처리합니다. 상태 시간의 단위도 수정했습니다([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). 긴급 취소·청산은 잘못된 레코드를 건너뛰고 나머지 주문과 포지션을 계속 처리합니다([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)). 증권사별 일정을 따로 저장하고 동시 쓰기와 부분 쓰기를 처리합니다. 미국 주식 단축 거래일의 조기 폐장 시간을 반영합니다([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). 보고서 수정은 검증된 수치를 유지하며 생략된 오류가 검증을 통과한 것처럼 안내하지 않습니다([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)). - **2026-10-06** 🛠️ **실거래 제어와 보고서 수정**: 실거래 러너 중지는 현재 분석을 취소하고 스케줄러 종료를 기다리며 시작 중 취소와 API 종료도 처리합니다. 상태 시간의 단위도 수정했습니다([#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)). 긴급 취소·청산은 잘못된 레코드를 건너뛰고 나머지 주문과 포지션을 계속 처리합니다([#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)). 증권사별 일정을 따로 저장하고 동시 쓰기와 부분 쓰기를 처리합니다. 미국 주식 단축 거래일의 조기 폐장 시간을 반영합니다([#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)). 보고서 수정은 검증된 수치를 유지하며 생략된 오류가 검증을 통과한 것처럼 안내하지 않습니다([#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)).
@@ -377,7 +380,7 @@ Vibe-Trading은 금융 질문을 실행 가능한 분석으로 바꾸는 오픈
| **애널리스트 팀 실행하기** | 투자, 퀀트, 크립토, 매크로, 리스크 워크플로를 위한 멀티 에이전트 리서치 리뷰. | | **애널리스트 팀 실행하기** | 투자, 퀀트, 크립토, 매크로, 리스크 워크플로를 위한 멀티 에이전트 리서치 리뷰. |
| **리서치를 IM 채널에 연결하기** | WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, Mochat에서 같은 session runtime을 CLI, REST, Web UI로 관리. | | **리서치를 IM 채널에 연결하기** | WebSocket, Telegram, Slack, Discord, Matrix, WhatsApp, Signal, QQ/NapCat, WeChat/WeCom, Feishu/Lark, DingTalk, Teams, email, Mochat에서 같은 session runtime을 CLI, REST, Web UI로 관리. |
| **사용 가능한 artifacts 만들기** | 리포트, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, 이후 리서치 세션. | | **사용 가능한 artifacts 만들기** | 리포트, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, 이후 리서치 세션. |
| **사전 빌드된 alpha zoo 벤치** | 462개의 alpha 인자(Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental)에 대해 한 줄 CLI로 IC + IR + alive/reversed/dead 분류 수행 | | **사전 빌드된 alpha zoo 벤치** | 475개의 alpha 인자(Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental)에 대해 한 줄 CLI로 IC + IR + alive/reversed/dead 분류 수행 |
| **상관관계 국면 포착하기** | `/correlation` 화면의 엣지 밀도 + 히스테리시스 타임라인으로 시장이 한 덩어리로 움직이기 시작하는 시점을 보여줍니다 — 시그널이 아니라 서술적 리스크 컨텍스트입니다. | | **상관관계 국면 포착하기** | `/correlation` 화면의 엣지 밀도 + 히스테리시스 타임라인으로 시장이 한 덩어리로 움직이기 시작하는 시점을 보여줍니다 — 시그널이 아니라 서술적 리스크 컨텍스트입니다. |
--- ---
@@ -461,7 +464,7 @@ vibe-trading connector install /tmp/my-broker
## 📡 데이터 소스 & 스마트 폴백 ## 📡 데이터 소스 & 스마트 폴백
`get_market_data` 한 번의 호출, **28개 시장 데이터 소스**(그중 **QVeris**는 선택형 유료 마켓플레이스). `source: "auto"`로 설정하면 로더가 심볼에 따라 소스를 고르고, 시장별 체인을 **IP 차단 위험** 순으로 따라갑니다: 절대 차단되지 않는 공개 소스를 먼저, 속도 제한 / 키 기반 소스를 마지막에 둡니다. 설정 불필요, 단일 장애 지점 없음. `get_market_data` 한 번의 호출, **29개 시장 데이터 소스**(그중 **QVeris**는 선택형 유료 마켓플레이스). `source: "auto"`로 설정하면 로더가 심볼에 따라 소스를 고르고, 시장별 체인을 **IP 차단 위험** 순으로 따라갑니다: 절대 차단되지 않는 공개 소스를 먼저, 속도 제한 / 키 기반 소스를 마지막에 둡니다. 설정 불필요, 단일 장애 지점 없음.
| Source | Markets | Auth | Role | | Source | Markets | Auth | Role |
|--------|---------|------|------| |--------|---------|------|------|
@@ -484,6 +487,7 @@ vibe-trading connector install /tmp/my-broker
| `tickerall` | forex / metals | key + 계정(읽기 전용) | 동일 브로커의 MT5 피드를 **호스팅형**으로 — 로컬 터미널 불필요, OS 무관(명시 지정 전용, auto 폴백에는 절대 들어가지 않음) | | `tickerall` | forex / metals | key + 계정(읽기 전용) | 동일 브로커의 MT5 피드를 **호스팅형**으로 — 로컬 터미널 불필요, OS 무관(명시 지정 전용, auto 폴백에는 절대 들어가지 않음) |
| `pykrx` | 한국 (KRX: KOSPI/KOSDAQ) | 없음 | `.KS` / `.KQ`용 KOSPI / KOSDAQ 일봉 (선택적 `krx` extra) | | `pykrx` | 한국 (KRX: KOSPI/KOSDAQ) | 없음 | `.KS` / `.KQ`용 KOSPI / KOSDAQ 일봉 (선택적 `krx` extra) |
| `india_broker` | 인도 (NSE/BSE) | 브로커 로그인 | `.NS` / `.BO`용 읽기 전용 Zerodha / Shoonya / Dhan 봉 (폴백 체인 말단) | | `india_broker` | 인도 (NSE/BSE) | 브로커 로그인 | `.NS` / `.BO`용 읽기 전용 Zerodha / Shoonya / Dhan 봉 (폴백 체인 말단) |
| `b3_cotahist` | Brazil (B3) | none | 로컬 B3 공식 현물·단주 일봉(미조정, BRL). `python -m src.br_data sync`로 가져오는 `.SA` 오프라인 대체 소스 |
| `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix | | `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix |
**폴백 체인 (IP 차단 위험 순):** **폴백 체인 (IP 차단 위험 순):**
@@ -492,6 +496,7 @@ vibe-trading connector install /tmp/my-broker
- **미국** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **미국** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **홍콩** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **홍콩** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **인도 (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local` - **인도 (NSE/BSE)** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **한국 (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local` - **한국 (KOSPI/KOSDAQ)** → `pykrx` · `yahoo` · `yfinance` · `local`
- **영국 (LSE)** → `yahoo` · `yfinance` · `local` *(명시된 GBP/GBp 호가만 허용)* - **영국 (LSE)** → `yahoo` · `yfinance` · `local` *(명시된 GBP/GBp 호가만 허용)*
- **크립토** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **크립토** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -662,9 +667,9 @@ Paper-vs-live는 **구조적 브로커별 런타임 가드**(account-id 형식,
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>5개 패밀리에 걸친 462개 사전 빌드된 quant alpha</sub></summary> <summary><b>Alpha Zoo</b> <sub>5개 패밀리에 걸친 475개 사전 빌드된 quant alpha</sub></summary>
- 🧬 operator 계층에서 lookahead가 금지된 462개 cross-sectional alpha - 🧬 operator 계층에서 lookahead가 금지된 475개 cross-sectional alpha
- 📈 한 줄 CLI로 IC + IR + alive/reversed/dead 분류 수행 - 📈 한 줄 CLI로 IC + IR + alive/reversed/dead 분류 수행
- 🔬 AST 순수성 게이트 + 300-row lookahead sentinel 테스트 + `pytest-socket` 네트워크 kill-switch - 🔬 AST 순수성 게이트 + 300-row lookahead sentinel 테스트 + `pytest-socket` 네트워크 kill-switch
- 📦 Qlib에 대한 Apache-2 출처 표기, zoo별 `LICENSE.md`에서 수식을 수학적 콘텐츠로 명시 - 📦 Qlib에 대한 Apache-2 출처 표기, zoo별 `LICENSE.md`에서 수식을 수학적 콘텐츠로 명시
@@ -683,7 +688,7 @@ Paper-vs-live는 **구조적 브로커별 런타임 가드**(account-id 형식,
</details> </details>
<details> <details>
<summary><b>백테스트 엔진</b> <sub>10개 엔진 + 옵션 포트폴리오, 크로스마켓 composite</sub></summary> <summary><b>백테스트 엔진</b> <sub>11개 엔진 + 옵션 포트폴리오, 크로스마켓 composite</sub></summary>
| Engine | Market | Notes | | Engine | Market | Notes |
|--------|--------|-------| |--------|--------|-------|
@@ -692,6 +697,7 @@ Paper-vs-live는 **구조적 브로커별 런타임 가드**(account-id 형식,
| **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack | | **IndiaEquity** | India (NSE/BSE) | T+1, circuit bands, config-driven STT / stamp / SEBI / GST cost stack |
| **KoreaEquity** | 한국 (KRX: KOSPI/KOSDAQ) | 롱 온리, 통합 호가 단위에서 ±30% 가격제한폭을 체결 시점에 판정, 2026년 0.20% 증권거래세 | | **KoreaEquity** | 한국 (KRX: KOSPI/KOSDAQ) | 롱 온리, 통합 호가 단위에서 ±30% 가격제한폭을 체결 시점에 판정, 2026년 0.20% 증권거래세 |
| **VietnamEquity** | 베트남 (HOSE) | 롱 온리, T+2 결제 보유, 10/50/100 VND 호가 단위에서 ±7% 가격제한폭, 100주 단위, 매도측 0.1% 세금 | | **VietnamEquity** | 베트남 (HOSE) | 롱 온리, T+2 결제 보유, 10/50/100 VND 호가 단위에서 ±7% 가격제한폭, 100주 단위, 매도측 0.1% 세금 |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split | | **Crypto** | crypto spot / USD-M perps | funding settlements, execution/mark split |
| **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers | | **ChinaFutures** · **GlobalFutures** | futures | margin, contract multipliers |
| **Forex** | FX / metals | via the `mt5` loader | | **Forex** | FX / metals | via the `mt5` loader |
@@ -974,7 +980,7 @@ Vibe-Trading은 tool-heavy agent입니다. skills, backtests, memory, swarms가
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # 사전 빌드된 462개 alpha 탐색; show / bench / compare / export-manifest 서브커맨드 사용 가능 vibe-trading alpha list # 사전 빌드된 475개 alpha 탐색; show / bench / compare / export-manifest 서브커맨드 사용 가능
vibe-trading playbook list # 예약 리서치 템플릿 5개; show / create 서브커맨드 사용 가능 vibe-trading playbook list # 예약 리서치 템플릿 5개; show / create 서브커맨드 사용 가능
vibe-trading channels status --local # IM 채널 설정과 설치 힌트 확인 vibe-trading channels status --local # IM 채널 설정과 설치 힌트 확인
vibe-trading provider doctor # 마스킹된 provider/proxy/패키지 진단 출력 vibe-trading provider doctor # 마스킹된 provider/proxy/패키지 진단 출력
@@ -1721,7 +1727,7 @@ Vibe-Trading/
│ │ ├── memory/ # Cross-session persistent memory │ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/) │ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │ │ │ │
│ │ ├── tools/ # 109 auto-discovered agent tools │ │ ├── tools/ # 110 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests │ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget) │ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file) │ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
@@ -1730,7 +1736,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo web search │ │ │ ├── web_search_tool.py # DuckDuckGo web search
│ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc. │ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo — 5개 패밀리에 걸친 462개 alpha │ │ ├── factors/ # Alpha Zoo — 5개 패밀리에 걸친 475개 alpha
│ │ │ ├── base.py # 19개 operator (rank/scale/ts_*/delta/decay_linear/safe_div/vwap) │ │ │ ├── base.py # 19개 operator (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # AST-only 메타데이터 로딩 + lazy compute + sanity gate │ │ │ ├── registry.py # AST-only 메타데이터 로딩 + lazy compute + sanity gate
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead 분류 │ │ │ ├── bench_runner.py # IC + alive/reversed/dead 분류
@@ -1746,8 +1752,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM provider abstraction │ │ └── providers/ # LLM provider abstraction
│ │ │ │
│ └── backtest/ # Backtest engines │ └── backtest/ # Backtest engines
│ ├── engines/ # 9 engines + composite cross-market engine + options_portfolio │ ├── engines/ # 10 engines + composite cross-market engine + options_portfolio
│ ├── loaders/ # 28 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata │ ├── loaders/ # 29 sources: tushare, okx, nobitex, wallex, binance, yfinance, akshare, baostock, tencent, mootdx, ccxt, futu, pykrx, local, eastmoney, sina, stooq, yahoo, finnhub, alphavantage, tiingo, fmp, longbridge, mt5, qveris, india_broker, tickerall, gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + auto-fallback chains │ │ └── registry.py # Registry + auto-fallback chains
│ └── optimizers/ # MVO, equal vol, max div, risk parity │ └── optimizers/ # MVO, equal vol, max div, risk parity
@@ -1813,7 +1819,7 @@ Vibe-Trading은 **[HKUDS](https://github.com/HKUDS)** agent ecosystem의 일부
| **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | 로컬 로더 출시 | | **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | 로컬 로더 출시 |
| **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Planned | | **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Planned |
| **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover-aware optimizer **0.1.11 출시 완료**; 나머지 Planned | | **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Turnover-aware optimizer **0.1.11 출시 완료**; 나머지 Planned |
| **Alpha Zoo** | 5개 패밀리에 걸친 462개의 사전 빌드된 alpha 인자(Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental), 한 줄 CLI 벤치, agent 통합, Web UI | **0.1.8 출시 완료**, 0.1.12까지 확장 | | **Alpha Zoo** | 5개 패밀리에 걸친 475개의 사전 빌드된 alpha 인자(Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental), 한 줄 CLI 벤치, agent 통합, Web UI | **0.1.8 출시 완료**, 0.1.12까지 확장 |
| **Strategy Development Manager** | 논문 / 브로커 리서치를 영속 store + 자동 IC/Sharpe decay 라이프사이클과 함께 팩터 & 전략으로 등록 | **0.1.11 출시 완료** | | **Strategy Development Manager** | 논문 / 브로커 리서치를 영속 store + 자동 IC/Sharpe decay 라이프사이클과 함께 팩터 & 전략으로 등록 | **0.1.11 출시 완료** |
| **Correlation Regime** | `/correlation` 위에 얹은 엣지 밀도 + 히스테리시스 레짐 타임라인 — 시장이 하나의 블록으로 융합되는 시점 포착 | **0.1.12 출시 완료** | | **Correlation Regime** | `/correlation` 위에 얹은 엣지 밀도 + 히스테리시스 레짐 타임라인 — 시장이 하나의 블록으로 융합되는 시점 포착 | **0.1.12 출시 완료** |
| **Research Delivery** | Slack / Telegram / email-style IM channels를 통한 예약 brief와 live research sessions | 스케줄러 + IM Runtime 출시 | | **Research Delivery** | Slack / Telegram / email-style IM channels를 통한 예약 brief와 live research sessions | 스케줄러 + IM Runtime 출시 |
README_zh.md
+21 -15
View File
@@ -52,16 +52,19 @@
> ⚠️ **安全警告：** X 账号 `VibeTrading_HKU`、Virtuals 项目 `101845` 及代币合约 `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` 均非 Vibe-Trading 官方。我们从未发行或背书任何代币或 meme 币。请勿购买、连接钱包或签名。[详细说明](SECURITY.md#official-channels--impersonation)。 > ⚠️ **安全警告：** X 账号 `VibeTrading_HKU`、Virtuals 项目 `101845` 及代币合约 `0x640BDBF77b6447E8b7DB7894cED84BD1c40571f4` 均非 Vibe-Trading 官方。我们从未发行或背书任何代币或 meme 币。请勿购买、连接钱包或签名。[详细说明](SECURITY.md#official-channels--impersonation)。
- **2026-10-12** 🧮 **研究与回测更新**：修正开仓成交、清仓时归因、回撤时长、估值容差及因子 IC 的 SEC 财报可见时点（[#1780](https://github.com/HKUDS/Vibe-Trading/pull/1780)–[#1794](https://github.com/HKUDS/Vibe-Trading/pull/1794)）。新增 13 个基本面／估值因子、巴西数据与回测、需显式启用的 ZimaOS 模板，以及飞书运行进度。范围与数据限制见 [CHANGELOG](CHANGELOG.md)。
- **2026-10-11** 🛠️ **五项优先修复**：派生数字的每个数据操作数都须有证据，不支持的实盘期权继续拦截。离线基准保持指定数据源，交易记录区分零盈亏平仓，Codex 工具保留可选参数（[#1622](https://github.com/HKUDS/Vibe-Trading/issues/1622)、[#1779](https://github.com/HKUDS/Vibe-Trading/pull/1779)、[#1781](https://github.com/HKUDS/Vibe-Trading/pull/1781)、[#1785](https://github.com/HKUDS/Vibe-Trading/pull/1785)）。
- **2026-10-10** 🛠️ **Slack/Discord 配置与可复现研究**：本地化配置表单可测试机器人凭证，测试不会保存改动（[#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)、[#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)）。组合研究可复用不可变快照，Web 可导出脱敏 JSON（[#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)）。技术指标报告日期边界和输入指纹；损坏的会话记录可恢复且不重写原文件。另修复记忆召回、凭证脱敏和交易流水格式识别。详见 [CHANGELOG](CHANGELOG.md)。 - **2026-10-10** 🛠️ **Slack/Discord 配置与可复现研究**：本地化配置表单可测试机器人凭证，测试不会保存改动（[#1764](https://github.com/HKUDS/Vibe-Trading/pull/1764)、[#1777](https://github.com/HKUDS/Vibe-Trading/pull/1777)）。组合研究可复用不可变快照，Web 可导出脱敏 JSON（[#1769](https://github.com/HKUDS/Vibe-Trading/pull/1769)）。技术指标报告日期边界和输入指纹；损坏的会话记录可恢复且不重写原文件。另修复记忆召回、凭证脱敏和交易流水格式识别。详见 [CHANGELOG](CHANGELOG.md)。
<details>
<summary>更早的新闻</summary>
- **2026-10-09** 🛠️ **渠道设置与回测修正**：原地更新渠道设置时保留 URL 内嵌凭证和微信认证后的服务地址，Web 以九种语言说明保存与重启结果（[#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)、[#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)、[#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)）。跨市场期权波动率仅使用截至当前时点的时间戳，回撤纳入初始资金。修正加密货币／贵金属数据源路由、港股重复识别、缓存恢复、科学计数金额解析及缺失净值验证；mootdx 移出默认 A 股链。全部十三项 PR 见 [CHANGELOG](CHANGELOG.md)。 - **2026-10-09** 🛠️ **渠道设置与回测修正**：原地更新渠道设置时保留 URL 内嵌凭证和微信认证后的服务地址，Web 以九种语言说明保存与重启结果（[#1737](https://github.com/HKUDS/Vibe-Trading/pull/1737)、[#1742](https://github.com/HKUDS/Vibe-Trading/pull/1742)、[#1744](https://github.com/HKUDS/Vibe-Trading/pull/1744)）。跨市场期权波动率仅使用截至当前时点的时间戳，回撤纳入初始资金。修正加密货币／贵金属数据源路由、港股重复识别、缓存恢复、科学计数金额解析及缺失净值验证；mootdx 移出默认 A 股链。全部十三项 PR 见 [CHANGELOG](CHANGELOG.md)。
- **2026-10-08** 🔐 **PDF 报告密码保护与券商读取修复**：定时邮件 PDF 可选 AES-256 密码保护，覆盖 Web、CLI 模板和 Agent 确认入口；密码只保存在渠道私有配置中（[#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)）。KIS 读取三个交易场所的订单，明确过期的 token 只刷新一次，不完整的读取返回错误（[#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)）。派生公式保留说明标签之后的完整算式，证据核验规则保持不变（[#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)）。 - **2026-10-08** 🔐 **PDF 报告密码保护与券商读取修复**：定时邮件 PDF 可选 AES-256 密码保护，覆盖 Web、CLI 模板和 Agent 确认入口；密码只保存在渠道私有配置中（[#1709](https://github.com/HKUDS/Vibe-Trading/pull/1709)）。KIS 读取三个交易场所的订单，明确过期的 token 只刷新一次，不完整的读取返回错误（[#1726](https://github.com/HKUDS/Vibe-Trading/pull/1726)）。派生公式保留说明标签之后的完整算式，证据核验规则保持不变（[#1728](https://github.com/HKUDS/Vibe-Trading/pull/1728)）。
<details>
<summary>更早的新闻</summary>
- **2026-10-07** 🛠️ **研究连续性与风险指标修正**：运行状态移出系统提示词，保留稳定前缀以利于缓存（[#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)）。跨市场日期正确对齐，缺失价格不再拼接跨期收益，期权 Sortino 按全样本计算下行偏差（[#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)、[#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)、[#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)）。数据源测试共用 OHLCV 契约，并修正信用估值、风格敞口、滚动验证和审计诊断。 - **2026-10-07** 🛠️ **研究连续性与风险指标修正**：运行状态移出系统提示词，保留稳定前缀以利于缓存（[#1708](https://github.com/HKUDS/Vibe-Trading/pull/1708)）。跨市场日期正确对齐，缺失价格不再拼接跨期收益，期权 Sortino 按全样本计算下行偏差（[#1710](https://github.com/HKUDS/Vibe-Trading/pull/1710)、[#1717](https://github.com/HKUDS/Vibe-Trading/pull/1717)、[#1727](https://github.com/HKUDS/Vibe-Trading/pull/1727)）。数据源测试共用 OHLCV 契约，并修正信用估值、风格敞口、滚动验证和审计诊断。
- **2026-10-06** 🛠️ **实盘控制与报告纠错修复**：停止实盘运行会取消当前分析并等待调度器完成清理，覆盖启动期间取消及 API 关闭，运行状态时间单位也已修正（[#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)）。紧急撤单／平仓扫描跳过异常记录，继续处理其余订单和持仓（[#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)）；不同券商的调度独立保存，持久化正确处理并发写入与短写入。美股半日交易遵循提前收市时间（[#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)）。报告纠错保留已核验通过的数字，反馈截断也不会暗示未列出的数字已通过（[#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)）。 - **2026-10-06** 🛠️ **实盘控制与报告纠错修复**：停止实盘运行会取消当前分析并等待调度器完成清理，覆盖启动期间取消及 API 关闭，运行状态时间单位也已修正（[#1704](https://github.com/HKUDS/Vibe-Trading/pull/1704)）。紧急撤单／平仓扫描跳过异常记录，继续处理其余订单和持仓（[#1703](https://github.com/HKUDS/Vibe-Trading/pull/1703)）；不同券商的调度独立保存，持久化正确处理并发写入与短写入。美股半日交易遵循提前收市时间（[#1706](https://github.com/HKUDS/Vibe-Trading/pull/1706)）。报告纠错保留已核验通过的数字，反馈截断也不会暗示未列出的数字已通过（[#1702](https://github.com/HKUDS/Vibe-Trading/pull/1702)）。
@@ -376,7 +379,7 @@ Vibe-Trading 是一个开源研究工作台，用于把金融问题转化为可
| **运行分析师团队** | 面向投资、量化、加密、宏观和风控工作流的多智能体研究评审。 | | **运行分析师团队** | 面向投资、量化、加密、宏观和风控工作流的多智能体研究评审。 |
| **把研究接入 IM 通道** | 通过 WebSocket、Telegram、Slack、Discord、Matrix、WhatsApp、Signal、QQ/NapCat、微信/企业微信、飞书/Lark、钉钉、Teams、email、Mochat，在 CLI、REST 和 Web UI 中管理同一套 session runtime。 | | **把研究接入 IM 通道** | 通过 WebSocket、Telegram、Slack、Discord、Matrix、WhatsApp、Signal、QQ/NapCat、微信/企业微信、飞书/Lark、钉钉、Teams、email、Mochat，在 CLI、REST 和 Web UI 中管理同一套 session runtime。 |
| **交付可用成果** | 报告、TradingView Pine Script、TDX、MetaTrader 5、MCP tools，以及可延续的研究 sessions。 | | **交付可用成果** | 报告、TradingView Pine Script、TDX、MetaTrader 5、MCP tools，以及可延续的研究 sessions。 |
| **跑预置 alpha zoo 横评** | 462 个 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental），一行 CLI 在你选的 universe 上算 IC + IR + alive/reversed/dead 分类 | | **跑预置 alpha zoo 横评** | 475 个 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + PIT-safe fundamental），一行 CLI 在你选的 universe 上算 IC + IR + alive/reversed/dead 分类 |
| **识别相关性状态** | `/correlation` 界面上的边密度 + 迟滞时间线，显示市场何时融合为一个板块——属于描述性风险上下文，而非交易信号。 | | **识别相关性状态** | `/correlation` 界面上的边密度 + 迟滞时间线，显示市场何时融合为一个板块——属于描述性风险上下文，而非交易信号。 |
--- ---
@@ -476,7 +479,7 @@ vibe-trading connector install /tmp/my-broker
## 📡 数据源与智能 Fallback ## 📡 数据源与智能 Fallback
一次 `get_market_data` 调用，**28 个行情数据源**（其中 **QVeris** 是可选的付费市场）。设 `source: "auto"`——loader 按符号自动选源，再沿按 **被封 IP 风险** 排序的同市场链向下走（永不封的公开源在前，限速 / 需 key 的在后）。零配置，无单点故障。 一次 `get_market_data` 调用，**29 个行情数据源**（其中 **QVeris** 是可选的付费市场）。设 `source: "auto"`——loader 按符号自动选源，再沿按 **被封 IP 风险** 排序的同市场链向下走（永不封的公开源在前，限速 / 需 key 的在后）。零配置，无单点故障。
| Source | Markets | Auth | Role | | Source | Markets | Auth | Role |
|--------|---------|------|------| |--------|---------|------|------|
@@ -499,6 +502,7 @@ vibe-trading connector install /tmp/my-broker
| `tickerall` | 外汇 / 贵金属 | key + 账户（只读） | 同一家券商的 MT5 数据源，**托管** —— 无需本地终端，任意操作系统（仅显式选用，绝不进 auto 链） | | `tickerall` | 外汇 / 贵金属 | key + 账户（只读） | 同一家券商的 MT5 数据源，**托管** —— 无需本地终端，任意操作系统（仅显式选用，绝不进 auto 链） |
| `pykrx` | 韩国（KRX：KOSPI/KOSDAQ） | 无 | `.KS` / `.KQ` 的 KOSPI / KOSDAQ 日线（可选 `krx` extra） | | `pykrx` | 韩国（KRX：KOSPI/KOSDAQ） | 无 | `.KS` / `.KQ` 的 KOSPI / KOSDAQ 日线（可选 `krx` extra） |
| `india_broker` | 印度（NSE/BSE） | 券商登录 | 只读 Zerodha / Shoonya / Dhan bars，服务 `.NS` / `.BO`（fallback 链尾） | | `india_broker` | 印度（NSE/BSE） | 券商登录 | 只读 Zerodha / Shoonya / Dhan bars，服务 `.NS` / `.BO`（fallback 链尾） |
| `b3_cotahist` | Brazil (B3) | none | 本地 B3 官方现货／零股日线（原始价格、BRL），由 `python -m src.br_data sync` 导入；`.SA` 的离线后备源 |
| `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix | | `local` | any | none | your own CSV / Parquet / DuckDB via `local:` prefix |
**Fallback 链（按被封 IP 风险排序）：** **Fallback 链（按被封 IP 风险排序）：**
@@ -507,6 +511,7 @@ vibe-trading connector install /tmp/my-broker
- **美股** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local` - **美股** → `yahoo` · `stooq` · `sina` · `eastmoney` · `yfinance` · `tiingo` · `fmp` · `finnhub` · `alphavantage` · `longbridge` · `akshare` · `local`
- **港股** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local` - **港股** → `tencent` · `eastmoney` · `yahoo` · `futu` · `akshare` · `yfinance` · `tushare` · `longbridge` · `local`
- **印度（NSE/BSE）** → `yahoo` · `yfinance` · `india_broker` · `local` - **印度（NSE/BSE）** → `yahoo` · `yfinance` · `india_broker` · `local`
- **Brazil (B3 `.SA`)** → `yahoo` · `yfinance` · `b3_cotahist` · `local`
- **韩国（KOSPI/KOSDAQ）** → `pykrx` · `yahoo` · `yfinance` · `local` - **韩国（KOSPI/KOSDAQ）** → `pykrx` · `yahoo` · `yfinance` · `local`
- **英国（LSE）** → `yahoo` · `yfinance` · `local` *（仅接受明示为 GBP/GBp 的报价）* - **英国（LSE）** → `yahoo` · `yfinance` · `local` *（仅接受明示为 GBP/GBp 的报价）*
- **加密** → `okx` · `binance` · `ccxt` · `yfinance` · `local` - **加密** → `okx` · `binance` · `ccxt` · `yfinance` · `local`
@@ -675,9 +680,9 @@ LONGBRIDGE_ACCESS_TOKEN=...
</details> </details>
<details> <details>
<summary><b>Alpha Zoo</b> <sub>462 个预置 alpha，覆盖 5 个家族</sub></summary> <summary><b>Alpha Zoo</b> <sub>475 个预置 alpha，覆盖 5 个家族</sub></summary>
- 🧬 462 个横截面 alpha，算子层即禁用 lookahead - 🧬 475 个横截面 alpha，算子层即禁用 lookahead
- 📈 一条 CLI 命令完成 IC + IR + alive/reversed/dead 分类 - 📈 一条 CLI 命令完成 IC + IR + alive/reversed/dead 分类
- 🔬 AST 纯函数门禁 + 300 行 lookahead 哨兵测试 + `pytest-socket` 网络阻断 - 🔬 AST 纯函数门禁 + 300 行 lookahead 哨兵测试 + `pytest-socket` 网络阻断
- 📦 Qlib 部分附 Apache-2 出处声明；每个 zoo 一份 `LICENSE.md`，声明公式属于数学内容 - 📦 Qlib 部分附 Apache-2 出处声明；每个 zoo 一份 `LICENSE.md`，声明公式属于数学内容
@@ -696,7 +701,7 @@ LONGBRIDGE_ACCESS_TOKEN=...
</details> </details>
<details> <details>
<summary><b>Backtest Engines</b> <sub>10 个引擎 + options portfolio，跨市场 composite</sub></summary> <summary><b>Backtest Engines</b> <sub>11 个引擎 + options portfolio，跨市场 composite</sub></summary>
| 引擎 | 市场 | 说明 | | 引擎 | 市场 | 说明 |
|------|------|------| |------|------|------|
@@ -705,6 +710,7 @@ LONGBRIDGE_ACCESS_TOKEN=...
| **IndiaEquity** | 印度（NSE/BSE） | T+1、熔断带、config 驱动的 STT / 印花税 / SEBI / GST 成本栈 | | **IndiaEquity** | 印度（NSE/BSE） | T+1、熔断带、config 驱动的 STT / 印花税 / SEBI / GST 成本栈 |
| **KoreaEquity** | 韩国（KRX：KOSPI/KOSDAQ） | 只做多，统一最小价位网格上于成交时刻判定 ±30% 涨跌停，2026 年 0.20% 证券交易税 | | **KoreaEquity** | 韩国（KRX：KOSPI/KOSDAQ） | 只做多，统一最小价位网格上于成交时刻判定 ±30% 涨跌停，2026 年 0.20% 证券交易税 |
| **VietnamEquity** | 越南（HOSE） | 只做多，T+2 交收锁定，10/50/100 越南盾最小价位网格上 ±7% 涨跌停，100 股整手，0.1% 卖出方税 | | **VietnamEquity** | 越南（HOSE） | 只做多，T+2 交收锁定，10/50/100 越南盾最小价位网格上 ±7% 涨跌停，100 股整手，0.1% 卖出方税 |
| **BrazilEquity** | Brazil (B3) | BRL, B3 trading + settlement fees, configurable broker fee, 1- or 100-share lots, R$0.01 tick, shorting allowed |
| **Crypto** | 加密现货 / USD-M 永续 | 资金费结算、成交价/标记价分离 | | **Crypto** | 加密现货 / USD-M 永续 | 资金费结算、成交价/标记价分离 |
| **ChinaFutures** · **GlobalFutures** | 期货 | 保证金、合约乘数 | | **ChinaFutures** · **GlobalFutures** | 期货 | 保证金、合约乘数 |
| **Forex** | 外汇 / 贵金属 | 经 `mt5` loader | | **Forex** | 外汇 / 贵金属 | 经 `mt5` loader |
@@ -981,7 +987,7 @@ Vibe-Trading 是高度依赖工具的智能体：skills、backtests、memory 和
vibe-trading # interactive TUI vibe-trading # interactive TUI
vibe-trading run -p "..." # single run vibe-trading run -p "..." # single run
vibe-trading serve # API server vibe-trading serve # API server
vibe-trading alpha list # 浏览 462 个预置 alpha；支持 show / bench / compare / export-manifest 子命令 vibe-trading alpha list # 浏览 475 个预置 alpha；支持 show / bench / compare / export-manifest 子命令
vibe-trading playbook list # 五个定时研究模板；支持 show / create 子命令 vibe-trading playbook list # 五个定时研究模板；支持 show / create 子命令
vibe-trading channels status --local # 检查 IM 通道配置和依赖安装提示 vibe-trading channels status --local # 检查 IM 通道配置和依赖安装提示
vibe-trading provider doctor # 打印脱敏后的 provider/proxy/依赖诊断 vibe-trading provider doctor # 打印脱敏后的 provider/proxy/依赖诊断
@@ -1719,7 +1725,7 @@ Vibe-Trading/
│ │ ├── memory/ # 跨 session 持久记忆 │ │ ├── memory/ # 跨 session 持久记忆
│ │ │ └── persistent.py # 基于文件的记忆（~/.vibe-trading/memory/） │ │ │ └── persistent.py # 基于文件的记忆（~/.vibe-trading/memory/）
│ │ │ │ │ │
│ │ ├── tools/ # 109 个自动发现的 agent 工具 │ │ ├── tools/ # 110 个自动发现的 agent 工具
│ │ │ ├── backtest_tool.py # 运行回测 │ │ │ ├── backtest_tool.py # 运行回测
│ │ │ ├── remember_tool.py # 跨 session 记忆（save/recall/forget） │ │ │ ├── remember_tool.py # 跨 session 记忆（save/recall/forget）
│ │ │ ├── skill_writer_tool.py # skill CRUD（save/patch/delete/file） │ │ │ ├── skill_writer_tool.py # skill CRUD（save/patch/delete/file）
@@ -1728,7 +1734,7 @@ Vibe-Trading/
│ │ │ ├── web_search_tool.py # DuckDuckGo 网络搜索 │ │ │ ├── web_search_tool.py # DuckDuckGo 网络搜索
│ │ │ └── ... # bash、文件 I/O、因子分析、期权、alpha 浏览 + 横评等 │ │ │ └── ... # bash、文件 I/O、因子分析、期权、alpha 浏览 + 横评等
│ │ │ │ │ │
│ │ ├── factors/ # Alpha Zoo —— 5 个家族共 462 个 alpha │ │ ├── factors/ # Alpha Zoo —— 5 个家族共 475 个 alpha
│ │ │ ├── base.py # 19 个算子（rank/scale/ts_*/delta/decay_linear/safe_div/vwap） │ │ │ ├── base.py # 19 个算子（rank/scale/ts_*/delta/decay_linear/safe_div/vwap）
│ │ │ ├── registry.py # 纯 AST 元数据加载 + 惰性计算 + sanity 校验 │ │ │ ├── registry.py # 纯 AST 元数据加载 + 惰性计算 + sanity 校验
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead 分类 │ │ │ ├── bench_runner.py # IC + alive/reversed/dead 分类
@@ -1744,8 +1750,8 @@ Vibe-Trading/
│ │ └── providers/ # LLM provider 抽象层 │ │ └── providers/ # LLM provider 抽象层
│ │ │ │
│ └── backtest/ # 回测引擎 │ └── backtest/ # 回测引擎
│ ├── engines/ # 9 个引擎 + 跨市场 composite 引擎 + options_portfolio │ ├── engines/ # 10 个引擎 + 跨市场 composite 引擎 + options_portfolio
│ ├── loaders/ # 28 个数据源：tushare、okx、nobitex、wallex、binance、yfinance、akshare、baostock、tencent、mootdx、ccxt、futu、pykrx、local、eastmoney、sina、stooq、yahoo、finnhub、alphavantage、tiingo、fmp、longbridge、mt5、qveris、india_broker、tickerall、gildata │ ├── loaders/ # 29 个数据源：tushare、okx、nobitex、wallex、binance、yfinance、akshare、baostock、tencent、mootdx、ccxt、futu、pykrx、local、eastmoney、sina、stooq、yahoo、finnhub、alphavantage、tiingo、fmp、longbridge、mt5、qveris、india_broker、tickerall、gildata, b3_cotahist
│ │ ├── base.py # DataLoader Protocol │ │ ├── base.py # DataLoader Protocol
│ │ └── registry.py # Registry + 自动 fallback 链路 │ │ └── registry.py # Registry + 自动 fallback 链路
│ └── optimizers/ # MVO、equal vol、max div、risk parity │ └── optimizers/ # MVO、equal vol、max div、risk parity
@@ -1811,7 +1817,7 @@ Vibe-Trading 是 **[HKUDS](https://github.com/HKUDS)** 智能体生态的一部
| **Data Bridge** | 自带数据：本地 CSV/Parquet/SQL connectors 与 schema mapping | 本地加载器已发布 | | **Data Bridge** | 自带数据：本地 CSV/Parquet/SQL connectors 与 schema mapping | 本地加载器已发布 |
| **Options Lab** | Vol surface、Greeks dashboard、payoff/scenario explorer | Planned | | **Options Lab** | Vol surface、Greeks dashboard、payoff/scenario explorer | Planned |
| **Portfolio Studio** | Risk x-ray、constraints、turnover-aware optimizer、rebalance notes | Turnover-aware optimizer **已发布 0.1.11**；其余 Planned | | **Portfolio Studio** | Risk x-ray、constraints、turnover-aware optimizer、rebalance notes | Turnover-aware optimizer **已发布 0.1.11**；其余 Planned |
| **Alpha Zoo** | 462 个预置 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental），一行 CLI 跑横评，agent 集成，Web UI 浏览 | **已发布 0.1.8**，延续至 0.1.12 | | **Alpha Zoo** | 475 个预置 alpha 因子（Qlib 158 + Kakushadze 101 + GTJA 191 + academic + fundamental），一行 CLI 跑横评，agent 集成，Web UI 浏览 | **已发布 0.1.8**，延续至 0.1.12 |
| **Strategy Development Manager** | 把论文 / 券商研报注册为因子与策略，配持久化 store + 自动化 IC/Sharpe 衰减生命周期 | **已发布 0.1.11** | | **Strategy Development Manager** | 把论文 / 券商研报注册为因子与策略，配持久化 store + 自动化 IC/Sharpe 衰减生命周期 | **已发布 0.1.11** |
| **Correlation Regime** | 叠加在 `/correlation` 上的边密度 + 迟滞状态时间线——识别市场何时融合为一个板块 | **已发布 0.1.12** | | **Correlation Regime** | 叠加在 `/correlation` 上的边密度 + 迟滞状态时间线——识别市场何时融合为一个板块 | **已发布 0.1.12** |
| **Research Delivery** | 通过 Slack / Telegram / email-style IM channels 发送定时 briefs 与实时研究 sessions | 调度器 + IM Runtime 已发布 | | **Research Delivery** | 通过 Slack / Telegram / email-style IM channels 发送定时 briefs 与实时研究 sessions | 调度器 + IM Runtime 已发布 |
agent/SKILL.md
+7 -7
View File
@@ -1,7 +1,7 @@
--- ---
name: vibe-trading name: vibe-trading
version: 0.1.16 version: 0.1.16
description: Professional finance research toolkit — backtesting (10 engines + benchmark comparison panel), factor analysis, Alpha Zoo (462 pre-built alphas across qlib158/alpha101/gtja191/academic/fundamental), options pricing, 90 finance skills, 30 multi-agent swarm teams, Trade Journal analyzer, and Shadow Account (extract → backtest → render) across 28 market-data sources (tushare, yfinance, okx, binance, akshare, baostock, tencent, mootdx, ccxt, futu, mt5, tickerall, local, eastmoney, sina, stooq, yahoo, pykrx, india_broker, qveris, longbridge, nobitex, wallex, plus optional-key finnhub/alphavantage/tiingo/fmp/gildata). description: Professional finance research toolkit — backtesting (11 engines + benchmark comparison panel), factor analysis, Alpha Zoo (475 pre-built alphas across qlib158/alpha101/gtja191/academic/fundamental), options pricing, 90 finance skills, 30 multi-agent swarm teams, Trade Journal analyzer, and Shadow Account (extract → backtest → render) across 29 market-data sources (tushare, yfinance, okx, binance, akshare, baostock, tencent, mootdx, ccxt, futu, mt5, tickerall, local, eastmoney, sina, stooq, yahoo, pykrx, india_broker, qveris, longbridge, nobitex, wallex, plus optional-key finnhub/alphavantage/tiingo/fmp/gildata).
dependencies: dependencies:
python: ">=3.11" python: ">=3.11"
pip: pip:
@@ -23,7 +23,7 @@ mcp:
# Vibe-Trading # Vibe-Trading
Professional finance research toolkit with AI-powered backtesting (10 engines), multi-agent teams, 90 specialized skills, the **Alpha Zoo** (462 pre-built quantitative alphas across qlib158 / alpha101 / gtja191 / academic / fundamental with one-line CLI benchmarking), and the Shadow Account loop — extract your implicit trading rules from a journal, backtest them across A股/港股/美股/crypto, then see where they would have served you better. Professional finance research toolkit with AI-powered backtesting (11 engines), multi-agent teams, 90 specialized skills, the **Alpha Zoo** (475 pre-built quantitative alphas across qlib158 / alpha101 / gtja191 / academic / fundamental with one-line CLI benchmarking), and the Shadow Account loop — extract your implicit trading rules from a journal, backtest them across A股/港股/美股/crypto, then see where they would have served you better.
## Setup ## Setup
@@ -53,7 +53,7 @@ Add to your agent's MCP config:
### API Key Requirements ### API Key Requirements
Core research MCP tools work with zero API keys for HK/US/Canada/crypto. After `pip install`, backtesting, market data, factor analysis, options pricing, chart patterns, web search, document reading, trade journal analysis, shadow-account extraction/backtest/report, the Alpha Zoo (462 pre-built alphas), and all 90 skills are ready to use. IBKR tools require a local TWS / IB Gateway session; `run_swarm` requires an LLM key. Core research MCP tools work with zero API keys for HK/US/Canada/crypto. After `pip install`, backtesting, market data, factor analysis, options pricing, chart patterns, web search, document reading, trade journal analysis, shadow-account extraction/backtest/report, the Alpha Zoo (475 pre-built alphas), and all 90 skills are ready to use. IBKR tools require a local TWS / IB Gateway session; `run_swarm` requires an LLM key.
| Feature | Key needed | When | | Feature | Key needed | When |
|---------|-----------|------| |---------|-----------|------|
@@ -74,7 +74,7 @@ Feed a CSV broker export (同花顺 / 东财 / 富途 / generic), and the agent
5. `scan_shadow_signals` — list today's symbols that match your shadow's entry cadence (research only). 5. `scan_shadow_signals` — list today's symbols that match your shadow's entry cadence (research only).
### Backtesting ### Backtesting
Create and run quantitative strategies across 10 engines (ChinaA, GlobalEquity, IndiaEquity, KoreaEquity, VietnamEquity, Crypto, ChinaFutures, GlobalFutures, Forex + options) with 28 market-data sources (auto-detect + ordered fallback; the hosted forex `tickerall` source is explicit-only): Create and run quantitative strategies across 11 engines (ChinaA, GlobalEquity, IndiaEquity, KoreaEquity, VietnamEquity, BrazilEquity, Crypto, ChinaFutures, GlobalFutures, Forex + options) with 29 market-data sources (auto-detect + ordered fallback; the hosted forex `tickerall` source is explicit-only):
- **HK/US equities** via yfinance / stooq / yahoo (free, no API key); optionally via **Longbridge** historical OHLCV (`longbridge`, requires the optional SDK and `LONGBRIDGE_APP_KEY` / `LONGBRIDGE_APP_SECRET` / `LONGBRIDGE_ACCESS_TOKEN`). To force it for a run, set `"source": "longbridge"` in `config.json`. - **HK/US equities** via yfinance / stooq / yahoo (free, no API key); optionally via **Longbridge** historical OHLCV (`longbridge`, requires the optional SDK and `LONGBRIDGE_APP_KEY` / `LONGBRIDGE_APP_SECRET` / `LONGBRIDGE_ACCESS_TOKEN`). To force it for a run, set `"source": "longbridge"` in `config.json`.
- **Canada equities (TSX/TSXV)** via yahoo / yfinance using Yahoo's canonical `<TICKER>.TO` (TSX, e.g. `TD.TO`) or `<TICKER>.V` (TSXV, e.g. `PNG.V`) suffixes — free, no API key. The GlobalEquity engine uses CAD identity, whole-share orders, configurable Canadian commission/slippage, and the TSX/TSXV price-increment grid. - **Canada equities (TSX/TSXV)** via yahoo / yfinance using Yahoo's canonical `<TICKER>.TO` (TSX, e.g. `TD.TO`) or `<TICKER>.V` (TSXV, e.g. `PNG.V`) suffixes — free, no API key. The GlobalEquity engine uses CAD identity, whole-share orders, configurable Canadian commission/slippage, and the TSX/TSXV price-increment grid.
- **India equities (NSE/BSE)** via yahoo / yfinance using `<SYMBOL>.NS` (NSE, e.g. `RELIANCE.NS`) or `<SCRIP>.BO` (BSE, e.g. `500325.BO`) — free, no API key. The `IndiaEquityEngine` models T+1 delivery, no overnight shorts (set `allow_short` for intraday), configurable circuit bands, 1-share lots, and the STT/stamp-duty/exchange/GST cost stack. Optionally back-fill from your live broker via the `india_broker` source (Shoonya/Dhan; requires broker login). - **India equities (NSE/BSE)** via yahoo / yfinance using `<SYMBOL>.NS` (NSE, e.g. `RELIANCE.NS`) or `<SCRIP>.BO` (BSE, e.g. `500325.BO`) — free, no API key. The `IndiaEquityEngine` models T+1 delivery, no overnight shorts (set `allow_short` for intraday), configurable circuit bands, 1-share lots, and the STT/stamp-duty/exchange/GST cost stack. Optionally back-fill from your live broker via the `india_broker` source (Shoonya/Dhan; requires broker login).
@@ -110,13 +110,13 @@ Example workflow:
Use `list_swarm_presets()` to see all teams, then `run_swarm()` to execute. Use `list_swarm_presets()` to see all teams, then `run_swarm()` to execute.
### Alpha Zoo (462 pre-built alphas) ### Alpha Zoo (475 pre-built alphas)
One-line cross-sectional IC / IR / alive-reversed-dead categorisation across five bundled zoos: One-line cross-sectional IC / IR / alive-reversed-dead categorisation across five bundled zoos:
- **qlib158** (154 alphas) — Microsoft Qlib's `Alpha158` feature handler, Apache-2.0 with pinned commit SHA. - **qlib158** (154 alphas) — Microsoft Qlib's `Alpha158` feature handler, Apache-2.0 with pinned commit SHA.
- **alpha101** (101 alphas) — Kakushadze (2015) "101 Formulaic Alphas" (arXiv:1601.00991), written from the paper appendix. - **alpha101** (101 alphas) — Kakushadze (2015) "101 Formulaic Alphas" (arXiv:1601.00991), written from the paper appendix.
- **gtja191** (191 alphas) — Guotai Junan 2014 "191 Short-period Trading Alpha Factors" research report. - **gtja191** (191 alphas) — Guotai Junan 2014 "191 Short-period Trading Alpha Factors" research report.
- **academic** (12 factors) — Fama-French 5 + Carhart momentum + Jegadeesh reversal + George-Hwang 52-week-high + Amihud illiquidity + Harvey-Siddique skew + Frazzini-Pedersen betting-against-beta (price-based proxies) + a correlation-rewiring stability score (from the in-repo correlation-regime skill). - **academic** (12 factors) — Fama-French 5 + Carhart momentum + Jegadeesh reversal + George-Hwang 52-week-high + Amihud illiquidity + Harvey-Siddique skew + Frazzini-Pedersen betting-against-beta (price-based proxies) + a correlation-rewiring stability score (from the in-repo correlation-regime skill).
- **fundamental** (4 factors) — PIT-safe earnings yield, ROE, gross profitability, and asset growth from daily fundamental panels. - **fundamental** (17 factors) — PIT-safe earnings yield, ROE, gross profitability and asset growth, plus ten SEC XBRL additions: book-to-market, FCF yield, accruals, ROA, CFO yield, operating margin, cash-to-assets, leverage, revenue growth and earnings growth, from daily fundamental panels; and three CN A-share daily-valuation factors (PB, PS_TTM, dividend yield) fed by the new ``val:*`` panel namespace from tushare daily_basic.
Each alpha ships with `__alpha_meta__` (formula LaTeX + theme + universe + warmup + columns required), guarded by an AST purity gate + 300-row lookahead sentinel test. Use the `vibe-trading alpha {list,show,bench,compare,export-manifest}` CLI, the `/alpha/*` REST routes (browser at `/alpha-zoo`), or compose multi-factor signals via `ZooSignalEngine.from_zoo(...)`. Each alpha ships with `__alpha_meta__` (formula LaTeX + theme + universe + warmup + columns required), guarded by an AST purity gate + 300-row lookahead sentinel test. Use the `vibe-trading alpha {list,show,bench,compare,export-manifest}` CLI, the `/alpha/*` REST routes (browser at `/alpha-zoo`), or compose multi-factor signals via `ZooSignalEngine.from_zoo(...)`.
@@ -150,7 +150,7 @@ Use `load_skill(name)` to access full methodology docs with code templates.
| `analyze_options` | Black-Scholes price + Greeks | None | | `analyze_options` | Black-Scholes price + Greeks | None |
| `analyze_options_payoff` | Multi-leg expiry payoff + spot/IV scenarios | None | | `analyze_options_payoff` | Multi-leg expiry payoff + spot/IV scenarios | None |
| `pattern_recognition` | Detect chart patterns (H&S, double top, etc.) | None | | `pattern_recognition` | Detect chart patterns (H&S, double top, etc.) | None |
| `get_market_data` | Fetch OHLCV data (auto-detect + ordered fallback across 28 sources) | None* | | `get_market_data` | Fetch OHLCV data (auto-detect + ordered fallback across 29 sources) | None* |
| `get_fund_flow` | Capital fund-flow (main/retail net inflow) | None* | | `get_fund_flow` | Capital fund-flow (main/retail net inflow) | None* |
| `get_dragon_tiger` | Dragon-tiger list (龙虎榜) top buyer/seller seats | None* | | `get_dragon_tiger` | Dragon-tiger list (龙虎榜) top buyer/seller seats | None* |
| `get_northbound_flow` | Northbound (Stock Connect) net flow | None* | | `get_northbound_flow` | Northbound (Stock Connect) net flow | None* |
agent/backtest/benchmark.py
+12 -8
View File
@@ -12,6 +12,7 @@ from typing import Any, Optional
import pandas as pd import pandas as pd
from backtest.loaders.base import resample_bars, source_interval from backtest.loaders.base import resample_bars, source_interval
from backtest.loaders.registry import is_no_network_fallback_source
from backtest.loaders.yfinance_loader import DataLoader as YfinanceLoader from backtest.loaders.yfinance_loader import DataLoader as YfinanceLoader
from backtest.metrics import bar_returns, buy_and_hold_return from backtest.metrics import bar_returns, buy_and_hold_return
@@ -24,6 +25,7 @@ MARKET_BENCHMARKS: dict[str, Optional[str]] = {
"us_equity": "SPY", "us_equity": "SPY",
"hk_equity": "HK.03100", # Hang Seng China Enterprises ETF "hk_equity": "HK.03100", # Hang Seng China Enterprises ETF
"ca_equity": "XIC.TO", # S&P/TSX Capped Composite ETF "ca_equity": "XIC.TO", # S&P/TSX Capped Composite ETF
"br_equity": "BOVA11.SA", # iShares Ibovespa ETF
"a_share": "000300.SH", # CSI 300 (China A-share core index) "a_share": "000300.SH", # CSI 300 (China A-share core index)
"crypto": "BTC-USDT", "crypto": "BTC-USDT",
"futures": "ES.CME", # E-mini S&P 500 futures "futures": "ES.CME", # E-mini S&P 500 futures
@@ -84,8 +86,8 @@ def resolve_benchmark(
explicit: Override ticker (e.g. "SPY" passed via config). explicit: Override ticker (e.g. "SPY" passed via config).
loader: Loader of the configured data source. When given, the loader: Loader of the configured data source. When given, the
benchmark is fetched through it first, falling back to benchmark is fetched through it first, falling back to
yfinance if it yields no data — except ``local``, yfinance if it yields no data, except sources whose
which fails closed to keep offline runs offline. contract prohibits a fallback to a different source.
Returns: Returns:
BenchmarkResult with return series and total return, or None if no BenchmarkResult with return series and total return, or None if no
@@ -95,18 +97,18 @@ def resolve_benchmark(
if ticker is None: if ticker is None:
return None return None
offline = source == "local" source_bound = is_no_network_fallback_source(source)
if offline and getattr(loader, "name", None) != source: if source_bound and getattr(loader, "name", None) != source:
# The runtime fallback chain in fetch_data_map() may have swapped in a # Do not let a swapped loader bypass the configured source's boundary:
# network loader while config["source"] still says local — never fetch # local must stay offline, and source-specific quote currencies and
# the benchmark through it. Fail closed instead. # explicit provider choices must not silently change for a benchmark.
loader = None loader = None
try: try:
bench_df = _fetch_benchmark( bench_df = _fetch_benchmark(
ticker, start_date, end_date, interval, ticker, start_date, end_date, interval,
loader=loader, loader=loader,
allow_fallback=not offline, allow_fallback=not source_bound,
) )
except Exception: except Exception:
return None return None
@@ -176,6 +178,8 @@ def _infer_market(codes: list[str], source: str) -> str:
return "ca_equity" return "ca_equity"
if first.endswith((".NS", ".BO")): if first.endswith((".NS", ".BO")):
return "india_equity" return "india_equity"
if first.endswith(".SA"):
return "br_equity"
if first.endswith((".KS", ".KQ")): if first.endswith((".KS", ".KQ")):
return "kr_equity" return "kr_equity"
crypto_quotes = ("-USDT", "-USDC", "-USD", "-BTC", "-ETH") crypto_quotes = ("-USDT", "-USDC", "-USD", "-BTC", "-ETH")
agent/backtest/engines/_market_hooks.py
+5 -1
View File
@@ -110,6 +110,9 @@ _MARKET_PATTERNS = [
# Argentina: BYMA listings use Yahoo's canonical .BA suffix. Keep this # Argentina: BYMA listings use Yahoo's canonical .BA suffix. Keep this
# as a distinct market so ARS can never be mixed with USD/CNY accounting. # as a distinct market so ARS can never be mixed with USD/CNY accounting.
(re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "ar_equity"), (re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "ar_equity"),
# Brazil: B3 listings (PETR4.SA, BOVA11.SA, HGLG11.SA, AAPL34.SA) use
# Yahoo's canonical .SA suffix; every line settles in BRL.
(re.compile(r"^[A-Z0-9]+\.SA$", re.I), "br_equity"),
# UK equities: London Stock Exchange (VOD.L, SHEL.L). Yahoo carries the # UK equities: London Stock Exchange (VOD.L, SHEL.L). Yahoo carries the
# suffix verbatim. # suffix verbatim.
(re.compile(r"^[A-Z0-9&.\-]+\.L$", re.I), "uk_equity"), (re.compile(r"^[A-Z0-9&.\-]+\.L$", re.I), "uk_equity"),
@@ -227,6 +230,7 @@ _MARKET_CURRENCY = {
"kr_equity": "KRW", "kr_equity": "KRW",
"ca_equity": "CAD", "ca_equity": "CAD",
"ar_equity": "ARS", "ar_equity": "ARS",
"br_equity": "BRL",
"uk_equity": "GBP", "uk_equity": "GBP",
"vietnam_equity": "VND", "vietnam_equity": "VND",
# Every crypto pattern in _MARKET_PATTERNS is USDT-quoted, and USDT is # Every crypto pattern in _MARKET_PATTERNS is USDT-quoted, and USDT is
@@ -347,7 +351,7 @@ def _detect_market(code: str) -> str:
Returns: Returns:
Market type (a_share/us_equity/hk_equity/india_equity/kr_equity/ Market type (a_share/us_equity/hk_equity/india_equity/kr_equity/
ca_equity/ar_equity/crypto/futures/forex). ca_equity/ar_equity/br_equity/crypto/futures/forex).
Bare 1-5 letter alphabetic tickers resolve to ``us_equity``; Bare 1-5 letter alphabetic tickers resolve to ``us_equity``;
bare 6-letter codes that start with a precious-metal or G10 bare 6-letter codes that start with a precious-metal or G10
currency code (whitelist) resolve to ``forex``; concatenated currency code (whitelist) resolve to ``forex``; concatenated
agent/backtest/engines/base.py
+31 -4
View File
@@ -971,6 +971,7 @@ class BaseEngine(ABC):
) )
bench_ret = ret_df.mean(axis=1) if ret_df.shape[1] > 0 else pd.Series(0.0, index=dates) bench_ret = ret_df.mean(axis=1) if ret_df.shape[1] > 0 else pd.Series(0.0, index=dates)
benchmark_metadata = {} benchmark_metadata = {}
benchmark_warning: str | None = None
# ── External benchmark fetch ────────────────────────────────────────── # ── External benchmark fetch ──────────────────────────────────────────
bench_ticker = config.get("benchmark") bench_ticker = config.get("benchmark")
@@ -1006,6 +1007,16 @@ class BaseEngine(ABC):
window_ret if window_ret is not None else bench_result.total_ret window_ret if window_ret is not None else bench_result.total_ret
), ),
} }
else:
# The equal-weight basket stands in for the declared benchmark,
# so every benchmark-relative metric (excess_return,
# information_ratio, tracking_error, benchmark_beta) is graded
# against something the caller did not ask for. Say so.
benchmark_warning = (
f"declared benchmark {bench_ticker!r} could not be resolved; "
"benchmark-relative metrics are graded against the "
"equal-weight basket of the traded symbols instead"
)
# ── External benchmark fetch ────────────────────────────────────────── # ── External benchmark fetch ──────────────────────────────────────────
bench_equity = self.initial_capital * (1 + bench_ret).cumprod() bench_equity = self.initial_capital * (1 + bench_ret).cumprod()
@@ -1151,6 +1162,8 @@ class BaseEngine(ABC):
card_warnings.append(config["_run_card_caliber_warning"]) card_warnings.append(config["_run_card_caliber_warning"])
if config.get("_run_card_annualisation_warning"): if config.get("_run_card_annualisation_warning"):
card_warnings.append(config["_run_card_annualisation_warning"]) card_warnings.append(config["_run_card_annualisation_warning"])
if benchmark_warning:
card_warnings.append(benchmark_warning)
from backtest.run_card import write_run_card from backtest.run_card import write_run_card
write_run_card( write_run_card(
run_dir, run_dir,
@@ -2013,6 +2026,8 @@ class BaseEngine(ABC):
commission=entry_commission + order.exit_commission, commission=entry_commission + order.exit_commission,
entry_margin=order.released_margin, entry_margin=order.released_margin,
exit_margin=exit_margin, exit_margin=exit_margin,
open_price=before.open_price,
open_size=before.open_size,
)) ))
self._record_fill( self._record_fill(
symbol=before.symbol, symbol=before.symbol,
@@ -2053,6 +2068,8 @@ class BaseEngine(ABC):
leverage=order.leverage, leverage=order.leverage,
entry_bar_idx=self._bar_idx, entry_bar_idx=self._bar_idx,
entry_commission=order.commission, entry_commission=order.commission,
open_price=order.price,
open_size=order.size,
) )
self._record_fill( self._record_fill(
symbol=order.symbol, symbol=order.symbol,
@@ -2146,6 +2163,8 @@ class BaseEngine(ABC):
commission=pos.entry_commission + exit_comm, commission=pos.entry_commission + exit_comm,
entry_margin=margin, entry_margin=margin,
exit_margin=exit_margin, exit_margin=exit_margin,
open_price=pos.open_price,
open_size=pos.open_size,
)) ))
self._record_fill( self._record_fill(
symbol=symbol, symbol=symbol,
@@ -2273,21 +2292,28 @@ class BaseEngine(ABC):
target_out.index.name = "timestamp" target_out.index.name = "timestamp"
target_out.to_csv(out / "target_positions.csv") target_out.to_csv(out / "target_positions.csv")
# Trades (compatible format) # Trades (compatible format): two rows per round trip, entry then exit.
# The entry row is the opening fill as executed. A TradeRecord's
# entry_price is the weighted cost after any scale-ins while its
# entry_time stays on the first open, so pairing those two would
# report a fill that never happened. Scale-ins stay in fills.jsonl.
trade_rows = [] trade_rows = []
for t in self.trades: for t in self.trades:
# Entry event # Entry event
entry_price = t.entry_price if t.open_price is None else t.open_price
entry_qty = t.size if t.open_size is None else t.open_size
trade_rows.append({ trade_rows.append({
"timestamp": str(t.entry_time.date()) if hasattr(t.entry_time, "date") else str(t.entry_time), "timestamp": str(t.entry_time.date()) if hasattr(t.entry_time, "date") else str(t.entry_time),
"code": t.symbol, "code": t.symbol,
"side": "buy" if t.direction == 1 else "sell", "side": "buy" if t.direction == 1 else "sell",
"price": round(t.entry_price, 4), "price": round(entry_price, 4),
"qty": round(t.size, 6), "qty": round(entry_qty, 6),
"reason": "signal", "reason": "signal",
"pnl": 0.0, "pnl": 0.0,
"holding_days": 0, "holding_days": 0,
"holding_bars": 0.0, "holding_bars": 0.0,
"return_pct": 0.0, "return_pct": 0.0,
"event": "entry",
}) })
# Exit event # Exit event
try: try:
@@ -2305,11 +2331,12 @@ class BaseEngine(ABC):
"holding_days": hold_days, "holding_days": hold_days,
"holding_bars": t.holding_bars, "holding_bars": t.holding_bars,
"return_pct": round(t.pnl_pct, 2), "return_pct": round(t.pnl_pct, 2),
"event": "exit",
}) })
trade_cols = [ trade_cols = [
"timestamp", "code", "side", "price", "qty", "reason", "pnl", "timestamp", "code", "side", "price", "qty", "reason", "pnl",
"holding_days", "holding_bars", "return_pct", "holding_days", "holding_bars", "return_pct", "event",
] ]
pd.DataFrame(trade_rows or [], columns=trade_cols).to_csv(out / "trades.csv", index=False) pd.DataFrame(trade_rows or [], columns=trade_cols).to_csv(out / "trades.csv", index=False)
agent/backtest/engines/brazil_equity.py
+95
View File
@@ -0,0 +1,95 @@
"""Brazil equity (B3) backtest engine.
Covers everything B3 lists under Yahoo's ``.SA`` suffix: shares (PETR4.SA),
units, ETFs (BOVA11.SA), FIIs (HGLG11.SA) and BDRs (AAPL34.SA). All settle in
BRL.
Market rules:
- Same-session round trips are allowed (day trade). Settlement is D+2, which
restricts cash, not the ability to sell shares bought today.
- Short selling is allowed (via the B3 securities-lending facility, BTC).
Set ``allow_short=False`` to model a long-only account.
- No hard daily price limits: B3 uses auction tunnels that pause trading
rather than blocking a price, so every bar is executable.
- Whole shares. The standard lot is 100, but the odd-lot (fractional)
market accepts single shares, so the default lot is 1. Set
``br_lot_size=100`` to trade only round lots.
- Price increment R$0.01, rounded against the trader.
Cost stack (all config-driven). B3 tariffs change; verify against the current
B3 price list before relying on absolute figures:
- B3 trading fee (emolumentos): 0.005% bilateral [br_trading_fee]
- B3 settlement fee (liquidação): 0.025% bilateral [br_settlement_fee]
- Broker commission: 0% (most retail brokers) [br_brokerage_rate]
- Fixed broker fee per order: R$0 [br_brokerage_fixed]
Income tax (15% swing / 20% day trade on net gains, monthly R$20k sales
exemption for shares) is assessed on the investor's monthly result, not per
trade, so it is not charged here.
"""
from __future__ import annotations
import math
import pandas as pd
from backtest.engines.base import BaseEngine
_TICK = 0.01
class BrazilEquityEngine(BaseEngine):
"""B3 cash-equity engine.
Config keys (all optional; defaults shown in the module docstring):
- allow_short: bool, default True
- slippage_br: default 0.0005
- br_lot_size: default 1
- br_trading_fee / br_settlement_fee / br_brokerage_rate /
br_brokerage_fixed
"""
def __init__(self, config: dict):
config = {**config, "leverage": config.get("leverage", 1.0)}
super().__init__(config)
self.allow_short: bool = bool(config.get("allow_short", True))
self.slippage_br: float = config.get("slippage_br", 0.0005)
self.br_lot_size: int = max(int(config.get("br_lot_size", 1)), 1)
self.br_trading_fee: float = config.get("br_trading_fee", 0.00005)
self.br_settlement_fee: float = config.get("br_settlement_fee", 0.00025)
self.br_brokerage_rate: float = config.get("br_brokerage_rate", 0.0)
self.br_brokerage_fixed: float = config.get("br_brokerage_fixed", 0.0)
def can_execute(self, symbol: str, direction: int, bar: pd.Series) -> bool:
"""Every bar is executable; only opening a short can be disabled."""
if direction == -1 and not self.allow_short:
return False
return True
def round_size(self, raw_size: float, price: float) -> float:
"""Floor to whole lots (1 share by default, 100 for round lots)."""
lot = self.br_lot_size
return float(math.floor(max(raw_size, 0.0) / lot) * lot)
def calc_commission(self, size: float, price: float, direction: int, is_open: bool) -> float:
"""B3 fees (bilateral) plus the configured broker commission."""
notional = size * price
if notional <= 0:
return 0.0
comm = notional * (self.br_trading_fee + self.br_settlement_fee)
comm += notional * self.br_brokerage_rate
comm += self.br_brokerage_fixed
return comm
def apply_slippage(self, price: float, direction: int) -> float:
"""Apply slippage, then snap to the R$0.01 grid against the trader."""
slipped = price * (1 + direction * self.slippage_br)
units = slipped / _TICK
if direction > 0:
steps = math.ceil(units - 1e-9)
elif direction < 0:
steps = math.floor(units + 1e-9)
else:
steps = round(units)
return round(steps * _TICK, 2)
agent/backtest/engines/composite.py
+3
View File
@@ -55,6 +55,9 @@ def _build_rule_engines(config: dict, codes: List[str]) -> Dict[str, BaseEngine]
"Argentina .BA market data is supported, but Argentina backtest " "Argentina .BA market data is supported, but Argentina backtest "
"execution rules are not modeled yet" "execution rules are not modeled yet"
) )
elif market == "br_equity":
from backtest.engines.brazil_equity import BrazilEquityEngine
engines["br_equity"] = BrazilEquityEngine(config)
elif market == "ca_equity": elif market == "ca_equity":
from backtest.engines.global_equity import GlobalEquityEngine from backtest.engines.global_equity import GlobalEquityEngine
engines["ca_equity"] = GlobalEquityEngine(config, market="ca") engines["ca_equity"] = GlobalEquityEngine(config, market="ca")
agent/backtest/loader_health.py
+3
View File
@@ -44,6 +44,9 @@ CANARY_SYMBOLS = {
} }
EXCLUDED_PUBLIC_SOURCES = { EXCLUDED_PUBLIC_SOURCES = {
"local": "operator files, not a public endpoint", "local": "operator files, not a public endpoint",
# Reads the local SQLite store that `python -m src.br_data sync` fills
# from B3's COTAHIST files; it never calls a network endpoint itself.
"b3_cotahist": "local br-data store, synced separately",
# Retired 2026-10-08 (HKUDS/Vibe-Trading#1729): TDX servers stopped # Retired 2026-10-08 (HKUDS/Vibe-Trading#1729): TDX servers stopped
# answering the mootdx/tdxpy client protocol in 2026-07 # answering the mootdx/tdxpy client protocol in 2026-07
# (mootdx/mootdx#157) and upstream is dormant. The loader stays # (mootdx/mootdx#157) and upstream is dormant. The loader stays
agent/backtest/loaders/_fundamental_schema.py
+27
View File
@@ -150,6 +150,8 @@ SEC_CONCEPT_MAP: dict[str, list[str]] = {
"Cash", "Cash",
"CashAndDueFromBanks", "CashAndDueFromBanks",
], ],
# Basic shares are not diluted shares, even when rounded EPS values match.
# Missing diluted counts remain unavailable instead of guessing dilution.
"shares_diluted": [ "shares_diluted": [
"WeightedAverageNumberOfDilutedSharesOutstanding", "WeightedAverageNumberOfDilutedSharesOutstanding",
"WeightedAverageDilutedSharesOutstanding", "WeightedAverageDilutedSharesOutstanding",
@@ -194,6 +196,31 @@ DERIVED_FIELDS: dict[str, DerivedFieldSpec] = {
) )
- 1, - 1,
}, },
# Year-over-year revenue growth, looked up by the same calendar lag as
# asset_growth (see _lag_365d) rather than row position, since the real
# pipeline feeds this a daily-densified panel, not one row per annual
# period_end.
"revenue_growth": {
"dependencies": ["revenue"],
"compute": lambda data: _safe_divide(
data["revenue"].astype("float64"),
_lag_365d(data["revenue"].astype("float64")),
)
- 1,
},
# Year-over-year earnings growth, sign-safe: the denominator is the
# ABSOLUTE value of the lagged net_income, so a negative prior year does
# not flip the growth sign (NI -10 -> +5 must read as positive growth).
# A zero or NaN lag yields NaN via _safe_divide's zero-masking. Calendar
# lag for the same daily-densified-panel reason as asset_growth above.
"earnings_growth": {
"dependencies": ["net_income"],
"compute": lambda data: _safe_divide(
data["net_income"].astype("float64")
- _lag_365d(data["net_income"].astype("float64")),
_lag_365d(data["net_income"].astype("float64")).abs(),
),
},
"accruals": { "accruals": {
"dependencies": ["net_income", "cfo", "total_assets"], "dependencies": ["net_income", "cfo", "total_assets"],
"compute": lambda data: _safe_divide( "compute": lambda data: _safe_divide(
agent/backtest/loaders/b3_cotahist_loader.py
+90
View File
@@ -0,0 +1,90 @@
"""B3 COTAHIST loader: official daily bars for every B3 instrument.
Reads the local SQLite store filled by ``python -m src.br_data sync`` (see
:mod:`src.br_data.cotahist`); it never touches the network, so it is the
offline fallback of the ``br_equity`` chain after Yahoo/yfinance.
Symbol convention: ``PETR4.SA`` -> COTAHIST ``CODNEG`` ``PETR4``. Round-lot
rows (TPMERC 010) are served; ``PETR4F.SA`` reads the odd-lot market (020).
Prices are as traded (caliber ``raw``): no dividend/split adjustment.
"""
from __future__ import annotations
import logging
from typing import Dict, List, Optional
import pandas as pd
from backtest.loaders.base import normalize_declared_quote_currency, validate_date_range
from backtest.loaders.registry import register
logger = logging.getLogger(__name__)
_OUTPUT_COLUMNS = ["open", "high", "low", "close", "volume"]
_CASH_MARKET = 10
_ODD_LOT_MARKET = 20
def map_symbol(code: str) -> tuple[str, int]:
"""``PETR4.SA`` -> (``PETR4``, 10); ``PETR4F.SA`` -> (``PETR4F``, 20)."""
ticker = code.strip().upper()
if ticker.endswith(".SA"):
ticker = ticker[:-3]
market = _ODD_LOT_MARKET if ticker.endswith("F") and ticker[-2:-1].isdigit() else _CASH_MARKET
return ticker, market
@register
class DataLoader:
"""Offline B3 daily bars from the official COTAHIST files."""
name = "b3_cotahist"
markets = {"br_equity"}
# QUATOT is the number of securities traded.
volume_units = {"br_equity": "shares"}
requires_auth = False
def is_available(self) -> bool:
"""Available once a sync has created the store."""
from src.br_data.store import db_path
return db_path().exists()
def fetch(
self,
codes: List[str],
start_date: str,
end_date: str,
*,
interval: str = "1D",
fields: Optional[List[str]] = None,
) -> Dict[str, pd.DataFrame]:
"""Return ``{code: OHLCV frame}`` for codes present in the store."""
validate_date_range(start_date, end_date)
if str(interval).strip().lower() not in {"1d", "d", "day", "daily"}:
logger.warning("b3_cotahist serves daily bars only; rejecting interval=%r", interval)
return {}
from src.br_data.store import connect
result: Dict[str, pd.DataFrame] = {}
try:
with connect() as conn:
for code in codes:
ticker, market = map_symbol(code)
rows = conn.execute(
"SELECT trade_date, open, high, low, close, volume FROM b3_quotes "
"WHERE ticker = ? AND market_type = ? AND trade_date BETWEEN ? AND ? "
"ORDER BY trade_date",
(ticker, market, start_date[:10], end_date[:10]),
).fetchall()
if not rows:
continue
frame = pd.DataFrame([tuple(r) for r in rows], columns=["trade_date", *_OUTPUT_COLUMNS])
frame["trade_date"] = pd.to_datetime(frame["trade_date"])
frame = frame.set_index("trade_date").astype(float)
# The importer admits only explicit R$ records, excluding older currencies.
result[code] = normalize_declared_quote_currency(frame, code, "BRL")
except FileNotFoundError as exc:
logger.warning("b3_cotahist: %s", exc)
return result
agent/backtest/loaders/base.py
+4 -1
View File
@@ -39,6 +39,7 @@ _UK_EQUITY_PATTERN = re.compile(r"^[A-Z0-9&.\-]+\.L$", re.I)
# these markets' chains is Yahoo or yfinance, so the declared currency decides. # these markets' chains is Yahoo or yfinance, so the declared currency decides.
_SINGLE_CURRENCY_VENUES: tuple[tuple[re.Pattern[str], str], ...] = ( _SINGLE_CURRENCY_VENUES: tuple[tuple[re.Pattern[str], str], ...] = (
(re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "ARS"), (re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "ARS"),
(re.compile(r"^[A-Z0-9]+\.SA$", re.I), "BRL"),
(re.compile(r"^[A-Z0-9&.\-]+\.(?:TO|V)$", re.I), "CAD"), (re.compile(r"^[A-Z0-9&.\-]+\.(?:TO|V)$", re.I), "CAD"),
) )
@@ -477,7 +478,9 @@ _LOADER_CACHE_TRUE_VALUES = {"1", "true", "yes", "on"}
# forward walk hold tail-truncated multi-year series and must never be served. # forward walk hold tail-truncated multi-year series and must never be served.
# v8: FMP/Tiingo keep one adjusted/raw basis and cached frames retain their # v8: FMP/Tiingo keep one adjusted/raw basis and cached frames retain their
# actual adjustment stamp. Older entries may contain artificial return jumps. # actual adjustment stamp. Older entries may contain artificial return jumps.
_LOADER_CACHE_VERSION = 8 # v9: SEC successor/predecessor facts extend historical coverage. Old cached
# panels may omit the predecessor's filings and must be rebuilt.
_LOADER_CACHE_VERSION = 9
_LOADER_FRAME_METADATA_ATTRS = ("quote_currency", "currency_conversion", "adjustment") _LOADER_FRAME_METADATA_ATTRS = ("quote_currency", "currency_conversion", "adjustment")
agent/backtest/loaders/registry.py
+8 -1
View File
@@ -61,6 +61,7 @@ VALID_SOURCES: set[str] = {
"longbridge", "longbridge",
"mt5", "mt5",
"tickerall", "tickerall",
"b3_cotahist",
"local", "local",
"auto", "auto",
} }
@@ -123,6 +124,7 @@ def _ensure_registered() -> None:
"backtest.loaders.longbridge", "backtest.loaders.longbridge",
"backtest.loaders.mt5_loader", "backtest.loaders.mt5_loader",
"backtest.loaders.tickerall_loader", "backtest.loaders.tickerall_loader",
"backtest.loaders.b3_cotahist_loader",
"backtest.loaders.local_loader", "backtest.loaders.local_loader",
] ]
import importlib import importlib
@@ -153,7 +155,7 @@ def _ensure_registered() -> None:
# as if it were Toman — a caliber error of about six orders of magnitude, not a # as if it were Toman — a caliber error of about six orders of magnitude, not a
# missing-data error. An unreachable Iranian endpoint must be visible. # missing-data error. An unreachable Iranian endpoint must be visible.
_NO_NETWORK_FALLBACK_SOURCES: frozenset[str] = frozenset( _NO_NETWORK_FALLBACK_SOURCES: frozenset[str] = frozenset(
{"local", "qveris", "tickerall", "fmp", "nobitex", "wallex"} {"local", "qveris", "tickerall", "fmp", "nobitex", "wallex", "b3_cotahist"}
) # QVERIS-INTEGRATION ) # QVERIS-INTEGRATION
@@ -226,6 +228,10 @@ FALLBACK_CHAINS: dict[str, list[str]] = {
"ca_equity": ["yahoo", "yfinance", "local"], "ca_equity": ["yahoo", "yfinance", "local"],
# Argentina (BYMA .BA): Yahoo public chart first, yfinance fallback. # Argentina (BYMA .BA): Yahoo public chart first, yfinance fallback.
"ar_equity": ["yahoo", "yfinance", "local"], "ar_equity": ["yahoo", "yfinance", "local"],
# Brazil (B3 .SA): Yahoo public chart (adjusted) first, yfinance second,
# then the official B3 COTAHIST files (unadjusted, daily, every listed
# instrument) synced by ``vibe-trading br-data sync``.
"br_equity": ["yahoo", "yfinance", "b3_cotahist", "local"],
# UK (LSE .L): direct Yahoo first, SDK fallback second. # UK (LSE .L): direct Yahoo first, SDK fallback second.
"uk_equity": ["yahoo", "yfinance", "local"], "uk_equity": ["yahoo", "yfinance", "local"],
# Vietnam (.VN): Yahoo lists HOSE only — HNX and UPCOM are unsupported, # Vietnam (.VN): Yahoo lists HOSE only — HNX and UPCOM are unsupported,
@@ -284,6 +290,7 @@ PRICE_CALIBER_BY_SOURCE: dict[str, str] = {
"sina": "raw", "sina": "raw",
"alphavantage": "raw", # TIME_SERIES_DAILY, not the _ADJUSTED endpoint "alphavantage": "raw", # TIME_SERIES_DAILY, not the _ADJUSTED endpoint
"longbridge": "raw", # pins AdjustType.NoAdjust "longbridge": "raw", # pins AdjustType.NoAdjust
"b3_cotahist": "raw", # B3 COTAHIST files carry traded prices only
} }
#: Per-(source, market) exceptions to the per-source table. #: Per-(source, market) exceptions to the per-source table.
agent/backtest/loaders/sec_edgar_client.py
+28 -1
View File
@@ -25,6 +25,9 @@ shape. The ticker->CIK table is fetched once per process and memoized.
from __future__ import annotations from __future__ import annotations
import logging import logging
import json
from copy import deepcopy
from pathlib import Path
import threading import threading
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
@@ -224,7 +227,31 @@ def get_company_facts(cik: str | int) -> Dict[str, Any]:
ValueError: If ``cik`` contains no digits. ValueError: If ``cik`` contains no digits.
requests.RequestException: On a non-2xx status or undecodable body. requests.RequestException: On a non-2xx status or undecodable body.
""" """
return _sec_get_json(_COMPANY_FACTS_URL.format(cik=_pad_cik(cik))) padded = _pad_cik(cik)
facts = _sec_get_json(_COMPANY_FACTS_URL.format(cik=padded))
# Resolve current filings through the SEC table; only the facts history is
# extended. Never redirect submissions or discard the successor's filings.
lineage = json.loads(Path(__file__).with_name("sec_filer_lineage.json").read_text())
predecessors = lineage.get(padded, [])
if not predecessors:
return facts
merged = deepcopy(facts)
for predecessor in predecessors:
old = _sec_get_json(_COMPANY_FACTS_URL.format(cik=predecessor["cik"]))
for namespace, concepts in old.get("facts", {}).items():
target = merged.setdefault("facts", {}).setdefault(namespace, {})
for concept, node in concepts.items():
for unit, rows in node.get("units", {}).items():
# Post-reorganization subsidiary reports need not describe
# the consolidated public company. Admit pre-change filings
# only; keep their original dates for PIT extraction.
eligible = [row for row in rows if row.get("filed")
and row["filed"] < predecessor["before"]]
if not eligible:
continue
destination = target.setdefault(concept, {"units": {}})
destination.setdefault("units", {}).setdefault(unit, []).extend(eligible)
return merged
def _reset_ticker_cache_for_tests() -> None: def _reset_ticker_cache_for_tests() -> None:
agent/backtest/loaders/sec_filer_lineage.json
+7
View File
@@ -0,0 +1,7 @@
{
"0002115436": [{
"cik": "0000034088",
"before": "2026-07-01",
"source": "https://www.sec.gov/Archives/edgar/data/2115436/000003408826000093/R9.htm"
}]
}
agent/backtest/loaders/valuation_loader.py
+166
View File
@@ -0,0 +1,166 @@
"""CN valuation panel loader from Tushare ``daily_basic``.
Supplies the ``val:*`` panel namespace (proposal §4 Tier 2): daily valuation
multiples keyed by ``trade_date``. Unlike the ``fund:*`` XBRL/statement panel
these are daily market data — PIT-safe by construction, with no filing lag to
anchor. Missing dates and non-CN symbols stay NaN; nothing is fabricated.
Field whitelist: only the ``daily_basic`` columns already proven by the
``extra_fields`` merge in ``backtest/loaders/tushare.py`` are served
(``pe``, ``pe_ttm``, ``pb``, ``ps_ttm``, ``dv_ttm``). Any other ``val:`` field
is rejected with a ``ValueError`` at load time rather than silently NaN-ed, so
a factor declaring an unsupported field fails loudly.
US/HK supply is deferred (proposal §4 Tier 2 note): non-CN symbols simply get
no data here, and factors declaring ``val:`` columns SkipAlpha on universes
without CN supply.
"""
from __future__ import annotations
import logging
from collections.abc import Iterable, Sequence
from typing import Any
import pandas as pd
from backtest.loaders._symbol_utils import _is_etf_listed
from backtest.loaders.tushare import (
TUSHARE_TOKEN_PLACEHOLDERS,
_call_with_backoff,
_is_index,
)
logger = logging.getLogger(__name__)
#: ``daily_basic`` columns the ``val:`` namespace serves. ``dv_ttm`` is in
#: percent units (股息率TTM, %); consumers z-score it, which is scale-invariant.
VALUATION_FIELDS: tuple[str, ...] = ("pe", "pe_ttm", "pb", "ps_ttm", "dv_ttm")
#: A-share exchange suffixes covered by ``daily_basic``.
_CN_EQUITY_SUFFIXES = (".SH", ".SZ", ".BJ")
def validate_valuation_fields(fields: Iterable[str]) -> list[str]:
"""Validate and de-duplicate requested ``val:`` field names.
Args:
fields: Field names with the ``val:`` prefix already stripped.
Returns:
Stable, de-duplicated field list.
Raises:
ValueError: If any field is outside :data:`VALUATION_FIELDS`.
"""
unknown = [field for field in fields if field not in VALUATION_FIELDS]
if unknown:
raise ValueError(
f"unsupported val: field(s) {unknown}; the valuation namespace "
f"serves only tushare daily_basic columns {list(VALUATION_FIELDS)}"
)
return list(dict.fromkeys(fields))
def _is_cn_equity(code: str) -> bool:
"""Detect A-share equity symbols served by ``daily_basic``.
``daily_basic`` is stock-only: indices and listed funds are excluded even
though they carry a ``.SH``/``.SZ`` suffix.
"""
if not code.upper().endswith(_CN_EQUITY_SUFFIXES):
return False
return not (_is_etf_listed(code) or _is_index(code))
def _build_api() -> Any:
"""Build a Tushare pro API client from the configured token.
Raises:
RuntimeError: If ``TUSHARE_TOKEN`` is unset or a placeholder.
"""
import tushare as ts
from src.config.accessor import get_env_config
token = get_env_config().data.tushare_token.strip()
if token in TUSHARE_TOKEN_PLACEHOLDERS:
raise RuntimeError(
"TUSHARE_TOKEN not configured; required for val:* valuation columns"
)
return ts.pro_api(token)
def load_valuation_panel(
*,
symbols: Sequence[str],
fields: Iterable[str],
start: str,
end: str,
index: pd.DatetimeIndex,
api: Any | None = None,
) -> dict[str, pd.DataFrame]:
"""Load whitelisted ``daily_basic`` valuation fields as wide panels.
Args:
symbols: Panel symbols; only CN A-share equities are fetched, the rest
stay NaN.
fields: Requested field names (``val:`` prefix stripped).
start: Start date (YYYY-MM-DD).
end: End date (YYYY-MM-DD).
index: Price-panel index to align to.
api: Optional pre-built Tushare pro API (tests inject fakes here).
Returns:
``field -> dates x symbols`` frames aligned to ``index``/``symbols``.
Every validated field is present; symbols or dates ``daily_basic`` did
not serve are NaN.
Raises:
ValueError: If a field is outside :data:`VALUATION_FIELDS`.
RuntimeError: If no API is injected and ``TUSHARE_TOKEN`` is missing.
"""
checked = validate_valuation_fields(fields)
if not checked:
return {}
def _nan_frame(per_code: dict[str, pd.Series] | None = None) -> pd.DataFrame:
if not per_code:
return pd.DataFrame(float("nan"), index=index, columns=list(symbols))
return pd.DataFrame(per_code).reindex(index=index, columns=list(symbols))
cn_symbols = [code for code in symbols if _is_cn_equity(code)]
if not cn_symbols:
return {field: _nan_frame() for field in checked}
if api is None:
api = _build_api()
sd = start.replace("-", "")
ed = end.replace("-", "")
series_by_field: dict[str, dict[str, pd.Series]] = {field: {} for field in checked}
for code in cn_symbols:
try:
basic = _call_with_backoff(
api.daily_basic,
ts_code=code,
start_date=sd,
end_date=ed,
fields="ts_code,trade_date," + ",".join(checked),
)
except Exception as exc: # noqa: BLE001 - one bad fetch must not raise
logger.warning("daily_basic valuation fetch failed for %s: %s", code, exc)
continue
if basic is None or basic.empty:
logger.warning("daily_basic returned empty valuation data for %s", code)
continue
basic = basic.copy()
basic["trade_date"] = pd.to_datetime(basic["trade_date"], errors="coerce")
basic = basic.dropna(subset=["trade_date"]).set_index("trade_date").sort_index()
for field in checked:
if field in basic.columns:
series_by_field[field][code] = pd.to_numeric(
basic[field], errors="coerce"
)
return {field: _nan_frame(per_code) for field, per_code in series_by_field.items()}
agent/backtest/loaders/yahoo_loader.py
+2 -2
View File
@@ -64,7 +64,7 @@ def _is_supported(code: str) -> bool:
return upper.endswith( return upper.endswith(
( (
".US", ".HK", ".NS", ".BO", ".KS", ".KQ", ".TO", ".V", ".VN", ".BA", ".US", ".HK", ".NS", ".BO", ".KS", ".KQ", ".TO", ".V", ".VN", ".BA",
".L", "=F", "=X", ".SA", ".L", "=F", "=X",
) )
) )
@@ -191,7 +191,7 @@ class DataLoader:
name = "yahoo" name = "yahoo"
markets = { markets = {
"us_equity", "hk_equity", "india_equity", "kr_equity", "ca_equity", "us_equity", "hk_equity", "india_equity", "kr_equity", "ca_equity",
"vietnam_equity", "uk_equity", "ar_equity", "vietnam_equity", "uk_equity", "ar_equity", "br_equity",
} }
# Yahoo chart volume is single shares for US/HK equities # Yahoo chart volume is single shares for US/HK equities
# (HKUDS/Vibe-Trading#1062; HK verified 2026-08-11, 00700.HK ratio 1.00 # (HKUDS/Vibe-Trading#1062; HK verified 2026-08-11, 00700.HK ratio 1.00
agent/backtest/loaders/yfinance_loader.py
+1 -1
View File
@@ -255,7 +255,7 @@ class DataLoader:
name = "yfinance" name = "yfinance"
markets = { markets = {
"us_equity", "hk_equity", "india_equity", "kr_equity", "ca_equity", "us_equity", "hk_equity", "india_equity", "kr_equity", "ca_equity",
"vietnam_equity", "uk_equity", "ar_equity", "crypto", "vietnam_equity", "uk_equity", "ar_equity", "br_equity", "crypto",
} }
# yfinance volume is single shares for US/HK equities # yfinance volume is single shares for US/HK equities
# (HKUDS/Vibe-Trading#1062; HK verified 2026-08-11, 00700.HK ratio 1.00 # (HKUDS/Vibe-Trading#1062; HK verified 2026-08-11, 00700.HK ratio 1.00
agent/backtest/metrics.py
+16
View File
@@ -44,6 +44,8 @@ _TRADING_DAYS = {
"india_broker": 252, "india_broker": 252,
# Korean equity (KRX) # Korean equity (KRX)
"pykrx": 252, "pykrx": 252,
# Brazilian equity (B3)
"b3_cotahist": 252,
} }
# mt5 is a forex/CFD feed: 24x5 sessions → 260 trading days, 24h intraday bars. # mt5 is a forex/CFD feed: 24x5 sessions → 260 trading days, 24h intraday bars.
# US equity (yfinance-style): 6.5h sessions → 390 1m bars/day. # US equity (yfinance-style): 6.5h sessions → 390 1m bars/day.
@@ -71,6 +73,8 @@ _BARS_PER_DAY = {
"india_broker": 375, "india_broker": 375,
# Korean equity (6.5h session, 09:00-15:30 KST) # Korean equity (6.5h session, 09:00-15:30 KST)
"pykrx": 390, "pykrx": 390,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 420,
}, },
"5m": {"yfinance": 78, "yahoo": 78, "finnhub": 78, "alphavantage": 78, "5m": {"yfinance": 78, "yahoo": 78, "finnhub": 78, "alphavantage": 78,
"tiingo": 78, "fmp": 78, "stooq": 78, "longbridge": 78, "tiingo": 78, "fmp": 78, "stooq": 78, "longbridge": 78,
@@ -82,6 +86,8 @@ _BARS_PER_DAY = {
"mt5": 288, "tickerall": 288, "mt5": 288, "tickerall": 288,
"india_broker": 75, "india_broker": 75,
"pykrx": 78, "pykrx": 78,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 84,
}, },
"15m": {"yfinance": 26, "yahoo": 26, "finnhub": 26, "alphavantage": 26, "15m": {"yfinance": 26, "yahoo": 26, "finnhub": 26, "alphavantage": 26,
"tiingo": 26, "fmp": 26, "stooq": 26, "longbridge": 26, "tiingo": 26, "fmp": 26, "stooq": 26, "longbridge": 26,
@@ -93,6 +99,8 @@ _BARS_PER_DAY = {
"mt5": 96, "tickerall": 96, "mt5": 96, "tickerall": 96,
"india_broker": 25, "india_broker": 25,
"pykrx": 26, "pykrx": 26,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 28,
}, },
"30m": {"yfinance": 13, "yahoo": 13, "finnhub": 13, "alphavantage": 13, "30m": {"yfinance": 13, "yahoo": 13, "finnhub": 13, "alphavantage": 13,
"tiingo": 13, "fmp": 13, "stooq": 13, "longbridge": 13, "tiingo": 13, "fmp": 13, "stooq": 13, "longbridge": 13,
@@ -104,6 +112,8 @@ _BARS_PER_DAY = {
"mt5": 48, "tickerall": 48, "mt5": 48, "tickerall": 48,
"india_broker": 13, "india_broker": 13,
"pykrx": 13, "pykrx": 13,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 14,
}, },
"1H": {"yfinance": 7, "yahoo": 7, "finnhub": 7, "alphavantage": 7, "1H": {"yfinance": 7, "yahoo": 7, "finnhub": 7, "alphavantage": 7,
"tiingo": 7, "fmp": 7, "stooq": 7, "longbridge": 7, "tiingo": 7, "fmp": 7, "stooq": 7, "longbridge": 7,
@@ -115,6 +125,8 @@ _BARS_PER_DAY = {
"mt5": 24, "tickerall": 24, "mt5": 24, "tickerall": 24,
"india_broker": 7, "india_broker": 7,
"pykrx": 7, "pykrx": 7,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 7,
}, },
"4H": {"yfinance": 2, "yahoo": 2, "finnhub": 2, "alphavantage": 2, "4H": {"yfinance": 2, "yahoo": 2, "finnhub": 2, "alphavantage": 2,
"tiingo": 2, "fmp": 2, "stooq": 2, "longbridge": 2, "tiingo": 2, "fmp": 2, "stooq": 2, "longbridge": 2,
@@ -126,6 +138,8 @@ _BARS_PER_DAY = {
"mt5": 6, "tickerall": 6, "mt5": 6, "tickerall": 6,
"india_broker": 2, "india_broker": 2,
"pykrx": 2, "pykrx": 2,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 2,
}, },
"1D": {"yfinance": 1, "yahoo": 1, "finnhub": 1, "alphavantage": 1, "1D": {"yfinance": 1, "yahoo": 1, "finnhub": 1, "alphavantage": 1,
"tiingo": 1, "fmp": 1, "stooq": 1, "longbridge": 1, "tiingo": 1, "fmp": 1, "stooq": 1, "longbridge": 1,
@@ -137,6 +151,8 @@ _BARS_PER_DAY = {
"mt5": 1, "tickerall": 1, "mt5": 1, "tickerall": 1,
"india_broker": 1, "india_broker": 1,
"pykrx": 1, "pykrx": 1,
# Brazilian equity (B3, 7h session 10:00-17:00 BRT; daily-only loader)
"b3_cotahist": 1,
}, },
} }
agent/backtest/models.py
+11
View File
@@ -23,6 +23,10 @@ class Position:
leverage: Effective leverage (1 for spot/stocks). leverage: Effective leverage (1 for spot/stocks).
entry_bar_idx: Index in the dates array at entry (for holding_bars). entry_bar_idx: Index in the dates array at entry (for holding_bars).
entry_commission: Commission paid at entry. entry_commission: Commission paid at entry.
open_price: Execution price of the fill that opened the position.
``entry_price`` becomes the weighted cost once the position is
scaled into; this keeps what was actually paid at the open.
open_size: Quantity of the fill that opened the position.
""" """
symbol: str symbol: str
@@ -33,6 +37,8 @@ class Position:
leverage: float = 1.0 leverage: float = 1.0
entry_bar_idx: int = 0 entry_bar_idx: int = 0
entry_commission: float = 0.0 entry_commission: float = 0.0
open_price: float | None = None
open_size: float | None = None
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -79,6 +85,9 @@ class TradeRecord:
commission: Total commission (entry + exit). commission: Total commission (entry + exit).
entry_margin: Actual margin allocated at entry, after size rounding. entry_margin: Actual margin allocated at entry, after size rounding.
exit_margin: Margin-equivalent value traded at the exit price. exit_margin: Margin-equivalent value traded at the exit price.
open_price: Execution price of the fill that opened the position,
or ``None`` when the position did not record it.
open_size: Quantity of that opening fill, or ``None``.
""" """
symbol: str symbol: str
@@ -96,6 +105,8 @@ class TradeRecord:
commission: float commission: float
entry_margin: float = 0.0 entry_margin: float = 0.0
exit_margin: float = 0.0 exit_margin: float = 0.0
open_price: float | None = None
open_size: float | None = None
@dataclass(frozen=True) @dataclass(frozen=True)
agent/backtest/runner.py
+202 -20
View File
@@ -59,6 +59,7 @@ _VALID_INTERVALS = {"1m", "5m", "15m", "30m", "1H", "4H", "1D", "1W", "1M"}
_VALID_ENGINES = {"daily", "options"} _VALID_ENGINES = {"daily", "options"}
_PRICE_PANEL_COLUMNS = ("open", "high", "low", "close", "volume", "vwap", "amount") _PRICE_PANEL_COLUMNS = ("open", "high", "low", "close", "volume", "vwap", "amount")
_FUND_PREFIX = "fund:" _FUND_PREFIX = "fund:"
_VAL_PREFIX = "val:"
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -851,6 +852,7 @@ _MARKET_TO_SOURCE = {
"kr_equity": "pykrx", "kr_equity": "pykrx",
"ca_equity": "yahoo", "ca_equity": "yahoo",
"ar_equity": "yahoo", "ar_equity": "yahoo",
"br_equity": "yahoo",
"uk_equity": "yahoo", "uk_equity": "yahoo",
"vietnam_equity": "yahoo", "vietnam_equity": "yahoo",
"crypto": "okx", "crypto": "okx",
@@ -1038,6 +1040,26 @@ def _fund_columns_required(selected_factors: Iterable[Any]) -> list[str]:
return out return out
def _val_columns_required(selected_factors: Iterable[Any]) -> list[str]:
"""Collect requested ``val:*`` panel columns from selected factors.
Args:
selected_factors: Factor metadata/spec objects.
Returns:
Stable, de-duplicated ``val:*`` column names.
"""
seen: set[str] = set()
out: list[str] = []
for spec in selected_factors:
for column in _columns_required_from_factor_spec(spec):
if not column.startswith(_VAL_PREFIX) or column in seen:
continue
seen.add(column)
out.append(column)
return out
def _build_price_panel(data_map: dict[str, pd.DataFrame]) -> dict[str, pd.DataFrame]: def _build_price_panel(data_map: dict[str, pd.DataFrame]) -> dict[str, pd.DataFrame]:
"""Convert ``code -> OHLCV frame`` rows into the factor panel shape. """Convert ``code -> OHLCV frame`` rows into the factor panel shape.
@@ -1139,6 +1161,85 @@ def _inject_fundamental_panel(
return panel return panel
def _inject_valuation_panel(
panel: dict[str, pd.DataFrame],
*,
symbols: list[str],
val_columns: Iterable[str],
start: str,
end: str,
index: pd.DatetimeIndex | None = None,
) -> dict[str, pd.DataFrame]:
"""Inject required ``val:*`` daily valuation fields into a factor panel.
Source is tushare ``daily_basic`` (CN A-share only): daily market data
keyed by ``trade_date``, so PIT-safe by construction — no filing lag.
Non-CN symbols and missing dates stay NaN; nothing is fabricated, and
factors declaring ``val:`` columns SkipAlpha or NaN-out naturally where
there is no CN supply.
Args:
panel: Existing factor panel keyed by column name.
symbols: Symbols to load.
val_columns: Requested ``val:*`` columns.
start: Start date string.
end: End date string.
index: Optional price index to align to. Defaults to ``panel["close"]``.
Returns:
The same panel dictionary with ``val:<field>`` frames added.
Raises:
ValueError: If a declared ``val:`` field is outside the daily_basic
whitelist (pe, pe_ttm, pb, ps_ttm, dv_ttm).
"""
fields = [column[len(_VAL_PREFIX):] for column in val_columns if column.startswith(_VAL_PREFIX)]
fields = list(dict.fromkeys(fields))
if not fields:
return panel
from backtest.loaders.valuation_loader import (
load_valuation_panel,
validate_valuation_fields,
)
# Whitelist rejection is a caller-fixable contract error, so it must
# propagate rather than degrade into NaN frames like a provider outage.
fields = validate_valuation_fields(fields)
price_index = index
if price_index is None:
close = panel.get("close")
price_index = close.index if close is not None else pd.DatetimeIndex([])
try:
loaded = load_valuation_panel(
symbols=symbols,
fields=fields,
start=start,
end=end,
index=price_index,
)
except Exception as exc: # noqa: BLE001 - data-source failure must not kill backtest
logger.warning(
"valuation panel load failed for fields=%s symbols=%s: %s; injecting NaN frames",
fields,
symbols,
exc,
exc_info=True,
)
loaded = {}
for field in fields:
frame = loaded.get(field)
if not isinstance(frame, pd.DataFrame):
frame = _nan_fundamental_frame(price_index, symbols)
else:
frame = frame.reindex(index=price_index, columns=symbols)
panel[f"{_VAL_PREFIX}{field}"] = frame
return panel
def _project_panel_fields_to_data_map( def _project_panel_fields_to_data_map(
data_map: dict[str, pd.DataFrame], data_map: dict[str, pd.DataFrame],
panel: dict[str, pd.DataFrame], panel: dict[str, pd.DataFrame],
@@ -1162,28 +1263,41 @@ def _maybe_inject_fundamentals_for_factor_panel(
data_map: dict[str, pd.DataFrame], data_map: dict[str, pd.DataFrame],
config: dict, config: dict,
) -> dict[str, pd.DataFrame]: ) -> dict[str, pd.DataFrame]:
"""Inject ``fund:*`` factor dependencies when selected factors request them.""" """Inject ``fund:*``/``val:*`` factor dependencies when selected factors request them."""
selected_factors = _selected_factor_specs(config) selected_factors = _selected_factor_specs(config)
fund_columns = _fund_columns_required(selected_factors) fund_columns = _fund_columns_required(selected_factors)
if not fund_columns: val_columns = _val_columns_required(selected_factors)
if not fund_columns and not val_columns:
return data_map return data_map
panel = _build_price_panel(data_map) panel = _build_price_panel(data_map)
close = panel.get("close") close = panel.get("close")
price_index = close.index if close is not None else pd.DatetimeIndex([]) price_index = close.index if close is not None else pd.DatetimeIndex([])
symbols = list(data_map) symbols = list(data_map)
_inject_fundamental_panel( if fund_columns:
panel, _inject_fundamental_panel(
symbols=symbols, panel,
fund_columns=fund_columns, symbols=symbols,
start=config.get("start_date", ""), fund_columns=fund_columns,
end=config.get("end_date", ""), start=config.get("start_date", ""),
freq="ttm", end=config.get("end_date", ""),
pit=True, freq="ttm",
source="auto", pit=True,
index=price_index, source="auto",
index=price_index,
)
if val_columns:
_inject_valuation_panel(
panel,
symbols=symbols,
val_columns=val_columns,
start=config.get("start_date", ""),
end=config.get("end_date", ""),
index=price_index,
)
return _project_panel_fields_to_data_map(
data_map, panel, [*fund_columns, *val_columns]
) )
return _project_panel_fields_to_data_map(data_map, panel, fund_columns)
# --- Main entry --- # --- Main entry ---
@@ -1306,8 +1420,17 @@ def main(run_dir: Path) -> None:
# Every source has already been fetched, sanitized, and enriched above. # Every source has already been fetched, sanitized, and enriched above.
# Reuse that exact snapshot so provider costs and run-card provenance stay # Reuse that exact snapshot so provider costs and run-card provenance stay
# aligned with the data consumed by the engine. # aligned with the data consumed by the engine. A source that prohibits
loader = _AutoLoader(data_map) # fallback keeps its own loader behind the snapshot: a declared benchmark
# is usually outside ``codes``, but must still come from that same source.
if is_no_network_fallback_source(source):
loader = _AutoLoader(
data_map,
inner=loader,
name=str(getattr(loader, "name", None) or source),
)
else:
loader = _AutoLoader(data_map)
if engine_type == "options": if engine_type == "options":
from backtest.engines.options_portfolio import run_options_backtest from backtest.engines.options_portfolio import run_options_backtest
@@ -1548,6 +1671,11 @@ def _create_market_engine(source: str, config: dict, codes: List[str]):
if "vietnam_equity" in markets: if "vietnam_equity" in markets:
from backtest.engines.vietnam_equity import VietnamEquityEngine from backtest.engines.vietnam_equity import VietnamEquityEngine
return VietnamEquityEngine(config) return VietnamEquityEngine(config)
# Brazil (B3) routing — same reason as India: ``yahoo`` has no Wave-1
# branch and would otherwise fall through to the crypto default.
if "br_equity" in markets:
from backtest.engines.brazil_equity import BrazilEquityEngine
return BrazilEquityEngine(config)
# Argentina market-data routing is supported, but BYMA execution rules # Argentina market-data routing is supported, but BYMA execution rules
# are not modeled yet. Fail closed instead of silently applying US/crypto # are not modeled yet. Fail closed instead of silently applying US/crypto
# commissions, lot sizes, settlement, or short-selling assumptions. # commissions, lot sizes, settlement, or short-selling assumptions.
@@ -1959,14 +2087,68 @@ def _sanitize_data_map(data_map: dict) -> dict:
class _AutoLoader: class _AutoLoader:
"""Loader adapter that returns a pre-fetched data map.""" """Loader adapter that returns a pre-fetched data map.
def __init__(self, data_map: dict): With an ``inner`` loader, symbols outside the snapshot are fetched through
it, and ``name`` reports that loader's source so benchmark resolution can
tell the adapter apart from a network loader swapped in by a fallback.
"""
def __init__(
self,
data_map: dict,
inner: Any = None,
name: str | None = None,
) -> None:
"""Wrap a fetched snapshot.
Args:
data_map: ``code -> DataFrame`` snapshot already consumed by the run.
inner: Loader that served the snapshot; consulted only for symbols
the snapshot does not hold. ``None`` serves the snapshot alone.
name: Source name of ``inner``. When ``None`` the adapter exposes no
``name``: a bare snapshot does not stand for any one source.
"""
self._data = data_map self._data = data_map
self._inner = inner
if name is not None:
self.name = name
def fetch(self, codes, start_date, end_date, fields=None, interval="1D"): def fetch(
"""Return preloaded rows for requested codes.""" self,
return {c: df for c, df in self._data.items() if c in codes} codes: List[str],
start_date: str,
end_date: str,
fields: Optional[List[str]] = None,
interval: str = "1D",
) -> dict:
"""Return snapshot rows, fetching symbols it lacks through ``inner``.
Frames from ``inner`` go through the same OHLC sanitisation as the
snapshot, so a benchmark is held to the bar quality the strategy was.
Args:
codes: Requested symbols.
start_date: Window start, passed through to ``inner``.
end_date: Window end, passed through to ``inner``.
fields: Extra fields, passed through to ``inner``.
interval: Bar interval, passed through to ``inner``.
Returns:
``code -> DataFrame`` for every requested symbol that could be served.
"""
served = {c: df for c, df in self._data.items() if c in codes}
missing = [c for c in codes if c not in served]
if missing and self._inner is not None:
source = str(getattr(self, "name", "") or "")
requested = _normalize_codes(missing, source)
fetched = self._inner.fetch(
requested, start_date, end_date, fields=fields, interval=interval
)
if isinstance(fetched, dict):
restored = _restore_original_codes(fetched, missing, requested)
served.update(_sanitize_data_map(restored))
return served
if __name__ == "__main__": if __name__ == "__main__":
agent/backtest/validation.py
+4 -2
View File
@@ -400,9 +400,11 @@ def _load_trades(run_dir: Path) -> List[TradeRecord]:
if df.empty: if df.empty:
return [] return []
# trades.csv has entry+exit row pairs; extract exit rows (they have pnl != 0) # New artifacts identify exits explicitly, including break-even trades.
# Unmarked historical/other-engine files retain their legacy convention.
trades = [] trades = []
exit_rows = df[df["pnl"] != 0].reset_index(drop=True) exit_mask = df["event"].eq("exit") if "event" in df.columns else df["pnl"].ne(0)
exit_rows = df[exit_mask].reset_index(drop=True)
for _, row in exit_rows.iterrows(): for _, row in exit_rows.iterrows():
hold = pd.to_numeric(row.get("holding_bars"), errors="coerce") hold = pd.to_numeric(row.get("holding_bars"), errors="coerce")
if pd.isna(hold): if pd.isna(hold):
agent/src/agent/context.py
+13 -5
View File
@@ -98,7 +98,7 @@ Decide which workflow to use based on the request:
Override: if the user explicitly requests full analysis regardless of routing, run all 4 layers. Override: if the user explicitly requests full analysis regardless of routing, run all 4 layers.
**Layer 1 — Trade Attribution** (always, if `artifacts/trades.csv` exists): **Layer 1 — Trade Attribution** (always, if `artifacts/trades.csv` exists):
- Read trades.csv. Exit rows have `pnl != 0` (entry rows have pnl = 0). Exit rows contain pnl, holding_days, return_pct — use exit rows directly, no pairing needed - Read trades.csv. With an `event` column, use `event == "exit"` for completed trades, including zero-PnL exits; `entry` marks openings. Only files without `event` use the legacy `pnl != 0` convention, whose zero-PnL exits are ambiguous. Exit rows contain pnl, holding_days, return_pct — use exit rows directly, no pairing needed
- Top-5 winners and losers: rank exit rows by pnl, show code, side, timestamp, pnl, return_pct, holding_days, reason - Top-5 winners and losers: rank exit rows by pnl, show code, side, timestamp, pnl, return_pct, holding_days, reason
- Robustness check: is the strategy still profitable after removing the top-5 winning trades? - Robustness check: is the strategy still profitable after removing the top-5 winning trades?
- Exit-reason breakdown: group by `reason`, show count, total_pnl, avg_pnl, win_rate per group - Exit-reason breakdown: group by `reason`, show count, total_pnl, avg_pnl, win_rate per group
@@ -218,9 +218,15 @@ Decide which workflow to use based on the request:
`observed` with the backtest's run directory as `ref`, e.g. `rp`, or the file `observed` with the backtest's run directory as `ref`, e.g. `rp`, or the file
you read, e.g. `rp/artifacts/target_positions.csv`; two backtests are two you read, e.g. `rp/artifacts/target_positions.csv`; two backtests are two
directories, so a comparison names each one (`rp::sharpe`, `ew::sharpe`); directories, so a comparison names each one (`rp::sharpe`, `ew::sharpe`);
`derived` — arithmetic on observed values (`note`: the formula; every number `derived` — arithmetic on observed values (`note`: the formula; ALL data
added or subtracted must itself be an observed value; `ref`: where the operands, including numerators and denominators, must be observed; `ref`:
operands came from, e.g. `rp, ew` for a difference between two backtests); where they came from, e.g. `rp, ew` for a difference between two backtests).
A chosen multiplier/divisor needs a `count` declaration AND its value and
note label on the same visible line of the answer, even for an integer
(e.g. "Discount factor: 97%.", declared `97% | count | discount factor`).
Literal 1 and square/cube exponents are structural constants. Every added
or subtracted term still needs observed data. Never relabel missing data
as a chosen parameter: fetch it or remove the calculation;
`proposed` — a price level you suggest, such as an entry, stop or target: inside `proposed` — a price level you suggest, such as an entry, stop or target: inside
the observed price range, or with a formula over observed values in `note`; a the observed price range, or with a formula over observed values in `note`; a
percentage is not a level, so state the price it implies; percentage is not a level, so state the price it implies;
@@ -228,10 +234,12 @@ Decide which workflow to use based on the request:
same sentence as the figure, and in `note`; same sentence as the figure, and in `note`;
`count` — a count, weight, threshold, window, probability or other parameter `count` — a count, weight, threshold, window, probability or other parameter
you chose, never a price or an amount. Plain integers, dates and security you chose, never a price or an amount. Plain integers, dates and security
codes need no line. Example (zh): codes need no line unless used as a chosen coefficient above. Example (zh,
with "折扣系数：97%。" visible in the answer):
```figures ```figures
0.666 | observed | 159516.SZ 收盘 2026-09-09 | 159516.SZ 0.666 | observed | 159516.SZ 收盘 2026-09-09 | 159516.SZ
0.646 | derived | 0.666 × 0.97 | 159516.SZ 0.646 | derived | 0.666 × 0.97 | 159516.SZ
97% | count | 折扣系数
37% | derived | (1.053 − 0.666) / 1.053 | 159516.SZ 37% | derived | (1.053 − 0.666) / 1.053 | 159516.SZ
0.62 | proposed | 买入参考，位于观测区间 0.567–1.053 内 0.62 | proposed | 买入参考，位于观测区间 0.567–1.053 内
``` ```
agent/src/agent/grounding/evidence.py
+5 -3
View File
@@ -1031,10 +1031,12 @@ class _EvidenceMixin:
arguments: Mapping[str, Any], arguments: Mapping[str, Any],
payload: dict[str, Any] | None, payload: dict[str, Any] | None,
call_id: str, call_id: str,
*,
tool_name: str = "get_market_data",
) -> None: ) -> None:
"""Convert full OHLCV payloads into source-linked evidence rows.""" """Convert full OHLCV payloads into source-linked evidence rows."""
if payload is None: if payload is None:
self._record_tool_failure("get_market_data", call_id, "malformed JSON result") self._record_tool_failure(tool_name, call_id, "malformed JSON result")
return return
requested_source = str(arguments.get("source") or "auto") requested_source = str(arguments.get("source") or "auto")
provenance = payload.get("_provenance") provenance = payload.get("_provenance")
@@ -1081,7 +1083,7 @@ class _EvidenceMixin:
self._evidence.append( self._evidence.append(
EvidenceRecord( EvidenceRecord(
call_id=call_id, call_id=call_id,
tool="get_market_data", tool=tool_name,
symbol=symbol, symbol=symbol,
source=actual_source, source=actual_source,
timestamp=timestamp, timestamp=timestamp,
@@ -1100,7 +1102,7 @@ class _EvidenceMixin:
self._evidence.append( self._evidence.append(
EvidenceRecord( EvidenceRecord(
call_id=call_id, call_id=call_id,
tool="get_market_data", tool=tool_name,
symbol=symbol, symbol=symbol,
source=requested_source, source=requested_source,
timestamp=None, timestamp=None,
agent/src/agent/grounding/identity.py
+3 -1
View File
@@ -125,7 +125,7 @@ _CANONICAL_SYMBOL_RE = re.compile(
# Every equity suffix the market-data layer routes (backtest.engines. # Every equity suffix the market-data layer routes (backtest.engines.
# _market_hooks._MARKET_PATTERNS); test_market_identity_parity keeps the # _market_hooks._MARKET_PATTERNS); test_market_identity_parity keeps the
# two in step, because .L / .VN / .BA each landed there without landing here. # two in step, because .L / .VN / .BA each landed there without landing here.
r"[A-Z][A-Z0-9&.-]{0,19}\.(?:US|NS|BO|FX|TO|V|BA|L|VN)|" r"[A-Z][A-Z0-9&.-]{0,19}\.(?:US|NS|BO|FX|TO|V|BA|SA|L|VN)|"
r"[A-Z0-9]{2,15}(?:-|/)(?:USDT|USDC|USD|BTC|ETH)|" r"[A-Z0-9]{2,15}(?:-|/)(?:USDT|USDC|USD|BTC|ETH)|"
r"[A-Z]{2,15}(?:" + "|".join(_JOINED_CRYPTO_QUOTE_SUFFIXES) + r")|" r"[A-Z]{2,15}(?:" + "|".join(_JOINED_CRYPTO_QUOTE_SUFFIXES) + r")|"
r"\^[A-Z0-9&.\-]{1,20}|" r"\^[A-Z0-9&.\-]{1,20}|"
@@ -255,6 +255,7 @@ def _infer_venue(symbol: str) -> str | None:
".TO": "toronto", ".TO": "toronto",
".V": "tsx_venture", ".V": "tsx_venture",
".BA": "buenos_aires", ".BA": "buenos_aires",
".SA": "b3",
".L": "lse", ".L": "lse",
".VN": "hose", ".VN": "hose",
} }
@@ -322,6 +323,7 @@ def _infer_currency(symbol: str) -> str | None:
".TO": "CAD", ".TO": "CAD",
".V": "CAD", ".V": "CAD",
".BA": "ARS", ".BA": "ARS",
".SA": "BRL",
# The UK loaders admit only a declared GBP / GBp quote and hand back # The UK loaders admit only a declared GBP / GBp quote and hand back
# GBP, the same contract as the backtest's _MARKET_CURRENCY. # GBP, the same contract as the backtest's _MARKET_CURRENCY.
".L": "GBP", ".L": "GBP",
agent/src/agent/grounding/ledger.py
+27
View File
@@ -308,6 +308,16 @@ class GroundingLedger(
self.persist() self.persist()
return return
if tool_name == "get_br_market_data":
# This tool serves B3 identifiers; bare native tickers must not be
# mistaken for US symbols by generic evidence ingestion.
tickers = arguments.get("tickers")
if isinstance(tickers, list):
arguments = {**arguments, "tickers": [
str(ticker).strip().upper() if str(ticker).strip().upper().endswith(".SA")
else str(ticker).strip().upper() + ".SA"
for ticker in tickers
]}
self._track_session_symbols(arguments, result) self._track_session_symbols(arguments, result)
self._note_model_write(tool_name, arguments, payload) self._note_model_write(tool_name, arguments, payload)
if tool_name in _ANALYSIS_TOOLS: if tool_name in _ANALYSIS_TOOLS:
@@ -316,6 +326,23 @@ class GroundingLedger(
self._ingest_resolution(arguments, payload, call_id) self._ingest_resolution(arguments, payload, call_id)
elif tool_name == "get_market_data": elif tool_name == "get_market_data":
self._ingest_market_data(arguments, payload, call_id) self._ingest_market_data(arguments, payload, call_id)
elif (tool_name == "get_br_market_data" and payload is not None
and isinstance(payload.get("data"), dict)
and payload["data"].get("action") == "quotes"):
data = payload["data"]
if data.get("currency") == "BRL":
quotes: dict[str, Any] = {"_provenance": {}}
for row in data.get("rows", []):
if not isinstance(row, dict) or not row.get("ticker"):
continue
ticker = str(row["ticker"]).strip().upper()
symbol = ticker if ticker.endswith(".SA") else ticker + ".SA"
quotes.setdefault(symbol, []).append({
key: row[key] for key in ("trade_date", "open", "high", "low", "close", "volume")
if key in row
})
quotes["_provenance"][symbol] = {"source": "b3_cotahist", "quote_currency": "BRL"}
self._ingest_market_data(arguments, quotes, call_id, tool_name=tool_name)
elif tool_name == "read_file" and payload is not None: elif tool_name == "read_file" and payload is not None:
self._ingest_engine_table(payload, call_id) self._ingest_engine_table(payload, call_id)
elif tool_name == "read_run_artifact" and payload is not None: elif tool_name == "read_run_artifact" and payload is not None:
agent/src/agent/grounding/policies.py
+65 -8
View File
@@ -90,6 +90,7 @@ _CURRENCY_ALIASES = {
"CAD": ("cad", "c$", "加元", "加拿大元"), "CAD": ("cad", "c$", "加元", "加拿大元"),
"GBP": ("gbp", "£", "英镑"), "GBP": ("gbp", "£", "英镑"),
"VND": ("vnd", "₫", "越南盾"), "VND": ("vnd", "₫", "越南盾"),
"BRL": ("brl", "r$", "real brasileiro", "reais", "巴西雷亚尔"),
"ARS": ("ars", "ar$", "阿根廷比索"), "ARS": ("ars", "ar$", "阿根廷比索"),
} }
@@ -493,6 +494,36 @@ class _PolicyMixin:
if evaluated is not None if evaluated is not None
for operand in evaluated[1] for operand in evaluated[1]
] ]
# The hidden declaration alone cannot turn invented data into a free
# coefficient. A chosen parameter needs a count declaration AND its
# value and label in the visible answer. No prose vocabulary decides
# its role; the same label-matching contract already protects citations.
scalar_parameters: list[float] = []
for parameter in figures:
declared = block.match(parameter.value, parameter.percent, parameter.digits)
if (
declared is None
or declared.role != "count"
or parameter.shape not in ("measured", "bare")
or parameter.currency
or parameter.column
):
continue
line = positions[parameter.line][0] if 0 <= parameter.line < len(positions) else ""
# Citation tokens cover labels embedded in ordinary prose. Also
# accept a complete label in any script, including a short symbol
# such as "w"; it must not match a letter inside an unrelated word.
label = declared.note.strip()
label_visible = bool(label) and (
bool(_note_tokens(label) & _note_tokens(line))
or re.search(
r"(?<![^\W\d_])" + re.escape(label) + r"(?![^\W\d_])",
line,
re.IGNORECASE,
) is not None
)
if label_visible:
scalar_parameters.append(parameter.value / 100 if parameter.percent else parameter.value)
checked_price = False checked_price = False
for figure in figures: for figure in figures:
if figure.shape not in ("measured", "bare"): if figure.shape not in ("measured", "bare"):
@@ -610,12 +641,12 @@ class _PolicyMixin:
checked_price = True checked_price = True
if role == "derived": if role == "derived":
issues.extend( issues.extend(
self._check_derived(figure, declaration, symbol, records) self._check_derived(figure, declaration, symbol, records, scalar_parameters)
) )
continue continue
if role == "proposed": if role == "proposed":
issues.extend( issues.extend(
self._check_proposed(figure, declaration, symbol, records) self._check_proposed(figure, declaration, symbol, records, scalar_parameters)
) )
continue continue
issues.extend(self._check_observed(figure, declaration, symbol, records)) issues.extend(self._check_observed(figure, declaration, symbol, records))
@@ -1743,13 +1774,16 @@ class _PolicyMixin:
records: Sequence[EvidenceRecord], records: Sequence[EvidenceRecord],
*, *,
money: bool = False, money: bool = False,
scalar_parameters: Sequence[float] = (),
) -> tuple[float, list[float]] | str | None: ) -> tuple[float, list[float]] | str | None:
"""Evaluate a declaration's note as an observation-anchored formula. """Evaluate a declaration's note as an observation-anchored formula.
Returns the ``(result, operands)`` pair when the note is arithmetic Returns the ``(result, operands)`` pair when the note is arithmetic
over at least two operands, at least one of which the run observed and over at least two operands. Data operands must be observed; chosen
every added or subtracted term of which holds an observed operand; scalar parameters must be declared and visible. Every added or
otherwise the reason it is not. subtracted term still needs an observed anchor. Literal 1 is the
multiplicative identity (also used in ratio-minus-one expressions),
and square/cube exponents belong to the operator, not the operands.
""" """
if declaration is None or not declaration.note.strip(): if declaration is None or not declaration.note.strip():
return "no_formula" return "no_formula"
@@ -1796,6 +1830,16 @@ class _PolicyMixin:
return "no_symbol" if unattributed else "formula_not_anchored" return "no_symbol" if unattributed else "formula_not_anchored"
if _unanchored_term(tree, observed): if _unanchored_term(tree, observed):
return "additive_operand_not_observed" return "additive_operand_not_observed"
if any(
not observed(operand)
and operand != 1
and not any(
math.isclose(abs(operand), abs(parameter), rel_tol=1e-12, abs_tol=0.0)
for parameter in scalar_parameters
)
for operand in operands
):
return "operand_not_observed"
return result, operands return result, operands
@staticmethod @staticmethod
@@ -1826,9 +1870,13 @@ class _PolicyMixin:
declaration: Declaration | None, declaration: Declaration | None,
symbol: str | None, symbol: str | None,
records: Sequence[EvidenceRecord], records: Sequence[EvidenceRecord],
scalar_parameters: Sequence[float] = (),
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
"""A derived figure must be the arithmetic its note states.""" """A derived figure must be the arithmetic its note states."""
derivation = self._derivation(declaration, symbol, records, money=figure.currency) derivation = self._derivation(
declaration, symbol, records, money=figure.currency,
scalar_parameters=scalar_parameters,
)
if isinstance(derivation, str): if isinstance(derivation, str):
return [ return [
self._figure_issue( self._figure_issue(
@@ -1839,7 +1887,12 @@ class _PolicyMixin:
"derived", "derived",
symbol, symbol,
derivation, derivation,
"is declared derived, but its note adds or subtracts an operand " "uses a data operand this session did not observe. Fetch the missing "
"data; do not relabel a numerator or denominator as a parameter. "
"For a genuinely chosen coefficient, declare its value as count and "
"show that value and its note label in the answer"
if derivation == "operand_not_observed"
else "is declared derived, but its note adds or subtracts an operand "
"this session did not observe" "this session did not observe"
if derivation == "additive_operand_not_observed" if derivation == "additive_operand_not_observed"
else "is declared derived, but its note is not arithmetic over at " else "is declared derived, but its note is not arithmetic over at "
@@ -1875,6 +1928,7 @@ class _PolicyMixin:
declaration: Declaration | None, declaration: Declaration | None,
symbol: str | None, symbol: str | None,
records: Sequence[EvidenceRecord], records: Sequence[EvidenceRecord],
scalar_parameters: Sequence[float] = (),
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
"""A proposed level (entry, target, stop) is derived or inside the observed range. """A proposed level (entry, target, stop) is derived or inside the observed range.
@@ -1907,7 +1961,10 @@ class _PolicyMixin:
"instrument and nothing attributes it to one", "instrument and nothing attributes it to one",
) )
] ]
derivation = self._derivation(declaration, symbol, records, money=figure.currency) derivation = self._derivation(
declaration, symbol, records, money=figure.currency,
scalar_parameters=scalar_parameters,
)
if not isinstance(derivation, str) and self._result_matches(figure, derivation[0]): if not isinstance(derivation, str) and self._result_matches(figure, derivation[0]):
return [] return []
candidates = self._price_candidates(symbol, records) candidates = self._price_candidates(symbol, records)
agent/src/agent/grounding/release.py
+4 -1
View File
@@ -88,6 +88,7 @@ _CORRECTION_REASONS = {
"unparseable_declaration": "the declaration line could not be read as `value | role | note | ref`", "unparseable_declaration": "the declaration line could not be read as `value | role | note | ref`",
"symbol_never_handled": "no tool call in this session passed in or returned that symbol", "symbol_never_handled": "no tool call in this session passed in or returned that symbol",
"additive_operand_not_observed": "a number its note adds or subtracts is not a value this session observed", "additive_operand_not_observed": "a number its note adds or subtracts is not a value this session observed",
"operand_not_observed": "a data operand is missing from this session's evidence; fetch it, or explicitly declare and show a genuinely chosen coefficient with its note label",
"role_in_price_column": "it sits in a price column, which holds only observed prints", "role_in_price_column": "it sits in a price column, which holds only observed prints",
"proposed_not_a_price": "a proposed level is a price, not a percentage; state the price it implies", "proposed_not_a_price": "a proposed level is a price, not a percentage; state the price it implies",
"symbol_mismatch": "it is declared for one instrument but the sentence writes it about another", "symbol_mismatch": "it is declared for one instrument but the sentence writes it about another",
@@ -242,7 +243,9 @@ class _ReleaseMixin:
"cell, one per line as `value | role | note | ref`, where role is one " "cell, one per line as `value | role | note | ref`, where role is one "
"of observed / derived / proposed / cited / count.", "of observed / derived / proposed / cited / count.",
"observed must appear in the tool results; derived needs a note that is " "observed must appear in the tool results; derived needs a note that is "
"the arithmetic itself, with one operand this session observed; " "the arithmetic itself, with every data operand observed. A chosen "
"multiplier/divisor needs a count declaration plus its value and note "
"label on the same visible line. Never relabel missing data as a parameter; "
"proposed must be derived or lie inside the observed price range; " "proposed must be derived or lie inside the observed price range; "
"cited needs a source in its note; count is not checked.", "cited needs a source in its note; count is not checked.",
"Reuse the exact locked symbol and venue.", "Reuse the exact locked symbol and venue.",
agent/src/agent/loop.py
+14 -109
View File
@@ -142,11 +142,11 @@ from src.agent.tool_results import ( # noqa: E402
_FORCED_TEXT_TOOL_CALL_RE as _FORCED_TEXT_TOOL_CALL_RE, _FORCED_TEXT_TOOL_CALL_RE as _FORCED_TEXT_TOOL_CALL_RE,
_DSML_BAR_TOOL_CALL_RE as _DSML_BAR_TOOL_CALL_RE, _DSML_BAR_TOOL_CALL_RE as _DSML_BAR_TOOL_CALL_RE,
_TARGET_PATH_RE as _TARGET_PATH_RE, _TARGET_PATH_RE as _TARGET_PATH_RE,
_TARGET_ACTION_RE, _TARGET_ACTION_RE as _TARGET_ACTION_RE,
_failure_code, _failure_code,
_is_tool_success, _is_tool_success,
_looks_like_tool_call_syntax, _looks_like_tool_call_syntax,
_named_target_paths, _named_target_paths as _named_target_paths,
_normalize_tool_run_dir, _normalize_tool_run_dir,
_previously_archived as _previously_archived, _previously_archived as _previously_archived,
_archive_backtest_result, _archive_backtest_result,
@@ -2170,119 +2170,24 @@ class AgentLoop:
return True return True
def _record_written_target(self, arguments: Mapping[str, Any]) -> None: def _record_written_target(self, arguments: Mapping[str, Any]) -> None:
"""Remember a file written by write_file/edit_file for completion checks.""" """Track writes with the active response's model identity."""
raw = arguments.get("path") or arguments.get("file_path") from src.agent.write_targets import record_written_target
if not raw:
return
p = Path(str(raw))
if not p.is_absolute() and self.memory.run_dir:
p = Path(self.memory.run_dir) / p
try:
resolved = p.resolve()
self._written_files.add(str(resolved).casefold())
except (OSError, ValueError):
resolved = p
self._written_files.add(str(p).casefold())
if not self.memory.run_dir: record_written_target(
return arguments, written_files=self._written_files,
run_root = Path(self.memory.run_dir).resolve() run_dir=self.memory.run_dir, llm_runtime=self._llm_runtime,
try: active_model_id=self._active_model_id,
relative = resolved.relative_to(run_root).as_posix() active_model_source=self._active_model_source,
except ValueError: )
return
if relative not in {"config.json", "code/signal_engine.py"}:
return
metadata_path = run_root / "strategy_provenance.json"
try:
metadata = json.loads(metadata_path.read_text(encoding="utf-8"))
except FileNotFoundError:
files = {}
except (OSError, json.JSONDecodeError) as exc:
logger.warning(
"Could not read strategy model provenance; update skipped: %s",
exc,
)
return
else:
if not isinstance(metadata, dict):
logger.warning("Invalid strategy model provenance; update skipped.")
return
files = metadata.get("files", {})
if not isinstance(files, dict):
logger.warning("Invalid strategy model provenance files; update skipped.")
return
files[relative] = {
"provider": self._llm_runtime.provider or None,
"model_id": self._active_model_id or None,
"model_source": self._active_model_source,
}
try:
metadata_path.write_text(
json.dumps({"files": files}, ensure_ascii=False, sort_keys=True) + "\n",
encoding="utf-8",
)
except OSError as exc:
logger.warning("Could not write strategy model provenance: %s", exc)
def _pending_write_directive( def _pending_write_directive(
self, user_message: str, run_started_wall: float self, user_message: str, run_started_wall: float
) -> str: ) -> str:
"""Return a directive naming task target files not yet written this run. """Name task targets not yet written in this run."""
from src.agent.write_targets import pending_write_directive
Empty when the message names no .md targets, carries no create/update return pending_write_directive(
intent, or every named target has been written (directly via user_message, run_started_wall, written_files=self._written_files,
write_file/edit_file, or by any process whose mtime is newer than the
run start - which covers the bash workaround).
"""
if not _TARGET_ACTION_RE.search(user_message or ""):
return ""
targets = _named_target_paths(user_message)
if not targets:
return ""
pending: list[str] = []
# A bare target ("create X.md at the same folder") carries no directory;
# resolve it against the folders of the absolute targets named in the
# same message, or the run would never see the file the model wrote to
# the intended folder and would report a false "not written".
base_dirs = [p.parent for p in targets if p.parent != Path(".")]
any_written = False
for p in targets:
candidates = [p]
if not p.is_absolute() and p.parent == Path("."):
candidates += [d / p.name for d in base_dirs]
written = False
for cand in candidates:
try:
key = str(cand.resolve()).casefold()
except (OSError, ValueError):
key = str(cand).casefold()
if key in self._written_files:
written = True
break
if not written:
try:
if cand.exists() and cand.stat().st_mtime >= run_started_wall:
written = True
break
except OSError:
pass
if written:
any_written = True
else:
pending.append(str(p))
# If any named target was written this run, the create/update task is
# addressed; reference files named alongside (e.g. "refer to A, create B")
# are never "written" and must not trigger the directive.
if any_written or not pending:
return ""
return (
"[SYSTEM] The following task target file(s) have NOT been written "
"yet in this run: " + ", ".join(pending) + ". Write them NOW using "
"write_file/edit_file (or bash for file operations). If your task "
"was to create or update a file, a plain-text answer without the "
"file write is a failure."
) )
def _context_budget(self, messages: list) -> CompactionBudget: def _context_budget(self, messages: list) -> CompactionBudget:
agent/src/agent/write_targets.py
+152
View File
@@ -0,0 +1,152 @@
"""Write-target tracking: written-file ledger + pending-write directive.
Extracted from ``AgentLoop`` (issue #1624, roadmap item: split ``loop.py``
along its existing seams). The regex/path helpers this builds on —
``_TARGET_ACTION_RE`` and ``_named_target_paths`` — already moved to
``src.agent.tool_results`` in the first extraction (#1636); this completes
that relocation by moving the two class methods that consumed them, so all
of write-target tracking lives together and ``loop.py`` stays the
orchestrator.
State stays on the loop: ``written_files`` is passed in and mutated in
place by ``record_written_target``; the provenance identity fields are
read at delegation time in ``loop.py``. Behavior is unchanged — every
branch condition, regex, and message string moved verbatim.
"""
from __future__ import annotations
import json
import logging
from pathlib import Path
from typing import Any, Mapping
from src.agent.tool_results import _TARGET_ACTION_RE, _named_target_paths
logger = logging.getLogger(__name__)
def record_written_target(
arguments: Mapping[str, Any],
*,
written_files: set[str],
run_dir: str | None,
llm_runtime: Any,
active_model_id: str | None,
active_model_source: str,
) -> None:
"""Remember a file written by write_file/edit_file for completion checks."""
raw = arguments.get("path") or arguments.get("file_path")
if not raw:
return
p = Path(str(raw))
if not p.is_absolute() and run_dir:
p = Path(run_dir) / p
try:
resolved = p.resolve()
written_files.add(str(resolved).casefold())
except (OSError, ValueError):
resolved = p
written_files.add(str(p).casefold())
if not run_dir:
return
run_root = Path(run_dir).resolve()
try:
relative = resolved.relative_to(run_root).as_posix()
except ValueError:
return
if relative not in {"config.json", "code/signal_engine.py"}:
return
metadata_path = run_root / "strategy_provenance.json"
try:
metadata = json.loads(metadata_path.read_text(encoding="utf-8"))
except FileNotFoundError:
files = {}
except (OSError, json.JSONDecodeError) as exc:
logger.warning(
"Could not read strategy model provenance; update skipped: %s",
exc,
)
return
else:
if not isinstance(metadata, dict):
logger.warning("Invalid strategy model provenance; update skipped.")
return
files = metadata.get("files", {})
if not isinstance(files, dict):
logger.warning("Invalid strategy model provenance files; update skipped.")
return
files[relative] = {
"provider": llm_runtime.provider or None,
"model_id": active_model_id or None,
"model_source": active_model_source,
}
try:
metadata_path.write_text(
json.dumps({"files": files}, ensure_ascii=False, sort_keys=True) + "\n",
encoding="utf-8",
)
except OSError as exc:
logger.warning("Could not write strategy model provenance: %s", exc)
def pending_write_directive(
user_message: str, run_started_wall: float, *, written_files: set[str]
) -> str:
"""Return a directive naming task target files not yet written this run.
Empty when the message names no .md targets, carries no create/update
intent, or every named target has been written (directly via
write_file/edit_file, or by any process whose mtime is newer than the
run start - which covers the bash workaround).
"""
if not _TARGET_ACTION_RE.search(user_message or ""):
return ""
targets = _named_target_paths(user_message)
if not targets:
return ""
pending: list[str] = []
# A bare target ("create X.md at the same folder") carries no directory;
# resolve it against the folders of the absolute targets named in the
# same message, or the run would never see the file the model wrote to
# the intended folder and would report a false "not written".
base_dirs = [p.parent for p in targets if p.parent != Path(".")]
any_written = False
for p in targets:
candidates = [p]
if not p.is_absolute() and p.parent == Path("."):
candidates += [d / p.name for d in base_dirs]
written = False
for cand in candidates:
try:
key = str(cand.resolve()).casefold()
except (OSError, ValueError):
key = str(cand).casefold()
if key in written_files:
written = True
break
if not written:
try:
if cand.exists() and cand.stat().st_mtime >= run_started_wall:
written = True
break
except OSError:
pass
if written:
any_written = True
else:
pending.append(str(p))
# If any named target was written this run, the create/update task is
# addressed; reference files named alongside (e.g. "refer to A, create B")
# are never "written" and must not trigger the directive.
if any_written or not pending:
return ""
return (
"[SYSTEM] The following task target file(s) have NOT been written "
"yet in this run: " + ", ".join(pending) + ". Write them NOW using "
"write_file/edit_file (or bash for file operations). If your task "
"was to create or update a file, a plain-text answer without the "
"file write is a failure."
)
agent/src/api/attribution_core.py
+36 -9
View File
@@ -514,10 +514,22 @@ def _attribution_run_brinson(
def _attribution_load_position_weights(run_dir: Path, notes: List[str]) -> Optional[Dict[str, float]]: def _attribution_load_position_weights(run_dir: Path, notes: List[str]) -> Optional[Dict[str, float]]:
"""Read the last row of ``positions.csv`` as per-symbol weights. """Read the book Brinson should attribute from ``positions.csv``.
Returns ``None`` (with a skip note) when the artifact is missing or cannot A run closes every position on its last bar (``end_of_backtest``), so the
be parsed into at least one weighted symbol column. final row is all zeros by construction and describes the liquidation, not
the portfolio. When that row is flat, the row before it is read instead,
and the note names its timestamp. Only one row is stepped back: a book that
was already flat before the close is reported as flat rather than replaced
by some older holding.
Args:
run_dir: Backtest run directory containing ``artifacts/positions.csv``.
notes: Report notes; skip and fallback reasons are appended here.
Returns:
Per-symbol weights, or ``None`` (with a skip note) when the artifact is
missing or has no weighted symbol column.
""" """
path = run_dir / "artifacts" / "positions.csv" path = run_dir / "artifacts" / "positions.csv"
try: try:
@@ -535,12 +547,27 @@ def _attribution_load_position_weights(run_dir: Path, notes: List[str]) -> Optio
if not symbols: if not symbols:
notes.append("brinson attribution skipped: positions.csv could not be parsed") notes.append("brinson attribution skipped: positions.csv could not be parsed")
return None return None
last_row = rows[-1]
weights: Dict[str, float] = {} def _row_weights(row: Dict[str, Any]) -> Dict[str, float]:
for symbol in symbols: out: Dict[str, float] = {}
weight = _attribution_finite_float(last_row.get(symbol)) for symbol in symbols:
weights[symbol] = weight if weight is not None else 0.0 weight = _attribution_finite_float(row.get(symbol))
return weights out[symbol] = weight if weight is not None else 0.0
return out
weights = _row_weights(rows[-1])
if any(weights.values()) or len(rows) < 2:
return weights
held = _row_weights(rows[-2])
if not any(held.values()):
# Flat before the close as well: the downstream zero-check reports it.
return weights
timestamp = str(rows[-2].get(fieldnames[0]) or "").strip()
notes.append(
"brinson attribution used the positions row before the end-of-backtest "
f"liquidation ({timestamp or 'second-to-last row'})"
)
return held
def _attribution_brinson_section( def _attribution_brinson_section(
agent/src/br_data/__init__.py
+17
View File
@@ -0,0 +1,17 @@
"""Free, official Brazilian market data synced into one local SQLite file.
Sources (all public, no key):
* B3 COTAHIST — daily quotes of every instrument traded on B3 (shares, units,
FIIs, ETFs, BDRs, options, forward market) since 1986.
* CVM Dados Abertos — daily NAV/quota of every registered investment fund,
the fund register, the listed-company register and annual/quarterly
financial statements (DFP/ITR).
* Banco Central (SGS) — Selic, CDI, IPCA, IGP-M, PTAX and any other series.
* Tesouro Transparente — Tesouro Direto bond prices and yields.
``python -m src.br_data sync`` downloads/refreshes everything;
``python -m src.br_data daemon`` repeats it once a day. Readers
(:mod:`backtest.loaders.b3_cotahist_loader`, ``get_br_market_data``) only
read the SQLite file.
"""
agent/src/br_data/__main__.py
+58
View File
@@ -0,0 +1,58 @@
"""CLI: ``python -m src.br_data {sync,daemon,status}``."""
from __future__ import annotations
import argparse
import json
import logging
import sys
from src.br_data.sync import SOURCES, run_daemon, run_sync
def _status() -> dict:
from src.br_data.store import connect, db_path
out: dict = {"db": str(db_path())}
try:
with connect() as conn:
for table, col in (
("b3_quotes", "trade_date"), ("cvm_fund_daily", "trade_date"),
("cvm_statements", "period_end"), ("bcb_series", "obs_date"),
("tesouro_prices", "base_date"),
):
row = conn.execute(f"SELECT COUNT(*), MIN({col}), MAX({col}) FROM {table}").fetchone()
out[table] = {"rows": row[0], "from": row[1], "to": row[2]}
for table in ("cvm_funds", "cvm_companies", "cvm_company_tickers"):
out[table] = {"rows": conn.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]}
except FileNotFoundError as exc:
out["error"] = str(exc)
return out
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="python -m src.br_data", description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
for name in ("sync", "daemon"):
cmd = sub.add_parser(name)
cmd.add_argument("--only", nargs="+", choices=SOURCES, help="sources to sync (default: all)")
if name == "sync":
cmd.add_argument("--cotahist-from", type=int, help="first COTAHIST year (1986+)")
cmd.add_argument("--cvm-from", type=int, help="first CVM year")
sub.add_parser("status")
args = parser.parse_args(argv)
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
if args.command == "status":
print(json.dumps(_status(), indent=2, ensure_ascii=False))
return 0
if args.command == "daemon":
run_daemon(args.only)
return 0
report = run_sync(args.only, cotahist_from=args.cotahist_from, cvm_from=args.cvm_from)
print(json.dumps(report, indent=2, ensure_ascii=False))
return 1 if any(isinstance(v, dict) and "error" in v for v in report.values()) else 0
if __name__ == "__main__":
sys.exit(main())
agent/src/br_data/_download.py
+78
View File
@@ -0,0 +1,78 @@
"""Download helpers shared by the Brazilian dataset syncs."""
from __future__ import annotations
import io
import logging
import zipfile
import math
from datetime import datetime
from typing import Optional
from backtest.loaders._http import throttled_get
logger = logging.getLogger(__name__)
_MIN_INTERVAL_S = 1.0
_TIMEOUT_S = 300.0
def fetch_bytes(url: str, host_key: str) -> Optional[bytes]:
"""GET ``url``; ``None`` on 404 (file not published), raise on other errors."""
response = throttled_get(url, host_key=host_key, min_interval=_MIN_INTERVAL_S, timeout=_TIMEOUT_S)
if response.status_code == 404:
return None
response.raise_for_status()
return response.content
def zip_members(payload: bytes) -> dict[str, bytes]:
"""Return ``{member name: bytes}`` for every file inside a ZIP payload."""
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
return {
info.filename: archive.read(info)
for info in archive.infolist()
if not info.is_dir()
}
def decode_text(raw: bytes) -> str:
"""Decode a CVM/B3 text file (UTF-8 when valid, else Latin-1)."""
try:
return raw.decode("utf-8-sig")
except UnicodeDecodeError:
return raw.decode("latin-1")
def to_float(value: str | None) -> Optional[float]:
"""Parse ``1234.56`` or Brazilian ``1.234,56``; blank -> ``None``."""
if value is None:
return None
text = str(value).strip()
if not text:
return None
if "," in text:
text = text.replace(".", "").replace(",", ".")
try:
number = float(text)
return number if math.isfinite(number) else None
except ValueError:
return None
def br_date(value: str | None) -> Optional[str]:
"""``dd/mm/yyyy`` or ``yyyy-mm-dd`` -> ``yyyy-mm-dd``; anything else -> ``None``."""
if not value:
return None
text = str(value).strip()[:10]
for fmt in ("%d/%m/%Y", "%Y-%m-%d"):
try:
return datetime.strptime(text, fmt).date().isoformat()
except ValueError:
continue
return None
def digits(value: str | None) -> str:
"""Keep only the digits (CNPJ normalization)."""
return "".join(ch for ch in str(value or "") if ch.isdigit())
agent/src/br_data/bcb.py
+97
View File
@@ -0,0 +1,97 @@
"""Banco Central do Brasil — SGS time series (Selic, CDI, IPCA, PTAX ...).
API: ``https://api.bcb.gov.br/dados/serie/bcdata.sgs.{code}/dados``
``?formato=json&dataInicial=dd/mm/yyyy&dataFinal=dd/mm/yyyy`` returns
``[{"data": "dd/mm/yyyy", "valor": "1.23"}, ...]``. Daily series are served in
windows of at most 10 years, so requests are chunked.
"""
from __future__ import annotations
import logging
import sqlite3
from datetime import date, timedelta
from typing import Optional
from backtest.loaders._http import throttled_get
from src.br_data._download import br_date, to_float
logger = logging.getLogger(__name__)
HOST_KEY = "bcb_sgs"
BASE_URL = "https://api.bcb.gov.br/dados/serie/bcdata.sgs.{code}/dados"
_WINDOW_DAYS = 365 * 5
# Series synced by default (code -> description).
SERIES: dict[int, str] = {
11: "Selic diária (% a.d.)",
432: "Meta Selic (% a.a.)",
12: "CDI diário (% a.d.)",
4389: "CDI anualizado base 252 (% a.a.)",
433: "IPCA mensal (%)",
13522: "IPCA acumulado 12 meses (%)",
189: "IGP-M mensal (%)",
188: "INPC mensal (%)",
1: "Dólar comercial venda - PTAX (R$/US$)",
21619: "Euro venda - PTAX (R$/€)",
4380: "PIB mensal - valores correntes (R$ milhões)",
24369: "Taxa de desocupação - PNAD (%)",
}
def fetch_series(code: int, start: date, end: date) -> list[tuple[str, Optional[float]]]:
"""Download ``code`` between ``start`` and ``end`` (inclusive)."""
out: list[tuple[str, Optional[float]]] = []
window_start = start
while window_start <= end:
window_end = min(window_start + timedelta(days=_WINDOW_DAYS), end)
response = throttled_get(
BASE_URL.format(code=code),
host_key=HOST_KEY,
min_interval=0.5,
params={
"formato": "json",
"dataInicial": f"{window_start:%d/%m/%Y}",
"dataFinal": f"{window_end:%d/%m/%Y}",
},
timeout=60,
)
# SGS answers 404 for a window with no observations (e.g. a monthly
# series queried over a few days); that is "no data", not an error.
if response.status_code == 404:
window_start = window_end + timedelta(days=1)
continue
response.raise_for_status()
for item in response.json() or []:
obs = br_date(item.get("data"))
if obs:
out.append((obs, to_float(item.get("valor"))))
window_start = window_end + timedelta(days=1)
return out
def store_series(conn: sqlite3.Connection, code: int, rows: list[tuple[str, Optional[float]]]) -> int:
conn.executemany(
"INSERT OR REPLACE INTO bcb_series (code, obs_date, value) VALUES (?, ?, ?)",
[(code, obs, value) for obs, value in rows],
)
return len(rows)
def sync(conn: sqlite3.Connection, start_year: int, today: date, codes: Optional[list[int]] = None) -> int:
"""Top up each series from its last stored observation (or ``start_year``)."""
total = 0
failures: list[str] = []
codes = codes or list(SERIES)
for code in codes:
row = conn.execute("SELECT MAX(obs_date) FROM bcb_series WHERE code = ?", (code,)).fetchone()
start = date.fromisoformat(row[0]) - timedelta(days=45) if row and row[0] else date(start_year, 1, 1)
try:
total += store_series(conn, code, fetch_series(code, start, today))
conn.commit()
except Exception as exc: # noqa: BLE001 - one series must not stop the rest
logger.warning("bcb: series %s failed: %s", code, exc)
failures.append(f"{code}: {type(exc).__name__}")
if failures and len(failures) == len(codes):
raise RuntimeError(f"every BCB series failed ({'; '.join(failures)})")
return total
agent/src/br_data/cotahist.py
+197
View File
@@ -0,0 +1,197 @@
"""B3 COTAHIST: daily quotes of every instrument traded on B3.
B3 publishes the "Séries Históricas" as fixed-width text files (245 chars per
record, Latin-1) inside ZIPs:
* yearly ``COTAHIST_A{YYYY}.ZIP`` (1986 onwards; the current year is updated
every trading day)
* daily ``COTAHIST_D{DDMMYYYY}.ZIP``
Record layout (type 01, 1-based positions from B3's published layout):
DATA 3-10, CODBDI 11-12, CODNEG 13-24, TPMERC 25-27, NOMRES 28-39,
ESPECI 40-49, PREABE 57-69, PREMAX 70-82, PREMIN 83-95, PREMED 96-108,
PREULT 109-121, PREOFC 122-134, PREOFV 135-147, TOTNEG 148-152,
QUATOT 153-170, VOLTOT 171-188, PREEXE 189-201, DATVEN 203-210,
FATCOT 211-217, CODISI 231-242. Prices carry two implied decimals and are
quoted per FATCOT units.
Prices are as traded (not adjusted for dividends or splits).
"""
from __future__ import annotations
import io
import logging
import sqlite3
import zipfile
from datetime import date, timedelta
from typing import Iterable, Iterator, Optional
from src.br_data._download import fetch_bytes
from src.br_data.store import is_done, mark_done
logger = logging.getLogger(__name__)
HOST_KEY = "b3_cotahist"
BASE_URL = "https://bvmf.bmfbovespa.com.br/InstDados/SerHist"
FIRST_YEAR = 1986
_COLUMNS = (
"trade_date", "ticker", "bdi", "market_type", "short_name", "spec",
"open", "high", "low", "avg", "close", "best_bid", "best_ask",
"trades", "volume", "turnover", "strike", "expiry", "quote_factor", "isin",
)
_INSERT = (
f"INSERT OR REPLACE INTO b3_quotes ({', '.join(_COLUMNS)}) "
f"VALUES ({', '.join('?' for _ in _COLUMNS)})"
)
def _num(field: str) -> int:
field = field.strip()
return int(field) if field.isdigit() else 0
def _iso(field: str) -> Optional[str]:
field = field.strip()
if len(field) != 8 or not field.isdigit() or field == "99991231":
return None
try:
return date(int(field[:4]), int(field[4:6]), int(field[6:8])).isoformat()
except ValueError:
return None
def parse_record(line: str) -> Optional[tuple]:
"""Parse one COTAHIST line; ``None`` for header/trailer/malformed lines."""
if len(line) < 245 or not line.startswith("01"):
return None
trade_date = _iso(line[2:10])
if trade_date is None:
return None
# Earlier COTAHIST history contains pre-real currencies. Never stamp those
# prices as BRL, nor fabricate missing quotation factors or OHLC values.
if line[52:56].strip() != "R$":
return None
factor = _num(line[210:217])
if factor <= 0 or any(not line[a:b].strip().isdigit()
for a, b in ((56, 69), (69, 82), (82, 95), (108, 121))):
return None
def price(start: int, end: int) -> float:
return _num(line[start:end]) / 100.0 / factor
return (
trade_date,
line[12:24].strip(),
line[10:12].strip(),
_num(line[24:27]),
line[27:39].strip(),
line[39:49].strip(),
price(56, 69),
price(69, 82),
price(82, 95),
price(95, 108),
price(108, 121),
price(121, 134),
price(134, 147),
_num(line[147:152]),
float(_num(line[152:170])),
_num(line[170:188]) / 100.0,
_num(line[188:201]) / 100.0 / factor,
_iso(line[202:210]),
factor,
line[230:242].strip(),
)
def parse_text(text: str) -> Iterator[tuple]:
"""Yield parsed records from a whole COTAHIST text body."""
for line in text.splitlines():
record = parse_record(line)
if record is not None:
yield record
def ingest_zip(conn: sqlite3.Connection, payload: bytes) -> int:
"""Insert every record of a COTAHIST ZIP; return the row count.
The yearly text is several hundred MB once unzipped, so it is streamed
line by line instead of being decoded in one piece.
"""
total = 0
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
for info in archive.infolist():
if info.is_dir():
continue
batch: list[tuple] = []
with archive.open(info) as raw:
for line in io.TextIOWrapper(raw, encoding="latin-1", newline=""):
record = parse_record(line.rstrip("\r\n"))
if record is None:
continue
batch.append(record)
if len(batch) >= 50_000:
conn.executemany(_INSERT, batch)
total += len(batch)
batch.clear()
if batch:
conn.executemany(_INSERT, batch)
total += len(batch)
return total
def _last_date(conn: sqlite3.Connection) -> Optional[date]:
row = conn.execute("SELECT MAX(trade_date) FROM b3_quotes").fetchone()
return date.fromisoformat(row[0]) if row and row[0] else None
def _weekdays(start: date, end: date) -> Iterable[date]:
day = start
while day <= end:
if day.weekday() < 5:
yield day
day += timedelta(days=1)
def sync(conn: sqlite3.Connection, start_year: int, today: date) -> int:
"""Download missing years in full, then the current year day by day.
Past years are fetched once (tracked in ``sync_log``). The current year is
fetched as one yearly file on first run, then topped up with the small
daily files for every weekday after the last stored date (holidays answer
404 and are skipped).
"""
start_year = max(start_year, FIRST_YEAR)
total = 0
for year in range(start_year, today.year):
item = f"A{year}"
if is_done(conn, "cotahist", item):
continue
logger.info("cotahist: downloading %s", item)
payload = fetch_bytes(f"{BASE_URL}/COTAHIST_A{year}.ZIP", HOST_KEY)
if payload is None:
logger.warning("cotahist: %s not published", item)
continue
rows = ingest_zip(conn, payload)
mark_done(conn, "cotahist", item, rows)
conn.commit()
total += rows
last = _last_date(conn)
if last is None or last.year < today.year:
logger.info("cotahist: downloading A%s (year to date)", today.year)
payload = fetch_bytes(f"{BASE_URL}/COTAHIST_A{today.year}.ZIP", HOST_KEY)
if payload is not None:
total += ingest_zip(conn, payload)
conn.commit()
last = _last_date(conn)
if last is not None and last.year == today.year:
for day in _weekdays(last + timedelta(days=1), today):
payload = fetch_bytes(f"{BASE_URL}/COTAHIST_D{day:%d%m%Y}.ZIP", HOST_KEY)
if payload is None:
continue
total += ingest_zip(conn, payload)
conn.commit()
return total
agent/src/br_data/cvm_companies.py
+169
View File
@@ -0,0 +1,169 @@
"""CVM listed companies: register, ticker map and financial statements.
* ``CIA_ABERTA/CAD/DADOS/cad_cia_aberta.csv`` — company register.
* ``CIA_ABERTA/DOC/FCA/DADOS/fca_cia_aberta_{YYYY}.zip`` — the
``valor_mobiliario`` file maps B3 tickers (``Codigo_Negociacao``) to CNPJ.
* ``CIA_ABERTA/DOC/DFP/DADOS/dfp_cia_aberta_{YYYY}.zip`` (annual) and
``.../ITR/DADOS/itr_cia_aberta_{YYYY}.zip`` (quarterly) — one CSV per
statement, e.g. ``dfp_cia_aberta_DRE_con_2025.csv``.
Only the current-period column (``ORDEM_EXERC = ÚLTIMO``) is kept and the
highest ``VERSAO`` wins. Consolidated statements are stored as ``DRE``,
``BPA`` ...; individual (parent-only) ones as ``DRE_ind`` ...
``ESCALA_MOEDA = MIL`` values are multiplied by 1000 so every value is BRL.
"""
from __future__ import annotations
import csv
import io
import logging
import re
import sqlite3
from datetime import date
from typing import Iterator, Optional
from src.br_data._download import br_date, decode_text, digits, fetch_bytes, to_float, zip_members
from src.br_data.store import is_done, mark_done
logger = logging.getLogger(__name__)
HOST_KEY = "cvm"
BASE_URL = "https://dados.cvm.gov.br/dados/CIA_ABERTA"
_STATEMENT_FILE = re.compile(
r"(?:dfp|itr)_cia_aberta_(?P<stmt>[A-Z_]+?)_(?P<scope>con|ind)_\d{4}\.csv$", re.I
)
_UPSERT_STATEMENT = (
"INSERT INTO cvm_statements (cnpj, doc, statement, period_end, period_start, account, "
"description, value, version) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) "
"ON CONFLICT(cnpj, doc, statement, period_end, period_start, account) DO UPDATE SET "
"description = excluded.description, value = excluded.value, version = excluded.version "
"WHERE excluded.version >= cvm_statements.version"
)
def _rows(text: str) -> Iterator[dict[str, str]]:
reader = csv.DictReader(io.StringIO(text), delimiter=";")
for row in reader:
yield {str(k or "").strip().upper(): (v or "").strip() for k, v in row.items()}
def _opt(value: Optional[str]) -> Optional[str]:
return value or None
def parse_register(text: str) -> Iterator[tuple]:
"""Yield ``cvm_companies`` tuples from cad_cia_aberta.csv."""
for row in _rows(text):
cnpj = digits(row.get("CNPJ_CIA"))
if not cnpj:
continue
yield (
cnpj,
_opt(row.get("CD_CVM")),
_opt(row.get("DENOM_SOCIAL")),
_opt(row.get("DENOM_COMERC")),
_opt(row.get("SIT")),
_opt(row.get("SETOR_ATIV")),
)
def parse_tickers(text: str) -> Iterator[tuple[str, str]]:
"""Yield ``(ticker, cnpj)`` from an FCA valor_mobiliario CSV."""
for row in _rows(text):
ticker = (row.get("CODIGO_NEGOCIACAO") or "").upper()
cnpj = digits(row.get("CNPJ_COMPANHIA") or row.get("CNPJ_CIA"))
if ticker and cnpj and re.fullmatch(r"[A-Z0-9]{4,12}", ticker):
yield ticker, cnpj
def parse_statement(text: str, doc: str, statement: str) -> Iterator[tuple]:
"""Yield ``cvm_statements`` tuples from one DFP/ITR statement CSV."""
for row in _rows(text):
order = row.get("ORDEM_EXERC", "").upper()
if order and not order.startswith(("ÚLTIMO", "ULTIMO")):
continue
cnpj = digits(row.get("CNPJ_CIA"))
period_end = br_date(row.get("DT_FIM_EXERC") or row.get("DT_REFER"))
account = row.get("CD_CONTA", "")
if not cnpj or not period_end or not account:
continue
currency = row.get("MOEDA", "").upper()
if currency and currency not in {"REAL", "REAIS", "BRL", "R$"}:
raise ValueError(f"Unsupported CVM statement currency: {currency}")
value = to_float(row.get("VL_CONTA"))
if value is not None and row.get("ESCALA_MOEDA", "").upper() == "MIL":
value *= 1000.0
version = to_float(row.get("VERSAO"))
yield (
cnpj,
doc,
statement,
period_end,
br_date(row.get("DT_INI_EXERC")) or "",
account,
_opt(row.get("DS_CONTA")),
value,
int(version) if version is not None else 0,
)
def ingest_statements_zip(conn: sqlite3.Connection, payload: bytes, doc: str) -> int:
"""Insert every statement CSV of a DFP/ITR ZIP; return the row count."""
total = 0
for name, raw in zip_members(payload).items():
match = _STATEMENT_FILE.search(name.rsplit("/", 1)[-1])
if not match:
continue
statement = match.group("stmt").upper()
if match.group("scope").lower() == "ind":
statement += "_ind"
batch = list(parse_statement(decode_text(raw), doc, statement))
conn.executemany(_UPSERT_STATEMENT, batch)
total += len(batch)
return total
def sync(conn: sqlite3.Connection, start_year: int, today: date) -> int:
"""Refresh register + tickers, then DFP/ITR from ``start_year``.
The current and previous year are always refreshed (filings and
restatements keep arriving); older years are fetched once.
"""
total = 0
register = fetch_bytes(f"{BASE_URL}/CAD/DADOS/cad_cia_aberta.csv", HOST_KEY)
if register is not None:
batch = list(parse_register(decode_text(register)))
conn.executemany("INSERT OR REPLACE INTO cvm_companies VALUES (?, ?, ?, ?, ?, ?)", batch)
total += len(batch)
conn.commit()
for year in (today.year, today.year - 1):
fca = fetch_bytes(f"{BASE_URL}/DOC/FCA/DADOS/fca_cia_aberta_{year}.zip", HOST_KEY)
if fca is None:
continue
for name, raw in zip_members(fca).items():
if "valor_mobiliario" in name.lower():
pairs = list(parse_tickers(decode_text(raw)))
conn.executemany("INSERT OR REPLACE INTO cvm_company_tickers VALUES (?, ?)", pairs)
total += len(pairs)
conn.commit()
break
for doc in ("DFP", "ITR"):
for year in range(start_year, today.year + 1):
item = f"{doc}{year}"
recent = year >= today.year - 1
if not recent and is_done(conn, "cvm_statements", item):
continue
url = f"{BASE_URL}/DOC/{doc}/DADOS/{doc.lower()}_cia_aberta_{year}.zip"
payload = fetch_bytes(url, HOST_KEY)
if payload is None:
continue
rows = ingest_statements_zip(conn, payload, doc)
if not recent:
mark_done(conn, "cvm_statements", item, rows)
conn.commit()
total += rows
return total
agent/src/br_data/cvm_funds.py
+183
View File
@@ -0,0 +1,183 @@
"""CVM investment funds: register and daily report (informe diário).
* Daily report — quota value, net assets, inflows/outflows and holders of
every registered fund/class, one ZIP per month:
``FI/DOC/INF_DIARIO/DADOS/inf_diario_fi_{YYYYMM}.zip`` (older years as
``.../DADOS/HIST/inf_diario_fi_{YYYY}.zip``). Since CVM Resolution 175 the
key columns are ``CNPJ_FUNDO_CLASSE``/``TP_FUNDO_CLASSE``/``ID_SUBCLASSE``;
older files use ``CNPJ_FUNDO``/``TP_FUNDO``. Both are accepted.
* Register — legacy ``FI/CAD/DADOS/cad_fi.csv`` plus the post-175
``FI/CAD/DADOS/registro_fundo_classe.zip`` (fund and class registers).
Column lookups are case-insensitive and tolerate either naming, because CVM
renames columns between layouts.
"""
from __future__ import annotations
import csv
import io
import logging
import sqlite3
from datetime import date
from typing import Iterable, Iterator, Optional
from src.br_data._download import br_date, decode_text, digits, fetch_bytes, to_float, zip_members
from src.br_data.store import is_done, mark_done
logger = logging.getLogger(__name__)
HOST_KEY = "cvm"
BASE_URL = "https://dados.cvm.gov.br/dados/FI"
_INSERT_DAILY = (
"INSERT OR REPLACE INTO cvm_fund_daily (cnpj, trade_date, fund_type, subclass, "
"total_value, quota, net_assets, inflows, outflows, holders) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
)
_UPSERT_FUND = (
"INSERT INTO cvm_funds (cnpj, name, status, fund_class, fund_type, manager, admin, "
"benchmark, start_date) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) "
"ON CONFLICT(cnpj) DO UPDATE SET "
"name = COALESCE(excluded.name, name), status = COALESCE(excluded.status, status), "
"fund_class = COALESCE(excluded.fund_class, fund_class), "
"fund_type = COALESCE(excluded.fund_type, fund_type), "
"manager = COALESCE(excluded.manager, manager), admin = COALESCE(excluded.admin, admin), "
"benchmark = COALESCE(excluded.benchmark, benchmark), "
"start_date = COALESCE(excluded.start_date, start_date)"
)
def _rows(text: str) -> Iterator[dict[str, str]]:
"""Iterate a ``;``-separated CVM CSV with upper-cased header keys."""
reader = csv.DictReader(io.StringIO(text), delimiter=";")
for row in reader:
yield {str(k or "").strip().upper(): (v or "").strip() for k, v in row.items()}
def _pick(row: dict[str, str], *names: str) -> str:
for name in names:
value = row.get(name.upper())
if value:
return value
return ""
def _opt(value: str) -> Optional[str]:
return value or None
def parse_daily(text: str) -> Iterator[tuple]:
"""Yield ``cvm_fund_daily`` tuples from one informe-diário CSV."""
for row in _rows(text):
cnpj = digits(_pick(row, "CNPJ_FUNDO_CLASSE", "CNPJ_FUNDO"))
trade_date = br_date(_pick(row, "DT_COMPTC"))
if not cnpj or not trade_date:
continue
holders = to_float(_pick(row, "NR_COTST"))
yield (
cnpj,
trade_date,
_opt(_pick(row, "TP_FUNDO_CLASSE", "TP_FUNDO")),
_pick(row, "ID_SUBCLASSE"),
to_float(_pick(row, "VL_TOTAL")),
to_float(_pick(row, "VL_QUOTA")),
to_float(_pick(row, "VL_PATRIM_LIQ")),
to_float(_pick(row, "CAPTC_DIA")),
to_float(_pick(row, "RESG_DIA")),
int(holders) if holders is not None else None,
)
def ingest_daily_zip(conn: sqlite3.Connection, payload: bytes) -> int:
"""Insert every CSV inside an informe-diário ZIP; return the row count."""
total = 0
for name, raw in zip_members(payload).items():
if not name.lower().endswith(".csv"):
continue
batch = list(parse_daily(decode_text(raw)))
conn.executemany(_INSERT_DAILY, batch)
total += len(batch)
return total
def parse_register(text: str) -> Iterator[tuple]:
"""Yield ``cvm_funds`` tuples from cad_fi.csv or registro_*.csv."""
for row in _rows(text):
cnpj = digits(_pick(row, "CNPJ_FUNDO", "CNPJ_CLASSE", "CNPJ_FUNDO_CLASSE"))
if not cnpj:
continue
yield (
cnpj,
_opt(_pick(row, "DENOM_SOCIAL", "DENOMINACAO_SOCIAL")),
_opt(_pick(row, "SIT", "SITUACAO")),
_opt(_pick(row, "CLASSE", "CLASSIFICACAO", "CLASSE_ANBIMA")),
_opt(_pick(row, "TP_FUNDO", "TIPO_CLASSE", "TIPO_FUNDO")),
_opt(_pick(row, "GESTOR")),
_opt(_pick(row, "ADMIN", "ADMINISTRADOR")),
_opt(_pick(row, "RENTAB_FUNDO", "INDICADOR_DESEMPENHO")),
br_date(_pick(row, "DT_INI_ATIV", "DATA_INICIO", "DT_REG", "DATA_REGISTRO")),
)
def _months(start: date, end: date) -> Iterable[tuple[int, int]]:
year, month = start.year, start.month
while (year, month) <= (end.year, end.month):
yield year, month
month += 1
if month == 13:
year, month = year + 1, 1
def sync_register(conn: sqlite3.Connection) -> int:
"""Refresh the fund register (both layouts, best effort each)."""
total = 0
legacy = fetch_bytes(f"{BASE_URL}/CAD/DADOS/cad_fi.csv", HOST_KEY)
if legacy is not None:
batch = list(parse_register(decode_text(legacy)))
conn.executemany(_UPSERT_FUND, batch)
total += len(batch)
current = fetch_bytes(f"{BASE_URL}/CAD/DADOS/registro_fundo_classe.zip", HOST_KEY)
if current is not None:
for name, raw in zip_members(current).items():
if name.lower().endswith(".csv") and "subclasse" not in name.lower():
batch = list(parse_register(decode_text(raw)))
conn.executemany(_UPSERT_FUND, batch)
total += len(batch)
conn.commit()
return total
def sync_daily(conn: sqlite3.Connection, start_year: int, today: date) -> int:
"""Fetch monthly reports; the last two months are always refreshed.
CVM revises recent months, so only months older than that are recorded as
done. A month that 404s falls back to the yearly HIST archive once.
"""
total = 0
recent = {(today.year, today.month)}
prev = (today.year - 1, 12) if today.month == 1 else (today.year, today.month - 1)
recent.add(prev)
hist_years_done: set[int] = set()
for year, month in _months(date(start_year, 1, 1), today):
item = f"{year}{month:02d}"
if (year, month) not in recent and is_done(conn, "cvm_fund_daily", item):
continue
payload = fetch_bytes(f"{BASE_URL}/DOC/INF_DIARIO/DADOS/inf_diario_fi_{item}.zip", HOST_KEY)
if payload is None:
if year in hist_years_done or is_done(conn, "cvm_fund_daily", f"HIST{year}"):
continue
hist = fetch_bytes(f"{BASE_URL}/DOC/INF_DIARIO/DADOS/HIST/inf_diario_fi_{year}.zip", HOST_KEY)
hist_years_done.add(year)
if hist is not None:
rows = ingest_daily_zip(conn, hist)
mark_done(conn, "cvm_fund_daily", f"HIST{year}", rows)
conn.commit()
total += rows
continue
rows = ingest_daily_zip(conn, payload)
if (year, month) not in recent:
mark_done(conn, "cvm_fund_daily", item, rows)
conn.commit()
total += rows
return total
agent/src/br_data/store.py
+180
View File
@@ -0,0 +1,180 @@
"""SQLite store for the Brazilian official datasets."""
from __future__ import annotations
import sqlite3
from contextlib import contextmanager
from pathlib import Path
from typing import Iterator
from src.config.accessor import get_env_config
DEFAULT_DB_PATH = Path.home() / ".vibe-trading" / "br-data" / "br_data.db"
_SCHEMA = """
CREATE TABLE IF NOT EXISTS b3_quotes (
trade_date TEXT NOT NULL, -- YYYY-MM-DD
ticker TEXT NOT NULL, -- CODNEG, e.g. PETR4, PETR4F, PETRJ300
bdi TEXT, -- CODBDI
market_type INTEGER, -- TPMERC: 10 cash, 20 odd lot, 70/80 options ...
short_name TEXT, -- NOMRES
spec TEXT, -- ESPECI (ON, PN, CI, DRN ...)
open REAL,
high REAL,
low REAL,
avg REAL,
close REAL,
best_bid REAL,
best_ask REAL,
trades INTEGER,
volume REAL, -- QUATOT, number of securities traded
turnover REAL, -- VOLTOT, BRL
strike REAL,
expiry TEXT,
quote_factor INTEGER,
isin TEXT,
PRIMARY KEY (ticker, trade_date, market_type)
) WITHOUT ROWID;
CREATE INDEX IF NOT EXISTS ix_b3_quotes_date ON b3_quotes(trade_date);
CREATE TABLE IF NOT EXISTS cvm_fund_daily (
cnpj TEXT NOT NULL, -- digits only
trade_date TEXT NOT NULL,
fund_type TEXT,
subclass TEXT NOT NULL DEFAULT '',
total_value REAL,
quota REAL,
net_assets REAL,
inflows REAL,
outflows REAL,
holders INTEGER,
PRIMARY KEY (cnpj, subclass, trade_date)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS cvm_funds (
cnpj TEXT PRIMARY KEY,
name TEXT,
status TEXT,
fund_class TEXT,
fund_type TEXT,
manager TEXT,
admin TEXT,
benchmark TEXT,
start_date TEXT
);
CREATE TABLE IF NOT EXISTS cvm_companies (
cnpj TEXT PRIMARY KEY,
cvm_code TEXT,
name TEXT,
trade_name TEXT,
status TEXT,
sector TEXT
);
CREATE TABLE IF NOT EXISTS cvm_company_tickers (
ticker TEXT PRIMARY KEY,
cnpj TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS cvm_statements (
cnpj TEXT NOT NULL,
doc TEXT NOT NULL, -- DFP (annual) or ITR (quarterly)
statement TEXT NOT NULL, -- BPA, BPP, DRE, DFC_MD, DFC_MI, DVA, DRA, DMPL
period_end TEXT NOT NULL,
period_start TEXT NOT NULL DEFAULT '',
account TEXT NOT NULL, -- CD_CONTA, e.g. 3.01
description TEXT,
value REAL, -- in BRL units (ESCALA_MOEDA applied)
version INTEGER,
PRIMARY KEY (cnpj, doc, statement, period_end, period_start, account)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS bcb_series (
code INTEGER NOT NULL,
obs_date TEXT NOT NULL,
value REAL,
PRIMARY KEY (code, obs_date)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS tesouro_prices (
bond TEXT NOT NULL, -- e.g. "Tesouro IPCA+"
maturity TEXT NOT NULL,
base_date TEXT NOT NULL,
buy_rate REAL,
sell_rate REAL,
buy_price REAL,
sell_price REAL,
base_price REAL,
PRIMARY KEY (bond, maturity, base_date)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS sync_log (
source TEXT NOT NULL,
item TEXT NOT NULL,
done_at TEXT NOT NULL,
rows INTEGER,
PRIMARY KEY (source, item)
);
"""
def db_path() -> Path:
"""Return the configured SQLite path (``VIBE_BR_DATA_DB`` or the default)."""
configured = get_env_config().data.vibe_br_data_db.strip()
return Path(configured).expanduser() if configured else DEFAULT_DB_PATH
def init_db(path: Path | None = None) -> Path:
"""Create the database file and tables when missing; return its path."""
target = path or db_path()
target.parent.mkdir(parents=True, exist_ok=True)
with sqlite3.connect(target) as conn:
conn.execute("PRAGMA journal_mode=WAL")
conn.executescript(_SCHEMA)
return target
@contextmanager
def connect(path: Path | None = None, *, write: bool = False) -> Iterator[sqlite3.Connection]:
"""Open the store. Read-only connections never create the file.
Raises:
FileNotFoundError: On a read-only open when no sync has run yet.
"""
target = path or db_path()
if write:
init_db(target)
conn = sqlite3.connect(target, timeout=60)
else:
if not target.exists():
raise FileNotFoundError(
f"Brazilian data store not found at {target}; run "
"`python -m src.br_data sync` first"
)
conn = sqlite3.connect(f"file:{target}?mode=ro", uri=True, timeout=60)
conn.row_factory = sqlite3.Row
try:
yield conn
if write:
conn.commit()
finally:
conn.close()
def mark_done(conn: sqlite3.Connection, source: str, item: str, rows: int) -> None:
"""Record that ``source/item`` was fully ingested."""
from datetime import datetime, timezone
conn.execute(
"INSERT OR REPLACE INTO sync_log(source, item, done_at, rows) VALUES (?, ?, ?, ?)",
(source, item, datetime.now(timezone.utc).isoformat(timespec="seconds"), rows),
)
def is_done(conn: sqlite3.Connection, source: str, item: str) -> bool:
"""Whether ``source/item`` was already ingested."""
row = conn.execute(
"SELECT 1 FROM sync_log WHERE source = ? AND item = ?", (source, item)
).fetchone()
return row is not None
agent/src/br_data/sync.py
+70
View File
@@ -0,0 +1,70 @@
"""Run every Brazilian dataset sync, once or as a daily daemon."""
from __future__ import annotations
import logging
import time
from datetime import date, datetime, timedelta
from typing import Callable, Iterable
from zoneinfo import ZoneInfo
from src.br_data import bcb, cotahist, cvm_companies, cvm_funds, tesouro
from src.br_data.store import connect, init_db
from src.config.accessor import get_env_config
logger = logging.getLogger(__name__)
TZ = ZoneInfo("America/Sao_Paulo")
SOURCES = ("cotahist", "cvm_funds", "cvm_companies", "bcb", "tesouro")
def _today() -> date:
return datetime.now(TZ).date()
def run_sync(only: Iterable[str] | None = None, *, cotahist_from: int | None = None,
cvm_from: int | None = None) -> dict[str, object]:
"""Sync the selected sources; a failing source is reported, not fatal."""
data_cfg = get_env_config().data
selected = [s for s in SOURCES if not only or s in set(only)]
today = _today()
cot_year = cotahist_from or data_cfg.vibe_br_cotahist_start_year
cvm_year = cvm_from or data_cfg.vibe_br_cvm_start_year
jobs: dict[str, Callable] = {
"cotahist": lambda c: cotahist.sync(c, cot_year, today),
"cvm_funds": lambda c: cvm_funds.sync_register(c) + cvm_funds.sync_daily(c, cvm_year, today),
"cvm_companies": lambda c: cvm_companies.sync(c, cvm_year, today),
"bcb": lambda c: bcb.sync(c, min(cot_year, cvm_year), today),
"tesouro": lambda c: tesouro.sync(c),
}
path = init_db()
report: dict[str, object] = {"db": str(path)}
for name in selected:
started = time.monotonic()
try:
with connect(path, write=True) as conn:
rows = jobs[name](conn)
report[name] = {"rows": rows, "seconds": round(time.monotonic() - started, 1)}
logger.info("br-data: %s ok (%s rows)", name, rows)
except Exception as exc: # noqa: BLE001 - keep the other sources going
report[name] = {"error": f"{type(exc).__name__}: {exc}"}
logger.warning("br-data: %s failed: %s", name, exc)
return report
def _next_run(now: datetime, hhmm: str) -> datetime:
hour, minute = (int(part) for part in hhmm.split(":", 1))
target = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
if target <= now:
target += timedelta(days=1)
return target
def run_daemon(only: Iterable[str] | None = None) -> None:
"""Sync now, then every day at ``VIBE_BR_SYNC_TIME`` (São Paulo time)."""
only = list(only or [])
while True:
logger.info("br-data: sync report %s", run_sync(only))
target = _next_run(datetime.now(TZ), get_env_config().data.vibe_br_sync_time)
logger.info("br-data: next sync at %s", target.isoformat())
time.sleep(max((target - datetime.now(TZ)).total_seconds(), 60))
agent/src/br_data/tesouro.py
+55
View File
@@ -0,0 +1,55 @@
"""Tesouro Direto prices and yields (Tesouro Transparente open data).
One ``;``-separated CSV with the whole history, Brazilian number format:
``Tipo Titulo;Data Vencimento;Data Base;Taxa Compra Manha;Taxa Venda Manha;
PU Compra Manha;PU Venda Manha;PU Base Manha``.
"""
from __future__ import annotations
import csv
import io
import sqlite3
from typing import Iterator
from src.br_data._download import br_date, decode_text, fetch_bytes, to_float
HOST_KEY = "tesouro"
URL = (
"https://www.tesourotransparente.gov.br/ckan/dataset/"
"df56aa42-484a-4a59-8184-7676580c81e3/resource/"
"796d2059-14e9-44e3-80c9-2d9e30b405c1/download/PrecoTaxaTesouroDireto.csv"
)
def parse(text: str) -> Iterator[tuple]:
"""Yield ``tesouro_prices`` tuples from the published CSV."""
reader = csv.DictReader(io.StringIO(text), delimiter=";")
for row in reader:
row = {str(k or "").strip().lower(): (v or "").strip() for k, v in row.items()}
bond = row.get("tipo titulo", "")
maturity = br_date(row.get("data vencimento"))
base = br_date(row.get("data base"))
if not bond or not maturity or not base:
continue
yield (
bond,
maturity,
base,
to_float(row.get("taxa compra manha")),
to_float(row.get("taxa venda manha")),
to_float(row.get("pu compra manha")),
to_float(row.get("pu venda manha")),
to_float(row.get("pu base manha")),
)
def sync(conn: sqlite3.Connection) -> int:
"""Download the full history (a few MB) and upsert it."""
payload = fetch_bytes(URL, HOST_KEY)
if payload is None:
return 0
rows = list(parse(decode_text(payload)))
conn.executemany("INSERT OR REPLACE INTO tesouro_prices VALUES (?, ?, ?, ?, ?, ?, ?, ?)", rows)
conn.commit()
return len(rows)
agent/src/channels/attempt_progress.py
+78
View File
@@ -0,0 +1,78 @@
"""Forward attempt status to IM adapters without releasing unchecked answers."""
from __future__ import annotations
from typing import Any
from src.channels.bus.events import InboundMessage, OutboundMessage
class AttemptProgress:
"""Poll buffered session events with an attempt-scoped replay cursor."""
def __init__(self, bus: Any, event_bus: Any, msg: InboundMessage,
session_id: str, attempt_id: str) -> None:
self.bus = bus
self.events = event_bus
self.msg = msg
self.session_id = session_id
self.attempt_id = attempt_id
self.cursor: str | None = None
self.streaming = bool(msg.metadata.get("_wants_stream"))
self.stream_started = False
self.reasoning_started = False
self.last_status = ""
async def _send(self, content: str, **flags: Any) -> None:
await self.bus.publish_outbound(OutboundMessage(
channel=self.msg.channel, chat_id=self.msg.chat_id, content=content,
metadata={**self.msg.metadata, "_channel_runtime": True,
"session_id": self.session_id, "attempt_id": self.attempt_id,
"_stream_id": self.attempt_id, **flags},
))
async def poll(self) -> None:
"""Forward status and opt-in reasoning, excluding answer drafts/results."""
for event in self.events.replay(self.session_id, self.cursor, replay_all=True):
self.cursor = event.event_id
data = event.data
if data.get("attempt_id") != self.attempt_id:
continue
kind = event.event_type
if kind == "reasoning_delta":
delta = data.get("delta", "")
if delta:
self.reasoning_started = True
await self._send(str(delta), _reasoning_delta=True)
continue
if kind in {"tool_call", "grounding_status", "attempt.completed", "attempt.failed"}:
await self._end_reasoning()
tool = str(data.get("tool") or "tool")
status = {
"attempt.created": "Working on your request…",
"tool_call": f"Using {tool}…",
"tool_heartbeat": f"Still running {tool}…",
"grounding_status": "Checking the figures…",
}.get(kind)
# Arbitrary progress payloads, tool results and text_delta may
# contain unvalidated figures. Only these status labels go out.
if not status or status == self.last_status:
continue
self.last_status = status
hint = kind in {"tool_call", "tool_heartbeat"}
flags = {"_progress": True, "_tool_hint": hint}
if self.streaming:
flags["_stream_delta"] = True
self.stream_started = True
await self._send(status + "\n", **flags)
async def _end_reasoning(self) -> None:
if self.reasoning_started:
await self._send("", _reasoning_end=True)
self.reasoning_started = False
async def close(self) -> None:
"""Close transient cards on completion, timeout or cancellation."""
await self._end_reasoning()
if self.stream_started:
await self._send("", _stream_end=True, _resuming=True)
self.stream_started = False
agent/src/channels/registry.py
+1
View File
@@ -21,6 +21,7 @@ logger = logging.getLogger(__name__)
_INTERNAL = frozenset( _INTERNAL = frozenset(
{ {
"attempt_progress",
"base", "base",
"bus", "bus",
"config", "config",
agent/src/channels/runtime.py
+14 -2
View File
@@ -192,7 +192,16 @@ class ChannelRuntime:
include_shell_tools=False, include_shell_tools=False,
) )
attempt_id = result.get("attempt_id") if isinstance(result, dict) else None attempt_id = result.get("attempt_id") if isinstance(result, dict) else None
reply = await self._wait_for_reply(session_id, attempt_id) from src.channels.attempt_progress import AttemptProgress
events = getattr(self.session_service, "event_bus", None)
progress = (AttemptProgress(self.bus, events, msg, session_id, attempt_id)
if events is not None and attempt_id else None)
try:
reply = await self._wait_for_reply(session_id, attempt_id, progress=progress)
finally:
if progress is not None:
await progress.close()
reply_content = reply.content reply_content = reply.content
try: try:
from src.scheduled_research.proposals import latest_pending_for_session from src.scheduled_research.proposals import latest_pending_for_session
@@ -233,6 +242,7 @@ class ChannelRuntime:
"_channel_runtime": True, "_channel_runtime": True,
"attempt_id": attempt_id, "attempt_id": attempt_id,
"session_id": session_id, "session_id": session_id,
**{key: msg.metadata[key] for key in ("chat_type", "root_id") if key in msg.metadata},
# QQ (and other platforms) need the originating message id # QQ (and other platforms) need the originating message id
# to reply as a passive message; without it, replies are # to reply as a passive message; without it, replies are
# treated as active messages and rejected for # treated as active messages and rejected for
@@ -355,10 +365,12 @@ class ChannelRuntime:
media.append(str(path)) media.append(str(path))
return media return media
async def _wait_for_reply(self, session_id: str, attempt_id: str | None) -> Message: async def _wait_for_reply(self, session_id: str, attempt_id: str | None, *, progress: Any = None) -> Message:
deadline = time.monotonic() + self.config.reply_timeout_s deadline = time.monotonic() + self.config.reply_timeout_s
last_assistant: Message | None = None last_assistant: Message | None = None
while time.monotonic() < deadline: while time.monotonic() < deadline:
if progress is not None:
await progress.poll()
messages = self.session_service.get_messages(session_id, limit=200) messages = self.session_service.get_messages(session_id, limit=200)
for message in reversed(messages): for message in reversed(messages):
if message.role != "assistant": if message.role != "assistant":
agent/src/config/env_schema.py
+11
View File
@@ -232,6 +232,16 @@ class DataConfig(_EnvBase):
vibe_trading_sec_ftd_files: int = Field(alias="VIBE_TRADING_SEC_FTD_FILES", default=1) vibe_trading_sec_ftd_files: int = Field(alias="VIBE_TRADING_SEC_FTD_FILES", default=1)
vibe_trading_openalex_mailto: str = Field(alias="VIBE_TRADING_OPENALEX_MAILTO", default="") vibe_trading_openalex_mailto: str = Field(alias="VIBE_TRADING_OPENALEX_MAILTO", default="")
vibe_tw_stock_db: str = Field(alias="VIBE_TW_STOCK_DB", default="") vibe_tw_stock_db: str = Field(alias="VIBE_TW_STOCK_DB", default="")
# Brazilian official data (B3 COTAHIST, CVM, BCB, Tesouro Direto), synced
# by ``vibe-trading br-data sync`` into one SQLite file. Empty = default
# ``~/.vibe-trading/br-data/br_data.db``.
vibe_br_data_db: str = Field(alias="VIBE_BR_DATA_DB", default="")
# First COTAHIST year a sync downloads (B3 publishes from 1986).
vibe_br_cotahist_start_year: int = Field(alias="VIBE_BR_COTAHIST_START_YEAR", default=2015)
# First year of CVM fund daily reports / company filings a sync downloads.
vibe_br_cvm_start_year: int = Field(alias="VIBE_BR_CVM_START_YEAR", default=2022)
# Local time (HH:MM, America/Sao_Paulo) of the daily sync in daemon mode.
vibe_br_sync_time: str = Field(alias="VIBE_BR_SYNC_TIME", default="21:30")
vibe_trading_data_cache: EnvBool = Field(alias="VIBE_TRADING_DATA_CACHE", default=False) vibe_trading_data_cache: EnvBool = Field(alias="VIBE_TRADING_DATA_CACHE", default=False)
vibe_trading_data_cache_root: str = Field(alias="VIBE_TRADING_DATA_CACHE_ROOT", default="") vibe_trading_data_cache_root: str = Field(alias="VIBE_TRADING_DATA_CACHE_ROOT", default="")
aliyun_iqs_api_key: str = Field(alias="ALIYUN_IQS_API_KEY", default="") aliyun_iqs_api_key: str = Field(alias="ALIYUN_IQS_API_KEY", default="")
@@ -259,6 +269,7 @@ class DataConfig(_EnvBase):
market_data_order_kr_equity: str = Field(alias="MARKET_DATA_ORDER_KR_EQUITY", default="") market_data_order_kr_equity: str = Field(alias="MARKET_DATA_ORDER_KR_EQUITY", default="")
market_data_order_ca_equity: str = Field(alias="MARKET_DATA_ORDER_CA_EQUITY", default="") market_data_order_ca_equity: str = Field(alias="MARKET_DATA_ORDER_CA_EQUITY", default="")
market_data_order_ar_equity: str = Field(alias="MARKET_DATA_ORDER_AR_EQUITY", default="") market_data_order_ar_equity: str = Field(alias="MARKET_DATA_ORDER_AR_EQUITY", default="")
market_data_order_br_equity: str = Field(alias="MARKET_DATA_ORDER_BR_EQUITY", default="")
market_data_order_uk_equity: str = Field(alias="MARKET_DATA_ORDER_UK_EQUITY", default="") market_data_order_uk_equity: str = Field(alias="MARKET_DATA_ORDER_UK_EQUITY", default="")
market_data_order_vietnam_equity: str = Field(alias="MARKET_DATA_ORDER_VIETNAM_EQUITY", default="") market_data_order_vietnam_equity: str = Field(alias="MARKET_DATA_ORDER_VIETNAM_EQUITY", default="")
market_data_order_crypto: str = Field(alias="MARKET_DATA_ORDER_CRYPTO", default="") market_data_order_crypto: str = Field(alias="MARKET_DATA_ORDER_CRYPTO", default="")
agent/src/factors/registry.py
+8 -4
View File
@@ -74,13 +74,16 @@ def validate_columns_required(cols: list[str]) -> None:
Raises: Raises:
ValueError: If a column is neither a known price column nor a ValueError: If a column is neither a known price column nor a
``fund:``-prefixed fundamental column. ``fund:``-prefixed fundamental or ``val:``-prefixed valuation
column.
""" """
for column in cols: for column in cols:
if column in _PRICE_COLS: if column in _PRICE_COLS:
continue continue
if column.startswith("fund:"): if column.startswith("fund:"):
continue continue
if column.startswith("val:"):
continue
raise ValueError(f"unknown panel column: {column}") raise ValueError(f"unknown panel column: {column}")
@@ -108,8 +111,9 @@ class AlphaMeta(BaseModel):
"""Validate factor panel dependencies. """Validate factor panel dependencies.
Price columns are fixed, while fundamental fields are an open namespace Price columns are fixed, while fundamental fields are an open namespace
under the ``fund:`` prefix. The runtime loader decides whether a under the ``fund:`` prefix and daily valuation fields are an open
particular fundamental field can be populated. namespace under the ``val:`` prefix. The runtime loader decides whether
a particular fundamental or valuation field can be populated.
Args: Args:
v: Declared panel columns. v: Declared panel columns.
@@ -119,7 +123,7 @@ class AlphaMeta(BaseModel):
Raises: Raises:
ValueError: If any column is outside the price set and the ValueError: If any column is outside the price set and the
``fund:`` namespace. ``fund:`` and ``val:`` namespaces.
""" """
validate_columns_required(v) validate_columns_required(v)
return v return v
agent/src/factors/zoo/fundamental/accruals.py
+30
View File
@@ -0,0 +1,30 @@
"""Fundamental accruals quality factor (Sloan 1996)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_accruals",
"nickname": "Accruals - inverse (NI-CFO)/TA, Sloan 1996",
"theme": ["quality"],
"formula_latex": r"-\mathrm{zscore}_{x}\left(\frac{\mathrm{NI}-\mathrm{CFO}}{\mathrm{TA}}\right)",
"columns_required": ["fund:accruals"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Balance-sheet accrual proxy: high accruals indicate low earnings "
"quality; sign inverted so high score = low accruals. PIT-safe from "
"the fundamental panel; missing statements remain NaN and are "
"excluded from that date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored inverse accruals."""
return -zscore(panel["fund:accruals"])
agent/src/factors/zoo/fundamental/book_to_market.py
+46
View File
@@ -0,0 +1,46 @@
"""Fundamental book-to-market value factor (real Fama-French HML characteristic)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import safe_div
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_book_to_market",
"nickname": "Book-to-market - equity over market cap (FF HML characteristic)",
"theme": ["value"],
"formula_latex": (
r"\mathrm{zscore}_{x}\left("
r"\frac{\mathrm{total\_equity}}{\mathrm{close}\times\mathrm{shares\_diluted}}"
r"\right)"
),
"columns_required": ["close", "fund:total_equity", "fund:shares_diluted"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Valuation proxy: close times weighted-average diluted shares is not "
"reported market capitalization; adjusted closes can change the level. "
"Fama-French (1993) HML-style book-to-market proxy built from "
"PIT-safe total equity and shares diluted aligned to the daily close "
"panel; the academic_hml factor in the academic zoo is only a price "
"proxy. High B/M (cheap relative to book value) scores high. "
"market_cap = close x shares_diluted is computed here in the factor "
"file by design: _fundamental_schema.py lines 210-212 deliberately "
"keep price-dependent value factors such as book_to_market out of "
"DERIVED_FIELDS because they need the aligned price panel. Zero or "
"non-finite market cap becomes NaN via safe_div before cross-sectional "
"z-scoring; missing statements remain NaN and are excluded from that "
"date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored book-to-market ratio."""
market_cap = panel["close"] * panel["fund:shares_diluted"]
book_to_market = safe_div(panel["fund:total_equity"], market_cap)
return zscore(book_to_market)
agent/src/factors/zoo/fundamental/cash_to_assets.py
+33
View File
@@ -0,0 +1,33 @@
"""Fundamental cash-to-assets balance-sheet strength factor."""
from __future__ import annotations
import pandas as pd
from src.factors.base import safe_div
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_cash_to_assets",
"nickname": "Cash to assets - balance-sheet cash ratio",
"theme": ["quality"],
"formula_latex": r"\mathrm{zscore}_{x}\left(\frac{\mathrm{cash}}{\mathrm{total\_assets}}\right)",
"columns_required": ["fund:cash", "fund:total_assets"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Balance-sheet cash ratio (cash and equivalents / total assets): "
"high cash holdings signal defensive capacity and financial "
"flexibility. Zero or missing total assets become NaN via safe "
"division before cross-sectional z-scoring; PIT-safe from the "
"filed-date-anchored fundamental panel."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored cash-to-assets ratio."""
cash_ratio = safe_div(panel["fund:cash"], panel["fund:total_assets"])
return zscore(cash_ratio)
agent/src/factors/zoo/fundamental/cfo_yield.py
+34
View File
@@ -0,0 +1,34 @@
"""Fundamental cash-flow yield quality factor (CFO / total assets)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import safe_div
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_cfo_yield",
"nickname": "CFO yield - operating cash flow over total assets",
"theme": ["quality"],
"formula_latex": r"\mathrm{zscore}_{x}\left(\frac{\mathrm{CFO}}{\mathrm{total\_assets}}\right)",
"columns_required": ["fund:cfo", "fund:total_assets"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Valuation proxy: close times weighted-average diluted shares is not "
"reported market capitalization; adjusted closes can change the level. "
"Cash-flow return on assets: the cash-backed counterpart of ROA, so "
"accrual-heavy earnings score lower here. Zero or missing total "
"assets become NaN via safe division before cross-sectional "
"z-scoring; PIT-safe from the filed-date-anchored fundamental panel."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored CFO-to-assets yield."""
cfo_yield = safe_div(panel["fund:cfo"], panel["fund:total_assets"])
return zscore(cfo_yield)
agent/src/factors/zoo/fundamental/dividend_yield.py
+34
View File
@@ -0,0 +1,34 @@
"""Fundamental dividend-yield (TTM) value factor (CN daily market data)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_dividend_yield",
"nickname": "Dividend yield TTM - dv_ttm (daily market data)",
"theme": ["value"],
"formula_latex": r"\mathrm{zscore}_{x}(\mathrm{DY_{TTM}})",
"columns_required": ["val:dv_ttm"],
"universe": ["equity_cn"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Trailing dividend yield from the val: namespace (tushare "
"daily_basic dv_ttm, CN A-share only), reported in percent units — "
"cross-sectional z-scoring is scale-invariant, so no unit "
"conversion is applied. High dividend yield scores high. Keyed by "
"trade_date, so PIT-safe by construction with no filing lag. "
"Missing dates or non-CN symbols stay NaN and are excluded from "
"that date's cross-section. Lineage: JoinQuant style-pro divyild; "
"akshare/Eastmoney dv_ttm convention."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored trailing dividend yield."""
return zscore(panel["val:dv_ttm"])
agent/src/factors/zoo/fundamental/earnings_growth.py
+40
View File
@@ -0,0 +1,40 @@
"""Fundamental earnings-growth factor (sign-safe YoY net-income growth)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_earnings_growth",
"nickname": "Earnings growth - sign-safe YoY net income expansion",
"theme": ["growth"],
"formula_latex": (
r"\mathrm{zscore}_{x}\left("
r"\frac{\Delta\,\mathrm{net\_income}}"
r"{\left|\mathrm{lag}_{365d}(\mathrm{net\_income})\right|}"
r"\right)"
),
"columns_required": ["fund:earnings_growth"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"PIT-safe earnings growth from the fundamental panel's derived field "
"``earnings_growth``. The schema-side formula is sign-safe: "
"delta net_income / abs(lag_365d net_income), so a negative year-ago "
"base does not flip the growth sign (the proposal #6 footnote "
"resolution; the alternative NaN-on-negative-base was not chosen). "
"Calendar-day lag per the #1564 lesson. High earnings growth scores "
"high, matching the JoinQuant net_profit_growth_rate convention. "
"Missing statements or a zero/missing year-ago base remain NaN and "
"are excluded from that date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored earnings growth."""
return zscore(panel["fund:earnings_growth"])
agent/src/factors/zoo/fundamental/fcf_yield.py
+46
View File
@@ -0,0 +1,46 @@
"""Fundamental free-cash-flow yield value factor (Lakonishok et al. 1994)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import safe_div
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_fcf_yield",
"nickname": "FCF yield - (CFO - capex) over market cap",
"theme": ["value"],
"formula_latex": (
r"\mathrm{zscore}_{x}\left("
r"\frac{\mathrm{CFO}-\mathrm{capex}}"
r"{\mathrm{close}\times\mathrm{shares\_diluted}}"
r"\right)"
),
"columns_required": ["close", "fund:cfo", "fund:capex", "fund:shares_diluted"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Valuation proxy: close times weighted-average diluted shares is not "
"reported market capitalization; adjusted closes can change the level. "
"PIT-safe free-cash-flow yield: FCF = CFO - capex, scaled by market "
"cap (close x shares_diluted) computed in the factor file per the "
"_fundamental_schema.py price-dependent convention. Sign assumption: "
"capex is reported as a positive magnitude (SEC concept "
"PaymentsToAcquirePropertyPlantAndEquipment), so subtracting it from "
"CFO yields FCF directly. High FCF yield (cheap on cash generation, "
"cf. Lakonishok-Shleifer-Vishny 1994 cash-flow yield value factors) "
"scores high. Zero/non-finite market cap becomes NaN via safe_div "
"before cross-sectional z-scoring; missing statements remain NaN and "
"are excluded from that date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored free-cash-flow yield."""
fcf = panel["fund:cfo"] - panel["fund:capex"]
market_cap = panel["close"] * panel["fund:shares_diluted"]
return zscore(safe_div(fcf, market_cap))
agent/src/factors/zoo/fundamental/leverage.py
+31
View File
@@ -0,0 +1,31 @@
"""Fundamental leverage factor (low leverage scores high)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_leverage",
"nickname": "Leverage - inverse debt/equity (low leverage scores high)",
"theme": ["leverage"],
"formula_latex": r"-\mathrm{zscore}_{x}\left(\frac{\mathrm{total\_debt}}{\mathrm{total\_equity}}\right)",
"columns_required": ["fund:leverage"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"PIT-safe market leverage (total debt / total equity) from the "
"fundamental panel; sign inverted so low leverage scores high — "
"heavily levered firms carry distress risk and lower expected "
"returns. Zero-equity or missing statements already yield NaN in the "
"derived field and are excluded from that date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return negative cross-sectional z-scored leverage."""
return -zscore(panel["fund:leverage"])
agent/src/factors/zoo/fundamental/operating_margin.py
+33
View File
@@ -0,0 +1,33 @@
"""Fundamental operating-margin quality factor."""
from __future__ import annotations
import pandas as pd
from src.factors.base import safe_div
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_operating_margin",
"nickname": "Operating margin - operating income over revenue",
"theme": ["quality"],
"formula_latex": r"\mathrm{zscore}_{x}\left(\frac{\mathrm{operating\_income}}{\mathrm{revenue}}\right)",
"columns_required": ["fund:operating_income", "fund:revenue"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Operating profitability (operating income / revenue), "
"cross-sectionally z-scored per date. Captures pricing power and cost "
"efficiency before financing and tax effects. Zero-revenue or missing "
"statements become NaN via safe division and are excluded from that "
"date's cross-section; PIT-safe from the fundamental panel."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored operating margin."""
margin = safe_div(panel["fund:operating_income"], panel["fund:revenue"])
return zscore(margin)
agent/src/factors/zoo/fundamental/pb.py
+35
View File
@@ -0,0 +1,35 @@
"""Fundamental price-to-book value factor (CN daily market data)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_pb",
"nickname": "P/B - inverse price-to-book (daily market data)",
"theme": ["value"],
"formula_latex": r"-\mathrm{zscore}_{x}(\mathrm{PB})",
"columns_required": ["val:pb"],
"universe": ["equity_cn"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Nonpositive multiples are excluded rather than ranked as cheap. "
"Inverse price-to-book from the val: namespace (tushare daily_basic "
"pb, CN A-share only): low PB (cheap relative to book value) scores "
"high, the daily-market-data counterpart of fund_book_to_market. "
"Keyed by trade_date, so PIT-safe by construction with no filing "
"lag. Missing dates or non-CN symbols stay NaN and are excluded "
"from that date's cross-section. Lineage: JoinQuant style "
"book_to_price_ratio; akshare/Eastmoney pb convention."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored inverse price-to-book."""
values = panel["val:pb"]
return -zscore(values.where(values > 0))
agent/src/factors/zoo/fundamental/ps_ttm.py
+35
View File
@@ -0,0 +1,35 @@
"""Fundamental price-to-sales (TTM) value factor (CN daily market data)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_ps_ttm",
"nickname": "P/S TTM - inverse price-to-sales (daily market data)",
"theme": ["value"],
"formula_latex": r"-\mathrm{zscore}_{x}(\mathrm{PS_{TTM}})",
"columns_required": ["val:ps_ttm"],
"universe": ["equity_cn"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"Nonpositive multiples are excluded rather than ranked as cheap. "
"Inverse trailing price-to-sales from the val: namespace (tushare "
"daily_basic ps_ttm, CN A-share only): low PS scores high. PS stays "
"informative for loss-making names where earnings yield fails. "
"Keyed by trade_date, so PIT-safe by construction with no filing "
"lag. Missing dates or non-CN symbols stay NaN and are excluded "
"from that date's cross-section. Lineage: JoinQuant style-pro "
"sales-to-price; akshare/Eastmoney ps_ttm convention."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored inverse price-to-sales (TTM)."""
values = panel["val:ps_ttm"]
return -zscore(values.where(values > 0))
agent/src/factors/zoo/fundamental/revenue_growth.py
+37
View File
@@ -0,0 +1,37 @@
"""Fundamental revenue-growth factor (YoY top-line growth)."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_revenue_growth",
"nickname": "Revenue growth - YoY top-line expansion",
"theme": ["growth"],
"formula_latex": (
r"\mathrm{zscore}_{x}\left("
r"\frac{\mathrm{revenue}}{\mathrm{lag}_{365d}(\mathrm{revenue})}-1"
r"\right)"
),
"columns_required": ["fund:revenue_growth"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"PIT-safe revenue growth from the fundamental panel's derived field "
"``revenue_growth`` (revenue / lag_365d(revenue) - 1, calendar-day "
"lag per the #1564 lesson rather than a row-wise shift). High "
"top-line growth scores high, matching the JoinQuant "
"operating_revenue_growth_rate convention. Missing statements or a "
"missing year-ago base remain NaN and are excluded from that date's "
"cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored revenue growth."""
return zscore(panel["fund:revenue_growth"])
agent/src/factors/zoo/fundamental/roa.py
+30
View File
@@ -0,0 +1,30 @@
"""Fundamental ROA quality factor."""
from __future__ import annotations
import pandas as pd
from src.factors.base import zscore
__alpha_meta__ = {
"id": "fund_roa",
"nickname": "ROA - return on assets (PIT-safe fundamentals)",
"theme": ["quality"],
"formula_latex": r"\mathrm{zscore}_{x}\left(\frac{\mathrm{net\_income}}{\mathrm{total\_assets}}\right)",
"columns_required": ["fund:roa"],
"universe": ["equity_us", "equity_cn", "equity_hk"],
"frequency": ["1d"],
"decay_horizon": 252,
"min_warmup_bars": 1,
"notes": (
"PIT-safe return on assets (net income / total assets) from the "
"fundamental panel; cross-sectional z-score per date. Higher ROA means "
"a more profitable asset base and scores higher. Missing statements "
"remain NaN and are excluded from that date's cross-section."
),
}
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return cross-sectional z-scored ROA."""
return zscore(panel["fund:roa"])
agent/src/live/enforcement.py
+12 -2
View File
@@ -537,8 +537,18 @@ def check_mandate(
detail=f"{intent.instrument_type.value} not in allowed_instruments", detail=f"{intent.instrument_type.value} not in allowed_instruments",
) )
# 3. Asset-class allowance (universe bucket). OPTION has no bucket and is # Share-notional checks cannot bound an option's contract multiplier or
# governed by allowed_instruments alone. An explicit intent.asset_class # maximum loss. Even an explicit instrument allowance cannot enable this
# unsupported path; both MCP and SDK orders pass through this check.
if intent.instrument_type == InstrumentType.OPTION:
return _breach(
broker=broker, remote_tool=remote_tool, intent=intent,
kind=BREACH_KIND_INSTRUMENT, limit="options_unsupported",
limit_value=0.0, attempted_value=0.0,
detail="options require an options-specific risk gate; order placement is unsupported",
)
# 3. Asset-class allowance (universe bucket). An explicit intent.asset_class
# (multi-market connectors: US/HK/CN equities) wins over the instrument # (multi-market connectors: US/HK/CN equities) wins over the instrument
# default so the gate buckets HK/A-share correctly. # default so the gate buckets HK/A-share correctly.
asset_class = intent.asset_class or _INSTRUMENT_ASSET_CLASS.get(intent.instrument_type) asset_class = intent.asset_class or _INSTRUMENT_ASSET_CLASS.get(intent.instrument_type)
agent/src/live/mandate/model.py
+4 -3
View File
@@ -18,9 +18,10 @@ class InstrumentType(str, Enum):
"""Instrument classes the broker may report or accept. """Instrument classes the broker may report or accept.
``CFD`` covers margin contracts-for-difference that are not spot forex ``CFD`` covers margin contracts-for-difference that are not spot forex
pairs (MT5 metals like XAUUSD, index/energy/crypto CFDs). Like ``OPTION`` pairs (MT5 metals like XAUUSD, index/energy/crypto CFDs). It has no universe
it has no universe asset-class bucket and is admitted only when the user's asset-class bucket and requires ``"cfd"`` in ``allowed_instruments``.
mandate explicitly lists ``"cfd"`` in ``allowed_instruments``. ``OPTION`` is representable for classification, but order enforcement
refuses it until an options-specific risk gate is available.
""" """
EQUITY = "equity" EQUITY = "equity"
agent/src/market_data.py
+2
View File
@@ -30,6 +30,8 @@ _SOURCE_PATTERNS = [
(re.compile(r"^[A-Z0-9&.\-]+\.(TO|V)$", re.I), "yahoo"), (re.compile(r"^[A-Z0-9&.\-]+\.(TO|V)$", re.I), "yahoo"),
# Argentina: BYMA equities and CEDEARs use Yahoo's canonical .BA suffix. # Argentina: BYMA equities and CEDEARs use Yahoo's canonical .BA suffix.
(re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "yahoo"), (re.compile(r"^[A-Z0-9&.\-]+\.BA$", re.I), "yahoo"),
# Brazil: B3 equities, FIIs, ETFs and BDRs use Yahoo's canonical .SA suffix.
(re.compile(r"^[A-Z0-9]+\.SA$", re.I), "yahoo"),
# UK: London Stock Exchange (VOD.L, SHEL.L). Yahoo serves the suffix # UK: London Stock Exchange (VOD.L, SHEL.L). Yahoo serves the suffix
# verbatim; without this they fell through to the tushare default and were # verbatim; without this they fell through to the tushare default and were
# routed to China-market loaders that cannot resolve them. # routed to China-market loaders that cannot resolve them.
agent/src/providers/openai_codex.py
+5
View File
@@ -558,6 +558,11 @@ def _convert_tools(tools: list[dict[str, Any]] | None) -> list[dict[str, Any]]:
"name": name, "name": name,
"description": fn.get("description") or "", "description": fn.get("description") or "",
"parameters": params if isinstance(params, dict) else {}, "parameters": params if isinstance(params, dict) else {},
# The Codex Responses endpoint treats a tool without an explicit
# ``strict`` as strict, which makes the model emit every property
# (``""`` for an optional string) instead of omitting it. Tool
# schemas here are plain JSON Schema with ``required`` honoured.
"strict": False,
} }
) )
return converted return converted
agent/src/quantlib/risk.py
+4 -2
View File
@@ -452,7 +452,8 @@ def max_drawdown_analysis(equity: pd.Series | np.ndarray | Sequence[float]) -> d
max_drawdown (float): Worst decline as a **positive** fraction, so max_drawdown (float): Worst decline as a **positive** fraction, so
0.25 means the curve fell 25% below its running peak. 0.0 when 0.25 means the curve fell 25% below its running peak. 0.0 when
the curve never declines. the curve never declines.
peak_date: Index label of the running peak preceding the trough. peak_date: Index label of the most recent running peak preceding
the trough; revisiting that peak resets the underwater period.
trough_date: Index label of the deepest point. trough_date: Index label of the deepest point.
recovery_date: Index label of the first observation at or above the recovery_date: Index label of the first observation at or above the
peak value, searched from the trough onward; None if the series peak value, searched from the trough onward; None if the series
@@ -491,7 +492,8 @@ def max_drawdown_analysis(equity: pd.Series | np.ndarray | Sequence[float]) -> d
running_peak = np.maximum.accumulate(values) running_peak = np.maximum.accumulate(values)
drawdown = values / running_peak - 1.0 # <= 0 everywhere drawdown = values / running_peak - 1.0 # <= 0 everywhere
trough_pos = int(np.argmin(drawdown)) trough_pos = int(np.argmin(drawdown))
peak_pos = int(np.argmax(values[: trough_pos + 1])) # A revisit closes the previous drawdown: start at the latest peak.
peak_pos = int(np.flatnonzero(values[: trough_pos + 1] == running_peak[trough_pos])[-1])
# A flat/rising curve has drawdown 0 at the trough, so the trough itself # A flat/rising curve has drawdown 0 at the trough, so the trough itself
# already satisfies "back at the peak" and recovery is instantaneous. # already satisfies "back at the peak" and recovery is instantaneous.
agent/src/quantlib/valuation/threestatement.py
+5 -4
View File
@@ -521,6 +521,7 @@ def _project_period(
minimum_cash: float, minimum_cash: float,
circularity_tolerance: float, circularity_tolerance: float,
max_circularity_iterations: int, max_circularity_iterations: int,
balance_tolerance: float = BALANCE_TOLERANCE_REL,
) -> PeriodResult: ) -> PeriodResult:
"""Project one period, solving the interest/revolver circularity by iteration. """Project one period, solving the interest/revolver circularity by iteration.
@@ -650,7 +651,7 @@ def _project_period(
paid_in_capital=prior_bs.paid_in_capital, paid_in_capital=prior_bs.paid_in_capital,
retained_earnings=prior_bs.retained_earnings + net_income - dividends_paid, retained_earnings=prior_bs.retained_earnings + net_income - dividends_paid,
) )
check_balance_sheet(balance_sheet) check_balance_sheet(balance_sheet, balance_tolerance)
return PeriodResult( return PeriodResult(
period=period, period=period,
@@ -751,10 +752,10 @@ def project_three_statement(
minimum_cash=float(drivers["minimum_cash"][index]), minimum_cash=float(drivers["minimum_cash"][index]),
circularity_tolerance=circularity_tolerance, circularity_tolerance=circularity_tolerance,
max_circularity_iterations=max_circularity_iterations, max_circularity_iterations=max_circularity_iterations,
balance_tolerance=balance_tolerance,
) )
# Defensive re-check with the caller's tolerance: _project_period already # Defensive re-check: the inner check already enforces the
# checked with the module default, but a caller-widened balance_tolerance # same tolerance, so this second pass is a safety net for future edits.
# must not be allowed to let a period through unchecked at its own bar.
check_balance_sheet(result.balance_sheet, balance_tolerance) check_balance_sheet(result.balance_sheet, balance_tolerance)
results.append(result) results.append(result)
prior_bs = result.balance_sheet prior_bs = result.balance_sheet
agent/src/skills/risk-analysis/SKILL.md
+1 -1
View File
@@ -110,7 +110,7 @@ dd["recovered"] # False when the series ends still underwater
Full return keys: `max_drawdown`, `peak_date`, `trough_date`, `recovery_date`, `recovered`, `peak_to_trough_periods`, `trough_to_recovery_periods`, `underwater_days`, `recovery_days`. Full return keys: `max_drawdown`, `peak_date`, `trough_date`, `recovery_date`, `recovered`, `peak_to_trough_periods`, `trough_to_recovery_periods`, `underwater_days`, `recovery_days`.
- Recovery means reaching the **peak** value again, not merely bouncing off the trough; `recovery_date` is None and `recovered` is False if it never happens. - Recovery means reaching the **peak** value again, not merely bouncing off the trough; `recovery_date` is None and `recovered` is False if it never happens. Revisiting the peak starts a new episode: its `peak_date` and underwater duration exclude an earlier, already recovered drawdown.
- `underwater_days` / `recovery_days` are calendar days and require a `DatetimeIndex`; on any other index they come back None and you should use the `*_periods` counts, which are always populated. - `underwater_days` / `recovery_days` are calendar days and require a `DatetimeIndex`; on any other index they come back None and you should use the `*_periods` counts, which are always populated.
- Non-positive equity raises — a drawdown *ratio* is undefined at or below zero. Rebase a signed PnL series to a positive net value first. - Non-positive equity raises — a drawdown *ratio* is undefined at or below zero. Rebase a signed PnL series to a positive net value first.
agent/src/strategy_discovery/evidence_harness.py
+3 -4
View File
@@ -4,10 +4,9 @@ Builds on the artifact readers in :mod:`run_artifacts`; this module owns the
regime labeling and the per-regime metrics. Conventions: regime labeling and the per-regime metrics. Conventions:
* Trade counting — engine ``trades.csv`` artifacts hold TWO rows per round * Trade counting — engine ``trades.csv`` artifacts hold TWO rows per round
trip (entry row ``pnl=0.0`` + exit row with realized pnl); only exit rows trip (``event=entry`` and ``event=exit``); only exits are counted, including
are counted, so a round trip contributes exactly one trade. Legacy zero-PnL exits. Unmarked files keep their legacy PnL-based interpretation
one-row-per-trade artifacts are detected via the marker column and keep (see :func:`run_artifacts.read_trade_activity`).
counting every row (see :func:`run_artifacts.read_trade_activity`).
* Regime labeling — each bar is labeled by the rolling cumulative benchmark * Regime labeling — each bar is labeled by the rolling cumulative benchmark
return over the trailing ``benchmark_window`` bars ending at that bar return over the trailing ``benchmark_window`` bars ending at that bar
(``<= bear_threshold`` → bear_market, ``>= bull_threshold`` → bull_market, (``<= bear_threshold`` → bear_market, ``>= bull_threshold`` → bull_market,
agent/src/strategy_discovery/run_artifacts.py
+31 -32
View File
@@ -7,23 +7,18 @@ Real engine schema (``backtest/engines/base.py::_write_artifacts``, the
writer every reproducible run goes through): writer every reproducible run goes through):
* ``trades.csv`` — columns ``timestamp, code, side, price, qty, reason, * ``trades.csv`` — columns ``timestamp, code, side, price, qty, reason,
pnl, holding_days, return_pct`` with TWO rows per round trip: an entry pnl, holding_days, holding_bars, return_pct, event`` with TWO rows per
row (``reason="signal"``, ``pnl=0.0``, ``holding_days=0``) followed by round trip: an entry row (``reason="signal"``, ``pnl=0.0``, ``holding_days=0``) followed by
the exit row carrying the realized ``pnl``. the exit row carrying the realized ``pnl``.
* ``equity.csv`` — date column ``timestamp`` (the engine's index name) * ``equity.csv`` — date column ``timestamp`` (the engine's index name)
plus ``ret, equity, drawdown, benchmark_equity, active_ret``. plus ``ret, equity, drawdown, benchmark_equity, active_ret``.
Entry/exit convention. The engine writes no dedicated marker column, and Entry/exit convention. New BaseEngine artifacts carry an explicit ``event``
``reason`` is not one either — exits can also carry ``reason="signal"`` column (``entry`` / ``exit``), so break-even exits count as completed trades.
(the base engine closes positions on signal). This reader therefore Unknown or blank event values are skipped, never inferred from PnL. Unmarked
mirrors the repo's own artifact-reload convention in files retain the legacy convention: finite nonzero PnL marks an exit, zero
``backtest/validation.py``: a row is a closed-trade (exit) row iff its marks an entry. Without any zero-PnL rows, every dated row counts as a legacy
``pnl`` parses to a nonzero finite value, while ``pnl == 0.0`` rows are closed trade. Break-even exits in unmarked files remain ambiguous.
entry rows. Files with no zero-pnl rows at all are treated as legacy
one-row-per-trade artifacts where every dated row is a meaningful closed
trade. Documented limitation (shared with ``validation.py``): an
exact-break-even exit (``pnl == 0.0``) is indistinguishable from an entry
row and is not counted as a round trip.
Legacy artifacts keep parsing through column aliases (``date`` for Legacy artifacts keep parsing through column aliases (``date`` for
``timestamp``, ``benchmark`` for ``benchmark_equity``). Unusable files — ``timestamp``, ``benchmark`` for ``benchmark_equity``). Unusable files —
@@ -136,14 +131,14 @@ def read_trade_activity(path: Path) -> tuple[list[date], int | None] | None:
Returns ``(exit_dates, max_concurrent)``: Returns ``(exit_dates, max_concurrent)``:
* ``exit_dates`` — one date per closed round trip, sorted. Engine * ``exit_dates`` — one date per closed round trip, sorted. Engine
artifacts hold two rows per round trip (entry row ``pnl=0.0`` plus artifacts hold two rows per round trip (``event=entry`` and
exit row with realized pnl); only exit rows count, so a round trip ``event=exit``); only exit rows count, so a round trip
is counted once, never twice. Legacy one-row-per-trade artifacts is counted once, never twice. Legacy one-row-per-trade artifacts
(no zero-pnl marker rows) count every dated row. (no zero-pnl marker rows) count every dated row.
* ``max_concurrent`` — peak simultaneously open positions derived * ``max_concurrent`` — peak simultaneously open positions derived
from the entry/exit pairs (FIFO per code, matching the engine's from the entry/exit pairs (FIFO per code, matching the engine's
one-position-per-symbol book), or ``None`` when the file lacks the one-position-per-symbol book), or ``None`` when the file lacks the
zero-pnl entry marker and concurrency is undetectable. entry marker and concurrency is undetectable.
Returns ``None`` when the file is unusable: no date column, decode / Returns ``None`` when the file is unusable: no date column, decode /
OS / CSV errors, or no usable content. Rows with unparseable dates OS / CSV errors, or no usable content. Rows with unparseable dates
@@ -165,7 +160,8 @@ def read_trade_activity(path: Path) -> tuple[list[date], int | None] | None:
pnl_column = _find_column(fieldnames, _PNL_ALIASES) pnl_column = _find_column(fieldnames, _PNL_ALIASES)
code_column = _find_column(fieldnames, _CODE_ALIASES) code_column = _find_column(fieldnames, _CODE_ALIASES)
rows: list[tuple[date, str, float | None]] = [] event_column = _find_column(fieldnames, ("event",))
rows: list[tuple[date, str, float | None, str]] = []
for record in reader: for record in reader:
trade_date = parse_iso_date(record.get(date_column)) trade_date = parse_iso_date(record.get(date_column))
if trade_date is None: if trade_date is None:
@@ -180,7 +176,14 @@ def read_trade_activity(path: Path) -> tuple[list[date], int | None] | None:
if code_column is not None if code_column is not None
else "" else ""
) )
rows.append((trade_date, code, pnl)) event = (
str(record.get(event_column) or "")
if event_column is not None
else "entry" if pnl == 0.0 else "exit"
)
if event_column is not None and event not in {"entry", "exit"}:
continue
rows.append((trade_date, code, pnl, event))
except _UNREADABLE_ERRORS as exc: except _UNREADABLE_ERRORS as exc:
logger.warning( logger.warning(
"trades.csv at %s is unreadable (%s); treated as unusable", path, exc "trades.csv at %s is unreadable (%s); treated as unusable", path, exc
@@ -190,32 +193,28 @@ def read_trade_activity(path: Path) -> tuple[list[date], int | None] | None:
if not rows: if not rows:
return [], None return [], None
# The engine writes entry rows with pnl=0.0 and exit rows with realized # Without explicit events, preserve the legacy one-row-per-trade rule.
# pnl (backtest/engines/base.py::_write_artifacts); backtest/validation.py has_entry_marker = any(event == "entry" for _, _, _, event in rows)
# reloads the same artifacts with `pnl != 0` as the exit-row filter. When if event_column is None and not has_entry_marker:
# no marker row exists, every dated row is a closed trade (legacy return sorted(trade_date for trade_date, _, _, _ in rows), None
# one-row-per-trade artifacts) and concurrency is undetectable.
has_entry_marker = any(pnl is not None and pnl == 0.0 for _, _, pnl in rows)
if not has_entry_marker:
return sorted(trade_date for trade_date, _, _ in rows), None
exit_dates: list[date] = [] exit_dates: list[date] = []
open_entries: dict[str, list[date]] = {} open_entries: dict[str, list[date]] = {}
intervals: list[tuple[date, date]] = [] intervals: list[tuple[date, date]] = []
for trade_date, code, pnl in rows: for trade_date, code, pnl, event in rows:
if pnl is None: if event_column is None and pnl is None:
continue # unclassifiable without a usable pnl marker continue # unclassifiable without a usable legacy pnl marker
if pnl != 0.0: if event == "exit":
# Exit row: one closed round trip. # Exit row: one closed round trip.
exit_dates.append(trade_date) exit_dates.append(trade_date)
pending = open_entries.get(code) pending = open_entries.get(code)
if pending: if pending:
intervals.append((pending.pop(0), trade_date)) intervals.append((pending.pop(0), trade_date))
else: else:
# Entry row (engine marker pnl == 0.0). # Explicit entry or legacy zero-PnL entry marker.
open_entries.setdefault(code, []).append(trade_date) open_entries.setdefault(code, []).append(trade_date)
exit_dates.sort() exit_dates.sort()
return exit_dates, _max_concurrent_positions(intervals) return exit_dates, _max_concurrent_positions(intervals) if has_entry_marker else None
def read_trade_dates(path: Path) -> list[date] | None: def read_trade_dates(path: Path) -> list[date] | None:
agent/src/tools/alpha_bench_tool.py
+80 -12
View File
@@ -699,6 +699,64 @@ def _validate_bench_data_requirements(registry: Any, alpha_ids: list[str], unive
return columns return columns
def _val_columns_required(registry: Any, alpha_ids: list[str], panel: dict) -> list[str]:
"""Collect declared ``val:*`` columns missing from the panel.
Args:
registry: Registry holding the selected factors.
alpha_ids: Factors to evaluate.
panel: Observed price panel.
Returns:
Sorted ``val:*`` column names not already present in the panel.
"""
return sorted({column for aid in alpha_ids
for column in (registry.get(aid).meta or {}).get("columns_required", [])
if column.startswith("val:") and column not in panel})
def _attach_valuation_columns(
enriched: dict, panel: dict, val_columns: list[str], start: str, end: str
) -> dict:
"""Attach ``val:*`` columns for a CN universe from tushare ``daily_basic``.
Daily market data keyed by ``trade_date`` — PIT-safe by construction, no
filing lag. Frames are masked to the price panel's own cross-section so a
date whose close was membership-masked carries no valuation either.
Args:
enriched: Panel copy being extended.
panel: Observed price panel (source of the cross-section mask).
val_columns: Declared ``val:*`` columns to attach.
start: Period start (YYYY-MM-DD).
end: Period end (YYYY-MM-DD).
Returns:
The enriched panel.
Raises:
ValueError: If a declared field is outside the daily_basic whitelist.
RuntimeError: If fewer than two stocks have data for a field.
"""
from backtest.loaders.valuation_loader import load_valuation_panel
close = panel["close"]
fields = [column.removeprefix("val:") for column in val_columns]
loaded = load_valuation_panel(
symbols=list(close.columns), fields=fields, start=start, end=end, index=close.index
)
for column, field in zip(val_columns, fields):
frame = loaded.get(field)
if frame is None or frame.notna().any().sum() < 2:
raise RuntimeError(
f"tushare daily_basic field {field!r} has data for fewer than two "
"stocks. Check TUSHARE_TOKEN permissions and CN coverage for this period."
)
enriched[column] = frame.reindex(index=close.index, columns=close.columns).where(close.notna())
enriched["_meta"] = {**enriched.get("_meta", {}), "valuation_source": "tushare_daily_basic"}
return enriched
def _prepare_bench_panel(registry: Any, alpha_ids: list[str], panel: dict, universe: str, period: str) -> dict: def _prepare_bench_panel(registry: Any, alpha_ids: list[str], panel: dict, universe: str, period: str) -> dict:
"""Load only the declared PIT fundamental inputs for the selected factors. """Load only the declared PIT fundamental inputs for the selected factors.
@@ -706,32 +764,42 @@ def _prepare_bench_panel(registry: Any, alpha_ids: list[str], panel: dict, unive
registry: Registry holding the factors being evaluated. registry: Registry holding the factors being evaluated.
alpha_ids: Selected factor identifiers. alpha_ids: Selected factor identifiers.
panel: Observed price panel; never mutated or filled with invented values. panel: Observed price panel; never mutated or filled with invented values.
universe: Stock universe; SEC fundamentals currently cover US stocks. universe: Stock universe; SEC fundamentals currently cover US stocks,
``val:*`` daily valuation columns cover the CN universe only —
other universes skip them and declaring alphas SkipAlpha.
period: Requested historical period. period: Requested historical period.
Returns: Returns:
A panel including the required fundamental columns. A panel including the required fundamental columns.
Raises: Raises:
ValueError: If SEC inputs are requested for a non-US universe. ValueError: If SEC inputs are requested for a non-US universe, or a
``val:`` field is outside the daily_basic whitelist.
RuntimeError: If fewer than two stocks have the required financial inputs. RuntimeError: If fewer than two stocks have the required financial inputs.
""" """
columns = [column for column in _validate_bench_data_requirements(registry, alpha_ids, universe) if column not in panel] columns = [column for column in _validate_bench_data_requirements(registry, alpha_ids, universe) if column not in panel]
if not columns: val_columns = _val_columns_required(registry, alpha_ids, panel) if universe == "csi300" else []
if not columns and not val_columns:
return panel return panel
from backtest.loaders.fundamentals_loader import load_fundamental_panel
start, end = _parse_period(period) start, end = _parse_period(period)
close = panel["close"] close = panel["close"]
fields = [column.removeprefix("fund:") for column in columns]
loaded = load_fundamental_panel(symbols=list(close.columns), fields=fields, start=start, end=end, pit=True, index=close.index)
enriched = dict(panel) enriched = dict(panel)
for column, field in zip(columns, fields): if columns:
frame = loaded.get(field) from backtest.loaders.fundamentals_loader import load_fundamental_panel
if frame is None or frame.notna().any().sum() < 2:
raise RuntimeError(f"SEC field {field!r} has data for fewer than two stocks. Check SEC connectivity and filing coverage for this period.") fields = [column.removeprefix("fund:") for column in columns]
enriched[column] = frame.reindex(index=close.index, columns=close.columns) loaded = load_fundamental_panel(symbols=list(close.columns), fields=fields, start=start, end=end, pit=True, index=close.index)
enriched["_meta"] = {**panel.get("_meta", {}), "fundamental_source": "sec", "fundamental_pit": True} for column, field in zip(columns, fields):
frame = loaded.get(field)
if frame is None or frame.notna().any().sum() < 2:
raise RuntimeError(f"SEC field {field!r} has data for fewer than two stocks. Check SEC connectivity and filing coverage for this period.")
# Date-only SEC filings can arrive after close(t). IC uses them
# on t+1; the engine's separate next-bar execution is unchanged.
enriched[column] = frame.reindex(index=close.index, columns=close.columns).shift(1)
enriched["_meta"] = {**enriched.get("_meta", {}), "fundamental_source": "sec", "fundamental_pit": True, "fundamental_pit_lag_bars": 1}
if val_columns:
enriched = _attach_valuation_columns(enriched, panel, val_columns, start, end)
return enriched return enriched
agent/src/tools/br_market_data_tool.py
+279
View File
@@ -0,0 +1,279 @@
"""Read-only tool: official Brazilian market data from the local store.
Serves what ``python -m src.br_data sync`` downloaded from B3 (COTAHIST),
CVM (funds, companies, financial statements), Banco Central (SGS) and
Tesouro Transparente. Everything is local SQLite, so calls are fast and
never rate-limited. BCB series that were not synced are fetched live.
"""
from __future__ import annotations
import json
import sqlite3
from datetime import date, datetime, timedelta
from typing import Any, Callable
from zoneinfo import ZoneInfo
from src.agent.tools import BaseTool
from src.config.limits import TOOL_RESULT_LIMIT
_DEFAULT_LIMIT = 250
_MAX_LIMIT = 5000
# Friendly names for common SGS series.
_BCB_ALIASES = {
"selic": 11, "selic_meta": 432, "cdi": 12, "cdi_anual": 4389,
"ipca": 433, "ipca_12m": 13522, "igpm": 189, "inpc": 188,
"dolar": 1, "usd": 1, "ptax": 1, "euro": 21619, "pib": 4380, "desemprego": 24369,
}
_STATEMENTS = ("BPA", "BPP", "DRE", "DRA", "DFC_MD", "DFC_MI", "DVA", "DMPL")
def _digits(value: Any) -> str:
return "".join(ch for ch in str(value or "") if ch.isdigit())
def _ticker(value: Any) -> str:
text = str(value or "").strip().upper()
return text[:-3] if text.endswith(".SA") else text
def _limit(value: Any) -> int:
try:
return max(1, min(int(value), _MAX_LIMIT))
except (TypeError, ValueError):
return _DEFAULT_LIMIT
def _dates(kwargs: dict) -> tuple[str, str]:
end = str(kwargs.get("end_date") or datetime.now(ZoneInfo("America/Sao_Paulo")).date().isoformat())[:10]
start = str(kwargs.get("start_date") or (date.fromisoformat(end) - timedelta(days=365)).isoformat())[:10]
if date.fromisoformat(start) > date.fromisoformat(end):
raise ValueError("start_date must not be after end_date")
return start, end
def _error(message: str) -> str:
return json.dumps({"ok": False, "error": message}, ensure_ascii=False)
def _envelope(action: str, rows: list[dict], extra: dict | None = None) -> str:
"""Return the newest rows that fit the result budget."""
count = len(rows)
while True:
data = {"action": action, "count": count, "available": len(rows),
"truncated": count < len(rows), "rows": rows[-count:] if count else []}
if extra:
data.update(extra)
text = json.dumps({"ok": True, "market": "BR", "source": "br_data", "data": data},
ensure_ascii=False, default=str)
if len(text) <= TOOL_RESULT_LIMIT:
return text
if count <= 1:
return _error("Result metadata exceeds the tool limit; narrow the query.")
count = max(1, min(count - 1, int(count * TOOL_RESULT_LIMIT / len(text))))
class BrMarketDataTool(BaseTool):
"""Brazilian official data: B3 quotes, CVM funds/companies, BCB, Tesouro."""
name = "get_br_market_data"
description = (
"Official, free Brazilian market data from the local store synced from "
"B3, CVM, Banco Central and Tesouro Direto. Actions: "
"'quotes' (daily B3 bars for any listed instrument — shares, FIIs, ETFs, "
"BDRs, options — e.g. tickers=['PETR4','HGLG11']; prices are as traded, "
"not adjusted); 'search' (find tickers, funds or companies by name); "
"'fund' (daily quota, net assets, flows and holders of any CVM fund by "
"CNPJ); 'financials' (DFP annual / ITR quarterly statements of a listed "
"company by ticker or CNPJ: DRE, BPA, BPP, DFC_MI ...); 'macro' (BCB "
"SGS series: selic, cdi, ipca, igpm, ptax or a numeric code); "
"'tesouro' (Tesouro Direto prices and yields); 'status' (what is synced). "
'Example: {"action": "fund", "cnpj": "00.017.024/0001-53", '
'"start_date": "2025-01-01"}.'
)
parameters = {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["quotes", "search", "fund", "financials", "macro", "tesouro", "status"],
},
"tickers": {
"type": "array", "items": {"type": "string"},
"description": "B3 tickers for 'quotes' (PETR4 or PETR4.SA), or one ticker for 'financials'.",
},
"query": {"type": "string", "description": "Name fragment for 'search' or bond name for 'tesouro'."},
"cnpj": {"type": "string", "description": "Fund CNPJ ('fund') or company CNPJ ('financials')."},
"statement": {"type": "string", "enum": list(_STATEMENTS), "description": "Statement for 'financials' (default DRE)."},
"doc": {"type": "string", "enum": ["DFP", "ITR"], "description": "DFP annual (default) or ITR quarterly."},
"series": {"type": "string", "description": "BCB alias (selic, cdi, ipca, igpm, ptax, dolar, euro) or SGS code."},
"start_date": {"type": "string", "description": "YYYY-MM-DD (default: one year before end_date)."},
"end_date": {"type": "string", "description": "YYYY-MM-DD (default: today)."},
"limit": {"type": "integer", "description": f"Max rows (1-{_MAX_LIMIT}, default {_DEFAULT_LIMIT}); newest kept."},
},
"required": ["action"],
}
repeatable = True
def execute(self, **kwargs: Any) -> str:
action = str(kwargs.get("action") or "").strip().lower()
handlers: dict[str, Callable[[sqlite3.Connection, dict], str]] = {
"quotes": self._quotes, "search": self._search, "fund": self._fund,
"financials": self._financials, "macro": self._macro,
"tesouro": self._tesouro, "status": self._status,
}
handler = handlers.get(action)
if handler is None:
return _error(f"unknown action {action!r}; use one of {sorted(handlers)}")
from src.br_data.store import connect
try:
with connect() as conn:
return handler(conn, kwargs)
except FileNotFoundError as exc:
if action == "macro":
return self._macro(None, kwargs)
return _error(str(exc))
except (ValueError, TypeError) as exc:
return _error(f"invalid request: {exc}")
except sqlite3.Error as exc:
return _error(f"store query failed: {exc}")
# -- actions -----------------------------------------------------------
def _quotes(self, conn: sqlite3.Connection, kwargs: dict) -> str:
tickers = [_ticker(t) for t in (kwargs.get("tickers") or []) if _ticker(t)]
if not tickers:
return _error("'tickers' is required for action 'quotes'")
start, end = _dates(kwargs)
marks = ",".join("?" for _ in tickers)
rows = conn.execute(
"SELECT trade_date, ticker, market_type, spec, open, high, low, close, avg, "
"volume, turnover, trades, strike, expiry FROM b3_quotes "
f"WHERE ticker IN ({marks}) AND trade_date BETWEEN ? AND ? ORDER BY trade_date DESC, ticker DESC LIMIT ?",
(*tickers, start, end, _limit(kwargs.get("limit"))),
).fetchall()
out = [dict(r) for r in reversed(rows)]
return _envelope("quotes", out, {"adjustment": "raw (as traded)", "currency": "BRL"})
def _search(self, conn: sqlite3.Connection, kwargs: dict) -> str:
query = str(kwargs.get("query") or "").strip()
if len(query) < 2:
return _error("'query' (2+ characters) is required for action 'search'")
like = f"%{query.upper()}%"
limit = min(_limit(kwargs.get("limit")), 50)
tickers = conn.execute(
"SELECT ticker, short_name, spec, MAX(trade_date) AS last_trade FROM b3_quotes "
"WHERE market_type = 10 AND (ticker LIKE ? OR UPPER(short_name) LIKE ?) "
"AND trade_date >= (SELECT date(MAX(trade_date), '-90 day') FROM b3_quotes) "
"GROUP BY ticker ORDER BY ticker LIMIT ?",
(like, like, limit),
).fetchall()
funds = conn.execute(
"SELECT cnpj, name, fund_type, fund_class, status FROM cvm_funds "
"WHERE UPPER(name) LIKE ? OR cnpj LIKE ? ORDER BY (status LIKE 'EM FUNCIONAMENTO%') DESC, name LIMIT ?",
(like, f"%{_digits(query) or '#'}%", limit),
).fetchall()
companies = conn.execute(
"SELECT c.cnpj, c.name, c.trade_name, c.sector, c.status, "
"GROUP_CONCAT(t.ticker) AS tickers FROM cvm_companies c "
"LEFT JOIN cvm_company_tickers t ON t.cnpj = c.cnpj "
"WHERE UPPER(c.name) LIKE ? OR UPPER(c.trade_name) LIKE ? OR t.ticker LIKE ? "
"GROUP BY c.cnpj LIMIT ?",
(like, like, like, limit),
).fetchall()
return _envelope("search", [], {
"tickers": [dict(r) for r in tickers],
"funds": [dict(r) for r in funds],
"companies": [dict(r) for r in companies],
})
def _fund(self, conn: sqlite3.Connection, kwargs: dict) -> str:
cnpj = _digits(kwargs.get("cnpj"))
if len(cnpj) != 14:
return _error("'cnpj' (14 digits) is required for action 'fund'; use action 'search' to find it")
start, end = _dates(kwargs)
info = conn.execute("SELECT * FROM cvm_funds WHERE cnpj = ?", (cnpj,)).fetchone()
rows = conn.execute(
"SELECT trade_date, subclass, quota, net_assets, total_value, inflows, outflows, holders "
"FROM cvm_fund_daily WHERE cnpj = ? AND trade_date BETWEEN ? AND ? ORDER BY trade_date DESC, subclass DESC LIMIT ?",
(cnpj, start, end, _limit(kwargs.get("limit"))),
).fetchall()
out = [dict(r) for r in reversed(rows)]
extra: dict[str, Any] = {"fund": dict(info) if info else None, "currency": "BRL"}
if len({r["subclass"] for r in out}) == 1 and len(out) >= 2 and out[0]["quota"] and out[-1]["quota"]:
extra["quota_return"] = round(out[-1]["quota"] / out[0]["quota"] - 1, 6)
return _envelope("fund", out, extra)
def _financials(self, conn: sqlite3.Connection, kwargs: dict) -> str:
cnpj = _digits(kwargs.get("cnpj"))
tickers = kwargs.get("tickers") or []
if not cnpj and tickers:
ticker = _ticker(tickers[0])
row = conn.execute("SELECT cnpj FROM cvm_company_tickers WHERE ticker = ?", (ticker,)).fetchone()
if row is None:
return _error(f"no CVM company mapped to ticker {ticker}; try action 'search'")
cnpj = row["cnpj"]
if len(cnpj) != 14:
return _error("'cnpj' or one entry in 'tickers' is required for action 'financials'")
statement = str(kwargs.get("statement") or "DRE").upper()
doc = str(kwargs.get("doc") or "DFP").upper()
rows = conn.execute(
"SELECT statement, period_end, period_start, account, description, value, version FROM cvm_statements "
"WHERE cnpj = ? AND doc = ? AND statement IN (?, ?) ORDER BY period_end, account",
(cnpj, doc, statement, f"{statement}_ind"),
).fetchall()
periods = sorted({r["period_end"] for r in rows})
keep = set(periods[-min(_limit(kwargs.get("limit")), 8 if doc == "DFP" else 12):])
out = [dict(r) for r in rows if r["period_end"] in keep]
company = conn.execute("SELECT * FROM cvm_companies WHERE cnpj = ?", (cnpj,)).fetchone()
return _envelope("financials", out, {
"company": dict(company) if company else {"cnpj": cnpj},
"doc": doc, "statement": statement, "currency": "BRL", "periods": sorted(keep),
"point_in_time": False, "basis": "latest stored restatement; scope is labelled per row",
})
def _macro(self, conn: sqlite3.Connection | None, kwargs: dict) -> str:
raw = str(kwargs.get("series") or "").strip().lower()
code = _BCB_ALIASES.get(raw) or (int(raw) if raw.isdigit() else None)
if code is None:
return _error(f"'series' must be a BCB SGS code or one of {sorted(_BCB_ALIASES)}")
start, end = _dates(kwargs)
rows: list[dict] = []
if conn is not None:
rows = [dict(r) for r in conn.execute(
"SELECT obs_date AS date, value FROM bcb_series WHERE code = ? "
"AND obs_date BETWEEN ? AND ? ORDER BY obs_date", (code, start, end),
).fetchall()]
if not rows:
from src.br_data.bcb import fetch_series
try:
rows = [{"date": d, "value": v} for d, v in
fetch_series(code, date.fromisoformat(start), date.fromisoformat(end))]
except Exception as exc: # noqa: BLE001 - surface as envelope
return _error(f"BCB SGS request failed: {exc}")
from src.br_data.bcb import SERIES
return _envelope("macro", rows[-_limit(kwargs.get("limit")):],
{"series": code, "name": SERIES.get(code)})
def _tesouro(self, conn: sqlite3.Connection, kwargs: dict) -> str:
start, end = _dates(kwargs)
query = f"%{str(kwargs.get('query') or '').upper()}%"
rows = conn.execute(
"SELECT base_date, bond, maturity, buy_rate, sell_rate, buy_price, sell_price "
"FROM tesouro_prices WHERE UPPER(bond) LIKE ? AND base_date BETWEEN ? AND ? "
"ORDER BY base_date DESC, bond DESC, maturity DESC LIMIT ?", (query, start, end, _limit(kwargs.get("limit"))),
).fetchall()
return _envelope("tesouro", [dict(r) for r in reversed(rows)], {"currency": "BRL"})
def _status(self, conn: sqlite3.Connection, kwargs: dict) -> str:
out = []
for table, col in (("b3_quotes", "trade_date"), ("cvm_fund_daily", "trade_date"),
("cvm_statements", "period_end"), ("bcb_series", "obs_date"),
("tesouro_prices", "base_date")):
row = conn.execute(f"SELECT COUNT(*), MIN({col}), MAX({col}) FROM {table}").fetchone()
out.append({"table": table, "rows": row[0], "from": row[1], "to": row[2]})
return _envelope("status", out)
agent/src/tools/financial_statements_tool.py
+92 -5
View File
@@ -384,6 +384,80 @@ def _fetch_yahoo_statement(
return {"periods": _cap_periods(periods)} return {"periods": _cap_periods(periods)}
# CVM statement codes per requested statement, consolidated first. CVM files
# carry no ratio/indicator block, so "indicators" always comes from Yahoo.
_CVM_STATEMENTS: dict[str, tuple[str, ...]] = {
"balance": ("BPA", "BPP"),
"income": ("DRE",),
"cashflow": ("DFC_MI", "DFC_MD"),
}
def _fetch_cvm_statement(code: str, *, statement: str, period: str) -> dict[str, Any] | None:
"""Read a B3 company's filed statement from the local CVM store.
The store is filled by ``python -m src.br_data sync`` (DFP annual, ITR
quarterly). Consolidated statements are preferred; parent-only (``_ind``)
ones are used when a company files no consolidated set.
Args:
code: B3 symbol, e.g. ``"PETR4.SA"``.
statement: ``balance``, ``income`` or ``cashflow``.
period: ``"annual"`` (DFP) or ``"quarter"`` (ITR).
Returns:
``{"periods": [...]}`` newest first, each period a flat
``{"period_end", "period_start", "<account> <description>": value}``
record in BRL, or ``None`` when the store, the ticker or the rows are
missing, so the caller can fall back to Yahoo.
"""
statements = _CVM_STATEMENTS.get(statement)
if not statements:
return None
try:
from src.br_data.store import connect
ticker = code.rpartition(".")[0].strip().upper()
doc = "DFP" if period == "annual" else "ITR"
with connect() as conn:
row = conn.execute(
"SELECT cnpj FROM cvm_company_tickers WHERE ticker = ?", (ticker,)
).fetchone()
if row is None:
return None
rows: list[Any] = []
for suffix in ("", "_ind"):
names = [name + suffix for name in statements]
marks = ",".join("?" for _ in names)
rows = conn.execute(
"SELECT period_end, period_start, account, description, value "
f"FROM cvm_statements WHERE cnpj = ? AND doc = ? AND statement IN ({marks}) "
"ORDER BY period_end DESC, period_start, account",
(row["cnpj"], doc, *names),
).fetchall()
if rows:
break
except Exception as exc: # noqa: BLE001 - missing store or schema: fall back to Yahoo
logger.info("cvm statement unavailable for %s: %s", code, exc)
return None
if not rows:
return None
periods: dict[tuple[str, str], dict[str, Any]] = {}
for r in rows:
key = (r["period_end"], r["period_start"])
record = periods.setdefault(
key, {"period_end": r["period_end"], "period_start": r["period_start"] or None}
)
record[f"{r['account']} {r['description'] or ''}".strip()] = r["value"]
ordered = sorted(
periods.values(),
key=lambda rec: (rec["period_end"], rec["period_start"] or ""),
reverse=True,
)
return {"periods": _cap_periods(ordered), "point_in_time": False,
"basis": "latest stored CVM revisions"}
def _to_number(value: Any) -> float | None: def _to_number(value: Any) -> float | None:
"""Coerce a numeric provider cell to float, preserving missing values.""" """Coerce a numeric provider cell to float, preserving missing values."""
if value in (None, "", "-"): if value in (None, "", "-"):
@@ -681,6 +755,9 @@ def _classify_market(code: str) -> str | None:
if suffix == "L": if suffix == "L":
# London Stock Exchange (#1206). # London Stock Exchange (#1206).
return "uk" return "uk"
if suffix == "SA":
# B3 (Brazil).
return "br"
return None return None
@@ -692,9 +769,10 @@ class FinancialStatementsTool(BaseTool):
"Fetch a single stock's financial statements: balance sheet, income " "Fetch a single stock's financial statements: balance sheet, income "
"statement, cash-flow statement, or key per-period indicators (margins, " "statement, cash-flow statement, or key per-period indicators (margins, "
"ROE, EPS, etc.). Markets: A-share (.SH/.SZ/.BJ), US (.US), " "ROE, EPS, etc.). Markets: A-share (.SH/.SZ/.BJ), US (.US), "
"Hong Kong (.HK) and UK LSE (.L). US uses SEC EDGAR companyfacts; " "Hong Kong (.HK), UK LSE (.L) and Brazil B3 (.SA). US uses SEC EDGAR "
"A-share and HK use Eastmoney; UK uses Yahoo quoteSummary (annual " "companyfacts; A-share and HK use Eastmoney; UK uses Yahoo quoteSummary "
"history). Reports come back newest-first as flat per-period rows. Use " "(annual history); Brazil uses the official CVM DFP/ITR filings from the "
"local br-data store (values in BRL), falling back to Yahoo. Reports come back newest-first as flat per-period rows. Use "
'this to read fundamentals before building a valuation or screen. Example: ' 'this to read fundamentals before building a valuation or screen. Example: '
'{"code": "600519.SH", "statement": "income", "period": "annual"}.' '{"code": "600519.SH", "statement": "income", "period": "annual"}.'
) )
@@ -705,7 +783,7 @@ class FinancialStatementsTool(BaseTool):
"type": "string", "type": "string",
"description": ( "description": (
"Single symbol with a market suffix, e.g. '600519.SH', " "Single symbol with a market suffix, e.g. '600519.SH', "
"'000001.SZ', 'AAPL.US', '00700.HK', or 'VOD.L'." "'000001.SZ', 'AAPL.US', '00700.HK', 'VOD.L', or 'PETR4.SA'."
), ),
}, },
"statement": { "statement": {
@@ -772,7 +850,7 @@ class FinancialStatementsTool(BaseTool):
market = _classify_market(code) market = _classify_market(code)
if market is None: if market is None:
return _error( return _error(
"code must carry a supported suffix: .SH/.SZ/.BJ, .US, .HK, or .L" "code must carry a supported suffix: .SH/.SZ/.BJ, .US, .HK, .L, or .SA"
) )
if market == "us": if market == "us":
@@ -781,6 +859,15 @@ class FinancialStatementsTool(BaseTool):
elif market == "uk": elif market == "uk":
result = _fetch_yahoo_statement(code, statement=statement, period=period) result = _fetch_yahoo_statement(code, statement=statement, period=period)
source = "yahoo" source = "yahoo"
elif market == "br":
# Official CVM filings (local store) first; Yahoo when not synced
# yet and for the indicator block CVM does not publish.
cvm = _fetch_cvm_statement(code, statement=statement, period=period)
if cvm is not None:
result, source = cvm, "cvm"
else:
result = _fetch_yahoo_statement(code, statement=statement, period=period)
source = "yahoo"
else: else:
result = _fetch_eastmoney_statement( result = _fetch_eastmoney_statement(
code, statement=statement, period=period code, statement=statement, period=period
agent/src/tools/stock_news_tool.py
+9 -4
View File
@@ -42,7 +42,7 @@ _EM_NEWS_URL = "https://search-api-web.eastmoney.com/search/jsonp"
# A-share / China-market suffixes that route to the Eastmoney news surface. # A-share / China-market suffixes that route to the Eastmoney news surface.
_EM_SUFFIXES = ("SH", "SZ", "BJ") _EM_SUFFIXES = ("SH", "SZ", "BJ")
# Suffixes that route to Yahoo's search-news surface. # Suffixes that route to Yahoo's search-news surface.
_YAHOO_SUFFIXES = ("US", "HK") _YAHOO_SUFFIXES = ("US", "HK", "SA")
# Default broad-market query used when ``scope='global'`` carries no code. # Default broad-market query used when ``scope='global'`` carries no code.
_GLOBAL_QUERY = "财经" _GLOBAL_QUERY = "财经"
@@ -249,8 +249,9 @@ class StockNewsTool(BaseTool):
description = ( description = (
"Fetch recent financial news headlines, read-only and no auth. Markets: " "Fetch recent financial news headlines, read-only and no auth. Markets: "
"China A-share (SH/SZ/BJ) returns Eastmoney news ARTICLES " "China A-share (SH/SZ/BJ) returns Eastmoney news ARTICLES "
"(title/url/source/published/snippet) under 'articles'. US (.US) and Hong " "(title/url/source/published/snippet) under 'articles'. US (.US), Hong "
"Kong (.HK) return Yahoo Finance news articles with the same fields. Use " "Kong (.HK) and Brazil (B3 .SA) return Yahoo Finance news articles with "
"the same fields. Use "
"scope 'stock' with a 'code', or scope 'global' (no code) for broad " "scope 'stock' with a 'code', or scope 'global' (no code) for broad "
"China-market finance articles. " "China-market finance articles. "
'Example: {"code": "600519.SH", "scope": "stock", "limit": 10}.' 'Example: {"code": "600519.SH", "scope": "stock", "limit": 10}.'
@@ -353,6 +354,10 @@ class StockNewsTool(BaseTool):
if suffix in _EM_SUFFIXES: if suffix in _EM_SUFFIXES:
return self._stock_via_eastmoney(code, query, limit) return self._stock_via_eastmoney(code, query, limit)
if suffix in _YAHOO_SUFFIXES: if suffix in _YAHOO_SUFFIXES:
# A bare B3 code (PETR4) is ambiguous on Yahoo search; its own
# spelling (PETR4.SA) is the listing.
if suffix == "SA":
query = code.upper()
return self._stock_via_yahoo(code, query, limit) return self._stock_via_yahoo(code, query, limit)
return self._error( return self._error(
f"unsupported market for code {code!r}; expected suffix in " f"unsupported market for code {code!r}; expected suffix in "
@@ -374,7 +379,7 @@ class StockNewsTool(BaseTool):
def _stock_via_yahoo(self, code: str, query: str, limit: int) -> str: def _stock_via_yahoo(self, code: str, query: str, limit: int) -> str:
"""Fetch US/HK news articles from Yahoo for one code.""" """Fetch US/HK news articles from Yahoo for one code."""
market = "hk" if _suffix_of(code) == "HK" else "us" market = {"HK": "hk", "SA": "br"}.get(_suffix_of(code), "us")
try: try:
articles = _fetch_yahoo_news(query, limit) articles = _fetch_yahoo_news(query, limit)
except Exception as exc: # noqa: BLE001 - surface any fetch failure as envelope except Exception as exc: # noqa: BLE001 - surface any fetch failure as envelope
agent/src/tools/stock_profile_tool.py
+5 -3
View File
@@ -275,6 +275,8 @@ def _market_for(ticker: str) -> str:
return "uk" return "uk"
if upper.endswith(".BA"): if upper.endswith(".BA"):
return "ar" return "ar"
if upper.endswith(".SA"):
return "br"
return "us" return "us"
@@ -283,8 +285,8 @@ class StockProfileTool(BaseTool):
name = "get_stock_profile" name = "get_stock_profile"
description = ( description = (
"Fetch a read-only company profile for a US, Hong Kong, UK (LSE .L), or " "Fetch a read-only company profile for a US, Hong Kong, UK (LSE .L), "
"Argentina (BYMA .BA) " "Argentina (BYMA .BA), or Brazil (B3 .SA) "
"listing from Yahoo Finance: valuation key statistics, analyst price " "listing from Yahoo Finance: valuation key statistics, analyst price "
"targets and earnings/revenue estimates, institutional and insider " "targets and earnings/revenue estimates, institutional and insider "
"ownership, and the analyst recommendation trend. Always includes " "ownership, and the analyst recommendation trend. Always includes "
@@ -301,7 +303,7 @@ class StockProfileTool(BaseTool):
"ticker": { "ticker": {
"type": "string", "type": "string",
"description": ( "description": (
"US, HK, UK (LSE .L), or Argentina (BYMA .BA) symbol. US uses a bare or .US suffix " "US, HK, UK (LSE .L), Argentina (BYMA .BA), or Brazil (B3 .SA, e.g. PETR4.SA) symbol. US uses a bare or .US suffix "
"(AAPL or AAPL.US); HK uses a zero-padded .HK code " "(AAPL or AAPL.US); HK uses a zero-padded .HK code "
"(00700.HK); UK uses the .L suffix (VOD.L)." "(00700.HK); UK uses the .L suffix (VOD.L)."
), ),
agent/src/tools/symbol_search_tool.py
+1
View File
@@ -160,6 +160,7 @@ _MARKET_BY_SUFFIX: Dict[str, str] = {
"TO": "ca", "TO": "ca",
"V": "ca", "V": "ca",
"BA": "ar", "BA": "ar",
"SA": "br",
"L": "uk", "L": "uk",
"VN": "vn", "VN": "vn",
} }
agent/src/trading/connectors/alpaca/sdk.py
+3 -1
View File
@@ -74,7 +74,9 @@ class AlpacaConfig:
profile: ``paper``, ``live-readonly`` or ``live``. profile: ``paper``, ``live-readonly`` or ``live``.
feed: Market-data feed, ``iex`` (free) or ``sip`` (paid). feed: Market-data feed, ``iex`` (free) or ``sip`` (paid).
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Always true for this layer; order methods are not exposed. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
api_key: str = "" api_key: str = ""
agent/src/trading/connectors/dhan/sdk.py
+3 -1
View File
@@ -80,7 +80,9 @@ class DhanConfig:
access_token: Dhan API access token (generated from dhan.co). access_token: Dhan API access token (generated from dhan.co).
profile: ``paper``, ``live-readonly`` or ``live``. profile: ``paper``, ``live-readonly`` or ``live``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Whether order placement is disabled. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
client_id: str = "" client_id: str = ""
agent/src/trading/connectors/etoro/client.py
+3 -1
View File
@@ -64,7 +64,9 @@ class EtoroConfig:
user_key: eToro user key (shared across demo and real paths). user_key: eToro user key (shared across demo and real paths).
profile: ``paper``, ``live-readonly`` or ``live``. profile: ``paper``, ``live-readonly`` or ``live``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Always true for this layer unless trade profiles call writes. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
api_key: str = "" api_key: str = ""
agent/src/trading/connectors/futu/sdk.py
+3 -1
View File
@@ -80,7 +80,9 @@ class FutuConfig:
filter_trdmarket: Trade market filter for the trade context, e.g. ``HK``. filter_trdmarket: Trade market filter for the trade context, e.g. ``HK``.
acc_id: Account id; ``0`` means resolve by ``trd_env``. acc_id: Account id; ``0`` means resolve by ``trd_env``.
timeout: Network/connect timeout in seconds. timeout: Network/connect timeout in seconds.
readonly: Always true for this layer; order methods are not exposed. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
host: str = DEFAULT_HOST host: str = DEFAULT_HOST
agent/src/trading/connectors/kis/sdk.py
+3 -1
View File
@@ -121,7 +121,9 @@ class KISConfig:
e.g. ``"01"``. e.g. ``"01"``.
profile: ``paper`` or ``live-readonly``. profile: ``paper`` or ``live-readonly``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Whether order placement is disabled. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
app_key: str = "" app_key: str = ""
agent/src/trading/connectors/longbridge/sdk.py
+3 -1
View File
@@ -66,7 +66,9 @@ class LongbridgeConfig:
profile: ``paper``, ``live-readonly`` or ``live`` (operator-declared). profile: ``paper``, ``live-readonly`` or ``live`` (operator-declared).
region: ``global`` or ``cn`` (host region; not a paper/live signal). region: ``global`` or ``cn`` (host region; not a paper/live signal).
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Always true for this layer; order methods are not exposed. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
app_key: str = field(default="", repr=False) app_key: str = field(default="", repr=False)
agent/src/trading/connectors/okx/sdk.py
+3 -1
View File
@@ -60,7 +60,9 @@ class OKXConfig:
host: REST host (default ``https://www.okx.com``). host: REST host (default ``https://www.okx.com``).
expected_uid: Optional account UID to pin in ``check_status``. expected_uid: Optional account UID to pin in ``check_status``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Always true for this layer; order methods are not exposed. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
api_key: str = "" api_key: str = ""
agent/src/trading/connectors/tiger/sdk.py
+3 -1
View File
@@ -68,7 +68,9 @@ class TigerConfig:
account: Tiger account number. A 17-digit number is a paper account. account: Tiger account number. A 17-digit number is a paper account.
profile: ``paper``, ``live-readonly`` or ``live``. profile: ``paper``, ``live-readonly`` or ``live``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Always true for this layer; order methods are not exposed. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
tiger_id: str = "" tiger_id: str = ""
agent/src/trading/connectors/upbit/sdk.py
+3 -1
View File
@@ -74,7 +74,9 @@ class UpbitConfig:
secret_key: Upbit Open API secret key. secret_key: Upbit Open API secret key.
profile: ``paper`` or ``live-readonly``. profile: ``paper`` or ``live-readonly``.
timeout: Network timeout in seconds. timeout: Network timeout in seconds.
readonly: Whether order placement is disabled. readonly: Not consulted by the order methods. Whether a profile may place or
cancel orders is gated first by its ``TradingProfile.readonly`` in the
trading service.
""" """
access_key: str = "" access_key: str = ""
agent/src/trading/service.py
+3 -3
View File
@@ -131,9 +131,9 @@ def _sdk_config(
# #
# Per-call overrides must still pass the connector's own allowlist. Every # Per-call overrides must still pass the connector's own allowlist. Every
# SDK connector deliberately narrows what a caller may override — OKX and # SDK connector deliberately narrows what a caller may override — OKX and
# Binance both exclude ``readonly`` ("always true for this layer") and # Binance both exclude ``readonly`` and ``timeout``; Longbridge's overlay
# ``timeout``; Longbridge's overlay is ``profile``/``region`` only, # is ``profile``/``region`` only, precisely so a caller "cannot mix or
# precisely so a caller "cannot mix or bypass the shared resolver". # bypass the shared resolver".
# ``build_config`` applies that filter; constructing from a raw merged # ``build_config`` applies that filter; constructing from a raw merged
# mapping would not, and ``overrides`` is the one part of this payload that # mapping would not, and ``overrides`` is the one part of this payload that
# reaches us from an MCP tool argument or a REST body. # reaches us from an MCP tool argument or a REST body.
agent/tests/factors/test_fundamental_expansion_price_samples.py
+211
View File
@@ -0,0 +1,211 @@
"""Sample tests for the Tier-1 fundamental expansion factors.
Covers the four price-dependent / derived-field factors added per
``research/fundamental_factor_expansion_proposal.md`` §4 Tier 1 (#1, #5, #6, #8):
- ``fund_book_to_market`` (real FF HML characteristic)
- ``fund_fcf_yield`` (CFO - capex over market cap)
- ``fund_revenue_growth`` (schema derived field ``revenue_growth``)
- ``fund_earnings_growth`` (schema derived field ``earnings_growth``)
All compute-level tests pass hand-built panel dicts directly to ``compute()``
— never through the loader/schema — so they are independent of when the
concurrent schema change adding ``revenue_growth``/``earnings_growth`` lands.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
import pytest
from src.factors.base import zscore
from src.factors.registry import AlphaMeta
from src.factors.registry import get_default_registry
from src.factors.zoo.fundamental import book_to_market
from src.factors.zoo.fundamental import earnings_growth
from src.factors.zoo.fundamental import fcf_yield
from src.factors.zoo.fundamental import revenue_growth
_DATES = pd.to_datetime(["2024-01-02", "2024-01-03"])
_CODES = ["A", "B", "C"]
_PRICE_FACTORS = [book_to_market, fcf_yield]
_GROWTH_FACTORS = [revenue_growth, earnings_growth]
_ALL_FACTORS = _PRICE_FACTORS + _GROWTH_FACTORS
def _frame(rows: list[list[float]]) -> pd.DataFrame:
"""Build a 2-date x 3-symbol frame from per-date row values."""
return pd.DataFrame(rows, index=_DATES, columns=_CODES, dtype=np.float64)
def _book_to_market_panel() -> dict[str, pd.DataFrame]:
# market cap: A=1000, B=2000, C=5000; equity 500/250/100
# -> b/m: A=0.50, B=0.125, C=0.02 (strictly decreasing)
return {
"close": _frame([[10.0, 20.0, 50.0], [10.0, 20.0, 50.0]]),
"fund:total_equity": _frame([[500.0, 250.0, 100.0], [500.0, 250.0, 100.0]]),
"fund:shares_diluted": _frame([[100.0, 100.0, 100.0], [100.0, 100.0, 100.0]]),
}
def _fcf_yield_panel() -> dict[str, pd.DataFrame]:
# A: cfo=100, capex=30 -> fcf=70, mcap=10*10=100 -> yield 0.70
# B: cfo=50, capex=15 -> fcf=35, mcap=20*10=200 -> yield 0.175
# C: cfo=20, capex=25 -> fcf=-5, mcap=50*10=500 -> yield -0.01
return {
"close": _frame([[10.0, 20.0, 50.0], [10.0, 20.0, 50.0]]),
"fund:cfo": _frame([[100.0, 50.0, 20.0], [100.0, 50.0, 20.0]]),
"fund:capex": _frame([[30.0, 15.0, 25.0], [30.0, 15.0, 25.0]]),
"fund:shares_diluted": _frame([[10.0, 10.0, 10.0], [10.0, 10.0, 10.0]]),
}
def _growth_panel(field: str) -> dict[str, pd.DataFrame]:
return {field: _frame([[0.30, 0.10, -0.20], [0.30, 0.10, -0.20]])}
def _valid_panel(module) -> dict[str, pd.DataFrame]:
if module is book_to_market:
return _book_to_market_panel()
if module is fcf_yield:
return _fcf_yield_panel()
if module is revenue_growth:
return _growth_panel("fund:revenue_growth")
return _growth_panel("fund:earnings_growth")
# ---------------------------------------------------------------------------
# (a) AlphaMeta gate accepts each factor's declared columns_required
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("module", _ALL_FACTORS, ids=lambda m: m.__alpha_meta__["id"])
def test_alpha_meta_gate_accepts_declared_columns(module) -> None:
meta = AlphaMeta(**module.__alpha_meta__)
assert meta.columns_required == module.__alpha_meta__["columns_required"]
assert meta.decay_horizon == 252
assert meta.min_warmup_bars == 1
assert meta.universe == ["equity_us", "equity_cn", "equity_hk"]
assert meta.frequency == ["1d"]
def test_alpha_meta_themes_match_proposal() -> None:
assert book_to_market.__alpha_meta__["theme"] == ["value"]
assert fcf_yield.__alpha_meta__["theme"] == ["value"]
assert revenue_growth.__alpha_meta__["theme"] == ["growth"]
assert earnings_growth.__alpha_meta__["theme"] == ["growth"]
# ---------------------------------------------------------------------------
# (b) compute() numeric correctness on hand-built panels
# ---------------------------------------------------------------------------
def test_book_to_market_matches_hand_computed_ratios() -> None:
panel = _book_to_market_panel()
expected_ratios = _frame([[0.50, 0.125, 0.02], [0.50, 0.125, 0.02]])
result = book_to_market.compute(panel)
pd.testing.assert_frame_equal(result, zscore(expected_ratios))
def test_book_to_market_direction_higher_equity_to_mcap_scores_higher() -> None:
panel = _book_to_market_panel()
result = book_to_market.compute(panel)
row = result.iloc[0]
assert row["A"] > row["B"] > row["C"]
def test_fcf_yield_subtracts_capex_from_cfo() -> None:
# A: cfo 100, capex 30 -> fcf 70 over mcap 100 -> yield 0.70
panel = _fcf_yield_panel()
expected_ratios = _frame([[0.70, 0.175, -0.01], [0.70, 0.175, -0.01]])
result = fcf_yield.compute(panel)
pd.testing.assert_frame_equal(result, zscore(expected_ratios))
@pytest.mark.parametrize(
("module", "field"),
[(revenue_growth, "fund:revenue_growth"), (earnings_growth, "fund:earnings_growth")],
ids=["fund_revenue_growth", "fund_earnings_growth"],
)
def test_growth_factors_zscore_declared_field(module, field: str) -> None:
panel = _growth_panel(field)
result = module.compute(panel)
pd.testing.assert_frame_equal(result, zscore(panel[field]))
row = result.iloc[0]
assert row["A"] > row["B"] > row["C"]
# ---------------------------------------------------------------------------
# (c) NaN contract: NaN in any declared dependency -> NaN output at that cell
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("module", _ALL_FACTORS, ids=lambda m: m.__alpha_meta__["id"])
def test_nan_in_any_declared_dependency_produces_nan_output(module) -> None:
for column in module.__alpha_meta__["columns_required"]:
panel = _valid_panel(module)
panel[column] = panel[column].copy()
panel[column].iloc[0, 1] = np.nan # symbol B on date 0
result = module.compute(panel)
assert np.isnan(result.iloc[0, 1]), f"{module.__alpha_meta__['id']}: {column}"
assert np.isfinite(result.iloc[0, 0]), f"{module.__alpha_meta__['id']}: {column}"
assert np.isfinite(result.iloc[0, 2]), f"{module.__alpha_meta__['id']}: {column}"
# ---------------------------------------------------------------------------
# (d) Zero market cap / zero denominator -> NaN, never inf
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("module", _PRICE_FACTORS, ids=lambda m: m.__alpha_meta__["id"])
def test_zero_close_produces_nan_not_inf(module) -> None:
panel = _valid_panel(module)
panel["close"] = panel["close"].copy()
panel["close"].iloc[0, 1] = 0.0 # zero market cap for B on date 0
result = module.compute(panel)
assert np.isnan(result.iloc[0, 1])
assert not np.isinf(result.to_numpy(dtype=np.float64)).any()
@pytest.mark.parametrize("module", _PRICE_FACTORS, ids=lambda m: m.__alpha_meta__["id"])
def test_zero_shares_produces_nan_not_inf(module) -> None:
panel = _valid_panel(module)
panel["fund:shares_diluted"] = panel["fund:shares_diluted"].copy()
panel["fund:shares_diluted"].iloc[1, 2] = 0.0 # zero share count for C on date 1
result = module.compute(panel)
assert np.isnan(result.iloc[1, 2])
assert not np.isinf(result.to_numpy(dtype=np.float64)).any()
# ---------------------------------------------------------------------------
# Registry load: the four ids are discoverable from the default registry
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"alpha_id",
["fund_book_to_market", "fund_fcf_yield", "fund_revenue_growth", "fund_earnings_growth"],
)
def test_factor_is_loadable_from_default_registry(alpha_id: str) -> None:
registry = get_default_registry()
alpha = registry.get(alpha_id)
assert alpha.id == alpha_id
agent/tests/factors/test_fundamental_expansion_samples.py
+231
View File
@@ -0,0 +1,231 @@
"""Registration + numeric regression for the six Tier-1 fundamental factors.
Covers ``fund_accruals``, ``fund_roa``, ``fund_leverage``, ``fund_cfo_yield``,
``fund_operating_margin`` and ``fund_cash_to_assets`` in ``zoo/fundamental/``
(research/fundamental_factor_expansion_proposal.md §4 Tier 1 #2/#3/#4/#7/#9/#10).
Assertions are membership-based, never an exact registry total: the zoo grows
concurrently. Expected values are recomputed here with an independent numpy
formula (mean / sample-std z-score, plain division) rather than reusing
``src.factors.base`` operators, so a regression in the shared operator does not
silently validate itself.
"""
from __future__ import annotations
import importlib
import numpy as np
import pandas as pd
import pytest
from src.factors.registry import Registry
_DATES = pd.to_datetime(["2024-01-01", "2024-01-02"])
_CODES = ["AAA", "BBB", "CCC"]
# alpha_id -> (theme, columns_required, module stem under zoo/fundamental/)
_EXPECTED_META: dict[str, tuple[list[str], list[str], str]] = {
"fund_accruals": (["quality"], ["fund:accruals"], "accruals"),
"fund_roa": (["quality"], ["fund:roa"], "roa"),
"fund_leverage": (["leverage"], ["fund:leverage"], "leverage"),
"fund_cfo_yield": (["quality"], ["fund:cfo", "fund:total_assets"], "cfo_yield"),
"fund_operating_margin": (
["quality"],
["fund:operating_income", "fund:revenue"],
"operating_margin",
),
"fund_cash_to_assets": (["quality"], ["fund:cash", "fund:total_assets"], "cash_to_assets"),
}
# One declared dependency per alpha, blanked wholesale in the NaN-contract test.
_BLANKED_DEPENDENCY: dict[str, str] = {
"fund_accruals": "fund:accruals",
"fund_roa": "fund:roa",
"fund_leverage": "fund:leverage",
"fund_cfo_yield": "fund:cfo",
"fund_operating_margin": "fund:operating_income",
"fund_cash_to_assets": "fund:cash",
}
# safe_div-based alphas and the denominator column driven to exactly zero.
_ZERO_DENOMINATOR: dict[str, str] = {
"fund_cfo_yield": "fund:total_assets",
"fund_operating_margin": "fund:revenue",
"fund_cash_to_assets": "fund:total_assets",
}
@pytest.fixture(scope="module")
def registry() -> Registry:
return Registry()
def _frame(rows: list[list[float]]) -> pd.DataFrame:
return pd.DataFrame(rows, index=_DATES, columns=_CODES, dtype="float64")
def _build_panel() -> dict[str, pd.DataFrame]:
"""Hand-built 2-date x 3-symbol panel; every row has distinct values."""
return {
"fund:accruals": _frame([[0.10, 0.02, -0.05], [0.08, 0.03, -0.01]]),
"fund:roa": _frame([[0.15, 0.05, -0.10], [0.12, 0.06, 0.00]]),
"fund:leverage": _frame([[2.0, 0.5, 0.1], [1.8, 0.6, 0.2]]),
"fund:cfo": _frame([[300.0, 200.0, 100.0], [320.0, 210.0, 90.0]]),
"fund:total_assets": _frame(
[[1000.0, 1000.0, 1000.0], [1100.0, 1050.0, 900.0]]
),
"fund:operating_income": _frame([[200.0, 100.0, 50.0], [220.0, 110.0, 40.0]]),
"fund:revenue": _frame([[1000.0, 1000.0, 1000.0], [1000.0, 1100.0, 800.0]]),
"fund:cash": _frame([[400.0, 100.0, 50.0], [420.0, 110.0, 60.0]]),
}
def _numpy_zscore(values: np.ndarray) -> np.ndarray:
"""Independent per-row z-score: (x - mean) / sample-std, NaN-preserving."""
mean = np.nanmean(values, axis=1, keepdims=True)
std = np.nanstd(values, axis=1, ddof=1, keepdims=True)
with np.errstate(invalid="ignore", divide="ignore"):
return (values - mean) / std
def _z(frame: pd.DataFrame) -> pd.DataFrame:
return pd.DataFrame(
_numpy_zscore(frame.to_numpy(dtype=np.float64)),
index=frame.index,
columns=frame.columns,
)
def _expected_outputs(panel: dict[str, pd.DataFrame]) -> dict[str, pd.DataFrame]:
"""Expected output frames per alpha, derived from the panel by numpy math."""
total_assets = panel["fund:total_assets"]
revenue = panel["fund:revenue"]
return {
"fund_accruals": -_z(panel["fund:accruals"]),
"fund_roa": _z(panel["fund:roa"]),
"fund_leverage": -_z(panel["fund:leverage"]),
"fund_cfo_yield": _z(panel["fund:cfo"] / total_assets),
"fund_operating_margin": _z(panel["fund:operating_income"] / revenue),
"fund_cash_to_assets": _z(panel["fund:cash"] / total_assets),
}
def _load_module(stem: str):
return importlib.import_module(f"src.factors.zoo.fundamental.{stem}")
# --------------------------------------------------------------- (a) registry
@pytest.mark.parametrize("alpha_id", sorted(_EXPECTED_META))
def test_registry_loads_factor_with_expected_meta(
registry: Registry, alpha_id: str
) -> None:
theme, columns, stem = _EXPECTED_META[alpha_id]
alpha = registry.get(alpha_id)
assert alpha.zoo == "fundamental"
assert alpha.module_path == f"src.factors.zoo.fundamental.{stem}"
assert alpha.meta["theme"] == theme
assert alpha.meta["columns_required"] == columns
assert alpha.meta["universe"] == ["equity_us", "equity_cn", "equity_hk"]
assert alpha.meta["frequency"] == ["1d"]
assert alpha.meta["decay_horizon"] == 252
assert alpha.meta["min_warmup_bars"] == 1
assert alpha.meta["notes"].strip()
# ------------------------------------------------- (b) numeric correctness
@pytest.mark.parametrize("alpha_id", sorted(_EXPECTED_META))
def test_compute_matches_independent_numpy_math(
registry: Registry, alpha_id: str
) -> None:
panel = _build_panel()
result = registry.compute(alpha_id, panel)
expected = _expected_outputs(panel)[alpha_id]
assert list(result.columns) == _CODES
assert list(result.index) == list(_DATES)
# atol covers values that are zero up to fp noise: when a cell equals the
# row mean exactly (e.g. cfo/TA = [0.3, 0.2, 0.1]), its z-score is ~1e-16
# with a sign that depends on summation order (pandas vs numpy).
np.testing.assert_allclose(
result.to_numpy(dtype=np.float64),
expected.to_numpy(dtype=np.float64),
rtol=1e-9,
atol=1e-12,
equal_nan=True,
err_msg=f"{alpha_id} diverged from independent numpy expectation",
)
def test_accruals_sign_direction(registry: Registry) -> None:
"""Sloan 1996: higher (NI-CFO)/TA must score LOWER on every date."""
out = registry.compute("fund_accruals", _build_panel())
accruals = _build_panel()["fund:accruals"]
for date in _DATES:
# CCC has the lowest accruals, AAA the highest, on both dates.
assert accruals.loc[date, "AAA"] > accruals.loc[date, "BBB"]
assert out.loc[date, "CCC"] > out.loc[date, "BBB"] > out.loc[date, "AAA"]
def test_leverage_sign_direction(registry: Registry) -> None:
"""Low leverage scores high: sign inverted versus the raw ratio."""
out = registry.compute("fund_leverage", _build_panel())
leverage = _build_panel()["fund:leverage"]
for date in _DATES:
assert leverage.loc[date, "AAA"] > leverage.loc[date, "BBB"]
assert out.loc[date, "CCC"] > out.loc[date, "BBB"] > out.loc[date, "AAA"]
def test_roa_sign_direction(registry: Registry) -> None:
"""Positive factor: higher ROA scores higher (no inversion)."""
out = registry.compute("fund_roa", _build_panel())
for date in _DATES:
assert out.loc[date, "AAA"] > out.loc[date, "BBB"] > out.loc[date, "CCC"]
# ------------------------------------------------------------ (c) NaN contract
@pytest.mark.parametrize("alpha_id", sorted(_BLANKED_DEPENDENCY))
def test_blank_dependency_bar_yields_all_nan(
registry: Registry, alpha_id: str
) -> None:
"""Blanking a declared dependency on one bar must NaN that whole bar.
The registry masks output wherever a declared dependency is missing
(#1377), and zscore/safe_div propagate NaN — the second date must come
back all-NaN while the first date is untouched.
"""
panel = _build_panel()
panel[_BLANKED_DEPENDENCY[alpha_id]].loc[_DATES[1]] = np.nan
out = registry.compute(alpha_id, panel)
assert out.loc[_DATES[1]].isna().all(), (
f"{alpha_id}: emitted a value on a bar whose declared dependency was NaN"
)
assert out.loc[_DATES[0]].notna().all()
# --------------------------------------------------- (d) zero denominator
@pytest.mark.parametrize("alpha_id", sorted(_ZERO_DENOMINATOR))
def test_zero_denominator_is_nan_not_inf(alpha_id: str) -> None:
"""safe_div contract: exact-zero denominator -> NaN, never +/-inf."""
stem = _EXPECTED_META[alpha_id][2]
module = _load_module(stem)
panel = _build_panel()
panel[_ZERO_DENOMINATOR[alpha_id]].loc[_DATES[0], "BBB"] = 0.0
out = module.compute(panel)
arr = out.to_numpy(dtype=np.float64, na_value=np.nan)
assert np.isnan(out.loc[_DATES[0], "BBB"])
assert not np.isinf(arr).any()
# The other two symbols on that date still score against each other.
assert out.loc[_DATES[0], ["AAA", "CCC"]].notna().all()
assert out.loc[_DATES[1]].notna().all()
agent/tests/factors/test_fundamental_valuation_samples.py
+361
View File
@@ -0,0 +1,361 @@
"""Registration + numeric regression for the three Tier-2 CN valuation factors.
Covers ``fund_pb``, ``fund_ps_ttm`` and ``fund_dividend_yield`` in
``zoo/fundamental/`` (research/fundamental_factor_expansion_proposal.md §4
Tier 2 #11/#12/#13), plus the ``val:`` namespace plumbing: the AlphaMeta
gate, the runner injection through a mocked tushare ``daily_basic`` API, the
whitelist rejection and the NaN-on-missing-data contract.
Assertions are membership-based, never an exact registry total: the zoo grows
concurrently. Expected values are recomputed here with an independent numpy
formula (mean / sample-std z-score) rather than reusing ``src.factors.base``
operators, so a regression in the shared operator does not silently validate
itself.
"""
from __future__ import annotations
import importlib
import sys
from types import SimpleNamespace
from typing import Any
import numpy as np
import pandas as pd
import pytest
from backtest import runner
from src.factors.registry import AlphaMeta, Registry
_DATES = pd.to_datetime(["2024-01-01", "2024-01-02"])
_CODES = ["AAA", "BBB", "CCC"]
# alpha_id -> (theme, columns_required, module stem under zoo/fundamental/)
_EXPECTED_META: dict[str, tuple[list[str], list[str], str]] = {
"fund_pb": (["value"], ["val:pb"], "pb"),
"fund_ps_ttm": (["value"], ["val:ps_ttm"], "ps_ttm"),
"fund_dividend_yield": (["value"], ["val:dv_ttm"], "dividend_yield"),
}
# One declared dependency per alpha, blanked wholesale in the NaN-contract test.
_BLANKED_DEPENDENCY: dict[str, str] = {
"fund_pb": "val:pb",
"fund_ps_ttm": "val:ps_ttm",
"fund_dividend_yield": "val:dv_ttm",
}
@pytest.fixture(scope="module")
def registry() -> Registry:
return Registry()
def _frame(rows: list[list[float]]) -> pd.DataFrame:
return pd.DataFrame(rows, index=_DATES, columns=_CODES, dtype="float64")
def _build_panel() -> dict[str, pd.DataFrame]:
"""Hand-built 2-date x 3-symbol panel; every row has distinct values."""
return {
# PB: CCC cheapest, AAA most expensive on both dates.
"val:pb": _frame([[3.0, 1.5, 0.5], [2.8, 1.6, 0.6]]),
"val:ps_ttm": _frame([[4.0, 2.0, 1.0], [3.6, 2.2, 0.8]]),
# dv_ttm in percent units, as tushare daily_basic reports it.
"val:dv_ttm": _frame([[0.5, 2.0, 4.5], [0.6, 2.1, 4.4]]),
}
def _numpy_zscore(values: np.ndarray) -> np.ndarray:
"""Independent per-row z-score: (x - mean) / sample-std, NaN-preserving."""
mean = np.nanmean(values, axis=1, keepdims=True)
std = np.nanstd(values, axis=1, ddof=1, keepdims=True)
with np.errstate(invalid="ignore", divide="ignore"):
return (values - mean) / std
def _z(frame: pd.DataFrame) -> pd.DataFrame:
return pd.DataFrame(
_numpy_zscore(frame.to_numpy(dtype=np.float64)),
index=frame.index,
columns=frame.columns,
)
def _expected_outputs(panel: dict[str, pd.DataFrame]) -> dict[str, pd.DataFrame]:
"""Expected output frames per alpha, derived from the panel by numpy math."""
return {
"fund_pb": -_z(panel["val:pb"]),
"fund_ps_ttm": -_z(panel["val:ps_ttm"]),
"fund_dividend_yield": _z(panel["val:dv_ttm"]),
}
def _load_module(stem: str):
return importlib.import_module(f"src.factors.zoo.fundamental.{stem}")
# ------------------------------------------------------- (a) gate + registry
def test_alpha_meta_gate_accepts_val_columns() -> None:
meta = AlphaMeta(
id="test_val_alpha",
theme=["value"],
formula_latex="x",
columns_required=["val:pb"],
universe=["equity_cn"],
frequency=["1d"],
decay_horizon=252,
min_warmup_bars=1,
)
assert meta.columns_required == ["val:pb"]
@pytest.mark.parametrize("alpha_id", sorted(_EXPECTED_META))
def test_registry_loads_factor_with_expected_meta(
registry: Registry, alpha_id: str
) -> None:
theme, columns, stem = _EXPECTED_META[alpha_id]
alpha = registry.get(alpha_id)
assert alpha.zoo == "fundamental"
assert alpha.module_path == f"src.factors.zoo.fundamental.{stem}"
assert alpha.meta["theme"] == theme
assert alpha.meta["columns_required"] == columns
assert alpha.meta["universe"] == ["equity_cn"]
assert alpha.meta["frequency"] == ["1d"]
assert alpha.meta["decay_horizon"] == 252
assert alpha.meta["min_warmup_bars"] == 1
assert alpha.meta["notes"].strip()
# ------------------------------------------------- (b) numeric correctness
@pytest.mark.parametrize("alpha_id", sorted(_EXPECTED_META))
def test_compute_matches_independent_numpy_math(
registry: Registry, alpha_id: str
) -> None:
panel = _build_panel()
result = registry.compute(alpha_id, panel)
expected = _expected_outputs(panel)[alpha_id]
assert list(result.columns) == _CODES
assert list(result.index) == list(_DATES)
np.testing.assert_allclose(
result.to_numpy(dtype=np.float64),
expected.to_numpy(dtype=np.float64),
rtol=1e-9,
atol=1e-12,
equal_nan=True,
err_msg=f"{alpha_id} diverged from independent numpy expectation",
)
def test_pb_sign_direction(registry: Registry) -> None:
"""Low PB scores high: cheaper-on-book names rank above expensive ones."""
out = registry.compute("fund_pb", _build_panel())
pb = _build_panel()["val:pb"]
for date in _DATES:
assert pb.loc[date, "AAA"] > pb.loc[date, "BBB"] > pb.loc[date, "CCC"]
assert out.loc[date, "CCC"] > out.loc[date, "BBB"] > out.loc[date, "AAA"]
def test_dividend_yield_sign_direction(registry: Registry) -> None:
"""High dv_ttm scores high (no inversion)."""
out = registry.compute("fund_dividend_yield", _build_panel())
for date in _DATES:
assert out.loc[date, "CCC"] > out.loc[date, "BBB"] > out.loc[date, "AAA"]
# ------------------------------------------------------------ (c) NaN contract
@pytest.mark.parametrize("alpha_id", sorted(_BLANKED_DEPENDENCY))
def test_blank_dependency_bar_yields_all_nan(
registry: Registry, alpha_id: str
) -> None:
"""Blanking a declared val: dependency on one bar must NaN that whole bar.
The registry masks output wherever a declared dependency is missing
(#1377), and zscore propagates NaN — the second date must come back
all-NaN while the first date is untouched.
"""
panel = _build_panel()
panel[_BLANKED_DEPENDENCY[alpha_id]].loc[_DATES[1]] = np.nan
out = registry.compute(alpha_id, panel)
assert out.loc[_DATES[1]].isna().all(), (
f"{alpha_id}: emitted a value on a bar whose declared dependency was NaN"
)
assert out.loc[_DATES[0]].notna().all()
# ------------------------------------------- (d) no inf on degenerate inputs
@pytest.mark.parametrize("alpha_id", sorted(_EXPECTED_META))
def test_extreme_and_zero_values_never_produce_inf(
registry: Registry, alpha_id: str
) -> None:
"""Invalid multiples stay NaN; zero dividends remain valid observations."""
panel = _build_panel()
dep = _BLANKED_DEPENDENCY[alpha_id]
panel[dep].loc[_DATES[0]] = [0.0, 0.0, 0.0]
panel[dep].loc[_DATES[1]] = [0.0, -1.5, 1e12]
if alpha_id in {"fund_pb", "fund_ps_ttm"}:
from src.factors.registry import RegistryError
with pytest.raises(RegistryError, match=">95% NaN"):
registry.compute(alpha_id, panel)
out = _load_module(_EXPECTED_META[alpha_id][2]).compute(panel)
assert out.isna().all().all() # one valid multiple cannot form a cross-section
else:
out = registry.compute(alpha_id, panel)
assert out.loc[_DATES[1]].notna().all()
assert not np.isinf(out.to_numpy(dtype=np.float64)).any()
assert out.loc[_DATES[0]].isna().all(), "constant row must z-score to NaN"
# ------------------------------------------------- runner injection (mocked)
class _FakeDailyBasicApi:
"""Mimics ``pro.daily_basic``: serves only 000001.SZ, one date short."""
def __init__(self) -> None:
self.calls: list[dict[str, Any]] = []
def daily_basic(self, **kwargs: Any) -> pd.DataFrame:
self.calls.append(kwargs)
if kwargs["ts_code"] != "000001.SZ":
return pd.DataFrame()
return pd.DataFrame(
{
"ts_code": ["000001.SZ", "000001.SZ"],
"trade_date": ["20240102", "20240103"],
"pb": [1.1, 1.2],
"ps_ttm": [2.1, 2.2],
"dv_ttm": [3.1, 3.2],
}
)
@pytest.fixture
def fake_tushare(monkeypatch: pytest.MonkeyPatch) -> _FakeDailyBasicApi:
api = _FakeDailyBasicApi()
monkeypatch.setenv("TUSHARE_TOKEN", "ts-test-token")
monkeypatch.setitem(
sys.modules, "tushare", SimpleNamespace(pro_api=lambda token="": api)
)
return api
def _cn_data_map() -> dict[str, pd.DataFrame]:
index = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04"])
frame = lambda: pd.DataFrame( # noqa: E731
{
"open": [10.0, 10.5, 10.2],
"high": [10.6, 10.7, 10.8],
"low": [9.9, 10.1, 10.0],
"close": [10.1, 10.3, 10.4],
"volume": [1000, 1100, 1200],
},
index=index,
)
return {"000001.SZ": frame(), "600000.SH": frame()}
def test_runner_injects_declared_val_columns_for_cn_symbols(
fake_tushare: _FakeDailyBasicApi,
) -> None:
index = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04"])
result = runner._maybe_inject_fundamentals_for_factor_panel(
_cn_data_map(),
{
"start_date": "2024-01-02",
"end_date": "2024-01-04",
"selected_factors": [{"columns_required": ["close", "val:pb"]}],
},
)
injected = result["000001.SZ"]["val:pb"]
assert injected.loc[index[0]] == pytest.approx(1.1)
assert injected.loc[index[1]] == pytest.approx(1.2)
# 2024-01-04 was not served by daily_basic: stays NaN, never fabricated.
assert pd.isna(injected.loc[index[2]])
# 600000.SH was not served either: whole column NaN, still injected.
assert result["600000.SH"]["val:pb"].isna().all()
assert [call["ts_code"] for call in fake_tushare.calls] == [
"000001.SZ",
"600000.SH",
]
assert fake_tushare.calls[0]["fields"] == "ts_code,trade_date,pb"
def test_runner_val_injection_skips_non_cn_symbols(
fake_tushare: _FakeDailyBasicApi,
) -> None:
index = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04"])
data_map = {
"AAPL.US": _cn_data_map()["000001.SZ"],
"00700.HK": _cn_data_map()["000001.SZ"],
}
result = runner._maybe_inject_fundamentals_for_factor_panel(
data_map,
{
"start_date": "2024-01-02",
"end_date": "2024-01-04",
"selected_factors": [{"columns_required": ["val:dv_ttm"]}],
},
)
assert fake_tushare.calls == []
assert result["AAPL.US"]["val:dv_ttm"].reindex(index).isna().all()
assert result["00700.HK"]["val:dv_ttm"].reindex(index).isna().all()
def test_runner_rejects_val_field_outside_whitelist() -> None:
with pytest.raises(ValueError, match="unsupported val: field"):
runner._maybe_inject_fundamentals_for_factor_panel(
_cn_data_map(),
{
"start_date": "2024-01-02",
"end_date": "2024-01-04",
"selected_factors": [{"columns_required": ["val:total_mv"]}],
},
)
def test_val_whitelist_covers_exactly_the_daily_basic_fields() -> None:
from backtest.loaders.valuation_loader import VALUATION_FIELDS
assert VALUATION_FIELDS == ("pe", "pe_ttm", "pb", "ps_ttm", "dv_ttm")
def test_missing_token_degrades_to_nan_frames(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""No TUSHARE_TOKEN: the run survives on NaN frames instead of crashing."""
monkeypatch.setenv("TUSHARE_TOKEN", "")
index = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04"])
panel = {"close": pd.DataFrame({"000001.SZ": [10.1, 10.3, 10.4]}, index=index)}
runner._inject_valuation_panel(
panel,
symbols=["000001.SZ"],
val_columns=["val:pb"],
start="2024-01-02",
end="2024-01-04",
)
assert panel["val:pb"].isna().all().all()
assert list(panel["val:pb"].index) == list(index)
@pytest.mark.parametrize('stem,column', [('pb', 'pb'), ('ps_ttm', 'ps_ttm')])
def test_nonpositive_multiples_are_not_ranked_as_cheap(stem, column):
frame = pd.DataFrame([[-1., 0., 1., 2., 3.]], columns=list('ABCDE'))
actual = _load_module(stem).compute({f'val:{column}': frame})
assert actual[['A', 'B']].isna().all().all()
np.testing.assert_allclose(actual[['C', 'D', 'E']].to_numpy(), [[1., 0., -1.]])
agent/tests/factors/test_missing_bar_fabrication.py
+29 -2
View File
@@ -93,8 +93,35 @@ def _base() -> dict[str, np.ndarray]:
volume = rng.integers(1000, 50000, (_T, _N)).astype(float) volume = rng.integers(1000, 50000, (_T, _N)).astype(float)
vwap = (high + low + 2 * close) / 4.0 vwap = (high + low + 2 * close) / 4.0
base = {"close": close, "open": open_, "high": high, "low": low, "volume": volume, "vwap": vwap} base = {"close": close, "open": open_, "high": high, "low": low, "volume": volume, "vwap": vwap}
# Drawn last so the price arrays above stay what they were. # Drawn last so the price arrays above stay what they were. The original
for name in ("fund:asset_growth", "fund:net_income", "fund:shares_diluted", "fund:gross_profitability", "fund:roe"): # five keep their draw order (identical values); the remaining schema
# fields are appended so every fundamental alpha's declared dependencies
# are present and the oracles actually exercise it instead of SkipAlpha.
for name in (
"fund:asset_growth",
"fund:net_income",
"fund:shares_diluted",
"fund:gross_profitability",
"fund:roe",
"fund:revenue",
"fund:cogs",
"fund:gross_profit",
"fund:operating_income",
"fund:total_assets",
"fund:total_equity",
"fund:total_debt",
"fund:cash",
"fund:cfo",
"fund:capex",
"fund:roa",
"fund:accruals",
"fund:leverage",
"fund:revenue_growth",
"fund:earnings_growth",
"val:pb",
"val:ps_ttm",
"val:dv_ttm",
):
base[name] = rng.integers(32, 96, (_T, _N)) / 64.0 base[name] = rng.integers(32, 96, (_T, _N)) / 64.0
return base return base
agent/tests/grounding_parameter_helpers.py
+6
View File
@@ -0,0 +1,6 @@
"""Explicit chosen coefficients for derivation fixtures."""
def declared_parameter(value: str, label: str = "Chosen coefficient") -> str:
"""Show a parameter to the reader and declare the same value and label."""
return f"\n\n{label}: {value}.\n\n```figures\n{value} | count | {label}\n```\n"
agent/tests/quantlib/test_drawdown_revisited_peak.py
+60
View File
@@ -0,0 +1,60 @@
"""A recovered high-water mark starts a new underwater episode."""
from __future__ import annotations
import json
import pandas as pd
import pytest
from src.quantlib.risk import max_drawdown_analysis
from src.tools.quantlib_tool import QuantlibCallTool
@pytest.mark.parametrize(
"values,peak,trough,recovery",
[
([100, 90, 100, 80, 100], 2, 3, 4),
([100, 100, 80, 100], 1, 2, 3),
([100, 80, 100, 70], 2, 3, None),
([90, 100, 80, 110], 1, 2, 3),
([100, 70, 100, 80], 0, 1, 2),
([100, 100, 100], 0, 0, 0),
],
)
def test_episode_starts_at_last_high_water_mark_before_worst_trough(
values, peak, trough, recovery
):
result = max_drawdown_analysis(values)
assert result["peak_date"] == peak
assert result["trough_date"] == trough
assert result["recovery_date"] == recovery
assert result["peak_to_trough_periods"] == trough - peak
assert result["max_drawdown"] == pytest.approx(1 - values[trough] / values[peak])
def test_calendar_underwater_duration_excludes_an_already_recovered_episode():
equity = pd.Series(
[100, 90, 100, 80, 100],
index=pd.to_datetime(
["2025-01-01", "2025-01-02", "2025-01-10", "2025-01-12", "2025-01-15"]
),
)
result = max_drawdown_analysis(equity)
assert result["peak_date"] == pd.Timestamp("2025-01-10")
assert result["underwater_days"] == 2
assert result["recovery_days"] == 3
def test_registered_quantlib_tool_reports_the_current_drawdown_episode():
result = json.loads(
QuantlibCallTool().execute(
action="call",
module="risk",
function="max_drawdown_analysis",
kwargs={"equity": [100, 90, 100, 80, 100]},
)
)
assert result["ok"] is True
assert result["result"]["peak_date"] == 2
assert result["result"]["peak_to_trough_periods"] == 1
agent/tests/quantlib/valuation/test_threestatement.py
+45
View File
@@ -374,6 +374,51 @@ def test_project_three_statement_raises_on_broken_opening_balance_sheet():
assert excinfo.value.residual == pytest.approx(-1.0, abs=TOL) assert excinfo.value.residual == pytest.approx(-1.0, abs=TOL)
def test_balance_tolerance_is_honored_for_projected_periods():
"""A caller-widened balance_tolerance must apply to every projected period, not just the opening.
The rollforwards preserve the residual exactly: expanding assets - liabilities - equity for period t
with cash_end = prior_cash + NI + D&A - ΔNWC - capex + draw - repay - div, ppe_end = prior_ppe + capex - D&A,
RE_end = prior_RE + NI - div, debt_end = prior_debt + draw - repay, every term except nwc_end - ΔNWC = prior_nwc
cancels, leaving residual_t ≡ residual_0. So a residual the caller accepts at period 0 is transported unchanged
into period 1 and then rejected if _project_period ignores the tolerance.
This test builds an opening sheet with a known 0.5 residual (residual/assets = 9.09e-4), which passes at
tolerance=1e-3 but would fail at the default 1e-9. Without the fix, period 1 raises BalanceSheetError even
though the residual is exactly conserved.
"""
opening_with_residual = {
"revenue": 1000.0,
"cash": 100.0,
"net_working_capital": 50.0,
"ppe": 400.0,
"revolver_balance": 200.0,
"paid_in_capital": 200.0,
"retained_earnings": 149.5, # Makes residual = 100+50+400 - (200+200+149.5) = 0.5
}
drivers_one_period = {
"revenue_growth": [0.05],
"gross_margin": [0.30],
"opex_pct_revenue": [0.20],
"capex_pct_revenue": [0.06],
"nwc_pct_revenue": [0.10],
"tax_rate": [0.25],
"dividend_payout_ratio": [0.20],
"depreciation_amortization": [30.0],
"interest_rate": [0.08],
"minimum_cash": [50.0],
}
# At tolerance=1e-3, both opening (residual/assets = 0.5/550 ≈ 9.09e-4) and period 1 should pass.
result = project_three_statement(opening_with_residual, drivers_one_period, balance_tolerance=1e-3)
assert len(result.periods) == 1
# Verify the residual is exactly conserved in period 1.
bs1 = result.periods[0].balance_sheet
assets1 = bs1.cash + bs1.net_working_capital + bs1.ppe
liab_eq1 = bs1.revolver_balance + bs1.paid_in_capital + bs1.retained_earnings
residual1 = assets1 - liab_eq1
assert residual1 == pytest.approx(0.5, abs=1e-9)
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
# Interest/revolver circularity: convergent (multi-iteration) and divergent. # Interest/revolver circularity: convergent (multi-iteration) and divergent.
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
agent/tests/test_agent_grounding.py
+6 -6
View File
@@ -1522,9 +1522,9 @@ def test_price_validation_ignores_symbol_date_and_quantity_digits(tmp_path: Path
for draft in ( for draft in (
"000543.SZ 截至 8 月 3 日收盘价 8.20 CNY（source: tencent）", "000543.SZ 截至 8 月 3 日收盘价 8.20 CNY（source: tencent）",
"000543.SZ 买入价 8.20 CNY（100 股成本 820 CNY；source: tencent）\n\n" "000543.SZ 买入价 8.20 CNY（股数100，成本 820 CNY；source: tencent）\n\n"
"```figures\n8.20 | observed | close | prices\n" "```figures\n8.20 | observed | close | prices\n"
"820 | derived | 100 × 8.20 | prices\n```", "820 | derived | 100 × 8.20 | prices\n100 | count | 股数\n```",
"000543.SZ 建议持有 1–4 周，买入价 8.20 CNY（source: tencent）", "000543.SZ 建议持有 1–4 周，买入价 8.20 CNY（source: tencent）",
): ):
result = ledger.validate_final_answer(draft) result = ledger.validate_final_answer(draft)
@@ -1729,9 +1729,9 @@ def test_screening_run_reaches_a_final_answer_through_the_agent_loop(
assert market.calls == 1 assert market.calls == 1
validation = agent._grounding.validate_final_answer( validation = agent._grounding.validate_final_answer(
"000543.SZ 买入价 8.20 CNY（100 股成本 820 CNY；source: tencent）\n\n" "000543.SZ 买入价 8.20 CNY（股数100，成本 820 CNY；source: tencent）\n\n"
"```figures\n8.20 | observed | close | prices\n" "```figures\n8.20 | observed | close | prices\n"
"820 | derived | 100 × 8.20 | prices\n```" "820 | derived | 100 × 8.20 | prices\n100 | count | 股数\n```"
) )
assert validation.valid is True, validation.issues assert validation.valid is True, validation.issues
@@ -1825,9 +1825,9 @@ def test_plan_level_alone_reaches_a_valid_answer(tmp_path: Path) -> None:
assert inside_range.valid is True, inside_range.issues assert inside_range.valid is True, inside_range.issues
derived_outside = ledger.validate_final_answer( derived_outside = ledger.validate_final_answer(
"000543.SZ 收盘价 8.20 CNY（source: tencent）。目标位 9.02。\n\n" "000543.SZ 收盘价 8.20 CNY（source: tencent）。目标位 9.02。选定系数110%。\n\n"
"```figures\n8.20 | observed | close | prices\n" "```figures\n8.20 | observed | close | prices\n"
"9.02 | proposed | 8.20 × 1.10 | prices\n```" "9.02 | proposed | 8.20 × 1.10 | prices\n110% | count | 选定系数\n```"
) )
assert derived_outside.valid is True, derived_outside.issues assert derived_outside.valid is True, derived_outside.issues
agent/tests/test_alpha_bench_pit_lag.py
+120
View File
@@ -0,0 +1,120 @@
"""Regression: after-close SEC filings must not leak into the IC one bar early.
SEC ``filed`` has no acceptance time. A filing on a trading date may be
published after its close. Same-day IC alignment can therefore include a
market reaction that preceded availability of the factor.
The known-answer check below plants a factor equal to the next day's return
(the strongest possible leak). Before the fix the bench reports a near-perfect
IC; with the filed-on-t value made first visible on bar t+1 the IC collapses
to ~0. These tests are offline: the loader is mocked, exactly like
``test_fundamental_factor_dependencies_are_loaded_at_filing_time``.
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import Mock
import numpy as np
import pandas as pd
from src.factors.factor_analysis_core import compute_ic_series
from src.tools.alpha_bench_tool import _compute_forward_returns, _prepare_bench_panel
_DATES = pd.to_datetime(["2026-01-05", "2026-01-06", "2026-01-07", "2026-01-08", "2026-01-09", "2026-01-12"])
def _close_panel() -> pd.DataFrame:
"""Eight deterministic closes; cross-sectional IC needs >= 5 symbols.
The series is the seed-73 draw from ``default_rng(73).normal(0, 0.02)``
compounded from 100 — chosen so consecutive-day cross-sectional return
ranks are uncorrelated (|lag-IC| <= 0.12 every day), keeping the
known-answer test about the filing lag, not about the fixture.
"""
return pd.DataFrame(
{
"AAPL.US": [100.00, 97.81, 95.38, 98.83, 99.43, 96.99],
"MSFT.US": [100.00, 101.56, 103.82, 102.25, 99.69, 98.64],
"NVDA.US": [100.00, 99.24, 96.04, 92.59, 93.58, 91.74],
"AMZN.US": [100.00, 100.59, 97.40, 98.70, 99.25, 98.15],
"GOOG.US": [100.00, 100.67, 98.98, 99.41, 102.24, 104.99],
"META.US": [100.00, 96.11, 98.81, 99.49, 98.70, 100.09],
"TSLA.US": [100.00, 101.00, 102.87, 100.62, 101.13, 101.96],
"AMD.US": [100.00, 98.07, 97.74, 98.84, 100.68, 101.34],
},
index=_DATES,
)
def _leak_frame(close: pd.DataFrame) -> pd.DataFrame:
"""A fundamental whose value on bar t is exactly the t->t+1 return.
This is what a same-day ``filed`` value looks like when the filing is
accepted after the close: the value encodes information the market has
not priced in yet at close(t).
"""
return close.pct_change(fill_method=None).shift(-1)
def _registry(columns: list[str]) -> SimpleNamespace:
return SimpleNamespace(get=lambda _aid: SimpleNamespace(meta={"columns_required": columns}))
def _ic(factor: pd.DataFrame, close: pd.DataFrame) -> float:
"""Mean cross-sectional Spearman IC between the factor and forward returns."""
returns = _compute_forward_returns({"close": close})
ic_series = compute_ic_series(factor, returns)
assert not ic_series.empty
return float(ic_series.mean())
def test_unshifted_pit_panel_would_report_a_perfect_ic() -> None:
"""The known-answer sanity check: without the lag the leak is real.
Guards against someone removing the shift in ``_prepare_bench_panel``
without noticing it re-opens issue #1790.
"""
close = _close_panel()
ic = _ic(_leak_frame(close), close)
assert abs(ic) > 0.9
def test_pit_panel_shifts_filing_one_bar_later(monkeypatch) -> None:
"""A filed-on-t value must first appear on bar t+1 in the bench panel."""
close = _close_panel()
# What the loader hands back for a filing anchored on bar 2: NaN before
# the filing, forward-filled after it (fundamentals_loader L194 ffills).
frame = pd.DataFrame(np.nan, index=close.index, columns=close.columns)
frame.iloc[2] = 5.0
frame = frame.ffill()
loader = Mock(return_value={"net_income": frame})
monkeypatch.setattr("backtest.loaders.fundamentals_loader.load_fundamental_panel", loader)
result = _prepare_bench_panel(_registry(["close", "fund:net_income"]), ["aid"], {"close": close}, "sp500", "2026-2026")
column = result["fund:net_income"]
# Before the filing bar and on it: NaN. From t+1 on: the filed value.
assert column.iloc[2].isna().all()
assert (column.iloc[3:] == 5.0).all().all()
# The lag is recorded so downstream consumers can see the semantics.
assert result["_meta"]["fundamental_pit_lag_bars"] == 1
def test_after_close_filing_does_not_leak_into_the_ic(monkeypatch) -> None:
"""Known-answer: a factor equal to next-day returns must not produce IC.
This is the reproduce step from the issue, planted offline: the loader
returns a value anchored on the filed date (as for an after-close
filing); after the fix the bench must not report a signal.
"""
close = _close_panel()
loader = Mock(return_value={"net_income": _leak_frame(close)})
monkeypatch.setattr("backtest.loaders.fundamentals_loader.load_fundamental_panel", loader)
result = _prepare_bench_panel(_registry(["close", "fund:net_income"]), ["aid"], {"close": close}, "sp500", "2026-2026")
assert abs(_ic(result["fund:net_income"], close)) < 0.1
agent/tests/test_attribution_liquidated_last_row.py
+113
View File
@@ -0,0 +1,113 @@
"""A run's final positions row is its liquidation, not its book.
Every run exits on the last bar (``end_of_backtest``), so ``positions.csv``
ends with an all-zero row. Reading only that row made Brinson report a
portfolio that had been invested for years as having "all position weights
zero" -- a one-row problem that nulled the whole section.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from src.api.attribution_core import _attribution_load_position_weights
SYMBOLS = ["AAA.HK", "BBB.HK"]
def _write_positions(run_dir: Path, rows: list[list[Any]]) -> None:
"""Write ``timestamp`` + per-symbol weight columns, the real artifact layout."""
path = run_dir / "artifacts" / "positions.csv"
path.parent.mkdir(parents=True, exist_ok=True)
lines = ["timestamp," + ",".join(SYMBOLS)]
lines += [",".join(str(cell) for cell in row) for row in rows]
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
def test_end_of_backtest_flat_row_is_skipped(tmp_path: Path) -> None:
"""The held row before the liquidation is the one that describes the book."""
_write_positions(
tmp_path,
[
["2026-09-23", 0.40, 0.35],
["2026-09-24", 0.42, 0.33],
["2026-09-25", 0.0, 0.0], # end-of-backtest liquidation
],
)
notes: list[str] = []
weights = _attribution_load_position_weights(tmp_path, notes)
assert weights == {"AAA.HK": 0.42, "BBB.HK": 0.33}
assert any("end-of-backtest liquidation (2026-09-24)" in note for note in notes)
def test_a_held_final_row_is_used_unchanged(tmp_path: Path) -> None:
"""A run still holding on its final bar must read exactly that row."""
_write_positions(
tmp_path,
[
["2026-09-24", 0.42, 0.33],
["2026-09-25", 0.50, 0.20],
],
)
notes: list[str] = []
weights = _attribution_load_position_weights(tmp_path, notes)
assert weights == {"AAA.HK": 0.50, "BBB.HK": 0.20}
assert notes == []
def test_a_book_that_never_held_still_reads_flat(tmp_path: Path) -> None:
"""True negative: all-zero throughout must stay all-zero.
This is the case the downstream zero-check was written for, and the fix
must not paper over it.
"""
_write_positions(
tmp_path,
[
["2026-09-23", 0.0, 0.0],
["2026-09-24", 0.0, 0.0],
["2026-09-25", 0.0, 0.0],
],
)
notes: list[str] = []
weights = _attribution_load_position_weights(tmp_path, notes)
assert weights == {"AAA.HK": 0.0, "BBB.HK": 0.0}
assert not any("end-of-backtest" in note for note in notes)
def test_a_book_flat_before_the_close_is_not_replaced_by_an_older_one(
tmp_path: Path,
) -> None:
"""Only the liquidation row is stepped over, never an earlier exit."""
_write_positions(
tmp_path,
[
["2026-09-22", 0.40, 0.35],
["2026-09-23", 0.0, 0.0], # the strategy itself exited here
["2026-09-24", 0.0, 0.0],
["2026-09-25", 0.0, 0.0], # end-of-backtest liquidation
],
)
notes: list[str] = []
weights = _attribution_load_position_weights(tmp_path, notes)
assert weights == {"AAA.HK": 0.0, "BBB.HK": 0.0}
assert not any("end-of-backtest" in note for note in notes)
def test_missing_artifact_still_reports_not_found(tmp_path: Path) -> None:
"""The other skip reasons must keep their own wording."""
notes: list[str] = []
weights = _attribution_load_position_weights(tmp_path, notes)
assert weights is None
assert notes == ["brinson attribution skipped: positions.csv not found"]
Loaded 100 of 143 files, more files were not shown because too many files have changed in this diff. Show more