* feat(bench): add horizon parameter for monthly-frequency IC revalidation
h-bar forward returns (close[t+h]/close[t]-1, no ffill — a NaN close never
bridges) sampled non-overlapping end-anchored, so the naive-iid t-stat and
the alive/dead thresholds stay honest without new statistics. Threaded
through run_bench / run_bench_strict / the alpha_bench tool + MCP wrapper /
CLI --horizon (argparse type + handler guard, exit 2 on bad input).
horizon=1 is byte-identical to the previous next-bar formula.
Follow-up #1 of issue #1789 (monthly-horizon revalidation of the 13
fundamental factors — the conventional reading for fundamental factors).
* fix(bench): exclude SEC-fundamental alphas off sp500 instead of failing the bench
--zoo fundamental --universe csi300 (the repro command printed in issue
#1789) hard-failed with 'Choose the S&P 500 universe' because the
validator rejects fund:* inputs on any non-sp500 universe, while the
mirror case (CN val:* alphas on sp500) skips cleanly per alpha. The
runners now split the selection up front, disclose the excluded ids in
the result envelope, and error only when nothing is left to bench; the
validator stays as the backstop for direct callers.
The CLI/tool JSON envelopes also state their own horizon,
forward_return_sampling and universe_excluded — a saved summary must
show the configuration its numbers came from.
* fix(bench): close review findings on horizon/exclusion disclosure
- multi-zoo CLI aggregation threads horizon / forward_return_sampling /
universe_excluded from the sub-runs into the aggregate envelope (a saved
all-zoos JSON now states its own horizon and reconciles its exclusions)
- alpha compare surfaces universe-excluded shortlist members as skipped
entries with the exclusion reason instead of silently dropping them
- horizon upper bound (1.._MAX_HORIZON) enforced consistently at the CLI
parser, the handler, and both runner entries BEFORE any panel load —
previously only the tool/MCP capped it and an unbounded CLI run fetched
the full panel before failing late
- non-int horizon can no longer TypeError in the entry schema construction
- categorise docstring notes the daily calibration of the |IC|>0.02 floor
at h>1; the sampling comment no longer overclaims (residual
autocorrelation from persistent factor exposures is not modeled)
- MCP wrapper uses _MAX_HORIZON instead of a hardcoded duplicate
Pinned by new tests: parser/handler/runner cap rejection (fetch not
called), multi-zoo disclosure aggregation, compare exclusion disclosure.
* feat(factors): industry-neutralized CN value alphas + PIT SW2021 sector tag
Three within-industry z-scored variants of the CN value factors
(fund_pb/fund_ps_ttm/fund_dividend_yield -> *_ind_neutral, zoo 475 -> 478):
raw A-share PB/PS/dy are heavily industry-confounded, so each name is
z-scored against its Shenwan L1 peers instead of the whole cross-section.
- industry_loader.py: tushare index_member_all (SW2021) membership with
in_date/out_date -> point-in-time labels per trading day; deterministic
overlap resolution; 2000-row truncation guard refetches per L1; degrades
to an unmapped frame on provider failure, never invents a label.
- csi300 bench attaches panel['sector'] outside the pickled price cache,
masked to the PIT membership cross-section, behind the same 0.9 coverage
gate the sp500 GICS tag uses; meta discloses sector_source/coverage.
- base.zscore_by_group: the z-scoring sibling of alpha101's _ind_neutralize
demean - no global fallback (an unlabelled cell stays NaN rather than
wearing an industry-neutral label it did not earn).
- README x7 + SKILL.md counts (incl. the stale 462 prose sites), i18n zh/en.
Follow-up #2 of issue #1789.
* fix(loaders): index_member_all takes no src param; fetch PIT history via is_new Y+N
Live-verified endpoint contract (pinned by test): index_member_all accepts
only the documented l1/l2/l3_code, ts_code and is_new filters — an
undocumented src param silently returns an empty frame (HTTP 200, 0 rows),
which degraded the first csi300 sector attach to coverage 0.0. src now goes
to index_classify (truncation fallback) only.
The server also defaults is_new to 'Y' (current members, no out_date rows),
so the historical half ('N', ~2000 rows with out_date back to the 1990s) is
fetched separately and concatenated — without it every label would be
today's classification and the loader's PIT claim false.
Live smoke: 350/350 csi300 PIT constituents covered (100%), 0.9 gate passes.
* docs(factors): surface the SW2021 taxonomy-retroactivity caveat; pin cache immutability
Review follow-ups: the loader docstring disclosed that SW2021
classifications are applied retroactively, but no user-facing surface
did — one sentence added to each ind-neutral alpha's notes, the i18n
en/zh entries and SKILL.md (membership is point-in-time; the taxonomy
version itself is not). The sector-attach test now pins that the
original pickled panel dict is never mutated by the attach, and the
sector_coverage meta semantics (labelled-somewhere fraction, not
per-date) are documented at the attach site.
* fix(factors): align ind-neutral PB/PS with the upstream nonpositive-multiple guard
The maintenance batch (d3c76819) added a guard to fund_pb/fund_ps_ttm:
nonpositive multiples are excluded rather than ranked as cheap. The
industry-neutral variants follow suit — the within-group z-score sees
the same values.where(values > 0) pre-filter, the guard sentence leads
the notes/i18n entries, and the upstream parametrized test is mirrored
for the neutral stems.
Fix live option guards, grounding operand evidence, source-bound benchmarks, trade artifacts, attribution, drawdowns, valuation tolerances, and SEC point-in-time factor inputs.
Add Brazil research data and market support, 13 fundamental and valuation factors, channel progress forwarding, and opt-in ZimaOS templates. Update seven README News sections with three visible dates.
Validation: 17,635 backend tests passed; 1,084 final targeted tests and 103 frontend tests passed. Frontend build, package build, documentation checks, and CI gates passed.
Preserve scientific-notation amounts and add regression coverage for the five review findings. Update README News after successful backend, frontend, and contract checks.
Signed-off-by: Haozhe Wu <haozhe_wu@connect.hku.hk>
TDX quote servers stopped answering the client protocol mootdx/tdxpy
speak (2026-07, mootdx/mootdx#157): every kline category, market and
server returns empty, realtime quotes return zero rows, and get_k_data
dies on a KeyError inside mootdx. Upstream has been dormant since 2024,
and the weekly health lane has been red on mootdx since it started.
Each auto-mode fetch still spent ~9.5s inside mootdx before falling
through to eastmoney, so the dead slot was pure latency tax on every
A-share call.
Drop mootdx from FALLBACK_CHAINS["a_share"] and from the canary set
(tracked under EXCLUDED_PUBLIC_SOURCES with the upstream issue as the
reason). The loader stays registered: explicit source="mootdx" calls
keep working if a compatible fork ever restores the protocol. Docs
(en/zh/ja/ko/es/ar/id), tool schema hints, env examples and the
data-routing/mootdx skills updated to match.
Closes#1729
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Wait for actual session terminal events, cancel in-flight analysis on runner stop, and finish scheduler teardown across startup, concurrent stops and API shutdown. Correct heartbeat status units and remove stopped sentinels.
Isolate broker job stores with legacy migration and complete atomic writes under concurrency. Skip malformed emergency-close symbols and overflowing quantities, and make capped grounding feedback explicit about omitted findings.
Validation: 16,817 backend tests passed (190 skipped), 38 frontend status tests passed, and changed Python syntax/lint passed. Update seven README News entries.
The doc guards require every shipped connector to be named in all seven
README broker tables and paragraphs, the pinned broker count to match the
registry, and every portfolio-eligible connector to carry an explicit
compatibility tier.
- bump EXPECTED_BROKERS 18 -> 19 and every broker badge
- add an mStock row to each broker table (REST Type B, read-only live,
no paper account) and name it in the no-paper paragraph
- register mstock as experimental in the portfolio compatibility map
(Type B reply shapes stay unverified until the HTTP read mapping lands)
The remaining test_readme_counts failures are pre-existing on main (the
southbound tool batch updated counts without touching the READMEs) and
are unchanged by this patch.
Signed-off-by: Jackzigen <jackzigen@users.noreply.github.com>
Goal: give MCP clients token-cheap structured access to backtest run
artifacts. read_file returns raw CSV text and truncates mid-file; this
tool parses the artifact once and serves typed JSON pages a client can
page through or chart directly. Implements #1645.
Design: the run_card.json artifacts manifest (path/sha256/size_bytes per
file) is the trust anchor. Friendly aliases (equity, trades, metrics,
positions, target_positions, ohlcv:<CODE>, run_card) resolve by a fixed
mapping and work without a card; any other value is treated as a
run_dir-relative path accepted only when the manifest lists it, so a new
artifact kind becomes readable with zero code change. Manifest entries
are themselves validated — no absolute paths, no drive prefixes, no '..'
segments under either separator style — and every resolution is
re-checked with resolve() + is_relative_to(run_root), so a tampered card
can never point the reader outside the run directory. Three formats:
rows (lossless offset paging with honest truncated/next_offset),
downsample (equal-stride, first+last pinned), meta (columns/total_rows/
size_bytes, plus the manifest sha256 and manifest_size_bytes when
listed). Envelopes stay within a 120K-character budget by shrinking
whole records following _result_paging's proportional-shrink pattern —
never mid-record, never broken JSON.
Contract: purely additive read-only surface (75th MCP tool, 108th
registry tool); errors return {"ok": false, "error", "hint"} and success
envelopes carry no ok/status field.
Tests: 44 cases — manifest acceptance and refusal (unlisted relative
paths, '..' and absolute tampered entries, symlinked manifest entries,
non-served suffixes), aliases with absent/corrupt cards, meta sha256
exposure, 3821-row downsample endpoint pinning, lossless paging walk,
traversal refusals, coercion, projection, budget shrink/re-stride/
floor, clamping, exact envelope key sets. README/SKILL count pins
updated (74->75 MCP, 107->108 registry); test_readme_counts,
test_distribution_skill_manifest and the MCP smoke suite are green.
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
Unreleased entries for #1572, #1634, #1636, #1578, #1602 and #1637.
#1572's own CHANGELOG hunk was written into the 0.1.16 section before
that release was cut, so it is dropped from the merge and rewritten
here under its own PR number (it cited #1569, the Email/WebSocket issue).
Crypto read okx · ccxt · binance and forex mt5 · yfinance · akshare in
all seven READMEs, while FALLBACK_CHAINS walks binance before ccxt and
akshare before yfinance. The note "futures / fund / macro → tushare /
akshare" was stale too: futures is akshare → local (#1395), funds
tushare → akshare, macro akshare → tushare. Canada, Vietnam, Argentina
and indices had no bullet.
test_fallback_chains_are_written_in_the_order_the_loader_walks compares
every chain a README writes with _DEFAULT_CHAINS, both ways, as
sequences, so translated labels need no mapping.
Rolls up 492 commits / 116 merged pull requests since 0.1.15, from 16
contributors.
The final-answer grounding gate stopped guessing what a number is from
the words around it: the model declares each figure's role (observed,
derived, proposed, cited or count), the gate checks the declaration
against the session's tool evidence, and a figure that fails is cut from
the answer instead of the whole answer being refused. A backtest's own
output now grounds the report written about it, run cards cite a metric
only where the saved CSV agrees, and the generated broker matrix states
each profile's permissions as declarations. Research runs keep what they
fetched: compaction follows the model's real context window instead of a
fixed 40K estimate. The missing-data sweep reached 59 more alphas.
Argentina (BYMA), weekly and monthly bars, Bahasa Indonesia, Gildata,
OpenCode, Email and WebSocket channels and four broker connectors (KIS,
Upbit, Toss Securities, Scalable Capital) land alongside.
Verified before release: the reporter's risk-parity prompt, which had
stopped with no_progress after 93 seconds on DeepSeek V4 Pro, completed
on 4a40a7f6's parent in 8m13s with both backtests run, 13 steps saved and
the report released with its core figures grounded.
Version declarations bumped in pyproject.toml, agent/SKILL.md,
Dockerfile, frontend/package.json, both package-lock.json entries, all
nine locales and the Layout test mock. CHANGELOG cites 116/116 merged
pull requests and credits 16/16 authors. README News carries the release
in seven languages, three entries visible.
Bound BaoStock messages and serialize SDK sessions; verify Copilot credential
lookup expiry without changing the SDK's per-call credential resolution.
Run public-loader health checks in isolated, credential-free processes and a
separate dependency environment, with explicit failure reports and a weekly
workflow. Render broker capabilities per profile and document verification
requirements. Update September 28 News in all seven README languages.
Closes#1626Closes#1623Closes#1617Closes#1492
Mirrors the DingTalk/QQ sentence in the "Configure from the Web UI"
paragraph across README.md and the six translations, in lockstep.
Part of #1569.
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
The BYMA / TSX fix could not close Hong Kong. The hk_equity chain starts with
Tencent and Eastmoney, which declare no currency, so a guard on Yahoo's
declared currency only moved a counter to the next source. The venue gives a
better handle. HKEX's Stock Code Allocation Plan (updated 2026-03-12) assigns
80000-89999 to "Products traded in Renminbi", and 09000-09199, 09200-09399,
09400-09499, 09500-09599, 09700-09799, 09800-09849, 10900-10999 and
41500-41599 to products traded in USD. Every other .HK code trades in HKD, and
that includes 09600-09699 and 09850-09999 (9988.HK, 9618.HK). Checked against
the currency Yahoo declared for 24 codes on 2026-09-24: 6 CNY, 10 USD and
8 HKD, all matching.
Before: 80700.HK (Tencent's RMB counter, 375.40 CNY) or 9834.HK (a USD ETF)
ran in the HK engine's HKD pool next to 00700.HK with no check at all, because
Composite's mixed-currency guard only runs across markets. The grounding gate
also required the answer to call their prices HKD.
- _market_hooks.hk_counter_currency(): the one table. code_currency() uses it,
so Composite's guard sees CNY / USD counters.
- runner._create_market_engine refuses a Hong Kong code traded in another
currency, for single-market and cross-market runs alike, before any engine
prices it.
- grounding identity._infer_currency reads the same table (lazy import, as
context.py already imports from backtest), so 80700.HK is quoted in CNY
whichever source served it.
Tests cover the 24 live codes, both sides of every range boundary, the refusal
(mixed, pure RMB, USD, cross-market) and an HKD-only run that still builds.
Mutants each fail: no RMB range (12), a USD range one code too wide (1),
code_currency ignoring the counter (1), the runner guard off (4), grounding
ignoring the counter (1). Related suites: 2769 passed.
openai 3.19.0 still built request headers with the case-sensitive
_merge_mappings; 3.19.2 switched to _merge_headers. Main's pre-fix code with
langchain-openai 1.6.5 passes 9/9 on openai 3.19.0 and fails the one test on
3.19.2. The comments, test docstring, CHANGELOG and news said 3.x.
The English README's "Configure from the Web UI" paragraph gained the
Email/WebSocket guided-channel sentence in the previous commit; mirror it
into the six translated READMEs (zh, ja, ko, ar, es, id) at the same slot
— after the QQ sentence, before the Test-connection sentence — keeping
IMAP/SMTP/TLS/WebSocket/Test connection verbatim. Prose-only; no count or
roster change, so test_readme_counts stays green (108 passed).
Follow-up to #1519 (extends #1520).
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
Weekly and monthly bars (#1479), price indices stamped raw (#1541), the
compaction replay (#1488), field-ref tail-risk identity (#1444) and the
QVeris refusal (#1494). The 2026-09-20 entry, which announced that 1M was
rejected, moves under Earlier news.
get_market_data and backtests accept 1W and 1M. A monthly review over
21 months used to mean 440 daily rows that microcompaction cleared
faster than the agent could aggregate them, until the no-progress
budget killed the run; one call now returns 21 bars.
Every loader is still asked for daily bars, and loaders.base.resample_bars
builds the period bar at the three fetch boundaries (runner.fetch_data_map,
market_data.fetch_market_data, the benchmark fetch). Few sources serve
weekly or monthly bars and those that do disagree on where a week ends
and how its bar is adjusted, so one resample keeps a weekly bar the same
thing whichever source served the days, at the daily caliber. A week runs
Monday to Sunday, a month is a calendar month, and each bar is stamped
with the last trading day inside it, when its close is known: first open,
highest high, lowest low, last close, summed volume and amount, vwap
weighted by volume, mean funding rate, last value of any other column.
The places that read an interval token learned the two new ones, and
1M is matched before any case fold, which would read it as a minute:
- calc_bars_per_year: 52 and 12 whatever the market's trading days;
- the runner's spacing check: a weekly or monthly file declared 1D now
matches 1W / 1M; the calendar branch is left for quarterly files;
- _interval_span_hours: 168h and a mean month of 730.5h, so a weekly
perpetual bar settles 21 funding periods instead of falling back to one;
- strict 100x compared the lower-cased token, so 1M would have passed as
one minute; attribution lower-cased too and would annualise a monthly
run at 98,280 bars a year;
- get_market_data serves 1W / 1M and the 1wk / 1mo spellings; the #1480
guard that refused them is gone, and the case fold no longer maps 1m and
1M onto one key;
- the technical-indicator tool documented 1wk / 1mo but handed them to the
loaders verbatim, and fetched two days per bar whatever the interval, so
200 weekly bars came back as 57.
READMEs x7, the MCP docstring and the strategy-generate skill list them.
Haozhe Wu