24 Commits
Author SHA1 Message Date
shadowinlife ee03474268 feat(factors): industry-neutralized CN value alphas + PIT SW2021 sector tag (csi300) (#1802)
* feat(bench): add horizon parameter for monthly-frequency IC revalidation

h-bar forward returns (close[t+h]/close[t]-1, no ffill — a NaN close never
bridges) sampled non-overlapping end-anchored, so the naive-iid t-stat and
the alive/dead thresholds stay honest without new statistics. Threaded
through run_bench / run_bench_strict / the alpha_bench tool + MCP wrapper /
CLI --horizon (argparse type + handler guard, exit 2 on bad input).
horizon=1 is byte-identical to the previous next-bar formula.

Follow-up #1 of issue #1789 (monthly-horizon revalidation of the 13
fundamental factors — the conventional reading for fundamental factors).

* fix(bench): exclude SEC-fundamental alphas off sp500 instead of failing the bench

--zoo fundamental --universe csi300 (the repro command printed in issue
#1789) hard-failed with 'Choose the S&P 500 universe' because the
validator rejects fund:* inputs on any non-sp500 universe, while the
mirror case (CN val:* alphas on sp500) skips cleanly per alpha. The
runners now split the selection up front, disclose the excluded ids in
the result envelope, and error only when nothing is left to bench; the
validator stays as the backstop for direct callers.

The CLI/tool JSON envelopes also state their own horizon,
forward_return_sampling and universe_excluded — a saved summary must
show the configuration its numbers came from.

* fix(bench): close review findings on horizon/exclusion disclosure

- multi-zoo CLI aggregation threads horizon / forward_return_sampling /
  universe_excluded from the sub-runs into the aggregate envelope (a saved
  all-zoos JSON now states its own horizon and reconciles its exclusions)
- alpha compare surfaces universe-excluded shortlist members as skipped
  entries with the exclusion reason instead of silently dropping them
- horizon upper bound (1.._MAX_HORIZON) enforced consistently at the CLI
  parser, the handler, and both runner entries BEFORE any panel load —
  previously only the tool/MCP capped it and an unbounded CLI run fetched
  the full panel before failing late
- non-int horizon can no longer TypeError in the entry schema construction
- categorise docstring notes the daily calibration of the |IC|>0.02 floor
  at h>1; the sampling comment no longer overclaims (residual
  autocorrelation from persistent factor exposures is not modeled)
- MCP wrapper uses _MAX_HORIZON instead of a hardcoded duplicate

Pinned by new tests: parser/handler/runner cap rejection (fetch not
called), multi-zoo disclosure aggregation, compare exclusion disclosure.

* feat(factors): industry-neutralized CN value alphas + PIT SW2021 sector tag

Three within-industry z-scored variants of the CN value factors
(fund_pb/fund_ps_ttm/fund_dividend_yield -> *_ind_neutral, zoo 475 -> 478):
raw A-share PB/PS/dy are heavily industry-confounded, so each name is
z-scored against its Shenwan L1 peers instead of the whole cross-section.

- industry_loader.py: tushare index_member_all (SW2021) membership with
  in_date/out_date -> point-in-time labels per trading day; deterministic
  overlap resolution; 2000-row truncation guard refetches per L1; degrades
  to an unmapped frame on provider failure, never invents a label.
- csi300 bench attaches panel['sector'] outside the pickled price cache,
  masked to the PIT membership cross-section, behind the same 0.9 coverage
  gate the sp500 GICS tag uses; meta discloses sector_source/coverage.
- base.zscore_by_group: the z-scoring sibling of alpha101's _ind_neutralize
  demean - no global fallback (an unlabelled cell stays NaN rather than
  wearing an industry-neutral label it did not earn).
- README x7 + SKILL.md counts (incl. the stale 462 prose sites), i18n zh/en.

Follow-up #2 of issue #1789.

* fix(loaders): index_member_all takes no src param; fetch PIT history via is_new Y+N

Live-verified endpoint contract (pinned by test): index_member_all accepts
only the documented l1/l2/l3_code, ts_code and is_new filters — an
undocumented src param silently returns an empty frame (HTTP 200, 0 rows),
which degraded the first csi300 sector attach to coverage 0.0. src now goes
to index_classify (truncation fallback) only.

The server also defaults is_new to 'Y' (current members, no out_date rows),
so the historical half ('N', ~2000 rows with out_date back to the 1990s) is
fetched separately and concatenated — without it every label would be
today's classification and the loader's PIT claim false.

Live smoke: 350/350 csi300 PIT constituents covered (100%), 0.9 gate passes.

* docs(factors): surface the SW2021 taxonomy-retroactivity caveat; pin cache immutability

Review follow-ups: the loader docstring disclosed that SW2021
classifications are applied retroactively, but no user-facing surface
did — one sentence added to each ind-neutral alpha's notes, the i18n
en/zh entries and SKILL.md (membership is point-in-time; the taxonomy
version itself is not). The sector-attach test now pins that the
original pickled panel dict is never mutated by the attach, and the
sector_coverage meta semantics (labelled-somewhere fraction, not
per-date) are documented at the attach site.

* fix(factors): align ind-neutral PB/PS with the upstream nonpositive-multiple guard

The maintenance batch (d3c76819) added a guard to fund_pb/fund_ps_ttm:
nonpositive multiples are excluded rather than ranked as cheap. The
industry-neutral variants follow suit — the within-group z-score sees
the same values.where(values > 0) pre-filter, the guard sentence leads
the notes/i18n entries, and the upstream parametrized test is mirrored
for the neutral stems.
2026-10-12 19:22:41 +08:00
Haozhe Wu d3c76819c3 fix: complete October 11-12 maintenance and research updates
Fix live option guards, grounding operand evidence, source-bound benchmarks, trade artifacts, attribution, drawdowns, valuation tolerances, and SEC point-in-time factor inputs.

Add Brazil research data and market support, 13 fundamental and valuation factors, channel progress forwarding, and opt-in ZimaOS templates. Update seven README News sections with three visible dates.

Validation: 17,635 backend tests passed; 1,084 final targeted tests and 103 frontend tests passed. Frontend build, package build, documentation checks, and CI gates passed.
2026-10-12 02:29:46 +08:00
Haozhe Wu 6365a9eba9 factors: pin Carhart's skip month and correct the text that described the old formula
#1578 fixed academic_carhart_mom to the return from t-252 to t-21, and
regenerated its golden with it, so the golden alone cannot tell the two
formulas apart. A new test scales one stock's closes inside the skipped
month and requires its last-bar value unchanged, and moves when the close
21 or 252 bars back changes; the old r_252 - r_21 fails it.

The alpha's notes, the zoo LICENSE and the wiki's Chinese note still said
'12-month return minus 1-month return'. The strategy-dev-manager example
taught the same subtraction, and named the alpha academic_carhart_momentum,
an id that does not exist.
2026-09-30 16:33:13 +08:00
Haozhe Wu 36f1698e1f wiki: Chinese alpha notes, and the GTJA 191 post re-measured
Alpha Library: the 191 notes, 117 nicknames and 10 theme tags now switch
to Chinese with the rest of the site. wiki/alpha-library/i18n.zh.json keeps
each translation next to the English it came from; the generator puts the
Chinese in data-text-zh only while that English still matches the
registry, and tests/factors/test_alpha_library_translations.py fails on a
missing, stale or orphaned translation or an unnamed theme.

GTJA 191 post: the 2026-05-17 numbers came from Tushare prices that were
not split-adjusted and from today's CSI 300 roster applied to the whole
window, both fixed on 2026-08-04 (d52199ff). Re-ran the bench on the
current code (point-in-time membership, qfq prices): 8 alive, 14 reversed,
168 dead, gtja191_138 untested (99% NaN), where the first version said
10 / 15 / 165 and never accounted for the 191st. Both languages now carry
a revision note and the new counts, theme table, top 5 by IR and three
most-negative alphas, and fix what the translation surfaced or the code
contradicts: the top 5 are ranked by IR, not IC; the theme table counts
every tag an alpha carries; the reflection matches the table; the Method
universe matches the Caveats; Kakushadze's preprint came a year after the
report; forward returns are simple, not log; no GTJA formula needs sector
tags; the bench measures no turnover and draws no IC plot; five alphas use
a proxy input (named); the reproduce command covers 2018-2025. A new
caveat reports the bench's multiple-testing deflation: 3 of the 8 alive
alphas clear the IR expected from 190 null formulas.
2026-09-29 18:36:52 +08:00
Haozhe Wu a7f645f368 wiki: one header on every page, English by default, one switch for 中文
Each section carried its own copy of the header: Home's nav had no Home,
the docs' nav had no Docs, the research post had no star or theme button,
and the generated Alpha Library pages had no header at all. Only the docs
could switch language, and they guessed Chinese from the browser while
every other page stayed English.

- partials/ holds one head block, header and footer; sync_site_chrome.py
  writes them into every page between markers and, with --check in CI,
  also requires <body data-section> (the current section is highlighted by
  CSS), the language attributes and a single ?v= token site-wide. The
  Alpha Library generator inserts the same partials.
- site.js is the shared shell (theme, stars, language, footer traffic on
  home only). English is the HTML; Chinese comes from locales/zh.json via
  data-i18n hooks, and check_i18n.py fails on a key with no Chinese or a
  Chinese string nothing uses. A URL that names a language (/docs/zh/,
  the two articles) shows it and remembers it; everything else follows the
  stored choice, English by default. theme-init.js keeps a page hidden until
  its Chinese text is in, so it never flashes English.
- English tutorial and Chinese research post added. The Chinese tutorial
  gets the corrections its translation surfaced: 462 alphas in five zoos,
  AAPL.US, the mandate caps trades per day (it has no daily loss limit),
  Shadow Account extracts up to five rules, the web UI needs Docker or a
  checkout.
- Alpha Library: the zoo pages were never deployed — content/ is
  gitignored and nothing built it, so every zoo link served the root
  redirect. The deploy now exports the manifest and renders the pages (and
  redeploys when agent/src/factors changes); the fifth zoo, fundamental, is
  listed; generated pages allow same-origin scripts only.
- Phones keep the nav as a scrollable row instead of hiding it.
2026-09-29 17:53:13 +08:00
Haozhe Wu 8a6d1934bc docs(wiki): rewrite the docs for 0.1.16, in English and Chinese
The docs still described 0.1.11: seven engines, six data sources, a
Quick start whose `vibe-trading serve` step opens no UI after a PyPI
install. Every count on the pages is now measured from the code (28
sources, 11 engines, 90 skills, 30 presets, 74 MCP tools, 18 brokers,
462 alphas), and five pages are new: Troubleshooting (no_progress,
"0 steps", cut figures, bare US tickers), Checked numbers, Alpha Zoo,
Brokers and live trading, Web UI and API.

Chinese pages are served at /docs/zh/<page>. The header toggle switches
language and remembers it; a first visit from a Chinese browser opens
the Chinese pages. The version selector offered 0.1.7-0.1.11 while
rendering one content set; it now lists only 0.1.16.
wiki/scripts/check_docs_parity.mjs (run in CI) fails when the two
languages differ in pages, sections, order, empty fields or internal
links. A #section in a docs URL now scrolls to it.

Landing page: "Seven engines" -> eleven, the uv tab no longer ends in
`serve`, and a 中文文档 button.
2026-09-29 17:18:19 +08:00
Haozhe Wu fa5e098143 fix(docs): propagate the fundamental zoo family + 0.1.11 counts to frontend, skills, and wiki
Post-release cross-audit findings: the 5th Alpha Zoo family never reached the
frontend (AlphaZoo card grid missing fundamental, academic still 6, '4 zoos'
i18n strings x5 locales), factors/__init__ docstring, or the asset-allocation
skill ('4 optimizers'). Wiki: 29->30 presets, docs version selector bumped to
0.1.11 (labels only; content is unversioned). README x5: locale parity for the
07-04 UTC news entry, india_broker in the loaders tree comment (20 sources),
ja/ko contributor headings translated.
2026-07-11 01:31:22 +08:00
Haozhe Wu c00de34676 Merge pull request #396 from kadaliao/codex/alpha-library-social-card
Add Alpha Library social card
2026-07-07 13:04:29 +08:00
Kada Liao 63f825ca77 fix(wiki): add alpha library social card
Signed-off-by: Kada Liao <kadaliao@gmail.com>
2026-07-05 02:09:09 +08:00
Kada Liao 2e5ea4fc17 docs: add Chinese beginner tutorial
Add a Chinese beginner-facing wiki tutorial and link it from the tutorial index, keeping broker/trading guidance conservative around read-only, paper, mandate, kill switch, and fail-closed boundaries.

Local verification: diff check, HTML/resource/id validation, sensitive-claim scan, and local wiki HTTP fetches passed.
2026-07-04 13:07:09 +08:00
Matt Van Horn d8f88d45da fix(wiki): harden alpha-library CSP by externalizing inline scripts 2026-07-01 15:40:27 +08:00
Haozhe Wu 399b5b4e23 fix(wiki): harden alpha library hydration 2026-06-29 15:51:47 +08:00
Haozhe Wu c2257fe6f3 fix(wiki): make footer counter all-time cumulative, not a 30-day window
Sum the full visits history instead of windowing to 30 days, so the agent/human counts grow monotonically since launch (data is never GC'd).
2026-06-05 14:54:43 +08:00
Haozhe Wu ee28390a2c fix(wiki): cache-bust main.js so browsers load the traffic-counter JS immediately
main.js is browser-cached up to 4h; the fresh HTML still pointed at the unversioned /main.js, so returning visitors kept running the pre-counter JS and the footer never populated. Versioned query forces an immediate fetch of the new module.
2026-06-05 14:47:39 +08:00
Haozhe Wu cb82c6262c fix(wiki): align live-trading copy with shipped Robinhood Agentic support; hide footer counter until data loads
The wiki still claimed 'no live trading / does not execute live trades', which contradicts the shipped opt-in bounded Robinhood Agentic channel (and the README). Aligns home, docs, and alpha-library copy with the README's framing. Also fixes a CSS specificity bug where .footer-traffic{display:flex} overrode the [hidden] attribute, showing placeholder dashes before /api/stats data arrived.
2026-06-05 14:39:55 +08:00
Haozhe Wu e2a42aa4ab fix(wiki): cache PyPI installs in D1, drop redundant GitHub call from /api/stats
pypistats/GitHub calls from the Worker intermittently rate-limit, blanking the footer. PyPI now reads through a D1 last-good cache so it never goes empty; GitHub stars are dropped here since main.js already loads them client-side.
2026-06-05 14:14:07 +08:00
Haozhe Wu 25b6ee5d72 feat(wiki): add agent-vs-human traffic counter to vibetrading.wiki
First-party, anonymous analytics for the public wiki — no client-side telemetry in the pip package. Pages Function middleware classifies each page request (AI agent / bot / human) by User-Agent server-side and counts it into D1; /api/stats serves the footer's aggregate counts alongside public PyPI install and GitHub star numbers.
2026-06-05 14:03:17 +08:00
Haozhe Wu 5a06371a94 chore(release): 0.1.9
Roll-up release since 0.1.8 — connector-first broker profiles (IBKR +
Robinhood Agentic Trading), Research Goal runtime, swarm reconcile/MCP
keepalive + worker MCP tools + strict alpha-bench + retry_run, the
agent/cli/ package refactor, the mootdx A-share loader, a robustness pass,
and the session/version fixes (#150 #156 #157 #158 + UI indicator).

Version → 0.1.9 across pyproject, SKILL.md, frontend APP_VERSION, and the
wiki docs version selector. MCP tool count corrected to 36 (adds retry_run)
across all 5 READMEs, with the ar/ko tool lists rebuilt from a stale 22 and
the ar skill count fixed 75→77. CHANGELOG [0.1.9] + 0.1.9 news entry added
to all 5 READMEs. Home backtest card data-source count corrected 3→7.
2026-06-01 23:00:43 +08:00
Haozhe Wu 6b8241a356 fix(wiki): route docs rewrites to docs app 2026-05-25 22:43:33 +08:00
Haozhe Wu d2c94396aa chore: move tweet draft out of public wiki
wiki/research-lab/posts/alpha-191-in-2026.tweet.md was an internal
marketing draft (pre-flight checklist + draft tweet text) that should
not have been committed to a public path. Moved to docs/alpha-zoo/
(gitignored) and removed from main.

Note: the file remains in git history at commit 137689c. If history
scrub is required, a force-push rewrite is needed (separate decision).
2026-05-17 20:23:44 +08:00
Haozhe Wu 07e1adcbba chore(0.1.8): version, CHANGELOG, packaging, News across 5 READMEs, SKILL.md
Release prep for Alpha Zoo v1. Wraps up the 0.1.8 main-branch
content so PyPI + ClawHub publish can run from a clean checkout.

Version
- pyproject.toml: 0.1.7 → 0.1.8
- wiki/docs/content.js: DOCS_DEFAULT_VERSION 0.1.7 → 0.1.8 + 0.1.7
  retained in the version dropdown
- agent/SKILL.md frontmatter: 0.1.7 → 0.1.8

Packaging
- pyproject.toml [tool.setuptools.package-data] adds the new
  src.factors zoo asset globs (zoo/**/*.yaml, zoo/**/*.md,
  zoo/**/NOTICE) alongside the existing src and backtest entries
- MANIFEST.in adds `recursive-include agent/src/factors *.yaml *.md`
  and `include NOTICE`

CHANGELOG.md (first changelog file in the repo)
- Full 0.1.8 entry covering the Alpha Zoo (4 zoos × 452 alphas),
  Web UI + 4 REST routes, CLI subcommand suite, safety floor
  (AST purity gate + lookahead test + pytest-socket + grep gates),
  community governance (CONTRIBUTING + DCO + NOTICE + per-zoo
  LICENSE.md), and the IC/IR core extraction.
- Known limitations subsection: btc-usdt is single-asset; cross-
  sectional IC needs ≥2 instruments; crypto-majors basket planned
  for 0.2.
- [Unreleased] anchor at top for the next cycle.

agent/SKILL.md (ClawHub package descriptor)
- Frontmatter description gains Alpha Zoo + 75 skills (was 74)
- New `### Alpha Zoo (452 pre-built alphas)` section with the
  4-zoo breakdown + CLI / REST / signal-engine entry points
- Skill count 74 → 75 wherever it appears
- Trademarked-name string intentionally absent (CI gate b enforces)

5 READMEs (EN/ZH/JA/KO/AR) per `docs/alpha-zoo/readme-update-spec.md`
- Roadmap: Alpha Zoo `Planned` → `Shipped 0.1.8` with expanded
  description (452 alphas, 4 zoo sources, CLI + agent + Web UI)
- Skill count 74 → 75 (Tool category 10 → 11, `alpha-zoo` added)
- Agent tools count 29 → 31 (alpha_zoo_tool, alpha_bench_tool)
- Project structure tree: new `factors/` block (base / registry /
  bench_runner / zoo), new `api/` block (alpha_routes), new
  top-level `tools/` block (ci_grep_gates.sh). Frontend pages list
  expanded (AlphaZoo, Correlation, Settings)
- Quick Example: second snippet showing `alpha bench` one-liner
- New "What You Can Do" row for the bench-a-pre-built-alpha-zoo path
- New `<details>` block under Detailed Capabilities listing the
  4 zoos + counts + sources + license stance
- Examples → Strategy & Backtesting: 3 snippets (bench, list+show,
  multi-factor compose via ZooSignalEngine.from_zoo)
- API Server route table: 4 `/alpha/*` rows (list, get, bench,
  bench stream SSE)
- CLI Reference: alpha subcommand introduced in the top block;
  single-run examples block extended with 3 alpha commands

`### Key Features` 4-card grid (#7) and Demo section (#13) are
intentionally NOT touched here — they need layout redesign + new
GIFs/screenshots; design notes in docs/alpha-zoo/readme-redesign-7-13.md
will be implemented in a follow-up PR.

969 tests pass; CI grep gates green (no yaml.load misuse, no
trademarked-name leak, no per-stock-code data in wiki); TypeScript
clean on the frontend.
2026-05-17 20:19:04 +08:00
Haozhe Wu 137689c5c1 feat(wiki): Alpha Library auto-render + "Alpha 191 in 2026" blog
wiki/scripts/build_alpha_library.py (475L)
- Reads manifest.json produced by `vibe-trading alpha export-manifest`
- Emits per-alpha pages (content/<zoo>/<id>.html) and per-zoo
  overview tables (content/<zoo>/index.html) — 452 + 4 = 456 pages
- Jinja2 with select_autoescape(["html","xml"]) mandatory; no |safe
  on user-derived content; strict CSP on every generated page
  (default-src 'self'; style-src 'self' 'unsafe-inline';
   script-src 'none'; img-src 'self' data:)
- Emits content/index.json for the landing page to hydrate per-zoo
  counts via fetch()
- Friendly exit when manifest.json missing
- End-of-main self-test compares manifest formula content against
  rendered HTML to verify the escape pipeline; prints
  "html escape ok" or "html escape FAIL"

wiki/alpha-library/index.html (rewritten from 78-line shell to 192L)
- Hero + 4 zoo cards (Kakushadze 101 Formulaic Alphas, qlib158,
  gtja191, academic) + stats grid + 3-line CLI quickstart + GitHub
  footer
- Per-zoo counts hydrated at runtime from content/index.json
- Loose CSP (script-src 'self' 'unsafe-inline') only on this landing
  page to keep the site theme-init inline script and the inline
  fetch() — generated /content/** pages stay on script-src 'none'

wiki/research-lab/index.html
- Post grid replacing the empty hero stub; links to the new blog
- One-line lede explaining the section

wiki/research-lab/posts/alpha-191-in-2026.html (~675L, ~2400 words)
- 6-section deep-dive: TL;DR / Background / Method / Findings /
  Caveats / Reproduce
- Numbers from a real bench run on CSI 300 / 2018-2025: 10 alive,
  15 reversed, 165 dead (of 190 tested, 1 skipped). Top 5 by IR:
  gtja191_171 / _111 / _163 / _002 / _054. Theme survival rates:
  microstructure 22%, reversal 11%, volatility 8%, volume 5%,
  momentum 2%, liquidity 0%
- Survivorship-bias caveat for cross-zoo readers (the current
  constituent list of any index is a forward-biased proxy for the
  point-in-time constituents during the window)
- Strict CSP (script-src 'none'); no MathJax/KaTeX — formula_latex
  rendered as <code class="formula">; aggregate stats only (no
  per-stock per-date values, Tushare ToS)

wiki/research-lab/posts/alpha-191-in-2026.tweet.md
- HKUDS-style draft tweet (Chao Huang voice): emoji-decorated
  project name, real numbers (10 / 15 / 165), 8-year OOS qualifier,
  GitHub + pip links, ≤2 hashtags. READY to post once the release
  goes live.

.gitignore adds wiki/alpha-library/manifest.json and
wiki/alpha-library/content/ as generated, never-checked-in artifacts.
2026-05-17 20:04:24 +08:00
Haozhe Wu eba0f1f8a1 docs: add wiki section shells 2026-05-14 22:02:32 +08:00
Haozhe Wu 7b27f6e3e2 Add static Vibe-Trading wiki
Add a static Cloudflare Pages wiki under wiki/, including the landing page, docs app, redirects/headers, maintainer README, and wiki validation workflow.
2026-05-14 19:31:31 +08:00