79 Commits
Author SHA1 Message Date
Haozhe Wu d2c94396aa chore: move tweet draft out of public wiki
wiki/research-lab/posts/alpha-191-in-2026.tweet.md was an internal
marketing draft (pre-flight checklist + draft tweet text) that should
not have been committed to a public path. Moved to docs/alpha-zoo/
(gitignored) and removed from main.

Note: the file remains in git history at commit 137689c. If history
scrub is required, a force-push rewrite is needed (separate decision).
2026-05-17 20:23:44 +08:00
Haozhe Wu 07e1adcbba chore(0.1.8): version, CHANGELOG, packaging, News across 5 READMEs, SKILL.md
Release prep for Alpha Zoo v1. Wraps up the 0.1.8 main-branch
content so PyPI + ClawHub publish can run from a clean checkout.

Version
- pyproject.toml: 0.1.7 → 0.1.8
- wiki/docs/content.js: DOCS_DEFAULT_VERSION 0.1.7 → 0.1.8 + 0.1.7
  retained in the version dropdown
- agent/SKILL.md frontmatter: 0.1.7 → 0.1.8

Packaging
- pyproject.toml [tool.setuptools.package-data] adds the new
  src.factors zoo asset globs (zoo/**/*.yaml, zoo/**/*.md,
  zoo/**/NOTICE) alongside the existing src and backtest entries
- MANIFEST.in adds `recursive-include agent/src/factors *.yaml *.md`
  and `include NOTICE`

CHANGELOG.md (first changelog file in the repo)
- Full 0.1.8 entry covering the Alpha Zoo (4 zoos × 452 alphas),
  Web UI + 4 REST routes, CLI subcommand suite, safety floor
  (AST purity gate + lookahead test + pytest-socket + grep gates),
  community governance (CONTRIBUTING + DCO + NOTICE + per-zoo
  LICENSE.md), and the IC/IR core extraction.
- Known limitations subsection: btc-usdt is single-asset; cross-
  sectional IC needs ≥2 instruments; crypto-majors basket planned
  for 0.2.
- [Unreleased] anchor at top for the next cycle.

agent/SKILL.md (ClawHub package descriptor)
- Frontmatter description gains Alpha Zoo + 75 skills (was 74)
- New `### Alpha Zoo (452 pre-built alphas)` section with the
  4-zoo breakdown + CLI / REST / signal-engine entry points
- Skill count 74 → 75 wherever it appears
- Trademarked-name string intentionally absent (CI gate b enforces)

5 READMEs (EN/ZH/JA/KO/AR) per `docs/alpha-zoo/readme-update-spec.md`
- Roadmap: Alpha Zoo `Planned` → `Shipped 0.1.8` with expanded
  description (452 alphas, 4 zoo sources, CLI + agent + Web UI)
- Skill count 74 → 75 (Tool category 10 → 11, `alpha-zoo` added)
- Agent tools count 29 → 31 (alpha_zoo_tool, alpha_bench_tool)
- Project structure tree: new `factors/` block (base / registry /
  bench_runner / zoo), new `api/` block (alpha_routes), new
  top-level `tools/` block (ci_grep_gates.sh). Frontend pages list
  expanded (AlphaZoo, Correlation, Settings)
- Quick Example: second snippet showing `alpha bench` one-liner
- New "What You Can Do" row for the bench-a-pre-built-alpha-zoo path
- New `<details>` block under Detailed Capabilities listing the
  4 zoos + counts + sources + license stance
- Examples → Strategy & Backtesting: 3 snippets (bench, list+show,
  multi-factor compose via ZooSignalEngine.from_zoo)
- API Server route table: 4 `/alpha/*` rows (list, get, bench,
  bench stream SSE)
- CLI Reference: alpha subcommand introduced in the top block;
  single-run examples block extended with 3 alpha commands

`### Key Features` 4-card grid (#7) and Demo section (#13) are
intentionally NOT touched here — they need layout redesign + new
GIFs/screenshots; design notes in docs/alpha-zoo/readme-redesign-7-13.md
will be implemented in a follow-up PR.

969 tests pass; CI grep gates green (no yaml.load misuse, no
trademarked-name leak, no per-stock-code data in wiki); TypeScript
clean on the frontend.
2026-05-17 20:19:04 +08:00
Haozhe Wu 137689c5c1 feat(wiki): Alpha Library auto-render + "Alpha 191 in 2026" blog
wiki/scripts/build_alpha_library.py (475L)
- Reads manifest.json produced by `vibe-trading alpha export-manifest`
- Emits per-alpha pages (content/<zoo>/<id>.html) and per-zoo
  overview tables (content/<zoo>/index.html) — 452 + 4 = 456 pages
- Jinja2 with select_autoescape(["html","xml"]) mandatory; no |safe
  on user-derived content; strict CSP on every generated page
  (default-src 'self'; style-src 'self' 'unsafe-inline';
   script-src 'none'; img-src 'self' data:)
- Emits content/index.json for the landing page to hydrate per-zoo
  counts via fetch()
- Friendly exit when manifest.json missing
- End-of-main self-test compares manifest formula content against
  rendered HTML to verify the escape pipeline; prints
  "html escape ok" or "html escape FAIL"

wiki/alpha-library/index.html (rewritten from 78-line shell to 192L)
- Hero + 4 zoo cards (Kakushadze 101 Formulaic Alphas, qlib158,
  gtja191, academic) + stats grid + 3-line CLI quickstart + GitHub
  footer
- Per-zoo counts hydrated at runtime from content/index.json
- Loose CSP (script-src 'self' 'unsafe-inline') only on this landing
  page to keep the site theme-init inline script and the inline
  fetch() — generated /content/** pages stay on script-src 'none'

wiki/research-lab/index.html
- Post grid replacing the empty hero stub; links to the new blog
- One-line lede explaining the section

wiki/research-lab/posts/alpha-191-in-2026.html (~675L, ~2400 words)
- 6-section deep-dive: TL;DR / Background / Method / Findings /
  Caveats / Reproduce
- Numbers from a real bench run on CSI 300 / 2018-2025: 10 alive,
  15 reversed, 165 dead (of 190 tested, 1 skipped). Top 5 by IR:
  gtja191_171 / _111 / _163 / _002 / _054. Theme survival rates:
  microstructure 22%, reversal 11%, volatility 8%, volume 5%,
  momentum 2%, liquidity 0%
- Survivorship-bias caveat for cross-zoo readers (the current
  constituent list of any index is a forward-biased proxy for the
  point-in-time constituents during the window)
- Strict CSP (script-src 'none'); no MathJax/KaTeX — formula_latex
  rendered as <code class="formula">; aggregate stats only (no
  per-stock per-date values, Tushare ToS)

wiki/research-lab/posts/alpha-191-in-2026.tweet.md
- HKUDS-style draft tweet (Chao Huang voice): emoji-decorated
  project name, real numbers (10 / 15 / 165), 8-year OOS qualifier,
  GitHub + pip links, ≤2 hashtags. READY to post once the release
  goes live.

.gitignore adds wiki/alpha-library/manifest.json and
wiki/alpha-library/content/ as generated, never-checked-in artifacts.
2026-05-17 20:04:24 +08:00
Haozhe Wu d77c64e2ce feat(alpha-zoo): bench runner + CLI + Web UI + 4 REST routes
End-to-end product surfaces over the factor framework: a CLI for
power users, an agent-tool layer for natural-language workflows,
a REST + SSE API for browser clients, and the Web UI itself.

CLI (vibe-trading alpha <subcommand>)
- list / show / bench / compare / export-manifest with security
  guards (path-traversal, alpha_id allowlist before any FS read)
- agent/cli.py wired (7-line glue), all logic in cli_handlers.py
- alpha bench: requires --zoo (or --yes); drives bench_runner with
  an on_progress callback feeding a Rich Progress bar; prints ETA
  banner; emits stderr summary "✓ Bench complete. N tested, M
  skipped. Report: <path>"; returns exit 1 on status != ok
- alpha list: --limit N / --json flags; TTY next-step hints
- alpha show: --brief; Syntax(word_wrap=True) so source survives
  narrow terminals
- TUSHARE_TOKEN errors append a 3-step fix banner
- ConstantInputWarning silenced at module level

Agent tools (auto-discovered via BaseTool.__subclasses__)
- AlphaZooTool: list_alphas / get_alpha / health
- AlphaBenchTool: orchestrator with Jinja2 autoescape HTML report +
  strict CSP meta. Module-level run_alpha_bench() reused by CLI

Universe loader (agent/src/tools/alpha_bench_tool.py)
- csi300 → Tushare index_weight + pro.daily, in-house panel build
  because the bundled tushare loader drops `amount`. Parallel fetch
  via ThreadPoolExecutor(max_workers=4); cache at ~/.vibe-trading/
  cache/<universe>_<start>_<end>.pkl with sha256 sidecar integrity
  guard
- sp500 → Wikipedia constituent list (UA-spoofed; bare pd.read_html
  returns 403) → yfinance loader; surfaces survivorship_bias=True
  in panel._meta because the constituent set is point-in-NOW
- btc-usdt → bundled okx loader; fails fast (ValueError) on single-
  asset cross-sectional IC degeneracy

REST API (agent/src/api/alpha_routes.py + agent/api_server.py +6 lines)
- GET /alpha/list?zoo=&theme=&universe=&limit= — paginated catalogue
- GET /alpha/{alpha_id} — meta + source code via Registry.get_source
- POST /alpha/bench — kicks off asyncio.create_task running the
  bench in asyncio.to_thread; in-memory job state with threading.Lock
  + 1-hour TTL + asyncio.Semaphore(2) concurrency cap (429 on
  saturation)
- GET /alpha/bench/{job_id}/stream — Server-Sent Events emitting
  progress / result / done / error events. Heartbeat ": ping\n\n"
  every 15s so reverse proxies don't close idle streams. Unexpected
  exceptions are sanitised to "internal error; see server logs"

Web UI (frontend/src/pages/AlphaZoo.tsx ~950 LOC, 3 sub-views)
- Browse: 4 gradient zoo cards (qlib158 / Kakushadze 101 / gtja191 /
  academic) + filter bar (zoo, theme, universe, id substring) +
  paginated table (50 rows + Load more)
- Detail: formula_latex in <code> block (no MathJax under strict
  CSP); metadata table; collapsible <details> source code; "Run
  bench" CTA prefilled
- Bench: form → SSE-streamed progress bar + currently-computing
  alpha id → Alive/Reversed/Dead/Skipped stat cards + Top-5-by-IR +
  Most-Reversed tables + by-theme echarts bar chart
- New /alpha-zoo route triplet wired in router.tsx; Layout.tsx
  nav entry with the Layers icon
- Raw EventSource pattern (useSSE hard-codes agent-stream event
  types and would drop our progress/result/done/error)
- a11y: htmlFor/id pairs on every label, aria-hidden on decorative
  lucide icons, sr-only table caption, CategoryBadge component
- BTC universe fail-fast surfaces as inline form error with hint
  to try sp500/csi300

Registry singleton via get_default_registry() so API requests don't
each pay the 452-module AST scan. compute_ic_series vectorised via
pandas corrwith(method="spearman"); ~5-10x speedup, numerically
identical to the prior scipy loop within 1.1e-16.

VWAP 千元 fix: empirical Tushare probe against 000001.SZ confirmed
daily.amount is in 千元 (not raw CNY). base.vwap(EQUITY_CN) now does
safe_div(amount * 1000.0, volume * 100.0 + 1.0) — previously
~1/1000th of true vwap. Affects factors mixing vwap with price
additively (alpha101_093 etc.).

969 tests pass; registry health 452 loaded / 0 failed; CI grep
gates green; TypeScript clean.
2026-05-17 20:04:02 +08:00
Haozhe Wu 94c2a8ffb2 feat(factors): port 452 alphas across 4 zoos
Implements four production-ready alpha catalogues, each in its own
agent/src/factors/zoo/<zoo>/ subdirectory with a per-zoo LICENSE.md
declaring provenance and treating the formulas as mathematical
content (paper prose, tables, and figures are not reproduced):

qlib158 — 154 alphas
  Port of Microsoft Qlib's Alpha158 handler.py feature definitions
  (github.com/microsoft/qlib @ d5379c5, Apache-2.0). Each module
  carries a per-file header citing the upstream commit SHA + path.
  Includes the upstream NOTICE per Apache-2.0 §4(d). 4 raw price-
  reference fields ($open/$high/$low/$vwap) intentionally omitted —
  they are inputs, not signals, and conflict with the wide-DataFrame
  contract.

alpha101 — 101 alphas
  Implementation of Kakushadze (2015) "101 Formulaic Alphas"
  (arXiv:1601.00991), written from the paper appendix. 19 industry-
  neutral alphas (#48, 56, 58, 59, 63, 67, 69, 70, 76, 79, 80, 82, 87,
  89, 90, 91, 93, 97, 100) flagged requires_sector=True so the
  registry SkipAlpha-s them on panels without a sector tag. Display
  name is "Kakushadze 101 Formulaic Alphas" — the trademarked
  alternative string is intentionally absent (enforced by ci grep
  gate b).

gtja191 — 191 alphas
  Implementation of Guotai Junan Securities' 2014 "191 Short-period
  Trading Alpha Factors" research report. Ambiguous formulas
  (MAX/MIN set vs ts, sign convention, SMA(x,n,m) → x.ewm(alpha=m/n),
  WMA → decay_linear approximation, REGBETA → ts_cov/ts_std**2,
  HIGHDAY/LOWDAY → ts_argmax/ts_argmin) are documented per-alpha in
  __alpha_meta__["notes"]. Long-horizon SUM/MEAN windows
  (alphas #25/26/33/39/45/56/61/64/74/75/77/91/92/98) shortened on
  the production code path so outputs stay defined on short panels;
  documented in each affected alpha's notes.

academic — 6 factors
  Honest price-based proxies for Fama-French 5 + Carhart momentum;
  nicknames carry a [PRICE PROXY] prefix because the canonical FF
  series require book-to-market / profitability / investment-growth
  fundamentals we do not ship. Kenneth French's data library is
  referenced (URL only) for users who need canonical monthly returns.
  Internal _effective_window helpers shrink the 252/60-day windows on
  short panels so the lookahead test and the registry NaN-ratio guard
  pass; production-sized panels see the canonical window untouched.

Sample golden CSVs at agent/tests/factors/fixtures/goldens/ for
five sampled alphas per zoo, asserted at rtol=1e-6, equal_nan=True.
ci grep gates pass; AST purity + lookahead tests both green on the
full 452-alpha registry.
2026-05-17 20:03:28 +08:00
Haozhe Wu 1676eaa328 feat(factors): Alpha Zoo framework — registry, operators, safety gates
New module agent/src/factors/ providing the foundation for the Alpha
Zoo:

- base.py: 19 cross-sectional + time-series operators on wide
  pd.DataFrame panels (rank/scale/ts_*/delta/decay_linear/signed_power/
  safe_div + market-aware vwap). NaN propagates strictly; delta(d>=1)
  enforced at the operator layer; no negative shifts allowed.
- registry.py: AST-extracts __alpha_meta__ from each zoo .py module
  without importing it; pydantic AlphaMeta (extra="forbid",
  frozen=True). Lazy importlib.import_module on compute(); filesystem-
  loader fallback for plugin/test trees. Sanity gates reject +/-inf,
  NaN ratio > 95%, and shape mismatches.
- factor_analysis_core.py: IC/IR + layered backtest math extracted
  from agent/src/tools/factor_analysis_tool.py so it's reusable from
  CLI, bench tool, and API.
- bench_runner.py: per-alpha compute → IC vs forward return →
  alive/reversed/dead categorisation (mean IC > 0.02, pos_ratio >=
  0.55, |t-stat| > 2). Shared by CLI driver and API worker.

Multi-factor skill gains a panel-style ZooSignalEngine.from_zoo()
factory (cross-sectional standardisation + weighting + optional
top-N / bottom-N long-short conversion). The legacy per-symbol
example_signal_engine.py is left untouched. SKILL.md edits in
multi-factor and factor-research cross-reference the new zoo.

Safety floor:
- test_alpha_purity.py: AST allowlist scan of every zoo/**/*.py
  module. Allowed imports: pandas, numpy, scipy.*, src.factors.base,
  __future__, typing, math, dataclasses. Banned names: os/sys/
  subprocess/socket/urllib/requests/httpx/pathlib/Path/open/eval/exec/
  compile/__import__, plus dunder-access via getattr(_, "__*"),
  breakpoint, input, help, memoryview, globals, locals, vars, and
  the __class__/__bases__/__mro__/__globals__/__builtins__ chain.
  Module-level statements limited to docstring + imports + functions
  + ALPHA_ID/__alpha_meta__ assignments with literal RHS.
- test_lookahead.py: 300-row synthetic panel; corrupt rows >= 270;
  assert factor at probe row 260 unchanged within 1e-9.
- test_base.py + test_registry.py + conftest.py with opt-in
  pytest-socket integration that hard-fails outbound network during
  the factors test suite.
- tools/ci_grep_gates.sh: rejects yaml.load( without safe_load,
  any trademarked-name leak in shipped artifacts, and any per-stock-
  code data leak in wiki/**/*.{json,csv,html}.

Governance:
- CONTRIBUTING.md with the Developer Certificate of Origin trailer
  for community PRs and the alpha PR reviewer checklist.
- NOTICE at the repo root with Apache-2.0 attribution for the
  upstream qlib NOTICE and the formulas-as-mathematical-content
  stance for Kakushadze / GTJA / Fama-French sources.
2026-05-17 20:03:02 +08:00
Haozhe Wu 6d7d365892 refactor(loaders): extract bounded retry/budget pattern into base.py
ccxt_loader and okx each carried their own copy of the same 25-line
retry+budget loop introduced in #121. Hoist the pattern into two
helpers — check_budget() and retry_with_budget() — so any future loader
(yfinance, akshare, tushare) inherits the same guarantees: per-call
transient retries, a wall-clock budget, non-transient exceptions left
unwrapped, original exception preserved as __cause__.

ccxt_loader / okx now declare only what is loader-specific (the
exception class for transient errors and the budget env var); the
scheduling, sleep math, and TimeoutError wrapping live in base.py.

Adds 11 helper-level tests + the 11 pre-existing integration tests
still pass against the refactored loaders.

Follow-up to #121.
2026-05-17 00:13:28 +08:00
Haozhe Wu fd9e5fc689 refactor(swarm): _is_error_result via JSON parse with substring fallback
The substring head-match (a) false-positived on nested status fields and
(b) false-negated when the error envelope sat past the 160-char head.
Parse the result as JSON and check only the top-level status field; fall
back to the original substring scan when the payload is truncated /
unparseable so the classifier never raises on the worker hot path.

Adds 7 unit tests pinning the new behavior — nested-status non-match,
past-head match, truncated fallback, non-error statuses (degenerate /
warning) correctly classified as non-error.

Follow-up to #119.
2026-05-17 00:13:19 +08:00
Haozhe Wu 2f9fa22a56 perf(redaction): cache _internal_roots() to avoid recomputation per call
_internal_roots resolves 8 candidate paths + builds 3 separator variants
+ sorts on every redact_internal_paths call. On the worker hot path this
runs once per task error / event / file-tool failure. Result is process-
stable (paths don't change without a restart), so @cache it.

Follow-up to #119.
2026-05-17 00:13:10 +08:00
Teerapat VatpitakandHaozhe Wu 39aaa62008 fix(swarm): surface errors, output contract, Windows-safe store, redact paths (#119)
* fix(swarm): surface task error at read boundaries

run_swarm / get_swarm_status / get_run_result and the in-process
run_swarm tool each hand-maintained a field allowlist that omitted
SwarmTask.error. A misconfigured provider returned status="failed"
with no diagnosable reason, though the error was captured on disk
the whole time.

Add src/swarm/serialization.py as the single source of truth for the
per-task projection (now includes error + iterations) plus a run-level
error summary; route all three boundaries through it so the allowlist
cannot silently drift again. Additive JSON change; the only shape
normalization is _format_result missing-summary "" -> null.

(cherry picked from commit 95f2717ac32bd02829f650bc971e3cd6dd7fffa6)

* fix(swarm): enforce worker output contract

A worker that emitted only its plan ("Phase 1 — Plan"), mock data,
unparsed tool markup, or (for a data agent) made no tool call and
wrote no report was returned status="completed". runtime.py also
folded timeout/token_limit into completed. The run reported success
with a stub as final_report (P01); degraded workers were invisible
(P03).

Add WorkerStatus.incomplete and a hybrid output contract in worker.py:
content-sanity for every agent, tool-evidence only for data agents so
tool-less synthesis/editor roles are not false-rejected. Fail-fast to
incomplete (no corrective retry — left for a follow-up). runtime.py
only maps a true completed to TaskStatus.completed; anything else
fails the run, so final_report can no longer be a non-deliverable.

(cherry picked from commit 774b26824509678c953cd0200ef7a56c03f23201)

* fix(swarm): thread shell tools + unify runs root

The MCP run_swarm wrapper called start_run() without
include_shell_tools, so stdio swarm workers silently lost bash and
could not run the scripts their own execution rules mandate (P03-B).
build_filtered_registry also dropped a requested-but-unavailable
tool with no log, hiding the contradiction.

Pass the server's shell-tool policy into start_run and warn on a
dropped tool. Add swarm_runs_root() as the single source of truth
for the run store location, shared by mcp_server and the path_utils
run-dir allow-list so the two can no longer drift (P03-A: an
installed-wheel layout had put every worker run_dir outside it).

(cherry picked from commit 2975fda09b5ad44d15580d9330890f72f43c00ce)

* fix(swarm): make store atomic write Windows-safe

SwarmStore._atomic_write did tmp.replace(target) with no retry. On
Windows os.replace raises PermissionError WinError 5/32 when the
target is concurrently open by load_run on the poll path, crashing
the worker thread and leaving the run stuck pending (P13). POSIX
os.replace is atomic and never raises these.

Add a WinError-scoped, bounded retry around the rename and a matching
transient read/parse retry in load_run (the other side of the same
race). Non-transient errors re-raise immediately; off-Windows the
loop runs once, so POSIX behavior is unchanged. Pre-existing, not
introduced by the P01/P03/P04 fixes — surfaced by E2E dogfooding.

(cherry picked from commit 46a779cf436353591e7840879f8c32f763dde826)

* style(swarm): drop unused imports (ruff F401)

Remove an unused `from pathlib import Path` in the PR03 test added by
2975fda, and the pre-existing unused `from src.swarm.mailbox import
Mailbox` in runtime.py (file already in this branch's diff). Net ruff
delta for PR1-PR4 is now -1. The remaining mcp_server.py:41 E402 is
pre-existing on upstream and structural — left out of scope.

(cherry picked from commit e96f56af1ead9b5b7058c90ec0d4df738b45e473)

* fix: redact internal absolute paths in errors

Unknown preset / workspace-escape / missing run-or-task dir errors
embedded the full install path (OS username, .venvs/site-packages
topology) — CWE-209/497, reachable via MCP/API/agent/CLI (P10).

Drop the absolute path while keeping the actionable bits: preset
name + Available list, the workspace-root boundary, and the logical
run/task id (basename only). Two test_path_safety assertions that
matched the old leaking message are updated to the redacted wording
(behavior unchanged — still raises and rejects). Verified-clean
sites (caller's own input / env-var / names list) left untouched.

(cherry picked from commit 69ea45295410cd25ea3079c25b7e3c4c73390b80)

* fix(security): redact internal paths in errors

Add agent/src/tools/redaction.py: anchored prefix redaction (no
regex, idempotent, None-safe) that hides only known-internal root
prefixes while keeping the relative tail for diagnosability.

Wire it through the swarm error read boundaries (serialization,
runtime, store) and the file tools' final broad except so a leaked
absolute path cannot reach a user-facing string (CWE-209/497, P10).
ValueError branches and the protected providers/llm.py are left
untouched.

(cherry picked from commit 21dc9cdfacf17cccd31cacfbe34481f62f1390af)

* fix(security): redact swarm event error payloads

---------

Co-authored-by: Haozhe Wu <haozhe_wu@connect.hku.hk>
2026-05-16 23:50:37 +08:00
Teerapat Vatpitak 220abb10a1 fix(mcp): unresolved symbols, finite options validation, row cap (#120)
* fix(mcp): surface unresolved market-data symbols

A typo / wrong-suffix / delisted / no-data code vanished from
get_market_data's result (the dict only held winners), so it was
indistinguishable from "no data", and a loader exception lost every
already-resolved symbol (P05).

Report any requested code that produced nothing under a reserved
`_unresolved` key (additive — omitted when all codes resolve, so the
all-resolved payload is byte-identical to before) and wrap the
per-group loader.fetch so a blow-up is contained, not opaque. Richer
per-symbol status objects are a deliberate breaking follow-up.

(cherry picked from commit eba2044e982c5d467c808bad05386de3d856d635)

* fix(tools): reject invalid analyze_options inputs

Invalid inputs (volatility<=0, spot/strike<=0, negative expiry,
bad option_type) and T=0 all returned status:"ok" with silently
wrong / zeroed Greeks, and NaN could leak into the JSON (P06).

Validate at the boundary and reject invalid inputs with an error
envelope; flag the legitimate T=0 expiry as status:"degenerate"
with a warning (intrinsic value still returned); serialize with
allow_nan=False. The normal path is numerically unchanged. The
Greeks-convention rework (delta=0.5 at expiry) is a larger
follow-up, out of scope here.

(cherry picked from commit 823dc2382d73b97919807cf14a3644073886247d)

* fix(mcp): cap get_market_data row output

Every row of every symbol was emitted, so the canonical "1 symbol,
1 year, daily" request (~251 rows) breached the MCP token budget and
had to spool to a file, defeating an inline data tool (P07).

Add a max_rows cap (default 250): a symbol exceeding it returns a
head+tail window plus truncation metadata (rows/returned/truncated/
hint); symbols within the cap are unchanged (plain list, so small
queries are byte-identical); max_rows=0 restores the unbounded legacy
behavior. Columnar encoding and envelope de-duplication are broader
follow-ups, out of scope here.

(cherry picked from commit 0e426b1d0523e8c8144c699f89cd4847199e6448)

* fix(options): reject nan/inf + log dropped loader

options_pricing previously accepted NaN/Inf numeric inputs and
silently produced unreliable Greeks. Reject any non-finite spot,
strike, expiry_days, volatility, or risk_free_rate at the
validation boundary before pricing, and treat a missing result
key as non-finite so a dropped field is flagged degenerate
instead of defaulting to a confident 0.0.

mcp_server.get_market_data: the contained loader-failure path
(P05) now logs the swallowed exception with logger.exception
instead of vanishing silently; the _unresolved fallthrough is
unchanged.

* fix(mcp): stride row cap, reject neg max_rows

_cap_rows previously returned a head+tail window with a synthetic
{_gap: N} sentinel, hiding the middle of the series and emitting a
fake non-bar record. Replace with an even-stride downsample (every
step-th bar, last bar pinned) so the capped series spans the full
date range with only real bars.

A negative max_rows now enforces the default cap instead of being
treated as <=0 (which silently disabled the cap and returned an
unbounded payload). max_rows=0 still means unbounded; the in-cap
plain-list path is unchanged.
2026-05-16 23:47:32 +08:00
Teerapat Vatpitak 14ba958513 fix(loaders): bound ccxt + okx fetch (timeout, retry, budget) (#121)
* fix(loaders): bound ccxt fetch timeout + retry

ccxt_loader set no request timeout and paginated fetch_ohlcv in an
unbounded loop with no retry budget, so a transient disconnect hung
get_market_data for 10+ minutes (P12-b — it stalled a crypto E2E).

Set an explicit per-call timeout on the exchange, retry the transient
ccxt.NetworkError family with bounded backoff, and enforce a hard
wall-clock budget that raises a clear TimeoutError instead of
hanging. Non-network errors (e.g. bad symbol) are not retried; the
happy path still issues one call per page (no behavior change).
okx.py has the same shape — noted as a follow-up, out of scope here.

(cherry picked from commit ef934fbd70209d74a110e81757dbe322a2b717ab)

* fix(loaders): bound okx fetch retry + budget

okx.py set a per-request timeout but had no retry and no wall-clock
budget, so a transient blip dropped the whole symbol via the outer
silent skip, and a slow keyless tier could stall roughly
max_pages * timeout (P12-b parity with the ccxt loader).

Retry the transient requests.RequestException family with bounded
backoff and enforce a hard wall-clock budget that raises a clear
TimeoutError; non-network errors are not retried; the happy path
still issues exactly one request per page (no behavior change).

(cherry picked from commit e5030bf3edc3e44cc27003b3ac23e9e4fefc3c97)
2026-05-16 23:47:10 +08:00
Haozhe Wu 0e42cbe4c8 docs: sync readme roadmap status 2026-05-16 17:23:22 +08:00
Haozhe Wu 9769e5b543 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	agent/src/tools/web_reader_tool.py
2026-05-16 17:10:58 +08:00
Haozhe Wu 101eb9b614 feat: add security scanner and hypothesis registry 2026-05-16 17:10:06 +08:00
Teerapat Vatpitak bdb9dfca55 fix(tools): disclose read_url Jina dependency + cache opt-out (#122)
- SKILL.md now explicitly notes that read_url forwards the full URL to the third-party Jina Reader (r.jina.ai), and warns against passing credentials or internal URLs.
- New optional no_cache=True parameter sends x-no-cache to bypass Jina's cache when freshness matters.
- Responses that contain Jina's cached-snapshot marker now surface a cached: true flag.
- Error paths keep the upstream HTTP body and exception text in the returned error string (and also log them via logger.warning) so operators can diagnose remote failures without grepping logs.
2026-05-16 14:51:27 +08:00
Haozhe Wu b1acd830d2 fix(loop): populate reason field on AgentLoop cancelled / max-iter exits (#116)
Reporter saw 'Execution failed: unknown' when AgentLoop hit
max_iterations. The run's state.json recorded a useful reason but
loop.run()'s returned dict omitted it on the cancelled and failed
branches, so SessionService.mark_failed got 'unknown'.

Populate reason on both branches with actionable phrasing and add
iterations + max_iterations to the result dict.

Fixes #114
2026-05-16 01:56:11 +08:00
Haozhe Wu 8f49fe4902 feat(swarm): universal data-citation discipline in worker prompt (#115)
Lift the anti-fabrication rule from grounding.py into build_worker_prompt
as a universal Data Citation Discipline section that renders
unconditionally. Free-form prompts without explicit symbols previously
escaped the rule; aggregator / synthesizer agents with no data tools
had no defense against inventing numbers upstream omitted. Applies to
all 29 swarm presets via the central prompt builder.

Fixes #106
2026-05-16 01:49:53 +08:00
shadowinlifeandshadowinlife e346fa1adf feat(agent): add MCP client integration (stdio, v1) (#83)
Adds a stdio MCP client so the built-in agent can load external MCP tools alongside local tools.

- Pydantic `AgentConfig` + loader (JSON/YAML, ~/.vibe-trading discovery) with safe fallback
- `MCPServerAdapter` wraps FastMCP stdio client: per-tool timeout, single transient retry, schema normalization, sync-from-async bridge
- Per-server failure isolation in `build_registry` — one broken server never blocks local tools or other MCP servers
- Sanitization gate: `mcpServers` in session-level overrides is stripped by default (RCE prevention); opt-in via `ALLOW_SESSION_MCP_SERVERS=1`
- Server-name collision resolver with operator-facing warnings propagated through CLI + SessionService SSE
- v1 scope: stdio only, serial execution, tools only, no Swarm propagation
- 58 new tests (16 adapter + 17 config + 11 registry + 6 stdio subprocess + 6 regression + 2 sanitize/merge invariant)

Co-authored-by: shadowinlife <shadowinlife@users.noreply.github.com>
2026-05-16 01:17:47 +08:00
Haozhe Wu 55d81a6c53 docs: update May 15 README news 2026-05-16 00:18:27 +08:00
Haozhe Wu db21715cf9 feat(run-card): surface trust layer card in web UI 2026-05-15 11:51:18 +08:00
Teerapat Vatpitak dce3eb94c2 feat(memory): harden PersistentMemory.add() for control bytes, length, empty names (#112)
* feat(memory): harden PersistentMemory.add() for control bytes, length, empty names

Bundles three follow-up fixes for the same module, requested by
@warren618 in the issue triage of #108, #109, and #110.

#109 (highest priority of the three) — `add()` previously wrote the
full body to disk while `_scan_entries` clipped reads at
`MAX_ENTRY_CHARS`, so any content past the limit was "saved but
invisible" from inside Vibe-Trading. Enforce the limit at write time
and append a visible ``[truncated at N chars]`` marker so anyone
inspecting the file sees the clip explicitly.

#110 — Empty / whitespace-only names now raise ``ValueError``
("memory name must not be empty or whitespace-only") instead of
collapsing to the same ``{type}_.md`` filename. For names whose
sanitized slug normalizes to just underscores (emoji-only,
punctuation-only, etc.), append a 6-char SHA-256 prefix so distinct
inputs produce distinct files. The same input still produces the
same filename, so the "edit memory" workflow is unaffected.

#108 — Strip C0 (``U+0000-U+001F`` except ``\t`` ``\n``) and C1
(``U+0080-U+009F``) control bytes from agent-supplied content before
persisting. These cannot legitimately appear in finance notes and
would otherwise replay back through `memory show` as ANSI sequences
into the user's terminal.

Implementation:

- New module-level helpers `_sanitize_body` and `_truncate_body`
  (small, pure, regression-tested in isolation).
- New `_CONTROL_CHAR_RE` regex constant + `_TRUNCATION_MARKER`
  template, named for readable error/diff output.
- `add()` now strips control bytes before truncation so the marker is
  computed against user-visible content length.

Tests:

- 5 unit tests for `_sanitize_body` (ANSI escape, NUL/BEL, tab/newline
  preservation, C1 range, empty input).
- 4 unit tests for `_truncate_body` (short passthrough, at-limit,
  over-limit with marker, custom limit).
- 3 tests for empty/whitespace/tab-only name rejection.
- 3 tests for hash-suffix slug disambiguation (two emoji names produce
  distinct files; same name is deterministic; punctuation-only also
  gets a hash).
- 2 integration tests for `add()` end-to-end (control byte stripped on
  disk; long body truncated with marker on disk).

All 108 existing memory/CLI/remember-tool tests still pass.

Closes #108, #109, #110.

* fix(memory): reserve truncation marker space inside MAX_ENTRY_CHARS limit

E2E showed that appending the marker AFTER `body[:MAX_ENTRY_CHARS]`
makes the marker live past byte 8000 — _scan_entries clips reads at
MAX_ENTRY_CHARS, so consumers that inspect `entry.body` (rich.print,
recall scoring) never see the marker, defeating the "explicit
truncation" goal of #109.

Reserve the marker space *inside* the limit instead: head =
content[:limit - len(marker)] + marker. Total on-disk body stays ≤
MAX_ENTRY_CHARS, the marker survives the read-side clip, and
programmatic readers see the marker exactly the same as `cat` does.

Trade-off: when content exceeds the limit, ~28 chars of head content
is dropped to fit the marker. That is intentional — surfacing the
truncation event is more valuable than retaining 28 more bytes of
content that nobody could see was clipped.

Tests updated to assert the new invariant (`len(out) <= MAX`,
`out.endswith("chars]\n")`).
2026-05-15 11:28:02 +08:00
Haozhe Wu 1190ee9735 docs: update README news for wiki launch 2026-05-14 22:14:41 +08:00
Haozhe Wu eba0f1f8a1 docs: add wiki section shells 2026-05-14 22:02:32 +08:00
Haozhe Wu 298ada79b7 ci: deploy wiki to Cloudflare Pages 2026-05-14 21:26:15 +08:00
Haozhe Wu 7b27f6e3e2 Add static Vibe-Trading wiki
Add a static Cloudflare Pages wiki under wiki/, including the landing page, docs app, redirects/headers, maintainer README, and wiki validation workflow.
2026-05-14 19:31:31 +08:00
Teerapat Vatpitak 02a16a6422 fix(memory): extend tokenizer + slug regex to Thai/Arabic/Hebrew/Cyrillic (#104)
The previous CJK tokenizer ranges (#87, #95) only matched ``一-鿿``
and ``㐀-䶿``, so memory entries with Thai, Arabic, Hebrew, or
Cyrillic titles:

- Tokenized to the empty set, making recall always miss (e.g.
  ``find_relevant("ถัวเฉลี่ย")`` returned nothing even when the body
  contained the word).
- Had their slug characters stripped to ``_``, so two distinct Thai
  titles of equal length silently overwrote each other on disk.

The new ``_NON_LATIN_SCRIPT_RANGES`` constant covers CJK + Thai +
Arabic + Hebrew + Cyrillic and is reused by:

- ``_TOKEN_RE`` — single alternation pattern, one ``re.findall`` per
  ``_tokenize`` call (one text scan instead of two; precompiled at
  module level so it doesn't go through ``re.compile`` cache lookup on
  each invocation).
- ``_SLUG_DISALLOWED_RE`` — negation pattern used by ``add()``.

Arabic and Hebrew are deliberately narrowed to the basic letter blocks
(U+0620-U+064A, U+05D0-U+05EA) to keep bidi-control codepoints like
U+061C ARABIC LETTER MARK and combining marks out of on-disk slugs,
where they would render as invisible-but-distinct filenames.

Tests cover tokenization for each script, slug preservation
(parametrized across the four new scripts), and a Thai collision-
distinction regression.

Out of scope: ``agent/src/session/search.py`` has the same CJK-only
range in its FTS sanitizer; worth a follow-up PR to consume the same
constant.
2026-05-14 18:46:40 +08:00
Haozhe Wu 6b6683a026 docs: sync localized READMEs and feature assets 2026-05-14 02:48:18 +08:00
Haozhe Wu 45a9f525e4 docs: refresh README positioning and feature art 2026-05-14 02:21:48 +08:00
Haozhe Wu 75ecbce1fc docs: update README news for memory CLI 2026-05-14 00:46:09 +08:00
Haozhe Wu 4604d4c67a Merge pull request #102 from Teerapat-Vatpitak/feat/memory-introspection-cli
feat(cli): add memory introspection commands
2026-05-14 00:22:14 +08:00
Teerapat-Vatpitak 16102825b4 fix(memory): coerce frontmatter values to str in _scan_entries
The frontmatter parser returns lists for ``[a, b]`` syntax and bools
for ``true``/``false``, but MemoryEntry annotates ``title``,
``description``, and ``memory_type`` as ``str``. The mismatch was latent
until the new memory CLI tried to render those fields through
``rich.markup.escape``, which raised ``TypeError: expected string or
bytes-like object, got 'list'`` for any description that looked like a
YAML list (e.g. ``description: [red]inject[/red]``).

Add a small ``_coerce_str`` helper and route the three frontmatter
fields through it in ``_scan_entries`` so the dataclass type contract is
truthful at the storage boundary. Other consumers (recall scoring,
``remember_tool``) also benefit transparently.

Tests:

- Unit coverage of ``_coerce_str`` for str/None/list/bool inputs
- Storage-layer regression: a markdown file with a bracketed description
  parses to a string-typed description
- CLI regression: ``cmd_memory_list`` no longer crashes on entries with
  a bracketed description
2026-05-13 16:38:55 +07:00
Teerapat-Vatpitak 2d80b6248c style(cli): revert unintended formatter spillover on unrelated functions
The previous two commits inadvertently captured whole-file output from a
local pre-tool-edit formatter, reformatting ~30 functions
(`_print_status_bar`, `_format_tool_call_args`, `_build_benchmark_table`,
`_print_result`, `_build_welcome_panel`, ...) that have nothing to do
with the memory introspection feature.

Restore those functions to upstream `main` while keeping only the
intentional changes for this PR:

- `from datetime import datetime` hoisted to top imports (so the new
  CLI commands can use it without a per-call local import)
- `from rich.markup import escape as rich_escape` for safely rendering
  user-supplied memory titles/descriptions
- New `memory` subparser block in `_build_parser`
- New `cmd_memory_list / show / search / forget` plus the local
  `_MEMORY_TYPES` constant and `_MEMORY_TYPE_STYLES` dict
- Dispatch for `args.command == "memory"` in `main()`
- Drop the now-redundant local `from datetime import datetime` inside
  `cmd_trace`

Net diff for agent/cli.py shrinks from +296/-53 to +170/-1.
2026-05-13 16:27:40 +07:00
Teerapat-Vatpitak 0e0a584d85 perf(cli): avoid loading agent runtime on cli startup
Hoisting PersistentMemory to top-level cli.py made every CLI invocation
import `src.agent.frontmatter`, which transitively triggers
`src/agent/__init__.py` and eagerly loads AgentLoop, SkillsLoader (74
skills), and ToolRegistry. `vibe-trading --version` measured 7.6s
because of this chain.

- Move `from src.memory.persistent import PersistentMemory` back into
  each `cmd_memory_*` so only memory commands pay the import cost
- Mirror MEMORY_TYPES as a local `_MEMORY_TYPES` tuple inside cli.py so
  argparse `choices` and `_MEMORY_TYPE_STYLES` no longer require the
  module-level import. The module-load assertion still guards against
  drift between the local tuple and the style dict

`vibe-trading --version` drops from 7.6s to 0.66s (11.5x). Memory
commands keep the same latency since they need the module anyway.
2026-05-13 15:02:24 +07:00
Teerapat-Vatpitak 32a96fb5b2 feat(cli): add memory introspection commands
Add `vibe-trading memory list/show/search/forget` so users can inspect
and manage persistent memory without opening the file store directly.
Two recent memory bugs (#87 underscore tokenization, #95 CJK slug
collisions) were hard to spot without observability into what is
actually saved and recalled.

Also expand the PersistentMemory public surface to keep CLI lookup
logic out of storage internals:

- MEMORY_TYPES tuple as the single source of truth for type names,
  shared by argparse choices and the CLI style table
- list_entries() / find() / remove_entry() back the new commands;
  find() resolves by title first, then by on-disk filename stem so
  users can paste either form from the index
- remove_entry(entry) skips the re-scan that remove(title) does

Existing remove(title) is unchanged; remember_tool stays green.
2026-05-13 13:59:04 +07:00
Haozhe Wu b41141fa8f docs: update May 13 README news 2026-05-13 01:40:03 +08:00
Haozhe Wu 0678fd195a Merge commit 'refs/pr/93' 2026-05-13 01:15:28 +08:00
Haozhe Wu fb7efe76f6 Merge commit 'refs/pr/84' 2026-05-13 01:15:28 +08:00
Haozhe Wu 9e70dca97c fix(swarm): thread grounding data safely 2026-05-13 01:13:44 +08:00
Haozhe Wu e7d72be574 fix(swarm): redact worker event previews 2026-05-13 01:13:44 +08:00
Haozhe Wu 3981eb53f8 feat: add trust layer run cards 2026-05-12 18:44:19 +08:00
Peng QianandClaude Opus 4.6 ca26b0660a fix: resolve report.md into worker summary and sanitize event payloads
Address PR review feedback (HKUDS#84):

1. Read report.md back into WorkerResult.summary:
   - Add `_resolve_summary()` that checks for report.md in artifact dir
   - Applied at all 5 exit points (timeout, token_limit, llm_failure,
     normal_completion, iteration_limit)
   - Ensures API/UI shows the full report, not just the brief closing text

2. Sanitize event logging:
   - Truncate tool arguments to 200 chars per value
   - Rename "result" to "result_preview" and reduce from 5000 to 500 chars
   - Avoids persisting/streaming large or sensitive tool payloads

3. Add regression tests for report → summary handoff:
   - Test report.md content is returned when present
   - Test fallback behavior when report.md missing or empty
   - Test error handling for unreadable paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 19:15:12 +08:00
Peng QianandClaude Opus 4.6 5d90f3317f fix: add swarm runs to allowed paths and increase all preset timeouts
- Add `agent_root / ".swarm" / "runs"` to `_default_run_roots()` in
  path_utils.py. Without this, all `write_file` / `edit_file` / `read_file`
  calls from swarm workers fail with "run_dir is outside allowed run roots".
- Increase `timeout_seconds` from 600 to 1800 for all 29 swarm presets.
  The aggregator agent in commodity_research_team timed out at 615s, and
  complex analysis tasks routinely exceed 600s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 19:15:12 +08:00
Peng QianandClaude Opus 4.6 37e287c6c7 fix: persist swarm worker reports and improve summary extraction
Swarm workers frequently produced no usable output because:
- The Execution Rules prompt asked agents to write summaries "directly in
  your response," but the worker only captured `last_assistant_content`
  (often a mid-process message like "let me fix the script").
- On timeout/iteration limit, all in-memory messages were discarded.
- Event logging recorded only tool names and elapsed time, not I/O.

Changes to worker.py:
- Force agents to write final reports via `write_file report.md` (Phase 3),
  decoupling report persistence from the in-memory message lifecycle.
- Add `_best_summary()` to extract the longest assistant message (>100
  chars) instead of relying on the last fragment.
- Add `_persist_messages()` to save full conversation to messages.json on
  timeout/iteration-limit for post-mortem debugging.
- Include tool arguments in tool_call events and result content (up to 5000
  chars) in tool_result events for better observability.

Changes to equity_research_team.yaml:
- Increase timeout_seconds from 600 to 900 for all four agents, as the
  aggregator previously timed out at 615s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 19:15:12 +08:00
Haozhe Wu 3daa8dfae8 docs: update May 11 README news 2026-05-11 12:15:42 +08:00
ykj@hku 7320ac8d01 Improve CLI run UI and preflight startup check
Refresh the CLI run experience and add a startup preflight check for common environment issues.
2026-05-11 12:09:32 +08:00
Teerapat Vatpitak 96dbdb8050 feat(swarm): use real provider tokens for SwarmRun totals
Prefer provider-reported token usage for swarm runs while keeping the existing estimate fallback.
2026-05-11 12:09:21 +08:00
d 🔹 ac955c5692 fix(memory): preserve CJK chars in slug to avoid silent filename collisions
Preserve CJK characters in persistent memory slug generation and add regression coverage.
2026-05-11 12:09:11 +08:00
Teerapat Vatpitak 5731868a20 fix(swarm): route grounding fetch through _detect_market(code)
Empirical end-to-end verification on a real run revealed the dispatch
in fetch_grounding_data was wrong:

    loader_cls = resolve_loader(code)   # resolve_loader takes a market key
    loader     = loader_cls()           # resolve_loader already returns
                                        # a ready instance, not a class

So a real call with code="NVDA.US" produced
"No available data source for market 'NVDA.US'. Tried: []." — the
fallback chain table is keyed by market type ("us_equity",
"hk_equity", "crypto", ...), not by raw symbol shape.

The unit tests passed only because they monkey-patched resolve_loader
to a `lambda code: lambda: _StubLoader(...)` that happened to satisfy
both the wrong call site and the wrong return-type expectation. A
useful reminder that mock-shaped tests can hide real interface bugs.

Fix
---
- Import the same `_detect_market` runner.py uses for source routing
  and feed its output to `resolve_loader`. The private import is a
  reasonable pragmatic choice — there is no public equivalent today,
  and replicating the regex table in two places would invite drift.
- Drop the `loader_cls()` instantiation step; `resolve_loader` already
  returns a ready loader instance.

Test
----
Updated `test_fetch_returns_normalized_bars` to assert the call is
routed through `"us_equity"` (captured via a stub `resolve_loader`).
This pins the contract: a regression that passes the raw code as the
market key, or rewrites the helper so dispatch goes elsewhere, will
fire here instead of waiting for someone to actually run a swarm.

Verified with the local pipeline:
    NVDA.US latest close = $215.20 on 2026-05-08  (7 bars, yfinance)
i.e. the same number this PR was supposed to make available to the
worker prompt, instead of the empty block the original commit
silently produced.

`pytest agent/tests/test_swarm_grounding.py -v` -> 12/12 pass.
Full sweep at the same baseline (877 passed, 3 pre-existing Windows-
specific failures untouched, separately addressed in #88).
2026-05-10 20:19:29 +07:00
Teerapat Vatpitak b1d363b33c feat(swarm): pre-fetch market data and ground worker prompts in real prices
Symptom
-------
Workers in fundamental_research_team and similar presets routinely quote
prices from their model's training cutoff. A run targeting NVDA.US in
2026-05 produced a research report anchored on $145.89 while the actual
yfinance close that day was $215.20. Subsequent recommendations
("Hold/Accumulate, target $148-160") were therefore wrong by 30%+ before
any analyst even saw them. issue #42-class "weird outputs" land here too
when a worker silently substitutes a remembered number for a missing
data call.

Why prompt rules alone don't fix it
-----------------------------------
worker.py:158-171 already prints generic "use tools when needed"
guidance. LLMs nevertheless prefer the cheap path of recalling a number
over the expensive path of issuing a tool call — especially under the
20-tool-call hard limit imposed two paragraphs later. Adding a stronger
"never quote prices" rule helps but does not eliminate the failure mode.

Fix
---
Make the data unavoidable. SwarmRuntime.start_run now:

  1. Scans every value in user_vars for suffixed symbols
     (NVDA.US / 700.HK / 600519.SH / BTC-USDT / 000001.SZ / 430090.BJ).
  2. Pre-fetches the last 30 days of OHLCV via the existing loader
     registry (resolve_loader handles per-market routing). Failures
     are non-fatal — the run still proceeds, just without grounding.
  3. Pins the result on a new SwarmRun.grounding_data field and
     persists it via the existing run.json path.

SwarmRuntime._execute_run renders the data into a markdown "Ground
Truth" block once per run and threads it through
_run_worker_with_retries -> run_worker -> build_worker_prompt, which
splices it ahead of Execution Rules. The block contains the recent
window range, last close, the last 5 daily closes, and an explicit
instruction to prefer these prices over training-data prices and to
call get_market_data for any range outside the window.

Backward compatibility
----------------------
* SwarmRun.grounding_data defaults to None; existing run.json files
  parse unchanged (Pydantic deserialises the missing key to the
  default). The legacy regression case is covered by
  test_swarm_run_metadata.py once that lands; this PR's tests cover the
  new module and prompt plumbing in isolation.
* build_worker_prompt and run_worker each gain a grounding_block= kw
  with default "" so callers outside the swarm runtime keep working.
* Empty grounding (no symbols detected, or every fetch failed) skips
  the entire section — no empty heading rendered.

What's not in scope
-------------------
* Bare tickers without a suffix (just "NVDA") are intentionally not
  matched. The false-positive cost on common English words is too high
  and every shipped preset already supplies suffixed symbols.
* The block is a one-shot snapshot at run start. Long-running swarms
  may see stale data after many minutes — still strictly better than
  training-data prices from a year ago. A refresh hook can land later.

Tests
-----
agent/tests/test_swarm_grounding.py covers symbol extraction (all four
loader-supported suffix shapes, ordering, non-string values, word
boundaries), fetch behaviour with a stubbed loader (normalised bars,
empty-frame skip, empty-input no-op), markdown rendering (range / last
close / explicit instruction text), and end-to-end prompt plumbing
(grounding block lands before Execution Rules; absent block omits the
section). All 12 pass; full pytest sweep against main shows the same
pre-existing 3 failures (test_backtest_runner_security x2,
test_loop_helpers normalize) as main and 875 passed (= baseline 863 +
the new 12 grounding tests).
2026-05-10 20:19:29 +07:00
Haozhe Wu fbe37e0f63 docs: update May 10 README news 2026-05-10 20:36:15 +08:00
Haozhe Wu 1e0a7a0dd8 Merge pull request #92 from Teerapat-Vatpitak/feat/swarm-record-provider-model
feat(swarm): record provider/model on SwarmRun for cost audit + debug
2026-05-10 20:24:34 +08:00
Haozhe Wu bb977f2614 Merge pull request #91 from Teerapat-Vatpitak/fix/runner-validate-run-dir
fix(backtest): validate run_dir in CLI entry against allowed run roots
2026-05-10 20:24:24 +08:00
Haozhe Wu 209334d609 Merge pull request #90 from Teerapat-Vatpitak/fix/api-server-bare-except
fix(api_server): replace bare except: with specific exception types
2026-05-10 20:24:17 +08:00
Haozhe Wu f55ef1ffb9 Merge pull request #88 from Teerapat-Vatpitak/fix/windows-test-compat
test: make path-handling tests pass on Windows runners
2026-05-10 20:24:10 +08:00
Teerapat Vatpitak ba7cd19564 feat(swarm): record provider/model on SwarmRun for cost audit + debug
run.json currently has no field describing which LLM provider/model the
swarm was actually launched against — SwarmAgentSpec.model_name carries
per-agent overrides but defaults to None ("use the global config"), and
that resolved config is never captured anywhere. Cost auditing,
side-by-side provider experiments, and post-hoc bug triage all need this
information; today it has to be reconstructed from .env timestamps.

This adds two optional fields to SwarmRun:

- provider: lowercase form of LANGCHAIN_PROVIDER, the same env var
  src/providers/llm.py:136,195 reads when constructing ChatLLM.
- model: LANGCHAIN_MODEL_NAME (raw, since model strings are
  case-sensitive — "gpt-4o" vs "openai/gpt-5").

Both default to None and are populated in SwarmRuntime.start_run before
the run is persisted (runtime.py:start_run), so every new run.json from
this point forward records the active provider/model. Per-agent
overrides remain visible on SwarmAgentSpec.model_name; this field is the
run-level default.

Backward compat: existing run.json files lack the keys entirely. Pydantic
deserializes missing optional fields to their default (None), so
SwarmStore.get_run / list_runs continue to work on legacy files. A
parametrized regression test covers this in
agent/tests/test_swarm_run_metadata.py along with construction,
serialization round-trip, and acceptance of multiple provider names.
2026-05-10 15:49:32 +07:00
Teerapat Vatpitak 80e177228f fix(backtest): validate run_dir in CLI entry against allowed run roots
backtest.runner.main() previously accepted any Path it was handed and
went straight to reading config.json + importing code/signal_engine.py
from that location. The MCP backtest tool already routes through
safe_run_dir (src/tools/backtest_tool.py:23), but
"python -m backtest.runner <path>" did not — meaning a contributor or
agent that can drop config.json + code/signal_engine.py anywhere on
disk could have it loaded and executed by the CLI path.

The AST scrubber at runner.py:217-239 blocks executable top-level
statements, but any function or method body still runs on instantiation
or method call (BaseEngine.run_backtest -> SignalEngine().__init__ ->
.generate(...)), so the AST guard is not a substitute for a path
boundary.

Reuse the existing safe_run_dir() (src/tools/path_utils.py:190) at the
top of main() so the CLI honours the same VIBE_TRADING_ALLOWED_RUN_ROOTS
whitelist as the MCP path. Errors are surfaced through the existing
{"error": "..."} JSON-on-stdout + sys.exit(1) convention this entry
point already uses.

Existing test_path_safety.py exercises safe_run_dir directly and still
passes; behaviour for run dirs under the default roots
(agent/runs, $cwd/runs, ~/.vibe-trading/shadow_runs) and any path listed
in VIBE_TRADING_ALLOWED_RUN_ROOTS is unchanged.
2026-05-10 15:49:31 +07:00
Teerapat Vatpitak 51d0188cc5 fix(api_server): replace bare except: with specific exception types
Three bare except: clauses in api_server.py silently caught BaseException
(KeyboardInterrupt, SystemExit, GeneratorExit) in addition to ordinary
errors. That makes the API server unresponsive to Ctrl+C inside these
blocks and hides real bugs (NameError, AttributeError) as silent passes.

The blocks themselves are best-effort metadata extraction from per-run
artifact files, where falling through with the default values is the
right behavior — but only for the specific I/O / parse failures these
blocks can actually produce. The new clauses spell those out:

- lines 1015, 1022 (req.json / planner_output.json prompt extraction):
    except (json.JSONDecodeError, OSError)
- line 1042 (metrics.csv numeric parse):
    except (OSError, ValueError)

Behavior is unchanged for the failures the blocks are designed to swallow;
KeyboardInterrupt, SystemExit, and programming errors now propagate as
they should.
2026-05-10 15:49:31 +07:00
Teerapat Vatpitak 5eb8390921 test: make path-handling tests pass on Windows runners
Three tests fail when the suite is run on Windows even though the
underlying behaviour is correct on both platforms — the failures are
test-side platform assumptions, not application bugs. Linux CI is
unaffected; this is purely a quality-of-life fix for contributors who
run pytest on Windows.

1. test_backtest_runner_security.py — rce path interpolation
   ----------------------------------------------------------
   Both ``test_signal_engine_rejects_top_level_execution`` and
   ``test_signal_engine_rejects_class_level_execution`` write a
   ``signal_engine.py`` whose source contains the pytest tmp_path
   formatted into an ``os.system('touch ...')`` call. On Windows,
   tmp_path looks like ``C:\Users\teera\AppData\Local\Temp\...``;
   interpolated into a single-quoted Python string literal, the leading
   ``\U`` is interpreted as a (truncated) unicode escape and ``ast.parse``
   raises ``SyntaxError`` before the security scrubber under test ever
   runs. Switching to ``Path.as_posix()`` produces forward-slash paths
   that survive Python source escaping intact while pointing at the same
   filesystem location on every OS.

2. test_loop_helpers.py — absolute-path preservation
   --------------------------------------------------
   ``test_preserves_absolute_run_dir`` passed the literal
   ``/var/tmp/custom_run`` and expected it back unchanged.
   ``_normalize_tool_run_dir`` calls ``Path.is_absolute()``, which on
   Windows returns False for a bare drive-less path — so the function
   correctly treats the input as relative and joins it with
   ``memory_run_dir``. The test was wrong, not the function: it asserted
   POSIX semantics on a cross-platform helper. Resolving the input
   through ``os.path.abspath`` before handing it to the function
   produces the platform's actual absolute form (``/var/tmp/custom_run``
   on POSIX, ``C:\var\tmp\custom_run`` on Windows) and the assertion now
   holds on both.

Verified locally: the targeted three now pass; the full pytest sweep
goes from "3 failed, 863 passed" on main to "866 passed, 0 failed".
No production code is touched.
2026-05-10 15:47:09 +07:00
Hitesh PandClaude Opus 4.7 09dd59e9aa fix(memory): treat underscores as word boundaries in _tokenize (#87)
Memory titles saved through RememberTool are slugified to snake_case
(e.g. "mcp_wiring_test"), so the previous regex `[a-zA-Z0-9_]{3,}`
collapsed them into a single token. A natural-language recall query
like "mcp wiring" produces tokens {"mcp", "wiring"} and never
intersects {"mcp_wiring_test"} — recall returned 0 results despite
the memory existing on disk.

Drop `_` from the character class so underscores act as token
boundaries. Verbatim queries still work because the same string
splits into the same tokens on both sides of the comparison.

Adds a regression test for the snake_case case.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 15:43:20 +08:00
Teerapat Vatpitak 5eef5f3bd3 test(mcp_server): add smoke test for tools/call deadlock regression (#86)
Spawns vibe-trading-mcp via subprocess and walks through the JSON-RPC
happy path: initialize + tools/list + tools/call analyze_options. The
critical assertion is that tools/call does not hang — without the
registry pre-warm in mcp_server.main() (PR #85), the lazy registry build
runs inside FastMCP's worker thread and deadlocks on the first call.

The test also verifies (a) the tool catalogue has at least 20 entries
including analyze_options/get_market_data/list_skills, and (b) the
analyze_options result has a positive price and a delta in [0, 1] —
covering tool-registration regressions and Black-Scholes formula breaks
in addition to the deadlock.

analyze_options is chosen over network-dependent tools (get_market_data,
web_search) so the test stays deterministic and runs offline in CI.

Verified locally with differential validation:
- with pre-warm: PASS in 3.91s
- pre-warm commented out: FAIL with 15s timeout on tools/call
- pre-warm restored: PASS in 3.81s
2026-05-10 15:35:06 +08:00
Haozhe Wu 9fe6698d17 docs: update May 9 README news 2026-05-09 22:00:58 +08:00
Haozhe Wu 79efe1d6a8 Merge pull request #85 from Teerapat-Vatpitak/fix/mcp-server-tools-call-deadlock
fix(mcp_server): pre-warm registry in main thread to avoid tools/call deadlock
2026-05-09 15:13:48 +08:00
Teerapat Vatpitak dd594080c3 fix(mcp_server): pre-warm registry in main thread to avoid tools/call deadlock
_get_registry() lazy-imports src.tools.build_registry on first call. Every
caller is a FastMCP tool handler, so the first tools/call triggers the lazy
init inside FastMCP's asyncio worker thread, where the import chain (notably
src.tools.shell.*) deadlocks. initialize and tools/list succeed because
they don't touch the registry.

Pre-warming _get_registry() in main() — after _include_shell_tools is set
and before mcp.run() — performs the import on the main thread and lets
worker threads only ever read the cached singleton. _get_registry() is
idempotent (if _registry is None guard), so the 14 existing call sites are
unaffected.

Verified deterministically: removing the line reintroduces the hang within
15s; adding it back fixes tools/call within 0.01s. End-to-end through
Claude Code's MCP client returns in <2s.
2026-05-09 12:36:43 +07:00
Haozhe Wu 5f411ddaf7 Merge pull request #80 from SJoon99/fix/validate-path-params
fix(api): validate run_id and session_id path parameters
2026-05-09 09:52:11 +08:00
Haozhe Wu b9a82655bd fix(api): reject newline path parameters 2026-05-09 09:51:50 +08:00
Haozhe Wu bd722334d3 Merge pull request #82 from voidborne-d/fix/frontend-proxy-vite-api-url
fix(frontend): honor VITE_API_URL env for vite dev proxy target
2026-05-08 22:30:24 +08:00
voidborne-d 977fddfbff fix(frontend): honor VITE_API_URL env for vite dev proxy target
vite.config.ts hardcoded http://localhost:8899 across 14 proxy entries,
ignoring the VITE_API_URL env that docker-compose.yml passes to the
frontend service (http://vibe-trading:8899). Inside the frontend
container, localhost resolves to the frontend's own network namespace,
so all proxied requests (sessions, runs, skills, etc.) ECONNREFUSED.

Read VITE_API_URL via loadEnv with the original localhost:8899 default,
and DRY the per-route target into a single PROXY_PATHS list driven by
Object.fromEntries.

Closes #78
2026-05-08 21:21:59 +08:00
SJoon99 e2535cc7c3 fix(api): validate run_id and session_id path parameters
Endpoints under /runs/{run_id}, /swarm/runs/{run_id} and /sessions/{session_id}
constructed filesystem paths as `RUNS_DIR / run_id / ...` without validating
the path-parameter value. An authenticated caller could submit values like
`..` or `foo/../bar` and reach paths outside the intended directory.

The same file already validates `{shadow_id}` with `_SHADOW_ID_RE`. This
change applies the same hardening to `run_id` and `session_id` via a shared
`_validate_path_param()` helper that enforces a safe character class
(`^[A-Za-z0-9_-]{1,128}$`). The class accepts every ID format produced by the
codebase today (regular run_id from `state.create_run_dir`, swarm
`swarm-{ts}-{short_uuid}`, session_id from `uuid.uuid4().hex[:12]`) and
rejects path separators, dots, NUL, whitespace, and oversize input.

13 endpoints are guarded:
  - GET /runs/{run_id}/code, /pine, /
  - GET|DELETE|PATCH /sessions/{session_id}
  - POST /sessions/{session_id}/messages, /cancel
  - GET /sessions/{session_id}/messages, /events
  - GET /swarm/runs/{run_id}, /events
  - POST /swarm/runs/{run_id}/cancel

Tests added in `test_security_auth_api.py`:
  - parametrized validator unit tests (5 valid IDs / 9 invalid inputs)
  - per-endpoint integration tests using FastAPI TestClient
2026-05-08 07:50:36 +00:00
Haozhe Wu 52b624d3c8 docs: update news for Tushare statement filters 2026-05-08 10:53:35 +08:00
Haozhe Wu 4115960a42 Merge pull request #77 from HKUDS/fix/fundamental-fields-hard-fail
fix: fail fast on Tushare fundamental enrichment errors
2026-05-08 10:41:11 +08:00
Haozhe Wu 4fd82a0952 fix: fail fast on Tushare fundamental enrichment errors 2026-05-08 10:38:06 +08:00
Haozhe Wu 4ae673b6d1 Merge pull request #76 from mrbob-git/codex/vibe-trading-issue-62-prefilter
feat: wire Tushare fundamentals into filters
2026-05-08 10:33:56 +08:00
MrBob 94d0589638 feat: wire tushare fundamentals into filters 2026-05-07 15:16:07 -03:00
Haozhe Wu 0f4e3d7110 docs: sync May 7 translated news 2026-05-07 13:40:17 +08:00
Haozhe Wu d1273171a6 docs: update May 7 news 2026-05-07 13:38:09 +08:00
Andrew Barnes 947d945827 add Tushare fundamental provider contract
Adds a point-in-time Tushare fundamental provider and tests the project TUSHARE_TOKEN path.
2026-05-07 13:27:49 +08:00
Haozhe Wu 41fa4ad0b0 docs: fold older README news entry 2026-05-06 20:31:06 +08:00
Haozhe Wu 2fc8c90d9f docs: update README for 0.1.7 release 2026-05-06 20:21:13 +08:00
685 changed files with 49343 additions and 2049 deletions

No files matched your search

+32
View File
@@ -0,0 +1,32 @@
name: Deploy Wiki
on:
push:
branches:
- main
paths:
- "wiki/**"
- ".github/workflows/wiki-deploy.yml"
concurrency:
group: wiki-deploy
cancel-in-progress: true
jobs:
deploy:
name: Deploy to Cloudflare Pages
runs-on: ubuntu-latest
permissions:
contents: read
deployments: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy wiki
uses: cloudflare/wrangler-action@v3
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
command: pages deploy wiki --project-name=vibetrading-wiki --branch=main
gitHubToken: ${{ secrets.GITHUB_TOKEN }}
+52
View File
@@ -0,0 +1,52 @@
name: Wiki
on:
push:
branches: [main]
paths:
- "wiki/**"
- ".github/workflows/wiki.yml"
pull_request:
branches: [main]
paths:
- "wiki/**"
- ".github/workflows/wiki.yml"
jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Check JavaScript syntax
run: |
node --check wiki/main.js
node --check wiki/theme.js
node --check wiki/docs/content.js
node --check wiki/docs/main.js
- name: Check locale JSON
run: python -m json.tool wiki/locales/en.json >/dev/null
- name: Check static site files
run: |
test -f wiki/index.html
test -f wiki/home/index.html
test -f wiki/docs/index.html
test -f wiki/tutorials/index.html
test -f wiki/alpha-library/index.html
test -f wiki/research-lab/index.html
test -f wiki/_redirects
test -f wiki/_headers
test -f wiki/assets/icon.png
! grep -R "localhost\\|nanobot" wiki --exclude=README.md
+14
View File
@@ -21,6 +21,12 @@ agent/tests/*_result.json
.ruff_cache/
frontend/dist/
frontend/tsconfig.tsbuildinfo
wiki/node_modules/
wiki/dist/
wiki/.astro/
wiki/.vite/
wiki/.wrangler/
wiki/.cache/
.vibe-dev/
.vibe-record/
.cache/
@@ -35,6 +41,8 @@ data/backup/
# Internal docs (plans, specs)
docs/
!wiki/docs/
!wiki/docs/**
data/minutes/
data/overnight*.log
data/*.log
@@ -63,3 +71,9 @@ NOTES.md
# Agent tool aliases (auto-generated mirror of CLAUDE.md)
AGENTS.md
.DS_Store
# Alpha Zoo wiki artifacts (generated, not source)
wiki/alpha-library/manifest.json
wiki/alpha-library/content/
+122
View File
@@ -0,0 +1,122 @@
# Changelog
All notable changes to Vibe-Trading are documented in this file.
This project adheres to [Semantic Versioning](https://semver.org/).
## [Unreleased]
### Added
### Changed
### Fixed
## [0.1.8] — 2026-05-17
### Added — Alpha Zoo (450+ pre-built quant alphas)
- `agent/src/factors/` — base operators (`rank`, `scale`, `ts_*`, `delta`,
`decay_linear`, `signed_power`, `safe_div`, market-aware `vwap`) and a
registry that AST-extracts metadata from each alpha module without
importing it. Lookahead is enforced at the operator level
(`delta(d>=1)`), and registry sanity checks reject `+/-inf` and
outputs that are more than 95 % NaN.
- 4 zoos shipping 452 alphas total:
- **qlib158** (154 alphas) — port of Microsoft Qlib's `Alpha158`
feature handler under Apache-2.0, with pinned commit SHA per file.
- **alpha101** (101 alphas) — implementation of Kakushadze (2015)
*"101 Formulaic Alphas"* (arXiv:1601.00991), written from the paper
appendix; the relevant trademarked string is intentionally absent.
- **gtja191** (191 alphas) — implementation of Guotai Junan's 2014
*"191 Short-period Trading Alpha Factors"* research report.
- **academic** (6 factors) — Fama-French 5 + Carhart momentum, shipped
as honest price-based proxies (not the canonical FF series).
- `vibe-trading alpha {list,show,bench,compare,export-manifest}` CLI
subcommand. `show` and `export-manifest` enforce path-traversal guards.
- New agent tools: `AlphaZooTool` (browse) and `AlphaBenchTool`
(orchestrator with Jinja2 autoescape + strict CSP HTML report).
- `ZooSignalEngine.from_zoo(...)` — composite multi-factor signal engine
with cross-sectional standardisation, weighting, and optional top-N /
bottom-N long-short conversion.
- `wiki/scripts/build_alpha_library.py` — Alpha Library renderer.
Reads `manifest.json` produced by `vibe-trading alpha export-manifest`
and emits 452 per-alpha HTML pages plus 4 per-zoo overviews, each with
`script-src 'none'` CSP. The landing page hydrates per-zoo counts
from `content/index.json`.
- New blog post: *"Which of the 191 GTJA alphas still work in 2026?"*
with aggregate IC statistics, theme breakdown, and the top alphas
that survive eight years of out-of-sample data.
### Added — Web UI for Alpha Zoo
- New page at `/alpha-zoo` in the Vite + React frontend with three
views: browse (4 zoo cards + filter bar + paginated table), detail
(formula, metadata, collapsible source code), and bench-runner
(form → SSE-streamed progress + Alive/Reversed/Dead stat cards +
Top-5-by-IR table + by-theme breakdown chart). "Alpha Zoo" nav
entry added to the layout.
- Four new REST routes in the FastAPI server:
- `GET /alpha/list` — filterable alpha catalogue
- `GET /alpha/{alpha_id}` — meta + source code
- `POST /alpha/bench` — kicks off a background bench job and
returns a `job_id`
- `GET /alpha/bench/{job_id}/stream` — Server-Sent Events with
`progress`, `result`, `done`, and `error` event types. In-memory
job state with a 1-hour TTL; no Redis/Celery dependency.
- Bench math is refactored into `agent/src/factors/bench_runner.py`
so the CLI driver (`agent/scripts/w4a_run_benches.py`) and the new
API worker share a single implementation.
### Added — Safety floor
- `agent/tests/factors/test_alpha_purity.py` — AST allowlist scan over
every `zoo/**/*.py` module (whitelist: pandas, numpy, scipy.\*,
`src.factors.base`, `__future__`, `typing`, `math`, `dataclasses`;
banned: `os`, `sys`, `subprocess`, `socket`, `urllib`, `requests`,
`httpx`, `pathlib`, `Path`, `open`, `eval`, `exec`, `compile`,
`__import__`, and `getattr(_, "__*")`).
- `agent/tests/factors/test_lookahead.py` — sentinel future-row
injection on a 300-row synthetic panel; corrupting rows after the
probe must leave the probe value unchanged within 1e-9.
- `tools/ci_grep_gates.sh` — CI gate that rejects `yaml.load(` without
`safe_load`, any trademarked-name leak in shipped artifacts, and any
per-stock-code data leak in `wiki/**/*.{json,csv,html}`.
- `agent/tests/factors/conftest.py` — opt-in `pytest-socket` integration
that hard-fails any test attempting outbound network during the
factors test suite.
### Added — Community governance
- `CONTRIBUTING.md` — Developer Certificate of Origin sign-off
requirement and a contributor checklist for new alpha PRs (purity,
lookahead, `__alpha_meta__` shape, LaTeX-matches-code, per-zoo
LICENSE.md, DCO).
- `NOTICE` (repo root) — Apache-2.0 attribution for Qlib and a
declaration that the bundled formulas from Kakushadze, GTJA, and the
academic baselines are mathematical content (paper prose, tables, and
figures are not reproduced here).
- Per-zoo `LICENSE.md` for each of `qlib158/`, `alpha101/`, `gtja191/`,
and `academic/`, plus an upstream `NOTICE` for `qlib158/`.
### Changed
- `agent/src/tools/factor_analysis_tool.py` extracted its IC/IR and
layered-backtest helpers to `agent/src/factors/factor_analysis_core.py`
so the new `alpha_bench_tool` reuses the same maths. Public tool
signature is unchanged; `_compute_ic_series` and `_compute_group_equity`
remain importable as backward-compatible aliases.
- `agent/cli.py` grew by 7 lines to register the `alpha` subcommand;
all handler logic lives in `agent/src/factors/cli_handlers.py`.
- Packaging: `pyproject.toml` now ships `zoo/**/*.yaml`, `zoo/**/*.md`,
and `zoo/**/NOTICE` as package data; `MANIFEST.in` recursively
includes `agent/src/factors`.
### Known limitations
- The `btc-usdt` universe is single-asset; cross-sectional IC requires
≥2 instruments, so the bundled `alpha101_btc` bench run returns
alive/reversed/dead = 0/0/0 by construction. Use a multi-symbol crypto
basket (e.g. BTC + ETH + SOL + the top-N perpetuals) for meaningful
cross-sectional results; a curated `crypto-majors` universe is planned
for 0.2.
### Internal
- `wiki/alpha-library/manifest.json` and `wiki/alpha-library/content/`
are generated artifacts and gitignored. Run
`vibe-trading alpha export-manifest --out wiki/alpha-library/manifest.json
--force` followed by `python wiki/scripts/build_alpha_library.py` to
regenerate the static site.
+123 -140
View File
@@ -1,169 +1,152 @@
# Contributing to Vibe-Trading
Thanks for your interest in contributing! This guide will help you get started.
Vibe-Trading is a natural-language finance research AI agent (FastAPI + ReAct
agent backend, Vite+React frontend, vectorized daily and options backtesting).
This guide covers contribution governance: the Developer Certificate of Origin
(DCO) sign-off requirement, the reviewer checklist for Alpha Zoo factor
contributions, and the quickstart for adding a new alpha.
## Quick Links
For general project setup (`pip install -e ".[dev]"`, dev servers,
`pytest --ignore=agent/tests/e2e_backtest`), see the README. For bug reports
and feature requests, use the GitHub issue templates.
- [Discord](https://discord.gg/2vDYc2w5) — ask questions, discuss ideas
- [Issues](https://github.com/HKUDS/Vibe-Trading/issues) — bug reports & feature requests
- [Good First Issues](https://github.com/HKUDS/Vibe-Trading/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) — great starting points
## Developer Certificate of Origin (DCO)
## Development Setup
Every commit in a community pull request MUST carry a `Signed-off-by:`
trailer. We do not require a CLA — the DCO is a lightweight per-commit
attestation that you wrote the code or have the right to submit it under
the project's MIT license. Maintainer-direct commits to `main` are not
subject to the trailer requirement (the maintainer's authorship is
already attested by the commit's author field), but community PRs are.
Sign your commits with `-s`:
```bash
git clone https://github.com/HKUDS/Vibe-Trading.git
cd Vibe-Trading
python -m venv .venv
source .venv/bin/activate # Linux/macOS
# .venv\Scripts\Activate.ps1 # Windows
pip install -e ".[dev]"
cp agent/.env.example agent/.env
# Edit agent/.env — set your LLM provider
git commit -s -m "feat(factors): add gtja191 alpha 042"
```
### Run tests
This appends a trailer like:
```bash
pytest --ignore=agent/tests/e2e_backtest --tb=short -q
```
Signed-off-by: Your Name <you@example.com>
```
### Start dev servers
PRs without a `Signed-off-by:` on every commit will be asked to rebase and
resign. To fix an unsigned series, run
`git rebase --signoff <base-branch>` and force-push the branch.
```bash
# Terminal 1: API server
vibe-trading serve --port 8899
### DCO 1.1 (full text)
# Terminal 2: Frontend
cd frontend && npm install && npm run dev
```
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```
## Project Structure
## Alpha PR Reviewer Checklist
| Directory | What lives here | Open to contribute? |
|-----------|----------------|:-------------------:|
| `agent/src/skills/` | 74 finance skill definitions (SKILL.md) | Yes |
| `agent/src/tools/` | 21 agent tools | Yes |
| `agent/backtest/` | Backtest engines, loaders, optimizers | Yes |
| `agent/src/swarm/presets/` | 29 swarm preset YAMLs | Yes |
| `frontend/` | React 19 + Vite web UI | Yes |
| `agent/src/agent/` | ReAct agent core (loop, context, skills) | Ask first |
| `agent/src/session/` | Session management | Ask first |
| `agent/src/providers/` | LLM provider abstraction | Ask first |
This checklist applies to any PR adding or modifying files under
`agent/src/factors/zoo/**/*.py`. Reviewers MUST verify every box before
merging. Authors are strongly encouraged to self-check first.
> **Core modules** (`agent/`, `session/`, `providers/`) are protected — please open an issue to discuss before modifying.
- [ ] **Purity gate**: file passes `pytest agent/tests/factors/test_alpha_purity.py`.
The AST scan rejects any imports outside the allowlist
(`pandas`, `numpy`, `scipy.*`, `src.factors.base`, `__future__`, `typing`,
`math`, `dataclasses`) and any reference to forbidden names: `os`,
`subprocess`, `socket`, `urllib`, `requests`, `httpx`, `pathlib`, `Path`,
`eval`, `exec`, `compile`, `__import__`, bare `open`, or `getattr` with a
second argument beginning with `"__"`.
- [ ] **Lookahead gate**: file passes `pytest agent/tests/factors/test_lookahead.py`.
No negative shifts, no forward leakage. `delta(df, d)` must have `d >= 1`.
- [ ] **`__alpha_meta__` present** with all required pydantic-validated fields:
`id`, `theme`, `formula_latex`, `columns_required`, `universe`, `frequency`,
`decay_horizon`, `min_warmup_bars`. Optional: `nickname`, `extras_required`,
`requires_sector`, `notes`.
- [ ] **`compute(panel)` contract**: returns a DataFrame with the same shape
as `panel["close"]`, NaN preserved at warmup / missing-data positions,
no `+/-inf` values.
- [ ] **LaTeX matches code**: the `formula_latex` in `__alpha_meta__` and the
formula in the module docstring describe what `compute()` actually does.
- [ ] **Per-zoo `LICENSE.md` updated** to cite the source paper / report and
to state that formulas are mathematical facts, not subject to copyright,
and that prose / tables / figures from papers and reports are not
reproduced in this repo. Do NOT frame the bundled formulas using US
affirmative-defense terminology — that is a litigation posture, not a
license grant. State the mathematical-facts rationale instead.
- [ ] **Apache-2 attribution**: if the alpha is adapted from an Apache-2.0
upstream (e.g. Microsoft Qlib), the file MUST carry a header of the form
`# Adapted from <repo>@<commit-sha>:<path> (Apache-2.0). Copyright (c) <holder>.`
- [ ] **DCO**: every commit in the PR carries `Signed-off-by:`.
## How to Contribute
## Adding a New Alpha (Quickstart)
### Reporting Bugs
Use the [Bug Report](https://github.com/HKUDS/Vibe-Trading/issues/new?template=bug_report.yml) template. Include:
- Steps to reproduce
- Expected vs actual behavior
- Your environment (OS, Python version, provider)
- Error logs if available
### Suggesting Features
Use the [Feature Request](https://github.com/HKUDS/Vibe-Trading/issues/new?template=feature_request.yml) template.
### Submitting Code
1. **Fork & branch** — create a feature branch from `main`
1. Pick the target zoo directory under `agent/src/factors/zoo/` (e.g.
`gtja191/`, `alpha101/`, `qlib158/`, `academic/`).
2. Create `<alpha_id_short>.py` in that directory. Define `__alpha_meta__`
(must satisfy the pydantic `AlphaMeta` schema in
`agent/src/factors/registry.py`) and a pure `compute(panel)` function
that imports only from `src.factors.base` plus the allowlisted stdlib /
numpy / pandas / scipy.
3. Run the purity and lookahead gates locally:
```bash
git checkout -b feat/my-feature
pytest agent/tests/factors/test_alpha_purity.py agent/tests/factors/test_lookahead.py -q
```
2. **Make changes** — follow the code standards below
3. **Test** — add or update tests, ensure `pytest` passes
4. (Optional but recommended) Run a quick bench:
```bash
pytest --ignore=agent/tests/e2e_backtest --tb=short -q
vibe-trading alpha bench --zoo <zoo_id> --universe csi300 --period 2020-2025
```
5. Open a PR. Every commit must include `Signed-off-by:` (use
`git commit -s`). Reviewers will walk the checklist above.
4. **Commit** — use [Conventional Commits](https://www.conventionalcommits.org/)
```
feat: add RSI divergence skill
fix: handle empty DataFrame in backtest loader
docs: update CLI reference table
test: add crypto engine edge cases
```
## Code Style
5. **Pull Request** — open a PR against `main` with:
- Summary of changes
- Related issue number (e.g., `Closes #42`)
- Test plan
- Format with `black`; lint with `ruff` (config in `pyproject.toml`).
- Type-annotate all public function and method signatures.
- Google-style docstrings (`Args:` / `Returns:` / `Raises:`).
- Keep files under 400 lines where practical, 800 hard cap.
- No hardcoded paths, secrets, or URLs — config via `.env`, YAML, or
module-level constants.
- Delete unused code rather than commenting it out.
## Code Standards
## Attribution
- **Python**: Google-style docstrings, type hints encouraged
- **No hardcoding**: config via `.env`, YAML, or constants
- **Delete unused code** — don't comment-preserve
- **OKX pairs**: use `BTC-USDT` format (hyphen, uppercase)
- **UI text**: English. LLM output follows user language
- **File size**: aim for < 400 lines, max 800
Do NOT add `Co-Authored-By:` trailers or AI-assistant attribution lines to
commit messages or PR descriptions. The DCO sign-off is the only required
trailer; keep commit metadata clean.
## Contribution Roadmap
### Easy: New Skill
Each skill is a single `SKILL.md` file in `agent/src/skills/<category>/<skill-name>/`. Categories: `data-source`, `strategy`, `analysis`, `asset-class`, `crypto`, `flow`, `tool`. Look at existing skills for the format.
### Easy: New Swarm Preset
A YAML file in `agent/src/swarm/presets/` defining agents, roles, and DAG workflow. See existing presets.
### Medium: New Data Source Loader
1. Create `agent/backtest/loaders/<source>.py`
2. Implement the `DataLoader` Protocol (see `agent/backtest/loaders/base.py`)
3. Register in `agent/backtest/loaders/registry.py`
4. Add tests in `agent/tests/`
### Medium: Portfolio Optimizer Constraints
Current optimizers (`agent/backtest/optimizers/`) support basic weight optimization. We need:
- **Leverage constraints** — max gross exposure limits
- **Sector/group caps** — weight limits per sector or asset group
- **Turnover penalties** — transaction cost-aware rebalancing
- Extend `BaseOptimizer` interface, keep backward compatibility
### Medium: Correlation Heatmap Dashboard
Add a cross-asset correlation matrix component to the frontend:
- Time-window slider (30d / 60d / 90d / 1Y)
- Hierarchical clustering for sector grouping
- ECharts heatmap or custom canvas renderer
- Wire to a new API endpoint that computes rolling correlations
### Hard: Intraday Backtest Engine
Current engines run on daily bars. We need sub-daily execution:
- Support 1m / 5m / 15m / 30m / 1H / 4H intervals
- Handle market session boundaries (pre-market, after-hours, overnight gaps)
- Intraday margin / buying power tracking
- Integrate with minute-level data loaders (OKX klines, yfinance intraday)
- Start from `BaseEngine` — see `agent/backtest/engines/base.py` for the abstract interface
### Hard: Options Volatility Surface & Greeks Visualization
Build 3D implied-vol surface and Greeks charts in the frontend:
- **Vol surface**: strike vs. expiry vs. implied vol (3D mesh or contour)
- **Greeks dashboard**: delta, gamma, theta, vega curves across strikes
- **Skew chart**: 25-delta risk reversal over time
- Backend: extend `options_pricing_tool.py` to return surface data
- Frontend: ECharts GL for 3D, or custom WebGL renderer
### Hard: Monte Carlo & Stress Testing
Add scenario simulation and tail-risk analysis:
- **Monte Carlo**: generate N return paths from fitted distribution, compute VaR/CVaR confidence intervals
- **Historical stress tests**: replay specific crisis periods (2008, 2020 COVID, 2022 crypto winter)
- **Regime switching**: HMM-based regime detection (bull/bear/sideways), conditional risk metrics
- Output: distribution plots, drawdown probability curves, worst-case tables
- Can be a new tool in `agent/src/tools/` or an extension to the backtest engine
## License
By contributing, you agree that your contributions will be licensed under the [MIT License](LICENSE).
By contributing, you agree that your contributions are licensed under the
project's MIT license (see `LICENSE`).
+4
View File
@@ -1,4 +1,5 @@
include LICENSE
include NOTICE
include README.md
include README_zh.md
include README_ja.md
@@ -11,6 +12,9 @@ include docker-compose.yml
# Developer entry-point template
include agent/.env.example
# Alpha Zoo factor metadata and per-zoo LICENSE / NOTICE files
recursive-include agent/src/factors *.yaml *.md
# Tests shipped so dev installs can run `pytest` against the package.
# Files prefixed with `_` are local debug scripts kept untracked — keep
# them out of the sdist even if present in the working tree.
+22
View File
@@ -0,0 +1,22 @@
Vibe-Trading
Copyright 2026 HKUDS contributors.
This product includes software developed at HKUDS (https://github.com/HKUDS).
This product bundles Microsoft Qlib feature definitions under the Apache 2.0
license. See agent/src/factors/zoo/qlib158/NOTICE and LICENSE.md for the
upstream NOTICE and attribution.
Mathematical formulas from the following sources are reimplemented in this
repository as factual mathematical content (formulas are not copyrightable):
- Kakushadze, Z. (2015) "101 Formulaic Alphas" (arXiv:1601.00991)
See agent/src/factors/zoo/alpha101/LICENSE.md
- Guotai Junan Securities (2014) "191 Short-period Trading Alpha Factors"
See agent/src/factors/zoo/gtja191/LICENSE.md
- Fama-French five-factor model, Carhart momentum, Hou-Xue-Zhang q-factor
See agent/src/factors/zoo/academic/LICENSE.md
Prose, tables, and figures from the source papers and reports are NOT
reproduced in this repository. Only the mathematical formulas — which are
factual content and not subject to copyright — are reimplemented here.
+322 -85
View File
@@ -19,45 +19,52 @@
<a href="https://pypi.org/project/vibe-trading-ai/"><img src="https://img.shields.io/pypi/v/vibe-trading-ai?style=flat&logo=pypi&logoColor=white" alt="PyPI"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow?style=flat" alt="License"></a>
<br>
<img src="https://img.shields.io/badge/Skills-74-orange" alt="Skills">
<img src="https://img.shields.io/badge/Swarm_Presets-29-7C3AED" alt="Swarm">
<img src="https://img.shields.io/badge/Tools-27-0F766E" alt="Tools">
<img src="https://img.shields.io/badge/Data_Sources-6-2563EB" alt="Data Sources">
<br>
<a href="https://github.com/HKUDS/.github/blob/main/profile/README.md"><img src="https://img.shields.io/badge/Feishu-Group-E9DBFC?style=flat-square&logo=feishu&logoColor=white" alt="Feishu"></a>
<a href="https://github.com/HKUDS/.github/blob/main/profile/README.md"><img src="https://img.shields.io/badge/WeChat-Group-C5EAB4?style=flat-square&logo=wechat&logoColor=white" alt="WeChat"></a>
<a href="https://discord.gg/2vDYc2w5"><img src="https://img.shields.io/badge/Discord-Join-7289DA?style=flat-square&logo=discord&logoColor=white" alt="Discord"></a>
</p>
<p align="center">
<a href="https://vibetrading.wiki/">Website</a> &nbsp;&middot;&nbsp;
<a href="https://vibetrading.wiki/docs/">Docs</a> &nbsp;&middot;&nbsp;
<a href="#-news">News</a> &nbsp;&middot;&nbsp;
<a href="#-key-features">Features</a> &nbsp;&middot;&nbsp;
<a href="#-shadow-account">Shadow Account</a> &nbsp;&middot;&nbsp;
<a href="#-demo">Demo</a> &nbsp;&middot;&nbsp;
<a href="#-what-is-vibe-trading">What Is It</a> &nbsp;&middot;&nbsp;
<a href="#-get-started">Get Started</a> &nbsp;&middot;&nbsp;
<a href="#-cli-reference">CLI</a> &nbsp;&middot;&nbsp;
<a href="#-api-server">API</a> &nbsp;&middot;&nbsp;
<a href="#-mcp-plugin">MCP</a> &nbsp;&middot;&nbsp;
<a href="#-project-structure">Structure</a> &nbsp;&middot;&nbsp;
<a href="#-quick-start">Quick Start</a> &nbsp;&middot;&nbsp;
<a href="#-examples">Examples</a> &nbsp;&middot;&nbsp;
<a href="#-api-server">API / MCP</a> &nbsp;&middot;&nbsp;
<a href="#-roadmap">Roadmap</a> &nbsp;&middot;&nbsp;
<a href="#-contributing">Contributing</a> &nbsp;&middot;&nbsp;
<a href="#contributors">Contributors</a>
<a href="#-contributing">Contributing</a>
</p>
<p align="center">
<a href="#-get-started"><img src="assets/pip-install.svg" height="45" alt="pip install vibe-trading-ai"></a>
<a href="#-quick-start"><img src="assets/pip-install.svg" height="45" alt="pip install vibe-trading-ai"></a>
</p>
---
## 📰 News
- **2026-05-05** 🛡️ **Security boundary follow-up**: Completes the remaining security-boundary hardening around explicit CORS origins, Settings credential indicators, web URL reading, and Shadow Account code generation, with regression tests added for each path. Normal localhost CLI/Web UI workflows stay the same; remote deployments should continue using `API_AUTH_KEY` and explicit trusted origins.
- **2026-05-04** 🖥️ **Interactive CLI UX + CI cleanup**: Interactive mode now has a live bottom status bar showing provider/model, session duration, last-run latency, and cumulative tool-call stats, plus prompt history navigation and cursor editing with arrow keys via `prompt_toolkit` ([#69](https://github.com/HKUDS/Vibe-Trading/pull/69)). The CLI still falls back to Rich prompts when `prompt_toolkit` or a TTY is unavailable. CI path expectations were also aligned with the hardened file-import sandbox and cross-platform `/tmp` resolution, returning main to green ([`bb67dc7`](https://github.com/HKUDS/Vibe-Trading/commit/bb67dc7cfcc11553c57d8962bee56381dca43758)).
- **2026-05-03** 🛡️ **Security hardening patch**: Tightens default API authentication for non-local deployments, protects sensitive run/session/swarm reads, restricts upload and local file-reading boundaries, gates shell-capable tools by entry point, validates generated strategy loading before import, and runs the Docker image as a non-root user with a localhost-only published port by default. Local CLI and localhost Web UI workflows remain low-friction; remote API/Web deployments should set `API_AUTH_KEY`.
- **2026-05-17** 🧬 **Alpha Zoo v1 (0.1.8)**: 452 pre-built quant alphas across 4 zoos — `qlib158` (Microsoft Qlib, Apache-2 attribution), `alpha101` (Kakushadze 101 Formulaic Alphas, paper rewrite from arXiv:1601.00991), `gtja191` (Guotai Junan 2014 short-horizon factor report), and `academic` (Fama-French 5 + Carhart price-based proxies). One-line CLI to bench any zoo on your universe: `vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025`. Ships with AST purity gate, lookahead-guard test, `pytest-socket` network kill-switch, per-zoo LICENSE.md, and a Developer Certificate of Origin (DCO) workflow for community PRs. Auto-rendered Alpha Library at [vibetrading.wiki/alpha-library/](https://vibetrading.wiki/alpha-library/) + research-lab post [Which of the 191 GTJA alphas still work in 2026?](https://vibetrading.wiki/research-lab/posts/alpha-191-in-2026.html).
- **2026-05-16** 🧪 **Research spine update**: Added a backend Hypothesis Registry with `create_hypothesis`, `update_hypothesis`, `link_backtest`, and `search_hypotheses`; external-content readers now attach warning-only `security_warnings`; and Shadow Account scanning now uses deterministic OHLCV feature evaluation instead of the old calendar-phase stub.
- **2026-05-15** 🪪 The run detail page now surfaces the Trust Layer run card alongside metrics and artifacts, completing the UI side of the `run_card.json` work landed on 2026-05-12. `PersistentMemory.add()` was also hardened on length, empty/whitespace-only names, and C0/C1 control bytes from the #108/#109/#110 triage ([#112](https://github.com/HKUDS/Vibe-Trading/pull/112), thanks @Teerapat-Vatpitak).
<details>
<summary>Earlier news</summary>
- **2026-05-14** 🌐 the public wiki is now live at [vibetrading.wiki](https://vibetrading.wiki/) with docs, tutorials, Research Lab, and Alpha Library sections deployed through Cloudflare Pages. Persistent memory is also inspectable from the CLI via `vibe-trading memory list/show/search/forget` ([#102](https://github.com/HKUDS/Vibe-Trading/pull/102), thanks @Teerapat-Vatpitak), and memory tokenization/slugs now support Thai, Arabic, Hebrew, and Cyrillic text ([#104](https://github.com/HKUDS/Vibe-Trading/pull/104)).
- **2026-05-13** 🧭 Swarm runs now ground workers with fetched market data and cleaner persisted reports ([#93](https://github.com/HKUDS/Vibe-Trading/pull/93), [#84](https://github.com/HKUDS/Vibe-Trading/pull/84)).
- **2026-05-12** 🧾 Backtests now emit `run_card.json` and `run_card.md` alongside artifacts for reproducible research runs.
- **2026-05-11** 🧭 **Memory slugs, swarm accounting, and CLI preflight**: Persistent memory now preserves CJK characters when generating file slugs, preventing silent filename collisions for Chinese/Japanese/Korean notes ([#95](https://github.com/HKUDS/Vibe-Trading/pull/95), thanks @voidborne-d). Swarm run totals now prefer provider-reported token usage with the existing estimate fallback ([#94](https://github.com/HKUDS/Vibe-Trading/pull/94), thanks @Teerapat-Vatpitak), and the CLI run UI gained a startup preflight check for common environment issues ([#96](https://github.com/HKUDS/Vibe-Trading/pull/96), thanks @ykykj).
- **2026-05-10** 🧱 **Regression guardrails + run metadata**: Memory recall now treats underscores as token boundaries, so snake_case saved memories such as `mcp_wiring_test` match natural-language queries like "mcp wiring" ([#87](https://github.com/HKUDS/Vibe-Trading/pull/87), thanks @hp083625). The MCP server has a subprocess smoke test covering initialize → `tools/list` → `tools/call` to guard the first-call deadlock path ([#86](https://github.com/HKUDS/Vibe-Trading/pull/86)), while low-risk hardening landed for Windows path-sensitive tests, API best-effort exception handling, backtest `run_dir` allowed-root validation, and SwarmRun provider/model metadata ([#88](https://github.com/HKUDS/Vibe-Trading/pull/88), [#90](https://github.com/HKUDS/Vibe-Trading/pull/90), [#91](https://github.com/HKUDS/Vibe-Trading/pull/91), [#92](https://github.com/HKUDS/Vibe-Trading/pull/92), thanks @Teerapat-Vatpitak).
- **2026-05-09** 🛡️ **API path hardening + MCP server stability**: API run/session routes now validate path IDs before lookup, rejecting malformed newline-containing parameters and pinning the behavior in the auth/security regression suite ([#80](https://github.com/HKUDS/Vibe-Trading/pull/80), thanks @SJoon99). The MCP server now pre-warms the tool registry on the main thread before serving `tools/call`, avoiding a first-call deadlock in lazy tool discovery ([#85](https://github.com/HKUDS/Vibe-Trading/pull/85), thanks @Teerapat-Vatpitak). The Vite dev proxy also honors `VITE_API_URL` for non-default backend targets ([#82](https://github.com/HKUDS/Vibe-Trading/pull/82), thanks @voidborne-d).
- **2026-05-08** 🧾 **Tushare statement fields in filters**: A-share daily backtests can now request PIT-safe financial statement fields through `fundamental_fields`, so signal engines can screen on `income_total_revenue`, `income_n_income`, `balancesheet_total_hldr_eqy_exc_min_int`, `fina_indicator_roe`, and similar table-prefixed columns after their announcement/disclosure dates ([#76](https://github.com/HKUDS/Vibe-Trading/pull/76), thanks @mrbob-git). Follow-up hardening makes explicit statement-field requests fail fast if Tushare enrichment cannot run, instead of silently falling back to raw price bars ([#77](https://github.com/HKUDS/Vibe-Trading/pull/77)).
- **2026-05-07** 📈 **Tushare fundamentals + community triage**: Added a point-in-time `TushareFundamentalProvider` contract for fundamental research workflows, with regression coverage for the project `TUSHARE_TOKEN` environment path ([#74](https://github.com/HKUDS/Vibe-Trading/pull/74)). Community triage also clarified that Vibe-Trading keeps rapid iteration focused on one UI language for now, avoids adding redundant search dependencies while DuckDuckGo-backed `web_search` is already bundled, and treats unofficial hosted deployments as untrusted places for API keys or data-source tokens.
- **2026-05-06** 🚀 **v0.1.7 released** ([Release notes](https://github.com/HKUDS/Vibe-Trading/releases/tag/v0.1.7), `pip install -U vibe-trading-ai`): Security-boundary hardening is now published on PyPI and ClawHub, covering safer API/read/upload/file/URL/generated-code/shell-tool/Docker defaults while keeping localhost CLI/Web UI workflows low-friction. This cycle also includes Web UI Settings, correlation heatmap, OpenAI Codex OAuth, A-share pre-ST filtering, interactive CLI UX, swarm preset inspection, dividend analysis, dev workflow polish, and audited frontend build-dependency floors. Thanks to the 0.1.7 contributors and to lemi9090 (S2W) for coordinated security validation.
- **2026-05-05** 🛡️ **Security boundary follow-up**: Completes the remaining security-boundary hardening around explicit CORS origins, Settings credential indicators, web URL reading, and Shadow Account code generation, with regression tests added for each path. Normal localhost CLI/Web UI workflows stay the same; remote deployments should continue using `API_AUTH_KEY` and explicit trusted origins.
- **2026-05-04** 🖥️ **Interactive CLI UX + CI cleanup**: Interactive mode now has a live bottom status bar showing provider/model, session duration, last-run latency, and cumulative tool-call stats, plus prompt history navigation and cursor editing with arrow keys via `prompt_toolkit` ([#69](https://github.com/HKUDS/Vibe-Trading/pull/69)). The CLI still falls back to Rich prompts when `prompt_toolkit` or a TTY is unavailable. CI path expectations were also aligned with the hardened file-import sandbox and cross-platform `/tmp` resolution, returning main to green ([`bb67dc7`](https://github.com/HKUDS/Vibe-Trading/commit/bb67dc7cfcc11553c57d8962bee56381dca43758)).
- **2026-05-03** 🛡️ **Security hardening patch**: Tightens default API authentication for non-local deployments, protects sensitive run/session/swarm reads, restricts upload and local file-reading boundaries, gates shell-capable tools by entry point, validates generated strategy loading before import, and runs the Docker image as a non-root user with a localhost-only published port by default. Local CLI and localhost Web UI workflows remain low-friction; remote API/Web deployments should set `API_AUTH_KEY`.
- **2026-05-02** 🧭 **Dividend analysis + sharper roadmap**: Added the `dividend-analysis` skill for income stocks, payout sustainability, dividend growth, shareholder yield, ex-dividend mechanics, and yield-trap checks, pinned by bundled-skill regression tests. The public roadmap now focuses on upcoming work: Research Autopilot, Data Bridge, Options Lab, Portfolio Studio, Alpha Zoo, Research Delivery, Trust Layer, and Community sharing.
- **2026-05-01** 🔥 **Correlation heatmap + OpenAI Codex OAuth + A-share pre-ST filter**: New correlation dashboard/API computes rolling return correlations and renders an ECharts heatmap for portfolio and symbol analysis ([#64](https://github.com/HKUDS/Vibe-Trading/pull/64)). OpenAI Codex provider support now uses ChatGPT OAuth via `vibe-trading provider login openai-codex`, with Settings metadata and adapter regression tests ([#65](https://github.com/HKUDS/Vibe-Trading/pull/65)). Added and hardened the `ashare-pre-st-filter` skill for A-share ST/*ST risk screening, including Sina penalty relevance filtering so securities-account mentions do not inflate E2 counts ([#63](https://github.com/HKUDS/Vibe-Trading/pull/63)).
- **2026-04-30** ⚙️ **Web UI Settings + validation CLI hardening**: New Settings page for LLM provider/model, base URL, reasoning effort, and data source credentials, backed by local/auth-protected settings APIs and data-driven provider metadata ([#57](https://github.com/HKUDS/Vibe-Trading/pull/57)). Also hardens `python -m backtest.validation <run_dir>` so missing, blank, malformed, non-existent, and non-directory inputs fail with clear operator-facing messages before validation starts ([#60](https://github.com/HKUDS/Vibe-Trading/pull/60)).
@@ -83,74 +90,137 @@
---
## 💡 What Is Vibe-Trading?
Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural language requests into executable trading strategies, research insights, and portfolio analysis across global markets.
### Key Capabilities:
• **Natural Language → Strategy** — Describe an idea; the agent writes, tests, and exports trading code<br>
• **6 Data Sources, Zero Config** — A-shares, HK/US, crypto, futures & forex with automatic fallback<br>
• **29 Expert Teams** — Pre-built multi-agent swarm workflows for investment, trading & risk<br>
• **Cross-Session Memory** — Remembers preferences and insights; creates & evolves reusable skills<br>
• **7 Backtest Engines** — Cross-market composite testing with statistical validation & 4 optimizers<br>
• **Multi-Platform Export** — One-click to TradingView, TDX (通达信/同花顺), and MetaTrader 5
---
## ✨ Key Features
<table width="100%">
<div align="center">
<table align="center" width="94%" style="width:94%; margin-left:auto; margin-right:auto;">
<tr>
<td align="center" width="25%" valign="top">
<img src="assets/scene-research.png" height="150" alt="Research"/><br>
<h3>🔍 DeepResearch for Trading</h3>
<img src="https://img.shields.io/badge/74_Skills-FF6B6B?style=for-the-badge&logo=bookstack&logoColor=white" alt="Skills" /><br><br>
<div align="left" style="font-size: 4px;">
• 74 specialist skills with persistent cross-session memory<br>
• Self-evolving: agent creates & refines workflows from experience<br>
• 5-layer context compression — no info lost in long sessions<br>
• Natural-language task routing across all finance domains
<td align="center" width="50%" valign="top">
<img src="assets/feature-self-improving-trading-agent.png" height="130" alt="Self-improving trading agent"/><br>
<h3>🔍 Self-Improving Trading Agent</h3>
<div align="left">
• Natural-language market research<br>
• Strategy drafts and file/web analysis<br>
• Memory-backed workflows
</div>
</td>
<td align="center" width="25%" valign="top">
<img src="assets/scene-swarm.png" height="150" alt="Swarm"/><br>
<h3>🐝 Swarm Intelligence</h3>
<img src="https://img.shields.io/badge/29_Trading_Teams-4ECDC4?style=for-the-badge&logo=hive&logoColor=white" alt="Swarm" /><br><br>
<td align="center" width="50%" valign="top">
<img src="assets/feature-multi-agent-trading-teams.png" height="130" alt="Multi-agent trading teams"/><br>
<h3>🐝 Multi-Agent Trading Teams</h3>
<div align="left">
• 29 out-of-the-box trading team presets<br>
• DAG-based multi-agent orchestration<br>
• Real-time streaming dashboard with live agent status<br>
• FTS5 session search across all past conversations
• Investment, quant, crypto, and risk teams<br>
• Streaming progress and persisted reports<br>
• Workers grounded with fetched market data
</div>
</td>
<td align="center" width="25%" valign="top">
<img src="assets/scene-backtest.png" height="150" alt="Backtest"/><br>
<h3>📊 Cross-Market Backtest</h3>
<img src="https://img.shields.io/badge/6_Data_Sources-FFD93D?style=for-the-badge&logo=bitcoin&logoColor=black" alt="Backtest" /><br><br>
</tr>
<tr>
<td align="center" width="50%" valign="top">
<img src="assets/feature-cross-market-data-backtesting.png" height="130" alt="Cross-market data and backtesting"/><br>
<h3>📊 Cross-Market Data & Backtesting</h3>
<div align="left">
• A-shares, HK/US equities, crypto, futures & forex<br>
• 7 market engines + composite cross-market engine with shared capital pool<br>
• Statistical validation: Monte Carlo, Bootstrap CI, Walk-Forward<br>
• 15+ performance metrics & 4 optimizers
• A/HK/US equities, crypto, futures, and forex<br>
• Data fallback and composite backtests<br>
• PIT data, validation, and run cards
</div>
</td>
<td align="center" width="25%" valign="top">
<img src="assets/scene-quant.png" height="150" alt="Quant"/><br>
<h3>🧮 Quant Analysis Toolkit</h3>
<img src="https://img.shields.io/badge/Quant_Tools-C77DFF?style=for-the-badge&logo=wolfram&logoColor=white" alt="Quant" /><br><br>
<td align="center" width="50%" valign="top">
<img src="assets/feature-shadow-account.png" height="130" alt="Shadow Account"/><br>
<h3>👥 Shadow Account</h3>
<div align="left">
• Factor IC/IR analysis & quantile backtesting<br>
• Black-Scholes pricing & full Greeks calculation<br>
• Technical pattern recognition & detection<br>
• Portfolio optimization via MVO/Risk Parity/BL
• Broker-journal behavior diagnostics<br>
• Rule-based Shadow Account comparisons<br>
• Exportable audit reports and strategy code
</div>
</td>
</tr>
</table>
</div>
## 74 Skills across 8 Categories
## 💡 What Is Vibe-Trading?
- 📊 74 specialized finance skills organized into 8 categories
Vibe-Trading is an open-source research workspace for turning finance questions into runnable analysis. It connects natural-language prompts to market-data loaders, strategy generation, backtest engines, reports, exports, and persistent research memory.
It is designed for research, simulation, and backtesting. It does not execute live trades.
---
## ✨ What You Can Do
| Task | Output |
|------|--------|
| **Ask a trading question** | Market research with tools, data, documents, and reusable session context. |
| **Backtest a strategy idea** | Strategy code, metrics, benchmark context, validation artifacts, and run cards. |
| **Review your own trades** | Broker-journal parsing, behavior diagnostics, rule extraction, and Shadow Account comparisons. |
| **Improve repeated research** | Persistent memory and editable skills turn useful routines into reusable workflows. |
| **Run analyst teams** | Multi-agent research reviews for investment, quant, crypto, macro, and risk workflows. |
| **Ship usable artifacts** | Reports, TradingView Pine Script, TDX, MetaTrader 5, MCP tools, and later research sessions. |
| **Bench a pre-built alpha zoo** | One-line IC + alive/reversed/dead categorisation across 452 alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + FF5 + Carhart) on your universe. |
---
## ⚡ Quick Example
```bash
pip install vibe-trading-ai
# Natural-language research
vibe-trading run -p "Backtest a BTC-USDT 20/50 moving-average strategy for 2024, summarize return and drawdown, then export the report"
# Bench a pre-built alpha zoo (one line)
vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20
```
```bash
vibe-trading --upload trades_export.csv
vibe-trading run -p "Analyze my trading behavior, extract my shadow strategy, and compare it with my actual trades"
```
---
## 👥 Shadow Account
Shadow Account starts from your own trading records instead of a generic strategy template.
Upload a broker export, let the agent summarize your behavior, then compare the actual trading path with a rule-based shadow strategy.
| Step | Agent output |
|------|--------------|
| **1. Read your journal** | Parses broker exports from 同花顺, 东方财富, 富途, and generic CSV formats. |
| **2. Profile your behavior** | Holding days, win rate, PnL ratio, drawdown, disposition effect, overtrading, momentum chasing, and anchoring checks. |
| **3. Extract your rules** | Turns recurring entries/exits into an explicit strategy profile instead of a hand-wavy summary. |
| **4. Run the shadow** | Backtests the extracted rules and highlights rule breaks, early exits, missed signals, and alternative trade paths. |
| **5. Deliver the report** | Produces an HTML/PDF report that can be inspected, archived, or refined in a later session. |
```bash
vibe-trading --upload trades_export.csv
vibe-trading run -p "Analyze my trading behavior, extract my shadow strategy, and compare it with my actual trades"
```
---
## 🧪 Research Workflow
Most runs follow the same evidence path: route the request, load the right market context, execute tools, validate outputs, and keep the artifacts inspectable.
| Layer | What happens |
|-------|--------------|
| **Plan** | Selects the relevant finance skills, tools, data sources, and swarm preset when useful. |
| **Ground** | Pulls A-shares, HK/US equities, crypto, futures, forex, documents, or web context through the available loaders. |
| **Execute** | Generates testable strategy code, runs tools, and uses the matching backtest engine or analysis workflow. |
| **Validate** | Adds metrics, benchmark comparison, Monte Carlo, Bootstrap, Walk-Forward, run cards, and warnings where applicable. |
| **Deliver** | Returns reports, artifacts, tool traces, and exports for TradingView, TDX, MetaTrader 5, MCP clients, or later sessions. |
---
## 🔩 Detailed Capabilities
Detailed inventories are folded below to keep the main README scannable. Open them when you want to inspect the available building blocks.
<details>
<summary><b>Finance Skill Library</b> <sub>75 skills across 8 categories</sub></summary>
- 📊 75 specialized finance skills organized into 8 categories
- 🌐 Complete coverage from traditional markets to crypto & DeFi
- 🔬 Comprehensive capabilities spanning data sourcing to quantitative research
@@ -162,10 +232,13 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
| Asset Class | 9 | `options-strategy`, `options-advanced`, `convertible-bond`, `etf-analysis`, `asset-allocation`, `sector-rotation` |
| Crypto | 7 | `perp-funding-basis`, `liquidation-heatmap`, `stablecoin-flow`, `defi-yield`, `onchain-analysis` |
| Flow | 7 | `hk-connect-flow`, `us-etf-flow`, `edgar-sec-filings`, `financial-statement`, `adr-hshare` |
| Tool | 10 | `backtest-diagnose`, `report-generate`, `pine-script`, `doc-reader`, `web-reader`, `vnpy-export` |
| Tool | 11 | `backtest-diagnose`, `report-generate`, `pine-script`, `doc-reader`, `web-reader`, `vnpy-export`, `alpha-zoo` |
| Risk Analysis | 1 | `ashare-pre-st-filter` |
## 29 Agent Swarm Team Presets
</details>
<details>
<summary><b>Preset Trading Teams</b> <sub>29 swarm presets</sub></summary>
- 🏢 29 ready-to-use agent teams
- ⚡ Pre-configured finance workflows
@@ -187,7 +260,29 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
</sub>
### 🎬 Demo
</details>
<details>
<summary><b>Alpha Zoo</b> <sub>452 pre-built quant alphas across 4 zoos</sub></summary>
- 🧬 452 cross-sectional alphas, lookahead-banned at the operator layer
- 📈 IC + IR + alive/reversed/dead categorisation in one CLI command
- 🔬 AST purity gate + 300-row lookahead sentinel test + `pytest-socket` network kill-switch
- 📦 Apache-2 attribution for Qlib; per-zoo `LICENSE.md` declaring formulas as mathematical content
- 🤝 Developer Certificate of Origin (DCO) sign-off workflow for community PRs
| Zoo | Count | Source | License |
|-----|-------|--------|---------|
| **qlib158** | 154 | Microsoft Qlib `Alpha158` (Apache-2.0, commit-pinned) | Apache-2.0 |
| **alpha101** | 101 | Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991 | Formulas are mathematical content |
| **gtja191** | 191 | Guotai Junan (2014), "191 Short-period Trading Alpha Factors" | Formulas are mathematical content |
| **academic** | 6 | Fama-French 5 + Carhart momentum (price-based proxies) | Public academic literature |
Run `vibe-trading alpha list` to browse, `vibe-trading alpha show <id>` for formulas + source, `vibe-trading alpha bench --zoo X --universe Y --period Z` to score a whole zoo.
</details>
## 🎬 Demo
<div align="center">
<table>
@@ -211,7 +306,7 @@ https://github.com/user-attachments/assets/3754a414-c3ee-464f-b1e8-78e1a74fbd30
---
## 🚀 Quick Started
## 🚀 Quick Start
### One-line install (PyPI)
@@ -219,6 +314,13 @@ https://github.com/user-attachments/assets/3754a414-c3ee-464f-b1e8-78e1a74fbd30
pip install vibe-trading-ai
```
Then run a first research task:
```bash
vibe-trading init
vibe-trading run -p "Backtest a BTC-USDT 20/50 moving-average strategy for 2024 and summarize return and drawdown"
```
> **Package name vs commands:** The PyPI package is `vibe-trading-ai`. Once installed, you get three commands:
>
> | Command | Purpose |
@@ -329,7 +431,7 @@ Copy `agent/.env.example` to `agent/.env` and uncomment the provider block you w
| `LANGCHAIN_PROVIDER` | Yes | Provider name (`openrouter`, `deepseek`, `groq`, `ollama`, etc.) |
| `<PROVIDER>_API_KEY` | Yes* | API key (`OPENROUTER_API_KEY`, `DEEPSEEK_API_KEY`, etc.) |
| `<PROVIDER>_BASE_URL` | Yes | API endpoint URL |
| `LANGCHAIN_MODEL_NAME` | Yes | Model name (e.g. `deepseek/deepseek-v3.2`) |
| `LANGCHAIN_MODEL_NAME` | Yes | Model name (e.g. `deepseek-v4-pro`) |
| `TUSHARE_TOKEN` | No | Tushare Pro token for A-share data (falls back to AKShare) |
| `TIMEOUT_SECONDS` | No | LLM call timeout, default 120s |
| `API_AUTH_KEY` | Recommended for network deployments | Bearer token required when the API is reachable from non-local clients |
@@ -348,10 +450,10 @@ Vibe-Trading is a tool-heavy agent — skills, backtests, memory, and swarms all
| Tier | Examples | When to use |
|------|----------|-------------|
| **Best** | `anthropic/claude-opus-4.7`, `anthropic/claude-sonnet-4.6`, `openai/gpt-5.4`, `google/gemini-3.1-pro-preview` | Complex swarms (3+ agents), long research sessions, paper-grade analysis |
| **Sweet spot** (default) | `deepseek/deepseek-v3.2`, `x-ai/grok-4.20`, `z-ai/glm-5.1`, `moonshotai/kimi-k2.5`, `qwen/qwen3-max-thinking` | Daily driver — reliable tool-calling at ~1/10 the cost |
| **Sweet spot** (default) | `deepseek-v4-pro`, `deepseek/deepseek-v4-pro`, `x-ai/grok-4.20`, `z-ai/glm-5.1`, `moonshotai/kimi-k2.5`, `qwen/qwen3-max-thinking` | Daily driver — reliable tool-calling at ~1/10 the cost |
| **Avoid for agent use** | `*-nano`, `*-flash-lite`, `*-coder-next`, small / distilled variants | Tool-calling is unreliable — the agent will appear to "answer from memory" instead of loading skills or running backtests |
The default `agent/.env.example` ships with `deepseek/deepseek-v3.2` — the cheapest option in the sweet-spot tier.
The default `agent/.env.example` ships with DeepSeek official API + `deepseek-v4-pro`; OpenRouter users can use `deepseek/deepseek-v4-pro`.
---
@@ -361,6 +463,7 @@ The default `agent/.env.example` ships with `deepseek/deepseek-v3.2` — the che
vibe-trading # interactive TUI
vibe-trading run -p "..." # single run
vibe-trading serve # API server
vibe-trading alpha list # browse 452 pre-built alphas; show / bench / compare / export-manifest sub-commands available
```
<details>
@@ -369,7 +472,7 @@ vibe-trading serve # API server
| Command | Description |
|---------|-------------|
| `/help` | Show all commands |
| `/skills` | List all 74 finance skills |
| `/skills` | List all 75 finance skills |
| `/swarm` | List 29 swarm team presets |
| `/swarm run <preset> [vars_json]` | Run a swarm team with live streaming |
| `/swarm list` | Swarm run history |
@@ -412,6 +515,12 @@ vibe-trading --continue <run_id> "refine the strategy"
vibe-trading --upload report.pdf
```
```bash
vibe-trading alpha list --zoo gtja191 --limit 10
vibe-trading alpha show gtja191_171
vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20
```
</details>
---
@@ -434,6 +543,25 @@ vibe-trading run -p "Backtest a momentum + value + quality multi-factor strategy
vibe-trading --pine <run_id>
```
**Bench a pre-built alpha zoo** (one line):
```bash
vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025 --top 20
```
**Browse the catalogue** and inspect a single alpha:
```bash
vibe-trading alpha list --zoo gtja191 --theme reversal --limit 10
vibe-trading alpha show gtja191_171
```
**Compose a multi-factor signal** from the zoo (Python):
```python
from src.skills.multi_factor.zoo_signal_engine import ZooSignalEngine
engine = ZooSignalEngine.from_zoo(["gtja191_171", "gtja191_111", "gtja191_163"])
panel = ... # your wide OHLCV panel
signal = engine.compute_signal(panel)
```
### Market Research
```bash
@@ -504,6 +632,10 @@ vibe-trading serve --port 8899
| `GET` | `/swarm/presets` | List swarm presets |
| `POST` | `/swarm/runs` | Start swarm run |
| `GET` | `/swarm/runs/{id}/events` | Swarm SSE stream |
| `GET` | `/alpha/list` | List alphas (filter by zoo/theme/universe) |
| `GET` | `/alpha/{alpha_id}` | Alpha metadata + source code |
| `POST` | `/alpha/bench` | Start a bench job (returns `job_id`) |
| `GET` | `/alpha/bench/{job_id}/stream` | SSE progress stream |
| `GET` | `/settings/llm` | Read Web UI LLM settings |
| `PUT` | `/settings/llm` | Update local LLM settings |
| `GET` | `/settings/data-sources` | Read local data source settings |
@@ -589,7 +721,7 @@ Browse on ClawHub: [clawhub.ai/skills/vibe-trading](https://clawhub.ai/skills/vi
<details>
<summary><b>OpenSpace — self-evolving skills</b></summary>
All 74 finance skills are published on [open-space.cloud](https://open-space.cloud) and evolve autonomously through OpenSpace's self-evolution engine.
All 75 finance skills are published on [open-space.cloud](https://open-space.cloud) and evolve autonomously through OpenSpace's self-evolution engine.
To use with OpenSpace, add both MCP servers to your agent config:
@@ -611,12 +743,93 @@ To use with OpenSpace, add both MCP servers to your agent config:
}
```
OpenSpace will auto-discover all 74 skills, enabling auto-fix, auto-improve, and community sharing. Search for Vibe-Trading skills via `search_skills("finance backtest")` in any OpenSpace-connected agent.
OpenSpace will auto-discover all 75 skills, enabling auto-fix, auto-improve, and community sharing. Search for Vibe-Trading skills via `search_skills("finance backtest")` in any OpenSpace-connected agent.
</details>
---
## 🔌 Loading Tools from External MCP Servers (MCP Client Mode)
> **This is the opposite direction from the MCP Plugin above.**
> The MCP Plugin lets *other* agents call Vibe-Trading tools.
> This section lets the *built-in* Vibe-Trading agent call tools from *your* external MCP servers.
### Quick start
Create `~/.vibe-trading/agent.json`:
```json
{
"mcpServers": {
"my-server": {
"command": "uvx",
"args": ["my-mcp-server"]
}
}
}
```
Run any CLI command — tools from `my-server` are automatically injected into the agent's registry after local tools:
```bash
vibe-trading run "use my-server to do X"
```
### Config reference
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `command` | string | required | Executable to spawn |
| `args` | array | `[]` | Command-line arguments |
| `env` | object | `{}` | Extra environment variables merged into the subprocess env |
| `toolTimeout` | number | `30` | Per-tool call timeout in seconds |
| `enabledTools` | array | `["*"]` | Tool allowlist. Use `["*"]` to expose all tools from the server |
Config file location: `~/.vibe-trading/agent.json` (JSON or YAML).
### Per-session overrides (API)
When creating a session via the API you can pass `mcpServers` inside `session.config` to extend or override the global config for that session only:
```json
{
"config": {
"mcpServers": {
"research-server": {
"command": "uvx",
"args": ["research-mcp"],
"enabledTools": ["search", "fetch"]
}
}
}
}
```
### Tool naming
Remote tools are exposed with stable names: `mcp_<server>_<tool>`.
If two server names produce the same ASCII-safe local prefix (e.g. `foo-bar` and `foo_bar` both become `foo_bar`), a deterministic hash suffix is appended at the server-segment level so names remain unique. The operator receives a warning:
```
WARNING: Configured MCP server 'foo-bar' collides with another server after local name
normalization. Using local tool prefix 'mcp_foo_bar_<hash>_<tool>' to keep generated
tool names unique. Rename the server in agent config if you want a different prefix.
```
### v1 limits
| Limit | Detail |
|-------|--------|
| Transport | stdio only (SSE / streamable HTTP excluded in v1) |
| Execution | serial only — MCP tools never enter the parallel readonly path |
| Surfaces | tools only (resources and prompts excluded in v1) |
| Hot reload | not supported — restart the process to pick up config changes |
| Swarm path | MCP tools are not available inside Swarm worker registries in v1 |
---
## 📁 Project Structure
<details>
@@ -633,7 +846,7 @@ Vibe-Trading/
│ │ ├── agent/ # ReAct agent core
│ │ │ ├── loop.py # 5-layer compression + read/write tool batching
│ │ │ ├── context.py # system prompt + auto-recall from persistent memory
│ │ │ ├── skills.py # skill loader (74 bundled + user-created via CRUD)
│ │ │ ├── skills.py # skill loader (75 bundled + user-created via CRUD)
│ │ │ ├── tools.py # tool base class + registry
│ │ │ ├── memory.py # lightweight workspace state per run
│ │ │ ├── frontmatter.py # shared YAML frontmatter parser
@@ -642,16 +855,25 @@ Vibe-Trading/
│ │ ├── memory/ # Cross-session persistent memory
│ │ │ └── persistent.py # file-based memory (~/.vibe-trading/memory/)
│ │ │
│ │ ├── tools/ # 27 auto-discovered agent tools
│ │ ├── tools/ # 31 auto-discovered agent tools
│ │ │ ├── backtest_tool.py # run backtests
│ │ │ ├── remember_tool.py # cross-session memory (save/recall/forget)
│ │ │ ├── skill_writer_tool.py # skill CRUD (save/patch/delete/file)
│ │ │ ├── session_search_tool.py # FTS5 cross-session search
│ │ │ ├── swarm_tool.py # launch swarm teams
│ │ │ ├── web_search_tool.py # DuckDuckGo web search
│ │ │ └── ... # bash, file I/O, factor analysis, options, etc.
│ │ │ └── ... # bash, file I/O, factor analysis, options, alpha browser + bench, etc.
│ │ │
│ │ ├── skills/ # 74 finance skills in 8 categories (SKILL.md each)
│ │ ├── factors/ # Alpha Zoo — 452 alphas across 4 zoos
│ │ │ ├── base.py # 19 operators (rank/scale/ts_*/delta/decay_linear/safe_div/vwap)
│ │ │ ├── registry.py # AST-only metadata load + lazy compute + sanity gates
│ │ │ ├── bench_runner.py # IC + alive/reversed/dead categorisation
│ │ │ └── zoo/ # qlib158 (154) + alpha101 (101) + gtja191 (191) + academic (6)
│ │ │
│ │ ├── api/ # FastAPI route modules
│ │ │ └── alpha_routes.py # /alpha/list, /alpha/{id}, /alpha/bench, SSE stream
│ │ │
│ │ ├── skills/ # 75 finance skills in 8 categories (SKILL.md each)
│ │ ├── swarm/ # Swarm DAG execution engine
│ │ │ └── presets/ # 29 swarm preset YAML definitions
│ │ ├── session/ # Multi-turn chat + FTS5 session search
@@ -666,13 +888,15 @@ Vibe-Trading/
│
├── frontend/ # Web UI (React 19 + Vite + TypeScript)
│ └── src/
│ ├── pages/ # Home, Agent, RunDetail, Compare
│ ├── pages/ # Home, Agent, AlphaZoo, RunDetail, Compare, Correlation, Settings
│ ├── components/ # chat, charts, layout
│ └── stores/ # Zustand state management
│
├── Dockerfile # Multi-stage build
├── docker-compose.yml # One-command deploy
├── pyproject.toml # Package config + CLI entrypoint
├── tools/ # Repo-level CI helpers
│ └── ci_grep_gates.sh # rejects yaml.load / trademark / per-stock-data leaks
└── LICENSE # MIT
```
@@ -713,13 +937,14 @@ Vibe-Trading is part of the **[HKUDS](https://github.com/HKUDS)** agent ecosyste
| Phase | Feature | Status |
|-------|---------|--------|
| **Research Autopilot** | Overnight research loop: hypothesis → data pull → backtest → evidence report | In Progress |
| **Trust Layer** | Reproducible run cards are emitted and shown in Run Detail; v1 adds tool traces and citations | v0 Shipped |
| **Hypothesis Registry** | Durable research hypotheses with lifecycle status, data sources, skills, run-card links, and invalidation notes | Backend MVP Shipped |
| **Research Autopilot** | Manual-first research loop: hypothesis → deterministic backtest → evidence report | Next |
| **Data Bridge** | Bring-your-own data: local CSV/Parquet/SQL connectors with schema mapping | Planned |
| **Options Lab** | Vol surface, Greeks dashboard, payoff/scenario explorer | Planned |
| **Portfolio Studio** | Risk x-ray, constraints, turnover-aware optimizer, rebalance notes | Planned |
| **Alpha Zoo** | Alpha101 / Alpha158 / Alpha191 factor libraries with screening + IC tests | Planned |
| **Alpha Zoo** | 452 pre-built alphas (Qlib 158 + Kakushadze 101 + GTJA 191 + FF5 + Carhart) with one-line bench, agent integration, and Web UI | **Shipped 0.1.8** |
| **Research Delivery** | Scheduled briefs to Slack / Telegram / email-style channels | Planned |
| **Trust Layer** | Reproducible run cards: tool trace, data sources, assumptions, citations | Planned |
| **Community** | Shareable skills, presets, and strategy cards | Exploring |
---
@@ -738,6 +963,18 @@ Want to contribute something bigger? Check the [Roadmap](#-roadmap) above and op
Thanks to everyone who has contributed to Vibe-Trading!
Recent v0.1.7 cycle contributors and credits:
- @GTC2080 / TaoMu — Web UI Settings and provider/data-source configuration APIs (#57)
- @BigNounce90 — validation CLI hardening for backtest `run_dir` input (#60)
- @shadowinlife — A-share pre-ST filter skill (#63)
- @MB-Ndhlovu — correlation heatmap dashboard and review fixes (#64, #66)
- @ykykj — OpenAI Codex OAuth provider option (#65)
- @RuifengFu — interactive CLI live status bar and prompt editing (#69)
- @SiMinus — swarm preset inspection command (#73)
- @warren618 / Haozhe Wu — security hardening, release integration, docs, Docker, packaging, and local dev workflow
- lemi9090 (S2W) — coordinated security research, validation, and disclosure support
<a href="https://github.com/HKUDS/Vibe-Trading/graphs/contributors">
<img src="https://contrib.rocks/image?repo=HKUDS/Vibe-Trading" />
</a>
+481 -253
View File
File diff suppressed because it is too large. Load diff
+505 -277
View File
File diff suppressed because it is too large. Load diff
+492 -262
View File
File diff suppressed because it is too large. Load diff
+518 -291
View File
File diff suppressed because it is too large. Load diff
+99 -7
View File
@@ -1,7 +1,7 @@
---
name: vibe-trading
version: 0.1.7
description: Professional finance research toolkit — backtesting (7 engines + benchmark comparison panel), factor analysis, options pricing, 74 finance skills, 29 multi-agent swarm teams, Trade Journal analyzer, and Shadow Account (extract → backtest → render) across 6 data sources (tushare, yfinance, okx, akshare, ccxt, futu).
version: 0.1.8
description: Professional finance research toolkit — backtesting (7 engines + benchmark comparison panel), factor analysis, Alpha Zoo (452 pre-built alphas across qlib158/alpha101/gtja191/academic), options pricing, 75 finance skills, 29 multi-agent swarm teams, Trade Journal analyzer, and Shadow Account (extract → backtest → render) across 6 data sources (tushare, yfinance, okx, akshare, ccxt, futu).
dependencies:
python: ">=3.11"
pip:
@@ -23,7 +23,7 @@ mcp:
# Vibe-Trading
Professional finance research toolkit with AI-powered backtesting (7 engines), multi-agent teams, 74 specialized skills, and the Shadow Account loop — extract your implicit trading rules from a journal, backtest them across A股/港股/美股/crypto, then see where they would have served you better.
Professional finance research toolkit with AI-powered backtesting (7 engines), multi-agent teams, 75 specialized skills, the **Alpha Zoo** (452 pre-built quantitative alphas across qlib158 / alpha101 / gtja191 / academic with one-line CLI benchmarking), and the Shadow Account loop — extract your implicit trading rules from a journal, backtest them across A股/港股/美股/crypto, then see where they would have served you better.
## Setup
@@ -53,7 +53,7 @@ Add to your agent's MCP config:
### API Key Requirements
**21 of 22 MCP tools work with zero API keys.** After `pip install`, backtesting, market data, factor analysis, options pricing, chart patterns, web search, document reading, trade journal analysis, shadow-account extraction/backtest/report, and all 74 skills are ready to use for HK/US equities and crypto.
**21 of 22 MCP tools work with zero API keys.** After `pip install`, backtesting, market data, factor analysis, options pricing, chart patterns, web search, document reading, trade journal analysis, shadow-account extraction/backtest/report, the Alpha Zoo (452 pre-built alphas), and all 75 skills are ready to use for HK/US equities and crypto.
| Feature | Key needed | When |
|---------|-----------|------|
@@ -99,7 +99,16 @@ Example workflow:
Use `list_swarm_presets()` to see all teams, then `run_swarm()` to execute.
### Finance Skills (74)
### Alpha Zoo (452 pre-built alphas)
One-line cross-sectional IC / IR / alive-reversed-dead categorisation across four bundled zoos:
- **qlib158** (154 alphas) — Microsoft Qlib's `Alpha158` feature handler, Apache-2.0 with pinned commit SHA.
- **alpha101** (101 alphas) — Kakushadze (2015) "101 Formulaic Alphas" (arXiv:1601.00991), written from the paper appendix.
- **gtja191** (191 alphas) — Guotai Junan 2014 "191 Short-period Trading Alpha Factors" research report.
- **academic** (6 factors) — Fama-French 5 + Carhart momentum (honest price-based proxies).
Each alpha ships with `__alpha_meta__` (formula LaTeX + theme + universe + warmup + columns required), guarded by an AST purity gate + 300-row lookahead sentinel test. Use the `vibe-trading alpha {list,show,bench,compare,export-manifest}` CLI, the `/alpha/*` REST routes (browser at `/alpha-zoo`), or compose multi-factor signals via `ZooSignalEngine.from_zoo(...)`.
### Finance Skills (75)
Comprehensive knowledge base covering:
- Technical analysis (candlestick, Elliott wave, Ichimoku, SMC, harmonic, chanlun)
- Quantitative methods (factor research, ML strategy, pair trading, multi-factor)
@@ -116,7 +125,7 @@ Use `load_skill(name)` to access full methodology docs with code templates.
| Tool | Description | API Key |
|------|-------------|---------|
| `list_skills` | List all 74 finance skills | None |
| `list_skills` | List all 75 finance skills | None |
| `load_skill` | Load full skill documentation | None |
| `backtest` | Run vectorized backtest engine | None* |
| `factor_analysis` | IC/IR analysis + layered backtest | None* |
@@ -147,7 +156,90 @@ Use `load_skill(name)` to access full methodology docs with code templates.
pip install vibe-trading-ai
```
That's it — no API keys needed for HK/US/crypto markets. Start using `backtest`, `get_market_data`, `analyze_options`, `analyze_trade_journal`, `extract_shadow_strategy`, `web_search`, and all 74 skills immediately.
That's it — no API keys needed for HK/US/crypto markets. Start using `backtest`, `get_market_data`, `analyze_options`, `analyze_trade_journal`, `extract_shadow_strategy`, `web_search`, the **Alpha Zoo** (`vibe-trading alpha bench --zoo gtja191 --universe csi300 --period 2018-2025`), and all 75 skills immediately.
## Loading Tools from External MCP Servers
The built-in agent can load tools from your own external MCP servers in addition to its local toolset.
> **Note:** This is the *MCP client* path — the opposite of the MCP plugin listed above. The plugin above makes Vibe-Trading's tools available to your agents. This section lets Vibe-Trading's own agent call tools from *your* servers.
### Setup
Create `~/.vibe-trading/agent.json`:
```json
{
"mcpServers": {
"my-server": {
"command": "uvx",
"args": ["my-mcp-server"],
"toolTimeout": 30,
"enabledTools": ["*"]
}
}
}
```
Remote tools appear automatically in every `vibe-trading run` / `vibe-trading chat` call. They are injected after local tools under stable names: `mcp_<server>_<tool>`.
### Config fields
| Field | Required | Default | Description |
|-------|----------|---------|-------------|
| `command` | yes | — | Executable to launch |
| `args` | no | `[]` | Command arguments |
| `env` | no | `{}` | Extra env vars for the subprocess |
| `toolTimeout` | no | `30` | Seconds before a tool call is cancelled |
| `enabledTools` | no | `["*"]` | Allowlist of remote tool names. `["*"]` enables all |
### Per-session override (API)
> **Security — disabled by default.** `mcpServers` defines subprocess `command`/`args`/`env` and is therefore restricted to operator-level trust. API callers **cannot** inject MCP server definitions through `POST /sessions` unless the server operator explicitly opts in.
To enable session-level MCP injection, set the environment variable on the server before starting the agent:
```bash
export ALLOW_SESSION_MCP_SERVERS=1
```
With the opt-in active, pass `mcpServers` inside `session.config` to extend or replace the global config for that session only:
```json
{
"config": {
"mcpServers": {
"research": {
"command": "uvx",
"args": ["research-mcp"],
"enabledTools": ["search"]
}
}
}
}
```
Without `ALLOW_SESSION_MCP_SERVERS=1`, any `mcpServers` key in `session.config` is silently stripped before config loading. The global operator config on disk (`~/.vibe-trading/agent.json`) is always respected regardless of this flag.
### v1 limits
- **Transport:** stdio only. SSE and HTTP transports are rejected.
- **Execution:** serial only. MCP tools never enter the parallel readonly path.
- **Surfaces:** tools only. Resources and prompts are not exposed.
- **Swarm:** MCP tools are excluded from Swarm worker registries in v1.
- **Hot reload:** not supported. Restart the process to pick up config changes.
### Failure handling
| Case | Behavior |
|------|----------|
| Missing config file | falls back to empty config — no MCP servers loaded |
| Invalid config file | logs a warning and falls back to empty config |
| Server fails to start | that server is skipped; local tools and other servers still load |
| Tool call times out | returns a normalized error payload instead of raising |
| Two server names collide after sanitization | deterministic hash suffix appended; operator warning emitted |
## Examples
+59 -4
View File
@@ -110,6 +110,7 @@ class RunResponse(BaseModel):
metrics: Optional[BacktestMetrics] = Field(None, description="Backtest metrics")
artifacts: List[Artifact] = Field(default_factory=list, description="Run artifacts")
run_card: Optional[Dict[str, Any]] = Field(None, description="Trust Layer run card payload")
equity_curve: Optional[List[Dict[str, Any]]] = Field(None, description="Equity preview")
trade_log: Optional[List[Dict[str, Any]]] = Field(None, description="Trade preview")
@@ -830,6 +831,13 @@ def _build_response_from_run_dir(run_dir: Path, elapsed: float, *, include_analy
if metrics_csv_path.exists():
response.artifacts_metrics_csv = _load_csv_to_dict(metrics_csv_path)
run_card_path = run_dir / "run_card.json"
if run_card_path.exists():
try:
response.run_card = json.loads(run_card_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
pass
trades_path = run_dir / "artifacts" / "trades.csv"
if trades_path.exists():
response.artifacts_trades_csv = _load_csv_to_dict(trades_path)
@@ -869,6 +877,31 @@ def _build_response_from_run_dir(run_dir: Path, elapsed: float, *, include_analy
return response
# ============================================================================
# Path-parameter validation
# ============================================================================
# ``run_id`` and ``session_id`` flow directly into filesystem paths
# (``RUNS_DIR / run_id`` etc.). Restrict to a safe character class so that
# values like ``..`` or ``foo/../bar`` cannot escape the parent directory.
_SAFE_PATH_PARAM_RE = __import__("re").compile(r"^[A-Za-z0-9_-]{1,128}$")
def _validate_path_param(value: str, kind: str) -> None:
"""Reject path parameters that could escape the parent directory.
Args:
value: User-supplied path-parameter value.
kind: Parameter name, used in the error detail.
Raises:
HTTPException: 400 when ``value`` does not match the safe character
class, mirroring the existing ``_SHADOW_ID_RE`` check.
"""
if not _SAFE_PATH_PARAM_RE.fullmatch(value or ""):
raise HTTPException(status_code=400, detail=f"invalid {kind}")
# ============================================================================
# API Endpoints
# ============================================================================
@@ -883,6 +916,7 @@ async def get_run_code(run_id: str):
Returns:
Map filename -> source text.
"""
_validate_path_param(run_id, "run_id")
run_dir = RUNS_DIR / run_id / "code"
if not run_dir.exists():
raise HTTPException(status_code=404, detail=f"Code directory for run {run_id} not found")
@@ -904,6 +938,7 @@ async def get_run_pine(run_id: str):
Returns:
Object with pine script content and exists flag.
"""
_validate_path_param(run_id, "run_id")
pine_path = RUNS_DIR / run_id / "artifacts" / "strategy.pine"
if not pine_path.exists():
return {"exists": False, "content": None}
@@ -916,6 +951,7 @@ async def get_run_pine(run_id: str):
@app.get("/runs/{run_id}", response_model=RunResponse, dependencies=[Depends(require_auth)])
async def get_run_result(run_id: str):
"""Fetch full details for a historical run by ``run_id``."""
_validate_path_param(run_id, "run_id")
run_dir = RUNS_DIR / run_id
if not run_dir.exists():
@@ -984,14 +1020,14 @@ async def list_runs(limit: int = 20):
try:
req_data = json.loads(req_file.read_text(encoding="utf-8"))
prompt = req_data.get("prompt")
except:
except (json.JSONDecodeError, OSError):
pass
if not prompt and planner_file.exists():
try:
planner_data = json.loads(planner_file.read_text(encoding="utf-8"))
prompt = planner_data.get("user_goal") or planner_data.get("goal")
except:
except (json.JSONDecodeError, OSError):
pass
if not prompt:
@@ -1011,7 +1047,7 @@ async def list_runs(limit: int = 20):
total_return = float(row.get('total_return', 0) or 0)
sharpe = float(row.get('sharpe', 0) or 0)
break
except:
except (OSError, ValueError):
pass
run_context = load_run_context(d)
@@ -1301,6 +1337,7 @@ async def list_sessions(limit: int = Query(50, ge=1, le=200)):
@app.get("/sessions/{session_id}", response_model=SessionResponse, dependencies=[Depends(require_auth)])
async def get_session(session_id: str):
"""Get one session by id."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1320,6 +1357,7 @@ async def get_session(session_id: str):
@app.delete("/sessions/{session_id}", dependencies=[Depends(require_auth)])
async def delete_session(session_id: str):
"""Delete a session."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1337,6 +1375,7 @@ class UpdateSessionRequest(BaseModel):
@app.patch("/sessions/{session_id}", dependencies=[Depends(require_auth)])
async def update_session(session_id: str, req: UpdateSessionRequest):
"""Update session fields (e.g. title)."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1354,6 +1393,7 @@ async def update_session(session_id: str, req: UpdateSessionRequest):
@app.post("/sessions/{session_id}/messages", dependencies=[Depends(require_auth)])
async def send_message(session_id: str, payload: SendMessageRequest, http_request: Request):
"""Send a user message and start the agent loop (natural language strategy)."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1371,6 +1411,7 @@ async def send_message(session_id: str, payload: SendMessageRequest, http_reques
@app.post("/sessions/{session_id}/cancel", dependencies=[Depends(require_auth)])
async def cancel_session(session_id: str):
"""Cancel the in-flight agent loop for this session."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1383,6 +1424,7 @@ async def cancel_session(session_id: str):
@app.get("/sessions/{session_id}/messages", response_model=List[MessageResponse], dependencies=[Depends(require_auth)])
async def get_messages(session_id: str, limit: int = Query(100, ge=1, le=1000)):
"""List messages for a session."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1408,6 +1450,7 @@ async def session_events(
last_event_id: Optional[str] = Query(None, alias="Last-Event-ID"),
):
"""SSE stream for agent events."""
_validate_path_param(session_id, "session_id")
svc = _get_session_service()
if not svc:
raise HTTPException(status_code=501, detail="Session runtime not enabled")
@@ -1609,6 +1652,7 @@ async def get_swarm_run(run_id: str):
"""Swarm run detail including task statuses."""
from src.swarm.task_store import TaskStore
_validate_path_param(run_id, "run_id")
runtime = _get_swarm_runtime()
run = runtime._store.load_run(run_id)
if not run:
@@ -1640,6 +1684,8 @@ async def get_swarm_run(run_id: str):
async def swarm_run_events(run_id: str, request: Request, last_index: int = Query(0, ge=0)):
"""SSE stream for a swarm run."""
import asyncio
_validate_path_param(run_id, "run_id")
runtime = _get_swarm_runtime()
async def event_stream():
@@ -1663,6 +1709,7 @@ async def swarm_run_events(run_id: str, request: Request, last_index: int = Quer
@app.post("/swarm/runs/{run_id}/cancel", dependencies=[Depends(require_auth)])
async def cancel_swarm_run(run_id: str):
"""Cancel an active swarm run."""
_validate_path_param(run_id, "run_id")
runtime = _get_swarm_runtime()
ok = runtime.cancel_run(run_id)
if not ok:
@@ -1670,6 +1717,14 @@ async def cancel_swarm_run(run_id: str):
return {"status": "cancelled"}
# ============================================================================
# Alpha Zoo routes (Web UI) — defined in src/api/alpha_routes.py
# ============================================================================
from src.api.alpha_routes import register_alpha_routes # noqa: E402
register_alpha_routes(app)
# ============================================================================
# Main Entry Point
# ============================================================================
+93 -9
View File
@@ -11,16 +11,19 @@ from __future__ import annotations
import importlib
import json
import logging
import re as _re
import sys
from abc import ABC, abstractmethod
from collections.abc import Iterable
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
import numpy as np
import pandas as pd
logger = logging.getLogger(__name__)
from backtest.loaders.tushare_fundamentals import (
TushareFundamentalProvider,
enrich_price_frames_with_fundamentals,
)
from backtest.metrics import (
by_exit_reason_stats,
by_symbol_stats,
@@ -28,11 +31,27 @@ from backtest.metrics import (
)
from backtest.models import EquitySnapshot, Position, TradeRecord
logger = logging.getLogger(__name__)
def _run_card_data_sources(config: Dict[str, Any], loader: Any) -> List[str]:
"""Return source names for run-card evidence."""
configured = config.get("_run_card_effective_sources")
if isinstance(configured, list):
return [str(source) for source in configured if str(source).strip()]
if isinstance(configured, str) and configured.strip():
return [configured.strip()]
loader_name = getattr(loader, "name", None)
if loader_name:
return [str(loader_name)]
source = config.get("source")
return [str(source)] if source else []
# ─── Market detection (lightweight, for signal alignment only) ───
import re as _re
_CRYPTO_RE = _re.compile(r"^[A-Z]+-USDT$|^[A-Z]+/USDT$", _re.I)
_FOREX_RE = _re.compile(r"^[A-Z]{3}/[A-Z]{3}$|^[A-Z]{6}\.FX$")
@@ -89,7 +108,7 @@ def _align(
logger.warning("Symbols dropped (no usable price data): %s", all_nan_cols)
codes = [c for c in codes if c not in all_nan_cols]
if not codes:
raise ValueError(f"All symbols have no data in the requested date range")
raise ValueError("All symbols have no data in the requested date range")
close = close[codes]
pos = pd.DataFrame(0.0, index=dates, columns=codes)
@@ -132,6 +151,57 @@ def _load_optimizer(config: Dict[str, Any]) -> Optional[Callable]:
return None
def _normalise_fundamental_fields(config: Dict[str, Any]) -> dict[str, list[str]]:
"""Read the optional statement-table field map from backtest config."""
raw_fields = config.get("fundamental_fields")
if raw_fields in (None, {}):
return {}
if not isinstance(raw_fields, dict):
raise ValueError("fundamental_fields must map table names to field-name lists")
normalized: dict[str, list[str]] = {}
for table, fields in raw_fields.items():
if not isinstance(table, str) or not table.strip():
raise ValueError("fundamental_fields table names must be non-empty strings")
if fields is None:
continue
if isinstance(fields, str) or not isinstance(fields, Iterable):
raise ValueError(f"fundamental_fields[{table!r}] must be a list of field names")
field_list = list(fields)
if not field_list:
continue
invalid = [field for field in field_list if not isinstance(field, str) or not field.strip()]
if invalid:
raise ValueError(f"fundamental_fields[{table!r}] contains invalid field names")
normalized[table.strip()] = field_list
return normalized
def _maybe_enrich_fundamentals(
data_map: Dict[str, pd.DataFrame],
config: Dict[str, Any],
) -> Dict[str, pd.DataFrame]:
"""Attach configured Tushare statement fields before signal generation."""
fields_by_table = _normalise_fundamental_fields(config)
if not fields_by_table:
return data_map
try:
provider = TushareFundamentalProvider()
return enrich_price_frames_with_fundamentals(
data_map,
provider,
fields_by_table,
as_of=config.get("end_date", ""),
periods=config.get("fundamental_periods"),
)
except Exception as exc:
raise RuntimeError(
f"fundamental_fields requested but Tushare enrichment failed: {exc}"
) from exc
# ─── Base Engine ───
@@ -277,6 +347,7 @@ class BaseEngine(ABC):
if not data_map:
print(json.dumps({"error": "No data fetched"}))
sys.exit(1)
data_map = _maybe_enrich_fundamentals(data_map, config)
# 2. Generate signals
signal_map = signal_engine.generate(data_map)
@@ -303,6 +374,7 @@ class BaseEngine(ABC):
index=[s.timestamp for s in self.equity_snapshots],
)
bench_ret = ret_df.mean(axis=1) if ret_df.shape[1] > 0 else pd.Series(0.0, index=dates)
benchmark_metadata = {}
# ── External benchmark fetch ──────────────────────────────────────────
bench_ticker = config.get("benchmark")
@@ -318,15 +390,17 @@ class BaseEngine(ABC):
)
if bench_result is not None:
bench_ret = bench_result.ret_series.reindex(dates).fillna(0.0)
bench_equity = self.initial_capital * (1 + bench_ret).cumprod()
m["benchmark_ticker"] = bench_result.ticker
m["benchmark_return"] = bench_result.total_ret
benchmark_metadata = {
"benchmark_ticker": bench_result.ticker,
"benchmark_return": bench_result.total_ret,
}
# ── External benchmark fetch ──────────────────────────────────────────
bench_equity = self.initial_capital * (1 + bench_ret).cumprod()
# 6. Metrics
m = calc_metrics(equity_series, self.trades, self.initial_capital, bars_per_year, bench_ret)
m.update(benchmark_metadata)
m["by_symbol"] = by_symbol_stats(self.trades)
m["by_exit_reason"] = by_exit_reason_stats(self.trades)
@@ -347,6 +421,16 @@ class BaseEngine(ABC):
target_pos, m, valid_codes,
)
# 9. Trust Layer run card
from backtest.run_card import write_run_card
write_run_card(
run_dir,
config,
m,
data_sources=_run_card_data_sources(config, loader),
strategy_path=run_dir / "code" / "signal_engine.py",
)
# Print scalar metrics (skip nested dicts for JSON compat)
print(json.dumps({k: v for k, v in m.items() if not isinstance(v, dict)}, indent=2))
return m
+9 -2
View File
@@ -16,7 +16,6 @@ Artifacts: equity.csv, metrics.csv, trades.csv, greeks.csv.
import json
import sys
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional
@@ -290,7 +289,6 @@ def run_options_backtest(
commission = config.get("commission", 0.001)
options_cfg = config.get("options_config", {})
risk_free_rate = options_cfg.get("risk_free_rate", 0.05)
iv_source = options_cfg.get("iv_source", "historical")
contract_multiplier = options_cfg.get("contract_multiplier", 1.0)
exercise_style = options_cfg.get("exercise_style", "european") # v2: "european" or "american"
iv_skew = options_cfg.get("iv_skew", 0.0) # v2: smile skew param (0 = flat)
@@ -554,6 +552,15 @@ def run_options_backtest(
pd.DataFrame(greeks_records).to_csv(out / "greeks.csv", index=False)
pd.DataFrame([metrics]).to_csv(out / "metrics.csv", index=False)
from backtest.run_card import write_run_card
write_run_card(
run_dir,
config,
metrics,
data_sources=[str(getattr(loader, "name", config.get("source", "")))],
strategy_path=run_dir / "code" / "signal_engine.py",
)
print(json.dumps(metrics, indent=2))
return metrics
+95 -2
View File
@@ -1,8 +1,16 @@
"""DataLoader Protocol and shared exceptions for all data source loaders."""
"""DataLoader Protocol, shared exceptions, and bounded-retry helpers.
The retry/budget helpers are the canonical pattern for any loader that calls
a flaky external API: a wall-clock deadline plus a small backoff schedule
applied only to a declared transient exception class. New loaders should
import :func:`check_budget` and :func:`retry_with_budget` rather than
re-implementing the loop.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
import time
from typing import Callable, Protocol, TypeVar, runtime_checkable
import pandas as pd
@@ -30,6 +38,91 @@ def validate_date_range(start_date: str, end_date: str) -> None:
raise ValueError(f"start_date ({start_date}) > end_date ({end_date})")
# ---------------------------------------------------------------------------
# Bounded retry / budget helpers (shared by ccxt_loader, okx, and any future
# loader calling a flaky external API).
# ---------------------------------------------------------------------------
DEFAULT_BACKOFF: tuple[float, ...] = (0.5, 1.5, 4.0)
DEFAULT_MAX_RETRIES = 3
def check_budget(deadline: float, label: str, budget_s: float | None = None) -> None:
"""Raise :class:`TimeoutError` if the monotonic clock has crossed ``deadline``.
Use this between pages of a paginated fetch to fail fast instead of
grinding through more requests once the wall-clock budget is gone.
Args:
deadline: ``time.monotonic()`` instant past which we abort.
label: Free-form label used in the exception message
(e.g. ``"ccxt fetch for BTC/USDT"``).
budget_s: Original budget in seconds, included verbatim in the
message when present.
"""
if time.monotonic() > deadline:
suffix = f" exceeded {budget_s:.0f}s budget" if budget_s is not None else " exceeded budget"
raise TimeoutError(f"{label}{suffix}")
_T = TypeVar("_T")
def retry_with_budget(
fn: Callable[[], _T],
*,
transient: type[BaseException] | tuple[type[BaseException], ...],
deadline: float,
label: str,
max_retries: int = DEFAULT_MAX_RETRIES,
backoff: tuple[float, ...] = DEFAULT_BACKOFF,
) -> _T:
"""Call ``fn`` with a bounded retry budget on declared transient errors.
Between attempts sleeps ``min(backoff[attempt], remaining_budget)`` so a
short remaining budget never spends the full backoff. The terminal
transient failure — whether ``max_retries`` is exhausted OR the deadline
has passed — is wrapped in :class:`TimeoutError`, preserving the original
exception as ``__cause__``. Anything not in ``transient`` propagates
unchanged on the first occurrence (we never retry an exception class
the caller didn't opt in to).
Args:
fn: Zero-arg callable producing the result.
transient: Exception class(es) considered transient and retryable.
deadline: ``time.monotonic()`` instant past which retries are aborted.
label: Free-form label used in the TimeoutError message
(e.g. ``"OKX fetch for BTC-USDT"``).
max_retries: Additional attempts after the first call. Total
attempts = ``max_retries + 1``.
backoff: Per-retry sleep seconds. Must have at least
``max_retries`` entries.
Returns:
Whatever ``fn`` returns.
Raises:
ValueError: ``backoff`` is shorter than ``max_retries``.
TimeoutError: All retries exhausted or the deadline crossed.
Any non-transient exception: Propagated unchanged from ``fn``.
"""
if len(backoff) < max_retries:
raise ValueError(
f"backoff has {len(backoff)} entries; need >= max_retries ({max_retries})"
)
for attempt in range(max_retries + 1):
try:
return fn()
except transient as exc:
remaining = deadline - time.monotonic()
if attempt == max_retries or remaining <= 0:
raise TimeoutError(
f"{label} failed after {attempt + 1} attempt(s): {exc}"
) from exc
time.sleep(min(backoff[attempt], max(0.0, remaining)))
raise AssertionError("unreachable: retry loop must return or raise") # pragma: no cover
@runtime_checkable
class DataLoaderProtocol(Protocol):
"""Interface that every data source loader must satisfy."""
+29 -3
View File
@@ -9,11 +9,16 @@ from __future__ import annotations
import logging
import os
import time
from typing import Dict, List, Optional
import pandas as pd
from backtest.loaders.base import validate_date_range
from backtest.loaders.base import (
check_budget,
retry_with_budget,
validate_date_range,
)
from backtest.loaders.registry import register
logger = logging.getLogger(__name__)
@@ -23,6 +28,14 @@ _INTERVAL_MAP = {
"1H": "1h", "4H": "4h", "1D": "1d",
}
# P12-b: ccxt had no request timeout and an unbounded paginated fetch with
# no retry budget, so a transient disconnect hung get_market_data for 10+
# minutes. Cap each HTTP call, bound transient retries, and enforce a hard
# wall-clock budget so the fetch fails fast instead of hanging. Retry
# scheduling is delegated to :mod:`backtest.loaders.base`.
_CCXT_TIMEOUT_MS = int(os.getenv("CCXT_TIMEOUT_MS", "15000"))
_CCXT_FETCH_BUDGET_S = float(os.getenv("CCXT_FETCH_BUDGET_S", "60"))
@register
class DataLoader:
@@ -51,7 +64,7 @@ class DataLoader:
if exchange_cls is None:
logger.warning("Unknown CCXT exchange %s, falling back to binance", exchange_id)
exchange_cls = ccxt.binance
return exchange_cls({"enableRateLimit": True})
return exchange_cls({"enableRateLimit": True, "timeout": _CCXT_TIMEOUT_MS})
def fetch(
self,
@@ -97,12 +110,25 @@ class DataLoader:
exchange, symbol: str, timeframe: str, since_ms: int, end_ms: int,
) -> Optional[pd.DataFrame]:
"""Paginated OHLCV fetch for one symbol."""
import ccxt
all_rows: list = []
cursor = since_ms
limit = 1000
deadline = time.monotonic() + _CCXT_FETCH_BUDGET_S
label = f"ccxt fetch for {symbol}"
for _ in range(200):
ohlcv = exchange.fetch_ohlcv(symbol, timeframe, since=cursor, limit=limit)
check_budget(deadline, label, budget_s=_CCXT_FETCH_BUDGET_S)
# ``ccxt.NetworkError`` covers RequestTimeout / DDoSProtection /
# ExchangeNotAvailable — the transient family. Anything else
# (e.g. ``ExchangeError`` for a bad symbol) is not retried.
ohlcv = retry_with_budget(
lambda: exchange.fetch_ohlcv(symbol, timeframe, since=cursor, limit=limit),
transient=ccxt.NetworkError,
deadline=deadline,
label=label,
)
if not ohlcv:
break
all_rows.extend(ohlcv)
+31 -3
View File
@@ -5,16 +5,28 @@ Supports 1m/5m/15m/30m/1H/4H/1D.
Up to 300 bars per request; paginates with ``after`` for longer history.
"""
import os
import time
from typing import Dict, List, Optional
import pandas as pd
import requests
from backtest.loaders.base import validate_date_range
from backtest.loaders.base import (
check_budget,
retry_with_budget,
validate_date_range,
)
from backtest.loaders.registry import register
BASE_URL = "https://www.okx.com/api/v5"
_MAX_PER_PAGE = 300
# P12-b parity: OKX already sets a per-request timeout but had no retry
# budget, so a transient blip dropped the whole symbol and a slow tier
# could stall ~max_pages*timeout. Bound it like the ccxt loader; retry
# scheduling is delegated to :mod:`backtest.loaders.base`.
_OKX_TIMEOUT = int(os.getenv("OKX_TIMEOUT_S", "15"))
_OKX_FETCH_BUDGET_S = float(os.getenv("OKX_FETCH_BUDGET_S", "60"))
@register
@@ -98,16 +110,32 @@ class DataLoader:
"""
all_rows: list = []
after = str(end_ts)
deadline = time.monotonic() + _OKX_FETCH_BUDGET_S
label = f"OKX fetch for {inst_id}"
for _ in range(max_pages):
check_budget(deadline, label, budget_s=_OKX_FETCH_BUDGET_S)
params = {
"instId": inst_id,
"bar": bar,
"limit": str(_MAX_PER_PAGE),
"after": after,
}
resp = requests.get(f"{BASE_URL}/market/candles", params=params, timeout=15)
data = resp.json()
def _do_request() -> dict:
resp = requests.get(
f"{BASE_URL}/market/candles",
params=params,
timeout=_OKX_TIMEOUT,
)
return resp.json()
data = retry_with_budget(
_do_request,
transient=requests.RequestException,
deadline=deadline,
label=label,
)
if data.get("code") != "0" or not data.get("data"):
break
@@ -0,0 +1,286 @@
"""Tushare fundamental data provider with point-in-time safeguards."""
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Iterable
import pandas as pd
TUSHARE_TOKEN_PLACEHOLDERS = {"", "your-tushare-token"}
class DataProviderError(Exception):
"""Base error for fundamental provider failures."""
class UnknownTableError(DataProviderError):
"""Raised when a requested fundamental table is not supported."""
class SchemaValidationError(DataProviderError):
"""Raised when provider output is missing required columns."""
@dataclass(frozen=True)
class ColumnSchema:
"""Machine-readable column metadata for a provider table."""
name: str
dtype: str
required: bool = False
@dataclass(frozen=True)
class TableSchema:
"""Machine-readable metadata for a Tushare fundamental table."""
name: str
api_name: str
point_in_time_column: str
columns: tuple[ColumnSchema, ...]
@property
def required_columns(self) -> tuple[str, ...]:
return tuple(column.name for column in self.columns if column.required)
_SCHEMAS: dict[str, TableSchema] = {
"balancesheet": TableSchema(
name="balancesheet",
api_name="balancesheet",
point_in_time_column="f_ann_date",
columns=(
ColumnSchema("ts_code", "str", required=True),
ColumnSchema("ann_date", "date", required=True),
ColumnSchema("f_ann_date", "date", required=False),
ColumnSchema("end_date", "date", required=True),
ColumnSchema("total_assets", "float"),
ColumnSchema("total_liab", "float"),
ColumnSchema("total_hldr_eqy_exc_min_int", "float"),
),
),
"cashflow": TableSchema(
name="cashflow",
api_name="cashflow",
point_in_time_column="f_ann_date",
columns=(
ColumnSchema("ts_code", "str", required=True),
ColumnSchema("ann_date", "date", required=True),
ColumnSchema("f_ann_date", "date", required=False),
ColumnSchema("end_date", "date", required=True),
ColumnSchema("net_profit", "float"),
ColumnSchema("n_cashflow_act", "float"),
ColumnSchema("c_cash_equ_end_period", "float"),
),
),
"fina_indicator": TableSchema(
name="fina_indicator",
api_name="fina_indicator",
point_in_time_column="ann_date",
columns=(
ColumnSchema("ts_code", "str", required=True),
ColumnSchema("ann_date", "date", required=True),
ColumnSchema("end_date", "date", required=True),
ColumnSchema("eps", "float"),
ColumnSchema("grossprofit_margin", "float"),
ColumnSchema("netprofit_margin", "float"),
ColumnSchema("roe", "float"),
ColumnSchema("debt_to_assets", "float"),
),
),
"income": TableSchema(
name="income",
api_name="income",
point_in_time_column="f_ann_date",
columns=(
ColumnSchema("ts_code", "str", required=True),
ColumnSchema("ann_date", "date", required=True),
ColumnSchema("f_ann_date", "date", required=False),
ColumnSchema("end_date", "date", required=True),
ColumnSchema("total_revenue", "float"),
ColumnSchema("revenue", "float"),
ColumnSchema("operate_profit", "float"),
ColumnSchema("n_income", "float"),
),
),
}
class TushareFundamentalProvider:
"""Small DataProvider contract for Tushare financial statement tables."""
def __init__(self, api: Any | None = None) -> None:
if api is None:
import tushare as ts
token = os.getenv("TUSHARE_TOKEN", "").strip()
if token in TUSHARE_TOKEN_PLACEHOLDERS:
token = ""
api = ts.pro_api(token)
self.api = api
def list_tables(self) -> list[str]:
"""Return supported fundamental tables in stable order."""
return sorted(_SCHEMAS)
def describe_table(self, table: str) -> TableSchema:
"""Return schema metadata for a supported table."""
try:
return _SCHEMAS[table]
except KeyError as exc:
raise UnknownTableError(f"Unsupported Tushare fundamental table: {table}") from exc
def query_fundamentals(
self,
table: str,
codes: Iterable[str],
*,
as_of: str | pd.Timestamp,
periods: Iterable[str] | None = None,
fields: Iterable[str] | None = None,
) -> pd.DataFrame:
"""Query a fundamental table and filter out rows unpublished by ``as_of``."""
schema = self.describe_table(table)
requested_periods = set(periods or [])
frames: list[pd.DataFrame] = []
api_method = getattr(self.api, schema.api_name, None)
if api_method is None:
raise DataProviderError(f"Tushare API object has no method: {schema.api_name}")
for code in codes:
frame = api_method(ts_code=code, period=None)
if frame is not None and not frame.empty:
frames.append(frame.copy())
if not frames:
return self._empty_frame(schema, fields)
result = pd.concat(frames, ignore_index=True)
self._validate_schema(schema, result)
if requested_periods:
result = result[result["end_date"].astype(str).isin(requested_periods)]
pit_column = schema.point_in_time_column
if pit_column not in result.columns or result[pit_column].isna().all():
pit_column = "ann_date"
as_of_date = _parse_tushare_date(as_of)
pit_values = result[pit_column]
if pit_column != "ann_date" and "ann_date" in result.columns:
pit_values = pit_values.where(pit_values.notna(), result["ann_date"])
pit_dates = pit_values.map(_parse_tushare_date)
result = result[pit_dates <= as_of_date]
output_columns = self._output_columns(schema, result, fields)
result = result.loc[:, output_columns].sort_values(["ts_code", "end_date"]).reset_index(drop=True)
return result
def _validate_schema(self, schema: TableSchema, frame: pd.DataFrame) -> None:
missing = [column for column in schema.required_columns if column not in frame.columns]
if missing:
raise SchemaValidationError(f"{schema.name} missing required columns: {', '.join(missing)}")
def _output_columns(
self,
schema: TableSchema,
frame: pd.DataFrame,
fields: Iterable[str] | None,
) -> list[str]:
identity = ["ts_code", "end_date", "ann_date"]
if schema.point_in_time_column in frame.columns and schema.point_in_time_column not in identity:
identity.append(schema.point_in_time_column)
wanted = identity + list(fields or [])
return [column for column in dict.fromkeys(wanted) if column in frame.columns]
def _empty_frame(self, schema: TableSchema, fields: Iterable[str] | None) -> pd.DataFrame:
columns = self._output_columns(schema, pd.DataFrame(columns=[c.name for c in schema.columns]), fields)
return pd.DataFrame(columns=columns)
def _parse_tushare_date(value: str | pd.Timestamp) -> pd.Timestamp:
"""Parse Tushare YYYYMMDD strings and common timestamp/date strings."""
if isinstance(value, pd.Timestamp):
return value.normalize()
text = str(value)
if len(text) == 8 and text.isdigit():
return pd.to_datetime(text, format="%Y%m%d")
return pd.to_datetime(text).normalize()
def enrich_price_frames_with_fundamentals(
data_map: dict[str, pd.DataFrame],
provider: TushareFundamentalProvider,
fields_by_table: dict[str, Iterable[str]],
*,
as_of: str | pd.Timestamp,
periods: Iterable[str] | None = None,
) -> dict[str, pd.DataFrame]:
"""Attach PIT-safe fundamental snapshots to daily price frames.
Fundamental columns are prefixed with their table name, for example
``income_total_revenue`` and ``fina_indicator_roe``. Each row becomes
visible only on or after its announcement/disclosure date.
"""
if not data_map or not fields_by_table:
return data_map
enriched = {code: frame.copy() for code, frame in data_map.items()}
codes = list(enriched)
for table, fields in fields_by_table.items():
field_list = list(fields or [])
fundamentals = provider.query_fundamentals(
table,
codes,
as_of=as_of,
periods=periods,
fields=field_list,
)
if fundamentals.empty:
continue
schema = provider.describe_table(table)
pit_column = schema.point_in_time_column
if pit_column not in fundamentals.columns or fundamentals[pit_column].isna().all():
pit_column = "ann_date"
for code, frame in enriched.items():
rows = fundamentals[fundamentals["ts_code"] == code].copy()
if rows.empty or frame.empty:
continue
pit_values = rows[pit_column]
if pit_column != "ann_date" and "ann_date" in rows.columns:
pit_values = pit_values.where(pit_values.notna(), rows["ann_date"])
rows["_pit_date"] = pit_values.map(_parse_tushare_date)
rows = rows.dropna(subset=["_pit_date"]).sort_values("_pit_date")
if rows.empty:
continue
value_columns = [column for column in rows.columns if column not in {"ts_code", "_pit_date"}]
right = rows[["_pit_date", *value_columns]].rename(
columns={column: f"{table}_{column}" for column in value_columns}
)
left = frame.copy()
original_index = left.index
left["_trade_date"] = pd.to_datetime(left.index).normalize()
left["_original_order"] = range(len(left))
merged = pd.merge_asof(
left.sort_values("_trade_date"),
right.sort_values("_pit_date"),
left_on="_trade_date",
right_on="_pit_date",
direction="backward",
)
merged = merged.sort_values("_original_order").drop(
columns=["_trade_date", "_original_order", "_pit_date"]
)
merged.index = original_index
enriched[code] = merged
return enriched
+200
View File
@@ -0,0 +1,200 @@
"""Trust Layer run card generator for backtest runs."""
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping, Sequence
SCHEMA_VERSION = "0.1"
BACKTEST_SUMMARY_KEYS = (
"codes",
"start_date",
"end_date",
"interval",
"engine",
"initial_cash",
"source",
)
def write_run_card(
run_dir: Path,
config: Mapping[str, Any],
metrics: Mapping[str, Any],
*,
data_sources: Sequence[str] | None = None,
strategy_path: Path | None = None,
warnings: Sequence[str] | None = None,
) -> dict[str, Any]:
"""Write JSON and Markdown run cards for a backtest run.
Args:
run_dir: Directory where run_card.json and run_card.md are written.
config: Full backtest configuration. Only a summary and hash are stored.
metrics: Backtest metrics. Scalar values are stored; ``validation`` is
stored separately when present.
data_sources: Data sources used by the run.
strategy_path: Optional strategy source file to hash for reproducibility.
warnings: Optional warnings to include in the card.
Returns:
The run card payload written to ``run_card.json``.
"""
run_dir = Path(run_dir)
run_dir.mkdir(parents=True, exist_ok=True)
config_file = run_dir / "config.json"
reproducibility: dict[str, Any] = {
"config_hash": _file_hash(config_file) if config_file.exists() else _json_hash(config),
}
if strategy_path is not None:
strategy_file = Path(strategy_path)
if strategy_file.exists() and strategy_file.is_file():
reproducibility["strategy_hash"] = _file_hash(strategy_file)
card: dict[str, Any] = {
"schema_version": SCHEMA_VERSION,
"generated_at": _utc_now(),
"run_dir": str(run_dir),
"backtest": _backtest_summary(config),
"reproducibility": reproducibility,
"data_sources": list(data_sources or []),
"metrics": _scalar_metrics(metrics),
"warnings": list(warnings or []),
"artifacts": _list_artifacts(run_dir),
}
if "validation" in metrics:
card["validation"] = metrics["validation"]
json_path = run_dir / "run_card.json"
md_path = run_dir / "run_card.md"
json_path.write_text(
json.dumps(card, ensure_ascii=False, indent=2, sort_keys=True, default=str) + "\n",
encoding="utf-8",
)
md_path.write_text(_render_markdown(card), encoding="utf-8")
return card
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _json_hash(value: Mapping[str, Any]) -> str:
payload = json.dumps(
{key: val for key, val in value.items() if not str(key).startswith("_")},
sort_keys=True,
default=str,
separators=(",", ":"),
ensure_ascii=False,
)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def _file_hash(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _backtest_summary(config: Mapping[str, Any]) -> dict[str, Any]:
return {key: config.get(key) for key in BACKTEST_SUMMARY_KEYS if key in config}
def _scalar_metrics(metrics: Mapping[str, Any]) -> dict[str, Any]:
return {
key: value
for key, value in metrics.items()
if key != "validation" and _is_scalar(value)
}
def _is_scalar(value: Any) -> bool:
return value is None or isinstance(value, (str, int, float, bool))
def _list_artifacts(run_dir: Path) -> list[dict[str, Any]]:
candidates: list[Path] = []
for relative in (Path("config.json"), Path("code/signal_engine.py")):
path = run_dir / relative
if path.exists() and path.is_file():
candidates.append(path)
artifacts_dir = run_dir / "artifacts"
if artifacts_dir.exists() and artifacts_dir.is_dir():
candidates.extend(path for path in artifacts_dir.rglob("*") if path.is_file())
artifacts = []
for path in sorted(candidates, key=lambda item: item.relative_to(run_dir).as_posix()):
artifacts.append(
{
"path": path.relative_to(run_dir).as_posix(),
"size_bytes": path.stat().st_size,
"sha256": _file_hash(path),
}
)
return artifacts
def _render_markdown(card: Mapping[str, Any]) -> str:
lines = [
"# Backtest Run Card",
"",
f"Generated: {card['generated_at']}",
f"Run directory: `{card['run_dir']}`",
"",
"## Backtest Summary",
]
backtest = card.get("backtest", {})
if backtest:
lines.extend(f"- {key}: {value}" for key, value in backtest.items())
else:
lines.append("- No backtest summary fields provided.")
lines.extend(["", "## Reproducibility"])
reproducibility = card.get("reproducibility", {})
lines.append(f"- config_hash: `{reproducibility.get('config_hash', '')}`")
if "strategy_hash" in reproducibility:
lines.append(f"- strategy_hash: `{reproducibility['strategy_hash']}`")
lines.extend(["", "## Data Sources"])
data_sources = card.get("data_sources", [])
lines.extend(f"- {source}" for source in data_sources) if data_sources else lines.append("- None recorded.")
lines.extend(["", "## Metrics"])
metric_values = card.get("metrics", {})
lines.extend(f"- {key}: {value}" for key, value in metric_values.items()) if metric_values else lines.append("- No scalar metrics recorded.")
lines.extend(["", "## Validation"])
if "validation" in card:
validation = card["validation"]
if isinstance(validation, Mapping):
lines.extend(f"- {key}: {value}" for key, value in validation.items())
else:
lines.append(f"- {validation}")
else:
lines.append("- Not present.")
warnings = card.get("warnings", [])
if warnings:
lines.extend(["", "## Warnings"])
lines.extend(f"- {warning}" for warning in warnings)
lines.extend(["", "## Artifacts"])
artifacts = card.get("artifacts", [])
if artifacts:
lines.extend(
f"- `{artifact['path']}` ({artifact['size_bytes']} bytes, sha256 `{artifact['sha256']}`)"
for artifact in artifacts
)
else:
lines.append("- None found.")
return "\n".join(lines) + "\n"
+40 -2
View File
@@ -14,7 +14,7 @@ import logging
import re
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Dict, List, Optional
import pandas as pd
from pydantic import BaseModel, ConfigDict, model_validator, field_validator
@@ -51,6 +51,7 @@ class BacktestConfigSchema(BaseModel):
source: str = "tushare"
interval: str = "1D"
engine: str = "daily"
fundamental_fields: Optional[Dict[str, List[str]]] = None
@field_validator("codes")
@classmethod
@@ -91,6 +92,21 @@ class BacktestConfigSchema(BaseModel):
raise ValueError(f"unsupported source {v!r}, must be one of {_VALID_SOURCES}")
return v
@field_validator("fundamental_fields")
@classmethod
def valid_fundamental_fields(
cls,
v: Optional[Dict[str, List[str]]],
) -> Optional[Dict[str, List[str]]]:
if v is None:
return v
for table, fields in v.items():
if not table.strip():
raise ValueError("fundamental_fields table names must be non-empty strings")
if any(not field.strip() for field in fields):
raise ValueError("fundamental_fields field names must be non-empty strings")
return v
@model_validator(mode="after")
def start_before_end(self) -> "BacktestConfigSchema":
if pd.Timestamp(self.start_date) > pd.Timestamp(self.end_date):
@@ -361,7 +377,24 @@ def main(run_dir: Path) -> None:
Args:
run_dir: Run directory containing ``config.json`` and ``code/signal_engine.py``.
The path is validated against the allowed run roots
(``VIBE_TRADING_ALLOWED_RUN_ROOTS`` plus the defaults) before any
file is read so an arbitrary filesystem location cannot be used
to source ``code/signal_engine.py``.
"""
# Guard the CLI entry point with the same root whitelist the MCP
# ``backtest`` tool already uses (src/tools/backtest_tool.py:23). Without
# this, ``python -m backtest.runner /tmp/attacker_path`` would happily
# import ``signal_engine.py`` from anywhere on disk; the AST scrubber
# below blocks executable top-level statements but a method body still
# runs on instantiation. See ``safe_run_dir`` for the policy.
from src.tools.path_utils import safe_run_dir
try:
run_dir = safe_run_dir(str(run_dir))
except ValueError as exc:
print(json.dumps({"error": str(exc)}))
sys.exit(1)
config_path = run_dir / "config.json"
if not config_path.exists():
print(json.dumps({"error": "config.json not found"}))
@@ -433,6 +466,11 @@ def main(run_dir: Path) -> None:
print(json.dumps({"error": "No data fetched"}))
sys.exit(1)
if source == "auto":
config["_run_card_effective_sources"] = sorted(_group_codes_by_source(codes))
else:
config["_run_card_effective_sources"] = [source]
# Engine
engine_type = config.get("engine", "daily")
signal_engine = engine_cls()
@@ -646,6 +684,6 @@ class _AutoLoader:
if __name__ == "__main__":
if len(sys.argv) < 2:
print(f"Usage: python -m backtest.runner <run_dir>")
print("Usage: python -m backtest.runner <run_dir>")
sys.exit(1)
main(Path(sys.argv[1]))
+861 -163
View File
File diff suppressed because it is too large. Load diff
+167 -64
View File
@@ -27,6 +27,8 @@ Claude Desktop config:
from __future__ import annotations
import json
import logging
import math
import os
import re
import sys
@@ -42,6 +44,8 @@ from fastmcp import FastMCP
mcp = FastMCP("Vibe-Trading")
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Lazy-loaded singletons
@@ -61,6 +65,7 @@ def _get_skills_loader():
global _skills_loader
if _skills_loader is None:
from src.agent.skills import SkillsLoader
_skills_loader = SkillsLoader()
return _skills_loader
@@ -69,6 +74,7 @@ def _get_registry():
global _registry
if _registry is None:
from src.tools import build_registry
_registry = build_registry(include_shell_tools=_include_shell_tools)
return _registry
@@ -77,6 +83,7 @@ def _get_registry():
# Skill tools
# ---------------------------------------------------------------------------
@mcp.tool
def list_skills() -> str:
"""List all available finance skills with names and descriptions.
@@ -111,6 +118,7 @@ def load_skill(name: str) -> str:
# Backtest tool
# ---------------------------------------------------------------------------
@mcp.tool
def backtest(run_dir: str) -> str:
"""Run a vectorized backtest using config.json and code/signal_engine.py.
@@ -133,6 +141,7 @@ def backtest(run_dir: str) -> str:
run_dir: Path to the run directory containing config.json and code/.
"""
from src.tools.backtest_tool import run_backtest
return run_backtest(run_dir)
@@ -140,6 +149,7 @@ def backtest(run_dir: str) -> str:
# Factor analysis tool
# ---------------------------------------------------------------------------
@mcp.tool
def factor_analysis(
codes: list[str],
@@ -165,17 +175,25 @@ def factor_analysis(
bottom_n: Number of bottom-ranked stocks per period.
"""
registry = _get_registry()
return registry.execute("factor_analysis", {
"codes": codes, "factor_name": factor_name,
"start_date": start_date, "end_date": end_date,
"source": source, "top_n": top_n, "bottom_n": bottom_n,
})
return registry.execute(
"factor_analysis",
{
"codes": codes,
"factor_name": factor_name,
"start_date": start_date,
"end_date": end_date,
"source": source,
"top_n": top_n,
"bottom_n": bottom_n,
},
)
# ---------------------------------------------------------------------------
# Options pricing tool
# ---------------------------------------------------------------------------
@mcp.tool
def analyze_options(
spot: float,
@@ -196,17 +214,24 @@ def analyze_options(
option_type: "call" or "put".
"""
registry = _get_registry()
return registry.execute("options_pricing", {
"spot": spot, "strike": strike, "expiry_days": expiry_days,
"risk_free_rate": risk_free_rate, "volatility": volatility,
"option_type": option_type,
})
return registry.execute(
"options_pricing",
{
"spot": spot,
"strike": strike,
"expiry_days": expiry_days,
"risk_free_rate": risk_free_rate,
"volatility": volatility,
"option_type": option_type,
},
)
# ---------------------------------------------------------------------------
# Pattern recognition tool
# ---------------------------------------------------------------------------
@mcp.tool
def pattern_recognition(run_dir: str) -> str:
"""Detect technical chart patterns (head-and-shoulders, double top/bottom,
@@ -226,6 +251,7 @@ def pattern_recognition(run_dir: str) -> str:
# Web & document reading tools
# ---------------------------------------------------------------------------
@mcp.tool
def read_url(url: str) -> str:
"""Fetch a web page and convert it to clean Markdown text.
@@ -237,6 +263,7 @@ def read_url(url: str) -> str:
url: Target URL to read.
"""
from src.tools.web_reader_tool import read_url as _read_url
return _read_url(url)
@@ -258,6 +285,7 @@ def read_document(file_path: str) -> str:
# Web search tool
# ---------------------------------------------------------------------------
@mcp.tool
def web_search(query: str, max_results: int = 5) -> str:
"""Search the web via DuckDuckGo and return top results.
@@ -270,15 +298,20 @@ def web_search(query: str, max_results: int = 5) -> str:
max_results: Maximum results to return (default 5, max 10).
"""
registry = _get_registry()
return registry.execute("web_search", {
"query": query, "max_results": min(max_results, 10),
})
return registry.execute(
"web_search",
{
"query": query,
"max_results": min(max_results, 10),
},
)
# ---------------------------------------------------------------------------
# File I/O tools (sandboxed to workspace)
# ---------------------------------------------------------------------------
@mcp.tool
def write_file(path: str, content: str) -> str:
"""Write content to a file. Used to create config.json and signal_engine.py
@@ -307,6 +340,7 @@ def read_file(path: str) -> str:
# Swarm team tool
# ---------------------------------------------------------------------------
@mcp.tool
def list_swarm_presets() -> str:
"""List available swarm multi-agent team presets.
@@ -316,6 +350,7 @@ def list_swarm_presets() -> str:
Returns preset names, descriptions, agent counts, and required variables.
"""
from src.swarm.presets import list_presets
presets = list_presets()
return json.dumps(presets, ensure_ascii=False, indent=2)
@@ -336,15 +371,17 @@ def run_swarm(preset_name: str, variables: dict[str, str]) -> str:
"""
import time
from src.swarm.runtime import SwarmRuntime
from src.swarm.store import SwarmStore
from src.swarm.store import SwarmStore, swarm_runs_root
from src.swarm.models import RunStatus
swarm_dir = AGENT_DIR / ".swarm" / "runs"
swarm_dir = swarm_runs_root()
store = SwarmStore(base_dir=swarm_dir)
runtime = SwarmRuntime(store=store)
try:
run = runtime.start_run(preset_name, variables)
run = runtime.start_run(
preset_name, variables, include_shell_tools=_include_shell_tools
)
except FileNotFoundError as exc:
return json.dumps({"status": "error", "error": str(exc)}, ensure_ascii=False)
except ValueError as exc:
@@ -357,19 +394,23 @@ def run_swarm(preset_name: str, variables: dict[str, str]) -> str:
if current is None:
return json.dumps({"status": "error", "error": "Run record lost"}, ensure_ascii=False)
if current.status in (RunStatus.completed, RunStatus.failed, RunStatus.cancelled):
tasks = [
{"id": t.id, "agent_id": t.agent_id, "status": t.status.value, "summary": t.summary}
for t in current.tasks
]
return json.dumps({
"status": current.status.value,
"preset": preset_name,
"run_id": current.id,
"final_report": current.final_report,
"tasks": tasks,
"total_input_tokens": current.total_input_tokens,
"total_output_tokens": current.total_output_tokens,
}, ensure_ascii=False, indent=2)
from src.swarm.serialization import run_level_error, serialize_task
tasks = [serialize_task(t) for t in current.tasks]
return json.dumps(
{
"status": current.status.value,
"preset": preset_name,
"run_id": current.id,
"final_report": current.final_report,
"error": run_level_error(current),
"tasks": tasks,
"total_input_tokens": current.total_input_tokens,
"total_output_tokens": current.total_output_tokens,
},
ensure_ascii=False,
indent=2,
)
return json.dumps({"status": "error", "error": "Swarm timed out after 30 minutes"}, ensure_ascii=False)
@@ -378,6 +419,8 @@ def run_swarm(preset_name: str, variables: dict[str, str]) -> str:
# Market data tool
# ---------------------------------------------------------------------------
DEFAULT_MAX_ROWS = 250
_SOURCE_PATTERNS = [
(re.compile(r"^\d{6}\.(SZ|SH|BJ)$", re.I), "tushare"),
(re.compile(r"^[A-Z]+\.US$", re.I), "yfinance"),
@@ -397,9 +440,40 @@ def _detect_source(code: str) -> str:
def _get_loader(source: str):
"""Get loader class via registry with fallback support."""
from backtest.loaders.registry import get_loader_cls_with_fallback
return get_loader_cls_with_fallback(source)
def _cap_rows(records: list, max_rows: int) -> list | dict[str, object]:
"""Bound a per-symbol row list to keep the MCP payload within budget.
max_rows==0 disables the cap (full list, unchanged shape). A negative
max_rows is invalid and enforces the default cap (never unbounded).
Otherwise an oversized symbol is *evenly strided* — every step-th bar,
with the last bar pinned — so the returned series spans the full range
(no head+tail gap, no synthetic ``_gap`` sentinel). Symbols within the
cap are returned unchanged (plain list) — small queries are
byte-identical.
"""
n = len(records)
if max_rows < 0:
max_rows = DEFAULT_MAX_ROWS # negative invalid -> enforce cap, never unbounded
if max_rows == 0 or n <= max_rows:
return records
step = math.ceil(n / max_rows)
sampled = records[::step]
if sampled[-1] is not records[-1]:
sampled = sampled + [records[-1]]
return {
"rows": n,
"returned": len(sampled),
"truncated": True,
"policy": f"every-{step}th-row (even stride; last bar pinned)",
"hint": "narrow the date range, coarsen interval, or set max_rows=0 for all rows",
"data": sampled,
}
@mcp.tool
def get_market_data(
codes: list[str],
@@ -407,6 +481,7 @@ def get_market_data(
end_date: str,
source: str = "auto",
interval: str = "1D",
max_rows: int = DEFAULT_MAX_ROWS,
) -> str:
"""Fetch OHLCV market data for stocks, crypto, or mixed symbols.
@@ -424,6 +499,11 @@ def get_market_data(
end_date: End date (YYYY-MM-DD).
source: Data source ("auto", "yfinance", "okx", "tushare", "akshare", "ccxt").
interval: Bar size (1m/5m/15m/30m/1H/4H/1D, default "1D").
max_rows: Per-symbol row cap (default 250) so the response stays
within the MCP token budget. A symbol exceeding it returns an
even-stride downsample (every step-th bar, last bar pinned)
plus truncation metadata. Set max_rows=0 for all rows
(unbounded, legacy behavior).
"""
results = {}
@@ -438,7 +518,14 @@ def get_market_data(
for src, src_codes in groups.items():
loader_cls = _get_loader(src)
loader = loader_cls()
data_map = loader.fetch(src_codes, start_date, end_date, interval=interval)
try:
data_map = loader.fetch(src_codes, start_date, end_date, interval=interval)
except Exception:
# A loader blow-up for one group must not lose already-resolved
# symbols or surface as an opaque MCP error; those codes fall
# through to _unresolved below (P05).
logger.exception("market-data loader %r failed for %s; codes fall through to _unresolved", src, src_codes)
data_map = {}
for symbol, df in data_map.items():
records = df.reset_index().to_dict(orient="records")
for r in records:
@@ -447,7 +534,16 @@ def get_market_data(
r[k] = v.isoformat()
elif hasattr(v, "item"):
r[k] = v.item()
results[symbol] = records
results[symbol] = _cap_rows(records, max_rows)
# P05: a typo / wrong-suffix / delisted / no-data symbol used to vanish
# silently (the dict only held winners), indistinguishable from "no data".
# Surface every requested code that produced nothing under a reserved key.
# Additive: omitted entirely when all codes resolved, so the happy-path
# payload is byte-identical to before.
unresolved = [c for c in codes if c not in results]
if unresolved:
results["_unresolved"] = unresolved
return json.dumps(results, ensure_ascii=False, indent=2)
@@ -456,28 +552,25 @@ def get_market_data(
# Swarm status & history tools
# ---------------------------------------------------------------------------
def _get_swarm_store():
swarm_dir = AGENT_DIR / ".swarm" / "runs"
from src.swarm.store import SwarmStore, swarm_runs_root
swarm_dir = swarm_runs_root()
swarm_dir.mkdir(parents=True, exist_ok=True)
from src.swarm.store import SwarmStore
return SwarmStore(base_dir=swarm_dir)
def _run_to_dict(run) -> dict:
from src.swarm.serialization import run_level_error, serialize_task
return {
"run_id": run.id,
"status": run.status.value,
"preset": run.preset_name,
"created_at": run.created_at,
"tasks": [
{
"id": t.id,
"agent_id": t.agent_id,
"status": t.status.value,
"summary": t.summary,
}
for t in run.tasks
],
"error": run_level_error(run),
"tasks": [serialize_task(t) for t in run.tasks],
"final_report": run.final_report,
"total_input_tokens": run.total_input_tokens,
"total_output_tokens": run.total_output_tokens,
@@ -532,14 +625,16 @@ def list_runs(limit: int = 20) -> str:
runs = store.list_runs(limit=limit)
items = []
for run in runs:
items.append({
"run_id": run.id,
"preset": run.preset_name,
"status": run.status.value,
"created_at": run.created_at,
"total_input_tokens": run.total_input_tokens,
"total_output_tokens": run.total_output_tokens,
})
items.append(
{
"run_id": run.id,
"preset": run.preset_name,
"status": run.status.value,
"created_at": run.created_at,
"total_input_tokens": run.total_input_tokens,
"total_output_tokens": run.total_output_tokens,
}
)
return json.dumps(items, ensure_ascii=False, indent=2)
@@ -547,6 +642,7 @@ def list_runs(limit: int = 20) -> str:
# Trade journal tool
# ---------------------------------------------------------------------------
@mcp.tool
def analyze_trade_journal(
file_path: str,
@@ -570,17 +666,21 @@ def analyze_trade_journal(
"symbol=600519.SH", "market=china_a").
"""
registry = _get_registry()
return registry.execute("analyze_trade_journal", {
"file_path": file_path,
"analysis_type": analysis_type,
"filter_expr": filter_expr,
})
return registry.execute(
"analyze_trade_journal",
{
"file_path": file_path,
"analysis_type": analysis_type,
"filter_expr": filter_expr,
},
)
# ---------------------------------------------------------------------------
# Shadow Account tools (4)
# ---------------------------------------------------------------------------
@mcp.tool
def extract_shadow_strategy(
journal_path: str,
@@ -600,11 +700,14 @@ def extract_shadow_strategy(
max_rules: Maximum rules to return (typically 3-5).
"""
registry = _get_registry()
return registry.execute("extract_shadow_strategy", {
"journal_path": journal_path,
"min_support": min_support,
"max_rules": max_rules,
})
return registry.execute(
"extract_shadow_strategy",
{
"journal_path": journal_path,
"min_support": min_support,
"max_rules": max_rules,
},
)
@mcp.tool
@@ -697,19 +800,19 @@ def scan_shadow_signals(
# Entry point
# ---------------------------------------------------------------------------
def main():
"""Entry point for `vibe-trading-mcp` CLI command."""
global _include_shell_tools, _registry
import argparse
parser = argparse.ArgumentParser(description="Vibe-Trading MCP Server")
parser.add_argument("--transport", choices=["stdio", "sse"], default="stdio",
help="MCP transport (default: stdio)")
parser.add_argument("--port", type=int, default=8900,
help="SSE port (only used with --transport sse)")
parser.add_argument("--transport", choices=["stdio", "sse"], default="stdio", help="MCP transport (default: stdio)")
parser.add_argument("--port", type=int, default=8900, help="SSE port (only used with --transport sse)")
args = parser.parse_args()
_include_shell_tools = True if args.transport == "stdio" else _env_shell_tools_enabled()
_registry = None
_get_registry() # pre-warm: avoids deadlock when first tools/call lazy-inits inside FastMCP worker thread
if args.transport == "sse":
mcp.run(transport="sse", port=args.port)
+250
View File
@@ -0,0 +1,250 @@
"""W4.a blog patcher: fill TBD placeholders in alpha-191-in-2026.html.
Reads ~/.vibe-trading/reports/bench_summary.json (produced by w4a_run_benches.py)
and patches the HTML in place.
Substitutions performed:
- num-alive / num-reversed / num-dead spans in the two stat blocks
- Theme survival table rows (count + survival rate per theme)
- Top 5 alphas (id + formula + IC mean)
- 3 famously-dead alphas (replace invented IDs with worst-IC from gtja191)
- Section-level "bench failed" callouts where the corresponding bench errored
The patcher never fabricates a number: if the gtja191 bench is missing, every
TBD that depends on it becomes a "bench failed" note instead. The script is
idempotent — re-running with the same summary produces the same HTML.
"""
from __future__ import annotations
import html
import json
import re
import sys
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2]
HTML_PATH = REPO / "wiki" / "research-lab" / "posts" / "alpha-191-in-2026.html"
SUMMARY_PATH = Path.home() / ".vibe-trading" / "reports" / "bench_summary.json"
GTJA_KEY = "gtja191_csi300"
# Themes shown in the table — these are the human labels in the HTML, mapped
# to the canonical theme tags in registry meta. "Volume-price interaction"
# requires *both* volume and reversal-or-momentum, so it gets a custom probe.
THEME_LABEL_TO_TAGS: dict[str, list[str]] = {
"Volume-price interaction": ["volume"],
"Short-horizon volatility": ["volatility"],
"Reversal": ["reversal"],
"Momentum": ["momentum"],
"Turnover / liquidity": ["liquidity"],
"Microstructure / range": ["microstructure"],
}
def _load_summary() -> dict[str, Any]:
if not SUMMARY_PATH.is_file():
raise SystemExit(f"summary not found: {SUMMARY_PATH}")
return json.loads(SUMMARY_PATH.read_text(encoding="utf-8"))
def _gtja_entry(summary: dict[str, Any]) -> dict[str, Any] | None:
entry = summary.get("zoos", {}).get(GTJA_KEY)
if not entry or entry.get("status") != "ok":
return None
return entry
def _patch_stats(text: str, entry: dict[str, Any]) -> str:
for span_class, key in (
("num-alive", "alive"),
("num-reversed", "reversed"),
("num-dead", "dead"),
):
value = entry.get(key)
if value is None:
continue
pattern = re.compile(
rf'(<span class="{span_class}">)\[TBD\](</span>)'
)
text = pattern.sub(rf"\g<1>{int(value)}\g<2>", text)
return text
def _theme_survival_rows(entry: dict[str, Any]) -> str:
"""Build the inner <tr> rows for the theme-survival table."""
by_theme = entry.get("by_theme", {})
rows: list[str] = []
definitions = {
"Volume-price interaction": "Correlation / covariance of volume with close, high-low range",
"Short-horizon volatility": "Rolling std / range over 5-20 day windows",
"Reversal": "Negative-sign return signals over 1-5 day horizons",
"Momentum": "Positive-sign return signals over 10-60 day horizons",
"Turnover / liquidity": "Volume ratios, turnover-rate transforms",
"Microstructure / range": "Open-close-high-low decompositions, intraday range proxies",
}
for label, tags in THEME_LABEL_TO_TAGS.items():
# Aggregate across all listed tags (most themes are single-tag).
alive = dead = reversed_ = count = 0
for tag in tags:
bucket = by_theme.get(tag)
if bucket is None:
continue
alive += bucket.get("alive", 0)
dead += bucket.get("dead", 0)
reversed_ += bucket.get("reversed", 0)
count += bucket.get("count", 0)
if count == 0:
count_cell = "n/a"
rate_cell = "n/a"
else:
count_cell = str(count)
rate_cell = f"{alive / count * 100:.0f}% ({alive}/{count})"
rows.append(
" <tr>\n"
f" <td>{html.escape(label)}</td>\n"
f" <td>{html.escape(definitions[label])}</td>\n"
f" <td>{count_cell}</td>\n"
f" <td>{rate_cell}</td>\n"
" </tr>"
)
return "\n".join(rows)
def _patch_theme_table(text: str, entry: dict[str, Any]) -> str:
new_rows = _theme_survival_rows(entry)
pattern = re.compile(
r'(<tbody>)(.*?)(</tbody>)',
re.DOTALL,
)
# The HTML has exactly one <tbody> inside the theme-table; surgical replace.
def _swap(match: re.Match[str]) -> str:
body = match.group(2)
if "Volume-price interaction" not in body:
return match.group(0) # not the theme table
return f"{match.group(1)}\n{new_rows}\n {match.group(3)}"
return pattern.sub(_swap, text, count=1)
def _formula_to_html(formula: str) -> str:
return html.escape(formula)
def _build_top5_cards(entry: dict[str, Any]) -> str:
rows = entry.get("top5_by_ir", [])[:5]
cards: list[str] = []
for row in rows:
aid = row["id"]
formula = _formula_to_html(row.get("formula_latex", ""))
ic = row.get("ic_mean", 0.0)
ir = row.get("ir", 0.0)
cards.append(
" <article class=\"alpha-card\">\n"
f" <span class=\"alpha-id\">{html.escape(aid)}</span>\n"
f" <pre class=\"formula-block\"><code>{formula}</code></pre>\n"
" <p class=\"alpha-note\">"
f"Mean IC = {ic:.4f}, IR = {ir:.4f} over the CSI 300 / 2018&ndash;2025 window. "
"Formula reproduced verbatim from the registry "
f"(<code>__alpha_meta__[\"formula_latex\"]</code> of <code>{html.escape(aid)}</code>)."
"</p>\n"
" </article>"
)
return "\n".join(cards)
def _build_dead_cards(entry: dict[str, Any]) -> str:
rows = entry.get("dead_examples", [])[:3]
cards: list[str] = []
for row in rows:
aid = row["id"]
formula = _formula_to_html(row.get("formula_latex", ""))
ic = row.get("ic_mean", 0.0)
cat = row.get("category", "dead")
ir = row.get("ir", 0.0)
cards.append(
" <article class=\"alpha-card\">\n"
f" <span class=\"alpha-id\">{html.escape(aid)} ({html.escape(cat)})</span>\n"
f" <pre class=\"formula-block\"><code>{formula}</code></pre>\n"
" <p class=\"alpha-note\">"
f"Mean IC = {ic:.4f}, IR = {ir:.4f}. Worst-performing slice of the gtja191 "
"zoo on CSI 300 / 2018&ndash;2025 by raw IC."
"</p>\n"
" </article>"
)
return "\n".join(cards)
def _patch_top5(text: str, entry: dict[str, Any]) -> str:
cards = _build_top5_cards(entry)
pattern = re.compile(
r'(<h3>Top 5 surviving alphas</h3>.*?</p>\s*)' # heading + lead paragraph
r'(?:<article class="alpha-card">.*?</article>\s*){5}',
re.DOTALL,
)
return pattern.sub(rf"\g<1>{cards}\n\n ", text, count=1)
def _patch_dead(text: str, entry: dict[str, Any]) -> str:
cards = _build_dead_cards(entry)
pattern = re.compile(
r'(<h3>Three famously dead alphas</h3>.*?</p>\s*)'
r'(?:<article class="alpha-card">.*?</article>\s*){3}',
re.DOTALL,
)
return pattern.sub(rf"\g<1>{cards}\n\n ", text, count=1)
def _patch_bench_failed_note(text: str) -> str:
"""Replace the W4-roadmap placeholder callout with a real timestamp note."""
return text.replace(
"Numbers will be filled in once the full bench finishes (Vibe-Trading\n roadmap W4.a). "
"This post is published as a methodology preview &mdash; the structure, definitions\n and caveats are final.",
"Numbers below are the live W4.a bench output on the bundled gtja191 zoo "
"(CSI 300, 2018&ndash;2025) &mdash; reproducible via the CLI snippet at the end of the post.",
)
def main() -> int:
summary = _load_summary()
entry = _gtja_entry(summary)
text = HTML_PATH.read_text(encoding="utf-8")
if entry is None:
# Surface the failure once at the top; leave all TBDs intact so the
# reader sees we have not invented numbers.
zoo_blob = summary.get("zoos", {}).get(GTJA_KEY, {})
reason = zoo_blob.get("error", "bench did not run")
note = (
'<p class="bench-failed"><strong>Note:</strong> the gtja191 / CSI 300 bench '
f"failed during W4.a (reason: {html.escape(str(reason))}). All TBD numbers below "
"remain to be filled in 0.1.9 once the bench is re-run.</p>"
)
text = text.replace(
'roadmap W4.a). This post',
f'roadmap W4.a). {note} This post'
)
HTML_PATH.write_text(text, encoding="utf-8")
print(f"bench failed; left TBDs intact and added note. tbd count = "
f"{text.count('[TBD]')}", file=sys.stderr)
return 0
text = _patch_stats(text, entry)
text = _patch_theme_table(text, entry)
text = _patch_top5(text, entry)
text = _patch_dead(text, entry)
text = _patch_bench_failed_note(text)
# Mean IC ~ [TBD] placeholders inside the per-card paragraphs are no longer
# needed (we rewrote the cards with real numbers). Sanity-check sweep.
HTML_PATH.write_text(text, encoding="utf-8")
remaining = text.count("[TBD]")
print(f"patched {HTML_PATH}; remaining [TBD] = {remaining}")
return 0
if __name__ == "__main__":
sys.exit(main())
+170
View File
@@ -0,0 +1,170 @@
"""W4.a bench driver: run 4 zoo×universe combinations and emit summary JSON.
Usage:
cd agent && python scripts/w4a_run_benches.py
Output:
~/.vibe-trading/reports/alpha_bench_*.html — per-run HTML reports
~/.vibe-trading/reports/bench_<zoo>_<uni>.json — per-run raw results
~/.vibe-trading/reports/bench_summary.json — aggregate summary
Safe to re-run; each bench result is cached by ``_load_universe_panel`` so the
expensive Tushare/yfinance fetches only happen once per (universe, period).
"""
from __future__ import annotations
import json
import logging
import os
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from dotenv import load_dotenv
# Repo-relative imports
HERE = Path(__file__).resolve().parent
AGENT_DIR = HERE.parent
sys.path.insert(0, str(AGENT_DIR))
load_dotenv(AGENT_DIR / ".env")
# Configure logging early so universe loaders log their progress.
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s | %(message)s",
)
logger = logging.getLogger("w4a")
from src.factors.bench_runner import run_bench # noqa: E402
from src.tools.alpha_bench_tool import run_alpha_bench # noqa: E402
REPORTS_DIR = Path.home() / ".vibe-trading" / "reports"
BENCHES = [
{"key": "gtja191_csi300", "zoo": "gtja191", "universe": "csi300", "period": "2018-2025"},
{"key": "alpha101_sp500", "zoo": "alpha101", "universe": "sp500", "period": "2020-2025"},
{"key": "qlib158_csi300", "zoo": "qlib158", "universe": "csi300", "period": "2020-2025"},
{"key": "alpha101_btc", "zoo": "alpha101", "universe": "btc-usdt", "period": "2022-2025"},
]
def _run_one(bench: dict) -> dict:
"""Run a single bench end-to-end. Returns a summary entry.
Delegates the math to ``src.factors.bench_runner.run_bench``; this wrapper
just adds the HTML report (via ``run_alpha_bench``) and persists the raw
per-alpha rows for downstream tooling, both of which the CLI driver wants
but the API route does not.
"""
key = bench["key"]
zoo = bench["zoo"]
universe = bench["universe"]
period = bench["period"]
logger.info("=== bench %s: zoo=%s universe=%s period=%s ===", key, zoo, universe, period)
start = time.monotonic()
entry: dict = {
"key": key,
"zoo": zoo,
"universe": universe,
"period": period,
"status": "pending",
}
# 1. HTML report (also exercises the universe loader so the panel is cached
# on disk for the run_bench call below).
try:
envelope = run_alpha_bench(
zoo=zoo, universe=universe, period=period, top=20
)
except Exception as exc: # noqa: BLE001 — never let one bench abort the suite
logger.exception("bench %s crashed (run_alpha_bench)", key)
entry["status"] = "error"
entry["error"] = f"unexpected: {exc!s}"
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
if envelope.get("status") != "ok":
entry["status"] = "error"
entry["error"] = envelope.get("error", "unknown")
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
# 2. Full bench (per-alpha rows + categorisation).
try:
result = run_bench(zoo=zoo, universe=universe, period=period, top=20)
except Exception as exc: # noqa: BLE001
logger.exception("bench %s crashed (run_bench)", key)
entry["status"] = "error"
entry["error"] = f"unexpected: {exc!s}"
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
if result.get("status") != "ok":
entry["status"] = "error"
entry["error"] = result.get("error", "unknown")
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
rows = result.pop("rows", [])
skipped = result.pop("skipped", [])
entry.update(result)
entry["report_path"] = envelope.get("report_path")
entry["wall_seconds"] = round(time.monotonic() - start, 2)
# Write per-bench raw rows for downstream tooling.
raw_path = REPORTS_DIR / f"bench_{key}.json"
raw_path.write_text(
json.dumps(
{"meta": {k: entry[k] for k in ("key", "zoo", "universe", "period", "status")},
"rows": rows, "skipped": skipped},
indent=2,
default=str,
),
encoding="utf-8",
)
logger.info(
"bench %s done: tested=%d skipped=%d alive=%d reversed=%d dead=%d (%.1fs)",
key, len(rows), len(skipped),
entry["alive"], entry["reversed"], entry["dead"],
entry["wall_seconds"],
)
return entry
def main() -> int:
REPORTS_DIR.mkdir(parents=True, exist_ok=True)
summary: dict = {
"generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
"zoos": {},
}
tushare_ok = bool(os.getenv("TUSHARE_TOKEN", "").strip())
if not tushare_ok:
logger.warning("TUSHARE_TOKEN unset; csi300 benches will be skipped")
for bench in BENCHES:
if bench["universe"] == "csi300" and not tushare_ok:
summary["zoos"][bench["key"]] = {
**{k: bench[k] for k in ("zoo", "universe", "period")},
"status": "skipped",
"error": "TUSHARE_TOKEN not set",
}
continue
entry = _run_one(bench)
summary["zoos"][bench["key"]] = entry
out = REPORTS_DIR / "bench_summary.json"
out.write_text(json.dumps(summary, indent=2, default=str), encoding="utf-8")
logger.info("wrote summary -> %s", out)
return 0
if __name__ == "__main__":
sys.exit(main())
+24 -5
View File
@@ -435,27 +435,46 @@ class AgentLoop:
"react_trace": react_trace,
}
# Determine final status
# Determine final status. The reason is also propagated into the
# returned dict so SessionService can surface a meaningful UI
# message instead of "Execution failed: unknown" (issue #114).
final_reason: str | None = None
if self._cancelled:
state_store.mark_failure(run_dir, "cancelled by user")
final_reason = "cancelled by user"
state_store.mark_failure(run_dir, final_reason)
final_status = "cancelled"
elif (run_dir / "artifacts" / "metrics.csv").exists() or final_content:
state_store.mark_success(run_dir)
final_status = "success"
else:
state_store.mark_failure(run_dir, "pipeline did not complete")
final_reason = (
f"reached max iterations ({self.max_iterations}) without final answer"
)
state_store.mark_failure(run_dir, final_reason)
final_status = "failed"
trace.write({"type": "end", "status": final_status, "iterations": iteration})
end_event: dict[str, Any] = {
"type": "end",
"status": final_status,
"iterations": iteration,
}
if final_reason is not None:
end_event["reason"] = final_reason
trace.write(end_event)
trace.close()
return {
result: dict[str, Any] = {
"status": final_status,
"run_dir": str(run_dir),
"run_id": run_dir.name,
"content": final_content,
"react_trace": react_trace,
"iterations": iteration,
"max_iterations": self.max_iterations,
}
if final_reason is not None:
result["reason"] = final_reason
return result
# -- Tool execution with read/write batching --------------------------------
+1
View File
@@ -0,0 +1 @@
"""HTTP route packages mounted onto the main FastAPI app in ``api_server.py``."""
+564
View File
@@ -0,0 +1,564 @@
"""Alpha Zoo HTTP routes for the Web UI.
Mounted by ``agent/api_server.py`` via ``register_alpha_routes(app, ...)``. Pulled
into its own module because ``api_server.py`` is already ~1800 lines.
Routes (auth via the caller-supplied ``require_auth`` /
``require_event_stream_auth`` dependencies):
- ``GET /alpha/list`` — list alphas with optional filters
- ``GET /alpha/{alpha_id}`` — single alpha meta + source
- ``POST /alpha/bench`` — kick off a background bench (returns job_id)
- ``GET /alpha/bench/{job_id}/stream`` — SSE: progress / result / done / error
Job state lives in the module-level ``ALPHA_BENCH_JOBS`` dict, guarded by
``_JOBS_LOCK``. No persistence — process restart wipes job state, which is
acceptable for v1 (a bench takes 5-10 min; users re-trigger).
Concurrency: at most ``MAX_CONCURRENT_BENCHES`` bench workers run at the same
time across the process. POST returns 429 when the cap is reached. Workers run
in a thread (``asyncio.to_thread``) and we hold task references in a module
set so the loop's garbage collector cannot cancel them mid-run.
Error surface: typed errors (``ValueError`` on bad period, ``KeyError`` on
unknown alpha_id) bubble out with their original message. Anything else is
logged with full traceback and reported to clients as ``"internal error; see
server logs"`` to avoid leaking stack frames / paths.
"""
from __future__ import annotations
import asyncio
import json
import logging
import re
import threading
import time
import uuid
from datetime import datetime, timezone
from typing import Any, Awaitable, Callable
from fastapi import Depends, FastAPI, HTTPException, Query, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field, field_validator
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Job store (in-memory, process-local)
# ---------------------------------------------------------------------------
ALPHA_BENCH_JOBS: dict[str, dict[str, Any]] = {}
_JOBS_LOCK = threading.Lock()
# Live background bench tasks. Holding strong refs prevents the asyncio GC from
# cancelling fire-and-forget tasks; ``add`` on create, ``discard`` on done.
_RUNNING_TASKS: set[asyncio.Task[Any]] = set()
_JOB_TTL_SECONDS = 60 * 60 # prune jobs older than 1 hour on each new POST
_POLL_INTERVAL_SECONDS = 0.5
_SSE_HEARTBEAT_SECONDS = 15.0
# Cap on simultaneous bench workers. A bench is pandas-heavy (5-10 min wall) so
# unbounded concurrency would DoS the server with two or three clicks. The
# semaphore is created lazily on the first request so we bind to the running
# event loop, not the import-time one.
MAX_CONCURRENT_BENCHES = 2
_BENCH_SEMAPHORE: asyncio.Semaphore | None = None
_BENCH_SEMAPHORE_LOCK = threading.Lock()
def _get_bench_semaphore() -> asyncio.Semaphore:
"""Return the process-wide bench semaphore, building it on first call."""
global _BENCH_SEMAPHORE
with _BENCH_SEMAPHORE_LOCK:
if _BENCH_SEMAPHORE is None:
_BENCH_SEMAPHORE = asyncio.Semaphore(MAX_CONCURRENT_BENCHES)
return _BENCH_SEMAPHORE
# Tighter alpha_id pattern: ``<zoo>_<short>``. Zoo prefix is short (a-z + a-z0-9),
# short id is 1-64 of [a-z0-9_]. Caps avoid pathological lookups.
_ALPHA_ID_RE = re.compile(r"^[a-z][a-z0-9]+_[a-z0-9_]{1,64}$")
_JOB_ID_RE = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
# Filter enums — keep in sync with src.factors.registry.Theme / Universe.
_VALID_ZOOS = {"alpha101", "gtja191", "qlib158", "academic"}
_VALID_THEMES = {
"momentum", "reversal", "volume", "volatility", "quality", "value",
"liquidity", "microstructure", "sentiment", "growth", "leverage",
}
_VALID_UNIVERSES = {"equity_us", "equity_cn", "equity_hk", "crypto", "futures"}
def _now_iso() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def _safe_error(exc: BaseException) -> str:
"""Sanitised user-facing error string for unexpected exceptions.
Always returns a fixed phrase — never echoes the exception message or
type, both of which can include filesystem paths, credentials, or stack
frame snippets. Callers should ``logger.exception`` BEFORE invoking this.
"""
return "internal error; see server logs"
def _prune_old_jobs() -> None:
"""Drop completed/errored jobs older than ``_JOB_TTL_SECONDS``."""
cutoff = time.time() - _JOB_TTL_SECONDS
with _JOBS_LOCK:
stale = [
jid for jid, job in ALPHA_BENCH_JOBS.items()
if job.get("status") in ("done", "error")
and job.get("_finished_at", 0) < cutoff
]
for jid in stale:
ALPHA_BENCH_JOBS.pop(jid, None)
# ---------------------------------------------------------------------------
# Request/response schemas
# ---------------------------------------------------------------------------
class BenchRequest(BaseModel):
"""POST /alpha/bench body."""
zoo: str = Field(..., min_length=1, max_length=64)
universe: str = Field(..., min_length=1, max_length=64)
period: str = Field(..., min_length=4, max_length=32)
top: int = Field(20, ge=1, le=500)
@field_validator("zoo")
@classmethod
def _zoo_known(cls, v: str) -> str:
if v not in _VALID_ZOOS:
raise ValueError(
f"unknown zoo {v!r}; expected one of {sorted(_VALID_ZOOS)}"
)
return v
@field_validator("universe")
@classmethod
def _universe_known(cls, v: str) -> str:
allowed = {"csi300", "sp500", "btc-usdt"}
if v not in allowed:
raise ValueError(
f"unknown universe {v!r}; expected one of {sorted(allowed)}"
)
return v
# ---------------------------------------------------------------------------
# Bench worker (runs in a thread; LLM-free, pandas-heavy)
# ---------------------------------------------------------------------------
def _make_progress_cb(job_id: str) -> Callable[[int, int, str], None]:
"""Return an on_progress closure that updates the job entry in-place."""
def _cb(n_done: int, n_total: int, alpha_id: str) -> None:
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is None:
return
job["progress"] = {
"n_done": int(n_done),
"n_total": int(n_total),
"current_alpha_id": alpha_id,
}
if job["status"] == "queued":
job["status"] = "running"
return _cb
def _run_bench_blocking(job_id: str, zoo: str, universe: str, period: str, top: int) -> None:
"""Synchronous bench worker (called via ``asyncio.to_thread``)."""
from src.factors.bench_runner import run_bench # local import: heavy deps
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is not None:
job["status"] = "running"
try:
result = run_bench(
zoo=zoo,
universe=universe,
period=period,
top=top,
on_progress=_make_progress_cb(job_id),
)
except Exception as exc: # noqa: BLE001 — worker must never crash the loop
logger.exception("alpha bench worker crashed (job=%s)", job_id)
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is not None:
job["status"] = "error"
job["error"] = _safe_error(exc)
job["_finished_at"] = time.time()
return
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is None:
return
if result.get("status") != "ok":
job["status"] = "error"
# bench_runner's own error string is curated (universe load failed,
# forward returns failed, etc.) — safe to surface.
job["error"] = result.get("error", "unknown")
else:
# Strip the bulky per-alpha lists — the API contract returns
# summary-only on the result event. We keep ``n_skipped`` (the
# count) which ``_result_for_wire`` reshapes into ``skipped``.
slim = {
k: v for k, v in result.items() if k not in ("rows", "skipped")
}
job["status"] = "done"
job["result"] = slim
job["_finished_at"] = time.time()
# ---------------------------------------------------------------------------
# Registration
# ---------------------------------------------------------------------------
AuthDep = Callable[..., Awaitable[Any] | Any]
def register_alpha_routes(
app: FastAPI,
require_auth: AuthDep | None = None,
require_event_stream_auth: AuthDep | None = None,
) -> None:
"""Mount the alpha routes onto ``app``.
Args:
app: The host FastAPI app.
require_auth: Header-auth dependency for JSON endpoints.
require_event_stream_auth: Query-param-auth dependency for SSE endpoints.
For backwards compatibility, when the dependency callables are not passed
explicitly we resolve them from the host ``api_server`` module via
``sys.modules``. Prefer the explicit form in new call sites.
"""
if require_auth is None or require_event_stream_auth is None:
import sys as _sys
host = _sys.modules.get("api_server") or _sys.modules.get("agent.api_server")
if host is None: # pragma: no cover — only triggers on weird import setups
raise RuntimeError(
"register_alpha_routes: api_server module not in sys.modules; "
"pass require_auth/require_event_stream_auth explicitly"
)
if require_auth is None:
require_auth = host.require_auth
if require_event_stream_auth is None:
require_event_stream_auth = host.require_event_stream_auth
# -----------------------------------------------------------------------
# GET /alpha/list
# -----------------------------------------------------------------------
@app.get("/alpha/list", dependencies=[Depends(require_auth)])
async def list_alphas(
zoo: str | None = Query(None, max_length=64),
theme: str | None = Query(None, max_length=64),
universe: str | None = Query(None, max_length=64),
limit: int = Query(100, ge=1, le=1000),
) -> dict[str, Any]:
"""List alphas, optionally filtered by zoo / theme / universe."""
if zoo is not None and zoo not in _VALID_ZOOS:
raise HTTPException(
status_code=400,
detail=f"unknown zoo {zoo!r}; expected one of {sorted(_VALID_ZOOS)}",
)
if theme is not None and theme not in _VALID_THEMES:
raise HTTPException(
status_code=400,
detail=f"unknown theme {theme!r}; expected one of {sorted(_VALID_THEMES)}",
)
if universe is not None and universe not in _VALID_UNIVERSES:
raise HTTPException(
status_code=400,
detail=f"unknown universe {universe!r}; expected one of {sorted(_VALID_UNIVERSES)}",
)
from src.factors.registry import get_default_registry
registry = get_default_registry()
try:
ids = registry.list(zoo=zoo, theme=theme, universe=universe)
except Exception as exc: # noqa: BLE001
logger.exception("registry.list failed")
raise HTTPException(status_code=500, detail=_safe_error(exc))
total = len(ids)
sliced = ids[:limit]
alphas: list[dict[str, Any]] = []
for aid in sliced:
try:
a = registry.get(aid)
except KeyError:
continue
meta = a.meta or {}
alphas.append(
{
"id": a.id,
"zoo": a.zoo,
"theme": meta.get("theme", []),
"universe": meta.get("universe", []),
"nickname": meta.get("nickname"),
"decay_horizon": meta.get("decay_horizon"),
"min_warmup_bars": meta.get("min_warmup_bars"),
"requires_sector": bool(meta.get("requires_sector", False)),
}
)
return {
"status": "ok",
"alphas": alphas,
"total": total,
"returned": len(alphas),
"truncated": total > len(alphas),
}
# -----------------------------------------------------------------------
# GET /alpha/{alpha_id}
# -----------------------------------------------------------------------
@app.get("/alpha/{alpha_id}", dependencies=[Depends(require_auth)])
async def get_alpha(alpha_id: str) -> dict[str, Any]:
"""Return alpha metadata + the source code of its zoo .py file."""
if not _ALPHA_ID_RE.fullmatch(alpha_id or ""):
raise HTTPException(status_code=400, detail="invalid alpha_id")
from src.factors.registry import RegistryError, get_default_registry
registry = get_default_registry()
try:
alpha = registry.get(alpha_id)
except KeyError:
raise HTTPException(
status_code=404,
detail={"status": "error", "error": "alpha_id not found"},
)
try:
source_code = registry.get_source(alpha_id)
except RegistryError as exc:
# Source-read failure is a degraded but recoverable case — log and
# surface a short placeholder. The reason here is a typed registry
# error (size cap or OS error from a known path), safe to expose.
logger.warning("failed to read source for %s: %s", alpha_id, exc)
source_code = f"# <source unavailable: {exc}>"
return {
"status": "ok",
"alpha": {
"id": alpha.id,
"zoo": alpha.zoo,
"module_path": alpha.module_path,
"meta": alpha.meta,
},
"source_code": source_code,
}
# -----------------------------------------------------------------------
# POST /alpha/bench
# -----------------------------------------------------------------------
@app.post(
"/alpha/bench",
status_code=202,
dependencies=[Depends(require_auth)],
)
async def kick_off_bench(payload: BenchRequest) -> dict[str, Any]:
"""Queue a background bench job and return a job_id."""
# Cheap period parse pre-check so we 400 here instead of letting the
# worker fail asynchronously.
from src.tools.alpha_bench_tool import _parse_period
try:
_parse_period(payload.period)
except ValueError as exc:
raise HTTPException(status_code=400, detail=f"invalid period: {exc}")
# Concurrency cap. We peek at the semaphore counter rather than
# ``acquire(block=False)`` so the actual acquire happens inside the
# worker (after the 202 is returned). _value is a CPython
# implementation detail but it's been stable since 3.0 and asyncio's
# own ``locked()`` uses it.
sem = _get_bench_semaphore()
# ``locked()`` returns True iff the counter is 0; defensive check.
if sem.locked() or getattr(sem, "_value", MAX_CONCURRENT_BENCHES) <= 0:
raise HTTPException(
status_code=429,
detail="too many running benches; wait for one to finish",
)
_prune_old_jobs()
job_id = uuid.uuid4().hex
with _JOBS_LOCK:
ALPHA_BENCH_JOBS[job_id] = {
"job_id": job_id,
"status": "queued",
"zoo": payload.zoo,
"universe": payload.universe,
"period": payload.period,
"top": payload.top,
"created_at": _now_iso(),
"progress": {"n_done": 0, "n_total": 0, "current_alpha_id": None},
"result": None,
"error": None,
}
async def _runner() -> None:
async with sem:
try:
await asyncio.to_thread(
_run_bench_blocking,
job_id,
payload.zoo,
payload.universe,
payload.period,
payload.top,
)
except Exception: # noqa: BLE001 — never escape the loop
logger.exception("bench runner outer task crashed (job=%s)", job_id)
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is not None and job["status"] not in ("done", "error"):
job["status"] = "error"
job["error"] = "internal error; see server logs"
job["_finished_at"] = time.time()
task = asyncio.create_task(_runner())
_RUNNING_TASKS.add(task)
task.add_done_callback(_RUNNING_TASKS.discard)
return {"status": "ok", "job_id": job_id}
# -----------------------------------------------------------------------
# GET /alpha/bench/{job_id}/stream
# -----------------------------------------------------------------------
@app.get(
"/alpha/bench/{job_id}/stream",
dependencies=[Depends(require_event_stream_auth)],
)
async def stream_bench(job_id: str, request: Request) -> StreamingResponse:
"""SSE: progress / result / done / error until the job terminates."""
if not _JOB_ID_RE.fullmatch(job_id or ""):
raise HTTPException(status_code=400, detail="invalid job_id")
with _JOBS_LOCK:
if job_id not in ALPHA_BENCH_JOBS:
raise HTTPException(status_code=404, detail=f"job {job_id} not found")
async def event_stream():
last_n_done = -1
last_emit = time.monotonic()
while True:
if await request.is_disconnected():
return
with _JOBS_LOCK:
job = ALPHA_BENCH_JOBS.get(job_id)
if job is None:
yield _sse("error", {"message": "job vanished"})
yield _sse("done", {"job_id": job_id})
return
status = job["status"]
progress = dict(job.get("progress") or {})
result = job.get("result")
error = job.get("error")
# Emit progress whenever n_done advances.
n_done = int(progress.get("n_done") or 0)
if n_done != last_n_done:
yield _sse("progress", progress)
last_n_done = n_done
last_emit = time.monotonic()
if status == "done":
if result is not None:
yield _sse("result", _result_for_wire(result))
yield _sse(
"done",
{
"job_id": job_id,
"wall_seconds": (result or {}).get("wall_seconds"),
},
)
return
if status == "error":
yield _sse("error", {"message": error or "unknown error"})
yield _sse("done", {"job_id": job_id})
return
# Heartbeat comment frame every ~15s — clients ignore comment
# frames; proxies see traffic and don't kill the idle connection.
if time.monotonic() - last_emit >= _SSE_HEARTBEAT_SECONDS:
yield ": ping\n\n"
last_emit = time.monotonic()
await asyncio.sleep(_POLL_INTERVAL_SECONDS)
return StreamingResponse(
event_stream(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
# ---------------------------------------------------------------------------
# SSE helpers
# ---------------------------------------------------------------------------
def _sse(event: str, data: dict[str, Any]) -> str:
"""Format a single SSE frame."""
payload = json.dumps(data, ensure_ascii=False, default=str)
return f"event: {event}\ndata: {payload}\n\n"
def _result_for_wire(result: dict[str, Any]) -> dict[str, Any]:
"""Project the bench-runner result onto the API contract's `result` event.
Keeps: alive / reversed / dead / skipped / n_alphas_tested / top5_by_ir /
dead_examples / by_theme / meta. The frontend renders skipped count and
surfaces universe meta (e.g. survivorship_bias) so users see degraded
runs clearly.
Drops anything internal (wall_seconds is reported on the `done` event;
rows / per-alpha skipped[] are stripped server-side to keep payloads small).
"""
wire: dict[str, Any] = {}
keep = (
"alive",
"reversed",
"dead",
"top5_by_ir",
"dead_examples",
"by_theme",
"n_alphas_tested",
"meta",
)
for k in keep:
if k in result:
wire[k] = result[k]
# bench_runner reports the skip count as ``n_skipped``; the wire contract
# exposes it as ``skipped``. Surface both keys to stay backward-compatible
# with any client that started reading ``n_skipped`` early.
if "n_skipped" in result:
wire["skipped"] = result["n_skipped"]
wire["n_skipped"] = result["n_skipped"]
return wire
+17
View File
@@ -0,0 +1,17 @@
"""Agent configuration helpers for MCP client integration."""
from src.config.loader import load_agent_config, load_runtime_agent_config, merge_agent_config_overrides, sanitize_session_overrides
from src.config.paths import get_config_path, get_data_dir, get_runtime_root
from src.config.schema import AgentConfig, MCPServerConfig
__all__ = [
"AgentConfig",
"MCPServerConfig",
"get_config_path",
"get_data_dir",
"get_runtime_root",
"load_agent_config",
"load_runtime_agent_config",
"merge_agent_config_overrides",
"sanitize_session_overrides",
]
+190
View File
@@ -0,0 +1,190 @@
"""Structured agent config loading utilities."""
from __future__ import annotations
import json
import logging
import os
from pathlib import Path
from typing import Any, Mapping
from pydantic import ValidationError
from src.config.paths import get_config_path
from src.config.schema import AgentConfig, AgentConfigOverride
logger = logging.getLogger(__name__)
try:
import yaml
except ImportError:
yaml = None # type: ignore
def load_agent_config(config_path: Path | None = None) -> AgentConfig:
"""Load structured agent config from disk with safe fallback.
Args:
config_path: Optional explicit config path. When omitted, the default
config discovery path is used.
Returns:
The validated agent config. Invalid or unreadable config files fall
back to ``AgentConfig()``.
"""
path = get_config_path(config_path)
if not path.exists():
return AgentConfig()
try:
raw = _read_config_file(path)
return AgentConfig.model_validate(raw)
except (OSError, ValueError, ValidationError) as exc:
logger.warning(
"Failed to load agent config from %s: %s",
path,
type(exc).__name__,
)
logger.debug("Agent config load error details: %s", exc)
return AgentConfig()
def merge_agent_config_overrides(
config: AgentConfig,
overrides: Mapping[str, Any] | None,
) -> AgentConfig:
"""Merge runtime overrides on top of a base config.
Overrides are validated against a partial schema first so both snake_case
and camelCase keys are accepted while only explicitly provided fields
override the base config.
Args:
config: Base agent config loaded from disk or defaults.
overrides: Runtime overrides, typically from session-level config.
Returns:
A new validated config containing the merged result.
"""
if not overrides:
return config
try:
override_model = AgentConfigOverride.model_validate(dict(overrides))
except ValidationError as exc:
logger.warning(
"Ignoring invalid agent config overrides (%s): %s — using base config",
type(exc).__name__,
[str(e["loc"]) for e in exc.errors()],
)
return config
merged = _merge_dicts(
config.model_dump(mode="json"),
override_model.model_dump(mode="json", exclude_unset=True),
)
return AgentConfig.model_validate(merged)
# Keys in session overrides that carry subprocess definitions and therefore
# require operator-level trust rather than API-caller trust.
_SESSION_RESTRICTED_KEYS: frozenset[str] = frozenset({"mcpServers", "mcp_servers"})
def sanitize_session_overrides(overrides: Mapping[str, Any]) -> dict[str, Any]:
"""Strip operator-only keys from API-caller-supplied session overrides.
``mcpServers`` / ``mcp_servers`` define subprocess ``command``/``args``/``env``
and therefore grant execution-level capabilities. They must originate from
the operator-controlled config file on disk, not from unauthenticated or
semi-trusted API callers. Operators who deliberately want to allow session-
level MCP injection can set ``ALLOW_SESSION_MCP_SERVERS=1``.
Args:
overrides: Raw session config dict received from the API caller.
Returns:
A new dict with restricted keys removed (or the original mapping
converted to dict if the env opt-in is active).
"""
if os.environ.get("ALLOW_SESSION_MCP_SERVERS", "").strip().lower() in {"1", "true", "yes"}:
return dict(overrides)
restricted_present = _SESSION_RESTRICTED_KEYS & overrides.keys()
if restricted_present:
logger.warning(
"Stripped %s from session config overrides: MCP server definitions "
"require operator-level trust (disk config). "
"Set ALLOW_SESSION_MCP_SERVERS=1 to allow session-level injection.",
sorted(restricted_present),
)
return {k: v for k, v in overrides.items() if k not in _SESSION_RESTRICTED_KEYS}
def load_runtime_agent_config(
config_path: Path | None = None,
overrides: Mapping[str, Any] | None = None,
) -> AgentConfig:
"""Load disk config and apply runtime overrides.
Args:
config_path: Optional explicit config file path.
overrides: Runtime override mapping applied on top of file-based config.
Returns:
The merged runtime config.
"""
config = load_agent_config(config_path)
return merge_agent_config_overrides(config, overrides)
def _read_config_file(path: Path) -> dict[str, Any]:
"""Read a supported config file format into a dictionary.
Args:
path: Config file path to decode.
Returns:
The decoded config object as a dictionary.
Raises:
ValueError: If the file format is unsupported, YAML support is
unavailable, or the decoded payload is not an object.
"""
suffix = path.suffix.lower()
text = path.read_text(encoding="utf-8")
if suffix == ".json":
data = json.loads(text)
elif suffix in {".yaml", ".yml"}:
if yaml is None:
raise ValueError("YAML config is not available because PyYAML is missing")
data = yaml.safe_load(text) or {}
else:
raise ValueError(f"Unsupported config file format: {suffix or '<none>'}")
if not isinstance(data, dict):
raise ValueError("Agent config must decode to a JSON/YAML object")
return data
def _merge_dicts(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
"""Recursively merge two plain dictionaries.
Args:
base: Base dictionary.
override: Override dictionary applied on top of ``base``.
Returns:
A merged dictionary where nested mappings are merged recursively and
scalar values from ``override`` replace those in ``base``.
"""
merged = dict(base)
for key, value in override.items():
current = merged.get(key)
if isinstance(current, dict) and isinstance(value, dict):
merged[key] = _merge_dicts(current, value)
else:
merged[key] = value
return merged
+71
View File
@@ -0,0 +1,71 @@
"""Path helpers for agent-level structured config."""
from __future__ import annotations
from pathlib import Path
_DEFAULT_FILENAMES = ("agent.json", "agent.yaml", "agent.yml")
def get_runtime_root(config_path: Path | None = None) -> Path:
"""Return the runtime root directory for user-level agent state.
Args:
config_path: Optional explicit config file path. When provided, the
runtime root is derived from that file's parent directory.
Returns:
The directory containing the explicit structured config file when one
is provided, otherwise the default ``~/.vibe-trading`` runtime root.
"""
if config_path is not None:
return config_path.expanduser().parent
return Path.home() / ".vibe-trading"
def get_config_candidates(config_path: Path | None = None) -> list[Path]:
"""Return supported config path candidates in lookup order.
Returns:
Candidate config paths ordered by lookup priority. When an explicit
config path is provided, only that path is returned.
"""
if config_path is not None:
return [config_path.expanduser()]
root = get_runtime_root()
return [root / filename for filename in _DEFAULT_FILENAMES]
def get_config_path(config_path: Path | None = None) -> Path:
"""Return the active config file path.
Prefers the first existing candidate. If an explicit path is provided,
returns that path directly. If no candidate exists yet, returns the
recommended default JSON path.
Args:
config_path: Optional explicit config file path.
Returns:
The selected config file path for the current runtime context.
"""
candidates = get_config_candidates(config_path)
for candidate in candidates:
if candidate.exists():
return candidate
return candidates[0]
def get_data_dir(config_path: Path | None = None) -> Path:
"""Return and create the runtime data directory derived from config path.
Args:
config_path: Optional explicit config file path.
Returns:
The directory containing the active config file. The directory is
created when it does not already exist.
"""
data_dir = get_config_path(config_path).parent
data_dir.mkdir(parents=True, exist_ok=True)
return data_dir
+103
View File
@@ -0,0 +1,103 @@
"""Structured agent config schema for MCP client integration."""
from __future__ import annotations
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
def _to_camel(name: str) -> str:
"""Convert snake_case names to camelCase aliases.
Args:
name: Field name in snake_case form.
Returns:
The camelCase alias used for external config compatibility.
"""
parts = name.split("_")
return parts[0] + "".join(part.capitalize() for part in parts[1:])
class ConfigBase(BaseModel):
"""Base config model accepting both snake_case and camelCase keys."""
model_config = ConfigDict(alias_generator=_to_camel, populate_by_name=True, extra="forbid")
class MCPServerConfig(ConfigBase):
"""Single external MCP server definition."""
type: Literal["stdio", "sse", "streamableHttp"] | None = None
command: str = ""
args: list[str] = Field(default_factory=list)
env: dict[str, str] = Field(default_factory=dict)
url: str = ""
headers: dict[str, str] = Field(default_factory=dict)
tool_timeout: float = Field(default=30.0, ge=0.1)
enabled_tools: list[str] = Field(default_factory=lambda: ["*"])
@model_validator(mode="after")
def validate_v1_stdio_only(self) -> "MCPServerConfig":
"""Reject non-stdio transports for the first release.
Returns:
The validated MCP server config instance.
Raises:
ValueError: If the config implies a non-stdio transport, uses
HTTP-only fields, or omits the command required for stdio.
"""
transport = self.type
if transport is None:
if self.command:
transport = "stdio"
elif self.url:
transport = "sse" if self.url.rstrip("/").endswith("/sse") else "streamableHttp"
if transport and transport != "stdio":
raise ValueError("Only stdio MCP servers are supported in v1")
if self.url or self.headers:
raise ValueError("HTTP MCP transports are not supported in v1")
if not self.command.strip():
raise ValueError("stdio MCP servers require a command")
return self
class MCPServerConfigOverride(ConfigBase):
"""Partial MCP server override used for runtime config layering."""
type: Literal["stdio", "sse", "streamableHttp"] | None = None
command: str | None = None
args: list[str] | None = None
env: dict[str, str] | None = None
url: str | None = None
headers: dict[str, str] | None = None
tool_timeout: float | None = Field(default=None, ge=0.1)
enabled_tools: list[str] | None = None
class AgentConfig(ConfigBase):
"""Top-level structured agent config."""
mcp_servers: dict[str, MCPServerConfig] = Field(default_factory=dict)
class AgentConfigOverride(ConfigBase):
"""Partial top-level config override used for runtime layering."""
model_config = ConfigDict(
alias_generator=_to_camel,
populate_by_name=True,
# Load-bearing: SessionService passes the entire session.config dict
# (which carries unrelated keys like include_shell_tools) through
# merge_agent_config_overrides. Flipping this back to "forbid" makes
# any such payload raise ValidationError and silently drops the whole
# override, including any valid mcpServers. Regression test:
# tests/test_agent_config.py::
# test_runtime_load_preserves_mcp_servers_when_opted_in_with_unknown_keys
extra="ignore",
)
mcp_servers: dict[str, MCPServerConfigOverride] = Field(default_factory=dict)
+2
View File
@@ -73,6 +73,8 @@ _ARTIFACTS_SPEC = {
"metrics": {"schema": "metrics_csv", "path": "artifacts/metrics.csv"},
"trades": {"schema": "trade_log", "path": "artifacts/trades.csv"},
"positions": {"schema": "positions_csv", "path": "artifacts/positions.csv"},
"run_card_json": {"schema": "json", "path": "run_card.json"},
"run_card_md": {"schema": "markdown", "path": "run_card.md"},
},
}
+48
View File
@@ -0,0 +1,48 @@
"""Alpha Zoo: 4 zoos (alpha101 / gtja191 / qlib158 / academic) × ~450 alphas.
See `docs/alpha-zoo/spec.md` for the panel format contract and operator semantics.
"""
from src.factors.base import (
Alpha,
AlphaCompute,
Market,
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
vwap,
)
__all__ = [
"Alpha",
"AlphaCompute",
"Market",
"decay_linear",
"delta",
"rank",
"safe_div",
"scale",
"signed_power",
"ts_argmax",
"ts_argmin",
"ts_corr",
"ts_cov",
"ts_max",
"ts_mean",
"ts_min",
"ts_rank",
"ts_std",
"vwap",
]
+273
View File
@@ -0,0 +1,273 @@
"""Alpha Zoo base operators.
Operators all act on **wide** ``pd.DataFrame`` where ``index = trading_date``
(DatetimeIndex) and ``columns = instrument_code`` (str). The factor compute
contract returns a DataFrame of the same shape — raw scores, NaN preserved
in warmup / missing data; +/- inf is forbidden (registry rejects it).
NaN policy: every operator propagates NaN; no silent ``fillna(0)``. A constant
window for ``ts_corr`` / ``ts_cov`` returns NaN, not zero.
Lookahead ban: ``delta(df, d)`` requires ``d >= 1``; the negative-shift
``Ref(df, -n)`` form is intentionally absent.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Any, Protocol, runtime_checkable
import numpy as np
import pandas as pd
class Market(str, Enum):
"""Market identifier used by ``vwap`` for market-specific formulas."""
EQUITY_US = "equity_us"
EQUITY_CN = "equity_cn"
EQUITY_HK = "equity_hk"
CRYPTO = "crypto"
FUTURES = "futures"
@dataclass(frozen=True, slots=True)
class Alpha:
"""Lightweight handle for a registered alpha (registry-owned)."""
id: str
zoo: str
module_path: str
meta: dict[str, Any] = field(default_factory=dict)
@runtime_checkable
class AlphaCompute(Protocol):
"""Structural protocol every alpha ``.py`` module satisfies."""
def __call__(self, panel: dict[str, pd.DataFrame]) -> pd.DataFrame: ...
def _as_float(df: pd.DataFrame) -> pd.DataFrame:
if df.dtypes.eq(np.float64).all():
return df
return df.astype(np.float64)
def rank(df: pd.DataFrame) -> pd.DataFrame:
"""Cross-sectional percentile rank per row (axis=1, ties=average, pct=True).
NaN inputs stay NaN. An all-NaN row returns an all-NaN row.
"""
return df.rank(axis=1, method="average", pct=True, na_option="keep")
def scale(df: pd.DataFrame, a: float = 1.0) -> pd.DataFrame:
"""Per-row L1 normalize so sum of absolute values equals ``a``.
Rows whose abs-sum is 0 (or all-NaN) become NaN — never silent zero.
"""
df = _as_float(df)
abs_sum = df.abs().sum(axis=1, skipna=True)
abs_sum = abs_sum.where(abs_sum > 0) # zero → NaN
return df.mul(a).div(abs_sum, axis=0)
def ts_rank(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling rank (last value's rank within the n-window), per column.
Warmup (first ``n-1`` rows per column) returns NaN. Result is a percentile
in [0, 1] so it is compositionally compatible with cross-sectional rank.
"""
if n < 1:
raise ValueError(f"ts_rank window must be >= 1, got {n}")
def _last_rank(arr: np.ndarray) -> float:
if np.isnan(arr).all():
return np.nan
last = arr[-1]
if np.isnan(last):
return np.nan
valid = arr[~np.isnan(arr)]
if valid.size == 0:
return np.nan
# average rank for ties; pct
less = (valid < last).sum()
eq = (valid == last).sum()
rank_avg = less + 0.5 * (eq + 1)
return float(rank_avg / valid.size)
return df.rolling(window=n, min_periods=n).apply(_last_rank, raw=True)
def ts_corr(x: pd.DataFrame, y: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling Pearson correlation per column, min_periods=n.
Constant series in the window → NaN (no silent zero). Pairs are inner-joined
on columns; columns missing from either side become NaN.
"""
if n < 2:
raise ValueError(f"ts_corr window must be >= 2, got {n}")
x = _as_float(x)
y = _as_float(y)
cols = x.columns.union(y.columns)
xa = x.reindex(columns=cols)
ya = y.reindex(columns=cols)
corr = xa.rolling(window=n, min_periods=n).corr(ya)
# corr above can produce +/- inf when one series is constant in some
# pandas versions; force to NaN.
return corr.replace([np.inf, -np.inf], np.nan)
def ts_cov(x: pd.DataFrame, y: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling sample covariance per column, min_periods=n."""
if n < 2:
raise ValueError(f"ts_cov window must be >= 2, got {n}")
x = _as_float(x)
y = _as_float(y)
cols = x.columns.union(y.columns)
xa = x.reindex(columns=cols)
ya = y.reindex(columns=cols)
cov = xa.rolling(window=n, min_periods=n).cov(ya)
return cov.replace([np.inf, -np.inf], np.nan)
def ts_mean(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling mean per column, warmup → NaN."""
if n < 1:
raise ValueError(f"ts_mean window must be >= 1, got {n}")
return df.rolling(window=n, min_periods=n).mean()
def ts_std(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling sample std (ddof=1) per column, warmup → NaN."""
if n < 2:
raise ValueError(f"ts_std window must be >= 2, got {n}")
return df.rolling(window=n, min_periods=n).std(ddof=1)
def ts_max(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling max per column, warmup → NaN."""
if n < 1:
raise ValueError(f"ts_max window must be >= 1, got {n}")
return df.rolling(window=n, min_periods=n).max()
def ts_min(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling min per column, warmup → NaN."""
if n < 1:
raise ValueError(f"ts_min window must be >= 1, got {n}")
return df.rolling(window=n, min_periods=n).min()
def _argmax_last(arr: np.ndarray) -> float:
if np.isnan(arr).all():
return np.nan
arr_filled = np.where(np.isnan(arr), -np.inf, arr)
return float(np.argmax(arr_filled))
def _argmin_last(arr: np.ndarray) -> float:
if np.isnan(arr).all():
return np.nan
arr_filled = np.where(np.isnan(arr), np.inf, arr)
return float(np.argmin(arr_filled))
def ts_argmax(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling argmax (0-based index into the window), warmup → NaN."""
if n < 1:
raise ValueError(f"ts_argmax window must be >= 1, got {n}")
return df.rolling(window=n, min_periods=n).apply(_argmax_last, raw=True)
def ts_argmin(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling argmin (0-based index into the window), warmup → NaN."""
if n < 1:
raise ValueError(f"ts_argmin window must be >= 1, got {n}")
return df.rolling(window=n, min_periods=n).apply(_argmin_last, raw=True)
def delta(df: pd.DataFrame, d: int) -> pd.DataFrame:
"""First difference at lag ``d``: ``df - df.shift(d)``.
Lookahead ban: ``d >= 1`` strictly. Negative lag forbidden.
"""
if d < 1:
raise ValueError(f"delta lag must be >= 1 (lookahead ban), got {d}")
return df - df.shift(d)
def decay_linear(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Linear decay-weighted moving average, weights ``n, n-1, ..., 1`` normalized.
Warmup (first ``n-1`` rows) → NaN.
"""
if n < 1:
raise ValueError(f"decay_linear window must be >= 1, got {n}")
weights = np.arange(n, 0, -1, dtype=np.float64)
weights /= weights.sum()
def _apply(arr: np.ndarray) -> float:
if np.isnan(arr).any():
return np.nan
return float(np.dot(arr, weights))
return df.rolling(window=n, min_periods=n).apply(_apply, raw=True)
def signed_power(df: pd.DataFrame, p: float) -> pd.DataFrame:
"""``sign(df) * |df|**p`` — preserves sign; never produces complex output."""
arr = df.to_numpy(dtype=np.float64, na_value=np.nan)
out = np.sign(arr) * np.power(np.abs(arr), p)
return pd.DataFrame(out, index=df.index, columns=df.columns)
def safe_div(a: pd.DataFrame, b: pd.DataFrame, eps: float = 1e-12) -> pd.DataFrame:
"""Safe division: ``a / (b + eps * sign(b))``.
Where ``b == 0`` exactly (or NaN), result is NaN — never silently inf or 0.
"""
a = _as_float(a)
b = _as_float(b)
sign = np.sign(b.to_numpy(dtype=np.float64, na_value=np.nan))
# b == 0 → sign == 0 → denom == 0 → NaN below
denom_arr = b.to_numpy(dtype=np.float64, na_value=np.nan) + eps * sign
denom = pd.DataFrame(denom_arr, index=b.index, columns=b.columns)
result = a.div(denom)
return result.replace([np.inf, -np.inf], np.nan)
def vwap(panel: dict[str, pd.DataFrame], market: Market | str) -> pd.DataFrame:
"""Market-aware VWAP-equivalent reference price.
- ``equity_cn``: ``(amount * 1000) / (volume * 100 + 1)`` — Tushare's
``daily.amount`` is in **千元 (thousand CNY)** and ``daily.vol`` is in
**手 (100 shares)**. Probe 2026-05-17 against ``000001.SZ`` shows
``amount/(vol*100) ≈ 0.0093`` for a close of 9.27 — confirming the 1000x
scale. We multiply ``amount`` by 1000 (CNY) and divide by
``volume * 100`` (shares); ``+1`` keeps the denominator positive on
suspended bars.
- ``equity_us`` / ``equity_hk`` / ``futures``: typical price ``(H + L + C + O) / 4``
when ``panel["vwap"]`` is absent.
- ``crypto``: prefer ``panel["vwap"]`` if provided, else typical price.
Any missing required column → NaN propagation; never silent zero.
"""
if isinstance(market, str):
market = Market(market)
if "vwap" in panel:
return panel["vwap"]
if market is Market.EQUITY_CN:
if "amount" not in panel or "volume" not in panel:
raise KeyError("vwap(equity_cn) requires panel['amount'] and panel['volume']")
return safe_div(panel["amount"] * 1000.0, panel["volume"] * 100.0 + 1.0)
required = ("open", "high", "low", "close")
missing = [k for k in required if k not in panel]
if missing:
raise KeyError(f"vwap({market.value}) requires panel keys {required}; missing {missing}")
return (panel["open"] + panel["high"] + panel["low"] + panel["close"]) / 4.0
+227
View File
@@ -0,0 +1,227 @@
"""Bench runner: compute IC stats for every alpha in a zoo over one universe.
Extracted from ``agent/scripts/w4a_run_benches.py`` so the same pipeline can be
called by:
- the CLI bench driver (``w4a_run_benches.py``)
- the Web UI background worker (``src/api/alpha_routes.py``)
The math is unchanged — only the carrier moved. Categorisation thresholds:
- ``alive`` : ic_mean > 0.02 and ic_positive_ratio >= 0.55 and |t| > 2
- ``reversed`` : ic_mean < -0.02 and |t| > 2
- ``dead`` : everything else
"""
from __future__ import annotations
import logging
import math
import time
from typing import Any, Callable
from src.factors.factor_analysis_core import compute_ic_series
from src.factors.registry import (
Registry,
RegistryError,
SkipAlpha,
get_default_registry,
)
from src.tools.alpha_bench_tool import _compute_forward_returns, _load_universe_panel
logger = logging.getLogger(__name__)
ProgressCb = Callable[[int, int, str], None]
"""Signature: ``on_progress(n_done, n_total, current_alpha_id)``."""
def t_stat(ic_mean: float, ic_std: float, n: int) -> float:
"""Two-sided t-statistic of the IC series."""
if not (n > 0 and ic_std > 0 and math.isfinite(ic_std)):
return 0.0
return ic_mean / (ic_std / math.sqrt(n))
def categorise(row: dict[str, Any]) -> str:
"""Bucket a per-alpha row into alive / reversed / dead."""
ic_mean = row["ic_mean"]
pos = row["ic_positive_ratio"]
t = t_stat(ic_mean, row["ic_std"], row["ic_count"])
if ic_mean > 0.02 and pos >= 0.55 and abs(t) > 2:
return "alive"
if ic_mean < -0.02 and abs(t) > 2:
return "reversed"
return "dead"
def theme_breakdown(rows: list[dict[str, Any]]) -> dict[str, dict[str, int]]:
"""Aggregate alive/reversed/dead counts by theme tag."""
by_theme: dict[str, dict[str, int]] = {}
for row in rows:
cat = row["_category"]
themes = row.get("theme", []) or ["uncategorised"]
for theme in themes:
bucket = by_theme.setdefault(
theme, {"alive": 0, "reversed": 0, "dead": 0, "count": 0}
)
bucket[cat] += 1
bucket["count"] += 1
return by_theme
def run_bench(
zoo: str,
universe: str,
period: str,
top: int = 20,
on_progress: ProgressCb | None = None,
registry: Registry | None = None,
) -> dict[str, Any]:
"""Run a bench end-to-end and return the API-shaped summary.
Args:
zoo: Zoo id (e.g. ``gtja191``, ``alpha101``, ``qlib158``).
universe: Universe key understood by ``_load_universe_panel``
(e.g. ``csi300``, ``sp500``, ``btc-usdt``).
period: ``YYYY-YYYY`` or ``YYYY-MM-DD/YYYY-MM-DD`` window.
top: Number of top-IR alphas to keep in ``top5_by_ir`` and
``dead_examples`` (capped at ``top``).
on_progress: Optional callback fired after every alpha completes
(success or skip). Signature: ``(n_done, n_total, alpha_id)``.
registry: Optional pre-built registry (test injection); defaults to a
fresh ``Registry()``.
Returns:
Dict with keys: ``status``, ``zoo``, ``universe``, ``period``,
``n_alphas_tested``, ``n_skipped``, ``alive``, ``reversed``, ``dead``,
``by_theme``, ``top5_by_ir``, ``dead_examples``, ``wall_seconds`` —
and on failure: ``status="error"``, ``error``.
"""
start = time.monotonic()
entry: dict[str, Any] = {
"status": "pending",
"zoo": zoo,
"universe": universe,
"period": period,
}
reg = registry if registry is not None else get_default_registry()
alpha_ids = reg.list(zoo=zoo)
if not alpha_ids:
entry["status"] = "error"
entry["error"] = f"no alphas registered under zoo={zoo!r}"
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
try:
panel = _load_universe_panel(universe, period)
except (ValueError, NotImplementedError, RuntimeError) as exc:
entry["status"] = "error"
entry["error"] = f"universe load failed: {exc}"
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
try:
return_df = _compute_forward_returns(panel)
except Exception as exc: # noqa: BLE001
entry["status"] = "error"
entry["error"] = f"forward returns failed: {exc}"
entry["wall_seconds"] = round(time.monotonic() - start, 2)
return entry
rows: list[dict[str, Any]] = []
skipped: list[dict[str, Any]] = []
n_total = len(alpha_ids)
for idx, aid in enumerate(alpha_ids, start=1):
try:
factor_df = reg.compute(aid, panel)
ic = compute_ic_series(factor_df, return_df)
if ic.empty:
skipped.append(
{"id": aid, "reason": "empty IC series", "kind": "typed"}
)
else:
ic_mean = float(ic.mean())
ic_std = float(ic.std())
ir = ic_mean / ic_std if ic_std > 0 else 0.0
meta = reg.get(aid).meta or {}
rows.append(
{
"id": aid,
"ic_mean": round(ic_mean, 6),
"ic_std": round(ic_std, 6),
"ir": round(ir, 4),
"ic_positive_ratio": round(float((ic > 0).mean()), 4),
"ic_count": int(len(ic)),
"theme": meta.get("theme", []),
"formula_latex": meta.get("formula_latex", ""),
}
)
except (SkipAlpha, RegistryError, RuntimeError, KeyError, ValueError) as exc:
skipped.append({"id": aid, "reason": str(exc), "kind": "typed"})
except Exception as exc: # noqa: BLE001
logger.exception("bench: unexpected failure on %s", aid)
skipped.append(
{"id": aid, "reason": f"unexpected: {exc}", "kind": "unexpected"}
)
if on_progress is not None:
try:
on_progress(idx, n_total, aid)
except Exception: # noqa: BLE001 — never let progress break the loop
logger.exception("on_progress callback raised; ignoring")
for row in rows:
row["_category"] = categorise(row)
counts = {"alive": 0, "reversed": 0, "dead": 0}
for row in rows:
counts[row["_category"]] += 1
rows_by_ir = sorted(rows, key=lambda r: r["ir"], reverse=True)
rows_by_ic = sorted(rows, key=lambda r: r["ic_mean"])
def _slim(r: dict[str, Any]) -> dict[str, Any]:
return {
"id": r["id"],
"ic_mean": r["ic_mean"],
"ir": r["ir"],
"theme": r["theme"],
"formula_latex": r["formula_latex"],
"category": r["_category"],
}
# Forward universe metadata (e.g. survivorship_bias flag from sp500
# loader). Panel meta lives in panel["_meta"] when the loader chooses to
# set it; absence is fine.
universe_meta_raw = panel.get("_meta") if isinstance(panel, dict) else None
universe_meta: dict[str, Any] = {}
if universe_meta_raw is not None:
try:
# Could be a pandas object or a plain dict.
if hasattr(universe_meta_raw, "to_dict"):
universe_meta = dict(universe_meta_raw.to_dict())
else:
universe_meta = dict(universe_meta_raw)
except Exception: # noqa: BLE001
universe_meta = {"raw": str(universe_meta_raw)}
entry.update(
{
"status": "ok",
"n_alphas_tested": len(rows),
"n_skipped": len(skipped),
"alive": counts["alive"],
"reversed": counts["reversed"],
"dead": counts["dead"],
"by_theme": theme_breakdown(rows),
"top5_by_ir": [_slim(r) for r in rows_by_ir[: min(5, top)]],
"dead_examples": [_slim(r) for r in rows_by_ic[:5]],
"rows": rows,
"skipped": skipped,
"meta": universe_meta,
"wall_seconds": round(time.monotonic() - start, 2),
}
)
return entry
+909
View File
@@ -0,0 +1,909 @@
"""CLI handlers for ``vibe-trading alpha {list,show,bench,compare,export-manifest}``.
All logic lives here; ``agent/cli.py`` only wires this in via :func:`add_subparser`
and :func:`dispatch`. Handlers print to stdout (Rich when available, plain
``print`` fallback) and return an int exit code. Errors are reported as a
one-line stderr message; tracebacks are suppressed unless ``--verbose`` is set
on the namespace.
Security gates:
* ``alpha show <id>`` validates ``id`` against ``Registry().list()`` before
touching the filesystem.
* ``alpha export-manifest --out PATH`` refuses to write outside the repo
root (``Path(__file__).resolve().parents[3]``) unless ``--force``.
"""
from __future__ import annotations
import argparse
import json
import sys
import time
import traceback
import warnings
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable
# Silence the noisy ConstantInputWarning that scipy emits from spearmanr when
# the IC slice has zero variance (very common in alpha bench loops).
try:
from scipy.stats import ConstantInputWarning # type: ignore[attr-defined]
warnings.filterwarnings("ignore", category=ConstantInputWarning)
except Exception: # pragma: no cover — older scipy / unexpected layout
try:
from scipy.stats._warnings_errors import ( # type: ignore[attr-defined]
ConstantInputWarning,
)
warnings.filterwarnings("ignore", category=ConstantInputWarning)
except Exception:
pass
try:
from rich.console import Console
from rich.progress import (
BarColumn,
Progress,
SpinnerColumn,
TextColumn,
TimeElapsedColumn,
)
from rich.syntax import Syntax
from rich.table import Table
_console: Console | None = Console()
except Exception: # pragma: no cover — rich is a project dep, fallback only
_console = None
Syntax = None # type: ignore[assignment]
Table = None # type: ignore[assignment]
Progress = None # type: ignore[assignment]
BarColumn = None # type: ignore[assignment]
SpinnerColumn = None # type: ignore[assignment]
TextColumn = None # type: ignore[assignment]
TimeElapsedColumn = None # type: ignore[assignment]
from src.factors.registry import Registry, RegistryError
# Resolve repo root once: this file lives at
# ``<repo>/agent/src/factors/cli_handlers.py``; parents[3] is the repo root.
# We treat ``<repo>`` (and any subdirectory thereof) as the write-allow root.
_REPO_ROOT = Path(__file__).resolve().parents[3]
_UNIVERSE_CHOICES = ["csi300", "sp500", "btc-usdt"]
def _print(msg: str) -> None:
if _console is not None:
_console.print(msg)
else:
print(msg)
def _err(msg: str) -> None:
print(msg, file=sys.stderr)
def _stdout_is_tty() -> bool:
try:
return sys.stdout.isatty()
except Exception: # noqa: BLE001
return False
_stderr_console: Console | None = None
if _console is not None:
try:
_stderr_console = Console(stderr=True)
except Exception: # noqa: BLE001
_stderr_console = None
def _hint(msg: str) -> None:
"""Print a next-step hint to stderr when stdout is a TTY.
Goes to stderr so it does not pollute JSON / pipe-friendly stdout output.
"""
if not _stdout_is_tty():
return
if _stderr_console is not None:
_stderr_console.print(f"[dim]{msg}[/dim]")
else:
print(msg, file=sys.stderr)
def _handle_exception(args: argparse.Namespace, prefix: str, exc: BaseException) -> int:
"""Print one-line error; emit traceback only when ``--verbose`` is set."""
_err(f"{prefix}: {exc}")
if getattr(args, "verbose", False):
traceback.print_exception(type(exc), exc, exc.__traceback__)
# Surface a helpful banner for the common TUSHARE_TOKEN error case.
if "TUSHARE_TOKEN" in str(exc):
_err("")
_err("How to fix:")
_err(" 1. Register for a free token at https://tushare.pro/register")
_err(" 2. Add 'TUSHARE_TOKEN=<your_token>' to agent/.env (or ~/.vibe-trading/.env)")
_err(" 3. Re-run this command")
return 1
# --------------------------------------------------------------------------- #
# Handlers #
# --------------------------------------------------------------------------- #
def cmd_alpha_list(args: argparse.Namespace) -> int:
"""``vibe-trading alpha list [--zoo X] [--theme Y] [--universe Z]``.
Supports ``--limit N`` (default 50) and ``--json`` for machine-readable
output. ``--include-load-errors`` (alias: ``--show-failed``) appends a
registry health block.
"""
try:
reg = Registry()
ids = reg.list(zoo=args.zoo, theme=args.theme, universe=args.universe)
limit = getattr(args, "limit", None)
total = len(ids)
truncated = False
# limit > 0 => cap; limit == 0 / None => no cap.
if isinstance(limit, int) and limit > 0 and total > limit:
ids = ids[:limit]
truncated = True
# JSON mode: machine-readable, no Rich, no hints.
if getattr(args, "json", False):
records: list[dict[str, Any]] = []
for alpha_id in ids:
alpha = reg.get(alpha_id)
meta = alpha.meta or {}
records.append(
{
"id": alpha.id,
"zoo": alpha.zoo,
"theme": meta.get("theme", []) or [],
"universe": meta.get("universe", []) or [],
"decay_horizon": meta.get("decay_horizon"),
}
)
print(json.dumps(records, indent=2, default=str))
if _include_load_errors(args):
_print_load_errors(reg)
return 0
if not ids:
_print("[dim]no alphas matched the filters[/dim]" if _console else "no alphas matched the filters")
elif _console is not None and Table is not None:
title = f"Alpha Zoo ({len(ids)} of {total} alphas)" if truncated else f"Alpha Zoo ({total} alphas)"
table = Table(title=title)
table.add_column("id", style="cyan", no_wrap=True)
table.add_column("zoo", style="magenta")
table.add_column("theme")
table.add_column("universe")
table.add_column("nickname")
for alpha_id in ids:
alpha = reg.get(alpha_id)
meta = alpha.meta
table.add_row(
alpha.id,
alpha.zoo,
", ".join(meta.get("theme", []) or []),
", ".join(meta.get("universe", []) or []),
meta.get("nickname") or "",
)
_console.print(table)
if truncated:
_print(f"[dim]Showing {len(ids)} of {total}. Pass --limit N (or --limit 0 for no cap).[/dim]")
else:
for alpha_id in ids:
alpha = reg.get(alpha_id)
meta = alpha.meta
print(
f"{alpha.id}\t{alpha.zoo}\t"
f"{','.join(meta.get('theme', []) or [])}\t"
f"{','.join(meta.get('universe', []) or [])}\t"
f"{meta.get('nickname') or ''}"
)
if truncated:
print(f"[showing {len(ids)} of {total}]")
if _include_load_errors(args):
_print_load_errors(reg)
if ids:
example_id = ids[0]
example_zoo = reg.get(example_id).zoo
_hint(
f"Next: vibe-trading alpha show {example_id} | "
f"Bench a zoo: vibe-trading alpha bench --zoo {example_zoo} "
f"--universe csi300 --period 2020-2025"
)
return 0
except Exception as exc: # noqa: BLE001
return _handle_exception(args, "alpha list failed", exc)
def _include_load_errors(args: argparse.Namespace) -> bool:
"""Honour both the new flag name and the deprecated alias."""
return bool(getattr(args, "include_load_errors", False) or getattr(args, "show_failed", False))
def _print_load_errors(reg: Registry) -> None:
health = reg.health()
errors = health.get("errors", [])
if not errors:
_print("[green]no load errors[/green]" if _console else "no load errors")
return
_print(
f"[yellow]{len(errors)} load error(s):[/yellow]"
if _console
else f"{len(errors)} load error(s):"
)
for entry in errors:
_print(f" - {entry.get('alpha_id')}: {entry.get('reason')}")
def cmd_alpha_show(args: argparse.Namespace) -> int:
"""``vibe-trading alpha show <alpha_id> [--brief]`` — metadata + source."""
try:
reg = Registry()
if args.alpha_id not in reg.list():
_err(f"alpha id not found: {args.alpha_id}")
return 1
alpha = reg.get(args.alpha_id)
meta = alpha.meta
brief = bool(getattr(args, "brief", False))
_print(f"[bold cyan]id[/bold cyan]: {alpha.id}" if _console else f"id: {alpha.id}")
if not brief:
_print(f"[bold]zoo[/bold]: {alpha.zoo}" if _console else f"zoo: {alpha.zoo}")
_print(
f"[bold]module_path[/bold]: {alpha.module_path}"
if _console
else f"module_path: {alpha.module_path}"
)
_print(
f"[bold]nickname[/bold]: {meta.get('nickname') or '-'}"
if _console
else f"nickname: {meta.get('nickname') or '-'}"
)
_print(
f"[bold]theme[/bold]: {', '.join(meta.get('theme', []) or [])}"
if _console
else f"theme: {', '.join(meta.get('theme', []) or [])}"
)
_print(
f"[bold]universe[/bold]: {', '.join(meta.get('universe', []) or [])}"
if _console
else f"universe: {', '.join(meta.get('universe', []) or [])}"
)
if not brief:
_print(
f"[bold]columns_required[/bold]: {', '.join(meta.get('columns_required', []) or [])}"
if _console
else f"columns_required: {', '.join(meta.get('columns_required', []) or [])}"
)
_print(
f"[bold]decay_horizon[/bold]: {meta.get('decay_horizon')}"
if _console
else f"decay_horizon: {meta.get('decay_horizon')}"
)
_print(
f"[bold]formula_latex[/bold]: {meta.get('formula_latex', '')}"
if _console
else f"formula_latex: {meta.get('formula_latex', '')}"
)
notes = meta.get("notes") or ""
if notes:
_print(f"[bold]notes[/bold]: {notes}" if _console else f"notes: {notes}")
if brief:
_hint(
f"Next: vibe-trading alpha bench --zoo {alpha.zoo} "
f"--universe csi300 --period 2020-2025 --top 20"
)
return 0
# Source code — read via the registry's _py_paths cache (semi-private but
# accepted in the parcel contract; falls back to importlib.getsource).
py_path: Path | None = reg._py_paths.get(alpha.id) # noqa: SLF001
source: str | None = None
if py_path is not None and py_path.is_file():
try:
source = py_path.read_text(encoding="utf-8")
except OSError as exc:
_err(f"warning: could not read source file {py_path}: {exc}")
if source is None:
try:
import importlib
import inspect
module = importlib.import_module(alpha.module_path)
source = inspect.getsource(module)
except Exception as exc: # noqa: BLE001
_err(f"warning: could not load source via importlib: {exc}")
source = None
if source:
_print("[bold]source[/bold]:" if _console else "source:")
if _console is not None and Syntax is not None:
try:
_console.print(
Syntax(
source,
"python",
line_numbers=True,
theme="monokai",
word_wrap=True,
),
soft_wrap=True,
)
except Exception: # noqa: BLE001 — fall back to raw print
print(source)
else:
print(source)
_hint(
f"Next: vibe-trading alpha bench --zoo {alpha.zoo} "
f"--universe csi300 --period 2020-2025 --top 20"
)
return 0
except Exception as exc: # noqa: BLE001
return _handle_exception(args, "alpha show failed", exc)
def _confirm_bench_all(n_total: int) -> bool:
"""Interactive y/N prompt before benching every alpha."""
msg = f"About to bench {n_total} alphas (~10-15 min). Continue? [y/N] "
try:
reply = input(msg).strip().lower()
except EOFError:
return False
return reply in {"y", "yes"}
def _zoo_for_id(reg: Registry, alpha_id: str | None) -> str:
"""Lookup an alpha's zoo (best-effort, returns '' on failure)."""
if not alpha_id:
return ""
try:
return reg.get(alpha_id).zoo
except Exception: # noqa: BLE001
return ""
def _run_single_zoo_with_progress(
*,
zoo: str,
universe: str,
period: str,
top: int,
n_target: int,
reg: Registry,
run_bench: Callable[..., dict[str, Any]],
) -> dict[str, Any]:
"""Run :func:`bench_runner.run_bench` for one zoo with a Rich progress bar."""
if _console is None or Progress is None:
return run_bench(
zoo=zoo,
universe=universe,
period=period,
top=top,
registry=reg,
)
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("{task.completed}/{task.total}"),
TextColumn("[cyan]{task.fields[current]}[/cyan]"),
TimeElapsedColumn(),
console=_console,
transient=False,
) as progress:
task_id = progress.add_task(
description="Benching",
total=n_target,
current="loading universe...",
)
def on_progress(n_done: int, n_total: int, aid: str) -> None:
progress.update(task_id, completed=n_done, total=n_total, current=aid)
return run_bench(
zoo=zoo,
universe=universe,
period=period,
top=top,
on_progress=on_progress,
registry=reg,
)
def _run_all_zoos_with_progress(
*,
target_ids: list[str],
universe: str,
period: str,
top: int,
start_ts: float,
reg: Registry,
run_bench: Callable[..., dict[str, Any]],
) -> dict[str, Any]:
"""Aggregate across every zoo for the ``--yes`` / no-``--zoo`` case."""
zoos = sorted({reg.get(aid).zoo for aid in target_ids})
total_alphas = sum(len(reg.list(zoo=z)) for z in zoos)
all_rows: list[dict[str, Any]] = []
all_skipped: list[dict[str, Any]] = []
def _run_zoo(z: str, base: int, update: Callable[[int, str], None] | None) -> None:
cb: Callable[[int, int, str], None] | None = None
if update is not None:
def cb(n_done: int, _n_total: int, aid: str) -> None:
update(base + n_done, f"{z}:{aid}")
sub = run_bench(
zoo=z,
universe=universe,
period=period,
top=top,
on_progress=cb,
registry=reg,
)
if sub.get("status") == "ok":
all_rows.extend(sub.get("rows", []) or [])
all_skipped.extend(sub.get("skipped", []) or [])
else:
all_skipped.append({"id": f"<zoo:{z}>", "reason": sub.get("error", "unknown")})
if _console is not None and Progress is not None:
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("{task.completed}/{task.total}"),
TextColumn("[cyan]{task.fields[current]}[/cyan]"),
TimeElapsedColumn(),
console=_console,
transient=False,
) as progress:
task_id = progress.add_task(
description="Benching (all zoos)",
total=total_alphas,
current="starting...",
)
def update(done: int, current: str) -> None:
progress.update(task_id, completed=done, current=current)
seen = 0
for z in zoos:
_run_zoo(z, seen, update)
seen += len(reg.list(zoo=z))
else:
for z in zoos:
_run_zoo(z, 0, None)
return {
"status": "ok" if all_rows else "error",
"zoo": "all",
"universe": universe,
"period": period,
"rows": all_rows,
"skipped": all_skipped,
"n_alphas_tested": len(all_rows),
"n_skipped": len(all_skipped),
"wall_seconds": round(time.monotonic() - start_ts, 2),
**({"error": "no alphas produced IC across any zoo"} if not all_rows else {}),
}
def cmd_alpha_bench(args: argparse.Namespace) -> int:
"""``vibe-trading alpha bench --zoo X --universe csi300 --period Y-Z [--top N]``.
Calls :func:`src.factors.bench_runner.run_bench` for the IC loop so we can
stream Rich progress, then renders the HTML report via helpers imported
from ``src.tools.alpha_bench_tool`` (we do NOT modify that module).
"""
try:
try:
from src.factors.bench_runner import run_bench
except ImportError as exc:
_err(f"alpha bench failed: bench_runner unavailable ({exc})")
return 1
# --- 1. Decide which zoo to run (handle no-args footgun) ----------- #
zoo = args.zoo
if zoo is None:
reg = Registry()
n_total = len(reg.list())
if getattr(args, "yes", False):
_print(
f"[yellow]No --zoo passed; benching ALL {n_total} alphas "
f"(confirmed via --yes).[/yellow]"
if _console
else f"No --zoo passed; benching ALL {n_total} alphas (--yes)."
)
else:
_print(
f"[yellow]Warning:[/yellow] no --zoo passed; this would bench "
f"every registered alpha ({n_total} total)."
if _console
else f"Warning: no --zoo passed; this would bench every alpha ({n_total})."
)
if not _confirm_bench_all(n_total):
_err("alpha bench: aborted (pass --zoo <id> or --yes to confirm).")
return 1
# --- 2. Pre-flight banner: counts + ETA ---------------------------- #
reg = Registry()
if zoo is not None:
target_ids = reg.list(zoo=zoo)
else:
target_ids = reg.list()
n_target = len(target_ids)
if n_target == 0:
_err(f"alpha bench failed: no alphas registered under zoo={zoo!r}")
return 1
_print(
f"[bold]Bench:[/bold] {n_target} alphas x {args.universe} x {args.period}"
if _console
else f"Bench: {n_target} alphas x {args.universe} x {args.period}"
)
_print(
"[dim]ETA: ~3-5 min (cache hit) / ~10-20 min (cold fetch)[/dim]"
if _console
else "ETA: ~3-5 min (cache hit) / ~10-20 min (cold fetch)"
)
# --- 3. Run the bench loop with a live progress bar --------------- #
start_ts = time.monotonic()
result: dict[str, Any]
if zoo is not None:
# Single-zoo path: one call to bench_runner.run_bench.
result = _run_single_zoo_with_progress(
zoo=zoo,
universe=args.universe,
period=args.period,
top=args.top,
n_target=n_target,
reg=reg,
run_bench=run_bench,
)
else:
# Multi-zoo path: aggregate across every zoo (bench_runner requires
# a non-empty zoo, so we loop per zoo and merge).
result = _run_all_zoos_with_progress(
target_ids=target_ids,
universe=args.universe,
period=args.period,
top=args.top,
start_ts=start_ts,
reg=reg,
run_bench=run_bench,
)
# --- 5. Handle bench-loop errors / propagate exit code ------------- #
status = result.get("status")
if status != "ok":
err_msg = result.get("error", "unknown error")
envelope = {
"status": "error",
"error": err_msg,
"zoo": zoo,
"universe": args.universe,
"period": args.period,
}
print(json.dumps(envelope, indent=2, default=str))
_err(f"alpha bench failed: {err_msg}")
if "TUSHARE_TOKEN" in str(err_msg):
_err("")
_err("How to fix:")
_err(" 1. Register for a free token at https://tushare.pro/register")
_err(" 2. Add 'TUSHARE_TOKEN=<your_token>' to agent/.env (or ~/.vibe-trading/.env)")
_err(" 3. Re-run this command")
return 1
# --- 6. Render HTML report (delegating to alpha_bench_tool helpers) -#
rows = result.get("rows", []) or []
skipped = result.get("skipped", []) or []
top_n = max(1, int(args.top))
rows_sorted = sorted(rows, key=lambda r: r.get("ir", 0.0), reverse=True)
top_rows_raw = rows_sorted[:top_n]
# Normalise rows + skipped for the report template (and for the JSON
# envelope so consumers don't see the internal _category key).
def _normalise_row(r: dict[str, Any]) -> dict[str, Any]:
return {
"id": r.get("id"),
"zoo": r.get("zoo") or _zoo_for_id(reg, r.get("id")),
"theme": r.get("theme") or [],
"formula_latex": r.get("formula_latex", ""),
"ic_mean": r.get("ic_mean", 0.0),
"ic_std": r.get("ic_std", 0.0),
"ir": r.get("ir", 0.0),
"ic_positive_ratio": r.get("ic_positive_ratio", 0.0),
"ic_count": r.get("ic_count", 0),
"category": r.get("_category") or r.get("category"),
}
def _normalise_skipped(s: dict[str, Any]) -> dict[str, Any]:
return {"alpha_id": s.get("id") or s.get("alpha_id"), "reason": s.get("reason", "")}
top_rows = [_normalise_row(r) for r in top_rows_raw]
failures_for_report = [_normalise_skipped(s) for s in skipped[:10]]
report_path: Path | None = None
try:
from src.tools.alpha_bench_tool import ( # type: ignore[import-not-found]
_CSP,
_REPORT_CSS,
_default_output_dir,
_render_html,
)
output_dir = _default_output_dir()
output_dir.mkdir(parents=True, exist_ok=True)
ts = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
report_path = output_dir / f"alpha_bench_{ts}.html"
context = {
"csp": _CSP,
"css": _REPORT_CSS,
"generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
"universe": args.universe,
"period": args.period,
"n_alphas_tested": len(rows),
"n_skipped": len(skipped),
"top": top_rows,
"failures": failures_for_report,
}
report_path.write_text(_render_html(context), encoding="utf-8")
except Exception as exc: # noqa: BLE001 — report is nice-to-have
_err(f"warning: could not write HTML report: {exc}")
report_path = None
envelope: dict[str, Any] = {
"status": "ok",
"zoo": zoo,
"universe": args.universe,
"period": args.period,
"n_alphas_tested": len(rows),
"n_skipped": len(skipped),
"top": top_rows,
"wall_seconds": result.get("wall_seconds"),
}
if report_path is not None:
envelope["report_path"] = str(report_path)
print(json.dumps(envelope, indent=2, default=str))
# Final stderr success line (M4).
tested = len(rows)
skipped_n = len(skipped)
summary = f"✓ Bench complete. {tested} tested, {skipped_n} skipped."
if report_path is not None:
summary += f" Report: {report_path}"
if _stderr_console is not None:
_stderr_console.print(summary, style="green")
else:
print(summary, file=sys.stderr)
return 0
except Exception as exc: # noqa: BLE001
return _handle_exception(args, "alpha bench failed", exc)
def cmd_alpha_compare(args: argparse.Namespace) -> int:
"""``vibe-trading alpha compare <id1> [<id2> ...] | --all | --zoo X``.
Stub for W4; reports validated targets so callers can wire scripts.
"""
try:
targets: list[str]
reg = Registry()
if getattr(args, "compare_all", False):
targets = reg.list()
elif args.zoo:
targets = reg.list(zoo=args.zoo)
else:
targets = list(args.alpha_ids or [])
if not targets:
_err(
"alpha compare: no targets supplied "
"(pass alpha ids, or --all for every alpha, or --zoo X to filter)"
)
return 1
_print(f"compare targets ({len(targets)}): {', '.join(targets)}")
_print(
"[yellow]alpha compare not yet implemented (W4)[/yellow]"
if _console
else "alpha compare not yet implemented (W4)"
)
print(json.dumps({"status": "stub", "targets": targets, "reason": "W4 not implemented"}, indent=2))
return 0
except Exception as exc: # noqa: BLE001
return _handle_exception(args, "alpha compare failed", exc)
def cmd_alpha_export_manifest(args: argparse.Namespace) -> int:
"""``vibe-trading alpha export-manifest --out PATH [--force]``."""
try:
out_path = Path(args.out).resolve()
# Refuse to write outside the repo root unless --force.
try:
out_path.relative_to(_REPO_ROOT)
inside_repo = True
except ValueError:
inside_repo = False
if not inside_repo and not args.force:
_err(
f"alpha export-manifest: refusing to write outside repo root "
f"({_REPO_ROOT}); pass --force to override. target={out_path}"
)
return 1
parent = out_path.parent
if not parent.exists():
parent.mkdir(parents=True, exist_ok=True)
reg = Registry()
manifest = reg.export_manifest()
out_path.write_text(json.dumps(manifest, indent=2, default=str), encoding="utf-8")
_print(
f"[green]wrote manifest[/green] ({manifest['health']['loaded']} alphas, "
f"{manifest['health']['failed']} errors) -> {out_path}"
if _console
else f"wrote manifest ({manifest['health']['loaded']} alphas, "
f"{manifest['health']['failed']} errors) -> {out_path}"
)
return 0
except (RegistryError, OSError) as exc:
return _handle_exception(args, "alpha export-manifest failed", exc)
except Exception as exc: # noqa: BLE001
return _handle_exception(args, "alpha export-manifest failed", exc)
# --------------------------------------------------------------------------- #
# Subparser + dispatch wiring #
# --------------------------------------------------------------------------- #
_DISPATCH: dict[str, Callable[[argparse.Namespace], int]] = {
"list": cmd_alpha_list,
"show": cmd_alpha_show,
"bench": cmd_alpha_bench,
"compare": cmd_alpha_compare,
"export-manifest": cmd_alpha_export_manifest,
}
# Module-level handle to the alpha parser so ``dispatch`` can print full help
# when no subcommand is supplied (M6).
_ALPHA_PARSER: argparse.ArgumentParser | None = None
def add_subparser(subparsers: Any) -> argparse.ArgumentParser:
"""Register ``alpha`` and its five sub-sub-commands on the given subparsers.
Args:
subparsers: The object returned by ``ArgumentParser.add_subparsers(...)``.
Returns:
The ``alpha`` parser (mostly for test introspection).
"""
global _ALPHA_PARSER
alpha_parser = subparsers.add_parser(
"alpha", help="Alpha Zoo: list / show / bench / compare / export-manifest"
)
alpha_parser.add_argument(
"--verbose", action="store_true", help="Show full traceback on errors"
)
alpha_sub = alpha_parser.add_subparsers(dest="alpha_command")
p_list = alpha_sub.add_parser("list", help="List registered alphas")
p_list.add_argument("--zoo", default=None, help="Filter by zoo (e.g. alpha101)")
p_list.add_argument("--theme", default=None, help="Filter by theme (e.g. momentum)")
p_list.add_argument(
"--universe",
default=None,
choices=_UNIVERSE_CHOICES,
help=f"Filter by universe ({', '.join(_UNIVERSE_CHOICES)})",
)
p_list.add_argument(
"--limit",
type=int,
default=50,
help="Max alphas to print (default: 50; pass 0 for no cap)",
)
p_list.add_argument(
"--json",
action="store_true",
help="Emit JSON array of {id, zoo, theme, universe, decay_horizon} instead of a table",
)
p_list.add_argument(
"--include-load-errors",
dest="include_load_errors",
action="store_true",
help="Also print load errors from registry.health()",
)
p_list.add_argument(
"--show-failed",
dest="show_failed",
action="store_true",
help="(deprecated) alias for --include-load-errors",
)
p_show = alpha_sub.add_parser("show", help="Show alpha metadata and source code")
p_show.add_argument("alpha_id", help="Alpha id, e.g. alpha101_001")
p_show.add_argument(
"--brief",
action="store_true",
help="Print only id/theme/universe/formula_latex/notes — omit source code",
)
p_bench = alpha_sub.add_parser("bench", help="Benchmark alphas in a zoo")
p_bench.add_argument("--zoo", default=None, help="Zoo to benchmark (required unless --yes is passed)")
p_bench.add_argument(
"--universe",
default="csi300",
choices=_UNIVERSE_CHOICES,
help=f"Universe (default: csi300; one of {', '.join(_UNIVERSE_CHOICES)})",
)
p_bench.add_argument(
"--period",
default="2020-2025",
help="Period spec: YYYY-YYYY or YYYY-MM-DD/YYYY-MM-DD (e.g. 2020-2025)",
)
p_bench.add_argument("--top", type=int, default=20, help="Top-N alphas to keep (default: 20)")
p_bench.add_argument(
"--yes",
action="store_true",
help="Skip the 'bench every alpha?' prompt when --zoo is omitted",
)
p_compare = alpha_sub.add_parser("compare", help="Compare alphas (W4 stub)")
p_compare.add_argument("alpha_ids", nargs="*", help="Alpha ids to compare")
p_compare.add_argument(
"--all",
dest="compare_all",
action="store_true",
help="Compare every alpha in the registry",
)
p_compare.add_argument(
"--zoo",
default=None,
help="Filter comparison to one zoo (e.g. alpha101)",
)
p_export = alpha_sub.add_parser("export-manifest", help="Export registry manifest as JSON")
p_export.add_argument("--out", required=True, help="Output JSON path")
p_export.add_argument("--force", action="store_true", help="Allow writing outside the repo root")
_ALPHA_PARSER = alpha_parser
return alpha_parser
def dispatch(args: argparse.Namespace) -> int:
"""Dispatch ``alpha <sub>`` to the matching handler.
Returns the exit code; ``cli.py`` propagates it via ``_coerce_exit_code``.
"""
sub = getattr(args, "alpha_command", None)
if sub is None:
if _ALPHA_PARSER is not None:
_ALPHA_PARSER.print_help()
else:
_err("alpha requires a subcommand. Try: vibe-trading alpha list")
return 1
handler = _DISPATCH.get(sub)
if handler is None:
_err(f"alpha: unknown subcommand {sub!r}")
return 1
return handler(args)
+127
View File
@@ -0,0 +1,127 @@
"""Pure IC/IR + layered backtest math shared by factor_analysis_tool and alpha_bench_tool."""
import numpy as np
import pandas as pd
from scipy.stats import spearmanr
_MIN_VALID_PER_DATE = 5
def compute_ic_series(factor_df: pd.DataFrame, return_df: pd.DataFrame) -> pd.Series:
"""Compute daily Spearman rank correlation (IC) between factor values and returns.
Vectorised via ``rank().corrwith()`` (Spearman = Pearson on ranks). A date
is dropped if fewer than 5 instruments have both a factor and return value
on that bar — matching the prior per-row guard.
Args:
factor_df: Factor values; index=date, columns=codes.
return_df: Returns; index=date, columns=codes.
Returns:
IC series indexed by date.
"""
common_dates = factor_df.index.intersection(return_df.index)
common_codes = factor_df.columns.intersection(return_df.columns)
if len(common_dates) == 0 or len(common_codes) == 0:
return pd.Series(dtype=float)
factor_df = factor_df.loc[common_dates, common_codes]
return_df = return_df.loc[common_dates, common_codes]
# Only rank cells where both factor and return are present (mirrors the
# per-date ``shared = f.dropna() ∩ r.dropna()`` from the loop version).
pair_mask = factor_df.notna() & return_df.notna()
n_valid = pair_mask.sum(axis=1)
factor_aligned = factor_df.where(pair_mask)
return_aligned = return_df.where(pair_mask)
# Spearman = Pearson on per-row ranks.
factor_ranks = factor_aligned.rank(axis=1, method="average")
return_ranks = return_aligned.rank(axis=1, method="average")
ic = factor_ranks.corrwith(return_ranks, axis=1, method="pearson")
ic = ic[n_valid >= _MIN_VALID_PER_DATE]
ic = ic.dropna()
if ic.empty:
return pd.Series(dtype=float)
return ic.astype(float)
# Kept available for callers/tests that want the slow reference implementation.
def _compute_ic_series_reference(
factor_df: pd.DataFrame, return_df: pd.DataFrame
) -> pd.Series:
"""Loop-based Spearman IC; retained as the numerical reference."""
common_dates = factor_df.index.intersection(return_df.index)
common_codes = factor_df.columns.intersection(return_df.columns)
if len(common_dates) == 0 or len(common_codes) == 0:
return pd.Series(dtype=float)
factor_df = factor_df.loc[common_dates, common_codes]
return_df = return_df.loc[common_dates, common_codes]
ic_values: dict = {}
for date in common_dates:
f = factor_df.loc[date].dropna()
r = return_df.loc[date].dropna()
shared = f.index.intersection(r.index)
if len(shared) < _MIN_VALID_PER_DATE:
continue
corr, _ = spearmanr(f[shared], r[shared])
if not np.isnan(corr):
ic_values[date] = corr
return pd.Series(ic_values, dtype=float)
def compute_group_equity(
factor_df: pd.DataFrame, return_df: pd.DataFrame, n_groups: int
) -> pd.DataFrame:
"""Layered backtest: rank by factor value daily, hold equal-weight, compute cumulative NAV.
Args:
factor_df: Factor values; index=date, columns=codes.
return_df: Returns; index=date, columns=codes.
n_groups: Number of quantile groups.
Returns:
DataFrame with index=date and columns Group_1 ... Group_N holding cumulative NAV.
"""
common_dates = sorted(factor_df.index.intersection(return_df.index))
common_codes = factor_df.columns.intersection(return_df.columns)
if len(common_dates) == 0 or len(common_codes) == 0:
return pd.DataFrame()
factor_df = factor_df.loc[common_dates, common_codes]
return_df = return_df.loc[common_dates, common_codes]
group_returns: dict[str, list[float]] = {f"Group_{i+1}": [] for i in range(n_groups)}
valid_dates = []
for date in common_dates:
f = factor_df.loc[date].dropna()
r = return_df.loc[date].dropna()
shared = f.index.intersection(r.index)
if len(shared) < n_groups:
continue
valid_dates.append(date)
ranked = f[shared].rank(method="first")
bins = pd.qcut(ranked, n_groups, labels=False, duplicates="drop")
if bins.nunique() < n_groups:
# Not enough distinct values; fall back to equal-width cut
bins = pd.cut(ranked, n_groups, labels=False)
for g in range(n_groups):
members = bins[bins == g].index
if len(members) > 0:
group_returns[f"Group_{g+1}"].append(r[members].mean())
else:
group_returns[f"Group_{g+1}"].append(0.0)
if not valid_dates:
return pd.DataFrame()
ret_df = pd.DataFrame(group_returns, index=valid_dates)
equity_df = (1 + ret_df).cumprod()
return equity_df
+411
View File
@@ -0,0 +1,411 @@
"""Alpha registry: AST-scan zoo modules, validate metadata, lazy-import on compute.
Design contract (frozen — referenced by W3 zoo-porting agents):
AlphaMeta (pydantic, ``extra="forbid", frozen=True``) — see fields below
Registry.list(zoo=None, theme=None, universe=None) -> list[str]
Registry.get(alpha_id) -> Alpha
Registry.compute(alpha_id, panel) -> pd.DataFrame
Registry.health() -> dict # {loaded, failed, errors}
Registry.export_manifest() -> dict
Registry.load_alpha_meta_from_py(path) -> AlphaMeta # AST, no import
Source of truth = ``__alpha_meta__`` dict literal in each ``zoo/<zoo>/<id>.py``.
``_meta.yaml`` files are generated by ``export-manifest`` for external consumers
(wiki) — not the load path. If we ever load a YAML, we use ``yaml.safe_load``
with a 5 MB size cap.
Module path derivation: ``f"src.factors.zoo.{zoo_id}.{alpha_id_short}"``. Both
``zoo_id`` and ``alpha_id_short`` must match ``^[a-z][a-z0-9_]{0,31}$``. We never
honour a ``py_module`` field from any data file.
"""
from __future__ import annotations
import ast
import importlib
import importlib.util
import logging
import re
import sys
import threading
from dataclasses import dataclass
from pathlib import Path
from types import ModuleType
from typing import Any, Literal
import numpy as np
import pandas as pd
from pydantic import BaseModel, ConfigDict, Field, ValidationError
from src.factors.base import Alpha
logger = logging.getLogger(__name__)
_ID_RE = re.compile(r"^[a-z][a-z0-9_]{0,31}$")
_MAX_YAML_BYTES = 5_000_000
_MAX_PY_BYTES = 200_000
Theme = Literal[
"momentum",
"reversal",
"volume",
"volatility",
"quality",
"value",
"liquidity",
"microstructure",
"sentiment",
"growth",
"leverage",
]
PanelColumn = Literal["open", "high", "low", "close", "volume", "vwap", "amount"]
Universe = Literal["equity_us", "equity_cn", "equity_hk", "crypto", "futures"]
class AlphaMeta(BaseModel):
"""Strict metadata schema; matches the ``__alpha_meta__`` dict literal."""
model_config = ConfigDict(extra="forbid", frozen=True)
id: str = Field(pattern=r"^[a-z][a-z0-9]+_[a-z0-9_]+$")
nickname: str | None = None
theme: list[Theme]
formula_latex: str
columns_required: list[PanelColumn]
extras_required: list[str] = Field(default_factory=list)
requires_sector: bool = False
universe: list[Universe]
frequency: list[str]
decay_horizon: int = Field(ge=0, le=60)
min_warmup_bars: int = Field(ge=0)
notes: str = ""
class SkipAlpha(Exception):
"""Raised when an alpha's preconditions (sector, columns) are not met."""
class RegistryError(Exception):
"""Raised on registry-level configuration errors."""
@dataclass(frozen=True, slots=True)
class _LoadError:
alpha_id: str
reason: str
def _validate_id_token(token: str, kind: str) -> None:
if not _ID_RE.fullmatch(token):
raise RegistryError(f"invalid {kind} {token!r}: must match {_ID_RE.pattern}")
def load_alpha_meta_from_py(path: Path) -> AlphaMeta:
"""AST-extract the ``__alpha_meta__`` dict literal from a zoo module.
No import is performed — purely static parsing. Raises ``RegistryError`` on
malformed metadata.
"""
size = path.stat().st_size
if size > _MAX_PY_BYTES:
raise RegistryError(f"{path.name}: {size}B exceeds {_MAX_PY_BYTES}B cap")
source = path.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(path))
meta_node: ast.expr | None = None
for stmt in tree.body:
if not isinstance(stmt, ast.Assign):
continue
targets = [t for t in stmt.targets if isinstance(t, ast.Name)]
if any(t.id == "__alpha_meta__" for t in targets):
meta_node = stmt.value
break
if meta_node is None:
raise RegistryError(f"{path.name}: __alpha_meta__ assignment not found")
try:
raw = ast.literal_eval(meta_node)
except (ValueError, SyntaxError) as exc:
raise RegistryError(f"{path.name}: __alpha_meta__ not a literal: {exc}") from exc
if not isinstance(raw, dict):
raise RegistryError(f"{path.name}: __alpha_meta__ must be dict, got {type(raw).__name__}")
try:
return AlphaMeta(**raw)
except ValidationError as exc:
raise RegistryError(f"{path.name}: AlphaMeta validation failed: {exc}") from exc
def _safe_yaml_load(path: Path) -> Any:
"""yaml.safe_load with hard 5 MB size cap (defence in depth)."""
import yaml # local import keeps registry import-light
size = path.stat().st_size
if size > _MAX_YAML_BYTES:
raise RegistryError(f"{path.name}: {size}B exceeds {_MAX_YAML_BYTES}B YAML cap")
text = path.read_text(encoding="utf-8")
return yaml.safe_load(text)
def _zoo_dir_default() -> Path:
return Path(__file__).parent / "zoo"
class Registry:
"""In-memory registry of all discoverable alphas across zoo subdirectories."""
def __init__(self, zoo_root: Path | None = None) -> None:
default_root = _zoo_dir_default()
self._zoo_root = (zoo_root or default_root).resolve()
# When the registry points at the bundled zoo dir, modules are loaded
# by package name (warm import cache, supports relative imports). For
# any other zoo_root (tests, plugins) we load by file path so the
# caller doesn't have to fiddle with sys.path.
self._use_filesystem_loader = self._zoo_root != default_root.resolve()
self._py_paths: dict[str, Path] = {}
self._alphas: dict[str, Alpha] = {}
self._load_errors: list[_LoadError] = []
self._scan()
# ------------------------- scanning -------------------------
def _scan(self) -> None:
if not self._zoo_root.is_dir():
return
for zoo_dir in sorted(self._zoo_root.iterdir()):
if not zoo_dir.is_dir():
continue
zoo_id = zoo_dir.name
if zoo_id.startswith("_") or zoo_id == "__pycache__":
continue
try:
_validate_id_token(zoo_id, "zoo_id")
except RegistryError as exc:
self._load_errors.append(_LoadError(zoo_id, str(exc)))
continue
for py_file in sorted(zoo_dir.glob("*.py")):
if py_file.name.startswith("_"):
continue
self._try_register(zoo_id, py_file)
def _try_register(self, zoo_id: str, py_file: Path) -> None:
short_id = py_file.stem
try:
_validate_id_token(short_id, "alpha_id_short")
except RegistryError as exc:
self._load_errors.append(_LoadError(f"{zoo_id}.{short_id}", str(exc)))
return
try:
meta = load_alpha_meta_from_py(py_file)
except RegistryError as exc:
self._load_errors.append(_LoadError(f"{zoo_id}.{short_id}", str(exc)))
return
# Cross-check: filename matches alpha id (defence vs typos)
# id is "<zoo>_<short>" by convention; not strictly required but flagged.
module_path = f"src.factors.zoo.{zoo_id}.{short_id}"
alpha = Alpha(id=meta.id, zoo=zoo_id, module_path=module_path, meta=meta.model_dump())
if alpha.id in self._alphas:
self._load_errors.append(_LoadError(alpha.id, "duplicate alpha id"))
return
self._alphas[alpha.id] = alpha
self._py_paths[alpha.id] = py_file
# ------------------------- public API -------------------------
def list(
self,
zoo: str | None = None,
theme: str | None = None,
universe: str | None = None,
) -> list[str]:
"""Return alpha IDs matching the (optional) filters."""
out: list[str] = []
for a in self._alphas.values():
if zoo is not None and a.zoo != zoo:
continue
if theme is not None and theme not in a.meta.get("theme", []):
continue
if universe is not None and universe not in a.meta.get("universe", []):
continue
out.append(a.id)
return sorted(out)
def get(self, alpha_id: str) -> Alpha:
if alpha_id not in self._alphas:
raise KeyError(f"alpha_id {alpha_id!r} not in registry")
return self._alphas[alpha_id]
def get_source(self, alpha_id: str) -> str:
"""Return the raw .py source of a registered alpha (size-capped).
Raises:
KeyError: alpha_id unknown.
RegistryError: source file exceeds ``_MAX_PY_BYTES`` or cannot be read.
"""
if alpha_id not in self._alphas:
raise KeyError(f"alpha_id {alpha_id!r} not in registry")
py_path = self._py_paths.get(alpha_id)
if py_path is None:
raise RegistryError(f"{alpha_id}: no source path recorded")
try:
size = py_path.stat().st_size
except OSError as exc:
raise RegistryError(f"{alpha_id}: cannot stat source: {exc}") from exc
if size > _MAX_PY_BYTES:
raise RegistryError(
f"{alpha_id}: source {size}B exceeds {_MAX_PY_BYTES}B cap"
)
try:
return py_path.read_text(encoding="utf-8")
except OSError as exc:
raise RegistryError(f"{alpha_id}: cannot read source: {exc}") from exc
def health(self) -> dict[str, Any]:
return {
"loaded": len(self._alphas),
"failed": len(self._load_errors),
"errors": [
{"alpha_id": e.alpha_id, "reason": e.reason} for e in self._load_errors
],
}
def compute(self, alpha_id: str, panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Lazy-import the alpha module and run its ``compute(panel)``.
Raises:
KeyError: alpha_id unknown.
SkipAlpha: required column / sector tag absent in panel.
RegistryError: import/compute failed or output failed sanity checks.
"""
alpha = self.get(alpha_id)
meta = alpha.meta
missing = [c for c in meta.get("columns_required", []) if c not in panel]
if missing:
raise SkipAlpha(f"{alpha_id}: panel missing required columns {missing}")
missing_extra = [c for c in meta.get("extras_required", []) if c not in panel]
if missing_extra:
raise SkipAlpha(f"{alpha_id}: panel missing extras {missing_extra}")
if meta.get("requires_sector") and "sector" not in panel:
raise SkipAlpha(f"{alpha_id}: panel missing sector tag")
try:
module = self._load_module(alpha)
except Exception as exc: # noqa: BLE001 — isolate import failure
raise RegistryError(f"{alpha_id}: import failed: {exc}") from exc
compute_fn = getattr(module, "compute", None)
if compute_fn is None:
raise RegistryError(f"{alpha_id}: module has no compute() function")
try:
result = compute_fn(panel)
except Exception as exc: # noqa: BLE001 — isolate compute failure
raise RegistryError(f"{alpha_id}: compute() raised: {exc}") from exc
return self._validate_output(alpha_id, result, panel)
def _load_module(self, alpha: Alpha) -> ModuleType:
if not self._use_filesystem_loader:
return importlib.import_module(alpha.module_path)
py_file = self._py_paths[alpha.id]
cached = sys.modules.get(alpha.module_path)
if cached is not None and getattr(cached, "__file__", None) == str(py_file):
return cached
spec = importlib.util.spec_from_file_location(alpha.module_path, py_file)
if spec is None or spec.loader is None:
raise RegistryError(f"{alpha.id}: could not build import spec for {py_file}")
module = importlib.util.module_from_spec(spec)
sys.modules[alpha.module_path] = module
try:
spec.loader.exec_module(module)
except Exception:
sys.modules.pop(alpha.module_path, None)
raise
return module
@staticmethod
def _validate_output(
alpha_id: str,
result: Any,
panel: dict[str, pd.DataFrame],
) -> pd.DataFrame:
if not isinstance(result, pd.DataFrame):
raise RegistryError(
f"{alpha_id}: compute() returned {type(result).__name__}, expected DataFrame"
)
ref = panel.get("close")
if ref is not None and result.shape != ref.shape:
raise RegistryError(
f"{alpha_id}: output shape {result.shape} != close shape {ref.shape}"
)
arr = result.to_numpy(dtype=np.float64, na_value=np.nan)
if np.isinf(arr).any():
raise RegistryError(f"{alpha_id}: output contains +/- inf")
nan_ratio = float(np.isnan(arr).mean()) if arr.size > 0 else 1.0
if nan_ratio > 0.95:
raise RegistryError(f"{alpha_id}: output >95% NaN (nan_ratio={nan_ratio:.3f})")
return result
def export_manifest(self) -> dict[str, Any]:
"""Return a JSON-serialisable snapshot for wiki rendering."""
from datetime import datetime, timezone
zoos: dict[str, list[dict[str, Any]]] = {}
for a in self._alphas.values():
zoos.setdefault(a.zoo, []).append(
{
"id": a.id,
"module_path": a.module_path,
"meta": a.meta,
}
)
return {
"generated_at": datetime.now(timezone.utc).isoformat(),
"zoos": [
{"zoo_id": zoo_id, "alphas": sorted(items, key=lambda x: x["id"])}
for zoo_id, items in sorted(zoos.items())
],
"health": self.health(),
}
# ---------------------------------------------------------------------------
# Process-wide singleton for hot paths (API, bench tool).
#
# Building a Registry() scans ~450 .py files via AST. Doing that on every API
# request is wasteful — the bundled zoo never changes inside a single process.
# Callers that need a custom zoo_root (tests, plugins) should keep instantiating
# Registry() directly; the singleton is only for the default bundled zoo.
# ---------------------------------------------------------------------------
_registry_cache: "Registry | None" = None
_registry_cache_lock = threading.Lock()
def get_default_registry() -> Registry:
"""Return a process-wide cached ``Registry`` for the bundled zoo.
Thread-safe. First call builds and caches; subsequent calls return the same
instance. Use this on hot paths (API handlers, bench tool); use
``Registry()`` directly when you need a custom ``zoo_root``.
"""
global _registry_cache
with _registry_cache_lock:
if _registry_cache is None:
_registry_cache = Registry()
return _registry_cache
def reset_default_registry() -> None:
"""Drop the cached registry (test hook; do not call in production)."""
global _registry_cache
with _registry_cache_lock:
_registry_cache = None
View File
Whitespace-only changes.
+64
View File
@@ -0,0 +1,64 @@
# Academic Baseline Factors — Attributions and Stance
This subdirectory contains price-based proxies for the canonical academic
factor portfolios used as long-short benchmarks in the empirical
asset-pricing literature. None of the original papers' prose is reproduced
here.
## Source attributions
- Sharpe, W. F. (1964). "Capital Asset Prices: A Theory of Market
Equilibrium under Conditions of Risk." *The Journal of Finance*,
19(3), 425-442. — market factor (MKT-RF).
- Fama, E. F., & French, K. R. (1993). "Common risk factors in the
returns on stocks and bonds." *Journal of Financial Economics*,
33(1), 3-56. — SMB (size) and HML (value).
- Carhart, M. M. (1997). "On persistence in mutual fund performance."
*The Journal of Finance*, 52(1), 57-82. — UMD (momentum).
- Fama, E. F., & French, K. R. (2015). "A five-factor asset pricing
model." *Journal of Financial Economics*, 116(1), 1-22. — RMW
(profitability) and CMA (investment).
- Hou, K., Xue, C., & Zhang, L. (2015). "Digesting anomalies: An
investment approach." *Review of Financial Studies*, 28(3), 650-705.
— Q-factor model (referenced for completeness; not implemented
separately because the investment and profitability legs overlap
with RMW / CMA in our OHLCV-only setting).
## Disclosure: these are price-based proxies, not the original factors
The implementations in this directory use only OHLCV panel inputs.
The original factor portfolios use fundamental data (book equity,
operating profitability, asset growth) that we do not carry in the
panel. We therefore replace each fundamental input with a price- or
volume-derived proxy:
- MKT-RF: 21-day per-stock total return (vs. value-weighted excess
market return in the original).
- SMB: negative log of 60-day average dollar volume (vs. market
capitalization in the original).
- HML: negative 252-day return (vs. book-to-market sort in the
original).
- RMW: negative 60-day return volatility (vs. operating profitability
sort in the original).
- CMA: negative 60-day change in log average volume (vs. asset-growth
sort in the original).
- Carhart UMD: 12-month minus 1-month return (matches the original
construction; cross-sectional z-score added for ranking).
Each module's `notes` field repeats this disclosure inline. Users
backtesting these factors as research-grade replacements for the
published series should obtain the authoritative monthly return series
from Kenneth French's data library:
https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html
We link to that resource and do not copy any of its data files.
## Stance on factor definitions
Factor definitions are mathematical / academic concepts and are not
copyrightable. We re-state each factor as a cross-sectional ranking
signal in our own words and our own code. None of the original papers'
prose, tables, or figures is reproduced. The Kenneth French data
library's return series are owned by their authors and remain
externally hosted.
Whitespace-only changes.
@@ -0,0 +1,59 @@
"""academic Carhart momentum: 12-month return excluding the most recent month.
Reference:
Carhart, M. M. (1997). "On persistence in mutual fund performance."
The Journal of Finance, 52(1), 57-82.
This is the canonical UMD (Up Minus Down) momentum factor: trailing 12-month
return less the most recent month, capturing intermediate-term momentum while
avoiding short-term reversal contamination. Computed directly from prices —
no fundamental data required — so this matches the original construction
modulo the cross-sectional z-score wrapper used for long-short ranking.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import delta, safe_div
__alpha_meta__ = {
'id': 'academic_carhart_mom',
'nickname': 'Carhart 1997 momentum — 12m-1m return',
'theme': ['momentum'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl((\mathrm{close}_t - \mathrm{close}_{t-252})/\mathrm{close}_{t-252} - (\mathrm{close}_t - \mathrm{close}_{t-21})/\mathrm{close}_{t-21}\bigr)',
'columns_required': ['close'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 60,
'min_warmup_bars': 252,
'notes': (
'Carhart (1997) UMD momentum factor. 12-month return minus 1-month return, '
'cross-sectional z-score per date for long-short ranking. Top z-scores = '
'winners. Constructed directly from prices, so this matches the original '
'definition modulo the z-score wrapper. Canonical 252d window; declared '
'decay_horizon=60 due to registry schema cap (le=60); real signal horizon=252.'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return 252-day minus 21-day return z-score (Carhart UMD).
Uses canonical (252, 21) windows without silent shrink on short panels.
Short panels produce all-NaN; the registry surfaces this as >95% NaN
(RegistryError) rather than returning a misleading shrunk-window value.
"""
close = panel['close']
ret_long = safe_div(delta(close, 252), close.shift(252))
ret_short = safe_div(delta(close, 21), close.shift(21))
return _cross_sectional_zscore(ret_long - ret_short)
+61
View File
@@ -0,0 +1,61 @@
"""academic CMA: investment factor as inverse 60-day volume growth.
Reference:
Fama, E. F., & French, K. R. (2015). "A five-factor asset pricing model."
Journal of Financial Economics, 116(1), 1-22.
This is a price-based proxy: the original CMA (Conservative Minus Aggressive)
sorts on asset growth from balance-sheet data. We use the negative of 60-day
log-volume change — firms aggressively scaling activity tend to show rising
trading volume; conservative firms show stable / shrinking volume. Higher
z-scores = volume contraction (conservative).
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import delta, ts_mean
__alpha_meta__ = {
'id': 'academic_cma',
'nickname': '[PRICE PROXY] FF2015 CMA — investment via inverse volume growth',
'theme': ['quality'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl(-\Delta_{60}\log(\mathrm{ts\_mean}(\mathrm{volume},\,60) + 1)\bigr)',
'columns_required': ['volume'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 60,
'min_warmup_bars': 120,
'notes': (
'[PRICE PROXY] for the Fama-French (2015) CMA (Conservative Minus '
'Aggressive) investment factor. The original definition uses total-asset '
'growth from fundamental data; here we use the negative 60-day change in '
'log average volume as an activity-growth proxy, then cross-sectional '
'z-score per date for long-short ranking. Top z-scores = volume contraction '
'(conservative / low-growth proxy).'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return inverse 60-day log-volume change z-score per stock.
Uses the canonical 60-bar rolling mean + 60-bar delta windows without
silent shrink on short panels. Short panels produce all-NaN; the
registry surfaces this as >95% NaN (RegistryError) so the user sees
"insufficient history" rather than a misleading shrunk-window value.
"""
volume = panel['volume']
log_avg_vol = np.log(ts_mean(volume, 60) + 1.0)
growth = delta(log_avg_vol, 60)
return _cross_sectional_zscore(-growth)
+60
View File
@@ -0,0 +1,60 @@
"""academic HML: value factor as inverse 252-day return (long-term reversal proxy).
Reference:
Fama, E. F., & French, K. R. (1993). "Common risk factors in the returns on
stocks and bonds." Journal of Financial Economics, 33(1), 3-56.
This is a price-based proxy: the original HML (High Minus Low) sorts on book-
to-market, which requires fundamental book equity. We use the negative of the
trailing 252-day return — value names tend to be long-term underperformers
whose prices have declined relative to book value. Higher z-scores = larger
long-term drawdowns (deeper value).
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import delta, safe_div
__alpha_meta__ = {
'id': 'academic_hml',
'nickname': '[PRICE PROXY] FF1993 HML — value via inverse 252d return',
'theme': ['value'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl(-(\mathrm{close}_t - \mathrm{close}_{t-252}) / \mathrm{close}_{t-252}\bigr)',
'columns_required': ['close'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 60,
'min_warmup_bars': 252,
'notes': (
'[PRICE PROXY] for the Fama-French (1993) HML (High Minus Low) value factor. '
'The original definition uses book-to-market ratio from fundamental data; here '
'we use the negative 252-day total return as a long-term reversal proxy, then '
'cross-sectional z-score per date for long-short ranking. Top z-scores = '
'long-term underperformers (deeper value). Canonical 252d window; declared '
'decay_horizon=60 due to registry schema cap (le=60); real signal horizon=252.'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return inverse 252-day return cross-sectional z-score per stock.
Uses the canonical 252-day window without silent shrink on short panels.
Short panels produce an all-NaN result, which the registry surfaces as a
>95% NaN error (RegistryError) so the user sees "insufficient history"
instead of a misleading shrunk-window value.
"""
close = panel['close']
ret = safe_div(delta(close, 252), close.shift(252))
return _cross_sectional_zscore(-ret)
+53
View File
@@ -0,0 +1,53 @@
"""academic MKT_RF: market factor as 21-day return, cross-sectionally z-scored.
Reference:
Sharpe, W. F. (1964). "Capital Asset Prices: A Theory of Market Equilibrium
under Conditions of Risk." The Journal of Finance, 19(3), 425-442.
This is a price-based proxy: the original CAPM market factor is the
value-weighted excess return of the market portfolio. We approximate the
per-stock market-factor exposure as the 21-day total return cross-sectionally
z-scored, suitable for ranking long-short portfolios.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import delta, safe_div
__alpha_meta__ = {
'id': 'academic_mkt_rf',
'nickname': '[PRICE PROXY] Market factor (Sharpe 1964) — 21d demeaned return',
'theme': ['momentum'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl((\mathrm{close}_t - \mathrm{close}_{t-21}) / \mathrm{close}_{t-21}\bigr)',
'columns_required': ['close'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 21,
'min_warmup_bars': 21,
'notes': (
'[PRICE PROXY] for the Sharpe (1964) / Fama-French market factor (MKT-RF). '
'The original definition uses value-weighted market excess returns; here we '
'use a 21-day per-stock total return and cross-sectional z-score per date '
'for long-short ranking. Top z-scores = strong recent winners; bottom = losers.'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
# std.where(std > 0) turns 0 / NaN std rows into NaN, which propagates
# through div to NaN — no silent inf.
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return 21-day return cross-sectional z-score per stock."""
close = panel['close']
ret = safe_div(delta(close, 21), close.shift(21))
return _cross_sectional_zscore(ret)
+54
View File
@@ -0,0 +1,54 @@
"""academic RMW: profitability factor as inverse 60-day return volatility.
Reference:
Fama, E. F., & French, K. R. (2015). "A five-factor asset pricing model."
Journal of Financial Economics, 116(1), 1-22.
This is a price-based proxy: the original RMW (Robust Minus Weak) sorts on
operating profitability from income statements. We use the negative of trailing
60-day realized return volatility — robust (profitable) firms historically
exhibit lower idiosyncratic volatility (the "low-vol anomaly" overlap). Higher
z-scores = lower volatility = quality proxy.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import delta, safe_div, ts_std
__alpha_meta__ = {
'id': 'academic_rmw',
'nickname': '[PRICE PROXY] FF2015 RMW — quality via inverse 60d volatility',
'theme': ['quality'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl(-\mathrm{ts\_std}((\mathrm{close}_t - \mathrm{close}_{t-1}) / \mathrm{close}_{t-1},\,60)\bigr)',
'columns_required': ['close'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 60,
'min_warmup_bars': 60,
'notes': (
'[PRICE PROXY] for the Fama-French (2015) RMW (Robust Minus Weak) '
'profitability factor. The original definition uses operating profitability '
'from fundamental data; here we use the negative of 60-day return volatility '
'as a low-vol-quality proxy, then cross-sectional z-score per date for '
'long-short ranking. Top z-scores = lower vol (quality / robust).'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return inverse 60-day return-volatility z-score per stock."""
close = panel['close']
ret_1d = safe_div(delta(close, 1), close.shift(1))
vol_60 = ts_std(ret_1d, 60)
return _cross_sectional_zscore(-vol_60)
+55
View File
@@ -0,0 +1,55 @@
"""academic SMB: size factor as inverse log of 60-day average dollar volume.
Reference:
Fama, E. F., & French, K. R. (1993). "Common risk factors in the returns on
stocks and bonds." Journal of Financial Economics, 33(1), 3-56.
This is a price-based proxy: the original SMB (Small Minus Big) sorts on market
capitalization, which we do not carry in the OHLCV panel. We use average daily
dollar volume (close * volume) as a liquidity-weighted size proxy — small caps
typically have low dollar volume. Higher z-scores = smaller (illiquid) names.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import ts_mean
__alpha_meta__ = {
'id': 'academic_smb',
'nickname': '[PRICE PROXY] FF1993 SMB — small-minus-big via inverse dollar-volume',
'theme': ['quality'],
'formula_latex': r'\mathrm{zscore}_{x}\bigl(-\log(\mathrm{ts\_mean}(\mathrm{volume} \cdot \mathrm{close},\,60) + 1)\bigr)',
'columns_required': ['close', 'volume'],
'universe': ['equity_us', 'equity_cn', 'equity_hk'],
'frequency': ['1d'],
'decay_horizon': 60,
'min_warmup_bars': 60,
'notes': (
'[PRICE PROXY] for the Fama-French (1993) SMB (Small Minus Big) size factor. '
'The original definition uses market capitalization from book equity data; here '
'we use the negative log of 60-day average dollar volume (close * volume) as a '
'liquidity-weighted size proxy, then cross-sectional z-score per date for '
'long-short ranking. Top z-scores = smaller / less liquid names.'
),
}
def _cross_sectional_zscore(df: pd.DataFrame) -> pd.DataFrame:
"""Per-row z-score: (x - row_mean) / row_std; zero/NaN std rows -> NaN."""
mean = df.mean(axis=1, skipna=True)
std = df.std(axis=1, ddof=1, skipna=True)
centered = df.sub(mean, axis=0)
result = centered.div(std.where(std > 0), axis=0)
return result.replace([np.inf, -np.inf], np.nan)
def compute(panel: dict[str, pd.DataFrame]) -> pd.DataFrame:
"""Return inverse log 60-day dollar-volume z-score per stock."""
close = panel['close']
volume = panel['volume']
dollar_volume = volume * close
avg = ts_mean(dollar_volume, 60)
log_size = np.log(avg + 1.0)
return _cross_sectional_zscore(-log_size)
+80
View File
@@ -0,0 +1,80 @@
# Kakushadze 101 Formulaic Alphas — License & Provenance
## Display Name
**Kakushadze 101 Formulaic Alphas**
This is the display name used throughout the Vibe-Trading codebase, manifests,
wiki, and documentation when referring to the 101 alphas implemented under
`agent/src/factors/zoo/alpha101/`. No other branding is used.
## Source
* **Paper**: Z. Kakushadze, *"101 Formulaic Alphas"*, 2015.
* **arXiv**: <https://arxiv.org/abs/1601.00991>
* **DOI**: 10.48550/arXiv.1601.00991
* **License of the paper**: arXiv non-exclusive distribution license; the
formulas themselves are presented as mathematical artifacts.
## What we reproduce
This subdirectory contains 101 standalone Python modules, one per alpha,
each implementing the formula given in the appendix of Kakushadze (2015) on
top of the operator surface in `agent/src/factors/base.py`. The
implementations were written from scratch against the formula table; we
did not copy or port source code from any third-party implementation.
## What we do NOT reproduce
* No prose, figures, tables, or discussion from the paper is included in
this repository.
* No commentary on alpha behaviour, holding periods, or empirical
performance is reproduced verbatim.
* The string commonly associated with the firm of one of the paper's
author affiliations **does not appear anywhere in this repository**;
the CI gate `tools/ci_grep_gates.sh` enforces this.
## Trademark / Naming Stance
Mathematical formulas are not subject to copyright. The formulas as listed
in the paper appendix are facts about real-valued functions of OHLCV data
and are reproducible by anyone with access to the paper. The choice of
display name **"Kakushadze 101 Formulaic Alphas"** credits the paper
author and avoids any trademark that may be associated with the
institutional affiliation of the work.
If you redistribute this subdirectory, please:
1. Keep the arXiv citation intact in each module docstring.
2. Keep the display name as **"Kakushadze 101 Formulaic Alphas"**.
3. Do not introduce trademark strings that this project deliberately
avoids (see CI gate `(b)` in `tools/ci_grep_gates.sh`).
## Numerical Cross-Validation
Public reference implementations of these formulas exist (for example
Menooker/KunQuant on GitHub). We used those only as a numerical sanity
check on small synthetic panels — never as a source of code. No file in
this subdirectory was copied, adapted, or text-spliced from a third-party
implementation; the Python code here is original and was generated from
the formula table by a deterministic templating step (the template
generator script is not retained in the published source tree).
## Implementation Notes
* The standard panel exposes `open`, `high`, `low`, `close`, `volume`,
`vwap`, and (optionally) `amount` and `sector`. Where the paper formula
references `cap` (market capitalisation) or `IndClass.industry /
subindustry / sector`, we either substitute a degraded value or gate
the alpha behind `requires_sector=True`; the per-alpha `notes` field
in `__alpha_meta__` records each such degradation.
* `delta(x, d)` is restricted to `d >= 1` (no negative-shift forms). The
paper's formulas use only positive lags so this is a no-op constraint.
* The `delay(x, n)` helper (`x.shift(n)` with `n >= 1`) is implemented
inside each alpha module as a small private function rather than
imported from `src.factors.base` to keep the public operator surface
small.
## Contact
Vibe-Trading project — <https://github.com/HKUDS/Vibe-Trading>
Whitespace-only changes.
@@ -0,0 +1,58 @@
"""Kakushadze Alpha #1.
Formula (paper appendix): rank(ts_argmax(SignedPower((returns<0)?stddev(returns,20):close, 2.), 5)) - 0.5
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 1.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_001"
__alpha_meta__ = {
'id': 'alpha101_001',
'nickname': 'Kakushadze Alpha #1',
'theme': ['reversal', 'volatility'],
'formula_latex': 'rank(ts_argmax(SignedPower((returns<0)?stddev(returns,20):close, 2.), 5)) - 0.5',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
cond = (returns < 0).astype(float)
x = ts_std(returns, 20) * cond + close * (1.0 - cond)
out = rank(ts_argmax(signed_power(x, 2.0), 5)) - 0.5
return out
@@ -0,0 +1,57 @@
"""Kakushadze Alpha #2.
Formula (paper appendix): -1 * correlation(rank(delta(log(volume), 2)), rank(((close-open)/open)), 6)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 2.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_002"
__alpha_meta__ = {
'id': 'alpha101_002',
'nickname': 'Kakushadze Alpha #2',
'theme': ['volume', 'reversal'],
'formula_latex': '-1 * correlation(rank(delta(log(volume), 2)), rank(((close-open)/open)), 6)',
'columns_required': ['open', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_corr(rank(delta(np.log(volume), 2)), rank((close - open_) / open_), 6)
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #3.
Formula (paper appendix): -1 * correlation(rank(open), rank(volume), 10)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 3.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_003"
__alpha_meta__ = {
'id': 'alpha101_003',
'nickname': 'Kakushadze Alpha #3',
'theme': ['volume', 'reversal'],
'formula_latex': '-1 * correlation(rank(open), rank(volume), 10)',
'columns_required': ['open', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
open_ = panel["open"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_corr(rank(open_), rank(volume), 10)
return out
@@ -0,0 +1,55 @@
"""Kakushadze Alpha #4.
Formula (paper appendix): -1 * Ts_Rank(rank(low), 9)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 4.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_004"
__alpha_meta__ = {
'id': 'alpha101_004',
'nickname': 'Kakushadze Alpha #4',
'theme': ['reversal'],
'formula_latex': '-1 * Ts_Rank(rank(low), 9)',
'columns_required': ['low', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 9,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
low = panel["low"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_rank(rank(low), 9)
return out
@@ -0,0 +1,63 @@
"""Kakushadze Alpha #5.
Formula (paper appendix): rank((open - sum(vwap,10)/10)) * (-1 * abs(rank((close - vwap))))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 5.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_005"
__alpha_meta__ = {
'id': 'alpha101_005',
'nickname': 'Kakushadze Alpha #5',
'theme': ['reversal'],
'formula_latex': 'rank((open - sum(vwap,10)/10)) * (-1 * abs(rank((close - vwap))))',
'columns_required': ['open', 'close', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
out = rank(open_ - rolling_sum(vwap, 10) / 10.0) * (-1.0 * rank(close - vwap).abs())
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #6.
Formula (paper appendix): -1 * correlation(open, volume, 10)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 6.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_006"
__alpha_meta__ = {
'id': 'alpha101_006',
'nickname': 'Kakushadze Alpha #6',
'theme': ['volume', 'reversal'],
'formula_latex': '-1 * correlation(open, volume, 10)',
'columns_required': ['open', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
open_ = panel["open"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_corr(open_, volume, 10)
return out
@@ -0,0 +1,86 @@
"""Kakushadze Alpha #7.
Formula (paper appendix): (adv20<volume)?((-1*ts_rank(abs(delta(close,7)),60))*sign(delta(close,7))):(-1)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 7.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_007"
__alpha_meta__ = {
'id': 'alpha101_007',
'nickname': 'Kakushadze Alpha #7',
'theme': ['momentum', 'volume'],
'formula_latex': '(adv20<volume)?((-1*ts_rank(abs(delta(close,7)),60))*sign(delta(close,7))):(-1)',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 67,
'notes': '',
}
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
make_one = _make_one
where_ternary = _where_ternary
d7 = delta(close, 7)
expr = (-1.0 * ts_rank(d7.abs(), 60)) * np.sign(d7)
out = where_ternary(adv20 < volume, expr, -1.0 * make_one(close))
return out
@@ -0,0 +1,71 @@
"""Kakushadze Alpha #8.
Formula (paper appendix): -1 * rank((sum(open,5)*sum(returns,5)) - delay(sum(open,5)*sum(returns,5),10))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 8.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_008"
__alpha_meta__ = {
'id': 'alpha101_008',
'nickname': 'Kakushadze Alpha #8',
'theme': ['reversal'],
'formula_latex': '-1 * rank((sum(open,5)*sum(returns,5)) - delay(sum(open,5)*sum(returns,5),10))',
'columns_required': ['open', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 15,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
s = rolling_sum(open_, 5) * rolling_sum(returns, 5)
out = -1.0 * rank(s - delay(s, 10))
return out
@@ -0,0 +1,80 @@
"""Kakushadze Alpha #9.
Formula (paper appendix): (0<ts_min(delta(close,1),5))?delta(close,1):((ts_max(delta(close,1),5)<0)?delta(close,1):(-1*delta(close,1)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 9.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_009"
__alpha_meta__ = {
'id': 'alpha101_009',
'nickname': 'Kakushadze Alpha #9',
'theme': ['momentum'],
'formula_latex': '(0<ts_min(delta(close,1),5))?delta(close,1):((ts_max(delta(close,1),5)<0)?delta(close,1):(-1*delta(close,1)))',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 6,
'notes': '',
}
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
where_ternary = _where_ternary
d1 = delta(close, 1)
cond1 = ts_min(d1, 5) > 0
cond2 = ts_max(d1, 5) < 0
out = where_ternary(cond1, d1, where_ternary(cond2, d1, -1.0 * d1))
return out
@@ -0,0 +1,81 @@
"""Kakushadze Alpha #10.
Formula (paper appendix): rank((0<ts_min(delta(close,1),4))?delta(close,1):((ts_max(delta(close,1),4)<0)?delta(close,1):(-1*delta(close,1))))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 10.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_010"
__alpha_meta__ = {
'id': 'alpha101_010',
'nickname': 'Kakushadze Alpha #10',
'theme': ['momentum'],
'formula_latex': 'rank((0<ts_min(delta(close,1),4))?delta(close,1):((ts_max(delta(close,1),4)<0)?delta(close,1):(-1*delta(close,1))))',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 5,
'notes': '',
}
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
where_ternary = _where_ternary
d1 = delta(close, 1)
cond1 = ts_min(d1, 4) > 0
cond2 = ts_max(d1, 4) < 0
inner = where_ternary(cond1, d1, where_ternary(cond2, d1, -1.0 * d1))
out = rank(inner)
return out
@@ -0,0 +1,58 @@
"""Kakushadze Alpha #11.
Formula (paper appendix): (rank(ts_max(vwap-close,3))+rank(ts_min(vwap-close,3)))*rank(delta(volume,3))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 11.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_011"
__alpha_meta__ = {
'id': 'alpha101_011',
'nickname': 'Kakushadze Alpha #11',
'theme': ['volume', 'reversal'],
'formula_latex': '(rank(ts_max(vwap-close,3))+rank(ts_min(vwap-close,3)))*rank(delta(volume,3))',
'columns_required': ['close', 'volume', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 5,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
diff = vwap - close
out = (rank(ts_max(diff, 3)) + rank(ts_min(diff, 3))) * rank(delta(volume, 3))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #12.
Formula (paper appendix): sign(delta(volume,1)) * (-1 * delta(close,1))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 12.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_012"
__alpha_meta__ = {
'id': 'alpha101_012',
'nickname': 'Kakushadze Alpha #12',
'theme': ['volume', 'reversal'],
'formula_latex': 'sign(delta(volume,1)) * (-1 * delta(close,1))',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 2,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = np.sign(delta(volume, 1)) * (-1.0 * delta(close, 1))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #13.
Formula (paper appendix): -1 * rank(covariance(rank(close), rank(volume), 5))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 13.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_013"
__alpha_meta__ = {
'id': 'alpha101_013',
'nickname': 'Kakushadze Alpha #13',
'theme': ['volume'],
'formula_latex': '-1 * rank(covariance(rank(close), rank(volume), 5))',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 5,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * rank(ts_cov(rank(close), rank(volume), 5))
return out
@@ -0,0 +1,57 @@
"""Kakushadze Alpha #14.
Formula (paper appendix): (-1*rank(delta(returns,3))) * correlation(open, volume, 10)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 14.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_014"
__alpha_meta__ = {
'id': 'alpha101_014',
'nickname': 'Kakushadze Alpha #14',
'theme': ['volume', 'momentum'],
'formula_latex': '(-1*rank(delta(returns,3))) * correlation(open, volume, 10)',
'columns_required': ['open', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
volume = panel["volume"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
out = (-1.0 * rank(delta(returns, 3))) * ts_corr(open_, volume, 10)
return out
@@ -0,0 +1,62 @@
"""Kakushadze Alpha #15.
Formula (paper appendix): -1 * sum(rank(correlation(rank(high), rank(volume), 3)), 3)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 15.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_015"
__alpha_meta__ = {
'id': 'alpha101_015',
'nickname': 'Kakushadze Alpha #15',
'theme': ['volume'],
'formula_latex': '-1 * sum(rank(correlation(rank(high), rank(volume), 3)), 3)',
'columns_required': ['high', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 6,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
out = -1.0 * rolling_sum(rank(ts_corr(rank(high), rank(volume), 3)), 3)
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #16.
Formula (paper appendix): -1 * rank(covariance(rank(high), rank(volume), 5))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 16.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_016"
__alpha_meta__ = {
'id': 'alpha101_016',
'nickname': 'Kakushadze Alpha #16',
'theme': ['volume'],
'formula_latex': '-1 * rank(covariance(rank(high), rank(volume), 5))',
'columns_required': ['high', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 5,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * rank(ts_cov(rank(high), rank(volume), 5))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #17.
Formula (paper appendix): ((-1*rank(ts_rank(close,10)))*rank(delta(delta(close,1),1)))*rank(ts_rank(volume/adv20,5))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 17.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_017"
__alpha_meta__ = {
'id': 'alpha101_017',
'nickname': 'Kakushadze Alpha #17',
'theme': ['volume', 'reversal'],
'formula_latex': '((-1*rank(ts_rank(close,10)))*rank(delta(delta(close,1),1)))*rank(ts_rank(volume/adv20,5))',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
out = (-1.0 * rank(ts_rank(close, 10))) * rank(delta(delta(close, 1), 1)) * rank(ts_rank(safe_div(volume, adv20), 5))
return out
@@ -0,0 +1,57 @@
"""Kakushadze Alpha #18.
Formula (paper appendix): -1 * rank(stddev(abs(close-open),5) + (close-open) + correlation(close,open,10))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 18.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_018"
__alpha_meta__ = {
'id': 'alpha101_018',
'nickname': 'Kakushadze Alpha #18',
'theme': ['volatility'],
'formula_latex': '-1 * rank(stddev(abs(close-open),5) + (close-open) + correlation(close,open,10))',
'columns_required': ['open', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
# Helper aliases (local closures keep the file standalone & purity-safe).
diff = (close - open_)
out = -1.0 * rank(ts_std(diff.abs(), 5) + diff + ts_corr(close, open_, 10))
return out
@@ -0,0 +1,70 @@
"""Kakushadze Alpha #19.
Formula (paper appendix): (-1*sign((close-delay(close,7))+delta(close,7))) * (1+rank(1+sum(returns,250)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 19.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_019"
__alpha_meta__ = {
'id': 'alpha101_019',
'nickname': 'Kakushadze Alpha #19',
'theme': ['momentum'],
'formula_latex': '(-1*sign((close-delay(close,7))+delta(close,7))) * (1+rank(1+sum(returns,250)))',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 250,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
out = (-1.0 * np.sign((close - delay(close, 7)) + delta(close, 7))) * (1.0 + rank(1.0 + rolling_sum(returns, 250)))
return out
@@ -0,0 +1,66 @@
"""Kakushadze Alpha #20.
Formula (paper appendix): (((-1*rank(open-delay(high,1)))*rank(open-delay(close,1)))*rank(open-delay(low,1)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 20.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_020"
__alpha_meta__ = {
'id': 'alpha101_020',
'nickname': 'Kakushadze Alpha #20',
'theme': ['reversal'],
'formula_latex': '(((-1*rank(open-delay(high,1)))*rank(open-delay(close,1)))*rank(open-delay(low,1)))',
'columns_required': ['open', 'high', 'low', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 2,
'notes': '',
}
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
high = panel["high"]
low = panel["low"]
# Helper aliases (local closures keep the file standalone & purity-safe).
delay = _delay
out = ((-1.0 * rank(open_ - delay(high, 1))) * rank(open_ - delay(close, 1))) * rank(open_ - delay(low, 1))
return out
@@ -0,0 +1,98 @@
"""Kakushadze Alpha #21.
Formula (paper appendix): complex piecewise; see paper
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 21.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_021"
__alpha_meta__ = {
'id': 'alpha101_021',
'nickname': 'Kakushadze Alpha #21',
'theme': ['momentum', 'volatility'],
'formula_latex': 'complex piecewise; see paper',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 20,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
make_one = _make_one
where_ternary = _where_ternary
m8 = rolling_sum(close, 8) / 8.0
s8 = ts_std(close, 8)
m2 = rolling_sum(close, 2) / 2.0
v_adv = safe_div(volume, adv20)
cond_a = (m8 + s8) < m2
cond_b = m2 < (m8 - s8)
cond_c = (v_adv >= 1.0)
one = make_one(close)
out = where_ternary(cond_a, -1.0 * one, where_ternary(cond_b, one, where_ternary(cond_c, one, -1.0 * one)))
return out
@@ -0,0 +1,57 @@
"""Kakushadze Alpha #22.
Formula (paper appendix): -1 * (delta(correlation(high,volume,5),5) * rank(stddev(close,20)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 22.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_022"
__alpha_meta__ = {
'id': 'alpha101_022',
'nickname': 'Kakushadze Alpha #22',
'theme': ['volume', 'volatility'],
'formula_latex': '-1 * (delta(correlation(high,volume,5),5) * rank(stddev(close,20)))',
'columns_required': ['high', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * (delta(ts_corr(high, volume, 5), 5) * rank(ts_std(close, 20)))
return out
@@ -0,0 +1,85 @@
"""Kakushadze Alpha #23.
Formula (paper appendix): ((sum(high,20)/20) < high) ? (-1*delta(high,2)) : 0
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 23.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_023"
__alpha_meta__ = {
'id': 'alpha101_023',
'nickname': 'Kakushadze Alpha #23',
'theme': ['momentum'],
'formula_latex': '((sum(high,20)/20) < high) ? (-1*delta(high,2)) : 0',
'columns_required': ['high', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 20,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
where_ternary = _where_ternary
mh = rolling_sum(high, 20) / 20.0
out = where_ternary(mh < high, -1.0 * delta(high, 2), 0.0 * close)
return out
@@ -0,0 +1,96 @@
"""Kakushadze Alpha #24.
Formula (paper appendix): complex piecewise; see paper
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 24.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_024"
__alpha_meta__ = {
'id': 'alpha101_024',
'nickname': 'Kakushadze Alpha #24',
'theme': ['momentum'],
'formula_latex': 'complex piecewise; see paper',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 200,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
where_ternary = _where_ternary
m100 = rolling_sum(close, 100) / 100.0
x = safe_div(delta(m100, 100), delay(close, 100))
cond = x <= 0.05
left = -1.0 * (close - ts_min(close, 100))
right = -1.0 * delta(close, 3)
out = where_ternary(cond, left, right)
return out
@@ -0,0 +1,58 @@
"""Kakushadze Alpha #25.
Formula (paper appendix): rank((((-1*returns)*adv20)*vwap)*(high-close))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 25.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_025"
__alpha_meta__ = {
'id': 'alpha101_025',
'nickname': 'Kakushadze Alpha #25',
'theme': ['momentum', 'volume'],
'formula_latex': 'rank((((-1*returns)*adv20)*vwap)*(high-close))',
'columns_required': ['high', 'close', 'volume', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 21,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
volume = panel["volume"]
vwap = panel["vwap"]
adv20 = ts_mean(volume, 20)
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
out = rank(((-1.0 * returns) * adv20) * vwap * (high - close))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #26.
Formula (paper appendix): -1 * ts_max(correlation(ts_rank(volume,5),ts_rank(high,5),5),3)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 26.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_026"
__alpha_meta__ = {
'id': 'alpha101_026',
'nickname': 'Kakushadze Alpha #26',
'theme': ['volume'],
'formula_latex': '-1 * ts_max(correlation(ts_rank(volume,5),ts_rank(high,5),5),3)',
'columns_required': ['high', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 13,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_max(ts_corr(ts_rank(volume, 5), ts_rank(high, 5), 5), 3)
return out
@@ -0,0 +1,92 @@
"""Kakushadze Alpha #27.
Formula (paper appendix): (0.5<rank((sum(correlation(rank(volume),rank(vwap),6),2)/2.0)))?(-1):1
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 27.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_027"
__alpha_meta__ = {
'id': 'alpha101_027',
'nickname': 'Kakushadze Alpha #27',
'theme': ['volume'],
'formula_latex': '(0.5<rank((sum(correlation(rank(volume),rank(vwap),6),2)/2.0)))?(-1):1',
'columns_required': ['volume', 'vwap', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
make_one = _make_one
where_ternary = _where_ternary
x = rank(rolling_sum(ts_corr(rank(volume), rank(vwap), 6), 2) / 2.0)
out = where_ternary(x > 0.5, -1.0 * make_one(close), make_one(close))
return out
@@ -0,0 +1,58 @@
"""Kakushadze Alpha #28.
Formula (paper appendix): scale((correlation(adv20,low,5) + (high+low)/2) - close)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 28.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_028"
__alpha_meta__ = {
'id': 'alpha101_028',
'nickname': 'Kakushadze Alpha #28',
'theme': ['volume'],
'formula_latex': 'scale((correlation(adv20,low,5) + (high+low)/2) - close)',
'columns_required': ['high', 'low', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
low = panel["low"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
out = scale(ts_corr(adv20, low, 5) + (high + low) / 2.0 - close)
return out
@@ -0,0 +1,83 @@
"""Kakushadze Alpha #29.
Formula (paper appendix): min(product(rank(rank(scale(log(sum(ts_min(rank(rank(-1*rank(delta(close-1,5)))),2),1))))),1),5) + ts_rank(delay(-1*returns,6),5)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 29.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_029"
__alpha_meta__ = {
'id': 'alpha101_029',
'nickname': 'Kakushadze Alpha #29',
'theme': ['reversal', 'volume'],
'formula_latex': 'min(product(rank(rank(scale(log(sum(ts_min(rank(rank(-1*rank(delta(close-1,5)))),2),1))))),1),5) + ts_rank(delay(-1*returns,6),5)',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 12,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _rolling_prod(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window product; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).apply(np.prod, raw=True)
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
rolling_prod = _rolling_prod
delay = _delay
inner = rank(rank(-1.0 * rank(delta(close - 1.0, 5))))
inner = ts_min(inner, 2)
inner = rolling_sum(inner, 1)
inner = np.log(inner.where(inner > 0))
inner = scale(inner)
inner = rank(rank(inner))
inner = rolling_prod(inner, 1)
term1 = ts_min(inner, 5)
term2 = ts_rank(delay(-1.0 * returns, 6), 5)
out = term1 + term2
return out
@@ -0,0 +1,71 @@
"""Kakushadze Alpha #30.
Formula (paper appendix): ((1-rank(sign(d1)+sign(d2)+sign(d3))) * sum(volume,5)) / sum(volume,20)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 30.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_030"
__alpha_meta__ = {
'id': 'alpha101_030',
'nickname': 'Kakushadze Alpha #30',
'theme': ['momentum', 'volume'],
'formula_latex': '((1-rank(sign(d1)+sign(d2)+sign(d3))) * sum(volume,5)) / sum(volume,20)',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 20,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
s = np.sign(close - delay(close, 1)) + np.sign(delay(close, 1) - delay(close, 2)) + np.sign(delay(close, 2) - delay(close, 3))
out = safe_div((1.0 - rank(s)) * rolling_sum(volume, 5), rolling_sum(volume, 20))
return out
@@ -0,0 +1,60 @@
"""Kakushadze Alpha #31.
Formula (paper appendix): rank(rank(rank(decay_linear(-1*rank(rank(delta(close,10))),10)))) + rank(-1*delta(close,3)) + sign(scale(correlation(adv20,low,12)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 31.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_031"
__alpha_meta__ = {
'id': 'alpha101_031',
'nickname': 'Kakushadze Alpha #31',
'theme': ['momentum'],
'formula_latex': 'rank(rank(rank(decay_linear(-1*rank(rank(delta(close,10))),10)))) + rank(-1*delta(close,3)) + sign(scale(correlation(adv20,low,12)))',
'columns_required': ['low', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
low = panel["low"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
t1 = rank(rank(rank(decay_linear(-1.0 * rank(rank(delta(close, 10))), 10))))
t2 = rank(-1.0 * delta(close, 3))
t3 = pd.DataFrame(np.sign(scale(ts_corr(adv20, low, 12)).to_numpy(dtype=np.float64, na_value=np.nan)), index=close.index, columns=close.columns)
out = t1 + t2 + t3
return out
@@ -0,0 +1,70 @@
"""Kakushadze Alpha #32.
Formula (paper appendix): scale(sum(close,7)/7 - close) + 20*scale(correlation(vwap, delay(close,5), 230))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 32.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_032"
__alpha_meta__ = {
'id': 'alpha101_032',
'nickname': 'Kakushadze Alpha #32',
'theme': ['momentum'],
'formula_latex': 'scale(sum(close,7)/7 - close) + 20*scale(correlation(vwap, delay(close,5), 230))',
'columns_required': ['close', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 235,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
out = scale(rolling_sum(close, 7) / 7.0 - close) + 20.0 * scale(ts_corr(vwap, delay(close, 5), 230))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #33.
Formula (paper appendix): rank(-1*(1-open/close))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 33.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_033"
__alpha_meta__ = {
'id': 'alpha101_033',
'nickname': 'Kakushadze Alpha #33',
'theme': ['reversal'],
'formula_latex': 'rank(-1*(1-open/close))',
'columns_required': ['open', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 1,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = rank(-1.0 * (1.0 - safe_div(open_, close)))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #34.
Formula (paper appendix): rank((1-rank(stddev(returns,2)/stddev(returns,5))) + (1-rank(delta(close,1))))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 34.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_034"
__alpha_meta__ = {
'id': 'alpha101_034',
'nickname': 'Kakushadze Alpha #34',
'theme': ['volatility'],
'formula_latex': 'rank((1-rank(stddev(returns,2)/stddev(returns,5))) + (1-rank(delta(close,1))))',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 6,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
out = rank((1.0 - rank(safe_div(ts_std(returns, 2), ts_std(returns, 5)))) + (1.0 - rank(delta(close, 1))))
return out
@@ -0,0 +1,58 @@
"""Kakushadze Alpha #35.
Formula (paper appendix): ts_rank(volume,32) * (1 - ts_rank((close+high-low),16)) * (1 - ts_rank(returns,32))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 35.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_035"
__alpha_meta__ = {
'id': 'alpha101_035',
'nickname': 'Kakushadze Alpha #35',
'theme': ['volume', 'momentum'],
'formula_latex': 'ts_rank(volume,32) * (1 - ts_rank((close+high-low),16)) * (1 - ts_rank(returns,32))',
'columns_required': ['high', 'low', 'close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 33,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
low = panel["low"]
volume = panel["volume"]
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
out = ts_rank(volume, 32) * (1.0 - ts_rank((close + high - low), 16)) * (1.0 - ts_rank(returns, 32))
return out
@@ -0,0 +1,77 @@
"""Kakushadze Alpha #36.
Formula (paper appendix): weighted sum; see paper
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 36.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_036"
__alpha_meta__ = {
'id': 'alpha101_036',
'nickname': 'Kakushadze Alpha #36',
'theme': ['momentum', 'volume'],
'formula_latex': 'weighted sum; see paper',
'columns_required': ['open', 'close', 'volume', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 200,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
volume = panel["volume"]
vwap = panel["vwap"]
adv20 = ts_mean(volume, 20)
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
t1 = 2.21 * rank(ts_corr((close - open_), delay(volume, 1), 15))
t2 = 0.7 * rank(open_ - close)
t3 = 0.73 * rank(ts_rank(delay(-1.0 * returns, 6), 5))
t4 = rank(ts_corr(vwap, adv20, 6).abs())
t5 = 0.6 * rank((rolling_sum(close, 200) / 200.0 - open_) * (close - open_))
out = t1 + t2 + t3 + t4 + t5
return out
@@ -0,0 +1,64 @@
"""Kakushadze Alpha #37.
Formula (paper appendix): rank(correlation(delay(open-close,1),close,200)) + rank(open-close)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 37.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_037"
__alpha_meta__ = {
'id': 'alpha101_037',
'nickname': 'Kakushadze Alpha #37',
'theme': ['momentum'],
'formula_latex': 'rank(correlation(delay(open-close,1),close,200)) + rank(open-close)',
'columns_required': ['open', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 201,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
# Helper aliases (local closures keep the file standalone & purity-safe).
delay = _delay
out = rank(ts_corr(delay(open_ - close, 1), close, 200)) + rank(open_ - close)
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #38.
Formula (paper appendix): (-1*rank(ts_rank(close,10))) * rank(close/open)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 38.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_038"
__alpha_meta__ = {
'id': 'alpha101_038',
'nickname': 'Kakushadze Alpha #38',
'theme': ['reversal'],
'formula_latex': '(-1*rank(ts_rank(close,10))) * rank(close/open)',
'columns_required': ['open', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
open_ = panel["open"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = (-1.0 * rank(ts_rank(close, 10))) * rank(safe_div(close, open_))
return out
@@ -0,0 +1,62 @@
"""Kakushadze Alpha #39.
Formula (paper appendix): (-1*rank(delta(close,7)*(1-rank(decay_linear(volume/adv20,9))))) * (1+rank(sum(returns,250)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 39.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_039"
__alpha_meta__ = {
'id': 'alpha101_039',
'nickname': 'Kakushadze Alpha #39',
'theme': ['momentum', 'volume'],
'formula_latex': '(-1*rank(delta(close,7)*(1-rank(decay_linear(volume/adv20,9))))) * (1+rank(sum(returns,250)))',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 250,
'notes': 'Very long lookback (>= ~100 bars); produces NaN warmup on short panels which may trigger the >95% NaN registry guard.',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
returns = close.pct_change()
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
out = (-1.0 * rank(delta(close, 7) * (1.0 - rank(decay_linear(safe_div(volume, adv20), 9))))) * (1.0 + rank(rolling_sum(returns, 250)))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #40.
Formula (paper appendix): (-1*rank(stddev(high,10))) * correlation(high,volume,10)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 40.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_040"
__alpha_meta__ = {
'id': 'alpha101_040',
'nickname': 'Kakushadze Alpha #40',
'theme': ['volatility', 'volume'],
'formula_latex': '(-1*rank(stddev(high,10))) * correlation(high,volume,10)',
'columns_required': ['high', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 10,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = (-1.0 * rank(ts_std(high, 10))) * ts_corr(high, volume, 10)
return out
@@ -0,0 +1,57 @@
"""Kakushadze Alpha #41.
Formula (paper appendix): (high*low)^0.5 - vwap
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 41.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_041"
__alpha_meta__ = {
'id': 'alpha101_041',
'nickname': 'Kakushadze Alpha #41',
'theme': ['reversal'],
'formula_latex': '(high*low)^0.5 - vwap',
'columns_required': ['high', 'low', 'vwap', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 1,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
low = panel["low"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = (high * low).pow(0.5) - vwap
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #42.
Formula (paper appendix): rank(vwap-close) / rank(vwap+close)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 42.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_042"
__alpha_meta__ = {
'id': 'alpha101_042',
'nickname': 'Kakushadze Alpha #42',
'theme': ['reversal'],
'formula_latex': 'rank(vwap-close) / rank(vwap+close)',
'columns_required': ['close', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 1,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
vwap = panel["vwap"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = safe_div(rank(vwap - close), rank(vwap + close))
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #43.
Formula (paper appendix): ts_rank(volume/adv20,20) * ts_rank(-1*delta(close,7),8)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 43.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_043"
__alpha_meta__ = {
'id': 'alpha101_043',
'nickname': 'Kakushadze Alpha #43',
'theme': ['volume', 'momentum'],
'formula_latex': 'ts_rank(volume/adv20,20) * ts_rank(-1*delta(close,7),8)',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 39,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
out = ts_rank(safe_div(volume, adv20), 20) * ts_rank(-1.0 * delta(close, 7), 8)
return out
@@ -0,0 +1,56 @@
"""Kakushadze Alpha #44.
Formula (paper appendix): -1 * correlation(high, rank(volume), 5)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 44.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_044"
__alpha_meta__ = {
'id': 'alpha101_044',
'nickname': 'Kakushadze Alpha #44',
'theme': ['volume'],
'formula_latex': '-1 * correlation(high, rank(volume), 5)',
'columns_required': ['high', 'volume', 'close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 5,
'notes': '',
}
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
high = panel["high"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
out = -1.0 * ts_corr(high, rank(volume), 5)
return out
@@ -0,0 +1,70 @@
"""Kakushadze Alpha #45.
Formula (paper appendix): -1 * (rank(sum(delay(close,5),20)/20)*correlation(close,volume,2)*rank(correlation(sum(close,5),sum(close,20),2)))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 45.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_045"
__alpha_meta__ = {
'id': 'alpha101_045',
'nickname': 'Kakushadze Alpha #45',
'theme': ['momentum', 'volume'],
'formula_latex': '-1 * (rank(sum(delay(close,5),20)/20)*correlation(close,volume,2)*rank(correlation(sum(close,5),sum(close,20),2)))',
'columns_required': ['close', 'volume'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
volume = panel["volume"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
out = -1.0 * (rank(rolling_sum(delay(close, 5), 20) / 20.0) * ts_corr(close, volume, 2) * rank(ts_corr(rolling_sum(close, 5), rolling_sum(close, 20), 2)))
return out
@@ -0,0 +1,93 @@
"""Kakushadze Alpha #46.
Formula (paper appendix): complex piecewise; see paper
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 46.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_046"
__alpha_meta__ = {
'id': 'alpha101_046',
'nickname': 'Kakushadze Alpha #46',
'theme': ['momentum'],
'formula_latex': 'complex piecewise; see paper',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 21,
'notes': '',
}
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
delay = _delay
make_one = _make_one
where_ternary = _where_ternary
x = ((delay(close, 20) - delay(close, 10)) / 10.0) - ((delay(close, 10) - close) / 10.0)
one = make_one(close)
out = where_ternary(0.25 < x, -1.0 * one, where_ternary(x < 0.0, one, -1.0 * (close - delay(close, 1))))
return out
@@ -0,0 +1,80 @@
"""Kakushadze Alpha #47.
Formula (paper appendix): ((rank(1/close)*volume/adv20) * (high*rank(high-close)/(sum(high,5)/5))) - rank(vwap-delay(vwap,5))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 47.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_047"
__alpha_meta__ = {
'id': 'alpha101_047',
'nickname': 'Kakushadze Alpha #47',
'theme': ['volume', 'momentum'],
'formula_latex': '((rank(1/close)*volume/adv20) * (high*rank(high-close)/(sum(high,5)/5))) - rank(vwap-delay(vwap,5))',
'columns_required': ['high', 'close', 'volume', 'vwap'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 25,
'notes': '',
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
high = panel["high"]
volume = panel["volume"]
vwap = panel["vwap"]
adv20 = ts_mean(volume, 20)
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
make_one = _make_one
t1 = safe_div(rank(safe_div(make_one(close), close)) * volume, adv20)
t2 = safe_div(high * rank(high - close), rolling_sum(high, 5) / 5.0)
out = t1 * t2 - rank(vwap - delay(vwap, 5))
return out
+104
View File
@@ -0,0 +1,104 @@
"""Kakushadze Alpha #48.
Formula (paper appendix): indneutralize(...subindustry...) / sum((delta(close,1)/delay(close,1))^2, 250)
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 48.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_048"
__alpha_meta__ = {
'id': 'alpha101_048',
'nickname': 'Kakushadze Alpha #48',
'theme': ['momentum', 'volatility'],
'formula_latex': 'indneutralize(...subindustry...) / sum((delta(close,1)/delay(close,1))^2, 250)',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': True,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 251,
'notes': "Industry neutralization implemented via per-row sector group demean (panel['sector'] required). When sector tag is absent the registry rejects via SkipAlpha; the compute() also has a degraded global demean fallback. This is a partial approximation of the paper's IndClass.industry/subindustry/sector neutralization.",
}
def _rolling_sum(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Rolling window sum; warmup -> NaN."""
return df.rolling(window=n, min_periods=n).sum()
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def _ind_neutralize(x: pd.DataFrame, panel: dict) -> pd.DataFrame:
"""Industry/sector neutralize: subtract the row-wise sector group mean.
If panel has a 'sector' DataFrame (same shape as close), subtract the
per-sector cross-sectional mean per row. If absent, degrade to global
cross-sectional demean (subtract row mean). This is a degraded fallback
relative to the paper's industry/subindustry neutralization; see notes.
"""
sector_df = panel.get("sector")
if sector_df is None:
row_mean = x.mean(axis=1, skipna=True)
return x.sub(row_mean, axis=0)
# Per-row group demean. Iterate rows; numpy-fast enough for small panels.
arr = x.to_numpy(dtype=np.float64, na_value=np.nan).copy()
sec_arr = sector_df.to_numpy()
n_rows = arr.shape[0]
for i in range(n_rows):
row = arr[i]
sec_row = sec_arr[i]
for tag in pd.unique(sec_row):
mask = sec_row == tag
vals = row[mask]
finite = vals[~np.isnan(vals)]
if finite.size == 0:
continue
mean = finite.mean()
row[mask] = vals - mean
arr[i] = row
return pd.DataFrame(arr, index=x.index, columns=x.columns)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
rolling_sum = _rolling_sum
delay = _delay
ind_neutralize = _ind_neutralize
num = (ts_corr(delta(close, 1), delta(delay(close, 1), 1), 250) * delta(close, 1)) / close
num = ind_neutralize(num, panel)
denom = rolling_sum(safe_div(delta(close, 1), delay(close, 1)).pow(2), 250)
out = safe_div(num, denom)
return out
@@ -0,0 +1,93 @@
"""Kakushadze Alpha #49.
Formula (paper appendix): (((delay(close,20)-delay(close,10))/10 - (delay(close,10)-close)/10) < -0.1) ? 1 : -1*(close-delay(close,1))
Source: Kakushadze (2015), "101 Formulaic Alphas", arXiv:1601.00991, eq. 49.
"""
from __future__ import annotations
import numpy as np
import pandas as pd
from src.factors.base import (
decay_linear,
delta,
rank,
safe_div,
scale,
signed_power,
ts_argmax,
ts_argmin,
ts_corr,
ts_cov,
ts_max,
ts_mean,
ts_min,
ts_rank,
ts_std,
)
ALPHA_ID = "alpha101_049"
__alpha_meta__ = {
'id': 'alpha101_049',
'nickname': 'Kakushadze Alpha #49',
'theme': ['momentum'],
'formula_latex': '(((delay(close,20)-delay(close,10))/10 - (delay(close,10)-close)/10) < -0.1) ? 1 : -1*(close-delay(close,1))',
'columns_required': ['close'],
'extras_required': [],
'requires_sector': False,
'universe': ['equity_us'],
'frequency': ['1D'],
'decay_horizon': 5,
'min_warmup_bars': 21,
'notes': '',
}
def _delay(df: pd.DataFrame, n: int) -> pd.DataFrame:
"""Backward shift by n (lookahead-safe; n>=1 required)."""
if n < 1:
raise ValueError("delay requires n >= 1 (lookahead ban)")
return df.shift(n)
def _make_one(ref: pd.DataFrame) -> pd.DataFrame:
"""A DataFrame of 1.0 with the same shape/index/columns as ``ref``."""
return pd.DataFrame(1.0, index=ref.index, columns=ref.columns)
def _where_ternary(cond, a, b):
"""Vectorised ternary `(cond ? a : b)` returning a DataFrame.
``cond`` is a boolean DataFrame; ``a`` / ``b`` may be DataFrame or scalar.
"""
if isinstance(a, (int, float)):
a_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(a))
else:
a_arr = a.to_numpy(dtype=np.float64, na_value=np.nan)
if isinstance(b, (int, float)):
b_arr = np.full_like(cond.to_numpy(dtype=np.float64), float(b))
else:
b_arr = b.to_numpy(dtype=np.float64, na_value=np.nan)
cond_arr = cond.to_numpy(dtype=bool, na_value=False) if hasattr(cond, "to_numpy") else np.asarray(cond, dtype=bool)
out = np.where(cond_arr, a_arr, b_arr)
out = np.where(np.isfinite(out), out, np.nan)
idx = cond.index if hasattr(cond, "index") else a.index
cols = cond.columns if hasattr(cond, "columns") else a.columns
return pd.DataFrame(out, index=idx, columns=cols)
def compute(panel: dict) -> pd.DataFrame:
"""Compute the alpha on the OHLCV+ panel and return a wide DataFrame."""
close = panel["close"]
# Helper aliases (local closures keep the file standalone & purity-safe).
delay = _delay
make_one = _make_one
where_ternary = _where_ternary
x = ((delay(close, 20) - delay(close, 10)) / 10.0) - ((delay(close, 10) - close) / 10.0)
one = make_one(close)
out = where_ternary(x < -0.1, one, -1.0 * (close - delay(close, 1)))
return out
Loaded 100 of 685 files, more files were not shown because too many files have changed in this diff. Show more