Add P@5/MRR/NDCG@5 evaluation framework for memory retrieval A/B testing.
Benchmark compares Baseline (pure token overlap) vs Treatment (BM25 + importance
decay) on a 200-entry corpus with 50 annotated queries.
The benchmark suite is NOT intended for CI — it requires corpus files
(tmp/benchmark_corpus/) that are not version-controlled. The suite
automatically skips when corpus files are absent.
Corpus is derived from anonymized production agent sessions.
Usage:
python -m pytest agent/tests/memory/benchmarks/ -v
Refs: #732
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
These files are not part of the community contribution scope.
Benchmark results are kept locally for validation only.
Refs: #732
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
Cherry-picked the test-infra commit from #175 (sambazhu): vitest setup + 19 frontend test files (197 tests), backend auth/path-traversal/CORS tests, yfinance crypto fallback (BTC-USDT->BTC-USD), and CI coverage + frontend-test steps. Dropped the PR's separate i18n commit to stay consistent with #163.
Fix: align @vitest/coverage-v8 to ^3.2.6 (was ^4.1.8, incompatible with vitest ^3.2.6 - blocked npm install and the new npm ci CI step).
First-party, anonymous analytics for the public wiki — no client-side telemetry in the pip package. Pages Function middleware classifies each page request (AI agent / bot / human) by User-Agent server-side and counts it into D1; /api/stats serves the footer's aggregate counts alongside public PyPI install and GitHub star numbers.
Refresh the interactive terminal transcript, activity rail, and prompt visuals. Document the 2026-05-23 CLI update in README and keep generated CLI screenshots out of git.
Co-authored-by: ykykj <140159248+ykykj@users.noreply.github.com>
wiki/scripts/build_alpha_library.py (475L)
- Reads manifest.json produced by `vibe-trading alpha export-manifest`
- Emits per-alpha pages (content/<zoo>/<id>.html) and per-zoo
overview tables (content/<zoo>/index.html) — 452 + 4 = 456 pages
- Jinja2 with select_autoescape(["html","xml"]) mandatory; no |safe
on user-derived content; strict CSP on every generated page
(default-src 'self'; style-src 'self' 'unsafe-inline';
script-src 'none'; img-src 'self' data:)
- Emits content/index.json for the landing page to hydrate per-zoo
counts via fetch()
- Friendly exit when manifest.json missing
- End-of-main self-test compares manifest formula content against
rendered HTML to verify the escape pipeline; prints
"html escape ok" or "html escape FAIL"
wiki/alpha-library/index.html (rewritten from 78-line shell to 192L)
- Hero + 4 zoo cards (Kakushadze 101 Formulaic Alphas, qlib158,
gtja191, academic) + stats grid + 3-line CLI quickstart + GitHub
footer
- Per-zoo counts hydrated at runtime from content/index.json
- Loose CSP (script-src 'self' 'unsafe-inline') only on this landing
page to keep the site theme-init inline script and the inline
fetch() — generated /content/** pages stay on script-src 'none'
wiki/research-lab/index.html
- Post grid replacing the empty hero stub; links to the new blog
- One-line lede explaining the section
wiki/research-lab/posts/alpha-191-in-2026.html (~675L, ~2400 words)
- 6-section deep-dive: TL;DR / Background / Method / Findings /
Caveats / Reproduce
- Numbers from a real bench run on CSI 300 / 2018-2025: 10 alive,
15 reversed, 165 dead (of 190 tested, 1 skipped). Top 5 by IR:
gtja191_171 / _111 / _163 / _002 / _054. Theme survival rates:
microstructure 22%, reversal 11%, volatility 8%, volume 5%,
momentum 2%, liquidity 0%
- Survivorship-bias caveat for cross-zoo readers (the current
constituent list of any index is a forward-biased proxy for the
point-in-time constituents during the window)
- Strict CSP (script-src 'none'); no MathJax/KaTeX — formula_latex
rendered as <code class="formula">; aggregate stats only (no
per-stock per-date values, Tushare ToS)
wiki/research-lab/posts/alpha-191-in-2026.tweet.md
- HKUDS-style draft tweet (Chao Huang voice): emoji-decorated
project name, real numbers (10 / 15 / 165), 8-year OOS qualifier,
GitHub + pip links, ≤2 hashtags. READY to post once the release
goes live.
.gitignore adds wiki/alpha-library/manifest.json and
wiki/alpha-library/content/ as generated, never-checked-in artifacts.
Add a static Cloudflare Pages wiki under wiki/, including the landing page, docs app, redirects/headers, maintainer README, and wiki validation workflow.
Pin PR #53 behavior with 4 cases:
- under-limit upload writes to UPLOADS_DIR
- exactly-at-limit upload succeeds
- over-limit returns 413 and removes the partial file
- blocked extension returns 400 with no file written
Tests shrink MAX_UPLOAD_SIZE to 4 KB via monkeypatch so the streaming
chunk loop and cleanup path run without allocating 50 MB.
Also gitignore AGENTS.md (local mirror of CLAUDE.md for non-Claude
agent tools, identical content, no need to track twice).
Users upload broker exports (CSV/Excel) and get a trading portrait
plus 4 bias diagnostics. Parses 同花顺/东方财富/富途/generic formats
with encoding fallback (utf-8/gbk/gb2312/big5).
Profile: holding days, win rate, PnL ratio, max drawdown, top
symbols, market/hourly distribution via FIFO lot matching.
Behavior: disposition effect, overtrading, chasing momentum,
anchoring — each with severity + numeric evidence derived from
the journal itself, no external market data pulls.
Also: gitignore agent/uploads/ so runtime user uploads never leak
into the repo; inline a tiny csv in test_doc_reader instead of
depending on a fixture file.
Haozhe Wu