Commit Graph
21 Commits
Author SHA1 Message Date
shadowinlifeandQoder b9507ee12b feat(memory): implement Tier 2 structural organization (#815)
* feat(memory): implement Tier 2 structural organization

Add four independent modules for memory structural organization:
- H-MEM hierarchical directory routing (hierarchy.py)
- A-MEM semantic linking via BM25 (semantic_links.py)
- Auto-compression pipeline via TF-IDF (compression.py)
- FTS5 full-text search index (search_index.py)

Integration:
- 4 feature flags in MemoryConfig (default: disabled)
- MemoryEntry extended with category + compression_level fields
- persistent.py wired with hierarchy routing, FTS5 search, link expansion
- lifecycle.py triggers compression in GC cycle
- remember_tool.py attaches related links in recall results
- FTS5 CJK bigram+unigram indexing for better Chinese recall
- Lazy auto-rebuild on first FTS5 search

Testing:
- 163 unit tests + 9 integration tests + 23 benchmarks (all passing)
- Benchmark evaluated on 200 real production memories + 50 labeled queries
- Compression retention: daily 91.4%, digest 97.1%
- Hierarchy routing: 93.3% P@5 parity with full scan

Closes #732 (Tier 2 scope)

Signed-off-by: shadowinlife <shadowinlife@gmail.com>

Co-Authored-By: Qoder <noreply@qoder.com>
AI-Contributed/Feature: 1582/1871
AI-Contributed/UT: 2123/2159

* feat(memory): add VT_MEMORY=off|on|full business-friendly preset

Simplify memory configuration from 7 individual boolean flags to a
single business-friendly environment variable:
  - off:  all features disabled (default, backward compatible)
  - on:   Tier 1 quality + decay + garbage collection
  - full: Tier 1 + Tier 2 hierarchy + links + compression + FTS

Individual VT_MEMORY_* flags still work as explicit overrides with
higher priority than the preset baseline.

Also adds a dependency-conflict warning in lifecycle.py when
compression is enabled without GC.

Signed-off-by: shadowinlife <shadowinlife@gmail.com>

AI-Contributed/Feature: 0/138
AI-Contributed/UT: 0/0

* docs: add Memory Tier 2 and VT_MEMORY preset to CHANGELOG

Document the new structural organization features (hierarchy, semantic
links, compression, FTS5) and the business-friendly VT_MEMORY=off|on|full
configuration preset.

Signed-off-by: shadowinlife <shadowinlife@gmail.com>

AI-Contributed/Feature: 0/15
AI-Contributed/UT: 0/0

---------

Co-authored-by: Qoder <noreply@qoder.com>
2026-07-24 22:13:16 +08:00
Haozhe Wu 8cb536fa61 chore(release): 0.1.12 — version bump + CHANGELOG
Bump version 0.1.11 → 0.1.12 across pyproject, frontend package(+lock),
i18n app.version, SKILL.md, and Dockerfile OCI label. CHANGELOG [0.1.12]
covers the 0.1.11→0.1.12 window (correlation regime timeline, 3 new LLM
providers, MT5 connector, SDM skill, USD-M perps, ~90 correctness fixes,
and the #758 zai streaming/base_url fix).
2026-07-22 13:14:11 +08:00
Haozhe Wu 11b4f62340 Merge PR #481: feat: MetaTrader 5 (Exness) broker connector + mt5 market-data source
Red-lines verified PASS (mandate gate, structural trade_mode discriminator, no MCP
order exposure, sandbox-barred loader). Resolved additive conflicts: registry (mt5
loader alongside longbridge), pyproject (mt5 extra), CHANGELOG (kept both entries);
README/SKILL counts reconciled in the news pass. market_data =F/=X (yahoo) and mt5
forex routing coexist.
2026-07-20 18:48:32 +08:00
Haozhe Wu facc97cec1 Merge PR #570 2026-07-17 21:29:25 +08:00
Ali EmadiandClaude Fable 5 e04c98ca5b feat(swarm): discover user presets from ~/.vibe-trading/swarm/presets
Preset YAMLs dropped into ~/.vibe-trading/swarm/presets/ are now discovered
alongside the bundled roster, mirroring the user-skills pattern
(src/agent/skills.py): the user directory is searched first, same-name files
override bundled presets, and user files survive pip install -U (previously
the only way to add a preset was copying into site-packages, which upgrades
wipe).

- load_preset()/resolve_preset_path(): user dir first, then bundled; the
  not-found error names both locations.
- list_presets(): union with user-wins-by-stem override; entries gain a
  source: "user"|"bundled" field (additive — CLI/MCP/API consumers read
  keys individually or serialize the whole dict).
- Preset names are validated to a single path segment before any filesystem
  lookup (rejects separators and '..'); the bundled lookup had the same
  latent exposure, so validation covers both paths.
- run_swarm(preset_name=...): an explicitly named user preset now resolves;
  keyword auto-routing intentionally stays limited to the curated table, so
  a user preset is reachable only by naming it.
- Packaging regression suite extended (6 new tests: discovery, override,
  absent-dir no-op, traversal rejection, error message, tool gating); the
  roster-count test now filters to bundled entries so a developer's own user
  presets cannot make it flap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Ali Emadi <ali@gatekeeperai.com>
2026-07-16 16:50:15 +02:00
xor_xe 80b564436b Merge remote-tracking branch 'upstream/main' into fix/550-local-source-engine-benchmark
Signed-off-by: xor_xe <bedauri.it@gmail.com>

# Conflicts:
#	CHANGELOG.md
2026-07-15 19:07:34 +04:00
Haozhe Wu 1ece159b61 fix(config): refresh cached env after dotenv load
Closes #541.
2026-07-15 20:33:25 +08:00
xor_xe 83e482ef46 fix(backtest): honor explicit local source for engine routing and benchmark
Sources without a dedicated routing branch (e.g. local) sent US/HK
equity datasets to the crypto engine, and explicit benchmarks were
always fetched via yfinance even when all required price data was
available locally.

Route unbranched sources by detected instrument market, and fetch the
explicit benchmark through the configured source's loader first,
keeping yfinance as the fallback. source: auto keeps the yfinance
default.

Fixes #550

Signed-off-by: xor_xe <bedauri.it@gmail.com>
2026-07-15 14:12:24 +04:00
Haozhe Wu 89f1883016 Merge remote-tracking branch 'origin/main' into codex/pr-532-followup
# Conflicts:
#	CHANGELOG.md
2026-07-15 12:04:57 +08:00
Haozhe Wu 9fa3a641c0 Merge remote-tracking branch 'origin/main' into codex/pr-531-followup
# Conflicts:
#	CHANGELOG.md
2026-07-15 11:57:57 +08:00
YZY0108 6b26f6bb0e fix(backtest): report turnover from executed trades
Signed-off-by: YZY0108 <u3629247@connect.hku.hk>
2026-07-14 09:33:51 +08:00
YZY0108 46e2e6cd7f fix(backtest): account for terminal close costs
Signed-off-by: YZY0108 <u3629247@connect.hku.hk>
2026-07-14 09:28:59 +08:00
YZY0108 cdd231cb41 fix(backtest): make open rebalances causal
Signed-off-by: YZY0108 <u3629247@connect.hku.hk>
2026-07-14 09:24:13 +08:00
Haozhe Wu 8855e654ed docs: log today's security-hardening + PR batch in CHANGELOG and news
Covers issue #476 (all 10 audit findings), #377/#478/#480/#487/#479/#484.
Propagated to all 5 locale READMEs' News section (3 visible + moved
2026-07-10 into <details>, per convention).
2026-07-13 12:55:36 +08:00
Yuan ZhiyiandYuan Zhiyi 9daccb427f fix(optimizers): exclude decision-bar returns (#487)
BaseOptimizer.optimize() built each estimation window with ret.loc[:dt], which
includes ret[dt] — a close-to-close return for the decision date. Since
weights at dt execute at that bar's open, ret[dt] isn't actually observable
yet at execution time, so every optimizer built on this base (mean-variance,
equal-volatility, max-diversification, risk-parity, turnover-aware) was
leaking one bar of future information into its own decision.

Fix restricts the window to ret.index < dt. Two regression tests prove it:
the window's latest timestamp always precedes the decision date, and shocking
only the decision-bar return leaves that bar's executed weights unchanged.
Backtests without an optimizer are unaffected; this does change historical
optimized results as intended (removing a look-ahead leak, not introducing
one).

Verified locally: full backend suite green (only 2 pre-existing/unrelated
failures, same as main), env-var CI gate passes, ruff clean, 18/18
optimizer-specific tests pass including both new causality tests.

Co-authored-by: Yuan Zhiyi <YZY0108@users.noreply.github.com>
2026-07-13 12:06:58 +08:00
staniell b159284af4 docs+packaging: mt5 extra, source/connector docs, CHANGELOG
- pyproject: [mt5] extra with a sys_platform == win32 marker so ubuntu
  CI and non-Windows installs are unaffected.
- README: data-source table + forex fallback chain + MetaTrader 5
  connector section documenting mt5.json, the four profiles, the
  demo/live identity guard, lot-aware mandate sizing, hedging
  close-by-ticket semantics, and the halt path.
- agent/SKILL.md + data-routing SKILL.md: sources 20 -> 21 (test-
  enforced counts), forex routing priority mt5 > akshare.
- CHANGELOG [Unreleased]: connector (11 -> 12) + data source (20 -> 21).

Signed-off-by: staniell <giostaniell12@gmail.com>
2026-07-13 00:21:39 +08:00
Haozhe Wu 9af8cb8c1e docs: log today's PR batch in CHANGELOG [Unreleased]
#457 SDM, #474 Requesty provider, #470 Binance USD-M slice 1,
#472 correlation bare tickers, #467 local loader interval, #469 FastMCP compat
2026-07-12 16:58:04 +08:00
Haozhe Wu ecd43ad7d2 release: v0.1.11 — India equity, fundamental factors, IM channels roll-up
Version bump + release paperwork: detailed CHANGELOG for 0.1.11 (plus the
missing 0.1.10 backfill), README news + counts across all 5 locales
(30 presets / 460 alphas across 5 families / 19 free sources + india_broker
row and India fallback chain / roadmap), v0.1.11 contributor cycle credits,
SKILL.md version, and the opt-in scheduler flag documented in .env.example.
2026-07-11 01:11:35 +08:00
Haozhe Wu 5a06371a94 chore(release): 0.1.9
Roll-up release since 0.1.8 — connector-first broker profiles (IBKR +
Robinhood Agentic Trading), Research Goal runtime, swarm reconcile/MCP
keepalive + worker MCP tools + strict alpha-bench + retry_run, the
agent/cli/ package refactor, the mootdx A-share loader, a robustness pass,
and the session/version fixes (#150 #156 #157 #158 + UI indicator).

Version → 0.1.9 across pyproject, SKILL.md, frontend APP_VERSION, and the
wiki docs version selector. MCP tool count corrected to 36 (adds retry_run)
across all 5 READMEs, with the ar/ko tool lists rebuilt from a stale 22 and
the ar skill count fixed 75→77. CHANGELOG [0.1.9] + 0.1.9 news entry added
to all 5 READMEs. Home backtest card data-source count corrected 3→7.
2026-06-01 23:00:43 +08:00
Haozhe Wuandykykj 74614f0e7c feat(cli): refresh interactive terminal UI
Refresh the interactive terminal transcript, activity rail, and prompt visuals. Document the 2026-05-23 CLI update in README and keep generated CLI screenshots out of git.

Co-authored-by: ykykj <140159248+ykykj@users.noreply.github.com>
2026-05-24 00:17:45 +08:00
Haozhe Wu 07e1adcbba chore(0.1.8): version, CHANGELOG, packaging, News across 5 READMEs, SKILL.md
Release prep for Alpha Zoo v1. Wraps up the 0.1.8 main-branch
content so PyPI + ClawHub publish can run from a clean checkout.

Version
- pyproject.toml: 0.1.7 → 0.1.8
- wiki/docs/content.js: DOCS_DEFAULT_VERSION 0.1.7 → 0.1.8 + 0.1.7
  retained in the version dropdown
- agent/SKILL.md frontmatter: 0.1.7 → 0.1.8

Packaging
- pyproject.toml [tool.setuptools.package-data] adds the new
  src.factors zoo asset globs (zoo/**/*.yaml, zoo/**/*.md,
  zoo/**/NOTICE) alongside the existing src and backtest entries
- MANIFEST.in adds `recursive-include agent/src/factors *.yaml *.md`
  and `include NOTICE`

CHANGELOG.md (first changelog file in the repo)
- Full 0.1.8 entry covering the Alpha Zoo (4 zoos × 452 alphas),
  Web UI + 4 REST routes, CLI subcommand suite, safety floor
  (AST purity gate + lookahead test + pytest-socket + grep gates),
  community governance (CONTRIBUTING + DCO + NOTICE + per-zoo
  LICENSE.md), and the IC/IR core extraction.
- Known limitations subsection: btc-usdt is single-asset; cross-
  sectional IC needs ≥2 instruments; crypto-majors basket planned
  for 0.2.
- [Unreleased] anchor at top for the next cycle.

agent/SKILL.md (ClawHub package descriptor)
- Frontmatter description gains Alpha Zoo + 75 skills (was 74)
- New `### Alpha Zoo (452 pre-built alphas)` section with the
  4-zoo breakdown + CLI / REST / signal-engine entry points
- Skill count 74 → 75 wherever it appears
- Trademarked-name string intentionally absent (CI gate b enforces)

5 READMEs (EN/ZH/JA/KO/AR) per `docs/alpha-zoo/readme-update-spec.md`
- Roadmap: Alpha Zoo `Planned` → `Shipped 0.1.8` with expanded
  description (452 alphas, 4 zoo sources, CLI + agent + Web UI)
- Skill count 74 → 75 (Tool category 10 → 11, `alpha-zoo` added)
- Agent tools count 29 → 31 (alpha_zoo_tool, alpha_bench_tool)
- Project structure tree: new `factors/` block (base / registry /
  bench_runner / zoo), new `api/` block (alpha_routes), new
  top-level `tools/` block (ci_grep_gates.sh). Frontend pages list
  expanded (AlphaZoo, Correlation, Settings)
- Quick Example: second snippet showing `alpha bench` one-liner
- New "What You Can Do" row for the bench-a-pre-built-alpha-zoo path
- New `<details>` block under Detailed Capabilities listing the
  4 zoos + counts + sources + license stance
- Examples → Strategy & Backtesting: 3 snippets (bench, list+show,
  multi-factor compose via ZooSignalEngine.from_zoo)
- API Server route table: 4 `/alpha/*` rows (list, get, bench,
  bench stream SSE)
- CLI Reference: alpha subcommand introduced in the top block;
  single-run examples block extended with 3 alpha commands

`### Key Features` 4-card grid (#7) and Demo section (#13) are
intentionally NOT touched here — they need layout redesign + new
GIFs/screenshots; design notes in docs/alpha-zoo/readme-redesign-7-13.md
will be implemented in a follow-up PR.

969 tests pass; CI grep gates green (no yaml.load misuse, no
trademarked-name leak, no per-stock-code data in wiki); TypeScript
clean on the frontend.
2026-05-17 20:19:04 +08:00