Commit Graph
1119 Commits
Author SHA1 Message Date
santhreal c671be6fba merge(main): sync before thermo hold 2026-07-22 12:21:18 -07:00
santhreal 87cab5ffa5 merge(main): sync before thermo hold 2026-07-22 12:21:15 -07:00
Haozhe Wu ef04bee26c Merge PR #777: feat(backtest): portfolio risk x-ray core + agent tool (Portfolio Studio slice)
feat(backtest): portfolio risk x-ray core + agent tool (Portfolio Studio slice)
2026-07-22 23:30:59 +08:00
Haozhe Wu 329b054128 docs(readme): backfill longbridge data-source parity in ja/ar
ja/ar were missing the longbridge Data Sources table row and its place in
the US/HK fallback chains (a pre-0.1.12 drift; en/ko/zh already had them).
Now all 5 locales agree on the 23-source table + chains. (The en-only
'Using Longbridge explicitly' config subsection remains en-only.)
2026-07-22 23:19:10 +08:00
Haozhe Wu 761d52155c docs(readme): sync all 5 READMEs to v0.1.12 features + counts
News: 🚀 v0.1.12 entry (3-visible, demote 07-19). Counts synced to code:
462 alphas (academic 12), 88 skills, 23 data sources, 12 broker connectors.
Data Sources: add binance + mt5 rows + forex/metals fallback chain. New
Broker Connectors (12) and Backtest Engines (8) capability tables. Providers
list + native Anthropic / SiliconFlow / NVIDIA. Roadmap: SDM + Correlation
Regime rows. en/ja/ko English descriptions; zh/ar translated.
2026-07-22 22:47:39 +08:00
Yufeng He 9889a74320 feat(backtest): portfolio risk x-ray core + agent tool (Portfolio Studio slice)
First slice of the Portfolio Studio roadmap item ("risk x-ray ... rest
Planned"): a pure-computation risk x-ray for weighted baskets, plus an
agent-facing tool that fetches closes through the standard loader
fallback chain.

backtest/risk_xray.py computes, over trailing returns only (no look-ahead
by construction): concentration (HHI / effective N / top-1 / top-3),
annualized and downside volatility, max drawdown with dates, historical
VaR + expected shortfall at 95/99, diversification ratio, and pairwise
correlation / beta vs an equal-weight proxy. Weights are validated and
renormalized with a warning; thin symbols fall out through a min-history
filter; every emitted float is finite-guarded so the payload always
survives json.dumps(allow_nan=False). Long-only for now — a short leg
changes concentration semantics and deserves its own slice.

The portfolio_risk_xray tool handles argument validation, shapes the
fetch envelope into a close panel (records with missing or mixed
timestamps fall back to loader order), and returns a strict-JSON
envelope with unresolved/skipped bookkeeping. Data fetching is
injectable for tests.

17 new tests cover the math on hand-built panels, the edge matrix
(renormalization, thin symbols, single asset, zero variance, bad
args), and the tool end to end with a stub fetcher. Full suite:
6037 passed, 12 skipped.

Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
2026-07-22 22:07:41 +08:00
shadowinlife 7adb2c6fb8 fix(mt5): convert numpy scalars to native Python types for JSON serialization
MetaTrader5 SDK copy_rates_from_pos() returns numpy structured arrays
containing numpy.int64/float64 scalars. These types are not natively
JSON-serializable, causing TypeError when trading_history responses
are serialized downstream.

Add _int_or_none() and _float_or_none() helpers in reads.py and apply
them to all numeric fields in _rate_to_dict(). Add regression tests
covering numpy, plain-dict, and None input scenarios.

Closes #774

Signed-off-by: shadowinlife <shadowinlife@gmail.com>
2026-07-22 21:25:31 +08:00
Haozhe Wu 43331c3221 docs(readme): v0.1.12 contributor credits across all 5 READMEs
Add the v0.1.12 cycle cohort (28 contributors) and demote the v0.1.11 list
into a collapsible block, in en/ja/ko/zh/ar. en/ja/ko keep English
descriptions; zh/ar translated.
v0.1.12
2026-07-22 21:13:25 +08:00
Haozhe Wu 8cb536fa61 chore(release): 0.1.12 — version bump + CHANGELOG
Bump version 0.1.11 → 0.1.12 across pyproject, frontend package(+lock),
i18n app.version, SKILL.md, and Dockerfile OCI label. CHANGELOG [0.1.12]
covers the 0.1.11→0.1.12 window (correlation regime timeline, 3 new LLM
providers, MT5 connector, SDM skill, USD-M perps, ~90 correctness fixes,
and the #758 zai streaming/base_url fix).
2026-07-22 13:14:11 +08:00
Haozhe Wu a066e5545e fix(providers): resolve zai streaming + base_url resolution (#758)
Three fixes for the native zai provider on glm-5.1:

- get_llm_credentials now falls back to the provider catalog's
  default_base_url when no *_BASE_URL is set, so CLI/manual-.env users
  reach the canonical endpoint (Z.ai /api/coding/paas/v4) instead of
  silently defaulting to api.openai.com. Web Settings already did this;
  the backend credential path did not. Benefits all 22 providers.
- stream_chat falls back to a non-streaming invoke when a provider
  streams zero chunks ('No generation chunks were returned'), instead of
  raising ProviderStreamError. Scoped to the empty-stream case; genuine
  provider errors still raise.
- ProviderStreamError appends a base-URL hint when the endpoint returns
  an HTML error page (a common misconfiguration signal).

Fixes #758.
2026-07-22 12:59:07 +08:00
klmtseng 4e7ddc5e4a fix(fundamentals): dedup restated rows and prevent period regression in PIT snapshots
Two point-in-time correctness bugs in tushare_fundamentals.py:

1. `query_fundamentals` did not deduplicate after the PIT cut, so the same
   (ts_code, end_date) could appear twice when both the original filing and a
   later restatement were visible as of the query date.  Fix: after the PIT
   cut, keep only the row with the latest effective pit date per
   (ts_code, end_date); ties preserve the last row in input order (natural
   Tushare ordering where restatements follow originals).

2. `enrich_price_frames_with_fundamentals` fed the deduped snapshot directly
   into merge_asof.  This caused two problems: (a) the per-day PIT visibility
   was evaluated on already-deduped data, making a restatement whose pit_date
   was in the future appear as if it were always visible; (b) when a late
   restatement for an *older* period (end_date < currently-visible end_date)
   arrived, merge_asof picked it as the nearest-prior announcement, regressing
   every subsequent snapshot to the older period.

   Fix: enrich now calls the new internal `_query_pit_cut` (raw PIT cut, no
   dedup) and builds an explicit forward-only effective timeline — a row is
   only accepted into the timeline when its end_date >= the currently-visible
   end_date.  Same-period restatements (same end_date, later pit_date) pass
   this check and appear as additional timeline entries; merge_asof then
   surfaces the restatement value only from its own pit_date onward.

Fixes #771

Signed-off-by: klmtseng <20399135+klmtseng@users.noreply.github.com>
2026-07-22 12:52:45 +08:00
Haozhe Wu f2c7256d56 Merge PR #756: feat(correlation): regime timeline endpoint and UI (edge density + hysteresis) 2026-07-22 12:51:20 +08:00
Haozhe Wu 93fa241fc8 Merge PR #757: feat(data): decouple maintenance-bracket acquisition from the historical fetch path 2026-07-22 12:51:19 +08:00
Haozhe Wu f06d399388 Merge PR #770: fix(journal): normalize Excel float-stringified A-share codes 2026-07-22 12:51:08 +08:00
Haozhe Wu 5eb45502c7 Merge PR #769: fix(tools): accept unicode dashes in PDF page ranges 2026-07-22 12:51:07 +08:00
Haozhe Wu d538bfa369 Merge PR #768: fix(api): treat export KEY= as KEY in dotenv settings helpers 2026-07-22 12:51:04 +08:00
Haozhe Wu 3f0e6b8c94 Merge PR #767: fix(factors): keep single-day IC std finite in factor analysis JSON 2026-07-22 12:51:03 +08:00
Haozhe Wu f1697117c1 Merge PR #766: fix(backtest): print scalar metrics as strict JSON 2026-07-22 12:51:02 +08:00
Haozhe Wu 566eb86ba3 Merge PR #765: fix(metrics): keep Sharpe/Sortino finite when equity hits zero mid-path 2026-07-22 12:51:02 +08:00
Haozhe Wu be0d0a7c2c Merge PR #764: fix(pattern): emit strict JSON when trend_slope mean is undefined 2026-07-22 12:51:01 +08:00
Haozhe Wu 6fffd52541 Merge PR #763: fix(session): skip schema-bad messages.jsonl lines when reading 2026-07-22 12:50:59 +08:00
Haozhe Wu e8241034e9 Merge PR #762: fix(session): skip corrupt session.json when listing sessions 2026-07-22 12:50:58 +08:00
Haozhe Wu c350cddf8c Merge PR #761: fix(loaders): keep yahoo 1m as minute bars, not monthly 2026-07-22 12:50:57 +08:00
santhreal 65e624ab8d fix(connectors): accept project 1H/4H periods on OKX bars
_BAR_MAP only listed lowercase 1h/4h, so runner-style 1H/4H fell
through to bar 1D and silently returned daily candles.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:27:34 -07:00
santhreal d4d4e2762b fix(connectors): accept project 1H/4H periods on Alpaca bars
_rest_timeframe only listed lowercase 1h/4h, so runner-style 1H/4H
fell through to 1Day and silently returned daily bars.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:27:33 -07:00
santhreal 92f0f6e35d fix(cli): keep empty edge columns when parsing markdown tables
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:26:40 -07:00
santhreal 90def21855 fix(feishu): keep empty edge columns when parsing markdown tables
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:26:39 -07:00
santhreal 3ab407d799 fix(loaders): reject unsupported akshare a-share intervals
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:26:37 -07:00
santhreal 78650f74aa fix(tiger): accept project-style 1H/4H period tokens
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:26:36 -07:00
santhreal d69be31f10 fix(loaders): map ccxt 1W/1M intervals instead of silent daily
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:26:34 -07:00
santhreal f14e701501 fix(loaders): map yfinance 1M to monthly bars, not minutes
Unmapped intervals were lowercased, so project monthly token 1M became
yfinance 1m (one minute). Map 1M/1W explicitly like the yahoo loader.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:25:16 -07:00
santhreal 19c1cbc6e4 fix(loaders): reject unsupported india broker intervals instead of daily
Unknown intervals (including runner 4H) fell through to broker period 1d,
so hourly requests silently returned daily bars. Reject and return no data.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:25:15 -07:00
santhreal 0f3a2d3aa0 fix(validation): tolerate blank holding_days cells in trades.csv
Blank holding_days become NaN under pandas and int(NaN) aborted the
validation CLI. Coerce non-numeric/blank values to 0 holding bars.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:25:13 -07:00
santhreal 9563c1a110 fix(tools): emit JSON-safe diffs when report audit hits a zero reported value
When reported_value is 0 and fetched_value is not, verdict used float('inf')
and json.dumps emitted non-RFC Infinity tokens. Record null percents instead
and serialize with allow_nan=False.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:25:12 -07:00
santhreal 4c6f23cb25 fix(journal): normalize Excel float YYYYMMDD dates in eastmoney parser
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:19:15 -07:00
santhreal 2b01bf46d0 fix(journal): normalize float-stringified A-share codes before qualifying
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:19:15 -07:00
santhreal 91c7cb23da fix(journal): normalize Excel float-stringified A-share codes
Numeric CSV/Excel cells often stringify as 600519.0; the '.' made
_qualify_a_share treat them as already-qualified and skip .SH/.SZ.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:18:35 -07:00
santhreal ab3e8076d6 fix(tools): accept unicode dashes in PDF page ranges
Word/LLM paste often uses en/em/minus dashes in ranges like 1–10;
those were silently dropped because only ASCII '-' was recognized.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:18:22 -07:00
santhreal c0f794c2c9 fix(api): treat export KEY= as KEY in dotenv settings helpers
Shell-style export KEY=value lines were read as a literal key named export KEY, so Settings missed configured secrets and upserts appended a duplicate KEY= instead of updating the export line. Strip the export prefix on read/write like python-dotenv.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:17:28 -07:00
santhreal e76e1dc6fb fix(factors): keep single-day IC std finite in factor analysis JSON
One IC observation makes Series.std() (ddof=1) NaN, which leaked into ic_summary.json and the tool envelope. Treat single-sample std as 0.0 (same guard as metrics vol) and dump with allow_nan=False.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:17:25 -07:00
santhreal d88a591272 fix(backtest): print scalar metrics as strict JSON
Explosive equity paths can set annual_return to +inf. The engine stdout
dump used default allow_nan=True and emitted bare Infinity tokens.
Sanitize non-finite scalars to null and dump with allow_nan=False,
matching options and run_card.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:16:54 -07:00
santhreal 3074b684e1 fix(metrics): keep Sharpe/Sortino finite when equity hits zero mid-path
A 100 → 0 → 50 equity path yields non-finite pct_change values that
poisoned Sharpe/Sortino/IR with NaN/inf. Skip those ratios (report 0)
when bar returns are non-finite, matching the options metrics path.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:16:52 -07:00
santhreal bd610d0b67 fix(pattern): emit strict JSON when trend_slope mean is undefined
Halt-style NaN gaps can leave every polyfit window empty, so mean_slope
became NaN and leaked a bare NaN token into the tool JSON. Treat that as
0.0 and serialize with allow_nan=False, matching options_pricing.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:16:48 -07:00
santhreal afd0046cbd fix(session): skip schema-bad messages.jsonl lines when reading
get_messages already skipped JSONDecodeError lines but still raised on
valid JSON that was not a Message mapping (null, extra keys). Skip those
too so later good lines remain readable.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:13:47 -07:00
santhreal 0b9df59cfe fix(session): skip corrupt session.json when listing sessions
One unreadable or schema-invalid session.json raised during list_sessions
and aborted the whole listing; skip that entry and keep siblings.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:13:46 -07:00
santhreal 13f71e5a6a fix(loaders): keep yahoo 1m as minute bars, not monthly
_to_yahoo_interval uppercased intervals before the map lookup, so project
interval 1m collided with monthly 1M and was sent to Yahoo as 1mo.

Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
2026-07-21 21:13:46 -07:00
shadowinlife d92e362cd6 style(memory): apply black formatting per CONTRIBUTING.md
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
2026-07-21 17:13:39 +08:00
shadowinlife 6f9e304987 test(memory): add retrieval quality benchmark suite
Add P@5/MRR/NDCG@5 evaluation framework for memory retrieval A/B testing.
Benchmark compares Baseline (pure token overlap) vs Treatment (BM25 + importance
decay) on a 200-entry corpus with 50 annotated queries.

The benchmark suite is NOT intended for CI — it requires corpus files
(tmp/benchmark_corpus/) that are not version-controlled. The suite
automatically skips when corpus files are absent.

Corpus is derived from anonymized production agent sessions.

Usage:
    python -m pytest agent/tests/memory/benchmarks/ -v

Refs: #732
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
2026-07-21 16:18:21 +08:00
zygora 5f9144946d feat(data): decouple maintenance-bracket acquisition from the historical fetch path
binanceusdm.fetch_leverage_tiers() routes to Binance's signed
GET /fapi/v1/leverageBracket USER_DATA endpoint and requires an API key,
even though the response carries no account-specific data. #716 called it
unconditionally for every -PERP fetch, so on merged main today any
perpetual fetch hard-crashes for callers without a Binance key.

- Remove the unconditional fetch_leverage_tiers() call (and the now-unused
  _fetch_maintenance_brackets live-fetch method entirely). Execution/mark
  OHLC and funding settlements go back to zero-credential, matching the
  rest of the stack.
- Introduce a caller-supplied, versioned bracket artifact as the only way
  to attach maintenance_brackets/maintenance_bracket_version: schema
  version, symbol, provenance timestamp, per-tier fields plus an optional
  notional_coefficient, and a content hash validated against the
  canonical bracket JSON. Never fetched live.
- fetch(..., bracket_artifacts={code: artifact}) attaches brackets only
  when a valid matching artifact is supplied for that code.
- fetch(..., require_brackets=True) makes a strict margin-risk consumer
  fail closed with an actionable error before any network call when no
  valid artifact is supplied for a -PERP code.

Per #462 / #716 follow-up discussion. No real bracket snapshot is
committed; test fixtures are synthetic. No Binance credentials are added
to the backtest path.

Signed-off-by: zygora <122336305+honginp@users.noreply.github.com>
2026-07-21 15:50:38 +08:00
shadowinlife 9ad5c9acf4 chore: remove test_hotness_decay.py (hotness decay out of scope)
Signed-off-by: shadowinlife <shadowinlife@gmail.com>
2026-07-21 15:22:31 +08:00