Concatenated pairs like BTCUSDT are alphabetic, so isalpha() labeled them
as US equities. Recognize USDT/USDC/BUSD suffixes before that branch.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Broker CSV cells like $1,234.56 or ¥1000 failed float() after only commas
were stripped, so price/fee silently became 0.0. Strip common currency
tokens and unicode minus too.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Notepad/Excel Unicode text exports start with a UTF-16 BOM; latin-1 never
raises so those bytes were accepted as mojibake. Detect BOM and decode as
utf-16 first.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
After loaders started accepting 1h/4h/1d, calc_bars_per_year still keyed
only 1H/1D, so hourly runs were annualised as daily (Sharpe/vol wrong).
Normalize interval case and alias yahoo→yfinance / binance→ccxt.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Non-ASCII skill names such as 动量策略 were scrubbed to "--", so distinct
Chinese names collided into one directory and overwrote each other.
Allow the same non-Latin ranges as persistent memory, and hash when the
slug would otherwise be only hyphens.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Excel "CSV UTF-16" / Unicode CSV writes a BOM that utf-8/gbk cannot
decode, so analyze_trade_journal rejected the file. Try utf-16 in the
encoding fallback before raising.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Two fixes that make historical funding actually flow into perp backtests:
1. Loader: Binance funding settlement timestamps carry millisecond
jitter (e.g. 2026-05-05T08:00:00.011Z). The raw timestamps never
intersect bar timestamps, so the missing-settlement guard raises
'funding settlement data is missing' for most real fetches. Round
settlement timestamps to the second before aligning.
2. Engine: the loader writes a historical funding_rate column into the
frame, but calc_crypto_funding_fee still charges the fixed config
rate (default 0.0001) — the fetched data is never consumed. Prefer
the bar's historical rate when present and finite; fall back to the
fixed rate otherwise (spot-proxy and daily-fallback paths keep
their existing behaviour).
Verified against live Binance USD-M data (BTC/ETH -USDT-PERP, 4H,
Jan-Jul 2026): fixed-rate short returned +44.5%, historical-funding
short +39.1% — the ~5pp delta matches 2026's predominantly negative
funding regime where shorts pay.
Tests: jitter alignment test in test_ccxt_perpetual_loader.py; three
override/fallback tests in test_crypto_engine.py.
AI-assisted-by: Claude (Anthropic)
Signed-off-by: Gordon Lee <xnegra80@gmail.com>
bench_runner.run_bench forwards the sp500 loader's survivorship_bias flag
as result["meta"] (bench_runner.py L319-347), and
alpha_routes._result_for_wire already keeps that key for the SSE/frontend
path (alpha_routes.py L777). cmd_alpha_bench built its own JSON envelope
and HTML report context from hand-enumerated dict literals that never
included "meta", so the CLI and the HTML report silently dropped the
disclosure while the web UI kept it.
Forward result.get("meta") into both dict literals when present, mirroring
how the wire path already treats it: an opaque passthrough, not a new
rendered field. Verified with a fresh regression test
(agent/tests/test_alpha_bench_meta_survivorship.py) that fails on main and
passes on this branch, plus a third case asserting the key stays absent
when the loader supplies no meta.
Fixes#797
Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
_make_jsonable only converted Pydantic BaseModel instances (via
model_dump), list, and dict. fastmcp's client auto-types
CallToolResult.data from a tool's output schema via
json_schema_to_type, which produces a plain stdlib dataclass (not a
BaseModel) whenever the schema doesn't set additionalProperties: true
-- the common case for a typical Pydantic-model tool return type. Such
an instance fell through to the final `return value` unconverted, so
json.dumps's `default` callback received the same unconvertible object
back and raised ValueError: Circular reference detected (its cycle
detector can't distinguish "default() gave up" from an actual cycle).
Adds a dataclasses.is_dataclass branch that recurses per-field (not a
raw asdict(), which would leave nested datetime/dataclass values
unconverted), plus a datetime/date branch, since a date-time schema
field validates to a real datetime object and would hit the same
failure one level down.
Reproduced against the real fastmcp dependency using an MCP tool's
output schema shaped like this (plain object properties, no
additionalProperties, nested datetime field): old _make_jsonable raises
the reported Circular reference detected error; the patched version
serializes correctly, verified both from direct construction and via
pydantic.TypeAdapter validation (the path fastmcp's client actually
uses).
Signed-off-by: ShaunM <echoandelementwebsites@gmail.com>
Co-authored-by: ShaunM <echoandelementwebsites@gmail.com>
Pure, immutable state and risk contracts for historical USD-M perpetual
backtests: MaintenanceBracket/MaintenanceSchedule, PositionState,
AccountState, ExecutionFrame, MarketRiskFrame. No engine wiring yet — that's
a later slice.
Reapplied against the current data contract rather than cherry-picked from
the old codex/perp-engine-fidelity branch, which predates #757 and still
carries the pre-#470 loader shape. Two real changes from that branch's
version:
- MaintenanceBracket gains bracket_tier (ordering) and optional
notional_coefficient, matching the loader's validated artifact tier
fields exactly (backtest.loaders.ccxt_loader._validate_bracket_artifact).
- MaintenanceSchedule drops its own independent SHA-256 hash computation
in favor of a single caller-supplied version string — the loader's
maintenance_bracket_version content hash. Content integrity is already
checked once, at the artifact-validation boundary; this class only
re-checks structural invariants (tier and notional-cap ordering) as
defense in depth, not a second, incompatible hash scheme.
MaintenanceSchedule.from_loader_columns(symbol, maintenance_brackets,
maintenance_bracket_version) builds a schedule straight from the
loader's DataFrame columns.
MarketRiskFrame.schedule is Optional: a -PERP fetch without a supplied
bracket artifact still produces a usable execution/mark/funding frame.
Part of #462's PR 2 (margin-risk model), sliced small per maintainer
guidance. Isolated/cross liquidation logic and engine integration are
separate follow-up slices.
Signed-off-by: zygora <122336305+honginp@users.noreply.github.com>
broker_sdk connectors (Longbridge, etc.) return a balances list but no
accounts key, causing the CLI to print "Accounts: (none)" before the
balances table. Fall back to result.profile_id — which _with_profile()
always injects — so the label is informative.
Closes#846
Signed-off-by: Robin1987China <41602358+Robin1987China@users.noreply.github.com>
str.strip('|') dropped leading/trailing empty cells in pipe tables, so
row-label columns and trailing blanks collapsed when rendering for Signal.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
str.strip('|') dropped leading/trailing empty cells in pipe tables, so
row-label columns and trailing blanks collapsed when rendering for Telegram.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Unknown periods such as 1H were silently served as daily candles. Fail
closed with an explicit unsupported-period error instead.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Project-style 1H fell through to daily K lines, and 4h incorrectly mapped
to K_60M instead of K_240M.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Generic CSV/XLSX loads stringify serial day numbers, which then failed
to parse and became NaT in the standardized journal frame.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
load_dataframe reads CSV/XLSX as strings, so Date serials never hit the
numeric Excel path and stayed as raw "45321.0".
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
load_dataframe reads CSV/XLSX as strings, so serial day numbers never
hit the numeric Excel path and stayed as raw "45321.375".
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Portfolio Studio step 2 (#456): per-name cap, per-name floor, and per-group exposure caps as a shared layer on top of every optimizer, not just turnover_aware's built-in bounds. Configured via constraints in config.json, off by default.
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Markets like European day-ahead power clear negative routinely, but the
loader's validate_ohlc drops any bar with a non-positive OHLC value and the
engine rejects any open at price <= 0. On real EU power data this silently
deletes ~27% of DE-LU days (every bar with a sub-zero hourly-derived low),
punching holes in rolling statistics (e.g. a z-score entry signal) with no
warning.
Add an opt-in `allow_nonpositive_prices` flag (default False — existing
behavior byte-identical for all positive-price markets):
- loaders/base.py: validate_ohlc gains the flag. Structural invariants
(high<low, high/low bracketing open/close) always enforced; when the flag
is set, negative prices pass through and only an exactly-zero price is
rejected. Zero is genuinely undefined for `size = notional / price` and
margin; a negative price is well-defined via abs().
- engines/base.py: BaseEngine reads the flag from config. _plan_open_order
rejects an exactly-zero open always, negatives only when the flag is off.
_calc_raw_size / _calc_margin divide by abs(price) so a negative entry
price yields a positive size magnitude (direction carries the sign) and
positive collateral — unchanged for positive prices. _calc_pnl already
uses a price difference, well-defined through zero.
Tests open a position on a negative-price bar and assert size/margin/PnL are
well-defined through zero, that an exactly-zero price is still rejected with
the flag on, and that the default still rejects negatives. A regression test
pins *why* zero stays dropped rather than kept: a surviving 0.00 close yields
inf at the `close.pct_change().fillna(0.0)` sites in benchmark.py and
engines/base.py (fillna fills NaN, not inf), so keeping it would corrupt the
return series. Fixture carries the 5 real non-positive-close bars from a 2025
NO2/DE-LU window (ENTSO-E via Energy-Charts / Bundesnetzagentur | SMARD,
CC BY 4.0). Existing engine and OHLC-validation suites pass unchanged
(140 tests total).
Co-authored-by: Ali Emadi <ali@gatekeeperai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
load_dataframe reads cells as strings, so day-count serials like
"44941.0" skipped the YYYYMMDD float path and stayed as raw date text.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Yahoo chart has no 4h interval; project 4H was lowercased to 4h and
sent unsupported to the API. Match yfinance's 4H → 1h approximation.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Docs say daily-only, but 1H/4H still logged in and returned frequency=d
bars. Reject non-daily intervals before login (tencent/fmp parity).
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
TIME_SERIES_DAILY is daily-only, but 1H/4H still hit the API and
returned day bars. Reject non-daily intervals before the request.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Finnhub always called resolution=D even for 1H/4H, so intraday runs
silently got day bars and stopped the fallback chain.
Reject unsupported intervals like the other daily-only loaders.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Tiingo always hit the daily EOD endpoint even for 1H/4H, so intraday
runs silently got day bars and stopped the fallback chain.
Reject unsupported intervals like the other daily-only loaders.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Stooq always fetched the daily CSV even for 1H/4H, so intraday runs
silently got day bars and stopped the fallback chain.
Reject unsupported intervals like tencent/fmp/india-broker.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Connector-style 1h/4h fell through the uppercase-only allowlist and were
substituted with 1D. Map the aliases like the OKX connector bar map, and
reject unknown intervals instead of silently fetching daily bars.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Lowercasing first left 1m/5m/15m/30m with empty wanted tokens, so daily
OHLCV capabilities outranked minute ones. Match minute tokens
case-sensitively and keep 1M as monthly.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
* fix(loaders): reject non-daily intervals for AKShare US/HK/ETF/forex
A-share already raised on unsupported intervals, but US/HK/ETF/forex
always called the daily endpoints, so interval=1H returned day bars
under a 1H cache key and stopped the fallback chain.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
* fix(loaders): tighten AKShare daily-only interval gate comment
---------
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Connector-style 1h/1d were rejected while project 1H/1D mapped to the
same broker tokens, so market_data and mixed-vocabulary callers got
empty frames for valid hour/day bars.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Connector-style hour/day tokens raised unsupported while 1H/4H/1D
already mapped to K_60M/K_240M/K_DAY, so mixed-vocabulary callers lost
intraday Futu history.
Signed-off-by: santhreal <64453045+santhreal@users.noreply.github.com>
Haozhe Wu