feat: data source expansion with auto-fallback + web search + skill categorization
- Add DataLoader Protocol + Registry with market-level fallback chains - New data sources: AKShare (A-shares/US/HK/futures, free) and CCXT (100+ crypto exchanges) - Auto-fallback: tushare unavailable -> akshare; OKX unavailable -> CCXT - Zero-config: all markets work without any API keys - New web_search tool (DuckDuckGo) for web information retrieval - Skill categorization: 68 skills organized into 7 categories (data-source, strategy, analysis, asset-class, crypto, flow, tool) - New data-routing meta-skill for intelligent data source selection - Registry self-initializes via _ensure_registered() — no manual imports needed - Pattern detection returns market type instead of source name (decoupled) - runner.py loads .env via dotenv for reliable token access
This commit is contained in:
warren618 committed
1 parent
dca916c1bd
commit
27354eb008
82 files changed
+1092
-85
No files matched your search
@@ -15,10 +15,10 @@
|
||||
<a href="https://pypi.org/project/vibe-trading-ai/"><img src="https://img.shields.io/pypi/v/vibe-trading-ai?style=flat&logo=pypi&logoColor=white" alt="PyPI"></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow?style=flat" alt="License"></a>
|
||||
<br>
|
||||
<img src="https://img.shields.io/badge/Skills-65-orange" alt="Skills">
|
||||
<img src="https://img.shields.io/badge/Skills-68-orange" alt="Skills">
|
||||
<img src="https://img.shields.io/badge/Swarm_Presets-29-7C3AED" alt="Swarm">
|
||||
<img src="https://img.shields.io/badge/Tools-20-0F766E" alt="Tools">
|
||||
<img src="https://img.shields.io/badge/Markets-3-2563EB" alt="Markets">
|
||||
<img src="https://img.shields.io/badge/Tools-21-0F766E" alt="Tools">
|
||||
<img src="https://img.shields.io/badge/Data_Sources-5-2563EB" alt="Data Sources">
|
||||
<br>
|
||||
<a href="https://github.com/HKUDS/.github/blob/main/profile/README.md"><img src="https://img.shields.io/badge/Feishu-Group-E9DBFC?style=flat-square&logo=feishu&logoColor=white" alt="Feishu"></a>
|
||||
<a href="https://github.com/HKUDS/.github/blob/main/profile/README.md"><img src="https://img.shields.io/badge/WeChat-Group-C5EAB4?style=flat-square&logo=wechat&logoColor=white" alt="WeChat"></a>
|
||||
@@ -44,7 +44,8 @@
|
||||
|
||||
## 📰 News
|
||||
|
||||
- **2026-04-09** 🔧 Multi-market backtest engines with per-market rules; **Pine Script v6 export** — convert strategies to TradingView in one command (`/pine`, API, frontend viewer).
|
||||
- **2026-04-09** 🔧 **Data source expansion**: 5 data sources with auto-fallback (tushare, yfinance, OKX, AKShare, CCXT) — zero-config backtesting across A-shares, US/HK equities, crypto, futures, and forex. Added `web_search` tool (DuckDuckGo), skill categorization (7 categories), and `data-routing` meta-skill for intelligent source selection.
|
||||
- **2026-04-08** 🔧 Multi-market backtest engines with per-market rules; **Pine Script v6 export** — convert strategies to TradingView in one command (`/pine`, API, frontend viewer).
|
||||
- **2026-04-01** 🚀 Released **v0.1.0** — Initial release: ReAct agent, 64 skills, 29 swarm presets, cross-market backtest, CLI + Web UI + MCP server.
|
||||
|
||||
---
|
||||
@@ -55,7 +56,7 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
|
||||
|
||||
### Key Capabilities:
|
||||
• **Strategy Generation** — Automatically writes trading code from your ideas<br>
|
||||
• **Smart Data Access** — Pulls market data from multiple sources seamlessly<br>
|
||||
• **Smart Data Access** — 5 data sources with automatic fallback; zero-config for all markets<br>
|
||||
• **Performance Testing** — Tests your strategies against historical market data<br>
|
||||
• **TradingView Export** — One-click convert strategies to Pine Script v6 for TradingView<br>
|
||||
• **Expert Teams** — Deploys specialized AI agents for complex research tasks<br>
|
||||
@@ -70,7 +71,7 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
|
||||
<td align="center" width="25%" valign="top">
|
||||
<img src="assets/scene-research.png" height="150" alt="Research"/><br>
|
||||
<h3>🔍 DeepResearch for Trading</h3>
|
||||
<img src="https://img.shields.io/badge/65_Skills-FF6B6B?style=for-the-badge&logo=bookstack&logoColor=white" alt="Skills" /><br><br>
|
||||
<img src="https://img.shields.io/badge/68_Skills-FF6B6B?style=for-the-badge&logo=bookstack&logoColor=white" alt="Skills" /><br><br>
|
||||
<div align="left" style="font-size: 4px;">
|
||||
• Multi-domain analysis coverage across markets<br>
|
||||
• Auto strategy and signal generation<br>
|
||||
@@ -92,10 +93,10 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
|
||||
<td align="center" width="25%" valign="top">
|
||||
<img src="assets/scene-backtest.png" height="150" alt="Backtest"/><br>
|
||||
<h3>📊 Cross-Market Backtest</h3>
|
||||
<img src="https://img.shields.io/badge/3_Markets-FFD93D?style=for-the-badge&logo=bitcoin&logoColor=black" alt="Backtest" /><br><br>
|
||||
<img src="https://img.shields.io/badge/5_Data_Sources-FFD93D?style=for-the-badge&logo=bitcoin&logoColor=black" alt="Backtest" /><br><br>
|
||||
<div align="left">
|
||||
• A-shares, HK/US equities & crypto<br>
|
||||
• 7 flexible intervals from 1 min to 1 day<br>
|
||||
• A-shares, HK/US equities, crypto, futures & forex<br>
|
||||
• Auto-fallback: always finds a working source<br>
|
||||
• Daily & options portfolio engines<br>
|
||||
• 15+ performance metrics & 4 optimizers
|
||||
</div>
|
||||
@@ -114,22 +115,21 @@ Vibe-Trading is an AI-powered multi-agent finance workspace that turns natural l
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 64 Skills across 8 Core Finance Domains
|
||||
## 68 Skills across 7 Categories
|
||||
|
||||
- 📊 65 specialized finance skills across 8 core domains
|
||||
- 📊 68 specialized finance skills organized into 7 categories
|
||||
- 🌐 Complete coverage from traditional markets to crypto & DeFi
|
||||
- 🔬 Comprehensive capabilities spanning technical analysis to quantitative research
|
||||
- 🔬 Comprehensive capabilities spanning data sourcing to quantitative research
|
||||
|
||||
| Area | Examples |
|
||||
|------|----------|
|
||||
| Technical Analysis | `technical-basic`, `candlestick`, `ichimoku`, `harmonic`, `elliott-wave`, `smc` |
|
||||
| Quant Research | `strategy-generate`, `pine-script`, `multi-factor`, `pair-trading`, `factor-research`, `ml-strategy` |
|
||||
| Fundamentals | `valuation-model`, `financial-statement`, `earnings-forecast`, `credit-analysis` |
|
||||
| HK/US Equities | `edgar-sec-filings`, `earnings-revision`, `us-etf-flow`, `hk-connect-flow`, `adr-hshare` |
|
||||
| Crypto Desk | `perp-funding-basis`, `liquidation-heatmap`, `stablecoin-flow`, `token-unlock-treasury`, `defi-yield` |
|
||||
| Macro | `macro-analysis`, `global-macro`, `asset-allocation`, `commodity-analysis`, `sector-rotation` |
|
||||
| Derivatives | `options-strategy`, `options-advanced`, `options-payoff`, `crypto-derivatives`, `hedging-strategy` |
|
||||
| Alt Data | `social-media-intelligence`, `sentiment-analysis`, `behavioral-finance`, `report-generate` |
|
||||
| Category | Skills | Examples |
|
||||
|----------|--------|----------|
|
||||
| Data Source | 6 | `data-routing`, `tushare`, `yfinance`, `okx-market`, `akshare`, `ccxt` |
|
||||
| Strategy | 16 | `strategy-generate`, `technical-basic`, `candlestick`, `ichimoku`, `elliott-wave`, `smc`, `multi-factor`, `ml-strategy` |
|
||||
| Analysis | 15 | `factor-research`, `macro-analysis`, `global-macro`, `valuation-model`, `earnings-forecast`, `credit-analysis` |
|
||||
| Asset Class | 9 | `options-strategy`, `options-advanced`, `convertible-bond`, `etf-analysis`, `asset-allocation`, `sector-rotation` |
|
||||
| Crypto | 7 | `perp-funding-basis`, `liquidation-heatmap`, `stablecoin-flow`, `defi-yield`, `onchain-analysis` |
|
||||
| Flow | 7 | `hk-connect-flow`, `us-etf-flow`, `edgar-sec-filings`, `financial-statement`, `adr-hshare` |
|
||||
| Tool | 8 | `backtest-diagnose`, `report-generate`, `pine-script`, `doc-reader`, `web-reader` |
|
||||
|
||||
## 29 Agent Swarm Team Presets
|
||||
|
||||
@@ -215,7 +215,7 @@ vibe-trading-mcp # start MCP server (stdio)
|
||||
- **Python 3.11+** for Path B
|
||||
- **Docker** for Path A
|
||||
|
||||
> **Tip:** yfinance (HK/US equities) and OKX (crypto) are free and require no API key. You only need a Tushare token for A-share data.
|
||||
> **Tip:** All markets work without any API keys thanks to automatic fallback. yfinance (HK/US), OKX (crypto), and AKShare (A-shares, US, HK, futures, forex) are all free. Tushare token is optional — AKShare covers A-shares as a free fallback.
|
||||
|
||||
### Path A: Docker (zero setup)
|
||||
|
||||
@@ -291,10 +291,11 @@ Edit `agent/.env`:
|
||||
| `OPENAI_BASE_URL` | Usually | API gateway URL (default: `https://openrouter.ai/api/v1`) |
|
||||
| `LANGCHAIN_PROVIDER` | Yes | LLM provider selector (e.g. `openrouter`) |
|
||||
| `LANGCHAIN_MODEL_NAME` | Yes | Model name (e.g. `deepseek/deepseek-v3.2`) |
|
||||
| `TUSHARE_TOKEN` | A-shares only | Tushare Pro token for A-share data |
|
||||
| `TUSHARE_TOKEN` | No | Tushare Pro token for A-share data (falls back to AKShare if missing) |
|
||||
| `CCXT_EXCHANGE` | No | CCXT exchange name, default `binance` |
|
||||
| `TIMEOUT_SECONDS` | No | Agent timeout, default 2400s |
|
||||
|
||||
**Free data (no key needed):** HK/US equities via yfinance, crypto via OKX public API.
|
||||
**Free data (no key needed):** A-shares via AKShare, HK/US equities via yfinance, crypto via OKX, 100+ crypto exchanges via CCXT. The system automatically selects the best available source for each market.
|
||||
|
||||
---
|
||||
|
||||
@@ -312,7 +313,7 @@ vibe-trading serve # API server
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `/help` | Show all commands |
|
||||
| `/skills` | List all 65 finance skills |
|
||||
| `/skills` | List all 68 finance skills |
|
||||
| `/swarm` | List 29 swarm team presets |
|
||||
| `/swarm run <preset> [vars_json]` | Run a swarm team with live streaming |
|
||||
| `/swarm list` | Swarm run history |
|
||||
@@ -446,7 +447,7 @@ Browse on ClawHub: [clawhub.ai/skills/vibe-trading](https://clawhub.ai/skills/vi
|
||||
<details>
|
||||
<summary><b>OpenSpace — self-evolving skills</b></summary>
|
||||
|
||||
All 65 finance skills are published on [open-space.cloud](https://open-space.cloud) and evolve autonomously through OpenSpace's self-evolution engine.
|
||||
All 68 finance skills are published on [open-space.cloud](https://open-space.cloud) and evolve autonomously through OpenSpace's self-evolution engine.
|
||||
|
||||
To use with OpenSpace, add both MCP servers to your agent config:
|
||||
|
||||
@@ -468,7 +469,7 @@ To use with OpenSpace, add both MCP servers to your agent config:
|
||||
}
|
||||
```
|
||||
|
||||
OpenSpace will auto-discover all 64 skills, enabling auto-fix, auto-improve, and community sharing. Search for Vibe-Trading skills via `search_skills("finance backtest")` in any OpenSpace-connected agent.
|
||||
OpenSpace will auto-discover all 68 skills, enabling auto-fix, auto-improve, and community sharing. Search for Vibe-Trading skills via `search_skills("finance backtest")` in any OpenSpace-connected agent.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -489,30 +490,33 @@ Vibe-Trading/
|
||||
│ ├── src/
|
||||
│ │ ├── agent/ # ReAct agent core
|
||||
│ │ │ ├── loop.py # main reasoning loop
|
||||
│ │ │ ├── skills.py # skill loader (64 SKILL.md files)
|
||||
│ │ │ ├── skills.py # skill loader (68 SKILL.md files, 7 categories)
|
||||
│ │ │ ├── tools.py # tool orchestration
|
||||
│ │ │ ├── context.py # system prompt builder
|
||||
│ │ │ ├── memory.py # run memory / artifact store
|
||||
│ │ │ └── trace.py # execution trace writer
|
||||
│ │ │
|
||||
│ │ ├── tools/ # 20 agent tools
|
||||
│ │ ├── tools/ # 21 agent tools
|
||||
│ │ │ ├── backtest_tool.py # run backtests
|
||||
│ │ │ ├── factor_analysis_tool.py
|
||||
│ │ │ ├── options_pricing_tool.py
|
||||
│ │ │ ├── pattern_tool.py # chart pattern detection
|
||||
│ │ │ ├── doc_reader_tool.py # PDF reader (OCR fallback)
|
||||
│ │ │ ├── web_reader_tool.py # web page reader (Jina)
|
||||
│ │ │ ├── web_search_tool.py # DuckDuckGo web search
|
||||
│ │ │ ├── swarm_tool.py # launch swarm teams
|
||||
│ │ │ └── ... # file I/O, bash, tasks, etc.
|
||||
│ │ │
|
||||
│ │ ├── skills/ # 64 finance skill definitions (SKILL.md each)
|
||||
│ │ ├── skills/ # 68 finance skills in 7 categories (SKILL.md each)
|
||||
│ │ ├── swarm/ # Swarm DAG execution engine
|
||||
│ │ ├── session/ # Multi-turn chat session management
|
||||
│ │ └── providers/ # LLM provider abstraction
|
||||
│ │
|
||||
│ ├── backtest/ # Backtest engines
|
||||
│ │ ├── engines/ # daily_portfolio + options_portfolio
|
||||
│ │ ├── loaders/ # tushare, okx, yfinance
|
||||
│ │ ├── loaders/ # 5 sources: tushare, okx, yfinance, akshare, ccxt
|
||||
│ │ │ ├── base.py # DataLoader Protocol
|
||||
│ │ │ └── registry.py # Registry + auto-fallback chains
|
||||
│ │ └── optimizers/ # MVO, equal vol, max div, risk parity
|
||||
│ │
|
||||
│ └── config/swarm/ # 29 swarm preset YAML definitions
|
||||
|
||||
@@ -0,0 +1,191 @@
|
||||
"""AKShare loader: free, no-auth data for A-shares, US, HK, futures, forex, macro.
|
||||
|
||||
AKShare (https://github.com/akfamily/akshare) is a completely free financial
|
||||
data aggregator covering Chinese and global markets. No API token required.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from backtest.loaders.registry import register
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_INTERVAL_MAP_DAILY = {
|
||||
"1D": "daily",
|
||||
"1W": "weekly",
|
||||
"1M": "monthly",
|
||||
}
|
||||
|
||||
|
||||
def _is_a_share(code: str) -> bool:
|
||||
return code.upper().endswith((".SZ", ".SH", ".BJ"))
|
||||
|
||||
|
||||
def _is_hk(code: str) -> bool:
|
||||
return code.upper().endswith(".HK")
|
||||
|
||||
|
||||
def _is_us(code: str) -> bool:
|
||||
return code.upper().endswith(".US")
|
||||
|
||||
|
||||
def _is_crypto(code: str) -> bool:
|
||||
return "-USDT" in code.upper() or "/USDT" in code.upper()
|
||||
|
||||
|
||||
@register
|
||||
class DataLoader:
|
||||
"""AKShare universal OHLCV loader (free, no auth)."""
|
||||
|
||||
name = "akshare"
|
||||
markets = {"a_share", "us_equity", "hk_equity", "futures", "fund", "macro", "forex"}
|
||||
requires_auth = False
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Available if akshare is installed."""
|
||||
try:
|
||||
import akshare # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
|
||||
def fetch(
|
||||
self,
|
||||
codes: List[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
interval: str = "1D",
|
||||
fields: Optional[List[str]] = None,
|
||||
) -> Dict[str, pd.DataFrame]:
|
||||
"""Fetch OHLCV data via AKShare.
|
||||
|
||||
Args:
|
||||
codes: Symbol list.
|
||||
start_date: YYYY-MM-DD.
|
||||
end_date: YYYY-MM-DD.
|
||||
interval: Bar size (only 1D supported currently).
|
||||
fields: Ignored.
|
||||
|
||||
Returns:
|
||||
Mapping symbol -> OHLCV DataFrame.
|
||||
"""
|
||||
result: Dict[str, pd.DataFrame] = {}
|
||||
for code in codes:
|
||||
try:
|
||||
df = self._fetch_one(code, start_date, end_date, interval)
|
||||
if df is not None and not df.empty:
|
||||
result[code] = df
|
||||
except Exception as exc:
|
||||
logger.warning("akshare failed for %s: %s", code, exc)
|
||||
return result
|
||||
|
||||
def _fetch_one(
|
||||
self, code: str, start_date: str, end_date: str, interval: str,
|
||||
) -> Optional[pd.DataFrame]:
|
||||
"""Fetch a single symbol."""
|
||||
import akshare as ak
|
||||
|
||||
if _is_a_share(code):
|
||||
return self._fetch_a_share(ak, code, start_date, end_date, interval)
|
||||
if _is_us(code):
|
||||
return self._fetch_us(ak, code, start_date, end_date)
|
||||
if _is_hk(code):
|
||||
return self._fetch_hk(ak, code, start_date, end_date)
|
||||
# Default: try A-share
|
||||
return self._fetch_a_share(ak, code, start_date, end_date, interval)
|
||||
|
||||
def _fetch_a_share(
|
||||
self, ak, code: str, start_date: str, end_date: str, interval: str,
|
||||
) -> Optional[pd.DataFrame]:
|
||||
"""Fetch A-share via stock_zh_a_hist."""
|
||||
symbol = code.split(".")[0]
|
||||
period = _INTERVAL_MAP_DAILY.get(interval, "daily")
|
||||
sd = start_date.replace("-", "")
|
||||
ed = end_date.replace("-", "")
|
||||
df = ak.stock_zh_a_hist(
|
||||
symbol=symbol,
|
||||
period=period,
|
||||
start_date=sd,
|
||||
end_date=ed,
|
||||
adjust="qfq",
|
||||
)
|
||||
if df is None or df.empty:
|
||||
return None
|
||||
return self._normalize(df, date_col="日期")
|
||||
|
||||
def _fetch_us(self, ak, code: str, start_date: str, end_date: str) -> Optional[pd.DataFrame]:
|
||||
"""Fetch US stock via stock_us_hist."""
|
||||
symbol = code.replace(".US", "")
|
||||
# akshare uses the format like "105.AAPL" for NASDAQ
|
||||
# Try common prefixes
|
||||
for prefix in ["105.", "106.", ""]:
|
||||
try:
|
||||
df = ak.stock_us_hist(
|
||||
symbol=f"{prefix}{symbol}",
|
||||
period="daily",
|
||||
start_date=start_date.replace("-", ""),
|
||||
end_date=end_date.replace("-", ""),
|
||||
adjust="qfq",
|
||||
)
|
||||
if df is not None and not df.empty:
|
||||
return self._normalize(df, date_col="日期")
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
def _fetch_hk(self, ak, code: str, start_date: str, end_date: str) -> Optional[pd.DataFrame]:
|
||||
"""Fetch HK stock via stock_hk_hist."""
|
||||
symbol = code.replace(".HK", "").zfill(5)
|
||||
df = ak.stock_hk_hist(
|
||||
symbol=symbol,
|
||||
period="daily",
|
||||
start_date=start_date.replace("-", ""),
|
||||
end_date=end_date.replace("-", ""),
|
||||
adjust="qfq",
|
||||
)
|
||||
if df is None or df.empty:
|
||||
return None
|
||||
return self._normalize(df, date_col="日期")
|
||||
|
||||
@staticmethod
|
||||
def _normalize(df: pd.DataFrame, date_col: str = "日期") -> pd.DataFrame:
|
||||
"""Normalize AKShare DataFrame to standard OHLCV schema.
|
||||
|
||||
AKShare Chinese column names: 日期, 开盘, 最高, 最低, 收盘, 成交量
|
||||
AKShare English column names: date, open, high, low, close, volume
|
||||
"""
|
||||
col_map_cn = {"开盘": "open", "最高": "high", "最低": "low", "收盘": "close", "成交量": "volume"}
|
||||
col_map_en = {"date": "trade_date", "open": "open", "high": "high", "low": "low", "close": "close", "volume": "volume"}
|
||||
|
||||
if date_col in df.columns:
|
||||
df = df.rename(columns={date_col: "trade_date"})
|
||||
elif "date" in df.columns:
|
||||
df = df.rename(columns={"date": "trade_date"})
|
||||
|
||||
# Try Chinese column names first, then English
|
||||
if "开盘" in df.columns:
|
||||
df = df.rename(columns=col_map_cn)
|
||||
else:
|
||||
df = df.rename(columns=col_map_en)
|
||||
|
||||
df["trade_date"] = pd.to_datetime(df["trade_date"])
|
||||
df = df.set_index("trade_date").sort_index()
|
||||
|
||||
for col in ["open", "high", "low", "close", "volume"]:
|
||||
if col in df.columns:
|
||||
df[col] = pd.to_numeric(df[col], errors="coerce")
|
||||
|
||||
ohlcv_cols = [c for c in ["open", "high", "low", "close", "volume"] if c in df.columns]
|
||||
df = df[ohlcv_cols].dropna(subset=["open", "high", "low", "close"])
|
||||
if "volume" not in df.columns:
|
||||
df["volume"] = 0.0
|
||||
return df
|
||||
@@ -0,0 +1,40 @@
|
||||
"""DataLoader Protocol and shared exceptions for all data source loaders."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
class NoAvailableSourceError(Exception):
|
||||
"""Raised when no data source is available for a given market."""
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class DataLoaderProtocol(Protocol):
|
||||
"""Interface that every data source loader must satisfy."""
|
||||
|
||||
name: str
|
||||
markets: set[str]
|
||||
requires_auth: bool
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check whether this data source is usable (token present, network ok, etc.)."""
|
||||
...
|
||||
|
||||
def fetch(
|
||||
self,
|
||||
codes: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
interval: str = "1D",
|
||||
fields: list[str] | None = None,
|
||||
) -> dict[str, pd.DataFrame]:
|
||||
"""Fetch OHLCV data.
|
||||
|
||||
Returns:
|
||||
Mapping ``{symbol: DataFrame(trade_date, open, high, low, close, volume)}``.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,128 @@
|
||||
"""CCXT loader: unified crypto exchange data (100+ exchanges).
|
||||
|
||||
Uses the CCXT library to fetch OHLCV candles from any supported exchange.
|
||||
Defaults to Binance; configurable via CCXT_EXCHANGE env var.
|
||||
No API key required for public market data.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from backtest.loaders.registry import register
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_INTERVAL_MAP = {
|
||||
"1m": "1m", "5m": "5m", "15m": "15m", "30m": "30m",
|
||||
"1H": "1h", "4H": "4h", "1D": "1d",
|
||||
}
|
||||
|
||||
|
||||
@register
|
||||
class DataLoader:
|
||||
"""CCXT-backed crypto OHLCV loader (100+ exchanges)."""
|
||||
|
||||
name = "ccxt"
|
||||
markets = {"crypto"}
|
||||
requires_auth = False
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Available if ccxt is installed."""
|
||||
try:
|
||||
import ccxt # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
|
||||
def _get_exchange(self):
|
||||
"""Create exchange instance."""
|
||||
import ccxt
|
||||
exchange_id = os.getenv("CCXT_EXCHANGE", "binance").lower()
|
||||
exchange_cls = getattr(ccxt, exchange_id, None)
|
||||
if exchange_cls is None:
|
||||
logger.warning("Unknown CCXT exchange %s, falling back to binance", exchange_id)
|
||||
exchange_cls = ccxt.binance
|
||||
return exchange_cls({"enableRateLimit": True})
|
||||
|
||||
def fetch(
|
||||
self,
|
||||
codes: List[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
interval: str = "1D",
|
||||
fields: Optional[List[str]] = None,
|
||||
) -> Dict[str, pd.DataFrame]:
|
||||
"""Fetch crypto OHLCV via CCXT.
|
||||
|
||||
Args:
|
||||
codes: Symbols like ``["BTC-USDT", "ETH-USDT"]``.
|
||||
start_date: Start date (YYYY-MM-DD).
|
||||
end_date: End date (YYYY-MM-DD).
|
||||
interval: Bar size.
|
||||
fields: Ignored.
|
||||
|
||||
Returns:
|
||||
Mapping symbol -> OHLCV DataFrame.
|
||||
"""
|
||||
exchange = self._get_exchange()
|
||||
timeframe = _INTERVAL_MAP.get(interval, "1d")
|
||||
since_ms = int(pd.Timestamp(start_date).timestamp() * 1000)
|
||||
end_ms = int((pd.Timestamp(end_date) + pd.Timedelta(days=1)).timestamp() * 1000)
|
||||
|
||||
result: Dict[str, pd.DataFrame] = {}
|
||||
for code in codes:
|
||||
try:
|
||||
ccxt_symbol = code.replace("-", "/").upper()
|
||||
df = self._fetch_one(exchange, ccxt_symbol, timeframe, since_ms, end_ms)
|
||||
if df is not None and not df.empty:
|
||||
result[code] = df
|
||||
except Exception as exc:
|
||||
logger.warning("CCXT failed for %s: %s", code, exc)
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _fetch_one(
|
||||
exchange, symbol: str, timeframe: str, since_ms: int, end_ms: int,
|
||||
) -> Optional[pd.DataFrame]:
|
||||
"""Paginated OHLCV fetch for one symbol."""
|
||||
all_rows: list = []
|
||||
cursor = since_ms
|
||||
limit = 1000
|
||||
|
||||
for _ in range(200):
|
||||
ohlcv = exchange.fetch_ohlcv(symbol, timeframe, since=cursor, limit=limit)
|
||||
if not ohlcv:
|
||||
break
|
||||
all_rows.extend(ohlcv)
|
||||
last_ts = ohlcv[-1][0]
|
||||
if last_ts >= end_ms or len(ohlcv) < limit:
|
||||
break
|
||||
cursor = last_ts + 1
|
||||
|
||||
if not all_rows:
|
||||
return None
|
||||
|
||||
df = pd.DataFrame(all_rows, columns=["timestamp", "open", "high", "low", "close", "volume"])
|
||||
df["trade_date"] = pd.to_datetime(df["timestamp"], unit="ms")
|
||||
df = df.set_index("trade_date").sort_index()
|
||||
|
||||
start_dt = pd.Timestamp(since_ms, unit="ms")
|
||||
end_dt = pd.Timestamp(end_ms, unit="ms")
|
||||
df = df[(df.index >= start_dt) & (df.index < end_dt)]
|
||||
|
||||
for col in ["open", "high", "low", "close", "volume"]:
|
||||
df[col] = pd.to_numeric(df[col], errors="coerce")
|
||||
|
||||
df = df[["open", "high", "low", "close", "volume"]].dropna(
|
||||
subset=["open", "high", "low", "close"]
|
||||
)
|
||||
return df if not df.empty else None
|
||||
@@ -10,13 +10,24 @@ from typing import Dict, List, Optional
|
||||
import pandas as pd
|
||||
import requests
|
||||
|
||||
from backtest.loaders.registry import register
|
||||
|
||||
BASE_URL = "https://www.okx.com/api/v5"
|
||||
_MAX_PER_PAGE = 300
|
||||
|
||||
|
||||
@register
|
||||
class DataLoader:
|
||||
"""OKX crypto OHLCV loader."""
|
||||
|
||||
name = "okx"
|
||||
markets = {"crypto"}
|
||||
requires_auth = False
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Always available (public API, no auth)."""
|
||||
return True
|
||||
|
||||
def __init__(self) -> None:
|
||||
"""No credentials required for public candles."""
|
||||
pass
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Loader registry with market-level fallback chains.
|
||||
|
||||
Loaders self-register via the ``@register`` decorator when their module is
|
||||
first imported. The ``_ensure_registered()`` helper lazily imports every
|
||||
known loader module so that callers of ``resolve_loader`` /
|
||||
``get_loader_cls_with_fallback`` never see an empty registry — regardless
|
||||
of import order.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Type
|
||||
|
||||
from backtest.loaders.base import NoAvailableSourceError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Global registry: source_name -> loader class
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
LOADER_REGISTRY: dict[str, Type[Any]] = {}
|
||||
|
||||
_registered = False
|
||||
|
||||
|
||||
def register(cls: Type[Any]) -> Type[Any]:
|
||||
"""Class decorator: register a loader into the global registry.
|
||||
|
||||
The class must have a ``name`` class attribute.
|
||||
"""
|
||||
LOADER_REGISTRY[cls.name] = cls
|
||||
return cls
|
||||
|
||||
|
||||
def _ensure_registered() -> None:
|
||||
"""Import every known loader module so ``@register`` decorators fire.
|
||||
|
||||
Safe to call multiple times — only runs the imports once.
|
||||
Loaders whose dependencies are missing (e.g. ``akshare`` not installed)
|
||||
are silently skipped.
|
||||
"""
|
||||
global _registered
|
||||
if _registered:
|
||||
return
|
||||
_registered = True
|
||||
|
||||
_loader_modules = [
|
||||
"backtest.loaders.tushare",
|
||||
"backtest.loaders.okx",
|
||||
"backtest.loaders.yfinance_loader",
|
||||
"backtest.loaders.akshare_loader",
|
||||
"backtest.loaders.ccxt_loader",
|
||||
]
|
||||
import importlib
|
||||
for mod in _loader_modules:
|
||||
try:
|
||||
importlib.import_module(mod)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fallback chains: market_type -> ordered list of source names
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
FALLBACK_CHAINS: dict[str, list[str]] = {
|
||||
"a_share": ["tushare", "akshare"],
|
||||
"us_equity": ["yfinance", "akshare"],
|
||||
"hk_equity": ["yfinance", "akshare"],
|
||||
"crypto": ["okx", "ccxt"],
|
||||
"futures": ["tushare", "akshare"],
|
||||
"fund": ["tushare", "akshare"],
|
||||
"macro": ["akshare", "tushare"],
|
||||
"forex": ["akshare", "yfinance"],
|
||||
}
|
||||
|
||||
|
||||
def resolve_loader(market: str) -> Any:
|
||||
"""Return the first *available* loader instance for *market*.
|
||||
|
||||
Walks the fallback chain and returns the first loader whose
|
||||
``is_available()`` returns ``True``.
|
||||
|
||||
Args:
|
||||
market: Market type key (e.g. ``"a_share"``, ``"crypto"``).
|
||||
|
||||
Returns:
|
||||
A loader instance.
|
||||
|
||||
Raises:
|
||||
NoAvailableSourceError: If every candidate is unavailable.
|
||||
"""
|
||||
_ensure_registered()
|
||||
chain = FALLBACK_CHAINS.get(market, [])
|
||||
tried: list[str] = []
|
||||
for name in chain:
|
||||
if name not in LOADER_REGISTRY:
|
||||
continue
|
||||
loader = LOADER_REGISTRY[name]()
|
||||
tried.append(name)
|
||||
if loader.is_available():
|
||||
return loader
|
||||
raise NoAvailableSourceError(
|
||||
f"No available data source for market '{market}'. "
|
||||
f"Tried: {tried or chain}. Check network and API token config."
|
||||
)
|
||||
|
||||
|
||||
def get_loader_cls_with_fallback(source: str) -> Type[Any]:
|
||||
"""Return a loader *class* for *source*, falling back if unavailable.
|
||||
|
||||
Args:
|
||||
source: Requested data source name.
|
||||
|
||||
Returns:
|
||||
A DataLoader class (not instance).
|
||||
|
||||
Raises:
|
||||
NoAvailableSourceError: If the source and all fallbacks are unavailable.
|
||||
"""
|
||||
_ensure_registered()
|
||||
if source not in LOADER_REGISTRY:
|
||||
raise NoAvailableSourceError(f"Unknown data source: {source}")
|
||||
|
||||
loader_cls = LOADER_REGISTRY[source]
|
||||
instance = loader_cls()
|
||||
if instance.is_available():
|
||||
return loader_cls
|
||||
|
||||
# Source unavailable — try same-market fallback
|
||||
for market in loader_cls.markets:
|
||||
try:
|
||||
fallback = resolve_loader(market)
|
||||
logger.warning(
|
||||
"%s is unavailable, falling back to %s for market %s",
|
||||
source, fallback.name, market,
|
||||
)
|
||||
return type(fallback)
|
||||
except NoAvailableSourceError:
|
||||
continue
|
||||
|
||||
raise NoAvailableSourceError(
|
||||
f"Data source '{source}' is unavailable and no fallback found."
|
||||
)
|
||||
@@ -9,10 +9,21 @@ from typing import Dict, List, Optional
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from backtest.loaders.registry import register
|
||||
|
||||
|
||||
@register
|
||||
class DataLoader:
|
||||
"""Tushare-backed OHLCV loader."""
|
||||
|
||||
name = "tushare"
|
||||
markets = {"a_share", "futures", "fund"}
|
||||
requires_auth = True
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Available when TUSHARE_TOKEN is set."""
|
||||
return bool(os.getenv("TUSHARE_TOKEN", ""))
|
||||
|
||||
def __init__(self) -> None:
|
||||
"""Initialize Tushare pro API."""
|
||||
import tushare as ts
|
||||
|
||||
@@ -8,6 +8,8 @@ from typing import Dict, List, Optional, Union
|
||||
import pandas as pd
|
||||
import yfinance as yf
|
||||
|
||||
from backtest.loaders.registry import register
|
||||
|
||||
_OHLCV_COLUMNS = ["open", "high", "low", "close", "volume"]
|
||||
_COLUMN_RENAMES = {
|
||||
"Open": "open",
|
||||
@@ -186,9 +188,18 @@ def _normalize_frame(frame: pd.DataFrame, requested_interval: str) -> pd.DataFra
|
||||
return normalized
|
||||
|
||||
|
||||
@register
|
||||
class DataLoader:
|
||||
"""Fetch HK/US equity bars from Yahoo Finance via yfinance."""
|
||||
|
||||
name = "yfinance"
|
||||
markets = {"us_equity", "hk_equity"}
|
||||
requires_auth = False
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Always available (free public data, no auth)."""
|
||||
return True
|
||||
|
||||
def __init__(self) -> None:
|
||||
"""Initialize the loader.
|
||||
|
||||
|
||||
@@ -14,15 +14,15 @@ from backtest.models import TradeRecord
|
||||
|
||||
# ─── Annualisation factor mapping ───
|
||||
|
||||
_TRADING_DAYS = {"tushare": 252, "yfinance": 252, "okx": 365}
|
||||
_TRADING_DAYS = {"tushare": 252, "yfinance": 252, "okx": 365, "akshare": 252, "ccxt": 365}
|
||||
_BARS_PER_DAY = {
|
||||
"1m": {"tushare": 240, "okx": 1440, "yfinance": 390},
|
||||
"5m": {"tushare": 48, "okx": 288, "yfinance": 78},
|
||||
"15m": {"tushare": 16, "okx": 96, "yfinance": 26},
|
||||
"30m": {"tushare": 8, "okx": 48, "yfinance": 13},
|
||||
"1H": {"tushare": 4, "okx": 24, "yfinance": 7},
|
||||
"4H": {"tushare": 1, "okx": 6, "yfinance": 2},
|
||||
"1D": {"tushare": 1, "okx": 1, "yfinance": 1},
|
||||
"1m": {"tushare": 240, "okx": 1440, "yfinance": 390, "akshare": 240, "ccxt": 1440},
|
||||
"5m": {"tushare": 48, "okx": 288, "yfinance": 78, "akshare": 48, "ccxt": 288},
|
||||
"15m": {"tushare": 16, "okx": 96, "yfinance": 26, "akshare": 16, "ccxt": 96},
|
||||
"30m": {"tushare": 8, "okx": 48, "yfinance": 13, "akshare": 8, "ccxt": 48},
|
||||
"1H": {"tushare": 4, "okx": 24, "yfinance": 7, "akshare": 4, "ccxt": 24},
|
||||
"4H": {"tushare": 1, "okx": 6, "yfinance": 2, "akshare": 1, "ccxt": 6},
|
||||
"1D": {"tushare": 1, "okx": 1, "yfinance": 1, "akshare": 1, "ccxt": 1},
|
||||
}
|
||||
|
||||
|
||||
|
||||
+111
-38
@@ -9,11 +9,27 @@ Usage: ``python -m backtest.runner <run_dir>``
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, List
|
||||
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
from backtest.loaders.registry import (
|
||||
LOADER_REGISTRY,
|
||||
get_loader_cls_with_fallback,
|
||||
resolve_loader,
|
||||
)
|
||||
from backtest.loaders.base import NoAvailableSourceError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_module_from_file(file_path: Path, module_name: str):
|
||||
"""Load a Python module from a file path via importlib.
|
||||
@@ -32,33 +48,80 @@ def _load_module_from_file(file_path: Path, module_name: str):
|
||||
return module
|
||||
|
||||
|
||||
# --- Market detection ---
|
||||
# --- Market detection (returns market type, NOT source name) ---
|
||||
|
||||
_PATTERNS = [
|
||||
(re.compile(r"^\d{6}\.(SZ|SH|BJ)$", re.I), "tushare"),
|
||||
(re.compile(r"^[A-Z]+\.US$", re.I), "yfinance"),
|
||||
(re.compile(r"^\d{3,5}\.HK$", re.I), "yfinance"),
|
||||
(re.compile(r"^[A-Z]+-USDT$", re.I), "okx"),
|
||||
_MARKET_PATTERNS = [
|
||||
(re.compile(r"^\d{6}\.(SZ|SH|BJ)$", re.I), "a_share"),
|
||||
(re.compile(r"^(51|15|56)\d{4}\.(SZ|SH)$", re.I), "a_share"),
|
||||
(re.compile(r"^[A-Z]+\.US$", re.I), "us_equity"),
|
||||
(re.compile(r"^\d{3,5}\.HK$", re.I), "hk_equity"),
|
||||
(re.compile(r"^[A-Z]+-USDT$", re.I), "crypto"),
|
||||
(re.compile(r"^[A-Z]+/USDT$", re.I), "crypto"),
|
||||
(re.compile(r"^[A-Z]{1,2}\d{3,4}\.(ZCE|DCE|SHFE|INE|CFFEX|GFEX)$", re.I), "futures"),
|
||||
(re.compile(r"^[A-Z]{3}/[A-Z]{3}$"), "forex"),
|
||||
(re.compile(r"^[A-Z]{6}\.FX$"), "forex"),
|
||||
]
|
||||
|
||||
# Back-compat: market type -> legacy source name (for engine selection & metrics)
|
||||
_MARKET_TO_SOURCE = {
|
||||
"a_share": "tushare",
|
||||
"us_equity": "yfinance",
|
||||
"hk_equity": "yfinance",
|
||||
"crypto": "okx",
|
||||
"futures": "tushare",
|
||||
"fund": "tushare",
|
||||
"macro": "akshare",
|
||||
"forex": "akshare",
|
||||
}
|
||||
|
||||
def _detect_source(code: str) -> str:
|
||||
"""Infer data source from symbol format.
|
||||
|
||||
def _detect_market(code: str) -> str:
|
||||
"""Infer market type from symbol format.
|
||||
|
||||
Args:
|
||||
code: Ticker / symbol string.
|
||||
|
||||
Returns:
|
||||
Source name (tushare/okx/yfinance); unknown defaults to ``tushare``.
|
||||
Market type (a_share/us_equity/hk_equity/crypto/futures/forex);
|
||||
unknown defaults to ``a_share``.
|
||||
"""
|
||||
for pattern, source in _PATTERNS:
|
||||
for pattern, market in _MARKET_PATTERNS:
|
||||
if pattern.match(code):
|
||||
return source
|
||||
return "tushare"
|
||||
return market
|
||||
return "a_share"
|
||||
|
||||
|
||||
def _detect_source(code: str) -> str:
|
||||
"""Infer legacy source name from symbol (back-compat for metrics/engine).
|
||||
|
||||
Args:
|
||||
code: Ticker / symbol string.
|
||||
|
||||
Returns:
|
||||
Source name (tushare/okx/yfinance/akshare).
|
||||
"""
|
||||
market = _detect_market(code)
|
||||
return _MARKET_TO_SOURCE.get(market, "tushare")
|
||||
|
||||
|
||||
def _group_codes_by_market(codes: List[str]) -> Dict[str, List[str]]:
|
||||
"""Group symbols by detected market type.
|
||||
|
||||
Args:
|
||||
codes: List of symbol strings.
|
||||
|
||||
Returns:
|
||||
Mapping market_type -> list of codes.
|
||||
"""
|
||||
groups: Dict[str, List[str]] = {}
|
||||
for code in codes:
|
||||
market = _detect_market(code)
|
||||
groups.setdefault(market, []).append(code)
|
||||
return groups
|
||||
|
||||
|
||||
def _group_codes_by_source(codes: List[str]) -> Dict[str, List[str]]:
|
||||
"""Group symbols by inferred source.
|
||||
"""Group symbols by inferred source (back-compat).
|
||||
|
||||
Args:
|
||||
codes: List of symbol strings.
|
||||
@@ -74,23 +137,21 @@ def _group_codes_by_source(codes: List[str]) -> Dict[str, List[str]]:
|
||||
|
||||
|
||||
def _get_loader(source: str):
|
||||
"""Return the DataLoader class for a source name.
|
||||
"""Return a DataLoader class for a source name, with fallback.
|
||||
|
||||
Args:
|
||||
source: Source name.
|
||||
source: Source name (tushare/okx/yfinance/akshare/ccxt).
|
||||
|
||||
Returns:
|
||||
DataLoader class.
|
||||
"""
|
||||
if source == "okx":
|
||||
from backtest.loaders.okx import DataLoader
|
||||
elif source == "yfinance":
|
||||
from backtest.loaders.yfinance_loader import DataLoader
|
||||
elif source == "tushare":
|
||||
from backtest.loaders.tushare import DataLoader
|
||||
else:
|
||||
from backtest.loaders.tushare import DataLoader
|
||||
return DataLoader
|
||||
try:
|
||||
return get_loader_cls_with_fallback(source)
|
||||
except NoAvailableSourceError:
|
||||
# Ultimate fallback for unknown sources
|
||||
if "tushare" in LOADER_REGISTRY:
|
||||
return LOADER_REGISTRY["tushare"]
|
||||
raise
|
||||
|
||||
|
||||
def _normalize_codes(codes: List[str], source: str) -> List[str]:
|
||||
@@ -103,7 +164,7 @@ def _normalize_codes(codes: List[str], source: str) -> List[str]:
|
||||
Returns:
|
||||
Normalized codes.
|
||||
"""
|
||||
if source == "okx":
|
||||
if source in ("okx", "ccxt"):
|
||||
return [c.replace("/", "-").upper() for c in codes]
|
||||
return codes
|
||||
|
||||
@@ -185,17 +246,23 @@ def _create_market_engine(source: str, config: dict, codes: List[str]):
|
||||
"""Create the appropriate market engine based on data source.
|
||||
|
||||
Args:
|
||||
source: Data source (okx / tushare / yfinance).
|
||||
source: Data source (okx/ccxt/tushare/akshare/yfinance).
|
||||
config: Backtest configuration.
|
||||
codes: Instrument codes.
|
||||
|
||||
Returns:
|
||||
BaseEngine subclass instance.
|
||||
"""
|
||||
if source == "okx":
|
||||
if source in ("okx", "ccxt"):
|
||||
from backtest.engines.crypto import CryptoEngine
|
||||
return CryptoEngine(config)
|
||||
elif source == "tushare":
|
||||
elif source in ("tushare", "akshare"):
|
||||
# Determine if codes look like A-shares or global equities
|
||||
markets = {_detect_market(c) for c in codes}
|
||||
if markets & {"us_equity", "hk_equity"}:
|
||||
from backtest.engines.global_equity import GlobalEquityEngine
|
||||
market = _detect_submarket(codes)
|
||||
return GlobalEquityEngine(config, market=market)
|
||||
from backtest.engines.china_a import ChinaAEngine
|
||||
return ChinaAEngine(config)
|
||||
elif source == "yfinance":
|
||||
@@ -203,7 +270,6 @@ def _create_market_engine(source: str, config: dict, codes: List[str]):
|
||||
market = _detect_submarket(codes)
|
||||
return GlobalEquityEngine(config, market=market)
|
||||
else:
|
||||
# Default: crypto (most permissive rules)
|
||||
from backtest.engines.crypto import CryptoEngine
|
||||
return CryptoEngine(config)
|
||||
|
||||
@@ -243,7 +309,7 @@ def _detect_primary_source(codes: List[str], source: str) -> str:
|
||||
|
||||
|
||||
def _fetch_auto(codes: List[str], config: dict, interval: str = "1D") -> dict:
|
||||
"""Auto mode: fetch per source and merge into one data map.
|
||||
"""Auto mode: route each market group through fallback chain.
|
||||
|
||||
Args:
|
||||
codes: All symbols.
|
||||
@@ -253,18 +319,25 @@ def _fetch_auto(codes: List[str], config: dict, interval: str = "1D") -> dict:
|
||||
Returns:
|
||||
Merged ``code -> DataFrame`` map.
|
||||
"""
|
||||
groups = _group_codes_by_source(codes)
|
||||
market_groups = _group_codes_by_market(codes)
|
||||
merged = {}
|
||||
start_date = config.get("start_date", "")
|
||||
end_date = config.get("end_date", "")
|
||||
|
||||
for src, src_codes in groups.items():
|
||||
src_codes = _normalize_codes(src_codes, src)
|
||||
LoaderCls = _get_loader(src)
|
||||
loader = LoaderCls()
|
||||
# Only tushare supports extra_fields
|
||||
fields = config.get("extra_fields") if src == "tushare" else None
|
||||
result = loader.fetch(src_codes, start_date, end_date, fields=fields, interval=interval)
|
||||
for market, market_codes in market_groups.items():
|
||||
try:
|
||||
loader = resolve_loader(market)
|
||||
except NoAvailableSourceError as exc:
|
||||
# Fallback: try legacy source mapping
|
||||
legacy_src = _MARKET_TO_SOURCE.get(market, "tushare")
|
||||
logger.warning("Fallback chain failed for %s: %s — trying %s", market, exc, legacy_src)
|
||||
LoaderCls = _get_loader(legacy_src)
|
||||
loader = LoaderCls()
|
||||
|
||||
src_name = getattr(loader, "name", "unknown")
|
||||
normalized_codes = _normalize_codes(market_codes, src_name)
|
||||
fields = config.get("extra_fields") if src_name == "tushare" else None
|
||||
result = loader.fetch(normalized_codes, start_date, end_date, fields=fields, interval=interval)
|
||||
merged.update(result)
|
||||
|
||||
return merged
|
||||
|
||||
@@ -9,8 +9,8 @@ from src.agent.memory import WorkspaceMemory
|
||||
from src.agent.skills import SkillsLoader
|
||||
from src.agent.tools import ToolRegistry
|
||||
|
||||
_SYSTEM_PROMPT = """You are a finance research agent with 64 specialist skills, 19 tools, and 29 multi-agent swarm teams.
|
||||
You handle backtesting, factor analysis, options pricing, risk audits, research reports, document/web reading, and team-based workflows.
|
||||
_SYSTEM_PROMPT = """You are a finance research agent with 68 specialist skills, 21 tools, 5 data sources (with auto-fallback), and 29 multi-agent swarm teams.
|
||||
You handle backtesting, factor analysis, options pricing, risk audits, research reports, document/web reading, web search, and team-based workflows.
|
||||
|
||||
## Tools
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@ class Skill:
|
||||
Attributes:
|
||||
name: Skill name.
|
||||
description: Skill description.
|
||||
category: Skill category for grouped display.
|
||||
body: SKILL.md body text.
|
||||
dir_path: Skill directory path (used for on-demand loading of supporting files).
|
||||
metadata: Parsed frontmatter metadata.
|
||||
@@ -27,6 +28,7 @@ class Skill:
|
||||
|
||||
name: str
|
||||
description: str = ""
|
||||
category: str = "other"
|
||||
body: str = ""
|
||||
dir_path: Optional[Path] = None
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
@@ -108,6 +110,7 @@ def _load_skill_dir(dir_path: Path) -> Optional[Skill]:
|
||||
return Skill(
|
||||
name=name,
|
||||
description=meta.get("description", ""),
|
||||
category=meta.get("category", "other"),
|
||||
body=body,
|
||||
dir_path=dir_path,
|
||||
metadata=meta,
|
||||
@@ -141,17 +144,33 @@ class SkillsLoader:
|
||||
if skill:
|
||||
self.skills.append(skill)
|
||||
|
||||
# Display order for categories (unlisted categories appear at the end).
|
||||
_CATEGORY_ORDER = [
|
||||
"data-source", "strategy", "analysis", "asset-class",
|
||||
"crypto", "flow", "tool", "other",
|
||||
]
|
||||
|
||||
def get_descriptions(self) -> str:
|
||||
"""Return a one-line summary of every skill for the system prompt (low token cost).
|
||||
"""Return skills grouped by category for the system prompt.
|
||||
|
||||
Returns:
|
||||
One line per skill with name + description.
|
||||
Grouped skill list with category headers.
|
||||
"""
|
||||
if not self.skills:
|
||||
return "(no skills)"
|
||||
lines = []
|
||||
|
||||
groups: Dict[str, List[Skill]] = {}
|
||||
for skill in self.skills:
|
||||
lines.append(f" - {skill.name}: {skill.description}")
|
||||
groups.setdefault(skill.category, []).append(skill)
|
||||
|
||||
ordered_cats = [c for c in self._CATEGORY_ORDER if c in groups]
|
||||
ordered_cats += [c for c in sorted(groups) if c not in ordered_cats]
|
||||
|
||||
lines: List[str] = []
|
||||
for cat in ordered_cats:
|
||||
lines.append(f"\n### {cat}")
|
||||
for skill in groups[cat]:
|
||||
lines.append(f" - {skill.name}: {skill.description}")
|
||||
return "\n".join(lines)
|
||||
|
||||
def get_content(self, name: str) -> str:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: adr-hshare
|
||||
description: ADR/H-share/A-share cross-listing premium analysis — track pricing gaps between US-listed ADRs, HK-listed H-shares, and A-shares for arbitrage signals, dual-listing valuation, and delisting risk assessment.
|
||||
category: flow
|
||||
---
|
||||
# ADR / H-Share / A-Share Cross-Listing Analysis
|
||||
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
name: akshare
|
||||
category: data-source
|
||||
description: AKShare financial data aggregator (18k+ stars). Free, no API key. Covers A-shares, US, HK, futures, macro, forex. Primary fallback for tushare and yfinance.
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
AKShare is a completely free, open-source Python financial data library. No registration or API key required. It aggregates data from public sources (Sina, East Money, etc.) covering Chinese and global markets.
|
||||
|
||||
- GitHub: https://github.com/akfamily/akshare (18k+ stars)
|
||||
- Install: `pip install akshare`
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import akshare as ak
|
||||
|
||||
# A-share daily OHLCV (前复权)
|
||||
df = ak.stock_zh_a_hist(symbol="000001", period="daily",
|
||||
start_date="20240101", end_date="20260101", adjust="qfq")
|
||||
|
||||
# US stock daily
|
||||
df = ak.stock_us_hist(symbol="105.AAPL", period="daily",
|
||||
start_date="20240101", end_date="20260101", adjust="qfq")
|
||||
|
||||
# HK stock daily
|
||||
df = ak.stock_hk_hist(symbol="00700", period="daily",
|
||||
start_date="20240101", end_date="20260101", adjust="qfq")
|
||||
```
|
||||
|
||||
## Top 10 High-Frequency Interfaces
|
||||
|
||||
### A-shares
|
||||
|
||||
| Function | Description | Key Params |
|
||||
|----------|-------------|------------|
|
||||
| `stock_zh_a_hist()` | A-share OHLCV | symbol, period, start_date, end_date, adjust |
|
||||
| `stock_zh_a_spot_em()` | Real-time A-share quotes | (none) |
|
||||
| `stock_individual_info_em()` | Stock basic info | symbol |
|
||||
| `stock_zh_a_hist_min_em()` | Intraday bars | symbol, period(1/5/15/30/60) |
|
||||
|
||||
### US / HK
|
||||
|
||||
| Function | Description | Key Params |
|
||||
|----------|-------------|------------|
|
||||
| `stock_us_hist()` | US stock OHLCV | symbol (e.g. "105.AAPL"), period, start_date, end_date |
|
||||
| `stock_hk_hist()` | HK stock OHLCV | symbol (e.g. "00700"), period, start_date, end_date |
|
||||
|
||||
### Macro / Forex / Futures
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `macro_china_gdp()` | China GDP data |
|
||||
| `macro_china_cpi()` | China CPI data |
|
||||
| `futures_main_sina()` | Futures main contract quotes |
|
||||
| `currency_boc_sina()` | BOC forex rates |
|
||||
|
||||
## Column Names
|
||||
|
||||
AKShare returns Chinese column names by default:
|
||||
|
||||
| Chinese | English | Description |
|
||||
|---------|---------|-------------|
|
||||
| 日期 | date | Trade date |
|
||||
| 开盘 | open | Open price |
|
||||
| 最高 | high | High price |
|
||||
| 最低 | low | Low price |
|
||||
| 收盘 | close | Close price |
|
||||
| 成交量 | volume | Volume |
|
||||
| 成交额 | amount | Turnover |
|
||||
| 涨跌幅 | pct_change | % change |
|
||||
| 换手率 | turnover_rate | Turnover rate |
|
||||
|
||||
## Date Format
|
||||
|
||||
- Input: `YYYYMMDD` string (e.g. `"20240101"`)
|
||||
- Output: `日期` column as string, convert with `pd.to_datetime()`
|
||||
|
||||
## Symbol Format
|
||||
|
||||
- A-shares: pure digits `"000001"` (no .SZ suffix)
|
||||
- US stocks: `"105.AAPL"` (NASDAQ prefix 105), `"106.BABA"` (NYSE prefix 106)
|
||||
- HK stocks: `"00700"` (5-digit zero-padded)
|
||||
|
||||
## Built-in Loader
|
||||
|
||||
The project has a built-in AKShare DataLoader at `backtest/loaders/akshare_loader.py`. When backtesting, the runner automatically falls back to AKShare when tushare/yfinance are unavailable.
|
||||
|
||||
## Reference Docs
|
||||
|
||||
For less common interfaces, see the `references/` subdirectory or the official docs at https://akshare.akfamily.xyz/
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: asset-allocation
|
||||
description: Asset allocation theory and optimizer usage — MPT / Black-Litterman / risk budgeting / all-weather strategy, including guides for 4 optimizers and rebalancing rules.
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# Asset Allocation and Portfolio Optimization
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: backtest-diagnose
|
||||
description: Diagnose failed or underperforming backtests, locate the root cause, and fix the issue
|
||||
category: tool
|
||||
---
|
||||
|
||||
# Backtest Diagnosis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: behavioral-finance
|
||||
description: "Behavioral finance applications: theories of overreaction and underreaction, behavioral explanations for momentum and reversal, investor sentiment cycles, cognitive-bias checklists, and debiasing quantitative strategies."
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Behavioral Finance Applications
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: candlestick
|
||||
description: Candlestick pattern recognition engine, pure pandas vectorized implementation of 15 classic candlestick patterns (5 single-candle + 5 double-candle + 4 triple-candle + 1 trend confirmation), generating a composite signal from bullish/bearish pattern scores.
|
||||
category: strategy
|
||||
---
|
||||
# Candlestick Pattern Recognition
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: ccxt
|
||||
category: data-source
|
||||
description: CCXT unified crypto exchange library (100+ exchanges). Free public market data. Fallback when OKX is unavailable.
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
CCXT is a unified cryptocurrency exchange trading library supporting 100+ exchanges including Binance, Bybit, OKX, Coinbase, Kraken, and more. Public market data (OHLCV, tickers, order books) requires no API key.
|
||||
|
||||
- GitHub: https://github.com/ccxt/ccxt (35k+ stars)
|
||||
- Install: `pip install ccxt`
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import ccxt
|
||||
|
||||
exchange = ccxt.binance({"enableRateLimit": True})
|
||||
|
||||
# Fetch daily OHLCV
|
||||
ohlcv = exchange.fetch_ohlcv("BTC/USDT", "1d", limit=100)
|
||||
# Returns: [[timestamp, open, high, low, close, volume], ...]
|
||||
|
||||
# Fetch ticker
|
||||
ticker = exchange.fetch_ticker("ETH/USDT")
|
||||
print(f"ETH price: {ticker['last']}")
|
||||
```
|
||||
|
||||
## Key Methods
|
||||
|
||||
| Method | Description | Returns |
|
||||
|--------|-------------|---------|
|
||||
| `fetch_ohlcv(symbol, timeframe, since, limit)` | Historical candles | `[[ts, o, h, l, c, v], ...]` |
|
||||
| `fetch_ticker(symbol)` | Latest quote | `{last, bid, ask, volume, ...}` |
|
||||
| `fetch_tickers(symbols)` | Batch quotes | `{symbol: ticker}` |
|
||||
| `fetch_order_book(symbol, limit)` | Order book | `{bids, asks, timestamp}` |
|
||||
| `fetch_trades(symbol, since, limit)` | Recent trades | `[{price, amount, side, timestamp}, ...]` |
|
||||
|
||||
## Timeframes
|
||||
|
||||
`1m`, `3m`, `5m`, `15m`, `30m`, `1h`, `2h`, `4h`, `6h`, `12h`, `1d`, `1w`, `1M`
|
||||
|
||||
Note: not all exchanges support all timeframes. Use `exchange.timeframes` to check.
|
||||
|
||||
## Symbol Format
|
||||
|
||||
CCXT uses slash format: `BTC/USDT`, `ETH/BTC`, `SOL/USDT`
|
||||
|
||||
The project's DataLoader automatically converts `BTC-USDT` (hyphen) to `BTC/USDT` (slash).
|
||||
|
||||
## Exchange Selection
|
||||
|
||||
Set via environment variable: `CCXT_EXCHANGE=binance` (default)
|
||||
|
||||
Popular exchanges: `binance`, `bybit`, `okx`, `coinbase`, `kraken`, `bitget`, `gate`
|
||||
|
||||
## Built-in Loader
|
||||
|
||||
The project has a built-in CCXT DataLoader at `backtest/loaders/ccxt_loader.py`. It serves as a fallback when the OKX loader is unavailable.
|
||||
|
||||
## Pagination
|
||||
|
||||
For long history, CCXT paginates via the `since` parameter (millisecond timestamp). The built-in loader handles this automatically (up to 200 pages).
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: chanlun
|
||||
description: 基于缠论（缠中说禅）的形态识别引擎，使用czsc库自动检测K线分型、笔、中枢，并生成一买/一卖/二买/二卖/三买/三卖等买卖点信号。支持多周期分析和形态分类（3/5/7/9/11笔形态）。
|
||||
category: strategy
|
||||
---
|
||||
# 缠论形态识别
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: commodity-analysis
|
||||
description: Commodity analysis (oil supply-demand balance / gold pricing / copper as an economic predictor / inventory cycles / futures premium-discount structure / seasonality), generating directional commodity signals.
|
||||
category: analysis
|
||||
---
|
||||
# Commodity Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: convertible-bond
|
||||
description: A股可转债分析——转股/纯债/期权三维估值、下修/强赎/回售博弈、双低策略与转债轮动选债框架
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# A股可转债分析
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: corporate-events
|
||||
description: 公司事件驱动分析：并购套利价差计算、大股东增减持信号、股权激励解读、定增配股影响评估、A股ST/退市预警
|
||||
category: flow
|
||||
---
|
||||
|
||||
# 公司事件驱动分析
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: correlation-analysis
|
||||
description: Correlation and cointegration analysis — co-movement discovery, deep return-correlation analysis, sector clustering, realized correlation, Engle-Granger / Johansen cointegration, half-life, Kalman dynamic hedge ratio, cross-market linkage analysis, and pair-trading signal generation
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Correlation and Cointegration Analysis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: credit-analysis
|
||||
description: "固收与信用分析：信用债评级、利差分析、违约风险评估、城投债研究、可转债定价与策略。"
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Credit Analysis Skill — 固收与信用分析
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: crypto-derivatives
|
||||
description: Crypto-derivatives strategies — perpetual funding-rate arbitrage, futures term-structure contango/backwardation trading, and option volatility-smile / Greeks analysis.
|
||||
category: crypto
|
||||
---
|
||||
|
||||
# Crypto-Derivatives Strategies
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: data-routing
|
||||
category: data-source
|
||||
description: Data source selection decision tree. Load this skill BEFORE any backtest or data-fetching task to choose the best available data source.
|
||||
---
|
||||
|
||||
## Data Source Overview
|
||||
|
||||
| Source | Markets | Auth Required | Network | Skill |
|
||||
|--------|---------|---------------|---------|-------|
|
||||
| tushare | A-shares, funds, futures, macro | Yes (`TUSHARE_TOKEN`) | China network | tushare |
|
||||
| akshare | A-shares, US, HK, futures, macro, forex | No | Unrestricted | akshare |
|
||||
| yfinance | US stocks, HK stocks, ETFs | No | Needs Yahoo Finance access | yfinance |
|
||||
| okx | Crypto (OKX exchange) | No | Needs okx.com access | okx-market |
|
||||
| ccxt | Crypto (100+ exchanges) | No | Needs exchange access | ccxt |
|
||||
|
||||
## Decision Tree
|
||||
|
||||
### Backtest Scenario (writing config.json)
|
||||
|
||||
Use `source: "auto"` — the runner automatically routes by symbol pattern and falls back to alternative sources if the primary one is unavailable.
|
||||
|
||||
You do NOT need to specify a concrete data source in config.json unless the user explicitly asks for one.
|
||||
|
||||
### Analysis / Research Scenario (writing Python scripts)
|
||||
|
||||
1. Identify the market type from the user's request
|
||||
2. Pick the source by priority:
|
||||
|
||||
**A-shares**: tushare (if TUSHARE_TOKEN is set) > akshare (free fallback)
|
||||
**US stocks**: yfinance > akshare
|
||||
**HK stocks**: yfinance > akshare
|
||||
**Crypto**: okx (single exchange) > ccxt (multi-exchange)
|
||||
**Futures**: tushare > akshare
|
||||
**Macro / economics**: akshare > tushare
|
||||
**Forex**: akshare > yfinance
|
||||
|
||||
3. Load the corresponding skill for API details: `load_skill("akshare")`
|
||||
|
||||
### Availability Check
|
||||
|
||||
- **tushare**: check if `TUSHARE_TOKEN` environment variable exists
|
||||
- **yfinance / okx / ccxt / akshare**: free but may have network restrictions
|
||||
- If the user reports "connection timeout" or "cannot access", switch to the same-market fallback
|
||||
|
||||
## Symbol Format Reference
|
||||
|
||||
| Market | Format | Examples |
|
||||
|--------|--------|---------|
|
||||
| A-shares | `NNNNNN.SZ/SH/BJ` | 000001.SZ, 600000.SH |
|
||||
| US stocks | `TICKER.US` | AAPL.US, MSFT.US |
|
||||
| HK stocks | `NNN(N).HK` | 700.HK, 9988.HK |
|
||||
| Crypto | `SYMBOL-USDT` | BTC-USDT, ETH-USDT |
|
||||
| Futures | `XXNNNN.EXCHANGE` | CU2406.SHFE |
|
||||
| Forex | `XXX/YYY` | USD/CNY, EUR/USD |
|
||||
|
||||
## Fallback Chain (Runner Layer)
|
||||
|
||||
The backtest runner implements automatic fallback at the market level:
|
||||
|
||||
```
|
||||
User requests 000001.SZ (A-share)
|
||||
-> detect market: a_share
|
||||
-> try tushare: TUSHARE_TOKEN missing -> skip
|
||||
-> try akshare: available -> use akshare
|
||||
-> success (zero config required)
|
||||
```
|
||||
|
||||
This is transparent to the user — they just see results.
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: defi-yield
|
||||
description: DeFi yield analysis and optimization — lending rates, LP yields, staking returns, yield farming strategies, risk-adjusted yield comparison, and protocol-level sustainability assessment.
|
||||
category: crypto
|
||||
---
|
||||
# DeFi Yield Analysis & Optimization
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: doc-reader
|
||||
description: Read PDF documents (papers, annual reports, research reports), automatically extracting text pages and applying OCR to image/scanned pages. Use the `read_document` tool.
|
||||
category: tool
|
||||
---
|
||||
# PDF Document Reading
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: earnings-forecast
|
||||
description: 盈利预测与一致预期分析（自上而下/自下而上预测法/SUE/PEAD/分析师预期修正），捕捉业绩超预期交易机会。
|
||||
category: analysis
|
||||
---
|
||||
# 盈利预测与一致预期
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: earnings-revision
|
||||
description: Earnings estimate revisions, guidance analysis, and post-earnings drift (PEAD) — track analyst consensus changes, earnings surprise patterns, and management guidance shifts for US/HK equities.
|
||||
category: analysis
|
||||
---
|
||||
# Earnings Revision & Guidance Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: edgar-sec-filings
|
||||
description: SEC EDGAR filing analysis — 10-K, 10-Q, 8-K, proxy statements, insider Form 4. Extract key financials, risk factors, management discussion, and generate investment signals from US public company filings.
|
||||
category: flow
|
||||
---
|
||||
# SEC EDGAR Filing Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: elliott-wave
|
||||
description: Elliott Wave Theory signal engine. Detects swing points through Zigzag, matches 5-wave impulse and 3-wave corrective structures, validates them with Fibonacci wave relationships, and generates trend-top / correction-complete signals. Pure in-house pandas implementation.
|
||||
category: strategy
|
||||
---
|
||||
# Elliott Wave Theory
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: etf-analysis
|
||||
description: "ETF分析：产品筛选、费率对比、跟踪误差、流动性评估、策略应用与中国市场ETF量化配置框架。"
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# ETF 分析 Skill
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: event-driven
|
||||
description: Event-driven strategy based on sentiment-scored signals from news, announcements, and macro events. The LLM acts as the NLP engine, and event data follows a CSV schema.
|
||||
category: strategy
|
||||
---
|
||||
# Event-Driven Strategy
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: execution-model
|
||||
description: Trade execution modeling (backtest only) — slippage formulas (linear / square-root impact), VWAP/TWAP execution logic, market-impact cost estimation, and execution-assumption configuration.
|
||||
category: strategy
|
||||
---
|
||||
|
||||
# Trade Execution Modeling
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: factor-research
|
||||
description: Factor research framework with IC/IR analysis, quantile backtesting, and factor combination. Suitable for cross-sectional factor evaluation across multiple instruments.
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Factor Research Framework
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: financial-statement
|
||||
description: 财报三表深度解读——三表勾稽关系、盈利质量(应计vs现金流)分析、杜邦分解、10+财务造假红旗指标
|
||||
category: flow
|
||||
---
|
||||
|
||||
# 财报三表解读
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: fund-analysis
|
||||
description: 基金分析与筛选：晨星评级/夏普比率/信息比率、Sharpe风格箱分析、风格漂移检测、基金经理评价、FOF组合构建、ETF选择
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# 基金分析与筛选
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: fundamental-filter
|
||||
description: Fundamental factor screening — filter stocks by PE/PB/ROE and other financial metrics for value or growth selection. Supports A-shares (via tushare extra_fields) and HK/US stocks (via yfinance Ticker info).
|
||||
category: flow
|
||||
---
|
||||
# Fundamental Factor Screening
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: geopolitical-risk
|
||||
description: "Geopolitical risk analysis: quantify crisis signals, identify precursors, and build event-driven strategies for war, sanctions, and supply disruption scenarios."
|
||||
category: tool
|
||||
---
|
||||
|
||||
# Geopolitical Risk Analysis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: global-macro
|
||||
description: Global macro analysis framework (central bank policy transmission / FX forecasting / geopolitical risk / capital flows), used to build macro factor signals that drive cross-asset allocation.
|
||||
category: analysis
|
||||
---
|
||||
# Global Macro Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: harmonic
|
||||
description: Harmonic Patterns signal engine. Identifies XABCD five-point structures such as Gartley/Bat/Butterfly/Crab based on Fibonacci geometry, and generates trading signals in the PRZ (Potential Reversal Zone).
|
||||
category: strategy
|
||||
---
|
||||
# Harmonic Patterns
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: hedging-strategy
|
||||
description: Hedging strategy design (beta hedge / option protection / tail risk / cross-asset hedging), including hedge-ratio calculation and cost evaluation.
|
||||
category: asset-class
|
||||
---
|
||||
# Hedging Strategy Design
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: hk-connect-flow
|
||||
description: Stock Connect (Shanghai/Shenzhen-Hong Kong) fund flow analysis — Northbound (foreign into A-shares), Southbound (mainland into HK), sector allocation tracking, and cross-border arbitrage signals.
|
||||
category: flow
|
||||
---
|
||||
# Stock Connect Fund Flow Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: ichimoku
|
||||
description: Ichimoku Kinko Hyo five-line system signal engine. A standalone Japanese technical-analysis school that generates trading signals from Tenkan/Kijun crossovers, cloud position, and Chikou confirmation. Pure pandas implementation.
|
||||
category: strategy
|
||||
---
|
||||
# Ichimoku Kinko Hyo
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: liquidation-heatmap
|
||||
description: Liquidation level analysis and heatmap interpretation — identify leveraged position concentration, liquidation cascades, stop-hunt zones, and use liquidation data as support/resistance signals.
|
||||
category: crypto
|
||||
---
|
||||
# Liquidation Heatmap & Level Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: macro-analysis
|
||||
description: Macroeconomic cycle positioning and central-bank policy interpretation, including GDP/CPI/PMI/rates/FX analysis, with output in the form of major-asset allocation tilts.
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Macroeconomic Analysis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: market-microstructure
|
||||
description: "Market microstructure: bid-ask spread analysis, order-flow toxicity metrics (VPIN / Kyle lambda), liquidity measures (Amihud / Roll), price-impact models, limit-order-book analysis, and China A-share call auction / block trade mechanics."
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Market Microstructure
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: minute-analysis
|
||||
description: Minute-level data analysis and backtesting. Retrieves minute candlesticks through OKX/Tushare/yfinance and can be used both for analysis and as input to the backtest engine.
|
||||
category: strategy
|
||||
---
|
||||
# Minute-Level Data Analysis and Backtesting
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: ml-strategy
|
||||
description: Machine-learning predictive strategy based on sklearn walk-forward training, feature engineering, and signal generation. Suitable for any OHLCV data.
|
||||
category: strategy
|
||||
---
|
||||
# Machine-Learning Predictive Strategy
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: multi-factor
|
||||
description: Multi-factor cross-sectional stock ranking. Combines factor standardization, equal-weight or IC-weighted scoring, and TopN portfolio construction. Suitable for multi-instrument portfolio strategies.
|
||||
category: strategy
|
||||
---
|
||||
# Multi-Factor Cross-Sectional Stock Ranking
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: okx-market
|
||||
description: OKX cryptocurrency market data interface. Uses the OKX V5 REST API to retrieve spot, derivatives, index, and other crypto market data, including real-time prices, candlesticks, funding rates, open interest, and more. No authentication required, free to use.
|
||||
category: data-source
|
||||
---
|
||||
# OKX Market
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: onchain-analysis
|
||||
description: On-chain data analysis — active addresses / whale tracking / TVL / DEX liquidity, interpretation and signal generation using on-chain valuation metrics such as MVRV / NVT / SOPR.
|
||||
category: crypto
|
||||
---
|
||||
|
||||
# On-Chain Data Analysis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: options-advanced
|
||||
description: "Advanced options strategies: volatility-surface modeling (SABR / Local Vol), dynamic Greeks rebalancing, calendar spreads, volatility arbitrage and skew trading, and option market-making basics."
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# Advanced Options Strategies
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: options-payoff
|
||||
description: "Option P&L analysis methodology: payoff diagrams, breakeven calculation, multi-leg strategy visualization, and Greeks-based scenario analysis."
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# Options Payoff — Option P&L Analysis Methodology
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: options-strategy
|
||||
description: Options strategy framework supporting Black-Scholes pricing, Greeks analysis, and multi-leg backtesting. Suitable for cryptocurrency and equity options.
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: pair-trading
|
||||
description: Pair trading strategy. Trades mean reversion using the spread/ratio Z-score of two correlated instruments. Requires at least two instruments.
|
||||
category: strategy
|
||||
---
|
||||
# Pair Trading Strategy
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: performance-attribution
|
||||
description: Performance attribution analysis — Brinson sector/stock-selection attribution, factor alpha/beta decomposition, market-timing evaluation, and benchmark comparison framework.
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Performance Attribution Analysis
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: perp-funding-basis
|
||||
description: Perpetual futures funding rate analysis and cash-carry basis trading — funding rate regimes, annualized basis signals, carry trade construction, and funding rate arbitrage between exchanges.
|
||||
category: crypto
|
||||
---
|
||||
# Perpetual Funding Rate & Basis Trading
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: pine-script
|
||||
description: Convert Python backtest strategies to TradingView Pine Script v6, or generate Pine Script from natural language descriptions.
|
||||
category: tool
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: quant-statistics
|
||||
description: "Quantitative statistical methods: ADF unit-root / cointegration tests, GARCH volatility modeling, regression diagnostics (heteroskedasticity / autocorrelation), Bootstrap, and hypothesis testing."
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Quantitative Statistical Methods
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: regulatory-knowledge
|
||||
description: 金融监管知识库：A股涨跌停/ST退市新规/融券、港股T+0/做空机制、美股PDT/熔断、加密监管政策、跨境税务基础
|
||||
category: tool
|
||||
---
|
||||
|
||||
# 金融监管知识库
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: report-generate
|
||||
description: Professional financial research report generation — standard structure (summary / views / main body / risks / recommendation), Markdown formatting standards, rating system, and terminology guide.
|
||||
category: tool
|
||||
---
|
||||
|
||||
# Professional Research Report Generation
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: risk-analysis
|
||||
description: Risk measurement and stress testing — VaR/CVaR/max drawdown calculation, Monte Carlo simulation, extreme-value tail-risk analysis, and historical scenario stress testing.
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Risk Measurement and Stress Testing
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: seasonal
|
||||
description: Seasonal/calendar-effect strategy. Generates trading signals from time-based patterns such as month-of-year effects and day-of-week effects. Suitable for any OHLCV data.
|
||||
category: strategy
|
||||
---
|
||||
# Seasonal / Calendar Effect Strategy
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: sector-rotation
|
||||
description: 行业轮动分析——申万行业景气度评分、行业动量排名、产业链传导、估值/盈利/资金流多维比较框架
|
||||
category: asset-class
|
||||
---
|
||||
|
||||
# 行业轮动分析
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: sentiment-analysis
|
||||
description: 市场情绪分析——恐贪指数/Put-Call Ratio/融资融券/北向资金信号解读、社交媒体舆情量化框架
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# 市场情绪分析
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: smc
|
||||
description: Smart Money Concepts (ICT) signal engine. Uses the smartmoneyconcepts library to implement institutional-trading-school analysis of BOS, ChoCH, FVG, and order blocks (OB).
|
||||
category: strategy
|
||||
---
|
||||
# Smart Money Concepts
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: social-media-intelligence
|
||||
description: "Social media intelligence: financial signal extraction from Twitter/X, Telegram, Discord, and Reddit for sentiment-driven trading strategies."
|
||||
category: tool
|
||||
---
|
||||
|
||||
# Social Media Intelligence
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: stablecoin-flow
|
||||
description: Stablecoin supply and flow analysis — USDT/USDC mint-burn signals, exchange stablecoin reserves, on-chain stablecoin velocity, and capital rotation indicators for crypto market timing.
|
||||
category: crypto
|
||||
---
|
||||
# Stablecoin Flow Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: strategy-generate
|
||||
description: Create, modify, and optimize quantitative trading strategies, then backtest and evaluate them.
|
||||
category: strategy
|
||||
---
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: technical-basic
|
||||
description: Core technical indicator collection (trend EMA/ADX + mean-reversion BB/RSI + volume-price OBV/volume ratio), generates a composite signal via three-dimensional voting. Pure pandas implementation for any OHLCV data.
|
||||
category: strategy
|
||||
---
|
||||
# Core Technical Indicator Collection
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: token-unlock-treasury
|
||||
description: Token unlock schedule analysis and project treasury tracking — vesting cliffs, linear unlocks, team/investor/ecosystem token releases, treasury diversification, and sell pressure forecasting.
|
||||
category: crypto
|
||||
---
|
||||
# Token Unlock & Treasury Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: tushare
|
||||
description: tushare是一个财经数据接口包，拥有丰富的数据内容，如股票、基金、期货、数字货币等行情数据，公司财务、基金经理等基本面数据。该模块通过标准化API方式统一了数据资产的对外服务方式，以帮助有需要的技术用户更实时、简洁、轻量的使用相关数据。
|
||||
category: data-source
|
||||
---
|
||||
# Tushare
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: us-etf-flow
|
||||
description: US ETF fund flow analysis, sector rotation breadth, and style factor flows — track institutional capital movement via ETF creation/redemption, sector breadth signals, and thematic momentum.
|
||||
category: flow
|
||||
---
|
||||
# US ETF Flow & Sector Breadth Analysis
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: valuation-model
|
||||
description: Valuation methodology — absolute valuation with DCF / DDM / SOTP, relative valuation with PE-Band / PB-ROE / EV-EBITDA, sensitivity analysis, and valuation-trap detection.
|
||||
category: analysis
|
||||
---
|
||||
|
||||
# Valuation Methodology
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: volatility
|
||||
description: Volatility strategy. Trades mean reversion based on percentile ranking of historical volatility (HV). Suitable for any OHLCV data.
|
||||
category: strategy
|
||||
---
|
||||
# Volatility Strategy
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: web-reader
|
||||
description: Read web pages, articles, and document links by converting URLs into Markdown text. Use the `read_url` tool directly, without bash.
|
||||
category: tool
|
||||
---
|
||||
# Web Reading
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
name: yfinance
|
||||
description: yfinance global market data interface — retrieve OHLCV, financials, insider transactions, and institutional holdings for US stocks, HK stocks, ETFs, and indices via Yahoo Finance. Free, no API key required.
|
||||
category: data-source
|
||||
---
|
||||
# yfinance
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ def build_registry() -> ToolRegistry:
|
||||
from src.tools.task_tools import TaskCreateTool, TaskUpdateTool, TaskListTool, TaskGetTool
|
||||
from src.tools.background_tools import BackgroundRunTool, CheckBackgroundTool
|
||||
from src.tools.web_reader_tool import WebReaderTool
|
||||
from src.tools.web_search_tool import WebSearchTool
|
||||
from src.tools.doc_reader_tool import DocReaderTool
|
||||
from src.tools.factor_analysis_tool import FactorAnalysisTool
|
||||
from src.tools.options_pricing_tool import OptionsPricingTool
|
||||
@@ -31,7 +32,7 @@ def build_registry() -> ToolRegistry:
|
||||
PatternTool(), CompactTool(), SubagentTool(),
|
||||
TaskCreateTool(), TaskUpdateTool(), TaskListTool(), TaskGetTool(),
|
||||
BackgroundRunTool(), CheckBackgroundTool(),
|
||||
WebReaderTool(), DocReaderTool(),
|
||||
WebReaderTool(), WebSearchTool(), DocReaderTool(),
|
||||
FactorAnalysisTool(), OptionsPricingTool(), SwarmTool()]:
|
||||
registry.register(tool)
|
||||
return registry
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
"""Web search tool: search the web via DuckDuckGo (free, no API key)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from src.agent.tools import BaseTool
|
||||
|
||||
|
||||
class WebSearchTool(BaseTool):
|
||||
"""Search the web via DuckDuckGo and return top results."""
|
||||
|
||||
name = "web_search"
|
||||
description = (
|
||||
"Search the web via DuckDuckGo. Returns top results with title, URL, "
|
||||
"and snippet. Use this to find information, news, or URLs before "
|
||||
"reading them with read_url."
|
||||
)
|
||||
parameters = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": "Search query",
|
||||
},
|
||||
"max_results": {
|
||||
"type": "integer",
|
||||
"description": "Maximum number of results to return (default 5, max 10)",
|
||||
"default": 5,
|
||||
},
|
||||
},
|
||||
"required": ["query"],
|
||||
}
|
||||
repeatable = True
|
||||
|
||||
def execute(self, **kwargs: Any) -> str:
|
||||
"""Run a DuckDuckGo search.
|
||||
|
||||
Args:
|
||||
**kwargs: Must include query; optionally max_results.
|
||||
|
||||
Returns:
|
||||
JSON with search results or error.
|
||||
"""
|
||||
query = kwargs["query"]
|
||||
max_results = min(int(kwargs.get("max_results", 5)), 10)
|
||||
|
||||
try:
|
||||
try:
|
||||
from ddgs import DDGS
|
||||
except ImportError:
|
||||
from duckduckgo_search import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
raw = list(ddgs.text(query, max_results=max_results))
|
||||
|
||||
results = [
|
||||
{
|
||||
"title": r.get("title", ""),
|
||||
"url": r.get("href", ""),
|
||||
"snippet": r.get("body", ""),
|
||||
}
|
||||
for r in raw
|
||||
]
|
||||
return json.dumps(
|
||||
{"status": "ok", "query": query, "results": results},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
except ImportError:
|
||||
return json.dumps(
|
||||
{
|
||||
"status": "error",
|
||||
"error": "DuckDuckGo search package not installed. Run: pip install ddgs",
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
except Exception as exc:
|
||||
return json.dumps(
|
||||
{"status": "error", "error": str(exc)},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
Reference in new issue
Block a user