See also (wiki): wiki/quant-asset-management-ai.md · top-hedge-fund-ai-agents-failure-modes-2026.md · asset-management-quant-ai-gap-map-2026.md
Source ledger: sources/06-industry-verticals/quant-ai-infrastructure-moats-2026-raw.md · G-Research standalone source card: sources/06-industry-verticals/g-research-production-llm-code-review-raw.md
Executive Summary
The repo had good coverage of hedge-fund AI agents, finance benchmarks, and asset-manager operating models, but it was thin on a separate lane: infrastructure as the quant AI moat.
Two official-source signals fill that gap:
- XTX Markets: machine-learning price forecasts over 50,000+ instruments, 25,000 GPUs, 650 PB usable storage, and TernFS, an open-sourced filesystem born from internal ML storage needs.
- G-Research: a quant-ML platform that publicly documents a production LLM code-review tool with source-of-truth validation, structured output, provider abstraction, two-pass verification, cost telemetry, and CI/CD adoption.
These are not public alpha claims. They are better than alpha claims for benchmark design: they show what serious quant AI systems need underneath the model. The winning pattern is an untrusted model inside deterministic, observable, permissioned, reproducible infrastructure.
What This Lane Adds
| Layer | Public Signal | StateBench Implication |
|---|---|---|
| Research storage | XTX TernFS: 500+ PB production deployment, immutable files, multi-region, snapshots, TB/s peak | Score artifact immutability, lineage, retry semantics, and partial-write safety |
| Compute scale | XTX homepage: 25,000 GPUs, 650 PB usable storage | Record backend and hardware limits; do not confuse compute with agent quality |
| Data movement | TernFS Linux kernel module and API designed around ML cluster file access | Benchmark high-volume source ingestion and reproducible dataset snapshots |
| Permission boundary | TernFS defers permissions; S3 gateway is coupled to internal auth | Keep auth/entitlements outside the model and score them explicitly |
| LLM trust model | G-Research treats LLM output as unverified input | Validate every agent claim against source ledgers, schemas, and rule IDs |
| Structured outputs | G-Research JSON schema + Pydantic + rules index | StateBench outputs should be machine-validated, not manually eyeballed |
| Recovery | G-Research handles truncation, provider JSON quirks, and one repair attempt | Score graceful degradation and bounded repair loops |
| Evaluation | G-Research tests rule/diff/expected findings, with precision/recall thresholds | Adopt behavior-level evals for finance agents, not exact-word matching |
| Model switching | G-Research provider abstraction and model registry with pricing | Compare Qwen/Gemma/Llama/API models behind one interface with cost logged |
Source Findings
XTX: Storage And Compute Are Part Of The Model
XTX’s homepage is unusually explicit: the firm says it uses machine learning to produce price forecasts across more than 50,000 instruments and lists 25,000 GPUs and 650 PB of usable storage. The TernFS tech post is more useful than the headline compute numbers because it explains why the infrastructure exists.
TernFS grew out of machine-learning research demand. XTX says its research scale moved from desktops and an NFS server to tens of thousands of GPUs, hundreds of thousands of CPUs, and hundreds of PB of storage. Existing storage systems did not fit, so the firm built a filesystem intended for tens of exabytes, trillions of files, and millions of concurrent clients.
The design choices are benchmark-relevant:
- files are immutable once written;
- half-written files are not visible;
- snapshots protect against accidental deletion;
- metadata has no single point of failure;
- the system can span regions;
- Linux-kernel access was kept because ML workflows used file APIs directly;
- permissions are deliberately external to the filesystem.
For our work, this means a finance agent benchmark should not only ask whether a model can read a report. It should ask whether the system can create reproducible dataset snapshots, preserve source lineage, avoid partial artifacts, and keep permission/authentication decisions outside model generation.
G-Research: Production LLMs Are Untrusted Components
G-Research’s May 13, 2026 code-review article, now promoted as standalone
source-acquisition task sa-130, is not about alpha, but it is one of the
best public quant-firm LLM engineering disclosures. The Data and Analytics team
provides core infrastructure across research, cybersecurity, and risk
management. Their LLM tool checks git diffs against internal Python coding
standards, runs in CI/CD, and posts non-blocking PR comments.
The technical lesson is directly portable to finance agents. G-Research does not let the LLM define the rules. The standards document is the source of truth; the model emits structured findings; Pydantic validation checks every finding against an authoritative rules index; deterministic fields such as severity are derived outside the model.
The failure handling is equally useful:
- detect truncation from finish reason and retry with a lower findings cap;
- send validation failures back for one structural repair attempt;
- normalize provider-specific JSON wrappers;
- split recall and precision into two calls;
- test behavior, not wording;
- require 100% recall for mandatory rules, 75% for recommendations, no false positives, and overall precision above 85%;
- keep cost visibility per call;
- keep the tool non-blocking because a human still owns the review.
For StateBench, this should become a finance-agent harness pattern. A model that produces a research note, feature card, ESG claim, code diff, or backtest result should be treated as untrusted until schema validation, source-ledger validation, deterministic recomputation, and review gates pass.
What This Means For Systematic Quant Benchmarks
institutional quant AI evaluation should include the infrastructure work that makes research output trustworthy:
- Immutable research artifact task: create a source bundle, feature file, notebook, and report where every artifact has a stable path, hash, input manifest, and timestamp.
- Permissioned retrieval task: answer only from entitled sources, log the connector/tool used, and abstain when the requested source is not available.
- Schema-validation task: emit findings against an authoritative rule or metric index; invented rule IDs fail.
- Two-pass verification task: first maximize recall, then filter false positives with a separate precision pass.
- Provider-switch task: run the same evaluation through API, local MLX, llama.cpp/GGUF, CUDA, or Neuron-compatible backends and record cost, latency, schema failures, and quality deltas.
- Failure-recovery task: handle truncation, malformed JSON, provider wrapper drift, and partial artifacts with bounded repair attempts.
- Human-review task: produce non-blocking findings with clear evidence, reviewer options, and escalation paths.
Pass condition: the model improves research throughput without becoming the source of truth. Deterministic systems own identity, permissions, storage integrity, calculation, validation, and final authority.
Candidate Model And Runtime Implications
- Local small models: Qwen3/Gemma/Llama-family SLMs are plausible for rule checks, source-card extraction, CI review, and schema-constrained comments if they meet strict precision/recall thresholds.
- LiquidAI/LFM2-style runtime models: useful candidates for low-latency extraction, routing, and code/comment triage, but they need the same behavior-level tests.
- Frontier APIs: useful for first-pass recall and complex synthesis, but must be wrapped in provider abstraction, cost telemetry, and source-of-truth validation.
- llama.cpp/GGUF and MLX: good local-serving targets for CI-like tasks when quantization preserves schema fidelity and rule recall.
- AWS Neuron: only relevant if the selected model and operators are compatible with Inferentia/Trainium. The harness should record unsupported operators, context limits, latency, and cost rather than assuming CUDA parity.
Do not treat compute scale as evidence that a model can perform finance research. Treat compute scale as evidence that the benchmark must include dataset movement, artifact integrity, and reproducibility.