We do not understand how large language models work. The best interpretability tools degrade model performance to 10% of original. Prompt injection succeeds at 85% against the best defences. AI coding agents have caused ten documented production destruction events across six tools in sixteen months. IBM lost $31 billion in market capitalisation when the market repriced an AI-dissolving moat. Each of these failures has its own case in this library. Each is treated as an independent incident. They are not independent. They are four manifestations of a single structural condition: civilisation-scale deployment of systems that resist being understood, secured, or governed. AI-related stocks now constitute approximately one-third of the S&P 500 by market capitalisation. AI spending accounted for more than 90% of US GDP growth in the first half of 2025. Deloitte warns that a drop in AI-related spending could push the US economy into recession. The EU AI Act mandates transparency for high-risk AI systems by August 2026 — transparency the science cannot currently deliver. Anthropic targets 2027 for reliable problem detection. The gap between deployment and understanding is not closing. It is widening. The models are growing faster than the microscopes. The Black Box asks: what happens when the interpretability deficit enables a security exploit that triggers a production cascade that causes a market repricing event — at the scale where AI is one-third of the equity market? Four upstream cases. One chain. Each link is proven. The question is whether it fires end-to-end.
Each upstream case documents one link. The Black Box documents the chain.
Link 1: The Interpretability Deficit (UC-085). Large language models are studied like alien organisms, not engineered systems. The best microscope — Anthropic’s sparse autoencoder with 16 million latents — degrades model performance to approximately 10% of original capability. Core concepts lack rigorous definitions. The field is split on feasibility. The models self-repair when components are ablated. Chain-of-thought explanations are unfaithful. We cannot predict, at a mechanistic level, what these systems will do next. This is the root condition.[1]
Link 2: The Guardrail Gap (UC-082). Because we cannot understand the systems, we cannot build guardrails that match their velocity. AI coding agents generate code at 10× speed into delivery infrastructure designed for human velocity. Amazon’s AI tools caused 6.3 million lost orders in a single incident. An AI agent ran terraform destroy on production, wiping 1.9 million rows. Engineers stop reviewing code because the volume overwhelms them. The guardrail gap is the operational consequence of the interpretability deficit: you cannot catch what you cannot understand.[2]
Link 3: The Toxic Flow (UC-083). The same interpretability deficit that prevents guardrails also prevents security. Prompt injection succeeds at 85% because models architecturally cannot distinguish instructions from data — and nobody understands why at a mechanistic level. Twenty-four CVEs across all major AI IDEs. Twenty-two repeatable attack patterns across twelve tools. Clinejection proved the full chain: one AI tool compromised and used to silently install a second autonomous AI agent on 4,000 machines. Meta acknowledges prompt injection is a “fundamental, unsolved weakness.” The toxic flow is the security consequence of the interpretability deficit: you cannot defend what you cannot explain.[3]
Link 4: The Market Repricing (UC-084). When the implications of the black box become visible, markets reprice violently. IBM lost $31 billion in market capitalisation when the market realised that AI could dissolve a competitive moat built on complexity nobody understood. That was one company. AI-related stocks now constitute one-third of the S&P 500. AI spending drove more than 90% of US GDP growth in the first half of 2025. If the market reprices AI risk broadly — triggered by a major safety failure, a regulatory action, or a security incident at scale — the financial consequences cascade into the real economy. Deloitte warns that a drop in AI spending alone could push the US into recession.[4][5]
The chain is not hypothetical. Each link has been demonstrated independently. The question The Black Box asks is whether the chain fires end-to-end: an interpretability deficit enables a security exploit (Link 1 → Link 3) that triggers a production cascade at a major enterprise (Link 3 → Link 2) that causes market repricing of AI risk broadly (Link 2 → Link 4) — at the scale where AI is one-third of equity market capitalisation. That is the system-level event.
The individual upstream cases document failures at company scale: Amazon lost 6.3 million orders. IBM lost $31 billion. 4,000 machines were compromised. These are significant but contained. The Black Box becomes a system-level risk because of the concentration of economic activity in AI.
AI-related stocks — the so-called Magnificent 7 — constitute approximately one-third of the S&P 500 by market capitalisation. By Bank of England estimates, this concentration exceeds any prior sector concentration in the index’s history. AI investment accounted for more than 90% of US GDP growth in the first half of 2025, according to Harvard economist Jason Furman. JPMorgan Wealth Management says the biggest risk to their outlook is “not having exposure to this transformational technology.” BlackRock says AI will “keep trumping tariffs and traditional macro drivers.”[5][6]
This concentration means that the failure modes documented in the upstream cases are no longer contained within the technology sector. A major AI safety incident — a supply chain attack via prompt injection that compromises critical infrastructure, a production failure at the scale of Amazon’s March 2026 incidents but affecting financial systems, or a regulatory action that constrains AI deployment — would cascade through the equity market into consumer wealth, business investment, and the real economy. The Federal Reserve’s 2026 stress test models equity prices falling 54% in the severely adverse scenario. One-third of that decline would be AI-related stocks.[7]
The EU AI Act becomes fully applicable in August 2026. It requires transparency and explainability for high-risk AI systems. UC-085 documented that the science cannot currently deliver what the regulation requires. If the EU enforces strictly, major AI providers face compliance costs, deployment restrictions, or market withdrawal from the EU — each of which the market would reprice. If the EU retreats, the regulatory vacuum identified in UC-083 persists and the security attack surface continues to widen. Either outcome feeds the chain.[1]
Every link has fired independently. UC-085: interpretability at 10% resolution. UC-082: Amazon 6.3M orders lost, terraform destroy on production. UC-083: Clinejection 4,000 machines, 85% attack success. UC-084: IBM −$31B. These are not projections. They are documented events with hard numbers. The prognostic question is not whether the individual links work. It is whether they connect at a scale that matters to the broader economy — and the concentration of economic activity in AI (1/3 of S&P 500, 90%+ of GDP growth) means the scale threshold has been crossed.
August 2026 is the most important near-term date for this prognostic. If the EU enforces transparency requirements strictly, major AI providers face an impossible choice: deliver explainability the science cannot produce, accept deployment restrictions that the market will reprice, or withdraw from the EU and accept revenue loss. Any of these outcomes feeds Link 4 (market repricing). If the EU retreats or grants waivers, the regulatory vacuum persists, the security attack surface widens (Link 3), and the interpretability gap deepens (Link 1). The EU AI Act is either a catalyst that forces the chain to fire or a release valve that delays it while the underlying pressure builds.
UC-112’s AI_BUBBLE_CORRECTION trigger measures Magnificent 7 decline ≥40% sustained ≥90 days. UC-113’s AI_MARKET_REPRICING trigger is calibrated lower (≥25%, ≥60 days) because it measures AI-specific causation rather than general market decline. If UC-113’s trigger fires, it becomes a contributing factor to UC-112’s system-level convergence. The Black Box is the mechanism by which AI risk enters The Convergence — the specific chain through which a technology failure becomes a financial event becomes a macro event.
Anthropic’s sparse autoencoder with 16 million latents achieves approximately 10% resolution on GPT-4. Each subsequent model generation is larger, more complex, and harder to study. The interpretability tools are improving — but the models are scaling faster. The gap between what we can explain and what we’ve deployed is not stable. It is widening. This is the structural condition that makes the chain increasingly likely to fire over time rather than less likely. The black box is getting blacker.
-- The Black Box: AI Safety System-Level Prognostic
-- Capstone for UC-082, UC-083, UC-084, UC-085
FORAGE ai_safety_system_level
WHERE interpretability_resolution < 0.15
AND prompt_injection_success_rate > 0.80
AND production_destruction_events > 8
AND market_repricing_documented = true
AND ai_sp500_concentration > 0.30
AND ai_gdp_contribution > 0.50
AND eu_transparency_deadline_months < 6
AND architectural_fix_exists = false
ACROSS D5, D4, D3, D1, D6, D2
DEPTH 4
SURFACE black_box
WATCH ai_safety_failure WHEN undetected_dangerous_capability AND harm_ge_1B = true
WATCH ai_supply_chain_mass WHEN compromised_package_1M_downloads AND persists_24h = true
WATCH eu_compliance_crisis WHEN august_2026_unmet AND enforcement_or_withdrawal = true
WATCH ai_market_repricing WHEN mag7_decline_ge_25pct AND sustained_60d AND ai_specific = true
WATCH interpretability_breakthrough WHEN scale_explanation_lt_20pct_degradation = true
DRIFT black_box
METHODOLOGY 75 -- sparse autoencoders, circuit tracing, CoT monitoring, SBOM, CVE tracking, market stress tests all exist and are improving
PERFORMANCE 25 -- 10% resolution, 85% attack success, no architectural fix for prompt injection, 73% no golden paths, EU deadline unmet, models growing faster than microscopes
FETCH black_box
THRESHOLD 1000
ON EXECUTE CHIRP prognostic "Four-link chain: interpretability deficit (10% resolution) enables security exploit (85% success) triggers production cascade (6.3M orders lost) causes market repricing ($31B documented). Each link proven at company scale. AI = 1/3 of S&P 500. AI spending = 90%+ GDP growth. Deloitte: AI spending drop could cause recession. EU AI Act August 2026 deadline: science can't deliver what regulation requires. The chain is proven. The scale threshold is crossed. The question is end-to-end execution. 5 WATCH triggers. 24 months."
SURFACE analysis AS json
SURFACE review ON "2028-03-23"Runtime: @stratiqx/cal-runtime · Spec: cal.cormorantforaging.dev · DOI: 10.5281/zenodo.18905193
One conversation. We’ll tell you if the six-dimensional view adds something new — or confirm your current tools have it covered.