See It Think for Yourself
The Black Box is an air-gapped, self-contained evaluation environment. No telemetry. No network calls. Your data, your tests, your results.
Results achieved in a completely air-gapped environment with zero external connectivity.
What is the Black Box?
The Black Box is a completely self-contained instance of CosmicMind that runs on hardware with zero external connectivity. It's designed for organizations that need to verify our claims independently before committing.
Air-gapped by design: no telemetry, no phone-home, no external API calls. Everything runs locally on ollama/llama3.1:8b —load your own data, define your own test facts, and run the same Needle-in-a-Haystack protocol we use internally. The results speak for themselves.
What You'll See
The Black Box runs a Multiple Needle-in-a-Haystack (NIAH) protocol: 10 simultaneous needles hidden across 11 document depths and 9 context scales, from 8K to 20M tokens. Below is what the results look like.
NIAH accuracy (%) across context lengths and document depths. Tested on ollama/llama3.1:8b in a fully air-gapped environment.
How It Compares
CosmicMind maintains near-perfect accuracy while staying sub-second at every scale. See the full benchmark data for more.
Accuracy vs Context Stuffing
Retrieval Speed at Scale
Evaluator Capabilities
The CosmicMind Black Box is a fully self-contained, air-gapped demonstration unit designed for independent evaluation. It operates with no internet connectivity, no telemetry, and no data exfiltration of any kind.
Evaluators can:
- Load their own data — any text corpus, any size, any domain
- Define their own test facts — specific information to be retrieved
- Run the full benchmark suite independently
- Verify all inputs, outputs, and timing with complete transparency
- Confirm the capability without accessing any proprietary technology
The Black Box exists so that every claim can be independently verified. We do not ask anyone to take our word for it. We ask them to test it themselves.
To schedule a Black Box evaluation: info@pansynapse.com
Hardware & Honest Limits
The Black Box runs on modest hardware. Here's what the Light tier looks like.
Light
Edge & mobile deploymentHardware: M-series Mac · Jetson Orin · High-end laptop
Memory: 16–32 GB RAM, SSD
100%
NIAH 20M
87.5%
Fiction.liveBench
<1s
Retrieval
Where We Fall Short
We publish where CosmicMind falls short. Transparency is non-negotiable.
LongMemEval (s subset)
On the single-session subset of LongMemEval, RAG-based approaches outperform CosmicMind by 2.6 percentage points. CosmicMind is optimized for cross-session synthesis, not single-turn lookups.
EgoLifeQA
EgoRAG, a retrieval system purpose-built for egocentric video QA, outperforms CosmicMind by 10–20pp on this multimodal benchmark. CosmicMind is a text-native system and does not yet ingest video or image data.
See the full benchmarks page for complete methodology and results.
Schedule Your Evaluation
See CosmicMind think for yourself. We'll set up a Black Box instance on your infrastructure.