A Sovereign, Open-Source Foundation Model for German and English

Soofi S 30B-A3B is a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba–Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment.

Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14–27B models on aggregate English and German benchmarks, achieves the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison — including ones far larger in active parameters. Among fully open models, it obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC-scale AI infrastructure operated by Deutsche Telekom in Munich.

🧩 Highlights

1. 🏆 German–English champion: best English and German code aggregates among all 17 measured open base models; strongest fully open model on the English and German aggregates; matches or outperforms every European sovereign baseline on every German benchmark in our suite — at a fraction of the active-parameter cost of dense 14–27B models.
2. 📋 Full data transparency: complete per-source, per-language token accounting for all three training phases (including sources we evaluated and excluded), with reproducible corpus construction scripts. ~99% of the mixture can be independently reconstructed.
3. 🔁 Reproducible recipe: full Warmup–Stable–Decay learning-rate schedule, optimizer, all hyperparameters, per-phase token budgets, and phase boundaries — a third party can rebuild the run.
4. ⚡ Long-context serving efficiency: only 6 of 52 layers keep a KV cache, so the per-sequence cache stays near-constant with context length. Measured aggregate decode TPS/GPU is 8–9× that of dense 14–24B models at 40K context (batch 32) and stays essentially flat from 4K to 256K.
5. 🇩🇪 Sovereign end to end: trained from 24 March to 13 May 2026 on up to 512 NVIDIA B200 GPUs of the German Industrial AI Cloud (Deutsche Telekom, Munich), under European operational and data-protection requirements.

📊 Results

• Aggregates: English 70.1 / German 79.1 — highest among fully open models (Olmo 3 32B: 67.3 / 69.2; Apertus 70B: 62.4 / 72.8), on par with dense open-weight 14–27B models.
• Code: HumanEval 73.8, MBPP 70.2, MBPP-DE 84.2 — best in both comparison sets.
• Mathematics: GSM8K 86.1, GSM8K-Platinum-DE 87.1, Minerva-500 79.4.
• German: first on every German benchmark against European open-source baselines (GLP-DE 88.8, INCLUDE-DE 61.2, ARC-Challenge-DE 92.3).
• Serving: 4.82k aggregate decode TPS/GPU at 40K context (batch 32, single B200, vLLM) — 9.2× Ministral 3 14B — with near-flat decode throughput from 4K to 256K.

📁 Available Artifacts

• 🤗 Soofi S 30B-A3B Base weights and selected intermediate checkpoints (currently gated during the beta phase; freely available once the beta ends)
• 🛠️ Training code and reproducible data-construction scripts
• 🧪 Evaluation code (base-eval) and eval-hive, incl. parallel English/German suites
• 📈 Weights & Biases dashboard of the full ~27T-token run
• 📋 Exact per-source token accounting for all three phases; commercially licensed sources (Genios) documented via aggregate statistics

All artifacts are released under permissive licenses for transparent audit and extension.

📜 Citation

If you use Soofi S or its artifacts, please cite the report (PDF):

@techreport{soofi2026soofis,
  title       = {A Sovereign, Open-Source Foundation Model for German and English: Soofi S Pretraining Report v1.0},
  author      = {{The Soofi-Team}},
  institution = {Project Soofi},
  year        = {2026},
  note        = {\url{https://huggingface.co/Soofi-Project}}
}