Jev and System One Models: AI That Decides Without Talking — And How Open Source Replicated It in 4 Days

On September 15, 2026, TypeSafe AI (founded by former OpenAI researchers) unveiled a new model called Jev alongside bold marketing claims:

• “40–200x faster than traditional LLMs”
• “100x cheaper”
• “Cannot hallucinate”

Even more remarkably, within just 4 days of the announcement, open-source developers on GitHub launched dozens of replication projects. Some claimed to match Jev’s performance using models as small as 150 million parameters trained in just 30 minutes on a free Google Colab tier!

This article explores what Jev really is, why it operates so quickly, what “cannot hallucinate” actually means in practice, and 3 open-source techniques you can run today to achieve similar results.

---

Part 1: How Traditional LLMs Work (And Why They Are Slow for Decision-Making)

Standard Large Language Models (such as GPT, Claude, DeepSeek, and Qwen) operate in an Autoregressive fashion. They process and generate text word-by-word (token-by-token), feeding newly generated tokens back as input for the next cycle.

The execution is split into two primary phases:

1. Prefill (Input Reading): The model processes the entire input prompt in one parallel pass. On GPUs, this phase is extremely fast.
2. Decode (Generation): The model produces tokens sequentially (e.g., “This” $\rightarrow$ “email” $\rightarrow$ “is” $\rightarrow$ “spam”). This phase is memory-bandwidth bound and slow because each token depends on the previous one, forcing the model to reload its full weight parameters from VRAM on every step.

If you only need an AI to answer a simple question like “Is this email spam?”, a 27B parameter LLM might take 1.5–2 seconds generating 30 tokens of explanation—when all you actually needed was 1 bit of information (Yes or No).

Furthermore, unconstrained text outputs introduce parsing overheads and non-deterministic formatting (e.g., “It seems to be spam, but I’m not entirely sure”), requiring complex downstream logic to extract structured answers.

---

Part 2: What is Jev and System One Model?

TypeSafe AI coined the term System One Model for Jev, borrowing from Daniel Kahneman’s book Thinking, Fast and Slow:

• System 1: Fast, automatic, intuitive thinking (e.g., recognizing anger on a human face instantly).
• System 2: Slow, deliberate, step-by-step logical reasoning (e.g., multiplying three-digit numbers).

While standard LLMs are built for System 2 reasoning, Jev is specifically engineered to act as System 1 for software systems.

Inputs and Outputs of Jev

• Input: Takes a State (contextual text such as an email, chat history, or system state) along with a typed list of queries in 3 categories:
  1. Yes/No (Noul): e.g., “Is this email spam?” $\rightarrow$ Returns a probability score (e.g., 0.93).
  2. Choice: e.g., “Category: Billing / Technical / Cancellation” $\rightarrow$ Returns a probability distribution across choices.
  3. Score: e.g., “Urgency scale 1–5” $\rightarrow$ Returns a probability distribution over the numerical scale.
• Output: Evaluates all queries simultaneously in a single forward pass without generating any text. The model doesn’t “speak”—it merely “points” to probabilities.

How Does Jev Achieve Its Speed and Cost Claims?

• 40–200x Faster: Bypasses the sequential Decode phase entirely, running only the highly parallel Prefill phase.
• 100x Cheaper: Charges solely for input tokens ($0.042 per million tokens) with zero output token fees.
• Cannot Hallucinate: Constrained strictly to user-provided choices, making type/formatting errors impossible. However, “cannot hallucinate” means it cannot produce invalid output schema—it can still make incorrect classification decisions.

Another crucial feature of Jev is Calibration. If Jev reports an 80% confidence level, its predictions are calibrated to be accurate approximately 80% of the time. This is achieved via RLCD (Reinforcement Learning for Calibrated Decisions).

---

Part 3: Why This Paradigm Shift Matters

In production software engineering, most AI tasks are not conversational chats, but rather atomic micro-decisions:

• Categorizing support tickets or incoming emails.
• Risk assessment and spam filtering.
• AI Model Routing: Deciding whether a query requires a large (expensive) model or a small (cheap) model.
• Guardrails: Validating safety and compliance of outputs before showing them to users.

Decoupling the “Decision Model” from the “Speech Model” results in significantly higher throughput and lower compute costs.

---

Part 4: 3 Open-Source Approaches Replicating Jev

Following Jev’s release, the open-source community created 3 primary replication strategies:

Approach 1: Prompt-Logprob (Using Existing LLMs Without Generation)

This approach requires no model fine-tuning and leverages Logits / Prompt Logprobs features available in inference engines like vLLM or SGLang:

1. Append all queries to the State text with placeholder tokens in answer positions.
2. Pass the entire prompt through the model in a single Prefill pass.
3. Extract the Logits / Probability scores for choice tokens (e.g., “A”, “B”, “C”) at the placeholder positions, applying a Softmax normalization to derive choice probabilities.

Benchmark Results (Qwen3.6-27B 8-bit):

• RACE-H Reading Comprehension: 92.9% accuracy (4.6 queries/sec)
• MMLU: 84.2% accuracy

Pros: Works out-of-the-box with existing open LLMs, retains high Zero-shot capabilities.
Projects: ikermoel/open-alternative-jev (so1), ekzhang/openjev-sglang, bnsd55/openjev

---

Approach 2: Small Model + Scoring Head (Fine-Tuned Micro Models)

Designed for ultra-low latency requirements:

1. Use lightweight base models such as Gemma 3 270M or ModernBERT 150M.
2. Attach a small linear layer (Scoring Head) to map hidden states to scalar decision values.
3. Fine-tune the head or LoRA adapters using classification datasets.

Benchmark Results:

• akash-kamat/system-one-gemma (Gemma 3 270M + 2.6M LoRA params): Achieves ~50 ms per decision, 64.4% overall accuracy.
• Heman10x-NGU/openJev-verdict-2.0 (ModernBERT 151M): 77% accuracy, capable of running client-side in browsers via WebGPU.

Pros: Extremely fast (15–50 ms execution time), runs efficiently on edge CPUs or browsers.
Cons: Lower Zero-shot generalization compared to large LLMs.

---

Approach 3: Structured Read on Discrete Diffusion Models

Leverages discrete diffusion architectures such as DiffusionGemma 26B-A4B:

1. Format the answer template with noisy/masked tokens.
2. Execute a single denoising step.
3. Read token probabilities across all answer slots simultaneously.

Since diffusion models utilize bidirectional attention across the entire context window, individual queries do not interfere with each other.

Benchmark Results (RTX PRO 6000):

• Latency: 94 ms per request (3 queries).
• Throughput: Up to 57 requests/sec at 64 concurrent users.

---

Part 5: Comparison of the 3 Open-Source Approaches

Feature	Approach 1: Prompt-Logprob	Approach 2: Small Model + Head	Approach 3: Diffusion Model
Training Required	None	Yes (30 min – 1 hr)	None
Zero-shot Generalization	Very High	Low – Moderate	High
Decision Latency	~150–250 ms	15–50 ms (CPU compatible)	~100 ms
Hardware Requirement	Standard LLM GPU	Any hardware / Edge CPU	VRAM $\ge$ 24 GB
Multi-Language Support	Model dependent (e.g. Qwen)	Depends on training data	Multi-lingual out-of-the-box

---

Key Takeaways

1. Schema enforcement, not guaranteed truth: “Cannot hallucinate” means structured type safety, not immunity from decision errors.
2. Speed comes from skipping the Decode loop: Extracting parallel probabilities in Prefill yields orders-of-magnitude faster execution than sequential token generation.
3. Immediate Production Utility: If you run vLLM or SGLang, you can implement Prompt-Logprob decision pipelines today for routing, filtering, and guardrails without relying on proprietary APIs.

The rapid community response to Jev highlights a growing shift in software engineering: the industry increasingly needs AI models that make fast, structured decisions rather than conversational models that generate long-form text.