Remember when diffusion models for images were a curiosity that serious NLP people politely ignored? For a couple of years, the token-by-token autoregressive decoder was simply how text generation worked, and anything that generated all its output at once was filed under “interesting, but not for language.” Then the parallel-generation crowd kept publishing, kept shipping, and the polite ignoring stopped.
I bring that up because I went through a smaller version of the same arc. A year ago I built non-autoregressive decision models trained with reinforcement learning. In September 2026, a well-funded frontier lab called TypeSafe AI, founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI, launched Jev with the same non-autoregressive decision approach. A lab called it a breakthrough in 2026. My version is called Laya, it runs at 33ms, it handles multiple languages, and it outputs calibrated probabilities. You can install it with pip install laya.
This piece is not a grievance. It is a set of notes on why the idea kept nagging at me, what it feels like when a bigger name arrives at your intersection, and what bot builders should take from it.
Why I stopped generating decisions one token at a time
The motivation was boringly practical. I build bots. Bots make small decisions constantly: is this message a refund request, does this input need escalation, which tool should fire, is this user frustrated. Those are System 1 decisions. Fast, low-deliberation, high-volume.
Running each of them through an autoregressive decoder felt like hiring a philosopher to read a thermometer. You pay for sequential generation you do not need, because the answer is not a paragraph. It is a label and a confidence number. Generating that label one token at a time adds latency for no informational gain.
So the design goal became: produce the whole decision in a single forward pass, and make the confidence number mean something. That second half is the part people skip. A classifier that says 0.93 and is right 60% of the time is worse than useless in a bot pipeline, because you will build routing logic on top of that number and the logic will quietly rot.
Where RL came in
Reinforcement learning showed up because decision quality is not the same thing as label accuracy. In a live bot, a wrong-but-confident answer costs more than an uncertain-but-honest one. Training against that asymmetry is easier to express as a reward than as a loss over ground-truth labels.
That combination, non-autoregressive decoding plus RL on decision quality, is what TypeSafe AI proposed with Jev. Same shape of idea, different starting point. They came at it from frontier-lab research. I came at it from being tired of latency in my own bots.
What 33ms actually changes
Latency numbers get thrown around as bragging rights, but for bot architecture the threshold matters more than the digits. Laya’s 33ms figure sits in a range that changes what you are willing to do:
- You can put a decision model inside a loop instead of at the entrance to one.
- You can call it several times per user turn without the user feeling it.
- You can use it as a cheap gate in front of a large, slow model, which means the large model only runs when the decision engine says it should.
- You can afford to ask it questions you would never spend a full LLM call on.
That last point is the one I underestimated. When a decision costs almost nothing, you start asking for decisions you previously guessed at in code. The architecture gets simpler because fewer heuristics are hand-written.
On being early and unfunded
The honest lesson is that being first with an idea is not the same as being the reason the idea spreads. A frontier lab has distribution, a recognizable founder, and the kind of attention that turns a technique into a named category. My repo had a pip install and a demo space.
What being early does give you is a year of scars. I already know where calibration drifts, where multilingual behavior gets strange, and which parts of the pipeline break under real traffic. That knowledge does not come from the idea. It comes from running it.
If you are building bots right now, the practical takeaway is smaller than the news cycle around it. Look at your pipeline and count how many places you are using a sequential generator to produce something that is functionally a label plus a confidence score. Every one of those is a latency tax. Non-autoregressive decision models are not exotic anymore, the tooling exists, and two independent teams landing on the same design is a decent signal that the design is sound.
Being validated by someone with more money than you is a strange kind of compliment. I will take it, and keep shipping.
🕒 Published: