It’s 2 a.m. and the router has been chewing on the same design for six hours. You check the log. Ninety-something violations. Shorts where two nets crossed a channel that was never wide enough, spacing errors clustered around a macro edge, a handful of opens that nobody can explain. You bump a few constraints, change a seed, and start again. Tomorrow you’ll do it a third time.
Anyone who has built pathfinding bots knows that feeling. Your agent walks into the same dead end over and over because it has no idea it has been there before. Detailed routing in dense chip layouts is that problem at industrial scale, and a new approach reports a 92% reduction in routing violations with a 10% cut in runtime using history-aware offline reinforcement learning built on an LSTM.
Those two numbers together are what caught my attention. Most optimization tricks trade time for quality. This one reportedly improves both.
Why routing breaks the usual RL playbook
If you’ve written a grid-world agent, you already understand the shape of detailed routing. You have a start pin, an end pin, obstacles, and a cost function. Find a path. Repeat a few million times.
The complication is that every path you commit changes the board for every path after it. Net 4,000 fails because nets 1 through 3,999 already ate the space. A standard reactive policy, the kind that looks at the current state and picks an action, is missing the one thing that matters most: how the congestion got this bad in the first place.
This is the classic partially observable trap. The agent’s snapshot of the grid doesn’t contain the sequence that produced it. Two identical-looking congestion patterns can require completely different strategies depending on what came before. If you’ve ever watched a bot oscillate between two bad choices because each looked fine in isolation, you’ve seen the same failure mode on a smaller canvas.
What history-aware actually buys you
Adding an LSTM to the policy is the direct answer to that trap. Instead of judging the current grid alone, the agent carries a learned summary of the routing trajectory so far. Recurrent state acts as memory of pressure that has already built up.
For bot builders, the practical lesson generalizes well past silicon. If your agent operates in an environment its own past actions reshape, a memoryless policy is fighting with one hand tied. Sequential resource allocation, warehouse task assignment, network traffic scheduling, even multi-step tool use in an agent loop all have this property. The state you observe is a consequence, not a cause.
Offline is the part people underrate
The offline piece matters just as much as the memory. Offline RL learns from a fixed dataset of previously collected experience instead of interacting with a live environment.
For chip routing, that’s close to mandatory. You cannot let an agent freely explore a design flow where a single routing attempt burns hours of compute. Random exploration at that cost is not a training strategy, it’s a budget incident.
Plenty of us building bots are in the same position with cheaper stakes. If every environment step costs an API call, a rate limit, or a real side effect in somebody’s production system, online exploration stops being practical. Offline learning from logged runs is the more honest fit. You probably already have the logs. Past routing attempts, past agent traces, past decisions and their outcomes — that’s your dataset.
Reading the numbers with appropriate skepticism
A 92% drop in violations is a large claim, and the specific setup behind it matters. Violation counts depend heavily on design, technology node, and the baseline you’re comparing against. A 10% runtime improvement is more modest and, honestly, more believable as a sustained result.
Fewer violations is where the real time savings hide anyway. Each violation is manual cleanup or another full iteration. Cutting that loop compounds in a way raw runtime percentages don’t capture on a benchmark table.
Routing isn’t the only place RL is showing up in this problem space either. There’s work on network-on-chip routing algorithms for tiled multicore processors, and more exotic directions like photonic spiking RL for routing decisions. Different layers of the stack, same underlying idea: sequential decisions under congestion are a learning problem, not a heuristics problem.
What I’d take back to my own agents
Two things worth stealing here. First, audit whether your agent needs memory. If its actions change the environment it observes, a recurrent or history-conditioned policy is probably not optional. Second, check your logs before you build a simulator. Offline RL on data you already collected is often the faster path, and it sidesteps the cost of letting an untrained agent loose on something expensive.
Chip routing is a brutal testbed — millions of decisions, hard geometric constraints, no room for a sloppy path. When a technique earns numbers like these there, it’s usually pointing at something more general about how agents should handle their own history.
🕒 Published: