Effectively, underlying that claim is a comparison between two pathways:

You’ll notice that the new pathway bypasses or short-circuits an important layer of abstraction in modern computing, the Instruction Set Architecture (ISA). Associated with that, it also eliminates the compiler. Importantly, it doesn’t eliminate instructions; ultimately, those need to be emitted either way in order to drive the hardware.

The question of what characteristics the ISA should have spawned the great RISC vs. CISC debate in computer engineering. We still use both RISC and CISC architectures, so it’s a subtle debate with merits on both sides. It also had some formative effects on the history of computer engineering, such as:

• The nature of the hardware ecosystem — how much it favors incumbents vs. new entrants

• The nature of hardware innovation — how viable different hardware architectures are

• The process by which software is interfaced to and optimized for hardware — compilers, toolchains, etc.

The ISA debate itself is not the locus of innovation any longer, but it has had substantial influence on how desktop, mobile, and server computing have developed. That’s all history now.

However, what if AI’s capabilities had arrived 30 years sooner, and Musk’s proposed scheme was the accepted software development paradigm? Would the RISC vs. CISC debate look the same?

On a separate note, in my previous post I discussed the importance of action tokens in robotics. Their granularity is an important design choice for robotic systems, and we have seen them range from very fine (joint commands) to very coarse (behavioral primitives). These are analogous to RISC and CISC instructions! However, importantly, robotics does not have the notion of an ISA or a compiler. The results of this alternate history of computing may be quite relevant to answering some of the big questions about how a “robotics ecosystem” shapes up over the coming years.

In the following sections, we’ll go over:

• A very brief review of the key points in RISC vs. CISC debate, and how they have shaped computing’s landscape

• What the landscape may have looked like without the ISA layer

• What this means for robotics

Share

This article is co-written with . Subscribe to for narrative and insightful posts on computer architecture.

Chip Insights

Semiconductor Industry Deep Dives

By Bharath Suresh

The RISC vs. CISC Debate and its Implications

RISC instructions do very simple bits of work, which means many instructions are needed to solve a complex task. The compiler does the job of breaking up a complex task into these small steps. CISC instructions, on the other hand, can have the CPU do large bits of a task in a single dispatch. x86, a CISC architecture, has dominated desktop computing for several decades, while Arm, a RISC architecture, has dominated the mobile market. The RISC vs. CISC debate wouldn’t exist if there was a clear winner, which means that both sides have merits and tradeoffs. For curious readers, I’ve linked further resources below, but I wanted to highlight some of the broader ecosystem implications of one vs. the other:

The simplicity of the RISC instructions makes it easier to build hardware: the ISA is simpler to implement, and this leads to relatively more hardware entrants. This has been shown to be true in computing, with a Cambrian explosion of RISC entrants.1 Most computer organization and architecture courses build a RISC processor, and there are significantly more freely available resources that assist this process.

Because the RISC ecosystem is easier to enter, we tend to see more diversity in the microarchitecture. A good example is the Condor Cuzco RISC-V core. Considering memory access time, different instructions require different amounts of latency in processors, and optimal performance requires scheduling instructions at precise times so that their operands are available. NVIDIA does this scheduling in their GPU compiler, but the RISC-V ISA leaves room for it to be implemented in hardware. Condor implements a “Time-Resource Matrix” solver for this problem, an innovative solution that they can pursue by relying on existing RISC-V infrastructure to build on.

The burden of software optimization is moved to the compiler by RISC. As said in his article, RISC is an apt acronym of “Relegate Important Stuff To Compiler”. The saving grace is that this compiler complexity is shared over many hardware targets, enabled by the ISA. It would be impractical to expect LLVM to be able to produce optimized code for a processor I invent, whereas I can easily piggyback on existing work if I target an established ISA.

To summarize, RISC helps promote a more open, thriving ecosystem, encourages innovation and diversity in hardware, and spreads out the burden of optimization among different vertical parts of the stack (software, compiler, hardware).

A Thought Experiment: no ISA or Compiler

Underlying both sides of the ISA debate, computer engineering has a foundation with two critical pieces of technology: the ISA itself, and the compiler.

Let’s imagine Musk’s proposed alternate world where these do not exist. To be clear, we are not advocating for this proposal (there is much other writing on this topic) other than using it for a thought experiment. Programs would be converted directly into machine code from high-level intent by an LLM (though the specifics of the method aren’t important for the point). In this world, we could still have RISC and CISC processors; the LLM would just have to know and understand how to map to the different instruction sets. Essentially, the LLM becomes a general purpose compiler across multiple ISAs.

However, the ecosystem effects would not be the same, and in many ways would be inverted!

Underlying the vibrant RISC hardware ecosystem was the hidden assumption that the compiler abstracts over implementation differences. If we delegate that to an LLM, RISC’s simplicity becomes a liability. Every microarchitectural optimization or quirk is now the LLM’s responsibility to learn separately, and CISC’s philosophy of “let the hardware handle complexity” becomes an advantage.

It is still simpler to build RISC hardware, but without an ISA, the complexity of the full stack will fall entirely on the new hardware entrant. They cannot utilize work done by software or compiler authors, and will have to build or fine-tune a new LLM to target their hardware, with all the complexity that entails (datasets, benchmarks, training infrastructure, etc.). Practically, this makes it even more difficult than it already is for new hardware entrants.2

The LLM now needs to embed knowledge of all the microarchitectural variations in hardware. For economic viability, hardware entrants will be incentivized to maintain LLM compatibility in novel releases, suppressing microarchitectural innovation.

RISC vs. CISC also affects the complexity of the LLM that needs to be built to target the hardware. With CISC, fewer instructions are needed for the same program and the output space of the LLM is reduced. In fact, CISC was historically preferred for a similar reason: computer memory was slow and expensive.

In some cases, CISC can also help abstract hardware differences away. For example, the REP MOVS x86 instruction is a full translation of the intent (“move a string”), while in a load-store RISC architecture that does not support memory-memory operations, the LLM needs to perform many steps, and to keep track of which registers are free to implement the equivalent.

Processors built on the CISC architectures have largely overcome their microarchitectural weaknesses by translating complex instructions into micro-ops to achieve similar hardware efficiencies as their RISC counterparts. In the proposed alternate world, RISC would also lose out on the other advantage it currently holds: easier performance optimization by the compiler.

Putting this all together, we first realize that Musk’s proposal results in a significantly more consolidated ecosystem where innovation in hardware is more difficult across the board, and architectural diversity is stifled. A dark future indeed!

More unexpected is the realization that in this alternate future, RISC hardware becomes disproportionately more difficult to build. The ecosystem becomes more closed and concentrated, while CISC has a somewhat better chance of supporting different hardware with the same software.

The Effects on Robotics

We’ve written before about how computer engineering can anticipate aspects of technology development in the field of robotics. Could a RISC vs. CISC debate appear there, and have similar ecosystem effects?

Action tokens are analogous to instructions, and we have seen examples of fine- or coarse-grained tokens in robotics. In a previous article I demonstrated that both are viable for a tabletop manipulation task.

A Vision-Language-Action (VLA) model is an example of a “RISC” software layer that outputs fine-grained actions, and the outputs can be joint-level or Cartesian infinitesimal commands. The “CISC” software layer requires effectively a high-level task orchestrator outputting higher-level symbolic motion or task primitives (like “grasp object”). We have seen examples of this in classical AI dating back to STRIPS, as well as in modern AI with VLMs like Gemini ER.

Since an ISA or compiler equivalent does not exist today, we end up in the “alternate history” version of the world from the previous section. What can we learn from it about the future of robotics?

Building “RISC” robots with only joint-level control functions remains a simpler task than also endowing them with higher-level task functions and autonomy. However, for the robots to be useful, those software components still need to be obtained from somewhere. Without the ISA layer, this effectively means that the software layer needs to have native support for the new hardware.

For my VLA project, I made my simulated robot arm choice based on what was supported by a VLA out-of-the-box, and I assume this is a common phenomenon. The result is consolidation and reduction of hardware diversity.

A 2024 study on zero-shot fall recovery across seven humanoid morphologies found noted difficulty transferring to robots that are “top-heavy, long-armed, or otherwise distant in morphology space”. The CISC equivalent has been explored less, but this 2020 paper from Berkeley demonstrates that using “CISC” subgoals with a morphology dependent low-level policy aids policy transfer to new morphologies.

Lastly, with the RISC perspective, the burden of optimization is pushed entirely on to either the AI model builder, or the hardware vendor (who must now fine-tune the large model to work with their hardware). Unlike the multi-layer software → compiler → hardware stack, where smaller players can share the burden to build competent products, there is no way to spread this workload around. This affects ecosystem investment and control, both commercially and in academic research.

Leave a comment

Closing Thoughts

I have always had a fond impression of how RISC influenced the computing ecosystem, and it turns out that the ISA layer was a crucial reason for that. Elon Musk’s claim made me think of how that hypothetical scenario could have changed history, and it almost completely inverts the balance.

Robotics is early to this game, doesn’t have an equivalent of an ISA yet, and ecosystems are just beginning to take shape. For now, there are (very smart!) engineers at different research labs and companies choosing “instruction” granularity for their own technical reasons. As well they should — building a robotic solution poses enough challenges as it is. However, so were computer engineers at Intel, and it’s difficult to imagine the long-reaching ecosystem consequences of these decisions!

Even though robotics doesn’t have an “ISA” yet, there are some promising directions and paradigms I’d like to explore in an upcoming article. Make sure to subscribe for that, and thanks for reading!

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

Share

Further Reading

In this article I talked through an “action token” in robotics, an equivalent of an “instruction” in computing:

Read about the RISC-spurned ecosystem in computing:

The Chip Letter

The RISC Wars Part 1 : The Cambrian Explosion

I’d like to start this week’s post with a small confession. In last week’s post, I called Berkeley RISC-I ‘the first RISC microprocessor’. In fact, some believe that a derivative of the IBM 801, known as ROMP, was the first RISC microprocessor. The story of the 801’s successors, including ROMP, will be the subject of a later post. I believe that RISC-I …

Read more

3 years ago · 29 likes · 24 comments · Babbage

Users were similarly upset when Opus 4.8 would use up tokens for reasoning, costing $$ while not producing any actionable output:

More broadly, as LLMs continue to gain use, the answers to many questions seem to return to tokens. Q: How responsive is an LLM? A: tokens/second. Q: How much do you spend on your LLM? A: $/token.

In this article, we will go over:

• What tokens and tokenizers are in LLMs, and what the fundamental tradeoffs are (in algorithm characteristics, computation, and memory) that tokens induce.

• How this tradeoff is going to be even more central in robotics or physical AI, and where biology lies on the spectrum

There is also a very interesting parallel between an action token in robotics to an instruction in computing, and the juxtaposition it creates with the RISC vs. CISC debate. I’ll cover that in a follow-on post that I’m excited to write — make sure to subscribe to get it in your app or inbox:

Subscribe now

Tokens in LLMs

In LLMs, tokens are a numerical encoding of language. I’ll describe the tokenization process briefly here for the purposes of this article, with references to a fuller analysis in the TinyTorch course notes.

One idea is to make a word a token. That’s more or less how humans learn a language: a dictionary will have entries corresponding to words.

Neural networks process numbers, not text. When you pass the string “Hello” to a model, it must first become a sequence of integers. This transformation happens in four steps: split text into tokens (units of meaning), build a vocabulary mapping each unique token to an integer ID, encode text by looking up each token’s ID, and enable decoding to reconstruct the original text from IDs.

The tokenization process decides how the “split” step happens, and this can be done in various ways.

The simplest approach treats each character as a token. Consider the word “hello”: split into characters ['h', 'e', 'l', 'l', 'o'], build a vocabulary with IDs {'h': 1, 'e': 2, 'l': 3, 'o': 4}, encode to [1, 2, 3, 3, 4], and decode back by reversing the lookup.

Even at this point, some of the tradeoffs are becoming clear. This mapping (which is used in the forward direction for encoding, and in reverse direction for decoding) needs to somehow be stored in memory:

Character vocabularies are tiny (typically 50-200 tokens depending on language), which means small embedding tables. A 100-character vocabulary with 512-dimensional embeddings requires only 51,200 parameters, about 200 KB of memory. This is dramatically smaller than word-level vocabularies with 100,000+ entries.

However, a different problem arises when it comes time to actually process the encoded sequence. This processing is algorithm-dependent (more on this later). When an LLM receives a query, it will pass through a transformer block that receives the token sequence. The length of this sequence has a huge impact on the number of computations in the transformer.

Character tokenization has a fatal flaw for neural networks: sequences are too long. A 50-word sentence might produce 250 character tokens. Processing 250 tokens through self-attention layers is computationally expensive, and the model must learn to compose characters into words from scratch.

So, small vocabularies mean low memory usage for the dictionary, but long sequences for computation. Large vocabularies (like full words) have high memory usage, but produce shorter sequences.

The last sentence in the quote above brings another side of the argument into the picture: how generative are the tokens? Stringing words together to make sensible sentences is easier than string letters together! A letter by itself doesn’t “mean” much, but a word carries a lot more information.

There is a spectrum between these options, and all LLMs today use a “subword” as a token with a process called byte pair encoding (BPE). For LLMs, BPE is learned from data: commonly used character sequences like “un-” or “pre-” will become tokens. Importantly, for LLMs, this type of encoding is easily learned from data. If you have an LLM-sized training corpus, you can just check what character sequences make sense as tokens over all of it.

BPE solves this by learning subword units. The algorithm is elegant: start with a character-level vocabulary, count all adjacent character pairs in the corpus, merge the most frequent pair into a new token, and repeat until reaching the target vocabulary size.

Before we move on, let’s summarize some of the tradeoffs tokenization induces, in a spectrum between fine (like letters) and coarse (like words) tokens:

1. Memory: Embedding table size better with fine tokens

2. Computation: Sequence length shorter with coarse tokens, but depends on algorithm

3. Generation: Easier with coarse tokens

4. Transfer: Words might be more task specific (e.g. a medical journal may have different types of words than a novel), but capture reusable task structure better (e.g. you can use an English to Italian dictionary to get by in Italy, vs. learning Italian from scratch from letters)

5. Execution latency and architecture flexibility: Fine tokens require very fast inference loops, but coarse tokens relax that and give more flexibility.

Action Tokens in Robotics

Robotics does physical work in the real world, and the most important part of that is the action. In this post, we will focus on action tokens instead of tokens for sensing or task input.

By analogy, a token is a small component of a larger task-directed behavior. If the task is to pick up a block, should a token be to move the joints by a few degrees, a Cartesian waypoint, or to complete the full grasp?

In a previous post, I wrote about two ends of this spectrum for exactly a task like this:

You can try the demo here too.

Basically all VLA models output fine-grained tokens, and the alternative approach used a VLM outputting coarse “move”, “grasp”, etc. commands.

Unlike the LLM “chatbot” interaction model where encoding and decoding are both part of standard usage, robot actions strictly only need to be decoded at inference time. If we are learning from imitation or demonstration, then an encoding step would be a part of training.

The tradeoffs from above can be ported over to make sense for robotics actions too, with some interesting implications:

1. Memory: This corresponds to the size of the “library” of action primitives that the robot is capable of executing. You don’t need to really store anything if the primitives are small joint motions, but memory demand could balloon if the primitives are more complex. For example, a robot arm may have hundreds or thousands of preset configurations, reaching motions, etc.

2. Computation: As exemplified by the robotics pipeline demo above, the Gemini ER model has a computationally simpler problem producing sequences of commands.

3. Generation: Easier with coarse tokens, as above

4. Transfer: Motion primitives might be more task specific (e.g. a pick-and-place vocabulary would not have “catch” and “toss” actions for juggling), but capture reusable task structure better (e.g. the primitives in the demo above are useful in for package logistics as well as loading a dishwasher)

5. Execution latency and architecture flexibility: The coarse tokens in the demo allowed us to combine hierarchically with model-based planners, whereas the VLA needs to run end-to-end

Behavioral Primitives in Robotics and Biology

Coarse action tokens are sometimes referred to as behavioral primitives.

The parallel to language (building up sentences from subwords and words) has been quite literally expressed in some robotics research (as a disclaimer, I was a Ph.D. student in the same lab):

In robotic behavior cloning, object-level and action-level abstractions can be learned from training data (the BPE equivalent):

This paper shows the emergence of synergies from reinforcement learning of humanoid locomotion.

In biology, motor synergies coordinated in the spinal cord allow multiple joints to be coordinated to produce coarse tokens:

The resulting behavioral primitives can be recognizable subcomponents of tasks:

Closing Thoughts

A token in an LLM or a robot can be either very fine-grained or coarse. Due to the rich tradeoff space, there isn’t an absolute correct answer for what to choose, and the best decision may change over time and for different applications (as Anthropic found with Opus 4.7).

Not enough attention has been paid yet to the implications of this choice for robotics. In addition to the technical tradeoffs I discussed above, it can have some very important downstream effects on the ecosystem of research: fine-grained tokens make the robotic stack monolithic, while coarse ones allow for parallel progress in low-level control and higher-level cognition that can be vertically integrated for a particular application. This also has an impact on how the research ecosystem can be structured.

The course of world model research and continued robotic deployments in 2026 may shed some more light on the best choice for different applications. I’ll look forward to covering major developments in this area in future posts. Also, stay tuned for a follow-up on the parallel to instructions in computing and the RISC vs. CISC debate.

Thanks for reading!

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

Share

On one end of the spectrum, organizations are building AI brains using foundation models (principally VLAs at the moment), and deploying them to humanoid robots or robot arms for some tasks. Taking a term the public understands (artificial intelligence), and extending it to physical work via robotics, is being referred to as embodied or physical intelligence. For example, Physical Intelligence is the literal name of one of the forerunning organizations, Embodied AI is a labeled a priority in the CCP’s 15th Five-Year Plan for China’s socioeconomic development, etc.

On a different end of the spectrum, DARPA sent out a call (submissions due on the date this article is published!) to researchers requesting information about physical intelligence, referring to intelligent materials that implement sensing and actuation without needing any brain at all.

These apparently contradictory viewpoints are just the tips of the iceberg of a rich corpus of literature about embodied intelligence. In this article, we’ll review a bit of that history and then see if it can help us build better robots today.

This post may be truncated in email form — click on “View entire message” to view it in a browser.

A Brief Review of Embodied Intelligence

While approaches differ, it’s an uncontroversial opinion that having a physical instantiation (a body) is helpful to developing intelligence. Even the most practically-motivated robotics companies are drawing inspiration from comparative psychology:

A classic experiment in 1963 was conducted to investigate how vision and motor activity are linked in perceptual development. In the setup, kittens were placed in a carousel — some walking freely, others carried in harnesses. Though all of them saw the same scenes, only the active kittens developed normal depth perception. The study showed that motor activity plays a decisive role in visually guided perception and motor learning.

Robots with embodied intelligence learn in much the same way. When they move, probe and act, sensory input can be connected to the consequences of their own behaviour. This feedback allows them to build more accurate internal models.

This says that at least the development of intelligence requires having a body. Technically, after the “accurate internal models” are built, maybe you have a fully developed AI brain that can then function without that body. There are stronger views that the body can never be separated from the brain, that it is a constitutive part of intelligence.

I’ll go over some of the viewpoints along this spectrum below. If I missed an important reference, please comment below — it would help me as well as other readers!

Leave a comment

Gibson’s Affordances (1979)

Gibson’s seminal book claims that perception isn’t a passive process, but of action possibilities (“affordances”) relative to a particular body.

The basic assumption is that vision depends on the eye which is connected to the brain. The author suggests that natural vision depends on the eyes in the head on a body supported by the ground, the brain being only the central organ of a complete visual system. When no constraints are put on the visual system, people look around, walk up to something interesting and move around it so as to see it from all sides, and go from one vista to another.

As an example, a step affords climbing for a human leg but not for a mouse. The “climbability” is a coupled property of the body and the environment. Gibson’s view goes further that the animal perceives the affordance directly, immediately and without reconciliation with an internal model in the brain.

The takeaway here is that affordances inseparably tie any notion of intelligent perception to the actual sensor itself. You can’t move a “perception skill” from one embodiment to another because that changes the affordance structure.

Brooks’ Subsumption Architecture (1987)

Rodney Brooks’ 1987 paper “Intelligence Without Representation” showcases a couple of interesting concepts.

Conventional AI at the time utilized a centralized Sense → Plan → Act pipeline, a hierarchy of components performing separate functions. Instead, Brooks suggests a hierarchy by objective, where each component does all the sensing, planning, and acting it needs. For example, a component could be avoid_collision, and a second component could be move_to_goal. Each of these components has their own objective, and can take precedence over or “subsume” another.

The second concept is (as the article title suggests) a complete rejection of internal models and representations. Brooks insists that the world is the model, and the only way to develop autonomous functions is by directly interacting with the environment.

The important takeaway for us is that there is no brain at all. The body (comprising sensors, motors) is the cognitive system. This concept resulted in autonomy in a variety of robots built by Brooks and colleagues:

The other takeaway (which we will return to) is that algorithms can be distributed in parallel in different parts of the body. For examples, a small loop including only the distance sensors and a component of the motor commands would serve the obstacle avoidance function.

Predictive Coding (1999)

This seminal work by Rao and Ballard provides a neuroscientific foundation for internal “world” models. The initial 1999 paper focused on perception in the visual cortex, and is the largely accepted hypothesis for this process:

Rao and Ballard implemented a simple model of the visual cortex, using a 3-layer hierarchical neural network (they numbered the layers 0, 1, and 2) with two-way connections: predictions flowed from the higher to lower layers and errors, or residuals, went from lower to higher layers.

…

Rao and Ballard found that their network spontaneously discovered hierarchical processing: layer 1 learned to recognize bars and edges, while layer 2 (the topmost layer) learned to compose features learned by layer 1 to recognize more abstract features.

The first takeaway is that neither direction (eyes to brain, or brain to eyes) stands on its own. The prediction arrows justify world models that are currently under active research in AI and robotics, and the sensory stimulus arrows similarly justify that perceptual models in the brain don’t work without the sensors themselves.

The second takeaway here is the central role of a “brain,” which is a departure from Gibson or Brooks. Still, the necessity for a brain does not reduce the significance of the body (and the brain’s coupling to it) in this view.

Friston’s Free Energy (2006)

Friston extended the predictive processing idea from perception to action, and a broader principle for the working of the brain. Per this theory, the brain optimizes a free-energy which approximately contains terms related to prediction error (similar to above) and also value (attaining a goal).

It is easy to illustrate with an example of an arm asked to point to a green dot:

The steps in the loop are:

1. Brain predicts (“dreams”) sensory state: arm is at green circle

2. Sensor reports: arm is at red circle → prediction error

3. Action is triggered to make the prediction true → the arm moves

4. When the arm reaches green, sensor input matches prediction. The prediction error goes to zero, and free energy is minimized.

The takeaway here is that free energy crucially depends on both the perception and action systems, and is meaningless without them. The brain’s functioning cannot be separated from the body.

Mechanical Intelligence or Body Schema

Commonly, we think of a single holistic brain interfacing with all the sensors and controlling all the motor functions in an animal. Brooks’ subsumption architecture, and to some extent Gibson’s view, allowed for a more distributed notion of computation.

Computation can be embedded in parts of the body without requiring the brain to be involved. In the 2005 book How the Body Shapes the Mind Gallagher describes the concept of a “body schema” which operates without any conscious activity of the brain.

My body schema is what arranges that my hand shape itself just so in order to pick up a pencil without my paying any attention to how it is shaped, it is what tightens my back muscles and adjusts my posture when I shake hands so that I do not throw myself off balance with the movement, and so on. It operates (to a first approximation) independently of what I think or how I feel.

Reflexes or preflexes are also examples of this kind of mechanical intelligence or passive dynamics. Computers don’t always need to be built with chips, sometimes tendons and muscles can implement PID control!

In the history of robotics, some of the most exciting milestones in robotic mobility made use of this kind of mechanical intelligence. This includes Tad McGeer’s completely unpowered walkers (~1990),

the first outdoor running robot, RHex (~1999), the IHMC OutRunner running robot (~2014),

the ATRIAS robot (~2015) and its modern descendants at Agility Robotics, etc.

There’s a lot more to say about mechanical intelligence (general purpose vs. task-specific design, power amplification, latching mechanisms, etc.) that I plan to cover in a dedicated future article — subscribe to make sure you don’t miss it!

Subscribe now

The takeaway for this article is that computation need not always go through a central brain. Hierarchically distributed mechanical intelligence can complement a brain, as it surely does for animals, reducing the burden of the nervous system:

Returning to the DARPA Physical Intelligence RFI, it is clear that this type of distributed intelligence is what they are looking for:

Rather than relying on centralized processors and large data flows, DARPA is exploring materials that can perform computation directly.

Returning to the Present

With the practically motivated goal of utilizing the best technology at our disposal to build the most capable robots, which of these (potentially conflicting) ideas should we bring along?

Behavior Cloning and Multi-Robot Brains

Behavior cloning (imitating a human performing a task with a robot) is the de facto methodology for modern humanoid robotics. Data of humans doing a huge variety of tasks is plentiful and readily available.

This paradigm runs into some challenges in light of the embodiment theories we just discussed:

1. Mechanical intelligence is invisible in the dataset. If the body is performing stabilization effectively, the data only contains the stable configuration without any indication of the act of stabilization itself. This is one of the reasons VLA actions typically have some stability structure wrapped around them (like an impedance controller). It isn’t possible to directly learn a complex dynamical behavior like gymnastics purely from imitation (the next section discusses how this is actually done).

2. When training data is from even a very slightly different body, the Gibsonian view is that the affordances aren’t compatible. In fact, we do see that small-scale VLAs are actually quite poor at cross-embodiment generalization. Scale and cross-embodiment training show promising results, but have an efficiency cost — for a fuller coverage of this topic, check out my previous article on multi-robot brains.

3. Insects and elephants both walk, but they use very different types of actuators and sensors in their embodiment to do so. High-level task strategies may heavily rely on the differences in how these components work. Here is a very relevant passage from Gallagher:

At the age of nineteen, Waterman lost all sense of touch and proprioception below his neck. As a result, he instantly lost the ability to control the affected parts of his body. Slowly, he regained the ability to walk, dress, eat, and so on, but in order to do these things he had to learn to do them in a new way: by alert conscious control of his every movement. Waterman must consciously adjust his balance when turning a corner, think about swinging his leg to take a step, make an effort to shape his hand into a position suited to gripping a mug if he wants to pick it up, and so on. As a result of this, he remains substantially disabled in his behavior. His case demonstrates that, while one’s body schema is not strictly necessary for movement, it is necessary for movement in normal human beings and necessary for fluent movement even after extensive retraining.

Waterman is, as Gallagher puts it, a man with a body image but without a body schema.

A robot certainly has a different body schema from the human demonstrating a skill, and skill transfer should expect some of the same issues.

The takeaway here is that the architecture of embodied intelligence is not just a brain in a vat puppeting a body, and includes distributed, hierarchical computing.

A Potential Resolution

Despite the incompatibilities with embodiment theories, behavior cloning across embodiments does seem to empirically work, as evidenced by the growing number of robotic foundation model demonstrations. In addition, we see highly dynamic motion transferred to robots:

That last video’s most crucial component is something we haven’t discussed yet: reinforcement learning (RL). In this step, the robot is placed in a simulation with its embodiment in its environment — all the component’s of Gibson’s affordance structure — and set loose to optimize its behavior over thousands or millions of trials.1

This combination is powerful.

The AI brain (trained without preference for a particular embodiment) provides cognitive higher-level intelligence, a menu of task-solving strategies, and examples of basic perception and motor skills. Think of it as an instruction manual, or a compendium of videos of someone playing a sport that you want to learn, say tennis.

However, if you want to actually play tennis, there is no alternative to actually using your eyes to track the ball, and using your wrists to flick the racket for a forehand. The very term “muscle memory” indicates something that cannot be transferred from demonstration. However, RL (or its model-based alternative, trajectory optimization) provides a mechanism for robots to gain such practice, with its body, in the requisite environment.

A fruitful recipe for training a robot to play tennis could include a very high-level instruction manual in the form of human demonstration, followed by extensive RL with the actual sensors, potentially including sensory modalities not even available in the demonstrations.

This also makes room for incorporating innovation in meta-materials as in the DARPA RFI — those modalities can be introduced at the RL stage, enabling rich distributed, hierarchical, and feedback-enhanced architectures.

The RFI itself is a bit pessimistic about be-all-end-all human-like form factors:

Additionally, while industry has emphasized human-like form factors designed to operate in human environments, of interest here are systems optimized for mission needs. Depending on the application, this could include designs that are smaller, larger, softer, or structurally unconventional, prioritizing performance and adaptability over familiarity.

However, there’s a lot of potential in hierarchical integration of AI brains with smart materials that can perform sensing, actuation, and computation. They may come together in a humanoid form or not (we discussed form factor diversification in robotics in this post). It just needs a bit of care in the methods used!

Thanks for reading! If you liked this post, please like (❤️ button), restack, and subscribe — it helps others find my writing.

Share

Subscribe now

Further Reading

In related past articles, I’ve written about the architecture of end-to-end robotics pipelines, and architectural strengths and limitations of deep neural networks. If you liked this post, these would be great reads:

For this article, I returned to these great Substack posts by other authors. Check them out too:

WHERE MACHINES THINK

The Case For World Models, Part I: The Neuroscientific Reason

LOOK at the two images above. What do you see…

Read more

4 months ago · 55 likes · 9 comments · Anil Ananthaswamy

It Can Think!

How can we get enough data to train a robot GPT?

It’s no secret that large language models are trained on massive amounts of data - many trillions of tokens. Even the largest robot datasets are quite far from this; in a year, Physical Intelligence collected about 10,000 hours worth of robot data to train their first foundation model, PI0. Professor Ken Goldberg of UC Berkeley gave a talk which Andra K…

Read more

a year ago · 58 likes · 5 comments · Chris Paxton

It feels natural to try and compare this kind of brain-body interface to a software-hardware interface in computers. An Instruction Set Architecture (ISA) helps abstract software programming from processor hardware implementation. The first ever ISA, in the IBM System/360 (1964), had the explicit goal of making software compatible across different hardware generations, and this goal has remained the most important driving force behind their existence.

In this article, I wanted to see how this analogy could help us think about multi-robot brains.

The ISA Analogy

The benefits of an ISA become clear when examining a typical “stack” for a computer:

The columns of this chart are different “verticals”. If a company owns every item in the column, that would be “vertical integration”. The iPhone, and most of Apple’s products, are famously vertically integrated up to the application layer. The PC ecosystem does not exhibit the same kind of vertical integration.

Conversely, “horizontal integration” is when a particular product or vendor is appears in many columns across a row of the stack. For example, the Android OS is used in 70% of smartphones sold worldwide.

The ISA allows the stack to grow horizontally without recreating every piece of the stack. The marginal effort required to introduce new software, or a new chip, is reduced as long as it can bridge itself to the ISA.

Let’s compare to a robot stack1 for a couple of hypothetical robots:

Vertical integration is exemplified by Unitree, and it could be argued that NVIDIA is positioning itself very well to be horizontally integrated as the computing device of choice.

The multi-robot brain (a) vertically bridges hardware and software, and (b) horizontally bridges different robots. In contrast, while the ISA bridges vertically, it does not bridge horizontally — the iPhone and the Android phone need not share any software or hardware components. This is an important distinction, because the horizontal bridging hinders vertical integration.

Vertical Integration

An ISA promotes vertical bridging, but simply gets out of the way after that, allowing for vertical integration. This allows for products that are really optimized to work well in a way that isn’t possible with horizontally integrated components. An easy example of this is how well sleep and wake work on a MacBook due to the tight vertical integration, and how poor it still is in Windows laptops.

The Arm AGI CPU was introduced recently because when performance is critical, companies were still building bespoke software to take advantage of vertically-integrated optimizations. Arm saw this need and it was large enough to get them to break out of a decades-old strategic choice and build their own chip (emphasis mine):

“Delivering AI experiences at global scale demands a robust and adaptable portfolio of custom silicon solutions, purpose-built to accelerate AI workloads and optimize performance across Meta’s platforms,” said Santosh Janardhan, head of infrastructure, Meta.

For critical datacenter functionality, the last mile of performance optimization needed vertical integration. Everyone else, like Amazon, Google, was already doing it by customizing their stacks, and Arm decided to try and save its customers from having to do this work by doing it themselves.

Subscribe now

The Missing Analogue of Compilation

In computer-land, the ISA is paired with a compiler that builds a binary tailored to the target system. The compiler can optimize and produce a binary that runs well on a particular chip using vector instructions, reordering, and other optimizations within the constraints of program correctness.

There isn’t an equivalent of compilation in a multi-robot deep neural network brain (though maybe there are Turing-complete architectural alternatives that could emerge in the future). This results in a few cracks in the ISA analogy:

1) Memory / Performance

The current multi-robot brain is more like a fat binary in some ways. While the exact implementation varies, there will usually be a vector of latent variables that explicitly or implicitly encodes the details of the embodiment. So, while a Mac universal binary could use a single bit to pick x86 or Arm machine code, the robot embodiment is represented by an n-dimensional vector of latent state that can further weight or guide the network.

In the Skild AI blog post referenced above, this “fat binary” aspect is used to demonstrate recovery from limb loss, or other aspects of hardware failure. The tradeoff is the added network size, manifesting as increased memory and compute burden.

2) Hardware-Specific Optimization

A multi-robot brain needs to adapt for factors such as location and dimensions of limbs, cameras, as well as higher-level locomotion and task strategy. For the kinematics and sensor transforms, optimized code is akin to platform-specific subroutines, an analogy I made in my article series on VLAs.

A robotic behavior “compilation” step could likewise tailor task execution strategy to a particular body. For example, trajectory optimization or reinforcement learning (RL) can be used to optimize a behavior for a robot body, and like compilation, is typically done offline.

However, in a neural network multi-robot brain (which does not have this notion of compilation) designed to solve a task with B bodies, either the behavior is not optimized for each body type, or B different strategies need to be stored (bringing back the memory issue from the prior point). In reality, the multi-robot brain does store some number of distinct-looking strategies (see the Skild AI blog post for demonstrations).

However, by the nature of deep neural networks, a discontinuous divergence in strategies is not very efficiently representable. As an example of strategy discontinuity, consider that my cat needs to run or jump up human-sized stairs, whereas a larger animal could simply walk up them. The multi-robot brain needs to do a lot of work to represent effective locomotion strategies for these variations.

Common Sense Generalization

One of the main arguments in favor of cross-embodiment training is that people are finding that it really helps with generalization. For example, see this excerpt of Sergey Levine’s explanation on the Automated podcast two days ago, and Skild’s case in their blog post:

One way to do this is to train the AI to control not just one robot, but a whole multiverse of robots with different bodies. It cannot memorize the solution for one body, it must find a strategy that works across all of them. When faced with unpredictable scenarios, the AI can now use the strategies it learnt during training and keep going.

There is still ongoing research and development on this finding, but one implicit logical leap is that the strategy is being evaluated (in all these cases) with end-to-end networks. This means that the cross-embodiment training is for a deep neural network that is learning not just the particulars of one robot body, but also the functional form of the necessary mathematical transformations to control it.

As an analogy, consider training a network to perform the sine calculation,

y = f(x) := sin(x)

We don’t know what the “sin()” operation does, but we are learning it from inputs and outputs {x, y}. To train this network, we have the option of using data from one embodiment, which has x values in the range [0, 1], or data from multiple embodiments with ranges covering the range [-10, 10]. The extra data from the other embodiments are helpful to just understand how the sine function works.

Closing Thoughts

Cross-embodiment robot training creates an interesting software stack bridging software to hardware vertically, as well as horizontally across different tasks and robot bodies. The former aspect is evocative of the ISA abstraction layer in computing, but the latter aspect is more of a distinction.

I wonder if a “compilation” analogue could exist, allowing for optimization of a multi-robot policy to a slimmed-down robot-specific policy. It’s possible to use RL to post-train or to fine-tune a model such as the ones discussed in this article, but that interferes with the fault tolerance feature, and does not help reduce the model’s size.

Lastly, the size of these multi-robot policies is going to necessarily be large. Taking it to the scaling extreme, only the largest corporations may have the funding to train these models, resulting in potentially unwanted ecosystem consolidation.

To sum up, multi-robot brains have been showing impressive generalization ability, and I expect we will continue to see cutting-edge results from them. However, their generalization success is partially wrapped up with the end-to-end deep neural network architecture, and that there might be opportunities for significant optimization with architectural innovation.

Thanks for reading, and let me know your thoughts on this parallel!

Leave a comment

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

Subscribe now

Fortunately, we know a lot about what deep neural networks represent. As a starting point, the universal approximation theorem (UAT) says that a feed-forward neural network with at least one hidden layer can approximate any continuous function over a compact domain to any desired degree of accuracy, provided it has enough neurons and a non-linear activation function.

This begs a number of follow-up questions:

• What kinds of tasks are (not) solved by approximating a continuous function?

• For this purpose, are transformers equivalent to feedforward neural networks, or do they do something different?

• How do these map to computational hardware, like CPUs, GPUs, or NPUs?

Answering these questions requires a review of what “computation” means, looking all the way back to the writings of Turing, Minsky, and Chomsky. In exchange we get some insights into the versatility as well as the energetic cost of current AI.

I’ll provide some answers to the first two questions in this post, and a detailed look at the last one in a follow-up.

Subscribe now

Universal Approximation

The prototypical “feedforward neural network” from the UAT is a multi-layer perceptron (MLP). This is typically composed of linear layers (which multiply its inputs by a weighting matrix) and a nonlinear activation function.

In the plots below1, we’re approximating a quasi-sinusoidal curve on the left and a square wave on the right using an MLP.

With larger model width and depth:

You’ll notice that the square wave is much more difficult to approximate than the sinusoidal one. Why is that? If you recall from above, the UAT promised that the MLP would be good at approximating continuous functions, and the square wave has periodic discontinuities.

Before you think that this is some pedantic example that would never occur in practice, let me offer two more practical ones that are equivalent.

Suppose you have a drone flying through a forest of tall trees:

The task is obstacle avoidance: the input is the front camera view, and the output we’d like is a path that won’t collide with a tree. In such a view, if the view changes continuously in such a way that a path becomes too narrow to pass through, the safe path must jump to a different one discontinuously.

An MLP would need to sample the input space very densely to quickly interpolate between the left path and the right one (as in the square wave example above). This has a high model size penalty, and additionally needs to interpolate through an unsafe part of the output space.

Another practical example is related to the title of the article. Assuming an LLM’s output is a token view of an internal reasoning state, “changing its mind” on a yes / no question requires a similar jump in its state. However, the internal computing machinery of a modern LLM, the transformer, is more complex than an MLP. We’ll look into the two categories separately below.

Subscribe now

Lookup Tables to Turing Machines

The computation power guaranteed by the UAT is equivalent to a lookup table. A lookup table effectively pairs inputs and outputs so that it can “look up” the appropriate output when queried with an input. In continuous spaces, this can include some interpolation or extrapolation. The curve approximation figure above is a good visualization of this: the table would contain {x, y} entries. The compact domain condition of the UAT effectively ensures that the number of entries in the lookup table is finite.

On the other end of complexity, we have a Turing machine: an automaton that has access to unbounded memory, and is able to make discrete decisions based on what is in its memory. While this may sound foreign, it is actually a very familiar concept. A CPU paired with almost any programming language is a Turing machine (putting aside the implementation detail of potentially running out of memory). You can control a program’s flow using if, while, etc. and call subroutines, and with these building blocks, you can build any software that has ever been written.

It should be clear that a Turing machine can do fundamentally more than a lookup table:

1. It can process an input that is arbitrarily large, which a lookup table cannot do. For example, you can very easily write a CPU program that factorizes an integer, but we could never fit such an algorithm on a lookup table, since you could always input a larger integer. A more current example is a pre-transformer language model, which could not handle sequences of arbitrary length, and thus could not exhibit the level of capability we got with a GPT.

2. It can exhibit irregular flow control, like branching and jumping. In the “flying through forest” example above, it can do something like

if left_path_too_narrow:
    take_right_path()
else:
    take_left_path()

While this looks benign, it is deeply connected to the continuity clause of the UAT. An MLP cannot represent an algorithm that needs this kind of branching to have a discontinuous or symbolic jump.

In the example above, a square wave was still able to be approximated by an MLP, but at the expense of a large number of parameters. As a contrast, here’s an almost trivial program that could accomplish the requisite classification with very few parameters:

if x mod 2 < 1: # if the remainder of x/2 is < 1
    return 1
else:
    return -1

This shows the expressive power of a Turing machine compared to a lookup table. Adding a little structural or organizational complexity drastically reduced the number of required parameters.

Subscribe now

The Transformer Attention Mechanism

We discussed earlier how the UAT only addresses a finite set of inputs. This is true in practice for MLPs as well: it will typically be used to process a fixed image size, or in an transformer feedforward network, a fixed layer width.2

The attention mechanism of transformers is different. In an LLM, when a sequence of tokens is fed in, each token can attend to each other token, enabling a computation paradigm that can handle sequences of arbitrary length. This makes it different from a lookup table, because the input dimension itself is unbounded. You don’t need to retrain for longer sequences since the attention mechanism adapts the algorithm.

In practical terms, a transformer’s sequence length has to be limited to a maximum context length to manage the mapping to computational hardware. By the same token, CPUs also needed unbounded memory to be true Turing machines.

So, are implementable transformers, like general purpose CPU programs, Turing machines in all but the most pedantic terms?

Not quite — there’s still a fundamental gap that cannot be closed. Transformers are still continuous function approximators and cannot efficiently exhibit irregular flow control. A 2026 paper from Oracle AI looks at discrete reasoning with transformers, and I’ll let it speak for itself:

Through this synthesis, we provide readers with a cohesive understanding of why transformers succeed in interpolation tasks (e.g. summarization) but fall short in reliably executing symbolic algorithms.

Symbolic algorithms are characterized by discontinuous outputs that present a challenge to transformers. Like in the square wave example above, you can try to circumvent the issue by increasing model width or dataset size, but this comes at the cost of greatly increased model size and inefficiency. Moreover, as the paper points out, as you compose symbolic tasks (task A → task B → …) the number of switching boundaries grows combinatorially.

For an LLM to change its mind on a yes / no answer, architecturally it needs to continuously interpolate through reasoning trajectories, traversed by generating (lots of) reasoning tokens.

Closing Thoughts

Deep neural networks can solve a huge variety of problems, founded on their universal function approximation ability. Transformers’ ability to process arbitrary sequences advances them into a new computational category beyond lookup tables.

However, they are still not well suited to problems with symbolic or discontinuous outputs. This is common in problems to do with safety or symbolic reasoning. In current successes of deep learning, solutions to these kinds of problems are attained in a similar fashion as the square wave approximation above — it works, but is extremely inefficient.

These problems could potentially be solved with much smaller models if they had Turing machine-style universal computation capabilities. ’s article linked below advocates for the same thing, approaching it from the computational hardware perspective for some classes of problems. In a follow up post, I’ll tie together the first-principles analysis in this post to current computational hardware, to discuss how different algorithm classes effectively map.

Thanks for reading!

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

References and Further Reading

Are Transformers Turing-complete? — Hessam Akhlaghpour (2024)

Barriers to Discrete Reasoning with Transformers — Oracle AI (2026)

Artificial Intelligence Made Simple

The Great Compute Re-Architecture: Why Branching & Sparsity Will Define the Next Decade of Silicon [Breakdowns]

It takes time to create work that’s clear, independent, and genuinely useful. If you’ve found value in this newsletter, consider becoming a paid subscriber. It helps me dive deeper into research, reach more people, stay free from ads/hidden agendas, and supports my crippling chocolate milk addiction…

Read more

a year ago · 57 likes · 17 comments · Devansh

For robotics, we may be in the midst of the first time this has ever happened.1 The start of our research careers resembled the “wild west” of emerging techniques and technologies, but ideas have converged more now. On one hand, robotic hardware has gotten good enough to see thousands of robots of getting shipped and used, by consumers and researchers alike. On the algorithm side, the bitter lesson and its corollary — hypothesized “scaling laws” — have provided a scaffolding around which progress can be evaluated. End-to-end behavior cloning policies seem like they can generalize to all sorts of tasks, and performance predictably improves with more data. We’ll refer to these two trends as commoditization and architectural convergence, and discuss how they shape the current paradigm below.

The establishment of this current paradigm has also had side-effects on the nature of research that may in themselves be setting us up for paradigm shifts. While it is a bit of an overreach to use the term “revolution” for robotics (as Kuhn did for science), such a shift would be pivotal for researchers and is worth understanding.

This article is co-written by and , both robotics researchers with experience in academia as well as industry. Chris writes about AI and robotics, and Avik writes about robotics, computing, and AI.

min{power}

Explorations in computing and robotics focused on power-efficiency and safety -- personal posts by Avik De, robotics Ph.D. and founder

By Avik De

It Can Think!

Robotics and AI; the future we're building and how we'll get there

By Chris Paxton

Trends in Robotics and AI

1) Commoditization

Going back to 2013, Avik’s Ph.D. research included the development of an internal research robot, Minitaur:

It took a lot of (Ph.D. student) effort to build the infrastructure, but resulted in a unique development platform that was easy to program (as an Arduino), lightweight and relatively safe (5 kg), and capable of producing very agile and exciting-looking behaviors. There was nothing like it that you could buy. All in all, this endeavor to develop a new robot led to papers, cool movies to show in talks, and even a startup company.

In the decade after, four-legged robots started to get out of the research lab and into public consciousness. The show Silicon Valley had a Boston Dynamics Spot cameo in 2016, and robot videos designed to appeal to a broad audience, like dancing, started to appear. Four-legged robots were officially out of the lab and in the wild, and this led to increased expectations for what they should do. Stably walking around used to be cutting edge, but became table stakes. Expectations for specs such as reliability, battery life, compute capability, ruggedness drove designs to be more complex. It became much more difficult for a couple of researchers with minimal engineering experience to put together a new robot. Moreover, after Chinese company Unitree entered the market and dropped the asking price by almost 30x in 2021, it became not worth the time and dollars to even try.

The pre-paradigm period of lab-developed robotic hardware is being replaced by algorithm development for commoditized hardware.

We have seen this play out in several robotics research labs. DJI commoditized consumer drones aggressively from 2013 onward, making it hard to justify custom builds even for capability reasons. By the mid-2010s, labs doing serious flight research (e.g., Davide Scaramuzza’s group at University of Zurich) were exclusively using commercial platforms. ETH Zurich’s Robotic Systems Lab (which built ANYmal originally, and also STarLETH) now deploys their locomotion research on the ANYmal platform rather than building new hardware. Boston Dynamics has an article that talks about how commercial platforms let researchers hit the ground running.

Post-commoditization, researchers who want to demonstrate algorithms working on robots can reap the benefits. Humanoid research circa 2015 meant figuring out “how do we actually build these things and make them not fall over,” whereas post-commoditization, time can be spent on higher-level algorithms and methods — we refer to this phenomenon as “moving up the stack.”

A secondary effect of commoditization is that parts are now easier to get, and researchers can put together novel modular combinations of more mature components. The WidowX 250 Dynamixel-based arm from Trossen Robotics has become the default low-cost manipulation platform because it is cheap (~$3k) and can be used to create “leader-follower” setups for data collection. The ALOHA paper notes that the whole system with two arms costs ~$20k off-the-shelf. More recently, we have seen robots like the YOR assembled from off-the-shelf parts for research purposes. This effect enables new types and form-factors of robots to be built — we will return to this in the next section.

The same trend applies to non-hardware AI research. Frontier language models cannot really be trained by academic research labs any more — research in these areas moves to fine-tuning commercial models instead. The following plots were generated from arXiv data and confirm these trends toward pretrained model usage in research compared to building them from scratch.

The Qwen series of models by Alibaba have nearly taken over the research world by facilitating fine-tuning. In 2026, no academics would think of training their own language models or even vision-language models from scratch — why would you, when Qwen 3.5 can already beat anything that’s within reach of an ordinary academic lab?

Just like for robotics hardware, the pre-paradigm period of lab-developed models is being replaced by fine-tuning commercial models.

Here as well, there are research ideas which can be pursued by moving up the stack: agentic reasoning, reinforcement learning, world representations, novel model architectures, etc. Robotics models are not like language models; there are fewer real world benchmarks and it seems that even within the domain of end-to-end deep learning there are plenty of ideas left unexplored.

2) Architectural Convergence

Labs used to have a narrower focus where they could carve their niche, e.g. computer vision, legged locomotion, etc. However, for a robot to demonstrate complex sensorimotor tasks, you need all of the Sense-Plan-Act functions implemented in some way. If you subscribe to the bitter lesson, even the best computer vision algorithm, when connected using hand-crafted interfaces to a planner and other downstream systems, cannot compete with end-to-end systems. General-purpose manipulation / locomotion research is converging on behavior cloning and VLAs since it works well enough across many tasks, and performance improves with larger models and more data.

This trend has pushed many previously-diverse labs toward developing end-to-end models, which is a significant reduction in the diversity and richness in the research ecosystem. For better or worse, we appear to solidly be in a paradigm of behavior cloning with end-to-end models.

This has several benefits for researchers: they can build on existing work easily without re-inventing the wheel, and it creates a scaffolding for new contributions. However, it also has the side-effect of suppressing other schools of thought. In Kuhn’s somewhat ominous words,

But there are always some men who cling to one or another of the older views, and they are simply read out of the profession, which thereafter ignores their work. The new paradigm implies a new and more rigid definition of the field. Those unwilling or unable to accommodate their work to it must proceed in isolation or attach themselves to some other group.

How do research labs and out-of-paradigm ideas stand out in the face of homogenization and consolidation in this paradigm? We discuss what we can learn from computer engineering in the next section.

Subscribe now

What we can learn from Computer Engineering

By necessity, computer engineering has always been a bit ahead of the same technology curve as robotics. After all, we needed the chips to facilitate computations needed for robots to work.

We saw there a similar commoditization trend, with hardware complexity outgrowing what a research lab could build. The initial university fab era was anchored by DARPA’s VLSI Project, which produced BSD Unix, the RISC concept, and MOSIS (a shared fab for academia). Once that era ended, academic research pivoted to what could be done without a fab.

As a response, computer engineering therefore shows a good set of examples of moving up the stack (transistors → meta-design tools and ISAs). Circa 2010, rather than building chips, Krste Asanović’s group at Berkeley designed the open RISC-V ISA explicitly motivated by the problem of proprietary architectures impeding academic research. With Chisel (Berkeley), academics built better tools for designing chips, by expressing hardware designs in a high-level language, and it became the foundation for most RISC-V implementations.

In addition, CPU architectures converged to x86 for desktop and ARM for mobile because they worked well enough for most workloads, and design costs could be amortized across different applications — a general-purpose computing paradigm.

Hennessy and Patterson’s 2017 Turing Award lecture argued that the post-Dennard-scaling era opens up a new window for research in domain-specific accelerators, where the design space is exploratory again. Coincident with the success of deep neural networks, the last few years have seen a Cambrian explosion in AI accelerators, ushering in much more innovation in computer architecture and silicon than was possible in CPUs.

In other words, computer engineering’s paradigm shift resulted in domain-specific diversification.

How do these apply to robotics and AI?

Just as chip fabrication leaving academia didn’t end computer architecture research, robotics research will find a home in core algorithms, training methodologies, and novel architectures. While papers can continue to be written on new methods and algorithms, unfortunately, the flashy demonstrations (important for fundraising and PR) may go out of lab reach. Similar to how ChatGPT capitalized on published transformer research, companies will capitalize on published public-domain research. It may become crucial to have a credit mechanism for academics for commercial usage of their work (this is not covered by academic metrics such as h-index).

The largest robotics companies are converging on general-purpose humanoids, optimizing for the broadest possible applicability and commercial value. By analogy to computer engineering’s domain-specific diversification, the next productive frontier for academic labs may be task-specific robots: surgical, agricultural, soft robots, etc., which diverge enough from general-purpose designs to make bespoke solutions worthwhile. A positive side-effect of the commoditization of hardware components (like actuators, IMUs, perception systems like the Kinect) all come together to facilitate this kind of development.

The Future

While the external perception of robotics and AI research is that we are undergoing a revolution today, the internal view is more consistent with commoditization and convergence. This paradigm has had a lot of positive side-effects, like establishing a framework and shared infrastructure, but also some serious downsides, like stifling research that doesn’t fit the mold.

In response, we already see the reality of robotics research moving up the stack, and we will potentially begin to see examples of domain-specific diversification if the largest companies with the largest datasets corner the end-to-end behavior cloning approach.

Beyond that, it’s too early to predict if there is a paradigm shift coming. Kuhn says on this topic:

Sometimes a normal problem, one that ought to be solvable by known rules and procedures, resists the reiterated onslaught of the ablest members of the group within whose competence it falls. On other occasions a piece of equipment designed and constructed for the purpose of normal research fails to perform in the anticipated manner, revealing an anomaly that cannot, despite repeated effort, be aligned with professional expectation.

Will there be a “piece of equipment” or “normal problem” whose unexpected result paves the way for the next robotics revolution? Optimistically, it seems like the current paradigm still has legs for a little while longer, but there is already work at the fringes looking toward the next set of leaps, like world model research, neuromorphic computing, etc. We’ll be writing about these topics over the coming weeks and months; stay tuned!

Subscribe now

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

What is a good comparison is this performance to last year’s, when the best robot time was over 160 minutes, or more than 3x this year’s time. That’s a remarkable improvement in one year. My doctoral thesis involved building and controlling hopping and running robots, and since then I’ve tried to design and build efficient commercial legged robots, giving me a decent idea of the constraints involved. So, in this article I wanted to try and examine — how did they do it? Is there some magical technology or technique that unlocked this performance? How did they beat the significantly better-known Unitree (who reportedly had to supply an ice pack backpack to try and complete the race without overheating)? Could a western robot have won?

This publication and this post contain the author’s personal thoughts and opinions only, and do not reflect the views of any companies or institutions.

The basic physics of hopping and running

Hopping, very simply, consists of alternating phases of a leg pushing against the ground (“stance phase”) and the body flying through the air (“aerial phase”).

In aerial phase, the body simply free-falls (constant acceleration due to gravity). You can think of this as losing vertical momentum. In stance phase, the job of the leg is to push against the ground to reverse this vertical momentum. The job of the “knee” actuator is primarily to generate this force in stance phase.

The other basic leg function is repositioning for the next foothold. In bipedal running, while one leg is pushing against the ground, the other leg is swinging to reposition for the next step. The job of the “hip” actuator is primarily to swing the leg forward.

Bipedal running is simply these two functions alternating in the two legs — while the left leg pushes against the ground, the right leg swings forward, and vice versa. Of course, this is an oversimplification in many ways, but it still captures the main effects that contribute to running energetics. Namely, it becomes clear that:

• the knee actuator must produce enough torque to reverse the entire robot momentum in the stance duration T_s

• the hip actuator must product enough power to accelerate the leg forward in the swing duration T_sw

The way a robot runs faster is that it increases its stride length and/or shortens the stance duration.

Shortening the stance duration requires a higher amount of knee torque to be needed to accomplish the same momentum reversal. Swinging the leg faster, and covering a longer stride length requires more torque and power from the hip actuator.

And just like that, with very basic physics, we’ve recovered the dependence of running speed on the torque and power produced by the actuators.

The basic physics of motors

Electric motors dissipate energy in an exact relation to the amount of torque they produce, and these quantities are related by an appropriately-named constant termed the motor constant, K_m. If τ is the torque produced by the motor and Q is the heat it produces,

In the “New Motor Models” section in my thesis (2017) I described how a K_m scaling relation can be approximated from rough first-principles geometry arguments. In particular, for a fixed length scale, K_m scales with the square root of motor mass √m. In a recent post, longtime blogger and roboticist Ben Katz generalizes and gives this coefficient a name , the “figure of merit (FoM),” which we can use here:

The r above is the motor diameter. To estimate motor mass m, I decided to relate it to the motor diameter and (unknown) length. With these, and assuming a high but reasonable FoM of 15, we can extrapolate the likely K_m.

To estimate the rotor inertia, we can relate it to the motor mass and inertia as j ~ mr² as Ben Katz also does.

Adding a geartrain (gear ratio G) after the motor amplifies its torque and reduces its speed by G. So, it helps with torque production, but it has a very deleterious effect in legged systems when accelerating. Since the rotor of the motor itself has to spin faster, the rotor inertia j in the output frame appears scaled to G²j, which can quickly become very large. Thus, a small motor with large gearing becomes very sluggish at accelerating its output, even if it can statically produce a large torque. This is obviously bad for the “swing phase” described above.

The Honor Lightning’s technology

There isn’t a technical report on this robot as far as I know, but some online articles list a few specifications. I referred to this substack article for this post. A couple of notes:

• This article and a few others say that the robot has 55 joints, but that is definitely a mistake. Potentially with hands (that were not equipped on these half-marathon versions) it could have 55 joints, but as deployed, they probably had closer to half as many joints.

• The page also lists “Leaderdrive” as a harmonic reducer technology partner implying that strain wave gearing was used. However, based on the analysis below, a lower reduction-ratio planetary or another type of gearing is more appropriate, especially for this kind of efficiency-critical application.

Actuation: motor, gearing, gait

These three factors are all interrelated and have an effect on how much energy is required and how much heat is produced. To see how, let’s start with the motor.

Typically, the motor K_m can be found in the datasheet, but in this case there’s no public reporting on the motor specs.

However, we can see the size of the fairly large hip/knee motors attached to the upper leg (my rough estimation is that the outer diameter is somewhere between 110-150mm from the image above). We can look at a couple of potential options: first, a reasonable 115mm diameter catalog motor, which I chose from TQ’s frameless motor catalog for similar reasons to Ben Katz’s blog post — they are well-documented and have a large selection. Second, we can use the scaling principles to make some reasonably good approximations of K_m for a hypothetical larger motor. I extrapolated to a 150x25 sized motor to obtain a K_m of 1.52 Nm/sqrt(W), and a mass of almost 2 kg.

Since we don’t know the gear ratio, we can use our simple physics model (script linked in references below) to estimate the power consumption for running for the “small” and “big” motors above as a function of G:

Note that:

• A high gear ratio is nice to minimize the power in the knee actuator (since its job of supporting the robot weight is made easier with mechanical advantage), but a high gear ratio also makes the leg swing energetically difficult.1 There’s usually a middle-ground optimum.

• The larger motor (150x25) prefers a smaller gear ratio (~23:1), and the smaller motor (115x25) prefers a higher gear ratio (~40:1).2 These are shown with the dashed gray lines.

• Both of these options seem to be able to accomplish the basic push-ground and leg-swing functions, with modest robot power consumption of 400-500 W.

So, in sum, the motor is not magical technology and in fact, a range of existing or projected options would work, when appropriately sized for this task. I’ll get back to this last bit and the green lines in the plot later.

The dissipated knee power (which is typically the main thermal limiting factor) is ~150W for both solutions. This is almost an unavoidable consequence — due to the predictable scaling of motor K_m, running at human speeds with a humanoid-sized robot will inevitably generate this amount of heat!

This, finally, is where we would see a potentially large difference between the two motors. Motor cooling is affected by the surface area over which heat removal can occur, and the larger motor has 70% more surface area. Even so, over a prolonged period, 150W is a large amount of power to dissipate from a single motor, and this is where one of the stated innovations in this robot design appear to be coming to bear (source):

According to Honor, the liquid - cooling pipes penetrate deep into the motors like capillaries. The high - power liquid pump has a heat - exchange flow rate of more than 4 liters per minute. Each of the four drive motors in the lower limbs is equipped with an independent liquid - cooling circuit.

Liquid cooling is not new, but it’s definitely not what I would call a commodity. It has shown up in research periodically, and on the commercial side Apptronik tried it for a few of their prototypes but (to my knowledge) does not use it on their main Apollo platform. While it definitely is not magical technology, it has been niche so far. As described above, it is absolutely essential (and so far quite challenging) to be able to dissipate ~150W from a motor for running at these speeds. From that respect, the liquid cooling tech is a key enabler of this type of performance.

Caveat: The script I used to generate the plots above makes a lot of simplifying approximations. It doesn’t capture the energy dissipated in other motors (arms, ankles, abduction, etc.). The basic physics principles don’t lie about the periodic center-of-mass behavior, but this doesn’t model other oscillations in the orientation as the body sways etc., or losses like friction or air resistance. The inertia of the leg is left out of the swing inertia calculation, since there is no way to approximate it properly with the information available. Published materials emphasize a lightweight leg construction, which indicates that the rotor reflected inertia will likely dominate it (and so the script’s approximation is likely good). There are more accurate ways to estimate the swing energetics incorporating the leg kinematics and swing trajectory, but I wanted to not increase the complexity of this analysis and chose to err on the side of simplicity. Still, I think the main estimates and talking points (motor / gearing selection for the knee motor, and power dissipated in it) can be trusted.

AI and autonomy

There’s nothing to write home about here. The gait controller could have used either a reinforcement learning (RL) controller, which is easy to train for flat ground, or a model-based controller. The autonomous navigation system used a provided GNSS system and just had to follow the route waypoints. This is all very well-understood technology.

Battery

Let’s assume that the battery was chosen to last 1.5 hrs (the robot finished in < 1 hr). For 600 W consumption (based on the figures above with some buffer), the battery would have had to have 900 Wh capacity, and at 300 Wh/kg energy density, the pack would have weighed 3 kg. This is well within reason for a 45 kg robot. Additionally, a 1.5 hour discharge time indicates a 1/1.5 or 0.67C discharge, which is well within the ratings of most existing batteries.

The Unitree H1 reportedly needed “pit stops” and battery cooling ice, indicating that it was consuming much higher power. We’ll talk about that more next.

Engineering always involves tradeoffs

Engineering is always characterized by tradeoffs — that’s what makes it challenging but also fun. Especially today with ever-stronger AI language models, the very human skill of judgment and knowing how to made tradeoffs is much more important than the rote work of completing a design to spec.

Even with the very simple model above, it was not that complex to roughly design a drivetrain that is theoretically capable of this feat. Then why did the competitors in the race, including more established and widely-shipped humanoids such as from Unitree or Agibot, not compete as well?

We can use the simple model to generate an equivalent energetics plot for walking at 1.5 m/s, a much more modest but potentially more common activity for a commercial humanoid robot:

The gray lines are as before — gear ratios optimized for half-marathon running. The green lines are where the power is minimized for walking, and they are significantly different!

Let’s say you design your robot to excel at the normal walking task and chose the green gear ratios. The knee motor power to run a half marathon with that green design consumes > 300 W, more than 2x what we had with the running-optimized gray designs. It wouldn’t be so surprising to need ice packs!

Conversely, the running-optimized gray design, when used for the walking task, wastes significantly more motor power than the green designs (as seen from where they intersect the blue curves). We couldn’t model this effect with the information available, but using larger motors sized for running also increases the weight of the robot and constantly wastes power when it isn’t running at full speed. You can visually see the difference in motor sizes between the Unitree H1 and Honor Lightning:

The larger motors will have all sorts of practical (if not fundamental) consequences like bumping into objects while operating in homes or factories.

Closing thoughts

What should we conclude from Honor’s accomplishment? First, the capillary motor cooling solution, if mass manufacturable, is a genuine advance, and I suspect this running pace would not have been sustainable without it. Second, even if there wasn’t any “magic” needed, this was a really impressive engineering effort and result. For better or worse, it deserves to be a landmark tantamount to Deep Blue v. Kasparov.

Having said that, I don’t believe this says anything at all about human half-marathon performances. It doesn’t even imply that a humanoid robot could join a race among a sea of people without GPS and resiliently finish the race. I wish those comparisons would be left out of the press coverage.

Another thing I found interesting is that the Lightning robot was reportedly developed in about a year, between MWC in March 2025 and the April 2026 race. That is incredibly fast. However, what is even more stunning is that the R&D team reportedly had 2,600 people. Comparing to a few US humanoid robot companies, to my knowledge, that eclipses the headcounts of Boston Dynamics, Figure, Agility, and Apptronik combined (I am not sure of Tesla’s Optimus-specific headcount). On top of that, you have to account for the partner and manufacturing ecosystem that was brought to bear, as reported by the same linked article.

Is all this worth it? It probably isn’t for most of these companies who need to spend their resources developing applications customers need and will pay for, but the cooling and weight-reduction advances may well be useful for more practical purposes like carrying heavy payloads down the line.

If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts. All of this is greatly appreciated.

Subscribe now

References

• Script used for power estimates (Github gist)

• Spreadsheet with motor parameters and estimates

To showcase and demonstrate some of the insights from the past articles, I’ve put together a demonstration of the insights from this article series that you can try out, modify, and learn from. This demo combines the flexible task programming and reasoning of the Gemini ER Vision-Language-Model (what is the scene, and what should I do?) and classical camera calibration, kinematics, motion controllers.

This post describes how it is put together, goes over of some of its interesting capabilities, and the aspects of its design that directly impact its strengths and weaknesses. To conclude, we will try to compare this approach against fully modular (model-based) as well as fully end-to-end methods. The code is open source, and I’m putting the ideas out there for discussion and feedback.

This article is the last part of a series on end-to-end robotics pipelines. Links to the other articles are below.

Trying out the demo

To make it as accessible as possible, the demo runs in the browser with no software installation required, and can be accessed from your computer or even a phone. Click this button or to open the page:

Link to demo

The environment is set up for tabletop manipulation with a robot arm. The colored blocks are objects that we can instruct the arm to move, the “plates” can serve as potential goal locations, and the grey cylinders can serve as obstacles to be avoided.

The demo uses a Gemini Robotics ER model for task reasoning and perception. To try it out, you need to grab your own Gemini API key (free tier), or use the pre-baked fallback plan, which will execute the “Put the blocks away where they belong” default task. Correspondingly, click “Run Task” (with API key) or “Use Cached Task” and watch! Use the mouse to orbit the camera, and check the console for debug logs.

What it does well

Flexible task programming and reasoning. Tasks can be prompted without needing task-specific programming, which is a major selling-point: the possible tasks are not limited by what is programmed at the factory. Gemini processes the prompt together with the scene and can break down multi-step tasks. We’ll go over how Gemini’s outputs are used by the rest of the system below.

Results from some tasks:

Place the red block on the blue target

This simple task shows the VLM’s visual and task understanding. Additionally, its language understanding can parse semantically similar words in the context of the scene (e.g. block vs. cube, or plate vs. coaster vs. target).

The video also shows the reactive obstacle avoidance allowing the arm to not collide with the cylindrical obstacles. This capability, with associated safety benefits, does not require any training or motion primitives to be built into the VLM. More on that below.

“Put the blocks on matching targets”

The VLM successfully reasons that blocks go on color-matched plates, and breaks down the task into a number of steps (move red block, move blue block).

“Swap the red and blue blocks”

This task requires a multi-step plan to move one of the blocks out of the way first, and the selection of a free location to store it.

The wireframes displayed in the animation show the spatial understanding ability built from a combination of a Gemini VLM with classical computer vision. Objects in the scene are semantically classified — into objects (blue wireframes), potential goal locations (green), and potential obstacles (black) — by the VLM guided by prompting, without hardcoding.

As a note of caution, I had a few runs where it chose the “free” location incorrectly on top of another block.

“Put away the blocks”

The success of this (underspecified) prompt showcases the language and intent understanding of the VLM. However, I will temper with the note that in some runs, it did try to move the green block and confuse itself — feel free to try it yourself!

“Wave”

This silly task shows that the VLM’s task understanding goes beyond tabletop manipulation, as it can produce waypoints just intended for arbitrary motion. However, this demo will only successfully perform horizontal motion due to the 2-dimensional understanding of the VLM — more on that below.

What is challenging

The principal weaknesses are also to do with the 2-dimensional understanding of the VLM.

“Stack the blocks”

It correctly moves multiple blocks to the same horizontal position, but does not properly reason about the vertical location of each drop-off. This results in the later blocks getting smashed into the ones already placed.

The architecture explains strengths and weaknesses

The architecture of the demo is shown below:

Gemini (VLM) blocks are blue, and blocks built using classical methods are green. Each layer is independently swappable, and the AI model doesn’t need to know anything about the robot’s embodiment. This recreates the modularity of a Sense-Plan-Act architecture while retaining the semantic reasoning of a foundation AI model.

Vision-Language Model (VLM)

The demo uses Gemini ER, whose inclusion I previously motivated with a coding agent analogy. Its inputs are the text prompt and a single image, and its outputs are grounded in pixels in the same image. This keeps its behavior well-defined and decoupled from the robot embodiment, solving many of the issues with X-VLA in a similar setup.

However, it builds in a few assumptions that should be acknowledged. Most importantly, its understanding of the world is decidedly planar (pixels in the image plane).1 The view must therefore be chosen to avoid occlusion, parallax-related issues as the camera moves, and tasks that require positioning along the camera axis (like the block-stacking task above).

Gemini ER can be prompted to output structured JSON, which is easy to work with in downstream layers. The system is first prompted for “perception”, which does object detection, semantic classification, and bounding box identification. All of these are very common functions, and easy for this model to complete in ~1 second. An example output for the perception block is below:

[
   {
      "label":"green block", // <- a name
      "point":[637, 232], // <- position in image coordinates
      "box_2d":[531, 157, 743, 305], // <- image coordinates of bounding box
      "type":"block" // <- semantic classification
   }, // ... other detections 
]

The next step asks the model to plan the motion for a task. We specify an output format that limits the output to “calling functions” that the arm and its lower-level controller is capable of executing. Example output from Gemini (took anywhere from 4-10 seconds):

[
   {
      "function":"move", // a function that moves the arm
      "args":[584, 753, false], // position (image coords) + 1 bit indicating height
   },
   {
      "function":"setGripperState", // a function that closes or opens the gripper
      "args":[false] // false to close, true to open
   }, // ... other steps
]

The full prompts to get these outputs are part of the open-source package.

Spatial understanding

First, we convert the image-plane understanding of the VLM into spatially accurate waypoints that the arm can act on. For this conversion, I also sampled depth values from the camera location (easily reproducible with stereoscopy or a model like DepthAnything). I chose to use the bounding boxes to isolate the depth values in a region around the object center, and use that to fit primitive shapes to the detections (rendered with wireframes in the videos above). This can be done by well-understood camera geometry transformations, and also allows for relocation of the camera, unlike in a VLA where the camera geometry is inextricably linked into the rest of the model. The output of this block is 3D waypoints and a representation of the obstacles.

The object shape affects how well a bounding box captures inlying depth pixels. Gemini also has a native ability to output segmentation masks, which could allow for further refinement in this computational block.

Model-based local planner

The next part is a model-based local planner that actually generates control signals. This decouples the control rate from the slow runtime of the VLM completely, and no retraining is needed to generate novel motions for new scene compositions. Adaptations for payload could be built into this layer without affecting VLM.

For obstacle avoidance, we use a “potential field” that pushes the end-effector away from obstacles (you can see orange arrows appearing briefly in the animations above), while moving toward the desired goal. This is a classic reactive motion planning technique, one of a family of well-understood algorithms along with sampling-based and grid-search planners.

The interface is crucial

The VLA approach had no choices to be made about the type of interface — when trained on the same embodiment end-to-end, input pixels get mapped straight through to actions. However, with this hierarchical controller, the choice of interface is quite important. While it resolved many of the drawbacks of the full end-to-end approach that held back a demonstration like this, one of the interpretations of the bitter lesson is that any hand-crafted interface design hampers system performance.

For example, for grasp generation in this demo, we have to assume that knowing the location of block is sufficient to produce an action to grasp it. However, different grasping actions may be needed for soft or unusually-shaped objects, like eggs, cloth, etc. One possible extension to resolve this is to incorporate a grasp generation module seeded by the object centers and bounding boxes. A VLA will just directly output actions, which is not limited by this kind of architectural judgment, but also may require a lot more training data and fail unpredictably when out of distribution.

Scoring the criteria from the first article

The architecture described above is neither an end-to-end VLA, nor a modular model-based one. For specificity in this section, I’ll assume the former camp as being represented by something like Physical Intelligence’s models (a small version of which we tried hands-on with X-VLA), and the latter as being represented by the MIT 2014 Atlas method. Both were discussed in the first part in this series about end-to-end robotics pipelines.

All that said, where does this “hybrid” hierarchical strategy fall? We identified a number of criteria in previous articles, and can try to roughly size up where each falls:

My summary would be that the end-to-end method can be the best if it is scaled potentially ad infinitum and has very fast computational hardware, which has practical (data requirements) and efficiency drawbacks. I think the hybrid architecture could be a good middle ground to greatly expand capabilities with less data and added safety and efficiency, but has some bottlenecks from interface choices that may impact some applications (but in a predictable way). I’m open to your thoughts — let me know below!

Leave a comment

Closing thoughts

This demo was put together with models released within the last year, but also with ideas that have existed for decades. We’ve been seeing transformational improvement in the capabilities of deep neural networks, leading in many cases to large strategic shifts to embrace fully end-to-end architectures. However, this shift brings with it new problems in safety, efficiency, and predictability. This post goes over a proposal for a hybrid architecture that attempts to draw on the strengths of both camps.

There is room for improvement in end-to-end VLA approaches with scaling, as well as in this kind of hybrid architecture (faster VLM inference, multi-view VLM). “World model” methods are rapidly gaining popularity as a component of larger modular pipelines (stay tuned for future posts on this topic). I also plan to look more into how to build an “embodied reasoning” open-weight VLM in future posts.

Please check out the demo, and the source code.

If you liked this post, please like ♡, share, restack, and subscribe — it helps others find my writing.

Subscribe now

Further reading

Other articles in this series:

1. The architecture behind “end-to-end” robotics pipelines

2. Online motor adaptation

3. VLA debugging insights

4. This article

5. Demo combining the best features of end-to-end and classical approaches

In this part, we finally close the loop to get our WidowX robot arm in the MuJoCo simulation to execute some manipulation tasks. I’ll go over how to build up (from scratch) something like the following behavior from a text prompt, and what we can learn about the architecture of robotics pipelines in the process.

Result of “Place the red block on the blue target”:

An end-to-end Vision-Language-Action (VLA) model is the obvious modern technology researchers and companies are moving toward for this kind of functionality, and part 3 of this series was dedicated to understanding them from the inside. The deployment exercise for this part made clear that a small VLA’s failure modes are difficult—to the point of impossible—to eliminate without retraining.

That observation ultimately forced a pivot to a different architecture, where the flexible programming and semantic reasoning layer delegates physical grounding to explicitly separate tools. This post explains how that architecture works, and what it says about robotics pipelines more broadly.

In addition to the story, all the code is open-source — feel free to learn from, star, and fork! Also, if you like this kind of post, please like, share, and subscribe:

Moravec’s paradox and VLAs: control bandwidth problem

In the previous post, we spent some time interpreting the perception and language understanding in VLAs — specifically, the Vision and Language parts. In many ways, the action head exhibits the most architectural diversity in VLAs.

There are broadly two types of action heads. (Auto)-regressive action heads generate actions sequentially, one at a time. This is similar to how most LLMs work today, generating tokens one after the other. Generative (or diffusion / flow-matching) action heads, in contrast, generate a whole action sequence at a time and incrementally refine it, similar to diffusion-based image generators.

Regressive action generators have a fundamental difficulty when used for behavior cloning in continuous action spaces. As Max Simchowitz presents in his recent CMU RI seminar1, the issue is that a small deviation takes the red robot trajectory off the training (expert) demonstration distribution, and it is unable to recover.

The same problem doesn’t occur in discrete spaces (like text generation) because they can be trained with a 0-1 or cross-entropy loss function, encouraging very aggressive contraction to the training distribution. Simchowitz identifies this challenge in continuous spaces with Moravec’s paradox (why learning hasn’t been as effective in physical tasks as in symbolic tasks like language).

Action chunking2 presents a way to get around this problem. By producing an action sequence, over which the natural dynamics of the system is assumed to prevent compounding error, the rate of divergence is kept under control:

A key assumption there was that the underlying system needs to have some strong stability properties. I won’t go into definitions here, but in practice, this means that VLA actions are almost exclusively desired positions (as opposed to velocity or torque). More generally, this means that the behavior is what is called “quasi-static”, i.e. the robot goes through a sequence of statically stable configurations. As an aside, this is also why VLA-implemented manipulation behaviors are slow and wouldn’t apply to dynamic behaviors like agile locomotion; quoting this Quanta magazine article, “Atlas moves like molasses while grasping auto parts but glides like a gymnast when it’s not touching anything except the floor”.

So, action chunking is a way to address the control bandwidth problem from part 1 for regressive policies. Generative action heads don’t have the same inherent divergence issue, and it does seem like in practice most VLAs use that strategy — this also applies to our demo setup with X-VLA from part 3. They learn a distribution over actions, and at inference time, start with a pure “noise” action and iteratively denoise it. One thing to note is that the trajectory horizon does not impact how long the inference takes (it is just the size of the action distribution learned during training). This means that shortening the action horizon size in order to get faster results isn’t an option like it typically is in model-predictive control.

Now that we understand VLA action heads a little better, let’s move on to closing the action loop.

Closing the loop with X-VLA: generalization and separation problems

The VLA outputs action chunks (a sequence of desired poses), and we now need to control the motors to reach them. The model for the WidowX arm in our simulation is set up for position control on the joints. This is in part due to how most people are using this arm (in some cases due to algorithmic constraints as mentioned above). For this article, I chose to keep that as is, and as a first pass, implement the most reasonable control method in this situation: inverse kinematics (IK). The implementation uses gradient descent to iteratively find the joint angles that reach a certain pose. This is a generalizable and quick method that will probably get replaced in the last part of the series by a non-IK solution.

After closing the control loop, prompting the VLA to “pick up the red block”, and running the simulation — well, it didn’t work. At this point, it was a little bit of the same challenge of “black box debugging” as in part 3, but now with more (literal) moving pieces.

It’s important to remember that X-VLA is a small VLA, and its generalization capabilities are limited by model size. As we saw in part 3, the model’s spatial reasoning (how far to reach, when to close) is tightly coupled to the training camera viewpoints. The camera intrinsic and extrinsic parameters are wrapped up in the full X-VLA policy and not separable3, and so I tried to modify the images received by the policy to try and match the training dataset.

I went into the BridgeData training dataset, and found the most similar task in the training data, grabbed the training video, and tried to make my scene resemble it as closely as possible. To do this, I manually tuned the camera position, robot gripper initial pose and framing (camera extrinsics), image field of view (intrinsics), aspect ratio “squishing” to match training data, lighting / shadows, table appearance:

Unfortunately, despite the manual tuning, and also completely decluttering the scene, the policy didn’t succeed with the prompt “Pick up the red block”:

It consistently overshot the block, which indicated to me that the visual processing had a consistent error, but fiddling with the camera settings didn’t yield a better result. The structural issue with VLAs (non-separability of camera and kinematics parameters) makes this debugging quite challenging, even beyond the techniques from part 3. If you know of anything that could have gotten this to work, let me know in the comments!

Leave a comment

I suspect that the generalization abilities of this size of VLA are just not sufficient to be able to use the policy zero-shot. There are two reasons why that is a roadblock: First (isolated to my usage here), I didn’t have a leader-follower arm or space mouse to collect more training data and go through a fine-tuning process. The second (and more fundamental) issue is that this limits how this kind of strategy can be used by robot end-users in ad-hoc unknown environments.

The flexible task programming and semantic task understanding of VLAs were some of the motivations for this project. Is there an alternative solution that can keep those strengths while adding some needed structure?

An “agentic” modular alternative

For scene and task understanding combined with flexible programming, we need some kind of VLM, but is there a way to get information out of the VLM in a more structured way?

In late 2025, Google released Gemini Robotics 1.5, which consists of two models designed to have a hierarchical interface:

Out of the two, I only used the ER (Embodied Reasoning) model, which has been trained to output structured text combining the spatial understanding and function calling capabilities of the impressive Gemini model family. As documented here, the “pointing” feature is effectively a customizable vision processing pipeline, and I found it to be incredibly robust:

The function calling capabilities can also be used to break down complex tasks into sub-steps, which is what I used for the working demo in the first section of this article. Here you can see that it is flexible to different prompts with no other changes:

Most shockingly, I spent only a couple of hours with the Gemini models to get to the successful end result above, after unsuccessful attempts over a significantly longer period with X-VLA.

So, why does this work so much more easily?

Just like Simchowitz did in his RI seminar, I think I’d have a pragmatic answer to do with scale, as well as an algorithmic answer independent of it. On model size, Gemini ER 1.5 is described as achieving “the low latency of a Gemini Flash model” for spatial tasks, which suggests it's Flash-scale (~8B range) but much larger than X-VLA (0.9B). On the algorithmic side, the difficulties we ran into with the VLA often had to do with inseparability of concerns (kinematics, calibration parameters not separable from the policy), and generalizability (difficult to tell when we were out of distribution).

I think an appropriate analogy here is between an LLM (even a coding-tuned one) to a coding agent like Claude Code (an LLM in a larger system that can interact with “tools”). A coding agent doesn’t ask the LLM to emit machine code directly; it keeps the model in the semantic reasoning layer and delegates execution to existing well-understood tools. In this analogy, I’m suggesting that camera calibration, kinematics, motion controllers are tools that the VLM can benefit from interfacing with. Gemini ER just works on images; a well-defined, separable concern without introducing variability due to the robot morphology. Our known camera transformations then lift its image-space outputs into 3D. If we move the camera (impossible with X-VLA without retraining), we can simply replace the camera calibration parameters.

However, this structural separation appears to contradict the pure end-to-end view that goes back to the “bitter lesson.” Overall, in my opinion, the bitter lesson essay has been interpreted more broadly than current evidence supports, and we will continue to see reinterpretations and corrections.

Closing thoughts

In this part of our series on robotics pipelines, we demonstrated a simple setup that exhibits flexible task programming. Despite our best efforts with an end-to-end VLA, this success came from coupling a strong VLM with model-based “tools” such as camera geometry and inverse / forward kinematics. This seems to me to reflect some of the strengths of agents that interacts with tools vs. an equivalent chatbot-style LLM. It certainly provided a clean way to integrate the strengths of a large learning-based model with structured model-based methods — something I’d set as a goal in part 1 of this series.

While this is a nice result, there are still a number of limitations: Gemini’s task planning is slow, even with cloud hardware. In the current implementation, the full plan is created at startup and there is no replanning for dynamic environments. The model is also not “open” and likely an order of magnitude larger than X-VLA. In the future, I may look into what it takes to develop an “embodied reasoning” model — it seems like the Gemini ER model appears to build on the ideas of the published SpatialVLM.

In the last part of this series, I will plan to improve the lower-level controller from its naive IK implementation to show more responsive and adaptive behavior. I will also aim to publish it in a browser-runnable format so you can very easily and rapidly see the effects of different prompts. As a reminder, the code is all open-source.

If you liked this post, please like (♡), share, restack, and subscribe — it helps others find my writing.

Recognizing performance deterioration

This Verge article from March 19 reports that there could be an impending recall of Tesla’s Full-Self Driving (FSD) service. I’m not interested in making any judgments about self-driving capability, but rather whether the root cause has anything we can learn from in broader robotics.

The issue appears to be that the system didn’t know when it wasn’t working well (causing the issues in the NHTSA filing), or that it did and didn’t notify the driver (which is unlikely, so we’ll assume the former).

This phenomenon isn’t isolated to AVs. The latest article in my Vision-Language-Action (VLA) robotics pipeline series went hands-on into debugging one, and while we found some techniques that can aid developers, they didn’t directly help at inference time. Item 1 in Rui Xu’s candid post-mortem of K-Scale Labs mentions the pitfalls of trusting a “large model” vs. dedicated safety features. Recent papers on VLAs mention the fragility when moving away from the training distribution (e.g. Fang et al Jun 2025, Hu et al Jan 2026).

Potential solutions: redundancy, confidence, architecture

NVIDIA recently announced their new Alpamayo model and accompanying AV stack as a reference open model and toolchain. During the CES 2026 keynote, Jensen Huang said something intriguing about safety:

This parallel or hybrid architecture with a classical stack and a policy arbitrator were also covered in this CounterPoint research article. Interestingly, I can’t find references from NVIDIA themselves about this parallel system other than Jensen’s keynote — it’s possible it is just early in development.

A related approach is to have the VLA output some kind of confidence (vs. a separate “policy arbitrator”). Zollo et al (Dec 2025) formalizes the problem of confidence calibration for VLA policies, describes how to extract confidence estimates from contemporary VLA architectures, and notes that current VLAs lack a reliable mechanism for quantifying the uncertainty of their chosen action sequences. It also introduces two potential remedies: prompt ensembles and action-wise Platt scaling.

Lastly, inserting some debuggable interfaces into end-to-end pipelines can facilitate inspection and safety — lower-level controllers can apply dedicated safety constraints based on the information passed down from a higher-level controller. This appears to still be possible in most successful humanoid robotics demonstrations of today due to a combination of factors. Keeping that architectural feature around may have long-standing benefits, based on current events in the AV industry!

Thanks for reading! I have been working on the next part of the end-to-end pipeline series, with a deep dive into the action head and closed-loop behavior. If you liked this post, please share and subscribe.

All these matrix multiplication units are based on the systolic array, an architectural concept invented by HT Kung at Carnegie Mellon University in the late 1970’s. And Kung’s group didn’t stop at matrix multiplication, they presented a concept of systolic networks of arbitrary processing nodes that could do way more. While some of those concepts appear in niche signal-processing ASICs today, the dominance of deep neural networks over the last decade has caused this history and potential to be significantly overlooked in my opinion.

My interest in this (and the goal of this article) is twofold: (1) Shine a spotlight on this fascinating research and preview the types of problems that can be solved with systolic architectures. (2) Dig into and potentially uncover jumps in performance and efficiency for AI and robotics. I believe that holistic full-stack understanding and optimization (bringing together algorithms and hardware) will be key in advancing these technologies.

Beyond this post, we won’t stop at a theoretical overview — leveraging the computer engineering experience and story-telling of we will actually build up accelerators to use in general-purpose robotics, AI, and scientific applications. We have an article coming soon with the first step, so make sure to subscribe!

Subscribe now

Why systolic architectures

A systolic architecture is characterized by a network of processing elements (PE) that feed data to each other instead of going to the memory hierarchy for operands.

The core benefits are:

• It alleviates memory bottlenecks by allowing multiple compute operations to occur without going to memory (as nicely depicted by the figure above). The design can allow computation time to be balanced with I/O if designed properly, avoiding one stalling due to the other.

• It can create simple, regular designs → a modular setup that can be extended for different functions. It is relatively easy to write the RTL!

• 2D arrays can very easily be deeply pipelined (as we will see below), naturally taking advantage of algorithm concurrency.

The PE network can look like a 1D array (pictured above), 2D array (the most common today), or even other connections for specialized computations.

Data flows between cells in a pipelined fashion, and communication with the outside world is at boundary cells.

The foundation of TPU — a MAC systolic network

A multiply-accumulate (MAC) PE has two input edges and two output edges. In the form drawn below (“weight-stationary”), the weight w is a parameter loaded into the PE. The data x flows in and is passed unchanged left to right, and the current “accumulation” b flows in from the top (usually from a PE connected to the north). The PE does the multiply-accumulate (x * w + b) and passes the accumulated sum down. We assume that the calculation happens in a single “tick” or clock cycle.

A PE in a systolic network is typically a simple compute primitive. Its power comes from connections to other PEs to express complex calculations.

The easiest way to understand how a weight-stationary systolic array works is to understand how a dot product is computed. This is shown in the following image for 3 cycles, and we will walk through the computation.

In each cycle, a new entry of x appears from the left, and one term is added to the dot product. The column of PEs contains a vector of weights. In each cycle, one term of the dot product is accumulated, and after 3 cycles, we have accumulated the full dot product b + w·x.

We now draw the exact same operation, but in an abridged form (not showing the intermediate calculations and instead just showing the inputs and outputs at the ticks they appear).

• A column of the array is drawn as vector weight w_i

• The inputs are drawn as a diagonal (and enters the array skewed in time)

• The output is shown at the bottom, appearing after R cycles from when the input hits row 1, where R is the number of PE’s in the column

In this form it is easy to see that we are pipelining different x_i by starting the second one the cycle after the first one. The initial accumulator value can be set to b_i, an affine bias.

So, with a single column systolic array, holding a column vector w, we are computing y = b + X·w, where the rows of X are x₁, x₂, …

It is also noting the latency between when the x_i starts getting input to when we receive the output: The first element of x₁ enters the array at time t=1, and we get the result out at t=R, so the latency is R-1 cycles.

Making this a 2D array (recall that the input x’s are bypassed to the right from each PE), we see that x_i will just arrive to interact with w₂ one cycle later. We can appropriately skew the columns of the B matrix:

The operation that is executes is Y = B + X·W, where b_ij above is in row i and column j of B, W = [w1, w2] is the fixed weight matrix loaded in first. If W is n×n, and X is m×n, the matrix product is O(mn²) operations (as is standard), but due to the structurally-enforced pipelining, it was completed in O(m+n) cycles!

And just like that, with a very simple MAC-computing PE, we can build up the matrix multiplication hardware unit that is the core of most AI hardware accelerators.

There is much more to be said about how it is implemented in RTL, how it performs, how the matrix shapes affect utilization, the total latency, throughput and efficiency benefits. We will go over that and intuitive insights in an upcoming post. In the remainder of this article, we will turn our attention to other, more overlooked, uses of systolic networks in applications to broader AI, robotics, and numerical methods.

Moving beyond MAC

Keeping the exact same 2D array structure and the skewed input feeding, observe that we had two underlying binary operations: the PE (node) computed multiply (·) and the arrow (edge) computed sum (+). In general, the array will compute the result with those operators replaced by any counterparts: (x₁⊙w₁) ⊕ (x₂⊙w₂) ⊕ ⋯

I’ll be brief with these and list further reading below, and try to draw special attention to ones that are interesting for applications in robotics and AI.

1) Pattern matching (Kung group)

Using logical and (∧) and logical or (∨) as the operations: y = (x₁∧w₁) ∨ (x₂∧w₂) ∨ ⋯

This will return 1 if the vector x matches the vector w, and 0 otherwise.

2) Sorting (Kung group)

This one is fascinating and intuitive. Each PE performs a simple compare and swap operation, and passes the max downward and the min rightward. With n rows and n columns, it will execute the odd-even sort algorithm and produce the sorted array.

A glance at that wikipedia page reveals both a weakness and a strength of systolic arrays. They can only execute algorithms that can work based on local connections (the odd-even sort takes O(n²) operations, vs. more optimal algorithms), but as in matrix multiplication above, the latency is O(n). While the best sort algorithm takes O(n log n) steps sequentially in scalar hardware, the systolic network lets suboptimal algorithms complete with lower latency.

3) 2D motion planning

Deterministic motion planning (identifying environmental obstacles and planning a path through free areas respecting the system dynamics) is a fundamental problem in robotics. About 10 years ago there was even attempt to build chips to solve this problem.

Dynamic programming solutions (including Dijkstra’s algorithm, A*) can be implemented by local and iterative propagation from the goal, and just as with odd-even sort, the nearest-neighbor connection pattern can be mapped well to a systolic array.

Unfortunately, the number of grid cells grows exponentially with the dimension of the ambient space, and this is problematic if we need to have one PE per cell. This makes systolic motion planning impractical unless we only have a 2D problem to solve, but I think it is an interesting application nonetheless.

4) Stereo vision semi-global matching

A PE that accumulates matching costs along a scanline can be used to form a systolic array that implements semi-global matching (SGM). This algorithm is used to calculate disparity in the very popular Intel RealSense camera ASICs.

SGM systolic arrays on FPGAs run this at pixel rate, processing one scanline per clock, and deterministic low-latency computation is obviously paramount here.

5) Matrix decompositions for numerical methods

To some extent, I’ve saved the most promising (at least in my view) for last. Matrix decompositions that aid in factorization are key to solving systems of equations, and this is ubiquitous in all sorts of robotics and general problems.

5.1) QR decomposition. This matrix factorization is the numerically stable way to solve least squares or pseudoinverses in overdetermined systems, and has applications to robot kinematics, SLAM, sensor fusion, online parameter estimation, etc. Additionally, it is a key component of quadratic program (QP) solvers: in active set solvers after the active set is identified, for Jacobian factorization in SQP, etc. These workloads are important in low-level control in robotics and typically need deterministic and low-latency solutions. The Givens rotations method (gentle explainer here) performs local operations on 2x2 submatrices, which lends itself very well to locally-connected CORDIC-implementing PEs in a systolic array.

5.2) Cholesky decomposition for symmetric positive-definite (SPD) matrices. This is a slightly easier factorization if the matrix is SPD, which comes up for example in state estimation, Kalman filtering, normal equations in interior point methods, etc. These workloads would come up in dedicated state estimation blocks in robotics pipelines. For decomposing A = LL^T with lower-triangular L, each PE computes one entry of L using only its left and upper neighbors, making the data dependencies purely local. This is repeated on the smaller matrix till completion.

Both of the systolic implementations referred to above use non-MAC PEs, and a triangular (not rectangular) network — this is very uncommon in current hardware, but was represented in the Kung references above.

For this post, I wanted to stick to high-level intuitive descriptions, but in the open-source TinyXPU project, we will aim to implement and analyze some of these non-traditional systolic networks for robotics and AI pipelines. Stay tuned for the upcoming post introducing this project!

Subscribe now

Conclusion

Systolic arrays were both invented before people think, and are more general than people think. They can have deterministic (no cache misses) high throughput and energy efficiency for algorithms which can work on local data. However, they are bad for working with sparse data (e.g. for sparse linear system solving), and bad for algorithms that need global data (e.g. Householder QR, which needs to operate on a full matrix column at a time).

In the deep neural network boom, the MAC array is so dominant in workload (>95% of operations in any DNN) that the non-MAC compute takes a tiny fraction of time. Dedicating a full systolic array with n² PEs to non-MAC operations would be area-inefficient for neural net workloads. This is why commercial vendors have not explored the co-design of systolic networks with algorithms, including PEs that can do MAC but also other functions like Givens rotations on one chip. For robotics workloads and other general scientific methods, the mix of primitives is different and (in my opinion) worth revisiting.

Further reading

• Kung 1982: Why Systolic Architectures - Great high-level overview of the motivation beyond systolic architectures

• Kung 1982: VLSA Array Processors - Further detail on applications such as QR decomposition

• Google TPU v1 paper

Related posts:

Chip Insights

Mapping algorithms to custom silicon - Part 1

Read more

4 months ago · 22 likes · Bharath Suresh and Avik De

1. The architecture behind “end-to-end” robotics pipelines

2. Online motor adaptation

3. This article

4. Closing the action loop with a VLM “agent”

5. Demo combining the best features of end-to-end and classical approaches

In this part, we get hands-on and build a VLA pipeline from scratch. I’ll be transparent about my starting point: while I have experience with model-based methods, RL controllers, and LLMs/VLMs, generalist end-to-end policies — almost exclusively being realized today as Vision-Language-Action (VLA) models — were new territory. That makes this post a useful vantage point to evaluate their strengths and weaknesses from first principles, and should be interesting to those who have never heard of VLAs as well as those who use them daily.

“Pick up the red block”

The demo is simple: take a specified prompt (like the one in the heading above), run it through the model, and visualize the actions that the model outputs. Obviously, when it is run in closed loop, you would get motion that hopefully results in the action described by the prompt, but there was so much to dig into with just this visualization that it made sense to spend an article on it. In the next part, we will close the action loop and explore some of the low-level controller facets mentioned in part 1.

In the following animation, the configuration of the arm is changed using the sliders (while being given the same prompt), showing that the output action is responsive to the robot and environment state.

The learning journey for this article is captured in a Jupyter notebook that can be accessed and run for free on colab — click here. All details on the software stack are in the open-source github repository for this project (which is where the notebook file also is). If it is a helpful learning tool or template, I’d welcome any feedback, fixes, contributions, stars, forks, etc.

First, let’s quickly go over what a VLA is.

Anatomy of a Vision-Language-Action (VLA) model

A Vision-Language-Action model has three functional components: a vision encoder, a language encoder, and an action head. In practice, the vision and language encoders are almost always a single pretrained VLM, i.e. the vision and language processing are already jointly trained before the action head is added. This means the “vision encoder” and “language encoder” aren’t independently tunable modules; they’re entangled by pretraining.

The architecturally interesting variation is in how the action head attaches to the VLM, and how much of the VLM is modified during robot training. This single design choice has large downstream consequences for what you can and cannot inspect at inference time.

VLA “action head” architectures

Two illuminating (but not exhaustive) designs:

Octo uses a dedicated readout token — a learned embedding (~384-dim) that aggregates action-relevant information from the transformer before a small decode network produces actions. This bottleneck is the closest thing to an inspectable interface in any current VLA: you can probe whether the readout encodes directional intent, object identity, or nothing interpretable.

Transformer → readout_action embedding (384-dim)
                        ↓
            Action Head (direct decode)
                        ↓
                    Actions

X-VLA processes images, language, proprioception, and noisy action candidates together in a single 24-layer transformer, conditioned by 32 learnable soft prompt tokens selected per embodiment. Flow matching then iteratively refines the action chunk over 10 steps. Action-relevant information is distributed across all layers and token types simultaneously.

Input: Images + Language + Proprio + Domain ID
               ↓
┌───────────────────────────────────────────┐
│  Soft Prompt Selection (per embodiment)   │
│  Domain 0 → Prompt_0 (32 learnable tokens)│
│  Domain 1 → Prompt_1 (32 learnable tokens)│
│  Domain N → Prompt_N (32 learnable tokens)│
└──────────────┬────────────────────────────┘
               ↓
┌──────────────────────────────────────────┐
│  Unified Transformer Stack (24 layers)   │
│  ┌────────────────────────────────────┐  │
│  │ [Soft Prompt | Vision | Lang |     │  │
│  │  Proprio | Noisy Actions]          │  │
│  │                                    │  │
│  │  All processed together with       │  │
│  │  standard self-attention           │  │
│  └────────────┬───────────────────────┘  │
└───────────────┼──────────────────────────┘
                ↓
       Flow Matching (10 steps)
                ↓
       Action Chunk (32 actions)

The soft prompts enable efficient cross-embodiment adaptation: only ~9M parameters (1% of the model) need updating for a new robot. But they also mean embodiment-specific behavior is encoded in vectors with no interpretable structure.

The deeper point applies to both architectures: even where a vector interface exists between components (Octo’s readout token, X-VLA’s soft prompts), end-to-end training means those vectors don’t have a physical interpretation that safety constraints can be applied to.

There is more to be said on action chunking and control bandwidth, which I’ll plan to do in the next part of the series.

Subscribe now

Model choice

I approached this as a user rather than a researcher: published weights only, no training data collection, and no fine-tuning iterations before deployment. The target application is tabletop pick-and-place on a WidowX, which is a common manipulation benchmark and exposes the control and perception properties I care about. Another soft constraint was that I’d be able to run it on my personal laptop (12GB VRAM).

These three criteria limit which VLAs can be tried. OpenVLA-7B requires task-specific fine-tuning and won’t fit in 12GB without quantization. π0 needs 24GB+. GR00T requires a Jetson Thor. Gemini Robotics On-Device is trained on dual-arm configurations and isn’t publicly accessible. Octo (93M params) fits the hardware but its pretraining doesn’t support zero-shot transfer without fine-tuning. SmolVLA likewise requires fine-tuning.

X-VLA seems to fit the bill. Its soft-prompt architecture was designed for cross-embodiment zero-shot transfer, and xvla-widowx provides a checkpoint fine-tuned on BridgeData for the WidowX embodiment specifically, meaning embodiment adaptation is handled, while task generalization remains zero-shot. It also has a ee6d (end-effector coordinates) action output mode, which appealed to me because it would allow elimination of kinematics-related variability.

What’s different about VLAs: task programming

VLAs have been heralded as revolutionary for robotics, and it’s true: the prospect of robot programming with natural language is a decided shift. Thinking about my own fielded robotics experience at Ghost Robotics, either customers would (a) directly command the robot, (b) pick between preprogrammed tasks (which can be considered a fixed small vocabulary of commands), or the robot would start its own tasks. Giving natural language commands increases the set of tasks without retraining or reprogramming. The natural language interface changes who can program a robot, not just what it can do. With a VLA, a non-technical operator can in principle specify novel tasks.

The flip side worth mentioning fairly: natural language as an interface trades a small precise vocabulary (preprogrammed tasks) for a large ambiguous one. “Pick up the red block” sounds more expressive than running the “pick_red” preprogrammed task, but as the next section will show, the boundary of what the model actually understands is opaque in a way that a fixed command vocabulary is not.

What’s different about VLAs: calibration and debugging

With classical methods, the process of setting up and debugging a task includes several well-delineated steps:

• calibrate cameras → check camera detection overlay → perception ✅

• calibrate joints → send arm “move up” command and ensure it moves as expected → actuators ✅

With VLAs, there are a few reasons why this kind of unit testing or debugging is simply not possible.

1. Camera extrinsics or joint torque constant parameters will not be isolated: datasets are typically trained with multiple camera angles without explicit calibration, and network learns spatial transforms end-to-end.1 Another example: swapping the camera lens for a fisheye for a wider FOV won’t generalize without retraining, unlike traditional vision where you just recalibrate intrinsics.

2. There aren’t obvious equivalents of non-end-to-end interfaces such as the camera detection overlay or a “move up” command, but we will try to come up with methods to work around this in the next section.

3. Randomness: the trajectory will vary with no environmental change. Flow matching stochasticity is in the action head specifically; the VLM backbone is deterministic given the same input. X-VLA uses 10-step flow matching. Even with same seed, numerical precision in GPU ops causes drift by step 5-6

Due to a combination of 1 and 2 (and slightly exacerbated by 3), it can be complex to reason about the root cause of a failure. Is failure due to (a) vision error, (b) action discretization error, (c) world model mismatch, or (d) all three? When can you dismiss a failure as being out of distribution vs. not?

For a developer, this ambiguity is an inconvenience; with enough time, you can run more experiments and form hypotheses (as we do in the next section). For a deployed system in customer hands, the same ambiguity becomes a safety property: the robot has no reliable mechanism to detect that it is out of distribution and should stop. Classical systems fail loudly (joint limit hit, object not detected, planner infeasible); VLAs fail silently, producing plausible-looking but wrong trajectories. This isn’t a criticism of VLAs specifically, but it is a structural consequence of end-to-end training, and it applies equally to any system where the failure boundary is defined implicitly by a training distribution rather than explicitly by an engineer.

VLA debugging ideas and techniques

Despite the structural challenges mentioned above, I had a fascinating experience coming up with ways to probe and understand what the VLA was doing.

Passive debugging: inspect what the model is already computing

1. Interpret VLM output (infeasible). My first instinct was to query the VLM backbone directly, e.g. by asking something like “Is there a red cube?” or “What objects are on the table?” to verify perception. This turns out not to be feasible for most architectures. In X-VLA and SmolVLA, the action head attaches to the VLM’s final hidden states and generates actions through flow matching in a continuous space, bypassing the text vocabulary entirely. You could query the underlying base VLM (e.g. SmolVLM2 for SmolVLA) separately, but that’s not a fair proxy: fine-tuning on robot manipulation data shifts the VLM’s internal representations, so its text generation behavior no longer reflects what the VLA backbone actually sees. This technique only works cleanly in text-token VLAs like VLA-0-Smol, where actions are generated as autoregressive text strings from the same output head as language. There, scene description quality and action quality share a representation and if the model produces a poor scene description, it will likely produce poor action tokens.

2. Visualize attention on tokens. The ubiquity of transformer-based architectures means that we can leverage the HuggingFace transformer’s output_attentions feature to try to visualize where the vision and text encoders are spending their attention, and if it is appropriate for the task specified. E.g. if we ask it to pick up a red block, is the vision encoder indeed looking at the red block?

Active debugging: intervene on inputs and observe behavioral change

1. Camera ablations (test whether vision is doing object detection or spatial template matching). Move the camera position, and introduce occlusions into one of the views if there are multiple. If attentions fail to track the desired object, it suggests the model learned spatial heuristics tied to camera geometry rather than object identity. In a classical pipeline, object detection is camera-pose-invariant by design (you’d re-project into robot frame), but here, camera pose is baked into the learned policy implicitly through the training distribution.

2. Counterfactual prompting to test semantic understanding. Use variations of the prompt (e.g. red block vs. red cube) that effectively mean the same thing and observe if the output stays consistent. Different outputs exposes that the action head is sensitive to tokenization differences that the VLM alone would smooth over. Also,

3. Primitive action prompts (tests action head’s semantic understanding of motion). E.g. if “don’t move” produces as much motion as “pick up block”, it shows that the action head is always generating motion from its training distribution, v.s. containing a deeper understanding of what motion is.

I suspect that some (if not all) of these will be familiar to seasoned VLA users, but please let me know in the comments if you’re aware of a better technique — chances are that it will many prospective and current VLA users!

Leave a comment

Debugging results

For each experiment, I’ll write what a reasonable expectation would be, the result we see, and the resulting insight or the deeper reason why.

Baseline: pick up the red block

In this baseline, the attention mask on the image looks like it is looking at the red block as well as the gripper. The reaching trajectory output looks like it moves to directly over the red block. Overall, this looks to be a great initial result.

Experiment 1: Picking a different block in view

Expectation: Symmetric action based on spatial understanding from multiple views

Result: The visualized attention shows that it is looking at approximately the correct part of the primary image, though it appears a little offset to the outside of the block. The reaching action appears to not reach as far toward the blue block.

Insight: Most likely, the “3D” spatial understanding from the images is not exactly what we would expect from an exactly calibrated perception and object identification setup.

Experiment 2: Swap blue / red positions

Expectation: Symmetric behavior from previous experiment.

Result: Blue trajectory overshoots more compared to initial red block trajectory, and red trajectory overshoots.

Insight: Spatial understanding and behavior is not symmetric when it is expected to be, indicating a bigger effect of things like training data distribution.

Experiment 3: Altered primary camera view

Expectation: Same behavior as the initial camera view.

Result: The red block trajectory now exhibits the under-reaching from the blue trajectory before, and vice versa.

Insight: The actions are inseparably tied to the camera view and not associated with absolute spatial understanding.

Experiment 4: Remove second camera view

Expectation: Slight degradation in performance.

Result: Removing the side view has minimal effect, but removing the over-the-shoulder view has a disastrous effect on performance.

Insight: It appears that BridgeData has a disproportionately high number of trials with the over-the-shoulder view and significantly altered view points may silently produce much worse results.

Experiment 5: Occluded primary view

Expectation: Second view provides redundancy.

Result: Trajectory moves away from the red block.

Insight: The side camera view seems to not be useful in X-VLA.

Experiment 6: Prompt variations

Expectation: Similar-meaning prompts will produce similar actions.

Result: All these similar prompts largely resulted in similar actions.

Insight: The language encoder is effective at collapsing equivalent prompts to the same actions.

Experiment 7: Don’t move

Expectation: No motion.

Result: Approximately as much motion as when asked to pick the red cube with the left shoulder view.

Insight: the model is still interpolating / extrapolating from training samples and does not have an explicit understanding of commands.

Experiment 8: Change picking position

Expectation: The output trajectory moves to the modified block position.

Result: The visual attention is strangely not on the block in the second example, but largely, the trajectory is responsive to the environment change.

Insight: As long as the prompt is visually grounded, the results generalize in the expected way. Soft prompt for WidowX likely encodes “approach visible object” as primitive (trained on Bridge dataset).

Experiment 9: Move forward / backward / up / down

Expectation: Move as asked.

Result: Approximately the same motion toward the tabletop, largely uncorrelated with the prompt.

Insight: No visual grounding for blind motions. The model has no spatial primitive vocabulary because VLMs are trained on image-caption pairs where “up” describes scene composition, not robot workspace direction.

Experiment 10: Move toward / away from base

Expectation: Move as instructed.

Result: Discernible difference in the two trials accordingly, suggesting some comprehension of the prompt.

Insight: The introduction of the robot base as a (visible) target makes things significantly easier for the model compared to the previous experiment.

Experiment 11: Move away from block

Expectation: Motion away from the block.

Result: Motion largely toward the tabletop.

Insight: The word “away” is probably not having the spatial effect that it should in this context, exposing the ambiguity inherent in using language for robot programming. Whether we like it or not, at least unless the language model is huge, it is safer to assume that the prompt effectively indexes or extrapolates among training data, and that positional prepositions (commonly used by humans to communicate spatial commands) are not reliable to use.

What the experiments reveal about current VLAs

1. Camera view is tied to the behavior, not a calibrated parameter.
Experiments 2, 3, 4, and 5 collectively show that the model’s spatial behavior is tied to the training distribution’s camera geometry rather than to a camera-pose-invariant object representation. Swapping shoulders changes reach distance; replacing the over-shoulder view with a side view breaks the policy entirely even though the scene is identical. This is a consequence of any VLA trained end-to-end without explicit camera calibration. The practical implication is that deployment requires camera placement matching the training distribution, and the model will fail silently when out of distribution.

2. The action manifold is object-centric, not spatially general.
Experiments 7, 9, 10, and 11 collectively show that the model has no spatial primitive vocabulary independent of objects. “Move up/forward/back” all produce similar grasping-like motions; “don’t move” produces motion; “move away from block” produces motion toward the block. “Move toward/away from base” works only because the base is a visually grounded object in the scene. This generalizes beyond X-VLA: any VLA at this scale trained predominantly on pick-and-place demonstrations will have an action manifold that approximates “move toward salient object and grasp.” Spatial relation commands only work when they can be reduced to object identity. This has a direct safety implication: you cannot issue a recovery command (”stop,” “move away,” “back off”) and expect it to override the trained behavioral prior.

3. VLAs at this scale appear to lack compositional generalization.
Experiments 7, 9, and 11 show that novel combinations of spatial primitives and objects (even using vocabulary the model demonstrably knows) produce behavior dominated by the training distribution rather than the instruction. This is distinct from the question of whether larger VLAs generalize better, which is likely true, but out of scope for this article. But it does suggest that for sub-1B parameter VLAs, natural language commands are most reliable when they closely match the task distribution the model was trained on, which significantly narrows the practical definition of "zero-shot generalization" for deployment.

Closing thoughts

For flow-matching VLAs like X-VLA, the classical debugging question “is this a vision problem or a control problem?” is not just difficult to answer but structurally unanswerable. End-to-end training eliminates the interfaces that would make the question meaningful.

The debugging ideas presented here offer partial remedies: passive inspection via attention visualization and active intervention via camera ablations and language variation. These experiments also surfaced three concrete findings: spatial understanding is tied to training-distribution camera geometry rather than calibrated object pose; the action manifold is object-centric and lacks spatial primitive vocabulary; and compositional generalization breaks down for novel combinations of known concepts. These are echoes of the reliability concerns of consistency, robustness, predictability, and safety that are crucially important to evaluate robotics progress.

None of this diminishes what VLAs actually deliver — flexible task programming and meaningful robustness to environmental variation, without any robot-specific programming. The path to reliable deployment is augmenting the strengths of VLAs with explicit interfaces for safety constraints, reducing complexity by utilizing known tools for camera and kinematics calibration, and out-of-distribution detection.

In the next part, we will close the loop with this demo’s action outputs to try and leverage the strengths of VLAs in conjunction with low-level control ideas from parts 1 and 2.

If you liked this kind of analysis, please subscribe for future posts, and thanks for reading!

Wiener was a prodigious child, receiving a PhD by age 18 from Harvard, and becoming MIT mathematics faculty. By the account of Dark Hero of the Information Age, the biography by Flo Conway and Jim Siegelman, he was simultaneously one of the most intellectually alive and emotionally turbulent figures in twentieth-century science: touched by manic-depressive episodes and collegial feuds, yet capable of a mathematical breadth that few of his contemporaries could match.

That breadth is visible in the book he published in 1948: Cybernetics, or Control and Communication in the Animal and the Machine. Its thesis was that information flow and message-passing are central to control and communication in both animals and machines. It appeared the same year as Shannon’s “A Mathematical Theory of Communication” and the year before Shockley’s transistor paper. Wiener was at the center of the founding of the information age, and yet he has been largely forgotten in the recent technological development. His legacy was overshadowed by Shannon, who had the more implementable theory, and by von Neumann, who had the more implementable architecture.

Reading Cybernetics now, almost 80 years later, is awe-inspiring and unsettling in equal measure. It is mathematically dense in places and dated in others, but the program it laid out is strikingly relevant to modern AI development. Here are the ideas from it that I found most resonant.

1. Feedback

“Cybernetics” originates from the Ancient Greek κυβερνήτης (kybernētēs), meaning “steersman” — the same root that, via Latin, gave us the word “governor.” It is perhaps not coincidence that Maxwell’s paper on governors was the first known exposition on feedback control. I don’t need to elaborate on the value of feedback in modern technology, but two nontrivial leaps Wiener makes are worth highlighting.

First, he draws a connection between communication and control in neurology. The feedback loop (sense the error, apply a correction, repeat) describes voluntary movement in biological systems. When this feedback is damaged, as in cerebellar injury, the result is a tremor or oscillation: too aggressive a correction followed by too aggressive a counter-correction. This convergence of engineering control theory and neurology was a founding observation of cybernetics: the same mathematics governs servomechanisms and nervous systems.

The second leap is the identification of a fundamental tradeoff: do you invest in modeling or in feedback? Wiener’s answer depends on how constant and knowable your system is. He called systems that leverage explicit models compensators, contrasting them with pure feedback mechanisms. In today’s terms, Wiener’s compensator needs a world model: an internal representation of how the system behaves that allows action without waiting for error to accumulate. The model vs. feedback tradeoff he identified has strong echoes of the one playing out now in the debate between scaling-based and structured AI architectures, not to mention in robotics. Model-free reinforcement learning is a direct descendant of the feedback side of this tradeoff: an agent interacts with an environment, receives a reward signal reflecting the gap between its behavior and a desired outcome, and adjusts its policy accordingly.

2. Neuron structure: digital vs. analog

Wiener asks in the book: in what ways are the computational substrates of brains and machines alike, and in what ways are they fundamentally different?

Wiener’s first observation is that neurons obey an “all-or-none” law (they fire fully or not at all) and in this sense are digital. This is in tension with von Neumann’s later analysis, covered in a prior post: von Neumann argued that individual neurons function more like small analog computers, with temporal dynamics and nonlinear integration beyond what a simple threshold element can do. The understanding of neuronal computation has deepened considerably since both accounts, and the honest answer is that neurons are neither purely one nor the other.

What followed from the digital view, at least in the engineering tradition, was eventually deep learning: stack enough simple threshold units to sufficient depth, and powerful computation emerges. But as the next sections argue, Wiener himself was skeptical that the generic stacking of simple units was sufficient.

3. Neuron organization: flexible vs. dedicated

What Wiener does not dispute is that even if neurons are digital in their firing, their organization is anything but generic. He writes:

The structure of our visual cortex is too highly organized, too specific, to lead us to suppose that it operates by what is after all a highly generalized mechanism.

As a mathematician, he frames this in terms of group theory: the visual system is built to be invariant under transformations of position, rotation, scale, and illumination. Image recognition is comparison at the level of structural properties that persist across transformations, and not comparison of photoreceptor signals. The retina has broadly distributed and low-resolution rod cells and foveally-concentrated cones, and layers beyond it extracting features at multiple spatial frequencies in parallel. Structure encoded by biology is doing work from the very first stage.

The unifying point is that the brain does not apply a general-purpose function to raw sensory data and let structure emerge. It applies a pipeline in which each stage is specifically organized to extract the right kind of information. Most modern vision models posit that this structure will emerge from scale and data; capsule networks, group-equivariant CNNs etc. attempt to encode it explicitly but remain outside the mainstream. This is the same tension at the heart of the world models debate: whether sufficient scale applied to a general architecture will recover the structure that biology built in deliberately, or whether that structure ought to be encoded.

4. The switchboard analogy

Wiener is next interested in how neurons are organized, and here his analysis diverges sharply from the digital computer model he was comparing against.

A digital computer of his era had specific circuits for specific operations: an adder, a multiplier, a comparator, each doing one thing reliably and repeatedly. The brain, he argues, does not work this way. Rather than dedicated permanent circuits, the brain reconfigures its functional connections dynamically, routing signals through different pathways depending on context. He uses the telephone switchboard as his analogy: the same physical wires serve different conversations depending on how the exchange is configured at any moment.

He makes this concrete with a visual processing example depicted above: recognizing a letter regardless of its size (a large “A” and a small “A”) with a fixed array of photocells. His proposed solution uses a switchable connection layer between the photocell array (bottom) and a fixed set of processing elements (top). By selecting different connection patterns (the diagonal lines), photocell activations at different scales get mapped onto the same processing elements, achieving scale invariance through reconfigurable routing rather than through a learned function. In deep learning perception, this is similar to ideas like spatial pyramid pooling or adaptive pooling.

In contrast, a vision transformer applies the same operation at every layer to every token, with flexibility coming entirely from learned weights at massive scale. There is no dynamic routing or reconfiguration based on the nature of the input. Wiener pointed out that this approach carries a cost: a large fixed architecture must run in its entirety even when most of it is irrelevant to the current input. A 175B parameter model processing a simple query still activates the full machinery, paying the energy and latency cost of elements that contribute nothing to that particular computation.

Some modern work moves toward Wiener’s direction. Mixture-of-experts architectures route inputs to specialized sub-networks rather than running everything; sparse transformers use dynamic attention patterns; early-exit networks use only as much compute as the input requires. These remain the exception rather than the rule, but they are each, in a real sense, implementations of the switchboard principle Wiener described in 1948.

Subscribe now

5. Spatial efficiency: foveation

A thread running through Wiener’s treatment of vision is that the brain achieves capable perception not by processing everything uniformly and in parallel, but by being strategically non-uniform in both space and time.

The spatial side is foveation. The fovea provides high-resolution detail while the periphery offers broad, low-resolution motion detection. The brain doesn’t passively receive a full image, it actively steers the fovea toward informative regions via saccades, driven by a continuous feedback loop. The implication is that high-resolution processing is a scarce resource allocated dynamically, not applied uniformly.

6. Temporal efficiency: the television analogy

The temporal side is more surprising. Wiener observes that the brain may serialize what would otherwise require parallel hardware, using alpha waves (the ~10 Hz electrical rhythms visible in EEGs) as a scanning clock. Just as a television converts a two-dimensional image into a sequential stream by sweeping line by line, the brain may sweep through its representational space cyclically, interrogating stored patterns at each clock cycle. The efficiency principle is time-multiplexing: reuse the same hardware over time rather than duplicate it in space.

Together these describe a coherent alternative to the architecture modern AI has converged on. Transformers process all positions in a spatially uniform and temporally instantaneous manner, which is expensive in both compute and energy. Biology does neither: it allocates spatial resolution selectively and serializes computation over time. Foveation-inspired architectures (glimpse networks, recurrent attention models) and ideas like conditional computation point in this direction but remain outside the mainstream, largely because uniform dense operations map cleanly onto GPU hardware. Wiener’s architectural intuitions may become increasingly relevant if the AI energy crisis makes the efficiency argument more economically compelling.

7. Avoiding blunders: redundancy and verification

The brain produces behavior of remarkable precision despite individual neurons being surprisingly unreliable: they fire spontaneously, transmit probabilistically, and have far worse signal-to-noise ratios than transistors. Wiener’s answer, developed in the psychopathology chapter, is that there are two complementary strategies for error correction. The first is the “what I tell you three times is true” strategy: running two or three computing mechanisms simultaneously on the same problem, so that errors can be recognized by agreement across parallel channels. The second is backtracking: sequential verification where the system checks its own output and revises when something goes wrong. One is spatial (parallel redundancy), the other is temporal (serial correction) — the same tradeoff from before, now applied to reliability rather than perception.

This maps directly onto one of the most discussed failure modes in LLMs: hallucinations. Wiener suggests that they are the expected behavior of a system optimized for speed without redundancy or verification, not simply a quirk to be patched. A single forward pass through a transformer produces an answer with no mechanism for catching its own errors. Reasoning models which iterate, self-check, and backtrack are exploiting exactly the reliability/overhead tradeoff Wiener described:

But verification has limits. As I argued in the world models post, a system that lacks a grounded semantic model of the world can cross-check its outputs without ever catching the deeper class of errors that stem from not understanding what it’s talking about.

The human use of human beings

Wiener was clearly one of the founders of the information age, but he was also deeply worried about what was being built. A passage from his follow-up book The Human Use of Human Beings reads like something written last week:

The first industrial revolution was the devaluation of the human arm by the competition of machinery. The modern industrial revolution is similarly bound to devalue the human brain, at least in its simpler and more routine decisions. The average human being of mediocre attainments or less has nothing to sell that it is worth anyone’s money to buy.

He was not predicting this as an inevitable law of nature. His proposed answer was equally striking: rather than trying to preserve the market value of human labor artificially, he argued that society would need to restructure itself around non-market values like dignity, community, creativity, meaning. He wrote letters to labor unions warning them of what was coming, but he was not listened to.

In 2026, AI systems are starting to now inexpensively perform many of the cognitive tasks (writing, coding, analysis, translation, legal research) that defined middle-class professional employment in the twentieth century. The policy infrastructure to manage this transition does not exist. The urgency Wiener felt in 1950, when he had no working computer to point to, is more justified now.

Brian Christian, in his introduction to the recent reissue, calls Wiener “the progenitor of contemporary AI safety discourse.” That may be the most accurate short description of the man. He was not a pessimist or a technophobe — he was a technologist who had thought seriously about what he was building and felt obligated to say what it implied. That combination of technical depth, ethical seriousness, and willingness to deliver uncomfortable conclusions publicly is just one more reason to read and remember him.

I’m a strong proponent of reading and non-echo-chamber thinking. If you know of any other writing of this ilk, please let me know in the comments. If you liked this post, please share it, and subscribe!

Leave a comment

Share

This post draws primarily on “Cybernetics: Or Control and Communication in the Animal and the Machine” and the biography “Dark Hero of the Information Age”. It continues themes from previous posts on von Neumann and world models in AI.

When battery life is crucial, and when microseconds matter to ensure control loops remain stable, the hardware doesn’t care about elegant class hierarchy or beautiful code. The end-product of programming is data transformation, and not the code itself.

This post, written with my friend (software engineer and CS lecturer), started as a simple teaching example about Array-of-Structures vs Structure-of-Arrays (AoS vs. SoA) layouts. We thought we’d show a clean and universal performance curve demonstrating cache effects tied to C++ code.

We built what seemed like a straightforward benchmark: measure access time for different memory strides. What we didn’t anticipate was encountering five distinct issues spanning multiple abstraction layers—from compiler behavior to microarchitecture to hardware characteristics:

1. The compiler deleted our measurement code and unpredictably stored variables in memory vs. registers

2. The CPU’s pipeline hazards dominated our memory access time

3. The CPU’s dynamic frequency scaling skewed our results

4. The hardware prefetcher made our predictions wrong

5. Different processors gave wildly different results

This illustrates the gap between abstraction and performance. Programming languages provide abstraction above the hardware, but achieving good performance requires understanding how code executes on the underlying architecture. While some of our issues may be familiar to experienced programmers, others might be surprising even to veterans.

Four examples showing why you should care

There are many real-world examples using an "array of structures" organization for good reasons: it's faster to prototype, easier to reason about when objects manage their own state, and typically more readable for developers.

Example 1: PCL (Point Cloud Library) PointXYZRGB structure.

When you have pcl::PointCloud<PointXYZRGB> with millions of points, the memory layout looks like

[x0, y0, z0, pad, rgb0, x1, y1, z1, pad, rgb1, ...]

For an example task of filtering by distance (operating on the xyz only), we get 40% extra cache misses. For a color segmentation task operating on rgb only, 4x extra cache misses.

Example 2: Unity GameObject-Component System. GameObjects directly contain Component instances by value, e.g. a GameObject with Transform, Rigidbody, and Collider components stored as member data. This is classic AoS: each GameObject owns its component data, providing flexible composition but poor cache locality when iterating over many objects.

Example 3: Box2D (version 2.x). Each b2Body contains position, velocity, and force data as members (e.g. b2Vec2 m_linearVelocity). Most traditional object-oriented game engines before the ECS trend used composition with value semantics—each enemy/player/NPC object contained all its data directly. However, Box2D v3.0 (2024) moved away from this, now using handle-based IDs and storing body data separately for better performance.

Example 4: Humanoid joints. Last but not least, here is a practical example of a humanoid robot joint that should be quite relatable:

A Limb would be composed of Joints, with Joint specialized into different joint types.

Suppose the humanoid robot has 50 joints. Computing Jacobians requires accessing each joint’s position: 5.5kB loaded into cache (87 cache misses), when we only need 200 bytes (4 cache misses if organized as an array of positions).

Now that we have shown that this organization occurs commonly, we will dig in and try to measure the effect it has.

An even simpler example to dig into

We created an even simpler example with a single data array and a parameterized “stride” for a strided access pattern. This would occur in the example above with stride = sizeof(Joint). Our goal was to time how long it takes to access a fixed number of elements with different strides, as in the code below.

The actual code for replicating all these measurements, and more, is on github: feel free to try it out.

What we expect to see: Effectively, as we access data, the processor can load a segment from main memory into cache, in blocks.

Data:    |x| | | | |x| |...
          ← stride →
Cache:   |y|y|y|y|y|y|y|y|z|z|...
          ← line size  →

As stride increases, visiting the same number of elements requires caching more blocks. If memory movement dominates, we expect a linear rise in time as stride increases and more cache lines are touched. (More on what happens after each access hits a separate cache line below.)

Understanding the results from this "simple" example felt like peeling endless layers of an onion, but was very gratifying at the same time!

Issue 1: Controlling compiler optimizations

With the code snipped above, the assembler output showed:

testStride(unsigned long):
        ret
data:
        .zero   256000000

Of course! sink was being optimized out, and my firmware programming background caused me to add a volatile to its declaration. However, something in the asm output for the loop looked amiss. Can you spot it?

While data should be loaded from memory to a register, sink should be able to remain in a register. However, volatile forces it to be loaded and stored because the compiler must assume that it can be modified externally. So we get rid of volatile, and uncomment the last line:

if (sink == -1.0f) std::cout << "";

The new loop looks like

Comparing to the assembly above, the extra load-store are gone - first mystery solved.

Issue source: compiler / programming language

Issue 2: Data dependency hazard

The relevant part of the loop looked like this:

Timing this loop as we varied stride showed that for the first few strides, increasing stride had no effect on the time (solid lines in the plot below). With an Apple M2:

With the size of our loop, increasing stride definitely means more cache lines are touched, but it is making no difference. What’s going on?

Let’s look back at the assembly (same as the previous snippet).

If we manually unroll a few iterations, we have the following pattern:

FP add 2 cannot issue until FP add 1 has been committed, a classic Read-After-Write hazard. While a chip designer understands this very well, a programmer rarely needs to understand data dependency hazards in CPU pipelining. In this example, the float add dominates the effects from the load/store due to the data dependency and the long latency of floating-point add.

We add an unrolled version:

The unrolled version time is significantly smaller, as visible in the dashed lines in the plot above, and more importantly, now we see the linear rise we had predicted.

Issue source: microarchitecture, not visible in assembly instructions

Issue 3: Warmup effects

After root-causing issue 2, to avoid dealing with the unrolled loop, we changed the accumulate to a Read-Modify-Write. The time for each iteration is now longer because a load and store are required for each iteration, which should make data movement costs the dominating factor.

A number of stateful microarchitectural effects unrelated to the data cache contribute to performance characteristics, yet produce data cache-like behaviors. Such factors may include page table caching, page walk caching, prefetcher training, the memory controller, and even frequency ramping.

We attempted to stabilize the effect of these factors before running trials by running a warmup function at the beginning of the program. The warmup simply iterates over every element of data once to have the cache in a predictable state.

The results:

The warmup appears to universally make the program faster, irrespective of the stride (more pronounced effect on a different system in plots below). Our best guess is that the warmup ramping up the CPU frequency is the dominant effect. We also considered a trial for one stride affecting another, but running a single stride per run of the program didn’t yield clearer results (and took much longer).

Again, if you have any better ideas, we would love to know - please leave a comment!

Leave a comment

Issue source: microarchitecture / hardware

Issue 4: Initial no-effect; second slope after cache line boundary

4a) In the previous plot, there is an initial part till about stride ~5 (20 bytes) where we predicted a linear rise, but instead see no effect of stride on timing.

While we are not sure, this is likely due to hardware prefetching: Modern CPUs have hardware prefetchers that detect sequential / strided access patterns and automatically fetch data ahead of time. Once the stride grows large enough (~20-64 bytes), the prefetcher can no longer keep up—either because it can’t fetch far enough ahead, or because the access pattern becomes too sparse for it to predict. At this point, we finally see the expected linear increase as each access genuinely waits for data from main memory.

4b) We expected the access time to plateau after each access was already hitting a different cache line. However, there appears to be a slower rise after the cache line boundary at least on the Apple M2 processor

Some (unconfirmed) hypotheses for the slower rise after the boundary:

• L1 → L2 spilling if the working set exceeds L1 capacity, incorporating L2 access times

• TLB misses as large strides access many different memory pages

Issue sources: microarchitecture / hardware

Issue 5: Different behavior on different processors

Throughout uncovering the previous issues, we ran a few tests on other processors, and unfortunately that only served to increase the number of unknowns. In this section we will show you some of those results, but only be able to speculate about what causes them.

With an AMD Zen5 processor:

We see a plateau after an initial rise, which matches our naive prediction.

However, we observe a peak around 32 floats (128 bytes) followed by a drop. We don’t have an explanation for this behavior, which may be to do with advanced prefetcher behavior. In other words, the hardware may be making assumptions about our access pattern, and stride = 64-128 bytes hits the worst-case scenario where those assumptions fail. If you have any ideas about the cause, let us know in the comments!

Leave a comment

We also tested on an Intel processor on Windows, which confirmed that some of the strangest aspects of the two plots above are to do with AMD, and not the compiler.

This resembles our Apple M2 plots more closely, including the slower rise after the cache line boundary. It also adds an even slower rise after 2x the cache line boundary.

Issue sources: secret microarchitectural optimizations

Back to programming

Through this journey, it is safe to say we learned a lot about the complexity of modern processors. Fortunately, though, our central point about the (initial, then plateauing) rise of access time with stride still stands as universally true. Phew!

How do we utilize this knowledge as a programmer? The key is to ensure that commonly-accessed data is packed tightly in contiguous memory.

The naive OOP concept of owning data:

• The class directly contains/owns the data as member variables

• Example: Joint class with float sensed_position (and other things) embedded in it

• This creates the AoS memory layout problem

Instead store indices. In the literature on data-oriented design, this is sometimes called: Entity-Component-System (ECS) pattern, or data-oriented design with handles.

• The class contains references, pointers, or indices to data stored elsewhere

• This allows you to keep polymorphism while avoiding AoS layout issues

It isn’t object-oriented vs. polymorphism. Just to reiterate that data-oriented is not opposed to OOP conveniences, consider that Pinocchio uses polymorphism to specialize functions, but stores indices to the vectors, not the data itself. The actual positions and velocities live in contiguous arrays, giving cache-friendly SoA layout, while the polymorphic joint models provide the OOP interface. You can have the benefits of polymorphism (different joint types with specialized behavior) without the memory layout problems of AoS. This is the middle ground between pure OOP with composition and abandoning OOP entirely for data-oriented design.

Closing thoughts

In this post, we first showed how OOP-thinking can naturally lead to suboptimal cache usage, with several real examples. Then we looked at the effects this can have, uncovering many interesting “side-quest” root-causing exercises.

It isn’t coincidence that modern performance-critical systems say no to naive composed OOP:

• Machine learning libraries will often select the data layout (NCHW etc.) transparently, optimizing for cache locality.

• Pinocchio, a robotics kinematics / dynamics library, has its functions access array data.

• Drake, a larger robotics-oriented library, eventually has data in arrays below abstraction layers.

• Unity DOTS stores all Transform data in packed arrays, not in GameObjects.

• Box2D v3.0 switched from OOP bodies to ID-based handles with SoA storage.

• Unreal Mass Entity is an ECS system for high-object-count scenarios.

Even if in an isolated example the performance gain seems small, these patterns occur so frequently that they can add up to large losses that are difficult to eliminate.

Thanks for reading! If you enjoyed this kind of full-stack analysis and root-causing, please share and subscribe for more posts on robotics, AI, and computing.

Share

References and further reading

• Code for demonstrations in this post, and more on github

• “Better memory representation” in Jeff Dean’s “Performance Hints”

• “Efficiency with Algorithms, Performance with Data Structures” - Chandler Carruth [CppCon 2014]. Note: I don’t fully agree with the statement (10:45) that “efficiency is only affected by algorithms” - a good example is the energetic cost of moving a byte from DRAM -> core being significantly higher than from L1, meaning the same algorithm with poor cache performance actually consumes more energy, in addition to completing slower.

• Data-Oriented Design and C++ - Mike Acton [CppCon 2014]

• Explicit cache control via software prefetching

1. Architecture of end-to-end: learning → control

2. This article

3. Dissecting a VLA

4. Closing the action loop with a VLM “agent”

5. Demo combining the best features of end-to-end and classical approaches

Last week, I wrote about modern end-to-end robotics pipelines; why this is the new north star, and the hidden architecture behind successful implementations. Part 1 reviewed some implementations showing signs of a cascade of a high-level (HL) → low-level (LL) controller in the actuation end of the pipeline:

I have first-hand experience demonstrating walking robots to customers in a sandy desert, and as the robots slipped they asked, “is it learning?” With the prior of people adapting their gait as they walk on ice (for example), a reasonable expectation is that an isolated robot can adjust its behavior after some trial & error or adaptation period.

However, this is not how naive foundation-model end-to-end pipelines (such as those covered in part 1) work today; a particular robot can only change its behavior once the “hive brain” is updated with new data in its training. Due to the size of these models, it is impractical that training happens on-device or frequently.

So, in part 2, we ask: how can a fielded robot adapt to unexpected conditions? Why do we even need adaptability? Given the HL → LL controller cascade structure in modern end-to-end pipelines from part 1, where does this adaptability live, and how does it affect the mapping to computing hardware? Lastly, we will also look at some published implementations and see how they approach or ignore this issue.

Updates to part 1, hot off the presses

Before we dig into part 2, I need to add a couple of updates from relevant news releases that I wasn’t able to review before part 1 was published (Jan 26):

1. Microsoft’s Rho-Alpha model announcement with commentary on Tech talks (Jan 24) reveals “split architecture” including dedicated low-level controller, underscoring at least two points in the part 1 post. (a) Tactile and proprioceptive information is incorporated in the action expert, showing that the action head facilitates feedback loops; (b) higher control bandwidth via so-called bypass mechanism. Quoting the post, “The long-term goal, Kolobov said, is to have the action expert or a part of it operate on proprioception and physical sensing modalities at a significantly higher frequency than on visual and language data.”

2. Figure Helix 02 Jan 27 update reveals new “System 0” controller, underscoring at least four points in the part 1 post. The “system 0” implementation is described as a dedicated whole-body controller (WBC), which conventionally converts desired accelerations or velocities to joint torques based on a model of the robot. (a) S1 went from controlling the upper body to the whole body, and this reduced the overall system complexity by separating concerns; (b) S0 and S1 incorporate tactile data in tighter feedback loops, without adding complexity to the large VLM S2; (c) S0 runs at a KHz rate increasing the last-level control bandwidth; (d) it is trained for that specific robot (vs. cross-embodiment), localizing robot body-related parameters in one place (and presumably enabling generalization of S2/S1 to a different robot). The purpose of the WBC is similar to the model-based reference in part 1, but the difference here is that it is also a neural network trained from data.1

I expect that we will continue to see further evidence and refinement of hierarchical control structures in commercial robots, vs. unstructured end-to-end pipelines. Make sure to subscribe to get future updates:

This publication and this post contain the author’s personal thoughts and opinions only, and do not reflect the views of any companies or institutions.

Why do we need adaptability?

When a robot leaves the lab and is in customers’ hands, it will at some point inevitably be subjected to an unexpected operating condition, stemming from component failure, perturbation, environmental condition, or operating condition (e.g. payload). To address this, one recourse is to build a large-enough model that has enough experience to handle all these situations (i.e. domain randomization, multi-embodiment, etc.). This of course takes (much) more data and more training, as OpenAI showed from their dexterity result in 2019:

The other option is adaptation in a strategic part of the pipeline to address as many of these variations as possible. In this post, we are focused on the action end of the pipeline, and the classes of variation we are interested in include variability in joints / motors (friction, motor torque), terrain properties, payload.

Let’s clarify the timescale hierarchy, because the word “adaptation” can refer to changes at various timescales. Within-movement corrections can happen in milliseconds, and is typically part of reactive control within the low-level controller. Skill acquisition across many tasks using large datasets during training will typically happen offline. The intermediate adjustment occurring in the seconds-to-minutes timescale, which we refer to as motor adaptation, is the focus of this post.

Historical context from biology, control theory, and LLMs

Cerebellar timescales (seconds to minutes) match closely with the motor adaptation timescale referred to above, and several research efforts identify its role in adaptation of behavior in that time range.

Morton (2006) further associates the cerebellum with motor adaptation and the spinal column to reactive control:

Cerebellar damage does not impair the ability to make reactive feedback-driven motor adaptations, but significantly disrupts predictive feedforward motor adaptations during splitbelt treadmill locomotion … The cerebellum seems to play an essential role in predictive but not reactive locomotor adjustments. We postulate that reactive adjustments may instead be predominantly controlled by lower neural centers, such as the spinal cord or brainstem.

In control theory, there is a long tradition of adaptive control and model-reference adaptive control (MRAC) which utilize a (model-based) adjustment mechanism to modify the parameters of the controller.

The arrows in the figure above reveal a degree of interconnectedness beyond the cascade connections we primarily reviewed in part 1. The adjustment mechanism can also act in discrete steps instead of continuously, or without a model, in which cases it is called “gain scheduling”.

Self-improving learning systems are beginning to appear in the news more frequently in the LLM world: Ilya Sutskever said in Nov 2025, “There has been one big idea that everyone has been locked into, which is the self-improving AI”. The aforementioned Rho-Alpha model has an ability to update weights while running using teleoperation feedback. However, this can lead to a common side-effect called “catastrophic forgetting” due to all weights being in one huge monolithic structure, and so updates needed to be made either in judicious layers or in careful batches.

Motor adaptation in practice

One advantage in robotics pipelines is that they may (as we saw in part 1) have a hierarchical HL → LL structure. In such a situation, there are motor adaptations that can be integrated the LL controller without impacting the behavior of the HL controller, sidestepping the catastrophic forgetting issue.

I’ll go over a few illustrative examples, and especially discuss their ability to handle unexpected conditions. If I missed an idea that is pertinent and relevant, let me know in the comments:

Leave a comment

Adaptation in model-based LL control: robot arms, drones, humanoids

The pre-”end-to-end” era had many examples of adaptation in practice. Old ideas such as MRAC show up in industrial and commercial manipulators, such as in the payload estimation feature in Universal Robots arms. Commercial drones estimate wind to remain stable, sometimes using neural networks. In a 2019 HyQ paper, an explicit terrain compliance estimation module estimates parameters used by the LL controller. In a 2023 demonstration of the Atlas robot using model-based controllers while picking up heavy objects, Atlas “has access to the mass properties” of the object it is picking up, which I would lump into a gain-scheduling type of approach.

In all these examples, because the LL controller is model-based, it is easier to adapt for quantities like payload mass because it is clear where those terms appear in the controller. This is an advantage of having physically interpretable parameters, compared to black-box latent-space interconnections.

Mapping to computation:

HL → WBC inverse dynamics/QP (CPU) → Joint/servo controllers (microcontroller/CPU) → Torques
                ↑
    *Adjustment mechanism (CPU/GPU)*

Meta-learning for adapting among training environments

The concept of meta-learning (2019) is targeted at the motor adaptation problem, but needs samples over environments during training. This leads to the aforementioned prolonged training and large models, as well as susceptibility to truly unexpected (out-of-distribution) conditions. The authors of the paper are among the founders of Physical Intelligence, so it is possible that they could institute meta-learning-type methods for online adaptation in their action expert (not the case today as far as I can tell).

Mapping to computation of this hypothetical scenario:

VLM (GPU) → Action expert *with internal model and meta-learning* (GPU/CPU) → Trajectory tracking (CPU) → Torques

Learning-based latent parameter estimation for locomotion

As my sand locomotion example above might hint at, unexpected payload and terrain conditions are particularly prevalent in locomotion.

RMA: Rapid Motor Adaptation (2021) introduces a dedicated adaptation module that predicts a set of “latent parameters” that can adjust the action policy to better suit different conditions. These varied conditions are trained by randomizing in simulation, potentially suffering from a few of the same issues with out-of-distribution encounters and training difficulty.

One of the authors founded Skild.AI, and this quote from their Aug 2025 blog post

A striking aspect of our model is that it is not just robust, but it is also adaptive and graceful

(emphasis theirs) suggests incorporation of something like RMA. Absent too many details, here is my best guess of the composed pipeline mapped to computational hardware:

HL action policy (GPU) → *Adaptation module (GPU)* → LL action policy (GPU) → Torques

Where RMA had a large-ish latent vector, there are similar approaches toward predicting parameters with more physical meaning, from a reduced or a full state estimate. These state-estimation networks concurrently learn base state and contact probabilities alongside policy, enabling better perception of ground interactions.

The end-result pipeline is quite similar, just potentially decomposing the adaptation module a bit:

HL command → *History encoder (GPU) → Estimator (GPU)* → Actor (GPU) → Impedance control (CPU) → Torques

In-context learning to fix recent mistakes

A different method called in-context learning (appearing in Covariant.AI’s Mar 2024 blog post, and in the RICL method from Aug 2025) attends to recent action history as opposed to encoded observation history. These relevant demonstrations are added to the VLA context before its forward pass.

The end-result pipeline adds retrieval buffer of demonstrations before the VLA, and an interpolation unit after the action module:

*Retrieval buffer* → VLM (GPU) → Action expert (GPU/CPU) → *Action interpolation (CPU/GPU)* → Trajectory tracking (CPU) → Torques

This method is in a slightly different category, where relevant demonstrations need to occur and be reflected upon to adapt, compared to the potentially faster adaptation enabled by the previous methods. This strategy would not be sensible for time-sensitive or safety-critical tasks, but is categorically different and seemed worth reviewing.

Closing thoughts

In part 2 of this article series reviewing modern end-to-end robotics pipelines, we discussed why it may be useful to have some adaptation capability for fielded robots to handle unexpected conditions, and some examples of how it can be implemented. We also discussed some historical context from biology and control theory.

In part 3, we will try to get more hands-on and utilize what we learned from the first two parts to build up an effective pipeline from scratch. I’m still debating whether to use existing tools such as Isaac sim or build even more from first principles for clarity, so it may take some time before we get there. If you have any suggestions or feedback, let me know in the comments. If you found this article interesting, please share and subscribe for future posts. Thanks for reading!

Share

Leave a comment

1. This article

2. Online motor adaptation

3. Dissecting a VLA

4. Closing the action loop with a VLM “agent”

5. Demo combining the best features of end-to-end and classical approaches

Recent progress and excitement in humanoid robotics are largely driven by rapid gains in generalist capabilities. Historically, most robots were engineered for narrow, well-defined tasks. The current wave of companies, in contrast, is pursuing systems intended to operate across a broad range of activities, shifting both public and economic expectations toward robots that can serve as general-purpose physical agents.

A central part of this shift is the widespread claim of end-to-end pipelines, often described as going from “pixels to actions,” in contrast to earlier approaches built from hand-designed perception, planning, and control modules. This post examines what “end-to-end” means in practice: where the pipeline actually begins and ends, the tradeoffs between different architectural choices, and how the algorithms map to computing hardware.

Part 1 focuses on the “actions” side of “pixels to actions”: how learned systems interface with the physical control of the robot body. Part 2 will examine how these architectures adapt to environmental uncertainty and contact-rich interaction. Later parts will include hands-on comparisons using small standalone examples to make these differences concrete.

This publication and this post contain the author’s personal thoughts and opinions only, and do not reflect the views of any companies or institutions.

Why “end-to-end”

Classical AI was built up from a strict idea of separation of sensing, planning, and action. To my knowledge, the first robot to embody Sense-Plan-Act was Shakey the robot (~1970), which also employed one of the first symbolic AI systems. This tiered structure was so formative to robotics research that most research labs today are dedicated to different portions of this hierarchy, such as “perception”, “planning”, or “locomotion”.

The sense-plan-act view today is dying a very rapid death. The modern narrative of general-purpose robotics holds that modular pipelines often fail because of limitations imposed by this decoupling; for example, perception errors break planners, planners produce infeasible motions, and most importantly, interfaces encode wrong assumptions.

As influencial AI researcher Sergey Levine puts it,

for any learning-enabled system, any component that is not learned but instead designed by hand will eventually become the bottleneck to its performance

End-to-end training avoids hand-designed intermediate representations, manually tuned cost functions, and any bottlenecks imposed by module interfaces.

Additionally, “end-to-end” sends a sociological signal to do with modern AI foundation-model alignment, scalability with data, and positions the company as an AI lab instead of a controls shop.

The action end in practice

The practical reality of “end-to-end” is more subtle than it might seem. In this section we’ll review what some published academic and commercial implementations actually appear to be doing today, and also try to outline how the implementation is mapped to computational hardware.

The old way: model-based stacks (~2014)

It is very common to have a whole-body controller at the low-level, as exemplified by the 2014 MIT Atlas team’s report. After a high-level plan is created, a tracking controller is implemented as a quadratic program, and that generates the signals sent to the actuators:

Mapping to computational hardware:

Trajectory optimizer (CPU) → WBC inverse dynamics/QP (CPU) → Joint/servo controllers (microcontroller/CPU) → Torques

Learning followed by impedance controller (~2017-2020)

To my knowledge, the first fielded robots using learning-based locomotion controllers appeared ~2018 from Google (using Minitaur) and in Marco Hutter’s group. As documented in the 2018 paper from Google and the highly-cited Hwangbo et al (2019) paper, the most effective choice of action space was an impedance controller in turn influenced by Peng et al (2017):

Our experiments suggest that action parameterizations that include basic local feedback, such as PD target angles, MTU activations, or target velocities, can improve policy performance and learning speed across different motions and character morphologies

The policy outputs desired joint positions and sometimes velocity offsets or gain modulation, and the torque applied is a simple algebraic equation:

The virtue of this architecture is that it is very generic, and succeeds in decoupling the fast time-scales and discontinuities of making and breaking contact from the learning algorithm.

Mapping to computational hardware:

Policy eval (CPU/embedded GPU) → Impedance controller (CPU) → Actuators

Figure AI’s “System 1” policy (2025)

Figure AI’s Feb 2025 blog post describes a “System 2 / System 1” design where a high-level vision-language model (S2) reasons about goals and semantics at low frequency, and a fast visuomotor network (S1) executes continuous control at high frequency. While this reflects a separation of timescales and roles, both modules are trained end-to-end with an abstract latent interface, meaning there is not a principled, physically interpretable handoff between high-level strategy and low-level control. As a result, Helix achieves generalization in perception and task reasoning but does not isolate physical control concerns (such as dynamics stabilization, contact interaction, or actuation abstraction) into structured model-based or classical control modules.

In a Mar 2025 blog post, they describe what sounds more like the impedance controller above than the system 1 design, so it’s possible some combination of both architectures is utilized:

We additionally run the policy output through kHz-rate closed-loop torque control to compensate for errors in actuator modeling

Mapping to computational hardware:

System 2 (Transformer, GPU) → System 1 (Network, GPU) → [Impedance control (CPU)] → Torques

Physical Intelligence’s action expert (2025)

The architecture described is similar to the system 1 above, but specifically suggests that the end-to-end training causes problems:

When adapting a VLM to a VLA in this action expert design, the VLM backbone representations are exposed to the gradients from the action expert. Our experiments show that those gradients from the action expert lead to unfavorable learning dynamics, which not only results in much slower learning, but also causes the VLM backbone to lose some of the knowledge acquired during web-scale pre-training.

This is conceptually analogous to known problems like vanishing/exploding gradients in deep nets, where lower layers dominate or drown out meaningful gradients for higher layers.

Another blog post describes issues to do with the mismatched control bandwidth of foundation model output to robot dynamics, solved by outputting short horizon trajectories that are played out by a low-level controller.

Mapping to computation:

VLM (GPU) → Action expert (GPU/CPU) → Trajectory tracking (CPU) → Torques

Boston Dynamics + TRI’s pose tracking (2025)

Their blog post describes an architecture with the higher-level cognitive layer outputs joint positions and end-effector poses. While there isn’t an explicit decription of how these position setpoints are tracked, the post mentions Atlas’s MPC, and it is reasonable to assume that that is the lower-level controller.

Mapping to computational hardware:

LBM inference (GPU) → MPC (CPU) → Actuator torques

1X’s inverse dynamics model IDM (2026)

1X also describes a hierarchy in their blog post:

World Model Backbone (WM): A text-conditioned video prediction model trained on internet-scale video data and fine-tuned on robot sensorimotor data. It predicts future visual states based on current observations and candidate actions.

Inverse Dynamics Model (IDM): Converts predicted future states into feasible robot action sequences that will produce those outcomes in the real world. The use of the term “inverse dynamics” suggests that the output actions are torques, though that isn’t specified.

Mapping to computational hardware:

World model (GPU) → IDM (GPU) → Actuator torques

Why not end-to-end

From the previous section, it is apparent that “end-to-end” doesn’t usually mean that a single algorithm or network is going from pixels to torques. In this section, we’ll try to list some potential intuitive reasons for this.

Separation of concerns

We saw above on Physical Intelligence’s blog post that there are difficulties in training an end-to-end policy that does so many different things. Another quote:

One hypothesis of why this is happening is the following. A pre-trained VLM, by its nature, pays attention to language inputs well. The gradients from the action expert now severly interfere with the model’s ability to process language, which leads the model to pick up on other correlations first.

These problems are a side-effect of one network trying to solve a lot of different problems. The old Sense-Plan-Act schema enforced a separation of concerns very strictly, but even with a more relaxed architecture, low-level control priors drastically reduce the policy search space.

A human nervous exhibits similar separation with a cortex (goal-directed commands), cerebellum (fast adaptation, prediction), spinal reflexes (fast control loops), and even mechanical impedance control in muscles / tendons.

Training complexity

Related to the separation of concerns above, an end-to-end network must learn contact mechanics, actuator dynamics, delays, friction, impact stabilization, as well as task-level planning, all in one gradient signal.

This creates extremely long credit chains and high sample complexity. Hierarchical control factorizes the learning problem.

Feedback control loops; tactile and force feedback

With a fully end-to-end system, any feedback on how the executing is going can only come in at the top. In contrast, a dedicated low-level control unit can run its own feedback controller that performs stabilization functions. This is in effect what we saw above with the selection of the impedance controller in the Peng and Hutter papers above.

Secondly, a low-level controller also provides a great opportunity to incorporate a rich set of sensory signals such as tactile and force feedback information. Rodney Brooks underlines the importance of non-visual feedback in his Sep 2025 essay, going as far as to flag it as a roadblock. The problem is, if you must have force feedback in an end-to-end model, you first have to contend with the lack of large-scale force data to train it from, as well as the much larger end-to-end model you now have to train and evaluate at inference-time. As I responded to a Substack comment here, a low-level control unit is a potential way that that data could be incorporated, without increasing the dimensionality of the higher-level brain.

Control bandwidth

Real-world physics and dynamics don’t wait for end-to-end inference to complete, and most implementations (Physical Intelligence’s action chunking, Figure’s rate-decoupled system 1, etc.) need to decouple the control bandwidth of the cognitive layer from the low-level controller.

talks about this aspect as an action inference limitation in his excellent post about VLA’s which you should read if you haven’t.

Sim2real transfer

As discussed in my recent world models post, almost all these implementations that utilize large-scale demonstration data need to follow it up with reinforcement learning post-training in simulation. This surfaces an issue that has been named “sim2real transfer,” where the simulator’s accuracy can limit the deployed behavior. This has a number of solutions including domain randomization and actuator networks, but alternatively, having a low-level controller can in many cases absorb modeling error with their inverse dynamics functionality. Physics errors affect torque-level policies massively, but impedance control, whole-body control, or model-predictive control absorb modeling error by actively driving mismatch errors to zero.

Safety constraints

We can explicitly add torque constraints, joint kinematic limits, self-collision avoidance, to a low-level controller. This is intuitively true, but I’ll leave an example of a recent research paper which found out exactly this. Quoting the author:

Introducing UMI-FT: the UMI gripper equipped with force/torque sensors (CoinFT) on each finger. Multimodal data from UMI-FT, combined with diffusion policy and compliance control, enables robots to apply sufficient yet safe force for task completion.

Generalization across hardware embodiment (*maybe)

In principle, if the low-level controller completely abstracts the hardware, the higher-level brain’s functionality can be kept the same with different embodiments. Intuitively, you can reuse high-level policies if low-level layers abstract hardware, and you can improve low-level stability without retraining ML.

However, this intuitive point is difficult to verify due to the methodology of how the cognitive models are developed today. The end-to-end pixel → action policies always incorporate some amount of information about the embodiment, so it isn’t possible to train an abstract cognitive model. In practice, the foundation models of today train on cross-embodiment data to obtain generalizable knowledge. To get to the bottom of this facet, we would need to understand what constitutes a cognitive model separate from embodiment, and that is not known yet as discussed in my previous world models post:

Closing thoughts

With the end of Sense-Plan-Act, the new robotics north star is an end-to-end pipeline that does away with the need for any task-specific pipeline architecture or programming. However, today’s successful implementations tell a different story, and there are a number of intuitive reasons for this.

Foundation models excel at semantic, perceptual, and strategic reasoning, but they are mismatched to high-bandwidth, stability-critical motor control. A robust robotic architecture separates concerns into layers aligned with physical timescales and modeling regimes.

In this (part 1) article, we focused on standard visuomotor task execution. In part 2 of this series, we’ll look at how unexpected events and motor adaptation are handled in these architectures. After that, to continue this series, I’d also like to explore a standalone demonstration that can be published as an open-source repo that examines a few of these architectures and compares them fairly.

If you found this post interesting, please let me know in the comments, and share, and subscribe. Thanks for reading!

Leave a comment

Share

I’ve known of von Neumann’s huge legacy on modern computing from a college computer organization course, but I was stunned at how much he was able to extrapolate into ideas about computation in general. His writings, from before the first transistor-based computer was built, are ever-relevant after 70 years of exponential growth in computing technology. He wasn’t correct about everything—that would be impossible—but the ways in which he was wrong are even more revealing and thought-provoking. They anticipate the reason deep learning has been so capable, and also predict the architectural limits we are now running into—memory bottlenecks, brute-force scale, and energy-hungry intelligence. They also anticipate the future directions we can go in to overcome these deficiencies.

The book is very short and absolutely worth a read if you can pick it up from a library or used bookstore, but I had four broad and powerful takeaways that contextualized decades of development for me.

1. Scale & memory: Basic operations force massive memory movement

2. Precise vs. statistical: Deep learning (DL) escapes numerical fragility by becoming brain-like

3. Depth vs. architecture: DL substitutes scale for structural sophistication in the brain

4. Representation & substrate: DL is rigid where the brain is fluid

I’ll explain these four aspects below, but together they point to the same overall thesis:

Modern AI succeeded by replicating the statistical aspect of natural computation, but suffers from brute-force scaling inside an architecture that von Neumann already suspected was fundamentally mismatched to cognition.

1. Scale & memory

As the book says, the principle of “one organ for each basic operation” necessitates memory for intermediate values, on top of instruction and data memory. Von Neumann predicted that computation systems built from simple primitives can only scale by also scaling memory.

Scalar CPU architecture is still very close to von Neumann’s artificial automaton. Post-von-Neumann architectures include systolic arrays (TPUs) and near-memory compute; GPUs are a bit of a hybrid with shared memory (scratchpads), and tiled matrix multiply (data reuse). Even heavily optimized post-von-Neumann machines are still dominated by data movement, because the algorithmic structure forces it.

Modern deep learning vindicates this: intelligence is achieved not through complex operations but through scale, which makes memory movement, not computation, the central bottleneck of contemporary hardware. We have been talking about the AI memory wall for a few years, but it was inevitable from these predictions 70 years ago.

A related aspect which von Neumann couldn’t have anticipated was the energetic impact of memory access. He did write about the energetic cost of logic operations, but today, moving 1 bit from DRAM costs more energy than 1 FLOP, to the tune of 100× a multiply. This pressure is driving technology development in near-memory compute, in-memory analog MACs, optical interconnects. The same architectural tension von Neumann identified today drives the economics1 of AI hardware.

2. Precise vs. statistical

One of the central topics in the book is how digital computation needs very high precision because of the high arithmetic depth of repeated basic operations. If each operation has error ε, after N steps you expect error ≈ O(Nε). Deep networks have extreme arithmetic depth with thousands of layers and trillions of operations. However, empirically, 4-bit quantization in deep learning works with nearly no drop in accuracy.

Why doesn’t error compound the way von Neumann predicted?

The key: von Neumann analyzed precise numerical methods (like solving equations, integrating trajectories), but neural networks are different in a couple of important ways:

1. Noise is inherent in the training process, resulting in a function approximator with inherent robustness to input noise.

2. In the accumulation function, the errors are mixed across thousands of dimensions, clipped by nonlinear saturating functions, and averaged out statistically.

The overall error is clamped and damped, and does not propagate in the same way that von Neumann assumed.

Relatedly, von Neumann argued that the brain works with low precision (1-10 bits), and performs a different type of computation than digital computers (32-64 bits). He referred to the brain as performing “statistical computing”. So deep learning is not violating von Neumann, but it is occupying the biological side of his dichotomy.

3. Depth vs. architecture

Von Neumann emphasizes three biological facts about neurons:

1. Low precision

2. Low speed (~10 ms per spike, though they can respond slightly faster under extreme stimulation)

3. Shallow circuits

We discussed the precision above; let’s dig into the others next. The nervous system is very slow, with each “layer” taking on the order of 10 ms to fire and reset (compared to digital lines changing state in < 1 ns). This means that while it is feasible to have a “deep” digital computation, that would be infeasible in a natural system.

The shallowness is also important: a crucial example in the book is that the retina does significant computation using three synapse layers, which is orders of magnitude smaller than is needed for modern Vision Transformer (ViT) encoders (hundreds of layers, billions of parameters).

How is this possible? The answer is that a biological neuron is not a basic linear unit + nonlinearity; it is more like a small analog computer. Each neuron has temporal dynamics, neuromodulators, and plasticity rules. Its connections are even more complex: each has hundreds of synapses, nonlinear integration of activations with potential spatial and geometric relations.

So the contrast is stark:

• The brain has shallow compositions of slow and low-precision units

• Deep nets have very deep compositions of very fast and medium-low-precision units

Von Neumann predicted that these fundamental differences in the basic blocks would result in different natural vs. artificial computing paradigms:

Hence the logical approach and structure in natural automata may be expected to differ widely from those in artificial automata.

Modern deep learning compensates for architectural simplicity with scale. Biology compensates for slow, noisy hardware with architectural sophistication and better primitives. This distinction strongly influences why our systems are large, power-hungry, data-hungry, and memory bound.

I hadn’t anticipated this connection when I started reading the book, but my article from last week also visits this architectural distinction from a world-model-representation perspective:

Why did we choose scaling of simple units for computing? Among other reasons (as discussed in the previous article), deep learning was built around universality + scalability, not biological realism. Simple units have advantages: easy to parallelize, easy to implement on GPUs, and easy to map to silicon.

What does this mean for general artificial intelligence? Von Neumann suspected that digital logic gates were too primitive to model cognition efficiently, and with today’s technology it is certainly true that the brain’s performance at 10W cannot be matched even at much higher power.

4. Representation & substrate

Von Neumann observes that representations of quantities which go through the nervous system may change from digital to analog and vice versa repeatedly. They can also have adaptive precision representations2. In contrast, digital machines commit very early to fixed-width numbers everywhere (FP32, FP16, INT8, etc.), and even “mixed precision” is coarse and static.

These points seem to suggest another architectural dichotomy (not just the connections between units, but also in how numerical quantities are represented). The brain has adaptive primitives, precisions, and numerical representations, whereas they are all fixed in the digital computing paradigm.

Is the answer analog computing? Von Neumann himself rejected naive analog computing due to its problems of scalability and reliability. The brain may be powerful while being efficient because it is representationally flexible, not because it is analog per se.

Neuromorphic computing is exactly about this axis, with conceptual departures such as event-driven computation, mixed analog/digital circuits, co-located computation and memory. My knowledge of the field is limited and I am not sure that any of the existing research in that area truly captures what von Neumann was hinting at, but I suspect that in the long-term future of this publication, neuromorphic computing will come up again.

Thanks for reading! Let me know if you’d suggest any related historical or modern writing on this topic, and please share and subscribe if you liked the essay.

Share

In parallel with these debates, especially on the heels of numerous announcements at CES 2026, the cultural focus is increasingly driving toward robotics or “physical AI”; is there a physical equivalent to this intellectual debate between scaling and structured models?

Here, we’ll try to go over some of the key aspects of this intellectual and conceptual spectrum starting with the informational world, and examine the implications of the equivalent schools of thought in the physical world.

This publication and this post contain the author’s personal thoughts and opinions only, and do not reflect the views of any companies or institutions.

Today’s AI is a product of scaling a simple architecture (mostly)

Breaking down this heading, by “today’s AI,” I’m referring to the most pervasive products, such as chatbots, search, coding and writing assistants. These systems are typically based on large transformer architectures composed of many repeated layers and trained on vast datasets, with models today having hunders of billions of parameters.1 In simplified form, these systems operate by mapping input tokens into embeddings, processing them through a stack of transformer blocks, and producing probability distributions over possible next tokens via a final linear projection and softmax layer.

Since the initial release of ChatGPT, the dominant trend in the development of these models has been to increase their size and the amount of data used for training, rather than to introduce fundamentally new architectural principles.

Given this architectural simplicity, the range of capability expressed by LLM-based tools is frankly impressive. Much of this capability therefore arises from the interaction between large model size and extensive training data, rather than from task-specific design and bespoke computational structures.

Sam Altman’s early 2025 blog post and the empirical observations of companies building LLMs added on evidence and expectation of continued scaling of intelligence this way. These observations led to a “scale is all you need” movement that has had enormous impact on our society and economy, with $1.7 trillion of projected investment by 2030.

The larger debate we’re looking at in this post is about the prediction that scale is sufficient (more below), but it is also important to ask if it is necessary. I.e. is scale required to exhibit the same progress? The answer to this is likely yes; as stated in a Dec 2025 preprint by Quattrociocchi et al, when the models are restricted to the transformer architecture described above, it appears to be true that “their apparent intelligence emerges only under conditions of massive scale”.

Another natural question is why there has been so much investment into exploiting scaling, vs. exploration of other architectures. The first is that progress is consistent and predictable (even suggesting scaling “laws” as in the Altman blog post) which enable predictable engineering and financial projections. Innovation and development of new architectures is a relatively unpredictable and risky process. Another very prominent virtue is that simple architectures are much easier for collaboration with other parts of the engineering stack, and has been key for the adoption of hardware acceleration for deep learning.

Many leading researchers such as Demis Hassabis, Geoffrey Hinton, and teams at OpenAI and Anthropic maintain that scaling remains a primary driver of progress.

The other side of the AI debate

Over the recent past, there have been an increasing number of arguments disagreeing with the claim that scaling is sufficient to get to arbitrary “intelligence.”

Per the March 2025 findings of the annual meeting of the AAAI, including responses from more than 475 members (67% of them academics),

More than three-quarters of respondents said that enlarging current AI systems ― an approach that has been hugely successful in enhancing their performance over the past few years ― is unlikely to lead to what is known as artificial general intelligence (AGI).

Well-respected AI researchers are starting to form the next wave of AI companies that try to encode some kind of “world model” or semantic understanding of the world: Dr. Fei-Fei Li’s World Labs generates images and videos but only via an intermediating representation of a 3D world. Yann LeCun’s new startup AMI labs is likely also building world models via some form of his published JEPA work. Ilya Sutskever (one of OpenAI’s founders, who had a large contribution to Sam Altman’s perspective above) went on Dwarkesh’s podcast and said that scaling alone would not carry us to AGI and that “something crucial is missing.” Cognitive scientist Gary Marcus has frequently writes about the need for symbolic reasoning for AI and is often in the thick of the debate on how to get there.

What is a world model?

There is at present no clearly-victorious architecture for how to encode added structure in large AI models. Consider a few examples from the AI world:

• World Labs, whose product generates consistent images and video, would define it as metric information about a 3D scene

• Many AI researchers using a working definition for a world model as a (potentially latent-space) dynamical model that predicts how the state of the world evolves under actions.

  • Schmidhuber wrote a paper about world models in 1991, with the working definition as “predicting future sensory data given our current motor actions”

  • Yann LeCun proposes learning and predicting latent-space dynamics in his JEPA research (papers 2022-2025) — crucially, the projection to latent space is also learned from data, making it more general but less grounded in physical laws

  • The 1x world model is described in Jan 2026 as having latent space prediction capability and used to generated predicted future video states

  • DeepMind’s 2025 paper also seeks a ”predictive model of its environment” — In the paper it is a markov process, but for a continuous system such as a robot, it would be continuous or discretized dynamics governed by physics. It does not, however, specify how one would design architectures to take advantage of world models: “Future work should explore developing scalable algorithms for eliciting these world models and using them to improve agent safety.”

Zooming out to broader science, models have been developed and used in almost all fields; biologists have been discovering models for navigation in animal brains, physicists have been developing models for the behavior of the universe from quantum to astronomical scales for centuries, civil engineers have been using models of mechanics to build our houses and bridges, etc. Gary Marcus defines a cognitive world model as “a computational framework that a system (a machine, or a person or other animal) uses to track what is happening in the world … persistent, stable, updatable (and ideally up-to-date) internal representations of some set of entities within some slice of the world.” Each of these parties would likely have different opinions on models of the world / universe that AI should be imbued with.

In this post, we’ll stay focused on whether the added structure is important, but not discuss the relative merits of these varied proposals. (That is a potential topic for future posts; make sure to subscribe to get notified when they get published)

Subscribe now

Why do we need world models?

The critical view is that while LLMs are designed to predict what to do next, but are not designed to build an underlying semantic understanding, and that there are many examples of errors (or “hallucinations”) that can ultimately be root-caused to this:

LLMs can parrot rules of chess but will make illegal moves at the same time, they do not generalize well to out-of-training scenarios or under uncertainty and can produce unpredictable responses to uncommon inputs such as SolidGoldMagikarp, they exhibit “semantic leakage” of concepts and semantics in their input streams, with real-world impacts on usage of AI for policing. While capabilities of LLMs do keep increasing, there is concern that errors such as these cannot be universally eradicated without an architectural shift.

Is there an equivalent debate in robotics?

Humanoid robotics in particular has been having a prominent rise into the cultural and economic consciousness in the last few years. Humanoids have been featured at NVIDIA keynotes for about two years now, clearly signaling that the time is here for robotics companies to show their products and get mass-market adoption. While the field of robotics has existed for a long time, it is undeniable that the capabilities demonstrated have been seeing large improvements along with this increased exposure to the public eye.

Does the same architectural divide we just discussed for LLMs also exist in robotics? Less is known (much less agreed upon) about the best way to develop advanced capabilities in these robots, but we can use public information from some companies that have made product announcements to guess some patterns:

• The Boston Dynamics CEO says that they “need to be able to bring a new task to bear in a day or two … because, I think in a factory, there’s literally hundreds of tasks and the tasks evolve,” and their 60 minutes feature shows the ability to rapidly deploy motion capture or VR demonstration data to their Atlas robot

• Figure describes its “Project Go-Big” as an effort to collect human demonstration data in the form of first-person video for pre-training2 a navigation model

• 1x described its plan to collect teleoperated demonstration data with its robot in people’s homes for continued training of its AI model in Oct 2025, and released an update in Jan 2026 suggesting learning from internet-scale videos as demonstration followed by RL in simulation

I want to note that all these companies have very intelligent researchers and engineers on their staff, and it is very possible (and likely) that there is more going on in these particular demos; I only include these specific reference points as context to pick out broad themes. Some surfacing patterns are that (a) the rate at which different tasks are demonstrated is a high priority for these companies, (b) many of them are looking to pre-training with motion data collected from humans, and (c) this will be followed by post-training using reinforcement learning (most likely in simulation) where the system’s reward will include matching the demonstration.

My rough summary here is largely echoed by Rodney Brooks in his 2025 post on humanoid robot dexterity:

How the humanoid companies and academic researchers have chosen to do this is largely through having a learning system watch movies of people doing manipulation tasks, and try to learn what the motions are for a robot to do the same tasks. In a few cases humans teleoperate a robot, that they can see, along with the objects being manipulated ...

A robotics parallel of LLM development

Very roughly, the training process for both status-quo approaches have similar-looking steps:

• pre-training - reading internet-scale text (LLMs), vs. watching internet-scale human demonstration video or motion data (robots);

• post-training - RLHF and its modern equivalents vs. RL in simulation followed by sim-to-real porting and deployment

With this grounding, we can ask whether robotics applications will run into the same problems and debates as we discussed for LLMs above.

One unknown is whether motion data is the best analogue of text data. Rodney Brooks articulates some concerns about this in his dexterous manipulation essay, suggesting that tactile sensing data is needed (but internet-scale tactile sensing data, or any other kind of robot data, doesn’t exist). It is likely that all the robots will include tactile sensors in some form, but it isn’t clear yet how they will fit into this human demonstration large-data paradigm.

The larger question is whether a navigation capability trained with motion data will generalize to unseen and unexpected situations, since it is not designed to encode an explicit understanding of “objects” or “inertias” or “positions”. This concern exactly mirrors the ones about semantic understanding in LLMs. It is likely that the rate of this class of error will go down with larger models trained with more data (effectively, the scaling argument). To accomplish that goal, the “robot data gap” will need to be closed, which will take a lot of compute for data generation and training larger models due to the large dimensionality of the sensory and action spaces in robotics.

It is also relatively more difficult to “scale-up” in robotics for several reasons. First, latency and real-time reaction is much more important than in a chatbot setting, and so increasing model size at the cost of latency is not viable. In Figure’s Feb 2025 blog post, we can see that a 7B parameter VLM is used, at a time when when much larger (and presumably more accurate) models were available, and 1x states that 11 seconds of thinking are required for 5 second tasks. Second, as Chris Paxton has written about many times, getting diverse and useful data to feed a larger model has a lot of challenges. Third, robots need to carry their own battery packs, and so adding a larger GPU to run larger models introduces runtime and thermal management concerns.

On the other hand, the architecture (albeit with many details glossed over) seems to be consistent across many tasks and does not require too many architectural decisions to be made or parameters to be tuned (except for training metaparameters). It also allows for a myriad of types of demonstrations to be stood up quickly for garnering buy-in and support from customers or investors, which is a significant benefit. This is understandably a parallel of some of the observations that led to the scaling-based improvements of LLMs.

World models in robotics

First, we must observe that the post-training process described above will typically use simulation environments (with simulated physics) for the training process. Despite having the appearance of being model free, the properties of the simulator (which itself uses physics models) are implicitly embedded into the learned policy.3 The 2025 DeepMind paper referenced above suggests that it may be possible to prove that this implicitly captured information can be used to extract an explicit physics model after training.

So, does that mean we should put world models out of mind and learn an implicit one as needed (or not)? Well, this is a very inefficient way to learn physics equations and parameters: Euler-Lagrange equations, and classical system ID or adaptive control methods may be able to capture the same model much more easily and in a way that is more easily generalizable. RL in general can require a large amount of training for the results they produce because rewards are typically sparse. In other words, an RL-trained policy possesses “a lot of knowledge, and in some ways far more than most, if not all, humans,” to have human-level performance on specific task. Of course, humans rely on enormous evolutionary and developmental pretraining, encoding which into specialized structures is exactly the pro-world-models argument in the debate.

In terms of the models themselves, for tasks like locomotion, Newtonian physics is very well understood, and roboticists have been building on it to develop and use models like ZMP, LIP for decades. For more abstract control systems, the concept of a “plant model” in control theory is not dissimilar to the abstract state-prediction models referred to in the section above.

Some classical methods to utilize these models are to use trajectory optimization subject to the model, model-predictive control, etc. These can impose constraints on future states, so that within the bounds of the model’s accuracy, some aspects of safety can be encoded in way that isn’t possible otherwise.

How to compare the two approaches

Now that we can recognize the “world model” debate in applications for informational and physical AI, it’s helpful to (in rough, broad strokes) know how to compare the two strategies from a number of perspectives:

1. Performance: Can the method produce results that are compelling? There are umpteen benchmarks to compare language models. The next generation of world-model-equipped LLMs aren’t here yet, so we’ll wait to wait a little while to see how they stack up. There aren’t robotics benchmarks of the sort yet, though some informal efforts are underway.

2. Scalability and time-to-market: This is a huge advantage of scaling a simple architectures. Deep neural networks with consistently-repeating matrix multiplication and reduction primitives have been able to be mapped to SIMT processors like GPUs and systolic array processors (NPUs, TPUs) with incredible performance gains. At the moment there is not even enough information about non-trivial architectures to consider mapping them to computational hardware. It is also possible that world models can be mapped into the existing computational frameworks (and we can assume that the first generation of them will have to do so to compete). Eventually, if the computations are quite different, modified paradigms and accelerators may be needed, and scaling those may require more care and thought than the straightforward process we have followed for scaling LLMs. Based on the current state of language models and humanoid robotics as recapped above, it is clearly easier to get initial proofs-of-concept working with model-free approaches scaling a simple architecture.

3. Computational efficiency: Newton’s equations descibe motion of bodies with very few parameters in great generality, and it is impractical to capture them with a “transformer-like” structure without significantly higher number of parameters. This is especially true where equations are discontinuous, which happens in robotics problems like locomotion and manipulation. AI is currently up against the so-called “memory wall” due to the fact that these models need to be so large, and the most recent innovations and movements in ML accelerators have been to do with addressing it. Utilizing appropriate models with differently-architected communications may completely sidestep this memory wall, as well as drastically improve the efficiency4 of equivalent computations.

4. Generalization: It should be clear that some of the models that need to be learned for robot motion have very applicable and general models that have been known for centuries, and the same holds for biologists, cognitive scientists, and psychologists in their fields. Ilya Sutskever, one of the architects of the current LLM era, says that their structure is weak at generalization and that generalizing in the way that humans can needs new architectures. The aforementioned DeepMind paper also cites domain adaptation and generalization to unseen tasks as something that could be improved by using world models.

5. Safety: We’ve discussed hallucinations in this post already, and the aforementioned Quattrociocchi paper makes an argument about the reliability of results from LLMs. The point of concern is how the system will react to unseen circumstances and whether it can extrapolate in reasonable ways. It may be especially important to have mechanisms for guaranteeing the possible range of actions the robot can take and explaining its decisions.

I didn’t feel like there is sufficient information to score the approaches yet, but it is clear that model-based approaches may offer advantages in generalization and interpretability, while model-free scaling currently dominates in deployment speed and tooling maturity.

Closing thoughts

Before closing out this article, I must point out that this “divide” is really a spectrum—there is likely a rich space of hybrids of the two approaches, which may consist of hierarchical structures combining the strengths of each. Deep learning excels at parsing and summarization of text and images, automatically finding the most appropriate dimensional reduction techniques. World models, when coupled with methods that know how to use them, are strong at generalization, abstraction, and can produce very computationally-efficient algorithms.

In future posts, I plan to write about any new developments on the informational or physical sides that are demonstrating usage and adoption of world models, or of new hybrid architectures. I will also be plan to write some posts where I construct simple scenarios to fairly evaluate competing architectures along the different metrics above. Last but not least, I will plan to go into more details on computational hardware acceleration of non-trivial architectures.

I believe this is going to be an ongoing recurring topic in this publication, so make sure to subscribe and share if you found this interesting.

Share

Just to provide brief context,

• the RoboBee hardware was at this point fairly mature, and on the “Split Dual-Actuator Bee” generation;

• the state-of-the-art flight controller was a capable, but task-specific hovering controller with limited generalizability.

The goal for this project was to develop a controller that could be easily generalized to more complex tasks using modern control methods. The resulting paper1 with Dr Rebecca McGill made some demonstrable advances in terms of better operation away from an upright configuration, the ability to stabilize tasks like following a desired path or executing more dynamic behaviors like perching and flipping, as well as robustness to suboptimal gain tuning and manufacturing variability.

Here are some hovering clips (short 32s video; no audio):

The remainder of this post explains how this result was achieved, and potential future extensions of the idea.

Background

RoboBee and flapping flight

The RoboBee is a 100mg flapping robot developed by Dr. Rob Wood, capable of hovering and controlled flight. To put it in perspective, a US nickel weights 5g or the equivalent of 50x RoboBees. Having spent a lot of time in the Harvard microrobotics lab fabricating them, it is no exaggeration to say that a sneeze can literally destroy weeks of work.

Along with a family of similarly-fabricated robotic systems developed at the Harvard microrobotics lab, they are actuated by piezoelectric bending actuators. The piezoelectric effect is commonly seen in the working of microphones, which convert vibrations created by acoustic pressure waves into electric signals. They also do that in reverse, converting electric pulses into vibratory motion. The RoboBee uses piezoelectric bending actuators, constructed similarly to a bimetallic strip, converting slight expansion and contraction of the piezoelectric material into a bending motion.

Generally, the piezoelectric actuators produce very small motions that need to be amplified to produce the requisite aerodynamic work. After the conversion to the bending motion, they also go through another transmission that converts the small bending translational motion into a rotational motion. In a previous post, I went into the details of how this transmission works, and a project I worked on to optimize it.

OK, now we are at the stage of converting electrical signals into rotational motion of the wing. The wing itself is attached to the end of the transmission via a passive hinge, so that when the base of the wing is flapped, it not only flaps, but also pivots about its hinge, thereby actively changing its pitch, or angle-of-attack. This motion is common among flapping animals:

RoboBee’s clever design allows the wing pitch to change passively as the wing flaps, i.e. only one actuator is needed per wing to obtain something resembling the complex wing motion of the hummingbird above:

Modeling RoboBee’s flight

A model of the motion produced is very important to understand how to use the available wing input signals to get to a desired goal. There is a large debate between model-based vs. model-free methods (which eschew models in gathering a lot of data with the black box system and approximating its behavior). Increasing computational power recently has resulted in increased temptation to abandon models, though in many sim2real reinforcement learning approaches, models are used in developing the simulation.

In the case of RoboBee, the difficulty with pursuing a fully model-based method is that aerodynamics is quite difficult to model. Nonetheless, some work in the early 2010’s on blade-element modeling has proved quite useful for understanding the relation of RoboBee wing motion to the produced lift and drag forces. Using that model, we developed a RoboBee simulator, which is open-sourced2. We will discuss the software supporting this work further below, but here is an animation of some fixed control inputs (similar to the 2013 flight control work) producing simulated flapping flight, complete with passive wing pitching (short 17s video; no audio):

The disadvantage of the model above that it is very complex and not possible to use to directly develop a controller. However, the other components of RoboBee dynamics (excluding how the wing produces lift and drag) are well-explained by Newtonian physics. In this latter area, there is a great degree of similarity to the control of legged robots.

Typically the world of flapping flight and legged control do not overlap, but there are a number of similarities that motivate the use of similar methods. They are both

• cyclic (though in the RoboBee case, the wings are assumed massless and flap so fast that their dynamics are considered decoupled from the body);

• mechanics-dominated (it is very important to consider the physics of ground interactions and aerodynamics); and

• underactuated (we don’t have enough actuators to fully stabilize the motion, and typically in these scenarios some amount of “lookahead planning” is required).

In the legged robotics field, there is a long tradition of using simplified models to aid in control development (so-called “spring-mass” models), as I have discussed before. In this paper, we introduce for the first time an equivalent for RoboBee-like flapping flight.

Model-predictive control (MPC)

As discussed above, in underactuated scenarios, it is typically the case that some knowledge about the future behavior of the system can be predicted in order to decide which inputs to supply. As a simple example, how should the cart be moved in order to get the attached pole to swing up?

A global understanding of the future behavior of the system can be summarized in a so-called value function, and knowing this function can tell us exactly which way we should move to get to our goal from all states.

The problem is, the value function is not “known.” It can be estimated by exhaustively poking and prodding the system (which is an approach that resembles reinforcement learning). However, when we know of a dynamical model for the system, it is sensible to use it, because it greatly reduces the dimensionality of the control system to treat the dynamics as fixed.

Model-predictive control (MPC) tries to create a small local approximation of the value function online by using the future state of the system over a short prediction horizon (subject to a model) as a proxy for the value of the current state. MPC is now an old technique, but widely used in industrial process automation, aerospace, etc.

Approach: model-based MPC and model-free inverse dynamics

Here is the overall plan:

1. Develop a simplified model capturing the desired behavior: for this step, I noted that we do not care about the heading, but instead simply that the robot stays upright.

2. “Anchor” the behavior on to the RoboBee: convert to signals that get sent to the acuators.

The system architecture figure below makes this explicit. The purple “flying brick” is the model, whose future states we can predict for known inputs. The MPC can then effectively back out the best inputs for that model to get to a desired state.

However, as the blue and green arrows in the figure show, that only partially solves our problem because the RoboBee is not a flying brick. To address the gap, we need to define the operations undertaken by the arrows:

• State projection (blue arrow): This process is relatively simple for this instance, because the state of the flying brick is effectively a subset of the state of the actual RoboBee. It has an elevation and body tilt angles just like the RoboBee, and we simply project the coordinates to those of the flying brick.

• Inverse dynamics (green arrow): The other direction is more complex—in essence, we want to go from the abstracted thrust/roll/pitch torque inputs for the brick → RoboBee wing actuator signals. This process is complex because of a couple of reasons:

  • The mapping is much more complex than any kind of projection; for the various components:

    • Wing voltage → actuator motion (depends on piezoelectric actuator electrical and mechanical properties)

    • Actuator motion → wing base motion (depends on transmission and its stiffness)

    • Wing base motion → wing motion (depends on hinge and wing mechanical properties)

    • Wing motion → reaction forces and torques (depends on wing aerodynamic interactions, ground effect, etc.)

  • Manufacturing variability makes this mapping inexact (if you manufactured two RoboBees, they may require different wing signals to produce the same wing motion)

For these reasons, models have limited utility for the green arrows, and so, the paper proposed a model-free method for that part.

Model-based MPC

Template: upright rigid body

First, we need to pick the model. As the saying goes, all models are wrong, but the goal here is to capture the most important parts of the dynamics, and the objective.

The RoboBee’s wings are very light, and so most of its mass is truly contained in its body (more on this below). Dynamically, this is well-approximated by the flying brick, with no other moving parts.

To capture the objective, we note that we do not particularly care about the heading of the RoboBee when we just want it to hover, or fly controllably. This allows us to effectively remove one degree of freedom from our specification of the objective, and capture the state of the flying brick with:

• To capture the position, we use the (x, y, z) Cartesian coordinates of the center of mass as expected.

• To capture the orientation, we only look at the components of the “upright vector” (a vector pointing up in the body frame). Note that an objective of hovering can be simply stated as the desire to have the upright vector point vertically up.

Waypoint tracking MPC

We write the dynamical equations for the flying brick using the Newton-Euler equations for the motion of a rigid body. After a small approximation as described in the paper, we get

where p represents the Cartesian position, s represents the upright vector, T represents the (scalar) upward thrust, and τ represents the (2-dimensional) roll, pitch torque vector.

These equations are quite simple, owing mainly to the fact that the wings are quite light, and so their flapping does not significantly impact the motion of the much more massive body. This concept is also utilized in many legged running robots, referred to there as “massless legs.” It’s worth taking a minute to appreciate the significance of this: in practice, human limbs are not massless, which allows (for example) a gymnast to adjust their body orientation while flying through the air by controllably moving their limbs and landing a flip. However, mastering that kind of control is much more difficult than the massless legs (or wings) paradigm, where we can safely make the assumptions that the appendages simply produce a force or torque that acts on the body. A helpful picture to have in mind is that in the massless appendage paradigm, we can substitute the appendages for thrusters attached at appendage base, and pretend we are controlling the thrust vector instead.

Upon further inspection, the equations are second-order (as expected for any mechanical system). The orientation equation is also unfortunately nonlinear, as can be seen from the product of s, T, and τ appearing on the right side. This is also normal for such systems, but adds a challenge to our MPC transcription.

To resolve this difficulty, we linearize these dynamics at the current orientation and thrust (s₀, T₀) before incorporating them into the model-predictive controller. The controller will reason about the best inputs based on how they act on the current state, which intuitively is fine for a short enough planning horizon.

As an analogy, a car driver on the highway will turn their steering wheel slightly to change lanes (an action that is appropriate for a planning horizon for a few seconds), even though that action would not be appropriate on a long enough horizon that they drive off the highway. Similar to the car driver, the RoboBee in this scenario will re-evaluate its inputs with a new state soon enough. MPC always works this way, with a finite planning horizon a short duration from the current time.

The objective for the MPC is to track a trajectory of future states, including a position and velocity. For example, to hover, the desired position is the hovering goal position, and the desired velocity is zero. To follow a particular path in space, that path can be discretized and substituted into the desired positions.

Simulation evaluation

To evaluate if the MPC with the linearized dynamics above works appropriately, we can compare the performance of the controller in a number of simple tasks in an apples-to-apples comparison with the prior state-of-the-art reactive controller.

Hovering, trajectory following

The tasks, as shown above, were:

• Hover task starting off withan initial orientation with roll​ and pitch angles set to 0.5 rad, -0.5 rad, and initial velocity 0.1 m/s in the x-direction​

• Waypoint tracking on an​ “S”-shaped trajectory in the xz-plane.​

• Tracking a commanded velocity of 2m/s for 0.5 seconds before stopping.

In each of these scenarios, the MPC performs better than the reactive controller (notes on tuning below), which is promising.

Perching, flipping

Specifying a task in terms of a reference trajectory can be onerous, for example, if we want the bee to do a backflip, it isn’t clear what sequences of positions and velocities are appropriate for the horizon.3 To test the robustness of the MPC, here we feed it “made up” infeasible trajectories and see how well it can track them.

The tasks we choose to test include the aforementioned flip, and a wall-perching behavior inspired by this past research:

The reference trajectories are selected intentionally naively:

• For the perch task, the desired position translates smoothly to the right, and the desired orientation steadily rotates to 90 degrees at the end of the motion

• For the flip task, the desired position is fixed, and the desired orientation smoothly rotates 360 degrees.

The results show that the MPC is able to compensate for the naiveté of the reference trajectories to accomplish the task to satisfaction. The reactive hover controller cannot solve these tasks.

A note on tuning the controllers

Something that most research papers will sweep under the rug is the process of how the controllers were tuned. The previous state-of-the-art reactive controller has hand-tuned PD gains, and the MPC has weights on the objective. To make a fair comparison, we have to tune both as best as possible.

In general, there is a tradeoff between tracking error and tracking effort. As an analogy, cruise control in cars often have an eco mode, where they may deviate from the speed setpoint a bit more, but waste less fuel. Similarly, you can spend less actuator effort in exchange for tracking the goal a little less precisely. This is usually one of the ways in which controllers are tuned in practive.

The plot on the right shows the MPC and the reactive controllers fairly compared with a variety of tuning gains, showing that the MPC is significantly easier to tune, and can track better with lower actuator effort than is possible with the reactive controller.

Data-driven inverse dynamics

As we discussed above, the mapping from actuator signal → produced force/torque is unknown/uncertain due to the system complexity and manufacturing variability.

An example of a common type of manufacturing variability is that some RoboBee transmissions just exhibit higher stiffness than others. If the left wing has a stiffer transmission than the right wing, the left wing may flap with a smaller wing amplitude than the right one when driven equivalently, and produce much less lift force.

In this project we took the approach of breaking down the components of this mapping, and just using data to characterize the variable parts. This meant collecting data of wing kinematics as a function of actuator signals and then fitting a function to approximate some “kinematics features” that could be expected for each actuator signal:​

We then used the blade-element model to predict the reaction force/torque from the wing kinematics.

To show the effect of this kind of mapping, we performed the same operation in the RoboBee simulator, and simulated the effect of adding a force bias of 3 mN to one of the actuators. With no force bias, the data-driven mapping and the manually-tuned mapping both work, but with the force bias, the data-driven mapping can still work while the manually-tuned mapping fails.

Hardware integration

Setup

Encouraged by the simulation results, we pushed ahead to integrate the MPC into the physical RoboBee control system, which looks as follows:

The actuators were connected to a Simulink real-time control PC, which was new to me. The setup encourages code to mostly be compiled from graphical blocks such as filters, delays, etc., but does allow for custom blocks written as MATLAB functions. While the state estimator and some other components were in fact MATLAB functions, we implemented the MPC in C using OSQP, as part of a more forward-looking architecture that could also run onboard the RoboBee on a microcontroller.

When run from the Simulink target PC, the iteration frequency was 5KHz for everything, locked together due to the Simulink architecture. The MPC itself also ran at 100-200Hz on small STM32G4 MCU​ that fell within the 25mg payload constraints of the RoboBee. We tested that the controller could successfully stabilize the simulator when run at rates of 100Hz.

Experimental results for hovering

A video clip of some of the hovering results were linked to in the introduction of this post. Some overlaid trajectories from those trials are shown in the figure below.

Each trial ended due to the motion capture system losing track of the RoboBee, or by a command we sent. We were able to keep the orientation stabilized in each trial, though the horizontal position drifted more than desired.

The hovering task was overall a good demonstration of the feasibility of integrating this much more advanced controller paradigm into the RoboBee.

In the future, it would be very exciting to see either or both:

• some of the tasks we tested (and compared to the reactive controller) in the simulation section running on the RoboBee

• the controller running on a microcontroller, along with onboard sensing and power, for fully untethered complex flight

Implementation details and replicating results

In the interests of open science, the code for various parts of this project are all online. While I don’t have continued access to the Simulink software and experimental setup, if you need support, please comment below—continued progress and replicability are well worth the support and debugging.

MPC

This is implemented as a quadratic program with OSQP.

• The quadratic program is defined in genqp.py. When that file is run as a script, it instantiates the controller and runs a test, or in the commented-out section at the bottom, run’s OSQP’s codegen feature to generate a standalone set of C files that can solve the QP. The codegen output is stored in the uprightmpc2 directory (though it can be regenerated as well).

• The codegen outputs define the structure of the problem, but the variables need to be updated as the current state of the RoboBee or the reference trajectory changes. To do this, the uprightmpc2.h file provides some simple interfaces with named parameters that can be called. The C file of the same name contains its implementation.

• The C code in the uprightmpc2 file can be built using CMake; something like

cd uprightmpc2
mkdir -p build && cd build
cmake ..

Simulations

• The simulations testing the MPC with the upright template model can be run from the template directory.

• The uprightmpc2.py file should recreate the test scenarios covered in plots above and in the paper when run as a script. The bottom of the file contains code describing the test scenarios that can be uncommented.

• The 3D pybullet simulation can be run by executing the robobee.py script.

Simulink setup

• The C code is integrated into the Simulink real-time setup as an S-function; the legacy_code_gen.m file configures the inputs and outputs of the block that will appear in Simulink. See this page for more guidance on this process, which was quite tricky.

• The simulink model files are slx files, and can be found here.

Modularity

In this talk, I started a bottom-up exploration of composition in robotics.

Dynamic legged locomotion

As with I’m sure many others, as a young graduate student, I was inspired by the dynamic legged locomotion work at the MIT Leg Lab in the 1980’s:

In his thought-provoking book, Raibert articulated an intriguing idea called “Control of Running Decomposed into Three Parts.” Researchers have been trying to understand when and how this may be possible, and how it generalizes, since then.

My Ph.D. advisor, Koditschek, has been doing that for decades. In the 1990’s, his research group built and impressive array of juggling robots (as a less-power-hungry proxy for cyclic dynamical behavior):

In the course of the juggling research, they introduced a formal idea of sequential composition with an intuitive but mathematically rigorous and useful idea:

Analogously, we can retroactively label Raibert’s “control in three parts” idea as an example of parallel composition. While the term is not extremely common in the robotics literature, similar concepts appear with names such as “decoupled control”. The idea has clearly been empirically useful, but formalizing it has been quite tricky with any degree of generality.

Sequential and parallel composition are a very intuitive idea with equivalents in programming and spoken language. Consider the example of generating spoken language – instead of outputting the sounds corresponding to an entire sentence at once, we may want to start by assembling words from phonemes, and assembling those into sentences. On the other hand, modern deep learning speech synthesis may not have any such compositional properties, which is an intentional counterpoint that we will return to.

Modularity elsewhere

Deep learning did evolve from neural networks, which evoke biology right in the name. Biology has inspired many of the working principles of quadrupedal robots, including behavioral modularity.

Animals have an abundance of sensory inputs and muscle, but the number of task-level variables important to any particular task is a lot smaller (Ting (2007)). Going further, Ting et. al. (2015) argues that motor modules arise from neural plasticity in spinal structures that selective coordinate and co-activate multiple muscles. The result is that animals can control tasks like balancing in a hierarchical fashion, keeping the dimension of the task-space control low.

While robots typically have fewer actuators than an animal has muscles, each individual task will typically be overactuated for a general-purpose robot. For example, a humanoid robot will not need its arms to maintain a standing posture.

If we accept the presence of these motor modules, these patterns of activation could be re-used for different behaviors. Quoting Ting et. al. (2015):

Multifunctionality: muscles can contribute to many actions; a few muscles can be combined in many ways to produce a wide range of different actions.

Making equivalences to the synthetic disciplines, there is a clear connection to the idea of re-using behavioral modules, as we showed with Minitaur vertical hopper compositions.

Putting it all together, I’d argue that there are equivalences between biology and robotics in three distinct aspects of modularity:

Modularity benefits

Some of the benefits of modularity that are enjoyed by biological systems can also apply to the synthetic disciplines as well.

Motor modules can help navigate a “difficult-to-search and nonlinear set of neuromechanical solutions for movement” (Ting et. al. (2015)) as well as the “curse of dimensionality” in various engineering disciplines. This has clear implications on the computational requirements for algorithms.

A slightly less obvious use case for modularity is for optimizing robot design for flapping, jumping, etc., using coordinated movement patterns (or, template trajectories).

Real-world robotics

As robotics tools proliferate, their side-effects will start to also have a larger and larger impact on society.

Safety and predictability

The autonomous vehicle industry is possibly the first (but certainly not the last) subfield that has been thrust into the limelight of the question of safety of autonomous systems. The responsible peer-reviewed efforts of the first-party companies (e.g. Waymo) are huge steps in the right direction, but that is certainly not the end of the story.

Robustness and multiple solutions inherent to a modular structure (as we saw above) is in stark contrast to the weakness of monolithic AI structures when subject to uncertainty (Cummings).

Intuitively, a modular architecture can be “debugged” and intermediate outputs can be logged and inspected. Just like a black box recording of an aircraft allows review of inputs made from the pilot to the machine, a modular structure allows insight into, and thresholding of, the function of individual modules:

Energy

While mechanical work done by robots necessarily needs energetic input (and the conversion efficiency can be quite high), the cost of computational work is nowhere close to the only known fundamental energetic limit based on Landauer’s principle.

Even as chips get more and more efficient, our appetite for computation outstrips those benefits, raising continual concern.

As already recognized by biology, a growing community of researchers are exploiting the fact that modular neural networks reduce power consumption.

The case for compositionality

Modularity comes with a price. The motor modules in humans have appeared over the (long) course of animal evolution, and the modular control structures developed for robots need to be hand-crafted. These processes are much less automatic, and need more work than scaling a simple structure with more data. In fact, the importance of pushing for architectural progress may not be limited to robotics (LeCun).

Additionally, modularity necessarily imposes limits on the space of usable methods or algorithms. For example, a modular controller reasoning with the equivalent of “motor modules” for a triple pendulum would never be able to accomplish this:

Nevertheless, the question of system abstraction with modularity has come up before in other fields such as digital VLSI and programming languages, and has clearly won out, in part due to the reasons discussed above.

We don’t yet have a generally accepted methodology or architecture in robotics that could be a foundation for symbolic behavior programming.

End-to-end deep neural networks have become a useful and generally-accepted architecture without compositional properties, but neural networks are not necessarily incompatible with compositionality (Hinton, Marcus). For more on this topic, I highly recommend the proceedings of this workshop on The Challenge of Compositionality for AI.

What is the path forward?

If we value the benefits of modularity discussed above, it will take more work to develop the correct architectures, but this work is essential to get to the point of robotics becoming a true scientific discipline with predictable outcomes.