I’m curious on your thoughts of having more layers instead of a pure 2 level HL->LL framework. It seems like humans do something like this with the cortex -> motor cortex -> brain stem/spinal cord. It’s interesting to see that Figure adopted this kind of hierarchy, any thoughts on the pros/cons on splitting the layered control architecture even more?
Also for what it’s worth I would vote for an IsaacSim implementation since it might be easier to have an RL pipeline that’s already kind of bundled together with active developer support than piecing together your own RL stack, sim, evals, etc. But idk it is always satisfying to piece together something from scratch haha
Agreed - more hierarchical layers mean more potentially performance-limiting interfaces, but also potentially better performance and debug ability. Another way to think about it is what happens if the higher level is switched off (maybe it needs to rethink or “reason”, in LLM terms)—it is nice if the lower levels can prevent safety issues by at least taking care of balance and other safety concerns independently.
The progression from high-level to low-level control in end-to-end robotics is a crucial design pattern. Your breakdown of how motor adaptation fits into this hierarchy is insightful, especially the comparison to biological systems like the cerebellum's role in adaptation. The challenge of handling unexpected conditions without retraining the entire foundation model is exactly where adaptive low-level controllers shine. Looking forward to Part 3!
I’m curious on your thoughts of having more layers instead of a pure 2 level HL->LL framework. It seems like humans do something like this with the cortex -> motor cortex -> brain stem/spinal cord. It’s interesting to see that Figure adopted this kind of hierarchy, any thoughts on the pros/cons on splitting the layered control architecture even more?
Also for what it’s worth I would vote for an IsaacSim implementation since it might be easier to have an RL pipeline that’s already kind of bundled together with active developer support than piecing together your own RL stack, sim, evals, etc. But idk it is always satisfying to piece together something from scratch haha
Agreed - more hierarchical layers mean more potentially performance-limiting interfaces, but also potentially better performance and debug ability. Another way to think about it is what happens if the higher level is switched off (maybe it needs to rethink or “reason”, in LLM terms)—it is nice if the lower levels can prevent safety issues by at least taking care of balance and other safety concerns independently.
Yeah I was thinking of that too! Especially after seeing Matt Mason's Inner Robot blogpost [https://mtmason.com/the-inner-robot/] and some more neuroscience evidence of discrete functional structures (even at the high level) [https://www.cambridge.org/zw/universitypress/subjects/life-sciences/animal-behaviour/divided-brains-biology-and-behaviour-brain-asymmetries?format=PB]. I'm not fully convinced by arguments from pure learning people claiming that any engineered structure will always become the bottleneck, it seems clear that biological counterparts do it for efficiency of digesting complex information flow -> actions.
I'm looking forward to your third installment!
The progression from high-level to low-level control in end-to-end robotics is a crucial design pattern. Your breakdown of how motor adaptation fits into this hierarchy is insightful, especially the comparison to biological systems like the cerebellum's role in adaptation. The challenge of handling unexpected conditions without retraining the entire foundation model is exactly where adaptive low-level controllers shine. Looking forward to Part 3!