ethanmclark1

• LeRobot integration into LuckyEngine (LE, LR, LL)
  • LeRobot already setup inside of LuckyLab and was able to train a policy
  • Focus is on inference into LuckyEngine right now
  • Currently, inference is setup and gRPC connects then sends observations but robot doesn't move inside of LuckyEngine
  • Scene setting is not working correctly in this scenario: piper-blockstacing scene, blocks get teleported out of workspace without robot even moving
• Setup LuckyLab with CLI to start LuckyEngine in specified scene and autonomously start training/inference (LE, LR, LL)
  • Enable command line control from LuckyLab to start LuckyEngine into a specified scene and play scene automatically for full control from command line with luckylab
• Configure mjwarp in LuckyEngine (LE, LR, LL)
  • Setup mjwarp inside of LucykEngine

• Enable multiple robots to be controlled inside of LuckyEngine through Learn/ API

ethanmclark1

Replace gRPC with Shared Memory for Local Zero-Copy Communication

Context

LuckyEngine and luckyrobots (Python) always run on the same machine — even in cloud (GCP/AWS), both are co-located on the same node. No remote transport needed. The goal is true zero-copy shared memory replacing gRPC+Protobuf for all communication.

Key insight: AgentBatch.StateBuffer is already a flat float[num_envs * obs_size] (AgentBatch.cs:74). By allocating this directly in shared memory, agents write observations straight into shared memory with zero copies end-to-end.

Architecture