LeRobot Improves VLM Subtask Labeling: Readable Contact Sheets, Seed Relabeling, and Self-Hosted vLLM Support
Decision Brief
This update focuses on the VLM subtask labeling pipeline. First, contact sheet timestamps are now scaled proportionally to tile height (previously ~10px font), ensuring readability after VLM downsampling to 768px, improving boundary labeling resolution. Second, an optional seed relabeling channel (plan.subtask_seeded_relabel) minimally corrects labels using pre/post-segment contacts while keeping boundaries fixed, costing one extra VLM call per segment (off by default). The OpenAI-compatible client now safely handles empty model responses (e.g., safety filters or exhausted reasoning budgets) without crashing the labeling run. It adds a `reasoning_effort` parameter to limit reasoning budgets for models like Gemini. Additionally, the environment variable `LEROBOT_PROMPT_OVERRIDE_` allows remote injection of candidate prompts for prompt search without branching. Finally, the previous attempt to use a simplified GEPA alignment prompt was reverted, as it reduced Seg F1 from 0.259 to 0.189 on a 30-episode test subset, restoring the original prompt as the best configuration.
Sources
- LeRobot(HuggingFace 机器人)
HuggingFace LeRobot — embodied-AI / robotics learning library; new models, datasets, and policies.
- LeRobot(HuggingFace 机器人)
留言
登入后即可留言,和其他 builder 交换实测心得。
还没有留言,抢头香。