---
title: "VLA models for robots: what's happening in the industry and how to build a DIY sorting robot"
date: 2026-10-19T12:00
aiAuthorship: 70
tags: [robotics, ai, vla, ml, hardware]
description: "A short overview of Vision-Language-Action models (ABC, Microsoft Rho, π0.5), the key companies, and a practical plan for building a sorting robot for a small warehouse."
---

After the [VLA models workshop](/en/events/2026-10-16-vision-language-action-workshop/) I wanted to sort out what is currently happening with "brains" for robot arms and estimate how realistic it is to build something useful on your own. For example, a robot that sorts parcels, trash or groceries in a small warehouse.

![A robot arm sorting parcels in a warehouse](/ru/blog/tech/img/vla-robots-cover.jpg)

<!--truncate-->

## What is a VLA

A **VLM** (Vision-Language Model) is a model that understands images and text. A **VLA** (Vision-Language-Action) is the same VLM with an action "head" added. As input the model gets camera frames, current joint angles and a text command ("put the cube in the box"), and as output it produces not text but a trajectory: joint and gripper positions roughly one second ahead. Such a piece of trajectory is called an **action chunk**. The robot executes part of it, then the model looks at the scene again and recomputes the plan.

The action head is usually trained with diffusion or flow matching - the same techniques that generate images, except here they generate motion.

The training recipe mirrors LLMs:

1. **Pretraining** on thousands of hours of recordings from different robots.
2. **Fine-tuning for a specific robot** - its kinematics, cameras, gripper.
3. **Fine-tuning for a task** - usually 50-300 demonstrations.
4. **Post-deployment corrections** (DAgger): a human takes over control when the robot makes a mistake, and these fragments go into further training.

Demonstrations are recorded via **teleoperation**: a human moves a "leader" arm, the robot repeats the motion, and cameras and sensors write everything into a dataset. The cheaper such recording is, the cheaper training gets, and this is exactly where most progress is happening now.

## Three works worth knowing

### ABC - an open stack from Berkeley, MIT, CMU and Amazon FAR

[ABC](https://abc.bot/) is an attempt to do for two-arm manipulators what ImageNet did for computer vision:

- the **ABC-130K** dataset: 3,553 hours, ~135 thousand episodes, 195 tasks;
- two open models: **ABC-DiT** (2B, diffusion transformer) and **ABC-VLA** (4.3B, based on Gemma 3);
- 400 hours of simulated data in MuJoCo, and success in simulation correlates well with the real world (r ≈ 0.85-0.91), so architectures can be compared without hardware;
- DAgger corrections raised box-folding success from 24% to 85%.

![Frames from the ABC-130K dataset: two-arm YAM stations and wrist camera views](/ru/blog/tech/img/vla-abc-dataset.jpg)
*Frames from the ABC-130K dataset. Source: [abc.bot](https://abc.bot/)*

![ABC simulation in MuJoCo: an arm drops a bottle into a bin](/ru/blog/tech/img/vla-abc-sim.jpg)
*The same kind of task in MuJoCo simulation - used to compare architectures without hardware. Source: [abc.bot](https://abc.bot/)*

Alongside this, i2rt released [yam-abc-reproduce](https://github.com/i2rt-robotics/yam-abc-reproduce) - the full "record → clean → train → deploy" cycle on a single station with one RTX 5090. It supports π0/π0.5, MolmoAct2 and ABC-DiT.

### Microsoft Rho - robot first, task second

[Rho](https://microsoft.github.io/rhobotics/) is an open 5B-parameter VLA: the Phi-Phy VLM plus a flow-matching action head. The main idea is a separate **midtraining** stage: about 300 hours of data for a specific robot before training on tasks. After it, a new task needs roughly half as many demonstrations, and 15 corrections raise success from 30% to 70%. Quality-wise Rho is on par with π0.5 or better.

![Rho action expert diagram: 12 blocks with self- and cross-attention to the VLM context, outputting flow velocity for the action chunk](/ru/blog/tech/img/vla-rho-action-expert.jpg)
*Rho's action head: it takes the robot state and a noisy action chunk, attends to the VLM context via cross-attention and predicts how to "denoise" the trajectory. Source: [Microsoft Rho](https://microsoft.github.io/rhobotics/)*

### Physical Intelligence π - model as a service

[Physical Intelligence](https://www.pi.website/) develops the π0 → π0.5 → π0.7 line. The π0 and π0.5 weights are openly available in the openpi repository and have become the de facto baseline for comparison. In October 2026 the company opened the [Pi API beta](https://www.pi.website/blog/api-beta): you upload your demonstrations, they fine-tune the model and host inference. Essentially it's OpenAI fine-tuning, but for robots.

![Collage of Pi API partner robots: kitchens, warehouses, agriculture, humanoids](/ru/blog/tech/img/vla-pi-api-partners.jpg)
*Early Pi API users - from burger assembly to warehouses and construction machinery. Source: [Physical Intelligence](https://www.pi.website/blog/api-beta)*

## The industry

The money in the field is already at the level of LLM startups, and approaches differ mainly in where the data comes from:

| Company | Bet | What they showed |
|---|---|---|
| **Physical Intelligence** | a universal "brain" for any robot | π0.7, Pi API; valuation $5.6B+ |
| **Skild AI** | one model for any body - from quadrupeds to humanoids | Skild Brain; $1.4B Series C |
| **Generalist AI** | own dataset of 270K+ hours, scaling laws | GEN-1.5 learns a task from a single in-context demonstration |
| **Rhoda AI** | training on internet video instead of teleoperation | DVA: predicts future video and converts it into actions |
| **Genesis AI** | own physics simulator + real data | GENE-26.5, its own anthropomorphic hand |
| **Dyna Robotics** | reliability in real production | Dyna-2.1, the Taku robot for entire workflows |
| **Weave, Ultra** | applied products on π models | Isaac 1 home robot, order packing in warehouses |
| **1X** | home humanoid | NEO for $20,000, a world model trained on video |

Main trends:

- **Data is the main bottleneck.** It is collected in three ways - teleoperation, internet video and simulation - and usually combined.
- **Hardware is getting cheaper.** The [YAM](https://doc.i2rt.com/products/yam-standard.html) research arm costs about $3,000, and training fits on a single gaming GPU.
- **Data collection is getting cheaper.** Instead of labs with operators - recording how people simply do their work.
- **Training doesn't end at deployment.** The robot improves through corrections right in the course of its work.

## Limitations that press releases don't mention

At the workshop it was clearly visible that there is a long way from a demo to a stable system:

- **Models are narrow.** They do well what they were taught (pick up a cube and put it in a box), but barely generalize to new tasks.
- **Sensitivity to the environment.** If the dataset was shot on a white table, the model will rely on the white background. Change the background, lighting or camera position - and quality drops. I stepped on the same rake about five years ago when I tried to hook up [BeeAlarmed](https://github.com/BeeAlarmed/BeeAlarmed) to my beehives: the model was trained on a green background, and in my conditions it worked poorly.
- **Not quite realtime.** The model looks at a frame, builds a trajectory, the robot executes part of it, and the cycle repeats. In practice the system processes around 10 frames per second. That's enough for sorting, but not for catching objects in flight.
- **Many video streams.** A typical station has three cameras (one on each wrist and one on top), and each must be run through the model. Hence the GPU requirements and latency.
- **Station geometry is part of the model too.** The distance between the arms and the position and angle of the cameras must match those used when recording the data.

## How to build a DIY sorting robot

Suppose the task is to distribute a stream of items into several bins: parcels by size, trash by type, groceries by category. Here's a realistic plan.

### 1. First ask yourself whether you need a VLA at all

If items lie on the table one at a time and are sorted by appearance, the classical approach is often more reliable: a detector like YOLO determines the class and position, and a simple algorithm with a vacuum suction cup moves the item into the right bin. It's cheaper, faster and more predictable. A VLA is worth it when grasping is non-trivial: soft bags, items in a pile, something needs to be flipped or taken out of a box.

### 2. Hardware

| Component | Budget option | "Like at the workshop" option |
|---|---|---|
| Arm | [SO-101](https://huggingface.co/docs/lerobot/so101) from Hugging Face LeRobot, ~$150-250 for a leader + follower pair | i2rt YAM, ~$3,000 per arm, plus a leader arm |
| Cameras | 2 USB cameras (top and wrist) | 3 Intel RealSense cameras |
| Compute | rented cloud GPU for training | your own RTX 5090 (32 GB) |
| Gripper | parallel two-finger | plus a vacuum suction cup for boxes |

It's better to start with one arm: a two-arm station doubles both the price and the amount of data.

### 3. Fix the scene, but diversify the data

This is the main lesson about stability. Cameras and arms are mounted rigidly and never move again. But during demonstration recording, deliberately vary everything else: background (lay down different mats), lighting, position and orientation of items, the items themselves. Otherwise the model will learn not "where the box is" but "where the white spot is".

### 4. Record 100-300 demonstrations

Using the leader arm, show the robot each operation. Short episodes of 10-30 seconds, one clear command per episode ("put the bottle in the blue bin"). Failed episodes are better deleted: [yam-abc-reproduce](https://github.com/i2rt-robotics/yam-abc-reproduce) and [LeRobot](https://github.com/huggingface/lerobot) provide a viewer for this cleanup.

### 5. Fine-tune an existing model

There's no need to train from scratch. Take π0.5 via openpi, ABC-DiT or Rho and fine-tune it on your data. If you don't have your own GPU, you can try the Pi API.

### 6. Deployment and corrections

Run the robot on a real stream and take over control when it makes mistakes. Dozens of such corrections often give more than hundreds of new demonstrations. Don't forget a speed limit and an emergency stop button.

### 7. What to improve later

- **One stereo camera instead of three.** Fewer video streams, less GPU load, plus depth out of the box.
- **Force and touch sensors** in the gripper. The model will know the item has been grasped, not just see it.
- **A multi-finger hand with tactile feedback.** Then you can pick up fragile, soft and small things, and that's a whole different class of tasks. For now it's expensive and there's little data for such hands, but Genesis and other companies are actively moving there.

## Summary

VLA models are now roughly where LLMs were before ChatGPT: they already work, they're already open, they can already be fine-tuned on a single GPU, but they are still narrow and finicky. For your own warehouse it's quite realistic to build a prototype for a few thousand euros and a couple of months of experiments. The key is to narrow down the task, rigidly fix the station and not skimp on data diversity.

For the hardware side - which motors go into such arms - there's a separate article: [Intro to robotics: motor types](/en/blog/tech/robot-motors/).