Vision-Language-Action workshop (October 16, 2026)

Went to another Tallinn Robotics & Computer Vision meetup - Vision-Language-Action models for robot manipulation tasks. It was again hosted by Sergii Kharagorgiiev, in the same lab as the Robot Actuators Workshop. This time about 10 people came.

The topic was VLA (Vision-Language-Action) models - language models with vision that output robot movements instead of text. Before the meetup I collected information about the companies and research that were mentioned, and it grew into a separate article: VLA models for robots: what's happening in the industry and how to build a DIY sorting robot.

The station

Sergii has a two-arm YAM Box station from i2rt and an Nvidia RTX 5090 on which he runs different VLA models. There are three cameras: one on each wrist and one on top.

The whole cycle - recording demonstrations, training, deployment and reviewing episodes - runs through yam-abc-reproduce. A person controls the robot via leader arms, and the program records video from all cameras along with joint positions. This way ordinary people create the dataset simply by recording their own actions, and model training gets noticeably cheaper: no labeling and no expensive labs needed.

How it works and why it's still "dumb"

Sergii honestly showed that the models are still quite narrow. They are trained on specific actions - pick up a cube and put it in a box - and handle other tasks poorly. Details differ between models, but the essence is the same: looking at camera frames, the model builds an optimal motion trajectory, the robot executes part of it, and after a few frames the computation repeats. The result is around 10 frames per second, so true realtime is still far away.

The main problem is stability. Public datasets contain thousands of hours of video, but, for example, they are shot on a white background, and the model becomes sensitive to it. I ran into the same thing about five years ago when I tried to hook up BeeAlarmed to my beehives: training there was tied to a green background, and it worked poorly for me.

My thoughts

• Too many video streams. Three cameras put a heavy load on video processing. I think ideally one stereo camera plus pressure sensors in the gripper would be enough.
• Geometry matters. For such models the distance between the arms and the camera position are critical - everything must match what the data was recorded with.
• More interesting hands. It would be interesting to try not a simple two-finger gripper, but a multi-finger hand with tactile feedback. Then much more complex things become possible.

Robot dog

At the end someone from the University of Tartu brought a robot dog (looks like a Unitree Go2), and it was interesting to see how it works.

• It is controlled with a remote, there is no AI inside and almost no smart features.
• There is a rotating lidar in front: the dog doesn't bump into walls, but it can when walking backwards.
• It can't carry a person, but something small - sure.
• The battery lasts a few hours, there is no auto-charging.
• It is expensive.

If you mounted an arm like Sergii's on top, you'd get a much more useful machine.