Action Primitives#

Where the planner chooses what to do, the action primitive chooses how it happens. A primitive turns a tool call (pi0_pick, move_to, rotate_wrist, …) into an executable action chunk for the environment.

RPent supports two families of primitives out of the box:

  • VLA policies (Vision-Language-Action models). These run in the dedicated vla_server process, keep GPU weights isolated from the physics engine, and are called by the toolkit through a per-env model client. Examples: Pi0.5 (LIBERO), RLDX-1 (RoboCasa).

  • Scripted primitives. Deterministic motions such as move_to, rotate_wrist, release, or back_project. They live on the agent side (no VLA weights needed) and are wired directly to env_server RPCs.

For the concrete per-robot configuration (which VLA runs against which robot, checkpoint paths, tool surface), see the robot pages: LIBERO, RoboCasa, Franka, Dual Franka, SO-101.

Which VLA runs where#

Environment / robot

Default VLA

Transport

Server

LIBERO (sim)

Pi0.5

HTTP or socket RPC

rpent/robots/components/pi05_vla_server.py

RoboCasa (sim)

RLDX-1

HTTP or socket RPC

robots/robocasa/vla_server.py

Franka (real)

Pi0.5 or RLDX-1 (task-dependent)

HTTP or socket RPC

robots/franka/vla_server.py (planned)

SO-101 (real)

RLDX-1 (task-dependent)

HTTP or socket RPC

robots/so101/vla_server.py (planned)

The VLA server exposes the same predict / healthz methods over both HTTP (JSON) and socket (pickle-framed) transports. When starting the server directly, select the transport with the server’s --transport {http,socket} option (defaults to http). See Add a New Robot for the design rationale.

For standalone services, remote endpoints, and cross-run model reuse, see Remote Services.

Adding a brand-new primitive family#

If the primitive you want is neither a VLA nor a scripted motion — say a WAM (World Action Model), a diffusion planner, or a Model Predictive Control primitive — see Add an Action Primitive.