[AI Robotics Field Notes (1)] The Difficulty of Robotic Automation Is Not Determined by Robot Specifications

In robot automation meetings, the first thing we check is not the robot's specifications but the nature of the task. We ask whether the workpiece arrives in the same position every time, whether its position varies, or whether nothing can be known until contact is made.

There is a reason we keep asking the same questions. The difficulty gap between these three cases is far larger than any gap between robot specifications. Two robots with identical payload and repeatability will require entirely different setup timelines and technology stacks depending on which kind of task they are assigned to.

Level 1: The workpiece and the path are the same every time

The first level is fixed, teaching-based automation. It runs on the premise that the work environment and the workpiece are always in the same position. An engineer teaches the coordinates one by one, and the robot repeats that path. Car-body spot welding, transferring and stacking aligned parts, and painting that sweeps a fixed path all belong here.

At this level, what separates the skilled from the rest is not the robot but the jigs and fences — that is, fixture design. Instead of making the robot smart, you control the environment completely. And when the environment wavers, performance wavers with it.

Level 2: The position varies, and the robot sees and adjusts

At the second level — vision-based automation — that premise breaks. With a camera attached, the workpiece no longer has to be in the same place every time.

Bin picking — pulling parts out of a randomly piled heap — is the example most often cited, but this level is not limited to picking. Tracking a weld seam to correct the path, fitting a sealant dispensing trajectory to the target's geometry, and inspecting appearance all live here.

Recognition accuracy is the starting point. If the coordinates wobble by a few millimeters, nothing you attach downstream will help. And this accuracy is determined less by the algorithm than by the choice of optics. Highly reflective metal scatters the projected pattern and leaves holes in the point cloud; widen the field of view to capture the whole pallet, and resolution drops. Choosing the method against the target material, the required precision, and the cycle time comes first.

Even when recognition is right, whether the robot can reach those coordinates is a separate matter. The grasp pose comes out, but the gripper hits the bin wall, or the path interferes with a jig along the way. A solution existing and a solution the robot can reach are two different things. In the end, recognition, path generation, and collision checking have to be interlocked to produce a result.

Level 3: The target responds differently every time

The third level is adaptive, high-difficulty automation — the domain where delicate force control is needed, the workpiece is flexible, or real-time correction is required mid-task. F/T sensor control, compliance control, and real-time trajectory correction are the foundation, with imitation learning and reinforcement learning being layered on top in recent years.

Grinding, deburring, and connector and cable insertion are the representative cases. What they share is that seeing alone is not enough. Grinding requires reading the reaction force at the instant the tool touches the surface; connector insertion requires sensing the snag and making fine adjustments to the pose.

Whether the target deforms is a variable of equal weight. Cables and wiring harnesses, tubes, thin sheets, and sealing materials change shape the moment they are grasped. The geometry read by vision is valid only until the gripper makes contact, and taught coordinates stop matching from the second unit onward. The state representation that ends at six values — position and orientation — for a rigid body is, here, effectively open-ended. So what is needed is not a see-once-then-execute approach, but one that watches, reads forces, and corrects on the fly.

Up through the previous levels, a plan can be made before the task begins. From this level on, it cannot. The plan becomes subject to revision from the moment of contact.

Level 4: Never-before-seen situations enter the mix

The last level is autonomous, learning-based automation. Built on end-to-end foundation models, it learns the movements of skilled workers from data without explicit coding, and handles never-before-seen objects through inference. The material for training these models ultimately comes from the floor at Level 3 — the force-torque data and process parameters that accumulate in contact-rich work.

Where the previous three levels were about how well one task gets done, this level is about how many tasks one system can carry. It is the picture of a robot that does not get re-taught at every variant change, and that carries assembly, inspection, and transfer in sequence. High-mix low-volume lines, household work like cooking and tidying, and autonomous laboratories that move reagents and operate instruments are the settings most often discussed. What they share is that the list of things to do cannot be written out in advance.

This is also why humanoids come up alongside this level. To use the spaces and tools made for people as-is, shaping the machine like a person is advantageous. But form and intelligence are separate matters. Two arms and a pair of legs do not lift a robot above Level 2 if it cannot divide up its own work and recover from failure — and conversely, a single articulated arm that can make those judgments belongs at this level.

At this point the robot comes closer to a physical AI agent than a machine: it takes a goal, composes its own procedure, chooses its tools, and, when it fails, tries again another way. For now, most of this still lives in laboratories and demo videos.

In closing

What divides robot automation is not robot specifications but how much uncertainty the task allows. If the workpiece and the path are the same every time, it is Level 1; if the position varies but seeing is enough to adjust, Level 2; if nothing can be known until contact, Level 3; and if the list of things to do cannot be written out in advance, Level 4.

Which is why an automation review should start with task analysis, not robot selection. If you have a process under review, check four things first:

• Are the workpiece's position and orientation the same every time?

• If they vary, is seeing enough?

• Does the target deform the moment it is grasped?

• Does the task require force control along the way?

Processes that end at the first two questions already have plenty of proven methods. If even one of the last two applies, the required technology stack and setup approach change completely.