We define functional generalization as the ability to repurpose diverse tools to achieve the same function. While humans can effortlessly adapt their actions across tools with different shapes and affordances, robots still struggle to transfer functional intent from perception to control. FuncBridge addresses this challenge by learning functional intermediate representations that bridge visual understanding and grounded action, enabling robots to accomplish the same function with arbitrary tools.
Three functions, three unseen tools. Compare FM, ATM, and FuncBridge on the same tool-use task.
Strike the red cube with the designated hitting point, adapting the motion to each unseen tool.
FM and ATM can pass over the target, miss the specified hitting point, or drop the tool. FuncBridge predicts function-aware keypoint trajectories to align the hitting region and complete the strike.
Push the blue cylinder with the designated sweeping point while maintaining the required contact.
FM and ATM can pass over the target, sweep imprecisely, or drop the tool. FuncBridge guides the sweeping point toward the cylinder and carries through the pushing motion.
Pull the green ball inward with the designated hooking point, coordinating alignment and the pulling motion.
FM and ATM can pass over the target, hook imprecisely, or drop the tool. FuncBridge aligns the functional region with the ball and follows through with an inward pull.
Functional generalization on a Franka robot. Compare FM and FuncBridge across unseen tools and two patterns per function.
Use the specified part of each unseen tool to strike the target. Each tool is evaluated with two hitting patterns.
Hitting Pattern 1 requires a strike with the hammer-like head. FM incorrectly uses the handle. FuncBridge strikes with the specified head.
Hitting Pattern 2 requires a strike with the handle. FM misses the hit. FuncBridge aligns the handle with the target and completes the strike.
Hitting Pattern 1 requires a strike with the front of the shoe. FM misses the hit. FuncBridge brings the specified front point onto the target.
Hitting Pattern 2 requires a strike with the rear of the shoe. FM misses the hit. FuncBridge adjusts the motion to strike with the rear point.
Hitting Pattern 1 requires a strike with the bottom edge. FM fails to complete the downward motion. FuncBridge follows through to hit the target.
Hitting Pattern 2 requires a strike with the book spine. FM misses the hit. FuncBridge aligns the spine and completes the strike.
Repurpose an unseen book to sweep the target using two different functional patterns.
Sweeping Pattern 1 requires the book to contact and push the target along the specified sweep. FM misses the sweep. FuncBridge aligns the book with the target and completes the pushing motion.
Sweeping Pattern 2 requires a different sweeping region and motion. FM uses the wrong sweeping pattern. FuncBridge follows the specified pattern to sweep the target.
Repurpose an unseen book to hook and pull the target inward using two different functional patterns.
Hooking Pattern 1 requires the book to engage the target and pull it inward. FM misses the hook. FuncBridge aligns the hooking region and completes the inward pull.
Hooking Pattern 2 requires a different hooking region and motion. FM does not complete the hook. FuncBridge maintains the interaction and follows through with the pull.
View full-size table
View full-size table