Skip to main content
A controller is a React component that calls useBeforePhysicsStep to write data.ctrl each frame and renders null. The useIkController() hook follows this same pattern. You can use it, swap in your own IK solver, or write your own controller from scratch.

​
Pattern: Simple Keyboard Bindings

For robots where arm control comes from <IkGizmo />, the controller only adds extra bindings (gripper, etc.): 
import { useKeyboardTeleop } from "mujoco-react";

export function FrankaController() {
  useKeyboardTeleop({
    bindings: {
      v: { actuator: "gripper", toggle: [0, 255] },
    },
  });
  return null;
}
Drop it into your scene as a child of <MujocoCanvas>: 
function FrankaScene() {
  const ik = useIkController({ siteName: "tcp", numJoints: 7 });
  return (
    <>
      {ik && <IkGizmo controller={ik} />}
      <FrankaController />
    </>
  );
}

<MujocoCanvas config={frankaConfig}>
  <FrankaScene />
</MujocoCanvas>

​
Pattern: Custom Physics-Step Control

For more complex control (IK solvers, velocity control, state machines), use useBeforePhysicsStep to write directly to data.ctrl each frame.

​
Keyboard State

Read keyboard input via window event listeners and a ref: 
import { useEffect, useRef } from "react";
import { useBeforePhysicsStep } from "mujoco-react";

function MyController() {
  const keys = useRef<Record<string, boolean>>({});

  useEffect(() => {
    const down = (e: KeyboardEvent) => { keys.current[e.code] = true; };
    const up = (e: KeyboardEvent) => { keys.current[e.code] = false; };
    window.addEventListener("keydown", down);
    window.addEventListener("keyup", up);
    return () => {
      window.removeEventListener("keydown", down);
      window.removeEventListener("keyup", up);
    };
  }, []);

  useBeforePhysicsStep((_model, data) => {
    const k = keys.current;
    if (k["KeyW"]) data.ctrl[0] += 0.01;
    if (k["KeyS"]) data.ctrl[0] -= 0.01;
  });

  return null;
}

​
Config-Driven Arm Controller

A generic hook that accepts a static config object makes it easy to support multiple robots. Each robot is a different config: 
interface ArmConfig {
  indices: number[];        // Actuator indices for this arm
  keys: string[];           // Key codes for movement
  initialJoints?: number[]; // Starting joint positions
}

interface ArmControllerConfig {
  numActuators: number;
  arms: ArmConfig[];
  base?: { ... };  // Mobile base drive
  head?: { ... };  // Pan/tilt head
}
The hook reads keyboard state and writes to the correct actuator indices each frame: 
function useArmController(config: ArmControllerConfig) {
  const keys = useRef<Record<string, boolean>>({});

  // ... keyboard listeners ...

  useBeforePhysicsStep((_model, data) => {
    for (const arm of config.arms) {
      // Read keys, solve IK, write to data.ctrl[arm.indices[i]]
    }
  });
}
Then each robot controller is just a config: 
const SO101_CONFIG: ArmControllerConfig = {
  numActuators: 6,
  arms: [{
    indices: [0, 1, 2, 3, 4, 5],
    keys: ["KeyD", "KeyA", "KeyW", "KeyS", "KeyQ", "KeyE",
           "KeyR", "KeyF", "KeyZ", "KeyC", "KeyV"],
    initialJoints: [0.0158, 2.052, 2.1307, -0.0845, 1.5857, -0.3745],
  }],
};

export function SO101Controller() {
  useArmController(SO101_CONFIG);
  return null;
}

​
Custom IK Solvers

Three options for IK:

​
1. Use the built-in solver

The default useIkController() uses Damped Least-Squares: 
const ik = useIkController({ siteName: "tcp", numJoints: 7 });
return ik ? <IkGizmo controller={ik} /> : null;

​
2. Plug in your own solver

Pass ikSolveFn to replace the built-in solver while keeping the gizmo, reset handling, and context: 
import type { IKSolveFn } from "mujoco-react";

const myIK: IKSolveFn = (pos, quat, currentQ) => {
  return myAnalyticalSolver(pos, currentQ); // return joint angles or null
};

const ik = useIkController({ siteName: "tcp", numJoints: 7, ikSolveFn: myIK });
return ik ? <IkGizmo controller={ik} /> : null;

​
3. Skip useIkController entirely

Solve IK yourself inside useBeforePhysicsStep with full access to the model and data: 
function MyIKController({ targetRef }) {
  useBeforePhysicsStep((model, data) => {
    const target = targetRef.current;
    if (!target) return;

    const joints = myCustomIKSolve(model, data, target);
    if (joints) {
      for (let i = 0; i < joints.length; i++) data.ctrl[i] = joints[i];
    }
  });
  return null;
}
This gives you full access to model/data for whatever solver you want.

​
Pattern: Reusable Plugins with createController

For reusable controllers with typed config and default merging, use the createController factory. It handles config merging, display names, and metadata. 
import { createController, useBeforePhysicsStep } from "mujoco-react";

// 1. Define your config type
interface MyConfig {
  gain: number;
  actuatorIndex: number;
  frequency?: number;
}

// 2. Write the implementation component
function MyControllerImpl({ config, children }: { config: MyConfig; children?: React.ReactNode }) {
  useBeforePhysicsStep((_model, data) => {
    const freq = config.frequency ?? 1.0;
    data.ctrl[config.actuatorIndex] = config.gain * Math.sin(data.time * freq);
  });
  return <>{children}</>;
}

// 3. Create the controller with the factory
export const MyController = createController<MyConfig>(
  { name: "MyController", defaultConfig: { gain: 1.0, frequency: 1.0 } },
  MyControllerImpl,
);

// Usage: <MyController config={{ gain: 2.0, actuatorIndex: 0 }} />

​
createController API

function createController<TConfig>(
  options: { name: string; defaultConfig?: Partial<TConfig> },
  Impl: React.FC<{ config: TConfig; children?: React.ReactNode }>,
): ControllerComponent<TConfig>;
The returned component accepts config (merged with defaults) and optional children. It also exposes static metadata: MyController.controllerName and MyController.defaultConfig.

​
Providing Context to Children

Controllers can provide state to descendants via React context: 
const MyContext = createContext<MyContextValue | null>(null);

function MyControllerImpl({ config, children }) {
  const value = useMemo(() => ({ /* state + methods */ }), []);
  return <MyContext.Provider value={value}>{children}</MyContext.Provider>;
}

​
Listening for Resets

Register a callback to reset your controller state when the simulation resets: 
function MyControllerImpl({ config, children }) {
  const { resetCallbacks } = useMujoco();

  useEffect(() => {
    const cb = () => { /* reset your state */ };
    resetCallbacks.current.add(cb);
    return () => { resetCallbacks.current.delete(cb); };
  }, [resetCallbacks]);

  return <>{children}</>;
}
The library’s useIkController() hook demonstrates all these patterns: reset handling, useBeforePhysicsStep for solving, and useFrame for gizmo animation.

​
Coexisting with IK Gizmo

When a robot supports both gizmo drag and keyboard control, the controller needs to:
  1. Accept ik as a prop (the IkContextValue from useIkController())
  2. Sync state on transition: when the user switches from gizmo to keyboard, read data.ctrl to avoid a position jump
  3. Disable IK via ik.setIkEnabled(false) when taking over
import { useBeforePhysicsStep } from "mujoco-react";
import type { IkContextValue } from "mujoco-react";

function MyArmController({ ik }: { ik?: IkContextValue | null }) {
  useBeforePhysicsStep((_model, data) => {
    const anyKeyPressed = /* check keyboard state */;

    if (anyKeyPressed && ik?.ikEnabledRef.current) {
      // Sync from current gizmo position
      for (let i = 0; i < arm.indices.length; i++) {
        targetJoints[i] = data.ctrl[arm.indices[i]];
      }
      ik.setIkEnabled(false);
    }

    if (!ik?.ikEnabledRef.current) {
      // Keyboard is in control, solve IK and write ctrl
      for (let i = 0; i < arm.indices.length; i++) {
        data.ctrl[arm.indices[i]] = targetJoints[i];
      }
    }
  });
}
Pass the ik value from useIkController() to the controller as a prop. The gizmo re-enables IK automatically when dragged.

​
Composing Controllers in Your Scene

Controllers are React children. Swap them based on state: 
function SceneChildren({ robotKey, ikConfig, showGizmo }) {
  const ik = useIkController(ikConfig);

  return (
    <>
      {ik && showGizmo && <IkGizmo controller={ik} />}
      <DragInteraction />

      {robotKey === "franka" && <FrankaController />}
      {robotKey === "so101" && <SO101Controller ik={ik} />}
      {robotKey === "xlerobot" && <XLeRobotController ik={ik} />}
    </>
  );
}

<MujocoCanvas config={entry.config}>
  <SceneChildren robotKey={robotKey} ikConfig={ikConfig} showGizmo={showGizmo} />
</MujocoCanvas>

​
Performance Tips

  • Use for loops instead of .forEach / .map in useBeforePhysicsStep (it runs every physics tick)
  • Store keyboard state in a useRef, not useState (avoids re-renders at 60fps)
  • Cache actuator IDs once (not every frame) using findActuatorByName
  • Keep the callback closure stable; avoid creating new functions each render