Architecture

Provider pattern, physics loop, controller plugins, and rendering pipeline

Provider Hierarchy

mujoco-react uses a layered provider pattern. Two setup options:

MujocoCanvas wraps R3F <Canvas> and forwards all Canvas props. MujocoPhysics provides just the physics context inside your own Canvas.

MujocoCanvas                          MujocoPhysics

MujocoProvider                        MujocoProvider
  └─ MujocoCanvas                       └─ Canvas
       └─ MujocoSimProvider                  └─ MujocoPhysics
            ├─ (SceneRenderer — auto)             └─ MujocoSimProvider
            └─ YourComponents                          ├─ (SceneRenderer — auto)
                 ├─ IkGizmo                            └─ YourComponents
                 └─ Controllers

MujocoProvider

The outermost wrapper. Loads the @mujoco/mujoco WASM module and provides it to all children. Must wrap your entire app (or at least the part that uses mujoco-react).

MujocoCanvas

A thin wrapper around R3F’s <Canvas>. It accepts a SceneConfig and all standard Canvas props (camera, shadows, style, etc.). Internally creates a MujocoSimProvider that loads the model and starts the physics loop.

MujocoPhysics

For use inside your own <Canvas>. Accepts the same physics props as MujocoCanvas (config, paused, speed, gravity, etc.) but doesn’t create a Canvas. This gives you control over gl settings, post-processing pipelines, and R3F context composition.

<MujocoProvider>
  <Canvas shadows camera={...} gl={{ antialias: true }}>
    <MujocoPhysics ref={apiRef} config={config} paused={paused}>
      <MyController />
    </MujocoPhysics>
    <OrbitControls />
  </Canvas>
</MujocoProvider>

MujocoSimProvider (internal)

You don’t use this directly. MujocoCanvas and MujocoPhysics create it. It:

• Loads the model from SceneConfig
• Runs the physics loop via useFrame at priority -1
• Exposes the MujocoSimAPI via React context
• Provides callback registration for useBeforePhysicsStep, useAfterPhysicsStep, and resetCallbacks

Controller Plugins

Controllers are React components that call useBeforePhysicsStep to write data.ctrl. The library ships useIkController() as one example; the same pattern works for any control logic.

// This is a complete controller:
import { ModelActuators, useBeforePhysicsStep, useCtrl } from "mujoco-react";

function MyController() {
  const shoulder = useCtrl(ModelActuators.franka.actuator1);

  useBeforePhysicsStep(({ data }) => {
    shoulder.write(Math.sin(data.time));
  });
  return null;
}

See Building Controllers for full patterns including custom IK solvers, config-driven controllers, and the createController factory.

Physics Loop

The physics loop runs inside useFrame at priority -1, ensuring it executes before any rendering:

┌─────────────────────────────────────────────────────┐
│  Priority -1 (MujocoSimProvider)                    │
│                                                      │
│  1. Zero qfrc_applied                               │
│  2. Run useBeforePhysicsStep callbacks               │
│  3. mj_step x substeps (loop until sim catches up)  │
│  4. Run useAfterPhysicsStep callbacks                │
│  5. Fire onStep callback                             │
└─────────────────────────────────────────────────────┘
          │
          ▼
┌─────────────────────────────────────────────────────┐
│  Priority 0 (default)                               │
│                                                      │
│  - SceneRenderer (internal): sync body meshes       │
│  - useIkController: gizmo animation                  │
│  - ContactMarkers: update contact positions          │
│  - Your useFrame callbacks                           │
└─────────────────────────────────────────────────────┘
          │
          ▼
        Render

Note that IK solving happens inside useBeforePhysicsStep (registered by useIkController), while gizmo animation runs at default priority via useFrame.

Timing

Physics stepping is decoupled from the render frame rate. Each render frame, the provider advances simulation time to match wall-clock time (scaled by speed). If the browser drops frames, multiple mj_step calls run in a single render frame to catch up.

// Pseudocode of the stepping loop
while (simTime < wallTime * speed) {
  mj_step(model, data);
  simTime += timestep;
}

State Management

All mutable simulation state lives in refs, not React state. React state updates trigger re-renders, and at 60fps that kills performance.

// Correct: refs for physics data
const mjModelRef = useRef<MujocoModel | null>(null);
const mjDataRef = useRef<MujocoData | null>(null);

// Wrong: React state for per-frame data
const [qpos, setQpos] = useState<Float64Array>();  // Don't do this

Hooks like useBodyState and useJointState return refs that update every frame without re-rendering:

const { position } = useBodyState("block");

useFrame(() => {
  // Read current position, no re-renders
  console.log(position.current.x);
});

Composability

Everything inside <MujocoCanvas> (or <MujocoPhysics>) is a standard R3F child. You mix library components with your own:

import { IkGizmo, ModelSites, useIkController } from "mujoco-react";

function Scene() {
  const ik = useIkController({ siteName: ModelSites.franka.tcp });

  return (
    <>
      {/* Library: opt-in IK gizmo */}
      {ik && <IkGizmo controller={ik} />}

      {/* Library: debugging */}
      <Debug showSites showJoints />
      <ContactMarkers />

      {/* Yours: scene decoration */}
      <OrbitControls makeDefault />
      <ambientLight intensity={0.7} />
      <Grid args={[10, 10]} />

      {/* Yours: game logic */}
      <MyController ik={ik} />
      <RewardVisualizer />
    </>
  );
}

<MujocoCanvas config={config}>
  <Scene />
</MujocoCanvas>

All visual components accept standard R3F group props (position, rotation, scale, visible, etc.), so you can position and transform them like any Three.js group.

Accessing the API

Two ways to access the simulation API:

1. Ref (outside R3F)

function App() {
  const apiRef = useRef<MujocoSimAPI>(null);

  return (
    <MujocoCanvas ref={apiRef} config={config}>
      {/* ... */}
    </MujocoCanvas>
  );

  // apiRef.current is the MujocoSimAPI once loaded
}

2. useMujoco() hook (inside R3F)

function MyComponent() {
  const sim = useMujoco();

  useEffect(() => {
    if (sim.isReady) sim.api.setSpeed(2.0);
  }, [sim]);

  return null;
}

useMujoco() can only be called from components that are children of MujocoCanvas or MujocoPhysics. Calling it outside will throw an error.