Introduction

Composable React Three Fiber wrapper around mujoco-js

mujoco-react

mujoco-react is a composable React Three Fiber wrapper around mujoco-js. It provides components for loading models, stepping physics, and rendering bodies. Controllers like IK are opt-in plugins.

Composable

<IkGizmo />, <Debug />, <ContactMarkers /> etc. are R3F children you add to your scene.

Controller Pattern

Controllers are React components that call useBeforePhysicsStep. Write your own or use the built-in ones.

Pluggable IK

Swap in any IK solver via ikSolveFn, or skip useIkController entirely and solve IK yourself in useBeforePhysicsStep.

Full MuJoCo

Contacts, sensors, tendons, flex bodies, raycasting, domain randomization. The full MuJoCo API via React hooks.

Quick Start

npm install mujoco-react mujoco-js @react-three/fiber @react-three/drei three

import {
  MujocoProvider, MujocoCanvas, useIkController, IkGizmo,
} from "mujoco-react";
import { OrbitControls } from "@react-three/drei";

function Scene() {
  const ik = useIkController({ siteName: "tcp", numJoints: 7 });
  return (
    <>
      <OrbitControls makeDefault />
      {ik && <IkGizmo controller={ik} />}
      <ambientLight intensity={0.7} />
      <directionalLight position={[1, 2, 5]} castShadow />
    </>
  );
}

function App() {
  return (
    <MujocoProvider>
      <MujocoCanvas
        config={{
          src: "https://raw.githubusercontent.com/google-deepmind/mujoco_menagerie/main/franka_emika_panda/",
          sceneFile: "scene.xml",
          homeJoints: [1.707, -1.754, 0.003, -2.702, 0.003, 0.951, 2.490],
        }}
        camera={{ position: [2, -1.5, 2.5], up: [0, 0, 1], fov: 45 }}
        shadows
        style={{ width: "100%", height: "100vh" }}
      >
        <Scene />
      </MujocoCanvas>
    </MujocoProvider>
  );
}

This loads the Franka Panda from DeepMind Menagerie, renders all bodies, and adds an interactive IK gizmo on the end-effector.

Controllers

A controller is a React component that calls useBeforePhysicsStep to write data.ctrl each frame:

import { useBeforePhysicsStep } from "mujoco-react";

function MyController() {
  useBeforePhysicsStep((_model, data) => {
    data.ctrl[0] = Math.sin(data.time);        // sine wave on actuator 0
    data.ctrl[1] = data.sensordata[0] * -0.5;  // feedback from a sensor
  });
  return null;
}

Add it as a child of <MujocoCanvas>:

<MujocoCanvas config={config}>
  <MyController />
</MujocoCanvas>

IK, teleoperation, RL policies, state machines all follow this pattern. The useIkController() hook follows this same pattern. You can also plug in your own IK solver or skip it entirely.

See the Building Controllers guide for config-driven patterns, createController factory, IK gizmo coexistence, and multi-arm support.

Design Philosophy

mujoco-react follows the same pattern as react-three-rapier:

• Library handles MuJoCo engine concerns only: WASM lifecycle, physics stepping, body rendering
• Controllers are opt-in plugins: IK, teleoperation, custom controllers are composable components
• Consumers compose everything else: lights, grid, camera controls, UI, game logic
• All scene elements are R3F children: no config objects for visual-only things

<MujocoCanvas> wraps R3F <Canvas> and forwards all Canvas props. For full control over the Canvas (gl settings, post-processing, etc.), use <MujocoPhysics> inside your own:

MujocoCanvas

<MujocoCanvas config={config} shadows camera={...}>
  <Scene />
  <OrbitControls />
</MujocoCanvas>

MujocoPhysics

<Canvas shadows camera={...} gl={{ antialias: true }}>
  <MujocoPhysics config={config}>
    <MyController />
  </MujocoPhysics>
  <OrbitControls />
  <EffectComposer>...</EffectComposer>
</Canvas>

Next Steps

Building Controllers

Custom controllers, IK solvers, and the createController factory

Installation

Peer dependencies, bundler setup, and WASM notes

Architecture

Provider → Canvas → children pattern and physics loop

Loading Models

SceneConfig, Menagerie, custom model sources