> ## Documentation Index
> Fetch the complete documentation index at: https://dadd.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# useCameraFrameCapture

> Capture offscreen frames from a chosen camera pose

Capture a still image from a specific camera pose without moving the user's
interactive viewport. This is useful for MuJoCo-mounted dataset cameras, policy
rollout reports, thumbnails, and visual regression checks.

Unlike `useFrameCapture`, this renders the scene to an offscreen render target.
It does not require `preserveDrawingBuffer`.

<Tip>
  For the distinction between viewer cameras, MuJoCo cameras, explicit capture
  poses, and virtual debug camera overlays, see [Cameras and Captures](/guides/cameras-and-captures).
</Tip>

## Signature

```tsx theme={null}
useCameraFrameCapture(options?: {
  camera?: THREE.Camera
  cameraName?: string
  siteName?: string
  bodyName?: string
  position?: THREE.Vector3 | readonly [number, number, number]
  lookAt?: THREE.Vector3 | readonly [number, number, number]
  quaternion?: THREE.Quaternion | readonly [number, number, number, number]
  up?: THREE.Vector3 | readonly [number, number, number]
  positionOffset?: THREE.Vector3 | readonly [number, number, number]
  quaternionOffset?: THREE.Quaternion | readonly [number, number, number, number]
  width?: number
  height?: number
  type?: string
  quality?: number
  fov?: number
  near?: number
  far?: number
  projectionMatrix?: THREE.Matrix4 | readonly number[]
  mujocoCameraCompatibility?: boolean | {
    useResolution?: boolean
    useIntrinsics?: boolean
    useClipping?: boolean
    preserveAspect?: boolean
    preferResolution?: boolean
  }
  hiddenGeomGroups?: readonly number[]
  visibleGeomGroups?: readonly number[]
  hiddenGeomNames?: readonly string[]
  background?: THREE.ColorRepresentation
  backgroundAlpha?: number
  visualOverrides?: CameraFrameVisualOverrides
  renderIsolation?: boolean | CameraFrameRenderIsolationOptions
  flipX?: boolean
}): {
  status: "idle" | "capturing" | "captured" | "error"
  error: Error | null
  isCapturing: boolean
  capture: (options?: CameraFrameCaptureOptions) => Promise<CameraFrameCaptureResult>
  captureBlob: (options?: CameraFrameCaptureOptions) => Promise<CameraFrameCaptureBlobResult>
  reset: () => void
}
```

`CameraFrameCaptureResult` and `CameraFrameCaptureBlobResult` include
`source.kind`, which is one of `mujoco-camera`, `mujoco-site`, `mujoco-body`,
`custom-camera`, `explicit-pose`, or `fallback-camera`. Dataset recorders should
prefer the MuJoCo-mounted source kinds.

## Usage

```tsx theme={null}
import { useCameraFrameCapture } from "mujoco-react";

function DatasetCaptureButton() {
  const capture = useCameraFrameCapture({
    width: 1024,
    height: 1024,
    siteName: "head_camera_rgb_optical_frame",
    type: "image/png",
  });

  async function saveFrame() {
    const frame = await capture.captureBlob();
    await uploadDatasetImage(frame.blob);
  }

  return (
    <button onClick={saveFrame} disabled={capture.isCapturing}>
      Capture dataset view
    </button>
  );
}
```

## API Ref Path

`MujocoSimAPI` exposes the same offscreen camera capture methods:

```tsx theme={null}
const frame = await apiRef.current?.captureCameraFrame({
  width: 1024,
  height: 1024,
  cameraName: "head_camera",
  mujocoCameraCompatibility: true,
});
if (frame?.source.kind !== "mujoco-camera") {
  throw new Error("Expected a mounted MuJoCo camera frame.");
}

const blob = await apiRef.current?.captureCameraFrameBlob({
  width: 1024,
  height: 1024,
  siteName: "wrist_camera_rgb_optical_frame",
});
```

`mujocoCameraCompatibility` is only applied to named MuJoCo `<camera>` captures.
It derives width/height from MJCF `resolution`, uses MJCF `fovy`, copies visual
clip planes when available, and installs an asymmetric projection matrix from
`cam_intrinsic` and `cam_sensorsize` when those arrays are exposed by the WASM
model. Leave it off for synthetic `position` + `lookAt` probes where you are
manually matching a dataset view.

Use `projectionMatrix` when you already have calibrated camera intrinsics. Use
`visualOverrides` to temporarily override scene background, environment, fog,
shadow maps, tone mapping, or output color space for one capture. Use
`renderIsolation` when policy or training images should be rendered with an
independent offscreen `WebGLRenderer` instead of inheriting viewer renderer
state.

## Policy Image Payloads

Use `usePolicyCameraFramesFromMountedStreams` or
`capturePolicyCameraFramesFromMountedStreams` when a policy expects image keys
such as `observation.images.front` and `observation.images.wrist`.
The helper resolves dataset stream names to mounted MuJoCo cameras, sites, or
bodies, captures each stream, and returns both raw frames and payload-ready
data URLs.

```tsx theme={null}
const policyCameras = usePolicyCameraFramesFromMountedStreams({
  cameraKeys: ["front", "wrist"],
  aliases: {
    front: [{ cameraName: "realsense_d435i" }],
    wrist: [{ cameraName: "wrist_cam" }],
  },
  defaults: {
    width: 640,
    height: 480,
    type: "image/jpeg",
    quality: 0.82,
  },
  streamOptions: {
    front: {
      aliases: ["realsense"],
      hiddenGeomGroups: [3, 4],
      background: "#d8ddd8",
    },
    wrist: {
      quaternionOffset: [1, 0, 0, 0],
    },
  },
  requireAll: true,
});

const result = await policyCameras.capture();

await fetch("/infer", {
  method: "POST",
  body: JSON.stringify({
    state,
    images: result.images,
  }),
});
```

Use the non-hook helper when policy execution happens outside React:

```ts theme={null}
const result = await capturePolicyCameraFramesFromMountedStreams(api, {
  cameraKeys: ["front", "wrist"],
  aliases: {
    front: [{ cameraName: "realsense_d435i" }],
    wrist: [{ cameraName: "wrist_cam" }],
  },
  requireAll: true,
});
```

Explicit Three camera poses are also valid policy streams. They do not move the
user's orbit camera:

```tsx theme={null}
const result = await capturePolicyCameraFramesFromMountedStreams(api, {
  cameraKeys: ["front", "wrist"],
  aliases: {
    wrist: [{ cameraName: "wrist_cam" }],
  },
  streamOptions: {
    front: {
      position: [1.1, -0.3, 1.3],
      lookAt: [0.45, -0.3, 0.8],
      up: [0, 0, 1],
      fov: 48.5,
    },
  },
});
```

Policy stream keys stay stable even when they resolve through aliases. In the
example above, `result.frames.wrist`, `result.images.wrist`, and
`result.images["observation.images.wrist"]` are populated even though the
mounted MuJoCo camera is named `wrist_cam`.

## Notes

* Use `cameraName` for MuJoCo `<camera>` elements, or `siteName` / `bodyName`
  for robot-mounted camera frames.
* Use `positionOffset` and `quaternionOffset` to calibrate a mounted camera
  pose without editing MJCF. Offsets are applied in the mounted camera/site/body
  local frame. Quaternion arrays use Three.js order `[x, y, z, w]`; convert
  MuJoCo `[w, x, y, z]` quaternions before passing them.
* Use `position` + `lookAt` only for synthetic fixed debug cameras.
* Use `quaternion` when replaying a recorded camera pose.
* Use `camera` to clone an existing `THREE.Camera`.
* The returned canvas is generated from an offscreen render target.
* Use `hiddenGeomGroups`, `visibleGeomGroups`, or `hiddenGeomNames` to match a
  dataset renderer without changing the interactive scene.
* Use `visualOverrides` and `renderIsolation` for canonical policy/training
  captures that should not inherit the interactive viewer's visual effects.
* Use `flipX` to mirror captured frames horizontally when matching a policy
  dataset or camera convention that stores mirrored images.
* Advanced renderers can attach a function to
  `CAMERA_FRAME_CAPTURE_RENDER_USER_DATA_KEY` in scene object `userData` to
  provide custom Three pixels for capture.