AI Agent Setup (MCP)

What This Gives You

Your own AI assistant can connect to you live inside Figmin XR and actively help inside the scene.

🤝

Live AI Assistance

Work with your assistant in real time instead of only chatting about ideas outside the app.

📦

Spatial App Creation

Ask your assistant to generate full Figmin XR apps for you, from quick utilities to interactive experiences.

🏷️

No Added Cost

Use your own AI assistant and computer. These features do not require a paid Figmin XR cloud service.

🗣️

Speech + Voice

Talk naturally with your assistant using local speech recognition and spoken replies.

👁️

Scene Awareness

Your assistant can inspect the scene, capture views, and help based on what is actually happening in Figmin XR.

⚡

Rapid Prototyping

Move from idea to working scene or app much faster by giving high-level instructions in plain language.

What You Can Ask It To Do

Once connected, your assistant can do much more than answer questions — it can actively help inside Figmin XR.

Retro paddle game made from 3D objects in Figmin XR

Build a retro paddle game I can play with my motion controller.

RC airplane flying around a room in Figmin XR

Make an RC airplane I can fly around the room.

Multiplayer quiz panel with four options in Figmin XR

Create a multiplayer quiz panel with A, B, C, and D options.

Selected object with a glowing label and spin effect in Figmin XR

I selected this object. Add a glowing label and make it gently spin.

Reference photo recreated as a small 3D scene in Figmin XR

Recreate this reference photo as a small 3D scene in Figmin XR.

Agent camera view inspecting an area in Figmin XR

Use your agent camera to look at this area and tell me what you think.

These prompts are illustrative. For best results, give your assistant more context and expect to iterate. Start with a clear request, review what it creates, then ask for changes until it feels right.

Speech, Voice & Vision

Talk to your assistant and let it see what is happening in your scene.

Figmin XR can support a more natural conversational workflow: dictate prompts in the headset, hear spoken replies, and let your assistant use its agent camera to inspect the scene, understand what you are working on, and check the things it creates.

Hardware acceleration: PCs with NVIDIA graphics cards can use GPU acceleration for faster speech recognition and voice replies.

How It Works

Figmin XR connects to your AI assistant through a small bridge that runs on your computer.

A small local bridge runs on your computer. Both Figmin XR and your AI tool connect to it, allowing the assistant to interact with Figmin XR in a live session.

The local machine can also offload heavier tasks such as speech recognition and voice generation, which makes the in-headset experience much more practical.

Connection Flow

XR

Figmin XR

AI

Claude / Codex / Other

PC

Local Bridge MCP + Speech + Voice

Setup

Do the first two steps once. After that, start the bridge whenever you want to use Figmin XR with your AI assistant.

The setup is more manual than we would like right now, but it is not difficult once you know the steps.

1

Install UV

UV is a small tool that downloads and runs Figmin XR's MCP tools.

On Windows, open Command Prompt and install UV. After it finishes, close Command Prompt before continuing.

winget install astral-sh.uv

2

Configure Your AI Client

Pick the AI app you want to use with Figmin XR. You can configure Codex Desktop, Claude Desktop, or both.

Codex Desktop

Ask Codex to add Figmin XR MCP

Open a new Codex chat and paste this prompt:

I'd like to add the Figmin XR MCP server to Codex Desktop:

uvx figmin-mcp

Could you please set that up for me?

Restart Codex Desktop after Codex finishes updating the MCP settings.

Claude Desktop

Add Figmin XR to Claude Desktop

Open Claude Desktop's developer settings and add Figmin XR MCP to Claude's configuration file.

1. Click Edit Config

Claude Desktop Developer settings with Edit Config highlighted

2. Add the Figmin XR entry

Claude Desktop MCP config file with the Figmin XR server entry highlighted

{
  "mcpServers": {
    "figmin-xr": {
      "command": "uvx",
      "args": ["figmin-mcp"]
    }
  }
}

Restart Claude Desktop after saving the config file.

Start a Session

When you want to work with Figmin XR over MCP, start the local bridge, connect from Figmin XR, then ask your assistant to join.

1

Start the Local Bridge

Open a new Command Prompt and run one of the bridge commands below. Keep this terminal window open while you are using Figmin XR with your AI assistant.

Standard bridge

Use this on most computers.

uvx figmin-bridge

NVIDIA GPU bridge

Use this on PCs with NVIDIA graphics for faster local speech recognition and voice replies.

uvx figmin-bridge-gpu

Important: Keep the bridge running while you use Figmin XR with your AI assistant. It is what lets your assistant connect to Figmin XR.

2

Wait Until the Bridge Is Ready

The first launch may take a few minutes while files are downloaded. When you see the ready message, note the address shown in your terminal.

Figmin XR bridge terminal ready message with the connection address highlighted

Your bridge will show your own address. Enter the address shown in your terminal, not the one in this example.

3

Connect From Figmin XR

In Figmin XR, open the MCP Agent app, enter the IP address from the bridge, and press Connect. Your headset and the computer running the bridge must be on the same local network.

You can find the MCP Agent app in the Featured section of the Figmin XR Discover panel.

4

Ask Your Agent to Join

Once Figmin XR says it is connected, ask your assistant to join the session. Try the prompt below.

Please connect to Figmin XR with MCP and start a realtime chat with me.

Before we begin, please read the Figmin XR help tool thoroughly so you understand how it all works.

For best results, make sure your assistant studies the MCP help tool before it starts. Even better, give it the Figmin XR agent skill so it has a focused guide to build from.

Level Up Your Agent

Give your assistant the right references, examples, and app patterns so it can build more confidently inside Figmin XR.

Download Agent Skill Give your assistant a focused Figmin XR guide before it starts building. Example Apps Use example projects as prompts and starting points Explore the API Learn what your assistant can build and control

Troubleshooting

If something does not work the first time, these are the most common places to check. If you still need help, join us on Discord.

Command not found

Make sure uv installed successfully, then close and reopen your terminal before trying again.

AI Client Configuration

If your AI client is not configured correctly, ask your assistant to help set up its MCP configuration. The goal is for your AI client to be able to run uvx figmin-mcp.

Figmin XR not connecting

Make sure the bridge is running on your PC and that Figmin XR is trying to connect to that same machine on the same local network.

Voice Replies Are Slow

If a chat bubble appears but the spoken reply arrives several seconds later, text to speech is probably running on your CPU. If you cannot use GPU acceleration, ask your assistant to reply without text to speech.

Changing Language

To speak in a language other than English, open dictation and use the language button in the lower-left corner. If spoken replies are not pronounced correctly, ask your assistant to switch to a voice that supports your language.

Agent Needs More Guidance

Ask your assistant to carefully read the Figmin XR help tool, or configure your AI client with the Figmin XR agent skill file. This gives it more context before it tries to build or edit your scene.

Spatial App Errors

If your assistant is building a spatial app and something is not working, open Settings (click the cog button), enable the Console toggle, reload the app, and copy any errors back to your assistant.

Uninstalling the Bridge

Close any running bridge terminal, then remove the cached bridge package with uv cache clean figmin-bridge or uv cache clean figmin-bridge-gpu. To remove the MCP server from your AI client, ask your assistant how to remove figmin-mcp from that client's configuration.