Use Verifyax with Claude

Run AI agent evaluations from Claude. Describe the test you want in plain language. Claude registers your agent, generates scenarios, runs the simulation and returns the scores with a full transcript.

This guide covers Claude.ai, the Claude desktop app, Cowork and Claude Code.

Last updated 2 October 2026. Current plugin versions: verifyax-mcp 0.3.5, verifyax-api 0.3.0, verifyax-claude-agent 0.1.2. The verifyax-mcp plugin pins @verifyax/mcp-server 0.3.5. The npm package and the hosted server can be newer: on 2 October 2026, npm @verifyax/mcp-server was 0.3.7.

Before you start

You need four things:

  1. 1.A VerifyAX account. Sign up or log in at console.verifyax.com.
  2. 2.An API key. In the console, go to Settings > API Keys and create a key. Keys start with sk-ver-api-. The console shows the full key once, so copy it straight away.
  3. 3.Credits in your workspace. Generating scenarios and running simulations use credits. If your balance is empty, runs fail with a payment error.
  4. 4.A paid Claude plan (Pro, Max, Team or Enterprise) to use plugins and connectors. On Team and Enterprise, an owner or admin must first enable them for your organization.

Set up in Claude.ai

Install the plugin

  1. 1.In Claude.ai, open Customize > Plugins.
  2. 2.Add the VerifyAX marketplace: verifyax/verifyax-plugins-claude
  3. 3.Install verifyax-mcp for conversational use, or verifyax-api for scripts and code.
  4. 4.Start a new chat and ask Claude to use VerifyAX.

Or add the MCP server as a connector

Prefer not to install a plugin? Add the hosted MCP server as a custom connector. This route needs no Node.js. On Team and Enterprise plans, only an organization owner can add a custom connector. If the option is greyed out, ask an owner to add it for your team.

  1. 1.In Claude.ai, open Customize > Connectors, click Add, then choose Add custom connector.
  2. 2.Enter the server URL: https://mcp.verifyax.com/mcp
  3. 3.Set Authentication to No sign-in. The server uses your API key, not OAuth.
  4. 4.Add a request header named Authorization with the value Bearer sk-ver-api-..., using your own API key. This needs VerifyAX MCP server 0.3.5 or later.

Fallback: upload the skill manually

  1. 1.Download the VerifyAX skill file for verifyax-api 0.3.0: verifyax-api.zip
  2. 2.In Claude.ai, open Customize > Skills and click Add.
  3. 3.Upload the file and switch the skill on.

Set up in the Claude desktop app and Cowork

The desktop app and Cowork connect to the hosted VerifyAX MCP server through mcp-remote, which needs Node.js 18 or later. To skip Node.js entirely, add the custom connector described under Set up in Claude.ai. It works here too.

  1. 1.Open Settings > Developer > Edit Config.
  2. 2.Add this to claude_desktop_config.json, replacing the key with your own:
JSON
{
  "mcpServers": {
    "verifyax": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://mcp.verifyax.com/mcp",
        "--transport",
        "http-only",
        "--header",
        "Authorization:${VERIFYAX_AUTH}"
      ],
      "env": {
        "VERIFYAX_AUTH": "Bearer sk-ver-api-..."
      }
    }
  }
}
  1. 3.Save the file and restart the app.

You can send your key in an X-VerifyAX-API-Key header instead of Authorization: Bearer. Change the header line to X-VerifyAX-API-Key:${VERIFYAX_AUTH} and set VERIFYAX_AUTH to the key alone, without the Bearer prefix.

Set up in Claude Code

1. Add the VerifyAX marketplace

COMMAND
/plugin marketplace add verifyax/verifyax-plugins-claude

This adds the marketplace from GitHub. After that, Claude Code refers to it as verifyax-plugins.

2. Install the plugin you need

COMMAND
/plugin install verifyax-mcp@verifyax-plugins
/plugin install verifyax-api@verifyax-plugins
/plugin install verifyax-claude-agent@verifyax-plugins

verifyax-claude-agent installs verifyax-api automatically.

3. Add your API key

verifyax-mcp asks for your key when you enable the plugin and stores it securely.

verifyax-api reads your key from an environment variable:

TERMINAL
export VERIFYAX_API_KEY=sk-ver-api-...

verifyax-claude-agent uses the same VERIFYAX_API_KEY. It also needs the Claude Code CLI and Python. Install the Python dependencies in your terminal:

TERMINAL
pip install -r ~/.claude/plugins/marketplaces/verifyax-plugins/plugins/verifyax-claude-agent/adapter/requirements.txt

Then start the guided setup in Claude Code:

COMMAND
/verifyax-claude-agent:connect-to-verifyax

The plugin opens a temporary public tunnel to your agent (see Security below), then runs the evaluation. It offers two modes, tools off and tools on. The full guide on GitHub explains both and lists exactly what gets sent to VerifyAX.

Prefer to add the MCP server directly?

This route runs the MCP server locally and needs Node.js 20 or later.

TERMINAL
claude mcp add verifyax --env VERIFYAX_API_KEY=sk-ver-api-... -- npx -y -p @verifyax/mcp-server verifyax-mcp-server

Run your first evaluation

Once the plugin is set up, describe what you want. For example:

Using VerifyAX, register my A2A agent at https://my-agent.example.com, generate one info_exchange scenario tagged empathy, run my agent against it and show me the scores when it's done.

Claude then:

  1. 1.Registers your agent and checks it responds.
  2. 2.Generates the scenario and waits until it's ready.
  3. 3.Runs your agent through the scenario.
  4. 4.Scores the run.
  5. 5.Returns the scores and the transcript.

Scenario generation and evaluation take between 30 seconds and 5 minutes. Wait for the result rather than asking again.

More prompts to try

  • •"Register this A2A agent on VerifyAX and run a quick interview-style evaluation."
  • •"List my failed simulations from the last week and show the error details for each."
  • •"Generate 5 info_exchange scenarios with the empathy and active_listening tags."
  • •"How many credits will this run cost?"

Ask Claude to estimate credits before a run and it shows you the cost first.

Glossary

Agent
The AI endpoint you're testing. An A2A or REST service you register with VerifyAX.
Scenario
The test environment your agent runs through. Two types: info_exchange (several agents, the default) and interview (one to one).
Skill tag
A label such as empathy or active_listening that sets what a scenario measures.
Simulation run
One run of your agent through a scenario. Produces a transcript.
Evaluation
The score for a completed run, measured against the scenario's expected outcome.
Job
A tracker for longer tasks. Status moves from PENDING to PROCESSING, then COMPLETED, FAILED or CANCELLED.
Credits
What your workspace spends on scenario generation, simulations and evaluations.

Every evaluation follows the same order: register agent, create scenario, run simulation, evaluate, review results.

Keep your plugins up to date

In Claude Code, refresh the marketplace, then reinstall each plugin you use. Installed plugins do not update themselves.

COMMAND
/plugin marketplace update verifyax-plugins
/plugin install verifyax-mcp@verifyax-plugins
/plugin install verifyax-api@verifyax-plugins
/plugin install verifyax-claude-agent@verifyax-plugins

If you uploaded the skill manually in Claude.ai, download verifyax-api.zip again and upload it in place of the old one. This link points to verifyax-api 0.3.0, and we update it here each time verifyax-api gets a new release.

Security

  • •Your API key stays in your own client config or environment. The hosted MCP server does not store keys and sends no telemetry.
  • •Claude never needs your API key in the chat. If anything asks you to paste it into a conversation, don't. Keys belong in your client config or environment only.
  • •Every action runs inside the permissions of the workspace tied to your key.
  • •Revoke a key in the console under Settings > API Keys to cut off access at once.
  • •Deleting an agent or scenario is permanent. Check any delete request before you approve it.
  • •verifyax-claude-agent makes your local agent reachable through a temporary Cloudflare tunnel so VerifyAX can call it. Only the agent adapter is exposed, and the tunnel URL lasts only while the connect session runs. It changes on every run unless you set a fixed URL.
  • •In tools-off mode (the default), the agent can't run commands, read or edit files, or reach the network, so VerifyAX can only talk to it. Tools-on mode lets the agent use tools without approval prompts. Only use it inside the plugin's sandbox folder.

Troubleshooting

"VERIFYAX_API_KEY is not set"
Your key isn't reaching VerifyAX. Check the environment variable, or the env block in your config file.
Authentication failed
The key is wrong, revoked or from a different environment. Create a new key in the console.
VerifyAX tools don't appear
Claude loads tools at startup. Restart the app or start a new Claude Code session.
Payment error
Your workspace is out of credits. Top up in the console.

Get help