Testing WebMCP contracts in CI

WebMCP has a failure mode that makes automated testing more valuable than usual: when it breaks, nothing tells you.

The entry point that tool registration hangs off was renamed during incubation. Code written against the old name does not throw. It registers nothing. Your page loads, your console is clean, your existing tests pass, and to an agent your site simply has no tools, exactly like a site that never implemented WebMCP at all.

A build is the only thing that will notice.

Accurate as of July 2026. WebMCP is a Chrome origin trial and the API can still change, which is itself the argument for this.

Why unit tests are not enough

You can unit test the object you pass to provideContext. That confirms your schema is well formed, and it confirms nothing about whether registration actually happened in a browser.

The three things that break are all runtime and environmental:

• Calling an entry point that no longer exists
• Registering too late, after an agent would have looked
• A build or bundling change that drops the registration entirely

None of these are visible from the source. You have to load the page and ask what is registered.

The shape of the test

Load the page in a real browser, read back the registered tools, and assert on them.

// tests/webmcp.spec.js — Playwright
import { test, expect } from '@playwright/test'

/**
 * Reads the tools the page actually registered. Returns null when the API
 * itself is absent, which distinguishes "the browser does not support this"
 * from "we registered nothing".
 */
async function registeredTools (page) {
  return page.evaluate(() => {
    if (!('modelContext' in navigator)) return null
    return (navigator.modelContext.getRegisteredTools?.() ?? [])
      .map(({ name, description, inputSchema }) => ({ name, description, inputSchema }))
  })
}

test('the page registers its tools', async ({ page }) => {
  await page.goto('/')

  const tools = await registeredTools(page)

  // The failure this whole file exists to catch.
  expect(tools, 'navigator.modelContext is unavailable in this browser').not.toBeNull()
  expect(tools.length, 'no tools registered: check the entry point name').toBeGreaterThan(0)

  expect(tools.map(t => t.name)).toEqual(
    expect.arrayContaining(['searchProducts', 'addToCart'])
  )
})

The messages on those assertions matter. A bare expected 0 to be greater than 0 at three in the morning tells you nothing. no tools registered: check the entry point name points straight at the most likely cause.

Test the contract, not just the presence

A tool that exists with a useless description is barely better than no tool. An agent choosing between several reads the descriptions, so treat them as part of the contract.

test('every tool is described well enough to be chosen', async ({ page }) => {
  await page.goto('/')
  const tools = await registeredTools(page)

  for (const tool of tools) {
    expect(tool.description, `${tool.name} has no description`).toBeTruthy()

    // "Search" tells a model nothing. Set a floor.
    expect(tool.description.length,
      `${tool.name} description is too short to choose between tools`
    ).toBeGreaterThan(40)

    expect(tool.inputSchema?.type, `${tool.name} has no input schema`).toBe('object')

    // Every property needs its own description, or the model guesses.
    for (const [prop, def] of Object.entries(tool.inputSchema.properties ?? {})) {
      expect(def.description, `${tool.name}.${prop} is undocumented`).toBeTruthy()
    }
  }
})

Guard against silent removal

The most valuable assertion is the one that fires when a tool disappears. Snapshot the tool names and fail on drift.

// A deliberate list. Changing it should be a decision, not a side effect.
const EXPECTED_TOOLS = ['searchProducts', 'getProduct', 'addToCart', 'checkStock']

test('the tool surface has not changed unintentionally', async ({ page }) => {
  await page.goto('/')
  const names = (await registeredTools(page)).map(t => t.name).sort()

  expect(names,
    'The registered tools differ from the expected set. If this is intended, update EXPECTED_TOOLS.'
  ).toEqual([...EXPECTED_TOOLS].sort())
})

This is what catches the bundling change that quietly drops your registration module.

Registration timing

An agent that inspects the page before your late callback runs sees nothing. Assert that tools are present early rather than eventually.

test('tools are registered without waiting for full page load', async ({ page }) => {
  await page.goto('/', { waitUntil: 'domcontentloaded' })

  // No networkidle, no arbitrary sleep: if this needs more time than
  // domcontentloaded, registration is too late.
  const tools = await registeredTools(page)
  expect(tools?.length ?? 0).toBeGreaterThan(0)
})

Resist adding a waitForTimeout to make this pass. If it only passes with a sleep, the underlying problem is that you register too late, and an agent will hit exactly that.

Wiring it up

name: WebMCP contracts
on: [push, pull_request]

jobs:
  webmcp:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test tests/webmcp.spec.js

Because WebMCP is behind an origin trial, your test browser may need the feature enabled explicitly. Launch Chromium with the appropriate flag rather than skipping the tests, since a skipped test provides exactly the same false comfort as no test.

Also check the deployed site

Tests confirm what your application intends to serve. They do not confirm what your infrastructure actually serves, because CDNs, edge rules and content security policies are not present in a test run. A CSP that blocks your registration script produces a perfectly green test suite and a live site with no tools.

So run a check against the real URL after deploy:

- name: Verify WebMCP on the deployed site
  run: npx agentcheck https://example.com --spec webmcp --fail-on error

Our WebMCP checker does the same thing interactively, loading your page in a real browser and listing the tools an agent would actually receive. Worth running once by hand before you automate it, so you know what a good result looks like.

What to assert, in short

1. navigator.modelContext exists in the test browser
2. At least one tool is registered
3. The expected tool names are all present
4. No unexpected tools have appeared or vanished
5. Every tool has a description with real content
6. Every schema property is documented
7. Registration completes by domcontentloaded
8. The deployed URL agrees with the test environment

Points three and four are the ones that will actually save you, because they fail loudly on precisely the change that otherwise fails silently.

Read next

• How to expose WebMCP tools on your site — writing the tools in the first place
• Why agent access breaks silently — the wider pattern this belongs to
• UCP integration checklist for Laravel — the same discipline on the commerce side