Skip to content
Blog

Verifying A2A Interoperability in Your Test Suite

Turn A2A protocol conformance into CI: a pytest suite that validates Agent Cards, JSON-RPC envelopes, task state transitions, SSE streaming, and auth headers against any A2A server before a partner ever connects.

Published on • November 16, 2026

AI Assistant

Interop demos are easy and interop regressions are quiet. Two teams integrate over A2A, the happy path works in the demo, and then a quiet mismatch surfaces in production: your client sends taskId in snake_case while the new server build expects camelCase, a card loses its capabilities.streaming flag, a terminal state name drifts, or an SSE stream keeps emitting after completed. Nobody’s code “crashed” - the two sides simply stopped agreeing about the contract.

The fix is boring and effective: make protocol conformance a test suite. The A2A repository defines the contract precisely enough to assert against - the Agent Card schema, the JSON-RPC 2.0 envelope with PascalCase methods (SendMessage, GetTask, CancelTask, SubscribeToTask), the task state machine, SSE event shapes, and the well-known discovery URL. In this tutorial you will build a reusable pytest suite that verifies any A2A server - yours, a partner’s staging endpoint, or a reference sample - and wire it into CI so interoperability breakage fails the build instead of the integration.

Prerequisites

  • Python 3.11+ with pytest and httpx (pip install pytest httpx)
  • An A2A server to test: run the reference Helloworld sample from the A2A repository, or point the suite at your own
  • A passing familiarity with JSON-RPC 2.0 and the A2A task lifecycle
  • (Optional) jsonschema for full schema validation: pip install jsonschema

What “Interoperable” Means, Operationally

Interoperability is not a vibe; it is a set of checkable assertions. For an A2A server, the testable surface is:

LayerContractFailure it catches
DiscoveryGET /.well-known/agent-card.json returns a valid cardClients cannot find you at all
EnvelopeJSON-RPC 2.0 request/response with PascalCase methodsWrong method names, missing id/jsonrpc
Card semanticsRequired fields, declared capabilities match behaviorClaiming streaming but 404-ing on it
Task state machineLegal transitions only, terminal states terminalEmitting completed -> working
Message/payload shaperole, parts[].type, taskId/contextId namingSilent misparses across SDKs
Transport behaviorsSSE framing, error codes, auth challengesStreams that never close, unauthenticated servers accepting traffic

Structure the suite around exactly these six layers.

Layer 1: Discovery Tests

Start where every client starts:

import asyncio
import json

import httpx
import pytest

BASE = "http://127.0.0.1:9999"

async def get_card() -> dict:
    async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
        r = await c.get("/.well-known/agent-card.json")
        assert r.status_code == 200, f"card endpoint returned {r.status_code}"
        assert "application/json" in r.headers["content-type"]
        return r.json()

@pytest.mark.asyncio
async def test_card_is_reachable_at_well_known_url():
    card = await get_card()
    assert card, "card must not be empty"

@pytest.mark.asyncio
async def test_card_required_fields():
    card = await get_card()
    for field in ("name", "description", "version", "url", "skills"):
        assert field in card, f"card missing required field: {field}"
    assert isinstance(card["skills"], list) and card["skills"], "at least one skill"
    for skill in card["skills"]:
        for field in ("id", "name", "description"):
            assert field in skill, f"skill missing {field}"

@pytest.mark.asyncio
async def test_card_cache_headers():
    async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
        r1 = await c.get("/.well-known/agent-card.json")
        etag = r1.headers.get("etag")
        assert etag, "cards should carry an ETag for cheap revalidation"
        r2 = await c.get("/.well-known/agent-card.json",
                         headers={"If-None-Match": etag})
        assert r2.status_code in (200, 304), "conditional requests must be honored"

The last test is one partners notice: without ETag/Cache-Control, every orchestrator re-downloads every card on every discovery sweep.

Layer 2: JSON-RPC Envelope Tests

Every method call must be standard JSON-RPC with the protocol’s method names. Verify both directions - correct requests produce correct envelopes, and deliberately malformed requests produce standard errors:

async def rpc(method: str, params: dict | None = None,
              headers: dict | None = None, expect_error: bool = False) -> dict:
    payload = {"jsonrpc": "2.0", "id": "test-1", "method": method,
               "params": params or {}}
    async with httpx.AsyncClient(base_url=BASE, timeout=30) as c:
        r = await c.post("/rpc", json=payload, headers=headers or {})
        body = r.json()
        if expect_error:
            assert "error" in body, "malformed request must return an error object"
            assert "code" in body["error"] and "message" in body["error"]
        else:
            assert "error" not in body, f"unexpected error: {body.get('error')}"
            assert body["jsonrpc"] == "2.0"
            assert body["id"] == "test-1", "response must echo the request id"
        return body

@pytest.mark.asyncio
async def test_unknown_method_is_jsonrpc_error():
    await rpc("NoSuchMethod", expect_error=True)

@pytest.mark.asyncio
async def test_get_task_on_missing_id_is_error():
    await rpc("GetTask", {"id": "definitely-not-a-task"}, expect_error=True)

@pytest.mark.asyncio
async def test_method_names_are_pascal_case():
    # sending the legacy snake_case name must NOT silently succeed
    await rpc("tasks/send", {"message": {}}, expect_error=True)

The final assertion is the cheap one that catches accidental dual-support: if your server still answers tasks/send, document it as a compatibility shim or fail it - silent acceptance of both conventions is how clients end up depending on the wrong one.

Layer 3: Capability-Claim Tests

A card that lies is worse than a card that omits. Test that declared capabilities are real:

@pytest.mark.asyncio
async def test_declared_streaming_actually_streams():
    card = await get_card()
    streaming = (card.get("capabilities") or {}).get("streaming", False)
    payload = {"jsonrpc": "2.0", "id": 1, "method": "SendStreamingMessage",
               "params": {"message": {"role": "ROLE_USER",
                                       "parts": [{"type": "text", "text": "hi"}]}}}
    async with httpx.AsyncClient(base_url=BASE, timeout=30) as c:
        async with c.stream("POST", "/rpc", json=payload) as r:
            if not streaming:
                pytest.skip("server does not declare streaming")
            assert r.headers["content-type"].startswith("text/event-stream")
            frames = [line async for line in r.aiter_lines()
                      if line.startswith("data:")]
            assert frames, "declared streaming but produced no SSE frames"

Run the same pattern for pushNotifications (attempt CreateTaskPushNotificationConfig - if the card claims support, the call must not be rejected as unsupported) and extendedAgentCard (GetExtendedAgentCard must not return UnsupportedOperationError).

Layer 4: Task State Machine Tests

The state machine is the heart of the contract. Build a helper that drives a real task and records states, then assert legal transitions and terminality:

LEGAL = {
    "TASK_STATE_SUBMITTED": {"TASK_STATE_WORKING", "TASK_STATE_INPUT_REQUIRED",
                             "TASK_STATE_AUTH_REQUIRED", "TASK_STATE_COMPLETED",
                             "TASK_STATE_FAILED", "TASK_STATE_REJECTED",
                             "TASK_STATE_CANCELED"},
    "TASK_STATE_WORKING": {"TASK_STATE_INPUT_REQUIRED", "TASK_STATE_AUTH_REQUIRED",
                           "TASK_STATE_COMPLETED", "TASK_STATE_FAILED",
                           "TASK_STATE_CANCELED"},
    "TASK_STATE_INPUT_REQUIRED": {"TASK_STATE_WORKING", "TASK_STATE_COMPLETED",
                                  "TASK_STATE_FAILED", "TASK_STATE_CANCELED"},
    "TASK_STATE_AUTH_REQUIRED": {"TASK_STATE_WORKING", "TASK_STATE_FAILED",
                                 "TASK_STATE_CANCELED"},
}
TERMINAL = {"TASK_STATE_COMPLETED", "TASK_STATE_FAILED",
            "TASK_STATE_CANCELED", "TASK_STATE_REJECTED"}

async def drive_task(text: str) -> tuple[list[str], dict]:
    states, task = [], None
    body = await rpc("SendMessage",
                     {"message": {"role": "ROLE_USER",
                                  "parts": [{"type": "text", "text": text}]}})
    task = body["result"].get("task")
    if task:
        states.append(task["status"]["state"])
        tid = task["id"]
        for _ in range(30):
            if states[-1] in TERMINAL or states[-1] == "TASK_STATE_INPUT_REQUIRED":
                break
            await asyncio.sleep(0.5)
            got = await rpc("GetTask", {"id": tid})
            state = got["result"]["status"]["state"]
            if state != states[-1]:
                states.append(state)
    return states, task

@pytest.mark.asyncio
async def test_task_reaches_a_legal_terminal_or_input_state():
    states, task = await drive_task("Say hello.")
    assert states, "task never reported a state"
    for prev, cur in zip(states, states[1:]):
        allowed = LEGAL.get(prev, set())
        assert cur in allowed, f"illegal transition {prev} -> {cur}"
    assert states[-1] in TERMINAL | {"TASK_STATE_INPUT_REQUIRED"}

@pytest.mark.asyncio
async def test_terminal_tasks_stay_terminal():
    states, task = await drive_task("Say hello.")
    if task and states[-1] in TERMINAL:
        await asyncio.sleep(1)
        again = await rpc("GetTask", {"id": task["id"]})
        assert again["result"]["status"]["state"] == states[-1], \
            "terminal task changed state on re-read"

Layer 5: Payload Shape and Multi-Turn Tests

Cross-SDK breakage usually lives in field naming and continuation semantics:

@pytest.mark.asyncio
async def test_task_payload_shape():
    body = await rpc("SendMessage",
                     {"message": {"role": "ROLE_USER",
                                  "parts": [{"type": "text", "text": "hi"}]}})
    result = body["result"]
    assert "task" in result or "message" in result, \
        "SendMessage result must contain a task or message"
    task = result.get("task")
    if task:
        for field in ("id", "contextId", "status"):
            assert field in task, f"task missing camelCase field: {field}"
        assert "state" in task["status"]

@pytest.mark.asyncio
async def test_followup_continues_the_same_task():
    body = await rpc("SendMessage",
                     {"message": {"role": "ROLE_USER",
                                  "parts": [{"type": "text", "text": "hi"}]}})
    task = body["result"]["task"]
    follow = await rpc("SendMessage", {"message": {
        "role": "ROLE_USER",
        "task_id": task["id"],
        "context_id": task["contextId"],
        "parts": [{"type": "text", "text": "more detail"}]}})
    continued = follow["result"].get("task")
    if continued:  # if the server uses Message-only responses, skip gracefully
        assert continued["id"] == task["id"], \
            "follow-up without preserved id started a new task"

Layer 6: Transport and Auth Tests

@pytest.mark.asyncio
async def test_unauthenticated_request_is_rejected_with_challenge():
    card = await get_card()
    if not card.get("security"):
        pytest.skip("server declares no security requirement")
    async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
        r = await c.post("/rpc", json={"jsonrpc": "2.0", "id": 1,
                                        "method": "SendMessage",
                                        "params": {"message": {}}})
        if r.status_code == 401:
            assert r.headers.get("www-authenticate"), \
                "401 must carry a challenge per the spec"
        else:
            # JSON-RPC binding may surface auth failures as error objects
            # instead of HTTP 401; either shape is spec-legal, silence is not.
            body = r.json()
            assert "error" in body, \
                "unauthenticated SendMessage must be rejected somehow"

@pytest.mark.asyncio
async def test_sse_stream_closes_at_terminal_state():
    card = await get_card()
    if not (card.get("capabilities") or {}).get("streaming"):
        pytest.skip("no streaming")
    payload = {"jsonrpc": "2.0", "id": 1, "method": "SendStreamingMessage",
               "params": {"message": {"role": "ROLE_USER",
                                       "parts": [{"type": "text", "text": "hi"}]}}}
    async with httpx.AsyncClient(base_url=BASE, timeout=60) as c:
        async with c.stream("POST", "/rpc", json=payload) as r:
            last_state = None
            async for line in r.aiter_lines():
                if line.startswith("data:"):
                    event = json.loads(line[5:])
                    result = event.get("result", {})
                    st = (result.get("statusUpdate") or {}).get("status") or {}
                    if st.get("state"):
                        last_state = st["state"]
            assert last_state in TERMINAL, \
                "stream must close after reaching a terminal state"

This single test catches the most common streaming bug in the wild: servers that keep the SSE connection open after completed, leaving clients hanging until timeout.

Wiring It Into CI

The suite runs against a URL, so CI is straightforward - start the server, wait for the card, run the suite:

# .github/workflows/a2a-interop.yml
name: a2a-interop
on: [push, pull_request]
jobs:
  interop:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: {python-version: "3.12"}
      - run: pip install a2a-sdk httpx pytest pytest-asyncio jsonschema
      - run: python -m my_server &        # your A2A server
      - name: Wait for Agent Card
        run: |
          for i in $(seq 1 30); do
            curl -sf http://127.0.0.1:9999/.well-known/agent-card.json && exit 0
            sleep 1
          done; exit 1
      - run: pytest tests/a2a_interop -q
        env:
          A2A_BASE_URL: http://127.0.0.1:9999

Parameterize BASE from A2A_BASE_URL and the same suite becomes a cross-organization contract test: run your client half against a partner’s staging URL on a schedule, and you will learn about their breaking change before their production traffic does.

Two CI habits that pay off:

  • Version the suite alongside the spec. When the protocol adds a method or field, add its assertions in the same PR that bumps your SDK.
  • Fail on card diffs. Fetch the card in CI, diff against a committed snapshot, and require review for removed skills, changed version, or dropped capabilities - the card is a public API.

Putting It All Together

Layout:

tests/
  a2a_interop/
    conftest.py        # BASE from env, shared rpc() helper
    test_discovery.py
    test_envelope.py
    test_capabilities.py
    test_state_machine.py
    test_payloads.py
    test_transport.py

conftest.py holds the rpc() helper and card fixture; each file owns one contract layer. Total: roughly 150 lines that convert “we think we’re compatible” into a green check on every commit.

Conclusion & Next Steps

A2A’s value is interoperability, and interoperability is only real if it is asserted. By testing discovery, the JSON-RPC envelope, capability honesty, the task state machine, payload shape, and transport behavior, you turn the specification into a gate - one that catches contract drift in your build, and with the same suite pointed at a partner URL, catches their drift before your users do.

Next steps:

  • Add schema-level validation with jsonschema against the spec’s published types.
  • Run the suite as a scheduled job against partner staging endpoints.
  • Extend the suite with a fuzz pass over SendMessage params to find unhandled validation errors.

Sources: A2A Protocol repository (google/A2A), A2A Protocol Specification, A2A Python SDK