Verifying A2A Interoperability in Your Test Suite
Turn A2A protocol conformance into CI: a pytest suite that validates Agent Cards, JSON-RPC envelopes, task state transitions, SSE streaming, and auth headers against any A2A server before a partner ever connects.
Published on • November 16, 2026
AI Assistant

Interop demos are easy and interop regressions are quiet. Two teams integrate over A2A, the happy path works in the demo, and then a quiet mismatch surfaces in production: your client sends taskId in snake_case while the new server build expects camelCase, a card loses its capabilities.streaming flag, a terminal state name drifts, or an SSE stream keeps emitting after completed. Nobody’s code “crashed” - the two sides simply stopped agreeing about the contract.
The fix is boring and effective: make protocol conformance a test suite. The A2A repository defines the contract precisely enough to assert against - the Agent Card schema, the JSON-RPC 2.0 envelope with PascalCase methods (SendMessage, GetTask, CancelTask, SubscribeToTask), the task state machine, SSE event shapes, and the well-known discovery URL. In this tutorial you will build a reusable pytest suite that verifies any A2A server - yours, a partner’s staging endpoint, or a reference sample - and wire it into CI so interoperability breakage fails the build instead of the integration.
Prerequisites
- Python 3.11+ with
pytestandhttpx(pip install pytest httpx) - An A2A server to test: run the reference Helloworld sample from the A2A repository, or point the suite at your own
- A passing familiarity with JSON-RPC 2.0 and the A2A task lifecycle
- (Optional)
jsonschemafor full schema validation:pip install jsonschema
What “Interoperable” Means, Operationally
Interoperability is not a vibe; it is a set of checkable assertions. For an A2A server, the testable surface is:
| Layer | Contract | Failure it catches |
|---|---|---|
| Discovery | GET /.well-known/agent-card.json returns a valid card | Clients cannot find you at all |
| Envelope | JSON-RPC 2.0 request/response with PascalCase methods | Wrong method names, missing id/jsonrpc |
| Card semantics | Required fields, declared capabilities match behavior | Claiming streaming but 404-ing on it |
| Task state machine | Legal transitions only, terminal states terminal | Emitting completed -> working |
| Message/payload shape | role, parts[].type, taskId/contextId naming | Silent misparses across SDKs |
| Transport behaviors | SSE framing, error codes, auth challenges | Streams that never close, unauthenticated servers accepting traffic |
Structure the suite around exactly these six layers.
Layer 1: Discovery Tests
Start where every client starts:
import asyncio
import json
import httpx
import pytest
BASE = "http://127.0.0.1:9999"
async def get_card() -> dict:
async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
r = await c.get("/.well-known/agent-card.json")
assert r.status_code == 200, f"card endpoint returned {r.status_code}"
assert "application/json" in r.headers["content-type"]
return r.json()
@pytest.mark.asyncio
async def test_card_is_reachable_at_well_known_url():
card = await get_card()
assert card, "card must not be empty"
@pytest.mark.asyncio
async def test_card_required_fields():
card = await get_card()
for field in ("name", "description", "version", "url", "skills"):
assert field in card, f"card missing required field: {field}"
assert isinstance(card["skills"], list) and card["skills"], "at least one skill"
for skill in card["skills"]:
for field in ("id", "name", "description"):
assert field in skill, f"skill missing {field}"
@pytest.mark.asyncio
async def test_card_cache_headers():
async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
r1 = await c.get("/.well-known/agent-card.json")
etag = r1.headers.get("etag")
assert etag, "cards should carry an ETag for cheap revalidation"
r2 = await c.get("/.well-known/agent-card.json",
headers={"If-None-Match": etag})
assert r2.status_code in (200, 304), "conditional requests must be honored"
The last test is one partners notice: without ETag/Cache-Control, every orchestrator re-downloads every card on every discovery sweep.
Layer 2: JSON-RPC Envelope Tests
Every method call must be standard JSON-RPC with the protocol’s method names. Verify both directions - correct requests produce correct envelopes, and deliberately malformed requests produce standard errors:
async def rpc(method: str, params: dict | None = None,
headers: dict | None = None, expect_error: bool = False) -> dict:
payload = {"jsonrpc": "2.0", "id": "test-1", "method": method,
"params": params or {}}
async with httpx.AsyncClient(base_url=BASE, timeout=30) as c:
r = await c.post("/rpc", json=payload, headers=headers or {})
body = r.json()
if expect_error:
assert "error" in body, "malformed request must return an error object"
assert "code" in body["error"] and "message" in body["error"]
else:
assert "error" not in body, f"unexpected error: {body.get('error')}"
assert body["jsonrpc"] == "2.0"
assert body["id"] == "test-1", "response must echo the request id"
return body
@pytest.mark.asyncio
async def test_unknown_method_is_jsonrpc_error():
await rpc("NoSuchMethod", expect_error=True)
@pytest.mark.asyncio
async def test_get_task_on_missing_id_is_error():
await rpc("GetTask", {"id": "definitely-not-a-task"}, expect_error=True)
@pytest.mark.asyncio
async def test_method_names_are_pascal_case():
# sending the legacy snake_case name must NOT silently succeed
await rpc("tasks/send", {"message": {}}, expect_error=True)
The final assertion is the cheap one that catches accidental dual-support: if your server still answers tasks/send, document it as a compatibility shim or fail it - silent acceptance of both conventions is how clients end up depending on the wrong one.
Layer 3: Capability-Claim Tests
A card that lies is worse than a card that omits. Test that declared capabilities are real:
@pytest.mark.asyncio
async def test_declared_streaming_actually_streams():
card = await get_card()
streaming = (card.get("capabilities") or {}).get("streaming", False)
payload = {"jsonrpc": "2.0", "id": 1, "method": "SendStreamingMessage",
"params": {"message": {"role": "ROLE_USER",
"parts": [{"type": "text", "text": "hi"}]}}}
async with httpx.AsyncClient(base_url=BASE, timeout=30) as c:
async with c.stream("POST", "/rpc", json=payload) as r:
if not streaming:
pytest.skip("server does not declare streaming")
assert r.headers["content-type"].startswith("text/event-stream")
frames = [line async for line in r.aiter_lines()
if line.startswith("data:")]
assert frames, "declared streaming but produced no SSE frames"
Run the same pattern for pushNotifications (attempt CreateTaskPushNotificationConfig - if the card claims support, the call must not be rejected as unsupported) and extendedAgentCard (GetExtendedAgentCard must not return UnsupportedOperationError).
Layer 4: Task State Machine Tests
The state machine is the heart of the contract. Build a helper that drives a real task and records states, then assert legal transitions and terminality:
LEGAL = {
"TASK_STATE_SUBMITTED": {"TASK_STATE_WORKING", "TASK_STATE_INPUT_REQUIRED",
"TASK_STATE_AUTH_REQUIRED", "TASK_STATE_COMPLETED",
"TASK_STATE_FAILED", "TASK_STATE_REJECTED",
"TASK_STATE_CANCELED"},
"TASK_STATE_WORKING": {"TASK_STATE_INPUT_REQUIRED", "TASK_STATE_AUTH_REQUIRED",
"TASK_STATE_COMPLETED", "TASK_STATE_FAILED",
"TASK_STATE_CANCELED"},
"TASK_STATE_INPUT_REQUIRED": {"TASK_STATE_WORKING", "TASK_STATE_COMPLETED",
"TASK_STATE_FAILED", "TASK_STATE_CANCELED"},
"TASK_STATE_AUTH_REQUIRED": {"TASK_STATE_WORKING", "TASK_STATE_FAILED",
"TASK_STATE_CANCELED"},
}
TERMINAL = {"TASK_STATE_COMPLETED", "TASK_STATE_FAILED",
"TASK_STATE_CANCELED", "TASK_STATE_REJECTED"}
async def drive_task(text: str) -> tuple[list[str], dict]:
states, task = [], None
body = await rpc("SendMessage",
{"message": {"role": "ROLE_USER",
"parts": [{"type": "text", "text": text}]}})
task = body["result"].get("task")
if task:
states.append(task["status"]["state"])
tid = task["id"]
for _ in range(30):
if states[-1] in TERMINAL or states[-1] == "TASK_STATE_INPUT_REQUIRED":
break
await asyncio.sleep(0.5)
got = await rpc("GetTask", {"id": tid})
state = got["result"]["status"]["state"]
if state != states[-1]:
states.append(state)
return states, task
@pytest.mark.asyncio
async def test_task_reaches_a_legal_terminal_or_input_state():
states, task = await drive_task("Say hello.")
assert states, "task never reported a state"
for prev, cur in zip(states, states[1:]):
allowed = LEGAL.get(prev, set())
assert cur in allowed, f"illegal transition {prev} -> {cur}"
assert states[-1] in TERMINAL | {"TASK_STATE_INPUT_REQUIRED"}
@pytest.mark.asyncio
async def test_terminal_tasks_stay_terminal():
states, task = await drive_task("Say hello.")
if task and states[-1] in TERMINAL:
await asyncio.sleep(1)
again = await rpc("GetTask", {"id": task["id"]})
assert again["result"]["status"]["state"] == states[-1], \
"terminal task changed state on re-read"
Layer 5: Payload Shape and Multi-Turn Tests
Cross-SDK breakage usually lives in field naming and continuation semantics:
@pytest.mark.asyncio
async def test_task_payload_shape():
body = await rpc("SendMessage",
{"message": {"role": "ROLE_USER",
"parts": [{"type": "text", "text": "hi"}]}})
result = body["result"]
assert "task" in result or "message" in result, \
"SendMessage result must contain a task or message"
task = result.get("task")
if task:
for field in ("id", "contextId", "status"):
assert field in task, f"task missing camelCase field: {field}"
assert "state" in task["status"]
@pytest.mark.asyncio
async def test_followup_continues_the_same_task():
body = await rpc("SendMessage",
{"message": {"role": "ROLE_USER",
"parts": [{"type": "text", "text": "hi"}]}})
task = body["result"]["task"]
follow = await rpc("SendMessage", {"message": {
"role": "ROLE_USER",
"task_id": task["id"],
"context_id": task["contextId"],
"parts": [{"type": "text", "text": "more detail"}]}})
continued = follow["result"].get("task")
if continued: # if the server uses Message-only responses, skip gracefully
assert continued["id"] == task["id"], \
"follow-up without preserved id started a new task"
Layer 6: Transport and Auth Tests
@pytest.mark.asyncio
async def test_unauthenticated_request_is_rejected_with_challenge():
card = await get_card()
if not card.get("security"):
pytest.skip("server declares no security requirement")
async with httpx.AsyncClient(base_url=BASE, timeout=10) as c:
r = await c.post("/rpc", json={"jsonrpc": "2.0", "id": 1,
"method": "SendMessage",
"params": {"message": {}}})
if r.status_code == 401:
assert r.headers.get("www-authenticate"), \
"401 must carry a challenge per the spec"
else:
# JSON-RPC binding may surface auth failures as error objects
# instead of HTTP 401; either shape is spec-legal, silence is not.
body = r.json()
assert "error" in body, \
"unauthenticated SendMessage must be rejected somehow"
@pytest.mark.asyncio
async def test_sse_stream_closes_at_terminal_state():
card = await get_card()
if not (card.get("capabilities") or {}).get("streaming"):
pytest.skip("no streaming")
payload = {"jsonrpc": "2.0", "id": 1, "method": "SendStreamingMessage",
"params": {"message": {"role": "ROLE_USER",
"parts": [{"type": "text", "text": "hi"}]}}}
async with httpx.AsyncClient(base_url=BASE, timeout=60) as c:
async with c.stream("POST", "/rpc", json=payload) as r:
last_state = None
async for line in r.aiter_lines():
if line.startswith("data:"):
event = json.loads(line[5:])
result = event.get("result", {})
st = (result.get("statusUpdate") or {}).get("status") or {}
if st.get("state"):
last_state = st["state"]
assert last_state in TERMINAL, \
"stream must close after reaching a terminal state"
This single test catches the most common streaming bug in the wild: servers that keep the SSE connection open after completed, leaving clients hanging until timeout.
Wiring It Into CI
The suite runs against a URL, so CI is straightforward - start the server, wait for the card, run the suite:
# .github/workflows/a2a-interop.yml
name: a2a-interop
on: [push, pull_request]
jobs:
interop:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: {python-version: "3.12"}
- run: pip install a2a-sdk httpx pytest pytest-asyncio jsonschema
- run: python -m my_server & # your A2A server
- name: Wait for Agent Card
run: |
for i in $(seq 1 30); do
curl -sf http://127.0.0.1:9999/.well-known/agent-card.json && exit 0
sleep 1
done; exit 1
- run: pytest tests/a2a_interop -q
env:
A2A_BASE_URL: http://127.0.0.1:9999
Parameterize BASE from A2A_BASE_URL and the same suite becomes a cross-organization contract test: run your client half against a partner’s staging URL on a schedule, and you will learn about their breaking change before their production traffic does.
Two CI habits that pay off:
- Version the suite alongside the spec. When the protocol adds a method or field, add its assertions in the same PR that bumps your SDK.
- Fail on card diffs. Fetch the card in CI, diff against a committed snapshot, and require review for removed skills, changed
version, or dropped capabilities - the card is a public API.
Putting It All Together
Layout:
tests/
a2a_interop/
conftest.py # BASE from env, shared rpc() helper
test_discovery.py
test_envelope.py
test_capabilities.py
test_state_machine.py
test_payloads.py
test_transport.py
conftest.py holds the rpc() helper and card fixture; each file owns one contract layer. Total: roughly 150 lines that convert “we think we’re compatible” into a green check on every commit.
Conclusion & Next Steps
A2A’s value is interoperability, and interoperability is only real if it is asserted. By testing discovery, the JSON-RPC envelope, capability honesty, the task state machine, payload shape, and transport behavior, you turn the specification into a gate - one that catches contract drift in your build, and with the same suite pointed at a partner URL, catches their drift before your users do.
Next steps:
- Add schema-level validation with
jsonschemaagainst the spec’s published types. - Run the suite as a scheduled job against partner staging endpoints.
- Extend the suite with a fuzz pass over
SendMessageparams to find unhandled validation errors.
Sources: A2A Protocol repository (google/A2A), A2A Protocol Specification, A2A Python SDK