MCP vs. Graph: Benchmarking Cross-Domain AI Agent Tasks (v2)

We Compared MCPs and Graph Queries on the Same Tasks

During an incident or deployment review, agents must connect records across tools without confusing production with test copies. We gave both agents the same ten questions: one used native MCPs, the other Anyshift graph tools with prepared queries.

The environment included AWS, Kubernetes, Datadog, GitHub and PagerDuty. Each task used two to four systems.

Our first benchmark compared the graph with Kubernetes and AWS MCPs. On average, the graph used 86.1% fewer tokens for PR review (13,888 vs. 99,634) and 81.3% fewer for node-impact analysis (21,824 vs. 116,451). Both setups met the answer requirements in all three runs per task and setup.

A separate test added Datadog: the graph used 88.0% fewer tokens (25,033 vs. 209,188), but MCPs checked extra evidence.

The Agents Retrieved Data in Two Ways

An agent tracing an alert must connect it to an application and its host. We tested two retrieval methods:

1. Native MCPs: the agent queried each source and joined the responses. We suggested an investigation route; the agent chose its calls. MCP provides the interface for those calls.

2. Anyshift graph: the agent inspected the graph’s data structure and ran Cypher query. Cypher is the graph’s query language. The agent used the returned records to answer.

We Measured Tokens, Time and Tool Calls

We measured tokens, time and tool calls from tool startup to final answer. Tokens count the text a model reads and generates. We checked each answer against source data.

The averages cover ten pairs. A Datadog connection failure required one paired retry, bringing the total to 22 sessions. We kept that retry and the other nine original pairs, including incomplete answers.

Test 1: Find a Queue’s Applications and Hosting Machines

In this example, we tested whether each agent could find the applications using a message queue and the machines hosting them.

Why it is important: A message backlog can delay orders or background jobs. Identifying the senders, receivers and their machines helps engineers decide where to investigate the delay.

Both agents traced a RabbitMQ queue, which holds messages between applications, through Kubernetes to its AWS hosting machines.

Which apps use this queue, and where do they run?

Native MCPs

Separate source records

Queue activity

Apps & pods

Machines

↓

Agent finds and joins the records

16 calls · 147.7 s284,762 tokens

Graph

Relationships stored before the task

2 senders→Queue→1 receiver

Apps run on ↓

Pods→Node→EC2

↓

Agent runs a prepared query

2 calls · 61.7 s57,292 tokens

Both found: 2 senders, 1 receiver and their hosting machines.

Both agents found the same two senders, one receiver and hosting machines. Source checks confirmed these roles for the requested period.

The graph agent used two calls, versus 16 native calls, for matching evidence.

Test 2: Match Each App Copy to Its Incident Service

In this example, we tested whether each agent could distinguish applications with the same name across environments and find their incident-service connections.

Why it is important: Production and test applications can share the same name. Matching each copy to the correct incident service helps responders investigate the right environment.

Both agents separated three checkout-api copies by Kubernetes location and identifier, then found their records in PagerDuty, which manages incidents.

Which incident service belongs to each app?

Native MCPs

Separate source records

3 app copies

Incident services

↓

Agent finds and joins the records

33 calls · 156.3 s381,556 tokens

Graph

Relationships stored before the task

Production→Service A

Demo-prod→Service B

Staging→Unknown

↓

Agent runs a prepared query

2 calls · 53.4 s56,101 tokens

Both found: 3 distinct apps, 2 incident-service links, 1 unknown.

Both agents separated the three applications, found two PagerDuty records and left the third connection unknown.

Both Agents Used the Same Model and Time Limit

We held the settings constant, started fresh sessions and verified answers against source systems.

Most monitoring checks used the same two-hour period; historical questions used a separate fixed period. The checked resource layout stayed stable while Anyshift continued collecting data.

Token Counts Do Not Directly Measure Cost

Teams budgeting agent tasks must account for cheaper cached input. Our counts include that input, so 83% fewer tokens does not imply 83% less spend. We did not calculate dollar savings or include graph collection, storage and maintenance costs.

We Compared Time and Tokens Across All Ten Tasks

Each pair answered the same question. Choose Tokens or Time to compare the two methods. Open Why it matters to see the decision each task supports.