Why an API is More Efficient Than an MCP for RAG
Okay, the title might be a bit controversial, I get it. That is precisely why I want to dedicate this post to presenting an argument based on variables that are relevant when deploying a RAG system and its use cases.
What is RAG?
RAG stands for Retrieval-Augmented Generation. It is a way to improve an LLM’s output by adding context outside of the data it was trained on. Often, this is highly confidential information that must be subject to compliance, depending on the jurisdiction.
Real-World Use Case
Picture a team of developers, all using agents to work on the same project, who generally use Markdown files to track progress utilizing spec-driven development.
The Problem?
This workflow works perfectly for independent developers or small teams. However, as the project grows, we must refine the filtering criteria to avoid injecting useless or even detrimental context. Let’s not forget that context is a precious asset, and loading it unnecessarily can impact both financial costs (tokens are not free) and efficiency (garbage-in, garbage-out).
It becomes imperative to centralize into a single source of truth where the LLM can retrieve context on demand. This is where RAG starts to become an interesting solution.
Visualizing the Flow
Instead of querying local files, agents must be capable of performing semantic queries. This workflow can look like this if we decide to deploy the entire infrastructure on Google Cloud.
- A developer authenticates using their API key; depending on their access level, they can access certain projects or features.
- The agent performs queries using semantic search. In the diagram above, a Cloud SQL instance running Postgres with the pg_vector extension enabled is used.
- Each text string is converted into an embedding using Vertex AI.
- The mathematical distance between the query vector and the vectors stored in the database is calculated, returning the most relevant results.
- When the developer implements the feature they are working on, they ask the agent to store the details in the database.
- The cycle repeats for the entire team.
Okay, So API or MCP?
On the surface, they might seem interchangeable, right? What actually sets them apart? I see two primary variables:
The Communication Flow
A fundamental difference between the two approaches is how they communicate.
-
An API requires the client to send an HTTP request to an endpoint. Ideally, this request must be authenticated so the server can evaluate whether to authorize it. When developing with AI, the agent will make these calls, meaning a well-implemented SKILL is non-negotiable.
-
MCP establishes a bidirectional connection using SSE. This ensures that if there are changes on the server, it can notify the client. In AI development, the client is our agent, which can then decide what action to take in response.
Financial Efficiency
In either case, the workflow and deployed services remain the same. However, there is a critical factor regarding the Cloud Run instance that makes an API more attractive for RAG, at least for this specific use case.
- Scale-to-zero. - We can apply scale-to-zero settings for periods when the API is not under heavy usage. Yes, there will be cold starts, but that can be addressed in the SKILL’s configuration. MCP, on the other hand, requires an open tunnel, preventing it from scaling to zero.
- Request-based vs. Instance-based. - To optimize FinOps even further, you can choose request-based pricing in the Cloud Run configuration. This changes the instance lifecycle. It is highly cost-efficient because you are only billed for the requests processed in this case, specific semantic queries or write operations.
- If we were to use instance-based pricing with a minimum of one instance active per month, even with the most basic configuration, it would incur an additional cost of $25 USD (always adjustable based on usage; this is just a reference).
Conclusion
One could argue that an agent can run a local MCP server that connects directly to a remote vector database. However, that would require making the database public, allowing clients to connect to it directly and thereby increasing the attack surface.
Just because MCP has become a standard, which is indeed non-negotiable for other use cases, does not mean we should design our architecture around it by default. We must always design for the specific use case.
I wrote this article to provide perspective. I believe there are situations where MCP shines, particularly when you want to avoid polling, but this is an interesting case that I wanted to share.