Choosing a Serverless Internal Service Layer
Before building an internal service layer, decide whether to. What the layer costs per call, the volume where VPC Lattice wins, and when direct invoke still beats it.
When a serverless estate grows past one team, services start calling each other. The usual answer is a cross-team grant of lambda:InvokeFunction on someone else’s function ARN. That grant turns a function name into a public contract. The callee can no longer rename, split, or replace the function without breaking a consumer that lives in a repository they cannot see. The naive fix, wrapping the callee in an SDK client library, does not help: the coupling is in the IAM policy, not in the code.
Before any of that, answer a harder question: does the caller need the answer in-band at all? AWS’s own Lambda guide is blunt about the alternative. Under a heading called Anti-patterns in Lambda-based event-driven applications, it says that “Lambda functions directly calling other Lambda functions is generally an anti-pattern due to cost and complexity concerns”, and prescribes an SQS queue or Step Functions instead (event-driven architectures). The same page notes that “Most Lambda invocations in production systems are asynchronous”. Events are the default shape for serverless east-west traffic, and they are the right one. If nobody is waiting on the reply, stop reading and put a bus in the middle.
This series is about the calls that survive that filter. Some genuinely need a response: a checkout that must know whether the inventory reservation succeeded, a request that cannot be eventually consistent. AWS’s prescriptive guidance grants the point rather than scolding you, listing predictable flow control, strong consistency, and simple error handling as real benefits of synchronous calls (synchronous integration). Those calls exist in every estate. The question is not whether to make them, it is what to make them over.
You do not deploy a mesh for that traffic; you compose one out of primitives you already own. The default is a private REST API in API Gateway. Callers reach it through a shared execute-api interface VPC endpoint. A resource policy plus AWS_IAM authorization gates every call. No sidecars, no control plane. It is not an alternative to events. It is the replacement for the direct invoke that AWS just told you not to write.
Two things make it the default, and neither is identity. The fixed cost is shared: one interface endpoint serves every private API in the VPC for tens of dollars a month, while the obvious competitor bills per service. And API Gateway is an API management layer, so the callee can throttle a route, validate a request against a schema, and publish a contract. Those are the arguments, and this part makes them properly against VPC Lattice below, which does the identity half at least as well as this layer does. Be clear about that up front: if you are choosing this layer because you think it is the only way to get per-route IAM authorization on an internal call, you are choosing it for the wrong reason.
What a cross-team direct invoke actually costs
Direct RequestResponse invoke looks free because nothing sits between the two functions. Four costs are hiding in that gap.
The function name becomes the contract. Team B’s execution role holds lambda:InvokeFunction on arn:aws:lambda:...:function:teamA-orders. Every rename, every split into two handlers, every migration to a different compute model breaks a policy that Team A does not own and cannot grep for.
You pay twice for the same second. A synchronous invoke from A to B leaves A idle, billed at its own memory tier, for the whole duration of B. Chain three services deep and the same milliseconds land on three bills.
There is no throttling contract. The callee cannot cap one caller at 200 requests per second and have the platform enforce it. Its only lever is reserved concurrency, which is blunt: it throttles every caller at once, including the callee’s own async triggers.
Telemetry stops at the function boundary. There are per-function metrics, but no per-route latency or error rate for the orders API as a whole. East-west traffic also mixes with north-south traffic in the same numbers.
One more detail is worth pinning, because stale figures circulate. The Lambda invocation payload quota is 6 MB each way for synchronous calls and 1 MB for asynchronous ones. It is not the 256 KB that older posts still quote (Lambda quotas). Direct invoke also changes retry semantics based on a flag the caller picked, not a contract the callee published.
The shape of the composed layer
The substitution at the heart of this pattern is small: the caller’s execution role holds execute-api:Invoke on a route ARN instead of lambda:InvokeFunction on a function ARN. The callee’s function ARN never leaves the callee’s stack.
Two gates stack in that path and both must allow the call. The API resource policy decides which VPC or VPC endpoint may reach the API at all. The method authorization decides which IAM principal may call which route. A third gate is optional: a VPC endpoint policy on the interface endpoint itself. It limits which APIs that endpoint may reach, so a platform team can hold a perimeter the service teams cannot widen. AWS puts it higher than that, listing it as a best practice: “For the most secure data perimeter, you can create a VPC endpoint policy” (private REST APIs). Part 105 builds one.
What the layer costs
Prices below are us-east-1 list rates, read on 2026-07-14. Re-run them against your own Region before you commit.
| Component | Rate | Source |
|---|---|---|
| Private REST API requests | $3.50 per million (first 333M/month) | API Gateway pricing |
| Interface VPC endpoint | $0.01 per endpoint per AZ-hour | PrivateLink pricing |
| PrivateLink data processing | $0.01 per GB (first PB/month) | PrivateLink pricing |
The fixed part is small and shared. One endpoint across two AZs runs about $14.60 per month (2 x $0.01 x 730 hours), and that one endpoint serves every private API in the VPC. Partial endpoint-hours bill as full hours, so the floor is real but flat.
The variable part is the honest tax. REST is the most expensive API Gateway tier, and it is the only tier that can be private. There is no escape to the cheaper HTTP API rate. At 50 million internal calls per month across the whole estate you are paying roughly $175 per month for plumbing that used to appear free. State that number to your team before someone finds it in a bill. Note one thing about that figure: for this layer, the bill is the same whether those 50 million calls run through one service or a dozen. That is not true of the alternative, and the next section is about why.
It is worth paying when the calls are a contract boundary between teams. The $175 buys a throttle the callee sets on its own route, request validation against a schema, and an exportable contract. It does not buy per-route authorization, because as the next section shows, the alternative has that too. Per-route metrics are not free either. Latency, 4XXError and 5XXError are dimensioned per method and resource only when you turn on detailed CloudWatch metrics (MetricsEnabled). That is off by default and billed as custom metrics. On a wide estate the cardinality is the cost, so either budget for it or read per-route latency out of access logs instead. It is not worth paying inside a single service’s own boundary. Two Lambdas owned by the same team, deployed by the same pipeline, with no independent consumer, should keep calling each other directly. Better still, they should stop being two Lambdas.
When to override the default
Before choosing any transport, ask whether the call needs a response at all. If the caller does not need the callee’s answer in-band, EventBridge or SQS is cheaper, more available, and removes the coupling instead of formalising it. Only a genuine request/response call belongs on this layer.
VPC Lattice deserves more than a paragraph, because it is the strongest objection to this entire series and the case for it is better than most write-ups admit. State it at full strength first.
Lattice is the service AWS built for exactly this traffic. Lambda is a native target type. Its auth policies are IAM policy documents evaluated with SigV4, on the action vpc-lattice-svcs:Invoke.
Its edge is worth locating precisely, because the sloppy version of this argument is wrong in both directions. Per-route grants are not the edge. API Gateway expresses those through the resource ARN (.../v1/POST/orders), which is exactly what part 103 builds, and Lattice can do the same: scope the resource to a path, or condition on RequestMethod and RequestPath. The genuine edge is the rest of its condition set: RequestHeader, RequestQueryString, SourceVpc, and SourceVpcOwnerAccount (auth policies). An API Gateway resource policy has no header or query-string predicate to answer with, since execute-api ships exactly one service condition key, viaDomainArn. If you need to authorize on a request header, Lattice is the only one of the two that can. One caveat: on a TLS passthrough listener Lattice cannot read the headers either. Lattice also leaves the caller no endpoint plumbing to manage, associates VPCs to a service network directly, and shares across accounts through RAM.
So say it plainly. On identity, Lattice is not behind this layer. It is ahead of it. If you picked a private API Gateway because you believed it was the only way to get per-route IAM authorization on an internal call, you picked it for a reason that does not exist. AWS’s own framing points the same way: it positions API Gateway as the north-south front door and Lattice as the service-to-service layer (what is VPC Lattice). This series argues against that framing, and AWS publishes no decision guide comparing the two. The argument below is mine, not AWS’s.
The argument is cost shape and API management, in that order.
Cost shape. Lattice charges $0.025 per service per hour, roughly $18.25 per service per month, plus $0.025 per GB processed. Requests are free for the first 300,000 per hour per service, which works out to roughly 219 million per month if traffic is evenly spread, then $0.10 per additional million (VPC Lattice pricing, read 2026-07-14). Set that against the numbers above: a flat $14.60 per month for one shared endpoint, plus $3.50 per million requests. The two fixed costs behave differently, and that is the whole game. Lattice’s floor scales linearly with the number of services. API Gateway’s floor does not scale at all.
That asymmetry decides most estates before the request charge matters:
| Estate | Private API Gateway | VPC Lattice |
|---|---|---|
| 20 services, 200k requests each per month | about $29 | about $365 |
| 20 services, 5M requests each per month | about $368 | about $375 |
| 3 services, 50M requests each per month | about $545 | about $69 |
Per service, ignoring data processing, the crossover is where Lattice’s $18.25 is repaid by API Gateway’s $3.50 per million: roughly 5.2 million requests per service per month. Below it, the private API is cheaper, and a wide, shallow estate is not close. Above it, Lattice pulls away hard, because its marginal request price is 35 times cheaper.
API management. This is the argument that survives even when the money is a wash. API Gateway lets the callee defend itself and publish a contract. It has stage-level per-method throttling, so a team can cap a route it owns. It validates a request body against a schema before the integration runs. It exports an OpenAPI document. It takes WAF and it can cache. Lattice has none of these: no throttling of any kind, no request validation, no contract artifact, no WAF, no caching. Its listener rules match on header, method, and path, and that is the extent of it. In a multi-team estate, a callee that cannot refuse an authorized caller is a real operational hole. Any team that holds the grant can run you to the service quota, and you have no lever to say no. That is the hole this layer fills. One caveat, because it matters: AWS says usage-plan throttling and quotas “are not hard limits, and are applied on a best-effort basis”, and per-caller quotas require API keys, which AWS separately tells you not to use for authorization. The defensible claim is per-route throttling, not per-caller billing-grade quotas.
Take the override when any single service crosses roughly 5 million calls a month, when the targets are not all Lambda (Lattice covers ECS, EKS, EC2, ALB, and IP targets), when you need cross-VPC or cross-account discovery without per-caller endpoint plumbing, or when your authorization genuinely needs a header or query-string predicate. Note what is not on that list: gRPC. Switching to Lattice does not buy you gRPC while your compute is still Lambda functions, for the reasons part 102 spells out. And despite the reputation, Lattice does not hand you health checks in an all-Lambda estate: AWS states that health checks are not supported for Lambda target groups.
ECS Service Connect is the name people reach for the moment they hear App Mesh is being switched off. For an all-Lambda estate it is not a weaker option. It is an empty set.
Service Connect is not a network you join, it is a deployment feature of an ECS service. Its data plane is a managed Envoy sidecar that ECS injects into every task, its configuration lives in serviceConnectConfiguration on an ECS service, and its registry is a Cloud Map HTTP namespace that creates no Route 53 records. The considerations page is categorical (considerations): “Only the tasks that services create are supported.” Applications outside ECS, in AWS’s words, “can connect through the Service Connect proxy but can’t resolve Service Connect endpoint names” (Service Connect). A Lambda function has no task, no sidecar, and no name to resolve. AWS never writes the sentence “Lambda is not supported”, because it does not need to: the exclusion is structural. The roadmap request for Lambda support has been open since February 2023.
Two things stay true even if AWS shipped that support tomorrow. Service Connect has no IAM authorization. Access control is security groups, and its TLS is encryption without peer authentication, so there is no mutual TLS either. Trading this layer for Service Connect would swap an identity boundary for a network boundary. And the asymmetry runs one way only: an ECS task holding execute-api:Invoke on its task role is a first-class caller into the private API described here. This layer accepts ECS clients. Service Connect does not accept Lambda clients.
Service Connect is the right answer when the estate is ECS talking to ECS, in one Region and one namespace, and you want short names, retries, outlier detection, and per-service metrics with no code change and no extra service charge. That is the shape AWS recommends it for, and it is the App Mesh replacement for ECS. It is simply not a shape a Lambda can take. For a mixed estate the arithmetic is different again: Service Connect covers only the ECS half, so you would run two east-west mechanisms. Lattice is the one layer that spans both, because LAMBDA is a native target type alongside INSTANCE and IP. That is the real reason “targets are not all Lambda” points at Lattice rather than at Service Connect.
ALB with Lambda targets is cheaper per request than a REST API, and it is not a contract layer. It has no IAM authorization, no per-route usage plan, and an hourly plus LCU bill of its own. It balances load; it does not publish a contract.
Direct invoke remains correct inside a single service’s own boundary, where the caller and the callee ship together. It stops being correct at a team boundary. That is the whole rule.
What I plan to measure
The POC is not deployed, so these are targets rather than results. Gateway overhead per call is the first number: Latency minus IntegrationLatency at p50 and p99, per route. That difference is exactly what the per-call tax buys or costs. Then the monthly bill: requests times $3.50 per million, plus endpoint AZ-hours, plus PrivateLink GB processed, set against the Lattice service-hour equivalent at the estate’s real service count. Then per-route 4XXError and 5XXError rates, which direct invoke cannot produce at all. And one adoption metric that matters more than any of them: the count of cross-team lambda:InvokeFunction grants, where the target is zero.
Where this default holds
The default holds for a multi-team serverless estate with all-Lambda targets, request/response calls, and moderate volume. That default is a private REST API behind one shared execute-api interface endpoint, with AWS_IAM per-route grants. What it buys is not identity, which the alternative also has. It is a fixed cost that does not grow with the service count, plus the API management the alternative does not have: a throttle the callee sets, request validation, and a contract it can publish. No sidecar, no control plane.
Apply the filters in order, because the first one removes the most work. If nobody is waiting on the reply, use an event bus and none of this applies. If the targets are not all Lambda, or a service crosses roughly 5 million calls per month, Lattice starts to win. If you need gRPC, you are changing compute, not gateways. And whatever you ship the bytes as, keep the schema as the contract, because that is the part you will still be living with when the wire format changes.
The first step is cheaper than the migration: count the cross-team lambda:InvokeFunction grants in your account today. That number is the size of the coupling you are carrying.
That decides what to run the traffic over. It says nothing yet about what the traffic can carry: whether gRPC is on the table, whether protobuf earns its keep on this transport. Part 102 settles the wire format before anything gets built.
References
- Event-driven architectures in Lambda - AWS’s own anti-pattern section: “Lambda functions directly calling other Lambda functions is generally an anti-pattern”. The reason events are the default and this layer is the exception.
- Synchronous integration - AWS’s even-handed case for synchronous calls: predictable flow control, strong consistency, simple error handling, against tight coupling and cascading failures.
- AWS Lambda quotas - Invocation payload of 6 MB each way synchronous and 1 MB asynchronous. Not the 256 KB that older posts still quote.
- Private REST APIs in API Gateway - One shared endpoint for many private APIs, and the “For the most secure data perimeter, you can create a VPC endpoint policy” best practice.
- Amazon API Gateway pricing - REST at $3.50 per million for the first 333M requests; HTTP API at $1.00 per million. Prices read 2026-07-14, us-east-1.
- AWS PrivateLink pricing - Interface endpoints billed per endpoint per AZ-hour with partial hours billed as full, plus per-GB data processing.
- VPC Lattice auth policies - IAM policies on
vpc-lattice-svcs:Invoke, with condition keys for RequestMethod, RequestPath, RequestHeader and RequestQueryString. Why Lattice is ahead of this layer on identity, not behind it. - What is VPC Lattice - AWS’s “fully managed application networking service”, and the network-owner versus service-owner split it is built around.
- Amazon VPC Lattice pricing - $0.025 per service-hour, $0.025 per GB, first 300,000 requests per hour free, and the 100-service worked example behind the crossover math.
- Amazon ECS Service Connect - Applications outside ECS “can’t resolve Service Connect endpoint names”. Why it is an empty set for an all-Lambda estate.
- ECS Service Connect considerations - “Only the tasks that services create are supported”: the structural exclusion of Lambda.
Serverless Internal Service Layer: A Mesh You Compose, Not Deploy
Building service-mesh-like east-west communication for a multi-team serverless estate on a private API Gateway. Identity, encryption, wire formats, cost, and where the industry is actually heading.
All Posts in This Series
Related posts
A private REST API structurally cannot carry gRPC, and every AWS surface that speaks gRPC excludes Lambda targets. What to keep from gRPC, and what to drop.
The private REST API, the resource policy that switches it on, per-route AWS_IAM grants, the two CDK stacks, and signing the call from a Node 22 Lambda.
SigV4 proves which service is calling and nothing about which user it is for. How to propagate a verified subject, and what the transport actually encrypts.
Same-account, the resource policy and the caller's identity policy are an OR. Cross-account they become an AND, and silence denies. What that changes in the perimeter.
API Gateway shares one throttle bucket with your front door, never retries a Lambda integration, and cannot see loops through itself. What you have to rebuild.