Identity and Encryption Between Serverless Services

Part 103 built the layer: a private REST API reached over a shared execute-api interface endpoint, with every call signed with SigV4 and authorized per route. That layer proves the service, not the user. It answers one question well and the other one not at all.

SigV4 proves which service is calling. It proves nothing about which user the call is for. Those are different questions, and a layer that only answers the first is a layer where any service can act as any user. This part closes that gap, then states precisely what the transport does and does not encrypt.

SigV4 proves the service, not the user

Take the concrete case. A checkout Lambda signing with its execution role proves it is checkout-service. It does not prove that the cart it is asking about belongs to the person who opened the session. Nothing in the signature carries a subject.

Start with what the callee gets for free. Under AWS_IAM with a Lambda proxy integration, the caller’s IAM principal arrives in the event with no mapping template at all. AWS states it directly: “When the authorization type is AWS_IAM, the authorized user information includes $context.identity.* properties” (mapping template reference). The useful fields are $context.identity.userArn, defined as “The Amazon Resource Name (ARN) of the effective user identified after authentication”, plus .caller and .accountId. In a proxy integration they land under event.requestContext.identity. So the callee can add a rule of its own on top of the IAM grant: only the checkout-service role may post to /orders.

That is service identity. The user is a separate problem, and the gateway does not solve it for you.

The plain x-user-id header is a trap. It looks harmless next to a signed request, and it is an unauthenticated assertion. Any service that already holds execute-api:Invoke on the route can set that header to any value, and the callee acts on a claim it never verified. AWS has a name for this shape: the confused deputy, “a security issue where an entity that doesn’t have permission to perform an action can coerce a more-privileged entity to perform the action” (confused deputy). The service is authenticated. The subject is not.

A method has one authorizationType, and you already spent it. The obvious fix is to have the gateway verify the user token as well. API Gateway will not do it. A REST API method carries exactly one authorizationType; the valid values are NONE, AWS_IAM, CUSTOM, and COGNITO_USER_POOLS, and an authorizer can only attach when the type is CUSTOM or COGNITO_USER_POOLS (Method). So AWS_IAM and a Cognito or Lambda authorizer are mutually exclusive on the same method. Once the gateway slot goes to service identity, there is no gateway-side verifier left for the user token. Verifying the subject inside the callee is therefore not a preference. It is the only place left.

That constraint forces the design: sign the subject, and verify the signature where the work happens. Two shapes work.

Forward the original token. Pass the user’s OIDC or Cognito access token in a custom header and verify it in the callee against the issuer’s JWKS endpoint. For Node, AWS points at the awslabs aws-jwt-verify library: “In a Node.js app, AWS recommends the aws-jwt-verify library to validate the parameters in the token that your user passes to your app” (verifying a JWT). It is cheap: one dependency, no new infrastructure, JWKS cached in the execution environment. The cost is that the raw user token now travels across every hop, and its audience and scope were minted for the edge, not for orders.

Exchange the token at the edge. Trade the incoming token once, at the entry point, for a short-lived internal token that carries both the user and the calling workload down the chain. The IETF is formalising this shape as Transaction Tokens, “designed to maintain and propagate user identity, workload identity and authorization context throughout the Call Chain within a trusted domain” (Transaction Tokens). Be honest about its status: it is an IETF draft in working-group last call, not an RFC, and there is no AWS-managed implementation to switch on. Read it as the direction of travel and as a specification of what a hand-rolled internal token should carry, not as something you turn on this quarter.

The callee then runs both checks in the same handler:

import type { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
import { CognitoJwtVerifier } from 'aws-jwt-verify';

// Built once per execution environment; the JWKS is cached after the first fetch.
const verifier = CognitoJwtVerifier.create({
  userPoolId: process.env.USER_POOL_ID!,
  tokenUse: 'access',
  clientId: process.env.APP_CLIENT_ID!,
});

// Pinned at deploy time. Do not parse the ARN: its shape depends on how the
// caller obtained its credentials.
const allowedCallers = new Set(
  (process.env.ALLOWED_CALLER_ARNS ?? '').split(',').filter(Boolean),
);

export const handler = async (
  event: APIGatewayProxyEvent,
): Promise<APIGatewayProxyResult> => {
  // Check 1: which service is calling. API Gateway verified the SigV4 signature.
  const callerArn = event.requestContext.identity.userArn;
  if (!callerArn || !allowedCallers.has(callerArn)) {
    return { statusCode: 403, body: JSON.stringify({ message: 'caller not allowed' }) };
  }

  // Check 2: which user this call is for. Nothing above verified this.
  // Lower case name: private endpoints pass header names through as-is.
  const assertion = event.headers['x-user-assertion'];
  if (!assertion) {
    return { statusCode: 401, body: JSON.stringify({ message: 'missing user assertion' }) };
  }

  try {
    const claims = await verifier.verify(assertion);
    return { statusCode: 200, body: JSON.stringify({ subject: claims.sub }) };
  } catch {
    return { statusCode: 401, body: JSON.stringify({ message: 'invalid user assertion' }) };
  }
};

Two checks, not one. Which service is calling comes from IAM and arrives already verified. Which user this is for comes from a signed assertion the callee verifies itself.

One field to leave out of the handler: $context.identity.vpceId. AWS documents it as present only for private APIs, so it is tempting to check the endpoint in code. The documented enforcement point is elsewhere: the resource policy with aws:SourceVpce, which the service stack in part 103 already sets. Keep the network condition in the policy. Keep the handler to the two identity checks.

The encryption layer between services

The transport is encrypted already. It is worth being exact about how much that covers, because the phrase people reach for next is usually wrong.

Three facts are solid. API Gateway exposes HTTPS endpoints only: “API Gateway doesn’t support unencrypted (HTTP) endpoints” (data protection). Private APIs enforce a TLS 1.2 floor. Read AWS’s sentence carefully, because it is easy to misread: “Private APIs only support TLS 1.2. Earlier TLS versions are not supported.” (private REST APIs). That is a minimum, not a ceiling. The default TLS_1_2 security policy accepts both TLS 1.2 and TLS 1.3 (security policies). And the path stays inside AWS: “traffic to your private API uses secure connections and is isolated from the public internet. Traffic doesn’t leave the Amazon network.”

It is not end-to-end encryption. TLS terminates at API Gateway. The integration is a second connection with its own handshake. In the all-Lambda shape this series recommends, that second leg is also encrypted, since “Lambda API endpoints only support secure connections over HTTPS” (Lambda data protection). But AWS nowhere asserts one unbroken channel from caller code to callee code, so do not claim it in a design review. Two TLS legs with a service boundary between them is the accurate description. The sharp edge lives on the second leg. For the HTTP integration types, API Gateway states that “each integration can specify a protocol (http/https), port and path” (Integration). Plain http is a legal choice there. Passing through API Gateway does not by itself mean the second leg was encrypted.

Mutual TLS is not available on this layer. “Mutual TLS isn’t supported for private APIs” (mutual TLS). mTLS in API Gateway requires a custom domain name, and private APIs are excluded from it. There is no property to set. If a compliance requirement asks for mTLS on east-west calls, the workaround is to terminate it upstream of the gateway. AWS has written that up (consuming private APIs using mutual TLS), and it is a blog pattern, not documented service behaviour. Label it that way when someone puts it in an architecture document.

The custom header is where the user token can leak. This follows straight from the identity section. Under AWS_IAM, SigV4 already owns the Authorization header, so a forwarded user assertion has to ride in a custom header such as x-user-assertion or x-id-token. Now read the redaction promise carefully: “API Gateway redacts authorization headers, API key values, and similar sensitive request parameters from the logged data” (set up logging). AWS names authorization headers and API keys, then leaves “similar sensitive request parameters” open. It nowhere documents an arbitrary custom header as redacted. It nowhere documents that it is not. So do not claim your JWT is being logged. Claim the thing that matters: AWS gives you no promise that it is not.

This is not a leak that is on by default, and the narrow version of the point is the sharp one. Data tracing is opt-in, and AWS says it “can result in logging sensitive data” and that “We recommend that you don’t use Data tracing for production APIs”. But the one header AWS_IAM forced you into is the one header AWS declines to make a promise about. Turn tracing on to debug a route and you cannot prove the token in that header is being stripped. So keep data tracing off on any stage carrying a user assertion, and keep the assertion out of your own structured logs. Then go one step further, because config discipline loses to whoever is debugging a live incident under pressure. Part 107 builds the control that survives that: a data protection policy that masks the token on ingestion, whoever turned tracing on.

Encrypt the field when the channel is not the boundary you need. A callee that must never see a PII value, or a control that has to survive a misconfigured log, calls for payload-level encryption instead. The AWS Encryption SDK is “a client-side encryption library designed to make it easy for everyone to encrypt and decrypt data using industry standards and best practices” (AWS Encryption SDK), built on envelope encryption with KMS wrapping keys. The caller encrypts the field, the gateway and the logs see ciphertext, and only a principal holding kms:Decrypt on the wrapping key can read it. This is defense in depth, not the default for an internal layer. Reach for it per field, not per service, and only where cleartext on the second leg or in a log line is genuinely unacceptable.

Common pitfalls

1. Trusting an x-user-id header because the request was signed. SigV4 proves the calling service and nothing about the subject. Forward a signed token and verify it in the callee.
2. Turning on data tracing for a stage whose calls carry a user assertion. Redaction covers authorization headers and API key values, not the custom header AWS_IAM forced you into.
3. Promising mutual TLS on this layer in a design document. Private APIs are excluded from it, and there is no property to set.
4. Calling the path end-to-end encrypted. TLS terminates at the gateway, and the integration is a second connection.

Identity is settled in both directions. Everything so far has assumed one account. In a multi-team estate the team boundary usually lands on an account boundary, and that single move rewrites the authorization rules underneath the layer. Part 105 takes the layer across accounts and into the data perimeter.

References

• API Gateway REST API reference: Method - The single authorizationType per method, and the rule that an authorizer only attaches to CUSTOM or COGNITO_USER_POOLS. The constraint that forces user verification into the callee.
• API Gateway mapping template reference - Source of the $context.identity.* variables that AWS_IAM fills in, including the definition of userArn.
• The confused deputy problem - AWS’s definition of the failure an unsigned x-user-id header creates.
• aws-jwt-verify - The awslabs library AWS recommends for verifying a JWT inside a Node.js application.
• Amazon Cognito: Verifying a JSON Web Token - The Cognito Developer Guide passage that recommends aws-jwt-verify to validate a token inside a Node.js app; the source of that quote.
• IETF: OAuth 2.0 Transaction Tokens - The emerging standard for “on whose behalf”, explicit that a transaction token is not a workload identity credential. Still a working-group draft.
• Configuring mutual TLS for a REST API - “Mutual TLS isn’t supported for private APIs”, plus the custom domain name requirement behind that exclusion.
• Consuming private Amazon API Gateway APIs using mutual TLS - The AWS compute blog pattern for terminating mTLS upstream of a private API. A blog post, not documented service behaviour.
• Set up CloudWatch logging for a REST API - The scope of header redaction and the warning against data tracing on production APIs.
• Data protection in API Gateway - “API Gateway doesn’t support unencrypted (HTTP) endpoints” and the PrivateLink isolation statement for private APIs.
• Private REST APIs in API Gateway - “Private APIs only support TLS 1.2”, the TLS floor for the private layer.
• API Gateway security policies - The default TLS_1_2 policy accepts both TLS 1.2 and TLS 1.3; the floor is a minimum, not a ceiling.
• API Gateway REST API reference: Integration - “each integration can specify a protocol (http/https)”, where plain http on the second leg is a legal choice.
• Data protection in AWS Lambda - “Lambda API endpoints only support secure connections over HTTPS”, the second TLS leg of the path.
• AWS Encryption SDK - Client-side envelope encryption under KMS wrapping keys, for the fields that must not sit in cleartext between services.