Add mTLS app-to-app routing support (RFC draft)#535
Draft
Add mTLS app-to-app routing support (RFC draft)#535
Conversation
- Add MtlsDomainConfig struct with domain-specific CA pool and forwarding modes - Add MtlsDomains field and mtlsDomainMap for fast domain lookups - Implement GetMtlsDomainConfig() for exact and wildcard domain matching - Add processMtlsDomains() to validate and build CA pools per domain - Support wildcard domains like *.apps.internal This enables GoRouter to enforce different mTLS policies per domain.
- Add getTLSConfigForClient() callback to dynamically select TLS config - For mTLS domains: require and verify client certs with domain CA pool - For regular domains: use base TLS configuration - Use SNI to determine which domain configuration to apply This allows GoRouter to enforce client certificate validation only on designated mTLS domains while leaving other domains unchanged.
- Add config parameter to NewClientCert() and clientCert struct - Check if request is for an mTLS domain using GetMtlsDomainConfig() - Use domain-specific ForwardedClientCert mode when applicable - Update all call sites and tests to pass config This allows different XFCC header handling policies per domain, supporting both legacy and mTLS-secured routes simultaneously.
- Add router.mtls_domains property to gorouter job spec - Implement ERB template validation and processing logic - Validate required fields (domain, ca_certs) and optional forwarded_client_cert - Support both wildcard (*.apps.internal) and exact domain matching This completes Phase 1a, enabling operators to configure per-domain mTLS policies via BOSH deployment manifests.
- Add AllowedSources struct to subscriber.go with app_guids list - Add AllowedSources field to RegistryMessage for NATS route messages - Add AllowedSourceAppGUIDs to EndpointOpts and Endpoint structs - Update Endpoint.Equal() to compare allowed source GUIDs - Add helper function getAllowedSourceAppGUIDs() This enables route registrations to specify which source apps are authorized to access endpoints on mTLS domains. Work in progress for Phase 1b authorization enforcement.
Ensures the allowed source app GUIDs are properly propagated from EndpointOpts to the Endpoint instance.
- Create handlers/identity.go with XFCC header parsing logic - Add CallerIdentity struct containing app GUID - Add CallerIdentity field to RequestInfo - Extract app GUID from certificate OU field (format: app:<guid>) - Parse X-Forwarded-Client-Cert header with PEM certificate The identity handler extracts the calling application's identity from the client certificate, enabling authorization checks in downstream handlers.
- Create handlers/mtls_authorization.go to enforce authorization - Check if request is on an mTLS domain using config.IsMtlsDomain() - Verify endpoint has AllowedSourceAppGUIDs configured - Match caller identity app GUID against allowed sources list - Return 403 Forbidden if caller not in allowed list - Return 401 Unauthorized if no caller identity present - Skip authorization for non-mTLS domains This handler ensures only explicitly authorized apps can communicate on mTLS-secured domains, completing the authorization enforcement layer for Phase 1b.
- Add NewIdentity() handler after NewClientCert() - Add NewMtlsAuthorization() handler after NewIdentity() - Handlers execute in order: ClientCert -> Identity -> Authorization Handler chain ensures: 1. XFCC header is processed and validated (ClientCert) 2. Caller identity is extracted from certificate (Identity) 3. Authorization is enforced on mTLS domains (MtlsAuthorization) This completes the request processing pipeline for mTLS app-to-app authorization in Phase 1b.
- Add AllowedSourceAppGUIDs field to RouteSchema and Route structs - Add AllowedSources field to NATS Message struct - Implement mapAllowedSources() helper to convert config to message format - Include allowed_sources in route registration messages sent via NATS This enables route-registrar to communicate authorization policies to GoRouter when registering routes on mTLS domains. Route registrations can now specify which source apps are allowed to access the endpoint, completing the authorization data flow from app configuration through to GoRouter enforcement.
Change all example domain names from *.apps.internal to *.apps.mtls.internal to match the RFC specification. The .mtls. segment differentiates mTLS-authenticated routing from direct container-to-container networking. Changes: - Update test files to use *.apps.mtls.internal - Update BOSH job spec example to use *.apps.mtls.internal - All 312 handler tests passing
Implement complete RFC-compliant authorization supporting apps, spaces, organizations, and 'any authenticated app' policies per the app-to-app mTLS routing RFC specification. Changes: - Expand AllowedSources struct with Apps/Spaces/Orgs/Any fields - Update Endpoint and EndpointOpts to use expanded AllowedSources - Enhance CallerIdentity to extract space and org GUIDs from cert OUs - Implement multi-level authorization in mtls_authorization handler - Update route-registrar to support expanded AllowedSources - Add 12 new tests for space/org/any authorization scenarios Authorization logic: - If Any=true, allow any authenticated app (mutually exclusive) - If Any=false, check Apps list, Spaces list, then Orgs list - Allow if caller matches any level (app, space, or org) - Default-deny if no AllowedSources or empty lists All 324 handler tests passing
Add end-to-end integration tests covering: - mTLS domain configuration and client certificate requirements - App-level, space-level, and org-level authorization - Multi-level authorization with OR logic - 'Any authenticated app' authorization (any=true) - Default-deny behavior for mTLS domains - X-Forwarded-Client-Cert header forwarding Test helpers added: - CreateInstanceIdentityCert() generates certificates with app/space/org GUIDs in OUs - registerWithAllowedSources() registers routes with authorization policies Integration tests require: - NATS server (set NATS_SERVER_BINARY env var) - Full GoRouter runtime environment - Can be run with: ginkgo --focus='App-to-App mTLS' integration/ Complements 324 passing unit tests with full end-to-end scenarios.
- Restore locket/lock package required by routing-api - Restore cactus/go-statsd-client package - Remove routing-api from vendor (it's a local submodule, not a vendored dep) - Remove routing-api from go.mod (local packages are resolved directly) This fixes the BOSH release build that was broken by unintended vendor changes in the 'Fix domain names to match RFC specification' commit.
The authorization handler was checking reqInfo.RouteEndpoint which is nil at handler execution time. Changed to use reqInfo.RoutePool which is set by the Lookup handler earlier in the chain. Added AllowedSources() and ApplicationId() methods to EndpointPool to support the authorization check at the pool level.
CAPI stores allowed_sources inside the route options JSON field, while route-registrar uses top-level allowed_sources. Updated GoRouter to check both locations with top-level taking precedence. - Added AllowedSources field to RegistryMessageOpts struct - Added getEffectiveAllowedSources() method to check both locations - Exported MakeEndpoint method for testing - Added unit tests for nested allowed_sources parsing
The clientcert.go sanitize() function produces XFCC headers with raw base64 certificate data (no PEM markers). The identity handler previously only supported Envoy-style Cert="<PEM>" format. Now it supports both formats: 1. GoRouter format: raw base64 without PEM markers 2. Envoy format: Cert="<PEM>" for compatibility This fixes the 'no-caller-identity' error in mTLS authorization.
Rename the AllowedSources struct and related fields/methods to MtlsAllowedSources throughout the codebase for better clarity and consistency with the feature naming. Changes include: - gorouter/route/pool.go: Rename struct and methods - gorouter/mbus/subscriber.go: Rename struct, update JSON tags - gorouter/handlers/mtls_authorization.go: Update variable names - route-registrar/config/config.go: Rename struct, update JSON/YAML tags - route-registrar/messagebus/messagebus.go: Rename helper functions All tests updated to use new naming convention.
Add xfcc_format configuration option for mTLS domains: - 'raw' (default): Full base64-encoded certificate (~1.5KB) - 'envoy': Compact 'Hash=<sha256>;Subject="<DN>"' format (~250 bytes) The Envoy format is ~6x smaller, reducing header overhead for high-volume app-to-app communication. Identity extraction in the identity handler now supports both formats, parsing the Subject DN from either raw certificates or Envoy format to extract app/space/org GUIDs.
Set RouteEndpoint on RequestInfo before returning 401/403 responses so that access logs are emitted to the target app's log stream. This allows operators to see denied requests in 'cf logs <app>' for the backend app, which is essential for debugging authorization issues in mTLS app-to-app communication.
RFC-0027 requires options values to be only strings, numbers, or booleans - not nested objects/arrays. Updated: - RegistryMessageOpts: Use flat fields (mtls_allowed_apps, mtls_allowed_spaces, mtls_allowed_orgs, mtls_allow_any) with comma-separated GUIDs instead of nested MtlsAllowedSources struct - parseCommaSeparatedGUIDs(): New helper to split comma-separated GUID strings into slices - getEffectiveMtlsAllowedSources(): Parse flat options from Options struct while maintaining top-level MtlsAllowedSources precedence for route-registrar compatibility - Tests: Updated to verify flat options parsing
cf-foundation-community-automation
bot
moved this to Inbox
in Application Runtime Platform Working Group
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Phase 1 (1a + 1b) of the App-to-App mTLS Routing RFC.
Note: This PR is a draft because the RFC for App-to-App mTLS Routing has not been approved yet.
Phase 1a: mTLS Infrastructure
GetConfigForClientcallbacksanitize_setmode) for mTLS domainsraw(base64 cert) orenvoy(compact hash+subject)router.mtls_domainsPhase 1b: Authorization
mtls_allowed_apps,mtls_allowed_spaces,mtls_allowed_orgs(comma-separated GUIDs),mtls_allow_any(boolean)Testing
Key Files Changed
GoRouter:
src/code.cloudfoundry.org/gorouter/config/config.go- MtlsDomainConfig structsrc/code.cloudfoundry.org/gorouter/router/router.go- GetConfigForClient callbacksrc/code.cloudfoundry.org/gorouter/handlers/clientcert.go- Domain-aware XFCCsrc/code.cloudfoundry.org/gorouter/handlers/identity.go- XFCC parsingsrc/code.cloudfoundry.org/gorouter/handlers/mtls_authorization.go- Authorization handlersrc/code.cloudfoundry.org/gorouter/mbus/subscriber.go- Route message parsingsrc/code.cloudfoundry.org/gorouter/route/pool.go- AllowedSources storagesrc/code.cloudfoundry.org/gorouter/proxy/proxy.go- Handler wiringRoute Registrar:
src/code.cloudfoundry.org/route-registrar/config/config.go- AllowedSources in Optionssrc/code.cloudfoundry.org/route-registrar/messagebus/messagebus.go- NATS message formatBOSH:
jobs/gorouter/spec- router.mtls_domains propertyjobs/gorouter/templates/gorouter.yml.erb- Template configurationConfiguration Example
Related PRs