Implement Hash-Based Routing by hoffmaen · Pull Request #519 · cloudfoundry/routing-release

hoffmaen · 2025-11-11T15:22:59Z

Read the Contributing document.

Summary

This Pull-Request implements #505.

Implement hash-based routing in Gorouter
- Gorouter routes requests to endpoints based on the value of a configured request header
- Gorouter distributes requests to different predefined application instances, when the current request count to an app instance exceeds the balance factor
- Gorouter routes requests to the "next instance" in the lookup table on retries
Follow decisions documented in this comment

Backward Compatibility

Breaking Change? Yes/No

src/code.cloudfoundry.org/gorouter/proxy/round_tripper/proxy_round_tripper.go

src/code.cloudfoundry.org/gorouter/route/maglev.go

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball

First set of comments. I have not looked at the implementation of maglev and the hash based instance determination.

I've focussed on the integration with the existing Gorouter code, route lookup, LB algo handling, etc.

src/code.cloudfoundry.org/gorouter/route/pool.go

src/code.cloudfoundry.org/gorouter/route/hash_based.go

src/code.cloudfoundry.org/gorouter/registry/registry_test.go

src/code.cloudfoundry.org/gorouter/handlers/helpers.go

src/code.cloudfoundry.org/gorouter/route/hash_based.go

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball

Some more comments, now on the hash_based.go

src/code.cloudfoundry.org/gorouter/route/hash_based.go

src/code.cloudfoundry.org/gorouter/route/roundrobin.go

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball · 2026-01-22T12:40:56Z

LGTM. Once we have the performance tests, we should move to community review.

hoffmaen · 2026-01-28T13:20:26Z

Good news - we evaluated the performance of gorouter with hash-based routing support.

Our test setup constists of scaled out HAProxy (as a load balancer in front of gorouter), scaled out Diego Cells and a very simple and performant application with 30 instances. This allows us to performance test gorouter via external requests, where gorouter is the bottleneck.

We compared the performance of the latest stable routing-release, and the same release including changes from this Pull-Request. Using oha (inofficial successor of hey), we put various load patterns on the application and measured throughput, CPU usage of gorouter, and latency. In summary,

both versions performed absolutely the same when using round-robin
we achieved the same performance using hash-based routing and round-robin, when requests are distributed across all instances (achieved by running oha with multiple instances, each using a different hash_header value)
various mixed load scenarios (load on a route with round-robin parallel to load on a route with hash) showed no impact on performance

We also explored that requests are routed to next instances as expected, when the max. connection limit of one app instance was reached. The hash_balance factor is effective and allows distribution of requests to next instances before overloading it.

Please feel free to reach out for details on the results!

Changes were addressed. As this is still subject to change, I cannot give "final" approval yet.

b1tamara · 2026-02-05T08:20:45Z

During performance tests, we figured out that the Gorouter memory consumption is higher than expected if we keep the permutation table (table which is used to fill the final maglev lookup table) in memory. We want to investigate this finding further and optimize it.
Moving PR back to Draft.

hoffmaen

Only documentation / log related comments.

jobs/gorouter/spec

src/code.cloudfoundry.org/gorouter/route/maglev.go

src/code.cloudfoundry.org/gorouter/config/config.go

This commit provides the basic implementation for hash-based routing. It does not consider the balance factor yet. Co-authored-by: Clemens Hoffmann <clemens.hoffmann@sap.com> Co-authored-by: Tamara Boehm <tamara.boehm@sap.com> Co-authored-by: Soha Alboghdady <soha.alboghdady@sap.com>

Co-authored-by: Alexander Nicke <alexander.nicke@sap.com>

…set list

maxmoehl · 2026-03-10T07:21:35Z

Please make sure to compact the history before merging.

maxmoehl · 2026-03-10T07:24:05Z

jobs/gorouter/spec

+  router.hash_based_routing.lookup_table_size:
+    description: |
+      Size of the Maglev lookup table used for hash-based load balancing.
+      The size should be approximately chosen based on the expected maximum number of endpoints / application instances per route which use hash-based routing. The provided values are not hard limits but recommended values.


You say that they are not hard limits, yet you prevent the operator from choosing their own value. Wouldn't an integer and a comment explaining the rough ranges be a better approach here?

We assumed that it could be difficult for operators to determine the exact value for this table size.
However, the current "t-shirt size" approach is an MVP for this first pass and may be refined as we gather more data. We are still discussing internally how to dynamically calculate the maglev lookup table size based on endpoint numbers, rather than defining a static configuration value via the routing release.

maxmoehl · 2026-03-10T07:26:46Z

src/code.cloudfoundry.org/gorouter/mbus/subscriber.go

 	LoadBalancingAlgorithm string  `json:"loadbalancing"`
 	HashHeaderName         string  `json:"hash_header"`
-	HashBalance            float64 `json:"hash_balance"`
+	HashBalance            float64 `json:"hash_balance,string"`


Why isn't this just a number in the JSON that we are parsing?

@hoffmaen
Could you remember why we changed to string?

Yes. This is because route-emitter sends the value as a string. We didn't want to make changes to route-emitter, and having the value as a string has no practical downsides.
This is also the implementation in cloud-controller

hoffmaen mentioned this pull request Nov 11, 2025

[DRAFT] Implement Hash-Based Routing #518

Closed

1 task

cf-foundation-community-automation bot moved this to Inbox in Application Runtime Platform Working Group Nov 11, 2025

cf-foundation-community-automation bot added this to Application Runtime Platform Working Group Nov 11, 2025

b1tamara force-pushed the hash-based-routing-with-overload branch from 7c44bfe to 2c23efd Compare November 13, 2025 13:05

hoffmaen force-pushed the hash-based-routing-with-overload branch from 2c23efd to 91f5968 Compare November 13, 2025 13:47

b1tamara mentioned this pull request Nov 14, 2025

Implement Hash-Based Routing Support in Cloud Controller cloudfoundry/capi-release#589

Closed

maxmoehl reviewed Nov 27, 2025

View reviewed changes

src/code.cloudfoundry.org/gorouter/proxy/round_tripper/proxy_round_tripper.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/maglev.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/pool.go Show resolved Hide resolved

b1tamara force-pushed the hash-based-routing-with-overload branch from e9cbe0c to 7b456f8 Compare December 3, 2025 09:32

hoffmaen force-pushed the hash-based-routing-with-overload branch from 9f21f6b to 513c575 Compare December 9, 2025 08:24

peanball reviewed Dec 9, 2025

View reviewed changes

peanball requested changes Jan 16, 2026

View reviewed changes

src/code.cloudfoundry.org/gorouter/route/hash_based.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/pool.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/pool.go Outdated Show resolved Hide resolved

hoffmaen force-pushed the hash-based-routing-with-overload branch from 5030bbc to 560f935 Compare January 16, 2026 14:29

peanball previously requested changes Jan 16, 2026

View reviewed changes

b1tamara requested a review from peanball January 21, 2026 07:22

peanball reviewed Jan 22, 2026

View reviewed changes

src/code.cloudfoundry.org/gorouter/route/roundrobin.go Show resolved Hide resolved

peanball reviewed Jan 22, 2026

View reviewed changes

src/code.cloudfoundry.org/gorouter/route/pool.go Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

b1tamara requested a review from peanball January 28, 2026 13:35

hoffmaen marked this pull request as ready for review January 28, 2026 14:53

hoffmaen requested a review from a team as a code owner January 28, 2026 14:53

a18e force-pushed the hash-based-routing-with-overload branch from 21cc8b7 to b414a38 Compare January 30, 2026 10:31

b1tamara marked this pull request as draft February 5, 2026 08:21

hoffmaen commented Feb 16, 2026

View reviewed changes

jobs/gorouter/spec Outdated Show resolved Hide resolved

jobs/gorouter/spec Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/maglev.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/config/config.go Outdated Show resolved Hide resolved

hoffmaen mentioned this pull request Feb 26, 2026

Add hoffmaen as Networking approver cloudfoundry/community#1442

Open

hoffmaen and others added 18 commits March 6, 2026 08:03

Add LICENSE information for maglev.go

a5c516a

Implement overflow traffic

2528612

* Minor improvements and refactoring

92b1477

Refactor pool.Endpoints to apply review feedback

a2672fe

Fix bug related to all endpoints are overloaded

3b7b841

Fix compiler errors

d43fd2b

Update RouteSchema struct in route-emitter

648736a

Add json string property to hash_balance

30930ed

Refactor route/pool Endpoints function

319d182

Refactor locallyOptimistic and pass request header to Endpoints

7b4b94e

WIP: Introduce routing properties

e2c1939

Apply review feedback

632e313

Refactor hash-based.go

ca97ada

refactor looking for endpoint for sticky session

55ac4be

Refactor hash_based.go

e29e5fb

Use int16 instead of int for maglev lookup table and permutation table

f26243c

WIP: Calculate permutation table on the fly, keep only offset/skip

34c3c48

feat: Expose Maglev Lookup Table Size as a gorouter spec property

2dcfaf0

Co-authored-by: Alexander Nicke <alexander.nicke@sap.com>

b1tamara force-pushed the hash-based-routing-with-overload branch from d21580f to cd304af Compare March 9, 2026 11:37

mike-jc and others added 2 commits March 9, 2026 12:40

feat: Maglev Lookup Table Size can be only one of the values from pre…

7cb8359

…set list

Fix for hash_based iterator

69da88c

b1tamara force-pushed the hash-based-routing-with-overload branch from cd304af to 69da88c Compare March 9, 2026 11:41

b1tamara marked this pull request as ready for review March 10, 2026 06:11

maxmoehl reviewed Mar 10, 2026

View reviewed changes

Conversation

hoffmaen commented Nov 11, 2025 • edited by b1tamara Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Backward Compatibility

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball commented Jan 22, 2026

Uh oh!

hoffmaen commented Jan 28, 2026 • edited by peanball Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

b1tamara commented Feb 5, 2026

Uh oh!

hoffmaen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxmoehl commented Mar 10, 2026

Uh oh!

maxmoehl Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

b1tamara Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

maxmoehl Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

b1tamara Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

hoffmaen Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

hoffmaen commented Nov 11, 2025 •

edited by b1tamara

Loading

hoffmaen commented Jan 28, 2026 •

edited by peanball

Loading

hoffmaen Mar 11, 2026 •

edited

Loading