feat(lambda-runtime): log non-2xx Lambda Runtime API responses with status and body by darklight3it · Pull Request #1109 · aws/aws-lambda-rust-runtime

darklight3it · 2026-02-25T17:01:50Z

Summary

When the Lambda Runtime API returns a non-2xx response (e.g., HTTP 410 for timeout, 500 for internal errors), the runtime previously provided no visibility into these failures. Customers had no way to diagnose why their Lambda invocations were failing at the runtime API level.

Solution

Add comprehensive error logging for all non-2xx responses from the Lambda Runtime API, while maintaining backward compatibility by preserving the existing behavior of returning Ok(()) and avoiding backward compatibility issue with RuntimeApiClientFuture which is unfortunately public.

Changes

1. Created `log_or_print!` macro (`lambda-runtime/src/macros.rs`)

New crate-internal macro that dispatches to tracing::error! when a tracing dispatcher is set
Falls back to eprintln! when no tracing subscriber is configured
Consistent with existing codebase patterns for dual logging support

2. Enhanced error logging in `RuntimeApiClientFuture` (`lambda-runtime/src/layers/api_client.rs`)

Added logging for non-2xx HTTP responses with status code details
Special handling for HTTP 410 (Gone) with additional "Lambda function timeout" message
Added logging for request building errors (First state)
Added logging for network/transport errors (Second state)
Maintains backward compatibility: Still returns Ok(()) for non-2xx responses to avoid breaking existing runtime behavior

3. Added comprehensive test coverage

Added tracing-test dev dependency for testing logging behavior
Tests cover all error branches: 200 OK, 410 timeout, 500 error, 404 error, request build errors, and network errors
Tests verify both return values and log output using #[traced_test]

4. Updated `lambda-runtime/src/runtime.rs`

Applied log_or_print! macro to existing concurrency warning for consistency

Example log output

**410 Timeout:**
ERROR lambda_runtime::layers::api_client: Lambda Runtime API returned non-200 response status=410 Gone 

ERROR lambda_runtime::layers::api_client: Lambda function timeout!


**500 Internal Server Error:**
ERROR lambda_runtime::layers::api_client: Lambda Runtime API returned non-200 response status=500 Internal Server Error


**Network failure:**
ERROR lambda_runtime::layers::api_client: Lambda Runtime API request failed error=...

Backward Compatibility

This change is fully backward compatible:

No changes to public API surface
RuntimeApiClientFuture still returns Ok(()) for non-2xx responses (preserving existing behavior)
Runtime continues processing events after API errors (no change in control flow)
Only adds observability through logging

Testing

Run tests with:

cd lambda-runtime && cargo test layers::api_client::tests --lib
All 6 tests pass,

…useful for the customer for timeouts

…e a separate tracing_subscriber

darklight3it · 2026-02-25T17:54:28Z

Added an issue explaining the problem

#1110

jlizen

This approach looks good to me. It should keep the worker id span intact if using tracing, which is nice.

lambda-runtime/src/macros.rs

lambda-runtime/src/layers/api_client.rs

FullyTyped

exactly what i had in mind!

FullyTyped · 2026-03-03T12:24:22Z

lambda-runtime/src/layers/api_client.rs

+                RuntimeApiClientFutureProj::Second(fut) => match ready!(fut.poll(cx)) {
+                    Ok(resp) if !resp.status().is_success() => {
+                        let status = resp.status();
+
+                        // TODO
+                        // we should consume the response body of the call in order to give a more specific message.
+                        // https://github.com/aws/aws-lambda-rust-runtime/issues/1110
+
+                        log_or_print!(
+                            tracing: tracing::error!(status = %status, "Lambda Runtime API returned non-200 response"),
+                            fallback: eprintln!("Lambda Runtime API returned non-200 response: status={status}")
+                        );
+
+                        // Adding more information on top of 410 Gone, to make it more clear since we cannot access the body of the message
+                        if status == 410 {
+                            log_or_print!(
+                                tracing: tracing::error!("Lambda function timeout!"),
+                                fallback: eprintln!("Lambda function timeout!")
+                            );
+                        }
+
+                        // Return Ok to maintain existing contract - runtime continues despite API errors
+                        break Ok(());
+                    }
+                    Ok(_) => break Ok(()),
+                    Err(err) => {
+                        log_or_print!(
+                            tracing: tracing::error!(error = ?err, "Lambda Runtime API request failed"),
+                            fallback: eprintln!("Lambda Runtime API request failed: {err:?}")
+                        );
+                        break Err(err);
+                    }


Exactly what I had in mind.

FullyTyped · 2026-03-03T12:24:47Z

lambda-runtime/src/layers/api_client.rs

+    async fn start_mock_server(status: StatusCode) -> String {
+        let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        let url = format!("http://{}", addr);


Much easier than harness tests ;)

Davide Melfi added 2 commits February 25, 2026 16:55

feat: ⚡ Improved Logging when Lambda Runtime Api fails, this will be …

b553d1e

…useful for the customer for timeouts

chore: fix linting

2a86319

darklight3it mentioned this pull request Feb 25, 2026

feat(lambda-runtime): log non-2xx Lambda Runtime API responses with status and body #1105

Closed

darklight3it requested review from jlizen and maxday and removed request for jlizen and maxday February 25, 2026 17:05

Davide Melfi added 2 commits February 25, 2026 17:29

chore: add traced_test to test_concurrent_structured_logging to creat…

92367c3

…e a separate tracing_subscriber

chore: fmt

1a5204d

darklight3it requested review from jlizen and maxday February 25, 2026 17:34

darklight3it marked this pull request as ready for review February 25, 2026 17:35

darklight3it mentioned this pull request Feb 25, 2026

Fix RuntimeApiClientFuture Visibility and Response Handling #1110

Open

jlizen approved these changes Feb 26, 2026

View reviewed changes

lambda-runtime/src/macros.rs Show resolved Hide resolved

lambda-runtime/src/layers/api_client.rs Show resolved Hide resolved

chore: adding a todo for consuoing the body of the response

1a84beb

darklight3it self-assigned this Feb 26, 2026

darklight3it requested a review from rarepolz February 26, 2026 11:40

chore: adding format stuff

6ace45b

rarepolz approved these changes Feb 26, 2026

View reviewed changes

darklight3it merged commit 00e90ad into main Feb 26, 2026
29 checks passed

darklight3it deleted the dmelfi/timeout-fix-simple branch February 26, 2026 16:26

FullyTyped reviewed Mar 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(lambda-runtime): log non-2xx Lambda Runtime API responses with status and body#1109

feat(lambda-runtime): log non-2xx Lambda Runtime API responses with status and body#1109
darklight3it merged 6 commits intomainfrom
dmelfi/timeout-fix-simple

darklight3it commented Feb 25, 2026 •

edited

Loading

Uh oh!

darklight3it commented Feb 25, 2026

Uh oh!

jlizen left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FullyTyped left a comment

Uh oh!

FullyTyped Mar 3, 2026

Uh oh!

FullyTyped Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

darklight3it commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Solution

Changes

1. Created log_or_print! macro (lambda-runtime/src/macros.rs)

2. Enhanced error logging in RuntimeApiClientFuture (lambda-runtime/src/layers/api_client.rs)

3. Added comprehensive test coverage

4. Updated lambda-runtime/src/runtime.rs

Example log output

Backward Compatibility

Testing

Uh oh!

darklight3it commented Feb 25, 2026

Uh oh!

jlizen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FullyTyped left a comment

Choose a reason for hiding this comment

Uh oh!

FullyTyped Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

FullyTyped Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

darklight3it commented Feb 25, 2026 •

edited

Loading

1. Created `log_or_print!` macro (`lambda-runtime/src/macros.rs`)

2. Enhanced error logging in `RuntimeApiClientFuture` (`lambda-runtime/src/layers/api_client.rs`)

4. Updated `lambda-runtime/src/runtime.rs`