feat: add api gateway emulator skeleton #1902

philasmar · 2024-12-10T17:16:17Z

Issue #, if available:
DOTNET-7837

Description of changes:

Added an API Gateway Emulator process
Added new command line arguments for the emulator
Added a service to process Lambda configuration through environment variables
Added the ability to match incoming HTTP requests with existing Lambda configurations
Added a catch-all HTTP endpoint
Updated base Spectre Command to support Cancellation tokens in order to facilitate unit tests

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Commands/RunCommand.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Commands/CancellableAsyncCommand.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Processes/ApiGatewayEmulatorProcess.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/IApiGatewayRouteConfigService.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

gcbeattyAWS · 2024-12-10T19:06:22Z

can you double check the line file endings. i was getting warnings in my ide that they were not set as CRLF

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Constants.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Processes/ApiGatewayEmulatorProcess.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

philasmar · 2024-12-11T15:25:18Z

can you double check the line file endings. i was getting warnings in my ide that they were not set as CRLF

Since we are working on different platforms we're bound to run into some line ending issues. I think what we did with the deploy tool was update our git config to retrieve files as LF. (something like that)

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Processes/ApiGatewayEmulatorProcess.cs

normj · 2024-12-11T22:35:58Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Processes/ApiGatewayEmulatorProcess.cs

+
+        var runTask = app.RunAsync(cancellationToken);
+
+        return new ApiGatewayEmulatorProcess


At this point do a LogInformation that the API Gateway emulator is available at the serviceUrl endpoint. Imagine this is what non-Aspire users will learn where to send requests to.

This is a bit redundant since the dotnet process will print the following to the console:

info: Microsoft.Hosting.Lifetime[14] Now listening on: **http://localhost:5051** dbug: Microsoft.AspNetCore.Hosting.Diagnostics[13] Loaded hosting startup assembly Amazon.Lambda.TestTool info: Microsoft.Hosting.Lifetime[0] Application started. Press Ctrl+C to shut down. info: Microsoft.Hosting.Lifetime[0] Hosting environment: Development info: Microsoft.Hosting.Lifetime[0] Content root path: /Users/asmarp/CodeBase/aws-lambda-dotnet/Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool dbug: Microsoft.Extensions.Hosting.Internal.Host[2] Hosting started

Also, at this point in time, the process may not successfully start. So if I was to print the service URL and then the task failed, it would be kind of weird. I'd prefer if we rely on the default .NET logging for this.

The ASP.NET Core log line below doesn't give users context that this is the API Gateway emulator versus Lambda runtime API.

info: Microsoft.Hosting.Lifetime[14] Now listening on: **http://localhost:5051**

I believe in the published version of the tool we will default all of the Microsoft logging default level to Error so that our console window is focused on giving logging messages focused on our tool. When users see the console of our tool that should be thinking this is a log message for a lambda emulator tool not a web server.

So we need somewhere in the output that says basically the API Gateway Emulator endpoint is here. If this isn't that right spot that is fine but needs to be somewhere.

Imagine a user installing who just does dotnet install lambda-test-tool and dotnet lambda-test-tool. There mind set should only be how to I test my Lambda function and what is informative to me as a Lambda developer debugging. Not how to a debug some webserver, that is our implementation detail.

normj · 2024-12-11T22:40:13Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+        foreach (var routeConfig in _routeConfigs)
+        {
+            _logger.LogDebug("Checking if '{Path}' matches '{Template}'.", path, routeConfig.Path);
+            var template = TemplateParser.Parse(routeConfig.Path);


I don't see any tests for handling API Gateway's style of using {proxy+} to indicate wild card character. I really want to know if Microsoft's TemplateParser is handling that or if we need to roll our own solution. Have you done any side by side comparisons with API Gateway and this library to see if they resolve the same.

TemplateParser supports a catch-all variable similar to the greedy variable in API gateway. I have updated the implementation to account for it as well as add sorting based on the route priority logic from API Gateway. I have added comments to explain everything in the code.

normj · 2024-12-11T22:44:52Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+        return null;
+    }
+
+    private RouteValueDictionary GetDefaults(RouteTemplate parsedTemplate)


Are defaults even a think in API Gateway routing? I thought with API Gateway routing you can just have the name of the variable. I could be wrong but be sure we are not just implementing a feature of the Microsoft library and not API Gateway.

You are right. The function was not doing anything for our use case. I removed it.

normj · 2024-12-11T22:54:19Z

...Tool-v2/tests/Amazon.Lambda.TestTool.UnitTests/Services/ApiGatewayRouteConfigServiceTests.cs

+
+namespace Amazon.Lambda.TestTool.UnitTests.Services;
+
+public class ApiGatewayRouteConfigServiceTests


You need tests with {proxy+} especially when there are other routes. And you need tests with some overlap in the routes. For example a test with routeA -> /foo/{value} and routeB -> /foo/bar and you test with requests to /foo, /foo/bar and /foo/fred. Similar idea with {proxy+} and make sure we mimic API Gateway's behavior when choosing to use the proxy endpoint versus a more direct path. Another example routeA /{proxy+} and routeB /phil. Requests going to /norm and /phil and /phil/dotnet. Don't expect the Microsoft template library to match API Gateway's behavior.

I have updated the implementation to account for it as well as add sorting based on the route priority logic from API Gateway. I have added comments to explain everything in the code.
I also added 1 big test that lists a bunch of different routes with overlap and tests which function was matched. This should cover everything.

normj · 2024-12-12T18:23:26Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Processes/ApiGatewayEmulatorProcess.cs

+
+        var runTask = app.RunAsync(cancellationToken);
+
+        return new ApiGatewayEmulatorProcess


The ASP.NET Core log line below doesn't give users context that this is the API Gateway emulator versus Lambda runtime API.

info: Microsoft.Hosting.Lifetime[14] Now listening on: **http://localhost:5051**

I believe in the published version of the tool we will default all of the Microsoft logging default level to Error so that our console window is focused on giving logging messages focused on our tool. When users see the console of our tool that should be thinking this is a log message for a lambda emulator tool not a web server.

So we need somewhere in the output that says basically the API Gateway Emulator endpoint is here. If this isn't that right spot that is fine but needs to be somewhere.

Imagine a user installing who just does dotnet install lambda-test-tool and dotnet lambda-test-tool. There mind set should only be how to I test my Lambda function and what is informative to me as a Lambda developer debugging. Not how to a debug some webserver, that is our implementation detail.

normj · 2024-12-12T18:24:05Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+        _logger.LogDebug("Updating the metadata needed to properly sort the Lambda config");
+        foreach (var routeConfig in _routeConfigs)
+        {
+            if (routeConfig.Path.Contains("{proxy+}"))


What if you have a resource path like /resource/{id}/subsegment/{proxy+} so you have both a proxy and a variable?

normj · 2024-12-12T18:26:16Z

...Tool-v2/tests/Amazon.Lambda.TestTool.UnitTests/Services/ApiGatewayRouteConfigServiceTests.cs

+    }
+
+    [Fact]
+    public void ProperlySortRouteConfigs()


Do you verify with an API Gateway deployed with routes like this that returns behaves the same?

Yes the values I have in the test have been verified in API Gateway

normj · 2024-12-14T01:56:04Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Models/ApiGatewayRouteType.cs

+/// <summary>
+/// The type of API Gateway Route. This is used to determine the priority of the route when there is route overlap.
+/// </summary>
+public enum ApiGatewayRouteType


I assume this can be deleted now.

normj · 2024-12-14T02:02:55Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+    /// <returns>true if route is valid, false if not</returns>
+    private bool IsRouteConfigValid(ApiGatewayRouteConfig routeConfig)
+    {
+        var occurrences = routeConfig.Path.Split("{proxy+}").Length - 1;


Validate that path, httpmethod and Lambda name are set.

normj · 2024-12-14T02:16:22Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+            }
+        }
+
+        // If there are leftover route segments


Can you explain what would be an example route and request path that would go into this section. It would be good to add examples in the comment block to help readers. I'm struggling to read this and understand in what scenario would the route ever be a matched if there are more route segments then were in the request path.

normj · 2024-12-14T02:17:16Z

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs

+        }
+
+        // If request has leftover segments that aren't matched
+        if (matched && requestPathIndex < requestSegments.Length)


If the variableCount doesn't match the variables found in the request path should that be an false match.

feat: add api gateway emulator skeleton

1af7cf4

philasmar added the Release Not Needed Add this label if a PR does not need to be released. label Dec 10, 2024

philasmar requested review from normj, 96malhar and gcbeattyAWS December 10, 2024 17:16

gcbeattyAWS requested changes Dec 10, 2024

View reviewed changes

gcbeattyAWS mentioned this pull request Dec 10, 2024

Add ApiGatewayHttpApiV2ProxyRequestTranslator and ApiGatewayProxyRequestTranslator #1901

Open

normj requested changes Dec 11, 2024

View reviewed changes

reply to comments

cd3607e

philasmar requested a review from gcbeattyAWS December 11, 2024 15:27

gcbeattyAWS reviewed Dec 11, 2024

View reviewed changes

Tools/LambdaTestTool-v2/src/Amazon.Lambda.TestTool/Services/ApiGatewayRouteConfigService.cs Show resolved Hide resolved

add trimming to the environment var parsing logic

65267f3

philasmar requested a review from gcbeattyAWS December 11, 2024 15:38

gcbeattyAWS approved these changes Dec 11, 2024

View reviewed changes

update return when config not found

693fcfa

philasmar requested a review from normj December 11, 2024 21:57

normj requested changes Dec 11, 2024

View reviewed changes

added priority to routes based on API Gateway logic

253d032

philasmar requested a review from normj December 12, 2024 17:42

normj reviewed Dec 12, 2024

View reviewed changes

updated the algorithm for finding the best match

37aa2b7

normj reviewed Dec 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add api gateway emulator skeleton #1902

feat: add api gateway emulator skeleton #1902

philasmar commented Dec 10, 2024

gcbeattyAWS commented Dec 10, 2024

philasmar commented Dec 11, 2024

normj Dec 11, 2024

philasmar Dec 12, 2024

philasmar Dec 12, 2024

normj Dec 12, 2024

normj Dec 11, 2024

philasmar Dec 12, 2024

normj Dec 11, 2024

philasmar Dec 12, 2024

normj Dec 11, 2024

philasmar Dec 12, 2024

normj Dec 12, 2024

normj Dec 12, 2024

normj Dec 12, 2024

philasmar Dec 12, 2024

normj Dec 14, 2024

normj Dec 14, 2024

normj Dec 14, 2024

normj Dec 14, 2024


		var runTask = app.RunAsync(cancellationToken);

		return new ApiGatewayEmulatorProcess


		namespace Amazon.Lambda.TestTool.UnitTests.Services;

		public class ApiGatewayRouteConfigServiceTests

feat: add api gateway emulator skeleton #1902

Are you sure you want to change the base?

feat: add api gateway emulator skeleton #1902

Conversation

philasmar commented Dec 10, 2024

gcbeattyAWS commented Dec 10, 2024

philasmar commented Dec 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment