Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add remote invoke implementation for Kinesis stream service for put_record API #6063

Merged
merged 6 commits into from
Oct 12, 2023

Conversation

hnnasit
Copy link
Contributor

@hnnasit hnnasit commented Oct 10, 2023

Which issue(s) does this change fix?

N/A

Why is this change necessary?

This PR adds the implementation for remote invoke support for Kinesis stream service. The boto3 API put_record is called when sam remote invoke is run to put a record to the provided kinesis stream name using event or event-file.

How does it address the issue?

  • The Data of the record which is a required field can be provided using event or event-file parameters which will get encoded as base64 and put into Kinesis stream. If no event or event-file is provided, {} is used as Data for putting a record. The event is loaded using json before passing to boto3 API and the event-file is read and directly passed to boto3 API.
  • The PartitionKey is a required field for boto3 API and so it is randomly generated if not provided as input. This is just used to determine which Shard does the record get added to using a MD5 hash function.
  • The other fields such as ExplicitHashKey, SequenceNumberForOrdering, and StreamARN can be provided as strings using the --parameter field

Use cases:

  1. sam remote invoke KinesisStreamLogicalId --stack-name sam-app --event '{"hello": "world", "foo": 1, "bar": {}}': Puts a record with provided event as data to the KinesisStreamLogicalId Kinesis stream.
{
  "ShardId": "shardId-000000000000",
  "SequenceNumber": "sequence-number"
}
  1. sam remote invoke arn:aws:kinesis:us-east-1:12345678910:stream/kinesis-stream-name --event "hello world": Kinesis stream Arn used as resource-id to put a record
  2. sam remote invoke kinesis-stream-name --event "hello world" --output json: Kinesis stream name used as resource-id to put a record
{
  "ShardId": "shardId-000000000000",
  "SequenceNumber": "sequence-number",
  "ResponseMetadata": {}
}

What side effects does this change have?

No side effects.

Mandatory Checklist

PRs will only be reviewed after checklist is complete

  • Add input/output type hints to new functions/methods
  • Write design document if needed (Do I need to write a design document?)
  • Write/update unit tests
  • Write/update integration tests
  • Write/update functional tests if needed
  • make pr passes
  • make update-reproducible-reqs if dependencies were changed
  • Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@hnnasit hnnasit marked this pull request as ready for review October 10, 2023 21:42
@hnnasit hnnasit requested a review from a team as a code owner October 10, 2023 21:42
@hnnasit hnnasit requested review from mndeveci and lucashuy October 10, 2023 21:42
Copy link
Contributor

@hawflau hawflau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. just one minor typo

samcli/commands/remote/remote_invoke_context.py Outdated Show resolved Hide resolved
@hnnasit hnnasit enabled auto-merge October 12, 2023 21:46
@hnnasit hnnasit added this pull request to the merge queue Oct 12, 2023
Merged via the queue into aws:develop with commit 6ccd872 Oct 12, 2023
76 checks passed
moelasmar pushed a commit to moelasmar/aws-sam-cli that referenced this pull request Oct 22, 2023
… put_record API (aws#6063)

* Add mypy-boto3 stubs for kinesis

* Add remote invoke implementation for kinesis service

* unit tests

* fix typo in comment

Co-authored-by: Wing Fung Lau <[email protected]>

---------

Co-authored-by: Wing Fung Lau <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants