Releases: aws-powertools/powertools-lambda-python
v2.14.0
Summary
Duplicate: due to a GitHub glitch this notification might have appeared earlier.
This release adds support for 3.10.
Changes
🌟New features and non-breaking changes
- feat(runtime): add support for python 3.10 (#2137) by @rubenfonseca
🔧 Maintenance
- chore(deps-dev): bump filelock from 3.11.0 to 3.12.0 (#2142) by @dependabot
- feat(runtime): add support for python 3.10 (#2137) by @rubenfonseca
- chore(deps-dev): bump importlib-metadata from 6.3.0 to 6.4.1 (#2134) by @dependabot
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.114 to 1.26.115 (#2135) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.77.0 to 0.77.1 (#2133) by @dependabot
- chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.97.post1 to 1.26.115 (#2132) by @dependabot
- chore(github): new tech debt issue form (#2131) by @heitorlessa
- chore(deps-dev): bump pytest from 7.3.0 to 7.3.1 (#2127) by @dependabot
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.109 to 1.26.114 (#2126) by @dependabot
- chore(ci): add support for x86-64 regions only (#2122) by @rubenfonseca
This release was made possible by the following contributors:
@dependabot, @dependabot[bot], @heitorlessa, @rubenfonseca and Release bot
v2.13.0
Summary
This release improves the developer experience using the Parameters utility, fixes a bug on the CodePipeline event source and adds several documentation improvements!
⭐ Huge thanks to our new contributor @neilramsay
Improved DX on the Parameters utility
Docs: Cache TTL, SSMProvider
You can now set the POWERTOOLS_PARAMETERS_MAX_AGE
and POWERTOOLS_PARAMETERS_SSM_DECRYPT
environment variables to set defaults values for all parameters fetched by this utility. This avoids having to pass max_age
and decrypt
when you fetch several parameters.
Changes
🌟New features and non-breaking changes
- feat(parameters): Configure max_age and decrypt parameters via environment variables (#2088) by @leandrodamascena
📜 Documentation updates
- feat(parameters): Configure max_age and decrypt parameters via environment variables (#2088) by @leandrodamascena
- fix(ci): update layer version on logger, tracer and metrics docs (#2120) by @rubenfonseca
🐛 Bug and hot fixes
- fix(event_sources): Update CodePipeline event source to include optional encryption_key field and make user_parameters field optional (#2113) by @neilramsay
- fix(ci): update layer version on logger, tracer and metrics docs (#2120) by @rubenfonseca
🔧 Maintenance
- chore(deps-dev): bump flake8-comprehensions from 3.11.1 to 3.12.0 (#2124) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.73.0 to 2.74.0 (#2125) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.73.0 to 2.74.0 (#2123) by @dependabot
- chore(ci): bump the cdk-aws-lambda-powertools-layer version (#2121) by @rubenfonseca
- chore(maintenance): validate acknowledgement section is present (#2112) by @leandrodamascena
- chore(deps): bump codecov/codecov-action from 3.1.1 to 3.1.2 (#2110) by @dependabot
- chore(deps-dev): bump httpx from 0.23.3 to 0.24.0 (#2111) by @dependabot
- chore(deps-dev): bump mypy from 1.1.1 to 1.2.0 (#2096) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.76.2 to 0.77.0 (#2107) by @dependabot
- chore(deps-dev): bump pytest from 7.2.2 to 7.3.0 (#2106) by @dependabot
- chore(deps-dev): bump importlib-metadata from 6.1.0 to 6.3.0 (#2105) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.5 to 9.1.6 (#2104) by @dependabot
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.80 to 1.26.109 (#2103) by @dependabot
This release was made possible by the following contributors:
@dependabot, @dependabot[bot], @djfurman, @leandrodamascena, @neilramsay, @rubenfonseca and Release bot
v2.12.0
Summary
This release improves the authoring experience for Batch processing (-boilerplate), fix a bug between Batch and Pydantic integration for poison pills, and several documentation and static typing improvements!
⭐ Huge thanks to new contributors: @michael-k (docs) and @LuckIlNe (Batch pydantic bugfix)
No more decorators for Batch Processing
Docs: Amazon SQS, Amazon Kinesis Data Streams, Amazon DynamoDB Streams
Batch feature is generally used for building more resilient data pipelines ranging from ETL to Change Data Capture to AWS automation.
This release significantly removes boilerplate and makes it less error prone to author batch processors. You can now use process_partial_response
and async_process_partial_response
within your Lambda handler to kick off processing, handle failures and return the expected response for partial failures in Lambda.
Before
After
Addressing poison pill issue with Batch and Pydantic model validation
Pydantic provides parsing and deep data validation mechanisms. It's commonly used with Batch feature to author and validate data models.
Thanks to @LuckIlNe, we now correctly handle malformed messages that fail validation early (poison pills), thus not breaking the partial failure mechanism. Previously, any record that doesn't conform with an early model validation would not fail gracefully.
Documentation improvements
Idempotency
Docs: Idempotency new diagrams, terminology, and Idempotency record expiration vs DynamoDB time-to-live (TTL).
Thanks to a discussion with @pmarko1711, we revamped the Idempotency documentation to include more sequence diagrams to visually explain the different idempotent scenarios. We also clarified terminologies like Idempotency Record, and a confusion between Idempotency Record Expiration and DynamoDB TTL.
The latter can be used as a fine-grained concurrency mechanism.
Batch
Pydantic integration examples now use the new Json
model from Pydantic to auto-deserialize JSON strings into Pydantic models. This removes excessive boilerplate when working with SQS and Kinesis Data Streams in Batch.
All examples now recommend use of process_partial_response
over the legacy decorator, including a new question in the FAQ section to clarify them.
Static typing improvements in Parser
When using Mypy for applications using Pydantic models provided by Parser, you couldn't use nested models for SQS, Kinesis, and DynamoDB Streams.
This quality of life improvement also allows you to use higher-level models like Json
to auto-deserialize JSON strings in common Lambda Event Source fields like SQS.body
.
from aws_lambda_powertools.utilities.parser import BaseModel, validator
from aws_lambda_powertools.utilities.parser.models import (
DynamoDBStreamChangedRecordModel,
DynamoDBStreamRecordModel,
KinesisDataStreamRecord,
KinesisDataStreamRecordPayload,
SqsRecordModel,
)
from aws_lambda_powertools.utilities.parser.types import Json, Literal
class Order(BaseModel):
item: dict
# SQS example
class OrderSqs(SqsRecordModel):
body: Json[Order]
# Kinesis example
class OrderKinesisPayloadRecord(KinesisDataStreamRecordPayload):
data: Json[Order]
class OrderKinesisRecord(KinesisDataStreamRecord):
kinesis: OrderKinesisPayloadRecord
# DynamoDB Streams example
class OrderDynamoDBChangeRecord(DynamoDBStreamChangedRecordModel):
NewImage: Optional[OrderDynamoDB]
OldImage: Optional[OrderDynamoDB]
class OrderDynamoDBRecord(DynamoDBStreamRecordModel):
dynamodb: OrderDynamoDBChangeRecord
Changes
🌟New features and non-breaking changes
- feat(batch): reduce boilerplate with process_partial_response (#2090) by @heitorlessa
- feat(idempotency): allow custom sdk clients in DynamoDBPersistenceLayer (#2087) by @heitorlessa
📜 Documentation updates
- docs(batch): use newly supported Json model (#2100) by @heitorlessa
- feat(batch): reduce boilerplate with process_partial_response (#2090) by @heitorlessa
- docs(homepage): remove banner for end-of-support v1 (#2098) by @leandrodamascena
- docs(idempotency): fixes to testing your code section (#2073) by @leandrodamascena
- docs(parser): fix highlighted line (#2064) by @michael-k
- docs(idempotency): new sequence diagrams, fix idempotency record vs DynamoDB TTL confusion (#2074) by @heitorlessa
🐛 Bug and hot fixes
- fix(batch): handle early validation errors for pydantic models (poison pill) #2091 (#2099) by @LuckIlNe
🔧 Maintenance
- chore(deps-dev): bump aws-cdk-lib from 2.72.1 to 2.73.0 (#2097) by @dependabot
- chore(deps-dev): bump filelock from 3.10.7 to 3.11.0 (#2094) by @dependabot
- chore(deps-dev): bump coverage from 7.2.2 to 7.2.3 (#2092) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.72.1 to 2.73.0 (#2093) by @dependabot
- chore(deps-dev): bump mypy-boto3-cloudformation from 1.26.60 to 1.26.108 (#2095) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.11 to 2.8.19.12 (#2085) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.76.1 to 0.76.2 (#2084) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.72.0 to 2.72.1 (#2081) by @dependabot
- chore(deps): bump aws-xray-sdk from 2.11.0 to 2.12.0 (#2080) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.4 to 9.1.5 (#2077) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.72.0 to 2.72.1 (#2076) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.99 to 1.26.104 (#2075) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.71.0 to 2.72.0 (#2071) by @dependabot
- chore(deps): bump peaceiris/actions-gh-pages from 3.9.2 to 3.9.3 (#2069) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.71.0 to 2.72.0 (#2070) by @dependabot
- chore(deps-dev): bump black from 23.1.0 to 23.3.0 (#2066) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.70.0 to 2.71.0 (#2067) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.70.0 to 2.71.0 (#2065) by @dependabot
This release was made possible by the following contributors:
@LuckIlNe, @dependabot, @dependabot[bot], @heitorlessa, @leandrodamascena, @michael-k and Release bot
v2.11.0
Summary
This release adds support for S3 Event Notifications via EventBridge in Event Source Data Classes, and a bug fix for Feature Flags to handle falsy context values correctly.
⭐ Huge welcome to new contributors: @ajwad-shaikh and @ivica-k
S3 Event Notifications via EventBridge
A new use case is to process S3 Events with the newer and faster EventBridge integration. Previously, EventBridge integration with S3 would use CloudTrail leading up to a few minutes delays.
With the new integration, there is a slightly new payload that we had to support in order to ease that integration.
Thanks to @ivica-k, you can now use S3EventBridgeNotificationEvent
!
from aws_lambda_powertools.utilities.data_classes import S3EventBridgeNotificationEvent, event_source
from aws_lambda_powertools.utilities.typing import LambdaContext
@event_source(data_class=S3EventBridgeNotificationEvent)
def lambda_handler(event: S3EventBridgeNotificationEvent, context: LambdaContext):
bucket_name = event.detail.bucket.name
file_key = event.detail.object.key
new_storage_class = event.detail.destination_storage_class
...
Fixing falsy values in Feature Flags
Thanks to @ajwad-shaikh for reporting and fixing this bug (incl. @ran-isenberg), you can now handle flag variants when their context values are falsy, e.g., empty lists, empty strings, False
, 0, etc.
Example that wasn't possible before this release
Flag
{
"sales_offer": {
"default": false,
"rules": {
"unpaid_users": {
"when_match": true,
"conditions": [ {"action": "EQUALS", "key": "is_paid", "value": 0} ]
}
}
}
Context
{
"is_paid": 0,
}
Changes
🌟New features and non-breaking changes
📜 Documentation updates
- fix(feature_flags): make test conditions deterministic (#2059) by @ajwad-shaikh
- docs(logger): warn append_keys on not being thread-safe (#2046) by @heitorlessa
- feat(event_sources): support for S3 Event Notifications through EventBridge (#2024) by @ivica-k
🐛 Bug and hot fixes
- fix(feature_flags): make test conditions deterministic (#2059) by @ajwad-shaikh
- fix(feature_flags): handle expected falsy values in conditions (#2052) by @ran-isenberg
🔧 Maintenance
- chore(deps-dev): bump types-requests from 2.28.11.16 to 2.28.11.17 (#2061) by @dependabot
- chore(deps-dev): bump filelock from 3.10.4 to 3.10.7 (#2055) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.75.1 to 0.76.1 (#2056) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.10 to 2.8.19.11 (#2057) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.97.post2 to 1.26.99 (#2054) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.3 to 9.1.4 (#2050) by @dependabot
- chore(deps-dev): bump filelock from 3.10.2 to 3.10.4 (#2048) by @dependabot
- chore(deps-dev): bump mypy-boto3-cloudwatch from 1.26.52 to 1.26.99 (#2049) by @dependabot
- chore(deps-dev): bump filelock from 3.10.1 to 3.10.2 (#2045) by @dependabot
- chore(deps-dev): bump types-requests from 2.28.11.15 to 2.28.11.16 (#2044) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.97 to 1.26.97.post2 (#2043) by @dependabot
- chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.97 to 1.26.97.post1 (#2042) by @dependabot
- chore(deps-dev): bump filelock from 3.10.0 to 3.10.1 (#2036) by @dependabot
- chore(deps-dev): bump aws-cdk from 2.69.0 to 2.70.0 (#2039) by @dependabot
- chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.87 to 1.26.97 (#2035) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.62 to 1.26.97 (#2037) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.69.0 to 2.70.0 (#2038) by @dependabot
- chore(deps): bump pydantic from 1.10.6 to 1.10.7 (#2034) by @dependabot
- chore(deps-dev): bump mypy-boto3-ssm from 1.26.77 to 1.26.97 (#2033) by @dependabot
- chore(deps-dev): bump flake8-comprehensions from 3.11.0 to 3.11.1 (#2029) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.75.0 to 0.75.1 (#2027) by @dependabot
- chore(deps-dev): bump pytest-asyncio from 0.20.3 to 0.21.0 (#2026) by @dependabot
- feat(event_sources): support for S3 Event Notifications through EventBridge (#2024) by @ivica-k
This release was made possible by the following contributors:
@ajwad-shaikh, @dependabot, @dependabot[bot], @heitorlessa, @ivica-k, @ran-isenberg and Release bot
v2.10.0
Summary
This release brings support for parsing S3 Event Notifications via EventBridge and support for custom properties on the ActiveMQ event source data class.
⭐️ Huge thanks for our first-time contributors @alexaiss and @ivica-k, and also @tibbe for the typing hot-fix!
Parsing S3 event notifications via EventBridge
Amazon S3 can send event notifications with Amazon EventBridge. We added support for parsing those events in Lambda using the new S3EventNotificationEventBridgeModel
.
Thank you @robert-malai for capturing this feature gap!
Custom properties on ActiveMQ event
When integrating with Active MQ, Lambda supports JMS custom properties. After this change, you can also access them when using the ActiveMQ event source data class.
Changes
🌟New features and non-breaking changes
- feat(event_sources): support for custom properties in ActiveMQEvent (#1999) by @alexaiss
- feat(parser): support for S3 Event Notifications via EventBridge (#1982) by @ivica-k
📜 Documentation updates
- docs(metrics): fix high-resolution metrics announcement link (#2017) by @rubenfonseca
- feat(parser): support for S3 Event Notifications via EventBridge (#1982) by @ivica-k
- docs(homepage): revamp install UX & share how we build Lambda Layer (#1978) by @heitorlessa
🐛 Bug and hot fixes
- fix(build): auto-generate setup.py for legacy build tools (#2013) by @sthulb
- fix(typing): swap NoReturn with None for methods with no return value (#2004) by @tibbe
🔧 Maintenance
- chore(deps-dev): bump cfn-lint from 0.74.3 to 0.75.0 (#2020) by @dependabot
- chore(deps-dev): bump coverage from 7.2.1 to 7.2.2 (#2021) by @dependabot
- chore(deps-dev): bump filelock from 3.9.1 to 3.10.0 (#2019) by @dependabot
- fix(build): auto-generate setup.py for legacy build tools (#2013) by @sthulb
- chore(deps-dev): bump aws-cdk from 2.67.0 to 2.69.0 (#2010) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.2 to 9.1.3 (#2009) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.74.2 to 0.74.3 (#2008) by @dependabot
- chore(deps-dev): bump filelock from 3.9.0 to 3.9.1 (#2006) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.68.0 to 2.69.0 (#2007) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.74.1 to 0.74.2 (#2005) by @dependabot
- chore(deps-dev): bump mypy from 0.982 to 1.1.1 (#1985) by @dependabot
- chore(deps-dev): bump pytest-xdist from 3.2.0 to 3.2.1 (#2000) by @dependabot
- chore(deps-dev): bump flake8-bugbear from 23.2.13 to 23.3.12 (#2001) by @dependabot
- chore(deps-dev): bump bandit from 1.7.4 to 1.7.5 (#1997) by @dependabot
- chore(deps): bump docker/setup-buildx-action from 2.4.1 to 2.5.0 (#1995) by @dependabot
- chore(deps-dev): bump mypy-boto3-secretsmanager from 1.26.49 to 1.26.89 (#1996) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.1 to 9.1.2 (#1994) by @dependabot
- chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.84 to 1.26.87 (#1993) by @dependabot
- chore(deps): bump pydantic from 1.10.5 to 1.10.6 (#1991) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.67.0 to 2.68.0 (#1992) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.74.0 to 0.74.1 (#1988) by @dependabot
- chore(deps): bump aws-actions/configure-aws-credentials from 1 to 2 (#1987) by @dependabot
- chore(deps-dev): bump pytest from 7.2.1 to 7.2.2 (#1980) by @dependabot
- chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.1.1 to 2.1.2 (#1979) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.9 to 2.8.19.10 (#1973) by @dependabot
- chore(deps-dev): bump hvac from 1.0.2 to 1.1.0 (#1983) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.1.0 to 9.1.1 (#1984) by @dependabot
- chore(deps-dev): bump mypy-boto3-dynamodb from 1.26.24 to 1.26.84 (#1981) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.15 to 9.1.0 (#1976) by @dependabot
- chore(deps-dev): bump cfn-lint from 0.67.0 to 0.74.0 (#1974) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.66.1 to 2.67.0 (#1977) by @dependabot
This release was made possible by the following contributors:
@alexaiss, @dependabot, @dependabot[bot], @heitorlessa, @ivica-k, @rubenfonseca, @sthulb, @tibbe and Release bot
2.9.1
Summary
This patch release reverts a regression in Idempotency when using static_pk_value
for custom composite keys in DynamoDB Persistence Layer - huge thanks to @Tankanow for reporting and fixing it.
⭐ Big thanks to new contributors: @KeltonKarboviak on inconsistencies on Parameters docs (#1966), and @cibinmathew on upgrading GitHub Actions set-output
command in our workflows.
Changes
📜 Documentation updates
- docs(appsync): add mutation example and infrastructure fix (#1964) by @heitorlessa
- docs(parameters): fix typos and inconsistencies (#1966) by @KeltonKarboviak
🐛 Bug and hot fixes
🔧 Maintenance
- chore(parser): add workaround to make API GW test button work (#1971) by @heitorlessa
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.55 to 1.26.80 (#1967) by @dependabot
- chore(ci): replace deprecated set-output commands (#1957) by @cibinmathew
- chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.1.0 to 2.1.1 (#1958) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.14 to 9.0.15 (#1959) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.8 to 2.8.19.9 (#1960) by @dependabot
- chore(deps): bump fastjsonschema from 2.16.2 to 2.16.3 (#1961) by @dependabot
- chore(deps-dev): bump coverage from 7.2.0 to 7.2.1 (#1963) by @dependabot
- chore(deps-dev): bump types-requests from 2.28.11.14 to 2.28.11.15 (#1962) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.66.0 to 2.66.1 (#1954) by @dependabot
- chore(deps-dev): bump coverage from 7.1.0 to 7.2.0 (#1951) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.13 to 9.0.14 (#1952) by @dependabot
- chore(deps-dev): bump mypy-boto3-ssm from 1.26.43 to 1.26.77 (#1949) by @dependabot
- chore(deps): bump release-drafter/release-drafter from 5.22.0 to 5.23.0 (#1947) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.65.0 to 2.66.0 (#1948) by @dependabot
- chore(deps-dev): bump types-requests from 2.28.11.13 to 2.28.11.14 (#1946) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.7 to 2.8.19.8 (#1945) by @dependabot
This release was made possible by the following contributors:
@KeltonKarboviak, @cibinmathew, @Tankanow, @dependabot, @dependabot[bot], @heitorlessa and Release bot
v2.9.0
Summary
This release contains a new feature on Batch Processing, along with enhancements and bug fixes to Logger and Feature Flags utilities.
⭐ Huge welcome to the new contributors: @royygael, @iago1460
SQS FIFO Batch Processing
You can now use SQS FIFO queues with the Batch Processing utility. We stop processing messages after the first failure and return all failed and unprocessed messages natively. This helps preserve the ordering of messages in your queue.
⭐ Thanks to @whoDoneItAgain for reporting this feature gap.
Logger improvements
We now allow exception
and exception_name
attributes as extra fields in all logging levels. Previously, we dropped those attributes in non-error logging levels (e.g: INFO
).
⭐ Thanks to @iago1460 for reporting this bug and submitting the fix!
New community content
Finally, we have two new community posts on Feature Flags from @ran-isenberg:
- How CyberArk uses Powertools to implement Feature Flags with AWS AppConfig: link
- A talk about the added value of using feature flags as part of your CI/CD process and how AWS Lambda Powertools can help: link
Changes
🌟New features and non-breaking changes
- feat(batch): add support to SQS FIFO queues (SqsFifoPartialProcessor) (#1934) by @rubenfonseca
📜 Documentation updates
- docs(we-made-this): add CI/CD using Feature Flags video (#1940) by @leandrodamascena
- feat(batch): add support to SQS FIFO queues (SqsFifoPartialProcessor) (#1934) by @rubenfonseca
- docs(event-handler): Fix REST API - HTTP Methods documentation (#1936) by @leandrodamascena
- docs(we-made-this): add Feature Flags post (#1939) by @leandrodamascena
🐛 Bug and hot fixes
- fix(metrics): clarify no-metrics user warning (#1935) by @rubenfonseca
- fix(logger): support exception and exception_name fields at any log level (#1930) by @iago1460
- fix(feature-flags): revert RuleAction Enum inheritance on str (#1910) by @royygael
🔧 Maintenance
- chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.0.5 to 2.1.0 (#1943) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.12 to 9.0.13 (#1944) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.64.0 to 2.65.0 (#1938) by @dependabot
- chore(deps-dev): bump types-python-dateutil from 2.8.19.6 to 2.8.19.7 (#1932) by @dependabot
- chore(deps-dev): bump types-requests from 2.28.11.12 to 2.28.11.13 (#1933) by @dependabot
- chore(deps): bump pydantic from 1.10.4 to 1.10.5 (#1931) by @dependabot
- chore(deps-dev): bump mypy-boto3-appconfig from 1.26.63 to 1.26.71 (#1928) by @dependabot
- chore(deps-dev): bump flake8-bugbear from 23.1.20 to 23.2.13 (#1924) by @dependabot
- chore(deps-dev): bump mypy-boto3-appconfigdata from 1.26.0.post1 to 1.26.70 (#1925) by @dependabot
This release was made possible by the following contributors:
@dependabot, @dependabot[bot], @heitorlessa, @iago1460, @leandrodamascena, @royygael, @rubenfonseca and Release bot
v2.8.0
Summary
This release is packed with new features and enhancements, specifically to Batch Processing, Metrics and Idempotency utility.
⭐ Huge welcome to new contributors: @BakasuraRCE, @prudnikov, @kavichu, @marcraminv
High-concurrency async processor
Thanks to @BakasuraRCE, we’re adding a new specialized processor for high concurrency jobs that can process thousands of messages sub-second (thanks to @sthulb for load testing) - AsyncBatchProcessor
and async_batch_processor
.
These will use Python’s asyncio.gather
to call your record handler with all records at the same time. During load testing, we’ve managed to process 5500 messages in approximately 300ms compared to 1.2 seconds with BatchProcessor
synchronous counterpart.
1-second resolution metrics
You can now create high-resolution (1s) metrics that are common for use cases like IoT, telemetry, time-series, real-time incident, etc. Previously, metrics were only created with 60-second resolution. This is an opt-in feature.
Inheriting dimensions in single_metric
Thanks to @prudnikov, you can now optionally set default dimensions for single metrics. These are individual application or operational metrics that despite being isolated from a larger set of metrics, they do share the same set of dimensions (avoid repetition, sparse metrics).
Idempotent_function decorator is now thread-safe
Thanks to @mploski, any Python function decorated with idempotent_function
can now safely be called from threads. Previously, we used to create higher-level AWS SDK clients which didn’t support threading - calling functions decorated back then could result in unforeseen side-effects.
Example
import os
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from threading import current_thread
from aws_lambda_powertools.utilities.idempotency import (
DynamoDBPersistenceLayer,
idempotent_function,
)
TABLE_NAME = os.getenv("IdempotencyTable", "")
persistence_layer = DynamoDBPersistenceLayer(table_name=TABLE_NAME)
threads_count = 2
@idempotent_function(persistence_store=persistence_layer, data_keyword_argument="record")
def record_handler(record):
time_now = time.time()
return {"thread_name": current_thread().name, "time": str(time_now)}
def lambda_handler(event, context):
with ThreadPoolExecutor(max_workers=threads_count) as executor:
futures = [executor.submit(record_handler, **{"record": event}) for _ in range(threads_count)]
return [
{"state": future._state, "exception": future.exception(), "output": future.result()}
for future in as_completed(futures)
]
Changes
🌟New features and non-breaking changes
- feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
- feat(metrics): add default_dimensions to single_metric (#1880) by @prudnikov
📜 Documentation updates
- docs(homepage): Replace poetry command to add group parameter (#1917) by @marcraminv
- feat(metrics) - add support for high resolution metrics (#1915) by @leandrodamascena
- feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
- fix(idempotency): make idempotent_function decorator thread safe (#1899) by @mploski
- docs(idempotency): add IAM permissions section (#1902) by @leandrodamascena
- docs(homepage): set url for end-of-support in announce block (#1893) by @kavichu
- feat(metrics): add default_dimensions to single_metric (#1880) by @prudnikov
🐛 Bug and hot fixes
🔧 Maintenance
- feat(batch): add async_batch_processor for concurrent processing (#1724) by @BakasuraRCE
- chore(deps-dev): bump mkdocs-material from 9.0.11 to 9.0.12 (#1919) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.63.2 to 2.64.0 (#1918) by @dependabot
- chore(pypi): add new links to Pypi package homepage (#1912) by @leandrodamascena
- chore(deps-dev): bump types-requests from 2.28.11.8 to 2.28.11.12 (#1906) by @dependabot
- chore(deps-dev): bump pytest-xdist from 3.1.0 to 3.2.0 (#1905) by @dependabot
- chore(deps): bump docker/setup-buildx-action from 2.4.0 to 2.4.1 (#1903) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.63.0 to 2.63.2 (#1904) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.10 to 9.0.11 (#1896) by @dependabot
- chore(deps-dev): bump mypy-boto3-appconfig from 1.26.0.post1 to 1.26.63 (#1895) by @dependabot
- docs(homepage): set url for end-of-support in announce block (#1893) by @kavichu
- chore(deps-dev): bump mkdocs-material from 9.0.9 to 9.0.10 (#1888) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.58 to 1.26.62 (#1889) by @dependabot
- chore(deps-dev): bump black from 22.12.0 to 23.1.0 (#1886) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.62.2 to 2.63.0 (#1887) by @dependabot
This release was made possible by the following contributors:
@BakasuraRCE, @dependabot, @dependabot[bot], @heitorlessa, @kavichu, @leandrodamascena, @marcraminv, @mploski, @prudnikov and Release bot
v2.7.1
Summary
This patch release addresses three areas:
Big welcome to our new contributors: @ashutoshojha5, @meckhardt
- Licensing. PyPi/Poetry license correction to
MIT No Attribution
(previouslyMIT-0
was unrecognized) - huge thanks to @meckhardt. - Documentation. Recreates 42 code snippets in the Parameters documentation to be more realistic, formatted, and linted as part of our larger campaign. It also corrects a CloudWatch Logs code snippet in the Event Source Data Classes documentation - thanks to @ashutoshojha5!
- End of support for v1. We formally set March 31st as our last day to support new bugfixes to V1, and recommend customers to follow our Upgrade Guide.
Changes
📜 Documentation updates
- docs(homepage): add banner for end-of-support v1 (#1879) by @leandrodamascena
- docs(parameters): snippets split, improved, and lint (#1564) by @leandrodamascena
- docs(event-source): fix incorrect method in example CloudWatch Logs (#1857) by @ashutoshojha5
🐛 Bug and hot fixes
- fix(license): correction to MIT + MIT-0 (no proprietary anymore) (#1883) by @meckhardt
- fix(ci): add auth to API HTTP Gateway and Lambda Function Url (#1882) by @rubenfonseca
- fix(license): add MIT-0 license header (#1871) by @meckhardt
- fix(tests): make logs fetching more robust (#1878) by @rubenfonseca
- fix(tests): make sure multiple e2e tests run concurrently (#1861) by @rubenfonseca
🔧 Maintenance
- fix(license): correction to MIT + MIT-0 (no proprietary anymore) (#1883) by @meckhardt
- fix(ci): add auth to API HTTP Gateway and Lambda Function Url (#1882) by @rubenfonseca
- fix(license): add MIT-0 license header (#1871) by @meckhardt
- chore(deps-dev): bump flake8-bugbear from 22.12.6 to 23.1.20 (#1854) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.6 to 9.0.8 (#1874) by @dependabot
- chore(deps): bump docker/setup-buildx-action from 2.0.0 to 2.4.0 (#1873) by @dependabot
- chore(deps-dev): bump isort from 5.11.4 to 5.11.5 (#1875) by @dependabot
- fix(tests): make sure multiple e2e tests run concurrently (#1861) by @rubenfonseca
- chore(deps-dev): bump aws-cdk-lib from 2.62.1 to 2.62.2 (#1869) by @dependabot
- chore(deps-dev): bump mypy-boto3-s3 from 1.26.0.post1 to 1.26.58 (#1868) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.62.0 to 2.62.1 (#1866) by @dependabot
- chore(deps-dev): bump mypy-boto3-cloudformation from 1.26.35.post1 to 1.26.57 (#1865) by @dependabot
- chore(deps-dev): bump coverage from 7.0.5 to 7.1.0 (#1862) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.61.1 to 2.62.0 (#1863) by @dependabot
- chore(deps): bump dependabot/fetch-metadata from 1.3.5 to 1.3.6 (#1855) by @dependabot
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.49 to 1.26.55 (#1856) by @dependabot
This release was made possible by the following contributors:
@ashutoshojha5, @dependabot, @dependabot[bot], @leandrodamascena, @meckhardt, @rubenfonseca and Release bot
v2.7.0
Summary
This release adds support for time-based Feature Flags, and simplified support for returning a custom HTTP status code on the event handler.
Feature Flags
This release enhanced the Feature Flags utility with a new time-based feature. You can now enable certain features in your application during the weekends, at a certain period or day, across one or more time zones. Big thanks to @ran-isenberg! Check out this sample that enables a feature on weekends in NYC TZ:
Event Handler
We also have a nice addition to our event handler utility. You can now add a second return argument if you want to customize the returned HTTP status code. ⭐ Big thanks to @theoutsider24 for your first contribution! ⭐
Changes
🌟New features and non-breaking changes
- feat(feature_flags): Add Time based feature flags actions (#1846) by @ran-isenberg, @rubenfonseca
- feat(event_handlers) Add support for returning dict, status-code tuples from resolvers (#1853) by @theoutsider24
📜 Documentation updates
- feat(feature_flags): Add Time based feature flags actions (#1846) by @ran-isenberg
🔧 Maintenance
- chore(deps-dev): bump mkdocs-material from 9.0.5 to 9.0.6 (#1851) by @dependabot
- chore(deps-dev): bump mypy-boto3-logs from 1.26.49 to 1.26.53 (#1850) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.60.0 to 2.61.1 (#1849) by @dependabot
- chore(deps-dev): bump mypy-boto3-cloudwatch from 1.26.30 to 1.26.52 (#1847) by @dependabot
- feat(feature_flags): Add Time based feature flags actions (#1846) by @leandrodamascena
- chore(deps-dev): bump types-requests from 2.28.11.7 to 2.28.11.8 (#1843) by @dependabot
- chore(deps): bump peaceiris/actions-gh-pages from 3.9.1 to 3.9.2 (#1841) by @dependabot
- chore(deps-dev): bump pytest from 7.2.0 to 7.2.1 (#1838) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.4 to 9.0.5 (#1840) by @dependabot
- chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions from 2.0.4 to 2.0.5 (#1837) by @dependabot
- chore(deps): bump future from 0.18.2 to 0.18.3 (#1836) by @dependabot
- chore(deps-dev): bump mkdocs-material from 9.0.3 to 9.0.4 (#1833) by @dependabot
- chore(deps-dev): bump mypy-boto3-logs from 1.26.43 to 1.26.49 (#1834) by @dependabot
- chore(deps-dev): bump mypy-boto3-secretsmanager from 1.26.40 to 1.26.49 (#1835) by @dependabot
- chore(deps-dev): bump mypy-boto3-lambda from 1.26.18 to 1.26.49 (#1832) by @dependabot
- chore(deps-dev): bump aws-cdk-lib from 2.59.0 to 2.60.0 (#1831) by @dependabot
This release was made possible by the following contributors:
@dependabot, @dependabot[bot], @leandrodamascena, @ran-isenberg, @rubenfonseca, @theoutsider24 and Release bot