You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/lambda-features/durable-functions.md
+16-53Lines changed: 16 additions & 53 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,54 +14,44 @@ description: Using Powertools for AWS Lambda (Python) with Lambda Durable Functi
14
14
|**Durable execution**| Complete lifecycle of a durable function, from start to completion |
15
15
|**Checkpoint**| Saved state that tracks progress through the workflow |
16
16
|**Replay**| Re-execution from the beginning, skipping completed checkpoints |
17
-
|**Steps**| Business logic with built-in retries and progress tracking |
18
-
|**Waits**| Suspend execution without incurring compute charges |
17
+
|**Step**| Business logic with built-in retries and progress tracking |
18
+
|**Wait**| Suspend execution without incurring compute charges |
19
19
20
20
## How it works
21
21
22
22
Durable functions use a **checkpoint/replay mechanism**:
23
23
24
-
1. Your code runs from the beginning
24
+
1. Your code runs always from the beginning
25
25
2. Completed operations are skipped using stored results
26
-
3. Execution continues from where it left off
26
+
3. Execution of new steps continues from where it left off
27
27
4. State is automatically managed by the SDK
28
28
29
29
## Powertools integration
30
30
31
-
Powertools for AWS Lambda (Python) works seamlessly with Durable Functions. The [Durable Execution SDK](https://github.com/aws/aws-durable-execution-sdk-python){target="_blank" rel="nofollow"} has native integration with Powertools Logger via `context.set_logger()`.
31
+
Powertools for AWS Lambda (Python) works seamlessly with Durable Functions. The [Durable Execution SDK](https://github.com/aws/aws-durable-execution-sdk-python){target="_blank" rel="nofollow"} has native integration with Logger via `context.set_logger()`.
32
32
33
33
???+ note "Found an issue?"
34
-
If you encounter any issues using Powertools for AWS with Durable Functions, please [open an issue](https://github.com/aws-powertools/powertools-lambda-python/issues/new?template=bug_report.yml){target="_blank"}.
34
+
If you encounter any issues using Powertools for AWS Lambda (Python) with Durable Functions, please [open an issue](https://github.com/aws-powertools/powertools-lambda-python/issues/new?template=bug_report.yml){target="_blank"}.
35
35
36
36
### Logger
37
37
38
-
The Durable Execution SDK provides a `context.logger` that automatically handles **log deduplication during replays**. You can integrate Powertools Logger to get structured JSON logging while keeping the deduplication benefits.
38
+
The Durable Execution SDK provides a `context.logger`instance that automatically handles **log deduplication during replays**. You can integrate Logger to get structured JSON logging while keeping the deduplication benefits.
39
39
40
-
#### Using Powertools Logger with context.set_logger
40
+
For the best experience, set the Logger on the durable context. This gives you structured JSON logging with automatic log deduplication during replays:
41
41
42
-
For the best experience, set the Powertools Logger on the durable context:
43
-
44
-
```python hl_lines="5 10" title="Integrating Powertools Logger with Durable Functions"
42
+
```python hl_lines="5 11 14 22" title="Integrating Logger with Durable Functions"
If you use the Powertools Logger directly (not through `context.logger`), logs will be emitted on every replay:
54
+
If you use the Logger directly (not through `context.logger`), logs will be emitted on every replay:
65
55
66
56
```python
67
57
# Logs will duplicate during replays
@@ -76,23 +66,20 @@ When you use `context.logger`, the SDK prevents duplicate logs during replays:
76
66
Tracer works with Durable Functions. Each execution creates trace segments.
77
67
78
68
???+ note "Trace continuity"
79
-
Due to the replay mechanism, traces may not show a continuous flow. Each execution (including replays) creates separate trace segments. Use the `execution_arn` to correlate traces.
69
+
Due to the replay mechanism, traces may be interleaved. Each execution (including replays) creates separate trace segments. Use the `execution_arn` to correlate traces.
80
70
81
71
```python hl_lines="5 9" title="Using Tracer with Durable Functions"
Metrics work with Durable Functions, but be aware that **metrics may be emitted multiple times** during replay if not handled carefully.
77
+
Metrics work with Durable Functions, but be aware that **metrics may be emitted multiple times** during replay if not handled carefully. Emit metrics at workflow completion rather than during intermediate steps to avoid counting replays as new executions.
88
78
89
-
```python hl_lines="6 10 21" title="Using Metrics with Durable Functions"
Emit metrics at workflow completion rather than during intermediate steps to avoid counting replays as new executions.
95
-
96
83
### Idempotency
97
84
98
85
The `@idempotent` decorator integrates with Durable Functions and is **replay-aware**. It's useful for protecting the Lambda handler entry point, especially for Event Source Mapping (ESM) invocations like SQS, Kinesis, or DynamoDB Streams.
@@ -111,14 +98,6 @@ The `@idempotent` decorator integrates with Durable Functions and is **replay-aw
111
98
112
99
- Steps within a durable function are already idempotent via the checkpoint mechanism
113
100
114
-
### Parser
115
-
116
-
Parser works with Durable Functions for validating and parsing event payloads.
117
-
118
-
```python hl_lines="9 14" title="Using Parser with Durable Functions"
@@ -128,26 +107,10 @@ Parameters work normally with Durable Functions.
128
107
```
129
108
130
109
???+ note "Parameter freshness"
131
-
For long-running workflows (hours/days), parameters fetched at the start may become stale. Consider fetching parameters within steps that need the latest values.
110
+
If the replay or execution happens within the cache TTL on the same execution environment, the parameter value may come from cache. For long-running workflows (hours/days), parameters fetched at the start may become stale. Consider fetching parameters within steps that need the latest values, and customize the caching behavior with `max_age` to control freshness.
132
111
133
112
## Best practices
134
113
135
-
### Use context.logger for log deduplication
136
-
137
-
Always use `context.set_logger()` and `context.logger` instead of using the Powertools Logger directly. This ensures logs are deduplicated during replays.
When your durable function is triggered by Event Source Mappings (SQS, Kinesis, DynamoDB Streams), use the `@idempotent` decorator to protect against duplicate invocations.
Copy file name to clipboardExpand all lines: docs/lambda-features/managed-instances.md
+17-81Lines changed: 17 additions & 81 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,84 +7,52 @@ description: Using Powertools for AWS Lambda (Python) with Lambda Managed Instan
7
7
8
8
[Lambda Managed Instances](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html){target="_blank" rel="nofollow"} enables you to run Lambda functions on Amazon EC2 instances without managing infrastructure. It supports multi-concurrent invocations, EC2 pricing models, and specialized compute options like Graviton4.
|**Concurrency**| Single invocation per execution environment | Multiple concurrent invocations per environment |
15
15
|**Python model**| One process, one request | Multiple processes, one request each |
16
16
|**Pricing**| Per-request duration | EC2-based with Savings Plans support |
17
-
|**Scaling**| Scale on demand with cold starts | Async scaling based on CPU, no cold starts|
17
+
|**Scaling**| Scale on demand with cold starts | Async scaling based on CPU|
18
18
|**Isolation**| Firecracker microVMs | Containers on EC2 Nitro |
19
19
20
20
## How Lambda Python runtime handles concurrency
21
21
22
-
Unlike Java or Node.js which use threads, the**Lambda Python runtime uses multiple processes** for concurrent requests. Each request runs in a separate process, which provides natural isolation between requests.
22
+
The**Lambda Python runtime uses multiple processes** for concurrent requests. Each request runs in a separate process, which provides natural isolation between requests.
23
23
24
24
This means:
25
25
26
-
-**Memory is not shared** between concurrent requests
27
-
-**Global variables** are isolated per process
26
+
-**Each process has its own memory** - global variables are isolated per process
28
27
-**`/tmp` directory is shared** across all processes - use caution with file operations
29
28
30
-
## Isolation model
31
-
32
-
Lambda Managed Instances use a different isolation model than Lambda (default):
|**Function isolation**| Strong isolation via microVMs | Container-based isolation within instances |
39
-
40
-
**Capacity providers** serve as the security boundary. Functions within the same capacity provider share the underlying EC2 instances. For workloads requiring strong isolation between functions, use separate capacity providers.
41
-
42
-
For Python specifically, the multi-process model adds another layer of isolation - each concurrent request runs in its own process with separate memory space.
29
+
For more details on the isolation model, see [Lambda Managed Instances documentation](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html){target="_blank" rel="nofollow"}.
43
30
44
31
## Powertools integration
45
32
46
33
Powertools for AWS Lambda (Python) works seamlessly with Lambda Managed Instances. All utilities are compatible with the multi-process concurrency model used by Python.
47
34
48
-
### Logger
49
-
50
-
Logger works without any changes. Each process has its own logger instance.
51
-
52
-
```python hl_lines="4 7" title="Using Logger with Managed Instances"
Tracer works without any changes. X-Ray traces are captured per request.
37
+
Core utilities work without any changes. Each process has its own instances, so correlation IDs and traces are naturally isolated per request.
59
38
60
39
???+ note "VPC connectivity required"
61
-
Lambda Managed Instances run in your VPC. Ensure you have [network connectivity](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-networking.html){target="_blank" rel="nofollow"} to send traces to X-Ray.
40
+
Lambda Managed Instances run in your VPC. Ensure you have [network connectivity](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-networking.html){target="_blank" rel="nofollow"} to send logs to CloudWatch, traces to X-Ray, and metrics to CloudWatch.
62
41
63
-
```python hl_lines="4 8 12" title="Using Tracer with Managed Instances"
42
+
```python hl_lines="5 6 7 10 11 12 20 25" title="Using Logger, Tracer, and Metrics with Managed Instances"
Metrics work without any changes. Each process flushes metrics independently.
70
-
71
-
???+ note "VPC connectivity required"
72
-
Ensure you have [network connectivity](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-networking.html){target="_blank" rel="nofollow"} to send metrics to CloudWatch.
73
-
74
-
```python hl_lines="5 9 12" title="Using Metrics with Managed Instances"
Since each process has its own cache, you might see more calls to SSM/Secrets Manager during initial warm-up. Once each process has cached the value, subsequent requests within that process use the cache.
55
+
Since each process has its own cache, you might see more calls to SSM/Secrets Manager during initial warm-up. Once each process has cached the value, subsequent requests within that process use the cache. You can customize the caching behavior with `max_age` to control the TTL.
88
56
89
57
### Idempotency
90
58
@@ -94,35 +62,6 @@ Idempotency works without any changes. It uses DynamoDB for state management, wh
All other Powertools for AWS utilities (Feature Flags, Validation, Parser, Data Masking, etc.) work without any changes. If you encounter any issues, please [open an issue](https://github.com/aws-powertools/powertools-lambda-python/issues/new?template=bug_report.yml){target="_blank"}.
107
-
108
-
## Working with shared resources
109
-
110
-
### The `/tmp` directory
111
-
112
-
The `/tmp` directory is **shared across all processes** in the execution environment. Use caution when writing files.
113
-
114
-
```python title="Safe file handling with unique names"
@@ -136,18 +75,19 @@ Configure connectivity using one of these options:
136
75
1.**VPC Endpoints** - Private connectivity without internet access
137
76
2.**NAT Gateway** - Internet access from private subnets
138
77
3.**Public subnet with Internet Gateway** - Direct internet access
78
+
4.**Egress-only Internet Gateway** - IPv6 outbound connectivity without inbound access ([learn more](https://docs.aws.amazon.com/vpc/latest/userguide/egress-only-internet-gateway.html){target="_blank" rel="nofollow"})
139
79
140
80
See [Networking for Lambda Managed Instances](https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-networking.html){target="_blank" rel="nofollow"} for detailed setup instructions.
141
81
142
82
## FAQ
143
83
144
84
### Does Powertools for AWS Lambda (Python) work with Lambda Managed Instances?
145
85
146
-
Yes, all Powertools for AWS utilities work seamlessly with Lambda Managed Instances. The multi-process model in Python provides natural isolation between concurrent requests.
86
+
Yes, all Powertools for AWS Lambda (Python) utilities work seamlessly with Lambda Managed Instances. The multi-process model in Python provides natural isolation between concurrent requests.
147
87
148
88
### Is my code thread-safe?
149
89
150
-
For Python, you don't need to worry about thread safety because Lambda Managed Instances uses **multiple processes**, not threads. Each request runs in its own process with isolated memory.
90
+
Lambda Managed Instances uses **multiple processes**, instead of threads. Each request runs in its own process with isolated memory. If you implement multi-threading within your handler, you are responsible for thread safety.
151
91
152
92
### Why is my cache not shared between requests?
153
93
@@ -157,10 +97,6 @@ Each process maintains its own cache (for Parameters, Feature Flags, etc.). This
157
97
158
98
Yes, but remember they are **per-process**, not shared across concurrent requests. This is actually safer than shared state.
159
99
160
-
### How should I handle files in `/tmp`?
161
-
162
-
Use unique file names (include request ID or UUID) to avoid conflicts between concurrent requests. Always clean up files after use to avoid filling the shared `/tmp` directory.
163
-
164
-
### Do I need to change my existing Powertools for AWS code?
100
+
### Do I need to change my existing Powertools for AWS Lambda (Python) code?
165
101
166
-
No changes are required. Your existing code will work as-is with Lambda Managed Instances.
102
+
No changes are required if you are running Powertools for AWS Lambda (Python) version **3.4.0** or later. Your existing code will work as-is with Lambda Managed Instances.
0 commit comments