|
1 | | -# Retry Mechanisms Enhancement |
| 1 | +# Retry Mechanisms |
2 | 2 |
|
3 | | -This document describes the improvements made to CodeceptJS retry mechanisms to eliminate overlaps and provide better coordination. |
| 3 | +This document describes the retry coordination system in CodeceptJS. |
4 | 4 |
|
5 | 5 | ## Problem Statement |
6 | 6 |
|
7 | | -CodeceptJS previously had multiple retry mechanisms that could overlap and conflict: |
| 7 | +CodeceptJS has multiple retry mechanisms at different levels: |
8 | 8 |
|
9 | 9 | 1. **Global Retry Configuration** - Feature and Scenario level retries |
10 | 10 | 2. **RetryFailedStep Plugin** - Individual step retries |
11 | 11 | 3. **Manual Step Retries** - `I.retry()` calls |
12 | 12 | 4. **Hook Retries** - Before/After hook retries |
13 | | -5. **Helper Retries** - Helper-specific retry mechanisms |
14 | 13 |
|
15 | 14 | These mechanisms could result in: |
16 | 15 |
|
17 | 16 | - Exponential retry counts (e.g., 3 scenario retries × 2 step retries = 6 total executions per step) |
18 | 17 | - Conflicting configurations with no clear precedence |
19 | | -- Confusing logging and unclear behavior |
20 | | -- Difficult debugging when multiple retry levels were active |
21 | 18 |
|
22 | 19 | ## Solution Overview |
23 | 20 |
|
24 | | -### 1. Enhanced Global Retry (`lib/listener/enhancedGlobalRetry.js`) |
| 21 | +CodeceptJS now includes a priority-based coordination system to prevent conflicts. |
25 | 22 |
|
26 | | -**New Features:** |
27 | | - |
28 | | -- Priority-based retry coordination |
29 | | -- Clear precedence system |
30 | | -- Enhanced logging with mechanism identification |
31 | | -- Backward compatibility with existing configurations |
32 | | - |
33 | | -**Priority System:** |
| 23 | +### Priority System |
34 | 24 |
|
35 | 25 | ```javascript |
36 | 26 | const RETRY_PRIORITIES = { |
37 | | - MANUAL_STEP: 100, // I.retry() or step.retry() - highest priority |
38 | | - STEP_PLUGIN: 50, // retryFailedStep plugin |
39 | | - SCENARIO_CONFIG: 30, // Global scenario retry config |
40 | | - FEATURE_CONFIG: 20, // Global feature retry config |
41 | | - HOOK_CONFIG: 10, // Hook retry config - lowest priority |
| 27 | + MANUAL_STEP: 100, // I.retry() or step.retry() - highest priority |
| 28 | + STEP_PLUGIN: 50, // retryFailedStep plugin |
| 29 | + SCENARIO_CONFIG: 30, // Global scenario retry config |
| 30 | + FEATURE_CONFIG: 20, // Global feature retry config |
| 31 | + HOOK_CONFIG: 10, // Hook retry config - lowest priority |
42 | 32 | } |
43 | 33 | ``` |
44 | 34 |
|
45 | | -### 2. Enhanced RetryFailedStep Plugin (`lib/plugin/enhancedRetryFailedStep.js`) |
46 | | - |
47 | | -**New Features:** |
48 | | - |
49 | | -- Automatic coordination with scenario-level retries |
50 | | -- Smart deferral when scenario retries are configured |
51 | | -- Priority-aware retry registration |
52 | | -- Improved logging and debugging information |
53 | | - |
54 | | -**Coordination Logic:** |
55 | | - |
56 | | -- When scenario retries are configured, step retries are automatically deferred to avoid excessive retry counts |
57 | | -- Users can override this with `deferToScenarioRetries: false` |
58 | | -- Clear logging explains coordination decisions |
59 | | - |
60 | | -### 3. Retry Coordinator (`lib/retryCoordinator.js`) |
61 | | - |
62 | | -**New Features:** |
63 | | - |
64 | | -- Central coordination service for all retry mechanisms |
65 | | -- Configuration validation with warnings for potential conflicts |
66 | | -- Retry registration and priority management |
67 | | -- Summary reporting for debugging |
68 | | - |
69 | | -**Key Functions:** |
70 | | - |
71 | | -- `validateConfig()` - Detects configuration conflicts and excessive retry counts |
72 | | -- `registerRetry()` - Registers retry mechanisms with priority coordination |
73 | | -- `getEffectiveRetryConfig()` - Returns the active retry configuration for a target |
74 | | -- `generateRetrySummary()` - Provides debugging information about active retry mechanisms |
75 | | - |
76 | | -## Migration Guide |
77 | | - |
78 | | -### Immediate Benefits (No Changes Needed) |
79 | | - |
80 | | -The enhanced retry mechanisms are **backward compatible**. Existing configurations will continue to work with these improvements: |
81 | | - |
82 | | -- Better coordination between retry mechanisms |
83 | | -- Enhanced logging for debugging |
84 | | -- Automatic conflict detection and resolution |
85 | | - |
86 | | -### Recommended Configuration Updates |
| 35 | +Higher priority retries will not be overwritten by lower priority ones. |
87 | 36 |
|
88 | | -#### 1. For Simple Cases - Use Scenario Retries Only |
| 37 | +## Configuration Examples |
89 | 38 |
|
90 | | -**Old Configuration (potentially conflicting):** |
| 39 | +### Scenario Retries Only |
91 | 40 |
|
92 | 41 | ```javascript |
93 | 42 | module.exports = { |
94 | 43 | retry: 3, // scenario retries |
95 | | - plugins: { |
96 | | - retryFailedStep: { |
97 | | - enabled: true, |
98 | | - retries: 2, // step retries - could result in 3 * 3 = 9 executions |
99 | | - }, |
100 | | - }, |
101 | | -} |
102 | | -``` |
103 | | - |
104 | | -**Recommended Configuration:** |
105 | | - |
106 | | -```javascript |
107 | | -module.exports = { |
108 | | - retry: 3, // scenario retries only |
109 | | - plugins: { |
110 | | - retryFailedStep: { |
111 | | - enabled: false, // disable to avoid conflicts |
112 | | - }, |
113 | | - }, |
114 | 44 | } |
115 | 45 | ``` |
116 | 46 |
|
117 | | -#### 2. For Step-Level Control - Use Step Retries Only |
118 | | - |
119 | | -**Recommended Configuration:** |
| 47 | +### Step Retries Only |
120 | 48 |
|
121 | 49 | ```javascript |
122 | 50 | module.exports = { |
123 | 51 | plugins: { |
124 | 52 | retryFailedStep: { |
125 | 53 | enabled: true, |
126 | 54 | retries: 2, |
127 | | - ignoredSteps: ['amOnPage', 'wait*'], // customize as needed |
| 55 | + ignoredSteps: ['amOnPage', 'wait*'], |
128 | 56 | }, |
129 | 57 | }, |
130 | | - // No global retry configuration |
131 | 58 | } |
132 | 59 | ``` |
133 | 60 |
|
134 | | -#### 3. For Mixed Scenarios - Use Enhanced Coordination |
| 61 | +### Mixed - With Auto Coordination |
135 | 62 |
|
136 | 63 | ```javascript |
137 | 64 | module.exports = { |
138 | 65 | retry: { |
139 | | - Scenario: 2, // scenario retries for most tests |
| 66 | + Scenario: 2, // scenario retries |
140 | 67 | }, |
141 | 68 | plugins: { |
142 | 69 | retryFailedStep: { |
143 | 70 | enabled: true, |
144 | 71 | retries: 1, |
145 | | - deferToScenarioRetries: true, // automatically coordinate (default) |
| 72 | + deferToScenarioRetries: true, // auto-coordinate (default) |
146 | 73 | }, |
147 | 74 | }, |
148 | 75 | } |
149 | 76 | ``` |
150 | 77 |
|
151 | | -### Testing Your Configuration |
152 | | - |
153 | | -Use the new retry coordinator to validate your configuration: |
154 | | - |
155 | | -```javascript |
156 | | -const retryCoordinator = require('codeceptjs/lib/retryCoordinator') |
157 | | - |
158 | | -// Validate your configuration |
159 | | -const warnings = retryCoordinator.validateConfig(yourConfig) |
160 | | -if (warnings.length > 0) { |
161 | | - console.log('Retry configuration warnings:') |
162 | | - warnings.forEach(warning => console.log(' -', warning)) |
163 | | -} |
164 | | -``` |
165 | | - |
166 | | -## Enhanced Logging |
167 | | - |
168 | | -The new retry mechanisms provide clearer logging: |
169 | | - |
170 | | -``` |
171 | | -[Global Retry] Scenario retries: 3 |
172 | | -[Step Retry] Deferred to scenario retries (3 retries) |
173 | | -[Retry Coordinator] Registered scenario retry (priority: 30) |
174 | | -``` |
175 | | - |
176 | | -## Breaking Changes |
177 | | - |
178 | | -**None.** All existing configurations continue to work. |
| 78 | +**Important:** When `deferToScenarioRetries` is true (default), step retries are automatically disabled if scenario retries are configured to avoid excessive total retries (2 scenario × 3 step = 6 executions per step). |
179 | 79 |
|
180 | 80 | ## New Configuration Options |
181 | 81 |
|
182 | | -### Enhanced RetryFailedStep Plugin |
183 | | - |
184 | | -```javascript |
185 | | -plugins: { |
186 | | - retryFailedStep: { |
187 | | - enabled: true, |
188 | | - retries: 2, |
189 | | - deferToScenarioRetries: true, // NEW: automatically coordinate with scenario retries |
190 | | - minTimeout: 1000, |
191 | | - maxTimeout: 10000, |
192 | | - factor: 1.5, |
193 | | - ignoredSteps: ['wait*', 'amOnPage'] |
194 | | - } |
195 | | -} |
196 | | -``` |
197 | | - |
198 | | -### New Options: |
| 82 | +### retryFailedStep Plugin |
199 | 83 |
|
200 | 84 | - `deferToScenarioRetries` (boolean, default: true) - When true, step retries are disabled if scenario retries are configured |
201 | 85 |
|
202 | | -## Debugging Retry Issues |
203 | | - |
204 | | -### 1. Check Configuration Validation |
205 | | - |
206 | | -```javascript |
207 | | -const retryCoordinator = require('codeceptjs/lib/retryCoordinator') |
208 | | -const warnings = retryCoordinator.validateConfig(Config.get()) |
209 | | -console.log('Configuration warnings:', warnings) |
210 | | -``` |
211 | | - |
212 | | -### 2. Monitor Enhanced Logging |
213 | | - |
214 | | -Run tests with `--verbose` to see detailed retry coordination logs. |
215 | | - |
216 | | -### 3. Generate Retry Summary |
217 | | - |
218 | | -```javascript |
219 | | -// In your test hooks |
220 | | -const summary = retryCoordinator.generateRetrySummary() |
221 | | -console.log('Retry mechanisms active:', summary) |
222 | | -``` |
223 | | - |
224 | 86 | ## Best Practices |
225 | 87 |
|
226 | | -1. **Choose One Primary Retry Strategy** - Either scenario-level OR step-level retries, not both |
227 | | -2. **Use Configuration Validation** - Check for conflicts before running tests |
228 | | -3. **Monitor Retry Logs** - Use enhanced logging to understand retry behavior |
229 | | -4. **Test Retry Behavior** - Verify your retry configuration works as expected |
230 | | -5. **Avoid Excessive Retries** - High retry counts often indicate test stability issues |
231 | | - |
232 | | -## Future Enhancements |
233 | | - |
234 | | -- Integration with retry coordinator for all retry mechanisms |
235 | | -- Runtime retry strategy adjustment |
236 | | -- Retry analytics and reporting |
237 | | -- Advanced retry patterns (exponential backoff, conditional retries) |
| 88 | +1. **Choose One Primary Retry Strategy** - Either scenario-level OR step-level retries |
| 89 | +2. **Use Auto Coordination** - Enable `deferToScenarioRetries` to avoid conflicts |
| 90 | +3. **Monitor Retry Behavior** - Use `DEBUG_RETRY_PLUGIN=1` to see retry details |
| 91 | +4. **Avoid Excessive Retries** - High retry counts often indicate test stability issues |
0 commit comments