Skip to content

Add dynamic sampling to frame screenshots#56048

Closed
huntie wants to merge 1 commit intofacebook:mainfrom
huntie:export-D95987488
Closed

Add dynamic sampling to frame screenshots#56048
huntie wants to merge 1 commit intofacebook:mainfrom
huntie:export-D95987488

Conversation

@huntie
Copy link
Member

@huntie huntie commented Mar 10, 2026

Summary:
Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

Motivation

  1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame screenshot loss.
  2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

Algorithm

Uses encodingInProgress atomic flag with single encoding thread and lastFrameBuffer storage for tail-capture of the last frame before idling (to capture settled animation states):

  • Not encoding: Frame passes directly to encoder → emits with screenshot when done
  • Encoding busy: Frame stored in lastFrameBuffer for tail-capture → any replaced frame emits without screenshot
  • Encoding done: Clears flag early, then opportunistically encodes tail frame without blocking new frames
  • Failed captures: Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

Remaining work

  • ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).
  • ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to Tracing.stop.

Changelog: [Internal]

Differential Revision: D95987488

@meta-codesync
Copy link

meta-codesync bot commented Mar 10, 2026

@huntie has exported this pull request. If you are a Meta employee, you can view the originating Diff in D95987488.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 10, 2026
@huntie huntie force-pushed the export-D95987488 branch from 9ff647c to 303d30b Compare March 11, 2026 22:27
huntie added a commit to huntie/react-native that referenced this pull request Mar 11, 2026
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame visual snapshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

NOTE: ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

NOTE: ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
@huntie huntie force-pushed the export-D95987488 branch from 303d30b to 553a459 Compare March 11, 2026 22:33
huntie added a commit to huntie/react-native that referenced this pull request Mar 11, 2026
Summary:
Pull Request resolved: facebook#56048

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame visual snapshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

NOTE: ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

NOTE: ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
@huntie huntie force-pushed the export-D95987488 branch from 553a459 to f2155ae Compare March 11, 2026 22:58
huntie added a commit to huntie/react-native that referenced this pull request Mar 11, 2026
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame visual snapshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

NOTE: ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

**Dynamic throttling algorithm**

- Use a single `lastFrameBuffer` to store the most recently captured frame (bitmap + metadata)
- Use a serial background thread (single-thread executor) for all encoding work
- Use an atomic `encodingInProgress` flag to track encoding state
- Always capture screenshots to `lastFrameBuffer` on every frame (even during encoding), replacing any previous waiting frame
- When not encoding: start encoding the captured frame
- When encoding finishes: loop back and check if buffer was refilled, encode that frame if present
- Always emit frame timing events immediately (without screenshot)

This ensures we don't queue up excessive encoding work while maintaining continuous frame timing data and guaranteeing the final settled state of animations is captured (tail-capture), minimizing the performance impact of recording on the device.

NOTE: ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
huntie added a commit to huntie/react-native that referenced this pull request Mar 11, 2026
Summary:
Pull Request resolved: facebook#56048

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame visual snapshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

NOTE: ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

**Dynamic throttling algorithm**

- Use a single `lastFrameBuffer` to store the most recently captured frame (bitmap + metadata)
- Use a serial background thread (single-thread executor) for all encoding work
- Use an atomic `encodingInProgress` flag to track encoding state
- Always capture screenshots to `lastFrameBuffer` on every frame (even during encoding), replacing any previous waiting frame
- When not encoding: start encoding the captured frame
- When encoding finishes: loop back and check if buffer was refilled, encode that frame if present
- Always emit frame timing events immediately (without screenshot)

This ensures we don't queue up excessive encoding work while maintaining continuous frame timing data and guaranteeing the final settled state of animations is captured (tail-capture), minimizing the performance impact of recording on the device.

NOTE: ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
@huntie huntie force-pushed the export-D95987488 branch from f2155ae to 57fe50b Compare March 11, 2026 23:05
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame screenshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

**Algorithm**

Uses `encodingInProgress` atomic flag with single-background-thread encoder and `lastFrameBuffer` for tail-capture:

- **Not encoding:** Frame passes directly to encoder → emits with screenshot when done
- **Encoding busy:** Frame stored in `lastFrameBuffer` for tail-capture → any replaced frame emits without screenshot
- **Encoding done:** Clears flag early, then opportunistically encodes tail frame without blocking new frames
- **Failed captures:** Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

**Remaining work**

- ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).
- ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
@huntie huntie force-pushed the export-D95987488 branch from 57fe50b to 3a23a1e Compare March 12, 2026 20:56
huntie added a commit to huntie/react-native that referenced this pull request Mar 12, 2026
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame screenshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

**Algorithm**

Uses `encodingInProgress` atomic flag with single encoding thread and `lastFrameBuffer` storage for tail-capture of the last frame before idling (to capture settled animation states):

- **Not encoding:** Frame passes directly to encoder → emits with screenshot when done
- **Encoding busy:** Frame stored in `lastFrameBuffer` for tail-capture → any replaced frame emits without screenshot
- **Encoding done:** Clears flag early, then opportunistically encodes tail frame without blocking new frames
- **Failed captures:** Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

**Remaining work**

- ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).
- ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
huntie added a commit to huntie/react-native that referenced this pull request Mar 12, 2026
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame screenshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

**Algorithm**

Uses `encodingInProgress` atomic flag with single encoding thread and `lastFrameBuffer` storage for tail-capture of the last frame before idling (to capture settled animation states):

- **Not encoding:** Frame passes directly to encoder → emits with screenshot when done
- **Encoding busy:** Frame stored in `lastFrameBuffer` for tail-capture → any replaced frame emits without screenshot
- **Encoding done:** Clears flag early, then opportunistically encodes tail frame without blocking new frames
- **Failed captures:** Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

**Remaining work**

- ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).
- ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
huntie added a commit to huntie/react-native that referenced this pull request Mar 12, 2026
Summary:
Pull Request resolved: facebook#56048

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame visual snapshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

NOTE: ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

**Algorithm**

Uses `encodingInProgress` atomic flag with single-background-thread encoder and `lastFrameBuffer` for tail-capture:

- **Not encoding:** Frame passes directly to encoder → emits with screenshot when done
- **Encoding busy:** Frame stored in `lastFrameBuffer` for tail-capture → any replaced frame emits without screenshot
- **Encoding done:** Clears flag early, then opportunistically encodes tail frame without blocking new frames
- **Failed captures:** Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

NOTE: ⚠️ We do still typically lose a small region of pending frames at the end of a trace, but this is now more reasonable. A further fix (likely unnecessary) is to add lifecycle methods to await frame processing before responding to `Tracing.stop`.

Changelog: [Internal]

Differential Revision: D95987488
huntie added a commit to huntie/react-native that referenced this pull request Mar 12, 2026
Summary:

Update Android frame screenshot processing to skip screenshot capture when encoding is already in progress — now limited to a single background thread — while always emitting frame timing events.

**Motivation**

1. Prevents truncated trace data on slower devices (e.g. missing screenshots for the last 1/3 of the trace), with the tradeoff of some intermediate frame screenshot loss.
2. Reduces total recording overhead by freeing up device threads - prevents excessive encoding work from blocking or slowing down the UI and other app threads.

**Algorithm**

Uses `encodingInProgress` atomic flag with single encoding thread and `lastFrameBuffer` storage for tail-capture of the last frame before idling (to capture settled animation states):

- **Not encoding:** Frame passes directly to encoder → emits with screenshot when done
- **Encoding busy:** Frame stored in `lastFrameBuffer` for tail-capture → any replaced frame emits without screenshot
- **Encoding done:** Clears flag early, then opportunistically encodes tail frame without blocking new frames
- **Failed captures:** Emit without screenshot immediately

Result: Every frame emitted exactly once. Encoding adapts to device speed. Settled animation state guaranteed captured.

**Remaining work**

- ⚠️ This still does not yet solve crashes (OkHttp network chunk size overflow) for heavy frame data at a high FPS on fast devices (coming next).

Changelog: [Internal]

Reviewed By: rubennorte

Differential Revision: D95987488
@meta-codesync meta-codesync bot closed this in d309cda Mar 12, 2026
@facebook-github-tools facebook-github-tools bot added the Merged This PR has been merged. label Mar 12, 2026
@meta-codesync
Copy link

meta-codesync bot commented Mar 12, 2026

This pull request has been merged in d309cda.

@react-native-bot
Copy link
Collaborator

This pull request was successfully merged by @huntie in d309cda

When will my fix make it into a release? | How to file a pick request?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged This PR has been merged. meta-exported p: Facebook Partner: Facebook Partner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants