feat: lean lane scopes — skip main history on export#10390
Open
davidfirst wants to merge 1 commit into
Open
Conversation
When a lane is exported to a scope different from its components' home scopes, the lane scope was being fattened with the full version history of every component, pulled in from each component's home scope as a side effect of the ExportValidate action. For lanes that are far behind main with many components, this fat-scope behavior is the main driver of OOM during export. This change makes lane scopes lean: 1. Client-side filter (export.main.runtime.ts): on lane export, drop refs whose origin.lane is undefined AND origin.id.scope differs from the destination lane scope. Those are main-origin snaps that already live on the component's home scope. 2. Server-side (export-validate.ts): ExportValidate no longer calls importMissingVersionHistory. The lane scope keeps only what the client sent (lane snaps + merge snap). Older history stays on each component's home scope. 3. getAllVersionParents (traverse-versions.ts) no longer throws on a second- pass when parents are still missing locally — for lean lane scopes, missing parents are expected. Returns what's reachable instead. 4. Client-side fallback in collectLogs (model-component.ts): when a lane's scope differs from a component's home scope, fall back to a second importWithoutDeps without lane after the first attempt, so the fetcher routes to the component's home scope and brings in the main chain for bit-log and similar parent-walking ops.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces memory usage (and prevents OOM) during lane exports to a separate “lane scope” by avoiding exporting/importing full main history into the lane scope, and instead fetching missing history on-demand from each component’s home scope when consumers traverse logs.
Changes:
- Client-side: filters out main-origin Version refs that belong to a component’s home scope (so they aren’t duplicated into the lane scope).
- Server-side: export validation no longer imports missing full version history for “external” components (lean lane scope behavior).
- Read-path:
collectLogs()retries history fetch without lane context when missing objects are detected and the lane scope differs from the component’s home scope. - Adds E2E coverage for “lane scope != component home scope” export + consumer behaviors.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
scopes/scope/remote-actions/export-validate.ts |
Stops importing full version history for external components during export validation (lean lane scope). |
scopes/scope/objects/models/model-component.ts |
Adds a log-collection fallback to fetch missing history from the component’s home scope when lane scope differs. |
scopes/scope/export/export.main.runtime.ts |
Filters out foreign main-origin Version refs during lane export to prevent duplicating main history into lane scopes. |
scopes/component/snap-distance/traverse-versions.ts |
Relaxes traversal behavior to allow missing parents (expected in lean lane scopes) when throws: false. |
e2e/harmony/lanes/lane-export-skip-main-history.e2e.ts |
New E2E tests validating lean lane scope export and consumer history traversal behavior. |
Comment on lines
49
to
63
| private async importAndThrowForMissingHistoryOnLane(bitObjectList: BitObjectList) { | ||
| const modelComponents = bitObjectList.getComponents(); | ||
| const externalComponents = modelComponents.filter((comp) => comp.scope !== this.scope.name); | ||
| if (!externalComponents.length) return; | ||
| await this.scope.scopeImporter.importMissingVersionHistory(externalComponents); | ||
| // this will throw in case the history is missing | ||
| await Promise.all( | ||
| externalComponents.map((modelComponent) => | ||
| getAllVersionHashes({ modelComponent, repo: this.scope.objects, throws: true }) | ||
| ) | ||
| // Lean-lane-scope: do NOT import full version-history into the lane scope. The history | ||
| // lives on each component's home scope (e.g. main snaps in component-scope, lane-b snaps in | ||
| // scope-b). Pulling them into the lane scope was the main source of fat-lane-scope OOM when | ||
| // a lane is far behind main. Consumers walking history will resolve missing parents by | ||
| // fetching from origin scopes on demand. We also skip the strict getAllVersionHashes check | ||
| // for the same reason — incomplete history on the lane scope is now expected. | ||
| logger.debug( | ||
| `export-validate, skipping importMissingVersionHistory for ${externalComponents.length} external components ` + | ||
| `(lean lane scope mode — their history stays on origin scopes)` | ||
| ); | ||
| } |
| const isLaneOrigin = Boolean(version.origin?.lane); | ||
| const isLocallyMutated = Boolean(version.squashed) || Boolean(version.unrelated); | ||
| const originScope = version.origin?.id?.scope; | ||
| const isForeignComponentScope = Boolean(originScope) && originScope !== remoteNameStr; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When a lane is exported to a scope different from its components' home scopes (typical for a feature lane against components on main), the lane scope was being fattened with the full version history of every component, pulled in from each component's home scope as a side effect of `ExportValidate`. For lanes that are far behind main with many components, this is the main driver of OOM during export.
What
Makes lane scopes lean. Main history stays on each component's home scope; the lane scope keeps only lane-origin snaps + the merge snap. Consumers' `bit log` walks into older history by fetching from the component's home scope transparently.
Implementation
Test plan
E2e suite `e2e/harmony/lanes/lane-export-skip-main-history.e2e.ts` — 8 tests, all passing: