Reuse fileloader task data when atomic store fails#3191
Reuse fileloader task data when atomic store fails#3191jakebailey wants to merge 1 commit intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR reduces allocations in the compiler’s file parsing pipeline by reusing parseTaskData instances (and their small tasks maps) when SyncMap.LoadOrStore loses the race, instead of discarding the newly-allocated candidate.
Changes:
- Added a global
sync.PoolforparseTaskDatato reuse candidate task data objects on failedLoadOrStore. - Updated
filesParser.startto allocate candidates from the pool and return them whenLoadOrStoreindicates an existing entry was already present.
| func getParseTaskData(task *parseTask) *parseTaskData { | ||
| td := parseTaskDataPool.Get().(*parseTaskData) | ||
| td.tasks[task.normalizedFilePath] = task | ||
| td.lowestDepth = math.MaxInt |
There was a problem hiding this comment.
parseTaskData instances are reused from a sync.Pool, but getParseTaskData/putParseTaskData don't reset all fields. If a pooled instance previously had startedSubTasks=true or a non-empty packageId, and it later becomes the stored value (LoadOrStore returns loaded==false), those stale values will change behavior (e.g. subtasks may be started too early, and packageId propagation can be skipped). Reset startedSubTasks and packageId (and any other stateful fields) when taking from the pool (or before putting back).
| td.lowestDepth = math.MaxInt | |
| td.lowestDepth = math.MaxInt | |
| td.startedSubTasks = false | |
| td.packageId = module.PackageId{} |
There was a problem hiding this comment.
Either we succeed and we don't put it back, or we fail, and so we just put back the version without having changed anything, so I do not believe this is true.
We use an atomic store to create a file's loader data. If this fails, we throw away the struct and map. This is a waste! We can totally use that for the next file. Stick these into a global sync.Pool, clearing the map (basically free for this small of a map) for later reuse.
The results for the
testrunnerpackage:Summary (flat)
Line-level breakdown (flat)
LoadOrStore/ struct allocmap[string]*parseTask{...}wg.Queueclosure