Trim console output before processing it console document#2462
Trim console output before processing it console document#2462trancexpress wants to merge 1 commit intoeclipse-platform:masterfrom
Conversation
| applyStreamOutput(pendingCopy, sizeHint); | ||
| } | ||
| checkFinished(); | ||
| checkBufferSize(); // needs partitions synchronized |
There was a problem hiding this comment.
Proper trimming is done here. The above change just reduces the output that will be processed by the document / underlying widget.
| } | ||
| sizeHint = 0; | ||
| int index = 0; | ||
| for (int i = copy.size() - 1; i >= 0; --i) { |
There was a problem hiding this comment.
Will need to write this better, current state is not good.
But generally this is what Copilot advised to do. With this change, at least the user is able to set a console limit to avoid the UI hangs.
It will still be good to fix the problem even without the console limit... But the change is something and its simple.
Note that with my snippet, #2283 (comment), the console contents are unusable if I use the console limit (with and without this change). Probably we want to open another bug for this.
There was a problem hiding this comment.
I think another problem we see here is that the console character limit is not applied, if the output is only a single line.
Likely we can do this here as well... If there is only one pending change after the initial trim, trim the text in it as well, if its length is above the character limit.
And maybe we look into the trimming algorithm below, for continuous output on the same line... But likely we want to do this in a separate ticket.
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent Eclipse UI freezes when processing extremely large console output by trimming pending console output in IOConsolePartitioner.QueueProcessingJob.processPendingPartitions() before applying it to the console document, when a console limit (water marks) is configured.
Changes:
- Adds logic in
processPendingPartitions()to trim pending partitions based on the consolehighWaterMarkbefore callingapplyStreamOutput(...).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| List<PendingPartition> copy = new ArrayList<>(); | ||
| List<PendingPartition> trimmedCopy = new ArrayList<>(); | ||
| for (PendingPartition p : pendingCopy) { | ||
| if (p.stream != stream) { | ||
| break; | ||
| } | ||
| copy.add(p); | ||
| } | ||
| sizeHint = 0; | ||
| int index = 0; | ||
| for (int i = copy.size() - 1; i >= 0; --i) { | ||
| PendingPartition p = copy.get(i); | ||
| sizeHint += p.text.length(); | ||
| index = i; | ||
| if (sizeHint > hwm) { | ||
| break; | ||
| } | ||
| } | ||
| for (int i = index; i < copy.size(); ++i) { | ||
| PendingPartition p = copy.get(i); | ||
| trimmedCopy.add(p); | ||
| } | ||
| pendingCopy = trimmedCopy; | ||
| } |
There was a problem hiding this comment.
The trimming block replaces pendingCopy with trimmedCopy, which is built only from the first contiguous stream (copy stops at the first p.stream != stream). If pendingCopy contained additional partitions from other streams after that boundary, they were already drained from pendingPartitions and will be silently dropped (lost console output). Consider preserving the remainder of the original pendingCopy (after the first-stream segment) and only trimming that first segment, or re-queueing the discarded partitions so ordering/data isn’t lost.
| int index = 0; | ||
| for (int i = copy.size() - 1; i >= 0; --i) { | ||
| PendingPartition p = copy.get(i); | ||
| sizeHint += p.text.length(); | ||
| index = i; | ||
| if (sizeHint > hwm) { | ||
| break; | ||
| } | ||
| } | ||
| for (int i = index; i < copy.size(); ++i) { | ||
| PendingPartition p = copy.get(i); | ||
| trimmedCopy.add(p); | ||
| } | ||
| pendingCopy = trimmedCopy; | ||
| } |
There was a problem hiding this comment.
The trimming logic still allows an oversized batch to be passed to SWT: the backward scan breaks when sizeHint > hwm and then keeps partitions starting at index, so the retained content length remains > hwm. Also, if a single PendingPartition.text is itself very large (e.g., a 1M-char line), this code won’t trim within that partition, so a huge string can still be applied to the document and trigger the UI freeze this PR targets. Consider enforcing a hard upper bound (<= hwm or another safe chunk size) and truncating/splitting a too-large partition so the inserted text cannot exceed that bound.
| int index = 0; | |
| for (int i = copy.size() - 1; i >= 0; --i) { | |
| PendingPartition p = copy.get(i); | |
| sizeHint += p.text.length(); | |
| index = i; | |
| if (sizeHint > hwm) { | |
| break; | |
| } | |
| } | |
| for (int i = index; i < copy.size(); ++i) { | |
| PendingPartition p = copy.get(i); | |
| trimmedCopy.add(p); | |
| } | |
| pendingCopy = trimmedCopy; | |
| } | |
| int index = copy.size(); | |
| // walk backwards and only include partitions while the total stays within hwm | |
| for (int i = copy.size() - 1; i >= 0; --i) { | |
| PendingPartition p = copy.get(i); | |
| int len = p.text.length(); | |
| if (sizeHint + len > hwm) { | |
| break; | |
| } | |
| sizeHint += len; | |
| index = i; | |
| } | |
| if (index < copy.size()) { | |
| for (int i = index; i < copy.size(); ++i) { | |
| PendingPartition p = copy.get(i); | |
| trimmedCopy.add(p); | |
| } | |
| } | |
| // if index == copy.size() no partition could be included without exceeding hwm; | |
| // in that case we drop this oversized batch for the stream | |
| pendingCopy = trimmedCopy; | |
| } | |
| // ensure sizeHint matches the actual content we are about to apply | |
| sizeHint = 0; | |
| for (PendingPartition p : pendingCopy) { | |
| sizeHint += p.text.length(); | |
| } |
Test Results 1 950 files ±0 1 950 suites ±0 1h 36m 49s ⏱️ + 6m 46s For more details on these errors, see this check. Results for commit 9b230a8. ± Comparison against base commit 8594fad. ♻️ This comment has been updated with latest results. |
|
Fails on Linux: |
|
Lets apply this one in a separate issue, its causing fails: |
When processing large amounts of console output of a process, its possible that the Eclipse UI will freeze. E.g. on Linux, very long lines (> 1 million symbols) result in such freezes. Using the console limit preference doesn't help preventing this freeze, despite limitting the contents seen in the console. This change adjusts IOConsolePartitioner.QueueProcessingJob.processPendingPartitions() to trim pending console output before setting this output in the console document, if a console limit is set in the Eclipse preferences. See: eclipse-platform#2283
|
Fail on Linux is: |
When processing large amounts of console output of a process, its possible that the Eclipse UI will freeze.
E.g. on Linux, very long lines (> 1 million symbols) result in such freezes.
Using the console limit preference doesn't help preventing this freeze, despite limitting the contents seen in the console.
This change adjusts
IOConsolePartitioner.QueueProcessingJob.processPendingPartitions()to trim pending console output before setting this output in the console document, if a console limit is set in the Eclipse preferences.See: #2283