Skip to content

Add ranked "and N more" reporting module#27

Open
Copilot wants to merge 5 commits intomainfrom
copilot/add-ranked-and-n-more-reporting-module
Open

Add ranked "and N more" reporting module#27
Copilot wants to merge 5 commits intomainfrom
copilot/add-ranked-and-n-more-reporting-module

Conversation

Copy link

Copilot AI commented Mar 19, 2026

Queries that report multiple related entities per finding (fields, parameters, call sites, etc.) have no standard way to cap result noise while communicating how many were omitted. This adds a reusable LimitResults module to handle that pattern.

New module: src/qtil/results/LimitResults.qll

  • LimitResultsConfigSig<Finding, Entity> — signature to implement (both types must have toString()):
    • problem/2 — finding-to-entity relationship
    • message/3 — message template; receives remaining (e.g. " (and 2 more)" or "")
    • default placeholderString/1 — display string for an entity; defaults to entity.toString()
    • default orderBy/1 — ascending sort key for entities; defaults to placeholderString(entity), allowing ordering to be overridden independently of the display string
    • default maxResults/0 — cap per finding; defaults to 3
    • default andMoreText/1 — customisable "and N more" suffix text; defaults to " (and N more)"
  • LimitResults<Finding, Entity, Config> — exposes:
    • query predicate problems(finding, msg, entity, entityStr): the primary query entry point; ranks entities per finding by orderBy(), reports the top maxResults(), appends andMoreText(n) when capped, and sets entityStr to placeholderString(entity)
    • hasLimitedResult(finding, entity, message): underlying predicate for advanced use

Usage

Only problem and message must be implemented — all other predicates have sensible defaults. Instantiating the module is all that's needed; Results::problems is automatically part of the query output with no from/where/select boilerplate:

module MyConfig implements LimitResultsConfigSig<IncompleteInitialization, Field> {
  predicate problem(IncompleteInitialization init, Field f) { f = init.getField() }

  bindingset[remaining]
  string message(IncompleteInitialization init, Field f, string remaining) {
    result = init.getKindStr() + " does not initialize non-static data member $@" + remaining + "."
  }
}

module Results = LimitResults<IncompleteInitialization, Field, MyConfig>;
// Results::problems is automatically part of the query output.

LimitResults is exported via Qtil.qll and documented in README.md.


⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.

Co-authored-by: MichaelRFairhurst <1627771+MichaelRFairhurst@users.noreply.github.com>
Copilot AI changed the title [WIP] Add ranked 'and n more' reporting module Add ranked "and N more" reporting module Mar 19, 2026
Copilot AI requested a review from MichaelRFairhurst March 19, 2026 17:32
@MichaelRFairhurst
Copy link
Collaborator

This code:

from IncompleteInitialization init, Field f, string message
where Results::hasLimitedResult(init, f, message)
select init, message, f, f.getName()

should not be necessary. It should automatically declared by means of a query predicate

module LimitResults<...> {
  ....
  query predicate problems(T1 t1, string msg, T2 t2, string t2str) {
     hasLimitedResult(t1, t2, msg) and
     t2str = t2.toString()
   }
 }

Add default predicates for ease of use, such as the following:

signature module ... {
  default int maxResults() { result = 3 }
  default string placeholderString(T2 t2) { result = t2.toString() }
}

allowing for users to override maxResults but they do not have to declare it.

Consider ordering by placeholderString.

Copy link
Collaborator

@MichaelRFairhurst MichaelRFairhurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment

…derString

Co-authored-by: MichaelRFairhurst <1627771+MichaelRFairhurst@users.noreply.github.com>
Copy link
Author

Copilot AI commented Mar 19, 2026

This code:

...

Done in ba7bd9d:

  • Added query predicate problems(Finding finding, string msg, Entity entity, string entityStr) — instantiating module Results = LimitResults<...> is all that's needed; no from/where/select boilerplate.
  • Replaced orderBy with default string placeholderString(Entity entity) { result = entity.toString() } — doubles as both the ordering key and the entityStr in problems.
  • Added default int maxResults() { result = 3 }.
  • Both Finding and Entity are now FiniteStringableType (required by CodeQL for query predicate arguments).

Copy link
Collaborator

@MichaelRFairhurst MichaelRFairhurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add documentation to README.md

*
* bindingset[remaining]
* string message(MyFinding finding, MyEntity entity, string remaining) {
* result = "Finding $@ has entity $@" + remaining + "."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

documentation error: should be "Finding " + finding.getName() + " has entity $@" + remaining + "."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d58a5a0 — the example now uses "Finding " + finding.getName() + " has entity $@" + remaining + ".".

* The instantiated module exposes a `problems` query predicate that can be used directly as
* the output of a query without any `from`/`where`/`select` boilerplate:
*
* ```ql
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete this code block, the above paragraph covers it already.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in d58a5a0.

string message(Finding finding, Entity entity, string remaining);

/**
* The display string for an entity, also used as the ordering key (ascending).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make "also used as the ordering key (ascending)" the default behavior.

E.g., add a new default string orderBy(Entity e) { result = getPlaceholderString(e) }.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d58a5a0 — added default string orderBy(Entity entity) { result = placeholderString(entity) } to the signature. The ranking now uses Config::orderBy(e), keeping placeholderString as purely the display string. Users can override orderBy independently.

*
* Defaults to `3`.
*/
default int maxResults() { result = 3 }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a default string andMoreText(int total) predicate

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d58a5a0 — added bindingset[n] default string andMoreText(int n) { result = " (and " + n + " more)" } to the signature. The hasLimitedResult predicate now calls Config::andMoreText(total - Config::maxResults()) for the suffix.

@@ -0,0 +1,146 @@
import qtil.results.LimitResults
import qtil.testing.Qnit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, we shouldn't use unit tests and should use a query test.

Simply import LimitResults and then expect

| BugA | "finding 1 for Bug A" | ....... |
| BugA | "finding 2 for Bug A" | .........
... etc

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted in d58a5a0. The test now just implements problem and message, instantiates the module, and re-exports Results::problems as a top-level query predicate. The .expected file contains the tabular rows directly.

.gitignore Outdated
@@ -0,0 +1 @@
codeql_home/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this .gitignore

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted in d58a5a0.

Copy link
Author

Copilot AI commented Mar 19, 2026

Add documentation to README.md

Added in d58a5a0LimitResults is documented in README.md under the Query Formatting section with a usage example and description of the defaults.

@MichaelRFairhurst MichaelRFairhurst marked this pull request as ready for review March 19, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ranked "and n more" reporting module

2 participants