fix: ensure working hostedSuppressions/RetireJS forceUpdate & correct cache validForHours checks#8494
Conversation
There was a problem hiding this comment.
Pull request overview
This PR restores and standardizes remote update behavior for Hosted Suppressions and RetireJS cached data sources, fixing incorrect staleness checks (seconds vs milliseconds) that caused unnecessary re-downloads, and aligning Hosted Suppressions behavior so packaged snapshot data is always available as an offline fallback.
Changes:
- Refactors Hosted Suppressions and RetireJS update logic to reuse
LocalDataSourcestaleness/timestamp handling and reinstateforceUpdatebehavior where it was lost. - Introduces
FileUtils.existsWithContentand applies it across multiple analyzers to consistently treat empty/near-empty files as non-actionable. - Adds/updates unit tests and documentation to cover and explain the new/clarified update semantics.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/src/test/java/org/owasp/dependencycheck/utils/FileUtilsTest.java | Adds coverage for the new existsWithContent helper. |
| utils/src/main/java/org/owasp/dependencycheck/utils/FileUtils.java | Adds existsWithContent(File) utility used across analyzers/data sources. |
| src/site/markdown/dependency-check-gradle/configuration.md | Clarifies meaning of hosted suppressions “enabled” flag in Gradle docs. |
| src/site/markdown/dependency-check-gradle/configuration-update.md | Same clarification for Gradle update task documentation. |
| src/site/markdown/dependency-check-gradle/configuration-aggregate.md | Same clarification for Gradle aggregate task documentation. |
| maven/src/site/markdown/configuration.md | Clarifies meaning of hostedSuppressionsEnabled in Maven docs. |
| maven/src/main/java/org/owasp/dependencycheck/maven/BaseDependencyCheckMojo.java | Updates parameter Javadoc to match new hosted suppressions semantics. |
| core/src/test/resources/suppressions-invalid.xml | Adds invalid XML fixture for hosted suppressions parsing tests. |
| core/src/test/resources/retirejs/jsrepository.json | Adds RetireJS repository fixture for update/analyzer tests. |
| core/src/test/resources/retirejs/jsrepository-invalid.json | Adds invalid JSON fixture for RetireJS parsing failure tests. |
| core/src/test/java/org/owasp/dependencycheck/data/update/RetireJSDataSourceTest.java | Adds tests for RetireJS update/purge/staleness/force-update behavior. |
| core/src/test/java/org/owasp/dependencycheck/data/update/HostedSuppressionsDataSourceTest.java | Adds tests for hosted suppressions update/purge/staleness/force-update behavior. |
| core/src/test/java/org/owasp/dependencycheck/BaseTest.java | Adds test helpers for reading resources as URI/URL/content strings. |
| core/src/test/java/org/owasp/dependencycheck/BaseDBTestCase.java | Switches resource resolution to the shared BaseTest helper. |
| core/src/test/java/org/owasp/dependencycheck/analyzer/RetireJsAnalyzerTest.java | Adds analyzer-level tests for RetireJS repository loading behavior. |
| core/src/test/java/org/owasp/dependencycheck/analyzer/AbstractSuppressionAnalyzerTest.java | Updates/extends suppression tests for packaged vs remote hosted suppressions behavior. |
| core/src/main/java/org/owasp/dependencycheck/data/update/RetireJSDataSource.java | Reworks RetireJS update logic; fixes staleness logic and restores “force update” semantics. |
| core/src/main/java/org/owasp/dependencycheck/data/update/LocalDataSource.java | Centralizes timestamp persistence and staleness calculation using Instant/Duration. |
| core/src/main/java/org/owasp/dependencycheck/data/update/HostedSuppressionsDataSource.java | Reworks hosted suppressions update behavior and separates “best effort” vs “unhandled” update. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/YarnAuditAnalyzer.java | Uses existsWithContent to skip empty/near-empty lock files. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/RetireJsAnalyzer.java | Delegates remote fetch/update to RetireJSDataSource and improves temp-copy handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/PnpmAuditAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/PipfilelockAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/PipfileAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/PipAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/PinnedMavenInstallAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/NodePackageAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/NodeAuditAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/LibmanAnalyzer.java | Uses existsWithContent for consistent empty-file handling. |
| core/src/main/java/org/owasp/dependencycheck/analyzer/AbstractSuppressionAnalyzer.java | Refactors suppression loading to always use hosted snapshot fallback and reuse data source update logic. |
| cli/src/site/markdown/arguments.md | Clarifies CLI wording for disabling hosted suppressions retrieval. |
| cli/src/main/java/org/owasp/dependencycheck/CliParser.java | Updates CLI option description to match new semantics. |
| ant/src/site/markdown/configuration.md | Clarifies hosted suppressions semantics in Ant docs. |
| ant/src/site/markdown/config-update.md | Clarifies hosted suppressions semantics in Ant update docs. |
| ant/src/main/java/org/owasp/dependencycheck/taskdefs/Update.java | Updates Ant task docs to match new hosted suppressions semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…back to snapshot suppressions even if failing Signed-off-by: Chad Wilson <29788154+chadlwilson@users.noreply.github.com>
After the switch from DB storage the logic was storing seconds since epoch, but comparing to milliseconds, so files were always considered stale, causing excessive updates. Signed-off-by: Chad Wilson <29788154+chadlwilson@users.noreply.github.com>
…milar to Hosted Suppressions Signed-off-by: Chad Wilson <29788154+chadlwilson@users.noreply.github.com>
Signed-off-by: Chad Wilson <29788154+chadlwilson@users.noreply.github.com>
Avoid duplicating all the logic for validating URLs and determining whether the repository should be updated Signed-off-by: Chad Wilson <29788154+chadlwilson@users.noreply.github.com>
9e67cd6 to
49e9ab8
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 35 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
utils/src/main/java/org/owasp/dependencycheck/utils/FileUtils.java:1
existsWithContenttreats 1-byte files as “no content”. Several analyzers previously processed any non-empty file (length() > 0), so this change can unintentionally skip legitimate very-small inputs (e.g., minimal requirements files). If the goal is “non-blank” rather than “>1 byte”, consider checking> 0and (optionally) verifying the first non-whitespace byte(s) instead of a hard> 1threshold.
Description of Change
As noted in #8478 (comment) the forceUpdate support was accidentally removed when introducing the "packaged" suppression fallback. So there was no way to force update of the remote source when
autoUpdate=false, except in "update only" mode.This PR
forceUpdatecapability for hosted suppressionshostedSuppressionsEnabledto refer only to retrieval from remote data source. The packaged version of hostedSuppressions are now always used as a fallback, just as for base suppressions.lastUpdatedtimestamp from DB to properties file. The timestamp was being stored in seconds, but then compare to milliseconds so it considered every single run as stale/needing update.Related issues
Have test cases been added to cover the new functionality?
yes