Skip to content

Load: Harden LOAD TSFILE authorization and source path validation#17624

Draft
Caideyipi wants to merge 4 commits intomasterfrom
load-pri
Draft

Load: Harden LOAD TSFILE authorization and source path validation#17624
Caideyipi wants to merge 4 commits intomasterfrom
load-pri

Conversation

@Caideyipi
Copy link
Copy Markdown
Collaborator

Description

This PR hardens LOAD TSFILE execution by adding explicit authorization checks and limiting user-issued load sources to
configured allowed directories.

Main changes:

  • Add a new system privilege LOAD_TSFILE.
    • Register it in PrivilegeType, AuthUtils, table-model privilege mapping, audit operation mapping, and SQL grammar.
    • Allow GRANT/REVOKE LOAD_TSFILE parsing in both tree and table authorization paths.
  • Enforce LOAD_TSFILE privilege for user-issued LOAD TSFILE.
    • TreeAccessCheckVisitor.visitLoadFile now checks global LOAD_TSFILE privilege.
    • Table-model StatementAnalyzer.visitLoadTsFile also checks missing LOAD_TSFILE privilege before analysis.
  • Restrict source paths for user-issued LOAD TSFILE.
    • Add load_tsfile_allowed_dirs config.
    • If unset, the allowed source directories default to IoTDB internal load TsFile directories.
    • Canonicalize source paths before validation to reject paths outside the configured allowlist, including
      traversal-style paths.
  • Preserve internal load flows.
    • Add unchecked constructors/factory methods for internal paths such as pipe receiver loading, active load,
      scheduler retry, and type-conversion retry paths.
  • Strengthen write-permission checks during load analysis.
    • Reuse tree write-data permission validation.
    • Ensure write permission is checked even when auto-create/verify schema is disabled.
  • Add tests covering:
    • LOAD TSFILE requires the new LOAD_TSFILE privilege.
    • LOAD_TSFILE can be granted/revoked by the authorization parser.
    • Source files outside load_tsfile_allowed_dirs are rejected.
    • Sub-statements preserve database information under the new path validation behavior.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@Caideyipi Caideyipi marked this pull request as draft May 9, 2026 07:16
@codecov
Copy link
Copy Markdown

codecov Bot commented May 9, 2026

Codecov Report

❌ Patch coverage is 43.75000% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.24%. Comparing base (d4be5c8) to head (5d2ac1c).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...ryengine/plan/analyze/load/LoadTsFileAnalyzer.java 0.00% 16 Missing ⚠️
...analyze/load/TreeSchemaAutoCreatorAndVerifier.java 0.00% 10 Missing ⚠️
...ngine/plan/statement/crud/LoadTsFileStatement.java 67.85% 9 Missing ⚠️
...ne/plan/relational/analyzer/StatementAnalyzer.java 0.00% 4 Missing ⚠️
...an/relational/security/TreeAccessCheckVisitor.java 33.33% 4 Missing ⚠️
...ueryengine/plan/relational/sql/ast/LoadTsFile.java 0.00% 4 Missing ⚠️
...ngine/plan/scheduler/load/LoadTsFileScheduler.java 0.00% 2 Missing ⚠️
...rageengine/load/active/ActiveLoadTsFileLoader.java 0.00% 2 Missing ⚠️
...ain/java/org/apache/iotdb/db/conf/IoTDBConfig.java 90.00% 1 Missing ⚠️
.../receiver/protocol/legacy/loader/TsFileLoader.java 0.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17624      +/-   ##
============================================
+ Coverage     40.23%   40.24%   +0.01%     
  Complexity     2554     2554              
============================================
  Files          5177     5177              
  Lines        348880   348952      +72     
  Branches      44624    44631       +7     
============================================
+ Hits         140363   140433      +70     
- Misses       208517   208519       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 9, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant