Show lineage features as on & provide information on required permissions

[ArBridgeman]

closes #238

[ayush-shah]

Thanks for adding the Exasol lineage permission notes. I validated the feature flag against the connector source, and lineage support itself looks real: the Exasol service spec wires ExasolLineageSource.

One detail looks inconsistent with the code path, though. The new text says OpenMetadata queries EXA_STATISTICS.EXA_DBA_AUDIT_SQL and that the user needs SELECT ANY DICTIONARY, but the connector query reads EXA_DBA_AUDIT_SQL and joins EXA_DBA_AUDIT_SESSIONS, and the test-connection metadata asks for select privileges on EXA_DBA_AUDIT_SQL. I could not find EXA_STATISTICS.EXA_DBA_AUDIT_SQL or SELECT ANY DICTIONARY in the Exasol connector path.

Could we update the permission wording to match the actual Exasol objects/grants required by the connector, including EXA_DBA_AUDIT_SESSIONS if that join also requires explicit access? That will make the setup instructions copy-pasteable for users enabling lineage.

[ArBridgeman]

Thanks for adding the Exasol lineage permission notes. I validated the feature flag against the connector source, and lineage support itself looks real: the Exasol service spec wires ExasolLineageSource.

One detail looks inconsistent with the code path, though. The new text says OpenMetadata queries EXA_STATISTICS.EXA_DBA_AUDIT_SQL and that the user needs SELECT ANY DICTIONARY, but the connector query reads EXA_DBA_AUDIT_SQL and joins EXA_DBA_AUDIT_SESSIONS, and the test-connection metadata asks for select privileges on EXA_DBA_AUDIT_SQL. I could not find EXA_STATISTICS.EXA_DBA_AUDIT_SQL or SELECT ANY DICTIONARY in the Exasol connector path.

Could we update the permission wording to match the actual Exasol objects/grants required by the connector, including EXA_DBA_AUDIT_SESSIONS if that join also requires explicit access? That will make the setup instructions copy-pasteable for users enabling lineage.

Thanks for the feedback! Both the EXA_DBA_AUDIT_SQL and EXA_DBA_AUDIT_SESSIONS are system tables inside of the schema EXA_STATISTICS. For the user to access the tables, they will need the SELECT ANY DICTIONARY system privilege or the DBA role. In this case, the SELECT ANY DICTIONARY system privilege is the better primary recommendation to follow the principle of least privilege.

Users who lack the system privilege would not be able to use SELECT on the tables.

[ArBridgeman]

Summary

So I double-checked this in my local openmetadata with a basic user.

CREATE USER audit_reader IDENTIFIED BY "StrongPassword123";
GRANT CREATE SESSION TO audit_reader;

I'm working on a PR for another feature in the Exasol connector, and I can update the error text so it's clearer how to resolve it, as I agree that this is not sufficient information. However, I'd argue that the check result (failing) is as expected, as the user lacks the system privileges needed to access these tables. If there's something beyond the text that should be modified, please let me know.

Further Details

GRANT SELECT ON EXA_DBA_AUDIT_SQL TO audit_reader;
--- returns SQL Error [42500]: EXA_DBA_AUDIT_SQL is not modifiable (Session: 1865694034563694592)

GRANT SELECT ANY DICTIONARY TO audit_reader;

[ayush-shah]

Hi @ArBridgeman 👋
Appreciate you checking this against a local Exasol instance. I checked the Exasol docs as well, and your explanation makes sense: EXA_STATISTICS is integrated into the namespace, and EXA_DBA_* system tables require SELECT ANY DICTIONARY, so the privilege recommendation looks correct.

The only small doc refinement I still see is that the connector query joins both EXA_DBA_AUDIT_SQL and EXA_DBA_AUDIT_SESSIONS, while the current text only mentions EXA_DBA_AUDIT_SQL. Could we phrase this as querying the Exasol audit system tables, for example EXA_STATISTICS.EXA_DBA_AUDIT_SQL joined with EXA_STATISTICS.EXA_DBA_AUDIT_SESSIONS?

The test-connection error text sounds fine to handle in your follow-up connector PR.

[ArBridgeman]

Hi @ArBridgeman 👋 Appreciate you checking this against a local Exasol instance. I checked the Exasol docs as well, and your explanation makes sense: EXA_STATISTICS is integrated into the namespace, and EXA_DBA_* system tables require SELECT ANY DICTIONARY, so the privilege recommendation looks correct.

The only small doc refinement I still see is that the connector query joins both EXA_DBA_AUDIT_SQL and EXA_DBA_AUDIT_SESSIONS, while the current text only mentions EXA_DBA_AUDIT_SQL. Could we phrase this as querying the Exasol audit system tables, for example EXA_STATISTICS.EXA_DBA_AUDIT_SQL joined with EXA_STATISTICS.EXA_DBA_AUDIT_SESSIONS?

The test-connection error text sounds fine to handle in your follow-up connector PR.

Hi @ayush-shah ,

That's a great point about mentioning both tables. Before your latest comment, I had updated the documentation in this PR with those changes, as well as a link to Exasol documentation. Can you double-check? You might need to refresh the page; we've noticed in our team that GitHub has been buggier today, though their status page hasn't mentioned much.