Skip to content

Fixes #26568 sigma lineage fixes#26569

Closed
thiago-costa-fanatic wants to merge 1328 commits intoopen-metadata:1.11.12from
thiago-costa-fanatic:thiago/sigma-lineage-fixes
Closed

Fixes #26568 sigma lineage fixes#26569
thiago-costa-fanatic wants to merge 1328 commits intoopen-metadata:1.11.12from
thiago-costa-fanatic:thiago/sigma-lineage-fixes

Conversation

@thiago-costa-fanatic
Copy link
Contributor

@thiago-costa-fanatic thiago-costa-fanatic commented Mar 18, 2026

Describe your changes:

Fixes #26568

I pushed these changes to main, which are scheduled to go to 1.12 but would like to have them (and another bug found) in 1.11 as well

This includes a fix for skipping the first page of charts in a dashboard and also the fact that it only uses the node based lineage while the configuration indicates we would use SQL based as well, so I'm implementing the query based lineage.

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

sonika-shah and others added 30 commits March 6, 2026 11:57
…mains, data products, and teams (open-metadata#26266)

* Record actual user in activity feeds for bulk asset operations on domains, data products, and teams

* address gitar bot comments

* Added @OverRide annotation to bulkAssetsOperation in both subclasses
* ui: fix vulnerabilities

* remove immutable

* remove duplicate pkg

* remove duplicate dependency

* update yarn.lock
…nents (open-metadata#26276)

* Initial plan

* Add 24-column Grid base component with Grid.Item, gap props, export, and Storybook story

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* add divider and update typography

* fix comments

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
…izer to DI index and combine filters (open-metadata#26177)

* Fix open-metadata#20052: Combine user filter with existing metric filter in Data Insights charts

* Fix open-metadata#20052: Add tier field mapping with lowercase_normalizer to DI index templates
…ry Lineage (open-metadata#26275)

* Fix open-metadata#26274: Powerbi - Add support for bigquery NativeQuery Lineage

* log invalid fqn table search

* sql line comment pattern constant

---------

Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com>
…flow test (open-metadata#26254)

* fix(test): resolve persona dropdown option not found in team persona flow test

* fix parallel runs

* address sonar comments

* address gitar

* address sonar

* revert usage of AGGREGATE_PAGE_SIZE_LARGE
…ata#26281)

The migrations renaming the 'preview' property to 'enabled' in apps
were incorrectly placed under 1.11.13. Move them to 1.13.0 where they
belong, since this change targets the next major release.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* add card component

* Update card.tsx
…pen-metadata#26206)

* Fix open-metadata#25800: Agent status not showing true last 5 runs

* Fix open-metadata#25800: Agent status not showing true last 5 runs

* added playwright tests

* address comments

---------

Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Rohit0301 <rj03012002@gmail.com>
…tadata#26297)

* Initial plan

* Cleanup unused DataPill.styled.tsx and remove related mock from SingleColumnProfile.test

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
* Make topDimensions configurable per dimensional test case

Previously the number of top dimension values was hardcoded to 5
(DEFAULT_TOP_DIMENSIONS). This adds a topDimensions field to the
test case schema so users can control how many dimension values
are shown individually before the rest are grouped as "Others".

* fix checkstyle

* i18n: Translate `top-dimension-plural` key in all non-English locale files (open-metadata#26123)

* Initial plan

* Translate top-dimension-plural string in all non-English locale files

Co-authored-by: IceS2 <4912399+IceS2@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: IceS2 <4912399+IceS2@users.noreply.github.com>

* Simplify Dimensionality code

* fix style

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
…pen-metadata#26294)

* ui: Optimize ColumnDetailPanel loading and remove redundant API call

* address gitar

* fix failing spec
…CaseResultsForTestSuite (MySQL) (open-metadata#26235)

* ISSUE-26225: add index idx_entity_timestamp_desc for data_quality_data_time_series

* ISSUE-26225: add index idx_entity_timestamp_desc for data_quality_data_time_series

* Update bootstrap/sql/migrations/native/1.12.2/mysql/schemaChanges.sql

* ISSUE-26225: fix the suggestion

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
…a#26268)

* feat(ui): move buttons to canvas for better performance

* update buttons rendering

* support pipeline buttons

* address comments

* address comments

* address comments

* fix issue with DQHighlight edges

* fix tests

* address sonar comments

* fix
…rkflows (open-metadata#26277)

* Fix: Include team fqn in the team response

* Added unit test for team response
* Add skills to build connectors

* Improve testing generation

* Improve  the test generation

* Fix comments

* fix tests

* Refactor template generation

* Add AI skills for connector developement

* Add AI skills for connector developement

* Fix comments

* Add tests to scaffold

* Address edge cases

* Address edge cases

* Address comments
* Register connector skills in .claude/skills for Claude Code discovery

Adds symlinks so `/skills` command shows scaffold-connector,
connector-review, and load-standards alongside existing playwright skills.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename load-standards to connector-standards

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@ulixius9 ulixius9 changed the base branch from main to 1.11.14 March 23, 2026 09:26
@ulixius9 ulixius9 changed the base branch from 1.11.14 to 1.11.4 March 23, 2026 09:26
@ulixius9 ulixius9 changed the base branch from 1.11.4 to 1.11.12 March 23, 2026 09:26
@ulixius9
Copy link
Member

Hey @thiago-costa-fanatic, thanks for the contribution the pr looks good but I'm unable to merge it because the base for this pr is 1.11.12

can you please raise a pr for main instead of this release branch?

shah-harshit and others added 6 commits March 23, 2026 10:11
…pen-metadata#26683)

- Zero vertical margin on .entity-summary-in-docs inside the doc panel
- Top padding on .entity-summary-panel-container for alignment with requirements content

Made-with: Cursor
…onents (open-metadata#26563)

* Migration: replace MUI components with custom style and core-ui components

* fixed sonar issues

* used core ui typography instead of antd

* used core ui typography instead of antd
Bumps [socket.io-parser](https://github.com/socketio/socket.io) from 4.2.5 to 4.2.6.
- [Release notes](https://github.com/socketio/socket.io/releases)
- [Changelog](https://github.com/socketio/socket.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io/compare/socket.io-parser@4.2.5...socket.io-parser@4.2.6)

---
updated-dependencies:
- dependency-name: socket.io-parser
  dependency-version: 4.2.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
…pen-metadata#26626)

Bumps [flatted](https://github.com/WebReflection/flatted) from 3.3.4 to 3.4.2.
- [Commits](WebReflection/flatted@v3.3.4...v3.4.2)

---
updated-dependencies:
- dependency-name: flatted
  dependency-version: 3.4.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
…ts (open-metadata#26590)

* Fix open-metadata#26577: Restrict Trigger Autopilot action based on permissions

* add app permission for trigger and deploy

* Showing the Trigger autopilot button on App.trigger permission

* fix tests

* addressed gitar comments

* Addressed PR comments

* removed test.slow

---------

Co-authored-by: Rohit0301 <rj03012002@gmail.com>
Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
* add unit and integration tests for MCP OAuth, tools, and impersonation features

* fix McpOAuthIT: pin status codes and add Assumptions.assumeTrue for test ordering

* fix McpOAuthIT assertions

* spotless fix
@thiago-costa-fanatic thiago-costa-fanatic force-pushed the thiago/sigma-lineage-fixes branch from 6dfd089 to ffcbb44 Compare March 23, 2026 15:15
harshach and others added 4 commits March 23, 2026 16:17
* Enable more service unit tests and fix uncovered regressions

* Fix remaining broadened unit-suite regressions

* Add meaningful Handlebars helper coverage

* Add formatter decorator unit coverage

* Improve formatter decorator coverage

* Improve utility, validator, and formatter coverage

* Expand OIDC validator coverage

* Tighten shared OIDC validator coverage

* Improve user and connection utility coverage

* Cover subscription utility workflows

* Cover entity field utility workflows

* Expand lineage and helper utility coverage

* Improve auth code flow handler coverage

* Expand auth code flow handler coverage

* Cover entity csv parsing flows

* Deepen entity csv parser coverage

* Fix search builder aggregation null handling

* Expand entity utility core coverage

* Cover search index utility workflows

* Expand search utility coverage

* Expand formatter message coverage

* Harden notification markdown rendering coverage

* Add notification card assembler coverage

* Expand EntityCsv coverage and dry-run fixes

* Expand K8s pipeline client coverage

* Expand saml validator coverage

* Expand rdf property mapper coverage

* Expand subscription utility coverage

* Fix schema field extractor coverage gaps

* Expand auth refresh flow coverage

* Add service unit test workflow

* Enforce new-code coverage on service PRs

* Add Unit Test Coverage

* Expand k8s pipeline and auth flow coverage

* Expand entity csv batch import coverage

* Expand entity csv entity creation coverage

* Expand entity csv user and flush coverage

* Expand entity csv typed import coverage

* Cover entity csv dependency validation paths

* Expand airflow and csv utility coverage

* Replace placeholder authorizer tests with real coverage

* Cover PII masking security flows

* Tighten async service retry and shutdown coverage

* Expand security util claim coverage

* Fix checkstyle

* Strengthen user bootstrap utility coverage

* Expand user activity tracker coverage

* Expand ODCS converter coverage

* Expand S3 log storage coverage

* Expand search repository and lineage coverage

* Expand search filter and index factory coverage

* Expand reindex handler coverage

* Expand inherited field search coverage

* Expand search cluster metrics coverage

* Expand search repository lifecycle coverage

* Expand slack client coverage and stabilize tests

* Expand search index executor control flow coverage

* Cover search index utility helpers

* Cover distributed indexing strategy flows

* Strengthen distributed search executor coverage

* Cover search reindex pipeline flows

* Cover search index logging flows

* Cover search index stats tracking

* Cover quartz search index progress flows

* Cover search index app coordination

* Cover slack progress listener behavior

* Cover polling job notifier behavior

* Cover redis job notifier behavior

* Expand Slack notifier coverage

* Cover partition worker processing flows

* Expand distributed participant coverage

* Cover orphan job monitor behavior

* Expand distributed stats aggregator coverage

* Expand distributed partition coverage

* Strengthen distributed coordinator coverage

* Expand search index and repository coverage

* Expand search executor control flow coverage

* Expand search repository delegation coverage

* Expand search index executor coverage

* Expand search repository helper coverage

* Expand search utility coverage

* Expand search index executor coverage

* Expand search repository coverage

* Strengthen search index manager coverage

* Strengthen distributed recovery and worker coverage

* Strengthen distributed executor coverage

* Fix index sink batching and stats coverage

* Expand elastic bulk sink behavior coverage

* Expand open search bulk sink behavior coverage

* Fix dropped bulk processor failure accounting

* Cover migration workflow discovery paths

* fix java checkstyle

* Fix permission debug effect normalization

* Cover migration FQN repair workflows

* Fix glossary workflow migration idempotency

* Cover v1100 migration utility flows

* Cover v1104 migration extension flows

* Fix and cover v160 migration policy flows

* fix java checkstyle

* Address PR review comments on vector search and csv docs

* fix java checkstyle

* Harden service unit test PR workflow

* Cover migration utility repair flows

* fix java checkstyle

* Fix service unit test regressions

* Split service new-code coverage check

* fix java checkstyle

* Fix service diff coverage regressions

* fix java checkstyle

* Clarify missing JaCoCo artifact failures

* fix java checkstyle

* Fix bulk sink lifecycle tests

* simplify CI

* Address PR review feedback after main merge

* Fix merged service unit test expectations

* Fix search repository bulk update tests

* Apply spotless formatting

* Use standard exception logging in search repository

* Stabilize multi-domain search integration test

* Apply spotless formatting

* Isolate web analytic event integration timestamps

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
- Fixed bug where first page of chart elements was skipped in get_chart_details
- Added column name truncation to prevent errors with long column names
- Added get_workbook_queries method to fetch SQL from workbooks
- Implemented SQL parsing to extract table references for lineage
- Enhanced get_lineage_details with better error handling
- Added WorkbookQuery and WorkbookQueriesResponse models
- Falls back to file-based lineage when queries unavailable
- Added 7 unit tests covering query and file-based lineage
- Updated documentation with lineage extraction details
- Change fallback dialect from Snowflake to ANSI for better SQL compatibility
- Add null guard in yield_dashboard_lineage_details to prevent exceptions
- Pre-index queries by elementId for O(n) lookup instead of O(n²)
- Add per-element file-based lineage fallback when SQL is missing
- Implement pagination in get_workbook_queries to fetch all query pages
- Add unit tests for pagination fix in get_chart_details
@thiago-costa-fanatic
Copy link
Contributor Author

Closing this in favor of #26699

@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed for 'open-metadata-ingestion'

Failed conditions
E Security Review Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.