Skip to content

exclude unused fields#25707

Open
amfor wants to merge 4 commits intoopen-metadata:mainfrom
amfor:feature/optimize_search_previews
Open

exclude unused fields#25707
amfor wants to merge 4 commits intoopen-metadata:mainfrom
amfor:feature/optimize_search_previews

Conversation

@amfor
Copy link
Contributor

@amfor amfor commented Feb 4, 2026

Describe your changes:

Fixes

I worked on ... because ...

Tested on our live deployment, who has many queries attributed to search results.

  ┌───────────────┬───────────────────────────┬───────────────────────────────────────┬───────────────┐
  │    Metric     │    Before (No Filter)     │ After (exclude_source_fields=queries) │  Improvement  │
  ├───────────────┼───────────────────────────┼───────────────────────────────────────┼───────────────┤
  │ Response Time │ 1.188 seconds             │ 0.377 seconds                         │ 68.3% faster  │
  ├───────────────┼───────────────────────────┼───────────────────────────────────────┼───────────────┤
  │ Payload Size  │ 5,957,627 bytes (5.68 MB) │ 1,181,706 bytes (1.13 MB)             │ 80.2% smaller │
  ├───────────────┼───────────────────────────┼───────────────────────────────────────┼───────────────┤
  │ HTTP Status   │ 200 OK                    │ 200 OK                                │ ✓             │
  └───────────────┴───────────────────────────┴───────────────────────────────────────┴───────────────┘

Summary by Gitar

  • Performance optimization:
    • Added excludeSourceFields parameter excluding columns, queries, and columnNames from search suggestion results
  • Targeted implementation:
    • Updated only Suggestions.tsx search query to avoid affecting other search functionality

This will update automatically on new commits.


Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@amfor amfor marked this pull request as ready for review February 5, 2026 23:49
@amfor amfor requested a review from a team as a code owner February 5, 2026 23:49
@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot
Copy link

gitar-bot bot commented Feb 6, 2026

🔍 CI failure analysis for 4f707dd: All 9 CI jobs failed due to missing 'safe to test' label. PR is approved and ready but awaiting maintainer label to enable CI validation.

Issue

All CI checks continue to fail because the PR is still missing the required "safe to test" label.

Root Cause

The OpenMetadata repository has a security workflow that validates PRs have the safe to test label before running CI jobs. This prevents untrusted code from external contributors from executing in the CI environment.

Details

All 9 failed CI jobs in this run encountered the same label validation error:

  • java-checkstyle (job id: 62694622981)
  • py-checkstyle (job id: 62694629631)
  • ui-coverage-tests (job id: 62694635866)
  • playwright-ci-postgresql (6 parallel jobs with ids: 62694622679, 62694622692, 62694622693, 62694622696, 62694622707, 62694622719)

Each job failed at the label validation step with error: This pull request does not contain any of the valid labels: ['safe to test']

Important: The PR code has been approved ✅ and is ready for CI validation. The author has addressed all code review feedback with a targeted, clean implementation. CI cannot run to validate these improvements until the label is added.

Code Review ✅ Approved 1 resolved / 1 findings

Clean performance optimization that adds excludeSourceFields to only the Suggestions component, correctly avoiding global changes. The targeted approach resolves the previous concern about affecting all search callers.

✅ 1 resolved
Bug: Global searchData function excludes fields for ALL callers

📄 openmetadata-ui/src/main/resources/ui/src/rest/miscAPI.ts:56-59
The exclude_source_fields parameter is hardcoded in the global searchData function, which means ALL search API calls throughout the application will exclude columns, queries, and columnNames fields.

This could break functionality in parts of the application that actually need these fields. For example:

  • Table detail views that need to display column information
  • Query analysis features that need the queries field
  • Any search result display that shows column names

Impact: Features depending on these fields in search results will silently fail to display data.

Suggested fix: Instead of hardcoding the exclusion in the global function, pass it as an optional parameter that callers can specify when needed:

export const searchData = <SI extends SearchIndex>(
  // ... existing params
  excludeSourceFields?: string[],
) => {
  // ... existing code
  return APIClient.get<SearchResponse<SI>>(`/search/query?q=${q}`, {
    params: {
      ...params,
      ...(excludeSourceFields && { exclude_source_fields: excludeSourceFields }),
    },
  });
};

Then in Suggestions.tsx, pass the exclusion fields explicitly for that specific use case.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant