Skip to content

feat: add opt-in dataplane format#596

Open
github-vincent-miszczak wants to merge 1 commit into
VictoriaMetrics:mainfrom
github-vincent-miszczak:dataplane_format
Open

feat: add opt-in dataplane format#596
github-vincent-miszczak wants to merge 1 commit into
VictoriaMetrics:mainfrom
github-vincent-miszczak:dataplane_format

Conversation

@github-vincent-miszczak
Copy link
Copy Markdown
Contributor

@github-vincent-miszczak github-vincent-miszczak commented Feb 26, 2026

Add an opt-in setting and the corresponding UI toggle to enable logs dataplane format.

Using this format allows Grafana correlations to work correctly with label values.
Without it, it's not possible to craft links that use label values when using Grafana correlations.

It's opt-in because the fields are renamed, which may break client-side transformations.


Summary by cubic

Adds an opt-in setting to output logs in Grafana dataplane format, enabling automatic ${labelKey} variables in correlations. Default is off to avoid breaking transformations that reference Time/Line.

  • New Features
    • New “Use dataplane format” toggle in datasource settings; provisioning key: jsonData.useDataplaneFormat.
    • When enabled: rename Timetimestamp and Linebody, and set frame meta to DataFrameType.LogLines with typeVersion [0,0] so ${labelKey} works in correlations.
    • Updated README, CHANGELOG, and tests to cover the new format.

Written for commit e49b2ea. Summary will update on new commits.

@github-vincent-miszczak github-vincent-miszczak marked this pull request as ready for review February 26, 2026 17:09
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

@arturminchukov
Copy link
Copy Markdown
Member

@github-vincent-miszczak Thanks for contributing! I didn't understand how it could help with the correlations. I'm just tried to set up a correlation but instead of Line field we just get the body field. I would be glad, if could share me a guide with setting up a correlation with a data plane format.

@arturminchukov arturminchukov added enhancement New feature or request vl-datasource labels Mar 30, 2026
@github-vincent-miszczak
Copy link
Copy Markdown
Contributor Author

Here is an example using the dataplane format without any transformation.
The link is correctly dynamically generated with our field value.

Label values get automagically promoted as top level variables.
Without this format, there is no syntax to access label values. At best we can workaround using transformation/regex, but that's not ideal.

image image image

@arturminchukov
Copy link
Copy Markdown
Member

@github-vincent-miszczak Could you please sign your commits? Here, you can find a guide on how to do it.

@arturminchukov
Copy link
Copy Markdown
Member

@github-vincent-miszczak
Right now, to use Grafana correlations (like linking from logs to traces), you have to enable the dataplane format. The problem is that it renames Time to timestamp and Line to body, which can break existing dashboards and panels that rely on those field names.

What I suggest: the plugin reads the correlation configs that the user has set up in Grafana, looks at which label field each correlation needs, pulls that value out of the log labels, and creates a clickable field for it — all at query time, automatically.

So the user just creates a correlation in the Grafana UI (say, logs → Tempo with trace_id), and it works. No need to enable any toggles, no field renaming, no risk of breaking anything.

To get the correlation configs at runtime, we use getCorrelationsService() from @grafana/runtime — specifically getCorrelationsBySourceUIDs([datasourceUid]) which returns all correlations where our datasource is
the source. Each correlation has a config.field telling us which label to extract.
What do you think?

@github-vincent-miszczak
Copy link
Copy Markdown
Contributor Author

Hi, thanks for taking the time look at this.

Yes, the field rename is annoying, but the opt-in toggle keeps it safe. It defaults to off, so no existing dashboard breaks. Users who need correlations flip it on knowingly.

It's the Grafana standard. The logs dataplane contract is the documented way for log datasources to expose label values to features like correlations, and Grafana core already special-cases it in many places (logsFrame.ts, LogDetails.tsx, LogLineDetailsComponent.tsx, the logs-table extractLogsFieldsTransform, panel suggestions, etc.). Building a plugin-side workaround duplicates what the platform already provides natively.

Correlations aren't the only consumer. Label promotion happens in getFieldLinksForExplore (https://github.com/grafana/grafana/blob/main/public/app/features/explore/utils/links.ts)): when frame.meta.type === DataFrameType.LogLines, every label is added to scopedVars as a flat ${labelKey}. That same path serves both correlation links and regular data links / derived fields — not correlation-specific.

Future-proofing label metadata. The dataplane spec also defines an optional labelTypes field (Grafana 12.4+) that lets us distinguish stream fields from parsed/computed fields in Log Details via DataSourceWithLogsLabelTypesSupport. That's the standard path for surfacing VictoriaLogs' stream-vs-log-field distinction in the UI.

Less plugin code, less drift. The dataplane path is just a field rename + frame meta — minimal surface area.

@arturminchukov
Copy link
Copy Markdown
Member

@github-vincent-miszczak could you please rebase on main?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request vl-datasource

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants