-
Notifications
You must be signed in to change notification settings - Fork 809
SOLR-17436: Create a v2 equivalent for /admin/metrics #4057
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add interface MetricsApi and class GetMetrics. Add MetricsUtil to merge the List<MetricSnapshot> in either MetricsHandler or GetMetrics. Implementations in MessageBodyWriters: not sure if needed. Note in SplitShardCmd: this appears to have been broken before. The Solr response might always be null now. More testing and cleaning-up to do.
Fix compilation and add unit tests
Adjust documentation. Set ResponseParser in MetricsRequest constructor. Clean-up.
Last clean-up after testing proxyToNodes() on a local cluster.
add changelog, and commit result of tidy
…olr.git into SOLR-17436-V2-Metrics-API
gerlowskija
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was very excited to see this PR @igiguere , thanks for putting this together!
I've only reviewed about half the PR but wanted to publish what I have so far.
Everything looks great overall so far, though there were a few small things I think will need tweaked that I've tried to call out inline. Lmk if any of those comments are unclear or you disagree and I'll try to respond when I come back for the second half later!
solr/api/src/java/org/apache/solr/client/api/model/MetricsResponse.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/cloud/api/collections/SplitShardCmd.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/handler/admin/api/GetMetrics.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/handler/admin/api/GetMetrics.java
Outdated
Show resolved
Hide resolved
gerlowskija
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, finished my review. Left a few more in-line comments, though things look very good overall. Just a few smaller things I think we'll need to change.
Let me know if you have any questions or I can help with anything!
solr/core/src/java/org/apache/solr/jersey/JerseyApplications.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/response/PrometheusResponseWriter.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java
Outdated
Show resolved
Hide resolved
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc
Outdated
Show resolved
Hide resolved
| === OpenMetrics | ||
|
|
||
| OpenMetrics format is available from the `/admin/metrics` endpoint by providing the `wt=openmetrics` parameter or by passing the Accept header `application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of the Prometheus format that adds additional metadata and exemplars. | ||
| OpenMetrics format is available from the `/metrics` endpoint by providing the `wt=openmetrics` parameter or by passing the Accept header `application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of the Prometheus format that adds additional metadata and exemplars. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Q] Huh - I'm surprised that these docs mention the "Accept" header even before you added a v2 endpoint in this PR. I didn't think anything on the v1 side looked at "Accept". I guess that's a good thing, I'm just surprised is all...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had written something custom to check content-type but just for OpenMetrics but not prometheus. See my comment above.
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc
Show resolved
Hide resolved
|
|
||
| /** Request to "/admin/metrics" */ | ||
| public class MetricsRequest extends SolrRequest<SolrResponse> { | ||
| public class MetricsRequest extends SolrRequest<InputStreamResponse> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Q] Why change the SolrJ class that corresponds to the v1 metrics API? Your PR only modifies the v2 API, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
V1, since Solr 10, returns a stream (prometheus or openmetrics), not a SolrResponse (where I would expect a NamedList... right?).
If I recall, the only place in Solr code where MetricsRequest is used is a method somewhere in the CLI tools that stars with something like : if (true) return; The method is disabled, and there's a couple of comments about tickets to be done.
I think the type param InputStreamResponse makes it clearer that this is not a typical Solr response.
solr/solr-ref-guide/modules/deployment-guide/pages/metrics-reporting.adoc
Show resolved
Hide resolved
mlbiscoc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @igiguere for doing this! The timing was great as the metrics api just had a huge overhaul in 10 so getting up to date on V2 gets us closer to a more modern path.
solr/api/src/java/org/apache/solr/client/api/model/MetricsResponse.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/handler/admin/api/GetMetrics.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/GetMetricsTest.java
Outdated
Show resolved
Hide resolved
| === OpenMetrics | ||
|
|
||
| OpenMetrics format is available from the `/admin/metrics` endpoint by providing the `wt=openmetrics` parameter or by passing the Accept header `application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of the Prometheus format that adds additional metadata and exemplars. | ||
| OpenMetrics format is available from the `/metrics` endpoint by providing the `wt=openmetrics` parameter or by passing the Accept header `application/openmetrics-text;version=1.0.0`. OpenMetrics is an extension of the Prometheus format that adds additional metadata and exemplars. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had written something custom to check content-type but just for OpenMetrics but not prometheus. See my comment above.
Address review comments. Add query params to the API method. Fix implementation and tests. Adjust documentation and comments. Remove class MetricsResponse. TODO : AdminHandlersProxy only supports V1
|
@gerlowskija, @mlbiscoc gradlew clean check in progress, locally. |
Adapt AdminHandlersProxy to handle V2 API. Change GetMetricsTest to send REST GET requests. Adapt MetricsRequest to support V1 and V2, and set the response parser, according to 'wt'... Should handle "Accept" header, but SolrRequest is not aware of HTTP headers. Adapt re-existing code (tests) accordingly. Add unit tests for MetricsRequest
remove temp file, remove ingore
Address review comments: Keep just one utility class for metrics (MetricUtils). Remove debug "System.out" form unit tests.
…olr.git into SOLR-17436-V2-Metrics-API
AdminHandlersProxy now supports both V1 and V2. |
solr/core/src/java/org/apache/solr/handler/admin/api/GetMetrics.java
Outdated
Show resolved
Hide resolved
solr/modules/jwt-auth/src/test/org/apache/solr/security/jwt/JWTAuthPluginTest.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/handler/admin/AdminHandlersProxy.java
Show resolved
Hide resolved
Remove unnecessary logging. Remove debug System.out
Merge remote-tracking branch 'origin-solr/main' into SOLR-17436-V2-Metrics-API
https://issues.apache.org/jira/browse/SOLR-17436
Description
Create a JAX-RS V2 equivalent for /admin/metrics.
Solution
Add interface MetricsApi, and implementation GetMetrics, as an extension of JerseyResource (via AdminAPIBase).
The new full URL path is /api/metrics. The metrics response is a StreamingOutput.
Add class MetricsUtil to re-group logic used by both MetricsHandler and GetMetrics. Both classes get their information directly from SolrMetricManager.
Adjust documentation.
Tests
Added unit tests for the GetMetrics implementation of MetricsApi.
Added unit tests for SolrJ MetricsRequest.
Functional tests on "devSlim", with a mini cluster of 2 nodes.
Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.