-
Notifications
You must be signed in to change notification settings - Fork 13.9k
[FLINK-27661][Runtime / Metrics] PrometheusPushGatewayReporter support pushgateway http authentication #27576
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…t pushgateway http authentication
| @@ -70,17 +72,28 @@ public PrometheusPushGatewayReporter createMetricReporter(Properties properties) | |||
| jobName = configuredJobName + new AbstractID(); | |||
| } | |||
|
|
|||
| String username = metricConfig.getString(USERNAME.key(), null); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor consistency nit: for consistency, we may want to use defaultValue() instead of null here as those would both functionally identical (null) but would follow the same pattern as the other available configuration options.
For example, HOST_URL:
String hostUrl = metricConfig.getString(HOST_URL.key(), HOST_URL.defaultValue());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for catching this! You're absolutely right - I've updated the code to use USERNAME.defaultValue() and PASSWORD.defaultValue() instead of hardcoded null to maintain consistency with other configuration options like HOST_URL.
| metricConfig.setProperty(USERNAME.key(), "flink-user"); | ||
| metricConfig.setProperty(PASSWORD.key(), "flink-password"); | ||
| // Both username and password configured - Basic Auth should be enabled | ||
| PrometheusPushGatewayReporter reporter = factory.createMetricReporter(metricConfig); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to add another assertion to this test to verify that authentication was actually enabled. We may want to consider following a pattern similar to hostUrl to expose it for testing (e.g. @VisibleForTesting package-private boolean field), something like basicAuthEnabled and then we could simply check that property here:
assertThat(reporter.basicAuthEnabled).isTrue();
Either that or rely on a reflection based approach to inspect the underlying connection factory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion! I've added a @VisibleForTesting boolean basicAuthEnabled field following the same pattern as hostUrl. Now all four basic auth test cases verify this property:
testBasicAuthNotEnabledWithoutCredentials - asserts basicAuthEnabled is false
testBasicAuthNotEnabledWithOnlyUsername - asserts basicAuthEnabled is false
testBasicAuthNotEnabledWithOnlyPassword - asserts basicAuthEnabled is false
testBasicAuthEnabledWithBothCredentials - asserts basicAuthEnabled is true
| hostUrl, | ||
| jobName, | ||
| randomSuffix, | ||
| deleteOnShutdown, | ||
| groupingKey); | ||
| groupingKey, | ||
| username != null && password != null); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nit: this can probably align with the recently added basicAuthEnabled flag to avoid the redundant logic.
What is the purpose of the change
This pull request adds HTTP Basic Authentication support for the PrometheusPushGatewayReporter. This enables users to connect Flink metrics reporting to secured PushGateway instances that require authentication.
Many production deployments secure their PushGateway endpoints with authentication to prevent unauthorized metrics submission. Without this feature, users would need to configure network-level security (firewalls, VPNs) to protect the PushGateway, which is often more complex and less flexible than application-level authentication.
Brief change log
usernameandpasswordconfiguration options toPrometheusPushGatewayReporterOptionsfor specifying HTTP Basic Auth credentialsPrometheusPushGatewayReporterto configureBasicAuthHttpConnectionFactorywhen both credentials are providedPrometheusPushGatewayReporterFactoryto read credentials from configuration and emit a warning log when only one of username/password is configured (incomplete configuration)Verifying this change
This change added tests and can be verified as follows:
PrometheusPushGatewayReporterTest#testBasicAuthNotEnabledWithoutCredentials- verifies reporter creation without credentialsPrometheusPushGatewayReporterTest#testBasicAuthNotEnabledWithOnlyUsername- verifies warning log when only username is configuredPrometheusPushGatewayReporterTest#testBasicAuthNotEnabledWithOnlyPassword- verifies warning log when only password is configuredPrometheusPushGatewayReporterTest#testBasicAuthEnabledWithBothCredentials- verifies reporter creation with complete credentialsDoes this pull request potentially affect one of the following parts:
simpleclient_pushgatewaydependency which already includesBasicAuthHttpConnectionFactory)@Public(Evolving): yes (added new configuration optionsusernameandpasswordtoPrometheusPushGatewayReporterOptionswhich is annotated with@PublicEvolving)Documentation
docs/content/docs/deployment/metric_reporters.mdanddocs/content.zh/docs/deployment/metric_reporters.md)