Skip to content

fix: prevent verbose/debug logs from leaking to Application Insights#740

Open
qtomlinson wants to merge 4 commits intoclearlydefined:masterfrom
qtomlinson:qt/log_transform
Open

fix: prevent verbose/debug logs from leaking to Application Insights#740
qtomlinson wants to merge 4 commits intoclearlydefined:masterfrom
qtomlinson:qt/log_transform

Conversation

@qtomlinson
Copy link
Collaborator

Problem: Verbose and debug log messages were being sent to Application Insights even when the logger level was set to 'info'.

Root cause: The previous implementation used logger.on('data', ...) to forward Winston logs to Application Insights. This stream-based event listener operated outside Winston's transport pipeline, bypassing the transport-level filtering that enforces log level thresholds.

Solution: Replace the 'data' event listener with a custom ApplicationInsightsTransport extending winston-transport. As a proper transport, it participates in Winston's built-in level filtering, so messages below the configured threshold are never forwarded.

Additional improvements:

  • Extract shared format (sanitize + errors) to logger level to avoid duplication across transports
  • Use raw info.message for AI telemetry instead of printf-formatted string, since AI tracks timestamps and severity separately
  • Preserve original stack trace in trackException instead of reconstructing a new Error
  • Add unit tests for level filtering, error routing, and severity

Task: #739

@qtomlinson qtomlinson marked this pull request as ready for review March 20, 2026 00:57
@qtomlinson qtomlinson requested a review from JamieMagee March 20, 2026 00:57
JamieMagee

This comment was marked as outdated.

Comment on lines +39 to +42
afterEach(() => {
clock.restore()
})

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These loggers are never closed. Winston holds transport streams open, so Mocha can hang. Worth hoisting logger and calling logger.close() in afterEach:

Suggested change
afterEach(() => {
clock.restore()
})
afterEach(() => {
logger.close()
clock.restore()
})

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The custom transport does not open a file stream or socket itself. It just forwards to trackTrace / trackException and calls the Winston callback. There is no transport-owned resource here that must be “closed” like a file transport. So the close might be optional.

There is an existing issue of the Application Insights SDK started may buffer telemetry. To avoid losing the last telemetry batch, we may want to consider calling AppInsight flush and wait for it to complete on graceful process shutdown. Can that be addressed as a separate PR?

Problem: Verbose and debug log messages were being sent to Application
Insights even when the logger level was set to 'info'.

Root cause: The previous implementation used logger.on('data', ...) to
forward Winston logs to Application Insights. This stream-based event
listener operated outside Winston's transport pipeline, bypassing the
transport-level filtering that enforces log level thresholds.

Solution: Replace the 'data' event listener with a custom
ApplicationInsightsTransport extending winston-transport. As a proper
transport, it participates in Winston's built-in level filtering, so
messages below the configured threshold are never forwarded.

Additional improvements:
- Extract shared format (sanitize + errors) to logger level to avoid
  duplication across transports
- Use raw info.message for AI telemetry instead of printf-formatted
  string, since AI tracks timestamps and severity separately
- Preserve original stack trace in trackException instead of
  reconstructing a new Error
- Add unit tests for level filtering, error routing, and severity
Coerce CRAWLER_ECHO with String(value ?? '').trim().toLowerCase() so boolean and mixed-case string values are handled consistently.

When CRAWLER_ECHO is unset, it behaves as false which reduces production console log noise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants