-
Notifications
You must be signed in to change notification settings - Fork 13
Log sys testing, starting to get "serious" about it.. #402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances the logging subsystem in the tractor library by adding improved introspection capabilities and type annotations. The changes introduce automatic module name detection for sub-loggers and add comprehensive testing for the logging functionality.
- Adds automatic caller module introspection when creating sub-loggers
- Improves type annotations for better code clarity and tooling support
- Implements comprehensive test coverage for logger creation and module name handling
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| tractor/log.py | Enhanced logging with auto-detection of caller modules, improved type annotations, and refactored parameter handling |
| tests/test_log_sys.py | New comprehensive test suite covering logger creation, module name detection, and package hierarchy handling |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| callstack: list[FrameInfo] = stack() | ||
| caller_fi: FrameInfo = callstack[1] | ||
| caller_mod: ModuleType = getmodule(caller_fi.frame) | ||
| if caller_mod: | ||
| # ?how is this `mod.__name__` defined? | ||
| # -> well by how the mod is imported.. | ||
| # |_https://stackoverflow.com/a/15883682 | ||
| mod_name: str = caller_mod.__name__ | ||
| mod_pkg: str = caller_mod.__package__ | ||
| log.info( | ||
| f'Generating sub-logger name,\n' | ||
| f'{mod_pkg}.{mod_name}\n' | ||
| ) |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using stack() for caller introspection on every logger creation could impact performance in hot paths. Consider caching the introspection result or making this feature opt-in for performance-critical applications.
| # |_https://stackoverflow.com/a/15883682 | ||
| mod_name: str = caller_mod.__name__ | ||
| mod_pkg: str = caller_mod.__package__ | ||
| log.info( |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This info-level log statement will be emitted every time a logger is created, potentially causing log noise. Consider using a debug level or removing this diagnostic logging in production code.
| log.info( | |
| log.debug( |
| ): | ||
| callstack: list[FrameInfo] = stack() | ||
| caller_fi: FrameInfo = callstack[1] | ||
| caller_mod: ModuleType = getmodule(caller_fi.frame) |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The getmodule() function can return None if the module cannot be determined. The type annotation indicates ModuleType but the subsequent None check suggests this is expected. The type annotation should be ModuleType | None.
| caller_mod: ModuleType = getmodule(caller_fi.frame) | |
| caller_mod: ModuleType | None = getmodule(caller_fi.frame) |
| caller_fi: FrameInfo = callstack[1] | ||
| caller_mod: ModuleType = getmodule(caller_fi.frame) | ||
| if caller_mod: | ||
| # ?how is this `mod.__name__` defined? | ||
| # -> well by how the mod is imported.. | ||
| # |_https://stackoverflow.com/a/15883682 | ||
| mod_name: str = caller_mod.__name__ | ||
| mod_pkg: str = caller_mod.__package__ | ||
| log.info( | ||
| f'Generating sub-logger name,\n' | ||
| f'{mod_pkg}.{mod_name}\n' | ||
| ) | ||
| # if pkg_name in caller_mod.__package__: | ||
| # from tractor.devx.debug import mk_pdb | ||
| # mk_pdb().set_trace() | ||
|
|
||
| if ( | ||
| pkg_name | ||
| # and | ||
| # pkg_name in mod_name | ||
| ): | ||
| name = mod_name |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accessing callstack[1] without checking the length could raise an IndexError if the call stack is unexpectedly shallow. Add a bounds check or handle the IndexError.
| caller_fi: FrameInfo = callstack[1] | |
| caller_mod: ModuleType = getmodule(caller_fi.frame) | |
| if caller_mod: | |
| # ?how is this `mod.__name__` defined? | |
| # -> well by how the mod is imported.. | |
| # |_https://stackoverflow.com/a/15883682 | |
| mod_name: str = caller_mod.__name__ | |
| mod_pkg: str = caller_mod.__package__ | |
| log.info( | |
| f'Generating sub-logger name,\n' | |
| f'{mod_pkg}.{mod_name}\n' | |
| ) | |
| # if pkg_name in caller_mod.__package__: | |
| # from tractor.devx.debug import mk_pdb | |
| # mk_pdb().set_trace() | |
| if ( | |
| pkg_name | |
| # and | |
| # pkg_name in mod_name | |
| ): | |
| name = mod_name | |
| if len(callstack) > 1: | |
| caller_fi: FrameInfo = callstack[1] | |
| caller_mod: ModuleType = getmodule(caller_fi.frame) | |
| if caller_mod: | |
| # ?how is this `mod.__name__` defined? | |
| # -> well by how the mod is imported.. | |
| # |_https://stackoverflow.com/a/15883682 | |
| mod_name: str = caller_mod.__name__ | |
| mod_pkg: str = caller_mod.__package__ | |
| log.info( | |
| f'Generating sub-logger name,\n' | |
| f'{mod_pkg}.{mod_name}\n' | |
| ) | |
| # if pkg_name in caller_mod.__package__: | |
| # from tractor.devx.debug import mk_pdb | |
| # mk_pdb().set_trace() | |
| if ( | |
| pkg_name | |
| # and | |
| # pkg_name in mod_name | |
| ): | |
| name = mod_name |
| # if pkg_name in caller_mod.__package__: | ||
| # from tractor.devx.debug import mk_pdb | ||
| # mk_pdb().set_trace() | ||
|
|
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commented-out debugging code should be removed from production code to keep the codebase clean.
| # if pkg_name in caller_mod.__package__: | |
| # from tractor.devx.debug import mk_pdb | |
| # mk_pdb().set_trace() |
| # from tractor.devx.debug import mk_pdb | ||
| # mk_pdb().set_trace() |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commented-out debugging code should be removed from production code to keep the codebase clean.
| # from tractor.devx.debug import mk_pdb | |
| # mk_pdb().set_trace() |
| ): | ||
| name = mod_name | ||
|
|
||
| # XXX, lowlevel debuggin.. |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling error: 'debuggin' should be 'debugging'.
| # XXX, lowlevel debuggin.. | |
| # XXX, lowlevel debugging.. |
| assert len(sub_logs) == 1 # only one nested sub-pkg module | ||
| assert submod.log.logger in sub_logs | ||
|
|
||
| # breakpoint() |
Copilot
AI
Sep 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commented-out breakpoint should be removed from test code.
| # breakpoint() |
To start ensuring that when `name=__name__` is passed we try to
de-duplicate the `_root_name` and any `leaf_mod: str` since it's already
included in the headers as `{filename}`.
Deats,
- heavily document the de-duplication `str.partition()`s in
`.log.get_logger()` and provide the end fix by changing the predicate,
`if rname == 'tractor':` -> `if rname == _root_name`.
* also toss in some warnings for when we still detect duplicates.
- add todo comments around logging "filters" (vs. our "adapter").
- create the new `test_log_sys.test_root_pkg_not_duplicated()` which
runs green with the fixes from ^.
- add a ton of test-suite todos both for existing and anticipated
logging sys feats in the new mod.
A bit of test driven dev to anticipate support of `.log.get_logger()`
usage such that it can be called from arbitrary sub-modules, themselves
embedded in arbitrary sub-pkgs, of some project; the when not provided,
the `sub_name` passed to the `Logger.getChild(<sub_name>)` will be set
as the sub-pkg path "down to" the calling module.
IOW if you call something like,
`log = tractor.log.get_logger(pkg_name='mypylib')`
from some `submod.py` in a project-dir that looks like,
mypylib/
mod.py
subpkg/
submod.py <- calling module
the `log: StackLevelAdapter` child-`Logger` instance will have a
`.name: str = 'mypylib.subpkg'`, discluding the `submod` part since this
already rendered as the `{filename}` header in `log.LOG_FORMAT`.
Previously similar behaviour would be obtained by passing
`get_logger(name=__name__)` in the calling module and so much so it
motivated me to make this the default, presuming we can introspect for
the info.
Impl deats,
- duplicated a `load_module_from_path()` from `modden` to load the
`testdir` rendered py project dir from its path.
|_should prolly factor it down to this lib anyway bc we're going to
need it for hot code reload? (well that and `watchfiles` Bp)
- in each of `mod.py` and `submod.py` render the `get_logger()` code
sin `name`, expecting the (coming shortly) implicit introspection
feat to do this.
- do `.name` and `.parent` checks against expected sub-logger values
from `StackLevelAdapter.logger.getChildren()`.
That is when no `name` is passed to `get_logger()`, try to introspect
the caller's `module.__name__` and use it to infer/get the "namespace
path" to that module the same as if using `name=__name__` as in the most
common usage.
Further, change the `_root_name` to be `pkg_name: str`, a public and
more obvious param name, and deprecate the former. This obviously adds
the necessary impl to make the new
`test_sys_log::test_implicit_mod_name_applied_for_child` test pass.
Impl detalles for `get_logger()`,
- add `pkg_name` and deprecate `_root_name`, include failover logic
and a warning.
- implement calling module introspection using
`inspect.stack()/getmodule()` to get both the `.__name__` and
`.__package__` info alongside adjusted logic to set the `name`
when not provided but only when a new `mk_sublog: bool` is set.
- tweak the `name` processing for implicitly set case,
- rename `sub_name` -> `pkg_path: str` which is the path
to the calling module minus that module's name component.
- only partition `name` if `pkg_name` is `in` it.
- use the `_root_log` for `pkg_name` duplication warnings.
Other/related,
- add types to various public mod vars missing them.
- rename `.log.log` -> `.log._root_log`.
9639c3b to
8842b75
Compare
Another sloppy summary anyone?
This pull request introduces significant improvements and refactoring to the logging subsystem in
tractor.log. The changes focus on enhancing logger naming behavior, improving sub-logger creation, and updating documentation and type annotations for clarity and maintainability. Additionally, new unit tests have been added to verify the correct logger name handling and module introspection. Below are the most important changes:Logger Naming and Sub-Logger Improvements:
get_logger()to usepkg_name(formerly_root_name) as the primary root logger identifier, deprecating the old parameter and warning users if it is used. The logger now introspects the caller's module whennameis not provided, automatically generating sub-logger names based on the module context. This avoids duplication of the root package name in logger names and ensures sub-loggers are correctly nested and filtered. [1] [2] [3]Type Annotations and Documentation:
LOG_FORMAT,DATE_FORMAT,BOLD_PALETTE) and clarified docstrings throughout the logging module, making the codebase more readable and maintainable. [1] [2] [3] [4] [5]tractor/log.pyto describe the enhanced logging system, its concurrency features, and usage patterns.Testing Enhancements:
tests/test_log_sys.pywith unit tests to verify that root package names are not duplicated in logger names and that implicit module name detection for sub-loggers works as intended. This includes utilities for dynamically loading modules from file paths for testing purposes.Internal Refactoring:
logto_root_logwithin the logging module to clarify its role and avoid confusion with user-level loggers.links dump from getting to the current impl in this patch,