-
Notifications
You must be signed in to change notification settings - Fork 2
Adding asyncio unit tests #22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bradpenney
wants to merge
2
commits into
firestoned:main
Choose a base branch
from
bradpenney:adding-asyncio-unit-tests
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # Firestone-lib Copilot Instructions | ||
|
|
||
| ## Project Scope | ||
| - Firestone-lib bundles shared capabilities used across the broader Firestone ecosystem: CLI utilities, schema tooling, and lightweight helpers. | ||
| - Consumers expect stable interfaces; keep changes backward-compatible unless a coordinated release states otherwise. | ||
|
|
||
| ## Ecosystem Awareness | ||
| - `firestone` consumes these helpers for spec generation; validate CLI or resource-facing changes against its workflows (see `firestone/__main__.py` and `firestone/spec/`). | ||
| - `forevd` depends on `firestone_lib.cli` for logging and config parsing—document any behavior shifts so the proxy can bump in lockstep. | ||
| - If a change requires synchronized updates, land `firestone-lib` first and surface follow-up PR references in the release notes or change description. | ||
|
|
||
| ## Architecture Overview | ||
| - `firestone_lib/cli.py`: click-based helpers (logging bootstrap, custom param types, async wrappers). Extend here when exposing new CLI behavior. | ||
| - `firestone_lib/resource.py`: schema ingestion/validation built around `jsonref` and `jsonschema`; reuse its loaders to keep `$ref` support intact. | ||
| - `firestone_lib/utils.py`: string-centric helpers used across Firestone services. | ||
| - `firestone_lib/resources/`: packaged configuration assets (for example, logging config in `logging/cli.conf`). | ||
| - Tests mirror the package layout under `test/`. | ||
|
|
||
| ## Build & Automation | ||
| - Poetry drives installs, builds, and test runs (`poetry install`, `poetry build`, `poetry run pytest`). | ||
| - Continuous integration is handled via `.github/workflows/pr.yml`; expect linting and tests to run on every pull request. | ||
| - Local pre-flight: run `poetry run pylint firestone_lib test` and `poetry run pytest --cov=firestone_lib --cov-report term-missing` before opening a PR. CI enforces both commands, and we aim to keep coverage at 100%. | ||
|
|
||
| ## Collaboration Notes | ||
| - Before adding new modules or resources, confirm they fit the existing structure; prefer evolving current packages. | ||
| - Keep logging messages meaningful and consistent with existing `_LOGGER` usage. | ||
| - When working on CLI or schema changes, ensure templating, environment-variable substitution, and file loading continue to behave as they do today. | ||
|
|
||
| ## Coding Conventions | ||
| - For language-level expectations (Python 3.9+ compatibility, mandatory type hints, pytest usage, formatting requirements), follow `.github/instructions/python.instructions.md`. | ||
| - When adding tests, include docstrings or inline comments for helper classes/overrides so pylint passes without silencing warnings globally. Use targeted `# pylint: disable=` comments with a short justification when you must access module-private APIs like `_jsonloader`. | ||
| - Resource loaders rely on the current `file://` / `file:` path normalization; keep that behavior intact and update tests accordingly if you touch it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| --- | ||
| applyTo: '**/*.py' | ||
| --- | ||
|
|
||
| # Python Coding Guidelines | ||
|
|
||
| These rules combine firestone-lib conventions with widely accepted Python practices. Where industry defaults differ, follow the repo-specific rule noted here. | ||
|
|
||
| They intentionally mirror the upstream Firestone project so moving changes between repositories stays predictable; if you spot a divergence, flag it so we can reconcile both sides. | ||
|
|
||
| ## Runtime & Dependencies | ||
| - Target Python **3.12+** as defined in `pyproject.toml`; keep code compatible with the supported range. | ||
| - Manage dependencies through Poetry and keep `pyproject.toml` and `poetry.lock` in sync when adding or bumping packages. | ||
|
|
||
| ## Type Hints & Docstrings | ||
| - Annotate every function and method (including inner helpers) with precise parameter and return types. | ||
| - Prefer modern `typing` / `collections.abc` generics (`Iterator`, `Mapping`, `Sequence`, etc.) in annotations. | ||
| - Keep docstrings concise one-liners or short paragraphs consistent with existing modules; use reST-style sections only when they add value. | ||
|
|
||
| ## Imports & Structure | ||
| - Use absolute imports; keep them grouped stdlib → third-party → local, separated by blank lines. | ||
| - Follow existing patterns (`os`, `io`, etc.) for filesystem helpers unless there is a clear benefit to switching to `pathlib.Path`. | ||
| - Keep module-level constants in uppercase with clear names. | ||
|
|
||
| ## Coding Style | ||
| - Format with Black (100-character line length, already configured). | ||
| - Obey the pylint settings in `pyproject.toml`; choose descriptive identifiers within those constraints. | ||
| - Use f-strings, `Enum`/`Literal` where they clarify intent, and explicit encodings when touching files. | ||
| - Favor context managers for I/O and structured error handling (catch specific exceptions only when you can recover). | ||
|
|
||
| ## Testing | ||
| - Place tests under `test/` and write them using pytest idioms (`assert`, fixtures, parametrization). Migrating legacy unittest code to pytest is encouraged when you touch it. | ||
| - Include or update tests for any behavioral change and keep them deterministic (use mocks, temp dirs, or fixtures as needed). | ||
| - Validate locally with `poetry run pytest --cov=firestone_lib --cov-report term-missing`. Keep coverage at, or very close to, 100%—justify any unavoidable exclusions. | ||
| - Run `poetry run pylint firestone_lib test` before submitting; lint failures will block CI. | ||
| - When tests need helper classes or overrides, add minimal docstrings and document any `# pylint: disable=` usage so the reason stays clear. | ||
|
|
||
| ## Error Handling & Logging | ||
| - Raise specific exceptions with actionable messages. | ||
| - Log through `logging.getLogger(__name__)` (existing `_LOGGER` pattern) and keep messages concise but informative. | ||
|
|
||
| ## Security & Robustness | ||
| - Avoid executing untrusted input and double-check file paths or templates before use. | ||
| - Preserve json/yaml loading safeguards already present in the repo (for example, `safe_load`). | ||
| - Access module-private helpers (such as `_jsonloader`) only from tests, and accompany the call with a comment/pylint waiver that explains the intent. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,94 @@ | ||
|  | ||
|
|
||
|
|
||
| # firestone-lib | ||
|
|
||
| This library is primarily used by the firestone project and anyone using the firestone project. | ||
| Firestone-lib is a shared toolkit that powers Firestone services and any downstream automation that needs to load schema resources, wire up CLI utilities, or work with reusable helpers. It packages common building blocks so projects across the ecosystem can stay consistent without duplicating code. | ||
|
|
||
| # building and testing | ||
| ## Firestone Ecosystem | ||
|
|
||
| ``` | ||
| brew install poetry | ||
| This library supports companion projects that build on the same patterns: | ||
|
|
||
| - [`firestone`](https://github.com/firestoned/firestone) turns JSON Schema resources into OpenAPI, AsyncAPI, CLI, and Streamlit artifacts. | ||
| - [`forevd`](https://github.com/firestoned/forevd) uses `firestone-lib`'s CLI and templating helpers to render Apache proxy configs with pluggable auth. | ||
|
|
||
| ## Features | ||
| - Schema helpers that load JSON or YAML, resolve `$ref` links with `jsonref`, and validate against canonical Firestone schemas. | ||
| - CLI utilities built on top of `click`, including rich parameter types, templated input handling, and logging bootstrap helpers. | ||
| - Lightweight string utilities (for example, `split_capitalize`) that keep naming logic consistent across projects. | ||
| - Bundled configuration assets such as default logging definitions. | ||
|
|
||
| ## Project Structure | ||
| - `firestone_lib/cli.py` – click helpers, logging setup, and custom parameter converters. | ||
| - `firestone_lib/resource.py` – schema loading, templating, and validation logic. | ||
| - `firestone_lib/utils.py` – general-purpose helpers shared by Firestone services. | ||
| - `firestone_lib/resources/` – packaged config files (for example, `logging/cli.conf`). | ||
| - `test/` – pytest-based regression and unit tests covering the public surface. | ||
|
|
||
| ## Installation | ||
|
|
||
| ### Prerequisites | ||
| - Python 3.12 or newer (see `pyproject.toml` for the precise compatibility range) | ||
| - [Poetry](https://python-poetry.org/docs/#installation) | ||
|
|
||
| ### From source | ||
| ```bash | ||
| poetry install | ||
| poetry build | ||
| poetry run pytest test | ||
| ``` | ||
|
|
||
| This installs the library and all development dependencies into Poetry's virtual environment. To enter the environment, run `poetry shell` or prefix commands with `poetry run`. | ||
|
|
||
| ### From PyPI (consumer projects) | ||
| ```bash | ||
| pip install firestone-lib | ||
| ``` | ||
|
|
||
| ## Quick Start | ||
| Loading and validating a resource definition: | ||
|
|
||
| ```python | ||
| from firestone_lib import resource | ||
|
|
||
| schema = resource.get_resource_schema("path/to/resource.yaml") | ||
| # Raises jsonschema.ValidationError on failure | ||
| resource.validate({"apiVersion": "v1", "kind": "Example", "spec": {}}) | ||
| ``` | ||
|
|
||
| Setting up CLI logging from a packaged configuration: | ||
|
|
||
| ```python | ||
| from firestone_lib import cli | ||
|
|
||
| cli.init_logging("firestone_lib.resources.logging", "cli.conf") | ||
| ``` | ||
|
|
||
| Normalizing identifiers for UI display: | ||
|
|
||
| ```python | ||
| from firestone_lib import utils | ||
|
|
||
| print(utils.split_capitalize("firestone_resource")) # Firestone Resource | ||
| ``` | ||
|
|
||
| ## Development Workflow | ||
| - Use type hints and reStructuredText docstrings throughout the codebase (see `.github/instructions/python.instructions.md` for the full style guide). | ||
| - Consult `.github/copilot-instructions.md` for architecture notes and expectations when planning larger changes. | ||
| - Format changes with `poetry run black .` (line length 100) only when necessary. | ||
| - Keep logging statements descriptive and consistent with existing `_LOGGER` usage. | ||
|
|
||
| ## Testing | ||
|
|
||
| Run the test suite before submitting changes: | ||
|
|
||
| ```bash | ||
| poetry run pytest | ||
| ``` | ||
|
|
||
| CI mirrors this command through `.github/workflows/pr.yml`. | ||
|
|
||
| ## Contributing | ||
|
|
||
| Issues and pull requests are welcome. When proposing a change, include the context, tests, and any documentation updates that ensure downstream services can adopt the update smoothly. | ||
|
|
||
| ## License | ||
|
|
||
| Licensed under the Apache License 2.0 – see `LICENSE` for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,7 +20,9 @@ | |
|
|
||
| def _jsonloader(uri, **kwargs): | ||
| if uri.startswith("file://"): | ||
| uri = uri[8:] | ||
| uri = uri[7:] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this needed?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| elif uri.startswith("file:"): | ||
| uri = uri[5:] | ||
|
|
||
| with io.open(uri, "r", encoding="utf-8") as fh: | ||
| if uri.endswith(".json"): | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.