Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Firestone-lib Copilot Instructions

## Project Scope
- Firestone-lib bundles shared capabilities used across the broader Firestone ecosystem: CLI utilities, schema tooling, and lightweight helpers.
- Consumers expect stable interfaces; keep changes backward-compatible unless a coordinated release states otherwise.

## Ecosystem Awareness
- `firestone` consumes these helpers for spec generation; validate CLI or resource-facing changes against its workflows (see `firestone/__main__.py` and `firestone/spec/`).
- `forevd` depends on `firestone_lib.cli` for logging and config parsing—document any behavior shifts so the proxy can bump in lockstep.
- If a change requires synchronized updates, land `firestone-lib` first and surface follow-up PR references in the release notes or change description.

## Architecture Overview
- `firestone_lib/cli.py`: click-based helpers (logging bootstrap, custom param types, async wrappers). Extend here when exposing new CLI behavior.
- `firestone_lib/resource.py`: schema ingestion/validation built around `jsonref` and `jsonschema`; reuse its loaders to keep `$ref` support intact.
- `firestone_lib/utils.py`: string-centric helpers used across Firestone services.
- `firestone_lib/resources/`: packaged configuration assets (for example, logging config in `logging/cli.conf`).
- Tests mirror the package layout under `test/`.

## Build & Automation
- Poetry drives installs, builds, and test runs (`poetry install`, `poetry build`, `poetry run pytest`).
- Continuous integration is handled via `.github/workflows/pr.yml`; expect linting and tests to run on every pull request.
- Local pre-flight: run `poetry run pylint firestone_lib test` and `poetry run pytest --cov=firestone_lib --cov-report term-missing` before opening a PR. CI enforces both commands, and we aim to keep coverage at 100%.

## Collaboration Notes
- Before adding new modules or resources, confirm they fit the existing structure; prefer evolving current packages.
- Keep logging messages meaningful and consistent with existing `_LOGGER` usage.
- When working on CLI or schema changes, ensure templating, environment-variable substitution, and file loading continue to behave as they do today.

## Coding Conventions
- For language-level expectations (Python 3.9+ compatibility, mandatory type hints, pytest usage, formatting requirements), follow `.github/instructions/python.instructions.md`.
- When adding tests, include docstrings or inline comments for helper classes/overrides so pylint passes without silencing warnings globally. Use targeted `# pylint: disable=` comments with a short justification when you must access module-private APIs like `_jsonloader`.
- Resource loaders rely on the current `file://` / `file:` path normalization; keep that behavior intact and update tests accordingly if you touch it.
45 changes: 45 additions & 0 deletions .github/instructions/python.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
applyTo: '**/*.py'
---

# Python Coding Guidelines

These rules combine firestone-lib conventions with widely accepted Python practices. Where industry defaults differ, follow the repo-specific rule noted here.

They intentionally mirror the upstream Firestone project so moving changes between repositories stays predictable; if you spot a divergence, flag it so we can reconcile both sides.

## Runtime & Dependencies
- Target Python **3.12+** as defined in `pyproject.toml`; keep code compatible with the supported range.
- Manage dependencies through Poetry and keep `pyproject.toml` and `poetry.lock` in sync when adding or bumping packages.

## Type Hints & Docstrings
- Annotate every function and method (including inner helpers) with precise parameter and return types.
- Prefer modern `typing` / `collections.abc` generics (`Iterator`, `Mapping`, `Sequence`, etc.) in annotations.
- Keep docstrings concise one-liners or short paragraphs consistent with existing modules; use reST-style sections only when they add value.

## Imports & Structure
- Use absolute imports; keep them grouped stdlib → third-party → local, separated by blank lines.
- Follow existing patterns (`os`, `io`, etc.) for filesystem helpers unless there is a clear benefit to switching to `pathlib.Path`.
- Keep module-level constants in uppercase with clear names.

## Coding Style
- Format with Black (100-character line length, already configured).
- Obey the pylint settings in `pyproject.toml`; choose descriptive identifiers within those constraints.
- Use f-strings, `Enum`/`Literal` where they clarify intent, and explicit encodings when touching files.
- Favor context managers for I/O and structured error handling (catch specific exceptions only when you can recover).

## Testing
- Place tests under `test/` and write them using pytest idioms (`assert`, fixtures, parametrization). Migrating legacy unittest code to pytest is encouraged when you touch it.
- Include or update tests for any behavioral change and keep them deterministic (use mocks, temp dirs, or fixtures as needed).
- Validate locally with `poetry run pytest --cov=firestone_lib --cov-report term-missing`. Keep coverage at, or very close to, 100%—justify any unavoidable exclusions.
- Run `poetry run pylint firestone_lib test` before submitting; lint failures will block CI.
- When tests need helper classes or overrides, add minimal docstrings and document any `# pylint: disable=` usage so the reason stays clear.

## Error Handling & Logging
- Raise specific exceptions with actionable messages.
- Log through `logging.getLogger(__name__)` (existing `_LOGGER` pattern) and keep messages concise but informative.

## Security & Robustness
- Avoid executing untrusted input and double-check file paths or templates before use.
- Preserve json/yaml loading safeguards already present in the repo (for example, `safe_load`).
- Access module-private helpers (such as `_jsonloader`) only from tests, and accompany the call with a comment/pylint waiver that explains the intent.
18 changes: 15 additions & 3 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,26 @@ on:
pull_request:
branches: [ "main" ]

permissions:
contents: read
pull-requests: write

jobs:
build:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11"]
python-version:
Comment thread
bradpenney marked this conversation as resolved.
- "3.12"
- "3.13"
- "3.14"

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
Expand All @@ -38,6 +45,11 @@ jobs:
run: |
poetry run pytest --cache-clear --cov=firestone_lib test/ > pytest-coverage.txt
- name: Comment coverage
if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}
uses: coroo/pytest-coverage-commentator@v1.0.2
with:
token: ${{ secrets.GITHUB_TOKEN }}
pytest-coverage: pytest-coverage.txt
- name: Coverage Badge
if: ${{ github.event_name == 'push' }}
uses: tj-actions/coverage-badge-py@v2
93 changes: 86 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,94 @@
![PR Build](https://github.com/ebourgeois/firestone-lib/actions/workflows/pr.yml/badge.svg)


# firestone-lib

This library is primarily used by the firestone project and anyone using the firestone project.
Firestone-lib is a shared toolkit that powers Firestone services and any downstream automation that needs to load schema resources, wire up CLI utilities, or work with reusable helpers. It packages common building blocks so projects across the ecosystem can stay consistent without duplicating code.

# building and testing
## Firestone Ecosystem

```
brew install poetry
This library supports companion projects that build on the same patterns:

- [`firestone`](https://github.com/firestoned/firestone) turns JSON Schema resources into OpenAPI, AsyncAPI, CLI, and Streamlit artifacts.
- [`forevd`](https://github.com/firestoned/forevd) uses `firestone-lib`'s CLI and templating helpers to render Apache proxy configs with pluggable auth.

## Features
- Schema helpers that load JSON or YAML, resolve `$ref` links with `jsonref`, and validate against canonical Firestone schemas.
- CLI utilities built on top of `click`, including rich parameter types, templated input handling, and logging bootstrap helpers.
- Lightweight string utilities (for example, `split_capitalize`) that keep naming logic consistent across projects.
- Bundled configuration assets such as default logging definitions.

## Project Structure
- `firestone_lib/cli.py` – click helpers, logging setup, and custom parameter converters.
- `firestone_lib/resource.py` – schema loading, templating, and validation logic.
- `firestone_lib/utils.py` – general-purpose helpers shared by Firestone services.
- `firestone_lib/resources/` – packaged config files (for example, `logging/cli.conf`).
- `test/` – pytest-based regression and unit tests covering the public surface.

## Installation

### Prerequisites
- Python 3.12 or newer (see `pyproject.toml` for the precise compatibility range)
- [Poetry](https://python-poetry.org/docs/#installation)

### From source
```bash
poetry install
poetry build
poetry run pytest test
```

This installs the library and all development dependencies into Poetry's virtual environment. To enter the environment, run `poetry shell` or prefix commands with `poetry run`.

### From PyPI (consumer projects)
```bash
pip install firestone-lib
```

## Quick Start
Loading and validating a resource definition:

```python
from firestone_lib import resource

schema = resource.get_resource_schema("path/to/resource.yaml")
# Raises jsonschema.ValidationError on failure
resource.validate({"apiVersion": "v1", "kind": "Example", "spec": {}})
```

Setting up CLI logging from a packaged configuration:

```python
from firestone_lib import cli

cli.init_logging("firestone_lib.resources.logging", "cli.conf")
```

Normalizing identifiers for UI display:

```python
from firestone_lib import utils

print(utils.split_capitalize("firestone_resource")) # Firestone Resource
```

## Development Workflow
- Use type hints and reStructuredText docstrings throughout the codebase (see `.github/instructions/python.instructions.md` for the full style guide).
- Consult `.github/copilot-instructions.md` for architecture notes and expectations when planning larger changes.
- Format changes with `poetry run black .` (line length 100) only when necessary.
- Keep logging statements descriptive and consistent with existing `_LOGGER` usage.

## Testing

Run the test suite before submitting changes:

```bash
poetry run pytest
```

CI mirrors this command through `.github/workflows/pr.yml`.

## Contributing

Issues and pull requests are welcome. When proposing a change, include the context, tests, and any documentation updates that ensure downstream services can adopt the update smoothly.

## License

Licensed under the Apache License 2.0 – see `LICENSE` for details.
4 changes: 3 additions & 1 deletion firestone_lib/resource.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@

def _jsonloader(uri, **kwargs):
if uri.startswith("file://"):
uri = uri[8:]
uri = uri[7:]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • _jsonloader now strips file:// URIs with uri[7:] and adds a fallback for plain file: . The old slice uri[8:] lopped off the first character of absolute paths like file:///tmp/schema.yaml, so jsonref tried to open tmp/schema.yaml relative to CWD and crashed whenever a schema contained $refs to other local files or when tests exercised those paths.
  • Guarding the plain file: case lets jsonref hand us either file:///... or file:/... and still resolves correctly, which is how coverage tests trigger the loader.
  • With this correction, nested schema references load reliably whether they arrive with file: or file:// URIs, eliminating the FileNotFound errors seen in test runs.

elif uri.startswith("file:"):
uri = uri[5:]

with io.open(uri, "r", encoding="utf-8") as fh:
if uri.endswith(".json"):
Expand Down
Loading