Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions DEVGUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ tagbot/
│ ├── changelog.py # Release notes generation (Jinja2)
│ ├── git.py # Git command wrapper
│ ├── gitlab.py # GitLab API wrapper (optional)
│ ├── graphql.py # GraphQL client for batched API operations
│ └── repo.py # Core logic: version discovery, release creation
├── local/
│ └── __main__.py # CLI entrypoint
Expand Down Expand Up @@ -99,6 +100,12 @@ tagbot/
- Extracts custom notes from registry PR (`<!-- BEGIN RELEASE NOTES -->`)
- Renders Jinja2 template

**`GraphQLClient` (graphql.py)** - Batched API operations:
- `fetch_tags_and_releases()` - Single query for tags + releases
- `fetch_commits_metadata()` - Batch commit metadata lookup
- `search_issues_and_pulls()` - Enhanced issue/PR search
- Provides 2x+ performance improvement over sequential REST calls

### Special Features

**Subpackages**: For monorepos with `subdir` input:
Expand All @@ -123,10 +130,13 @@ Performance: 600+ versions in ~4 seconds via aggressive caching.

| Cache | Purpose | Built By |
|-------|---------|----------|
| `__existing_tags_cache` | Skip existing tags | Single API call to `get_git_matching_refs("tags/")` |
| `__existing_tags_cache` | Skip existing tags | GraphQL query or `get_git_matching_refs("tags/")` |
| `__releases_cache` | Cached releases | Fetched alongside tags via GraphQL |
| `__tree_to_commit_cache` | Tree SHA → commit | Single `git log --all --format=%H %T` |
| `__registry_prs_cache` | Fallback commit lookup | Fetch up to 300 merged PRs |
| `__commit_datetimes` | "Latest" determination | Lazily built |
| `__commit_datetimes` | "Latest" determination | Single `git log --all --format=%H %aI` |

**GraphQL Optimization**: When available, `_build_tags_cache()` uses a single GraphQL query to fetch both tags and releases simultaneously, reducing API calls by 50% compared to separate REST calls.

**Pattern for new caches**:
```python
Expand Down
15 changes: 11 additions & 4 deletions IMPROVEMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,23 @@ The `Changelog._issues_and_pulls()` method now uses the GitHub search API to fil
---

### 1.4 Use GraphQL API for Batched Operations
**Status**: Not implemented
**Status**: ✅ Implemented (v1.24.4)
**Impact**: High
**Effort**: High

Many operations make multiple REST API calls that could be consolidated using GitHub's GraphQL API. A single GraphQL query could fetch:
Many operations that previously made multiple REST API calls are now consolidated using GitHub's GraphQL API. A single GraphQL query can fetch:
- All tags
- All releases
- Multiple commits' metadata
- Issues/PRs in a date range

**Implementation**: Created `graphql.py` module with `GraphQLClient` class that provides:
- `fetch_tags_and_releases()` - Single query to get tags + releases (replaces 2 separate REST calls)
- `fetch_commits_metadata()` - Batch commit metadata lookup
- `search_issues_and_pulls()` - Enhanced search with GraphQL

The implementation uses GraphQL as the primary method with graceful fallback to REST API on errors.

**Example**: Fetching tags and releases in one query:
```graphql
query {
Expand All @@ -72,7 +79,7 @@ query {
}
```

**Tradeoff**: Would require adding `gql` dependency and significant refactoring.
**Benefit**: Reduces API calls and improves performance. For repositories with many tags/releases, this can cut API calls by 50% or more.

---

Expand Down Expand Up @@ -279,7 +286,7 @@ Current Dockerfile uses `python:3.12-slim`. Could reduce further with:
| 1.1 | Git log primary lookup | High | Low | ✅ Done |
| 1.2 | Changelog API optimization | High | Medium | ✅ Done |
| 1.3 | Batch commit datetime lookups | Medium-High | Low | ✅ Done |
| 1.4 | GraphQL API | High | High | Not started |
| 1.4 | GraphQL API | High | High | ✅ Done |
| 2.1 | Split repo.py | Medium | Medium | Not started |
| 2.2 | Use tomllib | Low | Low | Not started |
| 2.3 | Structured logging | Medium | Medium | Not started |
Expand Down
479 changes: 209 additions & 270 deletions poetry.lock

Large diffs are not rendered by default.

3 changes: 0 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,6 @@ python-gitlab = { version = "^7.1.0", optional = true }
[tool.poetry.extras]
gitlab = ["python-gitlab"]

[tool.poetry.requires-plugins]
poetry-plugin-export = ">=1.8"

[tool.poetry.group.dev.dependencies]
black = "^25.12"
boto3 = "^1.42.21"
Expand Down
2 changes: 2 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
Flask==3.1.2
Jinja2>=3,<4
PyGithub>=2.7.0,<3
gql>=3.5.0,<4
requests-toolbelt>=1.0.0,<2
pylev>=1.3.0,<2
MarkupSafe==3.0.3
itsdangerous==2.2.0
Expand Down
Loading