Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ All under `/v1/`:
| `GET /search/explore?q=&platform=&page=` | User-triggered deep GitHub search, paginated, ingests into index. Also reads `X-GitHub-Token`. Cold-path latency is 10–30s — clients must use a 30s timeout. |
| `GET /categories/{trending\|new-releases\|most-popular}/{android\|windows\|macos\|linux}` | Pre-ranked repo lists. Sort order is `search_score DESC NULLS LAST, rank ASC` — static `rank` is only the tie-breaker once behavioral signals exist. |
| `GET /topics/{privacy\|media\|productivity\|networking\|dev-tools}/{platform}` | Topic-bucketed repos. Same dynamic ordering as categories. |
| `GET /repo/{owner}/{name}` | Single repo detail. Curated DB hit on the fast path; on miss, lazy-fetches metadata from GitHub via `GitHubResourceClient` and reads optional `X-GitHub-Token`. |
| `GET /repo/{owner}/{name}` | Single repo detail. Curated DB hit on the fast path; on miss, lazy-fetches metadata from GitHub via `GitHubResourceClient` and reads optional `X-GitHub-Token`. Response includes `openIssuesCount` (mirrors GitHub's `open_issues_count`, which counts open issues + open PRs together — same value the GitHub website's Issues tab shows). |
| `POST /repo/{owner}/{name}/refresh` | User-triggered refetch of a repo's metadata + latest release. Re-fetches from GitHub via `RepoRefreshCoordinator`, upserts Postgres + pushes Meili, returns the same shape as the GET. Per-repo cooldown 30s + global hourly budget 1000 prevent pool-token torch from spam clicks. Reads `X-GitHub-Token`. Response is `Cache-Control: no-store`; the GET path's CDN cache catches up via its own TTL (~5 min on `s-maxage=300`). |
| `GET /releases/{owner}/{name}?page=&per_page=` | Proxied list of GitHub releases. Reads optional `X-GitHub-Token`. Cached server-side for 1h. |
| `GET /readme/{owner}/{name}` | Proxied README JSON (base64-encoded content + metadata, GitHub's shape). Reads optional `X-GitHub-Token`. Cached 24h. |
Expand Down
118 changes: 118 additions & 0 deletions docs/client/open-issues-count.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Client Integration — `openIssuesCount`

**Audience:** client coding agent (KMP / Compose Multiplatform).
**Goal:** surface a repo's open-issues count on the details screen using the new `openIssuesCount` field on `RepoResponse`. No new endpoint, no pagination — the value rides on the existing GET response.

---

## 1. What changed

`RepoResponse` (the DTO returned by `GET /v1/repo/{owner}/{name}`, every category/topic list endpoint, and the new `POST /v1/repo/{owner}/{name}/refresh`) now carries a new field:

```kotlin
val openIssuesCount: Int = 0
```

Default is `0` so older clients that don't know the field still parse cleanly. Same back-compat story as every other additive field on `RepoResponse`.

---

## 2. What the value means

GitHub's `open_issues_count`. Identical to the number you see on the GitHub website's **Issues** tab badge. Two important properties:

1. **Includes open pull requests.** GitHub treats PRs as a kind of issue, so this count is `open_issues + open_prs` combined. There is no first-class field for "issues only — no PRs" on the GitHub API; getting that requires a separate paginated call. We do not paginate, so we accept GitHub's combined number.
2. **Is a snapshot, not realtime.** Updated when the row is upserted (passthrough ingest, refresh button, daily fetcher run, hourly RepoRefreshWorker). Worst case stale: 24h on a repo nobody has refreshed manually since the last fetcher run.

If the client UI labels it as "open issues" specifically, that's an acceptable simplification — the GitHub website itself uses the same number under "Issues."

---

## 3. Where the field appears in responses

Every `RepoResponse`-shaped payload, including:

| Endpoint | Behaviour |
|----------|-----------|
| `GET /v1/repo/{owner}/{name}` | DB-hit and lazy-fetch paths both fill it. DB-hit value may be up to 24h stale; lazy-fetch is live. |
| `POST /v1/repo/{owner}/{name}/refresh` | Fresh from GitHub, up-to-the-second. |
| `GET /v1/categories/.../...` | DB value (same staleness as the rest of the curated row). |
| `GET /v1/topics/.../...` | DB value. |
| `GET /v1/search?q=...` | Meilisearch index value (synced from DB by `meili_sync.py`). |

If you don't see the field, check that the row was last upserted before this change rolled out. Hit the refresh endpoint and the value will populate.

---

## 4. Display recommendations

- **Where:** details screen, near star count + fork count. Same row, same icon-with-number style.
- **Icon:** GitHub's open-circle "issue" glyph. Whatever your existing iconography uses.
- **Label:** "Issues" or "Open issues." Don't say "Bug count" — PRs are mixed in.
- **Tap behaviour:** open `https://github.com/{owner}/{name}/issues` in a browser tab. We do not currently expose a list endpoint; the user goes to GitHub for details.
- **Zero handling:** show "0" rather than hiding. Zero open issues is a meaningful signal (well-maintained or unused).
- **Large numbers:** format with `1.2k` / `12k` style above 1000. Current biggest repos (e.g. `facebook/react`, `microsoft/vscode`) show >5000. Don't overflow the layout.

---

## 5. Pseudo-code (Compose Multiplatform)

```kotlin
@Composable
fun RepoStatsRow(repo: RepoResponse) {
Row(...) {
StatChip(icon = Icons.Star, value = repo.stargazersCount.compactFormat(), label = "Stars")
StatChip(icon = Icons.Fork, value = repo.forksCount.compactFormat(), label = "Forks")
StatChip(
icon = Icons.Issue,
value = repo.openIssuesCount.compactFormat(),
label = "Issues",
onTap = { openInBrowser("https://github.com/${repo.fullName}/issues") },
)
}
}

private fun Int.compactFormat(): String = when {
this >= 1_000_000 -> "%.1fM".format(this / 1_000_000.0)
this >= 1_000 -> "%.1fk".format(this / 1_000.0)
else -> toString()
}
```

---

## 6. Backwards compatibility notes

- **Older app builds against the new server:** field is unknown, ignored by `kotlinx.serialization`'s `ignoreUnknownKeys = true`. Nothing breaks.
- **Newer app builds against the old server (during rollout):** field is missing, `Int = 0` default kicks in. UI shows "0 issues" briefly until backend deploy completes. Acceptable — rollout window is ~5 minutes.
- **Client deserializer:** must have `ignoreUnknownKeys = true` (you almost certainly already do; every other passthrough field has been added the same way).

---

## 7. What you do NOT need to do

- **No separate fetch.** Don't call `/v1/repo/{o}/{n}/issues` or anything like it — there is no such endpoint, and you don't need one for the count.
- **No pagination.** Count comes free with the repo lookup.
- **No client-side caching of the count.** The repo response is already cached client-side per your existing `/repo` cache strategy. The count rides along.
- **No special refresh UX.** If the user wants the count fresher than the cached repo response, they tap the refresh button (`POST /v1/repo/{o}/{n}/refresh`) — same as every other repo field.

---

## 8. Acceptance criteria

- [ ] `RepoResponse` deserializes with the new `openIssuesCount` field on every existing call site (details screen, search results, category/topic lists, repo refresh response).
- [ ] Details screen displays the count next to stars/forks.
- [ ] Tapping the issue chip opens `https://github.com/{owner}/{name}/issues` externally.
- [ ] No crash, no parse error, when an old server response (without the field) is consumed during rollout — `Int = 0` fallback works.
- [ ] Compact-format helper handles 0, small (`42`), thousands (`1.2k`), millions (`1.5M`).

---

## 9. Authoritative reference

Backend definitions:
- `model/RepoResponse.kt` — the DTO shape
- `db/migration/V14__open_issues_count.sql` — the column
- `routes/RepoRoutes.kt`, `routes/SearchRoutes.kt`, `db/RepoRepository.kt`, `ingest/GitHubSearchClient.kt` — all the mappers wired together

If the client and server disagree, the backend wins; file an issue on the backend repo.
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ object DatabaseFactory {
// delisted above so a fresh install never creates the table
// only for V13 to drop it seconds later.
"V13__drop_telemetry_events.sql",
"V14__open_issues_count.sql",
)
for (migration in migrations) {
val rawSql = this::class.java.classLoader
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ data class MeiliRepoHit(
val html_url: String = "",
val stars: Int = 0,
val forks: Int = 0,
val open_issues: Int = 0,
val language: String? = null,
val latest_release_date: String? = null,
val latest_release_tag: String? = null,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ class RepoRepository {
htmlUrl = this[Repos.htmlUrl],
stargazersCount = this[Repos.stars],
forksCount = this[Repos.forks],
openIssuesCount = this[Repos.openIssues],
language = this[Repos.language],
topics = this[Repos.topics],
releasesUrl = "${this[Repos.htmlUrl]}/releases",
Expand Down
1 change: 1 addition & 0 deletions src/main/kotlin/zed/rainxch/githubstore/db/Tables.kt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ object Repos : Table("repos") {
val htmlUrl = text("html_url")
val stars = integer("stars").default(0)
val forks = integer("forks").default(0)
val openIssues = integer("open_issues").default(0)
val language = text("language").nullable()
val topics = array<String>("topics", TextColumnType())
val latestReleaseDate = timestampWithTimeZone("latest_release_date").nullable()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -520,6 +520,7 @@ class GitHubSearchClient(
it[htmlUrl] = repo.htmlUrl
it[stars] = repo.stargazersCount
it[forks] = repo.forksCount
it[openIssues] = repo.openIssuesCount
it[language] = repo.language
it[topics] = repo.topics
it[latestReleaseDate] = releaseDate
Expand Down Expand Up @@ -555,6 +556,7 @@ class GitHubSearchClient(
html_url = r.repo.htmlUrl,
stars = r.repo.stargazersCount,
forks = r.repo.forksCount,
open_issues = r.repo.openIssuesCount,
language = r.repo.language,
topics = r.repo.topics,
latest_release_date = r.release.publishedAt,
Expand Down Expand Up @@ -601,6 +603,7 @@ class GitHubSearchClient(
htmlUrl = repo.htmlUrl,
stargazersCount = repo.stargazersCount,
forksCount = repo.forksCount,
openIssuesCount = repo.openIssuesCount,
language = repo.language,
topics = repo.topics,
releasesUrl = "${repo.htmlUrl}/releases",
Expand Down Expand Up @@ -654,6 +657,9 @@ data class GitHubRepo(
@SerialName("html_url") val htmlUrl: String,
@SerialName("stargazers_count") val stargazersCount: Int = 0,
@SerialName("forks_count") val forksCount: Int = 0,
// Includes open PRs (GitHub treats PRs as issues). Same number GitHub
// website's Issues tab shows.
@SerialName("open_issues_count") val openIssuesCount: Int = 0,
val language: String? = null,
val topics: List<String> = emptyList(),
val archived: Boolean = false,
Expand Down
4 changes: 4 additions & 0 deletions src/main/kotlin/zed/rainxch/githubstore/model/RepoResponse.kt
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ data class RepoResponse(
val htmlUrl: String,
val stargazersCount: Int,
val forksCount: Int,
// Mirrors GitHub's open_issues_count -- includes both open issues AND
// open PRs (GitHub treats PRs as a kind of issue). Same value as the
// GitHub website's Issues tab badge.
val openIssuesCount: Int = 0,
val language: String?,
val topics: List<String>,
val releasesUrl: String?,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ internal fun GitHubRepo.toMetadataOnlyResponse(): RepoResponse = RepoResponse(
htmlUrl = htmlUrl,
stargazersCount = stargazersCount,
forksCount = forksCount,
openIssuesCount = openIssuesCount,
language = language,
topics = topics,
releasesUrl = "$htmlUrl/releases",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ private fun zed.rainxch.githubstore.db.MeiliRepoHit.toRepoResponse() = RepoRespo
htmlUrl = html_url,
stargazersCount = stars,
forksCount = forks,
openIssuesCount = open_issues,
language = language,
topics = topics,
releasesUrl = "$html_url/releases",
Expand Down
19 changes: 19 additions & 0 deletions src/main/resources/db/migration/V14__open_issues_count.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
-- V14: track open_issues_count on the repos table so the details screen
-- can surface "X open issues" without an extra GitHub passthrough call.
--
-- GitHub's open_issues_count includes both issues and pull requests (GitHub
-- treats PRs as a kind of issue internally). The client surfaces it as the
-- same number the GitHub website shows on the Issues tab; no separation
-- attempted server-side.
--
-- Default 0 for existing rows. Backend writes the real value on:
-- * search passthrough ingest (`GitHubSearchClient.ingestToPostgres`)
-- * POST /v1/repo/{owner}/{name}/refresh
-- * RepoRefreshWorker's hourly cycle
-- * the Python fetcher's daily run (once the fetcher repo wires the field
-- into db_writer.py + meili_sync.py)
--
-- Idempotent: ADD COLUMN IF NOT EXISTS handles re-runs.

ALTER TABLE repos
ADD COLUMN IF NOT EXISTS open_issues INTEGER NOT NULL DEFAULT 0;
Loading