Skip to content

Always regenerate projects index so variant-flag toggles refresh links#168

Merged
cboos merged 2 commits into
mainfrom
dev/filtered-output-follow-up
May 23, 2026
Merged

Always regenerate projects index so variant-flag toggles refresh links#168
cboos merged 2 commits into
mainfrom
dev/filtered-output-follow-up

Conversation

@cboos
Copy link
Copy Markdown
Collaborator

@cboos cboos commented May 23, 2026

Summary

  • Drops the any_cache_updated gate on projects-index regeneration and the now-unused renderer.is_outdated(index_path) check.
  • Index is now always regenerated from the in-memory project_summaries aggregation (one template pass + one file write).

Why

The previous gate only fired when source JSONL files changed. Toggling a variant flag like --compact / --no-timestamps / --detail between runs produces new per-project filenames (e.g. combined_transcripts.compact.md) but doesn't touch the cache, so the index kept linking to the previous run's filenames. The bug also bit --expand-paths --combined no (per-session bullet links) and the symmetric toggle-back case (going back to the bare variant when the file already exists on disk).

A first attempt tracked output writes via a new any_output_written flag — that fixed the forward toggle but left the backward toggle stale. Always regenerating closes both directions and dispenses with the per-call freshness logic; the saved work on a true no-op run is negligible vs the per-project cache work that already ran.

Test plan

  • test/test_output_paths.py::TestProjectsIndexVariantRefresh — three new regression scenarios:
    • forward toggle (default → --compact)
    • --expand-paths --combined no + --compact
    • backward toggle (--compact → default, file already on disk)
  • test/test_html_regeneration.py::test_projects_index_regeneration_on_jsonl_change — updated to pin the always-regenerate contract and assert the removed "skipping regeneration" log line stays gone.
  • just ci clean (1791 unit + benchmark; ruff / pyright / ty all green).

Summary by CodeRabbit

  • Bug Fixes

    • Projects index is now regenerated on every run so it always reflects current project summaries and variant-specific output links.
  • Tests

    • Added and updated tests to ensure the index is rewritten on no-op runs, that timestamps advance, and that links refresh when output variant options toggle.

Review Change Stack

The projects index was gated on a `any_cache_updated` flag that only
flipped when source JSONL files changed. Toggling a variant flag like
`--compact` / `--no-timestamps` / `--detail` between runs produces new
per-project output filenames (e.g. `combined_transcripts.compact.md`)
but doesn't touch the cache — so the index kept linking to the
previous run's filenames. The same bug bit `--expand-paths --combined
no` (per-session bullet links) and the symmetric toggle-back case.

Switching to unconditional regeneration: the index is built from
`project_summaries` already in memory (one template pass + one file
write over already-aggregated data), so the saved work on a true
no-op run is negligible compared to the per-project cache check that
already ran. Drops the `any_cache_updated` flag, the
`renderer.is_outdated(index_path)` call, and the "Index ... is
current, skipping regeneration" / "Index regenerated" log lines that
are no longer informative.

Regression tests in `test_output_paths.py::TestProjectsIndexVariantRefresh`
cover the three scenarios:
- forward toggle (default → `--compact`),
- `--expand-paths --combined no` + `--compact`,
- backward toggle (`--compact` → default, file already on disk).

Existing `test_projects_index_regeneration_on_jsonl_change` updated
to pin the new always-regenerate contract (mtime advances on no-op
re-runs) and assert the removed log line stays gone.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 23, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 00bbeffc-05f0-44ab-8cab-da8d024fd0b9

📥 Commits

Reviewing files that changed from the base of the PR and between b682db2 and 16ed262.

📒 Files selected for processing (2)
  • test/test_html_regeneration.py
  • test/test_integration_realistic.py
✅ Files skipped from review due to trivial changes (1)
  • test/test_integration_realistic.py

📝 Walkthrough

Walkthrough

Index generation is refactored to always run unconditionally, removing cache-update tracking that previously gated regeneration. Tests confirm the index rewrites on every run and that links refresh when output format variants toggle across consecutive invocations.

Changes

Unconditional Index Regeneration

Layer / File(s) Summary
Index generation refactoring
claude_code_log/converter.py
Removes any_cache_updated tracking variable and changes index generation from conditional (checking staleness, date filters, and cache state) to unconditional. Index rendering now conditionally enables expand_paths_tree only for HTML/Markdown formats and always writes with errors="replace".
Index regeneration test validation
test/test_html_regeneration.py, test/test_integration_realistic.py
Updates test_projects_index_regeneration_on_jsonl_change to assert the index regenerates on every run (no-op reruns included) and that JSONL edits appear in the refreshed output. Switches modification-time checks from second to nanosecond precision for more reliable timing assertions; updates an integration test docstring to the always-regenerate wording.

Variant Refresh Regression Tests

Layer / File(s) Summary
Test setup and helper
test/test_output_paths.py
Imports process_projects_hierarchy and introduces _build_one_project_projects_dir() helper to create minimal on-disk projects structures for variant-refresh test cases.
Variant refresh test cases
test/test_output_paths.py
Adds TestProjectsIndexVariantRefresh class with three test cases verifying that toggling --compact flag, combining --combined no with expand_paths, and reverting to defaults all trigger index regeneration with links pointing to the correct variant-suffixed (or unsuffixed) filenames.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I nibble bytes beneath the moon,
I hop where indexes renew,
Flags flip, links twine and change their name,
Each run I write — no skip, no shame.
Hooray for fresh and tidy view!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: unconditional index regeneration to fix variant-flag toggle issues that previously left stale links.
Docstring Coverage ✅ Passed Docstring coverage is 90.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev/filtered-output-follow-up

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cboos cboos marked this pull request as ready for review May 23, 2026 20:45
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/test_html_regeneration.py (1)

133-202: 💤 Low value

Test correctly validates always-regenerate contract.

The switch to st_mtime_ns for nanosecond precision is more reliable for timing assertions, and the test properly verifies both the no-op always-regen behavior and regeneration after JSONL changes.

Optional enhancement: verify content changes

The test could strengthen the third-run validation by confirming the new JSONL content actually appears in the regenerated index:

 post_change_mtime_ns = index_file.stat().st_mtime_ns
 process_projects_hierarchy(projects_dir)
 assert index_file.stat().st_mtime_ns > post_change_mtime_ns
+# Verify the new content appears in the index
+new_content = index_file.read_text(encoding="utf-8")
+assert "updated content for index regeneration test" in new_content

This would fully validate the docstring's claim that "JSONL edits are picked up."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/test_html_regeneration.py` around lines 133 - 202, The test verifies
always-regenerate and JSONL-change detection but doesn't assert that the new
JSONL line actually appears in the regenerated index; after calling
process_projects_hierarchy(projects_dir) in the third run, read index_file (from
variable index_file) and assert the new_message (or a distinctive substring like
the summary text) is present to prove the content was incorporated; update
test_projects_index_regeneration_on_jsonl_change to open and read
index_file.read_text(encoding="utf-8") after the final
process_projects_hierarchy call and assert the expected JSON snippet or summary
string is in the content.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/test_html_regeneration.py`:
- Around line 133-202: The test verifies always-regenerate and JSONL-change
detection but doesn't assert that the new JSONL line actually appears in the
regenerated index; after calling process_projects_hierarchy(projects_dir) in the
third run, read index_file (from variable index_file) and assert the new_message
(or a distinctive substring like the summary text) is present to prove the
content was incorporated; update
test_projects_index_regeneration_on_jsonl_change to open and read
index_file.read_text(encoding="utf-8") after the final
process_projects_hierarchy call and assert the expected JSON snippet or summary
string is in the content.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e8a9e2d5-67af-4a36-9ce1-257022597d91

📥 Commits

Reviewing files that changed from the base of the PR and between 68e5bfb and b682db2.

📒 Files selected for processing (3)
  • claude_code_log/converter.py
  • test/test_html_regeneration.py
  • test/test_output_paths.py

Two cleanups on top of b682db2:

1. `test/test_html_regeneration.py::test_projects_index_regeneration_on_jsonl_change`:
   CR suggested asserting that the appended JSONL content appears in
   the regenerated index. The projects index surfaces project-level
   metadata (counts, date ranges, session AI titles) — not raw
   message text — so a string match on the appended message body
   isn't a reliable signal. Replaced with a direct before/after
   content comparison: catches the same "byte-equivalent rewrite"
   stale bug an mtime check alone would miss, without depending on
   internal template choices about which JSONL fields surface where.

2. `test/test_integration_realistic.py::test_index_regenerated_when_project_cache_updates`:
   Docstring rewritten — pre-PR-168 the test pinned "index regenerates
   when project cache changes", but with always-regenerate semantics
   the JSONL edit is now incidental cargo. New docstring describes
   what the assertion actually pins today (mtime advances on every
   call) and notes the JSONL edit's historical purpose.
@cboos cboos merged commit f4b2ea9 into main May 23, 2026
11 checks passed
@cboos cboos deleted the dev/filtered-output-follow-up branch May 23, 2026 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant