CB's ideas scratchpad

[cboos]

This "issue" is used for sharing my backlog of ideas for future claude-code-log features.
A bit like @daaain's TODO list in the README.md (there is some synergy!).
It's more a scratchpad really, feel free to comment. If something needs more discussion, I can open a dedicated issue.

Filtering

Older proposal:

• --depth option, numerical or symbolic:
  • 0 / "session": only the session summaries (should work on a folder; may also work on a single .jsonl if we also try to get the summary from the other sessions, as the summary message is not in the session .jsonl itself);
  • 1 / "user": stop at the user prompts; → Feature request: have claude to collect all stated requirements into a formal requirements.md file #26
  • 2 / "assistant": stop at the assistant answers; "assistant+": also include "thinking" → Detail levels and compact rendering of conversations #96 --detail minimal (no thinking)
  • 3 / "tools": stop at the tools level (no sidechains); "tools-"/"shallow": only the information gathering ones (Task, WebSearch); → --detail low
  • 4 / "subassistant": stop at the subassistant answers in the sidechains; → --detail low (to verify)
  • 5 / "full": all the way, i.e., like we have now → --detail full

Older proposal:

• --filter-out, --filter-in with fine grained control (e.g., --filter-out thinking, --filter-out tools + --filter-in edit)
  • the --depth option above would translate to filters:
    • --depth session = --filter-out user,command,assistant,thinking,system,tools,sub-assistant,sub-thinking,sub-tools
    • --depth user = --filter-out assistant,thinking,system,tools,sub-assistant,sub-thinking,sub-tools
    • --depth assistant = --filter-out thinking,system,tools,sub-assistant,sub-thinking,sub-tools
    • --depth assistant+ = --filter-out system,tools,sub-assistant,sub-thinking,sub-tools
    • --depth tools = --filter-out sub-assistant,sub-thinking,sub-tools
    • --depth shallow = --filter-out thinking,sub-assistant,sub-thinking,sub-tools + --filter-in task,websearch
    • --depth subassistant = --filter-out sub-thinking,sub-tools
    • --depth subassistant+ = --filter-out sub-tools

In light of #96, I wonder if we shouldn't simply have these two options to modulate what's hard-coded by default in the "low" detail mode...

Two most useful modes:

• "full" (what we have now) to fully understand what was going on by seeing everything that happened → --detail full

• "shallow" to have an overview of what happened at the information processing/gathering level, without the redundancy between thinking and assistant, and the details of what the tools did → --detail low

• --skip to start rendering at a given offset in the .jsonl, useful for deltas (generated output filename would contain the range: '...-start-end.fmt'); --skip could even take the previously generated file as input;

Rendering

HTML and Markdown

Structure

• pagination support: break a session in chunks delimited by n User messages

  • we can use that mode in Markdown previews in the TUI, so that it remains usable for long sessions
    • partially there, but problematic (see Markdown content jumbled up in TUI preview #84)

• -o should be respected when processing the projects directory as well; it's not that a good idea to create our logs below ~/.claude itself! Rather use $XDG_DOCUMENTS_DIR or ~/Documents, + /claude-code-log; if only for Bug #1972762 “Firefox and Chromium snaps preventing apps and dev...” : Bugs : snapd...

• -o is currently only half-way respected when processing one project directory:

  • only the combined transcript will take the specified name (fails when -o directory is given)
  • the individual sessions are still created below ~/.claude/projects/<projectdir>; instead they should be placed below the targeted directory (or siblings to -o file)

• combined transcript mode can be... too much (my current one for claude-code-log is now 146MB...); we already have --no-individual-sessions, so we should also have --no-combined-transcript (mutually exclusive; unless we want to use that combination for only doing a cache sync) => combined transcript is paginated now, so this is less urgent (well, with SQLite cache #59), but it would be nice to have only individual sessions (and those could be paginated as well, see above)

  • we could also have the "positive" ones, mutually exclusive: --individual-sessions, --combined-transcript

Content

• in the session summary, a session entry should include the start and end dates of the messages it contains, "(from - to)" suffix

  • but see DAG of Conversations #85 for the bigger picture

• --user=plain|markdown|md - how to interpret user input: possible values:

  • "plain": i.e., what we have now and the safe default;
  • "md" or "markdown": render as Markdown, a.k.a., "Trust me, I know what I'm doing" mode; -u as shortcut for --user=markdown;
  • "auto": auto-detect if it's a nicely formatted Markdown text or a copy/pasted error stack... delux version would auto-wrap such things in a raw code block (future) → Render user content as Markdown with raw fallback toggle #119

Impl note: ideas for the "auto" mode welcome!

• Specialized rendering for WebSearch tool; Add WebSearch tool renderer with structured parsing #83 / Add WebFetch tool renderer #87

• Differentiate a "Reject" error from a real error - there's some user prompt there that should "stand out":

• Better support for hooks:
  Even without the alignment glitch => Fix fold-bar colors and System Hook alignment #89, we can do better: pair them by uuid/parentUuid, and dedicated renderer.

  Excerpt from the corresponding input .jsonl

  {
    "type": "user",
    "message": {
      "role": "user",
      "content": "Stop hook feedback:\nYou've got mail! Use /clmail:read 10 to read message #10 and proceed (reply if requested)."
    },
    "isMeta": true,
    "uuid": "e9edc0d4-81b9-4d5f-9ad5-be56e588479e", // same
  },
  {
    "parentUuid": "e9edc0d4-81b9-4d5f-9ad5-be56e588479e", // same
    "type": "system",
    "subtype": "stop_hook_summary",
    "hookCount": 1,
    "hookInfos": [
      {
        "command": "callback"
      }
    ],
    "hookErrors": [
      "You've got mail! Use /clmail:read 10 to read message #10 and proceed (reply if requested)."
    ],
    "preventedContinuation": false,
    "stopReason": "",
    "hasOutput": true,
    "level": "suggestion",
  }

• Show transitions for cwd (now in ... directory) and gitBranch (now on ... branch)

HTML only

• Pygmentize old/new before diff? Easy for the red/green background, but what about intra-line highlights?

Markdown only

• User command, System info, Bash output in Markdown: support ANSI to Markdown (on the model of ANSI to HTML, only simpler, just bold and italics)

• EnterPlanMode: render as Markdown instead of code block ... or skip altogether

Others

• --format json - nothing like the "original" .jsonl, but a serialization of the high-level TemplateMessage objects export conversations to json #36

Styling

• Fix glitches as they're discovered (still quite a few)

  • Inverted colors for bottom bar: Fix fold-bar colors and System Hook alignment #89

• --compact option, for compact style:

  • for HTML:
    • For tools, something similar to
    • Less lines visible in <details><summary> (or none?);
    • Avoid the dates? (only as tooltips?)
    • Expand/collapse on the side? (my original idea, but never implemented)
  • for Markdown:
    • For tools, something similar to Add text, markdown, and chat output formats (#43) #44's "chat" mode (no headers)
    • Every tool Read/Edit/Write/Bash output in <details>
      • --no-details to suppress those

• too large spacing between "▼▼" in Chrome: fix with letter-spacing, though not too much as then this would break FF.

The TUI

claude-code-log --tui

Project list

• The projects are listed in some order... not sure I can figure out what order it is without looking at the code...; would be great if we had "last mod" and "created" date columns, and that we could sort according to name, or these dates
• Selecting a project with 's' - makes sense, but I keep hitting "[Enter]" ;-) Would be nice to have that as an alias!
• [ESC] alias to 'q'

Project view / Session list

• Better navigation: 'q' / '[ESC]' should go to project selection if we're coming from project selection, not quit immediately ('Q' could do that, but that saves one repeated keystroke...) => '[ESC]' alias to 'p', goes back to project list, 'q' still exits immediately, that's OK...

Markdown preview

• 't' doesn't do anything (supposed to toggle the outline view) => fixed
• '[Enter]' alias to 'v' (configurable?)

File system

~The FS caches were located in ~/.claude/projects/*/cache, and the new cache is located in' /.claude/projects/*/cache'.db is currently in ~/.claude/projects/.
If we keep that location, we should at least name it ~/.claude/projects/claude-code-log-cache.db => now ~/.claude/projects/claude-code-log-cache.db which is OK.

But I think we should have our own location. I'm no fan of ~/.claude-code-log, but as I wrote about, we should really put our generated output (.md and .html) in ~/Documents/ClaudeCodeLog (or ~/Documents/ClaudeCodeLog), we could also put the cache.db there.

Known issues

• ⛔ Pre-process "thinking" Markdown, typically missing required new lines, like in this example:
  ...but this is hopeless, the newlines are missing from the input:

  {"type":"thinking","thinking":"The user is right - the worker approach is over-engineering. The TUI is still blocking during generation anyway, and they don't want users to do other things during generation. A simpler approach is just to show a notification before starting the generation and then show the result when done.Let me revert to a simpler approach:\n1. Show \"Rendering Markdown...\" notification\n2. Do the synchronous generation3. Show success/error notification when doneThis is simpler and matches what the user wants.", ...

  => 2. Do the synchronous generation3. Show success/error (no space between "generation" and "3." in the input, therefore not much we could do about it.

[amjadtalib]

This is a great scratchpad. FWIW agree with you on separating FS cache from .claude project; also, not sure if this solves the need to mod the jsonl sessions but I find that a little annoying if you are sorting your FS by mod time.

Re the filters and depth, I think another layer that warrants distinction is file read vs write. I generally want to filter out all file-reads (since they can be re-read or attached separately if used to audit/feed into corrective session) and some tool calls but I like to see (and ideally export) the thought chain, even down to the subagents/assistant level. That said, it would be a lot easier to run through the TUI and blanket select certain types of templates with manual select/deselect as needed for export - just a thought.

Re rendering/content, I feel like interpreting user inputs is difficult. I personally use markdown double backticks around concepts and generally paste content within code fences (not defining lang); but the content within the fences itself is usually a mix between error stacks and markdown - with the latter potentially containing its own code fences that break on rendering but are generally interpreted correctly by the LLM. Fixing the reject as error bug would be nice!

Btw, personally really hoping markdown export and/or TUI copy to markdown support is close to being available in some form. Lmk what you think about being able to bulk select/de-select messages in the TUI for export.