Prepare local Telegram analytics setup

.github/workflows/cicd.yml

@@ -13,7 +13,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
       - run: npm ci
       - run: npm run format-check
 
@@ -23,7 +23,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
       - run: npm ci
       - run: npm run build:web
       - uses: actions/upload-artifact@v4
@@ -38,7 +38,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
       - run: npm ci
       - run: npm run build:node
       - uses: actions/upload-artifact@v4
@@ -54,7 +54,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
       - run: npm ci
       - uses: actions/download-artifact@v4
         with:
@@ -70,7 +70,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
       - run: npm ci
       - uses: actions/download-artifact@v4
         with:

.github/workflows/publish.yml

@@ -12,7 +12,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-node@v3
         with:
-          node-version: 16.x
+          node-version: 20.x
           registry-url: 'https://registry.npmjs.org'
       - run: npm ci
       - run: npm run build:web

.gitignore

@@ -6,5 +6,11 @@ report_sample*.data
 stats.json
 report.html
 
+# local private chat exports and generated reports
+IN/*
+!IN/README.md
+OUT/*
+!OUT/README.md
+
 # pack build
 chat-analytics-*.tgz

AGENTS.md

@@ -0,0 +1,55 @@
+# AGENTS.md
+
+## Environment
+
+- Use Node 20 only: `package.json` sets `engines.node` to `>=20 <21`.
+- Install with `npm ci`; do not use `npm install` for routine setup unless intentionally changing dependencies.
+
+## Commands agents usually need
+
+- Typecheck both Node and web projects: `npm run typecheck`.
+- Build Node CLI/library output: `npm run build:node` -> `dist/`.
+- Build web/report assets: `npm run build:web` -> `dist_web/`.
+- Run all tests with coverage: `npm run test`.
+- Run formatting check: `npm run format-check`.
+- Focused parser regression test: `npx jest tests/parse/Parsers.test.ts --runInBand --verbose`.
+- Generate local Telegram report after web+node builds: `node dist/lib/CLI.js -p telegram -i "IN/telegram_export_2026-04-18/result.json" -o OUT/report.html`.
+
+## Build order and output gotchas
+
+- Build `dist_web/` before using the Node CLI to generate reports; `generateReport()` loads `/report.html` from web assets.
+- Webpack has three entries: `app`, `report`, `reportWorker`; production inlines report CSS and the report worker into `report.html`.
+- Keep path aliases aligned across `tsconfig.json`, `webpack.config.js`, and Jest `moduleNameMapper` in `package.json`.
+- `tsconfig.web.json` uses `skipLibCheck: true` to avoid third-party Popper/Tippy declaration failures during web typecheck.
+
+## Architecture map
+
+- Library entrypoints: `pipeline/index.ts` (`generateDatabase`, `generateReport`) and `lib/CLI.ts`.
+- Parser layer: `pipeline/parse/*`; platform parsers live in `pipeline/parse/parsers/*` and emit `P*` interfaces from `pipeline/parse/Types.ts`.
+- Processing layer: `pipeline/process/*`; `DatabaseBuilder` orchestrates parser events, message grouping, NLP, indexing, and final DB construction.
+- Message storage is custom binary serialization in `pipeline/serialization/*`, compressed/encoded by `pipeline/compression/*`.
+- Aggregation blocks live in `pipeline/aggregate/blocks/*` and must be registered in `pipeline/aggregate/Blocks.ts`.
+- Report viewer UI is `report/*`; report block computation runs in `report/WorkerReport.ts`.
+- Report-builder web UI is `app/*`; CLI generation is in `lib/CLI.ts`.
+
+## Tests and coverage
+
+- Tests live only in `tests/`, usually mirroring source structure.
+- Current coverage targets `pipeline/**`, `lib/**`, and `assets/Plausible.ts`; React UI under `app/` and `report/` is not covered.
+- When touching Telegram parsing, keep regression coverage for nested text entities, poll question text entities, and reactions.
+
+## Private data and generated files
+
+- Private chat exports belong in `IN/`; generated reports belong in `OUT/`.
+- `.gitignore` ignores `IN/*` and `OUT/*` except `IN/README.md` and `OUT/README.md`.
+- Never commit real chat exports or generated `OUT/report.html`; report HTML embeds sensitive message data.
+- Do not paste private chat text into docs, tests, commits, or summaries; use synthetic Telegram JSON in tests.
+
+## Workflow rules
+
+- Before edits, branch switches, pulls, or rebases, run `git status --short --branch` and `git fetch --all --prune`.
+- Do not start active work directly on `main`; one task should map to one branch.
+- Prefer git worktrees for parallel or risky work, especially when the existing workspace is dirty.
+- Do not mix unrelated local edits into the same branch, PR, or commit.
+- Stop if staged changes include secrets, auth state, private chat data, generated reports, or unrelated runtime files.
+- Treat `todo.md` as the in-repo operator surface; update it when task status or verification evidence changes.

IN/README.md

@@ -0,0 +1,15 @@
+# IN
+
+Put private chat exports here. This folder is intentionally ignored by git, except for this README.
+
+Recommended Telegram Desktop layout:
+
+```text
+IN/
+  telegram_export_2026-04-18/
+    result.json
+    stickers/
+    video_files/
+```
+
+For Telegram analytics, `result.json` is the required input. Media folders can stay next to it, but the current CLI command only needs the JSON file.

OUT/README.md

@@ -0,0 +1,10 @@
+# OUT
+
+Generated reports go here. This folder is intentionally ignored by git, except for this README.
+
+Example output:
+
+```text
+OUT/
+  report.html
+```

README.md

@@ -37,7 +37,7 @@ You can generate reports from the following platforms:
 |-----------|----------------------------------------------------------------------------------|--------------|------------------|-------------------------------------------------------------------------------------|------------------|------------------------|-------------|-------|
 | Discord   | `json` from [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter) | ✅            | ✅                | ✅                                                                                   | ✅                | ✅ (until link expires) | ✅ (as text) | ✅ |
 | Messenger | `json` from [Facebook DYI export](https://www.facebook.com/dyi)                  | ✅            | ❌                | ✅                                                                                   | ❌                | ❌                      | ✅ (as text) | ❌ |
-| Telegram  | `json` from [Telegram Desktop](https://desktop.telegram.org/)                    | ✅            | ✅                | ✅                                                                                   | ❌ (not provided) | ❌                      | ✅ (as text) | ✅ |
+| Telegram  | `json` from [Telegram Desktop](https://desktop.telegram.org/)                    | ✅            | ✅                | ✅                                                                                   | ✅                | ❌                      | ✅ (as text) | ✅ |
 | WhatsApp  | `txt` or `zip` exported from a phone                                             | ✅            | ❌ (not provided) | ✅<strong>*</strong> (if exported from iOS)<br>🟦 (generic if exported from Android) | ❌ (not provided) | ❌                      | ✅ (as text) | ❌ |
 
 <strong>*</strong> not all languages are supported, check [WhatsApp.ts](/pipeline/parse/parsers/WhatsApp.ts).
@@ -74,6 +74,30 @@ For example:
 npx chat-analytics -p discord -i "exported/*.json" -o report.html
 ```
 
+### Local Telegram analysis workspace
+
+For local analysis, keep private chat exports in `IN/` and generated reports in `OUT/`. Both folders are ignored by git so chat data and reports stay local.
+
+Recommended Telegram Desktop export layout:
+
+```text
+IN/
+  telegram_export_2026-04-18/
+    result.json
+    stickers/
+    video_files/
+OUT/
+```
+
+Build and generate a local Telegram report:
+
+```sh
+npm run build:node
+node dist/lib/CLI.js -p telegram -i "IN/telegram_export_2026-04-18/result.json" -o OUT/report.html
+```
+
+`result.json` is the required Telegram input. Keep media folders beside it only if you want to preserve the original export structure.
+
 ## Docker Compose
 
 You can self-host the app using the official docker image provided at https://hub.docker.com/r/mlomb/chat-analytics. Check out the [Dockerfile](/Dockerfile).
@@ -93,7 +117,8 @@ You can map the web interface port as required by changing the port mapping, int
 
 ## Docs & Development
 
-You can read [docs/README.md](/docs/README.md) for technical details, and [docs/DEV.md](/docs/DEV.md) for development instructions.  
+You can read [docs/README.md](/docs/README.md) for technical details, and [docs/DEV.md](/docs/DEV.md) for development instructions.
+Use Node 20 and `npm ci` for reproducible local installs. `npm run typecheck` validates both Node and web TypeScript builds without emitting files.
 In [docs/TODO.md](/docs/TODO.md) you can find ideas and pending stuff to be implemented.
 
 ## Acknowledgements

docs/DEV.md

@@ -2,7 +2,7 @@
 
 Hi there!
 
-Don't forget to run `npm install` before starting to develop.
+Use Node 20 for development. Run `npm ci` before starting to develop so installs stay reproducible with `package-lock.json`.
 
 ## Developing UI
 
@@ -23,6 +23,8 @@ npm link
 npm run build:node && chat-analytics -p whatsapp -i "export/*"
 ```
 
+Run `npm run typecheck` to validate both Node and web TypeScript projects without writing build output.
+
 ## Testing
 
 Head to [TESTS.md](./TESTS.md) for information about testing!