Addition of diagnostic collectors and bug fixes

DATA.md

@@ -11,7 +11,7 @@ Complete reference of all data collected by radar.
 - **Per-Database**: Schema and object collectors (per database)
 - **pg_statviz (optional)**: Time-series statistics collectors (per database)
 
-**Output**: All data collected in a single ZIP file named `radar-{hostname}-{timestamp}.zip`
+**Output**: All data collected in a single ZIP file named `radar-{hostname}-{timestamp}.zip`. The archive root contains a `radar.out` entry identifying the radar binary that produced it (`version` from the build-time `-X main.version` stamp, `commit` from Go's embedded VCS info).
 
 ---
 
@@ -46,7 +46,8 @@ These collectors only run on Linux systems.
 | `system/dmesg_t.out` | `dmesg -T` | Kernel ring buffer with timestamps |
 | `system/free.out` | `free -h` | Memory usage summary |
 | `system/fstab.out` | `/etc/fstab` | Filesystem table |
-| `system/hostname.out` | `hostname -f` | Fully qualified hostname |
+| `system/hostname.out` | `hostname` | Kernel hostname (gethostname) |
+| `system/hostname_fqdn.out` | `hostname -f` | Fully qualified hostname (resolver canonical name) |
 | `system/hypervisor.out` | `systemd-detect-virt` | Hypervisor detection |
 | `system/ifconfig.out` | `ifconfig -a` | Network interfaces (legacy) |
 | `system/interfaces.out` | `ip -o address` | Network interfaces (one-line) |
@@ -181,7 +182,8 @@ These collectors only run on macOS systems.
 |------|--------|-------------|
 | `system/diskutil_info_all.out` | `diskutil info` (all disks) | Detailed disk information |
 | `system/diskutil_list.out` | `diskutil list` | Disk layout |
-| `system/hostname.out` | `hostname` | Hostname |
+| `system/hostname.out` | `hostname` | Kernel hostname (gethostname) |
+| `system/hostname_fqdn.out` | `hostname -f` | Fully qualified hostname (resolver canonical name) |
 | `system/hypervisor.out` | `sysctl kern.hv_vmm_present` | Hypervisor detection |
 | `system/ifconfig.out` | `ifconfig -a` | Network interfaces |
 | `system/iostat.out` | `iostat -c 5 -w 1` | I/O statistics (5 samples) |
@@ -222,7 +224,7 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | File | Source | Description |
 |------|--------|-------------|
 | `postgresql/archiver.tsv` | `pg_stat_archiver` | WAL archiver statistics |
-| `postgresql/available_extensions.tsv` | `pg_available_extensions` | Available extensions |
+| `postgresql/available_extensions.tsv` | `pg_available_extension_versions` | Available extension versions with installed flag, trust and requires metadata |
 | `postgresql/bgwriter.tsv` | `pg_stat_bgwriter` | Background writer statistics |
 | `postgresql/blocking_locks.tsv` | Complex query | Blocking/blocked lock pairs |
 | `postgresql/checkpointer.tsv` | `pg_stat_checkpointer` | Checkpointer statistics |
@@ -250,7 +252,7 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | `postgresql/replication_slots.tsv` | `pg_replication_slots` | Replication slots |
 | `postgresql/roles.tsv` | `pg_roles` | Database roles |
 | `postgresql/running_activity.tsv` | `pg_stat_activity` | Active connections and queries |
-| `postgresql/running_activity_maxage.tsv` | Complex query | Oldest queries/transactions |
+| `postgresql/running_activity_maxage.tsv` | Complex query | Oldest queries/transactions and longest lock wait |
 | `postgresql/running_locks.tsv` | `pg_locks WHERE granted` | Held locks |
 | `postgresql/shmem_allocations.tsv` | `pg_shmem_allocations` | Shared memory breakdown |
 | `postgresql/stat_io.tsv` | `pg_stat_io` | I/O statistics (PG16+) |
@@ -260,7 +262,9 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | `postgresql/stat_progress_copy.tsv` | `pg_stat_progress_copy` | COPY progress (PG14+) |
 | `postgresql/stat_progress_create_index.tsv` | `pg_stat_progress_create_index` | CREATE INDEX progress (PG12+) |
 | `postgresql/stat_progress_vacuum.tsv` | `pg_stat_progress_vacuum` | VACUUM progress (PG9.6+) |
+| `postgresql/stat_replication_slots.tsv` | `pg_stat_replication_slots` | Replication slot stats incl. spill counters (PG14+) |
 | `postgresql/stat_slru.tsv` | `pg_stat_slru` | SLRU cache statistics |
+| `postgresql/stat_ssl.tsv` | `pg_stat_ssl` | Per-backend SSL/TLS state |
 | `postgresql/stat_statements_calls.tsv` | `pg_stat_statements` | Top 100 queries by call count |
 | `postgresql/stat_statements_max_time.tsv` | `pg_stat_statements` | Top 100 queries by max execution time |
 | `postgresql/stat_statements_total_time.tsv` | `pg_stat_statements` | Top 100 queries by total execution time |
@@ -281,21 +285,24 @@ Collected for each accessible database. Files stored in `databases/{dbname}/`.
 
 | File | Source | Description |
 |------|--------|-------------|
+| `bloat.tsv` | `pg_stats` heuristic | Table bloat estimate (heuristic) |
 | `extensions.tsv` | `pg_extension` | Installed extensions |
 | `funcs.tsv` | `pg_proc WHERE prokind='f'` | Functions |
-| `indexes.tsv` | `pg_indexes` | Indexes |
+| `indexes.tsv` | `pg_index` + `pg_stat_all_indexes` | Top 1000 indexes by size with semantic key, validity, scan counters and size |
 | `languages.tsv` | `pg_language` | Procedural languages |
 | `operators.tsv` | `pg_operator` | Operators |
 | `partitioned_tables.tsv` | `pg_partitioned_table` | Partitioned tables (PG10+) |
 | `partitions.tsv` | `pg_inherits` | Partition relationships |
+| `pgstattuple.tsv` | `pgstattuple_approx()` | Authoritative bloat (when `pgstattuple` extension is installed). Disabled by default — run with `-include pgstattuple` because it touches every user table's heap pages. |
 | `procs.tsv` | `pg_proc WHERE prokind='p'` | Procedures (PG11+) |
 | `publication_tables.tsv` | `pg_publication_tables` | Tables in publications |
 | `publications.tsv` | `pg_publication` | Logical replication publications |
 | `schemas.tsv` | `pg_namespace` | Schemas |
+| `sequences.tsv` | `pg_sequences` | Sequences with last/min/max values (PG10+) |
 | `stat_database.tsv` | `pg_stat_database` | Per-database statistics |
 | `statistics.tsv` | `pg_statistic_ext` | Extended statistics (PG10+) |
 | `subscription_tables.tsv` | `pg_subscription_rel` | Subscription relation states |
-| `tables.tsv` | `pg_tables` | Tables |
+| `tables.tsv` | `pg_class` + `pg_stat_all_tables` | Top 1000 tables by size with persistence, options, heap and table sizes, toast mapping, dead-tup counters, vacuum/analyze timestamps and per-table vacuum/analyze ages in seconds |
 | `triggers.tsv` | `pg_trigger` | Triggers |
 | `types.tsv` | `pg_type` | Data types |
 

README.md

@@ -119,8 +119,12 @@ Options:
     	database name (default "postgres")
   -data-dir string
     	PostgreSQL data directory
+  -exclude string
+    	comma-separated task names to skip (e.g. -exclude bloat,stat_ssl)
   -h string
     	database host (default "localhost")
+  -include string
+    	comma-separated default-disabled task names to enable (e.g. pgstattuple, disabled by default)
   -p int
     	database port (default 5432)
   -skip-postgres
@@ -187,19 +191,20 @@ For a complete reference of all collected data, see [docs/data.md](docs/data.md)
 
 - **Configuration & files**: `pg_db_role_setting`, `pg_file_settings`, `pg_hba.conf`, `pg_hba_file_rules`, `pg_ident.conf`, `pg_settings`, `pg_tablespace`, `postgresql.auto.conf`, `postgresql.conf`, `recovery.conf`, `recovery.done`
 - **Activity & monitoring**: `pg_locks`, `pg_postmaster_start_time()`, `pg_prepared_xacts`, `pg_shmem_allocations`, `pg_stat_activity`
-- **Statistics views**: `pg_stat_archiver`, `pg_stat_bgwriter`, `pg_stat_checkpointer` (PG17+), `pg_stat_database_conflicts`, `pg_stat_io` (PG16+), `pg_stat_slru`, `pg_stat_statements` (if installed), `pg_stat_wal` (PG14+), `pg_stat_wal_receiver`
-- **Replication & WAL**: `pg_current_wal_lsn()`, `pg_replication_origin_status`, `pg_replication_slots`, `pg_stat_replication`, `pg_subscription`
+- **Statistics views**: `pg_stat_archiver`, `pg_stat_bgwriter`, `pg_stat_checkpointer` (PG17+), `pg_stat_database_conflicts`, `pg_stat_io` (PG16+), `pg_stat_slru`, `pg_stat_ssl`, `pg_stat_statements` (if installed), `pg_stat_wal` (PG14+), `pg_stat_wal_receiver`
+- **Replication & WAL**: `pg_current_wal_lsn()`, `pg_replication_origin_status`, `pg_replication_slots`, `pg_stat_replication`, `pg_stat_replication_slots` (PG14+, spill counters), `pg_subscription`
 - **Progress tracking**: `pg_stat_progress_analyze`, `pg_stat_progress_basebackup`, `pg_stat_progress_cluster`, `pg_stat_progress_copy`, `pg_stat_progress_create_index`, `pg_stat_progress_vacuum`
-- **Catalog**: `pg_available_extensions`, `pg_database`, `pg_database_size()`, `pg_roles`, `pg_tablespace_size()`, `version()`
+- **Catalog**: `pg_available_extension_versions`, `pg_database` (incl. `datfrozenxid`/`datminmxid` wraparound headroom and `datconnlimit`), `pg_database_size()`, `pg_roles`, `pg_tablespace_size()`, `version()`
 
 **Per-Database**
 
-- **Schema objects**: `pg_indexes`, `pg_namespace`, `pg_operator`, `pg_tables`, `pg_type`
+- **Schema objects**: `pg_class` + `pg_stat_all_tables` (tables incl. dead-tup counters, vacuum/analyze timestamps, `reloptions`, `reltoastrelid`, `relpersistence`), `pg_index` + `pg_stat_all_indexes` (indexes incl. validity, scan counters, size), `pg_namespace`, `pg_operator`, `pg_sequences`, `pg_type`
 - **Functions & procedures**: `pg_proc`
 - **Triggers & partitioning**: `pg_inherits`, `pg_partitioned_table`, `pg_trigger`
 - **Logical replication**: `pg_publication`, `pg_publication_tables`, `pg_subscription_rel`
-- **Extensions**: `pg_extension`, `pg_language`, `pg_statistic_ext`
+- **Extensions**: `pg_extension`, `pg_language`, `pg_statistic_ext`, `pgstattuple` (if installed — authoritative bloat estimate via `pgstattuple_approx()`)
 - **Statistics**: `pg_stat_database` (conflicts, deadlocks, temp files, stats reset)
+- **Bloat estimate**: heuristic from `pg_stats` (always emitted)
 
 **[pg_statviz](https://github.com/vyruss/pg_statviz) Extension** (if present)
 

docs/data.md

@@ -11,7 +11,7 @@ Complete reference of all data collected by radar.
 - **Per-Database**: Schema and object collectors (per database)
 - **pg_statviz (optional)**: Time-series statistics collectors (per database)
 
-**Output**: All data collected in a single ZIP file named `radar-{hostname}-{timestamp}.zip`
+**Output**: All data collected in a single ZIP file named `radar-{hostname}-{timestamp}.zip`. The archive root contains a `radar.out` entry identifying the radar binary that produced it (`version` from the build-time `-X main.version` stamp, `commit` from Go's embedded VCS info).
 
 ---
 
@@ -46,7 +46,8 @@ These collectors only run on Linux systems.
 | `system/dmesg_t.out` | `dmesg -T` | Kernel ring buffer with timestamps |
 | `system/free.out` | `free -h` | Memory usage summary |
 | `system/fstab.out` | `/etc/fstab` | Filesystem table |
-| `system/hostname.out` | `hostname -f` | Fully qualified hostname |
+| `system/hostname.out` | `hostname` | Kernel hostname (gethostname) |
+| `system/hostname_fqdn.out` | `hostname -f` | Fully qualified hostname (resolver canonical name) |
 | `system/hypervisor.out` | `systemd-detect-virt` | Hypervisor detection |
 | `system/ifconfig.out` | `ifconfig -a` | Network interfaces (legacy) |
 | `system/interfaces.out` | `ip -o address` | Network interfaces (one-line) |
@@ -181,7 +182,8 @@ These collectors only run on macOS systems.
 |------|--------|-------------|
 | `system/diskutil_info_all.out` | `diskutil info` (all disks) | Detailed disk information |
 | `system/diskutil_list.out` | `diskutil list` | Disk layout |
-| `system/hostname.out` | `hostname` | Hostname |
+| `system/hostname.out` | `hostname` | Kernel hostname (gethostname) |
+| `system/hostname_fqdn.out` | `hostname -f` | Fully qualified hostname (resolver canonical name) |
 | `system/hypervisor.out` | `sysctl kern.hv_vmm_present` | Hypervisor detection |
 | `system/ifconfig.out` | `ifconfig -a` | Network interfaces |
 | `system/iostat.out` | `iostat -c 5 -w 1` | I/O statistics (5 samples) |
@@ -222,7 +224,7 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | File | Source | Description |
 |------|--------|-------------|
 | `postgresql/archiver.tsv` | `pg_stat_archiver` | WAL archiver statistics |
-| `postgresql/available_extensions.tsv` | `pg_available_extensions` | Available extensions |
+| `postgresql/available_extensions.tsv` | `pg_available_extension_versions` | Available extension versions with installed flag, trust and requires metadata |
 | `postgresql/bgwriter.tsv` | `pg_stat_bgwriter` | Background writer statistics |
 | `postgresql/blocking_locks.tsv` | Complex query | Blocking/blocked lock pairs |
 | `postgresql/checkpointer.tsv` | `pg_stat_checkpointer` | Checkpointer statistics |
@@ -250,7 +252,7 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | `postgresql/replication_slots.tsv` | `pg_replication_slots` | Replication slots |
 | `postgresql/roles.tsv` | `pg_roles` | Database roles |
 | `postgresql/running_activity.tsv` | `pg_stat_activity` | Active connections and queries |
-| `postgresql/running_activity_maxage.tsv` | Complex query | Oldest queries/transactions |
+| `postgresql/running_activity_maxage.tsv` | Complex query | Oldest queries/transactions and longest lock wait |
 | `postgresql/running_locks.tsv` | `pg_locks WHERE granted` | Held locks |
 | `postgresql/shmem_allocations.tsv` | `pg_shmem_allocations` | Shared memory breakdown |
 | `postgresql/stat_io.tsv` | `pg_stat_io` | I/O statistics (PG16+) |
@@ -260,7 +262,9 @@ Instance-level PostgreSQL collectors. Files stored in `postgresql/`.
 | `postgresql/stat_progress_copy.tsv` | `pg_stat_progress_copy` | COPY progress (PG14+) |
 | `postgresql/stat_progress_create_index.tsv` | `pg_stat_progress_create_index` | CREATE INDEX progress (PG12+) |
 | `postgresql/stat_progress_vacuum.tsv` | `pg_stat_progress_vacuum` | VACUUM progress (PG9.6+) |
+| `postgresql/stat_replication_slots.tsv` | `pg_stat_replication_slots` | Replication slot stats incl. spill counters (PG14+) |
 | `postgresql/stat_slru.tsv` | `pg_stat_slru` | SLRU cache statistics |
+| `postgresql/stat_ssl.tsv` | `pg_stat_ssl` | Per-backend SSL/TLS state |
 | `postgresql/stat_statements_calls.tsv` | `pg_stat_statements` | Top 100 queries by call count |
 | `postgresql/stat_statements_max_time.tsv` | `pg_stat_statements` | Top 100 queries by max execution time |
 | `postgresql/stat_statements_total_time.tsv` | `pg_stat_statements` | Top 100 queries by total execution time |
@@ -281,21 +285,24 @@ Collected for each accessible database. Files stored in `databases/{dbname}/`.
 
 | File | Source | Description |
 |------|--------|-------------|
+| `bloat.tsv` | `pg_stats` heuristic | Table bloat estimate (heuristic) |
 | `extensions.tsv` | `pg_extension` | Installed extensions |
 | `funcs.tsv` | `pg_proc WHERE prokind='f'` | Functions |
-| `indexes.tsv` | `pg_indexes` | Indexes |
+| `indexes.tsv` | `pg_index` + `pg_stat_all_indexes` | Top 1000 indexes by size with semantic key, validity, scan counters and size |
 | `languages.tsv` | `pg_language` | Procedural languages |
 | `operators.tsv` | `pg_operator` | Operators |
 | `partitioned_tables.tsv` | `pg_partitioned_table` | Partitioned tables (PG10+) |
 | `partitions.tsv` | `pg_inherits` | Partition relationships |
+| `pgstattuple.tsv` | `pgstattuple_approx()` | Authoritative bloat (when `pgstattuple` extension is installed). Disabled by default — run with `-include pgstattuple` because it touches every user table's heap pages. |
 | `procs.tsv` | `pg_proc WHERE prokind='p'` | Procedures (PG11+) |
 | `publication_tables.tsv` | `pg_publication_tables` | Tables in publications |
 | `publications.tsv` | `pg_publication` | Logical replication publications |
 | `schemas.tsv` | `pg_namespace` | Schemas |
+| `sequences.tsv` | `pg_sequences` | Sequences with last/min/max values (PG10+) |
 | `stat_database.tsv` | `pg_stat_database` | Per-database statistics |
 | `statistics.tsv` | `pg_statistic_ext` | Extended statistics (PG10+) |
 | `subscription_tables.tsv` | `pg_subscription_rel` | Subscription relation states |
-| `tables.tsv` | `pg_tables` | Tables |
+| `tables.tsv` | `pg_class` + `pg_stat_all_tables` | Top 1000 tables by size with persistence, options, heap and table sizes, toast mapping, dead-tup counters, vacuum/analyze timestamps and per-table vacuum/analyze ages in seconds |
 | `triggers.tsv` | `pg_trigger` | Triggers |
 | `types.tsv` | `pg_type` | Data types |
 

docs/index.md

@@ -96,8 +96,12 @@ Options:
     	database name (default "postgres")
   -data-dir string
     	PostgreSQL data directory
+  -exclude string
+    	comma-separated task names to skip (e.g. -exclude bloat,stat_ssl)
   -h string
     	database host (default "localhost")
+  -include string
+    	comma-separated default-disabled task names to enable (e.g. pgstattuple, disabled by default)
   -p int
     	database port (default 5432)
   -skip-postgres
@@ -164,19 +168,20 @@ For a complete reference of all collected data, see [data.md](data.md).
 
 - **Configuration & files**: `pg_db_role_setting`, `pg_file_settings`, `pg_hba.conf`, `pg_hba_file_rules`, `pg_ident.conf`, `pg_settings`, `pg_tablespace`, `postgresql.auto.conf`, `postgresql.conf`, `recovery.conf`, `recovery.done`
 - **Activity & monitoring**: `pg_locks`, `pg_postmaster_start_time()`, `pg_prepared_xacts`, `pg_shmem_allocations`, `pg_stat_activity`
-- **Statistics views**: `pg_stat_archiver`, `pg_stat_bgwriter`, `pg_stat_checkpointer` (PG17+), `pg_stat_database_conflicts`, `pg_stat_io` (PG16+), `pg_stat_slru`, `pg_stat_statements` (if installed), `pg_stat_wal` (PG14+), `pg_stat_wal_receiver`
-- **Replication & WAL**: `pg_current_wal_lsn()`, `pg_replication_origin_status`, `pg_replication_slots`, `pg_stat_replication`, `pg_subscription`
+- **Statistics views**: `pg_stat_archiver`, `pg_stat_bgwriter`, `pg_stat_checkpointer` (PG17+), `pg_stat_database_conflicts`, `pg_stat_io` (PG16+), `pg_stat_slru`, `pg_stat_ssl`, `pg_stat_statements` (if installed), `pg_stat_wal` (PG14+), `pg_stat_wal_receiver`
+- **Replication & WAL**: `pg_current_wal_lsn()`, `pg_replication_origin_status`, `pg_replication_slots`, `pg_stat_replication`, `pg_stat_replication_slots` (PG14+, spill counters), `pg_subscription`
 - **Progress tracking**: `pg_stat_progress_analyze`, `pg_stat_progress_basebackup`, `pg_stat_progress_cluster`, `pg_stat_progress_copy`, `pg_stat_progress_create_index`, `pg_stat_progress_vacuum`
-- **Catalog**: `pg_available_extensions`, `pg_database`, `pg_database_size()`, `pg_roles`, `pg_tablespace_size()`, `version()`
+- **Catalog**: `pg_available_extension_versions`, `pg_database` (incl. `datfrozenxid`/`datminmxid` wraparound headroom and `datconnlimit`), `pg_database_size()`, `pg_roles`, `pg_tablespace_size()`, `version()`
 
 **Per-Database**
 
-- **Schema objects**: `pg_indexes`, `pg_namespace`, `pg_operator`, `pg_tables`, `pg_type`
+- **Schema objects**: `pg_class` + `pg_stat_all_tables` (tables incl. dead-tup counters, vacuum/analyze timestamps, `reloptions`, `reltoastrelid`, `relpersistence`), `pg_index` + `pg_stat_all_indexes` (indexes incl. validity, scan counters, size), `pg_namespace`, `pg_operator`, `pg_sequences`, `pg_type`
 - **Functions & procedures**: `pg_proc`
 - **Triggers & partitioning**: `pg_inherits`, `pg_partitioned_table`, `pg_trigger`
 - **Logical replication**: `pg_publication`, `pg_publication_tables`, `pg_subscription_rel`
-- **Extensions**: `pg_extension`, `pg_language`, `pg_statistic_ext`
+- **Extensions**: `pg_extension`, `pg_language`, `pg_statistic_ext`, `pgstattuple` (if installed — authoritative bloat estimate via `pgstattuple_approx()`)
 - **Statistics**: `pg_stat_database` (conflicts, deadlocks, temp files, stats reset)
+- **Bloat estimate**: heuristic from `pg_stats` (always emitted)
 
 **[pg_statviz](https://github.com/vyruss/pg_statviz) Extension** (if present)