Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,7 @@
* [PostgreSQL Requirements](database/postgresql-requirements.md)
* [Database schema](database/database-schema.md)
* [PostgreSQL Extensions](database/postgresql-extensions.md)
* [Database maintenance RPCs](database/database-maintenance-rpcs.md)

## Modules

Expand Down
Binary file added assets/database/running-queries.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/database/schema-explorer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/database/search-params-stats.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
224 changes: 224 additions & 0 deletions docs/database/database-maintenance-rpcs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
---
description: Inspect tables, observe active queries, and run maintenance operations (VACUUM, ANALYZE, REINDEX, TRUNCATE) against the Aidbox Postgres database via the aidbox.pg/* RPCs.
---

# Database maintenance RPCs

Available since Aidbox 2605. The `aidbox.pg/*` RPC family exposes a thin layer over Postgres' introspection (`pg_stat_*`, `pg_indexes`) and maintenance DDL (`VACUUM`, `ANALYZE`, `REINDEX`, `TRUNCATE`). The same RPCs power the [Database tab in Aidbox UI](../overview/aidbox-ui/README.md#database) — reach for them when scripting backups, building dashboards, or running ad-hoc maintenance.

{% hint style="success" %}
For interactive use, open the **Database** page in Aidbox UI. It calls the RPCs below and renders the results as a sortable, paginated table per section.
{% endhint %}

## Listing tables: `aidbox.pg/tables`

Returns one row per table with size, row-count estimate, index/toast share, and recency of vacuum/analyze.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/tables
params:
all-schemas: true # see all user schemas; omit for public-only
q: pat # ILIKE '<q>%' against table name
limit: 100 # default 100
```

| Parameter | Behavior |
|---|---|
| `q` | `ILIKE '<q>%'` filter on the table name. Optional. |
| `limit` | Max rows. Default `100`. |
| `schema` | Restrict to one schema. Optional. |
| `schemas` | Array of schemas. Optional. |
| `all-schemas` | `true` lists tables across every user schema. Without `schema`/`schemas`/`all-schemas`, only `public` is returned. `pg_catalog`, `information_schema`, and `pgagent` are always excluded. |

Each row carries `table_schema`, the table name, an estimated row count from `pg_class.reltuples`, total / index / toast sizes in both pretty (`text`) and bytes (`bigint`) forms, plus minutes since the last manual or autonomous vacuum/analyze.

```yaml
result:
- table_schema: public
table_name: patient
num_rows: 4231
total: 28 MB
total_size: 29360128
index: 6256 kB
index_size: 6406144
index_part: 21
toast: 8192 bytes
toast_size: 8192
toast_part: 0
options: null
last_autovacuum: 12 # minutes ago; null if never
last_vacuum: null
last_analyze: null
last_autoanalyze: 12
```

## Inspecting a table: `aidbox.pg/get-table`

One-shot detail view: the same row `aidbox.pg/tables` would return, plus a per-index breakdown from `pg_indexes` + `pg_stat_all_indexes` and a single sample row.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/get-table
params:
schema: public # optional; defaults to "public"
table: patient
```

```yaml
result:
table:
table_schema: public
table_name: patient
num_rows: 4231
# ... same shape as aidbox.pg/tables
indexes:
- index_name: patient_pkey
index_size: 152 kB
unique: Y
index_type: btree # pg_am.amname (btree/hash/gin/gist/spgist/brin)
index_def: >- # full CREATE INDEX statement from pg_get_indexdef
CREATE UNIQUE INDEX patient_pkey ON public.patient USING btree (id)
number_of_scans: 18221
tuples_read: 18221
tuples_fetched: 18221
row: # first row of the table; full resource JSON
id: pat-1
resource: { ... }
offset: 0
```

## Maintenance operations

All four maintenance RPCs accept the same shape: `{:table "<name>" :schema "<schema>"}`. `schema` is optional; identifiers are validated against `[A-Za-z0-9_]` and quoted before being spliced into the DDL.

### `aidbox.pg/vacuum-table`

Runs `VACUUM` on the table. Pass `analyze: true` to run `VACUUM ANALYZE`. Reclaims dead-tuple space and (optionally) refreshes planner statistics. Concurrent with reads and writes.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/vacuum-table
params:
schema: public
table: patient
analyze: true # optional — runs VACUUM ANALYZE
```

### `aidbox.pg/analyze-table`

Runs `ANALYZE` on the table to refresh planner statistics. Use after bulk loads or whenever query plans look stale.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/analyze-table
params:
schema: public
table: patient
```

### `aidbox.pg/reindex-table`

Runs `REINDEX TABLE` to rebuild every index on the table. Locks the table for writes until completion — prefer running it during a maintenance window.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/reindex-table
params:
schema: public
table: patient
```

### `aidbox.pg/truncate-table`

Runs `TRUNCATE TABLE` — **permanently deletes every row in the table**. There is no `WHERE` clause and the operation is not undoable. The Aidbox UI deliberately does not surface this RPC — call it only from scripts where you can wrap a confirmation step around it.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/truncate-table
params:
schema: public
table: patient
```

{% hint style="danger" %}
`TRUNCATE` removes all rows in one shot without going through the FHIR layer, so the `_history` table is left untouched and no `Provenance` / `AuditEvent` is recorded. For audited deletes — including patient-compartment cleanup with history — see [Delete data](../tutorials/crud-search-tutorials/delete-data.md) (`DELETE /fhir/<rt>/<id>` plus the [`$purge`](../api/bulk-api/purge.md) operation for bulk + history cleanup).
{% endhint %}

## Observing running queries

Backs the **Running Queries** subpage. Reads `pg_stat_activity` and lets you cancel or terminate individual backends.

### `aidbox.pg/active-queries`

Returns one row per active backend (excluding the caller's own), sorted by `query_start` so the longest-running shows first.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/active-queries
params: {}
```

```yaml
result:
- pid: 2391
usename: postgres
application_name: aidbox
state: active
query_start: 2026-05-12T09:14:21.117Z
duration: 32 # seconds since query_start
wait_event_type: Lock
wait_event: relation
query: SELECT * FROM patient WHERE …
```

### `aidbox.pg/cancel-query` and `aidbox.pg/terminate-query`

`cancel-query` asks Postgres to abort the currently running statement (`pg_cancel_backend`); the connection stays open. `terminate-query` kills the whole connection (`pg_terminate_backend`) — clients see a "terminating connection due to administrator command" error. Prefer cancel over terminate when possible.

```http
POST /rpc
Content-Type: application/json
```

```yaml
method: aidbox.pg/cancel-query # or aidbox.pg/terminate-query
params:
pid: 2391
```

Both return `{result: {pid: <pid>, result: <pg-message>}}`.

## See also

* [Aidbox UI — Database](../overview/aidbox-ui/README.md#database) — the page that ties these RPCs together.
* [Search Parameters Usage Statistics](../deployment-and-maintenance/indexes/search-parameter-usage-stats.md) — `aidbox.index/get-search-param-stats` and related RPCs.
* [SQL endpoints](../api/rest-api/other/sql-endpoints.md) — the `$psql` endpoint for arbitrary SQL.
Original file line number Diff line number Diff line change
Expand Up @@ -59,22 +59,29 @@ params:
flush-first: true
```

Parameter reference:

| Parameter | Behavior |
|---|---|
| `resource-type` | Single base. Optional. |
| `resource-types` | Array — for multi-base SearchParameters. Optional. |
| `search-param` | Limit to shapes containing this SP under any modifier. Optional. |
| `by` | `shape` (default) — one row per `(resource_type, search_params)`. |
| | `param` — one row per `(resource_type, single SP)`, modifiers rolled up. |
| `order-by` | `calls` (default). |
| | `mean-time-ms`. |
| | `total-time-ms`. |
| | `last-used`. |
| `limit` | Max rows. Default 100. |
| `offset` | Pagination offset. Default 0. |
| `flush-first` | Force a synchronous drain of the in-memory buffer before reading. |
Parameter reference.

| Parameter | Behavior |
|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `resource-type` | Exact match on a single base. Optional. |
| `resource-types` | Array — for multi-base SearchParameters. Optional. |
| `search-param` | Exact-match filter. With `by: shape`, matches shapes containing this SP under any modifier; with `by: param`, matches the unnested SP. Optional. |
| `resource-type-like` | Case-insensitive substring filter on `resource_type` (`ILIKE '%q%'`). Used by the UI search box; can be combined with the exact `resource-type` filter. Optional. |
| `search-param-like` | Case-insensitive substring filter on the SP key (`ILIKE '%q%'`). Optional. |
| `by` | `shape` (default) — one row per `(resource_type, search_params)` |
| | `param` — one row per `(resource_type, single SP)`, modifiers rolled up. |
| `order-by` | `calls` (default) |
| | `mean-time-ms` |
| | `total-time-ms` |
| | `min-time-ms` |
| | `max-time-ms` |
| | `last-used` |
| | `resource-type` |
| | `search-param` (the SP key in `by: param`; the `search_params` array in `by: shape`). |
| `order-dir` | `desc` (default) or `asc`. Applies to the `order-by` column; secondary tiebreakers (`resource_type`, then SP) stay ascending. |
| `limit` | Max rows. Default 100. |
| `offset` | Pagination offset. Default 0. |
| `flush-first` | Force a synchronous drain of the in-memory buffer before reading. |

With `by: shape` (the default), one row per `(resource_type, search_params)`:

Expand Down Expand Up @@ -103,6 +110,27 @@ result:
has_index: true
```

## Counting matching rows: `aidbox.index/count-search-param-stats`

Returns `{total: N}` — the row count `get-search-param-stats` would return under the same filters, ignoring `limit`/`offset`/`order-by`. Use it to drive a paginated UI; the Database tab calls it alongside the row RPC on every filter change.

```yaml
POST /rpc

method: aidbox.index/count-search-param-stats
params:
by: param
resource-type-like: patient
search-param-like: name
```

Accepts the same scope params as `get-search-param-stats`: `resource-type`, `resource-types`, `search-param`, `resource-type-like`, `search-param-like`, `by`, `flush-first`.

```yaml
result:
total: 42
```

## Resetting the stats: `aidbox.index/reset-search-param-stats`

Deletes rows from `aidbox_stat.search_param_stats` and drops matching entries from the in-memory buffer. Use it after running synthetic load you don't want to count, or to clear a stale baseline before a fresh measurement window.
Expand Down
45 changes: 27 additions & 18 deletions docs/overview/aidbox-ui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,13 @@ Some resource types have specialized views:
* **AccessPolicy** — includes a built-in dev tool for testing policies. Send a request directly from the editor and see which policies matched, the evaluation result, and the generated SQL — all without leaving the page.
* **ViewDefinition** — a visual builder for [SQL on FHIR](../../modules/sql-on-fhir/README.md) ViewDefinitions. Edit columns, preview the generated SQL, run the query, and inspect results side by side with a FHIRPath editor and schema browser.

<video src="../../../assets/overview/aidbox-ui/vd.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>

### REST Console

An interactive HTTP client built into Aidbox. Use it to execute REST API requests, inspect responses, and build collections of saved queries.
An interactive HTTP client built into Aidbox. Use it to execute REST API requests, inspect responses, and build collections of saved queries.

<video src="../../../assets/overview/aidbox-ui/rest-4k.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>

### SQL Console

Expand All @@ -36,31 +40,36 @@ Per-tab settings:

Stop button cancels the running query via [`$psql-cancel`](../../api/rest-api/other/sql-endpoints.md#usdpsql-cancel). The `Tab` key indents inside the editor; `EXPLAIN` plans render as a monospace block. Each tab keeps its own settings, query text, and running state. Backed by the [`$psql` endpoint](../../api/rest-api/other/sql-endpoints.md#usdpsql).

<video src="../../../assets/overview/aidbox-ui/db.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>

### Database

A DBA-focused page at `/u/database` with three subpages:

* **Schema Explorer** — every table across all user schemas, grouped into per-schema tabs. Per-table size, row count, index/toast share, and time since last (auto)vacuum/(auto)analyze. Expand a row to inspect its indexes (with their `CREATE INDEX` DDL on hover) and run `VACUUM`, `ANALYZE`, or `REINDEX`. Destructive operations like `TRUNCATE` are available only via the [`aidbox.pg/*` RPCs](../../database/database-maintenance-rpcs.md).

<figure><img src="../../../assets/database/schema-explorer.png" alt="Database → Schema Explorer — per-schema tabs over a fuzzy search; one row per table with size, row count, index/toast share, and recency of vacuum/analyze"><figcaption><p>Database → Schema Explorer.</p></figcaption></figure>

* **Running Queries** — `pg_stat_activity` snapshot of active queries, refreshed every 5 seconds. Cancel a statement (`pg_cancel_backend`) or terminate the whole connection (`pg_terminate_backend`) from the row.

<figure><img src="../../../assets/database/running-queries.png" alt="Database → Running Queries — live table of pg_stat_activity backends with PID, user, duration, wait event, app, query, and Cancel / Terminate buttons"><figcaption><p>Database → Running Queries.</p></figcaption></figure>

* **Search Params Stats** — paginated, sortable view of `aidbox_stat.search_param_stats`. Pick a resource type from the dropdown, filter by search-param substring, drop stats for the selected rows, or reset everything. See [Search Parameters Usage Statistics](../../deployment-and-maintenance/indexes/search-parameter-usage-stats.md) for the underlying data model.

<figure><img src="../../../assets/database/search-params-stats.png" alt="Database → Search Params Stats — sortable table of per-SP call counts, mean/min/max/total time, last-used timestamp, and an Index column; per-row checkboxes drive bulk drop"><figcaption><p>Database → Search Params Stats.</p></figcaption></figure>

Backed by the [`aidbox.pg/*`](../../database/database-maintenance-rpcs.md) and [`aidbox.index/*`](../../deployment-and-maintenance/indexes/search-parameter-usage-stats.md) RPCs.

### FHIR Packages

Browse, install, and manage FHIR Implementation Guides and NPM packages loaded into Aidbox via the [FHIR Artifact Registry](../../artifact-registry/artifact-registry-overview.md). Inspect individual resources within each package.

<video src="../../../assets/overview/aidbox-ui/far.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>

### Audit Events

View and search [AuditEvent](../../access-control/audit-and-logging.md) resources generated by Aidbox, with filtering by date, type, and agent.

### Settings

Configure Aidbox instance settings from the UI.

## Screenshots and videos

{% tabs %}
{% tab title="DB Console" %}
<video src="../../assets/overview/aidbox-ui/db.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>
{% endtab %}
{% tab title="REST Console" %}
<video src="../../assets/overview/aidbox-ui/rest-4k.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>
{% endtab %}
{% tab title="ViewDefinition Builder" %}
<video src="../../assets/overview/aidbox-ui/vd.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>
{% endtab %}
{% tab title="FHIR Packages" %}
<video src="../../assets/overview/aidbox-ui/far.mp4" autoplay loop muted playsinline controls class="w-full rounded-lg" loading="lazy"></video>
{% endtab %}
{% endtabs %}
Loading