Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
Until `1.0.0`, breaking changes may appear in any release and are flagged with **BREAKING** below.

## [Unreleased]

### Changed (BREAKING)

- **Outbox domain table renamed `outbox` → `okapi_outbox`.** Indexes follow the rename
(`idx_outbox_*` → `idx_okapi_outbox_*`). Host applications with a pre-existing `outbox`
table are no longer affected — okapi creates its own table under the `okapi_` prefix.
The new name is fixed; it is not configurable. ([#37](https://github.com/softwaremill/okapi/issues/37))
- **Liquibase tracking tables default to `okapi_databasechangelog` /
`okapi_databasechangeloglock`.** Previously okapi shared the application's
default `databasechangelog` / `databasechangeloglock`. Override the new defaults
via configuration to keep the shared-table layout (see Added below).
([#37](https://github.com/softwaremill/okapi/issues/37))

### Added

- `okapi.liquibase.changelog-table` — Spring Boot property that configures the
`databaseChangeLogTable` of okapi's autoconfigured `SpringLiquibase` beans
(`okapiPostgresLiquibase` / `okapiMysqlLiquibase`). Default: `okapi_databasechangelog`.
- `okapi.liquibase.changelog-lock-table` — likewise for `databaseChangeLogLockTable`.
Default: `okapi_databasechangeloglock`.

### Migration from 0.2.x

These are breaking changes; existing deployments must take action before the first
`0.3.0` startup. The README has the full SQL: see
[Database migrations § Upgrading from 0.2.x](README.md#upgrading-from-02x).
Two paths are documented: rename in place (recommended) or stay on the legacy
changelog table names by overriding `okapi.liquibase.changelog-table` /
`changelog-lock-table`. The domain-table rename has no opt-out — run the
provided `ALTER TABLE ... RENAME TO okapi_outbox` script.

## [0.2.0] — 2026-04-29

### Added

- Observability: `OutboxProcessorListener` API and the `okapi-micrometer` module
(counters, timers, gauges; Spring Boot Actuator integration). ([#27](https://github.com/softwaremill/okapi/pull/27))
- Multi-datasource transaction validation in `okapi-spring-boot`
(`SpringTransactionContextValidator`, `okapi.datasource-qualifier` property). ([#17](https://github.com/softwaremill/okapi/pull/17))
- `@JvmOverloads` / `@JvmStatic` annotations across the public API for Java interop. ([#24](https://github.com/softwaremill/okapi/pull/24))
- Maven Central release pipeline. ([#18](https://github.com/softwaremill/okapi/pull/18))

### Changed

- `OutboxStore` migrated from JetBrains Exposed to plain JDBC in
`okapi-postgres` and `okapi-mysql`. The Exposed-based path remains
available via the optional `okapi-exposed` module. ([#26](https://github.com/softwaremill/okapi/pull/26))
- Configuration unification: `Duration` types throughout, dedicated
`OutboxPurgerConfig` and `OutboxSchedulerConfig`. ([#16](https://github.com/softwaremill/okapi/pull/16))
- `OutboxProcessorScheduler` and `OutboxPurger` v2 — configurable interval,
batch size, retention; reliable shutdown via `SmartLifecycle`. ([#11](https://github.com/softwaremill/okapi/pull/11), [#14](https://github.com/softwaremill/okapi/pull/14))

### Fixed

- Actionable error message in `ExposedConnectionProvider` when no transaction is
bound to the current thread. ([#32](https://github.com/softwaremill/okapi/pull/32))
- `okapi-micrometer` artifact published to Maven Central; the
`okapi.metrics.refresh-interval` property documented. ([#29](https://github.com/softwaremill/okapi/pull/29))

## [0.1.0] — 2026-04-07

Initial public release.

### Added

- Transactional outbox pattern for Kotlin/JVM with PostgreSQL and MySQL stores.
- `okapi-http` and `okapi-kafka` deliverers; pluggable `MessageDeliverer` API.
- `OutboxProcessor` with configurable `RetryPolicy` and delivery-result
classification (`Success` / `RetriableFailure` / `PermanentFailure`).
- `okapi-spring-boot` autoconfiguration for stores, transports, scheduler, and purger.
- `okapi-exposed` integration (transaction runner, connection provider, validator).
- Concurrent processing via `FOR UPDATE SKIP LOCKED`.

[Unreleased]: https://github.com/softwaremill/okapi/compare/v0.2.0...HEAD
[0.2.0]: https://github.com/softwaremill/okapi/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/softwaremill/okapi/releases/tag/v0.1.0
64 changes: 64 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,70 @@ Okapi implements the [transactional outbox pattern](https://softwaremill.com/mic
- **Concurrent processing** — multiple processors can run in parallel using `FOR UPDATE SKIP LOCKED`, so messages are never processed twice simultaneously.
- **Delivery result classification** — each transport classifies errors as `Success`, `RetriableFailure`, or `PermanentFailure`. For example, HTTP 429 is retriable while HTTP 400 is permanent.

## Database migrations

Okapi ships Liquibase changelogs that create the outbox table and its indexes:

- `classpath:com/softwaremill/okapi/db/changelog.xml` — PostgreSQL (from `okapi-postgres`)
- `classpath:com/softwaremill/okapi/db/mysql/changelog.xml` — MySQL (from `okapi-mysql`)

When `okapi-spring-boot` is on the classpath, these run automatically against the configured `DataSource` on application startup. Without Spring Boot, point your own Liquibase setup at the paths above and pass an `outboxTable` change-log parameter (see below).

### Configuration

Okapi's table names are fixed under the `okapi_` prefix so its schema stays out of the way of any pre-existing tables in the host application (`outbox`, `databasechangelog`, etc.):

| Table | Purpose |
|-------|---------|
| `okapi_outbox` | Domain table holding outbox entries (created by the bundled Liquibase changesets, queried by `PostgresOutboxStore` / `MysqlOutboxStore`). |
| `okapi_databasechangelog` | Liquibase changeset history for okapi (configurable). |
| `okapi_databasechangeloglock` | Liquibase concurrency lock for okapi (configurable). |

The Liquibase tracking-table names are configurable in case the host application wants to share them with its own Liquibase setup:

| Property | Default | Description |
|----------|---------|-------------|
| `okapi.liquibase.changelog-table` | `okapi_databasechangelog` | Liquibase changeset history for okapi |
| `okapi.liquibase.changelog-lock-table` | `okapi_databasechangeloglock` | Liquibase concurrency lock for okapi |

These properties affect the autoconfigured `okapiPostgresLiquibase` / `okapiMysqlLiquibase` beans only. If you run Liquibase yourself, configure the table names there directly. The domain table name (`okapi_outbox`) is fixed.

### Upgrading from 0.2.x

Releases up to 0.2.x wrote to shared tables `databasechangelog` / `databasechangeloglock` and the domain table `outbox`. From 0.3.0 these are renamed to `okapi_*`. Two upgrade paths:

**Stay on the existing changelog tables** (simplest for the Liquibase tracking pair, zero-downtime) — opt out of the new defaults:

```yaml
okapi:
liquibase:
changelog-table: databasechangelog
changelog-lock-table: databasechangeloglock
```

The domain table `outbox` cannot be opted out via configuration — see the migration steps below.

**Migrate to dedicated tables** — run before the first 0.3.0 startup (PostgreSQL syntax shown):

```sql
-- Outbox domain table: rename in place. Indexes follow the table.
ALTER TABLE outbox RENAME TO okapi_outbox;
ALTER INDEX idx_outbox_status_last_attempt RENAME TO idx_okapi_outbox_status_last_attempt;
ALTER INDEX idx_outbox_status_created_at RENAME TO idx_okapi_outbox_status_created_at;

-- Liquibase tracking: split okapi rows into the new tables.
CREATE TABLE okapi_databasechangelog (LIKE databasechangelog INCLUDING ALL);
CREATE TABLE okapi_databasechangeloglock (LIKE databasechangeloglock INCLUDING ALL);
INSERT INTO okapi_databasechangelog
SELECT * FROM databasechangelog WHERE filename LIKE '%com/softwaremill/okapi/%';
INSERT INTO okapi_databasechangeloglock SELECT * FROM databasechangeloglock;
DELETE FROM databasechangelog WHERE filename LIKE '%com/softwaremill/okapi/%';
```

Without one of these steps, Liquibase will see an empty changelog table on the first 0.3.0 startup and try to re-run okapi's migrations — which fails if rows already exist under the legacy `outbox` table while okapi now writes to `okapi_outbox`.

Full release history: [CHANGELOG.md](CHANGELOG.md).

## Observability

Add `okapi-micrometer` alongside `okapi-spring-boot` (from the Quick Start above) to get Micrometer metrics:
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ org.gradle.caching=true
org.gradle.configuration-cache=true

GROUP=com.softwaremill.okapi
VERSION_NAME=0.2.0
VERSION_NAME=0.3.0
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ class PostgresBenchmarkSupport {
fun truncate() {
jdbc.withTransaction {
jdbc.withConnection { conn ->
conn.createStatement().use { it.execute("TRUNCATE TABLE outbox") }
conn.createStatement().use { it.execute("TRUNCATE TABLE okapi_outbox") }
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class MysqlTestSupport {
fun truncate() {
jdbc.withTransaction {
jdbc.withConnection { conn ->
conn.createStatement().use { it.execute("DELETE FROM outbox") }
conn.createStatement().use { it.execute("DELETE FROM okapi_outbox") }
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class PostgresTestSupport {
fun truncate() {
jdbc.withTransaction {
jdbc.withConnection { conn ->
conn.createStatement().use { it.execute("TRUNCATE TABLE outbox") }
conn.createStatement().use { it.execute("TRUNCATE TABLE okapi_outbox") }
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ class ConnectionLeakProofTest : FunSpec({

beforeEach {
counter.delegate.connection.use { conn ->
conn.createStatement().use { it.execute("TRUNCATE TABLE outbox") }
conn.createStatement().use { it.execute("TRUNCATE TABLE okapi_outbox") }
}
counter.opened.set(0)
counter.closed.set(0)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ class MultiDataSourceTransactionTest : FunSpec({

beforeEach {
outboxDataSource.connection.use { conn ->
conn.createStatement().use { it.execute("TRUNCATE TABLE outbox") }
conn.createStatement().use { it.execute("TRUNCATE TABLE okapi_outbox") }
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ class MysqlConnectionLeakProofTest : FunSpec({

beforeEach {
counter.delegate.connection.use { conn ->
conn.createStatement().use { it.execute("DELETE FROM outbox") }
conn.createStatement().use { it.execute("DELETE FROM okapi_outbox") }
}
counter.opened.set(0)
counter.closed.set(0)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class MysqlOutboxStore(

override fun persist(entry: OutboxEntry): OutboxEntry {
val sql = """
INSERT INTO outbox (id, message_type, payload, delivery_type, status, created_at, updated_at, retries, last_attempt, last_error, delivery_metadata)
INSERT INTO okapi_outbox (id, message_type, payload, delivery_type, status, created_at, updated_at, retries, last_attempt, last_error, delivery_metadata)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE
status = VALUES(status),
Expand Down Expand Up @@ -61,8 +61,8 @@ class MysqlOutboxStore(
// that FOR UPDATE SKIP LOCKED only row-locks the rows actually returned
// by LIMIT, rather than every row matching the WHERE clause.
val sql = """
SELECT * FROM outbox
FORCE INDEX (idx_outbox_status_created_at)
SELECT * FROM okapi_outbox
FORCE INDEX (idx_okapi_outbox_status_created_at)
WHERE status = ?
ORDER BY created_at ASC
LIMIT ?
Expand All @@ -84,9 +84,9 @@ class MysqlOutboxStore(

override fun removeDeliveredBefore(time: Instant, limit: Int): Int {
val sql = """
DELETE FROM outbox WHERE id IN (
DELETE FROM okapi_outbox WHERE id IN (
SELECT id FROM (
SELECT id FROM outbox
SELECT id FROM okapi_outbox
WHERE status = ?
AND last_attempt < ?
ORDER BY id
Expand All @@ -109,7 +109,7 @@ class MysqlOutboxStore(
override fun findOldestCreatedAt(statuses: Set<OutboxStatus>): Map<OutboxStatus, Instant> {
val result = statuses.associateWith { clock.instant() }.toMutableMap()
val placeholders = statuses.joinToString(",") { "?" }
val sql = "SELECT status, MIN(created_at) AS min_created_at FROM outbox WHERE status IN ($placeholders) GROUP BY status"
val sql = "SELECT status, MIN(created_at) AS min_created_at FROM okapi_outbox WHERE status IN ($placeholders) GROUP BY status"

connectionProvider.withConnection { conn ->
conn.prepareStatement(sql).use { stmt ->
Expand All @@ -126,7 +126,7 @@ class MysqlOutboxStore(
}

override fun countByStatuses(): Map<OutboxStatus, Long> {
val sql = "SELECT status, COUNT(*) AS count FROM outbox GROUP BY status"
val sql = "SELECT status, COUNT(*) AS count FROM okapi_outbox GROUP BY status"
val counts = mutableMapOf<OutboxStatus, Long>()

connectionProvider.withConnection { conn ->
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
--liquibase formatted sql
--changeset outbox:001

CREATE TABLE IF NOT EXISTS outbox
CREATE TABLE IF NOT EXISTS okapi_outbox
(
id CHAR(36) NOT NULL PRIMARY KEY,
message_type VARCHAR(255) NOT NULL,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
--liquibase formatted sql
--changeset outbox:002

CREATE INDEX idx_outbox_status_last_attempt ON outbox(status, last_attempt);
CREATE INDEX idx_okapi_outbox_status_last_attempt ON okapi_outbox(status, last_attempt);
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
--liquibase formatted sql
--changeset outbox:003

CREATE INDEX idx_outbox_status_created_at ON outbox (status, created_at);
CREATE INDEX idx_okapi_outbox_status_created_at ON okapi_outbox (status, created_at);
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class PostgresOutboxStore(

override fun persist(entry: OutboxEntry): OutboxEntry {
val sql = """
INSERT INTO outbox (id, message_type, payload, delivery_type, status, created_at, updated_at, retries, last_attempt, last_error, delivery_metadata)
INSERT INTO okapi_outbox (id, message_type, payload, delivery_type, status, created_at, updated_at, retries, last_attempt, last_error, delivery_metadata)
VALUES (?::uuid, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?::jsonb)
ON CONFLICT (id) DO UPDATE SET
status = EXCLUDED.status,
Expand Down Expand Up @@ -58,7 +58,7 @@ class PostgresOutboxStore(

override fun claimPending(limit: Int): List<OutboxEntry> {
val sql = """
SELECT * FROM outbox
SELECT * FROM okapi_outbox
WHERE status = ?
ORDER BY created_at ASC
LIMIT ?
Expand All @@ -80,8 +80,8 @@ class PostgresOutboxStore(

override fun removeDeliveredBefore(time: Instant, limit: Int): Int {
val sql = """
DELETE FROM outbox WHERE id IN (
SELECT id FROM outbox
DELETE FROM okapi_outbox WHERE id IN (
SELECT id FROM okapi_outbox
WHERE status = ?
AND last_attempt < ?
ORDER BY id
Expand All @@ -103,7 +103,7 @@ class PostgresOutboxStore(
override fun findOldestCreatedAt(statuses: Set<OutboxStatus>): Map<OutboxStatus, Instant> {
val result = statuses.associateWith { clock.instant() }.toMutableMap()
val placeholders = statuses.joinToString(",") { "?" }
val sql = "SELECT status, MIN(created_at) AS min_created_at FROM outbox WHERE status IN ($placeholders) GROUP BY status"
val sql = "SELECT status, MIN(created_at) AS min_created_at FROM okapi_outbox WHERE status IN ($placeholders) GROUP BY status"

connectionProvider.withConnection { conn ->
conn.prepareStatement(sql).use { stmt ->
Expand All @@ -120,7 +120,7 @@ class PostgresOutboxStore(
}

override fun countByStatuses(): Map<OutboxStatus, Long> {
val sql = "SELECT status, COUNT(*) AS count FROM outbox GROUP BY status"
val sql = "SELECT status, COUNT(*) AS count FROM okapi_outbox GROUP BY status"
val counts = mutableMapOf<OutboxStatus, Long>()

connectionProvider.withConnection { conn ->
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
--liquibase formatted sql
--changeset outbox:001

CREATE TABLE IF NOT EXISTS outbox
CREATE TABLE IF NOT EXISTS okapi_outbox
(
id UUID NOT NULL PRIMARY KEY,
message_type VARCHAR(255) NOT NULL,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
--liquibase formatted sql
--changeset outbox:002

ALTER TABLE outbox ADD COLUMN IF NOT EXISTS delivery_type VARCHAR(50);
ALTER TABLE okapi_outbox ADD COLUMN IF NOT EXISTS delivery_type VARCHAR(50);

UPDATE outbox SET delivery_type = delivery_metadata->>'type' WHERE delivery_type IS NULL;
UPDATE okapi_outbox SET delivery_type = delivery_metadata->>'type' WHERE delivery_type IS NULL;

ALTER TABLE outbox ALTER COLUMN delivery_type SET NOT NULL;
ALTER TABLE okapi_outbox ALTER COLUMN delivery_type SET NOT NULL;
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
--liquibase formatted sql
--changeset outbox:003

CREATE INDEX idx_outbox_status_last_attempt ON outbox(status, last_attempt);
CREATE INDEX idx_okapi_outbox_status_last_attempt ON okapi_outbox(status, last_attempt);
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
--liquibase formatted sql
--changeset outbox:004

CREATE INDEX IF NOT EXISTS idx_outbox_status_created_at ON outbox (status, created_at);
CREATE INDEX IF NOT EXISTS idx_okapi_outbox_status_created_at ON okapi_outbox (status, created_at);
Loading