Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 81 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,67 @@ To rollback instead of committing:
await tx.rollback()
```

### Cross-Database Queries

Attach other SQLite databases to run queries across multiple database files.
Each attached database gets a schema name that acts as a namespace for its
tables.

**Builder Pattern:** All query methods (`execute`, `executeTransaction`,
`fetchAll`, `fetchOne`) return builders that support `.attach()` for
cross-database operations.

```typescript
// Join data from multiple databases
const results = await db.fetchAll(
'SELECT u.name, o.total FROM users u JOIN orders.orders o ON u.id = o.user_id',
[]
).attach([
{
databasePath: 'orders.db',
schemaName: 'orders',
mode: 'readOnly'
}
])

// Update main database using data from attached database
await db.execute(
'UPDATE todos SET status = $1 WHERE id IN (SELECT todo_id FROM archive.completed)',
['archived']
).attach([
{
databasePath: 'archive.db',
schemaName: 'archive',
mode: 'readOnly'
}
])

// Atomic writes across multiple databases
await db.executeTransaction([
['INSERT INTO main.orders (user_id, total) VALUES ($1, $2)', [userId, total]],
['UPDATE stats.order_count SET count = count + 1', []]
]).attach([
{
databasePath: 'stats.db',
schemaName: 'stats',
mode: 'readWrite'
}
])
```

**Attached Database Modes:**

* `readOnly` - Read-only access (can be used with read or write operations)
* `readWrite` - Read-write access (requires write operation, holds write
lock)

**Important:**

* Attached database(s) automatically detached after query completion
* Read-write attachments acquire write locks on all involved databases
* Attachments are connection-scoped and don't persist across queries
* Main database is always accessible without a schema prefix

### Error Handling

```typescript
Expand All @@ -315,9 +376,9 @@ Common error codes:
### Closing and Removing

```typescript
await db.close() // Close this connection
await Database.closeAll() // Close all connections
await db.remove() // Close and DELETE database file(s) - irreversible!
await db.close() // Close this connection
await Database.close_all() // Close all connections
await db.remove() // Close and DELETE database file(s) - irreversible!
```

## API Reference
Expand All @@ -328,7 +389,7 @@ await db.remove() // Close and DELETE database file(s) - irreversible!
| ------ | ----------- |
| `Database.load(path, config?)` | Connect and return Database instance (or existing) |
| `Database.get(path)` | Get instance without connecting (lazy init) |
| `Database.closeAll()` | Close all database connections |
| `Database.close_all()` | Close all database connections |

### Instance Methods

Expand All @@ -342,6 +403,16 @@ await db.remove() // Close and DELETE database file(s) - irreversible!
| `close()` | Close connection, returns `true` if was loaded |
| `remove()` | Close and delete database file(s), returns `true` if was loaded |

### Builder Methods

All query methods (`execute`, `executeTransaction`, `fetchAll`, `fetchOne`)
return builders that are directly awaitable and support method chaining:

| Method | Description |
| ------ | ----------- |
| `attach(specs)` | Attach databases for cross-database queries, returns `this` |
| `await builder` | Execute the query (builders implement `PromiseLike`) |

### InterruptibleTransaction Methods

| Method | Description |
Expand All @@ -364,6 +435,12 @@ interface CustomConfig {
idleTimeoutSecs?: number // default: 30
}

interface AttachedDatabaseSpec {
databasePath: string // Path relative to app config directory
schemaName: string // Schema name for accessing tables (e.g., 'orders')
mode: 'readOnly' | 'readWrite'
}

interface SqliteError {
code: string
message: string
Expand Down
2 changes: 1 addition & 1 deletion api-iife.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions crates/sqlx-sqlite-conn-mgr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ times is safe (already-applied migrations are skipped).

### Attached Databases

Attach other SQLite databases to enable cross-database queries. Attachments are
Attach other SQLite databases to enable cross-database queries. Attached databases are
connection-scoped and automatically detached when the guard is dropped.

```rust
Expand Down Expand Up @@ -175,8 +175,8 @@ returned to pool on drop.

| Function | Description |
| -------- | ----------- |
| `acquire_reader_with_attached(db, specs)` | Acquire read connection with attached databases |
| `acquire_writer_with_attached(db, specs)` | Acquire writer connection with attached databases |
| `acquire_reader_with_attached(db, specs)` | Acquire read connection with attached database(s) |
| `acquire_writer_with_attached(db, specs)` | Acquire writer connection with attached database(s) |

Returns `AttachedConnection` or `AttachedWriteGuard` respectively. Both guards
deref to `SqliteConnection` and automatically detach databases on drop.
Expand Down
34 changes: 18 additions & 16 deletions crates/sqlx-sqlite-conn-mgr/src/attached.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ pub enum AttachedMode {
ReadWrite,
}

/// Guard holding a read connection with attached databases
/// Guard holding a read connection with attached database(s)
///
/// **Important**: Call `detach_all()` before dropping to properly clean up attached databases.
/// **Important**: Call `detach_all()` before dropping to properly clean up attached database(s).
/// Without explicit cleanup, attached databases persist on the pooled connection until
/// it's eventually closed. Derefs to `SqliteConnection` for executing queries.
#[must_use = "if unused, the attached connection and locks are immediately dropped"]
Expand Down Expand Up @@ -100,7 +100,7 @@ impl Drop for AttachedReadConnection {
}
}

/// Guard holding a write connection with attached databases
/// Guard holding a write connection with attached database(s)
///
/// **Important**: Call `detach_all()` before dropping to properly clean up attached databases.
/// Without explicit cleanup, attached databases persist on the pooled connection until
Expand Down Expand Up @@ -189,7 +189,7 @@ fn is_valid_schema_name(name: &str) -> bool {
&& !name.chars().next().unwrap().is_ascii_digit()
}

/// Acquire a read connection with attached databases
/// Acquire a read connection with attached database(s)
///
/// This function:
/// 1. Acquires a read connection from the main database's read pool
Expand Down Expand Up @@ -231,7 +231,7 @@ pub async fn acquire_reader_with_attached(
for spec in &specs {
let path = spec.database.path_str();
if !seen_paths.insert(path.clone()) {
return Err(Error::DuplicateDatabaseAttachment(path));
return Err(Error::DuplicateAttachedDatabase(path));
}
}

Expand Down Expand Up @@ -261,7 +261,7 @@ pub async fn acquire_reader_with_attached(
Ok(AttachedReadConnection::new(conn, Vec::new(), schema_names))
}

/// Acquire a write connection with attached databases
/// Acquire a write connection with attached database(s)
///
/// This function:
/// 1. Acquires the write connection from the main database
Expand Down Expand Up @@ -320,7 +320,7 @@ pub async fn acquire_writer_with_attached(
let mut seen_paths = HashSet::new();
for (path, _) in &db_entries {
if !seen_paths.insert(path.as_str()) {
return Err(Error::DuplicateDatabaseAttachment(path.clone()));
return Err(Error::DuplicateAttachedDatabase(path.clone()));
}
}

Expand Down Expand Up @@ -517,19 +517,21 @@ mod tests {
.fetch_one(&mut *conn)
.await
.unwrap();

let value1: String = row1.get(0);
assert_eq!(value1, "test_data");

let row2 = sqlx::query("SELECT value FROM db2.db2 LIMIT 1")
.fetch_one(&mut *conn)
.await
.unwrap();

let value2: String = row2.get(0);
assert_eq!(value2, "test_data");
}

#[tokio::test]
async fn test_readwrite_attachment_holds_writer_lock() {
async fn test_attached_database_in_readwrite_mode_holds_writer_lock() {
let temp_dir = TempDir::new().unwrap();
let main_db = create_test_db("main.db", &temp_dir).await;
let other_db = create_test_db("other.db", &temp_dir).await;
Expand All @@ -541,7 +543,7 @@ mod tests {
}];

// Acquire writer with attached database (holds other_db's writer)
let _attached_writer = acquire_writer_with_attached(&main_db, specs).await.unwrap();
let _guard = acquire_writer_with_attached(&main_db, specs).await.unwrap();

// Try to acquire other_db's writer directly - should block/timeout
let acquire_result = tokio::time::timeout(
Expand Down Expand Up @@ -571,7 +573,7 @@ mod tests {

// Acquire and drop
{
let _attached_writer = acquire_writer_with_attached(&main_db, specs).await.unwrap();
let _ = acquire_writer_with_attached(&main_db, specs).await.unwrap();
// Dropped at end of scope
}

Expand Down Expand Up @@ -646,7 +648,7 @@ mod tests {
}

#[tokio::test]
async fn test_attached_sorting_prevents_deadlock() {
async fn test_sorting_attached_databases_prevents_deadlock() {
let temp_dir = TempDir::new().unwrap();
let main_db = create_test_db("main.db", &temp_dir).await;
let db_a = create_test_db("a.db", &temp_dir).await;
Expand Down Expand Up @@ -675,7 +677,7 @@ mod tests {
}

#[tokio::test]
async fn test_cross_database_attachment_no_deadlock() {
async fn test_attaching_same_databases_in_different_order_concurrently_no_deadlock() {
// This test verifies the fix for the deadlock scenario:
// Thread 1: main=A, attach B
// Thread 2: main=B, attach A
Expand Down Expand Up @@ -765,7 +767,7 @@ mod tests {
}

#[tokio::test]
async fn test_duplicate_database_rejected() {
async fn test_duplicate_attached_database_rejected() {
let temp_dir = TempDir::new().unwrap();
let main_db = create_test_db("main.db", &temp_dir).await;
let other_db = create_test_db("other.db", &temp_dir).await;
Expand All @@ -786,8 +788,8 @@ mod tests {

let result = acquire_writer_with_attached(&main_db, specs).await;
assert!(
matches!(result, Err(Error::DuplicateDatabaseAttachment(_))),
"Should reject duplicate database attachment"
matches!(result, Err(Error::DuplicateAttachedDatabase(_))),
"Should reject duplicate attached database"
);
}

Expand All @@ -805,7 +807,7 @@ mod tests {

let result = acquire_writer_with_attached(&main_db, specs).await;
assert!(
matches!(result, Err(Error::DuplicateDatabaseAttachment(_))),
matches!(result, Err(Error::DuplicateAttachedDatabase(_))),
"Should reject attaching main database to itself"
);
}
Expand Down
9 changes: 5 additions & 4 deletions crates/sqlx-sqlite-conn-mgr/src/database.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,11 @@ pub struct SqliteDatabase {
impl SqliteDatabase {
/// Get the database file path as a string
///
/// Used internally for ATTACH DATABASE statements
/// Used internally (crate-private) for ATTACH DATABASE statements
pub(crate) fn path_str(&self) -> String {
self.path.to_string_lossy().to_string()
}
}

impl SqliteDatabase {
/// Connect to a SQLite database
///
/// If the database is already connected, returns the existing connection.
Expand Down Expand Up @@ -295,14 +293,17 @@ impl SqliteDatabase {
///
/// # Example
///
/// ```ignore
/// ```no_run
/// use sqlx_sqlite_conn_mgr::SqliteDatabase;
///
/// # async fn example() -> Result<(), Box<dyn std::error::Error>> {
/// // sqlx::migrate! is evaluated at compile time
/// static MIGRATOR: sqlx::migrate::Migrator = sqlx::migrate!("./migrations");
///
/// let db = SqliteDatabase::connect("test.db", None).await?;
/// db.run_migrations(&MIGRATOR).await?;
/// # Ok(())
/// # }
/// ```
pub async fn run_migrations(&self, migrator: &sqlx::migrate::Migrator) -> Result<()> {
// Ensure WAL mode is initialized via acquire_writer
Expand Down
6 changes: 4 additions & 2 deletions crates/sqlx-sqlite-conn-mgr/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ pub enum Error {
InvalidSchemaName(String),

/// Attempted to attach the same database multiple times
#[error("Database '{0}' appears multiple times in attachment list (would cause deadlock)")]
DuplicateDatabaseAttachment(String),
#[error(
"Database '{0}' appears multiple times in attached database list (would cause deadlock)"
)]
DuplicateAttachedDatabase(String),
}
Loading