Description
These lines fire a concurrent db insert task for each new alert. This is not a problem in and of itself, since version 3+ of SQLite implements a multi-read/single-write concurrency model, but there's a catch.
The process of writing to the database requires the following logical steps
- obtain a SHARED lock -> still readable, new readers allowed
- obtain a RESERVED lock -> still readable, new readers allowed
- write some stuff durably to the transaction log -> still readable, new readers allowed
- obtain a PENDING lock -> still readable, no new readers allowed
- obtain an EXCLUSIVE lock -> not readable anymore
Here's SQLite docs page.
Each of the statements goes through this pipe, which means we have a bunch of (green) threads who likely all acquired a PENDING lock, so no new readers are allowed.
Thanks to @blkt for the above investigation
To avoid issues as the ones seen in #924 we should implement batch inserts into alerts table.
Handling all of the recording logic in a single transaction would also be ideal. Right now, we're inserting independently prompts, outputs and alerts. We could write into the 3 tables acquiring a lock for the transaction. Docs
BEGIN IMMEDIATE
-- write to prompts, output, and alerts
COMMIT;
Additional Context
No response