Skip to content

feat(jsonrpc2): add manual receive mode to prevent message loss#835

Open
strichter wants to merge 2 commits into
bbernhard:masterfrom
strichter:feat/jsonrpc2-manual-receive-mode-upstream
Open

feat(jsonrpc2): add manual receive mode to prevent message loss#835
strichter wants to merge 2 commits into
bbernhard:masterfrom
strichter:feat/jsonrpc2-manual-receive-mode-upstream

Conversation

@strichter
Copy link
Copy Markdown

Fixes #834. Same root cause as the long-standing #255 ("Couldn't send message to golang channel, as there's no receiver").

Summary

In `MODE=json-rpc` with default `--receive-mode=on-start`, signal-cli auto-pulls inbound messages and pushes them as JSON-RPC notifications regardless of whether any websocket subscriber is currently attached. If no subscriber is reading the channel at the moment a notification arrives, the receiver loop in `src/client/jsonrpc2.go` drops the message via a non-blocking channel send and only logs a debug line. signal-cli has already acked the message to Signal at that point, so it is never redelivered.

This PR adds opt-in support for signal-cli's `--receive-mode=manual`, where the daemon does NOT auto-receive and clients must explicitly call `subscribeReceive` to start receiving. While no subscription is active, messages stay queued server-side under Signal's normal retention rules.

See #834 for the full reproduction, root cause analysis, and validation results.

Changes

  • `src/scripts/jsonrpc2-helper.go`: thread the new `JSON_RPC_RECEIVE_MODE` env var into the signal-cli daemon command line. Defaults unset / `on-start` to current behaviour. Setting it to `manual` adds `--receive-mode=manual` to the launch.
  • `src/client/jsonrpc2.go`:
    • Add `receiveSubscription` struct and per-account subscription tracking on `JsonRpc2Client`, with refcounting so multiple concurrent websocket subscribers for the same account share one signal-cli subscription.
    • Add `subscribeReceive` / `unsubscribeReceive` JSON-RPC method wrappers and `acquireReceiveSubscription` / `releaseReceiveSubscription` refcount helpers.
    • In `ReceiveData`, unwrap manual-mode notifications (`{subscription, result}`) so downstream consumers see the same envelope shape as in auto mode and don't have to know which mode is in use.
    • Modify `GetReceiveChannel` to accept an `account` parameter; if non-empty, acquire a subscription before registering the channel. Make the channel buffered (capacity 64) to absorb short subscriber stalls.
    • Modify `RemoveReceiveChannel` to look up the channel's account internally and release the subscription on detach.
  • `src/client/client.go`: thread `number` through `SignalClient.GetReceiveChannel(number string)` so the underlying `JsonRpc2Client` knows which account to subscribe.
  • `src/api/api.go`: pass `number` to `GetReceiveChannel` from `handleSignalReceive`.

Backward compatibility

Fully backward compatible:

  • When `JSON_RPC_RECEIVE_MODE` is unset or set to `on-start`, signal-cli runs in auto mode and the new `subscribeReceive` code path in `GetReceiveChannel` is gated by `if account != ""` — but in any case, calling `subscribeReceive` against an auto-mode daemon would just fail with no harm done, and the existing unsubscribed-broadcast path keeps working.
  • The notification unwrap step in `ReceiveData` is a no-op for auto-mode notifications because their params do not contain the `{subscription, result}` wrapper structure.
  • All public API signatures retain compatibility except `SignalClient.GetReceiveChannel`, which now takes a `number string` argument. The only in-tree caller (`handleSignalReceive`) is updated. External callers would need a one-line update.

Validation

Local validation against a real Signal account, real signal-cli-rest-api container in `MODE=json-rpc` mode, and a real bot consumer (nora-ng-2). Test scenario: stop the websocket client, send 10 messages from another Signal account into the group during a 60-second outage, restart the client, count delivered messages.

Configuration Result
Master (`on-start`, default) 10/10 lost (100% loss)
This PR (`JSON_RPC_RECEIVE_MODE=manual`) 0/10 lost (0% loss)

The signal-api log trace clearly shows the new lifecycle:

```
20:22:25 Subscribed to receive notifications for account +1XXXXXXXXXX (subscription=0)
20:22:47 Unsubscribed from receive notifications for account +1XXXXXXXXXX (subscription=0)
20:28:32 Subscribed to receive notifications for account +1XXXXXXXXXX (subscription=1)
```

Between unsubscribe (20:22:47) and resubscribe (20:28:32), 10 messages were sent into the group. After resubscribe, all 10 were delivered to the websocket client within ~10 seconds.

Notes for review

  • Refcounting assumes `subscribeReceive` and `unsubscribeReceive` are idempotent at the signal-cli layer. Multiple concurrent subscribers for the same account share one underlying signal-cli subscription. If signal-cli ever changes that contract this would need revisiting.
  • The notification format unwrap in `ReceiveData` is unconditional but harmless in auto mode (auto-mode `params` don't have a `subscription`/`result` shape, the unmarshal fails silently and `resp1.Params` is left untouched).
  • The channel buffer bump from unbuffered to capacity-64 in `GetReceiveChannel` is unrelated to manual mode; it just provides a small headroom so a subscriber that takes ~milliseconds to drain doesn't cause a single-message drop. Happy to split that into a separate commit if you'd prefer.
  • I have NOT updated the README or tests as part of this PR — happy to add those if you'd like the PR to land.

Open questions

  1. Default value: should this PR also flip the default to `manual`? I left it opt-in to be conservative, but auto mode is the buggy mode and I'd argue `manual` should be the new default (with `on-start` opt-in for users who want the old behaviour).
  2. Naming: `JSON_RPC_RECEIVE_MODE` follows the existing `JSON_RPC_*` env-var convention in the helper. Open to a different name if you prefer.
  3. Should I cherry-pick this against any active release branches?

Stephan Richter added 2 commits April 8, 2026 16:16
In the default 'on-start' receive mode, signal-cli's daemon
auto-pulls inbound messages and pushes them to its JSON-RPC clients
the moment they arrive from the Signal servers. If no websocket
subscriber is currently attached to /v1/receive/{number} — for
example during a brief subscriber redeploy — the receiver loop in
signal-cli-rest-api drops the message via a non-blocking channel
send (see the 'no receiver' debug log in ReceiveData). signal-cli
has already acknowledged the message to Signal, so it is never
redelivered.

Add support for signal-cli's --receive-mode=manual, opt-in via the
new JSON_RPC_RECEIVE_MODE=manual environment variable. In manual
mode signal-cli does NOT auto-receive; signal-cli-rest-api now
issues subscribeReceive when a websocket subscriber attaches and
unsubscribeReceive when the last subscriber detaches. While no
subscriber is attached, signal-cli does not pull messages from the
Signal servers, so they remain buffered server-side under Signal's
normal retention rules and are delivered to the next subscriber
that attaches.

Implementation:

* jsonrpc2-helper.go: thread JSON_RPC_RECEIVE_MODE through to the
  daemon launch command line.
* jsonrpc2.go: add per-account subscription tracking with
  refcounting; subscribeReceive/unsubscribeReceive helpers; an
  unwrap step in ReceiveData so manual-mode notifications
  ({subscription,result}) and auto-mode notifications (envelope
  directly) reach downstream consumers in the same shape.
* client.go, api.go: thread the account number through
  GetReceiveChannel so the JsonRpc2Client can attach the right
  subscription.

The change is fully backward compatible: when JSON_RPC_RECEIVE_MODE
is unset or set to 'on-start', signal-cli runs in auto mode and the
new subscribeReceive code path is bypassed (account is empty, no
RPC issued).

Closes bbernhard#255 (the same root cause: no subscriber → 'no receiver'
drop).
Copy link
Copy Markdown
Owner

@bbernhard bbernhard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR

Comment thread src/client/jsonrpc2.go
}

if account != "" {
if err := r.acquireReceiveSubscription(account); err != nil {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I don't understand that code part here. Isn't account always non-empty? So, I guess we would end up always in this code branch, no matter which mode is selected via JSON_RPC_RECEIVE_MODE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON-RPC mode: messages dropped when no websocket subscriber is attached

2 participants