Self-healing subscriptions for native pub sub

[Scooletz]

When using a transport that support a native pub sub mechanism, the subscribe part is handled in the following way: for every event that an endpoint wants to subscribe to a method is called resulting in calling a native transport API, registering the subscription.

Consider a following case. An endpoint is shut down and the file has some of the event entries removed. When an endpoint is started again, it does not unsubscribe from the events. We could provide a self-healing mechanism, calling an unsubscribe when an event of a type that should not be handled is received.

[Scooletz]

@timbussmann Could you take a look at it?

[timbussmann]

I like the idea, but thinking more about this I think there are some more questions on how transports providing native pubsub could benefit in the first place from file based pubsub (they can benefit from file based routing like any other transport).

With the current ideas

• Native pubsub mechanisms mostly require autosubscribe to create transport specific subscriptions (e.g. exchanges). Essentially, the broker takes the role of the subscription storage. Therefore Publish and Subscribe are still required to setup that "storage"
• Message driven pubsub mechanism only require a subscription storage to know where to send unicast messages to. Publish and Subscribe are not required, as the storage is already set up correctly and has all the information.

using both "storages" together becomes tricky. E.g.

• Removing an event handler from the routing file: "self-healing" could be applied to remove the subscription on the broker when a event is received
• Unsubscribing via code while the routing file still contains the event handler: code wins as the routing file is not used as a storage with native transports
• Adding an event handler to a routing file: does not lead to a subscribe operation therefore there will be no real subscription on the broker
• Subscribing via code: The routing file doesn't contain the event handler, "self-healing" would remove the subscription again

so this becomes quite tricky to manage properly. Thoughts @Scooletz ?

[SzymonPobiega]

@Scooletz @timbussmann the existence of Subscribe and Unsubscribe methods on IMessageSession is a bad API and in my opinion it cannot be fixed other than removed. Then the transport would be responsible for providing appropriate controls for subscribing and unsubscribing.

In case of RabbitMQ I managed to implement automatic subscribe and unsubscribe here: Particular/NServiceBus.RabbitMQ#247. I hope similar thing can be done with ASB. The idea is following:

• On start create a new set of subscriptions for the events that are not subscribed (might be smaller than previous set of subscriptions)
• When sure that new subscriptions are active, drop the old set

Rabbit ensures that no messages get duplicated because it automatically removes duplicates that end up in the same queue when routing. Not sure if ASB can do the same but even if no we can live with it because all ASB handlers are idempotent, are the? ;-)

[timbussmann]

I think the key takeaway here is that Subscribe and unsubscribe should not be provided by the core as there are mechanisms where they just make no sense, e.g.

• file based routing should require all modifications through the file, a call to subscribe and unsubscribe is actually even invalid
• native pubsub capabilities usually rely on some autosubscription mechanism, subscribing and unsubscribing at runtime is a very special edge case and can still be offered if possible

but it seems to me, that this doesn't necessarily belong to the transport, but to the pubsub mechanisms API?

[SzymonPobiega]

@timbussmann You are right. I did a mental shortcut and assumed that a pub/sub mechanism is provided by a transport but that is (and will be) not always the case.