Snooze streams

[MedAziz11]

Hello Faust team,

I am currently working on a project where I need to “snooze” streams to a specific time based on a process_time field specified in the record. I was wondering if there is an out-of-the-box feature in Faust for this, or if there is an example of the “Faust way” of implementing such functionality.

Use Case:

I have records with a process_time field, and I need to delay the processing of these records until the specified process_time.

Thank you for your assistance!

[Menziess]

Something like this?

[dada-engineer]

This is just not something that is supported by Kafka. What you could do is writing a transport for this and delay message serialization. Or you do this in your agent. Then a message is blocking though.

The big downside is that your consumer might get marked as dead by kafka when not commiting offsets of all pulled messages in session timeout time.

I would probably just reproduce this message to the topic so it is produced later then.