Async produce?

[gili8420]

I'm looking for a way to have Centrifugo publish messages directly to Kafka, similarly to how the Kafka consumer works internally — but in reverse.

I reviewed the internal Kafka consumer logic in Centrifugo, and I'm wondering if there's a built-in configuration or mechanism to achieve the opposite direction (i.e., producing messages to Kafka instead of consuming from it).

Would it be possible to implement this using something like Redpanda Benthos, or another streaming tool?
My goal is to avoid building a custom "centrifugo-exporter" service.

[FZambia]

Hello @awend0

There is no sth like this built-in right now. But you can achieve this with RPC proxy. I.e. implement some service which will receive proxy requests from Centrifugo (issued when the client side calls .rpc(...) method) and put data to Kafka. It can work as sidecar maybe.

It's not very clear to me which things such native integration may solve in practice. Maybe you can provide real-world use cases. Also, non-obvious part of this is data validation – should it be somehow validated before producing to Kafka.