Just wondering what the ideas are around running kafka or similar streaming systems 
At the moment I’m stuck at running my own kafka containers, but I’d much rather have Render run them 
Hi @harmw,
That’s a great question. I haven’t personally deployed kafka on Render, but I believe a single-node cluster would be relatively straighforward to run. An multi-node/multi-broker cluster would be a little more complicated. Since Render services with persistent disks can only have one instance, each node/broker would need to be a new Render service and setting up service discovery would likely be a bit more manual (since you can’t just refer to the nth instance of a multi-instance service).
Another snag you might run into is automatic rolling deploys. If you configure automatic deploys and have each node’s service backed by the same git repo (which seems like the most likely approach), they’d all get updated simultaneously. I don’t know the details of kafka’s multi-node architecture, so it may be resilient to this, but if it’s like etcd, you’ll want each node to update one-at-a-time.
If you make progress on this, I’d love to hear about it! And I’m happy to answer any questions that come up if you decide to move forward with experimenting.
1 Like
My team is exploring this currently, as it’s a big dependency looking at my current micro-services platform. Managing EC2-based Kafka (or Pulsar) is doable, but doing so in several Render services it’s looking to get a little too complex. It almost sounds like devops 
Is Render planning to bring support for something managed around event-sourcing?
1 Like
I know we are frequently thinking about what managed services make sense to build out, but as far as I know, we aren’t currently planning anything for event sourcing. I also checked https://feedback.render.com/features and couldn’t find any feature requests, but I encourage you to add one with details on your use case and what would be most valuable for you, since that definitely helps us prioritize new features
Hey folks, I have a related question, is there a way to run kafka consumer on render? I plan to have my kafka queue running inside confluent cloud
Hi there,
There shouldn’t be anything preventing you from running a Kafka consumer on Render.
Regards,
Keith
Render Support, UTC+10 