Kafka multiple producer writing to same topic - Ordering of message and data burst -

i trying learn kafka. have multiple producer each writing same topic. (cant have more topics because of design , using avro serialization) because our message big, need divide small parts , send kafka.

in scenario - can mesages different producers can intermix ? how can avoid scenario ? ideas

also , have huge databurst, there 1000s of message in 2 minutes, few message next 5 7 minutes, can in such scenarios

because our message big, need divide small parts , send kafka.

do you? runned basic tests , had issues? have tried adjust buffers? i'm pretty sure kafka can handle relatively big messages (tens of megabytes) without hassle. in fact, better throughput, comparing huge number of tiny messages.

also , have huge databurst, there 1000s of message in 2 minutes, few message next 5 7 minutes, can in such scenarios

single kafka broker on modern server machine can handle ~20k-40k messages/per second (batched of 1000 messages, each 2k size, sync mode). don't see problem there.

can messages different producers can intermix ?

yes, might intermix. in fact, moment not defined kafka protocol , particular implementation may change behaviour, stable solution take @ below section.

how can avoid scenario?

kafka has concept of partitions: each topic default has 1 partition, each partition can thought unit of parallelism. setup appropriate partitioner, such each producer writes it's own partition, in isolated manner.