Dynamic Topic Mapping on the basis of message content
Currently mongodb source connector only supports dynamic topic mapping on the bases of collections/databases. Can we extend it to support routing on the basis of message content?
Why it's important?
We've currently setup connector for a very large collection but since we wanted to route the data for different sections to different topics, we had to setup separate connector for each and define the filter in the pipeline section (it runs as an aggregate query on changestreams to filter out the relevant data). Now obviously it created performance concerns as large number of collscan queries started to run on these changestreams to filter out the data for respective connector. And also there's a restriction to create indexes on change streams so we can't take advantage of that. This became a problem for us.
Thus adding support for above would solve the issues for us as well for anyone who's following above as the workaround. I've seen some community posts of the same as well.