Connectors (BI, Kafka, Spark)
41 results found
-
Enable custom DRDL upload for Atlas BI Connector
We cannot upload custom DRDL to Atlas BI connector. This forces us to roll our own BI connector
25 votes -
Enable CSFLE decryption in BI Connector
Enable CSFLE decryption in BI Connector, in order to provide encrypted fields in MongoDB decrypted for ODBC exposition.
CSFLE is described here: https://www.mongodb.com/docs/manual/core/security-client-side-encryption/.
For exemple, CSFLE would then be an option triggerable at BI Connector Launch, that should come with KMS link or keyring if enabled.11 votes -
Bi connector - store schema information
We have the BI connector installed/configured via OPS manager on one of our deployments.
When we have to restart the node, the BI connector takes a long time to start as it has to rebuild the schema.
Hence we would like to file an enhancement request to be able to store the schema information and then retrieve it again after a restart.10 votes -
Allow separate whitelist for Atlas BI Connector
Currently, whitelisting is only possible at the Project level. We would like to allow whitelisting of the BI connector instance separately from the project.
The Users/IPs that connect to mongo are completely different for BI connector vs the actual DB
8 votes -
Support Prepared Statements in BI Connector
The Power Query Editor in Microsoft leverages a prepared statement for query/filtering.
Error message:
This command is not supported in the prepared statement protocol yetThis is a feature request to support prepared statements in the mongosqld BI Connector service.
7 votes -
Add X509 as a Authentication Method
The current authentication methods lack in their ability to control password complexity requirements (SCRAM-SHA-*) or require exposing on-premise LDAP servers to the public internet. The addition of X509 is worthwhile given that MongoDB Atlas already supports X509 as a user authentication mechanism.
6 votes -
BI Connector username and password in yaml file
Currently BI Connector has username and password in clear. this presents problem in terms of security. BI connector has to have a way to encrypt the password and then we store the encrypted password which BI connector can decrypt and use it. it can be any standard encrypt decrypt algorithm but the secret key can be generated by the person using this.
6 votes -
BI Connector Atlas - View current schema creation/update status
In Atlas the is no way to see the BI connector logs, so, the request is to be able to see the Current schema creation/update status or even better, be able to see the log.
6 votes -
Enable Write Permission In BI
Hope BI enable write permission because a lot of system especially Windows base using ODBC
6 votes -
Notification and alerts when BI Connector fails
When a BI Connector fails, no alerts are sent to admins, only the notification on the page is available indicating that it will "restart in 5 minutes".
6 votes -
Mongo Spark Connector Option to refresh the Schema
This w.r.t the ticket we raised "https://support.mongodb.com/case/01352011"
In the current spark connector to infer the schema automatically we have an option "stream.publish.full.document.only" to "true", once this configured there is no explicit schema we need to pass but the driver will infer the schema on the first document it streams and use/apply the schema for any future documents coming from that collection.
But the issue here is when there is any addition of new fields in the source collection the streams are not inferring the new changes and instead it is using the old schema.
We should either design…4 votes -
CFLE support for Kafka connector
Using MongoDB CFLE, The data need to be passed to downstream using Kafka connector which is consumed by downstream datalake for further processing. When data is pushed into Kafka it will remain encrypted and same encrypted data will land into the datalake, as encryption and decryption is happening at the driver level.
If Kafka connector supports CFLE encryption/decryption this would work seamlessly
4 votes -
Ignore heartbeats-mongodb topic by default
As per KAFKA-208, SMTs can't be applied to the heartbeats-mongodb topic. Users should not have to configure each connector to ignore this topic. Please either ignore this topic by default or provide a command-line switch so it can be ignored.
4 votes -
Get schema validation "feedback" in Kafka Mongo Sink Connector
Objective :
We want to be able to validate that data matches some requirements. We would like to to perform this data validation by adding a JSON schema in Mongo (such as it is described here : https://docs.mongodb.com/manual/core/schema-validation/).
Problem is that current implementation of the current Mongo DB Kafka Sink connector does not implement the required elements to benefit from features brought by this KIP : https://cwiki.apache.org/confluence/display/KAFKA/KIP-610%3A+Error+Reporting+in+Sink+Connectors
So if we define such a validation on Mongo, if a message has a value that does not match the definition, it would not go in the dead letter queue, and the…
4 votes -
MongoDB Sink Connector CDC default handler
I would like to have a default CDC handler that can process data produced from MongoDB Source Connector without Debezium https://docs.mongodb.com/kafka-connector/master/kafka-sink-cdc#cdc-handler-configuration
4 votes -
Kafka source connector once only semantics
Added as a suppport case here : https://support.mongodb.com/case/00634630
When using the connector as a Source, i.e we capture change streams from the Source Mongo DB and stream that to a Kafka endpoint.
Imagine these are updates on financial transactions in mongodb and they are NOT tolerant to
1) missed data and
2) duplicated data
in that order.So, we need to make sure that the Change Streams that we are observing(matching) on, are delivered once and exactly once to the Kafka pipeline. (Blog on the same : https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/). If exactly-once semantics are enabled, it makes commits transactional by default.
…
4 votes -
Retry/reconnect mechanism Mongodb Source Connectors on MongoTimeoutException
The MongoDB Kafka Source and Sink connectors for the data streaming are working fine seamlessly until there are any errors in Kafka Source connectors.
If there is any error occurs, the connectors are not recovered from the timeout exceptions and need to be reposted/restarted connectors.
Exception:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches3 votes -
Multiple clusters in a single mongosqld configuration.
The limitation of supporting multiple clusters in a single mongosqld configuration is now supported or not.
3 votes -
Power BI
The new Certified MongoDB Atlas Connector for Power BI seems to work great for Power BI Desktop, but does not work in Power BI Service as a dataflow and gives the error "An exception occurred: The given data source kind is not supported. Data source kind: MongoDBAtlasODBC. "
2 votes -
Make database clusters available to Grafana as datasource
We recently were trying to use timeseries collection to report some metrics and wanted to use atlas database clusters as datasource in Grafana. There exists a payed plugin from Grafana itself which is unfortunately only usable in conjunction with Grafana Cloud or Enterprise, which we both do not use as we host our grafana instances our selves.
It would be really nice to have the possibility to use Atlas DB clusters as datasource in Grafana.
2 votes
- Don't see your idea?