Data Federation and Data Lake
8 results found
-
Connect to Atlas Data Lake using Private Link
I need to connect to my Atlas Data Lake through Private Link
73 votesHello,
I’m thrilled to announce the general availability of AWS PrivateLink for Atlas Online Archive and Atlas Data Lake. You can now set up a private endpoint through the MongoDB Atlas UI or API to connect to your live and historical data, wherever it resides.
Please check out the Online Archive and Data Lake documentation for more information on getting started. Have questions? Visit our developer forums to get advice from other members in the MongoDB community.
Best,
Benjamin Flast
-
Deploy Atlas Data Lake From Terraform
Add the necessary resource types to the mongodb_atlas Terraform provider to enable Data Lake to be deployed by Terraform.
22 votesHello,
I’m happy to let you know you can now manage your Atlas Data Lakes using the terraform provider.
https://registry.terraform.io/providers/mongodb/mongodbatlas/latest
Best,
Ben
-
Support MongoDB Charts
We are using Data Lake and we would like to use MongoDB Charts for visualising our database.
20 votesMongoDB Charts can now query Atlas Data Lake. We’ll be continuing to work on and improve the integration in the near future as well. Please reach out if you have any questions or feedback.
Best,
Ben
-
Allow export of data from database to lake
It would be nice if we had a pipe which moves data, by query, from database to the lake with below options:
1. delete after copy
2. separate/chunk files
3. auto name formatting according to fields.this could be even awesome if it had both ways.
7 votesWe’ve released this functionality!
It is called “Online Archive” and can be found in the Atlas cluster card as a tab towards the right side. It is currently a Beta feature as we collect additional user feedback.
Please reach out with any questions or suggestions!
-
Expose Logs of my Data Lake Queries
I need to view past queries made on the platform to see successes and failures for trouble shooting purposes.
I also need to see who made queries or changes our configuration files for auditing purposes.
5 votesYou can now access Query Logs from your Atlas Data Lakes. Please let us know if you have any feedback or questions regarding the use of this feature.
Best,
Ben
-
Update storage config in the UI
I'd like to have the option to update the storage config in the UI instead of having to run a command in the DB.
This is more for when you first start using data lake and need to change the config a lot while testing.
3 votesWe have released the capability to edit the storage config through a JSON Editor in the Atlas UI.
Please let us know if you have any feedback or would like to see additional functionality around this domain!
-
Provenance Options in the Storage Configuration
It would be helpful to have options in the Storage Configuration that allow you to see what the source of a document retrieved through federation was.
For instance, it would be helpful if there was a simple way to know that a specific document came from the Collection Foo and Database Bar.
2 votesWe now have the ability to set provenance in the Storage Configuration to contain the source of documents.
-
Combine collections from various databases into one database
It would be helpful to be able to combine collections with the same name from various databases into a single "global" database in the Federated Database Storage Configuration.
Imagine you have a multi tenant application where you have DatabaseCustomerFoo and DatabaseCustomerBar, and each of them has a collection called "events". It would be great if you could easily combine those (at scale) through the storage configuration, so in this instance you would have a database: DatabaseCustomerGlobal and a collection: events which federated over both DatabaseCustomerBar.events and DatabaseCustomerFoo.events.
1 voteWe've added the capability to dynamically combine name spaces in underlying clusters using the Database Regex in the Storage Configuration for Atlas Data Federation.
- Don't see your idea?