Create databases and collections via API thorough Terraform
Create databases (DB) and collections via API thorough Terraform after the cluster has been created in Atlas. This would provide the ease of writing and running it in a single script before any data is loaded.
-
Martin commented
That would be really useful.
-
Spencer commented
I've been using a python script with null_resource buried within a module to create databases and indexes as a work around.
-
Phil Owen commented
Really difficult to make use of the mongodbatlas_event_trigger resource if you want one of type DATABASE and the collection needs to exist, but you can't create the collection using Terraform...
-
Ben Gesoff commented
The documentation for the online archive resource states that a collection must already exist in order to enable the online archive feature for it: https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/online_archive
Therefore I think it's pretty important to support creating a collection from Terraform, otherwise it's impossible to use the online archive resource as part of a declarative infrastructure-as-code configuration. Instead we have to resort to some sort of flakey script in a Terraform "provisioner" which is an antipattern.
-
Spencer Langley commented
This would fill a painful gap in the dev ops our practices that otherwise Atlas serves quite well.
-
John commented
Set up of indices should not be the responsibility of application code. Set up of indices is an infrastructure task. Managing IAC is commonly considered a critical piece.
-
Kareem Zock commented
Creating a database is core functionality. Im surprised that creating a database isn't available.
-
Dave Fournier commented
Creating a database is core functionality. I'm stunned that creating a database isn't available.
-
Frank Ledo commented
This is one of those features that should be expected from an enterprise product.
Not having this creates many challenges with creating search and global shard indexes, since those cannot be created during runtime.
-
Luis A commented
It is a very interesting feature for our organization because we can increase our automation capacity.
-
Ben Morris commented
I think not having this support encourages the antipattern that your blog post talks about:
https://www.mongodb.com/developer/languages/javascript/mongoose-versus-nodejs-driver/Specifically, the recommendation is to use database-level schema validation through the Mongo Node.js driver, rather than application-level models through an ODM like Mongoose.
I'd like to use the recommendation but a prerequisite is terraform support to define the schema validation programmatically for our multiple projects and databases. As it stands, this need a lot of manual work which would take a lot of time and lead to inconsistencies/mistakes so it's a non starter for us unfortunately.
-
AdminZuhair (Admin, MongoDB) commented
Thanks Joel / Srikaanth / Vineet / others. The Terraform MongoDB Atlas Provider as well as the Atlas Administration API (https://www.mongodb.com/docs/atlas/api/atlas-admin-api/) is mainly used to add, edit, or delete administrative objects within Atlas, including projects, users, and database deployments. However, you won't be able to read or write data, including creating Atlas databases and collections and we have no near term plans to enable this.
Instead, suggest you explore the MongoDB Atlas Data API to read and write data, which we recently GA: https://www.mongodb.com/docs/atlas/api/data-api/
Happy to chat more if this does not meet your need, feel free to email me directly at zuhair.ahmed@mongodb.com and we can find time to brainstorm live. Thank you.
-
Joel Carter commented
Seems like there's some questions here surrounding what this might look like from the Mongo team. As an example you could look at this Postgres provider (not a recommendation as I've never used it):
- https://registry.terraform.io/providers/cyrilgdn/postgresql/latest/docs/resources/postgresql_schema
- https://registry.terraform.io/providers/cyrilgdn/postgresql/latest/docs/resources/postgresql_databaseI know Mongo isn't a relational database but there are some schema-related tasks (creation of collections, addition of indexes, etc.) that would be great to manage via Terraform. It becomes even more important to add this for Mongo as there are many tools for schema versioning/migration for relational databases but an absolute absence in the Mongo space. Adding this capability would bring the familiar Infrastructure as Code (IaC) and migration tools from the relational world to the flexible world of Mongo.
-
srikaanth penugonda commented
This feature would help my team a lot as well. Is this work in progress
-
Vineet Saini commented
would be helpful
-
Mostefa commented
My team needs this feature aswell. Is it a wip?
-
Anton commented
At the moment we generate a lot of new DB and collections for our clients. Right now we use TF for create user and set privileges but have no possibilities to create DB and fill collections necessary data. for these purposes we use some executing scripts. Meanwhile Microsoft provide this option. You can find more details here https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/cosmosdb_mongo_collection
-
Hi Ravindra and other here, Currently the Atlas API focuses on administration functions and the authentication to that is via a programmatic key that does not have access to the data layer, since our Terraform provider covers this API it is providing functions inline with the capabilities of the API. However, it would be great to know more around use cases that are driving this request. If you'd like to share/discuss it would be great to talk. Feel free to reply here or reach out to me directly, melissa.plunkett@mongodb.com. -Melissa
-
Ravindra commented
Is there any updates on this one?
-
Siddharth commented
Allow creation of collections and indexes via terraform atlas plugin. The use case is that, when an atlas cluster is being set up, we want to enable the indexes on the collections from the start.