Expose auto cluster scaling options via Terraform
We would like the ability to configure an auto scaling cluster via Terraform. Currently we need to modify our cluster after its creation via terraform to setup auto scaling parameters.
Thank you to everyone who provided input here and directly. We are about to release the provider, v0.6.0, which exposes the autoscale attributes for clusters and includes a recommendation to use a lifecycle block to prevent Terraform and Atlas from conflicting (this will be included in the clusters documentation that will come out w/ the release). Please continue to provide suggestions here or reach out directly to melissa.plunkett@mongodb.com. Thank you!
-
Thank you all for the great feedback and suggestions! Iago, that is indeed a nice work around for that use case. It works great when it's a increase by count but in this case the value of provider_instance_size_name would change. I have a feeling Ward's suggestion of ignore_changes may be the best, basically once all set if provider_instance_size_name changes it can be set to be ignored. I've passed this onto our developers and we are going to work in implementing this in the best way possible.
-
Ward commented
Not as clean as what Iago suggested but it might be less work to implement this by making use of Terraform's ignore_changes: https://www.terraform.io/docs/configuration/resources.html#ignore_changes
-
Iago commented
Google Cloud provider has a very similar problem in the resource `google_container_node_pool`. The node pool can be automatically scaled by specifying the `autoscaling` config.
Their solution was very simple: they'd just added an option called `initial_node_count`. The the number of nodes is changed, the resource is not updated. If the user wants to enforce the number of nodes, it can be done by simply setting `node_count` directly.
For us, that works pretty well.
I would love to see this feature getting implemented. -
Jonas commented
I'd really like this
-
Jack commented
Is it possible for the provider to determine if Terraform is being run in an update mode and, if so, simply ignore the instance size property if the cluster autoscaling option was previously set to true and is not being changed?
-
Appreciate the questions - we have this item on our roadmap but no estimated ETA until we can either scope this to it's most simplest implementation (noted below) or schedule more time to experiment on if and how autoscaling will work with Terraform and/or how we could modify Atlas to better work with an IaC tool and still allow autoscaling. This is where those voting can help us with feedback on your expectations and needs.
As noted by the comment from Valery, if an Atlas user sets autoscaling via terraform, you'd need some way to validate that the refreshed values are within the min/max range or Terraform would see the configuration as out of sync with the starting size in the terraform config. Our developers are looking into ways to potentially do this but it starts to verge more on configuration management than IaC. Our assumption is that this is the ideal - to be able to either have Terraform somehow know autoscaling is okay, either by the way we do it in the provider or the way we expose these options in Atlas itself. Is this a correct assumption with those voting?
Another way we could do this is just expose the autoscale options in the terraform cluster resource and document that you'd need to likely modify your tf config on refresh to match the new size Atlas automatically set it to or else when you deploy Terraform will revert it. Would this be an acceptable stop-gap to have it sooner, but perhaps not in an ideal state?
-
James commented
Is there a timeline on getting this into the provider?
-
Valery commented
I guess at refresh provider need to verify that existing mongodb cluster size is inside MIN-MAX range values that can exists in provider arguments.
Rest of things are usual API implementation.
@melissa is there any ETA for this feature?Thank you
-
Valery commented
Hi
At this moment it can be done with provisioner "local-exec" and there CURL command to atlas API to enable instances autoscaling. I succeed with this.
And then (after enabled instance autoscaling, and autoscaling event occurs) instance type can be different from type that is in tfplan and then terraform tries to change instance type back at "apply", which can be harmful and then this became an issue - to use terraform with mongodb atlas if you need mongodb instance autoscaling. -
Appreciate the feedback and wanted to post that we have this item on our work list to research. We need to determine the best way support setting the auto scale options in a way that will work with Terraform. Basically if the system auto scales and one has defined a specific size in Terraform then Terraform's default action is to set that to match the Terraform config. It would be great to know how you would ideally like this feature to work if you've submitted here - feel free to comment or if you'd prefer email me directly at melissa.plunkett@mongodb.com. I have a few ideas but I'd like to know what you, our users, think would be most ideal to you.
-
Daniel commented
This should be a priority, that would help a lot to use this autoscale and not having issues with Terraform because of it. There is no workaround to not have conflicts between them.