Allow disabling TTL Monitor (to allow restoring a snapshot)
Problem:
If you are restoring a snapshot to check the data, or to look at old data, and the collection has a ttl index, then a snapshot, once restored, will immediately mark a bunch of records for deletion, thus making it impossible to examine, export, or otherwise use that data.
Solutions recommended by support include a) download the snapshot (which is very large) and load it on a local instance with the ttl monitor disabled. b) open a ticket to ask for the ttl monitor to be disabled on a target cluster, however once complete you also need to ask for the ttl monitor to be enabled, as otherwise "...eventually, the sessions collection may become full". This means that examining backup data now has multiple support delay loops involved.
We need a way to restore a snapshot so that we don't lose data to the ttl monitor, without requiring support involvement.
Possible Proposed Solutions:
- Allow project owners to disable the TTL Monitor for a cluster, OR
- Allow restoring a snapshot to include an option to disable TTL monitor for [ select a duration | v ], OR
- Allow a new cluster, on creation, to be created with the ttl monitor (permanently?) disabled (and some other mechanism clearing the sessions or anything else thats necessary) ,OR
- some other solution to allow marking a restore as "don't allow restored records to be ttl'd out"
- It might also be useful to provide a mechanism for cleaning the sessions while the TTL Monitor is disabled
Why this matters to me:
- We tried to pull data from a backup to calculate a longer history and didn't realize that the data was being deleted while we were trying to export the data, which meant we provided bad info.
- There was no real way for us to restore quickly - best case is a several hour delay after creating a cluster to restore to.
- Without some solution here, relying on snapshots as backups seems really.... bad, and needlessly slow to recover.