I'd like to have a scheduled mongodump backup option
I'm coming from mLab and I'm used to having an option in my backup strategy that includes a mongodump backup as well as what Atlas calls a CPS (cloud provider snapshot). The latter is actually a snapshot of the database directory at the filesystem level, which is great for instant restores within Atlas (same or different cluster). However, the mongodump backup (from mLab as an example) allows me to backup on a regular schedule and download those backups for restores locally and elsewhere outside of Atlas.
Yes, I can run mongodump operations myself from a system outside of Atlas, but having those mongodump archives available within a rolling window is really great to quickly grab a backup and examine offline or in another environment entirely, with no additional management or systems from me to do so.
Ryan Mitchell
shared this idea
We have released the ability to export a snapshot to your own S3 bucket. More details here —> https://docs.atlas.mongodb.com/backup/cloud-backup/export/
-
Nicolai
commented
Any plans to support Azure Blob Storage?
-
Jecsham
commented
Finally the development of this tool has started :D
-
Kristofer Pervin
commented
Yes, I came here to suggest this. Tried downloading one of the scheduled backups, and found that it was a raw database. Working with Meteor and coming from mLab, I swap between my development database and prod database frequently, so be able to mongorestore is very crucial for me.
-
Kamalakannan
commented
Yes. This is one of the useful future. Currently, there is a need to another client to connect and trigger the dump
Also provide option to upload into external system like AWS S3.
-
Andre
commented
This would be nice, its an extra step to restore for us since you have to launch a matching MongoDB Server and then dump the data just to make it usable
We created a k8s pod that we run that loads it after we have extracted it to a PVC:
```
---
apiVersion: v1
kind: Pod
metadata:
name: mongo-s3-restore-pod
namespace: mongo-s3-backup
spec:
containers:
- name: mongo-s3-restore-pod
image: "mongo:3.6-xenial"
imagePullPolicy: Always
envFrom:
- configMapRef:
name: mongo-s3-backup-config
volumeMounts:
- name: snapshot-volume
mountPath: /snapshots
- name: snapshot-volume
mountPath: /data/db
subPath: "path/to/dump"
restartPolicy: OnFailure
volumes:
- name: snapshot-volume
persistentVolumeClaim:
claimName: mongo-s3-restore-pod---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mongo-s3-restore-pod
namespace: mongo-s3-backup
labels:
app: mongo-s3-restore-pod
spec:
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: 750Gi```
Also we have an entire script that runs daily and archives the backups to our own S3, really would not like to have to do this as its prone to errors and we manage it.
-
Ryan
commented
That's actually a good thing to know. I haven't had the opportunity to play with the Data Lake service yet - I'll be sure to do that as soon as I can.
-
Definitely helps, thanks! BSON has another advantage by the way: it can be read (if persisted in S3) directly from our Atlas Data Lake service.
-
Ryan
commented
Thanks for the update Andrew. Yes, I know the API and the UI both allow me to download backups, but they are the filesystem-level backups rather than BSON format. BSON is valuable so I can easily restore into MongoDB instances via mongorestore rather than having to copy a filesystem dump over which requires more work with something like Docker running locally. Additionally, you can create an archive straight out of mongodump that can be restored from that same single archive file with mongorestore and it's very easy to get that moving regardless of where the server is you're restoring into.
That help clarify?
-
Hi Ryan thanks for sharing this. I do want to make sure that you're aware that you can download your MongoDB Atlas backups through the UI or via the API any time.
I will note that when doing so, the backup file format will be a MongoDB data directory rather than BSON (e.g. the output of mongodump) so it's not strictly apples to apples but wanted to make sure you knew about this option.
Is having the data in BSON an important attribute for your workflow?
Cheers
-Andrew
