mongoexport - generate multiple compress files as output
This is a feature request for https://github.com/mongodb/mongo-tools/blob/master/mongoexport/main/mongoexport.go
- similar to s3.format.maxFileSize parameter in ADL $out, cut a new file every xxxMB
- similar to s3.format.columnCompression parameter in ADL $out, use gzip/zstd to compress the output BSON or JSON
- support write to locally-mounted file system (EBS/NFS) and cloud object store S3/GCS
The similar logic has probably implemented in the ADL $out codebase already, we just need to port them over to mongoexport
1
vote
Eric
shared this idea