Raise maximum BSON document size bigger than 16 MB
Per https://www.mongodb.com/docs/manual/reference/limits/#:~:text=The%20maximum%20BSON%20document%20size,MongoDB%20provides%20the%20GridFS%20API, maximum BSON document size is 16 MB. I would like to request this to support bigger sizes like 32 MB, 64 MB or even bigger.
Limit is good, 16MB as a default is ok, 64MB is even better but to make MongoDB flexible and suitable for even more use cases, limit should be configurable.
How can you still think 16MB are enough in 2023? You defined this limit in 2011.
We have more data, larger document, sensors, socials, please raise the limit to 64MB.
I see a lot of "you're doing it wrong" comments in this thread: https://www.mongodb.com/community/forums/t/increase-max-document-size-to-at-least-64mb/124424/28 from Asya Kamsky. Unfortunately 5G configuration data cannot be broken out into separate collections/documents. It absolutely HAS to be in a single document. If the document size cannot be increased then unfortunately my organization WILL have to go to Couchbase to store our 5G data.
Is this being worked on? any update?
This limitation is a serious risk for a lot of production systems.
The only way to safely stay under the document size limit is to use MongoDB as a relational database: wherever a record might contain any non-fixed size lists, you have to put those in separate documents, as once in a million times everything _will_ grow longer than you've expected.
And once any of the document reaches 16MB, your production system is broken. You can patch up your system, to split that new thing that became too long in the "offending" document, but that is not operation/support, it's a never-ending crisis managent.
And once you've split them all up, you will realise that you are using MongoDB as a relational database, just without rdbms features. It just defeats the purpose.
Saying that 16MB is the maximum _recommended_ document size is all fine, a design can easily be built around it, ensuring that the wast majority of documents is going to fit within that size. And anything above that will be significantly slower. Once in a million, it can be slow all right. But if a document above the limit can not be handled at all with the system without significant development, that renders the system highly vulnerable and completely intolerant to a lot of unexpected events (and to DOS attacks). Which very significantly limits it's usability.
Why this limit, leave it to the customer. I find it incredibly hard as the crux of reason to use mongo has been destroyed. The worst part is the same limit applies to aggregation results. if you are documents are close to 16MB and you wanted a lookup, how can the output be less than 16mb.
This one issue is the primary if not sole reason why we're looking to move from Mongo to Postgres. Yes, we can change our data model to work within the 16MB limit. But that means modifying significant amounts of existing code. As long as we're going to modify significant amounts of corporate-wide code to handle this, we may as well move to a relational model which would entirely eliminate all size restrictions.
If this were bumped up to 32MB, it would likely stave off any move we might contemplate for years, long enough that Mongo could move again up to 64MB. But the reality is that as long as there is a limit, people will hit it, even sparingly, and if Mongo cannot handle it, people will be forced to make changes to work around this limit, and if we have to make changes, then it's an opportunity to revisit the costs and benefits of Mongo vs other options.
You can put a disclaimer that 16 MB is recommended but Customer should be able to configure this value to whatever he wants 64 MB, 128mb or whatever, knowing the performances implications. If We, the customers are ready to compromise on other things like performance what problem MongoDb should have?
Last year only, we started using mongo db. We were super happy in the begining. But almost after 3 months, We came to know this limitation and one of our Collection is at risk due to this limitation.
We have a Position document. multiple Employees can be assigned to this Position. In a large org there could be a million employees. All the changes in Position document as well as in nested document of employees array are revisioned.
I don’t wanna go in much details. But, 16 MB seems super scary as our world is growing more and more data. You can not put a limit. Rather than asking your customers counter questions why do we need more than 16 mb, please make it configurable and put a recommendation of 16 mb. Otherwise, we are left with no choice but to decommision mongoDb from our tech stack.
The discussion that @FeldrinH linked for us (and which took over a year) has plenty of use cases in which we explain the necessity to adjust the document size.
Adding a configuration parameter to change the BSON document size (e.g. bigger than 16 MB) would be extremely appreciated by many developers of existing or planned applications with MongoDB.
Some relevant use cases can be found in the related forum discussion:
AdminShameek (Admin, MongoDB) commented
Hey Albert, thanks for sharing this recommendation! What kind of scenarios would having this greater document size limit help you?
The reason MongoDB maintains a max doc size is so to ensure a large amount of documents of a large size don't induce a resource bottleneck - either with an excessive amount of RAM or, during networking, an excessive amount of bandwidth.
Through an improved data modelling, our users who have needed larger document size limits have been able to break up these large documents into smaller ones while still maintaining their use cases.