Hi Shameek,

Thank you for taking your time to respond with a lengthy answer.

With the information you provide it's to some extend understandble to why this ticket is a "Can't Do".

Nonetheless, my hope from this discussion was that there might be a chance (and this would be my suggestion here) to see a branched alpha version with an increased document size to e.g. 32 MB so that your customers can try it out and see what breaking changes would occur.
IF step 1 worked out well, and many devs out there would already enjoy that huge gain, the document size could be made configurable (within a reasonable range of sizes) in a second, much later step.

Please let us know if it would technically be possible to increase to 32MB or if developers with the requiremend of larger documents than 16MB will have to work with that size restriction for an undefined long time.

Kind regards from Germany!
Marcel

Large change, yes. The rest, no. You've made the change before, the only thing stopping you is will. All of these other issues are solvable, if you want to solve them. You do not. Other databases have made such breaking changes to their protocols simply by supporting both the old and new protocols for a period, and new features require the new protocol. This isn't hard (though the implementation is anything but trivial). The idea that you're pitching these excuses *to a group of developers who have to solve hard problems like this before lunch on a daily basis* is quite patronising and tells us a lot about what you think of us.

I'll need to figure out how to unsubscribe from this because we will be moving from Mongo to something that does support this ("Postgres") in the foreseeable future. This ship has sailed.

If we're going to have to work around a document database's limitations by being relational, we definitely should be using a relational database which is going to handle this properly.

Good luck to those that remain, you have my condolences.

At this time, MongoDB is not planning on increasing or making its document size limit configurable. This is due to fact that this large, customer-breaking change presents a significant element of risk for us (e.g. validations customers are used to, drivers expecting documents to be a certain size, the current size limit of the MongoDB Wire Protocol being 48MB, etc.). At this time, we are focused on spending our time improving other critical facets of the MongoDB database.

I understand this may be frustrating as with certain use cases, having a configurable document size limit may be desirable. However, we have seen most use cases successfully achieve their same goal by working with a number of our workarounds. Please contact MongoDB Support or engage in the MongoDB Community Forums for more in-depth explanations and examples.

This feedback has not gone unheard; several products have made improvements to allow for most of the use cases mentioned here. Notably, this includes $changeStreamSplitLargeEvent, which can be used to split events greater than 16MB into smaller fragments.

Thank you all for taking the time to leave this feedback and we apologize for the lack of activity on this ticket. Please reach out to me directly if there is further feedback or inquiry.

I wonder if there's a better way to reach out to the developers.

Obviously our voices are being ignored for years. It's a real shame actually.

Mongodb team changed the document size to 16 MB in 2009; since then, compute power has gone up quite a bit, and 64MB should be very much doable.

I think one thing has been overlooked in this conversation and that is $graphLookup. I work in the media industry and we have a fairly complex hierarchical model. $graphLookup is crucial to handle swift retrieval of related and sizeable portions of of our data. Our documents are relatively small, probably around 10kb max, but performing the lookup essentially creates a massive document which can easily exceed the 16MB limit. We cannot even limit the data returned during the process, in an effort to artificially reduce the size. Now I do not know this for sure, but I would take an educated guess that the $graphLookup facility was added after the 16MB limit was defined but can easily threaten it and therefore maybe the limit should have been reviewed at the point $graphLookup was introduced in order to effectively support the new functionality?

Is this being worked on ?

This feature implementation is both critical and crucial!
Implementing a feature that allows administrators to configure the maximum BSON document size limit is the only way to go forward.

When using Splunk to import Active Directory groups, users, and computers, this limitation causes users to not see all their data beyond some tens of thousands of objects. Many larger AD environments have more than this. They must reconfigure inputs to pull in one piece at a time, which causes delays and frustration. If MongoDB would increase this limit to 64MB, this could help modern IT shops manage their data more easily with Splunk. This is just one more use case to add to the list of reasons to increase this limit from 2009 or 2011.

Limit is good, 16MB as a default is ok, 64MB is even better but to make MongoDB flexible and suitable for even more use cases, limit should be configurable.

How can you still think 16MB are enough in 2023? You defined this limit in 2011.
We have more data, larger document, sensors, socials, please raise the limit to 64MB.

I see a lot of "you're doing it wrong" comments in this thread: https://www.mongodb.com/community/forums/t/increase-max-document-size-to-at-least-64mb/124424/28 from Asya Kamsky. Unfortunately 5G configuration data cannot be broken out into separate collections/documents. It absolutely HAS to be in a single document. If the document size cannot be increased then unfortunately my organization WILL have to go to Couchbase to store our 5G data.

Is this being worked on? any update?

This limitation is a serious risk for a lot of production systems.
The only way to safely stay under the document size limit is to use MongoDB as a relational database: wherever a record might contain any non-fixed size lists, you have to put those in separate documents, as once in a million times everything _will_ grow longer than you've expected.
And once any of the document reaches 16MB, your production system is broken. You can patch up your system, to split that new thing that became too long in the "offending" document, but that is not operation/support, it's a never-ending crisis managent.
And once you've split them all up, you will realise that you are using MongoDB as a relational database, just without rdbms features. It just defeats the purpose.

Saying that 16MB is the maximum _recommended_ document size is all fine, a design can easily be built around it, ensuring that the wast majority of documents is going to fit within that size. And anything above that will be significantly slower. Once in a million, it can be slow all right. But if a document above the limit can not be handled at all with the system without significant development, that renders the system highly vulnerable and completely intolerant to a lot of unexpected events (and to DOS attacks). Which very significantly limits it's usability.

Why this limit, leave it to the customer. I find it incredibly hard as the crux of reason to use mongo has been destroyed. The worst part is the same limit applies to aggregation results. if you are documents are close to 16MB and you wanted a lookup, how can the output be less than 16mb.

This one issue is the primary if not sole reason why we're looking to move from Mongo to Postgres. Yes, we can change our data model to work within the 16MB limit. But that means modifying significant amounts of existing code. As long as we're going to modify significant amounts of corporate-wide code to handle this, we may as well move to a relational model which would entirely eliminate all size restrictions.

If this were bumped up to 32MB, it would likely stave off any move we might contemplate for years, long enough that Mongo could move again up to 64MB. But the reality is that as long as there is a limit, people will hit it, even sparingly, and if Mongo cannot handle it, people will be forced to make changes to work around this limit, and if we have to make changes, then it's an opportunity to revisit the costs and benefits of Mongo vs other options.

You can put a disclaimer that 16 MB is recommended but Customer should be able to configure this value to whatever he wants 64 MB, 128mb or whatever, knowing the performances implications. If We, the customers are ready to compromise on other things like performance what problem MongoDb should have?

Last year only, we started using mongo db. We were super happy in the begining. But almost after 3 months, We came to know this limitation and one of our Collection is at risk due to this limitation.
We have a Position document. multiple Employees can be assigned to this Position. In a large org there could be a million employees. All the changes in Position document as well as in nested document of employees array are revisioned.
I don’t wanna go in much details. But, 16 MB seems super scary as our world is growing more and more data. You can not put a limit. Rather than asking your customers counter questions why do we need more than 16 mb, please make it configurable and put a recommendation of 16 mb. Otherwise, we are left with no choice but to decommision mongoDb from our tech stack.

The discussion that @FeldrinH linked for us (and which took over a year) has plenty of use cases in which we explain the necessity to adjust the document size.

Adding a configuration parameter to change the BSON document size (e.g. bigger than 16 MB) would be extremely appreciated by many developers of existing or planned applications with MongoDB.

Some relevant use cases can be found in the related forum discussion:
https://www.mongodb.com/community/forums/t/increase-max-document-size-to-at-least-64mb/124424/27

Hey Albert, thanks for sharing this recommendation! What kind of scenarios would having this greater document size limit help you?

The reason MongoDB maintains a max doc size is so to ensure a large amount of documents of a large size don't induce a resource bottleneck - either with an excessive amount of RAM or, during networking, an excessive amount of bandwidth.

Through an improved data modelling, our users who have needed larger document size limits have been able to break up these large documents into smaller ones while still maintaining their use cases.