Posting this idea at the request of one of the Jira users. You can find more technical details about this in the Jira issue:

https://jira.mongodb.org/browse/SERVER-61613

The gist of it is that I may have two collections, b2 and b3, that are not distinguishable in the Mongo Shell, like this:

db.b2.find()

[

  { _id: 1, created: ISODate("2021-11-18T23:16:33.149Z") },

  { _id: 2, created: ISODate("2021-11-18T23:16:33.149Z") }

] 

db.b3.find()

[

  { _id: 1, created: ISODate("2021-11-18T22:53:02.113Z") },

  { _id: 2, created: ISODate("2021-11-18T22:53:02.113Z") }

] ```

When I merge each into a collection `pg` with this syntax:

db.pg.aggregate([{$merge: {into: "b3", whenMatched: "merge", whenNotMatched: "fail"}} ]);
```

, it reports a cryptic message:

MongoServerError: $merge failed to update the matching document, did you attempt to modify the id or the shard key? :: caused by :: Performing an update on the path 'id' would modify the immutable field '_id '

The pg collection is just an example that in real life is a an aggregation stage before $merge.

The underlying issue is that b2._id is BSON int and b3._id is BSON double, which is not visible in Mongo Shell, and that Mongo DB server fails to recognize numeric types that would work just fine in other comparisons, like queries.

Please report an error or a warning that _id fields in merged collections are of different numeric types, so it would be clear what the problem is, which will save people a lot of time trying to figure out why the type of _id is being changed.

Given that _id is immutable, people would have to drop the collection and recreate it, which may be quite tricky for large databases. If there was a tool to do that or if the server would recognize compatible numeric values and report the error only when numeric types cannot be coalesced, it would also be helpful.

When we have large size mongo replica set, we may have to take out each nodes for maintenance for few hrs or days.

In that case we have to increase size of Oplog to keep several days of transactions to resync nodes after maintanance.

If the RS headroom falls below oplog window, then we have blow away data and do init sync to add node back after maintanence.

The problem with very large size Oplog, will slow down any Change Stream process. Also it will occupy un neccasary space within collections.

Can mongo offer alternative way to constantly dump oplog data to local file system? Then we can use this data to re-apply transactions, to sync failed nodes, instead of doing init sync. Some thing similar to archichive log mechanism for redo log, Oracle uses.

This will avoid init sync issue for very large replica sets.

Thanks
Rama Arumugam

If we have more shards, and if want to remove few shards (more than 1), we use below command.

db.adminCommand( { removeShard : "Shardname" } )

Ex, if I have Shard 1, 2, 3, 4, 5. want to remove Shard 2 & 5.

I want to remove one shard at a time to minimize impact to users , then want to remove, say Shard 2, then want to remove Shard 5.

If we do this, some chunks from Shard#2 also get moved to "Shard#5", which is suppose to removed later. This causes Shard#5 chunk size increase. Then takes more time.

Offcourse we can remove more than 1 shard in parellel (from version 4.4 onwards), but it may cause performance issue since balancer will be busy migrating chunks.

So, it will be good, mongo an add new command, "mark the shards to be removed". No new chunks should move to these shards. Then we can remove 1 shard at time, to reduce performance impact.

Thanks
Rama

I haven't tracked down this functionality yet, so excuse me if it is already implemented.

As a developer and system administrator I came to a dilemma how to prevent myself to make changes to documents.

I can limit myself as a programmer, but as a administrator I can always logon to console and make changes ''by hand''.

I was thinking of read-only field type, which once set, could not be updated nor removed from document (although backup and restore is the first problem which comes to mind). This limit should be set on a database level. Read-only field could typically hold some computed hash value, which would guaranty that the field or document was not changed in any way.

There could also be collections with read-only documents. For example when ''read-only-document'' field is set, database would prevent any updates (deletes) to document. Although more problems can arise here. Like how to remove outdated documents or move them to archive collection.

by
TheR

The syntax for joins that bring back multiple documents from foreign collections is very straightforward and yields exactly what one would expect, but simple joins that are bread and butter in SQL require very convoluted and expensive to run syntax.

Consider a product database that has products, categories and reviews collections. Each product has a unique category and may have multiple reviews. Getting all reviews in an aggregation is very straightforward (top stage), but getting categories, similar to SQL, is as convoluted as it gets (bottom stage).

db.products.aggregate(
[
    //
    // Document aggregates naturally aggregate foreign documents
    // into the primary document.
    //
    {$lookup: {
      from: "reviews",
      localField: "_id",
      foreignField: "product_id",
      as: "reviews"
    }},

    //
    // A simple join that is supposed to bring a couple of fields
    // into the primary document translates into this convoluted
    // syntax just to get those two fields.
    //
    {$lookup: {
      from: "categories",
      let: {category_id: "$category"},
      pipeline: [
        {$match: {
            $expr: {$eq: ["$_id", "$$category_id"]}
        }},
        {$project: {
            _id: 0,
            cat_name: "$category",
            cat_description: "$description"
        }}
      ],
      as: "category_details"
    }},
    {$replaceRoot: {
      newRoot: {
        $mergeObjects: [
          "$$ROOT", {$arrayElemAt: ["$category_details", 0]}
        ]
      }
    }},
    {$project: {
        category_details: 0
    }}
])

Notice that keeping category.category and category.description in products, the noSQL-way, isn't an option because updating 10000's of products when category or description change is just not a good idea any way one looks at it. Denormalization is useful in many noSQL scenarios, but this isn't one of them.

Having some simple join syntax that would identify the joined collection as a 1-to-1 join and would bring field names and values as if they existed in the primary document, possibly aliased via a projection-like syntax, would be extremely helpful in keeping aggregations readable, keeping data reasonably normalized, and would run much faster because it would be just multiple parallel BSON append calls when constructing aggregation results documents.

Data for the query:

products:

{"_id":{"$oid":"6140e6c6dfe7b9d6e597d07e"},"name":"Product A","category":1,"price":111}
{"_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"name":"Product B","category":2,"price":222}
{"_id":{"$oid":"6140e6c6dfe7b9d6e597d080"},"name":"Product C","category":1,"price":333}

categories:

{"_id":1,"category":"Clothing","description":"All kinds of clothing"}
{"_id":2,"category":"Electronics","description":"Cool gadgets"}

reviews:

{"_id":{"$oid":"6140e7cadfe7b9d6e597d08a"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07e"},"review":"Product A is great","rating":4}
{"_id":{"$oid":"6140e813dfe7b9d6e597d090"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"review":"This product is horrible","rating":1}
{"_id":{"$oid":"6140e8eff4ff76778b48ba88"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"review":"This product is not the best, but usable","rating":2}