AWS and GCP start with 2300-3000 IOPS for M10+ instance from the smallest disk size (8Gb), whereas on Azure we get 120 IOPS with 8Gb, 240 with 64Gb, 500 IOPS with 128Gb ... nowhere near to what AWS/GCP offer (120 - 500 IOPS for a database server is nothing!).

So if I take a bigger storage (512Gb for 2300 IOPS) to be on par with AWS/GCP then there is a dramatic price difference of your MongoDB Atlas offering for Azure compared to AWS/GCP - it gets 2 times more expensive!

I understand that there is a dependency on the cloud provider, however I am sure you have your direct channels to Microsoft ...

When upgrading the MongoDB server community edition from a very old version to the newest version one cannot skip installing intermediate versions, e. g. version 3.6 -> 4.4 does not work because the database files cannot be auto-migrated and the FeatureCompatibilityVersion is too low on 3.6 (or not existent).

Instead we have to install 3.6 -> 4.0 -> 4.2 -> 4.4 and execute db.adminCommand( { setFeatureCompatibilityVersion: "..." } ) appropriately in between installing each new version.

I propose creating an additional installer package for Linux which takes care of all that to migrate the internal database structure from the installed version to the latest but recent version available to apt - including setting FeatureCompatibilityVersion. This "upgrade-package" would contain all files necessary to upgrade from 3.6 -> 4.2 (in my example) so that the upgrade from 3.6 to 4.4 would not fail.

For enforcing uniqueness in a sharded cluster, the officially recommended approach provided here https://docs.mongodb.com/manual/tutorial/unique-constraints-on-arbitrary-fields/#std-label-shard-key-arbitrary-uniqueness is simplistic and in production environment it brings non-trivial amount of work. Some considerations:

1. Ephemeral issues might cause inconsistencies between the two collections (for example, unique index collection update succeeded but not the main collection) and make some unique keys not useable.
2. There are many changes needed (we're using ORM Mongoose, there are many hooks for it to change) for enforce this universally.

What we ended up doing is to use distributed ephemeral locks (a TTLed MongoDB collection) to lock on the unique keys before adding to the collection. Getting this correct / verifying its correct is non-trivial.

The syntax for joins that bring back multiple documents from foreign collections is very straightforward and yields exactly what one would expect, but simple joins that are bread and butter in SQL require very convoluted and expensive to run syntax.

Consider a product database that has products, categories and reviews collections. Each product has a unique category and may have multiple reviews. Getting all reviews in an aggregation is very straightforward (top stage), but getting categories, similar to SQL, is as convoluted as it gets (bottom stage).

db.products.aggregate(
[
    //
    // Document aggregates naturally aggregate foreign documents
    // into the primary document.
    //
    {$lookup: {
      from: "reviews",
      localField: "_id",
      foreignField: "product_id",
      as: "reviews"
    }},

    //
    // A simple join that is supposed to bring a couple of fields
    // into the primary document translates into this convoluted
    // syntax just to get those two fields.
    //
    {$lookup: {
      from: "categories",
      let: {category_id: "$category"},
      pipeline: [
        {$match: {
            $expr: {$eq: ["$_id", "$$category_id"]}
        }},
        {$project: {
            _id: 0,
            cat_name: "$category",
            cat_description: "$description"
        }}
      ],
      as: "category_details"
    }},
    {$replaceRoot: {
      newRoot: {
        $mergeObjects: [
          "$$ROOT", {$arrayElemAt: ["$category_details", 0]}
        ]
      }
    }},
    {$project: {
        category_details: 0
    }}
])

Notice that keeping category.category and category.description in products, the noSQL-way, isn't an option because updating 10000's of products when category or description change is just not a good idea any way one looks at it. Denormalization is useful in many noSQL scenarios, but this isn't one of them.

Having some simple join syntax that would identify the joined collection as a 1-to-1 join and would bring field names and values as if they existed in the primary document, possibly aliased via a projection-like syntax, would be extremely helpful in keeping aggregations readable, keeping data reasonably normalized, and would run much faster because it would be just multiple parallel BSON append calls when constructing aggregation results documents.

Data for the query:

products:

{"_id":{"$oid":"6140e6c6dfe7b9d6e597d07e"},"name":"Product A","category":1,"price":111}
{"_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"name":"Product B","category":2,"price":222}
{"_id":{"$oid":"6140e6c6dfe7b9d6e597d080"},"name":"Product C","category":1,"price":333}

categories:

{"_id":1,"category":"Clothing","description":"All kinds of clothing"}
{"_id":2,"category":"Electronics","description":"Cool gadgets"}

reviews:

{"_id":{"$oid":"6140e7cadfe7b9d6e597d08a"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07e"},"review":"Product A is great","rating":4}
{"_id":{"$oid":"6140e813dfe7b9d6e597d090"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"review":"This product is horrible","rating":1}
{"_id":{"$oid":"6140e8eff4ff76778b48ba88"},"product_id":{"$oid":"6140e6c6dfe7b9d6e597d07f"},"review":"This product is not the best, but usable","rating":2}


We frequently denormalise either full documents or subsets to different documents in order to speed up reading, create indexes or paginate/sort on fields.

Consider a user collection and a task collection, if a task can be assigned to the user, it makes sense to just put the user document on the task they are assigned. But an update to a user now requires you to update the user both in the user collection and all tasks with that user in the tasks collection.

This can be achieved but does introduce some complexity, however the introduction of updates using aggregation pipelines could eliminate this problem if they supported the $lookup operator.

So we could achieve the desired result using two simple operations

db.collection('users').updateOne({id: '1234'}, {$set: ...})
db.collection('tasks').updateMany({"user.id": '1234'}, [
{
$lookup: {
from: 'users',
localField: 'user.id',
foreignField: 'id',
as: 'user'
}
}
]);

This would make it really simple to synchronise data between documents in different collections

I have observed the different profile result with regard to $comment query operator in association with find and update operation respectively as follows;

• find operation shows a comment on the command.filter.$comment and the command.comment field in the system.profile collection.

  op: 'query',
  ns: 'db101.Bets',
  command: {
  find: 'Bets',
  filter: {
  _id: ObjectId("61a9db4b3bd34e4f68fb9abc"),
  '$comment': 'test-dba'
  },
  comment: 'test-dba',
  lsid: { id: UUID("43ebee67-3184-4ede-9cee-ecca7457861a") },
  '$db': 'db101'
  },

• update operation shows a comment only on the command.q.$comment field in the system.profile collection.

  op: 'update',
  ns: 'db101.Bets',
  command: {
  q: {
  _id: ObjectId("61a9db4b3bd34e4f68fb9abc"),
  '$comment': 'test-dba'
  },
  u: { '$set': { odds: 0.5 } },
  multi: false,
  upsert: false
  },

Posting this idea at the request of one of the Jira users. You can find more technical details about this in the Jira issue:

https://jira.mongodb.org/browse/SERVER-61613

The gist of it is that I may have two collections, b2 and b3, that are not distinguishable in the Mongo Shell, like this:

db.b2.find()

[

  { _id: 1, created: ISODate("2021-11-18T23:16:33.149Z") },

  { _id: 2, created: ISODate("2021-11-18T23:16:33.149Z") }

] 

db.b3.find()

[

  { _id: 1, created: ISODate("2021-11-18T22:53:02.113Z") },

  { _id: 2, created: ISODate("2021-11-18T22:53:02.113Z") }

] ```

When I merge each into a collection `pg` with this syntax:

db.pg.aggregate([{$merge: {into: "b3", whenMatched: "merge", whenNotMatched: "fail"}} ]);
```

, it reports a cryptic message:

MongoServerError: $merge failed to update the matching document, did you attempt to modify the id or the shard key? :: caused by :: Performing an update on the path 'id' would modify the immutable field '_id '

The pg collection is just an example that in real life is a an aggregation stage before $merge.

The underlying issue is that b2._id is BSON int and b3._id is BSON double, which is not visible in Mongo Shell, and that Mongo DB server fails to recognize numeric types that would work just fine in other comparisons, like queries.

Please report an error or a warning that _id fields in merged collections are of different numeric types, so it would be clear what the problem is, which will save people a lot of time trying to figure out why the type of _id is being changed.

Given that _id is immutable, people would have to drop the collection and recreate it, which may be quite tricky for large databases. If there was a tool to do that or if the server would recognize compatible numeric values and report the error only when numeric types cannot be coalesced, it would also be helpful.