9 results found

1. Should be possible to configure profiling output destination

   When enabling database profiling the output is sent to both the system.profile collection and system logs. Logging to a capped collection is fine but spamming the logs on disk is not good.

   We have a need to be able to react to changing query patterns quickly, so we have profiling enabled in production on a busy system and we do real-time analysis on the the system.profile collection. This works fine and the performance hit is acceptable but our system logs on disk grows a lot.

   Please make it possible to configure if profile logging should go to disk, collection or both. Ideally with individual levels.

   When enabling database profiling the output is sent to both the system.profile collection and system logs. Logging to a capped collection is fine but spamming the logs on disk is not good.

   We have a need to be able to react to changing query patterns quickly, so we have profiling enabled in production on a busy system and we do real-time analysis on the the system.profile collection. This works fine and the performance hit is acceptable but our system logs on disk grows a lot.

   Please make it possible to configure if profile logging should go to disk, collection or… more

   8 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

2. an aggregation stage to load data to DRAM for the fields that are only requested

   When we use $project stage, it loads whole document from disk to memory (if not in working set). Because of this, When we create a data model, We have to create separate collection if the field is not required in frequent access of data. Creating a view is an option, but what if $project itself or $project with some argument in it, or a new pipeline stage or operator gets introduced which fetches the data from disk only for the fields specified instead of loading whole document.

   With memory mapped file, retrieving fields specified alone would not be simply possible but just thinking of a possibility which can be really useful in a case.

   When we use $project stage, it loads whole document from disk to memory (if not in working set). Because of this, When we create a data model, We have to create separate collection if the field is not required in frequent access of data. Creating a view is an option, but what if $project itself or $project with some argument in it, or a new pipeline stage or operator gets introduced which fetches the data from disk only for the fields specified instead of loading whole document.

   With memory mapped file, retrieving fields specified alone would not be simply possible… more

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

3. Implement $bucket and $group on indexed values with sub-linear runtime

   We noticed that sum $bucket and $group aggregations such as $min, $max, $count are unexpectedly slow even when fully covered by an index, (partially) because the DB scans through the entire index rather than employing optimization approaches such as binary search.

   An example pipeline that should return instantaneous but scans through the entire index (confirmed on v4.4 and v5):
   [
   {
   $match: {
   status: "DELIVERED",
   },
   },
   {
   $group: {
   id: {
   status: "$status",
   },
   min: {
   $min: "$modifytime",
   },
   },
   },
   ]
   with an index { status: 1, modify_time: 1}

   Another example is $bucket (same index):
   [
   {
   $match: {
   status: "DELIVERED",
   },
   },
   {
   $bucket: {
   groupBy: "$creation_ts",
   boundaries: [
   Date(0),
   ISODate("2020-01-01T00:00:30Z"),
   ISODate("2022-01-01T00:00:30Z"),
   ],
   default: "default",
   output: {
   count: {
   $sum: 1,
   },
   status: {
   $first: "$status",
   },
   },
   },
   },
   ]

   We noticed that sum $bucket and $group aggregations such as $min, $max, $count are unexpectedly slow even when fully covered by an index, (partially) because the DB scans through the entire index rather than employing optimization approaches such as binary search.

   An example pipeline that should return instantaneous but scans through the entire index (confirmed on v4.4 and v5):
   [
   {
   $match: {
   status: "DELIVERED",
   },
   },
   {
   $group: {
   id: {
   status: "$status",
   },
   min: {
   $min: "$modifytime",
   },
   },
   },
   ]
   with an index { status: 1, modify_time: 1}

   Another example is $bucket (same index):
   [
   {… more

   6 votes

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

4. Avoid truncating the query on the Atlas profiler or system.profile collection

   Slow running queries that are captured in system.profile collection or on profiler page of Atlas are truncated if the query is too long. As an Application DBA, it would be difficult to analyse the query without figuring out the actual query. The current limitation of command document is 50Kb. Request you to consider this limitation to avoid truncation of queries.

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

5. Build MongoDB with PGO

   I would like to see support for PGO (and even LLVM Bolt) in the upstream. Would be awesome if MongoDB will distribute PGO-optimized binaries, so the users will be able to see an additional performance boost "for free". At least describe to the users somewhere in the documentation, how they could achieve a boost for their own scenarios with PGO.

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   1 comment  ·  Performance  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

6. throttle sessions which use too much resources

   We have different types of applications :
   1. Writer - to load data into mongodb from different data sources
   2. Reader - to read data and display to end user.

   Normally, there is strict SLA for reader , but no SLA (or less restricted) for writers. We want to make sure that writer will not impact reader in case when for some reason a lot of data arrived from external sources. So, we would like to slow down writers for the sake of readers.

   Writers can saturate CPUs and IO, that's why we want an option to leave some room for readers. Normally it's hard to limit on application level because applications can be deployed on many different host and may not communicate to each other.

   We have different types of applications :
   1. Writer - to load data into mongodb from different data sources
   2. Reader - to read data and display to end user.

   Normally, there is strict SLA for reader , but no SLA (or less restricted) for writers. We want to make sure that writer will not impact reader in case when for some reason a lot of data arrived from external sources. So, we would like to slow down writers for the sake of readers.

   Writers can saturate CPUs and IO, that's why we want an option to leave some room… more

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

7. Improve sorting performance

   Sorting always ends up doing a collection scan when the selected index for the find/match does meet the sort requirement. The sort effectively makes the performance worse by 15-25 times for the "matched" dataset which runs into 10s of thousands (not millions) of documents

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

8. Boost the performance of bioinformatic annotation queries

   The documents to be selected look something like this:

   {
   "_id": {
   "$oid": "6272c580d4400d8cb10d5406"
   },
   "#CHROM": 1,
   "POS": 286747,
   "ID": "rs369556846",
   "REF": "A",
   "ALT": "G",
   "QUAL": ".",
   "FILTER": ".",
   "INFO": [{
   "RS": 369556846,
   "RSPOS": 286747,
   "dbSNPBuildID": 138,
   "SSR": 0,
   "SAO": 0,
   "VP": "0x050100000005150026000100",
   "WGT": 1,
   "VC": "SNV",
   "CAF": [{
   "$numberDecimal": "0.9381"
   }, {
   "$numberDecimal": "0.0619"
   }],
   "COMMON": 1,
   "TOPMED": [{
   "$numberDecimal": "0.88411856523955147"
   }, {
   "$numberDecimal": "0.11588143476044852"
   }]
   },
   ["SLO", "ASP", "VLD", "G5", "KGPhase3"]
   ]
   }

   For a basic annotation (https://en.wikipedia.org/wiki/SNP_annotation) scenario, we need such query:

   {'ID': {'$in': ['rs369556846', 'rs2185539', 'rs2519062', 'rs149363311', 'rs55745762', <...>]}}
   , where <...> means hundreds/thousands of values.

   Such query is executed in a few seconds.

   More complex annotation queries:

   {'$or': [{'#CHROM': 1, 'POS': 1499125}, {'#CHROM': 1, 'POS': 1680158}, {'#CHROM': 1, 'POS': 1749174}, {'#CHROM': 1, 'POS': 3061224}, {'#CHROM': 1, 'POS': 3589337}, <...>]}

   {'$or': [{'ID': 'rs149434212', 'REF': 'C', 'ALT': 'T'}, {'ID': 'rs72901712', 'REF': 'G', 'ALT': 'A'}, {'ID': 'rs145474533', 'REF': 'G', 'ALT': 'C'}, {'ID': 'rs12096573', 'REF': 'G', 'ALT': 'T'}, {'ID': 'rs10909978', 'REF': 'G', 'ALT': 'A'}, <...>]}

   Despite the involvement of IXSCAN, they run many hours.

   Please test aforementioned queries thoroughly and improve the performance of their execution. This will help science!

   The documents to be selected look something like this:

   {
   "_id": {
   "$oid": "6272c580d4400d8cb10d5406"
   },
   "#CHROM": 1,
   "POS": 286747,
   "ID": "rs369556846",
   "REF": "A",
   "ALT": "G",
   "QUAL": ".",
   "FILTER": ".",
   "INFO": [{
   "RS": 369556846,
   "RSPOS": 286747,
   "dbSNPBuildID": 138,
   "SSR": 0,
   "SAO": 0,
   "VP": "0x050100000005150026000100",
   "WGT": 1,
   "VC": "SNV",
   "CAF": [{
   "$numberDecimal": "0.9381"
   }, {
   "$numberDecimal": "0.0619"
   }],
   "COMMON": 1,
   "TOPMED": [{
   "$numberDecimal": "0.88411856523955147"
   }, {
   "$numberDecimal": "0.11588143476044852"
   }]
   },
   ["SLO", "ASP", "VLD", "G5", "KGPhase3"]
   ]
   }

   For a basic annotation (https://en.wikipedia.org/wiki/SNP_annotation) scenario, we need such query:

   {'ID': {'$in': ['rs369556846', 'rs2185539', 'rs2519062', 'rs149363311', 'rs55745762', <...>]}}
   , where <...> means hundreds/thousands… more

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

9. There is a specific collection that I need more performance than others. Is there a way to assign more ram/memory to a specific collection?

   There is a specific collection that I need more performance than others. Is there a way to assign more ram/memory to a specific collection?

   1 vote

   Vote Vote Vote

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close

   Vote

   We’ll send you updates on this idea

   0 comments  ·  Performance  ·  Edit…  ·  Delete…  ·  Admin →
   How important is this to you?

   Not at all You must login first! Important You must login first! Critical You must login first!

   We're glad you're here

   Please sign in to leave feedback

   Signed in as (Sign out)

   Close

   Close