Database
272 results found
-
Add a "Limit" to Delete and Bulk Delete operations
Deleting tens of millions of documents can have a big impact on the performance of the Clusters, even using Bulk Delete. A "Limit" must be added to Delete and Bulk Delete to let us limit the number of operations, making sure we do not kill the Clusters' performance.
- For the delete, this would make sure we only delete n number of documents.
- For the Bulk Delete, this would also make sure we only delete n number of documents, or it could instead limit the number of batches/groups of documents to be deleted.
Right now, the only solution is a hack,…
9 votes -
Should be possible to configure profiling output destination
When enabling database profiling the output is sent to both the system.profile collection and system logs. Logging to a capped collection is fine but spamming the logs on disk is not good.
We have a need to be able to react to changing query patterns quickly, so we have profiling enabled in production on a busy system and we do real-time analysis on the the system.profile collection. This works fine and the performance hit is acceptable but our system logs on disk grows a lot.
Please make it possible to configure if profile logging should go to disk, collection or…
8 votes -
Data Replication from one MongoDB Cluster to another MongoDB Cluster
Create an option for the user to replicate data from one MongoDB Cluster to another MongoDB Cluster inside the account. We don't need to migrate data, I need replicate data from an Production Cluster to an Dev/Test Cluster. Thanks.
8 votes -
Allow custom autoscaling policies
Currently this is the following autoscaling policy in mongo:
Please note, if the next highest cluster tier is within your Maximum Cluster Size range, Atlas scales the cluster up to the next tier if one of the following is true for any node in the cluster:
Average CPU Utilization has exceeded 75% for the past hour, or
Memory Utilization has exceeded 75% for the past hour.The feature request is to allow a DBA to setup custom autoscaling policy
8 votes -
8 votes
-
Auditing requirement for the changes done through Ops Manager portal
Changes done through Ops Manager portal are visible only in Alerts(Activity Feeds). Though these Alerts(Activity Feeds) can be derived through API, our auditors may not accept API calls and we need ops manger to log these change in respective deployment mongo audit.log
Thank you
8 votes -
Lock on session start level with provided keys
Scenario:
Parallel DB updates with transactions on multiple collections that use same documents.
Example: Calculate some common stuff to embed it in extra collections to avoid lookups.Problem:
Lock and timeout on documents uses up valuable time and performance.
Also lock conflicts produce a huge amount of exceptions that need to be handled.Idea:
Key-based sessionsExample:
Session A with Keys 1 and 2 is started.
Session B tries to start with Key 1. Key 1 is locked with Session A. Session B waits till Session A is finished.In this case transaction doesn't start with updating and failing. Waiting…
7 votes -
Allow to define access to DBs/collections by prefix or pattern.
Please extend ACL to support prefixes (or regex) in the database name/collection.
Currently only allowed ALL (when empty string provided) or exact db/collection matching when provided.Use case: Several services are using the same cluster but need to be isolated. Every service can get readWriteAnyDatabase but only to databases prefixed by some prefix.
Services need to create new databases on the fly, so it is not possible to define a list of databases upfront.For example - rwRoleForService1 allows "update", "insert", and "remove" operations only on databases prefixed by "service1-" (sevice1-db1, sevice1-db2, ....)
{
role: "rwRoleForService1",
privileges: [
{
{…7 votes -
Handle Daylight Saving Time when $densify is used on a date field
When using "day" as "unit" for a $densify pipeline stage on a date field, the date is always advanced of 24 hours. This is however not always the expected result in timezones in which the year has one 23-hour and one 25-hour long day, because of Daylight Saving Time.
It would be useful to have the possibility to pass an optional timezone parameter in the $densify stage and, when present, have the stage account for these exceptions when appropriate.
Here follows an example.
Assume we have a collection containing the following documents:
…db.densifyDateExample.insertMany([ {_id: "a", d: ISODate("2022-10-28T22:00:00Z")}, {_id: "b", d:7 votes -
Raise the limit of 16 MB JSON between aggregation stages
When doing analytics, the 16 MB JSON limit between aggregation stages restricts the ability to process large amount of data. allowDiskUse does not help with all the various $operators and stages that we use. See ticket 00774514 for details.
7 votes -
provide faster way to perform a count() on million documents
a couple of customer a shifting from MongoDB to other DBs like Redis or even PostreSQL as count is too slow on MongoDB when there are a lot of document to count even with use of indexes
7 votes -
Elevate hidden replica to a voting member when quorum is lost for a Replica Set
The MongoDB engine should allow for hidden members in a replica set to be automatically elevated to a voting member. When quorum is lost and can't be met, the hidden replica is automatically reconfigured to a voting member to obtain quorum.
When the failed voting member comes back online and catches up the hidden replica is demoted back to a normal hidden member.7 votes -
Unique Indexes and Bulk Upserts for Time Series Collections
We would like to insert data in bulk into time series collections and identify the new data that has been inserted without the possibility of duplicates being inserted.
For regular collections this is achievable by adding a unique index and performing a bulk upsert (as any duplicates will be rejected due to the unique index).
For time series collections however unique indexes are not currently supported.
In addition performing an upsert with $setOnInsert option which should only action insert operations is also not currently supported for time series collections.
At the moment the only options appear to be:
(1) to…
6 votes -
Implement $bucket and $group on indexed values with sub-linear runtime
We noticed that sum $bucket and $group aggregations such as $min, $max, $count are unexpectedly slow even when fully covered by an index, (partially) because the DB scans through the entire index rather than employing optimization approaches such as binary search.
An example pipeline that should return instantaneous but scans through the entire index (confirmed on v4.4 and v5):
[
{
$match: {
status: "DELIVERED",
},
},
{
$group: {
id: {
status: "$status",
},
min: {
$min: "$modifytime",
},
},
},
]
with an index { status: 1, modify_time: 1}Another example is $bucket (same index):
[
{…6 votes -
Support compound/multiple grouping keys in $bucket
We often need to compute statistical/summarizing aggregations grouped by more than one field where all fields are of a $bucket-able type.
An example, would be to count all orders grouped by their status and some custom time ranges of their creation date.
This can be achieved by using $group in combination with a $switch expression (sometimes simplified with $trunc), however, that is cumbersome and prevents efficient grouping since e.g. no binary search can be employed to identify the bucket boundaries efficiently.The query syntax of $bucket would not need to change much. It would simply need to allow for nested…
6 votes -
sharding error shardsvr
Make it clear which node is causing the "shardsvr" error.
Spawned from support case 01042995
Our error occurred when the user tried to connect using Compass. The failure was to list the collection names on one database.
The error presented back to the user was merely
Cannot accept sharding commands if not started with --shardsvrWe found eventually that the primary changed on one of the shards, and that primary did not have the appropriate
clusterRolein the mongod.conf file. My concerns are that this took too long to track down and would be impossible in a 100-shard environment.- Nothing…
6 votes -
x509 authentication with other certificate's components than (O,OU,DC)
In some entities (e.g. ours), the O, OU, DC triplet is not detailled enough or not appropriate, which makes it impossible to authenticate through x509.
For exemple, in our entity, the O and OU are the same for all certificates (because all servers are in the same Organisation Unit), and the DC field is not used. We do use other fields though.
Because of that, we can't use the x509 authentication feature, although it is strongly asked for by the security Team.Would it be possible to enhance the x509 authentication mechanism to allow more flexibility for the authentication's fields?
6 votes -
`$getField` to work with a dynamic `field`
Currently
$getFieldworks only whenfieldresolves at query-compile-time to a string. It would be nice if it worked also whenfieldresolves to a string at runtime.See this Jira ticket - https://jira.mongodb.org/browse/SERVER-67030
5 votes -
Get metadata about source client connection that submitted a given change
Currently with change streams it is impossible to know who or what connection initiated the changes.
It would be a good feature to have to be able to receive some data about the source client connection that initiated a change.
My particular use case is the following:
I have an app that connects to Atlas. (source client connection)
I can subscribe to change streams and then execute some logic when it applies.That app can scale to multiple instances.
Each instance subscribes to the change streams.
But I only want each instance to execute the logic that applies to only…5 votes -
Unique index in sharded cluster
For enforcing uniqueness in a sharded cluster, the officially recommended approach provided here https://docs.mongodb.com/manual/tutorial/unique-constraints-on-arbitrary-fields/#std-label-shard-key-arbitrary-uniqueness is simplistic and in production environment it brings non-trivial amount of work. Some considerations:
- Ephemeral issues might cause inconsistencies between the two collections (for example, unique index collection update succeeded but not the main collection) and make some unique keys not useable.
- There are many changes needed (we're using ORM Mongoose, there are many hooks for it to change) for enforce this universally.
What we ended up doing is to use distributed ephemeral locks (a TTLed MongoDB collection) to lock on the unique keys before adding…
5 votes
- Don't see your idea?