Faster Sorting with Atlas Search

I am looking to achieve a faster sort when using Atlas Search.

Please comment if you tried Stored Source Fields and it did not meet the performance you were looking for.

Stored Source Fields Docs:
Index - https://www.mongodb.com/docs/atlas/atlas-search/stored-source-definition/
Query - https://www.mongodb.com/docs/atlas/atlas-search/return-stored-source/

46 votes

Elle shared this idea · Jun 16, 2022 · Report… · Admin →

completed ·

Elle responded · Jul 18, 2023

Sort in Search is now GA -- see changelog here for documentation.

Show previous admin responses (2)

An error occurred while saving the comment

Huu Phuoc NGUYEN commented · July 19, 2023 12:55 AM · Report

Thanks for compiling this! I now can use it successfully in Compass.

However, because of using dynamic SearchDefinitionBuilder, I'm still waiting for C# driver support this option.

Submitting...
Germán commented · July 18, 2023 12:46 PM · Report

Thanks for this! I'm trying it but I'm getting the following error:
"{{fieldName}} is not indexed as sortable."

I already rebuilt the index. Is there something else I'm missing? The example in the documentation uses facet types, maybe that?

Submitting...
Ed commented · June 10, 2023 6:16 PM · Report

I need to sort based on timestamp, i tried near but it's not working
{
near: {
path: "data.postingDate_timestamp",
origin: 1686433743913,
pivot: 1,
score: {
boost: {
value: 1000,
},
},
},
},

Submitting...
Huu Phuoc NGUYEN commented · May 18, 2023 12:46 AM · Report

Cluster tier M30, Version 4.4.21.
Atlas search
collection.Aggregate().Search(Builders<Rate>.Search.Compound().Must(clauses), indexName: "Dashboard-Index", returnStoredSource: true)
.SortByDescending(x => x.LastUpdate).Skip(0).Limit(200).ToList();

Statistic:
19k ~ 9 seconds
98k ~ 7 seconds
149k ~ 19 seconds
231k ~ 69 seconds

Too bad performance of sorting after search

Submitting...
Prasad Kini commented · January 8, 2023 11:28 AM · Report

Hi Admin, could you please share a high-level idea of the solution (that is currently being worked on) for faster sorting with Atlas search?

This would help us understand if it would meet our requirements and/or if we need to make changes to our module that builds the queries.

Thanks!

Submitting...
Prasad Kini commented · January 7, 2023 9:23 PM · Report

Any ETA on when faster sort inside Atlas search will be available? I have tried stored source and it has failed miserably for my requirement :(

Submitting...
Ofer Chacham commented · December 12, 2022 2:59 AM · Report

Hello,
Is there a ETA for this work to be completed ?

Submitting...
Lawrence Raj commented · August 22, 2022 3:13 AM · Report

How is this different from having a project stage after $search, also this approach doesn't seems to improve the performance on lage dataset.

Submitting...
Ofer Chacham commented · July 25, 2022 3:00 AM · Report

Having the the ability to sort big number of documents inside Atlas search is crucial. The alternative of using storedSource and sort stage after that is very slow and cannot be used in a user-facing application

Submitting...
David commented · July 7, 2022 4:01 PM · Report

While stored fields are definitely much faster than not using them, a $sort clause on a large result set is still way too slow (10 - 30 sec vs upwards of 90 sec+). The docs suggest that sorting can be done in $search, but does not show any examples of how.

"Using a $sort aggregation pipeline stage after a $search stage can drastically slow down query results. If possible, design your $search query so that all necessary sorting occurs in the $search stage to remove the need for a $sort stage. ..."

Submitting...
Kyle commented · June 25, 2022 2:22 PM · Report

This does not seem like an acceptable solution to the sort problem. Adding stored source is going to increase index overhead. It should use the already created field mappings and allow a native searchSort stage.

I have also tried stored source on a collection of around 300k documents and it is still way too slow. (M40 cluster)

Submitting...
Nadeem Ahmad commented · June 23, 2022 5:28 PM · Report

I've tried stored fields but it doesn't improve the performance of $sort or $skip to acceptable numbers on large data sets.

Index definition:-

{
"mappings": {
"dynamic": false,
"fields": {
"category": {
"fields": {
"name": {
"store": false,
"type": "string"
}
},
"type": "document"
},
"countries": {
"store": false,
"type": "string"
},
"manufacturer": {
"fields": {
"name": {
"store": false,
"type": "string"
}
},
"type": "document"
},
"tags": {
"store": false,
"type": "string"
},
"title": {
"store": false,
"type": "string"
}
}
},
"storedSource": {
"include": [
"dealPrice",
"createdOn"
]
}
}

Submitting...