cross cluster lookup (kind of)
Hi,
This is more of an add-on request for the aggregate().
Lets say we have two clusters , second clusters is derived from first cluster ( i-e some of the object ids in second cluster exist only in first cluster ).
Now I want to do a lookup for the id in the first cluster from the second cluster , which is not possible.
So , can we have another optional parameter in the aggregate function which allows us to pass the documents , over which we can apply the required stages ( i-e after using find() on cluster 2 , I will pass the result into an aggregate() call for cluster 1 ).
example query ->
db1 points to cluster1
db2 point to cluster2.
I am working on Node.js -
let result=[];
let response = await db2.find().forEach(item=>{
result.push(item);
});
let lookupResponse = await db1.aggregate([<stages>],{},result);
Here in the second query , the documents in which the aggregate stages should work
is the documents present in the variable 'result'.
I hope I was able to explain my idea.
It would really help if this is added in the next update.
Thanks,
Dushyant.
dushyant
shared this idea
-
AdminMassimiliano
(Admin, MongoDB)
commented
Thank you for submitting this request, Dushyant.
This is not something we support in Compass today, although we are looking at how we can enable users to work with multiple connections/clusters at the same time. We'll keep your request in mind and reach out if we have more questions.
In the meantime, you have 2 options:
1. if you are using Atlas, you can set up a Data Federation instance (https://www.mongodb.com/docs/atlas/data-federation/overview/) where you map your separate clusters to virtual collections and then run an aggregate as if they were the same cluster2. you can use the MongoDB shell from the terminal and do more or less what you expected to do in your example: you can keep 2 connections open at the same time (with the connect() method), run your first aggregate, store the results in an array, and then inject the array into the second pipeline using the $documents stage (https://www.mongodb.com/docs/manual/reference/operator/aggregation/documents/).