Atlas shards with different clusters tiers
For solutions of hot and cold storage, there is at least 2 options:
- Online archiving
- Hot and cold cluster (using a trigger and datalake)
But these solutions have limitations (https://www.mongodb.com/docs/datalake/limitations/)
There is another solution: using sharding.
We can set shards like so:
1 shard: Data from the last 2 months -> Hot data
1 shard: Data between 2 and 24 months -> Warm data
1 shard: Data more than 24 months -> Cold data
The usage pattern: 95% of the accesses/queries go to shard with the hot data.
On Atlas, we need to set the same Cluster Tier in all the shards.
But in this scenario, the shards with warm and cold data, doesn't need as much CPU and RAM as the other, but it needs more storage.
So, it would be good to have the availability to set custom cluster tiers for different shards.
Something like:
Hot data shard: M60 with 128GB
Warm data shard: M40 with 256GB
Cold data shad: M30 with 512GB