Data Usage Reporting & Improved Profiler
The Current Problem:
The existing issue stems from MongoDB's inability to provide concrete evidence supporting the data charges for your data usage. This predicament becomes especially troublesome when your system typically operates within a data usage threshold of less than 100GB daily. Suddenly, over a span of 7 days, you are billed for data usage exceeding 1000-2000GB daily, only to subsequently revert to using less than 100GB daily. The absence of substantiated evidence leaves you in a quandary, unsure whether the issue lies with your system or is a reporting error on MongoDB's part. MongoDB Support relies on the slowest queries from the profiler as evidence; however, it's essential to note that these queries have persisted both before and after the anomaly.
The proposed feature request centers on the creation of improved data usage reporting, aiming to address the aforementioned challenges. This feature would empower users to access comprehensive data consumption reports which would provide quantifiable evidence to reconcile data usage discrepancies. Key additions include enhancing the Profiler to provide an option to display all queries, not limited to just the slow queries. Additionally, a report should be introduced, offering precise insights into data usage, broken down by cluster, node, and collections per day. This report should tally the number of queries to each collection, along with the amount of data retrieved by each IP Address and user that queried that collection. These comprehensive reports are pivotal in facilitating a more effective understanding of invoices and Atlas Costs.