Provide an interface that can stream logs directly to Splunk.
I'm trying to get logs from MongoDB Atlas to Splunk, and there's currently no native interface to do it, so instead of streaming to Splunk directly, I'm having to execute https calls in intervals, unzip them and then ship them to Splunk. Many products have interfaces to stream to Splunk directly, and it would certainly be handy in this situation.
For others (like Qradar) it would be easier if we would have an API that would return the Audit or log messages in JSON format (and not as an archive) and which would strictly respect the requested time window.
The retrieving of log files as well as audit log files is very cumbersome for MongoDB Atlas. There is a delay in retrieving the log files (5 min), then you have to download the logs via the API and then you have to unpack it. This setup is not suitable for live trouble shooting.
I could use an integration to datadog for log streaming.
But, pushing to s3 would be a good step because at least that can go through a bucket event-based forwarder to Datadog.
I am in need of this feature. This will help us in resolving many critical issues in our development environment.
If MongoDB instance is able to push the logs stream to splunk, we can trace many events like slow running queries and other critical info and take action immediately. Now there is no automated way to find out this.
I'm interresting to have the functionnality to push automatically the logs (organisations events, projetcs events, clusters events, clusters logs, cluster audits logs) automaticaly to a cloud storage with the possibility to set the frequency.
It's can usefull to tranfert these events to our SIEM.
I would like to have this feature along with capability of sending the logs to on-prem Splunk system.
Would appreciate a streaming solution for the audit logs to Splunk as well. The solution mentioned by Andrew (push-based integration for logs to S3) would work for us as well. So where can I upvote for this solution ;-)
Out of curiosity, what is your preferred cloud provider? We plan to introduce push-based integrations for logs to cloud object storage in the future, targeting S3 first: we believe it should be easy to go to any SIEM or log solution like Splunk from there. I'm sorry we don't have something to serve your needs yet.
Note that we have pull-based integrations with jSonar (they can push to Splunk I believe) and Sumo Logic today.