Enable setting the security.redactClientLogData parameter.
We're using Atlas to store PII (HIPAA), and we need to store the log files for an extended period of time, but we don't want any PII / HIPAA data to be stored in the logs.
Dan
shared this idea
-
Kyle
commented
Any status on this? There are certain countries we can't build clusters in until these redaction options are in place.
-
Kyle
commented
Items we need to comply with data privacy rules:
1. Log redaction turned on (this appears to be available via a support ticket)
2. Ability to enable log redaction via automation (API, terraform, console)
3. Ability to validate that log redaction is enabled via API/console -
Kyle
commented
Hi Salman,
That's good to know. I was told this wasn't even an option right now.
When can we turn this on via automation (terraform)? Ideally this option would be allowed to be configured by your customers instead of having to open a ticket with support to enable it each time.
Thanks,
Kyle -
AdminSalman
(Admin, MongoDB)
commented
Hi Kyle,
You can reach out to support for enabling this feature. However, enabling this feature effectively disables the Performance Advisor and Query Profiler, so that is a tradeoff you may need to consider.
Salman
-
Kyle
commented
Any progress on this? There are certain government regulations that make this a requirement if personal data is being stored in log files. I'm curious how you are solving this for your customer bases that want to build clusters in places like South Korea.
-
Hi Kyle,
Conceptually this is something we would like to introduce in future.
-Andrew
-
Kyle
commented
Andrew, what about the ability to redact the logs on the fly when we download them (either through the UI or via API calls)?
We currently pull logs and store them in CloudWatch so we can search more easily for issues without the need to grant everyone access to Atlas in our organization. But we'd prefer to not have CRUD operations show up in those logs if possible.
Thinking the API call could include a redact option (true/false) and the UI can have a check box to turn it on when you pull it down manually.
-
Hi Dan,
It's important to emphasize that the only portion of logs that can contain query contents is the slow query logs: MongoDB Atlas provides a lot of high-value capability on top of these slow query logs, ranging from the Performance Advisor which provides index suggestions to the Query Profiler.
Importantly, access to database process logs is limited to Project Data Access Read Only users and above, and accesses of logs are audited events in the Project-level activity feed. You can lock down environments by managing your infrastructure in code, and give Project Read Only (as distinct from Product Data Access Read Only) to most users (this will mean they will have metadata access view only, including monitoring, without access to log files).
Longer-term we plan to move to model that can provide finer grained authorization for users to be granted the right to perform privileged actions on specific resources. We also aspire to provide richer, more configurable views into logs and other diagnostics data.
Also I should point out that MongoDB also offers Client-Side Field Level Encryption which allows you to encrypt data of the highest classification level before it ever leaves your network, with the tradeoff that you give up some queryability on those fields (point queries continue to work, but range queries do not). See more here: https://docs.mongodb.com/drivers/use-cases/client-side-field-level-encryption-guide
-Andrew
