Reduce the minimum value for watchdogPeriodSeconds
The storage watchdog attempts to create, write, and read a test file in critical directories every 10 seconds.
The watchdogPeriodSeconds parameter controls how often these a thread checks to ensure at least one check has succeeded since the last check.
The minimum value for watchdogPeriodSeconds is 60 seconds. This means that in the worst case, the mongod could be unable to write for up to 2 minutes before the watchdog asserts and kills the stalled node. That is a very long time for a primary node to be stalled in a busy cluster.
It does make sense that watchdogPeriodSeconds must be greater than the check frequency. However, reducing the minimum from 60 seconds to 20 or 30 seconds would still permit multiple check cycles per period, but would significantly reduce the maximum length of time that the node could be stalled.