in #10932, the local volume semantics are "if the disk is backed by /dev/zvol/rdsk/, it is assumed to be on the U.2s we test in the product, which to date and for the foreseeable future are not expected to report volatile write caches".
this is wrong in two important ways:
- it is technically possible that a U.2 in the field will spontaneously grow a write cache (this what NVMe
Get Log Page - SMART Health Information Log bit 4 is about: volatile memory is present and the memory backup solution failed)
- development environments (like my workstation!) probably have commodity storage devices that very much have volatile write caches.
#10932 elaborates on the costs of "wrong" here, but suffice to say we want to not be wrong.
in the U.2 case, there are many questions. do we place new local volumes on partially-degraded U.2s? should we consider this kind of degradation tolerable? does a local volume on such a U.device have the "faulted" state, or can new instances with existing volumes here start (say, to evacuate data)?
for the cases of disparate hardware, we'd really need sled-agent to query disks for write-cachefulness and consider that in the control plane. we're also assuming that, for example, no one is going to randomly change the enabledness of a disk's write cache.
for the "degraded-in-the-field" case, regardless of the InstanceSpec Nexus sends along, Propolis needs to be aware of the actual hardware state to tell guests if the storage device underlying a local volume has experienced a failure. that's a distinct issue, Propolis#1185. this may come with a device reporting a VWC when it did not before - how disks degrade here is (to me!) kind of a string of unknowns.
in #10932, the local volume semantics are "if the disk is backed by
/dev/zvol/rdsk/, it is assumed to be on the U.2s we test in the product, which to date and for the foreseeable future are not expected to report volatile write caches".this is wrong in two important ways:
Get Log Page - SMART Health Information Logbit 4 is about: volatile memory is present and the memory backup solution failed)#10932 elaborates on the costs of "wrong" here, but suffice to say we want to not be wrong.
in the U.2 case, there are many questions. do we place new local volumes on partially-degraded U.2s? should we consider this kind of degradation tolerable? does a local volume on such a U.device have the "faulted" state, or can new instances with existing volumes here start (say, to evacuate data)?
for the cases of disparate hardware, we'd really need sled-agent to query disks for write-cachefulness and consider that in the control plane. we're also assuming that, for example, no one is going to randomly change the enabledness of a disk's write cache.
for the "degraded-in-the-field" case, regardless of the InstanceSpec Nexus sends along, Propolis needs to be aware of the actual hardware state to tell guests if the storage device underlying a local volume has experienced a failure. that's a distinct issue, Propolis#1185. this may come with a device reporting a VWC when it did not before - how disks degrade here is (to me!) kind of a string of unknowns.