csi-driver fail to mount certs volume after running for some time

[guscott]

After multiple days of working correctly, csi-driver stops working on some Kubernetes nodes:

I0124 20:13:27.725185       1 app.go:59] main "msg"="building driver"  
I0124 20:13:27.727245       1 filesystem.go:91] storage "msg"="Mounted new tmpfs"  "path"="csi-data-dir/inmemfs"
I0124 20:13:27.828334       1 app.go:103] main "msg"="running driver"  
E0204 13:48:20.214196       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
E0204 13:48:20.315307       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
E0204 13:48:20.720442       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
E0204 13:48:20.821168       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
E0204 13:48:21.732976       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
E0204 13:48:21.835129       1 server.go:109] driver "msg"="failed processing request" "error"="mkdir csi-data-dir/inmemfs: no such file or directory" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume" 
[...]

The volume csi-data-dir is mounted on Host Directory (/tmp/cert-manager-csi-driver DirectoryOrCreate). This host is configured to auto-cleanup old stuff in /tmp.

Is it possible to either:

• periodically touch the directory csi-data-dir and/or its content so that it is not deleted by the /tmp cleanup job
• when receiving the error mkdir csi-data-dir/inmemfs: no such file or directory, recreate the tmpfs filesystem to allow next requests to work

[munnerz]

See #73 and #74

If you're running in an environment where /tmp is periodically cleaned up, you should change the path that these are persisted at using the option added there.

/var/run/cert-manager-csi-driver/ might be a good option - an in-memory filesystem will be created there anyway, so this shouldn't change your security posture.