Fix reads from local dir that changes directory

[phofl]

Not adding this to the token makes us thing this is the same operation, silently reading the wrong data

The example might be a little bit too constructed, but if we start caching more this can happen in different scenarios

[fjetter]

I'm pretty sure the parquet reader does not need this since we're actually computing a checksum for all the files. if that doesn't work, we should make sure that the checksum is reliable and we may want the same/similar mechanism for csv instead of relying on CWD. CWD feels odd when working with remote storages