Overlay fs not really rw?

[M-Gonzalo]

I'm trying to mount my dwarfs using the procedure described here, but I'm finding that I can only write to the root folder of the mounted filesystem.
For example, if I have

Documents           // Mounted fs
Documents.dwarfs

I can only write to Documents/file.bin, not to Documents/foo/file.bin. That is, I can't create or delete items.

What could be the reason for this?

[mhx]

I've just confirmed that the procedure still works. I can definitely use the DwarFS/OverlayFS setup just like any other file system, including creation and deletion of files.

My best guess as for why this isn't working for you is that when you created the DwarFS image, the folder foo was already not writeable. DwarFS fully preserves all permissions, so a non-writeable directory will still be non-writeable when you mount the image. However, OverlayFS also does allow you change the permissions, so you could make the directory writeable (if you're the owner) after setting up OverlayFS.

In case that's isn't the reason, could you attach a small DwarFS image here (or send it to my github email address, if you prefer) that reproduces the issue so I can take a look?

[M-Gonzalo]

Maybe it's a Manjaro bug. Because it's not a privilege or permissions error, i.e. can't be fixed using sudo or chmod, instead it says "not implemented". Like it knows it's on a read-only system, only it shouldn't.

sudo chmod gives me the same message

I'm attaching one sample file. But it happens with all of them.

IHP.zip

[mhx]

So this is the procedure I've just tried, without any obvious issues:

$ mkdir mnt-ro mnt-rw mnt-work mnt

$ ls
IHP.dwarfs  mnt  mnt-ro  mnt-rw  mnt-work

$ dwarfs IHP.dwarfs mnt-ro -oallow_root
I 09:30:26.763282 dwarfs (v0.5.6, fuse version 35)
I 09:30:26.763965 file system initialized [448.6us]

$ sudo mount -t overlay overlay -o lowerdir=mnt-ro,upperdir=mnt-rw,workdir=mnt-work mnt

$ cd mnt/blog/

$ ll
total 10
-rw-r--r-- 1 mhx 1000 1966 Aug 31 04:33 App.cabal
drwxr-xr-x 1 mhx 1000    4 Aug 31 04:33 Application
drwxr-xr-x 1 mhx 1000    3 Aug 31 04:37 Config
-rw-r--r-- 1 mhx 1000  425 Aug 31 04:36 Main.hs
-rw-r--r-- 1 mhx 1000  775 Aug 31 04:33 Makefile
-rw-r--r-- 1 mhx 1000   46 Aug 31 04:33 Setup.hs
drwxr-xr-x 1 mhx 1000    5 Aug 31 04:36 Web
drwxr-xr-x 1 mhx 1000    3 Aug 31 04:37 build
-rw-r--r-- 1 mhx 1000  520 Aug 31 04:33 default.nix
-rw-r--r-- 1 mhx 1000  113 Aug 31 04:33 hie.yaml
-rwxr-xr-x 1 mhx 1000 1052 Aug 31 04:33 start
drwxr-xr-x 1 mhx 1000    4 Aug 31 04:33 static

$ cp ../../IHP.dwarfs .

$ chmod -R g+w static

$ ll -d static
drwxrwxr-x 1 mhx 1000 88 Aug 31 04:33 static

$ cd ../..

$ sudo umount mnt

$ fusermount -u mnt-ro

One more idea: are you using fuse-overlayfs or the kernel-based overlayfs? Because I've only ever been using the kernel-based one, I have no idea if the fuse version works properly.

[M-Gonzalo]

How do you tell them apart? I tried whatis and which but that doesn't tell me anything.
BTW, is ...

-oallow_root

... correct, or is there a space missing? It'd be hilarious if all the fuss is because I typed one too many spaces

[mhx]

How do you tell them apart? I tried whatis and which but that doesn't tell me anything.

I'd reckon that you'd mount the FUSE version using fuse-overlayfs instead of mount -t overlay.

BTW, is ...

-oallow_root

... correct, or is there a space missing? It'd be hilarious if all the fuss is because I typed one too many spaces

It shouldn't make a difference. If you're using the command line I've used, you're most likely using the kernel driver. In that case, I honestly don't know what the problem is. One thing you can try is to run dwarfs in debug mode and see if you can spot anything interesting. This is the output I get when doing cp IHP.dwarfs mnt/blog/:

$ dwarfs IHP.dwarfs mnt-ro -d -oallow_root,debuglevel=debug
I 22:01:52.445133 [dwarfs.cpp:722] dwarfs (v0.5.6, fuse version 35)
D 22:01:52.445806 [filesystem_v2.cpp:316] section num=0, type=BLOCK, compression=ZSTD, length=1359652, checksum=0x9ac1ac27caa49627 @ 64 [1359652 bytes]
D 22:01:52.446107 [filesystem_v2.cpp:316] section num=1, type=METADATA_V2_SCHEMA, compression=ZSTD, length=484, checksum=0x786a2bb9c20f21da @ 1359780 [484 bytes]
D 22:01:52.446469 [filesystem_v2.cpp:316] section num=2, type=METADATA_V2, compression=ZSTD, length=14840, checksum=0x0cba64f6f5965694 @ 1360328 [14840 bytes]
D 22:01:52.447446 [string_table.cpp:69] imported dictionary for names string table [18.56us]
D 22:01:52.447807 [filesystem_v2.cpp:337] read 1 blocks and 29999 bytes of metadata
I 22:01:52.448247 [dwarfs.cpp:617] file system initialized [2.506ms]
FUSE library version: 3.9.0
unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.31
flags=0x03fffffb
max_readahead=0x00020000
D 22:01:52.455103 [dwarfs.cpp:112] op_init
   INIT: 7.31
   flags=0x00409029
   max_readahead=0x00020000
   max_write=0x00100000
   max_background=0
   congestion_threshold=0
   time_gran=1
   unique: 2, success, outsize: 80
unique: 4, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 6440
D 22:02:17.966146 [dwarfs.cpp:395] op_statfs
   unique: 4, success, outsize: 96
unique: 6, opcode: LOOKUP (1), nodeid: 1, insize: 45, pid: 6452
D 22:02:46.723602 [dwarfs.cpp:123] op_lookup(1, blog)
   unique: 6, success, outsize: 144
unique: 8, opcode: GETXATTR (22), nodeid: 2, insize: 71, pid: 6452
D 22:02:46.724157 [dwarfs.cpp:426] op_getxattr(2, trusted.overlay.origin, 0)
   unique: 8, error: -61 (No data available), outsize: 16
unique: 10, opcode: LOOKUP (1), nodeid: 2, insize: 51, pid: 6452
D 22:02:46.724926 [dwarfs.cpp:123] op_lookup(2, IHP.dwarfs)
   unique: 10, error: -2 (No such file or directory), outsize: 16
unique: 12, opcode: LISTXATTR (23), nodeid: 2, insize: 48, pid: 6452
   unique: 12, error: -38 (Function not implemented), outsize: 16
D 22:04:05.613347 [block_cache.cpp:220] stopping cache workers

Now that I'm seeing this, the lines

unique: 12, opcode: LISTXATTR (23), nodeid: 2, insize: 48, pid: 6452
   unique: 12, error: -38 (Function not implemented), outsize: 16

do look suspicious in the context of the error you've posted. However, I'm pretty certain it's not a necessity for FUSE drivers to implement listxattr. And it still works on my system without overlayfs complaining at all.

[M-Gonzalo]

I basically copied the commands on the doc, with the exception I prepended a dot to the auxiliary folders so they remain hidden. That's all.