Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-UTF8 filename support #975

Open
cgwalters opened this issue Dec 16, 2024 · 2 comments
Open

non-UTF8 filename support #975

cgwalters opened this issue Dec 16, 2024 · 2 comments
Labels
area/client Related to the client/CLI

Comments

@cgwalters
Copy link
Collaborator

Since its creation ostree (current bootc backend) intentionally didn't support non-UTF8 filenames. In theory, we could lift that restriction as we look towards a direct composefs backend.

I personally think it's almost always going to be a mistake to ship non-UTF8 filenames in containers, but just filing this so we have a tracker.

Also, it'd probably be good to add this to container lint.

@cgwalters cgwalters added the area/client Related to the client/CLI label Dec 16, 2024
@allisonkarlitskaya
Copy link
Contributor

composefs-rs is pretty agnostic about filename encoding. It processes filenames either as Path, OsString, or [u8], all of which should be fine. It never tries to stringify anything except for output/debugging purposes...

allisonkarlitskaya added a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

Signed-off-by: Allison Karlitskaya <[email protected]>
allisonkarlitskaya added a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.
allisonkarlitskaya added a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

Signed-off-by: Allison Karlitskaya <[email protected]>
@cgwalters
Copy link
Collaborator Author

Yeah, the UTF-8 restriction was an intentional choice many years ago for ostree. Another two example opinionated choices:

  • Not having modification times at all
  • No support for device files

The mtime thing is still a huge overall issue...and devices are basically always a bad idea (even the overlayfs whiteout nesting one is as discussed better done another way)

But of course since OCI didn't add these restrictions we need to be compatible and yes, composefs will help with that.

It's actually really nice that with composefs specifically the non-UTF8 filenames don't leak onto the physical filesystem and only impact the container using them.

allisonkarlitskaya added a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

Signed-off-by: Allison Karlitskaya <[email protected]>
allisonkarlitskaya added a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

Signed-off-by: Allison Karlitskaya <[email protected]>
cgwalters pushed a commit to allisonkarlitskaya/bootc that referenced this issue Dec 19, 2024
As mentioned in containers#975, we should add a lint to ensure that all filenames
are UTF-8, giving nice errors (with full pathnames) in the event that we
encounter invalid filenames.

We also check symlink targets.

Add a unit test that tries various valid and invalid scenarios.

Signed-off-by: Allison Karlitskaya <[email protected]>
Signed-off-by: Colin Walters <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/client Related to the client/CLI
Projects
None yet
Development

No branches or pull requests

2 participants