prefixing of binaries that are already present on uss

[mgrossmann]

Hello,

there is a flaw with shadowing binaries.
In my opinion it may be a good practice to prefixing binaries,
if they are not 100% compatible with the IBM ones.

e.g.

coreutilsport/cp is shadowing the IBM cp. This leads to a lot of problems, because the IBM cp hast some features that are not (yet) implemented in the gnu version. Should we deploy the binary as gcp?

makeport/make is shadowing the IBM make. Other OSes are already using gmake instead.

Just to start the discussion

Mike

[IgorTodorovskiIBM]

Thanks for bringing this discussion. This is definitely an issue. Here's a few other ideas:

• zopen could be configured to avoid generating the $ROOTFS/usr/local/bin/ symlinks for tools that override native z/OS tools. We could maintain a list or detect if the tools is already present /bin
• We could detect such tools and make them available under another directory such as bin_override/ rather than bin/. The user can optionally configure if the PATH bin_override is added via a zopen-config option or via zopen-init.

[MikeFultonDev]

One of the reasons we exposed 'cp' was so that our tools would build, and I put in a hack to support cp of datasets by failing over to /bin/cp (I agree it's a hack). If we call it gcp, it means our tools won't build.
Some thoughts:

• an alternate location like @IgorTodorovskiIBM suggests
• an option on install (in general) to not install conflicting tools (man has a similar issue, as does cat). What 'conflicting' means is interesting - do we not install grep, sed, ... ? I think we want to because the z/OS implementation is too archaeic

[DevonianTeuchter]

This is actually something that is an issue in the zopen-* scripts themselves - we hardcode certain calls [like /bin/echo, /bin/find] to ensure that we use the basic system-supplied versions rather than rely on any updated packages or non-standard builtins.

There is already options that either do not merge the new packages into the main system [so not on the path/libpath/.. etc] - users can still use the packages by sourcing from within the package directory as per pre-package-management

We could:

• add an option to check if shadowing and ask whether to install the updated version or install to a bin_override directory ?
• Append rather than prepend the zos open tools on the PATH - he "default" tools would be the preference while still allowing any extra tools (like those in coreutils) to be installed and available.
• Prefixing would lead to issues where one tool hardcodes reliance on another or if that prefix wasn't unique in itself and would look odd - suffixing with ".zot" or something would still show the original tool name but would still suffer from any hardcoding issues though..

Tbh, I would have the zos open tools first on the PATH and if there is a feature that a standard z/OS tool [like cp] can do that the z/OS Open Tools version can't, then that should be raised as an issue with the original still being available with a suitable prefix "/bin/cp" until such time as the port implements that feature [and being open source, anyone could implement the missing feature - hint, hint ;-) ]

[MikeFultonDev]

append instead of prepend is interesting - for a 'consumer' that might work (what I don't know is if any of our tools like git would have runtime dependencies on our tools and therefore break if we did that)

[IgorTodorovskiIBM]

We discussed this issue last week and came up with a few more ideas:

1. Enhance zopen's functionality to recognize conflicting native tools and enable users to selectively include them in the PATH.

./zopen-config by default will continue to prefix all z/OS Open Tools by default

We add an extra option (or environment variable):

zopen-config --usebintools=make,sed # this will choose /bin/make and /bin/sed over z/OS Open Tools make/sed

zopen could create aliases on the fly for make to /bin/make.

• Alternatively, we could fill in a $HOME/.zopen/bin/ directory (user specific) which dynamically create symlinks every time you source .zopen-config.

2. Two layer approach (Taken from @DevonianTeuchter 's summary):
   To summarize how to do the two-layer approach with the current tooling (at least as I understand the needs!):
   The sysprog would need to generate two zopen filesystems.
   run zopen init corefs and install only "core" tools to this system
   run zopen init otherfs to generate a second zopen filesystem where any other tools can be installed (any other tools that get pulled in as dependancies
   Users that only want to use the core/FMIDed tools would source otherfs/etc/zopen-config; users that also want the zopen-shipped core tools rather than the system tools would also source the corefs/etc/zopen-config setup.
   There was discussion a long time back about assigning tools into groups/categories; maybe something to resurrect as we didn't really implement it fully.

[lbdyck]

why not add the IBM differences to the open-source and upstream them?

[IgorTodorovskiIBM]

That's the longer term vision, yeah, but it will take time.

There's also some cases where the open tools provide an alternative option. e.g. /bin/make -c to change the dir is different and incompatible with GNU Make's option (make -C). Also, some changes the upstream community may not agree with - will they even be ok with adopting support for a non-UNIX file system like datasets?