Collecting statistics in zopen

[IgorTodorovskiIBM]

Should zopen be enhanced to collect statistics similar to how other package manager like brew and npm collect stats. Collecting statistics can help us prioritize our efforts towards projects that are heavily used vs seldomly used projects.

Background:

• We currently report total downloads per project in our Progress page: https://zosopentools.github.io/meta/#/Progress?id=projects-with-the-most-downloads. However, some projects have inflated downloads because of our CI/CD automation and it is impossible to determine how many of those downloads are coming from real users. These numbers are extracted from Github's REST APIs.
• Github.com corrently provides insights: https://github.com/ZOSOpenTools/meta/pulse but nothing on package downloads.

Homebrew has an good article describing what data they collect here: https://docs.brew.sh/Analytics.

What data should be collected?

• Tool / Version being installed - could be collected via zopen install
• Tool / Version being upgraded - could be collected via zopen upgrade
• Tool / Version being uninstalled - could be collected via zopen remove
• Optional email / company / name - could be collected via zopen init . This might be too sensitive. Brew does only stores anonymous data.
• Are the tools being used? We currently have generic zoslib hooks in place for most C/C++ projects, so we could take advantage of this
• Should statistics be collected by default with opt-out? or via opt-in (at zopen init time)?

Data collected from brew:

Whether the data is being sent from CI, e.g. true if the CI environment variable is set.
Whether you are using the default install prefix (e.g. /opt/homebrew) or a custom one (e.g. /home/mike/.brew). If your prefix is custom, it will be sent as custom-prefix to preserve anonymity.
Whether you are a Homebrew Developer, e.g. true if the HOMEBREW_DEVELOPER environment variable is set.
Whether devcmdrun is set, e.g. true if you have ever run one of Homebrew’s developer commands.
Your CPU’s architecture, e.g. x86_64.
The OS you are using and its version number, e.g. macOS 13.
The version of Homebrew, e.g. 4.0.0.

Homebrew’s analytics records the following different events:

The install event category and the Homebrew formula from a non-private GitHub tap you install plus any used options (e.g. wget --HEAD) as the action. This allows us to identify which formulae where work should be prioritised, as well as how to handle possible deprecation or removal of any.
The install_on_request event category and the Homebrew formula from a non-private GitHub tap you have requested to install (e.g. when explicitly named with a brew install) plus options. This allows us to differentiate the formulae that users intend to install from those pulled in as dependencies.
The cask_install event category and the Homebrew cask from a non-private GitHub tap you install as the action. This allows us to identify which casks where work should be prioritised, as well as how to handle possible deprecation or removal of any.
The build_error event category and the Homebrew formula plus options that failed to install as the action, e.g. wget --HEAD. This allows us to identify formulae that may need fixing. The details or logs of the build error are not sent.

Brew's discussion on collecting statistics by default (with opt-out) vs opt-in: Homebrew/brew#142. They ultimately went with an opt-out approach.

[v1gnesh]

Considering the general predisposition of mainframe groups, I'm guessing opt-in will be preferred.
Install, upgrade, or uninstall counters can be added to those binaries as you say, however, reporting them back is a whole bunch of conversations (within companies), and I'm skeptical of what may come out of it.

How about adding a metadata field to track downloads in CI. That way, we use GH's stats, but subtract our CI counts.

[IgorTodorovskiIBM]

Brew uses:

analytics [subcommand]
Control Homebrew’s anonymous aggregate user behaviour analytics. Read more at https://docs.brew.sh/Analytics.

brew analytics [state]
Display the current state of Homebrew’s analytics.

brew analytics (on|off)
Turn Homebrew’s analytics on or off respectively.

[MikeFultonDev]

I like the UUID - that's nice and simple.
For consistency perhaps we have a --opt-out option (which is the default) but it might make people feel better to explicitly opt out.
Does gitrows require any new 'holes' people would need to punch in the firewall? If so, then that could be an issue, but then again, people that have strict firewalls may all be in the 'opt out' category anyway? I like the idea of being transparent.
So - agree with all your suggestions.

[v1gnesh]

If not provided, the user would be asked a question like this:

Below the question, it would be nice to specify an example of what's sent... with sample data.

[v1gnesh]

UUID may be helpful in future troubleshooting too.
Even for opt out folks, in the future, when there's proper support (or even just the existing support via GitHub), having their zopen-{verb} actions tracked & stored locally may be handy.

[DevonianTeuchter]

Even for opt out folks, in the future, when there's proper support (or even just the existing support via GitHub), having their zopen-{verb} actions tracked & stored locally may be handy.

There is an audit log that should be available (in rootfs/var/log) - utilising that more in the "verbs" would be good practice

[MikeFultonDev]

Brew uses:

analytics [subcommand]
Control Homebrew’s anonymous aggregate user behaviour analytics. Read more at https://docs.brew.sh/Analytics.

brew analytics [state]
Display the current state of Homebrew’s analytics.

brew analytics (on|off)
Turn Homebrew’s analytics on or off respectively.

Love it. So I like us creating zopen analytics with the on/off option. Consistency with other tools is nice.