Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spec: console output #190

Open
zkochan opened this issue Nov 29, 2016 · 10 comments
Open

Spec: console output #190

zkochan opened this issue Nov 29, 2016 · 10 comments

Comments

@zkochan
Copy link
Collaborator

zkochan commented Nov 29, 2016

I am not sure we should standardize this. However, logging is such a pain.... Just look at npm's logging, it has like dozens of supporting libraries and a lot of complexity to work with them.

I think we can have a standard like TAP and different reporters could make it prettier.

@alexanderGugel
Copy link
Owner

What about using something like ndjson? budo uses it and it looks great!

There are tons of formatters for it as well I think, e.g. garnish.

@zkochan
Copy link
Collaborator Author

zkochan commented Nov 30, 2016

ndjson looks good. We'll have to describe in what format should the logs be and some standard set of log types.

@billiegoose
Copy link
Collaborator

I like newline delimited JSON. I first encountered it in bunyan logging utility, but have seen it many places since, and have used it for many of my own projects since (and not just for logging, but any time series data).

But I'm not sure it's right for this use case. I assume the point is to standardize installer output so that debugging failed installs is easier. In my CI build steps I used to run npm installwith the verbosity level at 'silly', so I can try to figure out whether the install failed due to a network timeout, a package being unavailable (404), my npm cache server (50x), a package's node-gyp build step failing, or the OOM killer. However, the npm output would be enormous and be unmanageable, particularly in a CI web interface showing console output. I eventually wrote my own installer that reads package.json and executed 'npm install' for each dependency separately. This greatly reduced npm's memory usage, AND allowed me to handle errors individually. Instead of bailing at the first failed install, it would keep going, and at the end present a report listing which packages failed to install, so I could decide whether I actually needed to rebuild the image (I may not be testing the part of the app that requires those modules), instantly informed me whether the problem was with the new dependencies I just added, a problem with npmjs itself (symptom: 3-10 packages at random out of 100 failed), or with my build environment (node-gyp failed to compile X, or installing a dependency from Github failed because the git+ssh port was blocked).

@billiegoose
Copy link
Collaborator

So I had never thought of it before you mentioned TAP, but I think it would be really handy for installers to have a --output-tap flag. I'm less convinced that something as rich and flexible as ndjson would actually result in interoperability, because it would be so tempting to keep adding new fields.

Just to be clear, your not thinking of trying to standardize interactive installs right? I think there's too many options in terms of CLI interfaces... someone may decide to do more than just progress bars and write a full nurses interface. Or have dancing nyan cats.

@zkochan
Copy link
Collaborator Author

zkochan commented Dec 11, 2016

What do you mean by interactive installs?

I think we can standardize some basic log types like progress events, resolving, HTTP responses, errors. Everything else can be logged as some nonstandard log type maybe... But if these basic log types will be standardized, I think we could have some pretty nice, tool-agnostic install reporters

@billiegoose
Copy link
Collaborator

I mean installs run in a TTY. To quote an example from the Node manual

$ node -p "Boolean(process.stdout.isTTY)"
true
$ node -p "Boolean(process.stdout.isTTY)" | cat
false

In a TTY, I'd like to have spinners, progress bars, prompts ("This will install 3 million packages... Continue? [Y/n]"), ANSI colors, and other fancy features that look like absolute shit if written directly to a log file. I don't recall how many times, but I had to scroll through log files with npm2 that were 90% filled with that little spinner...

\
|
/
-
\
...

@billiegoose
Copy link
Collaborator

I think we can standardize some basic log types like progress events, resolving, HTTP responses, errors. ... But if these basic log types will be standardized, I think we could have some pretty nice, tool-agnostic install reporters

Sounds ambitious, but I'm not against trying, if only because it will make us carefully consider what type of events our programs could expose. Another possibility, I just want to bring up, would be to ditch the limitations of communicating through stdout and use an EventEmitter. If we choose to write our core installers (e.g. modinst) as a separate package from our cli programs (e.g. modinst-cli), then it would be easy for others to write not just reporters, but also input handling. For instance, right now there are big differences in the command names used by npm (npm install) and yarn (yarn add). I think the yarn names are attempting to improve upon the npm command names, but in doing so they've created friction. Converting old scripts that use "npm install -S" to use yarn would take effort. Instead, we could have one CLI interface, and allow specifying the underlying backend to use! So for example,

nodeinstall express --save --use=pnpm
nodeinstall express --save --use=ied --ied-specific-option
(and to round out)
nodeinstall express --save --use=mi --max-concurrent=4 --retry=3 --continue-on-fail --max-memory=200M

@zkochan
Copy link
Collaborator Author

zkochan commented Dec 12, 2016

Wouldn't that limit the supported package managers to be written in Node? ied is already being rewritten to Go

@billiegoose
Copy link
Collaborator

Grmph. Well.... good point. It has to use whatever the baseline for interprocess communication between languages is these days. Do we have something better than the old stdout & stderr yet? No? Darn.

</rant Like seriously. It's been decades since stdin, stdout & stderr were invented, and there's still no "better" way of passing a number between two processes that doesn't involve serializing it to text and back again? Why can't we at least pass floats... IEEE 754 standardized floating point numbers in 1985. What have Operating Systems designers been working on since then? /rant>

@zkochan
Copy link
Collaborator Author

zkochan commented Jan 14, 2017

I added ndjson to pnpm! Here's the PR: pnpm/pnpm#540

@alexanderGugel, it was a really good idea!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants