Colcon hangs when stdout / stderr is redirected and output line is longer than default buffer size

[vcomito-apexai]

I have to add a reproducer use case, but colcon seems to hang when a subprocess (e.g. cmake) writes a single stdout/stderr line longer than the default buffer size.

This is probably due to subprocess and PIPE which will hang when no newline is found in the buffer.

Increasing the buffersize in
https://github.com/colcon/colcon-core/blob/master/colcon_core/subprocess.py#L159 seems to help

I will add a reproducer

[cottsay]

Just how long are we talking?

On my Fedora 39 machine, I wasn't able to get a hang with a 4 million character long line. It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

[vcomito-apexai]

Just how long are we talking?

On my Fedora 39 machine, I wasn't able to get a hang with a 4 million character long line. It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

iirc 100'000 characters. Also, apparently, the default size is 64 KiB. Sorry for not adding further instructions for reproducing this yet.

It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

Was the output redirected to a file? I don't know exactly how this indirectly changes the behavior of the PIPE of the children processes

colcon ... > stdout.txt