cpu, Memory usage spike when endpoints returns a large json file

[zhiyanshao]

We got cpu, memory and file descriptors spikes when the endpoint returns a large json object. pprof shows most of them are spent on yaag/yaag.GenerateHtml and yaag/yaag.Init. Down the tree it, it calls encoding/json.Mashal (47%) and html/template.htmlReplacer(34%) which calls runtime.slicebytetostring(16%) and bytes.(*Buffer).WriteString.

The memory doesn't get GC-ed quickly.

[aniketawati]

This is a design issue. With current design, it can't handle large data properly.
For such use cases, yaag would need to support streaming writes to data object file. For that, it would most probably need to shift from json to protobuf. This can be picked up as a feature, so will mark it as such.

But for the current state, make sure that you don't run yaag in production. This is only designed to generate api docs with tests or manual api calls. Running this on production is not advisable at all.