Limit number of the transformations threads

[striker2000]

When a search bot requests many images at the same time and these images are not in the cache, all transformation goroutines start in parallel causing the huge memory allocation, that leads to the termination of the process by OOM.

This fix is adding the limiter in the TransformingTransport that limits the number of running transformation threads to the number of logical CPUs. The metric http_requests_in_flight is adding as well to monitor the length of the requests queue.

Fixes #200.

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[codecov]

Codecov Report

Merging #235 into master will increase coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #235      +/-   ##
==========================================
+ Coverage   89.16%   89.34%   +0.17%     
==========================================
  Files           6        6              
  Lines         674      685      +11     
==========================================
+ Hits          601      612      +11     
  Misses         50       50              
  Partials       23       23              

Impacted Files	Coverage Δ
imageproxy.go	85.00% <100.00%> (+0.55%)	⬆️
metrics.go	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c7d08f...b2a9772. Read the comment docs.

[striker2000]

@googlebot I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[willnorris]

Hey @striker2000, thanks for this patch! Sorry it's taken a couple of week to get back to you.

I think this is certainly a good approach; my only reservation is about where exactly inside of TransformingTransport.RoundTrip that we block. I haven't had a chance to profile this code to know exactly where the out of memory issue in #200 is occurring. If it's inside the cache, then where you have the limiter check is probably right. But if it's actually happening during the transform step, then we probably want to block right before calling Transform. That way, the should304 call can still return 304 responses without tying up the limiter.

Does that make sense?

[mdkent]

We're seeing fairly frequent OOM's (~10 a day) which are likely related to transformations. Is anyone using this patch successfully?

[Jorgevillada]

@mdkent thsi PR has a problem with inflight request.

I have updated with the current main branch and this PR. https://github.com/Jorgevillada/imageproxy

error copying response: readfrom tcp 127.0.0.1:8080->127.0.0.1:53150: write tcp 127.0.0.1:8080->127.0.0.1:53150: write: broken pipe

Some request never finish.

with main branch, the behavior with the ram is weird with invalid requests.