Support Offscreen Canvas, FileReader, Image, createImageBitmap APIs

[dtruffaut]

Please support:

• Offscreen Canvas
• FileReader: https://developer.mozilla.org/fr/docs/Web/API/FileReader
• Image: https://developer.mozilla.org/fr/docs/Web/API/HTMLImageElement/Image
• createImageBitmap: https://developer.mozilla.org/en-US/docs/Web/API/createImageBitmap

Just like a regular web worker in the browser would.

It would allow to redimension images and/or play with Tensorflow on the server, which is a secure environment VS the brower.
(Browser is not secure because extensions act as proxies and can break/rewrite the CSP before serving the page to the user)

This feature is constantly asked since 2019 (3 years):

• june 2019: https://community.cloudflare.com/t/rendering-svg-using-canvas-with-cloudflare-workers/90952
• january 2021: https://community.cloudflare.com/t/blob-is-not-defined/233614/4
• october 2021: https://community.cloudflare.com/t/feature-request-support-offscreencanvas-api/315006

Of course there is the option of using Cloudflare Images, but the use cases are limited (only for resizing).
Playing with composable primitives is prefered (image analysis, pixel per pixel, etc).

Examples

DataUrl helps browser to display inlined images.
It is useful for debugging, if you look at the second example below.

FileReader can be handy for snippets like:

// Read a blob and returns a dataUrl
// which is "basically" an inlined thing prefixed with its type
// data:[<mediatype>][;base64],<data>
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs

const toDataUrl = async (blob) => {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.addEventListener('error', () => reject(false), { passive: true });
    reader.addEventListener('load', () => resolve(reader.result), { passive: true });
    reader.readAsDataURL(blob);
  });
}

console.log(await toDataUrl(blob));

Here is an example of createImageBitmap, Offscreen Canvas, FileReader working together:

const resize = async ({ event, width }) => {

  // Get the image
  // While it would be possible to resize the image using this method, 
  // the resize is in reality performed later using OffscreenCanvas
  const image = await createImageBitmap(event.data.data);

  const height = Math.ceil(width * image.height / image.width);
  
  // Draw an Offscreen Canvas to convert into webp
  const canvas = new OffscreenCanvas(width, height);
  const contextOptions = {
    alpha: false,
    desynchronized: true,
    preserveDrawingBuffer: true
  };
  const ctx = canvas.getContext('2d', contextOptions);
  ctx['imageSmoothingEnabled'] = true;
  ctx['imageSmoothingQuality'] = 'high';
  
  // Resize happens here (image.width -> width ; image.height -> height)
  ctx['drawImage'](image, 0, 0, image.width, image.height, 0, 0, width, height); // source, destination
  
  // Convert into webp
  const webpBlob = await canvas.convertToBlob({ type: 'image/webp', quality: 0.8 });
  
  // Log the webp to debug
  const dataUrl = await toDataUrl(webpBlob);
  console.log(dataUrl);

  // Return the arrayBuffer for analysis, storage, ...etc.
  return webpBlob.arrayBuffer();
}

Having this kind of primitives is interesting, because in Machine Learning, developers often need to resize, grayscale, extract alpha... and perform various and lots of granular operations.

[dom96]

It would be interesting to explore what WASM implementations of image manipulation libraries are out there. https://silvia-odwyer.github.io/photon/ looks pretty nice, it is also possible to get Python's Pillow library running in Workers through WASM (with some big caveats currently).

[hkochniss]

the importance of theses usecases and the pain to put them into the runtime (this would be a really bad idea for many reasons) was prob. the reason for https://blog.cloudflare.com/introducing-workers-browser-rendering-api/
I think rendering API really covers most of the use cases, like social image generation IMO

[dtruffaut]

The browser rendering API covers SEO cases.

But it does not address the need for image manipulation (Filters, Analysis, Feature extraction, Machine Learning, ...etc.).

Machine Learning, which is a pretty important use case nowadays, always requires image manipulation before entering into the Learning process. The only way to perform those pre processing operations securely is on the server. There is a card to play in favor of Cloudflare Workers.

[CanRau]

Wow just now stumbled upon this 😍

I mean you might be able to manipulate an image on your website with a dynamic route using CSS & JS and then take a screenshot 🤔

Curious why it's been build on Puppeteer over Playwright❔

[klokan]

Wouldn't https://www.npmjs.com/package/canvaskit-wasm from Google Engineers be a possible solution?

Anybody could test this?

I believe the current default limit for Worker code size is set to 5MBytes, but it may be increased via https://www.cloudflare.com/larger-scripts-on-workers-early-access/ ...

[hkochniss]

Hrm I think the real size is more like 4,2MB, according to https://unpkg.com/browse/[email protected]/bin/
So when trimming away fonts (Github Readme mentions the option) and some other stuff it seems also be possible to half that even.. so I think it should already run in workers?

[waki285]

This does not seem to be designed for Edge Runtime... process is not defined error.