[Performance] Direct rendering to OffscreenCanvas(es)

[djahandarie]

Hi! I'm using resvg-js in wasm in the browser (in a Chrome extension, yomitan).

I use it basically like this:

const opts = {
    fitTo: {
        mode: 'width',
        value: m.width,
    },
};
const resvgJS = new Resvg(new Uint8Array(m.content), opts);
const pngData = resvgJS.render();
const pngBuffer = pngData.asPng();

const drawPromises = [];
const imageDecoder = new ImageDecoder({type: 'image/png', data: pngBuffer});
drawPromises.push(imageDecoder.decode().then((decodedImage) => {
    return {canvases: m.canvases, image: decodedImage.image};
}));
for (const {canvases, image} of await Promise.all(drawPromises)) {
    for (const c of canvases) {
        c.getContext('2d')?.drawImage(image, 0, 0, c.width, c.height);
    }
}

(Note: we Promise.all the decode promises in order to have a very tight drawImage loop)

Basically, we need to render a large number of SVGs in a limited amount of time budget, which we currently cannot meet, and we are observing the following:

• Almost 60% of a given render is spent in asPng, and 40% in render for resvg, suggesting a potential 2x speedup
• Even ignoring the numbers, we are clearly doing something very inefficient here by encoding to PNG and then immediately decoding it in order to draw to the canvas(es)

I could see any of the following resvg-js changes potentially helping:

1. Return a bitmap instead of a png. This might do the trick by itself, but there might be a bit of overhead from the much larger uncompressed bitmap being sent across the wasm-js boundary
2. Have resvg-js support drawing directly to a single OffscreenCanvas, and then transfer that canvas back to the caller so we can further copy it to our other canvases with drawImage
3. Have resvg-js support drawing directly to multiple OffscreenCanvas. This would be the most ideal API for us but maybe a bit too overspecialized? Not sure how many other people would need this feature of being able to write the same SVG to multiple canvases.

We would prefer it in order of 3, 2, 1. But ease of implementation is probably in order of 1, 2, 3. And 3 is probably a bit too niche (but don't let me stop you if you think it seems good as it's our ideal...).

What do you folks think? Do any of these options seem appropriate / feasible to include in the library?

[yisibl]

Have you tried .pixels? This doesn't need to be encoded as a PNG.
https://github.com/yisibl/resvg-js/blob/39eaa225b3d9b9274d4e27fe4f01922bd601c06a/__test__/wasm.spec.ts#L65

Also, enabling SIMD will result in faster speeds. #148

[djahandarie]

Thank you for the super fast response! 😄

I wasn't aware of that! It is basically what I was thinking of for my bullet 1.

I tried it out, and indeed it cut out all the useless png encoding/decoding.

const resvgJS = new Resvg(new Uint8Array(m.content), opts);
const pixels = resvgJS.render().pixels;
drawPromises.push(createImageBitmap(new ImageData(new Uint8ClampedArray(pixels), m.width, m.height)).then((bitmap) => {
    return {canvases: m.canvases, image: bitmap};
}));

(There might be some better way than using ImageData > ImageBitmap here but I couldn't figure it out lol... this part of web APIs is so confusing. Maybe something with VideoFrame would be a more direct way of using the array? No clue)

Anyways, this gave us a nice performance boost.

The SIMD stuff also looks super nice!! I'm not sure we want to take on the burden of compiling this in our build since we aren't a rust project, but if push comes to shove with performance we might do it...

Thank you so much.

[yisibl]

Can you just use putImageData? Here is an example: #304

Where does yomitan use resvg-js? I didn't find the code.

[djahandarie]

Yes, it took a tiny bit of refactoring but putImageData worked! Thanks.

I hadn't committed it yet -- just did now. You can see it here: https://github.com/yomidevs/yomitan/tree/performance-canvas in the latest commit. (The code is a huge mess right now unfortunately, will get it cleaned up soon so we can merge it to master)