malasada

A serverless function to resize and convert images on 52Poké Wiki.

It is inspired by this example of Serverless Framework, and contains two features:

• Generate thumbnails for media files on MediaWiki
• Convert images to WebP to reduce bandwidth costs

On Wikimedia wikis, Thumbor is used for similar tasks.

Pre-requisites

This function is designed for use on MediaWiki installations which have AWS extension configured to store images in S3. You also need the following:

• AWS Credentials for Serverless Framework.

Deployment

MediaWiki

This function assumes the images is stored in wiki/ path in a single S3 bucket, hashLevels be 2, and generating thumbnail on parse be disabled.

$wgFileBackends['s3'] = [
    'class'              => 'AmazonS3FileBackend',
    'name'               => 'AmazonS3',
    'wikiId'             => 'wiki',
    'lockManager'        => 'redisLockManager',  // remove this if a lock manager isn't in use
    'containerPaths'     => [
        'wiki-local-public'  => '<s3-bucket>/wiki',
        'wiki-local-thumb'   => '<s3-bucket>/wiki/thumb',
        'wiki-local-temp'    => '<s3-bucket>/wiki/temp',
        'wiki-local-deleted' => '<s3-bucket>/wiki/deleted',
    ]
];

$wgLocalFileRepo  =  [
    'class'              => 'LocalRepo',
    'name'               => 'local',
    'backend'            => 'AmazonS3',
    'scriptDirUrl'       => $wgScriptPath,
    'url'                => wfScript( 'img_auth' ),
    'hashLevels'         => 2,
    'deletedHashLevels'  => 3,
    'zones'             =>  [
        'public'  => [ 'url' => 'https://<s3-public-url>/wiki' ],
        'thumb'   => [ 'url' => 'https://<s3-public-url>/wiki/thumb' ],
        'temp'    => [ 'url' => false ],
        'deleted' => [ 'url' => false ]
    ],
    'transformVia404' => true
];

$wgGenerateThumbnailOnParse = false;

wfLoadExtension( 'AWS' );
$wgAWSCredentials = [
        'key'    => '<aws-api-key>',
        'secret' => '<aws-secret-key>',
        'token'  => false
];
$wgAWSRegion = '<s3-region>';

On 52Poké Wiki, we serve original images for logged-in users, and WebP-compressed images for anonymous users. This is configured via Lazyload extension.

Malasada

1. Set up AWS cridentials for Serverless.

2. Deploy malasada:

sls deploy --stage ${STAGE} --bucket ${BUCKET} --region ${REGION}

Nginx

Nginx (or any other front-end web server) needs to be configured to proxy requests to S3 and handle 404 errors to this lambda function.

This example uses separated domain for original and WebP images, but Accept header may be preferred.

For domains without WebP compression:

location / {
    add_header Access-Control-Allow-Methods GET;
    proxy_pass http://<s3-bucket>.<s3-region>.amazonaws.com;
    proxy_redirect     off;
}

location ~ ^/wiki/thumb/(archive/)?[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/([0-9]+)px-.*$ {
    proxy_pass http://<s3-bucket>.<s3-region>.amazonaws.com;
    proxy_redirect     off;
    proxy_intercept_errors on;
    error_page     404 502 504 = @thumb;
}

location @thumb {
    internal;
    proxy_pass         https://<api-gateway-endpoint>/<api-gateway-stage>$request_uri;
    proxy_ssl_server_name on;
}

location ~ ^/wiki/deleted/ {
    deny all;
}

For domains with WebP compression:

location / {
    proxy_pass http://<s3-bucket>.<s3-region>.amazonaws.com/webp-cache$request_uri;
    proxy_redirect     off;
    proxy_intercept_errors on;
    error_page     404 502 504 = @webp;
    break;
}

location @webp {
    internal;
    proxy_pass https://<api-gateway-endpoint>/<api-gateway-stage>/webp$request_uri;
    proxy_ssl_server_name on;
}

location ~ ^/wiki/deleted/ {
    deny all;
}

Cache Purging

WebP cache needs be deleted when the upstream media file is updated. A DELETE request can be sent to https://<api-gateway-endpoint>/<api-gateway-stage>/webp/<s3 object path> to delete the WebP cache.

On 52Poké Wiki, we use timburr to handle cache purging.

purge:
  entries:
  - host: <s3-public-url>
    method: DELETE
    uris:
    - "https://<api-gateway-endpoint>/<api-gateway-stage>/webp#url#"

rules:
- name: purge
  topic: cdn-url-purges
  taskType: purge

Configuration

Feel free to look into and modify the TypeScript files and serverless.yml to customize this project for your own needs.

License

MIT