Extensions

What Are Extensions?

Extensions are additional features that do not exist in the core Redash implementation but are loaded from a library. The library can be independently maintained in a separate repository, distributed on pypi, and installed along with other packages in requirements.txt An extension can have two parts, a front-end consisting of HTML, CSS, and/or Javascript code and a back-end consisting of Python code. These two parts are packaged and loaded in different ways.

How Do Backend Extensions Work?

Packaging Backend Extensions

Backend extensions are created with a mechanism in Python called "entry points". Entry points allow for individual components of a library to be discovered and loaded by other code. In order for a component to be loaded as an extension, first it needs to be registered as an entry point in the extension library's setup.py file.

setup(
    name='extension_package_name'
    entry_points={
        'redash.extensions': [
            'entry_point_name = package_name.file_name:entry_function_name',
            ...
        ]
    }
    ...
)

A full example can be found here

Loading Backend Extensions

Once the extensions library is installed in the the Redash process's Python environment (e.g. pip install redash-stmo), the extensions can then be discovered and loaded as needed like this:

from pkg_resources import iter_entry_points

extension_return_values = {}
for entry_point in iter_entry_points('redash.extensions'):
    extension = entry_point.load()
    extension_return_values[entry_point.name] = extension(flask_app)

Note:

• redash.extensions is the same as specified in setup.py
• entry_point.load() returns the function entry_function_name specified in setup.py
• entry_point.name comes from the entry_point_name specified in setup.py

A full example can be found here.

In Redash, back-end extensions are initialized at runtime when a flask app is created.

How Do Frontend Extensions Work?

Packaging Frontend Extensions

Before getting into how frontend extensions are packaged, it’s important to understand at a high level how Redash’s frontend is bundled and built. Redash uses webpack, a tool that minimizes and optimizes code from multiple files into bundles so they can be efficiently loaded and run in the browser. So the ultimate goal when packaging and loading the frontend extensions is to make them available in a directory that webpack will be looking at to bundle.

Frontend extensions are packaged in a similar manner as backend extensions, using "entry points". However, since we are dealing with non-python files, they are treated as static files when the extension package is installed. The installed static files must then be copied into the directory where webpack will look for them.

To start, we specify in setup.py that we want to include "static" files by adding an include_package_data parameter. We also add a line to MANIFEST.in: recursive-include src/redash_stmo *.html *.js *.css. As one might expect, this specifies that we recursively search for js/html/css files to be included when packaging. Also, for extensions that are frontend only, entry_function_name is not required in setup.py. Instead, entry_point_name must match the extension directory’s name. So now our setup.py may look like this:

setup(
    name='extension_package_name'
    include_package_data=True,
    entry_points={
        'entry.point.group.name': [
            'entry_point_name = package_name.file_name:entry_function_name',
            'entry_point_name = package_name.frontend_extension_name,
            ...
        ],
    }
    ...
)

Loading Frontend Extensions

Now the frontend extension files will be available in the installation directory of the extension library. But we need to move them into a directory where webpack can discover them in Redash. So we create a script, bin/bundle-extensions which will run as part of the Redash build process to help with this.

We start by creating a directory where the frontend extensions code will live, client/app/extensions. We save this directory's path in an environment variable so it can be referenced from webpack.config.js:

EXTENSIONS_RELATIVE_PATH = os.path.join('client', 'app', 'extensions')
EXTENSIONS_DIRECTORY = os.path.join(
       os.path.dirname(os.path.dirname(__file__)),
       EXTENSIONS_RELATIVE_PATH)

if not os.path.exists(EXTENSIONS_DIRECTORY):
    os.makedirs(EXTENSIONS_DIRECTORY)
os.environ["EXTENSIONS_DIRECTORY"] = EXTENSIONS_RELATIVE_PATH

Now we can iterate over all extensions in a similar fashion as we do for backend extensions. We look for frontend extensions by checking for a bundle directory then we copy them into our new extensions directory.

for entry_point in iter_entry_points('redash.extensions'):
    # This is where the frontend code for an extension lives
    # inside of its package.
    content_folder_relative = os.path.join(
        entry_point.name, 'bundle')
    (root_module, _) = os.path.splitext(entry_point.module_name)

    if not resource_isdir(root_module, content_folder_relative):
        continue

    content_folder = resource_filename(root_module, content_folder_relative)

    # This is where we place our extensions folder.
    destination = os.path.join(
        EXTENSIONS_DIRECTORY,
        entry_point.name)

    copy_tree(content_folder, destination)

We also add an alias in webpack.config.js called % to reference the EXTENSIONS_DIRECTORY environment variable created above.

Finally, once the build step is complete, extensions will be registered in a similar fashion to the components/services/pages etc. that are registered in client/app/config/index.js with this function:

function registerExtensions() {
  const context = require.context('%', true, /^((?![\\/]test[\\/]).)*\.js$/);
  registerAll(context);
}

This will call all available functions in the extensions directory that were exported as default (e.g. export default function init() {...})

How Do I Create a New Extension?

Now that we've covered how frontend and backend extensions work, creating a new extension is quite easy!

Creating a Backend Extension

Creating a backend extension is as simple as adding an entry point to the redash.extensions group in setup.py in the extensions codebase to point to your extension's entry function. Then you can fill in the code for your entry function as you wish.

Note that all backend entry functions are passed an instance of a Flask app that they can freely modify. So your entry function will need to receive app as a parameter.

Here is an example of an entry function and its entry point in setup.py

Creating a New Frontend Component

Creating a frontend extension will similarly require adding an entry point to the webpack.bundles group in setup.py. frontend_extensions.py will also need to be updated with a dict as described in Packaging Frontend Extensions above.

To create a new frontend component, you will need an extension point in the main Redash codebase for your extension to hook into. This will simply be an empty HTML tag for that component. e.g.:

<datasource-link></datasource-link>

Now your extension code can implement the datasource-link component in exactly the same way it would be done in the Redash codebase:

export default function init(ngModule) {
  ngModule.component('datasourceLink', {
    template,
    controller,
  });
}

General Notes on Extensions:

• The extensions run in scope of Redash and have full import access to all Redash modules. For example, the following is acceptable code to place in an extension:
  • from redash import models (Python)
  • import { Policy } from '@/services/policy'; (Javascript)
• It is possible to import a Redash module and monkey-patch it to add new functionality. However, there is a higher risk of losing forward compatibility this way. It is best whenever possible to create an extension point in the core Redash code (i.e. a function that can be called by an extension) to facilitate logical changes that an extension wants to make.