diff --git a/source/guides/configure-app-framework.rst b/source/guides/configure-app-framework.rst new file mode 100644 index 00000000000..a7c9b3c5f50 --- /dev/null +++ b/source/guides/configure-app-framework.rst @@ -0,0 +1,10 @@ +Configure the Apps Framework +============================ + +The Apps framework includes a built-in slash command, ``/apps settings``, that allows for the configuration of system-wide settings. + +Configuration settings +~~~~~~~~~~~~~~~~~~~~~~~ + +- **Enable Developers Mode**: Enable debugging feature for the Apps Plugin. Disabled by default. +- **Enable HTTP apps**: Allow HTTP apps to be installed. Disabled by default on Cloud servers. diff --git a/source/guides/deploy-aws.rst b/source/guides/deploy-aws.rst new file mode 100644 index 00000000000..33fe3882fd7 --- /dev/null +++ b/source/guides/deploy-aws.rst @@ -0,0 +1,157 @@ +Deploy Apps to AWS +================== + +Deployment in self-managed environments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An App designed and bundled for AWS Lambda can be deployed to the customer's own +AWS environment, and then installed on a self-managed ("on-prem") Mattermost +instance. Note that the only AWS-hosted apps available on Mattermost Cloud instances are the ones by Mattermost. + +For details on how to develop and package apps for AWS see `Package / AWS `_. + +There are three steps required to enable AWS applications on a self-managed Mattermost instance. + +1. Initialize the AWS environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set up AWS S3 bucket +^^^^^^^^^^^^^^^^^^^^ + +You will need to create an S3 bucket within AWS or use an existing bucket. + +1. Visit https://s3.console.aws.amazon.com +2. Create a bucket. +3. **Bucket name**: Give your bucket a name, later used in ``MM_APPS_S3_BUCKET`` environment variable. +4. **AWS Region**: + a. Select your region + b. Save the slug value for later, to be used in ``MM_APPS_AWS_REGION`` environment variable (Example: ``us-east-1``) + - Corresponding Lambdas will be deployed in the same region +5. **Block Public Access settings for this bucket** + a. (Optional) Check **Block *all* public access** +6. Select **Create Bucket**. + +Create a privileged IAM user access key and secret +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You will need an access key and secret so that ``appsctl`` can deploy the app. These credentials can come from creating an IAM user, using a privileged IAM user, or even using the AWS account owners personal access key. Please follow the instructions provided by AWS (https://aws.amazon.com/premiumsupport/knowledge-center/create-access-key/) to complete these steps and save the ``Access key ID`` and ``Secret access key`` values. + +Set AWS_DEPLOY environment variables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Open a terminal where you installed the Apps plugin and set the following variables to the AWS credentials just created and saved: + +- ``MM_APPS_DEPLOY_AWS_ACCESS_KEY`` +- ``MM_APPS_DEPLOY_AWS_SECRET_KEY`` + +Set the following environment variables based on bucket name and region from Step 1: + +- ``MM_APPS_S3_BUCKET`` +- ``MM_APPS_AWS_REGION`` + +Initialize the AWS resources +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following command will create Mattermost invocation credentials and policy for use with AWS. + +``go run ./cmd/appsctl aws init [flags...]`` + +It will create or update: + +- An IAM user that will be used by the server to invoke the app (``mattermost-apps-invoke``). +- An IAM group that the user belongs to, that will be used by the server to invoke the app (``mattermost-apps-invoke-group``). +- An IAM policy used to control access to the Lambda and S3 resources, used by the invoke group (``mattermost-apps-invoke-policy``). +- An IAM role used to execute app's functions (``mattermost-apps-execute-lambda-role``) with the ``AWSLambdaBasicExecutionRole`` attached to it. + +Optional Flags for the ``appsctl`` command: + +- ``--create-access-key`` create a new access key and print it out. +- ``--create`` create resources that don't yet exist (user, group, role, access policy). +- ``--group`` name the group to invoke function(s), default ``mattermost-apps-invoke-group``. +- ``--policy`` name the policy used to invoke function(s), default ``mattermost-apps-invoke-policy``. +- ``--user`` name the user to invoke the lambda function(s), default ``mattermost-apps-invoke``. + +The output of the command will contain two "Invoke" environment variables. + +2. Enable AWS Upstream for Mattermost Apps +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To enable AWS Apps installation and use, the Mattermost server must be (re-)started with the following environment variables: + +- ``MM_APPS_AWS_ACCESS_KEY`` An AWS Access Key ID for ``mattermost-apps-invoke`` user. If ``appsctl aws init --create-access-key`` was used, it can be found (once!) in the command's output. +- ``MM_APPS_AWS_SECRET_KEY`` The Secret Key for the access key above. +- ``MM_APPS_AWS_REGION`` The AWS region to use. +- ``MM_APPS_S3_BUCKET`` The name of the AWS bucket for static assets and manifests. + +Restart the Mattermost server to complete your Mattermost and AWS setup. You can now deploy an app to AWS. + +3. Deploy apps to AWS +~~~~~~~~~~~~~~~~~~~~~ + +Deploy an app to AWS +^^^^^^^^^^^^^^^^^^^^ + +To deploy AWS Apps from a bundle use ``appsctl aws deploy {aws-bundle.zip}`` command. It will deploy all necessary resources to AWS, update the invoke policy as needed, and "list" (upload the manifest of) the app in Mattermost server. ``--install`` can be used to automatically install the app once it's deployed. + +Flags: + +- ``--execute-role`` name of the role to use for executing the lambda function, default ``mattermost-apps-execute-lambda-role`` +- ``--install`` install the app onto the Mattermost server once it's been successfully deployed. +- ``--policy`` name of the policy to use to invoke the lambda function, default ``mattermost-apps-invoke-policy``. +- ``--update`` update the lambda function that already exists. + +The command requires that the following environment variables are set: + +- ``MM_SERVICESETTINGS_SITEURL`` must be set to where the Mattermost server APIs can be accessed. +- ``MM_ADMIN_TOKEN`` must be set to access the Mattermost REST APIs. +- ``MM_APPS_AWS_REGION`` must be set to the AWS region where the resources are deployed and accessed. +- ``MM_APPS_S3_BUCKET`` must be the name of the S3 bucket used to store manifests and static assets. +- ``MM_APPS_DEPLOY_AWS_ACCESS_KEY``, ``MM_APPS_DEPLOY_AWS_SECRET_KEY`` are the priviledged credentials used to deploy functions and files to AWS. +- ``MM_APPS_AWS_ACCESS_KEY``, ``MM_APPS_AWS_SECRET_KEY`` are the "unprivileged" credentials used to invoke the functions, used only for the ``appsctl aws test`` command. + +Once deployed, apps can be installed interactively in Mattermost using ``/apps install listed`` command which will show all the deployed apps available for installation in Mattermost. + +Deployed app details +~~~~~~~~~~~~~~~~~~~~ + +AWS Lambda Functions Naming +---------------------------- + +AWS Lambda functions have semantic names, meaning that a function described in the ``manifest.json`` file translates to AWS as ``$appID_$appVersion_$functionName`` to avoid collisions with other apps' or other versions' functions. **appsctl** deploys lambda functions using this naming convention. For example, the name of a ``servicenow`` app's lambda function might be ``com-mattermost-servicenow_0-1-0_go-function``. **appsctl** handles the naming of AWS Lambda functions. The dedicated S3 bucket name is stored in the environment variable: ``MM_APPS_S3_BUCKET``. + +Storage of App Assets and Manifests +------------------------------------ + +The AWS S3 bucket mentioned above is used to store all app static assets and manifest files. + +All files in the static folder of the bundle are considered to be the app's static assets and are stored in the above-mentioned bucket. Stored assets also have semantic keys and are generated using the rule: ``static/$appID_$appVersion/filename``. For example, the ``servicenow`` app's static file key can be accessed at ``"static/com.mattermost.servicenow_0.1.0_app/photo.png"``. + +Storage of Manifest File +------------------------ + +The ``manifest.json`` file of an app is stored in the same S3 bucket with the key: ``manifests/$appID_$appVersion.json``. + +Flow of Deploying Custom Apps to AWS +------------------------------------- + +.. image:: deploy-third-party-aws.png + :width: 800 + +Deployment in Mattermost Cloud +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to be deployed in Mattermost Cloud, an app bundle is uploaded to the specific S3 bucket. On a new app release, a bundle is created by GitLab CI and uploaded to S3. The |Mattermost apps cloud deployer|, running as a k8s cron job every hour, detects the S3 upload, and creates appropriate lambda functions, assets, and manifest the same way the **appsclt** does for the third-party accounts. + +The deployer needs lambda function names, asset keys, and the manifest key to deploy the app. It calls the ``aws.GetProvisionDataFromFile(/PATH/TO/THE/APP/BUNDLE)`` from the Apps Plugin to get the deploy data. Same data can be generated using the command: + +``appsctl generate-terraform-data /PATH/TO/YOUR/APP/BUNDLE`` + +Flow of deploying custom apps to AWS +------------------------------------- + +.. image:: deploy-mm-aws.png + :width: 800 + +.. |Mattermost apps cloud deployer| raw:: html + + Mattermost apps cloud deployer \ No newline at end of file diff --git a/source/guides/deploy-http.rst b/source/guides/deploy-http.rst new file mode 100644 index 00000000000..adbecdb699a --- /dev/null +++ b/source/guides/deploy-http.rst @@ -0,0 +1,8 @@ +Deploy HTTP Apps +================ + +External service providers can offer "remote" apps already deployed as HTTP services. + +Internal organization-specific apps can also be developed and run as HTTP services on the existing hosting infrastructure, for example, using ``systemd``. It's important that your internal apps are only reachable by the Mattermost server, and not from the public internet. + +To install an HTTP app, use the ``/apps install http {URL-to-manifest.json}`` command in Mattermost. diff --git a/source/guides/deploy-mattermost-apps.rst b/source/guides/deploy-mattermost-apps.rst new file mode 100644 index 00000000000..b34d3327dbb --- /dev/null +++ b/source/guides/deploy-mattermost-apps.rst @@ -0,0 +1,39 @@ +App deployment +============== + +.. toctree:: + :maxdepth: 1 + :hidden: + :titlesonly: + + Settings + HTTP + AWS + Package Apps for AWS + OpenFaaS + +- In **Mattermost Cloud**, all apps are deployed to the Marketplace by Mattermost staff. They can be installed onto a specific Mattermost instance using the ``/apps install listed`` command. No special configuration is required; the ``/apps install`` command should be enabled and functional by default. + +- **Self-managed Mattermost** installations can use external Apps as HTTP services that have already been deployed, or can deploy App bundles on self-managed hosting or serverless platforms. Currently, `AWS Lambda `_, `OpenFaaS `_, and Kubeless deployments are supported. The `appsctl` command can be used to deploy app bundles to these environments. + + Self-managed customers can also install external `HTTP `_ apps, with no need to deploy them. + + The ``appsctl`` CLI tool is provided to deploy AWS and OpenFaaS apps in self-managed environments. To install ``appsctl``, use the following command: + + .. code-block:: shell + + go install github.com/mattermost/mattermost-plugin-apps/cmd/appsctl@latest + +.. note:: + + If you have a self-hosted Mattermost instance running on AWS EC2, the default Golang version is 1.13.8. + +Because of this, the Golang install command will give an error stating: + +.. code-block:: shell + + can't load package: package github.com/mattermost/mattermost-plugin-apps/cmd/appsctl@latest: cannot use path@version syntax in GOPATH mode + +To fix this, update your Golang version to the latest release and run the command again. When deploying to AWS, the ``appsctl`` binary is present in the ``Home`` directory inside the ``go/bin`` folder. + +To use ``appsctl``, run the following command: ``./go/bin/appsctl`` diff --git a/source/guides/deploy-mm-aws.png b/source/guides/deploy-mm-aws.png new file mode 100644 index 00000000000..a07aaafa004 Binary files /dev/null and b/source/guides/deploy-mm-aws.png differ diff --git a/source/guides/deploy-openfaas.rst b/source/guides/deploy-openfaas.rst new file mode 100644 index 00000000000..0c101ef3ac9 --- /dev/null +++ b/source/guides/deploy-openfaas.rst @@ -0,0 +1,56 @@ +Deploy Apps to OpenFaas and faasd +================================= + +OpenFaaS Apps +------------- + +What is OpenFaaS? +~~~~~~~~~~~~~~~~~ + +OpenFaaS makes it relatively simple to deploy both serverless functions and existing code in stateless containers on Kubernetes or directly on Linux hosts, on any public or private cloud. Learn more at |OpenFaas.com|. + +Package an App for OpenFaaS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For details on how to develop and package apps for AWS, see |serverless example|. + +Deploy with ``appsctl`` +~~~~~~~~~~~~~~~~~~~~~~~~ + +An App designed and bundled for OpenFaaS can be deployed to the customer's own OpenFaaS or faasd environments, and then installed on a self-managed ("on-prem") Mattermost server, using the ``appsctl openfaas deploy`` command. + +This command requires that |faas-cli| is installed and configured, with credentials sufficiently privileged to deploy functions. + +OpenFaaS requires a |docker registry| to pass the images that it builds to the function instances, so ``docker`` must have been configured, and ``docker login`` done, with sufficient credentials to push to the registry of choice. + +To deploy OpenFaaS Apps use ``appsctl openfaas deploy {openfaas-bundle.zip}`` command. It will deploy all functions in the bundle, and "list" (upload the manifest of) the app in Mattermost server. ``--install`` can be used to automatically install the app once it's deployed. + +**Flags:** + +- ``--docker-registry``: the docker registry name (will be used as a prefix for the image name) to push the function images to. +- ``--install``: install the app onto the Mattermost server once it's been successfully deployed. +- ``--update``: update the function that already exists. + +The command requires that the following environment variables are set: + +- ``MM_SERVICESETTINGS_SITEURL``: must be set to where the Mattermost server APIs can be accessed. +- ``MM_ADMIN_TOKEN``: must be set to access the Mattermost REST APIs. +- ``OPENFAAS_URL``: must be set to the admin (root) URL of the OpenFaaS installation. + +Once deployed, apps can be installed interactively in Mattermost using ``/apps install listed`` command. + +.. |OpenFaas.com| raw:: html + + OpenFaas.com + +.. |serverless example| raw:: html + + serverless example + +.. |faas-cli| raw:: html + + faas-cli + +.. |docker registry| raw:: html + + docker registry \ No newline at end of file diff --git a/source/guides/deploy-third-party-aws.png b/source/guides/deploy-third-party-aws.png new file mode 100644 index 00000000000..e4d1a30fd71 Binary files /dev/null and b/source/guides/deploy-third-party-aws.png differ diff --git a/source/guides/deployment.rst b/source/guides/deployment.rst index 1162717dbb2..f7ff4d96314 100644 --- a/source/guides/deployment.rst +++ b/source/guides/deployment.rst @@ -11,20 +11,21 @@ Learn how to install, deploy, and scale Mattermost for teams and organizations o :hidden: :titlesonly: - Prepare your Mattermost database - Deploy on Ubuntu - Deploy using Docker - Deploy with a tarball - Deploy using Omnibus - Deploy using Kubernetes - Deploy on Red Hat - Prepare for your Mattermost deployment - Deployment guides - Upgrade Mattermost - Scale Mattermost - Deployment troubleshooting - Changelogs - Additional server installation guides + Prepare your Mattermost database + Deploy on Ubuntu + Deploy using Docker + Deploy with a tarball + Deploy using Omnibus + Deploy using Kubernetes + Deploy on Red Hat + Prepare for your Mattermost deployment + Deployment guides + Deploy Mattermost Apps + Upgrade Mattermost + Scale Mattermost + Deployment troubleshooting + Changelogs + Additional server installation guides .. tip:: @@ -39,7 +40,8 @@ Learn how to install, deploy, and scale Mattermost for teams and organizations o * :doc:`Deploy using Omnibus ` - An entire Mattermost installation on a single server. * :doc:`Deploy using Kubernetes ` - Install using the Mattermost Helm Chart or Operator and simplified updates. * :doc:`Deploy on Red Hat ` - Support for all current Red Hat Enterprise Linux platforms with a tarball. -* :doc:`Deployment guides ` - for administrators who are ready to integrate Mattermost with their organization’s IT infrastructure. +* :doc:`Deployment guides ` - for administrators who are ready to integrate Mattermost with their organization’s IT infrastructure. +* :doc:`Deploy Mattermost Apps ` - Learn how to deploy Mattermost Apps to your server. * :doc:`Prepare for your Mattermost deployment ` - Review software and hardware requirements for Mattermost server, and plan out your Mattermost rollout. * :doc:`Upgrade Mattermost ` - Learn how to stay up to date with the latest features and improvements. * :doc:`Scale Mattermost ` - Learn how to scale and monitor your Mattermost deployment. diff --git a/source/guides/package-aws.rst b/source/guides/package-aws.rst new file mode 100644 index 00000000000..adcfacbeb1d --- /dev/null +++ b/source/guides/package-aws.rst @@ -0,0 +1,65 @@ +Package Apps for AWS Lambda +=========================== + +Developers must prepare apps for deployment to AWS, which includes creating an app bundle and making the app runnable as an AWS Lambda function. + +App Bundle +^^^^^^^^^^ + +An app bundle is a convenient way to deliver an app to the Mattermost ecosystem. It provides a way to organize code and resources needed for an app to run. An app bundle is created by the developer of the app. Mattermost uses app bundles to deploy and install/uninstall apps. + +The app bundle contains a ``manifest.json`` file, a ``static/`` folder (optional), and one or several lambda function bundles. + +- The ``static/`` folder contains all the static files the app needs. For the Mattermost AWS apps, static files are automatically deployed and stored in the dedicated AWS S3 bucket. Apps have unlimited access to them by providing the static file name to the Apps Plugin. For the third-party hosted AWS apps, static files are stored in a different S3 bucket (specified by the third-party). For the HTTP apps, when creating a server, the developer should store the static files in the ``/static/$FILE_NAME`` relative URL. +- The ``manifest.json`` file contains details about the app such as appID, appVersion, appType (HTTP or an AWS app), requested permissions, requested locations, and information about the functions such as function path, name, runtime, and handler. +- Each of the lambda function bundles is a valid and runnable AWS Lambda function, deployed in AWS by the |Mattermost Apps Cloud Deployer|. The AWS Lambda function bundle is a ``.zip`` file which contains scripts or compiled programs and their dependencies. Note that it must be smaller than 50 MB. Exact specification of the bundle varies for different runtimes. For example, one can see more details for ``node.js`` bundles |bundles here|. +Making your app runnable as an AWS Lambda function +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In order for your app to run as an AWS Lambda function, it must use one of the supported languages for AWS Lambda. You can find the list |list here|. + +It's important to design an app in a _stateless_ way, as its lifetime only spans one request. No persistent information should be stored except using the |store API endpoints| provided by the Apps Framework. + +A language library is used to emulate an HTTP to your app. For Go, you might use |aws-lambda-go-api-proxy|. + +Finally, you need to define the AWS function in the manifest of your app by adding ``aws_lambda`` to it, which has the following fields: + +- ``path``: The lambda function with its path being the longest-matching prefix of the call's path which will be invoked for a call. +- ``name``: A human-readable name. +- ``handler``: The name of the handler function. +- ``runtime``: The AWS Lambda runtime to use. + +For a Go app, the manifest snippet would look like this: + +.. code-block:: json + + { + "aws_lambda": [ + { + "path": "/", + "name": "go-function", + "handler": "$YOUR_APP_NAME", + "runtime": "go1.x" + } + ] + } + +.. |Mattermost Apps Cloud Deployer| raw:: html + + Mattermost Apps Cloud Deployer + +.. |bundles here| raw:: html + + here + +.. |list here| raw:: html + + here + +.. |store API endpoints| raw:: html + + store API endpoints + +.. |aws-lambda-go-api-proxy| raw:: html + + aws-lambda-go-api-proxy \ No newline at end of file