Skip to content

Commit

Permalink
Documenting the new automated repo creation workflow (#4566)
Browse files Browse the repository at this point in the history
  • Loading branch information
AndrewRTsao authored Apr 12, 2024
1 parent cc69e13 commit 3a4f500
Show file tree
Hide file tree
Showing 11 changed files with 88 additions and 30 deletions.
83 changes: 68 additions & 15 deletions docs/docs/deploy/existing-project/existing-project.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,48 +26,99 @@ graph LR;

## Push the project to Github

Rill Cloud connects to a repository on Github containing a Rill project, and continuously deploys that project on every push. Therefore, your project must be on Github before you deploy it to Rill.
Rill Cloud connects to a repository on Github containing a Rill project, and continuously deploys that project on every push. Rill Cloud has the ability to auto-create a Git repository on your behalf when first deploying your project or you have the option to manually create the Git repository yourself before deploying the project to Rill Cloud.

- If your project is not yet on Github, follow the steps on [https://github.com/new](https://github.com/new) to create a new repository and push your project files to it.
- If your project is already on Github, make sure you have permission to grant access to it. If you're deploying a project controlled by someone else, first fork or copy it to a repository in your account.
### Automated repository creation

If you'd like Rill Cloud to automaticaly create the Git repository for a Rill project that you deploy, you can skip to the [next step](#deploy-to-rill-cloud).

:::note GitHub app permissions

This assumes that the installed Github app in your organization has write access. If unsure, please check with your Github admin.

:::

### Manual repository creation

If you'd like to create the Git repository manually, the project must be on Github <u>before</u> you deploy it to Rill.
- If your project is not yet on Github, you can follow the steps on Github [here](https://github.com/new) to create a new repository and push your project files to it.
- If your project is already on Github, make sure you have appropriate permissions to grant access to it. If you're deploying a project controlled by someone else, you may need to fork or copy it to a repository in your account.

:::info Custom Git repository name

When Rill attempts to create a Git repository on your behalf, _the new repository will mirror the name of your Rill project_. If you'd like more flexibility and/or to give the Git repository a different name, you should create the repository manually.

:::

## Deploy to Rill Cloud

With your project files on Github, you're ready to deploy the project. In the directory containing your project, run:
To deploy a project to Rill Cloud, from the directory containing your project, it's as simple as running:

```
rill deploy
```

The CLI will guide you through authenticating with Rill Cloud and granting read-only access to your Rill project on Github.
The CLI will guide you through authenticating with Rill Cloud and granting appropriate access to your Rill project on Github.

:::tip configure credentials
Cloud datastores will require service keys to access data. Make sure to create the necessary key and then run ```rill env configure``` with the correct credentials.
:::tip Configure credentials
Cloud datastores will typically require service keys to access data. Make sure to create the necessary key for your service account and then run ```rill env configure``` with the correct credentials. For more details, please refer to our [connector documentation](/build/credentials/credentials.md).

More details on credentials by source can be found in our [connectors section](/build/credentials/credentials.md).
:::

### First deployment

If this is your first deployment to Rill Cloud, you will get prompted to either sign up or log in (if you have an existing account on [Rill Cloud](https://ui.rilldata.com/)). Proceed with the sign up and email verification process for new users or authorization process for existing users. As a new user, you can expect to see the following page:

![Sign in to Rill Cloud](/img/deploy/existing-project/rill-cloud-sign-in.png)

After successfully signing in and/or authorizing the Rill CLI, you will get prompted to connect to Github when deploying your project.

![Connect to Github](/img/deploy/existing-project/connect-github.png)

:::warning Select the correct Github organization when installing the Rill Cloud app

Make sure that you are selecting the correct Github organization when installing / connecting the Rill Cloud app. The organization should correspond to where you want your repositories to belong to when deploying Rill projects.

:::

After connecting Rill Cloud to Github and selecting a [default organization](/reference/cli/org) in the CLI, you should now be able to continuously deploy new projects and/or update existing projects. These projects, [unless specified otherwise](/reference/cli/deploy), will be deployed to your selected [organization](/manage/user-management#managing-members-of-an-organization).

:::info Check with your Github organization admin

If you're not the admin of your Github organization, they will likely need to first install the Rill Cloud app in your organization before you can proceed with deploying a project. After the Rill Cloud app is installed, it should have the following privileges:

![Github app permissions](/img/deploy/existing-project/github-app-permissions.png)

:::

When deploying a project, Rill Cloud will first check whether there is a git remote present. If there is no git remote associated, you should get prompted whether you'd like Rill to create a Git repository on your behalf. If you enter **`Y`**, a new Git repository will be created and the project will be deployed.

:::warning Beware of existing repositories with the same name!

Rill Cloud will automatically attempt to create a Git repository using the <u>same name</u> as your Rill project for auto-deployments. If a Git repository with the same name already exists, you should get prompted and receive a warning in the CLI.

:::

## Checking deployment status

Once the deployment has completed, the browser will open on your project's status page. You can also check the project status from the command-line by running:
Once the deployment has completed, the browser will open on your project's status page. Alternatively, you can check the project status from the command-line (or CLI) by running the following command:
```
rill project status
```

:::info Resetting an Errored Project
Projects can sometimes be in an error state for a variety of a reasons. A hard reset can often clear these issues.

To execute a hard reset, use the following command from the CLI `rill project reconcile --reset`
To execute a hard reset of your project deployment, you can use the `rill project reset` command from the CLI.
:::

## Updating the deployment

Your project on Rill Cloud will automatically re-deploy every time you `git push` changes to Github.
Your project on Rill Cloud will automatically redeploy every time you `git push` changes to Github.

To refresh data sources without pushing code changes, run:
To manually refresh data sources without pushing code changes (or redeploying your project), run the following command:
```
rill project reconcile --refresh
rill project refresh
```

# Change your production branch
Expand All @@ -81,7 +132,7 @@ rill deploy --prod-branch [PROD-BRANCH]
```

## Deploying from a branch other than `main`
A branch from which continuous deployment is setup can be changed while editing the project. To change the branch, run:
A branch from which continuous deployment is setup can be changed while editing the project. To change the branch, run the following command:
```
rill project edit
```
Expand All @@ -92,5 +143,7 @@ If your Rill project is in a sub-directory of a Git repository, use the `--subpa
```
rill deploy --subpath path/to/rill/project
```
Note that you must run `rill deploy` from the root of the Git repository, not the root of the Rill project.
:::warning
Note that you must run `rill deploy` from the <u>root</u> of your Git repository, **not** the root of your Rill project.
:::

35 changes: 20 additions & 15 deletions docs/docs/deploy/existing-project/github-101.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,16 @@ To share dashboards with other users, Rill utilizes GitHub as a means of version

In this section, we will outline:

- Installing the Git desktop app (recommended for those less familiar with Git vs. using the command line)
- Installing the GitHub Desktop app (recommended for those less familiar with Git versus using the command line)
- Setting up your repository (repo) and cloning to your local
- Pushing changes to Git
- Deploying Rill to your new org to share dashboards, set up alerts and receive scheduled reports

![process](../../../static/img/deploy/github/process.png)
![process](/img/deploy/github-101/process.png)

## Installing Git

-To download the GitHub app, visit the [install site here](https://docs.github.com/en/desktop/installing-and-authenticating-to-github-desktop/installing-github-desktop).
-To download the GitHub app for a UI driven workflow, visit the [install site here](https://docs.github.com/en/desktop/installing-and-authenticating-to-github-desktop/installing-github-desktop).
- In your computer's Downloads folder, double-click the GitHub Desktop zip file.
- After the file has been unzipped, double-click the GitHub Desktop application file.
- GitHub Desktop will launch after installation is complete.
Expand All @@ -38,14 +38,14 @@ If you do not already have a GitHub account, you'll need to create one as part o
Once you have downloaded GitHub and connected your account, you'll be able to create a new repository. Think of the repository as a shared cloud drive for the configuration files (sources, models, dashboards) that power Rill.

- First, go to GitHub.com and create a new repository - clicking the ```New``` button on the homepage
![new](../../../static/img/deploy/github/new.png)
![new](/img/deploy/github-101/new.png)

- After you name your respository and click save, you'll see the option to ```Set up in Desktop```. This will open the GitHub desktop app directly to "clone" the repo to your local computer
![repo](../../../static/img/deploy/github/repo.png)
![repo](/img/deploy/github-101/repo.png)

- Select the folder on your computer where you'd like to build with Rill. Confirm the location you cloned for the next step.

![clone](../../../static/img/deploy/github/clone.png)
![clone](/img/deploy/github-101/clone.png)

- Open the Terminal and go to your local copy of the repo (e.g. the above would be ```cd Documents/GitHub/rill-poc/testrepo```)

Expand All @@ -55,26 +55,31 @@ Now - any changes you make locally can be synced directly with your GitHub repo

## Pushing Changes

After installing Rill and adding sources, your local project will now contain a variety of new files - source definitions for connectors, sql models for transformation and dashboards for analysis. GitHub automatically recognizes those files as new and you can now push those changes to your cloud repository.
After installing Rill and adding sources, your local project will now contain a variety of new files - source definitions, SQL models for transformation, and dashboards for analysis. GitHub automatically recognizes those files as new and you can now push those changes to your cloud repository.

- First, open the GitHub app and select your repo. You should now see a variety of new changes
First, open the GitHub app and select your repo. You should now see a variety of new changes (that can then be committed):

![commit](../../../static/img/deploy/github/commit.png)
![commit](/img/deploy/github-101/commit.png)

Now, in order to actually commit these changes to your repo, you can take the following steps:
- On the bottom left, you can describe the changes (or add something like "initial commit")
- Select the now blue "Commit to Main" button on the bottom left
- Click Publish Branch on the top right - and that's it. Any future changes would follow the same two steps - add a comment, commit and publish
- Select the now blue _Commit to main_ button on the bottom left
- Click _Publish Branch_ on the top right - and that's it. Any future changes would follow the same two steps - add a comment, commit and publish!

:::warning Avoid Syncing Data Files
During the commit process, uncheck files you do not wish to sync to your cloud repo. In particular, if you were working with local data files those will now be in the repo including ```tmp``` files and/or sources (e.g. parquet files). GitHub has a limit on cloud storage so instead add that data to a [cloud location (s3, gcs, azure)](../../reference/connectors/connectors.md) to access via the internet
During the commit process, make sure to **uncheck** files you do not wish to sync to your cloud repo (or add the appropriate exclusions in your `.gitignore` file). In particular, if you were working with local data files, those will now be in the repo including ```tmp``` files and/or sources (e.g. parquet or CSV files). GitHub has a limit on cloud storage so instead consider adding that data to a [cloud object storage location](/reference/connectors/connectors.md) (S3, GCS, Azure).
:::

:::tip Make small changes directly in github
For any larger changes, we would suggest developing locally to see the effects of that change. However, for smaller changes (add a simple metric, changing the name of a dimension, etc) some users prefer to edit directly inline on GitHub.com
For any larger changes, we would strongly suggest developing locally to see the effects of that change. However, for smaller changes (add a simple metric, changing the name of a dimension, etc.), some users _may_ prefer to edit directly inline on [Github](https://github.com) directly.
:::

## Deploying Rill

Now your dashboards are fully synced to GitHub, version controled and available to be edited by others. To make the dashboard fully cloud-enabled, you can return the Terminal and run ```rill deploy```. With everything synced, you should now be able to create a new Org within Rill and push your dashboard to the cloud. Any future changes would automatically be present in the cloud dashboards once committed and new dashboards would appear automatically as well (no deploy command needed). More details on [deployment here](../existing-project/existing-project.md).
Now your dashboards are fully synced to GitHub, version controled and available to be edited by others. To make the dashboard fully cloud-enabled, you can return to your terminal and run ```rill deploy```. With everything synced, you should now be able to create a [new organization](/manage/user-management#managing-members-of-an-organization) within Rill Cloud and push your dashboard to the cloud for shared collaboration. Any future changes should automatically be present in your deployed dashboards on Rill Cloud once committed and new dashboards will appear automatically as well (no deploy command needed). More details on [deployment here](../existing-project/existing-project.md).

We know that utilizing Git may be new to some - please reach out with questions on our [Discord channel](https://bit.ly/3unvA05) or [send us a note](mailto:[email protected])
:::info Have further questions? We'd love to hear from you!

We know that utilizing Git may be new to some - please don't hesitate to [reach out](contact.md) to us with any questions and we'd be happy to assist.

:::
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes

0 comments on commit 3a4f500

Please sign in to comment.