Merge branch 'master' into docs-document-gcp-backup

service-catalog/bastion/README.md

@@ -0,0 +1,15 @@
+# Bastion
+
+Bastion is an EC2 deployed in the `us-west-1` AWS region,
+used to access other servers inside our VPC.
+
+You can learn more about what a bastion is on
+[Wikipedia](https://en.wikipedia.org/wiki/Bastion_host)
+
+It is managed in the
+[terraform/bastion](https://github.com/rust-lang/simpleinfra/blob/master/terraform/bastion/README.md)
+module.
+
+## Versions
+
+We need to keep the Ubuntu version of the EC2 up-to-date.

service-catalog/bors/README.md

@@ -1,11 +1,14 @@
 # Bors
 
-The bors [module](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/bors)
-contains the infrastructure of [bors](https://github.com/rust-lang/bors), the Rust rewrite of
+[Bors](https://github.com/rust-lang/bors) is the Rust rewrite of
 [homu](https://github.com/rust-lang/homu).
 
-`Homu` is deployed in the legacy account, while `bors` is deployed in the
-`bors-staging` and `bors-prod` account, depending on the environment.
+- The bors infrastructure is managed in the
+  [bors](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/bors)
+  terragrunt module and it's deployed in the
+  `bors-staging` and `bors-prod` account, depending on the environment.
+- The `homu` infrastructure is managed in the
+  [terraform/bors](https://github.com/rust-lang/simpleinfra/tree/master/terraform/bors) module and it is deployed in the legacy account.
 
 Bors is deployed as a [Fargate](https://aws.amazon.com/fargate/) service
 ([ECS](https://aws.amazon.com/ecs/)) in the `us-east-2` region.
@@ -22,13 +25,15 @@ pipeline:
 
 These are the versions we need to keep up-to-date:
 
-- Operating system: Ubuntu. Version specified in the
-  [Dockerfile]
+- Ubuntu version specified in the [Dockerfile]
 - Rust toolchain: specified in the [Dockerfile]
 - Rust dependencies: specified in the
   [Cargo.toml](https://github.com/rust-lang/bors/blob/main/Cargo.toml)
 - PostgreSQL: version specified in
   [Terraform](https://github.com/rust-lang/simpleinfra/blob/master/terragrunt/modules/bors/main.tf)
+- GitHub Actions: specified in the
+  [workflows](https://github.com/rust-lang/bors/tree/main/.github/workflows)
+  directory
 
 No automation is in place as of August 2024.
 

service-catalog/crates-io/README.md

@@ -0,0 +1,31 @@
+# Crates.io
+
+Crates.io is the Rust community's package registry.
+The crates.io
+[team](https://www.rust-lang.org/governance/teams/dev-tools#team-crates-io)
+develops [crates-io](https://github.com/rust-lang/crates.io)
+and the Infra team helps with the infrastructure.
+
+The crates.io app is deployed in Heroku, you can learn more about it in the
+crates-io [docs](https://github.com/rust-lang/crates.io/blob/main/docs/ARCHITECTURE.md).
+
+Here are the details of the crates.io infrastructure managed by the infra team:
+
+- [crates-io-logs](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/crates-io-logs):
+  the infrastructure that counts crates downloads.
+  It is deployed in the AWS accounts `crates-io-prod` and `crates-io-staging`.
+- [crates-io](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/crates-io): Terraform module for the crates.io infrastructure.
+- [crates-io-heroku-metrics](https://github.com/rust-lang/simpleinfra/tree/master/terraform/crates-io-heroku-metrics):
+  Terraform module that deploys the Heroku metrics [collector](https://github.com/rust-lang/crates-io-heroku-metrics) used to gather crates.io's metrics.
+
+## Versions
+
+These are the versions we need to keep up-to-date:
+
+- Terraform providers in `simpleinfra` repo.
+- Rust dependencies in the
+  [Cargo.toml](https://github.com/rust-lang/simpleinfra/blob/master/terragrunt/modules/crates-io/compute-static/Cargo.toml)
+  of the fastly CDN (`Compute@Edge`).
+- crates-io-heroku-metrics [Dockerfile](https://github.com/rust-lang/crates-io-heroku-metrics/blob/main/Dockerfile):
+  - Ubuntu.
+  - Vector.

service-catalog/datadog/README.md

@@ -10,8 +10,20 @@ We use the following features:
 - Platform-level metrics from AWS and Fastly
 - Log pipelines for logs from our CDNs, applications, and servers
 
+## Team members
+
+User accounts on Datadog are managed in [terraform/team-members-datadog].
+
+## Integrations
+
+We use the following integrations:
+
+- [aws](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/datadog-aws)
+- [fastly](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/datadog-fastly)
+
 ## Explanations
 
 - [About Permissions](./about-permissions.md)
 
 [datadog]: https://www.datadoghq.com/
+[terraform/team-members-datadog]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/team-members-datadog

service-catalog/dev-desktops/how-to-update-system.md

@@ -29,3 +29,13 @@ are no longer needed:
 ```shell
 sudo apt autoremove
 ```
+
+## Update Ubuntu
+
+To update the Ubuntu version:
+
+1. Update the packages as described above.
+2. Run `sudo do-release-upgrade` to upgrade to the next LTS version.
+3. The system should reboot automatically after the upgrade.
+4. Apply the Ansible playbook again to ensure it still works
+5. Reboot the machine to ensure all services are running with the latest versions: `sudo reboot now`

service-catalog/discord-mods-bot/README.md

@@ -0,0 +1,29 @@
+# Discord moderation bot
+
+[Discord-mods-bot](https://github.com/rust-lang/discord-mods-bot)
+is a bot used by the moderation team to manage the Rust Discord server.
+
+The bot is managed in the
+[terraform/discord-mods-bot](https://github.com/rust-lang/simpleinfra/tree/master/terraform/discord-mods-bot)
+module and uses the `discord-mods-bot` database of
+[terraform/rds-databases](https://github.com/rust-lang/simpleinfra/tree/master/terraform/rds-databases)
+
+## Versions
+
+Here are the dependencies we need to keep up-to-date:
+
+`simpleinfra` repo:
+
+- terraform providers
+- `aws_ecs_service` platform version
+
+`discord-mods-bot` repo:
+
+- Ubuntu version specified in the
+  [Dockerfile](https://github.com/rust-lang/discord-mods-bot/blob/master/Dockerfile)
+- Rust dependencies: specified in the
+  [Cargo.toml](https://github.com/rust-lang/discord-mods-bot/blob/master/Cargo.toml)
+- Postgres version
+- GitHub Actions: specified in the
+  [workflows](https://github.com/rust-lang/discord-mods-bot/tree/master/.github/workflows)
+  directory

service-catalog/dns/README.md

@@ -0,0 +1,7 @@
+# DNS
+
+- The [terraform/dns] module configures DNS records of resources **not** managed by Terraform.
+- The [terraform/dns-delegation] module configures the `rust-lang.net` domain
+
+[terraform/dns]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/dns
+[terraform/dns-delegation]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/dns-delegation

service-catalog/fastly/README.md

@@ -3,9 +3,26 @@
 [Fastly] is a [Content Delivery Network] (CDN) that caches content and makes it
 available closer to the user.
 
+## Fastly Exporter
+
+[terraform/fastly-exporter] deploys the fastly-exporter, which makes the [Fastly Real-time analytics](https://www.fastly.com/documentation/reference/api/metrics-stats/realtime/) data available to Prometheus.
+
+## Team members
+
+User accounts on Fastly are managed in [terraform/team-members-fastly].
+
+### Versions
+
+These are the versions we need to keep up-to-date:
+
+- Docker version of `ghcr.io/fastly/fastly-exporter`
+- Terraform providers
+
 ## How-to Guides
 
 - [How to test cache invalidations](./how-to-test-cache-invalidations.md)
 
 [content delivery network]: https://en.wikipedia.org/wiki/Content_delivery_network
 [fastly]: https://www.fastly.com/
+[terraform/fastly-exporter]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/fastly-exporter
+[terraform/team-members-fastly]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/team-members-fastly

service-catalog/monitorbot/README.md

@@ -0,0 +1,19 @@
+# Monitorbot
+
+Bot that monitors various APIs and services that the infrastructure team uses.
+For example, it's used to check if GitHub tokens are getting close to their rate limits.
+
+[terraform/monitorbot] deploys the monitorbot as an ECS app.
+
+## Versions
+
+These are the versions we need to keep up-to-date:
+
+- Ubuntu version of the [Dockerfile]
+- Dependencies of [Cargo.toml]
+- GitHub Actions versions in [workflows]
+
+[Dockerfile]: https://github.com/rust-lang/monitorbot/blob/master/Dockerfile
+[Cargo.toml]: https://github.com/rust-lang/monitorbot/blob/master/Cargo.toml
+[workflows]: https://github.com/rust-lang/monitorbot/blob/master/.github/workflows/main.yml
+[terraform/monitorbot]: https://github.com/rust-lang/simpleinfra/blob/master/terraform/monitorbot/

service-catalog/playground/README.md

@@ -0,0 +1,20 @@
+# Rust Playground
+
+The [Rust Playground] is a service that allows users to run Rust code in a web
+browser. More info in the Playground [help page].
+
+[terraform/playground] deploys the Rust Playground infrastructure.
+
+The Playground is deployed in an AWS EC2. The [deployment]
+directory contains information and scripts on how to deploy it.
+
+## Versions
+
+- Ubuntu version of the EC2 instance.
+- [package.json](https://github.com/rust-lang/rust-playground/blob/main/ui/frontend/package.json).
+- [Cargo.lock](https://github.com/rust-lang/rust-playground/blob/main/ui/Cargo.lock).
+
+[Rust Playground]: https://play.rust-lang.org/
+[help page]: https://play.rust-lang.org/help
+[terraform/playground]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/playground
+[deployment]: https://github.com/rust-lang/rust-playground/tree/main/deployment

service-catalog/releases/README.md

@@ -0,0 +1,24 @@
+# Releases
+
+Here's the infrastructure we use to distribute the Rust releases:
+
+- [release-distribution](https://github.com/rust-lang/simpleinfra/blob/master/terragrunt/modules/release-distribution/):
+  Terraform module that uses Cloudfront and Fastly to manage the distribution of releases via `static-rust-lang-org`.
+- [releases](https://github.com/rust-lang/simpleinfra/blob/docs-document-release-distribution/terraform/releases/):
+  Terraform module managing the infrastructure that publishes Rust releases, including [promote-release](https://github.com/rust-lang/promote-release),
+  the tool used to publish new releases of the Rust toolchain.
+
+## Versions
+
+`simpleinfra` repo:
+
+- Terraform providers
+
+`promote-releases` repo:
+
+- [Cargo.toml](https://github.com/rust-lang/promote-release/blob/master/Cargo.toml) dependencies
+- GitHub Actions: specified in the
+  [workflows](https://github.com/rust-lang/promote-release/tree/master/.github/workflows)
+  directory
+- Ubuntu version specified in the
+  [Dockerfile](https://github.com/rust-lang/promote-release/blob/master/prod/Dockerfile)

service-catalog/rust-forge/README.md

@@ -0,0 +1,9 @@
+# Rust Forge
+
+The [Rust Forge] is a website that provides documentation for members
+of the Rust Project.
+
+It's a static website managed in [terraform/rust-forge].
+
+[Rust forge]: https://forge.rust-lang.org/
+[terraform/rust-forge]: https://github.com/rust-lang/simpleinfra/blob/master/terraform/rust-forge/

service-catalog/rust-log-analyzer/README.md

@@ -0,0 +1,20 @@
+# Rust Log Analyzer
+
+[rust-log-analyzer] is a tool that analyzes the CI build logs of the [rust] repository to automatically extract error messages from failed builds.
+
+This tool is deployed via the [terraform/rust-log-analyzer] module as an ECS service and uses S3 to store the GitHub actions index.
+
+## Versions
+
+These are the versions we need to keep up-to-date:
+
+- Ubuntu version in the [Dockerfile]
+- Dependencies in [Cargo.lock]
+- GitHub Actions versions in the [workflows]
+
+[Cargo.lock]: https://github.com/rust-lang/rust-log-analyzer/blob/master/Cargo.lock
+[Dockerfile]: https://github.com/rust-lang/rust-log-analyzer/blob/master/Dockerfile
+[rust-log-analyzer]: https://github.com/rust-lang/rust-log-analyzer
+[rust]: https://github.com/rust-lang/rust
+[terraform/rust-log-analyzer]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/rust-log-analyzer
+[workflows]: https://github.com/rust-lang/rust-log-analyzer/tree/master/.github/workflows

service-catalog/rustc-perf/README.md

@@ -0,0 +1,37 @@
+# Rustc perf
+
+[Rustc perf] is a website that tracks the performance of
+`rustc` over time.
+
+It is deployed using [terraform/rustc-perf].
+The app is in the [rustc-perf]
+repository and it's maintained by the [wg-compiler-performance] team.
+
+It uses an ECR repository to store the Docker image and an ECS service to run it.
+
+It also uses the `shared` db of [rds-databases] and the `rustc-perf` S3 bucket.
+
+The collector (the machine that runs the benchmarks) is a dedicated physical server running at [Hetzner].
+Rustc-perf runs on a bare-metal server because we need predictable performance.
+Otherwise, test results might be spoiled by noisy neighbors in the cloud.
+
+## Versions
+
+These are the versions we need to keep up-to-date:
+
+- Node and Ubuntu version specified in the [Dockerfile]
+- Dependencies of [Cargo.lock] and [package-lock.json]
+- Postgres version of the `shared` database
+- GitHub Actions versions in [workflows]
+- Operating system of the Hetzner server
+
+[Rustc perf]: https://perf.rust-lang.org/
+[rustc-perf]: https://github.com/rust-lang/rustc-perf
+[terraform/rustc-perf]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/rustc-perf
+[wg-compiler-performance]: https://github.com/orgs/rust-lang/teams/wg-compiler-performance
+[rds-databases]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/rds-databases
+[Hetzner]: https://www.hetzner.com/
+[Dockerfile]: https://github.com/rust-lang/rustc-perf/blob/master/Dockerfile
+[Cargo.lock]: https://github.com/rust-lang/rustc-perf/blob/master/Cargo.lock
+[package-lock.json]: https://github.com/rust-lang/rustc-perf/blob/master/site/frontend/package-lock.json
+[workflows]: https://github.com/rust-lang/rustc-perf/tree/master/.github/workflows

service-catalog/rustup/README.md

@@ -0,0 +1,18 @@
+# Rustup
+
+Rustup is the recommended tool for installing Rust.
+It is a command line tool that manages Rust versions and associated tools.
+
+The Infra team maintains the infrastructure for Rustup:
+
+- [rustup](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/rustup)
+  terragrunt module.
+  An S3 bucket called `rustup-build` for storing the Rustup binaries,
+  distributed through CloudFront CDN at `rustup-builds.rust-lang.org`.
+  The [rust-lang/rustup](https://github.com/rust-lang/rustup)
+  GitHub repository builds and uploads Rustup artifacts
+  to this S3 bucket.
+- [win-rustup-rs](https://github.com/rust-lang/simpleinfra/tree/master/terragrunt/modules/win-rustup-rs)
+  terragrunt module.
+  A Cloudfront distribution of the `/rustup/dist` path of the `static-rust-lang-org` bucket providing convenient short for downloading
+  rustup on Windows.

service-catalog/sync-team/README.md

@@ -0,0 +1,34 @@
+# Sync team
+
+[sync-team](https://github.com/rust-lang/sync-team) is a CLI tool used to
+synchronize the contents of the
+[rust-lang/team](https://github.com/rust-lang/team)
+repository with the services we use, such as GitHub, and Zulip.
+
+The infastructure lives in
+[terraform/team-repo](https://github.com/rust-lang/simpleinfra/tree/master/terraform/team-repo)
+and consists of:
+
+- ECR repository for storing the Docker image of the sync-team CLI.
+- A lambda function that runs sync-team through CodeBuild.
+
+## Versions
+
+`simpleinfra` repo:
+
+- Terraform providers
+- Node.js runtime for the lambda function
+
+`sync-team` repo:
+
+- Rust toolchain: specified in
+  [rust-toolchain.toml](https://github.com/rust-lang/sync-team/blob/master/rust-toolchain.toml)
+- Rust dependencies: specified in the
+  [Cargo.toml](https://github.com/rust-lang/sync-team/blob/master/Cargo.toml)
+- Ubuntu version specified in the
+  [Dockerfile](https://github.com/rust-lang/sync-team/blob/master/Dockerfile)
+- GitHub Actions: specified in the
+  [workflows](https://github.com/rust-lang/sync-team/tree/master/.github/workflows)
+  directory
+
+No automation is in place as of August 2024.

service-catalog/team-member-access/README.md

@@ -0,0 +1,11 @@
+# Team member access
+
+Access to cloud resources is managed in terraform.
+
+- [terraform/team-members-access]: Grants access to AWS resources.
+- [terraform/team-members-datadog]: Grants access to Datadog.
+- [terraform/team-members-fastly]: Grants access to Fastly.
+
+[terraform/team-members-access]: https://github.com/rust-lang/simpleinfra/tree/master/terraform/team-members-access
+[terraform/team-members-datadog]: ../datadog/README.md#team-members
+[terraform/team-members-fastly]: ../fastly/README.md#team-members