GCP GitLab Runner

This Terraform modules creates a GitLab CI runner.

The runners created by the module use preemptible instances by default for running the builds using the docker+machine executor.

• Shared cache in GCS with life cycle management to clear objects after x days.
• Runner agents registered automatically.

The runner supports:

GitLab CI docker-machine runner

In this scenario the runner agent is running on a GCP Compute Instance and runners are created by docker machine using preemptible instances. Runners will scale automatically based on the configuration. The module creates a GCS cache by default, which is shared across runners (preemptible instances).

GitLab CI docker runner

In this scenario not docker machine is used but docker to schedule the builds. Builds will run on the same compute instance as the agent.

GitLab CI Kubernetes Runner

See gke-runner

Autoscaling the Runners

Both docker-machine runner and docker runners autoscale using GCP Custom metrics. The runner publishes running jobs metrics to stackdriver which is then used to scale up/down the number of active runners. var.runners_min_replicas and var.runners_max_replicas defined variables for the minimum and maximum number of runners respectively. It uses Google Managed Instance Group Autoscaler to scale when the average of running jobs exceeds var.runners_concurrent.

⚠️ With scaling up and down, runners can get terminated without waiting for clean up when GCP scales down. GCP shutdown scripts have a maximum of atmost 90 seconds, during which it's forcefully terminated. Hence, it might not wait for your jobs to finish during scale down. This can leave orphan docker-machine-created instances and failed jobs. Use this with caution. Advisably, if you need more than one runner, set var.runner_min_replicas = var.runners_max_replicas = number of runners you need.

GitLab runner token configuration

The runner is registered on initial deployment. Each new runner registers itself with the same description and tag. To register the runner automatically set the variable gitlab_runner_registration_config["registration_token"]. This token value can be found in your GitLab project, group, or global settings. For a generic runner you can find the token in the admin section. By default the runner will be locked to the target project, not run untagged. Below is an example of the configuration map.

gitlab_runner_registration_config = {
  registration_token = "Required: <registration token>"
  tag_list           = "Optional: <your tags, comma separated>"
  description        = "Optional: <some description>"
  locked_to_project  = "optional: true"
  run_untagged       = "Optional: false"
  maximum_timeout    = "Optional: 3600"
  access_level       = "Optional: <not_protected OR ref_protected, ref_protected runner will only run on pipelines triggered on protected branches. Defaults to not_protected>"
}

GitLab runner cache

By default the module creates a a cache for the runner in Google Cloud Storage. Old objects are automatically removed via a configurable life cycle policy on the bucket.

Usage

module "runner" {
  source  = "DeimosCloud/gitlab-ci-runner/google"

  network    = "default"
  subnetwork = "someNetwork"
  region     = "europe-west1"
  project    = local.project_id

  runners_name       = "docker-default"
  runners_gitlab_url = "https://gitlab.com"

  gitlab_runner_registration_config = {
    registration_token = "my-token"
    tag_list           = "docker"
  }

}

NOTE: If runners are set to use internal IPs, a Cloud NAT must be deployed for runners to be able to reach internet

Contributing

Report issues/questions/feature requests on in the issues section.

Full contributing guidelines are covered here.

Requirements

Name	Version
terraform	>= 0.14

Providers

Name	Version
google	4.19.0
random	3.1.3

Modules

Name	Source	Version
cache_gcs	./modules/cache	n/a

Resources

Name	Type
google_compute_firewall.docker_machine	resource
google_compute_firewall.docker_machine_ssh	resource
google_compute_firewall.docker_machines	resource
google_compute_firewall.internet	resource
google_compute_firewall.ssh	resource
google_compute_instance_template.this	resource
google_compute_region_autoscaler.this	resource
google_compute_region_instance_group_manager.this	resource
google_monitoring_metric_descriptor.jobs	resource
google_project_iam_member.this	resource
google_service_account.agent	resource
google_service_account.runner	resource
google_service_account_iam_member.agent_runner	resource
google_service_account_key.agent	resource
random_id.this	resource
random_shuffle.zones	resource
google_compute_network.this	data source
google_compute_zones.available	data source

Inputs

Name	Description	Type	Default	Required
cache_bucket_versioning	Boolean used to enable versioning on the cache bucket, false by default.	bool	false	no
cache_expiration_days	Number of days before cache objects expires.	number	2	no
cache_location	The location where to create the cache bucket in. If not specified, it defaults to the region	any	null	no
cache_shared	Enables cache sharing between runners.	bool	true	no
cache_storage_class	The cache storage class	string	"STANDARD"	no
create_cache_bucket	Creates a cache cloud storage bucket if true	bool	true	no
docker_machine_disk_size	The disk size for the docker-machine instances.	number	20	no
docker_machine_disk_type	The disk Type for docker-machine instances.	string	"pd-standard"	no
docker_machine_download_url	Full url pointing to a linux x64 distribution of docker machine.	string	"https://gitlab-docker-machine-downloads.s3.amazonaws.com/main/docker-machine-Linux-x86_64"	no
docker_machine_image	A GCP custom image to use for spinning up docker-machines	string	""	no
docker_machine_machine_type	The Machine Type for the docker-machine instances.	string	"f1-micro"	no
docker_machine_options	List of additional options for the docker machine config. Each element of this list must be a key=value pair. E.g. '["google-zone=a"]'	list(string)	[]	no
docker_machine_preemptible	If true, docker-machine instances will be premptible	bool	false	no
docker_machine_tags	Additional Network tags to be attached to the docker-machine instances.	list(string)	[]	no
docker_machine_use_internal_ip	If true, docker-machine instances will have only internal IPs.	bool	false	no
gitlab_runner_registration_config	Configuration used to register the runner. Available at https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner.	map	{ "access_level": "not_protected", "description": "", "locked_to_project": "", "maximum_timeout": "", "registration_token": "", "run_untagged": "", "tag_list": "" }	no
gitlab_runner_version	Version of the GitLab runner. Defaults to latest	string	""	no
labels	Map of labels that will be added to created resources	map(string)	{}	no
network	The target VPC for the docker-machine and runner instances.	string	"default"	no
prefix	The prefix to apply to all GCP resource names (e.g. -runner, -agent-1).	string	"ci"	no
project	The GCP project to deploy the runner into.	string	n/a	yes
region	The GCP region to deploy the runner into.	string	n/a	yes
runner_additional_service_account_roles	Additional roles to pass to the Runner service account	list(string)	[]	no
runners_additional_volumes	Additional volumes that will be used in the runner config.toml, e.g Docker socket	list(any)	[]	no
runners_allow_ssh_access	Enables SSH Access to the runner instances.	bool	true	no
runners_concurrent	Concurrent value for the runners, will be used in the runner config.toml. Limits how many jobs globally can be run concurrently when running docker-machine.	number	10	no
runners_disable_cache	Runners will not use local cache, will be used in the runner config.toml	bool	false	no
runners_disk_size	The size of the created gitlab runner instances in GB.	number	20	no
runners_disk_type	The Disk type of the gitlab runner instances	string	"pd-standard"	no
runners_docker_runtime	docker runtime for runners, will be used in the runner config.toml	string	""	no
runners_enable_monitoring	Installs Stackdriver monitoring Agent on runner Instances to collect metrics.	bool	true	no
runners_environment_vars	Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml	list(string)	[]	no
runners_executor	The executor to use. Currently supports docker+machine or docker.	string	"docker+machine"	no
runners_gitlab_url	URL of the GitLab instance to connect to.	string	"https://gitlab.com"	no
runners_helper_image	Overrides the default helper image used to clone repos and upload artifacts, will be used in the runner config.toml	string	""	no
runners_idle_count	(docker-machine) Idle count of the runners, will be used in the runner config.toml.	number	0	no
runners_idle_time	(docker-machine) Idle time of the runners, will be used in the runner config.toml.	number	600	no
runners_image	Image to run builds, will be used in the runner config.toml	string	"docker:19.03"	no
runners_install_docker_credential_gcr	Install docker_credential_gcr inside startup_script_pre_install script	bool	true	no
runners_limit	Limit for the runners, will be used in the runner config.toml.	number	0	no
runners_machine_autoscaling	(docker-machine) Set autoscaling parameters based on periods, see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersmachine-section	list(object({ periods = list(string) idle_count = number idle_time = number timezone = string }))	[]	no
runners_machine_type	Instance type used for the GitLab runner.	string	"n1-standard-1"	no
runners_max_builds	(docker-machine) Max builds for each runner after which it will be removed, will be used in the runner config.toml. By default set to 0, no maxBuilds will be set in the configuration.	number	0	no
runners_max_growth_rate	(docker-machine) The maximum number of machines that can be added to the runner in parallel. Default is 0 (no limit).	number	0	no
runners_max_replicas	The maximum number of runners to spin up.For docker+machine, this is the max number of instances that will run docker-machine. For docker, this is the maximum number of runner instances.	number	1	no
runners_metadata	(Optional) Metadata key/value pairs to make available from within instances created from this template.	map	{}	no
runners_min_replicas	The minimum number of runners to spin up. For docker+machine, this is the min number of instances that will run docker-machine. For docker, this is the minimum number of runner instances	number	1	no
runners_name	Name of the runner, will be used in the runner config.toml.	string	n/a	yes
runners_output_limit	Sets the maximum build log size in kilobytes, by default set to 4096 (4MB)	number	4096	no
runners_post_build_script	Commands to be executed on the Runner just after executing the build, but before executing after_script.	string	"\"\""	no
runners_pre_build_script	Script to execute in the pipeline just before the build, will be used in the runner config.toml	string	"\"\""	no
runners_pre_clone_script	Commands to be executed on the Runner before cloning the Git repository. this can be used to adjust the Git client configuration first, for example.	string	"\"\""	no
runners_preemptible	If true, runner compute instances will be premptible	bool	true	no
runners_privileged	Runners will run in privileged mode, will be used in the runner config.toml	bool	true	no
runners_pull_policy	pull_policy for the runners, will be used in the runner config.toml	string	"always"	no
runners_request_concurrency	Limit number of concurrent requests for new jobs from GitLab (default 1)	number	1	no
runners_root_size	Runner instance root size in GB.	number	16	no
runners_services_volumes_tmpfs	n/a	list(object({ volume = string options = string }))	[]	no
runners_shm_size	shm_size for the runners, will be used in the runner config.toml	number	0	no
runners_ssh_allowed_cidr_blocks	List of CIDR blocks to allow SSH Access to the gitlab runner instance.	list(string)	[ "0.0.0.0/0" ]	no
runners_tags	Additional Network tags to be attached to the Gitlab Runner.	list(string)	[]	no
runners_target_autoscale_cpu_utilization	The target CPU utilization that the autoscaler should maintain. If runner CPU utilization gets above this, a new runner is created until runners_max_replicas is reached	number	0.9	no
runners_use_internal_ip	Restrict runners to the use of a Internal IP address. NOTE: NAT Gateway must be deployed in your network so that Runners can access resources on the internet	bool	false	no
runners_volumes_tmpfs	n/a	list(object({ volume = string options = string }))	[]	no
startup_script_post_install	Startup script snippet to insert after GitLab runner install	string	""	no
startup_script_pre_install	Startup script snippet to insert before GitLab runner install	string	""	no
subnetwork	Subnetwork used for hosting the gitlab-runners.	string	""	no

Outputs

No outputs.