Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add config for gcp ga4 project #1484

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

add config for gcp ga4 project #1484

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fce5eba
add config for gcp ga4 project
Nyzl Oct 28, 2024
4db673b
add dataform repo permissions
Nyzl Oct 28, 2024
27ed3db
code clean up based on PR comments
Nyzl Nov 12, 2024
57d89c3
import project & fix references
Nyzl Nov 12, 2024
f6a3c03
move imports to new file
Nyzl Nov 13, 2024
341ef3b
add space to the end of imports.tf
Nyzl Nov 13, 2024
48edf94
update logging custom role
Nyzl Nov 20, 2024
a9050c8
clean up resource names
Nyzl Nov 25, 2024
6da0102
use project reference in service account name
Nyzl Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions terraform/deployments/ga4-analytics/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
terraform {
cloud {
organization = "govuk"
workspaces {
tags = ["gcp-ga4-analytics", "gcp", "production"]
}
}
required_version = "~> 1.5"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.26.0"
}
}
}

provider "google" {
project = "ga4-analytics-352613"
}

resource "google_project" "project" {
name = "GA4 Analytics"
project_id = "ga4-analytics-352613"
folder_id = "278098142879"
billing_account = "015C7A-FAF970-B0D375"
}
theseanything marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 3 additions & 0 deletions terraform/deployments/ga4-analytics/outputs.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
output "google_project_id" {
value = google_project.project.project_id
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this being used by anything?

75 changes: 75 additions & 0 deletions terraform/deployments/ga4-analytics/project_iam_binding.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
resource "google_project_iam_binding" "project-owners" {
project = google_project.project.project_id
role = "roles/owner"

members = [
"group:[email protected]",
"serviceAccount:terraform-cloud-production@govuk-production.iam.gserviceaccount.com"
]
}

resource "google_project_iam_binding" "project-editors" {
project = google_project.project.project_id
role = "roles/editor"
members = [
"group:[email protected]",
]
}

resource "google_project_iam_binding" "project-viewers" {
project = google_project.project.project_id
role = "roles/viewer"
members = [
# purposfully empty
]
}

resource "google_project_iam_binding" "project-gds_bigquery_editor" {
project = google_project.project.project_id
role = "projects/ga4-analytics-352613/roles/GDS_BQ_editor"
theseanything marked this conversation as resolved.
Show resolved Hide resolved
members = [
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
theseanything marked this conversation as resolved.
Show resolved Hide resolved
]
depends_on = [
google_service_account.sa--ga-database,
google_service_account.sa--ga4-user-admin,
]
}

resource "google_project_iam_binding" "project-gds_bigquery_user" {
project = google_project.project.project_id
role = "projects/ga4-analytics-352613/roles/gds.bigquery.user"
members = [
"domain:digital.cabinet-office.gov.uk",
"user:[email protected]",
]
}

resource "google_project_iam_binding" "project-GDS_BQ_read_access" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
resource "google_project_iam_binding" "project-GDS_BQ_read_access" {
resource "google_project_iam_binding" "project-gds-bq-read-access" {

Can we at least normalise the names for the terraform resources?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, much cleaner and easier to read

Copy link
Contributor

@theseanything theseanything Nov 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: normalise dashes "-" to underscores "_" as per terraform conventions. https://developer.hashicorp.com/terraform/language/style#resource-naming

project = google_project.project.project_id
role = "projects/ga4-analytics-352613/roles/GDS_BQ_read_access"
members = [
"serviceAccount:analytics-events-pipeline@search-api-v2-integration.iam.gserviceaccount.com",
"serviceAccount:analytics-events-pipeline@search-api-v2-production.iam.gserviceaccount.com",
"serviceAccount:analytics-events-pipeline@search-api-v2-staging.iam.gserviceaccount.com",
"serviceAccount:chatbot-cloudrun@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:chatbot-cloudrun-dev@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:data-insights-experimentation@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:govuk-content-data-ga4@govuk-content-data.iam.gserviceaccount.com",
"serviceAccount:[email protected]",
]
}

resource "google_project_iam_binding" "project-GDS_log_alert_writer" {
project = google_project.project.project_id
role = "projects/ga4-analytics-352613/roles/GDS_log_alert_writer"
members = [
"group:[email protected]"
]
}
150 changes: 150 additions & 0 deletions terraform/deployments/ga4-analytics/project_iam_custom_roles.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
resource "google_project_iam_custom_role" "roles--GDS_BQ_read_access" {
description = "Created on: 2023-10-27"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a description which describes the purpose of the role?

permissions = [
"bigquery.datasets.get",
"bigquery.tables.get",
"bigquery.tables.getData"
]
project = google_project.project.project_id
role_id = "GDS_BQ_read_access"
stage = "GA"
title = "GDS BQ read access"
}

resource "google_project_iam_custom_role" "roles--GDS_BQ_saved_query_writer" {
description = "Created on: 2023-11-23"
permissions = [
"bigquery.savedqueries.create",
"bigquery.savedqueries.delete",
"bigquery.savedqueries.update",
"dataform.repositories.create",
"dataform.repositories.get",
"dataform.repositories.list"
]
project = google_project.project.project_id
role_id = "GDS_BQ_saved_query_writer"
stage = "GA"
title = "GDS BQ saved query writer"
}

resource "google_project_iam_custom_role" "roles--GDS_logging_alerts_writer" {
description = "Logger access to create alerts"
permissions = [
"logging.logEntries.create",
"logging.logEntries.list",
"logging.logMetrics.create",
"logging.logMetrics.delete",
"logging.logMetrics.get",
"logging.logMetrics.list",
"logging.logMetrics.update",
"monitoring.alertPolicies.create",
"monitoring.alertPolicies.delete",
"monitoring.alertPolicies.get",
"monitoring.alertPolicies.list",
"monitoring.alertPolicies.update"
]
project = google_project.project.project_id
role_id = "GDS_log_alert_writer"
stage = "GA"
title = "GDS log alert writer"
}

resource "google_project_iam_custom_role" "roles--GDS_BQ_user" {
description = "Adds transfers update to the standard BigQuery User role"
permissions = [
"bigquery.bireservations.get",
"bigquery.capacityCommitments.get",
"bigquery.capacityCommitments.list",
"bigquery.datasets.get",
"bigquery.datasets.getIamPolicy",
"bigquery.jobs.create",
"bigquery.jobs.get",
"bigquery.jobs.list",
"bigquery.jobs.listAll",
"bigquery.jobs.listExecutionMetadata",
"bigquery.models.list",
"bigquery.readsessions.create",
"bigquery.readsessions.getData",
"bigquery.reservationAssignments.list",
"bigquery.reservationAssignments.search",
"bigquery.reservations.get",
"bigquery.reservations.list",
"bigquery.savedqueries.get",
"bigquery.savedqueries.list",
"bigquery.tables.get",
"bigquery.tables.getData",
"bigquery.tables.list",
"bigquery.transfers.get",
"bigquery.transfers.update",
"bigquerymigration.translation.translate",
"resourcemanager.projects.get",
]
project = google_project.project.project_id
role_id = "gds.bigquery.user"
stage = "GA"
title = "GDS BQ user"
}

resource "google_project_iam_custom_role" "roles--GDS_BQ_editor" {
description = "Edit access to BQ"
permissions = [
"bigquery.capacityCommitments.get",
"bigquery.capacityCommitments.list",
"bigquery.config.get",
"bigquery.datasets.create",
"bigquery.datasets.get",
"bigquery.datasets.getIamPolicy",
"bigquery.datasets.update",
"bigquery.datasets.updateTag",
"bigquery.jobs.create",
"bigquery.jobs.get",
"bigquery.models.create",
"bigquery.models.delete",
"bigquery.models.export",
"bigquery.models.getData",
"bigquery.models.getMetadata",
"bigquery.models.list",
"bigquery.models.updateData",
"bigquery.models.updateMetadata",
"bigquery.models.updateTag",
"bigquery.readsessions.create",
"bigquery.readsessions.getData",
"bigquery.routines.create",
"bigquery.routines.delete",
"bigquery.routines.get",
"bigquery.routines.list",
"bigquery.routines.update",
"bigquery.routines.updateTag",
"bigquery.savedqueries.create",
"bigquery.savedqueries.delete",
"bigquery.savedqueries.get",
"bigquery.savedqueries.list",
"bigquery.savedqueries.update",
"bigquery.tables.create",
"bigquery.tables.createIndex",
"bigquery.tables.createSnapshot",
"bigquery.tables.delete",
"bigquery.tables.deleteIndex",
"bigquery.tables.export",
"bigquery.tables.get",
"bigquery.tables.getData",
"bigquery.tables.getIamPolicy",
"bigquery.tables.list",
"bigquery.tables.replicateData",
"bigquery.tables.restoreSnapshot",
"bigquery.tables.update",
"bigquery.tables.updateData",
"bigquery.tables.updateTag",
"bigquery.transfers.get",
"bigquery.transfers.update",
"bigquerymigration.translation.translate",
"dataform.locations.get",
"dataform.locations.list",
"dataform.repositories.create",
"resourcemanager.projects.get"
]
project = google_project.project.project_id
theseanything marked this conversation as resolved.
Show resolved Hide resolved
role_id = "GDS_BQ_editor"
stage = "GA"
theseanything marked this conversation as resolved.
Show resolved Hide resolved
title = "GDS BQ editor"
}
33 changes: 33 additions & 0 deletions terraform/deployments/ga4-analytics/service_account.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
resource "google_service_account" "sa--ga4-analytics-352613" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
resource "google_service_account" "sa--ga4-analytics-352613" {
resource "google_service_account" "ga4-analytics" {

Avoiding naming with arbitrary descriptors (we don't need to know the project id in the tf resource name)

account_id = "ga4-analytics-352613"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is meant to be the same as the project id - could we use a reference to the project id?

disabled = "false"
theseanything marked this conversation as resolved.
Show resolved Hide resolved
display_name = "App Engine default service account"
project = google_project.project.project_id
theseanything marked this conversation as resolved.
Show resolved Hide resolved
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--ga-database" {
account_id = "ga-database"
description = "Account for analytics_settings_database"
disabled = "false"
display_name = "ga-database"
project = google_project.project.project_id
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--ga4-user-admin" {
account_id = "ga4-user-admin"
disabled = "false"
display_name = "GA4 User Admin"
project = google_project.project.project_id
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--search-analytics-pipeline" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
resource "google_service_account" "sa--search-analytics-pipeline" {
resource "google_service_account" "search_analytics_pipeline" {

Don't think the "sa--" prefix adds anything to the resources names. Also use "_" rather than "-"s

account_id = "search-analytics-pipeline"
description = "Service account used by the Search Analytics Pipeline to access the GA4 API"
disabled = "false"
display_name = "search-analytics-pipeline"
project = google_project.project.project_id
create_ignore_already_exists = "true"
}
Loading