Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add config for gcp ga4 project #1484

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

add config for gcp ga4 project #1484

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fce5eba
add config for gcp ga4 project
Nyzl Oct 28, 2024
4db673b
add dataform repo permissions
Nyzl Oct 28, 2024
27ed3db
code clean up based on PR comments
Nyzl Nov 12, 2024
57d89c3
import project & fix references
Nyzl Nov 12, 2024
f6a3c03
move imports to new file
Nyzl Nov 13, 2024
341ef3b
add space to the end of imports.tf
Nyzl Nov 13, 2024
48edf94
update logging custom role
Nyzl Nov 20, 2024
a9050c8
clean up resource names
Nyzl Nov 25, 2024
6da0102
use project reference in service account name
Nyzl Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions terraform/deployments/ga4-analytics/imports.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import {
id = "ga4-analytics-352613"
to = google_project.project
}

import {
id = "GDS_BQ_read_access"
to = google_project_iam_custom_role.roles--gds_bigquery_read_access
}

import {
id = "GDS_BQ_saved_query_writer"
to = google_project_iam_custom_role.roles--gds_bigquery_saved_query_writer
}

import {
id = "GDS_log_alert_writer"
to = google_project_iam_custom_role.roles--gds_logging_alerts_writer
}

import {
id = "gds.bigquery.user"
to = google_project_iam_custom_role.roles--gds_bigquery_user
}

import {
id = "GDS_BQ_editor"
to = google_project_iam_custom_role.roles--gds_bigquery_editor
}
26 changes: 26 additions & 0 deletions terraform/deployments/ga4-analytics/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
terraform {
cloud {
organization = "govuk"
workspaces {
tags = ["ga4-analytics", "gcp", "production"]
}
}
required_version = "~> 1.5"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.26.0"
}
}
}

provider "google" {
project = "ga4-analytics-352613"
}

resource "google_project" "project" {
name = "GA4 Analytics"
project_id = "ga4-analytics-352613"
folder_id = "278098142879"
billing_account = "015C7A-FAF970-B0D375"
}
theseanything marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 3 additions & 0 deletions terraform/deployments/ga4-analytics/outputs.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
output "google_project_id" {
value = google_project.project.project_id
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this being used by anything?

75 changes: 75 additions & 0 deletions terraform/deployments/ga4-analytics/project_iam_binding.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
resource "google_project_iam_binding" "project-owners" {
project = google_project.project.project_id
role = "roles/owner"

members = [
"group:[email protected]",
"serviceAccount:terraform-cloud-production@govuk-production.iam.gserviceaccount.com"
]
}

resource "google_project_iam_binding" "project-editors" {
project = google_project.project.project_id
role = "roles/editor"
members = [
"group:[email protected]",
]
}

resource "google_project_iam_binding" "project-viewers" {
project = google_project.project.project_id
role = "roles/viewer"
members = [
# purposfully empty
]
}

resource "google_project_iam_binding" "project-gds_bigquery_editor" {
project = google_project.project.project_id
role = google_project_iam_custom_role.roles--gds_bigquery_editor.name
members = [
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
"serviceAccount:[email protected]",
google_service_account.sa--ga-database.member,
google_service_account.sa--ga4-user-admin.member,
]
depends_on = [
google_service_account.sa--ga-database,
google_service_account.sa--ga4-user-admin,
]
}

resource "google_project_iam_binding" "project-gds_bigquery_user" {
project = google_project.project.project_id
role = google_project_iam_custom_role.roles--gds_bigquery_user.name
members = [
"domain:digital.cabinet-office.gov.uk",
"user:[email protected]",
]
}

resource "google_project_iam_binding" "project-gds_bigquery_read_access" {
project = google_project.project.project_id
role = google_project_iam_custom_role.roles--gds_bigquery_read_access.name
members = [
"serviceAccount:analytics-events-pipeline@search-api-v2-integration.iam.gserviceaccount.com",
"serviceAccount:analytics-events-pipeline@search-api-v2-production.iam.gserviceaccount.com",
"serviceAccount:analytics-events-pipeline@search-api-v2-staging.iam.gserviceaccount.com",
"serviceAccount:chatbot-cloudrun@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:chatbot-cloudrun-dev@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:data-insights-experimentation@data-insights-experimentation.iam.gserviceaccount.com",
"serviceAccount:govuk-content-data-ga4@govuk-content-data.iam.gserviceaccount.com",
"serviceAccount:[email protected]",
]
}

resource "google_project_iam_binding" "project-gds_log_alert_writer" {
project = google_project.project.project_id
role = google_project_iam_custom_role.roles--gds_logging_alerts_writer.name
members = [
"group:[email protected]"
]
}
144 changes: 144 additions & 0 deletions terraform/deployments/ga4-analytics/project_iam_custom_roles.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
resource "google_project_iam_custom_role" "roles--gds_bigquery_read_access" {
description = "Created on: 2023-10-27"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a description which describes the purpose of the role?

permissions = [
"bigquery.datasets.get",
"bigquery.tables.get",
"bigquery.tables.getData"
]
role_id = "GDS_BQ_read_access"
title = "GDS BQ read access"
}

resource "google_project_iam_custom_role" "roles--gds_bigquery_saved_query_writer" {
description = "Created on: 2023-11-23"
permissions = [
"bigquery.savedqueries.create",
"bigquery.savedqueries.delete",
"bigquery.savedqueries.update",
"dataform.repositories.create",
"dataform.repositories.get",
"dataform.repositories.list"
]
role_id = "GDS_BQ_saved_query_writer"
title = "GDS BQ saved query writer"
}

resource "google_project_iam_custom_role" "roles--gds_logging_alerts_writer" {
description = "Logger access to create alerts"
permissions = [
"logging.logEntries.create",
"logging.logEntries.list",
"logging.logMetrics.create",
"logging.logMetrics.delete",
"logging.logMetrics.get",
"logging.logMetrics.list",
"logging.logMetrics.update",
"logging.notificationRules.create",
"logging.notificationRules.get",
"logging.notificationRules.list",
"logging.notificationRules.update",
"monitoring.alertPolicies.create",
"monitoring.alertPolicies.delete",
"monitoring.alertPolicies.get",
"monitoring.alertPolicies.list",
"monitoring.alertPolicies.update"
]
role_id = "GDS_log_alert_writer"
title = "GDS log alert writer"
}

resource "google_project_iam_custom_role" "roles--gds_bigquery_user" {
description = "Adds transfers update to the standard BigQuery User role"
permissions = [
"bigquery.bireservations.get",
"bigquery.capacityCommitments.get",
"bigquery.capacityCommitments.list",
"bigquery.datasets.get",
"bigquery.datasets.getIamPolicy",
"bigquery.jobs.create",
"bigquery.jobs.get",
"bigquery.jobs.list",
"bigquery.jobs.listAll",
"bigquery.jobs.listExecutionMetadata",
"bigquery.models.list",
"bigquery.readsessions.create",
"bigquery.readsessions.getData",
"bigquery.reservationAssignments.list",
"bigquery.reservationAssignments.search",
"bigquery.reservations.get",
"bigquery.reservations.list",
"bigquery.savedqueries.get",
"bigquery.savedqueries.list",
"bigquery.tables.get",
"bigquery.tables.getData",
"bigquery.tables.list",
"bigquery.transfers.get",
"bigquery.transfers.update",
"bigquerymigration.translation.translate",
"resourcemanager.projects.get",
]
role_id = "gds.bigquery.user"
title = "GDS BQ user"
}

resource "google_project_iam_custom_role" "roles--gds_bigquery_editor" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
resource "google_project_iam_custom_role" "roles--gds_bigquery_editor" {
resource "google_project_iam_custom_role" "gds_bigquery_editor" {

Nit: don't think the prefix adds anything to the name

description = "Edit access to BQ"
permissions = [
"bigquery.capacityCommitments.get",
"bigquery.capacityCommitments.list",
"bigquery.config.get",
"bigquery.datasets.create",
"bigquery.datasets.get",
"bigquery.datasets.getIamPolicy",
"bigquery.datasets.update",
"bigquery.datasets.updateTag",
"bigquery.jobs.create",
"bigquery.jobs.get",
"bigquery.models.create",
"bigquery.models.delete",
"bigquery.models.export",
"bigquery.models.getData",
"bigquery.models.getMetadata",
"bigquery.models.list",
"bigquery.models.updateData",
"bigquery.models.updateMetadata",
"bigquery.models.updateTag",
"bigquery.readsessions.create",
"bigquery.readsessions.getData",
"bigquery.routines.create",
"bigquery.routines.delete",
"bigquery.routines.get",
"bigquery.routines.list",
"bigquery.routines.update",
"bigquery.routines.updateTag",
"bigquery.savedqueries.create",
"bigquery.savedqueries.delete",
"bigquery.savedqueries.get",
"bigquery.savedqueries.list",
"bigquery.savedqueries.update",
"bigquery.tables.create",
"bigquery.tables.createIndex",
"bigquery.tables.createSnapshot",
"bigquery.tables.delete",
"bigquery.tables.deleteIndex",
"bigquery.tables.export",
"bigquery.tables.get",
"bigquery.tables.getData",
"bigquery.tables.getIamPolicy",
"bigquery.tables.list",
"bigquery.tables.replicateData",
"bigquery.tables.restoreSnapshot",
"bigquery.tables.update",
"bigquery.tables.updateData",
"bigquery.tables.updateTag",
"bigquery.transfers.get",
"bigquery.transfers.update",
"bigquerymigration.translation.translate",
"dataform.locations.get",
"dataform.locations.list",
"dataform.repositories.create",
"resourcemanager.projects.get"
]
role_id = "GDS_BQ_editor"
title = "GDS BQ editor"
}
25 changes: 25 additions & 0 deletions terraform/deployments/ga4-analytics/service_account.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
resource "google_service_account" "sa--ga4-analytics-352613" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
resource "google_service_account" "sa--ga4-analytics-352613" {
resource "google_service_account" "ga4-analytics" {

Avoiding naming with arbitrary descriptors (we don't need to know the project id in the tf resource name)

account_id = "ga4-analytics-352613"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is meant to be the same as the project id - could we use a reference to the project id?

display_name = "App Engine default service account"
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--ga-database" {
account_id = "ga-database"
description = "Account for analytics_settings_database"
display_name = "ga-database"
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--ga4-user-admin" {
account_id = "ga4-user-admin"
display_name = "GA4 User Admin"
create_ignore_already_exists = "true"
}

resource "google_service_account" "sa--search-analytics-pipeline" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
resource "google_service_account" "sa--search-analytics-pipeline" {
resource "google_service_account" "search_analytics_pipeline" {

Don't think the "sa--" prefix adds anything to the resources names. Also use "_" rather than "-"s

account_id = "search-analytics-pipeline"
description = "Service account used by the Search Analytics Pipeline to access the GA4 API"
display_name = "search-analytics-pipeline"
create_ignore_already_exists = "true"
}
Loading