From 05bbe5a890baeff4f70d2b55ffb85780882a4d0c Mon Sep 17 00:00:00 2001 From: dkrees Date: Wed, 3 Apr 2024 17:34:57 +0100 Subject: [PATCH 01/16] Making item cats and labels standardised. The html components now rely on the yml data to populate the details such as the text, style and links. Some new parameters allow these can be overridden for use in overlay views. --- _data/categoryItems.yml | 153 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 _data/categoryItems.yml diff --git a/_data/categoryItems.yml b/_data/categoryItems.yml new file mode 100644 index 0000000..eb89087 --- /dev/null +++ b/_data/categoryItems.yml @@ -0,0 +1,153 @@ +# These are the default values for the technology category items used in the _includes/categoryItem.html template. +# Only the 'id', 'selected', 'description' and 'link' parameters can be overridden when using categoryItem template. +# All other vales are the default values, used EVERYWHERE, so please be aware when making any changes here. + +# The Category Items are mapped and accessible using thier item name. +# - id: is the id attribute applied to the categoryItem. This is useful for anchor links. Useful for linking from a infographic item to an in page category item (optional, but defaults to the name of the item). +# - category: is actually a css class that is passed through to set the colour of the item (background to the icon). There maybe other future oppourtunities to use this. +# - title: used as the main title/descriptor of the item (required and default). +# - icon: used as the icon of the item (required and default). Maps to the assets/images/icons.svg. +# - selected: when true, this highlights the item. Used for the contextual overlay views (false by default). +# - description: adds additional text under the title providing som context. Used for the contextual overlay views (optional). +# - link: Creates a link to another page (further details). These are setup by default to the relevant pages, but may be overridden for specific use cases, typically in the overlay views (optional, but defaults to a page). +# - noLink: Provides an overide to prevent any navigation to a page. Renders the component in a
instead of an . + +CatUSoftware: + id: CatUSoftware + category: cat-u + title: Off The Shelf including Open-Source Software + icon: web + selected: false + description: + link: /categories/upstream#software + noLink: false + +CatUEmployeeHardware: + id: CatUEmployeeHardware + category: cat-u + title: User Hardware (laptops, printers, etc.) + icon: devices + selected: false + description: + link: /categories/upstream#employee-hardware + noLink: false + +CatUNetworkHardware: + id: CatUNetworkHardware + category: cat-u + title: Networking and Infrastructure Hardware + icon: router + selected: false + description: + link: /categories/upstream#networking-hardware + noLink: false + +CatUServerHardware: + id: CatUServerHardware + category: cat-u + title: Servers and Storage Hardware + icon: storage + selected: false + description: + link: /categories/upstream#data-center-and-server-hardware + noLink: false + +CatOServers: + id: CatOServers + category: cat-o + title: Servers and Storage + icon: storage + selected: false + description: + link: /categories/operational#servers-and-storage + noLink: false + +CatONetworkDevices: + id: CatONetworkDevices + category: cat-o + title: Networking and Infrastructure + icon: router + selected: false + description: + link: /categories/operational#networking-devices + noLink: false + +CatOUserDevices: + id: CatOUserDevices + category: cat-o + title: User Devices + icon: devices + selected: false + description: + link: /categories/operational#employee-devices + noLink: false + +CatGGenerators: + id: CatGGenerators + category: cat-g + title: Generators + icon: bolt + selected: false + description: + link: /categories/operational#generators + noLink: false + +CatCCloud: + id: CatCCloud + category: cat-c + title: Cloud Services + icon: cloud + selected: false + description: + link: /categories/operational#cloud-services + noLink: false + +CatCSaas: + id: CatCSaas + category: cat-c + title: SaaS + icon: web + selected: false + description: + link: /categories/operational#saas + noLink: false + +CatCManaged: + id: CatCManaged + category: cat-c + title: Managed Services + icon: storage + selected: false + description: + link: /categories/operational#managed-services + noLink: false + +CatDEndUserDevices: + id: CatDEndUserDevices + category: cat-d + title: End-User Devices + icon: devices + selected: false + description: + link: /categories/downstream#end-user-devices + noLink: false + +CatDNetworkDataTransfer: + id: CatDNetworkDataTransfer + category: cat-d + title: Network Data Transfer + icon: cell-tower + selected: false + description: + link: /categories/downstream#network-data-transfer + noLink: false + +CatDInfrastructure: + id: CatDInfrastructure + category: cat-d + title: Downstream Infrastructure + icon: hub + selected: false + description: + link: /categories/downstream#infrastructure + noLink: false \ No newline at end of file From 4528204449bd6dc733744725d5f60d6cbe4a3b10 Mon Sep 17 00:00:00 2001 From: dkrees Date: Wed, 3 Apr 2024 17:41:24 +0100 Subject: [PATCH 02/16] Update categoryLabels.yml From 211bcce272b210ede80dacc4324c05c7dcbdabf9 Mon Sep 17 00:00:00 2001 From: dkrees Date: Wed, 3 Apr 2024 17:50:41 +0100 Subject: [PATCH 03/16] Update categoryItem.html From 850d7b4a74247b46f27616c7b63108f4f6f954f7 Mon Sep 17 00:00:00 2001 From: dkrees Date: Wed, 3 Apr 2024 17:53:59 +0100 Subject: [PATCH 04/16] Update categoryItem.html From caefd548f78afff0cfec56ce608e8f8523d27114 Mon Sep 17 00:00:00 2001 From: dkrees Date: Wed, 3 Apr 2024 18:06:12 +0100 Subject: [PATCH 05/16] Update icons.svg adding downstream infra icon From e220719c7756092da11b00cfe00e92638cad8eed Mon Sep 17 00:00:00 2001 From: dkrees Date: Thu, 4 Apr 2024 10:13:09 +0100 Subject: [PATCH 06/16] Mapped the descriptive text for each category to data file. Renamed the categoryItems.yml to carbonStandard.yml to encapsulate all the things in the standard. --- _data/categoryItems.yml | 153 ---------------------------------------- 1 file changed, 153 deletions(-) delete mode 100644 _data/categoryItems.yml diff --git a/_data/categoryItems.yml b/_data/categoryItems.yml deleted file mode 100644 index eb89087..0000000 --- a/_data/categoryItems.yml +++ /dev/null @@ -1,153 +0,0 @@ -# These are the default values for the technology category items used in the _includes/categoryItem.html template. -# Only the 'id', 'selected', 'description' and 'link' parameters can be overridden when using categoryItem template. -# All other vales are the default values, used EVERYWHERE, so please be aware when making any changes here. - -# The Category Items are mapped and accessible using thier item name. -# - id: is the id attribute applied to the categoryItem. This is useful for anchor links. Useful for linking from a infographic item to an in page category item (optional, but defaults to the name of the item). -# - category: is actually a css class that is passed through to set the colour of the item (background to the icon). There maybe other future oppourtunities to use this. -# - title: used as the main title/descriptor of the item (required and default). -# - icon: used as the icon of the item (required and default). Maps to the assets/images/icons.svg. -# - selected: when true, this highlights the item. Used for the contextual overlay views (false by default). -# - description: adds additional text under the title providing som context. Used for the contextual overlay views (optional). -# - link: Creates a link to another page (further details). These are setup by default to the relevant pages, but may be overridden for specific use cases, typically in the overlay views (optional, but defaults to a page). -# - noLink: Provides an overide to prevent any navigation to a page. Renders the component in a
instead of an . - -CatUSoftware: - id: CatUSoftware - category: cat-u - title: Off The Shelf including Open-Source Software - icon: web - selected: false - description: - link: /categories/upstream#software - noLink: false - -CatUEmployeeHardware: - id: CatUEmployeeHardware - category: cat-u - title: User Hardware (laptops, printers, etc.) - icon: devices - selected: false - description: - link: /categories/upstream#employee-hardware - noLink: false - -CatUNetworkHardware: - id: CatUNetworkHardware - category: cat-u - title: Networking and Infrastructure Hardware - icon: router - selected: false - description: - link: /categories/upstream#networking-hardware - noLink: false - -CatUServerHardware: - id: CatUServerHardware - category: cat-u - title: Servers and Storage Hardware - icon: storage - selected: false - description: - link: /categories/upstream#data-center-and-server-hardware - noLink: false - -CatOServers: - id: CatOServers - category: cat-o - title: Servers and Storage - icon: storage - selected: false - description: - link: /categories/operational#servers-and-storage - noLink: false - -CatONetworkDevices: - id: CatONetworkDevices - category: cat-o - title: Networking and Infrastructure - icon: router - selected: false - description: - link: /categories/operational#networking-devices - noLink: false - -CatOUserDevices: - id: CatOUserDevices - category: cat-o - title: User Devices - icon: devices - selected: false - description: - link: /categories/operational#employee-devices - noLink: false - -CatGGenerators: - id: CatGGenerators - category: cat-g - title: Generators - icon: bolt - selected: false - description: - link: /categories/operational#generators - noLink: false - -CatCCloud: - id: CatCCloud - category: cat-c - title: Cloud Services - icon: cloud - selected: false - description: - link: /categories/operational#cloud-services - noLink: false - -CatCSaas: - id: CatCSaas - category: cat-c - title: SaaS - icon: web - selected: false - description: - link: /categories/operational#saas - noLink: false - -CatCManaged: - id: CatCManaged - category: cat-c - title: Managed Services - icon: storage - selected: false - description: - link: /categories/operational#managed-services - noLink: false - -CatDEndUserDevices: - id: CatDEndUserDevices - category: cat-d - title: End-User Devices - icon: devices - selected: false - description: - link: /categories/downstream#end-user-devices - noLink: false - -CatDNetworkDataTransfer: - id: CatDNetworkDataTransfer - category: cat-d - title: Network Data Transfer - icon: cell-tower - selected: false - description: - link: /categories/downstream#network-data-transfer - noLink: false - -CatDInfrastructure: - id: CatDInfrastructure - category: cat-d - title: Downstream Infrastructure - icon: hub - selected: false - description: - link: /categories/downstream#infrastructure - noLink: false \ No newline at end of file From e5e07a9184e973a7e66076ff99c5dfbe42f49e83 Mon Sep 17 00:00:00 2001 From: dkrees Date: Thu, 4 Apr 2024 11:36:40 +0100 Subject: [PATCH 07/16] making the infographic reusable with a variant template From 64a36384eca7f1118f3dca5b33043048ce35b7a0 Mon Sep 17 00:00:00 2001 From: dkrees Date: Thu, 4 Apr 2024 12:27:48 +0100 Subject: [PATCH 08/16] Adding TCS views From 378d56c5eafdd9452863d845e39240b1537e7c02 Mon Sep 17 00:00:00 2001 From: dkrees Date: Fri, 3 May 2024 15:20:40 +0100 Subject: [PATCH 09/16] Create template-view.yml Adding a template file for duplicating and using for overlay views. From 925d2d9d80df61cfa59787d314e9ce9ee312f18e Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Mon, 13 May 2024 11:26:40 +0100 Subject: [PATCH 10/16] Add data overlay (copied from architecture). Added first pass of content for data page. --- _data/overlays/data.yml | 72 ++++++++++++++++++++++++++++++++++++ pages/views/roles/data.md | 77 +++++++++++++++++++++++++++++++++++++++ pages/views/views.md | 1 + 3 files changed, 150 insertions(+) create mode 100644 _data/overlays/data.yml create mode 100644 pages/views/roles/data.md diff --git a/_data/overlays/data.yml b/_data/overlays/data.yml new file mode 100644 index 0000000..d463218 --- /dev/null +++ b/_data/overlays/data.yml @@ -0,0 +1,72 @@ +# Optionally change the title of the infographic. Defaults to the carbonStandard.title otherwise. +title: + +CatUSoftware: + selected: false + description: false + noLink: true + +CatUEmployeeHardware: + selected: false + description: + noLink: true + +CatUNetworkHardware: + selected: false + description: + noLink: true + +CatUServerHardware: + selected: true + description: Drives for data at rest + link: '#ServerHardware' + +CatOServers: + selected: true + description: Organisation of data at rest + link: "#Servers" + +CatONetworkDevices: + selected: true + description: Data sent over networks + noLink: "#NetworkDevices" + +CatOUserDevices: + selected: false + description: + noLink: true + +CatGGenerators: + selected: false + description: + noLink: true + +CatCCloud: + selected: false + description: + noLink: true + +CatCSaas: + selected: true + description: Data use in hosted software + link: '#Saas' + +CatCManaged: + selected: true + description: Data hosted externally + link: '#Managed' + +CatDEndUserDevices: + selected: true + description: Data on end user devices + link: '#EndUserDevices' + +CatDNetworkDataTransfer: + selected: true + description: Data in transit + link: '#NetworkTransfer' + +CatDInfrastructure: + selected: true + description: + link: '#Infrastructure' \ No newline at end of file diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md new file mode 100644 index 0000000..f69d821 --- /dev/null +++ b/pages/views/roles/data.md @@ -0,0 +1,77 @@ +--- +layout: default +title: Data View +permalink: views/roles/data +--- + +# Data View + +{% include carbonStandard.html variant=site.data.overlays.data hideOutOfScope=true %} + +Every bit of data, at rest or in transit, incurs a carbon cost. +If a user minimises the amount of data they store, then, they can significantly reduce carbon emissions due to storage. Doing so is also likely to reduce monetary costs: hosted services generally charge by data use, while local storage will use less energy to read and write the data, and can use fewer or smaller storage options, reducing embedded carbon. + +## Upstream Emissions + +## Operational Emissions + +While upstream data has a carbon cost, it is operational emissions where a company can make the choices for the greatest effect. + +The first, and perhaps biggest choice, is where to store data: on local servers, or on the cloud (or some other hosted solution). + +In either case, though, costs scale with the amount of data stored (and, to a lesser extent, with the rate at which that data is written or read). Therefore, compressing the data is likely to reduce storage costs; however, it will incur extra emissions in computing for compression and decompression. + +### Direct +{% include categoryLabel.html label="CatO" %} +{% include categoryItem.html item="CatONetworkDevices" id="NetworkDevices" noLink=true %} + + +Transferring data over networks incurs an electricity cost, and therefore a carbon cost. + +{% include categoryItem.html item="CatOServers" id="Servers" noLink=true %} + +When hosting locally, the biggest factor in emissions is likely to be the embodied carbon of the servers. + +| Storage Type | Embedded Carbon (kgCO2e/TB) | Operating Power (W/TB) | +| ----------------------- | --------------------------: | ---------------------: | +| Hard Disk Drive (HDD) | 38 | 1 | +| Solid State Drive (SSD) | 316 | 2.5 | + +(These figures are for a 8 TB HDD and a 1 TB SSD. Of course, there are reasons for using SSDs over HDDs, such as faster read/write speed.) + +A HDD will therefore have roughly equal amortised embodied carbon and operating emissions over 10 years of constant operation (while a SSD would require 35 years!). Since most storage media have a typical use life of around 5 years, increasing the lifespan of storage media is likely to have a significant effect on average emissions. + +#### Multiple Copies and Backups + +Backups are commonly recommended for most data, and certainly all important data. Guidance like the 3-2-1 rule suggest keeping multiple copies, across multiple media and multiple sites. + + + +### Indirect + +{% include categoryLabel.html label="CatC" %} + +{% include categoryItem.html item="CatCManaged" id="Managed" noLink=true %} + +Much data is now stored on cloud storage - that is, storage managed by a cloud provider and accessed over the internet. Almost all cloud storage exists, ultimately, in HDD or SSD, and so while embedded carbon costs are hidden from the user, they are still incurred by the provider. + +Cloud storage provision also incurs a carbon cost from use, and this can be more visible to the end user. Some cloud providers allow users to examine estimated emissions from their services, but not all do; still, emissions can be estimated since they are roughly proportional to the amount of data stored. Storing one terabyte of data on a major cloud storage provider for one year uses around 35 kilowatt-hours of electricity. + +It should be noted that, by default, most cloud providers duplicate data, keeping three copies on three different machines (to ensure availability and insure against drive failure). + +#### Location and Legislation + +Since users can typically select different regions for their cloud service hosting, they can select regions with different carbon intensities and so different resultant emissions. However, there may be data localisation laws, which limit the sites at which personal data may be stored. + +## Downstream Emissions + +{% include categoryLabel.html label="CatD" %} + +{% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" noLink=true %} + +The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) + +{% include categoryItem.html item="CatDNetworkDataTransfer" id="NetworkTransfer" noLink=true %} + + + \ No newline at end of file diff --git a/pages/views/views.md b/pages/views/views.md index 34d7a8d..686a16b 100644 --- a/pages/views/views.md +++ b/pages/views/views.md @@ -14,6 +14,7 @@ Here, you can find some created views that look at the Tech Carbon Standard with Role views look at the Technology Carbon Standard from the perspective of a business role and the concerns that a role would most likely focus its efforts on. - [Architecture Views](/views/roles/architecture) - how to incorporate sustainability into strategy and into NFRs +- [Data View](/views/roles/data) ## Organisation Types From 8a90fbe88a1a39857ca2ad92fa560f38f97dc8e5 Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Fri, 7 Jun 2024 15:51:07 +0100 Subject: [PATCH 11/16] Added data on tape drives --- pages/views/roles/data.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index f69d821..4d975a1 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -23,12 +23,11 @@ In either case, though, costs scale with the amount of data stored (and, to a le ### Direct {% include categoryLabel.html label="CatO" %} -{% include categoryItem.html item="CatONetworkDevices" id="NetworkDevices" noLink=true %} - +{% include categoryItem.html item="CatONetworkDevices" id="NetworkDevices" %} Transferring data over networks incurs an electricity cost, and therefore a carbon cost. -{% include categoryItem.html item="CatOServers" id="Servers" noLink=true %} +{% include categoryItem.html item="CatOServers" id="Servers" %} When hosting locally, the biggest factor in emissions is likely to be the embodied carbon of the servers. @@ -41,17 +40,21 @@ When hosting locally, the biggest factor in emissions is likely to be the embodi A HDD will therefore have roughly equal amortised embodied carbon and operating emissions over 10 years of constant operation (while a SSD would require 35 years!). Since most storage media have a typical use life of around 5 years, increasing the lifespan of storage media is likely to have a significant effect on average emissions. -#### Multiple Copies and Backups +#### Long-term Copies and Backups -Backups are commonly recommended for most data, and certainly all important data. Guidance like the 3-2-1 rule suggest keeping multiple copies, across multiple media and multiple sites. +Backups are commonly recommended for most data, and certainly all important data. Guidance like the 3-2-1 rule suggest keeping multiple copies, across multiple media and multiple sites. This will increase the carbon cost of storage accordingly, and should be accounted for. - +Removable media may also be considered for long-term storage. Their intermittent use makes it harder estimate energy use while active, though published data imply that energy use is significantly lower than that for fixed drives. Their embedded carbon may still be measured: + +| Storage Type | Embedded Carbon (kgCO2e) | Media Embedded Carbon (kgCO2e/TB) | +| ------------------ | -----------------------: | --------------------------------: | +| Tape Drive (LTO 8) | 46.6 | 0.62 | ### Indirect {% include categoryLabel.html label="CatC" %} -{% include categoryItem.html item="CatCManaged" id="Managed" noLink=true %} +{% include categoryItem.html item="CatCManaged" id="Managed" %} Much data is now stored on cloud storage - that is, storage managed by a cloud provider and accessed over the internet. Almost all cloud storage exists, ultimately, in HDD or SSD, and so while embedded carbon costs are hidden from the user, they are still incurred by the provider. @@ -67,11 +70,11 @@ Since users can typically select different regions for their cloud service hosti {% include categoryLabel.html label="CatD" %} -{% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" noLink=true %} +{% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" %} The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) -{% include categoryItem.html item="CatDNetworkDataTransfer" id="NetworkTransfer" noLink=true %} +{% include categoryItem.html item="CatDNetworkDataTransfer" id="NetworkTransfer" %} \ No newline at end of file From 9d5cc3d632a38d4ef7105417f754d465f4b93c38 Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Mon, 8 Jul 2024 09:01:35 +0100 Subject: [PATCH 12/16] More rework: feedback from review; section on ROT --- pages/views/roles/data.md | 65 ++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index 4d975a1..3d9b693 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -9,46 +9,42 @@ permalink: views/roles/data {% include carbonStandard.html variant=site.data.overlays.data hideOutOfScope=true %} Every bit of data, at rest or in transit, incurs a carbon cost. -If a user minimises the amount of data they store, then, they can significantly reduce carbon emissions due to storage. Doing so is also likely to reduce monetary costs: hosted services generally charge by data use, while local storage will use less energy to read and write the data, and can use fewer or smaller storage options, reducing embedded carbon. +If a user minimises the amount of data they store, then they can significantly reduce carbon emissions due to storage. Doing so is also likely to reduce monetary costs: hosted services generally charge by data use, while local storage will use less energy to read and write the data, and can use fewer or smaller storage options, reducing embodied carbon. ## Upstream Emissions -## Operational Emissions - -While upstream data has a carbon cost, it is operational emissions where a company can make the choices for the greatest effect. +### Servers and Storage Hardware +{% include categoryItem.html item="CatUServerHardware" id="ServerHardware" %} -The first, and perhaps biggest choice, is where to store data: on local servers, or on the cloud (or some other hosted solution). +The first, and perhaps biggest choice, is where to store data: on local servers, or on the cloud (for which, see direct, below). -In either case, though, costs scale with the amount of data stored (and, to a lesser extent, with the rate at which that data is written or read). Therefore, compressing the data is likely to reduce storage costs; however, it will incur extra emissions in computing for compression and decompression. +The embodied costs of storage hardware will likely be a major contribution from on-premise storage: -### Direct -{% include categoryLabel.html label="CatO" %} -{% include categoryItem.html item="CatONetworkDevices" id="NetworkDevices" %} +| Storage Type | Embodied Carbon | +| ----------------------- | -------------------------: | +| Hard Disk Drive (HDD) | 20 kgCO2e/TB | +| Solid State Drive (SSD) | 160 kgCO2e/TB | +| Tape Drive (LTO 8) | 46 kgCO2e + 0.62 kgCO2e/TB | -Transferring data over networks incurs an electricity cost, and therefore a carbon cost. +(These embodied figures are average estmates from Tammu/Nair[^dirtysecret] and a tape data sheet[^tapedrive]. Of course, there are other considerations between media, such as faster read/write speed.) -{% include categoryItem.html item="CatOServers" id="Servers" %} - -When hosting locally, the biggest factor in emissions is likely to be the embodied carbon of the servers. +#### Long-term Copies and Backups -| Storage Type | Embedded Carbon (kgCO2e/TB) | Operating Power (W/TB) | -| ----------------------- | --------------------------: | ---------------------: | -| Hard Disk Drive (HDD) | 38 | 1 | -| Solid State Drive (SSD) | 316 | 2.5 | +Backups are commonly recommended for most data, and certainly all important data. Guidance like the 3-2-1 rule[^321rule] suggest keeping multiple copies, across multiple media and multiple sites. This will increase the carbon cost of storage accordingly, and should be accounted for. -(These figures are for a 8 TB HDD and a 1 TB SSD. Of course, there are reasons for using SSDs over HDDs, such as faster read/write speed.) +Removable media should also be considered for long-term storage. Their intermittent use makes it harder to estimate energy use while active, though published data imply that energy use is significantly lower than that for fixed drives. -A HDD will therefore have roughly equal amortised embodied carbon and operating emissions over 10 years of constant operation (while a SSD would require 35 years!). Since most storage media have a typical use life of around 5 years, increasing the lifespan of storage media is likely to have a significant effect on average emissions. +## Operational Emissions -#### Long-term Copies and Backups +### Direct +{% include categoryLabel.html label="CatO" %} +{% include categoryItem.html item="CatONetworkDevices" id="NetworkDevices" %} -Backups are commonly recommended for most data, and certainly all important data. Guidance like the 3-2-1 rule suggest keeping multiple copies, across multiple media and multiple sites. This will increase the carbon cost of storage accordingly, and should be accounted for. +Transferring data over networks incurs an electricity cost, and therefore a carbon cost. As with data at rest, these costs can be minimised by compression. -Removable media may also be considered for long-term storage. Their intermittent use makes it harder estimate energy use while active, though published data imply that energy use is significantly lower than that for fixed drives. Their embedded carbon may still be measured: +{% include categoryItem.html item="CatOServers" id="Servers" %} -| Storage Type | Embedded Carbon (kgCO2e) | Media Embedded Carbon (kgCO2e/TB) | -| ------------------ | -----------------------: | --------------------------------: | -| Tape Drive (LTO 8) | 46.6 | 0.62 | +Running the drives incurs a cost in electricity and therefore carbon. Most devices, regardless of type or size, run at a power of single-digit Watts when reading or writing; idle powers can vary more significantly, even within single media types. High-power drives in frequent use may therefore have yearly power consumptions of hundreds of kWh, and concomitantly high emissions. ### Indirect @@ -56,7 +52,7 @@ Removable media may also be considered for long-term storage. Their intermittent {% include categoryItem.html item="CatCManaged" id="Managed" %} -Much data is now stored on cloud storage - that is, storage managed by a cloud provider and accessed over the internet. Almost all cloud storage exists, ultimately, in HDD or SSD, and so while embedded carbon costs are hidden from the user, they are still incurred by the provider. +Much data is now stored on cloud storage - that is, storage managed by a cloud provider and accessed over the internet. Almost all cloud storage exists, ultimately, in HDD or SSD, and so while embodied carbon costs are hidden from the user, they are still incurred by the provider. Cloud storage provision also incurs a carbon cost from use, and this can be more visible to the end user. Some cloud providers allow users to examine estimated emissions from their services, but not all do; still, emissions can be estimated since they are roughly proportional to the amount of data stored. Storing one terabyte of data on a major cloud storage provider for one year uses around 35 kilowatt-hours of electricity. @@ -64,7 +60,11 @@ It should be noted that, by default, most cloud providers duplicate data, keepin #### Location and Legislation -Since users can typically select different regions for their cloud service hosting, they can select regions with different carbon intensities and so different resultant emissions. However, there may be data localisation laws, which limit the sites at which personal data may be stored. +Since users can typically select different regions for their cloud service hosting, they can select regions with different carbon intensities and so different resultant emissions. However, data localisation laws, which limit the sites at which personal data may be stored, may constrain this choice. + +#### ROT Data and auditing + +ROT (Redundant, Obsolete or Trivial) is a term used to describe useless data; surveys indicate that around 30% of data stored on the cloud is ROT[^ROTscale]. Since this data is still hosted, it will contribute to emissions as with useful data. It can be minimised with regular data audits, which analyse stored data to detect and remove ROT. ## Downstream Emissions @@ -72,9 +72,10 @@ Since users can typically select different regions for their cloud service hosti {% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" %} -The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) - -{% include categoryItem.html item="CatDNetworkDataTransfer" id="NetworkTransfer" %} +The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured directly, since it will depend upon users' choices). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) - - \ No newline at end of file +## References +[^321rule]: [US-CERT Data Backup Options](https://www.cisa.gov/sites/default/files/publications/data_backup_options.pdf) +[^dirtysecret]: [Tannu/Nair 2023, The Dirty Secret of SSDs](https://arxiv.org/pdf/2207.10793). See also [Pure Storage on Embodied Carbon](https://blog.purestorage.com/perspectives/how-does-the-embodied-carbon-dioxide-equivalent-of-flash-compare-to-hdds/), a rebuttal (whose numbers do not include upstream emissions from e.g. mineral extraction). +[^tapedrive]: [IBM, Phyiscal Tape CO2, p. 6](https://www.ibm.com/downloads/cas/YE5WAQ0B) +[^ROTscale]: [Veritas Databerg Report, 2020](https://www.veritas.com/content/dam/www/en_us/documents/at-a-glance/AG_uk_databerg_report.pdf) \ No newline at end of file From 48346a03e55a540c316c0f07f5375f10d2da443b Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Thu, 18 Jul 2024 14:48:49 +0100 Subject: [PATCH 13/16] Fixed links; added data reference. --- _data/carbonStandard.yml | 2 +- _data/overlays/data.yml | 16 ++++++++-------- pages/views/roles/data.md | 7 ++++--- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/_data/carbonStandard.yml b/_data/carbonStandard.yml index 123c5da..0903e92 100644 --- a/_data/carbonStandard.yml +++ b/_data/carbonStandard.yml @@ -61,7 +61,7 @@ CatUServerHardware: icon: storage selected: false description: - link: /categories/upstream#data-center-and-server-hardware + link: /categories/upstream#data-centre-and-server-hardware noLink: false OperationalEmissions: diff --git a/_data/overlays/data.yml b/_data/overlays/data.yml index d463218..dc1aa5e 100644 --- a/_data/overlays/data.yml +++ b/_data/overlays/data.yml @@ -29,7 +29,7 @@ CatOServers: CatONetworkDevices: selected: true description: Data sent over networks - noLink: "#NetworkDevices" + link: "#NetworkDevices" CatOUserDevices: selected: false @@ -42,19 +42,19 @@ CatGGenerators: noLink: true CatCCloud: - selected: false - description: - noLink: true + selected: true + description: Data hosted on the cloud + link: '#Cloud' CatCSaas: selected: true description: Data use in hosted software - link: '#Saas' + noLink: true CatCManaged: - selected: true - description: Data hosted externally - link: '#Managed' + selected: false + description: + noLink: true CatDEndUserDevices: selected: true diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index 3d9b693..5d1f3c4 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -50,11 +50,11 @@ Running the drives incurs a cost in electricity and therefore carbon. Most devic {% include categoryLabel.html label="CatC" %} -{% include categoryItem.html item="CatCManaged" id="Managed" %} +{% include categoryItem.html item="CatCCloud" id="Cloud" %} Much data is now stored on cloud storage - that is, storage managed by a cloud provider and accessed over the internet. Almost all cloud storage exists, ultimately, in HDD or SSD, and so while embodied carbon costs are hidden from the user, they are still incurred by the provider. -Cloud storage provision also incurs a carbon cost from use, and this can be more visible to the end user. Some cloud providers allow users to examine estimated emissions from their services, but not all do; still, emissions can be estimated since they are roughly proportional to the amount of data stored. Storing one terabyte of data on a major cloud storage provider for one year uses around 35 kilowatt-hours of electricity. +Cloud storage provision also incurs a carbon cost from use, and this can be more visible to the end user. Some cloud providers allow users to examine estimated emissions from their services, but not all do; still, emissions can be estimated since they are roughly proportional to the amount of data stored. Storing one terabyte of data on a major cloud storage provider for one year uses around 32 kilowatt-hours of electricity[^CloudStorageRate]. It should be noted that, by default, most cloud providers duplicate data, keeping three copies on three different machines (to ensure availability and insure against drive failure). @@ -78,4 +78,5 @@ The same arguments for storage on servers (see above) apply to user devices (tho [^321rule]: [US-CERT Data Backup Options](https://www.cisa.gov/sites/default/files/publications/data_backup_options.pdf) [^dirtysecret]: [Tannu/Nair 2023, The Dirty Secret of SSDs](https://arxiv.org/pdf/2207.10793). See also [Pure Storage on Embodied Carbon](https://blog.purestorage.com/perspectives/how-does-the-embodied-carbon-dioxide-equivalent-of-flash-compare-to-hdds/), a rebuttal (whose numbers do not include upstream emissions from e.g. mineral extraction). [^tapedrive]: [IBM, Phyiscal Tape CO2, p. 6](https://www.ibm.com/downloads/cas/YE5WAQ0B) -[^ROTscale]: [Veritas Databerg Report, 2020](https://www.veritas.com/content/dam/www/en_us/documents/at-a-glance/AG_uk_databerg_report.pdf) \ No newline at end of file +[^ROTscale]: [Veritas Databerg Report, 2020](https://www.veritas.com/content/dam/www/en_us/documents/at-a-glance/AG_uk_databerg_report.pdf) +[^CloudStorageRate]: [Cloud Carbon Footprint methodology](https://www.cloudcarbonfootprint.org/docs/methodology/#storage) \ No newline at end of file From 0c82e8dfef426b68e2ede4042b217cd34778c975 Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Tue, 23 Jul 2024 16:29:25 +0100 Subject: [PATCH 14/16] Changed wording. --- pages/views/roles/data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index 5d1f3c4..587f7e8 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -72,7 +72,7 @@ ROT (Redundant, Obsolete or Trivial) is a term used to describe useless data; su {% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" %} -The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured directly, since it will depend upon users' choices). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) +The same arguments for storage on servers (see above) apply to user devices (though, depending upon the project, user devices' embodied carbon may be accounted for elsewhere). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) ## References [^321rule]: [US-CERT Data Backup Options](https://www.cisa.gov/sites/default/files/publications/data_backup_options.pdf) From fdf17a82b711b6cff48223b2b91a362555f52d7e Mon Sep 17 00:00:00 2001 From: Angus Balkham Date: Wed, 24 Jul 2024 10:38:08 +0100 Subject: [PATCH 15/16] Changes from David's feedback. --- pages/views/roles/data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index 587f7e8..5b69132 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -72,7 +72,7 @@ ROT (Redundant, Obsolete or Trivial) is a term used to describe useless data; su {% include categoryItem.html item="CatDEndUserDevices" id="EndUserDevices" %} -The same arguments for storage on servers (see above) apply to user devices (though, depending upon the project, user devices' embodied carbon may be accounted for elsewhere). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) +The same arguments for storage on servers (see above) apply to user devices (though embodied carbon can no longer be measured so confidently, since developers will typically not know the users' device choices). However, electricity use for storage can be estimated, using the knowledge of the type and size of data stored, and the likely storage medium. (For instance, smartphones store data in flash storage, while laptops and desktops may use SSDs or HDDs.) ## References [^321rule]: [US-CERT Data Backup Options](https://www.cisa.gov/sites/default/files/publications/data_backup_options.pdf) From 76bf1154ea317ab7a361afff827ab3e4fbe41dfc Mon Sep 17 00:00:00 2001 From: jcamilleri-scottlogic Date: Mon, 29 Jul 2024 11:09:43 +0100 Subject: [PATCH 16/16] SF-173 marked data overlay as unpublished until we can review the file and nav structure of overlays. --- pages/views/roles/data.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pages/views/roles/data.md b/pages/views/roles/data.md index 5b69132..8c643fb 100644 --- a/pages/views/roles/data.md +++ b/pages/views/roles/data.md @@ -2,6 +2,7 @@ layout: default title: Data View permalink: views/roles/data +published: false --- # Data View @@ -79,4 +80,4 @@ The same arguments for storage on servers (see above) apply to user devices (tho [^dirtysecret]: [Tannu/Nair 2023, The Dirty Secret of SSDs](https://arxiv.org/pdf/2207.10793). See also [Pure Storage on Embodied Carbon](https://blog.purestorage.com/perspectives/how-does-the-embodied-carbon-dioxide-equivalent-of-flash-compare-to-hdds/), a rebuttal (whose numbers do not include upstream emissions from e.g. mineral extraction). [^tapedrive]: [IBM, Phyiscal Tape CO2, p. 6](https://www.ibm.com/downloads/cas/YE5WAQ0B) [^ROTscale]: [Veritas Databerg Report, 2020](https://www.veritas.com/content/dam/www/en_us/documents/at-a-glance/AG_uk_databerg_report.pdf) -[^CloudStorageRate]: [Cloud Carbon Footprint methodology](https://www.cloudcarbonfootprint.org/docs/methodology/#storage) \ No newline at end of file +[^CloudStorageRate]: [Cloud Carbon Footprint methodology](https://www.cloudcarbonfootprint.org/docs/methodology/#storage)