AWS: decrease cost for ~Summer~ Autumn 2023

[dduportal]

Follow up of #3502

Top level goal is to decrease the AWS bill to $5,000 per month.

The June 2023 costs broken down by usage type:

The following tasks are required:

• Move the updates.jenkins.io (pkg.origin.jenkins.io) VM out from AWS: [INFRA-3100] Migrate updates.jenkins.io to another Cloud #2649
  • It is the reason of the "Data transfer Out" cost usage
  • It is an Ubuntu 18.04 machine (ref. Ubuntu 22.04 upgrade campaign #2982 )
  • It is not highly available
• Move the usage.jenkins.io VM to Azure: [INFRA-1972] Migrate usage.jenkins.io to the Azure account #1626
• Move the census.jenkins.io VM to Azure (issue to be done)
• Add a Garbage collector for the S3 artefacts from ci.jenkins.io (ref. Garbage Collector on the ci-jenkins-io-artifacts S3 storage account  #3663)

Tasks

Preview Give feedback

1. [INFRA-3100] Migrate updates.jenkins.io to another Cloud #2649
   aws imported-jira-issue jira-component:aws jira-component:update-center jira-type:update-center update-center +6
   
2. [INFRA-1972] Migrate usage.jenkins.io to the Azure account #1626
   azure imported-jira-issue jira-component:azure jira-component:usage.jenkins.io jira-type:usage.jenkins.io usage.jenkins.io +6
3. Garbage Collector on the ci-jenkins-io-artifacts S3 storage account #3663
   aws ci.jenkins.io +2
4. [ci.jenkins.io] Migrate ci.jenkins.io EKS clusters out from CloudBees AWS account #3954
   aws azure ci.jenkins.io sponsors +4
   

[dduportal]

Update: title updated to move Summer to Autumn...

[dduportal]

Update: Removed a bunch of S3 buckets in #3762

[dduportal]

Update:

• More dangling resources cleanup in cleanup: remove unused resources associated to former EC2 VM agents and their AMIs aws#473

TODO:

• Need to remove the following IAM users (and eventually associated resources):
  • jenkins-infra-ci (https://github.com/jenkins-infra/aws/pull/473/files#diff-09ee06ee2289b6361e23b5c23d9b90725e635e5684537cc6648819ac44c6f1e0L2-L4)
  • jenkins-ci (https://github.com/jenkins-infra/aws/pull/473/files#diff-09ee06ee2289b6361e23b5c23d9b90725e635e5684537cc6648819ac44c6f1e0L57-L59)
  • ci-packer (never tracked as code)
  • helpdesk-3762 (leftover of Migrate Terraform states from AWS S3 to Azure buckets  #3762)

[dduportal]

Update:

TODO:

* Need to remove the following IAM users (and eventually associated resources):
  
  * `jenkins-infra-ci` (https://github.com/jenkins-infra/aws/pull/473/files#diff-09ee06ee2289b6361e23b5c23d9b90725e635e5684537cc6648819ac44c6f1e0L2-L4)
  * `jenkins-ci` (https://github.com/jenkins-infra/aws/pull/473/files#diff-09ee06ee2289b6361e23b5c23d9b90725e635e5684537cc6648819ac44c6f1e0L57-L59)
  * `ci-packer` (never tracked as code)
  * `helpdesk-3762` (leftover of [Migrate Terraform states from AWS S3 to Azure buckets  #3762](https://github.com/jenkins-infra/helpdesk/issues/3762))

Done!

[dduportal]

Update on this topic:

• [ci.jenkins.io] Migrate ci.jenkins.io EKS clusters out from CloudBees AWS account #3954 has been added and done: all us-east-2 resources have been removed and are migrated somewhere else than CloudBees AWS account.
• As such, Garbage Collector on the ci-jenkins-io-artifacts S3 storage account  #3663 is obsolete and closed

(HARD) Work is in progress in #2649