r.2024.05.08.0

Summary:

• New management command for running Trino migrations
• Storing memory, OS, and storage information in support of new features that are in development

What's Changed

• COST-5009 Fix data details (cluster info) timestamps for start of new month by @lcouzens in #5079
• COST-4936 Add product_memory and product_operatingsystem columns to AWS line items tables in Trino by @djnakabaale in #5071
• COST-5007 Move trino migration to a masu management command by @maskarb in #4963
• COST-1665, COST-4866 Support new storage report columns by @maskarb in #5052
• Add inital backstage integration by @lcouzens in #5082
• Fix mkdocs.yml file location by @lcouzens in #5083
• COST-5012 Add cost models to internal sources endpoint by @maskarb in #5084
• COST-3617 Fix GCP end of month summary by @lcouzens in #5080

Full Changelog: r.2024.05.01.1...r.2024.05.08.0

r.2024.05.01.1

Summary:

• Improvements to bigquery queries to create manifests in sequential order

• The correlation logic between Microsoft Azure cloud integrations and OpenShift clusters has been updated to perform a partial match instead of an exact match. This change accommodates any additional prefixes or suffixes added to the Azure resource identifiers when matching with the names of OpenShift's persistent volumes.

  Azure	OpenShift
  az-123-volume_name1	volume_name1
  my-volume-name-OS_disk	my-volume-name

What's Changed

• [COST-4947] Migration to remove account and region from ProviderInfrastructureMap by @cgoodfred in #5077
• Add order by to bigquery queries (for manifest creation datetimes) by @lcouzens in #5074
• [COST-4967] Undo partial matching on node for ocp on azure by @myersCody in #5078

Full Changelog: r.2024.05.01.0...r.2024.05.01.1

r.2024.05.01.0

Summary:

• Development and operational enhancements
• Bug fixes around report status information for monitoring ingest process

What's Changed

• [COST-4967] Update Azure Openshift Correlation to be contains. by @myersCody in #5062
• [COST-4844] Add group by storageclass to OCP volume endpoint. by @myersCody in #5044
• [COST-4975] - Fixing GCP summary status for sources endpoint by @lcouzens in #5064
• [COST-4975] - Fix OCP infra state lookup by @lcouzens in #5070
• [COST-4759] Add all_labels to back populate for ocp on cloud. by @myersCody in #5001
• [COST-4762] Update HCS AWS Estimate pricing by @cgoodfred in #4979
• [COST-3761] API for node network costs returning example data by @samdoran in #5051
• Use multi-arch container images for development by @samdoran in #5045
• [COST-4807] Masu trino query readonly user by @cgoodfred in #5067
• [COST-4914] Update classification to include unattributed by @myersCody in #5073
• prevent running migration CI check if removal label is present by @cgoodfred in #5075
• [COST-4947] Remove account and region fields from ProviderInfrastructureMap by @cgoodfred in #5076

Full Changelog: r.2024.04.25.0...r.2024.05.01.0

r.2024.04.25.0

Summary:

Small bug fix around mid month finalization for AWS causing gaps in cost.

What's Changed

• COST-4871 mount OCI credentials as volume by @maskarb in #5048
• COST-4871 add oci env variables to subs workers by @maskarb in #5050
• COST-4947 Remove references to region and account in ProviderInfrastructureMap by @cgoodfred in #5053
• COST-861 Tokenizing error - skip bad rows when reading csv by @maskarb in #5031
• COST-4873 update ephemeral placeholder credentials by @maskarb in #5059
• COST-4974 Trino Migration to Drop Azure Line Items by @cgoodfred in #5061
• COST-4973 - Fix mid month finalization by @lcouzens in #5060
• Update view.py by @lcouzens in #5063
• update trino and hive images for local dev by @maskarb in #4717

Full Changelog: r.2024.04.18.0...r.2024.04.25.0

r.2024.04.18.0

Summary:

Small bug fix around OCP on Cloud infrastructure matching for correlating OpenShift metrics with Cloud billing costs
Updated dependencies and improving security around credentials

What's Changed

• update dependencies by @maskarb in #5035
• Bump sqlparse from 0.4.4 to 0.5.0 in the pip group across 1 directory by @dependabot in #5043
• Add dev doc about worker unallocated distribution by @myersCody in #5041
• COST-4807 mount aws credentials as volume by @maskarb in #5022
• update sqlparse and gunicorn by @maskarb in #5047
• COST-4947 Remove account and region from ProviderInfrastructureMap filter by @cgoodfred in #5049

Full Changelog: r.2024.04.16.0...r.2024.04.18.0

r.2024.04.16.0

Summary:

• COST-4372 fix inflated PVC capacity count on reports/openshift/volumes/ endpoint by aggregating the max counts
• updates in preparation for a major Trino upgrade

What's Changed

• add smoke_test shell script back by @cgoodfred in #5036
• Mark smoke_test.sh as an executable file by @cgoodfred in #5037
• rename components to trino in smoke_test.sh by @cgoodfred in #5038
• COST-4372 Use Max for volume capacity count by @djnakabaale in #5032
• Update delayed resummary logic for ocp on cloud. by @myersCody in #5027
• COST-4626 strip acct from schema in parquet converter by @maskarb in #5039
• COST-4626 bump parquet converter version by @maskarb in #5040

Full Changelog: r.2024.04.15.0...r.2024.04.16.0

r.2024.04.15.0

Summary:

• address numerous SAST results
• RBAC: enable OCP lower level access (project/node) when only the highest level (cluster) is enabled
• In this release, we've addressed an edge case concerning Worker unallocated distribution. Previously, the Worker unallocated cost was not negated after distributing costs for clusters whose worker nodes reported zero CPU capacity, resulting in inflated costs due to Worker unallocated costs counting twice.

What's Changed

• COST-4874 address shellcheck warnings by @maskarb in #5023
• Upgrade Codecov action by @samdoran in #5024
• COST-4807 Remove remove-stale from enabled tags masu endpoint options by @cgoodfred in #5007
• COST-4715 sentry -> glitchtip cleanup p2 by @maskarb in #5012
• COST-4764 Set lower level rbac roles when only top level are set for OpenShift cluster/node/project by @lcouzens in #5020
• COST-4757 Remove custom Unleash strategies by @lcouzens in #5021
• COST-4822 Remove cluster from OCPAWS tag matching SQL by @cgoodfred in #5026
• COST-4785 Worker unallocated cost not negated after distribution by @myersCody in #4999
• COST-3761 Add models for OCP node networking costs by @samdoran in #4968
• COST-4877 Report correct PostgreSQL version by @samdoran in #5019
• add provider-billing-source to sources masu endpoint by @maskarb in #4984
• rename variables for "COPY_PASTE_ERROR" by @maskarb in #5025
• chore(image): update base image by @github-actions in #5018
• Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #5017
• COST-4895 create tmp dir for build-deploy by @maskarb in #5028
• COST-4736 Fix local strategies by @lcouzens in #5029
• rename presto -> trino in app-interface by @maskarb in #5030
• COST-4626 return pd.NaT instead of None for custom date converters by @maskarb in #5034
• chore(image): update base image by @github-actions in #5033

Full Changelog: r.2024.04.08.0...r.2024.04.15.0

r.2024.04.08.0

Summary:

In this release, we've made some security improvements and other minor changes.

What's Changed

• Upgrade unleash for local development by @samdoran in #4996
• Add make target to run local snyk scan by @samdoran in #4906
• COST-4828 Handle 0 in divisor for cost distribution SQL by @cgoodfred in #5008
• COST-4715 sentry -> glitchtip cleanup p1 by @maskarb in #5010
• COST-4715 sentry -> glitchtip cleanup p1 by @maskarb in #5011
• COST-4704 Update django to 4.2.11 to address CVE-2024-27351 by @djnakabaale in #5009
• COST-4372 OCP: Fix capacity instance count by @djnakabaale in #4964
• COST-4835 validate tag uuids by @maskarb in #5014
• COST-4807 Make provider_uuid not a query param and parameterize bigquery queries by @cgoodfred in #5005
• chore(image): update base image by @github-actions in #5016
• COST-4775 add v3.2.1 commit shas by @maskarb in #4982

Full Changelog: r.2024.03.28.0...r.2024.04.08.0

r.2024.03.28.0

What's Changed

• COST-4802 Add conversion field to kafka message for Subscriptions by @cgoodfred in #4995
• Revert "[COST-4794] Deduplicate parent keys for tag mapping." by @myersCody in #4997
• COST-4423 Fix which manifest is getting updated during summary flows specifically for OCP on Cloud tasks by @lcouzens in #5000
• COST-3607, COST-4627] create model for daily key metrics data by @maskarb in #4989
• COST-4814 Fix inconsistent ordering for tag mapping by @myersCody in #5004
• COST-4801 Added a tracing id to the delayed celery task by @myersCody in #4998
• COST-4759 Add unleash flag for all labels field by @myersCody in #5002
• COST-4807 rename constant to prevent CWE-547 false-positive by @maskarb in #5003

Full Changelog: r.2024.03.25.0...r.2024.03.28.0

r.2024.03.25.0

Minor Changes

• COST-4390 OCP changing GB to GiB by @bacciotti in #4938
• COST-4394 Fixing OCI GB and TB calculations by @bacciotti in #4974
• COST-4219 Update masu manifest endpoint filters by @cgoodfred in #4944
• COST-4459 Normalize product IDs for subs variants by @cgoodfred in #4950
• COST-4121 OpenAPI spec update: new unit (GiB) used on OCP by @bacciotti in #4972
• COST-4578 Add v3.2.0 operator commit mapping by @maskarb in #4971
• COST-4423 Cluster info include timestamps by @lcouzens in #4970
• Increase gunicorn workers for sources client by @samdoran in #4988
• Add populate exchange rates make command by @maskarb in #4990

Bug Fixes

• COST-4714 Explicitly check for Tenant existence in KokuTenantSchemaExistsMiddleware by @maskarb in #4941
• COST-4537 Update crytography and oci by @samdoran in #4962
• COST-4643 Use cascade_delete to remove expired CostUsageReportManifest by @maskarb in #4969
• COST-4730 Fixing pagination to paginate by parent instead of by children by @bacciotti in #4965
• COST-4750 Fix regression caused by bad cast in trino by @myersCody in #4977
• COST-4765 Fix org unit join by @maskarb in #4973
• COST-4423 Fix OCP-GCP summary timestamp issue by @lcouzens in #4978
• Catch non-existent manifest for ocp-on-cloud infra provider by @maskarb in #4993

Upcoming Features

Tag Mapping

• COST-4686 Handling tag rates in tag mapping by @myersCody in #4939
• COST-4729 Fix ordering for tag mapping API by @myersCody in #4957
• COST-4716 Prevent removal of stale tags linked to a mapping by @myersCody in #4948
• COST-4750 OCP Scrub disabled tags in trino by @myersCody in #4961
• COST-4752 Azure scrub disabled tags in trino by @myersCody in #4967
• COST-4772 Tag mapping multiple filter functionality by @myersCody in #4976
• COST-4794 Deduplicate parent keys for tag mapping by @myersCody in #4991
• COST-4751 GCP Scrub disabled tags in Trino by @myersCody in #4966

Full Changelog: r.2024.03.05.0...r.2024.03.25.0