Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[COST-4915] Add unattributed distribution to cost model form #5072

Merged
merged 10 commits into from
May 10, 2024
Merged

[COST-4915] Add unattributed distribution to cost model form #5072

merged 10 commits into from
May 10, 2024

Conversation

myersCody
Copy link
Contributor

@myersCody myersCody commented Apr 26, 2024 •
edited
Loading

Jira Ticket

COST-4915

Description

This change will add the unattributed distribution options to the cost model.

Testing

  1. Checkout Branch
  2. Load test customer data
  3. You should see the new distribution options for our default providers
  4. Delete a cost model:
    DELETE method to:
    http://127.0.0.1:8000/api/cost-management/v1/cost-models/<cost-model-id>/
  5. Add a cost model:
{
  "name": "Cost Management OpenShift Cost Model",
  "description": "A cost model of on-premises OpenShift clusters.",
  "source_type": "OCP",
  "source_uuids": [
    "<your-source-id>"
  ],
  "rates": [
    {
      "metric": {
        "name": "node_cost_per_month",
        "label_metric": "Node",
        "label_measurement": "Currency",
        "label_measurement_unit": "node-month"
      },
      "description": "",
      "tag_rates": {
        "tag_key": "app",
        "tag_values": [
          {
            "unit": "USD",
            "usage": {
              "unit": "USD",
              "usage_end": null,
              "usage_start": null
            },
            "value": 24000,
            "default": true,
            "tag_value": "smoking",
            "description": ""
          },
          {
            "unit": "USD",
            "usage": {
              "unit": "USD",
              "usage_end": null,
              "usage_start": null
            },
            "value": 250,
            "default": false,
            "tag_value": "bigsmoke",
            "description": ""
          }
        ]
      },
      "cost_type": "Supplementary"
    },
    {
      "metric": {
        "name": "node_cost_per_month",
        "label_metric": "Node",
        "label_measurement": "Currency",
        "label_measurement_unit": "node-month"
      },
      "description": "",
      "tag_rates": {
        "tag_key": "app",
        "tag_values": [
          {
            "unit": "USD",
            "usage": {
              "unit": "USD",
              "usage_end": null,
              "usage_start": null
            },
            "value": 123,
            "default": true,
            "tag_value": "smoke",
            "description": ""
          }
        ]
      },
      "cost_type": "Infrastructure"
    },
    {
      "metric": {
        "name": "node_cost_per_month",
        "label_metric": "Node",
        "label_measurement": "Currency",
        "label_measurement_unit": "node-month"
      },
      "description": "",
      "tag_rates": {
        "tag_key": "web",
        "tag_values": [
          {
            "unit": "USD",
            "usage": {
              "unit": "USD",
              "usage_end": null,
              "usage_start": null
            },
            "value": 456,
            "default": true,
            "tag_value": "smoker",
            "description": ""
          }
        ]
      },
      "cost_type": "Infrastructure"
    },
    {
      "metric": {
        "name": "cluster_cost_per_month",
        "label_metric": "Node",
        "label_measurement": "Currency",
        "label_measurement_unit": "node-month"
      },
      "description": "",
      "tag_rates": {
        "tag_key": "app",
        "tag_values": [
          {
            "unit": "USD",
            "usage": {
              "unit": "USD",
              "usage_end": null,
              "usage_start": null
            },
            "value": 456,
            "default": true,
            "tag_value": "smoker",
            "description": ""
          }
        ]
      },
      "cost_type": "Infrastructure"
    }
  ],
  "distribution": "memory",
  "distribution_info": {
      "distribution_type": "memory",
      "platform_cost": true,
      "worker_cost": true,
      "network_unattributed": false,
      "storage_unattributed": true
  }
}
  1. Check the db:
select * from cost_model;

Example Return:

uuid              | 684987ab-fb3b-4d88-8f8d-a06c29fcc56f
name              | Cost Management OpenShift Cost Model
description       | A cost model of on-premises OpenShift clusters.
source_type       | OCP
created_timestamp | 2024-04-26 20:55:33.460185+00
updated_timestamp | 2024-04-26 20:55:33.460209+00
rates             | [{"metric": {"name": "node_cost_per_month"}, "cost_type": "Supplementary", "tag_rates": {"tag_key": "app", "tag_values": [{"unit": "USD", "usage": {"unit": "USD", "usage_end": null, "usage_start": null}, "value": "24000.0000000000", "default": true, "tag_value": "smoking", "description": ""}, {"unit": "USD", "usage": {"unit": "USD", "usage_end": null, "usage_start": null}, "value": "250.0000000000", "default": false, "tag_value": "bigsmoke", "description": ""}]}, "description": ""}, {"metric": {"name": "node_cost_per_month"}, "cost_type": "Infrastructure", "tag_rates": {"tag_key": "app", "tag_values": [{"unit": "USD", "usage": {"unit": "USD", "usage_end": null, "usage_start": null}, "value": "123.0000000000", "default": true, "tag_value": "smoke", "description": ""}]}, "description": ""}, {"metric": {"name": "node_cost_per_month"}, "cost_type": "Infrastructure", "tag_rates": {"tag_key": "web", "tag_values": [{"unit": "USD", "usage": {"unit": "USD", "usage_end": null, "usage_start": null}, "value": "456.0000000000", "default": true, "tag_value": "smoker", "description": ""}]}, "description": ""}, {"metric": {"name": "cluster_cost_per_month"}, "cost_type": "Infrastructure", "tag_rates": {"tag_key": "app", "tag_values": [{"unit": "USD", "usage": {"unit": "USD", "usage_end": null, "usage_start": null}, "value": "456.0000000000", "default": true, "tag_value": "smoker", "description": ""}]}, "description": ""}]
markup            | {}
distribution      | memory
currency          | USD
distribution_info | {"worker_cost": true, "platform_cost": true, "distribution_type": "memory", "network_unattributed": false, "storage_unattributed": true}

The distribution_info section is what we care about here.

  1. Make sure the platform & worker unallocated cost are still ran:
koku-worker-1  | [2024-04-29 15:55:31,001] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'triggering INSERT', 'tracing_id': '', 'table': 'reporting_ocpusagelineitem_daily_summary'}
koku-worker-1  | [2024-04-29 15:55:31,010] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'finished INSERT', 'tracing_id': '', 'row_count': -1, 'table': 'reporting_ocpusagelineitem_daily_summary', 'running_time': 0.009272575378417969}
koku-worker-1  | [2024-04-29 15:55:31,015] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'distributing platform_cost', 'tracing_id': '', 'start_date': datetime.date(2024, 4, 1), 'end_date': datetime.date(2024, 4, 29), 'schema': 'org1234567', 'report_period_id': 5, 'distribution': 'memory', 'source_uuid': 'e4cc78f5-9ca9-499d-85e6-f3afc9a18b72', 'populate': True}
koku-worker-1  | [2024-04-29 15:55:31,015] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'triggering INSERT', 'tracing_id': '', 'schema': 'org1234567', 'start_date': datetime.date(2024, 4, 1), 'end_date': datetime.date(2024, 4, 29), 'provider_uuid': 'e4cc78f5-9ca9-499d-85e6-f3afc9a18b72', 'table': 'reporting_ocpusagelineitem_daily_summary'}
koku-worker-1  | [2024-04-29 15:55:31,019] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'triggering INSERT', 'tracing_id': '', 'table': 'reporting_ocpusagelineitem_daily_summary'}
koku-worker-1  | [2024-04-29 15:55:31,032] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'finished INSERT', 'tracing_id': '', 'row_count': 207, 'table': 'reporting_ocpusagelineitem_daily_summary', 'running_time': 0.012488603591918945}
koku-worker-1  | [2024-04-29 15:55:31,035] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'distributing worker_cost', 'tracing_id': '', 'start_date': datetime.date(2024, 4, 1), 'end_date': datetime.date(2024, 4, 29), 'schema': 'org1234567', 'report_period_id': 5, 'distribution': 'memory', 'source_uuid': 'e4cc78f5-9ca9-499d-85e6-f3afc9a18b72', 'populate': True}
koku-worker-1  | [2024-04-29 15:55:31,035] INFO f9d2e1d1-2d51-4a64-b2dc-9f04d4c17f15 43 {'message': 'triggering INSERT', 'tracing_id': '', 'schema': 'org1234567', 'start_date': datetime.date(2024, 4, 1), 'end_date': datetime.date(2024, 4, 29), 'provider_uuid': 'e4cc78f5-9ca9-499d-85e6-f3afc9a18b72', 'table': 'reporting_ocpusagelineitem_daily_summary'}

Release Notes

  • proposed release note
* [COST-####](https://issues.redhat.com/browse/COST-4915) Add unattributed distribution to cost model form

@myersCody myersCody added the smoke-tests pr_check will build the image and run minimal required smokes label Apr 26, 2024
@myersCody
Copy link
Contributor Author

/retest

@codecov Codecov
Copy link

codecov bot commented Apr 26, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.1%. Comparing base (aa0379c) to head (fffba63).

Additional details and impacted files 
@@           Coverage Diff           @@
##            main   #5072     +/-   ##
=======================================
- Coverage   94.1%   94.1%   -0.0%     
=======================================
  Files        377     377             
  Lines      31427   31439     +12     
  Branches    3724    3725      +1     
=======================================
+ Hits       29583   29593     +10     
- Misses      1175    1176      +1     
- Partials     669     670      +1

@myersCody myersCody force-pushed the COST-4915-cost-model branch from 4378cdc to 0576694 Compare April 29, 2024 16:33
@myersCody myersCody marked this pull request as ready for review April 29, 2024 19:50
@myersCody myersCody requested review from a team as code owners April 29, 2024 19:50
samdoran
samdoran reviewed Apr 30, 2024
View reviewed changes
koku/api/metrics/constants.py Outdated Show resolved Hide resolved
koku/cost_models/serializers.py Outdated Show resolved Hide resolved
koku/masu/database/ocp_report_db_accessor.py Show resolved Hide resolved
samdoran
samdoran previously approved these changes May 9, 2024
View reviewed changes
@myersCody myersCody enabled auto-merge (squash) May 9, 2024 17:54
@myersCody
Copy link
Contributor Author

/retest

@lcouzens
Copy link
Contributor

/retest

@myersCody myersCody merged commit eb4ca82 into main May 10, 2024
11 checks passed
@myersCody myersCody deleted the COST-4915-cost-model branch May 10, 2024 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lcouzens lcouzens lcouzens approved these changes

@samdoran samdoran samdoran left review comments

Assignees
No one assigned
Labels
smoke-tests pr_check will build the image and run minimal required smokes smokes-required
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@myersCody @lcouzens @samdoran