Grafana dashboard for Iter8 experiments

[Alan-Cha]

• Design Grafana dashboard for Iter8 experiments #1537
• Implement Grafana dashboard for Iter8 experiments #1538
  • Design and implement http performance test dashboard for Grafana #1547
  • Design and implement grpc performance test dashboard for Grafana #1548
• Implement metrics server changes for Grafana dashboard for Iter8 experiments #1539
  • Metrics service API to retrieve http/grpc data for Grafana #1549
  • Storage client design to store http/grpc data #1550
  • Update http task to write fortio results to storage client #1551
  • Update grpc task to write ghz results to storage client #1552
• Delete cronjob #1553
• Remove assess task #1554
• Remove html/text reporters #1555
• Remove iter8 k report command #1556
• Remove iter8 k assess command #1557
• Remove autox #1558
• Write documentation for Grafana dashboard for Iter8 experiments #1541
• Outdated experiment result in Grafana #1580

Additional (optional) related issues

• Select app in Grafana dashboard #1524
• Number of versions in histogram #1526
• Number of buckets in histogram #1525
• Missing tests in workflows #1566
  • github and slack tasks #1568
• grpc tutorial example failing #1567
• Removal of old tutorials using custommetrics #1569
• Removal of old tutorials #1570

[kalantar]

Initial list of more specific items that need to be addressed:

• Design http performance test dashboard for Grafana
• Design grpc performance test dashboard for Grafana
• Metrics service API to retrieve http/grpc data for Grafana
• Storage client design to store http/grpc data
  • namespace-group::helm-revision::loop :: endpoint —> fortio (http)
  • namespace-group::helm-revision::loop :: endpoint —> ghz (grpc)
  • namespace-name::endpoint
• Update http task to write fortio results to storage client
  • Question: via metrics service? (ABn writes directly to storage client)
  • Question: implement via Driver implementation for this?
  • Question: modify ExperimentSpec.Insights or just write directly (delete Insights). MODIFY to have fortio/ghz sections (delete other sections)
• Update grpc task to write ghz results to storage client
  • Questions: same as for http task
• Other information in ExperimentResults (also store in metrics store)? YES -- all result
• Question: Move experiment spec to storage client ? NO
• Or have spec in Kubernetes (secret) and result in storage client. YES. Mount secret in job. Read it (file driver).
• delete cronjob
• Remove assess task
• Remove html/text reporters
• Remove iter8 k report command
• Remove iter8 k assess command
• Keeping: tasks -- http, grpc, readiness, notify; remove custommetrics, assess, abnmetrics
• keeping: cmd -- launch, run delete, controllers; remove report, assess, abn, log
• even more radical, remove lauch, delete (use helm). keep run, controllers; iter8 becomes "internal" only
• documentation changes
  • remove custommetrics, looped experiments, cronjob, slos, rewards, install, autox etc.
• consider renaming group to name
• remove autox

Implementation Steps:

• write user story / tutorial behavior
• dashboards
• storage design
• metrics http PUT
• metrics http fortio GET
• metrics grpc ghz GET
• metric serivce driver implementation
• experiment spec/results changes
• remove unused pieces

[kalantar]

User Story

1. User runs experiment namespace/name
   • helm install (or helm update) creates (updates) experiment spec
   • experiment spec (secret) is mounted to a job that runs iter8 k run
   • job reads spec using file driver from mounted secret
   • tasks run fortio/ghz and results are written to experiment result (Insight)
   • job PUTs experiment result to metrics service
   • metrics serice writes it to the storage client
2. (assume user has Grafana) User opens appropriate Grafana dashboard (http/grpc) specifying experiment name
   • if necessary user will need to define data source(s)
   • if necessary user will need to import dashboard definition
3. User inspects experiment result

[kalantar]

@Alan-Cha I think the last three items you've listed as "optional" are really required. We need to decide on strategy and implement.

[kalantar]

released in v0.16