Skip to content

Commit

Permalink
Runtime config endpoint now supports diff parameter (cortexproject#3700)
Browse files Browse the repository at this point in the history
* Runtime config endpoint now supports diff parameter

The runtime config endpoint now supports the diff parameter meaning that
using /runtime_config?mode=diff will show only the non-default values
for the configuration

Signed-off-by: Mario de Frutos <[email protected]>

* Fix lint errors

Signed-off-by: Mario de Frutos <[email protected]>

* Move runtime handler to the cortext package

To avoid having a dependency of the validation package in the api one we
have move the runtime configuration handle to the cortex package.

This change also led to move some function of the handle to the util
package

Signed-off-by: Mario de Frutos <[email protected]>

* Added kv.Multi diff too

Now we're doing diff with the kv.Multi component for the runtime config

Signed-off-by: Mario de Frutos <[email protected]>

* Refactor and improvement of the runtime config diff

Co-authored-by: Peter Štibraný <[email protected]>
Signed-off-by: Mario de Frutos <[email protected]>

Co-authored-by: Peter Štibraný <[email protected]>
  • Loading branch information
ethervoid and pstibrany authored Jan 27, 2021
1 parent e414714 commit ac54ba9
Show file tree
Hide file tree
Showing 9 changed files with 152 additions and 95 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
* Avoid unnecessary `runtime.GC()` during compactions.
* Prevent compaction loop in TSDB on data gap.
* [ENHANCEMENT] Return server side performance metrics for query-frontend (using Server-timing header). #3685
* [ENHANCEMENT] Runtime Config: Add a `mode` query parameter for the runtime config endpoint. `/runtime_config?mode=diff` now shows the YAML runtime configuration with all values that differ from the defaults. #3700
* [BUGFIX] HA Tracker: don't track as error in the `cortex_kv_request_duration_seconds` metric a CAS operation intentionally aborted. #3745

## 1.7.0 in progress
Expand Down
10 changes: 9 additions & 1 deletion docs/api/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ Displays the configuration currently applied to Cortex (in YAML format), includi
GET /config?mode=diff
```

Displays the configuration currently applied to Cortex (in YAML format) as before, but containing only the values that differ from the default values.
Displays the configuration currently applied to Cortex (in YAML format) as before, but containing only the values that differ from the default values.

```
GET /config?mode=defaults
Expand All @@ -133,6 +133,14 @@ GET /runtime_config

Displays the runtime configuration currently applied to Cortex (in YAML format), including default values. Please be aware that the endpoint will be only available if Cortex is configured with the `-runtime-config.file` option.

#### Different modes

```
GET /runtime_config?mode=diff
```

Displays the runtime configuration currently applied to Cortex (in YAML format) as before, but containing only the values that differ from the default values.

### Services status

```
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/arguments.md
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ multi_kv_config:

When running Cortex on Kubernetes, store this file in a config map and mount it in each services' containers. When changing the values there is no need to restart the services, unless otherwise specified.

The `/runtime_config` endpoint returns the runtime configuration, including the overrides.
The `/runtime_config` endpoint returns the whole runtime configuration, including the overrides. In case you want to get only the non-default values of the configuration you can pass the `mode` parameter with the `diff` value.

## Ingester, Distributor & Querier limits.

Expand Down
6 changes: 3 additions & 3 deletions pkg/api/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ import (
"github.com/cortexproject/cortex/pkg/storegateway"
"github.com/cortexproject/cortex/pkg/storegateway/storegatewaypb"
"github.com/cortexproject/cortex/pkg/util/push"
"github.com/cortexproject/cortex/pkg/util/runtimeconfig"
)

type Config struct {
Expand Down Expand Up @@ -179,10 +178,11 @@ func (a *API) RegisterAPI(httpPathPrefix string, actualCfg interface{}, defaultC
}

// RegisterRuntimeConfig registers the endpoints associates with the runtime configuration
func (a *API) RegisterRuntimeConfig(runtimeCfgManager *runtimeconfig.Manager) {
func (a *API) RegisterRuntimeConfig(runtimeConfigHandler http.HandlerFunc) {
a.indexPage.AddLink(SectionAdminEndpoints, "/runtime_config", "Current Runtime Config (incl. Overrides)")
a.indexPage.AddLink(SectionAdminEndpoints, "/runtime_config?mode=diff", "Current Runtime Config (show only values that differ from the defaults)")

a.RegisterRoute("/runtime_config", runtimeConfigHandler(runtimeCfgManager), false, "GET")
a.RegisterRoute("/runtime_config", runtimeConfigHandler, false, "GET")
}

// RegisterDistributor registers the endpoints associated with the distributor.
Expand Down
92 changes: 3 additions & 89 deletions pkg/api/handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,9 @@ package api

import (
"context"
"fmt"
"html/template"
"net/http"
"path"
"reflect"
"regexp"
"sync"

Expand All @@ -25,14 +23,12 @@ import (
v1 "github.com/prometheus/prometheus/web/api/v1"
"github.com/weaveworks/common/instrument"
"github.com/weaveworks/common/middleware"
"gopkg.in/yaml.v2"

"github.com/cortexproject/cortex/pkg/chunk/purger"
"github.com/cortexproject/cortex/pkg/distributor"
"github.com/cortexproject/cortex/pkg/querier"
"github.com/cortexproject/cortex/pkg/querier/stats"
"github.com/cortexproject/cortex/pkg/util"
"github.com/cortexproject/cortex/pkg/util/runtimeconfig"
)

const (
Expand Down Expand Up @@ -117,95 +113,24 @@ func indexHandler(httpPathPrefix string, content *IndexPageContent) http.Handler
}
}

func yamlMarshalUnmarshal(in interface{}) (map[interface{}]interface{}, error) {
yamlBytes, err := yaml.Marshal(in)
if err != nil {
return nil, err
}

object := make(map[interface{}]interface{})
if err := yaml.Unmarshal(yamlBytes, object); err != nil {
return nil, err
}

return object, nil
}

func diffConfig(defaultConfig, actualConfig map[interface{}]interface{}) (map[interface{}]interface{}, error) {
output := make(map[interface{}]interface{})

for key, value := range actualConfig {

defaultValue, ok := defaultConfig[key]
if !ok {
output[key] = value
continue
}

switch v := value.(type) {
case int:
defaultV, ok := defaultValue.(int)
if !ok || defaultV != v {
output[key] = v
}
case string:
defaultV, ok := defaultValue.(string)
if !ok || defaultV != v {
output[key] = v
}
case bool:
defaultV, ok := defaultValue.(bool)
if !ok || defaultV != v {
output[key] = v
}
case []interface{}:
defaultV, ok := defaultValue.([]interface{})
if !ok || !reflect.DeepEqual(defaultV, v) {
output[key] = v
}
case float64:
defaultV, ok := defaultValue.(float64)
if !ok || !reflect.DeepEqual(defaultV, v) {
output[key] = v
}
case map[interface{}]interface{}:
defaultV, ok := defaultValue.(map[interface{}]interface{})
if !ok {
output[key] = value
}
diff, err := diffConfig(defaultV, v)
if err != nil {
return nil, err
}
if len(diff) > 0 {
output[key] = diff
}
default:
return nil, fmt.Errorf("unsupported type %T", v)
}
}

return output, nil
}

func configHandler(actualCfg interface{}, defaultCfg interface{}) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
var output interface{}
switch r.URL.Query().Get("mode") {
case "diff":
defaultCfgObj, err := yamlMarshalUnmarshal(defaultCfg)
defaultCfgObj, err := util.YAMLMarshalUnmarshal(defaultCfg)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

actualCfgObj, err := yamlMarshalUnmarshal(actualCfg)
actualCfgObj, err := util.YAMLMarshalUnmarshal(actualCfg)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

diff, err := diffConfig(defaultCfgObj, actualCfgObj)
diff, err := util.DiffConfig(defaultCfgObj, actualCfgObj)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
Expand All @@ -222,17 +147,6 @@ func configHandler(actualCfg interface{}, defaultCfg interface{}) http.HandlerFu
}
}

func runtimeConfigHandler(runtimeCfgManager *runtimeconfig.Manager) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
runtimeConfig := runtimeCfgManager.GetConfig()
if runtimeConfig == nil {
util.WriteTextResponse(w, "runtime config file doesn't exist")
return
}
util.WriteYAMLResponse(w, runtimeConfig)
}
}

// NewQuerierHandler returns a HTTP handler that can be used by the querier service to
// either register with the frontend worker query processor or with the external HTTP
// server to fulfill the Prometheus query API.
Expand Down
2 changes: 1 addition & 1 deletion pkg/cortex/modules.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ func (t *Cortex) initRuntimeConfig() (services.Service, error) {

serv, err := runtimeconfig.NewRuntimeConfigManager(t.Cfg.RuntimeConfig, prometheus.DefaultRegisterer)
t.RuntimeConfig = serv
t.API.RegisterRuntimeConfig(t.RuntimeConfig)
t.API.RegisterRuntimeConfig(runtimeConfigHandler(t.RuntimeConfig, t.Cfg.LimitsConfig))
return serv, err
}

Expand Down
47 changes: 47 additions & 0 deletions pkg/cortex/runtime_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@ package cortex
import (
"errors"
"io"
"net/http"

"gopkg.in/yaml.v2"

"github.com/cortexproject/cortex/pkg/ring/kv"
"github.com/cortexproject/cortex/pkg/util"
"github.com/cortexproject/cortex/pkg/util/runtimeconfig"
"github.com/cortexproject/cortex/pkg/util/validation"
)
Expand Down Expand Up @@ -83,3 +85,48 @@ func multiClientRuntimeConfigChannel(manager *runtimeconfig.Manager) func() <-ch
return outCh
}
}
func runtimeConfigHandler(runtimeCfgManager *runtimeconfig.Manager, defaultLimits validation.Limits) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
cfg, ok := runtimeCfgManager.GetConfig().(*runtimeConfigValues)
if !ok || cfg == nil {
util.WriteTextResponse(w, "runtime config file doesn't exist")
return
}

var output interface{}
switch r.URL.Query().Get("mode") {
case "diff":
// Default runtime config is just empty struct, but to make diff work,
// we set defaultLimits for every tenant that exists in runtime config.
defaultCfg := runtimeConfigValues{}
defaultCfg.TenantLimits = map[string]*validation.Limits{}
for k, v := range cfg.TenantLimits {
if v != nil {
defaultCfg.TenantLimits[k] = &defaultLimits
}
}

cfgYaml, err := util.YAMLMarshalUnmarshal(cfg)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

defaultCfgYaml, err := util.YAMLMarshalUnmarshal(defaultCfg)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

output, err = util.DiffConfig(defaultCfgYaml, cfgYaml)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

default:
output = cfg
}
util.WriteYAMLResponse(w, output)
}
}
68 changes: 68 additions & 0 deletions pkg/util/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package util

import (
"fmt"
"reflect"
)

// DiffConfig utility function that returns the diff between two config map objects
func DiffConfig(defaultConfig, actualConfig map[interface{}]interface{}) (map[interface{}]interface{}, error) {
output := make(map[interface{}]interface{})

for key, value := range actualConfig {

defaultValue, ok := defaultConfig[key]
if !ok {
output[key] = value
continue
}

switch v := value.(type) {
case int:
defaultV, ok := defaultValue.(int)
if !ok || defaultV != v {
output[key] = v
}
case string:
defaultV, ok := defaultValue.(string)
if !ok || defaultV != v {
output[key] = v
}
case bool:
defaultV, ok := defaultValue.(bool)
if !ok || defaultV != v {
output[key] = v
}
case []interface{}:
defaultV, ok := defaultValue.([]interface{})
if !ok || !reflect.DeepEqual(defaultV, v) {
output[key] = v
}
case float64:
defaultV, ok := defaultValue.(float64)
if !ok || !reflect.DeepEqual(defaultV, v) {
output[key] = v
}
case nil:
if defaultValue != nil {
output[key] = v
}
case map[interface{}]interface{}:
defaultV, ok := defaultValue.(map[interface{}]interface{})
if !ok {
output[key] = value
}
diff, err := DiffConfig(defaultV, v)
if err != nil {
return nil, err
}
if len(diff) > 0 {
output[key] = diff
}
default:
return nil, fmt.Errorf("unsupported type %T", v)
}
}

return output, nil
}
19 changes: 19 additions & 0 deletions pkg/util/yaml.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
package util

import "gopkg.in/yaml.v2"

// YAMLMarshalUnmarshal utility function that converts a YAML interface in a map
// doing marshal and unmarshal of the parameter
func YAMLMarshalUnmarshal(in interface{}) (map[interface{}]interface{}, error) {
yamlBytes, err := yaml.Marshal(in)
if err != nil {
return nil, err
}

object := make(map[interface{}]interface{})
if err := yaml.Unmarshal(yamlBytes, object); err != nil {
return nil, err
}

return object, nil
}

0 comments on commit ac54ba9

Please sign in to comment.