diff --git a/README.md b/README.md index 45fa0c6..2d7d930 100644 --- a/README.md +++ b/README.md @@ -157,19 +157,19 @@ matches some expectation: * `kube.with.labels`: (optional) `map[string]string` containing the label keys and values to use in constructing an equality label selector (for all listed labels) -* `kube.assert`: (optional) object containing assertions to make about the +* `assert`: (optional) object containing assertions to make about the action performed by the test. -* `kube.assert.error`: (optional) string to match a returned error from the +* `assert.error`: (optional) string to match a returned error from the Kubernetes API server. -* `kube.assert.len`: (optional) int with the expected number of items returned. -* `kube.assert.notfound`: (optional) bool indicating the test author expects +* `assert.len`: (optional) int with the expected number of items returned. +* `assert.notfound`: (optional) bool indicating the test author expects the Kubernetes API to return a 404/Not Found for a resource. -* `kube.assert.unknown`: (optional) bool indicating the test author expects the +* `assert.unknown`: (optional) bool indicating the test author expects the Kubernetes API server to respond that it does not know the type of resource attempting to be fetched or created. -* `kube.assert.matches`: (optional) a YAML string, a filepath, or a +* `assert.matches`: (optional) a YAML string, a filepath, or a `map[string]interface{}` representing the content that you expect to find in - the returned result from the `kube.get` call. If `kube.assert.matches` is a + the returned result from the `kube.get` call. If `assert.matches` is a string, the string can be either a file path to a YAML manifest or inline an YAML string containing the resource fields to compare. Only fields present in the Matches resource are compared. There is a @@ -177,7 +177,7 @@ matches some expectation: the value of the fields match. Only scalar fields are matched entirely. In other words, you do not need to specify every field of a struct field in order to compare the value of a single field in the nested struct. -* `kube.assert.conditions`: (optional) a map, keyed by `ConditionType` string, +* `assert.conditions`: (optional) a map, keyed by `ConditionType` string, of any of the following: - a string containing the `Status` value that the `Condition` with the `ConditionType` should have. @@ -189,18 +189,18 @@ matches some expectation: `ConditionType` should have * `reason` which is the exact string that should be present in the `Condition` with the `ConditionType` -* `kube.assert.json`: (optional) object describing the assertions to make about +* `assert.json`: (optional) object describing the assertions to make about resource(s) returned from the `kube.get` call to the Kubernetes API server. -* `kube.assert.json.len`: (optional) integer representing the number of bytes in the +* `assert.json.len`: (optional) integer representing the number of bytes in the resulting JSON object after successfully parsing the resource. -* `kube.assert.json.paths`: (optional) map of strings where the keys of the map +* `assert.json.paths`: (optional) map of strings where the keys of the map are JSONPath expressions and the values of the map are the expected value to be found when evaluating the JSONPath expression -* `kube.assert.json.path_formats`: (optional) map of strings where the keys of the map are +* `assert.json.path_formats`: (optional) map of strings where the keys of the map are JSONPath expressions and the values of the map are the expected format of the value to be found when evaluating the JSONPath expression. See the [list of valid format strings](#valid-format-strings) -* `kube.assert.json.schema`: (optional) string containing a filepath to a +* `assert.json.schema`: (optional) string containing a filepath to a JSONSchema document. If present, the resource's structure will be validated against this JSONSChema document. @@ -227,8 +227,8 @@ name: test-nginx-pod-not-exist tests: - kube: get: pods/nginx - assert: - notfound: true + assert: + notfound: true ``` Testing that there are two Pods having the label `app:nginx`: @@ -242,8 +242,8 @@ tests: with: labels: app: nginx - assert: - len: 2 + assert: + len: 2 ``` Testing that a Pod with the name `nginx` exists by the specified timeout @@ -323,9 +323,9 @@ tests: - exec: ssh -T someuser@ip ``` -### Asserting resource fields using `kube.assert.matches` +### Asserting resource fields using `assert.matches` -The `kube.assert.matches` field of a `gdt-kube` test Spec allows a test author +The `assert.matches` field of a `gdt-kube` test Spec allows a test author to specify expected fields and those field contents in a resource that was returned by the Kubernetes API server from the result of a `kube.get` call. @@ -342,16 +342,16 @@ tests: - name: check deployment's ready replicas is 2 kube: get: deployments/my-deployment - assert: - matches: | - kind: Deployment - metadata: - name: my-deployment - status: - readyReplicas: 2 + assert: + matches: | + kind: Deployment + metadata: + name: my-deployment + status: + readyReplicas: 2 ``` -you don't even need to include the kind and metadata in `kube.assert.matches`. +you don't even need to include the kind and metadata in `assert.matches`. If missing, no kind and name matching will be performed. ```yaml @@ -359,10 +359,10 @@ tests: - name: check deployment's ready replicas is 2 kube: get: deployments/my-deployment - assert: - matches: | - status: - readyReplicas: 2 + assert: + matches: | + status: + readyReplicas: 2 ``` In fact, you don't need to use an inline multiline YAML string. You can @@ -373,15 +373,15 @@ tests: - name: check deployment's ready replicas is 2 kube: get: deployments/my-deployment - assert: - matches: - status: - readyReplicas: 2 + assert: + matches: + status: + readyReplicas: 2 ``` -### Asserting resource `Conditions` using `kube.assert.conditions` +### Asserting resource `Conditions` using `assert.conditions` -`kube.assertion.conditions` contains the assertions to make about a resource's +`assertion.conditions` contains the assertions to make about a resource's `Status.Conditions` collection. It is a map, keyed by the ConditionType (matched case-insensitively), of assertions to make about that Condition. The assertions can be: @@ -402,9 +402,9 @@ use lowercase strings: tests: - kube: get: pods/nginx - assert: - conditions: - ready: true + assert: + conditions: + ready: true ``` If we wanted to assert that the `ContainersReady` Condition had a status @@ -414,11 +414,11 @@ of either `False` or `Unknown`, we could write the test like this: tests: - kube: get: pods/nginx - assert: - conditions: - containersReady: - - false - - unknown + assert: + conditions: + containersReady: + - false + - unknown ``` Finally, if we wanted to assert that a Deployment's `Progressing` @@ -429,16 +429,16 @@ Condition had a Reason field with a value "NewReplicaSetAvailable" tests: - kube: get: deployments/nginx - assert: - conditions: - progressing: - status: true - reason: NewReplicaSetAvailable + assert: + conditions: + progressing: + status: true + reason: NewReplicaSetAvailable ``` -### Asserting resource fields using `kube.assert.json` +### Asserting resource fields using `assert.json` -The `kube.assert.json` field of a `gdt-kube` test Spec allows a test author to +The `assert.json` field of a `gdt-kube` test Spec allows a test author to specify expected fields, the value of those fields as well as the format of field values in a resource that was returned by the Kubernetes API server from the result of a `kube.get` call. @@ -446,7 +446,7 @@ the result of a `kube.get` call. Suppose you have a Deployment resource and you want to write a test that checks that a Deployment resource's `Status.ReadyReplicas` field is `2`. -You can specify this expectation using the `kube.assert.json.paths` field, +You can specify this expectation using the `assert.json.paths` field, which is a `map[string]interface{}` that takes map keys that are JSONPath expressions and map values of what the field at that JSONPath expression should contain: @@ -456,10 +456,10 @@ tests: - name: check deployment's ready replicas is 2 kube: get: deployments/my-deployment - assert: - json: - paths: - $.status.readyReplicas: 2 + assert: + json: + paths: + $.status.readyReplicas: 2 ``` JSONPath expressions can be fairly complex, allowing the test author to, for @@ -471,14 +471,14 @@ tests: - name: check deployment's pod template "app" label is "nginx" kube: get: deployments/my-deployment - assert: - json: - paths: - $.spec.template.labels["app"]: nginx + assert: + json: + paths: + $.spec.template.labels["app"]: nginx ``` You can check that the value of a particular field at a JSONPath is formatted -in a particular fashion using `kube.assert.json.path_formats`. This is a map, +in a particular fashion using `assert.json.path_formats`. This is a map, keyed by JSONPath expression, of the data format the value of the field at that JSONPath expression should have. Valid data formats are: @@ -511,11 +511,11 @@ date-time timestamp: tests: - kube: get: deployments/nginx - assert: - json: - path_formats: - $.metadata.uid: uuid4 - $.metadata.creationTimestamp: date-time + assert: + json: + path_formats: + $.metadata.uid: uuid4 + $.metadata.creationTimestamp: date-time ``` ### Updating a resource and asserting corresponding field changes @@ -564,10 +564,10 @@ tests: after: 20s kube: get: deployments/nginx - assert: - matches: - status: - readyReplicas: 2 + assert: + matches: + status: + readyReplicas: 2 - name: apply-deployment-change kube: apply: | @@ -582,10 +582,10 @@ tests: after: 20s kube: get: deployments/nginx - assert: - matches: - status: - readyReplicas: 1 + assert: + matches: + status: + readyReplicas: 1 - name: delete-deployment kube: delete: deployments/nginx @@ -619,16 +619,16 @@ tests: - name: deployment-exists kube: get: deployments/nginx - assert: - matches: - spec: - replicas: 2 - template: - metadata: - labels: - app: nginx - status: - readyReplicas: 2 + assert: + matches: + spec: + replicas: 2 + template: + metadata: + labels: + app: nginx + status: + readyReplicas: 2 - name: delete-deployment kube: delete: deployments/nginx @@ -685,13 +685,13 @@ ok command-line-arguments 3.683s You can see from the debug output above that `gdt` created the Deployment and then did a `kube.get` for the `deployments/nginx` Deployment. Initially -(attempt 1), the `kube.assert.matches` assertion failed because the +(attempt 1), the `assert.matches` assertion failed because the `status.readyReplicas` field was not present in the returned resource. `gdt` retried the `kube.get` call 4 more times (attempts 2-5), with attempts 2 and 3 failed the existence check for the `status.readyReplicas` field and attempt 4 failing the *value* check for the `status.readyReplicas` field being `1` instead of the expected `2`. Finally, when the Deployment was completely rolled -out, attempt 5 succeeded in all the `kube.assert.matches` assertions. +out, attempt 5 succeeded in all the `assert.matches` assertions. ## Determining Kubernetes config, context and namespace values diff --git a/eval.go b/eval.go index 738c136..48f58d7 100644 --- a/eval.go +++ b/eval.go @@ -77,7 +77,7 @@ func (s *Spec) get( Kind: kind, } res, err := c.gvrFromGVK(gvk) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) if !a.OK() { return result.New(result.WithFailures(a.Failures()...)) } @@ -145,7 +145,7 @@ func (s *Spec) doList( list, err := c.client.Resource(res).Namespace(namespace).List( ctx, opts, ) - return newAssertions(s.Kube.Assert, err, list) + return newAssertions(s.Assert, err, list) } // doGet performs the Get() call and assertion check for a supplied resource @@ -163,7 +163,7 @@ func (s *Spec) doGet( name, metav1.GetOptions{}, ) - return newAssertions(s.Kube.Assert, err, obj) + return newAssertions(s.Assert, err, obj) } // splitKindName returns the Kind for a supplied `Get` or `Delete` command @@ -212,7 +212,7 @@ func (s *Spec) create( ns = s.Namespace() } res, err := c.gvrFromGVK(gvk) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) if !a.OK() { return result.New(result.WithFailures(a.Failures()...)) } @@ -225,7 +225,7 @@ func (s *Spec) create( // object that was created, which is wrong. When I add the polymorphism // to the Assertions struct, I will modify this block to look for an // indexed set of error assertions. - a = newAssertions(s.Kube.Assert, err, obj) + a = newAssertions(s.Assert, err, obj) return result.New(result.WithFailures(a.Failures()...)) } return nil @@ -269,7 +269,7 @@ func (s *Spec) apply( ns = s.Namespace() } res, err := c.gvrFromGVK(gvk) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) if !a.OK() { return result.New(result.WithFailures(a.Failures()...)) } @@ -289,7 +289,7 @@ func (s *Spec) apply( // object that was applied, which is wrong. When I add the polymorphism // to the Assertions struct, I will modify this block to look for an // indexed set of error assertions. - a = newAssertions(s.Kube.Assert, err, obj) + a = newAssertions(s.Assert, err, obj) return result.New(result.WithFailures(a.Failures()...)) } return nil @@ -353,7 +353,7 @@ func (s *Spec) delete( for _, obj := range objs { gvk := obj.GetObjectKind().GroupVersionKind() res, err := c.gvrFromGVK(gvk) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) if !a.OK() { return result.New(result.WithFailures(a.Failures()...)) } @@ -379,7 +379,7 @@ func (s *Spec) delete( Kind: kind, } res, err := c.gvrFromGVK(gvk) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) if !a.OK() { return result.New(result.WithFailures(a.Failures()...)) } @@ -404,7 +404,7 @@ func (s *Spec) doDelete( name, metav1.DeleteOptions{}, ) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) return result.New(result.WithFailures(a.Failures()...)) } @@ -422,6 +422,6 @@ func (s *Spec) doDeleteCollection( metav1.DeleteOptions{}, metav1.ListOptions{}, ) - a := newAssertions(s.Kube.Assert, err, nil) + a := newAssertions(s.Assert, err, nil) return result.New(result.WithFailures(a.Failures()...)) } diff --git a/parse.go b/parse.go index d8150f5..6bf8cce 100644 --- a/parse.go +++ b/parse.go @@ -58,6 +58,15 @@ func (s *Spec) UnmarshalYAML(node *yaml.Node) error { return errors.ExpectedScalarAt(valNode) } s.KubeDelete = valNode.Value + case "assert": + if valNode.Kind != yaml.MappingNode { + return errors.ExpectedMapAt(valNode) + } + var e *Expect + if err := valNode.Decode(&e); err != nil { + return err + } + s.Assert = e default: if lo.Contains(gdttypes.BaseSpecFields, key) { continue @@ -189,8 +198,8 @@ func validateKubeSpec(s *Spec) error { } } } - if s.Kube.Assert != nil { - exp := s.Kube.Assert + if s.Assert != nil { + exp := s.Assert if exp.Matches != nil { if err := validateMatches(exp.Matches); err != nil { return err diff --git a/parse_test.go b/parse_test.go index 885af07..b47b53e 100644 --- a/parse_test.go +++ b/parse_test.go @@ -240,6 +240,7 @@ spec: - name: nginx image: nginx:1.7.9 ` + var zero int expTests := []gdttypes.Evaluable{ &gdtkube.Spec{ @@ -314,6 +315,9 @@ spec: Kube: &gdtkube.KubeSpec{ Get: "pods/foo", }, + Assert: &gdtkube.Expect{ + Len: &zero, + }, }, } assert.Equal(expTests, s.Tests) diff --git a/spec.go b/spec.go index b6fdda6..cd8512b 100644 --- a/spec.go +++ b/spec.go @@ -85,11 +85,6 @@ type KubeSpec struct { // app: nginx // ``` With *With `yaml:"with,omitempty"` - // Assert houses the various assertions to be made about the kube client - // call (Create, Apply, Get, etc) - // TODO(jaypipes): Make this polymorphic to be either a single assertion - // struct or a list of assertion structs - Assert *Expect `yaml:"assert,omitempty"` } // Spec describes a test of a *single* Kubernetes API request and response. @@ -133,6 +128,11 @@ type Spec struct { // having such a label. // * the string `--all` to delete all resources of that kind. KubeDelete string `yaml:"kube.delete,omitempty"` + // Assert houses the various assertions to be made about the kube client + // call (Create, Apply, Get, etc) + // TODO(jaypipes): Make this polymorphic to be either a single assertion + // struct or a list of assertion structs + Assert *Expect `yaml:"assert,omitempty"` } // Title returns a good name for the Spec diff --git a/testdata/apply-deployment.yaml b/testdata/apply-deployment.yaml index 00eada2..b449830 100644 --- a/testdata/apply-deployment.yaml +++ b/testdata/apply-deployment.yaml @@ -11,10 +11,10 @@ tests: after: 20s kube: get: deployments/nginx - assert: - matches: - status: - readyReplicas: 2 + assert: + matches: + status: + readyReplicas: 2 - name: apply-deployment-change kube: apply: | @@ -29,10 +29,10 @@ tests: after: 20s kube: get: deployments/nginx - assert: - matches: - status: - readyReplicas: 1 + assert: + matches: + status: + readyReplicas: 1 - name: delete-deployment kube: delete: deployments/nginx diff --git a/testdata/conditions.yaml b/testdata/conditions.yaml index 5805611..0f620c8 100644 --- a/testdata/conditions.yaml +++ b/testdata/conditions.yaml @@ -9,28 +9,27 @@ tests: - name: deployment-immediately-has-false-or-unknown kube: get: deployments/nginx - assert: - conditions: - available: [false, unknown] + assert: + conditions: + available: [false, unknown] - name: deployment-has-true-progressing-condition timeout: after: 2s kube: get: deployments/nginx - assert: - conditions: - progressing: true + assert: + conditions: + progressing: true - name: deployment-last-progressing-reason timeout: after: 20s kube: get: deployments/nginx - assert: - conditions: - progressing: - status: true - reason: NewReplicaSetAvailable + assert: + conditions: + progressing: + status: true + reason: NewReplicaSetAvailable - name: delete-deployment kube: delete: deployments/nginx - diff --git a/testdata/create-get-delete-pod.yaml b/testdata/create-get-delete-pod.yaml index eed4fca..b375ed1 100644 --- a/testdata/create-get-delete-pod.yaml +++ b/testdata/create-get-delete-pod.yaml @@ -15,5 +15,5 @@ tests: - name: pod-no-longer-exists kube: get: pods/nginx - assert: - notfound: true + assert: + notfound: true diff --git a/testdata/create-unknown-resource.yaml b/testdata/create-unknown-resource.yaml index 5cb67f1..e9eb234 100644 --- a/testdata/create-unknown-resource.yaml +++ b/testdata/create-unknown-resource.yaml @@ -9,5 +9,5 @@ tests: kind: unknown metadata: name: unknown - assert: - unknown: true + assert: + unknown: true diff --git a/testdata/delete-resource-not-found.yaml b/testdata/delete-resource-not-found.yaml index e6edf91..8f0500f 100644 --- a/testdata/delete-resource-not-found.yaml +++ b/testdata/delete-resource-not-found.yaml @@ -5,5 +5,5 @@ fixtures: tests: - kube: delete: pods/doesnotexist - assert: - notfound: true + assert: + notfound: true diff --git a/testdata/delete-unknown-resource.yaml b/testdata/delete-unknown-resource.yaml index 977db67..7d6fa16 100644 --- a/testdata/delete-unknown-resource.yaml +++ b/testdata/delete-unknown-resource.yaml @@ -5,5 +5,5 @@ fixtures: tests: - kube: delete: unknown/unknown - assert: - unknown: true + assert: + unknown: true diff --git a/testdata/envvar-substitution.yaml b/testdata/envvar-substitution.yaml index 1f68b74..d119c0b 100644 --- a/testdata/envvar-substitution.yaml +++ b/testdata/envvar-substitution.yaml @@ -15,5 +15,5 @@ tests: - name: ${pod_name}-no-longer-exists kube: get: pods/${pod_name} - assert: - notfound: true + assert: + notfound: true diff --git a/testdata/get-pod-not-found.yaml b/testdata/get-pod-not-found.yaml index 4a36a55..e9883b0 100644 --- a/testdata/get-pod-not-found.yaml +++ b/testdata/get-pod-not-found.yaml @@ -6,10 +6,10 @@ tests: - name: assert-len-zero kube: get: pods/doesnotexist - assert: - len: 0 + assert: + len: 0 - name: assert-not-found kube: get: pods/doesnotexist - assert: - notfound: true + assert: + notfound: true diff --git a/testdata/json.yaml b/testdata/json.yaml index 3f9fdd0..505f938 100644 --- a/testdata/json.yaml +++ b/testdata/json.yaml @@ -11,15 +11,15 @@ tests: after: 20s kube: get: deployments/nginx - assert: - json: - paths: - $.spec.replicas: 2 - $.spec.template.metadata.labels["app"]: nginx - $.status.readyReplicas: 2 - path_formats: - $.metadata.uid: uuid4 - $.metadata.creationTimestamp: date-time + assert: + json: + paths: + $.spec.replicas: 2 + $.spec.template.metadata.labels["app"]: nginx + $.status.readyReplicas: 2 + path_formats: + $.metadata.uid: uuid4 + $.metadata.creationTimestamp: date-time - name: delete-deployment kube: delete: deployments/nginx diff --git a/testdata/list-pods-empty.yaml b/testdata/list-pods-empty.yaml index bc5dce4..094f04e 100644 --- a/testdata/list-pods-empty.yaml +++ b/testdata/list-pods-empty.yaml @@ -6,5 +6,5 @@ tests: - name: verify-no-pods kube: get: pods - assert: - len: 0 + assert: + len: 0 diff --git a/testdata/list-pods-with-labels.yaml b/testdata/list-pods-with-labels.yaml index 517bba5..45cd872 100644 --- a/testdata/list-pods-with-labels.yaml +++ b/testdata/list-pods-with-labels.yaml @@ -12,16 +12,16 @@ tests: with: labels: app: nginx - assert: - len: 2 + assert: + len: 2 - name: verify-no-pods-with-app-noexist-label kube: get: pods with: labels: app: noexist - assert: - len: 0 + assert: + len: 0 - name: delete-deployment kube: delete: deployments/nginx diff --git a/testdata/matches.yaml b/testdata/matches.yaml index ddbd3b0..1be40d7 100644 --- a/testdata/matches.yaml +++ b/testdata/matches.yaml @@ -11,16 +11,16 @@ tests: after: 20s kube: get: deployments/nginx - assert: - matches: - spec: - replicas: 2 - template: - metadata: - labels: - app: nginx - status: - readyReplicas: 2 + assert: + matches: + spec: + replicas: 2 + template: + metadata: + labels: + app: nginx + status: + readyReplicas: 2 - name: delete-deployment kube: delete: deployments/nginx diff --git a/testdata/parse.yaml b/testdata/parse.yaml index fe7bc18..9db0037 100644 --- a/testdata/parse.yaml +++ b/testdata/parse.yaml @@ -35,3 +35,5 @@ tests: - name: fetch a pod with envvar substitution kube: get: pods/${pod_name} + assert: + len: 0 diff --git a/testdata/parse/fail/bad-matches-file-not-found.yaml b/testdata/parse/fail/bad-matches-file-not-found.yaml index 4cbdc1c..ec81cff 100644 --- a/testdata/parse/fail/bad-matches-file-not-found.yaml +++ b/testdata/parse/fail/bad-matches-file-not-found.yaml @@ -3,5 +3,5 @@ description: matches refers to unknown file tests: - kube: get: pods/mypod - assert: - matches: does/not/exist.yaml + assert: + matches: does/not/exist.yaml diff --git a/testdata/parse/fail/bad-matches-invalid-yaml.yaml b/testdata/parse/fail/bad-matches-invalid-yaml.yaml index fbbb537..cddf216 100644 --- a/testdata/parse/fail/bad-matches-invalid-yaml.yaml +++ b/testdata/parse/fail/bad-matches-invalid-yaml.yaml @@ -3,5 +3,5 @@ description: matches contains invalid YAML tests: - kube: get: pods/mypod - assert: - matches: :this-is-not-valid!YAML + assert: + matches: :this-is-not-valid!YAML diff --git a/testdata/parse/fail/bad-matches-not-map-any.yaml b/testdata/parse/fail/bad-matches-not-map-any.yaml index 96668c0..7a99c01 100644 --- a/testdata/parse/fail/bad-matches-not-map-any.yaml +++ b/testdata/parse/fail/bad-matches-not-map-any.yaml @@ -3,5 +3,5 @@ description: "matches is not a string or map[string]interface{}" tests: - kube: get: pods/mypod - assert: - matches: 42 + assert: + matches: 42