Skip to content

Commit

Permalink
feat(kafka_quota): added support for kafka quota
Browse files Browse the repository at this point in the history
  • Loading branch information
vmyroslav committed Dec 18, 2024
1 parent 9d58d73 commit 8ee0a6a
Show file tree
Hide file tree
Showing 12 changed files with 1,159 additions and 1 deletion.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ __debug_bin
# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Go workspace file
go.work
go.work.sum

# Dependency directories
vendor/
packrd/
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ nav_order: 1
<!-- Always keep the following header in place: -->
<!--## [MAJOR.MINOR.PATCH] - YYYY-MM-DD -->


## [MAJOR.MINOR.PATCH] - YYYY-MM-DD

- Add `aiven_kafka_quota` resource

## [4.31.0] - 2024-12-18

- Add `alloydbomni` BETA resource and datasource
Expand Down
79 changes: 79 additions & 0 deletions docs/resources/kafka_quota.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "aiven_kafka_quota Resource - terraform-provider-aiven"
subcategory: ""
description: |-
Creates and manages quotas for an Aiven for Apache Kafka® service user.
---

# aiven_kafka_quota (Resource)

Creates and manages quotas for an Aiven for Apache Kafka® service user.

## Example Usage

```terraform
resource "aiven_kafka_quota" "example_quota" {
project = data.aiven_project.foo.project
service_name = aiven_kafka.example_kafka.service_name
user = "example-kafka-user"
client_id = "example_client"
consumer_byte_rate = 1000
producer_byte_rate = 1000
request_percentage = 50
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `project` (String) The name of the project this resource belongs to. To set up proper dependencies please refer to this variable as a reference. Changing this property forces recreation of the resource.
- `service_name` (String) The name of the service that this resource belongs to. To set up proper dependencies please refer to this variable as a reference. Changing this property forces recreation of the resource.

### Optional

- `client_id` (String) Represents a logical group of clients, assigned a unique name by the client application.
Quotas can be applied based on user, client-id, or both.
The most relevant quota is chosen for each connection.
All connections within a quota group share the same quota.
It is possible to set default quotas for each (user, client-id), user or client-id group by specifying 'default'
- `consumer_byte_rate` (Number) Defines the bandwidth limit in bytes/sec for each group of clients sharing a quota.
Every distinct client group is allocated a specific quota, as defined by the cluster, on a per-broker basis.
Exceeding this limit results in client throttling.
- `producer_byte_rate` (Number) Defines the bandwidth limit in bytes/sec for each group of clients sharing a quota.
Every distinct client group is allocated a specific quota, as defined by the cluster, on a per-broker basis.
Exceeding this limit results in client throttling.
- `request_percentage` (Number) Sets the maximum percentage of CPU time that a client group can use on request handler I/O and network threads per broker within a quota window.
Exceeding this limit triggers throttling.
The quota, expressed as a percentage, also indicates the total allowable CPU usage for the client groups sharing the quota.
- `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
- `user` (String) Represents a logical group of clients, assigned a unique name by the client application.
Quotas can be applied based on user, client-id, or both.
The most relevant quota is chosen for each connection.
All connections within a quota group share the same quota.
It is possible to set default quotas for each (user, client-id), user or client-id group by specifying 'default'

### Read-Only

- `id` (String) The ID of this resource.

<a id="nestedblock--timeouts"></a>
### Nested Schema for `timeouts`

Optional:

- `create` (String)
- `default` (String)
- `delete` (String)
- `read` (String)
- `update` (String)

## Import

Import is supported using the following syntax:

```shell
terraform import aiven_kafka_quota.example_quota PROJECT/SERVICE_NAME
```
1 change: 1 addition & 0 deletions examples/resources/aiven_kafka_quota/import.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import aiven_kafka_quota.example_quota PROJECT/SERVICE_NAME
9 changes: 9 additions & 0 deletions examples/resources/aiven_kafka_quota/resource.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
resource "aiven_kafka_quota" "example_quota" {
project = data.aiven_project.foo.project
service_name = aiven_kafka.example_kafka.service_name
user = "example-kafka-user"
client_id = "example_client"
consumer_byte_rate = 1000
producer_byte_rate = 1000
request_percentage = 50
}
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.23

require (
github.com/aiven/aiven-go-client/v2 v2.33.0
github.com/aiven/go-client-codegen v0.71.0
github.com/aiven/go-client-codegen v0.73.0
github.com/avast/retry-go v3.0.0+incompatible
github.com/dave/jennifer v1.7.1
github.com/docker/go-units v0.5.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ github.com/aiven/go-api-schemas v1.106.0 h1:qncRsbiaGnU9JE9fmTFHclTCBem+t+6EPMXG
github.com/aiven/go-api-schemas v1.106.0/go.mod h1:z7dGvufm6If4gOdVr7dWTuFZmll9FOZr5Z5CSxGpebA=
github.com/aiven/go-client-codegen v0.71.0 h1:SGiHrfbU8RiqVegQGV3BStnbIdFke+15lxadlPORqfI=
github.com/aiven/go-client-codegen v0.71.0/go.mod h1:QKN/GgLMGWd6+gPEucXlZPi5vC3C6RpD3UeBRQOLI1Y=
github.com/aiven/go-client-codegen v0.73.0 h1:1xk7zmAqKxQYHWE4ARWFlKHZg8FB4VTDGxVua7iruRg=
github.com/aiven/go-client-codegen v0.73.0/go.mod h1:QKN/GgLMGWd6+gPEucXlZPi5vC3C6RpD3UeBRQOLI1Y=
github.com/apparentlymart/go-textseg/v12 v12.0.0/go.mod h1:S/4uRK2UtaQttw1GenVJEynmyUenKwP++x/+DdGV/Ec=
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
Expand Down
223 changes: 223 additions & 0 deletions internal/acctest/template.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
package acctest

import (
"bytes"
"fmt"
"sort"
"strings"
"testing"
"text/template"
)

// ResourceConfig is the interface that all resource configs must implement
type resourceConfig interface {
// ToMap converts the config to a map for template rendering
ToMap() map[string]any
}

// Template represents a single Terraform configuration template
type Template struct {
Name string
Template string
}

// TemplateRegistry holds templates for a specific resource type
type TemplateRegistry struct {
resourceName string
templates map[string]*template.Template
funcMap template.FuncMap
}

// NewTemplateRegistry creates a new template registry for a resource
func NewTemplateRegistry(resourceName string) *TemplateRegistry {
return &TemplateRegistry{
resourceName: resourceName,
templates: make(map[string]*template.Template),
funcMap: make(template.FuncMap),
}
}

// AddTemplate adds a new template to the registry
func (r *TemplateRegistry) AddTemplate(t testing.TB, name, templateStr string) error {
t.Helper()

tmpl := template.New(name)
if len(r.funcMap) > 0 {
tmpl = tmpl.Funcs(r.funcMap)
}

parsed, err := tmpl.Parse(templateStr)
if err != nil {
return fmt.Errorf("failed to parse template: %w", err)
}
r.templates[name] = parsed

return nil
}

// MustAddTemplate is like AddTemplate but panics on error
func (r *TemplateRegistry) MustAddTemplate(t testing.TB, name, templateStr string) {
t.Helper()

if err := r.AddTemplate(t, name, templateStr); err != nil {
t.Fatal(err)
}
}

// Render renders a template with the given config
func (r *TemplateRegistry) Render(t testing.TB, templateKey string, cfg map[string]any) (string, error) {
t.Helper()

tmpl, exists := r.templates[templateKey]
if !exists {
availableTemplates := r.getAvailableTemplates()

return "", fmt.Errorf("template %q does not exist for resource %s. Available templates: %v",
templateKey,
r.resourceName,
availableTemplates,
)
}

var buf bytes.Buffer
if err := tmpl.Execute(&buf, cfg); err != nil {
return "", fmt.Errorf("failed to render template: %w", err)
}

return buf.String(), nil
}

// MustRender is like Render but fails the test on error
func (r *TemplateRegistry) MustRender(t testing.TB, templateKey string, cfg map[string]any) string {
t.Helper()

result, err := r.Render(t, templateKey, cfg)
if err != nil {
t.Fatal(err)
}

return result
}

// AddFunction adds a custom function to the template registry
func (r *TemplateRegistry) AddFunction(name string, fn interface{}) {
if r.funcMap == nil {
r.funcMap = make(template.FuncMap)
}
r.funcMap[name] = fn
}

// HasTemplate checks if a template exists in the registry
func (r *TemplateRegistry) HasTemplate(key string) bool {
_, exists := r.templates[key]
return exists
}

// RemoveTemplate removes a template from the registry
func (r *TemplateRegistry) RemoveTemplate(key string) {
delete(r.templates, key)
}

// getAvailableTemplates returns a sorted list of available template keys
func (r *TemplateRegistry) getAvailableTemplates() []string {
templates := make([]string, 0, len(r.templates))
for k := range r.templates {
templates = append(templates, k)
}
sort.Strings(templates)

return templates
}

// compositionEntry represents a combination of template and its config
type compositionEntry struct {
TemplateKey string
Config map[string]any
}

// CompositionBuilder helps build complex compositions of templates
type CompositionBuilder struct {
registry *TemplateRegistry
compositions []compositionEntry
}

// NewCompositionBuilder creates a new composition builder
func (r *TemplateRegistry) NewCompositionBuilder() *CompositionBuilder {
return &CompositionBuilder{
registry: r,
compositions: make([]compositionEntry, 0),
}
}

// Add adds a new template and config to the composition
func (b *CompositionBuilder) Add(templateKey string, cfg map[string]any) *CompositionBuilder {
b.compositions = append(b.compositions, compositionEntry{
TemplateKey: templateKey,
Config: cfg,
})
return b
}

// AddWithConfig adds a new template and config to the composition using a resourceConfig
func (b *CompositionBuilder) AddWithConfig(templateKey string, cfg resourceConfig) *CompositionBuilder {
b.compositions = append(b.compositions, compositionEntry{
TemplateKey: templateKey,
Config: cfg.ToMap(),
})
return b
}

// AddIf conditional method to CompositionBuilder
func (b *CompositionBuilder) AddIf(condition bool, templateKey string, cfg map[string]any) *CompositionBuilder {
if condition {
return b.Add(templateKey, cfg)
}

return b
}

func (b *CompositionBuilder) Remove(templateKey string) *CompositionBuilder {
var newCompositions []compositionEntry
for _, comp := range b.compositions {
if comp.TemplateKey != templateKey {
newCompositions = append(newCompositions, comp)
}
}
b.compositions = newCompositions

return b
}

// Render renders all templates in the composition and combines them
func (b *CompositionBuilder) Render(t testing.TB) (string, error) {
t.Helper()

var renderedParts []string

Check failure on line 195 in internal/acctest/template.go

View workflow job for this annotation

GitHub Actions / make_lint

Consider pre-allocating `renderedParts` (prealloc)

Check failure on line 195 in internal/acctest/template.go

View workflow job for this annotation

GitHub Actions / make_lint

Consider pre-allocating `renderedParts` (prealloc)

// Render each template
for _, comp := range b.compositions {
rendered, err := b.registry.Render(t, comp.TemplateKey, comp.Config)
if err != nil {
return "", fmt.Errorf("failed to render template %s: %w", comp.TemplateKey, err)
}
renderedParts = append(renderedParts, rendered)
}

// Combine all rendered parts
combined := strings.Join(renderedParts, "\n\n")

//TODO: add HCL validation?

return combined, nil
}

// MustRender is like Render but fails the test on error
func (b *CompositionBuilder) MustRender(t testing.TB) string {
t.Helper()

result, err := b.Render(t)
if err != nil {
t.Fatal(err)
}
return result
}
Loading

0 comments on commit 8ee0a6a

Please sign in to comment.