Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cloudflare KV Storage #1298

Merged
merged 70 commits into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
c49806a
feat: add cloudflare kv
Geun-Oh Mar 21, 2024
79a9ae2
test: add test
Geun-Oh Mar 21, 2024
adad63a
chore: add config
Geun-Oh Mar 21, 2024
8b6245b
chore: initialize mod
Geun-Oh Mar 21, 2024
5817eaf
chore: add contribute configuration
Geun-Oh Mar 21, 2024
c424c66
fix: rollback lint
Geun-Oh Mar 21, 2024
85133f6
fix: change kv => cloudflarekv
Geun-Oh Mar 21, 2024
8028341
chore: remove godotenv package
Geun-Oh Mar 21, 2024
d6e0360
fix: remove local env testing
Geun-Oh Mar 21, 2024
7410373
add: add error return
Geun-Oh Mar 21, 2024
115d290
Update .github/workflows/release-drafter-cloudflarekv.yml
Geun-Oh Mar 21, 2024
551373a
fix: remove v2 path
Geun-Oh Mar 21, 2024
cc2ea9a
fix: remove line
Geun-Oh Mar 21, 2024
7a7807b
feat: preallocate key with make
Geun-Oh Mar 21, 2024
5f53fa0
feat: add benchmark tests
Geun-Oh Mar 21, 2024
c83d7f8
docs: kv => Cloudflare KV
Geun-Oh Mar 21, 2024
e9bd7e9
add: set default value
Geun-Oh Mar 21, 2024
398c183
fix: remove stale version
Geun-Oh Mar 22, 2024
19be481
feat: add DB Conn & its testing
Geun-Oh Mar 22, 2024
796363e
docs: add ref on root
Geun-Oh Mar 22, 2024
01d6a94
fix: update go version
Geun-Oh Mar 22, 2024
1c65d38
test: change Conn test with NotNil
Geun-Oh Mar 22, 2024
65db3ca
docs: add Conn() ref
Geun-Oh Mar 22, 2024
6a5b6eb
test: add test settings
Geun-Oh Mar 28, 2024
f022a14
Merge branch 'gofiber:main' into main
Geun-Oh Mar 28, 2024
4798a58
test: add testing shell yaml file
Geun-Oh Mar 28, 2024
154e150
fix: clarify go version for testing
Geun-Oh Mar 28, 2024
b8d752b
fix: fix testing jobs
Geun-Oh Mar 28, 2024
fdda850
fix: run wrangler dev server at background
Geun-Oh Mar 28, 2024
12eb0f1
chore: remove unused library
Geun-Oh Mar 28, 2024
8ef7cd7
Merge branch 'main' into main
Geun-Oh Mar 28, 2024
5e03e38
Update test-cloudflarekv.yml
gaby Mar 28, 2024
033ebf7
Update test-cloudflarekv.yml
gaby Mar 28, 2024
f0076b5
Update cloudflarekv/README.md
gaby Mar 30, 2024
5b26a0b
Merge branch 'gofiber:main' into main
Geun-Oh Mar 30, 2024
510d583
feat: add cloudflare worker init step at benchmark
Geun-Oh Mar 30, 2024
a5bee00
Update cloudflarekv/README.md
ReneWerner87 Apr 2, 2024
5ef27d1
fix: remove logging in benchmark test
Geun-Oh Apr 2, 2024
9d3cdce
Merge branch 'main' into main
Geun-Oh Apr 2, 2024
26e9d83
Merge branch 'main' into main
Geun-Oh Apr 8, 2024
d8f61c2
Update benchmark.yml
ReneWerner87 Apr 8, 2024
f4f69d7
Update benchmark.yml
ReneWerner87 Apr 8, 2024
f0533e6
fix: add git core.fileMode true
Geun-Oh Apr 8, 2024
bc9b6c3
Merge branch 'main' of https://github.com/Geun-Oh/storage
Geun-Oh Apr 8, 2024
554fa73
fix: remove chmod
Geun-Oh Apr 8, 2024
2c4232d
Merge branch 'main' into main
Geun-Oh Apr 9, 2024
64725d9
Update cloudflarekv/cloudflarekv.go
ReneWerner87 Apr 9, 2024
b72e733
Update cloudflarekv/cloudflarekv.go
ReneWerner87 Apr 9, 2024
235e47e
fix: add chmod
Geun-Oh Apr 9, 2024
126df74
Update cloudflarekv.go
ReneWerner87 Apr 9, 2024
ad8df22
Add Cloudflare KV Storage
ReneWerner87 Apr 9, 2024
6d5d57d
Add Cloudflare KV Storage
ReneWerner87 Apr 9, 2024
7e321c1
fix: check in github permission change
Geun-Oh Apr 9, 2024
73bb4f1
Add Cloudflare KV Storage
ReneWerner87 Apr 9, 2024
a5b09ad
Merge remote-tracking branch 'origin/main'
ReneWerner87 Apr 9, 2024
1ae7fa0
Add Cloudflare KV Storage
ReneWerner87 Apr 9, 2024
1527f88
fix: fix dev env with 8787 port
Geun-Oh Apr 9, 2024
64b08d4
Add Cloudflare KV Storage
ReneWerner87 Apr 9, 2024
9590f1a
Merge branch 'main' into main
Geun-Oh Apr 10, 2024
64ba9a2
fix: set cloudflarekv schedule interval daily
Geun-Oh Apr 11, 2024
5cd7843
fix: set setup-node version 4
Geun-Oh Apr 11, 2024
29245c6
fix: init storage per test & remove default storage
Geun-Oh Apr 11, 2024
ffeab66
fix: set key test
Geun-Oh Apr 11, 2024
8bbaa80
fix: add cloudflare testing validation
Geun-Oh Apr 11, 2024
b397ca3
fix: valid get with list-up method
Geun-Oh Apr 11, 2024
0f8625f
fix: test get event with listing kv pairs
Geun-Oh Apr 11, 2024
da47a1f
Merge branch 'main' into main
Geun-Oh Apr 12, 2024
d4b1c83
fix: make get works
Geun-Oh Apr 12, 2024
8e3d3f9
fix: revalidate cache of getter
Geun-Oh Apr 12, 2024
71bbf1e
Merge branch 'main' into main
Geun-Oh Apr 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@ updates:
- "🤖 Dependencies"
schedule:
interval: "weekly"
- package-ecosystem: "gomod"
directory: "/kv/" # Location of package manifests
Geun-Oh marked this conversation as resolved.
Show resolved Hide resolved
labels:
- "🤖 Dependencies"
schedule:
interval: "weekly"
- package-ecosystem: "gomod"
directory: "/memcache/" # Location of package manifests
labels:
Expand Down
44 changes: 44 additions & 0 deletions .github/release-drafter-kv.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name-template: "KV - v$RESOLVED_VERSION"
tag-template: "kv/v$RESOLVED_VERSION"
tag-prefix: kv/v
include-paths:
- kv
categories:
- title: "🚀 New"
labels:
- "✏️ Feature"
- title: "🧹 Updates"
labels:
- "🧹 Updates"
- "🤖 Dependencies"
- title: "🐛 Fixes"
labels:
- "☢️ Bug"
- title: "📚 Documentation"
labels:
- "📒 Documentation"
change-template: "- $TITLE (#$NUMBER)"
change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.
exclude-contributors:
- dependabot
- dependabot[bot]
version-resolver:
major:
labels:
- "major"
minor:
labels:
- "minor"
- "✏️ Feature"
patch:
labels:
- "patch"
- "📒 Documentation"
- "☢️ Bug"
- "🤖 Dependencies"
- "🧹 Updates"
default: patch
template: |
$CHANGES
**Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...kv/v$RESOLVED_VERSION
Thank you $CONTRIBUTORS for making this update possible.
19 changes: 19 additions & 0 deletions .github/workflows/release-drafter-kv.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Release Drafter KV
on:
push:
# branches to consider in the event; optional, defaults to all
branches:
- master
- main
paths:
- "kv/**"
jobs:
draft_release_dynamodb:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: release-drafter/release-drafter@v6
with:
config-name: release-drafter-kv.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
29 changes: 29 additions & 0 deletions .github/workflows/test-kv.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
on:
push:
branches:
- master
- main
paths:
- "kv/**"
pull_request:
paths:
- "kv/**"
name: "Tests KV"
jobs:
Tests:
runs-on: ubuntu-latest
strategy:
matrix:
go-version:
- 1.19.x
- 1.20.x
- 1.21.x
steps:
- name: Fetch Repository
uses: actions/checkout@v4
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: "${{ matrix.go-version }}"
- name: Run Test
run: cd ./kv && go test ./... -v -race
119 changes: 119 additions & 0 deletions kv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
id: kv
title: CloudflareKV
---

![Release](https://img.shields.io/github/v/tag/gofiber/storage?filter=kv*)
[![Discord](https://img.shields.io/discord/704680098577514527?style=flat&label=%F0%9F%92%AC%20discord&color=00ACD7)](https://gofiber.io/discord)
![Test](https://img.shields.io/github/actions/workflow/status/gofiber/storage/test-kv.yml?label=Tests)
![Security](https://img.shields.io/github/actions/workflow/status/gofiber/storage/gosec.yml?label=Security)
![Linter](https://img.shields.io/github/actions/workflow/status/gofiber/storage/linter.yml?label=Linter)

A kv storage driver using [cloudflare/cloudflare-go](https://github.com/cloudflare/cloudflare-go).

**Note: Requires Go 1.19 and above**

### Table of Contents

- [Signatures](#signatures)
- [Installation](#installation)
- [Examples](#examples)
- [Config](#config)
- [Default Config](#default-config)

### Signatures

```go
func New(config ...Config) Storage
func (s *Storage) Get(key string) ([]byte, error)
func (s *Storage) Set(key string, val []byte, exp time.Duration) error
func (s *Storage) Delete(key string) error
func (s *Storage) Reset() error
func (s *Storage) Close() error
```

### Installation

KV is tested on the 2 last [Go versions](https://golang.org/dl/) with support for modules. So make sure to initialize one first if you didn't do that yet:

```bash
go mod init github.com/<user>/<repo>
```

And then install the KV implementation:

```bash
go get github.com/gofiber/storage/kv/v2
```

### Examples

Import the storage package.

```go
import "github.com/gofiber/storage/kv/v2"
```

You can use the following possibilities to create a storage.

Key must be an API Token generated with at least `Account.Workers KV Storage` permission.

Check [Create API Token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/) docs to generate.

```go
// Initialize default config
store := kv.New()

store := kv.New(kv.Config{
Key: "",
Email: "",
AccountID: "fiber",
NamespaceID: "fiber",
Reset: false,
})

```

### Config

```go
type Config struct {

// Cloudflare Auth Token
//
// Optional. Default is ""
Key string

// Cloudflare Email
//
// Optional. Default is ""
Email string

// Account id
//
// Optional. Default is "fiber"
AccountID string

// Namespace id
//
// Optional. Default is "fiber"
NamespaceID string

// Reset clears any existing keys in existing Table
//
// Optional. Default is false
Reset bool
}
```

### Default Config

```go
var ConfigDefault = Config{
Key: "",
Email: "",
AccountID: "fiber",
NamespaceID: "fiber",
Reset: false,
}
```
52 changes: 52 additions & 0 deletions kv/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
package kv

// Config defines the config for storage.
type Config struct {

// Cloudflare Auth Token
//
// Optional. Default is ""
Key string

// Cloudflare Email
//
// Optional. Default is ""
Email string

// Account id
//
// Optional. Default is "fiber"
AccountID string

// Namespace id
//
// Optional. Default is "fiber"
NamespaceID string

// Reset clears any existing keys in existing Table
//
// Optional. Default is false
Reset bool
}

// ConfigDefault is the default config
var ConfigDefault = Config{
Key: "",
Email: "",
AccountID: "fiber",
NamespaceID: "fiber",
Reset: false,
}

// Helper function to set default values
func configDefault(config ...Config) Config {
// Return default config if nothing provided
if len(config) < 1 {
return ConfigDefault
}

// Override default config
cfg := config[0]

return cfg
}
23 changes: 23 additions & 0 deletions kv/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
module github.com/gofiber/storage/kv/v2

go 1.20

require (
github.com/cloudflare/cloudflare-go v0.90.0
github.com/joho/godotenv v1.5.1
github.com/stretchr/testify v1.9.0
)

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.5 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
golang.org/x/net v0.22.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.5.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
43 changes: 43 additions & 0 deletions kv/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
github.com/cloudflare/cloudflare-go v0.90.0 h1:hFfZNPEd9FFh6AJCxSiYwNXwHIkoSWCB5Iuc8J0o7s8=
github.com/cloudflare/cloudflare-go v0.90.0/go.mod h1:nUqvBUUDRxNzsDSQjbqUNWHEIYAoUlgRmcAzMKlFdKs=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-hclog v0.9.2/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ=
github.com/hashicorp/go-hclog v1.2.0 h1:La19f8d7WIlm4ogzNHB0JGqs5AUDAZ2UfCY4sJXcJdM=
github.com/hashicorp/go-retryablehttp v0.7.5 h1:bJj+Pj19UZMIweq/iie+1u5YCdGrnxCT9yvm0e+Nd5M=
github.com/hashicorp/go-retryablehttp v0.7.5/go.mod h1:Jy/gPYAdjqffZ/yFGCFV2doI5wjtH1ewM9u8iYVjtX8=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40=
github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.8.1 h1:geMPLpDpQOgVyCg5z5GoRwLHepNdb71NXb67XFkP+Eg=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Loading