Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discovery: SQLite-based server implementation #2589

Merged
merged 24 commits into from
Nov 30, 2023
Merged

Discovery: SQLite-based server implementation #2589

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
865d95e
Discovery: SQL based server implementation
gerardsn Nov 17, 2023
9dacfc4
credential properties index
reinkrul Nov 19, 2023
ff9b719
regen
reinkrul Nov 20, 2023
984f8aa
sqlite concurrency
reinkrul Nov 20, 2023
fe929e3
SQL schema feedback fixes
reinkrul Nov 20, 2023
8b7bce7
feedback
reinkrul Nov 20, 2023
6d8a88d
unusedfunc
reinkrul Nov 20, 2023
55dcede
godoc, moved errors to interface.go
reinkrul Nov 20, 2023
e44fa5f
PEX: Changed path_nested to be object instead of slice (#2609)
reinkrul Nov 20, 2023
4d815cb
wip
reinkrul Nov 21, 2023
28f2795
PR feedback
reinkrul Nov 24, 2023
f6f4caa
fixed tests
reinkrul Nov 24, 2023
ffd37cb
Storage: do not write sqlite.db to source dir
reinkrul Nov 24, 2023
b6957d3
added VP verify, more tests, organized tests
reinkrul Nov 24, 2023
442cab1
feedback
reinkrul Nov 24, 2023
4eaadb5
feedback
reinkrul Nov 29, 2023
91b0bef
config flags
reinkrul Nov 29, 2023
e751c88
copyriht
reinkrul Nov 29, 2023
96d172d
documentation
reinkrul Nov 29, 2023
1cd95f6
Merge branch 'master' into usecaselist-maintainer
reinkrul Nov 29, 2023
3a759fc
tests, todo
reinkrul Nov 29, 2023
2467cc7
feedback
reinkrul Nov 29, 2023
7c25c86
tweak error msg
reinkrul Nov 30, 2023
0306b39
remove requirement retraction ID is scoped to issuer
reinkrul Nov 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,9 @@ The following options can be configured on the server:
:widths: 20 30 50
:class: options-table

==================================== =============================================================================================================================================================================================================================================================================================================== ==================================================================================================================================================================================================================================
==================================== =============================================================================================================================================================================================================================================================================================================== ================================================================================================================================================================================================================================================================================================================================
Key Default Description
==================================== =============================================================================================================================================================================================================================================================================================================== ==================================================================================================================================================================================================================================
==================================== =============================================================================================================================================================================================================================================================================================================== ================================================================================================================================================================================================================================================================================================================================
configfile nuts.yaml Nuts config file
cpuprofile When set, a CPU profile is written to the given path. Ignored when strictmode is set.
datadir ./data Directory where the node stores its files.
Expand Down Expand Up @@ -207,6 +207,9 @@ The following options can be configured on the server:
crypto.vault.pathprefix kv The Vault path prefix.
crypto.vault.timeout 5s Timeout of client calls to Vault, in Golang time.Duration string format (e.g. 1s).
crypto.vault.token The Vault token. If set it overwrites the VAULT_TOKEN env var.
**Discovery**
discovery.definitions.directory Directory to load Discovery Service Definitions from. If not set, the discovery service will be disabled. If the directory contains JSON files that can't be parsed as service definition, the node will fail to start.
discovery.server.definition_ids [] IDs of the Discovery Service Definitions for which to act as server. If an ID does not map to a loaded service definition, the node will fail to start.
**Events**
events.nats.hostname 0.0.0.0 Hostname for the NATS server
events.nats.port 4222 Port where the NATS server listens on
Expand All @@ -224,7 +227,7 @@ The following options can be configured on the server:
http.default.auth.type Whether to enable authentication for the default interface, specify 'token_v2' for bearer token mode or 'token' for legacy bearer token mode.
http.default.cors.origin [] When set, enables CORS from the specified origins on the default HTTP interface.
**JSONLD**
jsonld.contexts.localmapping [https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json=assets/contexts/lds-jws2020-v1.ldjson,https://schema.org=assets/contexts/schema-org-v13.ldjson,https://nuts.nl/credentials/v1=assets/contexts/nuts.ldjson,https://www.w3.org/2018/credentials/v1=assets/contexts/w3c-credentials-v1.ldjson] This setting allows mapping external URLs to local files for e.g. preventing external dependencies. These mappings have precedence over those in remoteallowlist.
jsonld.contexts.localmapping [https://nuts.nl/credentials/v1=assets/contexts/nuts.ldjson,https://www.w3.org/2018/credentials/v1=assets/contexts/w3c-credentials-v1.ldjson,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json=assets/contexts/lds-jws2020-v1.ldjson,https://schema.org=assets/contexts/schema-org-v13.ldjson] This setting allows mapping external URLs to local files for e.g. preventing external dependencies. These mappings have precedence over those in remoteallowlist.
jsonld.contexts.remoteallowlist [https://schema.org,https://www.w3.org/2018/credentials/v1,https://w3c-ccg.github.io/lds-jws2020/contexts/lds-jws2020-v1.json] In strict mode, fetching external JSON-LD contexts is not allowed except for context-URLs listed here.
**Network**
network.bootstrapnodes [] List of bootstrap nodes ('<host>:<port>') which the node initially connect to.
Expand Down Expand Up @@ -252,12 +255,12 @@ The following options can be configured on the server:
storage.redis.sentinel.password Password for authenticating to Redis Sentinels.
storage.redis.sentinel.username Username for authenticating to Redis Sentinels.
storage.redis.tls.truststorefile PEM file containing the trusted CA certificate(s) for authenticating remote Redis servers. Can only be used when connecting over TLS (use 'rediss://' as scheme in address).
storage.sql.connection Connection string for the SQL database. If not set, it defaults to a SQLite database stored inside the configured data directory
storage.sql.connection Connection string for the SQL database. If not set it, defaults to a SQLite database stored inside the configured data directory. Note: using SQLite is not recommended in production environments. If using SQLite anyways, remember to enable foreign keys ('_foreign_keys=on') and the write-ahead-log ('_journal_mode=WAL').
**VCR**
vcr.openid4vci.definitionsdir Directory with the additional credential definitions the node could issue (experimental, may change without notice).
vcr.openid4vci.enabled true Enable issuing and receiving credentials over OpenID4VCI.
vcr.openid4vci.timeout 30s Time-out for OpenID4VCI HTTP client operations.
==================================== =============================================================================================================================================================================================================================================================================================================== ==================================================================================================================================================================================================================================
==================================== =============================================================================================================================================================================================================================================================================================================== ================================================================================================================================================================================================================================================================================================================================

This table is automatically generated using the configuration flags in the core and engines. When they're changed
the options table must be regenerated using the Makefile:
Expand Down
5 changes: 5 additions & 0 deletions cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"context"
"errors"
"fmt"
"github.com/nuts-foundation/nuts-node/discovery"
"github.com/nuts-foundation/nuts-node/vdr/resolver"

"github.com/nuts-foundation/nuts-node/golden_hammer"
Expand All @@ -46,6 +47,7 @@ import (
"github.com/nuts-foundation/nuts-node/didman"
didmanAPI "github.com/nuts-foundation/nuts-node/didman/api/v1"
didmanCmd "github.com/nuts-foundation/nuts-node/didman/cmd"
discoveryCmd "github.com/nuts-foundation/nuts-node/discovery/cmd"
"github.com/nuts-foundation/nuts-node/events"
eventsCmd "github.com/nuts-foundation/nuts-node/events/cmd"
httpEngine "github.com/nuts-foundation/nuts-node/http"
Expand Down Expand Up @@ -192,6 +194,7 @@ func CreateSystem(shutdownCallback context.CancelFunc) *core.System {
vdrInstance := vdr.NewVDR(cryptoInstance, networkInstance, didStore, eventManager)
credentialInstance := vcr.NewVCRInstance(cryptoInstance, vdrInstance, networkInstance, jsonld, eventManager, storageInstance, pkiInstance)
didmanInstance := didman.NewDidmanInstance(vdrInstance, credentialInstance, jsonld)
discoveryInstance := discovery.New(storageInstance, credentialInstance)
authInstance := auth.NewAuthInstance(auth.DefaultConfig(), vdrInstance, credentialInstance, cryptoInstance, didmanInstance, jsonld, pkiInstance)
statusEngine := status.NewStatusEngine(system)
metricsEngine := core.NewMetricsEngine()
Expand Down Expand Up @@ -233,6 +236,7 @@ func CreateSystem(shutdownCallback context.CancelFunc) *core.System {
system.RegisterEngine(credentialInstance)
system.RegisterEngine(networkInstance)
system.RegisterEngine(authInstance)
system.RegisterEngine(discoveryInstance)
system.RegisterEngine(didmanInstance)
system.RegisterEngine(goldenHammer)
// HTTP engine MUST be registered last, because when started it dispatches HTTP calls to the registered routes.
Expand Down Expand Up @@ -333,6 +337,7 @@ func serverConfigFlags() *pflag.FlagSet {
set.AddFlagSet(eventsCmd.FlagSet())
set.AddFlagSet(pki.FlagSet())
set.AddFlagSet(goldenHammerCmd.FlagSet())
set.AddFlagSet(discoveryCmd.FlagSet())

return set
}
2 changes: 1 addition & 1 deletion cmd/root_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ func Test_CreateSystem(t *testing.T) {
system.VisitEngines(func(engine core.Engine) {
numEngines++
})
assert.Equal(t, 15, numEngines)
assert.Equal(t, 16, numEngines)
}

func Test_ClientCommand_ErrorHandlers(t *testing.T) {
Expand Down
37 changes: 37 additions & 0 deletions discovery/cmd/cmd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Copyright (C) 2023 Nuts community
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*
*/

package cmd

import (
"github.com/nuts-foundation/nuts-node/discovery"
"github.com/spf13/pflag"
)

// FlagSet contains flags relevant for the module.
func FlagSet() *pflag.FlagSet {
defs := discovery.DefaultConfig()
flagSet := pflag.NewFlagSet("discovery", pflag.ContinueOnError)
flagSet.String("discovery.definitions.directory", defs.Definitions.Directory,
"Directory to load Discovery Service Definitions from. If not set, the discovery service will be disabled. "+
"If the directory contains JSON files that can't be parsed as service definition, the node will fail to start.")
flagSet.StringSlice("discovery.server.definition_ids", defs.Server.DefinitionIDs,
"IDs of the Discovery Service Definitions for which to act as server. "+
"If an ID does not map to a loaded service definition, the node will fail to start.")
return flagSet
}
Loading
Loading