This repository contains submodules of all the repositories used for skywire deployment
- Skywire Deployment
- Table of Contents
- Initialize the repo
- Code Formatting
- Code Linting
- Code Tests
- Building
- Runtime Dependencies by Service
- Required Services
- Key generation for services
- SKYCOIN-SERVICE-DISCOVERY Setup
- service-discovery
- DMSG Setup
- SKYWIRE-SERVICES Setup
- SKYWIRE Setup
- Using Dmsg to connect to the deployment
- HTTP Service Configuration
This repo contains all necessary repos to deploy Skywire as submodules. To set the source code up locally,
make sure you have make and git installed on your machine.
Clone this repo and all its submodules with:
git clone --recurse-submodules https://github.com/skycoin/skywire-deployment.git
Dependencies:
For any given submodule repository, included is a Makefile with directives such as format.
make format executes a variation of the following commands, depending on the repo:
go mod tidy -v
[[ -d pkg ]] && goimports -w -local github.com/skycoin/$(basename $(pwd)) ./pkg
[[ -d cmd ]] && goimports -w -local github.com/skycoin/$(basename $(pwd)) ./cmd
[[ -d internal ]] && goimports -w -local github.com/skycoin/$(basename $(pwd)) ./internal
find . -type f -name '*.go' -not -path "./.git/*" -not -path "./vendor/*" -exec goimports-reviser -project-name github.com/skycoin/$(basename $(pwd)) {} \;
Dependencies:
For any given submodule repository, included is a Makefile with a lint directive.
make lint executes a variation of the following commands, depending on the repo:
golangci-lint run -c .golangci.yml ./...
Dependencies:
For any given submodule repository, included is a Makefile with a test directive.
Note: make check combines the lint and test directives and is currently used by the CI to check pull requests.
make test executes a variation of the following commands:
go clean -testcache &>/dev/null
[[ -d internal ]] && go test -cover -timeout=5m -mod=vendor ./internal/...
[[ -d pkg ]] && go test -cover -timeout=5m -mod=vendor ./pkg/...
Note: The flags used for go test may vary from repo to repo
Dependencies:
For any given submodule repository (except skywire-utilities) included is a Makefile with a build directive.
make build executes go build for all the binaries that can be built from .go files in ./cmd subdirectories, for example
cd skywire
go build ./cmd/skywire-visor/skywire-visor.go
go build ./cmd/skywire-cli/skywire-cli.go
It is also possible to go run without explicitly compiling a binary
cd skywire
go run ./cmd/skywire-visor/skywire-visor.go --help
go run ./cmd/skywire-cli/skywire-cli.go --help
Note that some binaries may have gcc / libc6 dependency unless statically compiled with musl.
The releases for skywire now comprise a single binary executable file with subcommands for every previously separately-compiled binary. It is recommended, for the sake of simpliciity, to compile only this one binary from the skywire repo.
This document will henceforth assume the reader has compiled this unified binary and that it exists in the executable PATH.
The Makefile directives in skywire have been updated (on develop branch) to only produce the new unified binary; separate compilations are still posssible but should be considered deprecated in the case of visor native applications as there is no way to automatically generate a config for the separate apps since the release format has been changed to the single binary.
combined documentation can be found here
Redis setup simply entails installing redis and starting the service.
Some services use postgresql. The database setup is included in this document for each service which requires it.
As a brief overview of the process, one must run postgresql, and make a database with UTF-8 character-set. The name of the database is inconsequential. The database name as well as the postgres credentials are specified via environmental variables
Pass the database, user, and password as envs
export PG_USER=username
export PG_PASSWORD=pass
export PG_DATABASE=sampledb
Specify the postgres host and port via flags
--pg-host localhost --pg-port 5432
All tables are created automatically.
- address-resolver
- dmsg-discovery
- dmsg-server
- transport-discovery
- route-finder
- service-discovery
- setup-node
A list of endpoints corresponding to some of these services in the current deployment is provided here for reference:
- sd.skycoin.com/api/services?type=proxy
- ar.skywire.skycoin.com/transports
- tpd.skywire.skycoin.com/all-transports
- dmsgd.skywire.skycoin.com/dmsg-discovery/entries
- rf.skywire.skycoin.com/
Public/secret keypairs for all services have identical format can can be used interchangeably. Generate a keypair with skywire cli config gen-keys:
skywire cli config gen-keys
The keypair can be written to a file and used by a service which accepts it in the following way
skywire cli config gen-keys | tee dmsgd-config.json
skywire dmsg disc --sk $(tail -n1 dmsgd-config.json)
The following endpoint is queried by the visor on config gen:
https://conf.skywire.skycoin.com/
This endpoint contains json which will become part of the visor's config. The following file is created manually to reflect your deployment:
{
"dmsg_discovery": "http://dmsgd.skywire.skycoin.com",
"transport_discovery": "http://tpd.skywire.skycoin.com",
"address_resolver": "http://ar.skywire.skycoin.com",
"route_finder": "http://rf.skywire.skycoin.com",
"route_setup_nodes": [
"0324579f003e6b4048bae2def4365e634d8e0e3054a20fc7af49daf2a179658557",
"024fbd3997d4260f731b01abcfce60b8967a6d4c6a11d1008812810ea1437ce438",
"03b87c282f6e9f70d97aeea90b07cf09864a235ef718725632d067873431dd1015"
],
"transport_setup": [
"03530b786c670fc7f5ab9021478c7ec9cd06a03f3ea1416c50c4a8889ef5bba80e",
"03271c0de223b80400d9bd4b7722b536a245eb6c9c3176781ee41e7bac8f9bad21",
"03a792e6d960c88c6fb2184ee4f16714c58b55f0746840617a19f7dd6e021699d9",
"0313efedc579f57f05d4f5bc3fbf0261f31e51cdcfde7e568169acf92c78868926",
"025c7bbf23e3441a36d7e8a1e9d717921e2a49a2ce035680fec4808a048d244c8a",
"030eb6967f6e23e81db0d214f925fc5ce3371e1b059fb8379ae3eb1edfc95e0b46",
"02e582c0a5e5563aad47f561b272e4c3a9f7ac716258b58e58eb50afd83c286a7f",
"02ddc6c749d6ed067bb68df19c9bcb1a58b7587464043b1707398ffa26a9746b26",
"03aa0b1c4e23616872058c11c6efba777c130a85eaf909945d697399a1eb08426d",
"03adb2c924987d8deef04d02bd95236c5ae172fe5dfe7273e0461d96bf4bc220be"
],
"uptime_tracker": "http://ut.skywire.skycoin.com",
"service_discovery": "http://sd.skycoin.com",
"stun_servers": [
"192.53.117.238:3478",
"170.187.228.44:3478",
"192.53.117.237:3478",
"192.53.117.146:3478",
"192.53.117.60:3478",
"192.53.117.124:3478",
"170.187.228.178:3478",
"170.187.225.246:3478"
],
"dns_server": "1.1.1.1",
"survey_whitelist": [
"02b5ee5333aa6b7f5fc623b7d5f35f505cb7f974e98a70751cf41962f84c8c4637",
"03714c8bdaee0fb48f47babbc47c33e1880752b6620317c9d56b30f3b0ff58a9c3",
"020d35bbaf0a5abc8ec0ba33cde219fde734c63e7202098e1f9a6cf9daaeee55a9",
"027f7dec979482f418f01dfabddbd750ad036c579a16422125dd9a313eaa59c8e1",
"031d4cf1b7ab4c789b56c769f2888e4a61c778dfa5fe7e5cd0217fc41660b2eb65",
"0327e2cf1d2e516ecbfdbd616a87489cc92a73af97335d5c8c29eafb5d8882264a",
"03abbb3eff140cf3dce468b3fa5a28c80fa02c6703d7b952be6faaf2050990ebf4"
]
}
This endpoint is used to avoid relying on hardcoded defaults for the production deployment endpoints in the visor's config 9they still are hardcoded for fallback scenarios. Without it, any changes to the deployment would require an updated version of skywire to manifest correct config default values or significant user intervention.
A deployment of all services as a set will use this for visors to query the endpoint during config gen to get the services for the custom deployment, otherwise it is not strictly required.
Note as of v1.3.19 the services as defined in the conf service are distributed with the skywire binary releases as services-config.json
This config file is used in the instancers that the services cannot be fetched during config gen. It may also be explicitly specified, in order to provide another means of configuring the visor to connect to a different or alternative deployment.
This section deals with skycoin-service-discovery or SD which can be built from the skycoin/skycoin-service-discovery repo
Note: this service requires redis
Note: this service requires postgresql & initial DB setup
sudo -iu postgres createdb sd
Run the Service Discovery server
skywire cli config gen-keys > sd-config.json
PG_USER="postgres" PG_DATABASE="sd" PG_PASSWORD="" skywire svc sd --addr ":9098" --redis "redis://localhost:6379" --dmsg-disc "http://127.0.0.1:9090" --sk $(tail -n1 sd-config.json)
This section deals with services which can be built from the skycoin/dmsg repo
Note: this service requires redis
Run the Dmsg Discovery server
skywire cli config gen-keys > dmsgd-config.json
skywire dmsg disc --addr ":9090" --redis "redis://localhost:6379" --sk $(tail -n1 dmsgd-config.json)
Generate a config for the dmsg server
dmsg-server config gen -o dmsg-config.json
output
{
"public_key": "02a4072022716ee7fb8205edd4286dc73abe514ff8a7a74c6a27131c6efbe71876",
"secret_key": "62bcd18c5693ad1d2a24239f7303a84d6afb6966340fc70afe93579987cceb90",
"discovery": "http://dmsgd.skywire.skycoin.com",
"public_address": "127.0.0.1:8081",
"local_address": ":8081",
"health_endpoint_address": ":8082",
"log_level": "info",
"update_interval": 0,
"max_sessions": 2048
}
The IP address must be a public ip address.
The port on which the dmsg server is running must be forwarded or otherwise accessible for public servers.
The above "discovery" endpoint should be changed to match the endpoint of the dmsg discovery server referenced in the previous section. The default ports are shown.
Run the dmsg server
skywire dmsg server start dmsg-config.json
This section deals with services which can be built from the skycoin/skywire-services repo
address-resolver Note: the specified port must be on a public ip address or port forwarded for udp_ Note: this service requires redis
Run the address resolver
skywire cli config gen-keys > ar-config.json
skywire svc ar --addr ":9093" --redis "redis://localhost:6379" --dmsg-disc "http://127.0.0.1:9090" --sk $(tail -n1 ar-config.json)
Note: this service requires postgresql & initial DB setup
sudo -iu postgres createdb rf
Run the Route Finder
skywire cli config gen-keys > rf-config.json
PG_USER="postgres" PG_DATABASE="rf" PG_PASSWORD="" skywire svc rf --addr ":9092" --dmsg-disc "http://127.0.0.1:9090" --sk $(tail -n1 rf-config.json)
Note: this service requires redis
Note: this service requires postgresql & initial DB setup
sudo -iu postgres createdb tpd
Run the Transport Discovery server
keys-gen | tee tpd-config.json
PG_USER="postgres" PG_DATABASE="tpd" PG_PASSWORD="" transport-discovery --addr ":9091" --redis "redis://localhost:6379" --dmsg-disc "http://127.0.0.1:9090" --sk $(tail -n1 tpd-config.json)
This section deals with services which can be built from the skycoin/skywire repo
Running route setup-node requires a config as follows:
{
"public_key": "02a4072022716ee7fb8205edd4286dc73abe514ff8a7a74c6a27131c6efbe71876",
"secret_key": "62bcd18c5693ad1d2a24239f7303a84d6afb6966340fc70afe93579987cceb90",
"dmsg": {
"discovery": "http://dmsgd.skywire.skycoin.com",
"sessions_count": 1,
"servers": []
},
"transport_discovery": "http://tpd.skywire.skycoin.com",
"log_level": "debug"
}
This configuration can be parsed from the output of skywire-cli config gen -n and should be repopulated with the endpoints for your deployment. Example using jq
skywire cli config gen -n --loglvl debug | jq '{public_key: .pk, secret_key: .sk, dmsg: {discovery: .dmsg.discovery, sessions_count: .dmsg.sessions_count, servers: .dmsg.servers}, transport_discovery: .transport.discovery, log_level: .log_level}'
config generation for setup node will be provided in the future
Run the setup-node
setup-node setup-node-config.json
skywire-visor skywire-cli skywire
A brief overview of the visor's use with a new deployment is as follows
Generate a config with defaults for your deployment:
Note the -p flag is used for the linux package installation path
skywire-cli config gen -bpirxa conf.haltingstate.net -o skywire-config.json
example output
{
"version": "v1.3.20",
"sk": "30706958e03ef393418df2ed5a24da477730a0101da9b17959d4f1f231624877",
"pk": "03adb8df92b82320ee1507a131c8cde0a0fdf267b4e43ec1be1849e4f9dbc258c4",
"dmsg": {
"discovery": "http://dmsgd.haltingstate.net",
"sessions_count": 2,
"servers": [],
"servers_type": "all"
},
"dmsgpty": {
"dmsg_port": 22,
"cli_network": "unix",
"cli_address": "/tmp/dmsgpty.sock",
"whitelist": []
},
"skywire-tcp": {
"pk_table": null,
"listening_address": ":7777"
},
"transport": {
"discovery": "http://tpd.haltingstate.net",
"address_resolver": "http://ar.haltingstate.net",
"public_autoconnect": true,
"transport_setup": [
"03530b786c670fc7f5ab9021478c7ec9cd06a03f3ea1416c50c4a8889ef5bba80e",
"03271c0de223b80400d9bd4b7722b536a245eb6c9c3176781ee41e7bac8f9bad21",
"03a792e6d960c88c6fb2184ee4f16714c58b55f0746840617a19f7dd6e021699d9",
"0313efedc579f57f05d4f5bc3fbf0261f31e51cdcfde7e568169acf92c78868926",
"025c7bbf23e3441a36d7e8a1e9d717921e2a49a2ce035680fec4808a048d244c8a",
"030eb6967f6e23e81db0d214f925fc5ce3371e1b059fb8379ae3eb1edfc95e0b46",
"02e582c0a5e5563aad47f561b272e4c3a9f7ac716258b58e58eb50afd83c286a7f",
"02ddc6c749d6ed067bb68df19c9bcb1a58b7587464043b1707398ffa26a9746b26",
"03aa0b1c4e23616872058c11c6efba777c130a85eaf909945d697399a1eb08426d",
"03adb2c924987d8deef04d02bd95236c5ae172fe5dfe7273e0461d96bf4bc220be"
],
"log_store": {
"type": "file",
"location": "/opt/skywire/local/transport_logs",
"rotation_interval": "168h0m0s"
},
"stcpr_port": 0,
"sudph_port": 0
},
"routing": {
"route_setup_nodes": [
"0324579f003e6b4048bae2def4365e634d8e0e3054a20fc7af49daf2a179658557",
"024fbd3997d4260f731b01abcfce60b8967a6d4c6a11d1008812810ea1437ce438",
"03b87c282f6e9f70d97aeea90b07cf09864a235ef718725632d067873431dd1015"
],
"route_finder": "http://rf.haltingstate.net",
"route_finder_timeout": "10s",
"min_hops": 0
},
"uptime_tracker": {
"addr": "http://ut.skywire.skycoin.com"
},
"launcher": {
"service_discovery": "http://sd.haltingstate.net",
"apps": [
{
"name": "vpn-client",
"binary": "skywire",
"args": [
"app",
"vpn-client",
"--dns",
"1.1.1.1"
],
"auto_start": false,
"port": 43
},
{
"name": "skychat",
"binary": "skywire",
"args": [
"app",
"skychat",
"--addr",
":8001"
],
"auto_start": true,
"port": 1
},
{
"name": "skysocks",
"binary": "skywire",
"args": [
"app",
"skysocks"
],
"auto_start": true,
"port": 3
},
{
"name": "skysocks-client",
"binary": "skywire",
"args": [
"app",
"skysocks-client",
"--addr",
":1080"
],
"auto_start": false,
"port": 13
},
{
"name": "vpn-server",
"binary": "skywire",
"args": [
"app",
"vpn-server"
],
"auto_start": false,
"port": 44
}
],
"server_addr": "localhost:5505",
"bin_path": "/opt/skywire/bin",
"display_node_ip": false
},
"survey_whitelist": [
"02b5ee5333aa6b7f5fc623b7d5f35f505cb7f974e98a70751cf41962f84c8c4637",
"03714c8bdaee0fb48f47babbc47c33e1880752b6620317c9d56b30f3b0ff58a9c3",
"020d35bbaf0a5abc8ec0ba33cde219fde734c63e7202098e1f9a6cf9daaeee55a9",
"027f7dec979482f418f01dfabddbd750ad036c579a16422125dd9a313eaa59c8e1",
"031d4cf1b7ab4c789b56c769f2888e4a61c778dfa5fe7e5cd0217fc41660b2eb65",
"0327e2cf1d2e516ecbfdbd616a87489cc92a73af97335d5c8c29eafb5d8882264a",
"03abbb3eff140cf3dce468b3fa5a28c80fa02c6703d7b952be6faaf2050990ebf4"
],
"hypervisors": [],
"cli_addr": "localhost:3435",
"log_level": "",
"local_path": "/opt/skywire/local",
"dmsghttp_server_path": "/opt/skywire/local/custom",
"stun_servers": [
"192.53.117.238:3478",
"170.187.228.44:3478",
"192.53.117.237:3478",
"192.53.117.146:3478",
"192.53.117.60:3478",
"192.53.117.124:3478",
"170.187.228.178:3478",
"170.187.225.246:3478"
],
"shutdown_timeout": "10s",
"is_public": false,
"persistent_transports": null
}
Note the default ports:
stcp :7777
cli RPC :3435
appsrv :5505
HVUI :8000
skychat :8001
ports which are by default 0 are selected at random
"stcpr_port": 0,
"sudph_port": 0
Note: tcp, udp, and http ports in the config will always have a colon and will never be low ports by default!
The other ports are virtual dmsg ports
Run skywire-visor
skywire visor -c skywire-config.json
A JSON config file called dmsghttp-config.json can be created containing the dmsg addresses of the services for a deployment. This configuration is used automatically based on region (iran, china) with skywire-cli config gen -b or used by default with skywire-cli config gen -d and must be present at the expected path - either the current directory or the installation directory - for that configuration to be generated. This file is included in the skywire github repository as well as with our releases.
the following file is created manually to reflect the deployment
{
"test": {
"dmsg_servers": [
{
"static":"024716428e6315d954356e9ad72bea32bb2b41aab5a54a9b5cb4313964016e64d8",
"server":{
"address":"172.104.188.39:30080"
}
},
{
"static":"03f6b0a20be8fe0fd2fd0bd850507cfb10d0eaa37dce5c174654d07db5749a41bf",
"server":{
"address":"192.53.115.181:8083"
}
},
{
"static":"02ae49c901480b49f9c40f6f90fa8e775cf9b00b61ded673a9b0776dfb7cccd374",
"server":{
"address":"139.162.45.141:8083"
}
}
],
"dmsg_discovery": "dmsg://03cd2336e5de74bdab2bbdb44b06b0c8c713a5ee9029615f5526f8c99a6afe87b8:80",
"transport_discovery": "dmsg://02703cf828ea11d25b2c8eb0796132ecc7e53b22325b20ce3674ce5cd8693e4fb6:80",
"address_resolver": "dmsg://030eb7d8cf6eac40c19bbc433de6d6b9bb7a47f2e1d7095c6a01aa676471670ad2:80",
"route_finder": "dmsg://02ece5b69eaee13ef967b7eb67ca93f1dfddad3a51c9cb1808c4bd0d8d8aa32053:80",
"uptime_tracker": "dmsg://022c788cca11f208cdfd83ed0c2a8c7b661221736c461adc7c6738a2c1b041c7f8:80",
"service_discovery": "dmsg://038f751df4af75fb3d51f6693602bfe8289145e633ffdd1e67d686bea595f84d55:80"
},
"prod": {
"dmsg_servers": [
{
"static":"02a2d4c346dabd165fd555dfdba4a7f4d18786fe7e055e562397cd5102bdd7f8dd",
"server":{
"address":"dmsg.server02a2d4c3.skywire.skycoin.com:30081"
}
},
{
"static":"03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4",
"server":{
"address":"dmsg.server03717576.skywire.skycoin.com:30082"
}
},
{
"static":"0228af3fd99c8d86a882495c8e0202bdd4da78c69e013065d8634286dd4a0ac098",
"server":{
"address":"45.118.133.242:30084"
}
},
{
"static":"03d5b55d1133b26485c664cf8b95cff6746d1e321c34e48c9fed293eff0d6d49e5",
"server":{
"address":"dmsg.server03d5b55d.skywire.skycoin.com:30083"
}
},
{
"static":"0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb",
"server":{
"address":"192.53.114.142:30085"
}
},
{
"static":"02a49bc0aa1b5b78f638e9189be4ed095bac5d6839c828465a8350f80ac07629c0",
"server":{
"address":"dmsg.server02a4.skywire.skycoin.com:30089"
}
}
],
"dmsg_discovery": "dmsg://022e607e0914d6e7ccda7587f95790c09e126bbd506cc476a1eda852325aadd1aa:80",
"transport_discovery": "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80",
"address_resolver": "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80",
"route_finder": "dmsg://039d89c5eedfda4a28b0c58b0b643eff949f08e4f68c8357278081d26f5a592d74:80",
"uptime_tracker": "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80",
"service_discovery": "dmsg://0204890f9def4f9a5448c2e824c6a4afc85fd1f877322320898fafdf407cc6fef7:80"
}
}
The following caddy-server Caddyfile configuration reflects the default ports of the http services.
(common) {
header /* {
-Server
}
}
dmsgd.haltingstate.net {
reverse_proxy http://127.0.0.1:9090
import common
}
ar.haltingstate.net {
reverse_proxy http://127.0.0.1:9093
import common
}
rf.haltingstate.net {
reverse_proxy http://127.0.0.1:9092
import common
}
tpd.haltingstate.net {
reverse_proxy http://127.0.0.1:9091
import common
}
sd.haltingstate.net {
reverse_proxy http://127.0.0.1:9098
import common
}
conf.haltingstate.net {
header Content-Type application/json
respond {"dmsg_discovery":"http://dmsgd.haltingstate.net","transport_discovery":"http://tpd.haltingstate.net","address_resolver":"http://ar.haltingstate.net","route_finder":"http://rf.haltingstate.net","setup_nodes":["024fbd3997d4260f731b01abcfce60b8967a6d4c6a11d1008812810ea1437ce438"],"uptime_tracker":"http://ut.skywire.skycoin.com","service_discovery":"http://sd.haltingstate.net","stun_servers":["139.162.12.30:3478","170.187.228.181:3478","172.104.161.184:3478","170.187.231.137:3478","143.42.74.91:3478","170.187.225.78:3478","143.42.78.123:3478","139.162.12.244:3478"],"dns_server":"1.1.1.1"}
import common
}