Keto CLI cannot communicate with the containers

[nithinguruswamy]

@[](### Preflight checklist

• I could not find a solution in the existing issues, docs, nor discussions.
• I agree to follow this project's Code of Conduct.
• I have read and am following this repository's Contribution Guidelines.
• I have joined the Ory Community Slack.
• I am signed up to the Ory Security Patch Newsletter.

Ory Network Project

No response

Describe the bug

Hi, I was trying the Keto CLI for creating relationship tuple (the create relation-tuples command), but the command didn't seem to work. It just output "context deadline exceed". It looked like the CLI cannot connect with the serve. So, I did a further investigation by testing it with the status command. The output was "NOT_SERVING".

Reproducing the bug

" kind: Deployment
apiVersion: apps/v1
metadata:
name: "keto-service"
namespace: dev
labels:
app: "keto-service"
spec:
replicas: 1
revisionHistoryLimit: 3
selector:
matchLabels:
app: "keto-service"
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: "keto-service"
version: v1
spec:
initcontainers:
- name: keto-service-migrate
image: oryd/keto:v0.11.0-alpha.0
imagePullPolicy: IfNotPresent
command: ["keto"]
args: [ "migrate", "up", "-y", "--config", "/etc/config/keto.yaml" ]
envFrom:
- configMapRef:
name: "keto-service-migrate"
volumeMounts:
- name: keto-config-volume
mountPath: /etc/config
readOnly: true
podSecurityContext:
fsGroupChangePolicy: "OnRootMismatch"
runAsNonRoot: true
runAsUser: 65534
fsGroup: 65534
runAsGroup: 65534
seccompProfile:
type: RuntimeDefault
sysctls: []
supplementalGroups: []
containers:
- name: keto-service
securityContext:
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
allowPrivilegeEscalation: false
privileged: false
seLinuxOptions:
level: "s0:c123,c456"
image: oryd/keto:v0.11.0-alpha.0
imagePullPolicy: IfNotPresent
command: ["keto"]
args: [ "serve", "--config", "/etc/config/keto.yaml" ]
envFrom:
- configMapRef:
name: "keto-service"
volumeMounts:
- name: keto-config-volume
mountPath: /etc/config
readOnly: true
ports:
- name: grpc-write
containerPort: 4467
protocol: TCP
- name: grpc-read
containerPort: 4466
protocol: TCP
- name: http-metrics
containerPort: 4468
protocol: TCP
resources:
requests:
cpu: "300m"
memory: "768Mi"
livenessProbe:
httpGet:
path: /health/alive
port: 4467
httpHeaders:
- name: Host
value: '127.0.0.1'
imagePullSecrets:
- name: "cloud.docker.deployuser"
volumes:
- name: keto-config-volume
configMap:
name: keto-service "

`---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
namespace: dev
spec:
selector:
istio: ingressgateway
servers:
- hosts:
{{- range .Values.ory_keto_gateway_hosts_list }}
- {{ . | quote }}
{{- end }}
port:
name: http
number: 80
protocol: HTTP2
tls:
httpsRedirect: true
- hosts:
{{- range .Values.ory_keto_gateway_hosts_list }}
- {{ . | quote }}
{{- end }}
port:
number: 443
name: https
protocol: HTTP
tls:
mode: "SIMPLE"
credentialName: "dev-k8s-us-west-2-certificate"

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: keto-metrics
namespace: dev
spec:
hosts:
ory-keto-metrics.dev.exam.com/metrics
gateways:
- ory_keto_service_gateway
http:
- route:
- destination:
host: "keto-service-metrics"
port:
number: 80
corsPolicy:
allowOrigin:
- "*"
allowMethods:
- POST
- GET
- OPTIONS
- PUT
- DELETE
allowHeaders:
- content-type
- authorization
maxAge: 100000000s
exposeHeaders:
- authorization
- content-type
allowCredentials: true
debug: false
enabled: true

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: keto-write
namespace: dev
spec:
hosts:
https://ory-keto-write.dev.exam.com/write
gateways:
- ory_keto_service_gateway
http:
- match:
- uri:
prefix: /write
route:
- destination:
host: "keto-service-write"
port:
number: 80
corsPolicy:
allowOrigin:
- "*"
allowMethods:
- POST
- GET
- OPTIONS
- PUT
- DELETE
allowHeaders:
- content-type
- authorization
maxAge: 100000000s
exposeHeaders:
- authorization
- content-type
allowCredentials: true
debug: false
enabled: true

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: keto-read
namespace: dev
spec:
hosts:
https://ory-keto-read.dev.exam.com/read
- ory_keto_service_gateway
http:
- match:
- uri:
prefix: /read
route:
- destination:
host: "keto-service-read"
port:
number: 80
corsPolicy:
allowOrigin:
- "*"
allowMethods:
- POST
- GET
- OPTIONS
- PUT
- DELETE
allowHeaders:
- content-type
- authorization
maxAge: 100000000s
exposeHeaders:
- content-type
- authorization
allowCredentials: true
debug: false
enabled: true`

`kind: Service
apiVersion: v1
metadata:
name: "keto-service-write"
namespace: dev
labels:
app: "keto-service"
spec:
ports:
- name: write
port: 80
targetPort: 4467
protocol: TCP
appProtocol: grpc
selector:
app: keto-service

`kind: Service
apiVersion: v1
metadata:
name: "keto-service-read"
namespace: dev
labels:
app: "keto-service"
spec:
ports:
- name: grpc-read
port: 80
targetPort: 4466
protocol: TCP
appProtocol: grpc
selector:
app: "keto-service"

`kind: Service
apiVersion: v1
metadata:
name: "keto-service-metrics"
namespace: dev
labels:
app: "keto-service"
spec:
ports:
- name: http-metrics
port: 80
targetPort: 4468
protocol: TCP
selector:
app: "keto-service"

Relevant log output

~ $ keto status
NOT_SERVING
~ $ keto status --read-remote 4466
NOT_SERVING
~ $ keto status --write-remote 4467
NOT_SERVING

Relevant configuration

---

`kind: ConfigMap
apiVersion: v1
metadata:
name: "keto-service-migrate"
namespace: dev
annotations:
helm.sh/hook: pre-install, pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: '0'
data:
dsn: xyz/ory
"keto.yaml": |
namespaces:
- name: "sample"
id: 1
serve:
metrics:
port: 4468
host: 0.0.0.0
read:
port: 4466
host: 0.0.0.0
write:
port: 4467
host: 0.0.0.0

`kind: ConfigMap
apiVersion: v1
metadata:
name: "keto-service"
namespace: dev
data:
dsn: xyz/ory
"keto.yaml": |
namespaces:
- name: "sample"
id: 1
serve:
metrics:
port: 4468
read:
port: 4466
write:
port: 4467

Version

0.11.0

On which operating system are you observing this issue?

macOS

In which environment are you deploying?

Kubernetes with Helm

Additional Context

I am getting context deadline

~ $ keto relation-tuple get --insecure-disable-transport-security (base) 6.8s  Thu Jun 29 14:37:50 2023
Error: context deadline exceeded
Usage:
keto relation-tuple get [flags]

Flags:
--authority string Set the authority header for the remote gRPC server.
--format string Set the output format. One of table, json, yaml, json-pretty, jsonpath and jsonpointer. (default "default")
-h, --help help for get
--insecure-disable-transport-security Disables transport security. Do not use this in production.
--insecure-skip-hostname-verification Disables hostname verification. Do not use this in production.
--namespace string Set the requested namespace
--object string Set the requested object
--page-size int32 maximum number of items to return (default 100)
--page-token string page token acquired from a previous response
-q, --quiet Be quiet with output printing.
--read-remote string Remote address of the read API endpoint. (default "127.0.0.1:4466")
--relation string Set the requested relation
--subject-id string Set the requested subject ID
--subject-set string Set the requested subject set; format: "namespace:object#relation"
--write-remote string Remote address of the write API endpoint. (default "127.0.0.1:4467")

Global Flags:
-c, --config strings Config files to load, overwriting in the order specified. (default [/Users/christian/keto.yml])

context deadline exceeded)