Developers guide
Development environment prerequisites
- A dagger installation
How to test Klio in a kind cluster during development
During the development phase, it is recommended to test any changes in a test environment as soon as they are implemented. To perform such a test, a developer can build and deploy the binaries and the images in a reusable Kubernetes environment.
This section will guide you through the steps required to deploy the Klio operator and its resources inside a kind cluster.
- Create a kind cluster using the
hack/setup-cluster.shscript from the CNPG repository. - To build and deploy Klio images and operator, exec the command:
KIND_CLUSTER_NAME=$(kind get clusters) task integration:deploy-to-kind - Once the operator is in
Readystate, you can deploy the resources:
- Klio server:
kubectl apply -f operator/config/samples/klio_v1alpha1_server.yaml - CNPG Cluster with Klio plugin and related PluginConfiguration CR:
kubectl apply -f operator/config/samples/cluster-example-plugin.yamlThe PluginConfiguration CR name should be equal to the value ofrefparameter, set inside the Cluster manifest.spec.pluginssection.
plugins:
- name: klio.cnpg.io
enabled: true
parameters:
pluginConfigurationRef: client-config-example-plugin
Repeat step 2 to deploy the new changes applied to the code, and reset the operator. Then rollout the new binary in both server and cluster pods:
- delete the Klio Server pod:
kubectl delete pod server-sample-klio-0 - delete the Cluster's pods:
kubectl delete pod -l cnpg.io/cluster=cluster-example - delete the PluginConfiguration CR:
kubectl delete pluginconfiguration client-config-example-plugin
How to get performance profiles with pprof
- Open a port forward connection to port 6060:
kubectl port-forward pod/server-sample-klio-0 6060 - run pprof tool, e.g.:
go tool pprof -http=:9999 http://localhost:6060/debug/pprof/profile\?seconds=30
How to recreate the GRPC stub and skeleton
dagger call protoc --source . -o internal/grpc
How to setup manually a Klio server
Given the following configuration file in /home/klio/.klio.yaml:
tier1:
encryption_key: thiskey
base:
repository: /home/klio/klio_data/base
cache: /home/klio/klio_data/base
listen_address: 0.0.0.0:12000
wal:
listen_address: 0.0.0.0:12001
path: /home/klio/klio_data/wals
tls:
client_ca_cert: /home/klio/ca-cert.pem
cert: /home/klio/server.crt
key: /home/klio/server.key
A new Klio repository for both WAL and base backups can be bootstrapped with:
openssl genrsa -out ca-key.pem 2048
openssl req -new -key ca-key.pem -out ca-csr.pem -subj "/CN=KlioClientsCA"
openssl x509 -req -in ca-csr.pem -signkey ca-key.pem -out ca-cert.pem -days 365 -extfile <(printf "basicConstraints=CA:TRUE,pathlen:0\nkeyUsage=critical,cRLSign,keyCertSign")
openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 \
-nodes -keyout server.key -out server.crt -subj "/CN=klio-server" \
-addext "subjectAltName=DNS:klio-server,IP:127.0.0.1"
~/klio server initialize
The Klio server hosts both the WAL streaming server and the base backup server in a single process. Start them with:
~/klio server start
How to setup manually a Klio client
Given the following configuration file in /var/lib/postgresql/.klio.yaml:
client:
cluster_name: pg-dev-1
wal:
server_cert_path: /home/klio/server.crt
client_cert_path: /home/klio/client-cert.pem
client_key_path: /home/klio/client-key.pem
address: klio-server:12000
base:
url: https://klio-server:12001
server_cert_path: /home/klio/server.crt
client_cert_path: /home/klio/client-cert.pem
client_key_path: /home/klio/client-key.pem
source:
dsn: "user=postgres replication=yes"
standard_dsn: "dbname=postgres"
slot: klio
The client certificates can be created with:
openssl genrsa -out client-key.pem 2048
openssl req -new -key client-key.pem -out client-csr.pem -subj "/CN=klio@cluster-example"
cat <<EOF > client-ext.cnf
[ req ]
distinguished_name = req_distinguished_name
[ req_distinguished_name ]
CN = klio@cluster-example
[ usr_cert ]
keyUsage = critical, digitalSignature
extendedKeyUsage = clientAuth
EOF
openssl x509 -req -in client-csr.pem -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -days 365 -extfile client-ext.cnf -extensions usr_cert