k3s servers (master) and agents (workers)
k3s Master 1/Control Panel 1:⌗
export K3S_DATASTORE_ENDPOINT=‘mysql://username:password@tcp(dbk3s.local.yourdns.com:3306)/k3s’
export K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(dbk3s.local.yourdns.com:3306)/k3s'
more generic:
again dbk3s.local.youdns.com is the local dns that we can easily point to any machine!
No Encryption:
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server –datastore-cafile="/path/to/keys/db.crt" –datastore-certfile="/path/to/keys/k3s.crt" –datastore-keyfile="/path/to/keys/k3s.key" –tls-san=lbk3s.local.yourdns.com –node-taint CriticalAddonsOnly=true:NoExecute –disable servicelb
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server --datastore-cafile="/path/to/keys/db.crt" --datastore-certfile="/path/to/keys/k3s.crt" --datastore-keyfile="/path/to/keys/k3s.key" --tls-san=lbk3s.local.yourdns.com --node-taint CriticalAddonsOnly=true:NoExecute --disable servicelb
Notes on params:⌗
{K3s version} e.g. v1.27.0+k3s1 lbk3s.local.yourdns.com stands for load balancer of k3s [master servers/control panel] we disable servicelb because we are going to use metallb for our local cluster!
Note additional params below if you want encryption for secrets in the cluster:⌗
Note: (I did not use this option as i am noob) feel free to give it a try let me know [see about section for contact details]
–encryption-provider-config-automatic-reload=true –secrets-encryption
full cmd with encryption would look like this:
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server –datastore-cafile="/path/to/keys/db.crt" –datastore-certfile="/path/to/keys/k3s.crt" –datastore-keyfile="/path/to/keys/k3s.key" –tls-san=lbk3s.local.yourdns.com –node-taint CriticalAddonsOnly=true:NoExecute –disable servicelb –encryption-provider-config-automatic-reload=true –secrets-encryption
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server --datastore-cafile="/path/to/keys/db.crt" --datastore-certfile="/path/to/keys/k3s.crt" --datastore-keyfile="/path/to/keys/k3s.key" --tls-san=lbk3s.local.yourdns.com --node-taint CriticalAddonsOnly=true:NoExecute --disable servicelb --encryption-provider-config-automatic-reload=true --secrets-encryption
Finally on this machine we need the token i.e.:
sudo cat /var/lib/rancher/k3s/server/node-token
k3s Master 2/Control Panel 2 ad infinitum:⌗
use the same command as on the first with the addition of –token= option at the end! Remember to do the export first!!!
export K3S_DATASTORE_ENDPOINT=‘mysql://username:password@tcp(dbk3s.local.yourdns.com:3306)/k3s’
No encryption:
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server –datastore-cafile="/path/to/keys/db.crt" –datastore-certfile="/path/to/keys/k3s.crt" –datastore-keyfile="/path/to/keys/k3s.key" –tls-san=lbk3s.local.yourdns.com –node-taint CriticalAddonsOnly=true:NoExecute –disable servicelb –token=‘token’
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server --datastore-cafile="/path/to/keys/db.crt" --datastore-certfile="/path/to/keys/k3s.crt" --datastore-keyfile="/path/to/keys/k3s.key" --tls-san=lbk3s.local.yourdns.com --node-taint CriticalAddonsOnly=true:NoExecute --disable servicelb --token='token'
or with encryption just stay consistent with whatever you did on master1:
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server –datastore-cafile="/path/to/keys/db.crt" –datastore-certfile="/path/to/keys/k3s.crt" –datastore-keyfile="/path/to/keys/k3s.key" –tls-san=lbk3s.local.yourdns.com –node-taint CriticalAddonsOnly=true:NoExecute –disable servicelb –encryption-provider-config-automatic-reload=true –secrets-encryption –token=‘token’
sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - server --datastore-cafile="/path/to/keys/db.crt" --datastore-certfile="/path/to/keys/k3s.crt" --datastore-keyfile="/path/to/keys/k3s.key" --tls-san=lbk3s.local.yourdns.com --node-taint CriticalAddonsOnly=true:NoExecute --disable servicelb --encryption-provider-config-automatic-reload=true --secrets-encryption --token='token'
Sanity check before we proceed to configuring agents on master 1 server run the command:
sudo kubectl get nodes
you should see the 2 master nodes etc for example in my case:
NAME STATUS ROLES AGE VERSION
zelda2 Ready control-plane,master 6d23h v1.26.6+k3s1
zelda1 Ready control-plane,master 6d23h v1.26.6+k3s1
Configuring agents ad infinitum:⌗
Note: no need to export db.
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION={K3s version} sh -s - agent –server https://lbk3s.local.yourdns.live:6443 –disable servicelb –token=‘token’
Note: not sure if we must specify encryption here if you used it before should be an easy fix even if you do…[but tedious as you’d have to uninstall the agent and reinstall with those flags but i think your getting it at this point :)
NAME STATUS ROLES AGE VERSION
link1 Ready <none> 6d23h v1.26.6+k3s1
link2 Ready <none> 6d23h v1.26.6+k3s1
zelda2 Ready control-plane,master 6d23h v1.26.6+k3s1
zelda1 Ready control-plane,master 6d23h v1.26.6+k3s1
link3 Ready <none> 6d23h v1.26.6+k3s1
link4 Ready <none> 6d23h v1.26.6+k3s1
Obviously the age will be a few seconds/minutes old… in your case :)