Red Hat

You can install Kubernetes version 1.34 on RHEL (Red Hat Enterprise Linux) 8.x / 9.x and Rocky Linux operating systems. You can configure Containerd container runtime, create cluster network structure with Flannel network plugin, and configure multi-master.

Checks Required Before Starting Installation

Before starting installations, make sure that the server’s hostname is not localhost.localdomain and that each one is unique (with hostname command). If it is, be sure to change it before starting operations.

(If Necessary) Changing Hostname

hostnamectl set-hostname your-new-hostname

There should not be an IP block hostname assignment like 127.0.1.1 in the /etc/hosts file.

There should not be an entry like "nameserver 127.0.1.1" in the /etc/resolv.conf file.

If a Proxy is required for internet access, the following codes should be executed.

#Run the following on Linux Shell:
export http_proxy=http://proxyIp:port/
export https_proxy=http://proxyIp:port/
export no_proxy=localhost,127.0.0.1,SERVERIP,*.hostname

Add the following codes to the files below:

sudo vi /etc/apt/apt.conf

Acquire::http::Proxy "http://username:password@proxyIp:port";
Acquire::https::Proxy "https://username:password@proxyIp:port";

sudo mkdir -p /etc/systemd/system/containerd.service.d/

sudo vi /etc/systemd/system/containerd.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=http://proxyIp:port"
Environment="HTTPS_PROXY=https://proxyIp:port"
Environment="NO_PROXY=localhost,127.0.0.1,::1,SERVERIP,*.hostname"

# systemd'yi yeniden yüklenir
sudo systemctl daemon-reload
 
# containerd'yi yeniden başlatılır
sudo systemctl restart containerd
 
# Ayarların yüklendiğini kontrol edilir
sudo systemctl show --property=Environment containerd

1) Operating System Configurations (To be performed on all servers)

# Refresh your cache of enabled yum repositories.
dnf makecache --refresh

# Execute following dnf command to update your Rocky Linux server.
dnf update -y

# It is recommended that the following tools be installed on all servers.
sudo dnf install -y net-tools yum-utils bind-utils  device-mapper-persistent-data lvm2 telnet wget zip curl lsof

# Apinizer user is created and authorized.
sudo adduser apinizer
sudo passwd apinizer
sudo usermod -aG wheel apinizer

# Switch to the user to continue operations
su - apinizer

# Firewall is disabled
sudo systemctl stop firewalld
sudo systemctl disable firewalld

# SELinux is disabled to prevent communication problems on servers.
sudo setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# Swap is disabled and the swap line in /etc/fstab file is deleted to prevent it from restarting.
sudo swapoff -a
sudo vi /etc/fstab
# Then close the vi file (:wq)

2) Kubernetes Installation

2.1) Container Installation (To be performed on all Kubernetes servers)

The following steps are followed to prepare the system and install Docker before proceeding to Kubernetes installation.

#To load modules in the running system
sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF

#sysctl settings 
sudo vi /etc/sysctl.d/k8s.conf

The first three lines here are mandatory, and the others can be changed according to need.

net.ipv4.ip_local_port_range=1024 65535
fs.file-max=2097152
net.core.somaxconn=65535
net.ipv4.tcp_max_syn_backlog=65535
net.ipv4.tcp_max_tw_buckets=1440000
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
net.core.wmem_default=8388608
net.core.rmem_default=8388608
net.ipv4.tcp_sack=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_fin_timeout=15
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_moderate_rcvbuf=1
net.core.rmem_max=134217728
net.core.wmem_max=134217728
net.ipv4.tcp_mem=134217728 134217728 134217728
net.ipv4.tcp_rmem=4096 277750 134217728
net.ipv4.tcp_wmem=4096 277750 134217728
net.core.netdev_max_backlog=300000

Required modules are loaded.

#Loading configurations
sudo sysctl --system

#Defining Docker repository
sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf makecache

#Installing containerd module
sudo dnf install -y containerd.io

#Settings of containerd module
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml

sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

#Starting containerd module
sudo systemctl restart containerd
sudo systemctl enable containerd
systemctl status containerd

If you encounter errors like package runc-1:1.1.4-1.module+el8.7.0+16520+2db5507d.x86_64 is filtered out by modular filtering, apply the following command line.

sudo yum remove containerd.io | sudo yum remove runc

2.2) Kubernetes Installation (On Master and Worker servers)

Kubernetes keys and repository addresses are loaded to the system, Kubernetes is installed and started.

sudo vi /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.34/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.34/rpm/repodata/repomd.xml.key

#Kubernetes installation
sudo dnf makecache --refresh
sudo dnf install -y kubeadm kubelet kubectl
systemctl enable --now kubelet.service

#Kubernetes installation check and startup
kubectl version --client && kubeadm version

2.2.1) Bash Auto-Completion (Optional, On any Kubernetes Master Server)

This operation can speed up writing Kubernetes commands.

yum install bash-completion
echo 'source <(kubectl completion bash)' >>~/.bashrc
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
source ~/.bashrc

2.2.2) Starting Kubernetes Control Plane (Master)

Execute the following command to download the container images required to create the Kubernetes Cluster.

Since this Node is the first Node in the cluster, it will be selected as Kubernetes Control Plane.

kubeadm config images pull
sudo kubeadm init --pod-network-cidr "10.244.0.0/16" --control-plane-endpoint "<MASTER_SERVER_HOSTNAME>" --upload-certs

To use Multi-Master structure, other nodes that will be Master should be connected with the following code.

sudo kubeadm join <MASTER_SERVER_HOSTNAME>:6443 --token <XXX> --discovery-token-ca-cert-hash sha256:<YYY> --control-plane --certificate-key <ZZZ>

Very Important

If the connection command is desired to be recreated, the output of the second of the following codes should be added to the first;

kubeadm token create --print-join-command
sudo kubeadm init phase upload-certs --upload-certs

As a result, it should look like the following:

<output of join command in step 1> --control-plane --certificate-key <key value that is the output of step 2>

If the code is desired to be created manually, the following are used:

• For XXX → kubeadm token list
• For YYY → openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
• For ZZZ → sudo kubeadm init phase upload-certs --upload-certs

2.2.3) Setting User Configuration of Kubectl Command on Kubernetes Master Server (On Kubernetes Master Servers)

Required settings are made while on the user who will run kubectl commands.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown -R $(id -u):$(id -g) $HOME/.kube

Execute the following command to set the KUBECONFIG variable for all sessions.

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile.d/k8s.sh

2.2.4) Install Kubernetes Network Plugin (On Kubernetes Master Servers)

We will use the Flannel network plugin in this guide. You can choose other supported network plugins. Flannel is a simple and easy way to configure a Layer-3 network structure designed for Kubernetes.

curl https://raw.githubusercontent.com/projectcalico/calico/v3.31.3/manifests/calico.yaml -O 

kubectl apply -f calico.yaml

2.2.5) If Master Server is Also Desired to be Used as Worker (Optional)

This is not a recommended method for production environments.

To add Worker task to Master

# To make all Masters Workers
kubectl  taint nodes --all node-role.kubernetes.io/control-plane:NoSchedule-

# To give Worker task to all Masters, master needs to be written for some cases
kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule-

# To give Worker task to only one Master
kubectl taint nodes <NODE_NAME> node-role.kubernetes.io/control-plane:NoSchedule-

To remove Worker task from Master

# To remove Worker task from all Masters
kubectl taint nodes --all node-role.kubernetes.io/control-plane:NoSchedule

# For some cases, master needs to be written instead of control-plane
kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule

# To remove Worker task from only one Master
kubectl taint nodes <NODE_NAME> node-role.kubernetes.io/control-plane:NoSchedule

2.2.6) Registering Kubernetes Worker Servers to Master (On Kubernetes Worker Servers)

A token information is needed to connect the Worker server to Master. This will be visible in writing on the master node during this installation phase. But if it is skipped or if you want to view it again, the following command can be used.

On Master Node

sudo kubeadm token create --print-join-command

On Node(s) that will be Worker

sudo kubeadm join <MASTER_SERVER_HOSTNAME>:6443 --token xxx --discovery-token-ca-cert-hash sha256:yyy

2.2.7) Installation Check (On any Kubernetes Master Server)

If the Node created in addition to Master is also visible when the following code is run from Master, the installation has been completed successfully.

If it does not transition from NotReady status to Ready status after two minutes, the problem should be examined with the "kubectl describe node <NODE_NAME>" command.

kubectl get node

NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   5d    v1.34.0
k8s-worker1  Ready    <none>   5d    v1.34.0

2.3) DNS Test (Optional, On any Kubernetes Master Server)

kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
kubectl -n kube-system get configmap coredns -oyaml
kubectl exec -i -t dnsutils -- nslookup kubernetes.default
kubectl exec -ti dnsutils -- cat /etc/resolv.conf

For more information, please review the address at this link.