> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apinizer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Red Hat

> You can install Kubernetes version 1.34 on RHEL (Red Hat Enterprise Linux) 8.x / 9.x and Rocky Linux operating systems. You can configure Containerd container runtime, create cluster network structure with Flannel network plugin, and configure multi-master.

## Checks Required Before Starting Installation

Before starting installations, make sure that the server's hostname is not localhost.localdomain and that each one is unique (with hostname command). If it is, be sure to change it before starting operations.

<Warning>
  ### (If Necessary) Changing Hostname

  ```bash theme={null}
  hostnamectl set-hostname your-new-hostname
  ```

  There should not be an IP block hostname assignment like `127.0.1.1` in the `/etc/hosts` file.

  There should not be an entry like `"nameserver 127.0.1.1"` in the `/etc/resolv.conf` file.
</Warning>

<Warning>
  If a Proxy is required for internet access, the following codes should be executed.

  ```bash theme={null}
  #Run the following on Linux Shell:
  export http_proxy=http://proxyIp:port/
  export https_proxy=http://proxyIp:port/
  export no_proxy=localhost,127.0.0.1,SERVERIP,*.hostname
  ```

  Add the following codes to the files below:

  ```bash theme={null}
  sudo vi /etc/apt/apt.conf
  ```

  ```
  Acquire::http::Proxy "http://username:password@proxyIp:port";
  Acquire::https::Proxy "https://username:password@proxyIp:port";
  ```

  ```bash theme={null}
  sudo mkdir -p /etc/systemd/system/containerd.service.d/

  sudo vi /etc/systemd/system/containerd.service.d/http-proxy.conf
  ```

  ```
  [Service]
  Environment="HTTP_PROXY=http://proxyIp:port"
  Environment="HTTPS_PROXY=https://proxyIp:port"
  Environment="NO_PROXY=localhost,127.0.0.1,::1,SERVERIP,*.hostname"
  ```

  ```bash theme={null}
  # systemd'yi yeniden yüklenir
  sudo systemctl daemon-reload
   
  # containerd'yi yeniden başlatılır
  sudo systemctl restart containerd
   
  # Ayarların yüklendiğini kontrol edilir
  sudo systemctl show --property=Environment containerd
  ```
</Warning>

## 1) Operating System Configurations (To be performed on all servers)

```bash theme={null}
# Refresh your cache of enabled yum repositories.
dnf makecache --refresh

# Execute following dnf command to update your Rocky Linux server.
dnf update -y

# It is recommended that the following tools be installed on all servers.
sudo dnf install -y net-tools yum-utils bind-utils  device-mapper-persistent-data lvm2 telnet wget zip curl lsof

# Apinizer user is created and authorized.
sudo adduser apinizer
sudo passwd apinizer
sudo usermod -aG wheel apinizer

# Switch to the user to continue operations
su - apinizer

# Firewall is disabled
sudo systemctl stop firewalld
sudo systemctl disable firewalld

# SELinux is disabled to prevent communication problems on servers.
sudo setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# Swap is disabled and the swap line in /etc/fstab file is deleted to prevent it from restarting.
sudo swapoff -a
sudo vi /etc/fstab
# Then close the vi file (:wq)
```

## 2) Kubernetes Installation

### 2.1) Container Installation (To be performed on all Kubernetes servers)

The following steps are followed to prepare the system and install Docker before proceeding to Kubernetes installation.

```bash theme={null}
#To load modules in the running system
sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF

#sysctl settings 
sudo vi /etc/sysctl.d/k8s.conf
```

The first three lines here are mandatory, and the others can be changed according to need.

```
net.ipv4.ip_local_port_range=1024 65535
fs.file-max=2097152
net.core.somaxconn=65535
net.ipv4.tcp_max_syn_backlog=65535
net.ipv4.tcp_max_tw_buckets=1440000
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
net.core.wmem_default=8388608
net.core.rmem_default=8388608
net.ipv4.tcp_sack=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_fin_timeout=15
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_moderate_rcvbuf=1
net.core.rmem_max=134217728
net.core.wmem_max=134217728
net.ipv4.tcp_mem=134217728 134217728 134217728
net.ipv4.tcp_rmem=4096 277750 134217728
net.ipv4.tcp_wmem=4096 277750 134217728
net.core.netdev_max_backlog=300000
```

Required modules are loaded.

```bash theme={null}
#Loading configurations
sudo sysctl --system

#Defining Docker repository
sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf makecache

#Installing containerd module
sudo dnf install -y containerd.io

#Settings of containerd module
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml

sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

#Starting containerd module
sudo systemctl restart containerd
sudo systemctl enable containerd
systemctl status containerd
```

<Info>
  If you encounter errors like `package runc-1:1.1.4-1.module+el8.7.0+16520+2db5507d.x86_64` is filtered out by modular filtering, apply the following command line.

  ```bash theme={null}
  sudo yum remove containerd.io | sudo yum remove runc
  ```
</Info>

### 2.2) Kubernetes Installation (On Master and Worker servers)

Kubernetes keys and repository addresses are loaded to the system, Kubernetes is installed and started.

```bash theme={null}
sudo vi /etc/yum.repos.d/kubernetes.repo
```

```
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.34/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.34/rpm/repodata/repomd.xml.key
```

```bash theme={null}
#Kubernetes installation
sudo dnf makecache --refresh
sudo dnf install -y kubeadm kubelet kubectl
systemctl enable --now kubelet.service

#Kubernetes installation check and startup
kubectl version --client && kubeadm version
```

#### 2.2.1) Bash Auto-Completion (Optional, On any Kubernetes Master Server)

This operation can speed up writing Kubernetes commands.

```bash theme={null}
yum install bash-completion
echo 'source <(kubectl completion bash)' >>~/.bashrc
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
source ~/.bashrc
```

#### 2.2.2) Starting Kubernetes Control Plane (Master)

Execute the following command to download the container images required to create the Kubernetes Cluster.

Since this Node is the first Node in the cluster, it will be selected as Kubernetes Control Plane.

```bash theme={null}
kubeadm config images pull
sudo kubeadm init --pod-network-cidr "10.244.0.0/16" --control-plane-endpoint "<MASTER_SERVER_HOSTNAME>" --upload-certs
```

To use Multi-Master structure, other nodes that will be Master should be connected with the following code.

```bash theme={null}
sudo kubeadm join <MASTER_SERVER_HOSTNAME>:6443 --token <XXX> --discovery-token-ca-cert-hash sha256:<YYY> --control-plane --certificate-key <ZZZ>
```

<Warning>
  **Very Important**

  If the connection command is desired to be recreated, the output of the second of the following codes should be added to the first;

  ```bash theme={null}
  kubeadm token create --print-join-command
  sudo kubeadm init phase upload-certs --upload-certs
  ```

  As a result, it should look like the following:

  ```
  <output of join command in step 1> --control-plane --certificate-key <key value that is the output of step 2>
  ```

  If the code is desired to be created manually, the following are used:

  * For XXX → `kubeadm token list`
  * For YYY → `openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'`
  * For ZZZ → `sudo kubeadm init phase upload-certs --upload-certs`
</Warning>

#### 2.2.3) Setting User Configuration of Kubectl Command on Kubernetes Master Server (On Kubernetes Master Servers)

Required settings are made while on the user who will run kubectl commands.

```bash theme={null}
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown -R $(id -u):$(id -g) $HOME/.kube
```

Execute the following command to set the KUBECONFIG variable for all sessions.

```bash theme={null}
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile.d/k8s.sh
```

#### 2.2.4) Install Kubernetes Network Plugin (On Kubernetes Master Servers)

We will use the Flannel network plugin in this guide. You can choose other supported network plugins. Flannel is a simple and easy way to configure a Layer-3 network structure designed for Kubernetes.

```bash theme={null}
curl https://raw.githubusercontent.com/projectcalico/calico/v3.31.3/manifests/calico.yaml -O 
```

```bash theme={null}
kubectl apply -f calico.yaml
```

#### 2.2.5) If Master Server is Also Desired to be Used as Worker (Optional)

<Warning>
  This is not a recommended method for production environments.
</Warning>

<Info>
  To add Worker task to Master

  ```bash theme={null}
  # To make all Masters Workers
  kubectl  taint nodes --all node-role.kubernetes.io/control-plane:NoSchedule-

  # To give Worker task to all Masters, master needs to be written for some cases
  kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule-

  # To give Worker task to only one Master
  kubectl taint nodes <NODE_NAME> node-role.kubernetes.io/control-plane:NoSchedule-
  ```

  To remove Worker task from Master

  ```bash theme={null}
  # To remove Worker task from all Masters
  kubectl taint nodes --all node-role.kubernetes.io/control-plane:NoSchedule

  # For some cases, master needs to be written instead of control-plane
  kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule

  # To remove Worker task from only one Master
  kubectl taint nodes <NODE_NAME> node-role.kubernetes.io/control-plane:NoSchedule
  ```
</Info>

#### 2.2.6) Registering Kubernetes Worker Servers to Master (On Kubernetes Worker Servers)

A token information is needed to connect the Worker server to Master. This will be visible in writing on the master node during this installation phase. But if it is skipped or if you want to view it again, the following command can be used.

On Master Node

```bash theme={null}
sudo kubeadm token create --print-join-command
```

On Node(s) that will be Worker

```bash theme={null}
sudo kubeadm join <MASTER_SERVER_HOSTNAME>:6443 --token xxx --discovery-token-ca-cert-hash sha256:yyy
```

#### 2.2.7) Installation Check (On any Kubernetes Master Server)

If the Node created in addition to Master is also visible when the following code is run from Master, the installation has been completed successfully.

If it does not transition from NotReady status to Ready status after two minutes, the problem should be examined with the `"kubectl describe node <NODE_NAME>"` command.

```bash theme={null}
kubectl get node
```

```
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   5d    v1.34.0
k8s-worker1  Ready    <none>   5d    v1.34.0
```

### 2.3) DNS Test (Optional, On any Kubernetes Master Server)

```bash theme={null}
kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
kubectl -n kube-system get configmap coredns -oyaml
kubectl exec -i -t dnsutils -- nslookup kubernetes.default
kubectl exec -ti dnsutils -- cat /etc/resolv.conf
```

For more information, please review the address at this link.