快速部署k8s集群

本文使用kubeadm方式來快速部署k8s集群，操作系統(tǒng)基于ubuntu操作系統(tǒng)

基礎(chǔ)配置

IP網(wǎng)段規(guī)劃

192.168.50.31 master節(jié)點

192.168.50.32 node01節(jié)點

192.168.50.33 node02節(jié)點

操作系統(tǒng)版本

• Ubuntu 22.04 LTS amd64版本

環(huán)境準(zhǔn)備

增加hosts解析記錄

分別3臺機器配置hosts解析記錄

在所有機器的/etc/hosts文件中增加解析

192.168.50.31    master01.k8s.com
192.168.50.31    node01.k8s.com
192.168.50.32    node02.k8s.com

配置主機名

• master01機器配置(192.168.50.31)

1
~$ hostnamectl set-hostname master01.k8s.com

• node01機器配置(192.168.50.31)

1
~$ hostnamectl set-hostname node01.k8s.com

• node02機器配置(192.168.50.32)

1
~$ hostnamectl set-hostname node02.k8s.com

禁止所有機器的防火墻

~$ systemctl disable --now ufw.service

所有機器啟用IPV4內(nèi)核轉(zhuǎn)發(fā)模塊

~$ modprobe br_netfilter

所有機器配置ulimit參數(shù)

~$ ulimit -SHn 65535

永久配置

~$ cat /etc/security/limits.conf
    soft nofile 65535
    hard nofile 65535
    soft nproc 65535
    hard nproc 65535

所有機器內(nèi)核參數(shù)配置

~$ cat /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.netfilter.nf_conntrack_max=2310720

net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max=655535
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 16384
~$ sysctl --system

所有機器IPVS模塊安裝

~$ modprobe -- ip_vs
~$ modprobe -- ip_vs_rr
~$ modprobe -- ip_vs_wrr
~$ modprobe -- ip_vs_sh
~$ modprobe -- nf_conntrack
~$ cat /etc/modules-load.d/ipvs.conf
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip

所有機器檢查模塊是否加載

~$ lsmod |grep -e ip_vs -e nf_conntrack
~$ apt install ipset # 安裝ipset, ipvsadm 管理工具

所有機器同步服務(wù)器時間

~$ ln -sf /usr/share/zoneinfo/Asia/Shanghai  /etc/localtime # 配置時區(qū)
~$ ntpdate time.windows.com

所有機器關(guān)閉swap

~$ swapoff  -a
~$ cat /etc/fstab
# 注釋swap分區(qū)
#/swap.img  none    swap    sw  0   0

容器運行時安裝

所有機器依賴包安裝

~$ apt-get install -y libseccomp2

所有機器containerd下載部署

~$ wget https://github.com/containerd/containerd/releases/download/v1.6.18/cri-containerd-cni-1.6.18-linux-amd64.tar.gz
~$ tar -tf cri-containerd-cni-1.6.18-linux-amd64.tar.gz # 查看壓縮包包含哪些文件
~$ tar zxf cri-containerd-cni-1.6.18-linux-amd64.tar.gz -C / # 解壓包到跟路徑
~$ export PATH=$PATH:/opt/cin/bin # 增加環(huán)境變量

所有機器配置文件生成

~$ mkdir -p /etc/containerd
~$ containerd config default > /etc/containerd/config.toml

所有機器修改配置文件

配置文件路徑/etc/containerd/config.toml

root = "/var/lib/containerd" # 保存持久化數(shù)據(jù)目錄
....
sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6"
....
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
....
    SystemdCgroup = true
....
[plugins."io.containerd.grpc.v1.cri".registry] # 鏡像加速配置
....
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["https://bqr1dr1n.mirror.aliyuncs.com"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
          endpoint = ["https://registry.aliyuncs.com/k8sxio"]

所有機器啟動containerd服務(wù)

~$ systemctl  enable --now containerd

開始集群安裝

所有機器kubeadm安裝

apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
cat </etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet=1.23.13-00 kubeadm=1.23.13-00 kubectl=1.23.13-00
sudo apt-mark hold kubelet=1.23.13-00 kubeadm=1.23.13-00 kubectl=1.23.13-00 # 鎖定版本

所有機啟動kubelet服務(wù)

~$ systemctl enable --now kubelet

master節(jié)點初始化集群配置文件

~$ kubeadm config print init-defaults --component-configs KubeletConfiguration >/root/kubeadm.yaml

master節(jié)點修改配置文件

配置文件路徑?/root/kubeadm.yaml

.....
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.50.31 # 替換成master地址
    bindPort: 6443
nodeRegistration:
  criSocket: /run/containerd/containerd.sock   # 修改為containerd的sock文件
  imagePullPolicy: IfNotPresent
  name: master01.k8s.com                         # 注冊的master名稱
  taints:                                      # 污點防止調(diào)度到master節(jié)點
    - effect: "NoSchedule"
      key: "node-role.kubernetes.io/master"
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1     # 增加kube-proxy模式配置
kind: KubeProxyConfiguration
mode: ipvs   # kube-proxy 模式
.... 
imageRepository: registry.aliyuncs.com/k8sxio     # 修改鏡像獲取地址為國內(nèi)地址
kind: ClusterConfiguration
kubernetesVersion: 1.23.13                        # 獲取k8s版本
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: 10.244.0.0/16  # 指定pod子網(wǎng)
....

初始化master節(jié)點

# 獲取鏡像
~$ kubeadm  config images pull --config /root/kubeadm.yaml
# 手動拉取coredns鏡像，阿里云沒有這個鏡像
~$ ctr -n k8s.io image pull docker.io/coredns/coredns:1.8.6
~$ ctr -n k8s.io image tag docker.io/coredns/coredns:1.8.6 registry.aliyuncs.com/k8sxio/coredns:v1.8.6
# 初始化master節(jié)點, 根據(jù)提示可加入node節(jié)點
～$ kubeadm  init -v 5 --config /root/kubeadm.yaml

初始化master節(jié)點后，根據(jù)kubeadm join提示來完成node節(jié)點的加入 如果忘記join節(jié)點token可以通過命令重新獲取新的tokenkubeadm token create --print-join-command

網(wǎng)絡(luò)插件安裝

calico文件下載github地址

~$ kubectl apply -f calico.yaml
# 移除containerd的網(wǎng)絡(luò)配置，放置影響pod分配的IP地址
~$ mv /etc/cni/net.d/10-containerd-net.conflist  /etc/cni/net.d/10-containerd-net.conflist.old

清理集群

執(zhí)行kubeadm rest以后需要在對應(yīng)的節(jié)點執(zhí)行清理，次命令可以在所有節(jié)點執(zhí)行

~$ kubeadm rest
~$ ifconfig cni0 down && ip link delete cni0
~$ rm -rf /var/lib/cni/