最受關(guān)注的 Cilium Service Mesh 到底怎么玩？ - 上手實踐
大家好，我是張晉濤。
Cilium 是一個基于 eBPF 技術(shù)，用于為容器工作負(fù)載間提供安全且具備可觀測性的網(wǎng)絡(luò)連接的開源軟件。
如果你對 Cilium 還不太了解，可以參考我之前的兩篇文章：
最近 Cilium v1.11.0 正式發(fā)布了，增加 Open Telemetry 的支持以及其他一些增強(qiáng)特性。同時，也宣布了 Cilium Service Mesh 的計劃。當(dāng)前 Cilium Service Mesh 正處于測試階段，預(yù)期在 2022 年會合并到 Cilium v1.12 版本中。
Cilium Service Mesh 也帶來了一個全新的模式。
Cilium 直接通過 eBPF 技術(shù)實現(xiàn)的 Service Mesh 相比我們常規(guī)的 Istio/Linkerd 等方案，最顯著的特點(diǎn)就是將 Sidecar proxy 模型替換成了 Kernel 模型， 如下圖：
不再需要每個應(yīng)用程序旁邊都放置一個 Sidecar 了，直接在每臺 Node 上提供支持。
我在幾個月前就已經(jīng)知道了這個消息并且進(jìn)行了一些討論，最近隨著 isovalent 的一篇文章 How eBPF will solve Service Mesh - Goodbye Sidecars ，Cilium Service Mesh 也成為了大家關(guān)注的焦點(diǎn)。
本篇我?guī)銓嶋H體驗下 Cilium Service Mesh。
安裝部署
這里我使用 KIND 作為測試環(huán)境，我的內(nèi)核版本是 5.15.8 。
準(zhǔn)備 KIND 集群
關(guān)于 KIND 命令行工具的安裝這里就不再贅述了，感興趣的小伙伴可以參考我之前的文章 《使用KIND搭建自己的本地 Kubernetes 測試環(huán)境》。
以下是我創(chuàng)建集群使用的配置文件：
apiVersion:?kind.x-k8s.io/v1alpha4
kind:?Cluster
nodes:
-?role:?control-plane
-?role:?worker
-?role:?worker
-?role:?worker
networking:
??disableDefaultCNI:?true
創(chuàng)建集群：
???cilium-mesh?kind?create?cluster?--config?kind-config.yaml?
Creating?cluster?"kind"?...
???Ensuring?node?image?(kindest/node:v1.22.4)???
???Preparing?nodes??????????????
???Writing?configuration????
???Starting?control-plane?????
???Installing?StorageClass????
???Joining?worker?nodes????
Set?kubectl?context?to?"kind-kind"
You?can?now?use?your?cluster?with:
kubectl?cluster-info?--context?kind-kind
Not?sure?what?to?do?next??????Check?out?https://kind.sigs.k8s.io/docs/user/quick-start/
安裝 Cilium CLI
這里我們使用 Cilium CLI 工具進(jìn)行 Cilium 的部署。
???cilium-mesh?curl?-L?--remote-name-all?https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz\{,.sha256sum\}
[1/2]:?https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz?-->?cilium-linux-amd64.tar.gz
--_curl_--https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
??%?Total????%?Received?%?Xferd??Average?Speed???Time????Time?????Time??Current
?????????????????????????????????Dload??Upload???Total???Spent????Left??Speed
100???154??100???154????0?????0????243??????0?--:--:--?--:--:--?--:--:--???242
100???664??100???664????0?????0????579??????0??0:00:01??0:00:01?--:--:--???579
100?14.6M??100?14.6M????0?????0??2928k??????0??0:00:05??0:00:05?--:--:--?3910k
[2/2]:?https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz.sha256sum?-->?cilium-linux-amd64.tar.gz.sha256sum
--_curl_--https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz.sha256sum
??%?Total????%?Received?%?Xferd??Average?Speed???Time????Time?????Time??Current
?????????????????????????????????Dload??Upload???Total???Spent????Left??Speed
100???164??100???164????0?????0????419??????0?--:--:--?--:--:--?--:--:--???418
100???674??100???674????0?????0????861??????0?--:--:--?--:--:--?--:--:--???861
100????92??100????92????0?????0?????67??????0??0:00:01??0:00:01?--:--:--?????0
???cilium-mesh?ls
cilium-linux-amd64.tar.gz??cilium-linux-amd64.tar.gz.sha256sum??kind-config.yaml
???cilium-mesh?tar?-zxvf?cilium-linux-amd64.tar.gz?
cilium
加載鏡像
在部署 Cilium 的過程中需要一些鏡像，我們可以提前下載后加載到 KIND 的 Node 節(jié)點(diǎn)中。如果你的網(wǎng)絡(luò)比較順暢， 那這一步可以跳過。
???cilium-mesh?ciliumMeshImage=("quay.io/cilium/cilium-service-mesh:v1.11.0-beta.1"?"quay.io/cilium/operator-generic-service-mesh:v1.11.0-beta.1"?"quay.io/cilium/hubble-relay-service-mesh:v1.11.0-beta.1")
???cilium-mesh?for?i?in?${ciliumMeshImage[@]}
do
??docker?pull?$i
??kind?load?docker-image?$i
done
部署 cilium
接下來我們直接使用 Cilium CLI 完成部署。注意這里的參數(shù)。
???cilium-mesh?cilium?install?--version?-service-mesh:v1.11.0-beta.1?--config?enable-envoy-config=true?--kube-proxy-replacement=probe?--agent-image='quay.io/cilium/cilium-service-mesh:v1.11.0-beta.1'?--operator-image='quay.io/cilium/operator-generic-service-mesh:v1.11.0-beta.1'??--datapath-mode=vxlan?
???Auto-detected?Kubernetes?kind:?kind
??Running?"kind"?validation?checks
??Detected?kind?version?"0.12.0"
????using?Cilium?version?"-service-mesh:v1.11.0-beta.1"
???Auto-detected?cluster?name:?kind-kind
???Auto-detected?IPAM?mode:?kubernetes
???Custom?datapath?mode:?vxlan
???Found?CA?in?secret?cilium-ca
???Generating?certificates?for?Hubble...
???Creating?Service?accounts...
???Creating?Cluster?roles...
???Creating?ConfigMap?for?Cilium?version?1.11.0...
???Manual?overwrite?in?ConfigMap:?enable-envoy-config=true
???Creating?Agent?DaemonSet...
???Creating?Operator?Deployment...
??Waiting?for?Cilium?to?be?installed?and?ready...
??Cilium?was?successfully?installed!?Run?'cilium?status'?to?view?installation?health
查看狀態(tài)
在安裝成功后， 可以通過 cilium status命令來查看當(dāng)前 Cilium 的部署情況。
???cilium-mesh?cilium?status
????/ˉˉ\
?/ˉˉ\__/ˉˉ\????Cilium:?????????OK
?\__/ˉˉ\__/????Operator:???????OK
?/ˉˉ\__/ˉˉ\????Hubble:?????????disabled
?\__/ˉˉ\__/????ClusterMesh:????disabled
????\__/
Deployment????????cilium-operator????Desired:?1,?Ready:?1/1,?Available:?1/1
DaemonSet?????????cilium?????????????Desired:?4,?Ready:?4/4,?Available:?4/4
Containers:???????cilium?????????????Running:?4
??????????????????cilium-operator????Running:?1
Cluster?Pods:?????3/3?managed?by?Cilium
Image?versions????cilium?????????????quay.io/cilium/cilium-service-mesh:v1.11.0-beta.1:?4
??????????????????cilium-operator????quay.io/cilium/operator-generic-service-mesh:v1.11.0-beta.1:?1
啟用 Hubble
Hubble 主要是用來提供可觀測能力的。在啟用它之前，需要先加載一個鏡像，如果網(wǎng)絡(luò)暢通可以跳過。
docker.io/envoyproxy/envoy:v1.18.2@sha256:e8b37c1d75787dd1e712ff389b0d37337dc8a174a63bed9c34ba73359dc67da7
然后使用 Cilium CLI 開啟 Hubble ：
???cilium-mesh?cilium?hubble?enable?--relay-image='quay.io/cilium/hubble-relay-service-mesh:v1.11.0-beta.1'?--ui
???Found?CA?in?secret?cilium-ca??????????????????????????????????????????????????????????????????????
??Patching?ConfigMap?cilium-config?to?enable?Hubble...????????????????????
????Restarted?Cilium?pods?????????????????????????????????????????????????????????????????????????????
??Waiting?for?Cilium?to?become?ready?before?deploying?other?Hubble?component(s)...
???Generating?certificates?for?Relay...????????
??Deploying?Relay?from?quay.io/cilium/hubble-relay-service-mesh:v1.11.0-beta.1...
??Deploying?Hubble?UI?from?quay.io/cilium/hubble-ui:v0.8.3?and?Hubble?UI?Backend?from?quay.io/cilium/hubble-ui-backend:v0.8.3...
??Waiting?for?Hubble?to?be?installed...???????
????/ˉˉ\???????????????????????????????
?/ˉˉ\__/ˉˉ\????Cilium:?????????OK????????????????????????????????????????????????????????????????????
?\__/ˉˉ\__/????Operator:???????OK?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
?/ˉˉ\__/ˉˉ\????Hubble:?????????OK????????????????????????????????????????????????????????????????????
?\__/ˉˉ\__/????ClusterMesh:????disabled??????????????????????????????????????????????????????????????
????\__/?????????????????????????????????????????????????????????????????????????????????????????????
?????????????????????????????????????????????????????????????????????????????????????????????????????
DaemonSet?????????cilium?????????????Desired:?4,?Ready:?4/4,?Available:?4/4
Deployment????????cilium-operator????Desired:?1,?Ready:?1/1,?Available:?1/1
Deployment????????hubble-relay???????Desired:?1,?Ready:?1/1,?Available:?1/1
Deployment????????hubble-ui??????????Desired:?1,?Unavailable:?1/1
Containers:???????cilium?????????????Running:?4
??????????????????cilium-operator????Running:?1
??????????????????hubble-relay???????Running:?1
??????????????????hubble-ui??????????Running:?1
Cluster?Pods:?????5/5?managed?by?Cilium
Image?versions????cilium?????????????quay.io/cilium/cilium-service-mesh:v1.11.0-beta.1:?4
??????????????????cilium-operator????quay.io/cilium/operator-generic-service-mesh:v1.11.0-beta.1:?1
??????????????????hubble-relay???????quay.io/cilium/hubble-relay-service-mesh:v1.11.0-beta.1:?1
??????????????????hubble-ui??????????quay.io/cilium/hubble-ui:v0.8.3:?1
??????????????????hubble-ui??????????quay.io/cilium/hubble-ui-backend:v0.8.3:?1
??????????????????hubble-ui??????????docker.io/envoyproxy/envoy:v1.18.2@sha256:e8b37c1d75787dd1e712ff389b0d37337dc8a174a63bed9c34ba73359dc67da7:?1
測試 7 層 Ingress 流量管理
安裝LB
這里我們可以給 KIND 集群中安裝 MetaLB ，以便于我們可以使用 LoadBalancer 類型的 svc 資源（Cilium 會默認(rèn)創(chuàng)建一個 LoadBalancer 類型的 svc）。如果不安裝 MetaLB ，那也可以使用 NodePort 的方式來進(jìn)行替代。
具體過程就不一一介紹了，直接按下述操作步驟執(zhí)行即可。
???cilium-mesh?kubectl?apply?-f?https://raw.githubusercontent.com/metallb/metallb/master/manifests/namespace.yaml?
namespace/metallb-system?created
???cilium-mesh?kubectl?create?secret?generic?-n?metallb-system?memberlist?--from-literal=secretkey="$(openssl?rand?-base64?128)"??
secret/memberlist?created
???cilium-mesh?kubectl?apply?-f?https://raw.githubusercontent.com/metallb/metallb/master/manifests/metallb.yaml
Warning:?policy/v1beta1?PodSecurityPolicy?is?deprecated?in?v1.21+,?unavailable?in?v1.25+
podsecuritypolicy.policy/controller?created
podsecuritypolicy.policy/speaker?created
serviceaccount/controller?created
serviceaccount/speaker?created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller?created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker?created
role.rbac.authorization.k8s.io/config-watcher?created
role.rbac.authorization.k8s.io/pod-lister?created
role.rbac.authorization.k8s.io/controller?created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller?created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker?created
rolebinding.rbac.authorization.k8s.io/config-watcher?created
rolebinding.rbac.authorization.k8s.io/pod-lister?created
rolebinding.rbac.authorization.k8s.io/controller?created
daemonset.apps/speaker?created
deployment.apps/controller?created
???cilium-mesh?docker?network?inspect?-f?'{{.IPAM.Config}}'?kind
[{172.18.0.0/16??172.18.0.1?map[]}?{fc00:f853:ccd:e793::/64??fc00:f853:ccd:e793::1?map[]}]
???cilium-mesh?vim?kind-lb-cm.yaml
???cilium-mesh?cat?kind-lb-cm.yaml?
apiVersion:?v1
kind:?ConfigMap
metadata:
??namespace:?metallb-system
??name:?config
data:
??config:?|
????address-pools:
????-?name:?default
??????protocol:?layer2
??????addresses:
??????-?172.18.255.200-172.18.255.250
???cilium-mesh?kubectl?apply??-f?kind-lb-cm.yaml
configmap/config?created
加載鏡像
這里我們使用 hashicorp/http-echo:0.2.3作為示例程序，它們可以按照啟動參數(shù)的不同響應(yīng)不同的內(nèi)容。
???cilium-mesh?docker?pull?hashicorp/http-echo:0.2.3
0.2.3:?Pulling?from?hashicorp/http-echo
86399148984b:?Pull?complete?
Digest:?sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96
Status:?Downloaded?newer?image?for?hashicorp/http-echo:0.2.3
docker.io/hashicorp/http-echo:0.2.3
???cilium-mesh?kind?load?docker-image?hashicorp/http-echo:0.2.3?
Image:?"hashicorp/http-echo:0.2.3"?with?ID?"sha256:a6838e9a6ff6ab3624720a7bd36152dda540ce3987714398003e14780e61478a"?not?yet?present?on?node?"kind-worker",?loading...
Image:?"hashicorp/http-echo:0.2.3"?with?ID?"sha256:a6838e9a6ff6ab3624720a7bd36152dda540ce3987714398003e14780e61478a"?not?yet?present?on?node?"kind-worker2",?loading...
Image:?"hashicorp/http-echo:0.2.3"?with?ID?"sha256:a6838e9a6ff6ab3624720a7bd36152dda540ce3987714398003e14780e61478a"?not?yet?present?on?node?"kind-control-plane",?loading...
Image:?"hashicorp/http-echo:0.2.3"?with?ID?"sha256:a6838e9a6ff6ab3624720a7bd36152dda540ce3987714398003e14780e61478a"?not?yet?present?on?node?"kind-worker3",?loading...
部署測試服務(wù)
本文中的所有配置文件均可在 https://github.com/tao12345666333/practical-kubernetes/tree/main/cilium-mesh 代碼倉庫中獲取。
我們使用如下配置進(jìn)行測試服務(wù)的部署：
apiVersion:?v1
kind:?Pod
metadata:
??labels:
????run:?foo-app
??name:?foo-app
spec:
??containers:
??-?image:?hashicorp/http-echo:0.2.3
????args:
????-?"-text=foo"
????name:?foo-app
????ports:
????-?containerPort:?5678
????resources:?{}
??dnsPolicy:?ClusterFirst
??restartPolicy:?Always
status:?{}
---
apiVersion:?v1
kind:?Service
metadata:
??labels:
????run:?foo-app
??name:?foo-app
spec:
??ports:
??-?port:?5678
????protocol:?TCP
????targetPort:?5678
??selector:
????run:?foo-app
---
apiVersion:?v1
kind:?Pod
metadata:
??labels:
????run:?bar-app
??name:?bar-app
spec:
??containers:
??-?image:?hashicorp/http-echo:0.2.3
????args:
????-?"-text=bar"
????name:?bar-app
????ports:
????-?containerPort:?5678
????resources:?{}
??dnsPolicy:?ClusterFirst
??restartPolicy:?Always
---
apiVersion:?v1
kind:?Service
metadata:
??labels:
????run:?bar-app
??name:?bar-app
spec:
??ports:
??-?port:?5678
????protocol:?TCP
????targetPort:?5678
??selector:
????run:?bar-app
新建如下的 Ingress 資源文件：
apiVersion:?networking.k8s.io/v1
kind:?Ingress
metadata:
??name:?cilium-ingress
??namespace:?default
spec:
??ingressClassName:?cilium
??rules:
??-?http:
??????paths:
??????-?backend:
??????????service:
????????????name:?foo-app
????????????port:
??????????????number:?5678
????????path:?/foo
????????pathType:?Prefix
??????-?backend:
??????????service:
????????????name:?bar-app
????????????port:
??????????????number:?5678
????????path:?/bar
????????pathType:?Prefix
創(chuàng)建 Ingress 資源，然后可以看到產(chǎn)生了一個新的 LoadBalancer 類型的 svc 。
???cilium-mesh?kubectl?apply?-f?cilium-ingress.yaml
ingress.networking.k8s.io/cilium-ingress?created
???cilium-mesh?kubectl?get?svc
NAME????????????????????????????TYPE???????????CLUSTER-IP??????EXTERNAL-IP??????PORT(S)????????AGE
bar-app?????????????????????????ClusterIP??????10.96.229.141??????????????5678/TCP???????106s
cilium-ingress-cilium-ingress???LoadBalancer???10.96.161.128???172.18.255.200???80:31643/TCP???4s
foo-app?????????????????????????ClusterIP??????10.96.166.212??????????????5678/TCP???????106s
kubernetes??????????????????????ClusterIP??????10.96.0.1??????????????????443/TCP????????81m
???cilium-mesh?kubectl?get?ing
NAME?????????????CLASS????HOSTS???ADDRESS??????????PORTS???AGE
cilium-ingress???cilium???*???????172.18.255.200???80??????1m
測試
使用 curl 命令進(jìn)行測試訪問，發(fā)現(xiàn)可以按照 Ingress 資源中的配置得到正確的響應(yīng)。查看響應(yīng)頭，我們會發(fā)現(xiàn)這里的代理實際上還是使用的 Envoy 來完成的。
???cilium-mesh?curl?172.18.255.200
???cilium-mesh?curl?172.18.255.200/foo
foo
???cilium-mesh?curl?172.18.255.200/bar
bar
???cilium-mesh?curl?-I?172.18.255.200/bar
HTTP/1.1?200?OK
Content-Length:?4
Connection:?keep-alive
Content-Type:?text/plain;?charset=utf-8
Date:?Sat,?18?Dec?2021?06:02:56?GMT
Keep-Alive:?timeout=4
Proxy-Connection:?keep-alive
Server:?envoy
X-App-Name:?http-echo
X-App-Version:?0.2.3
X-Envoy-Upstream-Service-Time:?0
???cilium-mesh?curl?-I?172.18.255.200/foo
HTTP/1.1?200?OK
Content-Length:?4
Connection:?keep-alive
Content-Type:?text/plain;?charset=utf-8
Date:?Sat,?18?Dec?2021?06:03:01?GMT
Keep-Alive:?timeout=4
Proxy-Connection:?keep-alive
Server:?envoy
X-App-Name:?http-echo
X-App-Version:?0.2.3
X-Envoy-Upstream-Service-Time:?0
測試 CiliumEnvoyConfig
在使用上述方式部署 CIlium 后， 它其實還安裝了一些 CRD 資源。其中有一個是 CiliumEnvoyConfig用于配置服務(wù)之間代理的。
???cilium-mesh?kubectl?api-resources?|grep?cilium.io
ciliumclusterwidenetworkpolicies???ccnp???????????cilium.io/v2???????????????????????????false????????CiliumClusterwideNetworkPolicy
ciliumendpoints????????????????????cep,ciliumep???cilium.io/v2???????????????????????????true?????????CiliumEndpoint
ciliumenvoyconfigs?????????????????cec????????????cilium.io/v2alpha1?????????????????????false????????CiliumEnvoyConfig
ciliumexternalworkloads????????????cew????????????cilium.io/v2???????????????????????????false????????CiliumExternalWorkload
ciliumidentities???????????????????ciliumid???????cilium.io/v2???????????????????????????false????????CiliumIdentity
ciliumnetworkpolicies??????????????cnp,ciliumnp???cilium.io/v2???????????????????????????true?????????CiliumNetworkPolicy
ciliumnodes????????????????????????cn,ciliumn?????cilium.io/v2???????????????????????????false????????CiliumNode
部署測試服務(wù)
可以先進(jìn)行 Hubble 的 port-forward
???cilium-mesh?cilium?hubble?port-forward
默認(rèn)會監(jiān)聽到 4245 端口上，如果不提前執(zhí)行此操作就會出現(xiàn)下述內(nèi)容
???Enabling?Hubble?telescope...
????Unable?to?contact?Hubble?Relay,?disabling?Hubble?telescope?and?flow?validation:?rpc?error:?code?=?Unavailable?desc?=?connection?error:?desc?=?"transport:?Error?while?dialing?dial?tcp?[::1]:4245:?connect:?connection?refused"
如果已經(jīng)開啟 Hubble 的 port-forward ，正常情況下會得到如下輸出：
???cilium-mesh?cilium?connectivity?test?--test?egress-l7
????Monitor?aggregation?detected,?will?skip?some?flow?validation?steps
??[kind-kind]?Waiting?for?deployments?[client?client2?echo-same-node]?to?become?ready...
??[kind-kind]?Waiting?for?deployments?[echo-other-node]?to?become?ready...
??[kind-kind]?Waiting?for?CiliumEndpoint?for?pod?cilium-test/client-6488dcf5d4-pk6w9?to?appear...
??[kind-kind]?Waiting?for?CiliumEndpoint?for?pod?cilium-test/client2-5998d566b4-hrhrb?to?appear...
??[kind-kind]?Waiting?for?CiliumEndpoint?for?pod?cilium-test/echo-other-node-f4d46f75b-bqpcb?to?appear...
??[kind-kind]?Waiting?for?CiliumEndpoint?for?pod?cilium-test/echo-same-node-745bd5c77-zpzdn?to?appear...
??[kind-kind]?Waiting?for?Service?cilium-test/echo-other-node?to?become?ready...
??[kind-kind]?Waiting?for?Service?cilium-test/echo-same-node?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.5:32751?(cilium-test/echo-other-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.5:32133?(cilium-test/echo-same-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.3:32133?(cilium-test/echo-same-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.3:32751?(cilium-test/echo-other-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.2:32751?(cilium-test/echo-other-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.2:32133?(cilium-test/echo-same-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.4:32751?(cilium-test/echo-other-node)?to?become?ready...
??[kind-kind]?Waiting?for?NodePort?172.18.0.4:32133?(cilium-test/echo-same-node)?to?become?ready...
????Skipping?IPCache?check
??[kind-kind]?Waiting?for?pod?cilium-test/client-6488dcf5d4-pk6w9?to?reach?default/kubernetes?service...
??[kind-kind]?Waiting?for?pod?cilium-test/client2-5998d566b4-hrhrb?to?reach?default/kubernetes?service...
???Enabling?Hubble?telescope...
????Hubble?is?OK,?flows:?16380/16380
???Running?tests...
[=]?Skipping?Test?[no-policies]
[=]?Skipping?Test?[allow-all]
[=]?Skipping?Test?[client-ingress]
[=]?Skipping?Test?[echo-ingress]
[=]?Skipping?Test?[client-egress]
[=]?Skipping?Test?[to-entities-world]
[=]?Skipping?Test?[to-cidr-1111]
[=]?Skipping?Test?[echo-ingress-l7]
[=]?Test?[client-egress-l7]
..........
[=]?Skipping?Test?[dns-only]
[=]?Skipping?Test?[to-fqdns]
??All?1?tests?(10?actions)?successful,?10?tests?skipped,?0?scenarios?skipped.
我們也可以同時打開UI看看：
???cilium-mesh?cilium?hubble?ui???
????Opening?"http://localhost:12000"?in?your?browser...
效果圖如下：
這個操作實際上會進(jìn)行如下部署：
???cilium-mesh?kubectl?-n?cilium-test?get?all
NAME??????????????????????????????????READY???STATUS????RESTARTS???AGE
pod/client-6488dcf5d4-pk6w9???????????1/1?????Running???0??????????66m
pod/client2-5998d566b4-hrhrb??????????1/1?????Running???0??????????66m
pod/echo-other-node-f4d46f75b-bqpcb???1/1?????Running???0??????????66m
pod/echo-same-node-745bd5c77-zpzdn????1/1?????Running???0??????????66m
NAME??????????????????????TYPE???????CLUSTER-IP??????EXTERNAL-IP???PORT(S)??????????AGE
service/echo-other-node???NodePort???10.96.124.211???????????8080:32751/TCP???66m
service/echo-same-node????NodePort???10.96.136.252???????????8080:32133/TCP???66m
NAME??????????????????????????????READY???UP-TO-DATE???AVAILABLE???AGE
deployment.apps/client????????????1/1?????1????????????1???????????66m
deployment.apps/client2???????????1/1?????1????????????1???????????66m
deployment.apps/echo-other-node???1/1?????1????????????1???????????66m
deployment.apps/echo-same-node????1/1?????1????????????1???????????66m
NAME????????????????????????????????????????DESIRED???CURRENT???READY???AGE
replicaset.apps/client-6488dcf5d4???????????1?????????1?????????1???????66m
replicaset.apps/client2-5998d566b4??????????1?????????1?????????1???????66m
replicaset.apps/echo-other-node-f4d46f75b???1?????????1?????????1???????66m
replicaset.apps/echo-same-node-745bd5c77????1?????????1?????????1???????66m
我們也可以看看它的 label：
???cilium-mesh?kubectl?get?pods?-n?cilium-test?--show-labels?-o?wide?
NAME??????????????????????????????READY???STATUS????RESTARTS???AGE???IP?????????????NODE???????????NOMINATED?NODE???READINESS?GATES???LABELS
client-6488dcf5d4-pk6w9???????????1/1?????Running???0??????????67m???10.244.3.7?????kind-worker3??????????????????????????kind=client,name=client,pod-template-hash=6488dcf5d4
client2-5998d566b4-hrhrb??????????1/1?????Running???0??????????67m???10.244.3.18????kind-worker3??????????????????????????kind=client,name=client2,other=client,pod-template-hash=5998d566b4
echo-other-node-f4d46f75b-bqpcb???1/1?????Running???0??????????67m???10.244.1.146???kind-worker2??????????????????????????kind=echo,name=echo-other-node,pod-template-hash=f4d46f75b
echo-same-node-745bd5c77-zpzdn????1/1?????Running???0??????????67m???10.244.3.164???kind-worker3??????????????????????????kind=echo,name=echo-same-node,other=echo,pod-template-hash=745bd5c77
測試
這里我們在主機(jī)上進(jìn)行操作下， 先拿到 client2 的 Pod 名稱，然后通過 Hubble 命令觀察所有訪問此 Pod 的流量。
???cilium-mesh?export?CLIENT2=client2-5998d566b4-hrhrb
???cilium-mesh?hubble?observe?--from-pod?cilium-test/$CLIENT2?-f
Dec?18?14:07:37.200:?cilium-test/client2-5998d566b4-hrhrb:44805?<>?kube-system/coredns-78fcd69978-7lbwh:53?to-overlay?FORWARDED?(UDP)
Dec?18?14:07:37.200:?cilium-test/client2-5998d566b4-hrhrb:44805?->?kube-system/coredns-78fcd69978-7lbwh:53?to-endpoint?FORWARDED?(UDP)
Dec?18?14:07:37.200:?cilium-test/client2-5998d566b4-hrhrb:44805?<>?kube-system/coredns-78fcd69978-7lbwh:53?to-overlay?FORWARDED?(UDP)
Dec?18?14:07:37.200:?cilium-test/client2-5998d566b4-hrhrb:44805?->?kube-system/coredns-78fcd69978-7lbwh:53?to-endpoint?FORWARDED?(UDP)
Dec?18?14:07:37.200:?cilium-test/client2-5998d566b4-hrhrb:42260?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-endpoint?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:07:37.201:?cilium-test/client2-5998d566b4-hrhrb:42260?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:07:37.201:?cilium-test/client2-5998d566b4-hrhrb:42260?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:07:37.202:?cilium-test/client2-5998d566b4-hrhrb:42260?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:07:37.203:?cilium-test/client2-5998d566b4-hrhrb:42260?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:07:50.769:?cilium-test/client2-5998d566b4-hrhrb:36768?<>?kube-system/coredns-78fcd69978-7lbwh:53?to-overlay?FORWARDED?(UDP)
Dec?18?14:07:50.769:?cilium-test/client2-5998d566b4-hrhrb:36768?<>?kube-system/coredns-78fcd69978-7lbwh:53?to-overlay?FORWARDED?(UDP)
Dec?18?14:07:50.769:?cilium-test/client2-5998d566b4-hrhrb:36768?->?kube-system/coredns-78fcd69978-7lbwh:53?to-endpoint?FORWARDED?(UDP)
Dec?18?14:07:50.769:?cilium-test/client2-5998d566b4-hrhrb:36768?->?kube-system/coredns-78fcd69978-7lbwh:53?to-endpoint?FORWARDED?(UDP)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?<>?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-overlay?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-endpoint?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?<>?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-overlay?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?<>?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-overlay?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:07:50.770:?cilium-test/client2-5998d566b4-hrhrb:42068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:07:50.771:?cilium-test/client2-5998d566b4-hrhrb:42068?<>?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-overlay?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:07:50.771:?cilium-test/client2-5998d566b4-hrhrb:42068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:07:50.772:?cilium-test/client2-5998d566b4-hrhrb:42068?<>?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-overlay?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:07:50.772:?cilium-test/client2-5998d566b4-hrhrb:42068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-endpoint?FORWARDED?(TCP?Flags:?ACK)
以上輸出是由于我們執(zhí)行了下面的操作：
kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-v?echo-same-node:8080/
kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-v?echo-other-node:8080/
日志中基本上都是 to-endpoint 或者 to-overlay的。
測試使用 proxy
需要先安裝 networkpolicy ， 我們可以直接從 Cilium CLI 的倉庫中拿到。
kubectl?apply?-f?https://raw.githubusercontent.com/cilium/cilium-cli/master/connectivity/manifests/client-egress-l7-http.yaml
kubectl?apply?-f?https://raw.githubusercontent.com/cilium/cilium-cli/master/connectivity/manifests/client-egress-only-dns.yaml
然后重復(fù)上面的請求：
Dec?18?14:33:40.570:?cilium-test/client2-5998d566b4-hrhrb:44344?->?kube-system/coredns-78fcd69978-2ww28:53?L3-L4?REDIRECTED?(UDP)
Dec?18?14:33:40.570:?cilium-test/client2-5998d566b4-hrhrb:44344?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:33:40.570:?cilium-test/client2-5998d566b4-hrhrb:44344?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:33:40.570:?cilium-test/client2-5998d566b4-hrhrb:44344?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-other-node.cilium-test.svc.cluster.local.?A)
Dec?18?14:33:40.570:?cilium-test/client2-5998d566b4-hrhrb:44344?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-other-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?14:33:40.571:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?L3-L4?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?14:33:40.571:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:33:40.571:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:33:40.571:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:33:40.572:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?http-request?FORWARDED?(HTTP/1.1?GET?http://echo-other-node:8080/)
Dec?18?14:33:40.573:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:33:40.573:?cilium-test/client2-5998d566b4-hrhrb:42074?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
執(zhí)行另一個請求：
???cilium-mesh?kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-v?echo-same-node:8080/
也可以看到如下輸出，其中有 to-proxy的字樣。
Dec?18?14:45:18.857:?cilium-test/client2-5998d566b4-hrhrb:58895?->?kube-system/coredns-78fcd69978-2ww28:53?L3-L4?REDIRECTED?(UDP)
Dec?18?14:45:18.857:?cilium-test/client2-5998d566b4-hrhrb:58895?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:45:18.857:?cilium-test/client2-5998d566b4-hrhrb:58895?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:45:18.857:?cilium-test/client2-5998d566b4-hrhrb:58895?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?14:45:18.857:?cilium-test/client2-5998d566b4-hrhrb:58895?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?A)
Dec?18?14:45:18.858:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?L3-L4?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?14:45:18.858:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:45:18.858:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:45:18.858:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:45:18.858:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?http-request?FORWARDED?(HTTP/1.1?GET?http://echo-same-node:8080/)
Dec?18?14:45:18.859:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:45:18.859:?cilium-test/client2-5998d566b4-hrhrb:42266?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
其實看請求頭更加方便：
???cilium-mesh?kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-I?echo-same-node:8080/
HTTP/1.1?403?Forbidden
content-length:?15
content-type:?text/plain
date:?Sat,?18?Dec?2021?14:47:39?GMT
server:?envoy
之前都是如下：
#?沒有?proxy
?? cilium-mesh kubectl exec?-it?-n?cilium-test?$CLIENT2?--?curl?-v?echo-same-node:8080/??????????????
*???Trying?10.96.136.252:8080...?????????????????????????????????????????????????????????????????????
*?Connected?to?echo-same-node?(10.96.136.252)?port?8080?(#0)?????????????????????????????????????????
>?GET?/?HTTP/1.1?????????????????????????????????????????????????????????????????????????????????????
>?Host:?echo-same-node:8080??????????????????????????????????????????????????????????????????????????
>?User-Agent:?curl/7.78.0?????????????????????????
>?Accept:?*/*????????????????????????????????????????????????????????????????????????????????????????
>????????????????????????????????????????????????????????????????????????????????????????????????????
*?Mark?bundle?as?not?supporting?multiuse?????????????????????????????????????????????????????????????
true?????????????????????????????????????????????????????????????
"809-7438674ba0"???????????????????????????????????????????????????????????????????????????
請求一個不存在的地址：
以前請求響應(yīng)是 404 ，現(xiàn)在是 403 ,并得到如下內(nèi)容
???cilium-mesh?kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-v?echo-same-node:8080/foo
*???Trying?10.96.136.252:8080...
*?Connected?to?echo-same-node?(10.96.136.252)?port?8080?(#0)
>?GET?/foo?HTTP/1.1
>?Host:?echo-same-node:8080
>?User-Agent:?curl/7.78.0
>?Accept:?*/*
>?
*?Mark?bundle?as?not?supporting?multiuse
Access?denied
*?Connection?#0?to?host?echo-same-node?left?intact
日志中也都是 to-proxy的字樣。
Dec?18?14:50:39.185:?cilium-test/client2-5998d566b4-hrhrb:37683?->?kube-system/coredns-78fcd69978-7lbwh:53?L3-L4?REDIRECTED?(UDP)
Dec?18?14:50:39.185:?cilium-test/client2-5998d566b4-hrhrb:37683?->?kube-system/coredns-78fcd69978-7lbwh:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:50:39.185:?cilium-test/client2-5998d566b4-hrhrb:37683?->?kube-system/coredns-78fcd69978-7lbwh:53?to-proxy?FORWARDED?(UDP)
Dec?18?14:50:39.185:?cilium-test/client2-5998d566b4-hrhrb:37683?->?kube-system/coredns-78fcd69978-7lbwh:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?14:50:39.185:?cilium-test/client2-5998d566b4-hrhrb:37683?->?kube-system/coredns-78fcd69978-7lbwh:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?A)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?L3-L4?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?http-request?DROPPED?(HTTP/1.1?GET?http://echo-same-node:8080/foo)
Dec?18?14:50:39.186:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?14:50:39.187:?cilium-test/client2-5998d566b4-hrhrb:42274?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
我們使用如下內(nèi)容作為 Envoy 的配置文件，其中包含 rewrite 策略。
apiVersion:?cilium.io/v2alpha1
kind:?CiliumEnvoyConfig
metadata:
??name:?envoy-lb-listener
spec:
??services:
????-?name:?echo-other-node
??????namespace:?cilium-test
????-?name:?echo-same-node
??????namespace:?cilium-test
??resources:
????-?"@type":?type.googleapis.com/envoy.config.listener.v3.Listener
??????name:?envoy-lb-listener
??????filter_chains:
????????-?filters:
????????????-?name:?envoy.filters.network.http_connection_manager
??????????????typed_config:
????????????????"@type":?type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
????????????????stat_prefix:?envoy-lb-listener
????????????????rds:
??????????????????route_config_name:?lb_route
????????????????http_filters:
??????????????????-?name:?envoy.filters.http.router
????-?"@type":?type.googleapis.com/envoy.config.route.v3.RouteConfiguration
??????name:?lb_route
??????virtual_hosts:
????????-?name:?"lb_route"
??????????domains:?["*"]
??????????routes:
????????????-?match:
????????????????prefix:?"/"
??????????????route:
????????????????weighted_clusters:
??????????????????clusters:
????????????????????-?name:?"cilium-test/echo-same-node"
??????????????????????weight:?50
????????????????????-?name:?"cilium-test/echo-other-node"
??????????????????????weight:?50
????????????????retry_policy:
??????????????????retry_on:?5xx
??????????????????num_retries:?3
??????????????????per_try_timeout:?1s
????????????????regex_rewrite:
??????????????????pattern:
????????????????????google_re2:?{}
????????????????????regex:?"^/foo.*$"
??????????????????substitution:?"/"
????-?"@type":?type.googleapis.com/envoy.config.cluster.v3.Cluster
??????name:?"cilium-test/echo-same-node"
??????connect_timeout:?5s
??????lb_policy:?ROUND_ROBIN
??????type:?EDS
??????outlier_detection:
????????split_external_local_origin_errors:?true
????????consecutive_local_origin_failure:?2
????-?"@type":?type.googleapis.com/envoy.config.cluster.v3.Cluster
??????name:?"cilium-test/echo-other-node"
??????connect_timeout:?3s
??????lb_policy:?ROUND_ROBIN
??????type:?EDS
??????outlier_detection:
????????split_external_local_origin_errors:?true
????????consecutive_local_origin_failure:?2
測試請求時，發(fā)現(xiàn)可以正確的得到響應(yīng)了。
???cilium-mesh?kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-X?GET?-I?echo-same-node:8080/?
HTTP/1.1?200?OK
x-powered-by:?Express
vary:?Origin,?Accept-Encoding
access-control-allow-credentials:?true
accept-ranges:?bytes
cache-control:?public,?max-age=0
last-modified:?Sat,?26?Oct?1985?08:15:00?GMT
etag:?W/"809-7438674ba0"
content-type:?text/html;?charset=UTF-8
content-length:?2057
date:?Sat,?18?Dec?2021?15:00:01?GMT
x-envoy-upstream-service-time:?1
server:?envoy
并且請求 /foo地址時，也可以正確的得到響應(yīng)了。
???cilium-mesh?kubectl?exec?-it?-n?cilium-test?$CLIENT2?--?curl?-X?GET?-I?echo-same-node:8080/foo
HTTP/1.1?200?OK
x-powered-by:?Express
vary:?Origin,?Accept-Encoding
access-control-allow-credentials:?true
accept-ranges:?bytes
cache-control:?public,?max-age=0
last-modified:?Sat,?26?Oct?1985?08:15:00?GMT
etag:?W/"809-7438674ba0"
content-type:?text/html;?charset=UTF-8
content-length:?2057
date:?Sat,?18?Dec?2021?15:01:40?GMT
x-envoy-upstream-service-time:?2
server:?envoy
同時：請求 /foo 的時候，流量如下: ?直接轉(zhuǎn)換成功了對/的訪問
Dec?18?15:02:22.541:?cilium-test/client2-5998d566b4-hrhrb:38860?->?kube-system/coredns-78fcd69978-2ww28:53?L3-L4?REDIRECTED?(UDP)
Dec?18?15:02:22.541:?cilium-test/client2-5998d566b4-hrhrb:38860?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:02:22.541:?cilium-test/client2-5998d566b4-hrhrb:38860?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:02:22.541:?cilium-test/client2-5998d566b4-hrhrb:38860?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?15:02:22.541:?cilium-test/client2-5998d566b4-hrhrb:38860?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?A)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?none?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53048?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:02:22.542:?cilium-test/client2-5998d566b4-hrhrb:53048?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?http-request?FORWARDED?(HTTP/1.1?GET?http://echo-same-node:8080/)
Dec?18?15:02:22.543:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?15:02:22.544:?cilium-test/client2-5998d566b4-hrhrb:53062?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
多次請求看日志：
Dec?18?15:07:20.883:?cilium-test/client2-5998d566b4-hrhrb:49656?->?kube-system/coredns-78fcd69978-2ww28:53?L3-L4?REDIRECTED?(UDP)
Dec?18?15:07:20.883:?cilium-test/client2-5998d566b4-hrhrb:49656?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:07:20.883:?cilium-test/client2-5998d566b4-hrhrb:49656?->?kube-system/coredns-78fcd69978-2ww28:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:07:20.883:?cilium-test/client2-5998d566b4-hrhrb:49656?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?A)
Dec?18?15:07:20.884:?cilium-test/client2-5998d566b4-hrhrb:49656?->?kube-system/coredns-78fcd69978-2ww28:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?none?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53064?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:07:20.885:?cilium-test/client2-5998d566b4-hrhrb:53064?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?http-request?FORWARDED?(HTTP/1.1?GET?http://echo-same-node:8080/)
Dec?18?15:07:20.886:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?15:07:20.886:?cilium-test/client2-5998d566b4-hrhrb:53070?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?15:07:26.086:?cilium-test/client2-5998d566b4-hrhrb:53048?->?cilium-test/echo-same-node-745bd5c77-zpzdn:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?15:07:44.739:?cilium-test/client2-5998d566b4-hrhrb:39057?->?kube-system/coredns-78fcd69978-7lbwh:53?L3-L4?REDIRECTED?(UDP)
Dec?18?15:07:44.739:?cilium-test/client2-5998d566b4-hrhrb:39057?->?kube-system/coredns-78fcd69978-7lbwh:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:07:44.740:?cilium-test/client2-5998d566b4-hrhrb:39057?->?kube-system/coredns-78fcd69978-7lbwh:53?to-proxy?FORWARDED?(UDP)
Dec?18?15:07:44.740:?cilium-test/client2-5998d566b4-hrhrb:39057?->?kube-system/coredns-78fcd69978-7lbwh:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?AAAA)
Dec?18?15:07:44.740:?cilium-test/client2-5998d566b4-hrhrb:39057?->?kube-system/coredns-78fcd69978-7lbwh:53?dns-request?FORWARDED?(DNS?Query?echo-same-node.cilium-test.svc.cluster.local.?A)
Dec?18?15:07:44.741:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?none?REDIRECTED?(TCP?Flags:?SYN)
Dec?18?15:07:44.741:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?SYN)
Dec?18?15:07:44.741:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
Dec?18?15:07:44.741:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:07:44.742:?cilium-test/client2-5998d566b4-hrhrb:53068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?PSH)
Dec?18?15:07:44.742:?cilium-test/client2-5998d566b4-hrhrb:53068?->?cilium-test/echo-other-node-f4d46f75b-bqpcb:8080?http-request?FORWARDED?(HTTP/1.1?GET?http://echo-same-node:8080/)
Dec?18?15:07:44.744:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK,?FIN)
Dec?18?15:07:44.744:?cilium-test/client2-5998d566b4-hrhrb:53072?->?cilium-test/echo-same-node:8080?to-proxy?FORWARDED?(TCP?Flags:?ACK)
可以看到它真的成功的進(jìn)行了負(fù)載均衡。
總結(jié)
本文我?guī)悴渴鹆?Cilium Service Mesh，并通過兩個示例，帶你體驗了 Cilium Service Mesh 的工作情況。
整體而言， 這種方式能帶來一定的便利性，但它的服務(wù)間流量配置主要依靠于 CiliumEnvoyConfig ，不算太方便。
一起來期待它后續(xù)的演進(jìn)！
PS:?本文中的所有配置文件均可在 https://github.com/tao12345666333/practical-kubernetes/tree/main/cilium-mesh? 代碼倉庫中獲取。
評論
圖片
表情