eShopOnContainers 知多少[12]:Envoy gateways
1. 引言
在最新的eShopOnContainers ?3.0 中Ocelot 網(wǎng)關(guān)被Envoy Proxy 替換。下面就來簡要帶大家了解下Envoy,并嘗試梳理下為什么要使用Envoy替代Ocelot。
2. Hello EnvoyENVOY IS AN OPEN SOURCE EDGE AND SERVICE PROXY, DESIGNED FOR CLOUD-NATIVE APPLICATIONS.Enovy(信使) 是一款開源的專為云原生應(yīng)用設(shè)計的服務(wù)代理。
2.1. 快速體驗(yàn)
首先基于本地Dockers快速體驗(yàn)一下,先啟動本地Docker-Desktop,拉取Envoy鏡像:
>?docker?search?envoy-dev
NAME????????????????????????DESCRIPTION?????????????????????????????????????STARS???????????????OFFICIAL????????????AUTOMATED
envoyproxy/envoy????????????Images?for?tagged?releases.?Use?envoy-dev?fo…???96
>?docker?image?pull?envoyproxy:envoy-dev
latest:?Pulling?from?envoyproxy/envoy-dev
171857c49d0f:?Pull?complete
419640447d26:?Pull?complete
61e52f862619:?Pull?complete
3f2a8c910457:?Pull?complete
b2ce823b3fd3:?Pull?complete
ec09faba9bc7:?Pull?complete
b0b9168845d0:?Pull?complete
39a220277151:?Pull?complete
9081a11f5983:?Pull?complete
1880b475bc3a:?Pull?complete
Digest:?sha256:cd8dbbbd8ce4c8c6eb52e4f8eebf55f29d1e597ca8311fecf9eda08b8cca813a
Status:?Downloaded?newer?image?for?envoyproxy/envoy-dev:latest
docker.io/envoyproxy/envoy-dev:latest
該Docker 鏡像將包含最新版本的 Envoy 和一個基本的 Envoy 配置,可以將10000端口的入站請求路由到www.google.com。下面啟動容器測試:
>?docker?run?-d?--name?envoy?-p?10000:10000?envoyproxy/envoy-dev:latest
27e422f34b389d99e9180e47d8109a19975ccd139f42ac2f4fa9f724906b72f6
>?docker?ps?|?findstr?'envoy'
27e422f34b38????????envoyproxy/envoy-dev:latest???"/docker-entrypoint.?????2?minutes?ago????????Up?2?minutes???????0.0.0.0:10000->10000/tcp???envoy
>?curl?-I?http://localhost:10000
HTTP/1.1?200?OK
content-type:?text/html;?charset=ISO-8859-1
p3p:?CP="This?is?not?a?P3P?policy!?See?g.co/p3phelp?for?more?info."
date:?Sat,?17?Oct?2020?04:38:38?GMT
server:?envoy
x-xss-protection:?0
x-frame-options:?SAMEORIGIN
expires:?Sat,?17?Oct?2020?04:38:38?GMT
cache-control:?private
set-cookie:?1P_JAR=2020-10-17-04;?expires=Mon,?16-Nov-2020?04:38:38?GMT;?path=/;?domain=.google.com;?Secure
set-cookie:?NID=204=h0EoJXNOTbQA11L-tVowqcwloS0-BCTR71IeN4irsmpubdPIIS4sU8Gco79pt1NhONAxxFdUJ46SKvbX4Ni-jKMWbSW0k_kn3fFkVrfLm7OOBbAtUWtxGGOCRJGbSNIRyOPfDB7_wMngEWW3yoFEs9diSCtZK9DWFZdtJJZtWuI;?expires=Sun,?18-Apr-2021?04:38:38?GMT;?path=/;?domain=.google.com;?HttpOnly
alt-svc:?h3-Q050=":443";?ma=2592000,h3-29=":443";?ma=2592000,h3-27=":443";?ma=2592000,h3-T051=":443";?ma=2592000,h3-T050=":443";?ma=2592000,h3-Q046=":443";?ma=2592000,h3-Q043=":443";?ma=2592000,quic=":443";?ma=2592000;?v="46,43"
x-envoy-upstream-service-time:?37
transfer-encoding:?chunked
PS: 請確保本地機(jī)器能訪問Google,否則curl -I http://localhost:10000 會出錯。
接下來我們進(jìn)入容器內(nèi)部,查看下配置文件,默認(rèn)路徑為/etc/envoy/envoy.yaml:
docker?exec?-it?envoy?/bin/bash
root@27e422f34b38:/#?cat?/etc/envoy/envoy.yaml
admin:
??access_log_path:?/tmp/admin_access.log
??address:
????socket_address:
??????protocol:?TCP
??????address:?127.0.0.1
??????port_value:?9901
static_resources:
??listeners:
??-?name:?listener_0
????address:
??????socket_address:
????????protocol:?TCP
????????address:?0.0.0.0
????????port_value:?10000
????filter_chains:
????-?filters:
??????-?name:?envoy.filters.network.http_connection_manager
????????typed_config:
??????????"@type":?type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
??????????stat_prefix:?ingress_http
??????????route_config:
????????????name:?local_route
????????????virtual_hosts:
????????????-?name:?local_service
??????????????domains:?["*"]
??????????????routes:
??????????????-?match:
??????????????????prefix:?"/"
????????????????route:
??????????????????host_rewrite_literal:?www.google.com
??????????????????cluster:?service_google
??????????http_filters:
??????????-?name:?envoy.filters.http.router
??clusters:
??-?name:?service_google
????connect_timeout:?30s
????type:?LOGICAL_DNS
????#?Comment?out?the?following?line?to?test?on?v6?networks
????dns_lookup_family:?V4_ONLY
????lb_policy:?ROUND_ROBIN
????load_assignment:
??????cluster_name:?service_google
??????endpoints:
??????-?lb_endpoints:
????????-?endpoint:
????????????address:
??????????????socket_address:
????????????????address:?www.google.com
????????????????port_value:?443
????transport_socket:
??????name:?envoy.transport_sockets.tls
??????typed_config:
????????"@type":?type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
????????sni:?www.google.com
我們把上面的配置文件拷貝到本地,將上面的www.google.com改為www.baidu.com,將admin.address.socket_address.address: 127.0.0.1改為0.0.0.0,然后把配置文件命名為envoy-baidu.yaml,然后掛載到容器的/etc/envoy/envoy.yaml。
>?docker?run?--rm?-d?--name?envoy-baidu-v?$Home/k8s/envoy-baidu.yaml:/etc/envoy/envoy.yaml?-p?9901:9901?-p?15001:15001?envoyproxy/envoy-dev:latest
>?docker?ps?|?findstr?'envoy'
f07f6a1e9305????????envoyproxy/envoy-dev:latest???"/docker-entrypoint.?????2?minutes?ago???????Up?2?minutes????????10000/tcp,?0.0.0.0:9901->9901/tcp,?0.0.0.0:15001->15001/tcp???envoy-baidu
3cd12b5f6ddd????????envoyproxy/envoy-dev:latest???"/docker-entrypoint.?????About?an?hour?ago???Up?About?an?hour????0.0.0.0:10000->10000/tcp??????????????envoy
>?curl?-I?http://localhost:15001
HTTP/1.1?200?OK
accept-ranges:?bytes
cache-control:?private,?no-cache,?no-store,?proxy-revalidate,?no-transform
content-length:?277
content-type:?text/html
date:?Sat,?17?Oct?2020?05:41:01?GMT
etag:?"575e1f65-115"
last-modified:?Mon,?13?Jun?2016?02:50:13?GMT
pragma:?no-cache
server:?envoy
x-envoy-upstream-service-time:?24
使用瀏覽器訪問http://localhost:9901即可訪問envoy管理頁面,如下圖所示:
2.2. 配置簡介
第一次看Envoy的配置文件,和第一次接觸Nginx的配置文件一樣,絕對一臉懵逼。沒關(guān)系,咱們來理一理。
作為一個代理,不管是Nginx、HAProxy,還是Envoy,其處理流程都是一樣的。其首先都是要監(jiān)聽指定端口獲取請求流量,然后分析請求數(shù)據(jù),進(jìn)行請求轉(zhuǎn)發(fā)。腦補(bǔ)完大致流程后,再來看 Envoy 是如何組織配置信息的。先來了幾個核心配置:
- listener : Envoy 的監(jiān)聽地址,用來接收請求,處理入站請求。Envoy 會暴露一個或多個 Listener 來監(jiān)聽客戶端的請求。
- filter : 過濾器是處理入站和出站流量的鏈?zhǔn)浇Y(jié)構(gòu)的一部分。在過濾器鏈上可以集成很多特定功能的過濾器,例如,通過集成 GZip 過濾器可以在數(shù)據(jù)發(fā)送到客戶端之前壓縮數(shù)據(jù)。
- route_config : 路由規(guī)則配置。即將請求路由到后端的哪個集群。
- cluster : 集群定義了流量的目標(biāo)端點(diǎn),同時還包括一些其他可選配置,如負(fù)載均衡策略等。
整體流程如下圖所示:
圖片來源:https://fuckcloudnative.io/envoy-handbook/docs/gettingstarted/quick-start/2.3. 代理 ASP.NET Core WebApi
有了上面的基礎(chǔ),下面嘗試使用Envoy代理ASP.NET Core WebApi。首先創(chuàng)建兩個簡單API,然后創(chuàng)建一個Envoy配置文件,最后通過docker compose啟動三個容器進(jìn)行測試。由于項(xiàng)目文件結(jié)構(gòu)簡單,這里不再過多闡述,主要包含四個部分:
- City Api
- Weather Api
- Envoy 代理配置
- docker compose 配置
整體解決方案如下圖所示。源碼路徑:K8S.NET.Envoy。

Envoy 代理配置基于第一節(jié)的基礎(chǔ)上進(jìn)行修改,如下所示:
admin:
??access_log_path:?/tmp/admin_access.log
??address:
????socket_address:
??????protocol:?TCP
??????address:?0.0.0.0
??????port_value:?9903
static_resources:
??listeners:
????-?name:?listener_0
??????address:
????????socket_address:
??????????protocol:?TCP
??????????address:?0.0.0.0
??????????port_value:?10003
??????filter_chains:
????????-?filters:
????????????-?name:?envoy.filters.network.http_connection_manager
??????????????typed_config:
????????????????"@type":?type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
????????????????stat_prefix:?ingress_http
????????????????route_config:
??????????????????name:?local_route
??????????????????virtual_hosts:
????????????????????-?name:?local_service
??????????????????????domains:?["*"]
??????????????????????routes:
????????????????????????-?match:
????????????????????????????prefix:?"/c"
??????????????????????????route:
????????????????????????????prefix_rewrite:?"/city"
????????????????????????????cluster:?city_service
????????????????????????-?match:
????????????????????????????prefix:?"/w"
??????????????????????????route:
????????????????????????????prefix_rewrite:?"/weather"
????????????????????????????cluster:?weather_service
????????????????http_filters:
??????????????????-?name:?envoy.filters.http.router
??clusters:
????-?name:?city_service
??????connect_timeout:?0.25s
??????type:?LOGICAL_DNS
??????#?Comment?out?the?following?line?to?test?on?v6?networks
??????dns_lookup_family:?V4_ONLY
??????lb_policy:?ROUND_ROBIN
??????load_assignment:
????????cluster_name:?city_service
????????endpoints:
??????????-?lb_endpoints:
??????????????-?endpoint:
??????????????????address:
????????????????????socket_address:
??????????????????????address:?cityapi
??????????????????????port_value:?80
????-?name:?weather_service
??????connect_timeout:?0.25s
??????type:?LOGICAL_DNS
??????#?Comment?out?the?following?line?to?test?on?v6?networks
??????dns_lookup_family:?V4_ONLY
??????lb_policy:?ROUND_ROBIN
??????load_assignment:
????????cluster_name:?weather_service
????????endpoints:
??????????-?lb_endpoints:
??????????????-?endpoint:
??????????????????address:
????????????????????socket_address:
??????????????????????address:?weatherapi
??????????????????????port_value:?80
以上配置Envoy監(jiān)聽10003端口,通過指定prefix_rewrite重寫前綴,將/c路由至cityapi的/city路徑,將/w路由至weatherapi的/weather路徑。
docker-compose配置如下:
version:?'3'
services:
??envoygateway:
????build:?Envoy/
????ports:
??????-?"9903:9903"
??????-?"10003:10003"
????volumes:
??????-?./Envoy/envoy.yaml:/etc/envoy/envoy.yaml
??cityapi:
????build:?K8S.NET.CityApi/
????ports:
??????-?"8080:80"
????environment:
??????ASPNETCORE_URLS:?"http://+"
??????ASPNETCORE_ENVIRONMENT:?"Development"
??weatherapi:
????build:?K8S.NET.WeatherApi/
????ports:
??????-?"8082:80"
????environment:
??????ASPNETCORE_URLS:?"http://+"
??????ASPNETCORE_ENVIRONMENT:?"Development"
從上可以看到,主要用來啟動三個服務(wù):
- envoy gateway:其中將項(xiàng)目路徑下
/Envoy/envoy.yaml掛載到容器目錄/etc/envoy/envoy.yaml。同時暴露2個端口,9903,10003。 - city api
- weather api
因此最終可以通過以下路徑進(jìn)行訪問:
- http://localhost:10003/c 訪問city api。
- http://localhost:10003/w 訪問weather api。
執(zhí)行以下命令,啟動應(yīng)用和代理,并測試:
>?docker-compose?up?-d
Starting?k8snetenvoy_envoygateway_1?...?done
Starting?k8snetenvoy_cityapi_1??????...?done
Starting?k8snetenvoy_weatherapi_1???...?done
>?docker-compose?ps
???????????Name?????????????????????????Command???????????????State?????????????????????????Ports
-----------------------------------------------------------------------------------------------------------------------
k8snetenvoy_cityapi_1????????dotnet?K8S.NET.CityApi.dll???????Up??????443/tcp,?0.0.0.0:8080->80/tcp
k8snetenvoy_envoygateway_1???/docker-entrypoint.sh?envo?...???Up??????10000/tcp,?0.0.0.0:10003->10003/tcp,
??????????????????????????????????????????????????????????????????????0.0.0.0:9903->9903/tcp
k8snetenvoy_weatherapi_1?????dotnet?K8S.NET.WeatherApi.dll????Up??????443/tcp,?0.0.0.0:8082->80/tcp
>?curl?http://localhost:10003/c
Shanghai
>?curl?http://localhost:10003/w
Cool
3. eShopOnContainers 中的應(yīng)用eShopOnContainer 中主要定義了四個API 網(wǎng)關(guān)(BFF 模式),服務(wù)間通信方式主要有兩種,一種是HTTP,一種是gRPC。如果啟用Service Mesh并且部署至K8S,服務(wù)整體通信架構(gòu)如下圖所示:

有兩點(diǎn)需要補(bǔ)充說明:
- Linkerd是一種Service Mesh,其核心思想是借助Sidecar模式無侵入式對應(yīng)用進(jìn)行服務(wù)治理,包括服務(wù)發(fā)現(xiàn)、流量管理、負(fù)載均衡、路由等。
- 了解過Istio(目前比較流行的Service Mesh)應(yīng)該知道,Envoy在Istio中作為Sidecar而存在,而在eShopOnContainers中Envoy被充當(dāng)API Gateways。
基于上面的基礎(chǔ),再來看eShopOnContainers中的配置,其實(shí)就很明白了,主要是配置文件從Ocelot 轉(zhuǎn)變到envoy.yaml,配置如下圖所示。
路由配置如下:
- /m/ 、/marketing-api/ 路由至:marketing api
- /c/、/catalog-api/ 路由至:catalog api
- /o/、/ordering-api/ 路由至:ordering api
- /b/、/basket-api/ 路由至:basket api
- / 路由至:web bff aggregator api
部署時,基于helm將envoy.yaml保存至ConfigMap,在基于envoyproxy/enovy鏡像構(gòu)建容器,將配置從ConfigMap掛載到容器中,容器內(nèi)部即可基于配置啟動Envoy 網(wǎng)關(guān)了。
經(jīng)過上面的了解發(fā)現(xiàn),Envoy還是充當(dāng)?shù)木W(wǎng)關(guān)角色,那為什么要替換呢?先來了解下Envoy的優(yōu)勢:
非侵入式架構(gòu)?:
Envoy?基于Sidecar模式,是一個獨(dú)立進(jìn)程,對應(yīng)用透明。(在eShopOnContainer中還是獨(dú)立的網(wǎng)關(guān)項(xiàng)目,并非以Sidecar模式注入到服務(wù)中。)
基于C++開發(fā)實(shí)現(xiàn):擁有強(qiáng)大的定制化能力和優(yōu)異的性能。
L3/L4/L7 架構(gòu)?: 傳統(tǒng)的網(wǎng)絡(luò)代理,要么在?
HTTP?層工作,要么在?TCP?層工作。而Envoy?同時支持 3/4 層和 7 層代理。頂級 HTTP/2 支持?: 它將?
HTTP/2?視為一等公民,并且可以在?HTTP/2?和?HTTP/1.1?之間相互轉(zhuǎn)換(雙向),建議使用?HTTP/2。gRPC 支持?:?Envoy 完美支持 HTTP/2,也可以很方便地支持?
gRPC(gRPC?使用?HTTP/2?作為底層多路復(fù)用傳輸協(xié)議)。服務(wù)發(fā)現(xiàn)和動態(tài)配置?: 與?
Nginx?等代理的熱加載不同,Envoy?可以通過?API?接口動態(tài)更新配置,無需重啟代理。特殊協(xié)議支持?: Envoy 支持對特殊協(xié)議在 L7 進(jìn)行嗅探和統(tǒng)計,包括:MongoDB、DynamoDB 等。
可觀測性?:?
Envoy?內(nèi)置?stats?模塊,可以集成諸如?prometheus/statsd?等監(jiān)控方案。還可以集成分布式追蹤系統(tǒng),對請求進(jìn)行追蹤。
再來看下Ocelot:其本質(zhì)還是ASP.NET Core中的一個請求中間件。只能進(jìn)行7層代理,不支持 gRPC,不支持監(jiān)控。因此總體而言,Envoy更契合云原生對網(wǎng)絡(luò)代理的訴求。
5. 總結(jié)本文簡要梳理了Envoy的基本用法,以及其在eShopOnContainers中的運(yùn)用。Envoy作為一個比肩Nginx的服務(wù)代理,其特性在Service Mesh中有著靈活的運(yùn)用。本文就講到這里了,下次有機(jī)會在和大家分享下Envoy在Service Mesh中的應(yīng)用。
參考資料:
- Envoy 介紹 - Envoy 中文指南
- Build an API Gateway with Envoy and use with .NET Core APIs
