Istio实践

本文主要涉及Istio学习过程中使用过的一些yaml文件的整理,便于后续作为模板快速拷贝^_^

组件调试

在Istio使用中,遇到被悲催的事莫过于:”配置下发了,但是系统中流量规则始终不生效”(这个直接影响到业务,所以真心要在生产环境中慎用istio)。此时,需要对istio中各组件的工作原理以及其debug方式有一定的了解,下面是一些简单的信息收集方式。

Pilot-Discovery

Pilot启动以后,监听端口 15010(gRPC)和 8080(HTTP)。当应用的Sidecar(Envoy,Istio-Proxy)启动以后,它将会连接 pilot.istio-system:15010,获取初始配置,并保持长连接。

调试方式为:

1
2
3
4
5
6
7
8
9
10
11
PILOT=istio-pilot.istio-system:9093

# What is sent to envoy
# Listeners and routes
curl $PILOT/debug/adsz

# Endpoints
curl $PILOT/debug/edsz

# Clusters
curl $PILOT/debug/cdsz

这里是代码中对应可以用于调试的URL信息:

Envoy

Envoy作为客户端,是通过静态配置一个对应的xds server来通过gRPC与server建立长连接并主动获取配置的。对于XDS server这块,有java和golang的control-plane项目,可以实现配置的集中管理。Envoy启动之后,监听 15000 来作为本地的admin端口, 使用15001作为本地的业务端口。

调试的时候,可以通过以下命令来在istio-proxy中获取到动态配置的内容:

1
curl http://127.0.0.1:15000/config_dump > config_dump

network.istio.io

主要涉及virtualService,destinationRule,gateway以及serviceEntry这几个概念的理解。

VirtualServices

按权重比例路由

需要结合destinationRule先指定subsets。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
http:
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1
weight: 70
- destination:
host: flaskapp.default.svc.cluster.local
subset: v2
weight: 30

按HTTP header匹配

http报文header带lab:canary的走v2, 其他走v1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
http:
- match:
- headers:
lab:
exact: canary
route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v2
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1

按labels路由

基于source service的labels走v2, 其他走v1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
http:
- match:
- sourceLabels:
app: sleep
version: v1
route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v2
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1

URI重定向

基于URI做重定向,将”/env/HOSTNAME”替换为对”/env/version”的访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
http:
- match:
- sourceLabels:
app: sleep
version: v1
uri:
exact: "/env/HOSTNAME"
redirect:
uri: /env/version
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1

URI重写

基于uri做rewrite, 和redirect不同的是:必须包含route,且不能够和redirect共存

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
http:
- match:
- uri:
exact: "/get"
rewrite:
uri: /post
route:
- destination:
host: flaskapp.default.svc.cluster.local
- route:
- destination:
host: flaskapp.default.svc.cluster.local

超时

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.default.svc.cluster.local
http:
- timeout: 3s
route:
- destination:
host: httpbin.default.svc.cluster.local

重试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.default.svc.cluster.local
http:
- route:
- destination:
host: httpbin.default.svc.cluster.local
retries:
attempts: 3
perTryTimeout: 1s
timeout: 7s

故障注入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.default.svc.cluster.local
http:
- route:
- destination:
host: httpbin.default.svc.cluster.local
fault:
delay:
fixDelay: 3s
percent: 100

中断注入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.default.svc.cluster.local
http:
- route:
- destination:
host: httpbin.default.svc.cluster.local
fault:
abort:
httpStaus: 500
percent: 100

mirror

将流量路由到v1,同时,mirror一份到v2。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.default.svc.cluster.local
http:
- route:
- destination:
host: httpbin.default.svc.cluster.local
subsets: v1
mirror:
host: httpbin.default.svc.cluster.local
subsets: v2

DestinationRules

设置subsets

1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: flaskapp
spec:
host: flaskapp
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2

熔断设置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin
spec:
host: httpbin
trafficPolicy:
connectionPool:
tcp:
maxConnections: 1
http:
httplMaxPendingRequests: 1
maxRequestsPerConnection: 1
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 3m
maxEjectionPercent: 100

Gateway

网关例子

只会接受请求,但不知道路由到哪里,需要配合virtualService来设置。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: example-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*.microservice.rocks"
- "*.microservece.xyz"

指定路由

virtualService配合gateway路由流量到后端服务。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
- flaskapp.microservice.rocks
gateways:
- mesh
- example-gateway
http:
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1

证书支持

  1. 创建secret文件
1
kubectl create -n istio-system secret tls istio-ingressgateway-certs --key rocks/key.pem --cert rocks/cert.pem
  1. 将secret挂载到/etc/istio/ingressgateway-ca-certs目录
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: example-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*.microservice.rocks"
- "*.microservece.xyz"
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
serviceCertificate: /etc/istio/ingressgateway-ca-certs/tls.crt
privateKey: /etc/istio/ingressgateway-ca-certs/tls.key
hosts:
- "flask.microservice.rocks"
- "flask.microservece.xyz"

virtualService match网关

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: flaskapp
spec:
hosts:
- flaskapp.default.svc.cluster.local
- flaskapp.microservice.rocks
gateways:
- mesh
- example-gateway
http:
- match:
- gateways:
- example-gateway
route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v2
- route:
- destination:
host: flaskapp.default.svc.cluster.local
subset: v1

serviceEntry

创建条目

通过hosts来匹配路由,也需要通过virtualService配合。

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number : 80
name: http
protocol: HTTP
resolution: DNS

匹配并设置超时

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin-service
spec:
hosts:
- httpbin.org
http:
- timeout: 3s
route:
- destination:
host: httpbin.org

config.istio.io

Mixer

  • Instance: 声明一个模板, 用模板将传给 Mixer 的数据转换为适合特定适配器的输出格式
  • Handler: 声明一个适配器的配置
  • Rule: 将Instance和Handler连接起来,确认处理关系

handler列表

名称 作用
Denier 根据自定义条件判断是否拒绝服务
Fluentd 向Fluentd服务提交日志
List 用于执行白名单或者黑名单检查
MemQuota 以内存为存储后端,提供简易的配额控制功能
Prometheus 为Prometheus提供Istio的监控指标
RedisQuota 基于Redis存储后端,提供配额管理功能
StatsD 向StatsD发送监控指标
Stdio 用于在本地输出日志和指标

denier

  • handler
1
2
3
4
5
6
7
8
apiVersion: "config.istio.io/v1alpha2" 
kind: denier
metadata:
name: code-7
spec :
status:
code: 7
message: Not allowed
  • instance
1
2
3
4
5
apiVersion: "config.istio.io/v1alpha2" 
kind: checknothing
metadata:
name: palce-holder
spec:
  • rule
1
2
3
4
5
6
7
8
9
apiVersion: "config.istio.io/v1alpha2" 
kind: rule
metadata:
name: deny-sleep-v1-to-httpbin
spec:
match: destination.labels["app"] == "httpbin" && source.labels["app"] == "sleep" && source.labels["version"] == "v1"
actions:
- handler: code-7.denier
instances: [place-holder.checknothing]

listchecker

  • handler
1
2
3
4
5
6
7
apiVersion: config.istio.io/v1alpha2 
kind: listchecker
metadata:
name: chaos
spec:
overrides: ["v1","v3"]
blacklist: true
  • instance
1
2
3
4
5
6
apiVersion: config.istio.io/v1alpha2 
kind: listentry
metadata:
name: version
spec:
value: source.labels["version"]
  • rule
1
2
3
4
5
6
7
8
9
10
apiVersion: "config.istio.io/v1alpha2" 
kind: rule
metadata:
name: checkversion
spec:
match: destination.labels["app"]=="httpbin"
actions:
- handler: chaos.listchecker
instances:
- version.listentry

prometheus

  • instance
1
2
3
4
5
6
7
8
[root@k8s istio]# kci get metrics
NAME AGE
requestcount 3d1h
requestduration 3d1h
requestsize 3d1h
responsesize 3d1h
tcpbytereceived 3d1h
tcpbytesent 3d1h
  • handler
1
2
3
[root@k8s istio]# kci get prometheus
NAME AGE
handler 3d1h
  • rules
1
2
3
4
[root@k8s istio]# kci get rules
NAME AGE
promhttp 3d1h
promtcp 3d1h

log(stdio)

满足条件的访问日志会被记录到mixer的stdout上面,通过kubectl logs查看。

1
2
3
4
[root@k8s istio]# kci get logentry 
NAME AGE
accesslog 3d1h
tcpaccesslog 3d1h
1
2
3
[root@k8s istio]# kci get stdio
NAME AGE
handler 3d1h
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[root@k8s istio]# kci get rules
NAME AGE
stdio 3d1h

[root@k8s istio]# kci get rule stdio -o yaml
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"config.istio.io/v1alpha2","kind":"rule","metadata":{"annotations":{},"name":"stdio","namespace":"istio-system"},"spec":{"actions":[{"handler":"handler.stdio","instances":["accesslog.logentry"]}],"match":"context.protocol == \"http\" || context.protocol == \"grpc\""}}
creationTimestamp: "2019-03-31T13:55:50Z"
generation: 1
name: stdio
namespace: istio-system
resourceVersion: "16200"
selfLink: /apis/config.istio.io/v1alpha2/namespaces/istio-system/rules/stdio
uid: b1935286-53bc-11e9-ad8b-525400ff729a
spec:
actions:
- handler: handler.stdio
instances:
- accesslog.logentry
match: context.protocol == "http" || context.protocol == "grpc"

log(fluentd)

需要先在k8s上部署fluentd,并暴露service为fluentd-listener:24224

  • handler
1
2
3
4
5
6
apiVersion: config.istio.io/v1alpha2
kind: fluentd
metadata:
name: handler
spec:
address: "fluentd-listener:24224"
  • instance

定义logentry(sleep-log)并应用;

  • rule
1
2
3
4
5
6
7
8
9
10
apiVersion: config.istio.io/v1alpha2 
kind: rule
metadata:
name: fluentd
spec:
actions:
- handler: handler.fluentd
instances:
- sleep-log.logentry
match: context.protocol == "http" && source.labels["app"] == "sleep"
0%