Istio实践

发表于 | 分类于 |
阅读时长 ≈ 10 mins.

本文主要涉及Istio学习过程中使用过的一些yaml文件的整理，便于后续作为模板快速拷贝^_^

组件调试

在Istio使用中，遇到被悲催的事莫过于：”配置下发了，但是系统中流量规则始终不生效”（这个直接影响到业务，所以真心要在生产环境中慎用istio）。此时，需要对istio中各组件的工作原理以及其debug方式有一定的了解，下面是一些简单的信息收集方式。

Pilot-Discovery

Pilot启动以后，监听端口 15010(gRPC)和 8080(HTTP)。当应用的Sidecar（Envoy，Istio-Proxy）启动以后，它将会连接 pilot.istio-system:15010，获取初始配置，并保持长连接。

调试方式为：

1
2
3
4
5
6
7
8
9
10
11
PILOT=istio-pilot.istio-system:9093

# What is sent to envoy
# Listeners and routes
curl $PILOT/debug/adsz

# Endpoints
curl $PILOT/debug/edsz

# Clusters
curl $PILOT/debug/cdsz

这里是代码中对应可以用于调试的URL信息：

Envoy

Envoy作为客户端，是通过静态配置一个对应的xds server来通过gRPC与server建立长连接并主动获取配置的。对于XDS server这块，有java和golang的control-plane项目，可以实现配置的集中管理。Envoy启动之后，监听 15000 来作为本地的admin端口， 使用15001作为本地的业务端口。

调试的时候，可以通过以下命令来在istio-proxy中获取到动态配置的内容：

1
curl http://127.0.0.1:15000/config_dump > config_dump

network.istio.io

主要涉及virtualService，destinationRule，gateway以及serviceEntry这几个概念的理解。

VirtualServices

按权重比例路由

需要结合destinationRule先指定subsets。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  http:
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1
      weight: 70
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v2
      weight: 30

按HTTP header匹配

http报文header带lab:canary的走v2, 其他走v1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  http:
  - match:
    - headers:
        lab:
          exact: canary
    route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v2
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1

按labels路由

基于source service的labels走v2, 其他走v1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  http:
  - match:
    - sourceLabels:
        app: sleep
        version: v1
    route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v2
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1

URI重定向

基于URI做重定向，将”/env/HOSTNAME”替换为对”/env/version”的访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  http:
  - match:
    - sourceLabels:
        app: sleep
        version: v1
      uri:
        exact: "/env/HOSTNAME"
    redirect:
      uri: /env/version
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1

URI重写

基于uri做rewrite, 和redirect不同的是：必须包含route，且不能够和redirect共存

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  http:
  - match:
    - uri:
        exact: "/get"
    rewrite:
      uri: /post
    route:
    - destination:
        host: flaskapp.default.svc.cluster.local
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local

超时

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.default.svc.cluster.local
  http:
  - timeout: 3s
    route:
    - destination:
        host: httpbin.default.svc.cluster.local

重试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.default.svc.cluster.local
  http:
  - route:
    - destination:
        host: httpbin.default.svc.cluster.local
    retries:
      attempts: 3
      perTryTimeout: 1s
    timeout: 7s

故障注入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.default.svc.cluster.local
  http:
  - route:
    - destination:
        host: httpbin.default.svc.cluster.local
    fault:
      delay:
        fixDelay: 3s
        percent: 100

中断注入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.default.svc.cluster.local
  http:
  - route:
    - destination:
        host: httpbin.default.svc.cluster.local
    fault:
      abort:
        httpStaus: 500
        percent: 100

mirror

将流量路由到v1，同时，mirror一份到v2。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - httpbin.default.svc.cluster.local
  http:
  - route:
    - destination:
        host: httpbin.default.svc.cluster.local
        subsets: v1
    mirror:
      host: httpbin.default.svc.cluster.local
      subsets: v2

DestinationRules

设置subsets

1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: flaskapp
spec:
  host: flaskapp
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

熔断设置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: httpbin
spec:
  host: httpbin
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 1
      http: 
        httplMaxPendingRequests: 1
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutiveErrors: 1 
      interval: 1s 
      baseEjectionTime: 3m 
      maxEjectionPercent: 100

Gateway

网关例子

只会接受请求，但不知道路由到哪里，需要配合virtualService来设置。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: example-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
      - "*.microservice.rocks"
      - "*.microservece.xyz"

指定路由

virtualService配合gateway路由流量到后端服务。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  - flaskapp.microservice.rocks
  gateways:
  - mesh
  - example-gateway
  http:
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1

证书支持

  1. 创建secret文件
1
kubectl create -n istio-system secret tls istio-ingressgateway-certs --key rocks/key.pem --cert rocks/cert.pem
  1. 将secret挂载到/etc/istio/ingressgateway-ca-certs目录
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: example-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
      - "*.microservice.rocks"
      - "*.microservece.xyz"
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        serviceCertificate: /etc/istio/ingressgateway-ca-certs/tls.crt
        privateKey: /etc/istio/ingressgateway-ca-certs/tls.key
      hosts:
      - "flask.microservice.rocks"
      - "flask.microservece.xyz"

virtualService match网关

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - flaskapp.default.svc.cluster.local
  - flaskapp.microservice.rocks
  gateways:
  - mesh
  - example-gateway
  http:
  - match:
    - gateways:
      - example-gateway
    route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v2
  - route:
    - destination:
        host: flaskapp.default.svc.cluster.local
        subset: v1

serviceEntry

创建条目

通过hosts来匹配路由，也需要通过virtualService配合。

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: httpbin-ext
spec: 
  hosts:
  - httpbin.org 
  ports:
  - number : 80
    name: http
    protocol: HTTP 
  resolution: DNS

匹配并设置超时

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin-service
spec:
  hosts:
  - httpbin.org
  http:
  - timeout: 3s
    route:
    - destination:
        host: httpbin.org

config.istio.io

Mixer

  • Instance: 声明一个模板, 用模板将传给 Mixer 的数据转换为适合特定适配器的输出格式
  • Handler: 声明一个适配器的配置
  • Rule: 将Instance和Handler连接起来，确认处理关系

handler列表

名称 作用
Denier 根据自定义条件判断是否拒绝服务
Fluentd 向Fluentd服务提交日志
List 用于执行白名单或者黑名单检查
MemQuota 以内存为存储后端，提供简易的配额控制功能
Prometheus 为Prometheus提供Istio的监控指标
RedisQuota 基于Redis存储后端，提供配额管理功能
StatsD 向StatsD发送监控指标
Stdio 用于在本地输出日志和指标

denier

  • handler
1
2
3
4
5
6
7
8
apiVersion: "config.istio.io/v1alpha2" 
kind: denier
metadata:
  name: code-7 
spec :
  status:
    code: 7
    message: Not allowed
  • instance
1
2
3
4
5
apiVersion: "config.istio.io/v1alpha2" 
kind: checknothing
metadata:
  name: palce-holder
spec:
  • rule
1
2
3
4
5
6
7
8
9
apiVersion: "config.istio.io/v1alpha2" 
kind: rule
metadata:
  name: deny-sleep-v1-to-httpbin 
spec:
  match: destination.labels["app"] == "httpbin" && source.labels["app"] == "sleep" && source.labels["version"] == "v1"
  actions:
  - handler: code-7.denier
    instances: [place-holder.checknothing]

listchecker

  • handler
1
2
3
4
5
6
7
apiVersion: config.istio.io/v1alpha2 
kind: listchecker
metadata:
  name: chaos
spec:
  overrides: ["v1","v3"] 
  blacklist: true
  • instance
1
2
3
4
5
6
apiVersion: config.istio.io/v1alpha2 
kind: listentry
metadata:
  name: version 
spec:
  value: source.labels["version"]
  • rule
1
2
3
4
5
6
7
8
9
10
apiVersion: "config.istio.io/v1alpha2" 
kind: rule
metadata:
  name: checkversion
spec:
  match: destination.labels["app"]=="httpbin"
  actions:
  - handler: chaos.listchecker
    instances:
    - version.listentry

prometheus

  • instance
1
2
3
4
5
6
7
8
[root@k8s istio]# kci get metrics
NAME              AGE
requestcount      3d1h
requestduration   3d1h
requestsize       3d1h
responsesize      3d1h
tcpbytereceived   3d1h
tcpbytesent       3d1h
  • handler
1
2
3
[root@k8s istio]# kci get prometheus
NAME      AGE
handler   3d1h
  • rules
1
2
3
4
[root@k8s istio]# kci get rules
NAME                     AGE
promhttp                 3d1h
promtcp                  3d1h

log(stdio)

满足条件的访问日志会被记录到mixer的stdout上面，通过kubectl logs查看。

1
2
3
4
[root@k8s istio]# kci get logentry 
NAME           AGE
accesslog      3d1h
tcpaccesslog   3d1h
1
2
3
[root@k8s istio]# kci get stdio
NAME      AGE
handler   3d1h
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[root@k8s istio]# kci get rules
NAME                     AGE
stdio                    3d1h

[root@k8s istio]# kci get rule  stdio -o yaml
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"config.istio.io/v1alpha2","kind":"rule","metadata":{"annotations":{},"name":"stdio","namespace":"istio-system"},"spec":{"actions":[{"handler":"handler.stdio","instances":["accesslog.logentry"]}],"match":"context.protocol == \"http\" || context.protocol == \"grpc\""}}
  creationTimestamp: "2019-03-31T13:55:50Z"
  generation: 1
  name: stdio
  namespace: istio-system
  resourceVersion: "16200"
  selfLink: /apis/config.istio.io/v1alpha2/namespaces/istio-system/rules/stdio
  uid: b1935286-53bc-11e9-ad8b-525400ff729a
spec:
  actions:
  - handler: handler.stdio
    instances:
    - accesslog.logentry
  match: context.protocol == "http" || context.protocol == "grpc"

log(fluentd)

需要先在k8s上部署fluentd，并暴露service为fluentd-listener:24224

  • handler
1
2
3
4
5
6
apiVersion: config.istio.io/v1alpha2
kind: fluentd
metadata:
  name: handler 
spec:
  address: "fluentd-listener:24224"
  • instance

定义logentry(sleep-log)并应用；

  • rule
1
2
3
4
5
6
7
8
9
10
apiVersion: config.istio.io/v1alpha2 
kind: rule
metadata:
  name: fluentd
spec:
  actions:
  - handler: handler.fluentd
    instances:
    - sleep-log.logentry
  match: context.protocol == "http" && source.labels["app"] == "sleep"
0%