Mutual TLS and Istio

Before Start

You should have NO virtualservice, destinationrule, gateway or policy (in tutorial namespace) kubectl get virtualservice kubectl get destinationrule kubectl get gateway kubectl get policy if so run:

./scripts/clean.sh

In this chapter, we are going to see how to secure the communication between all services by enabling mutual TLS in Istio.

Testing mTLS

Check the mTLS by sniffing traffic between services, which is a bit more tedious, open a new terminal tab and run next command:

CUSTOMER_POD=$(oc get pod | grep cust | awk '{ print $1}' ) (1)

or in Kuberentes:

CUSTOMER_POD=$(kubectl get pod | grep cust | awk '{ print $1}' )


oc exec -it $CUSTOMER_POD -c istio-proxy /bin/bash (2)

or in Kubernetes:

kubectl exec -it $CUSTOMER_POD -c istio-proxy /bin/bash

# Inside pod shell

ifconfig (3)
sudo tcpdump -vvvv -A -i eth0 '((dst port 8080) and (net 172.17.0.10))' (4)
1 Get customer pod name
2 Open a shell inside pod
3 Get IP of current pod (probably the IP represented at eth0 interface)
4 Capture traffic from eth0 (or your interface) of port 8080 and network 172.17.0.10 (your IP from ifconfig)

Now all communication that happens between customer service and preference service is dumped in the console.

So now go to a terminal and execute:

curl http://istio-ingressgateway-istio-system.$(minishift ip).nip.io/customer
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

Obviously, the response is exactly the same, but if you go to the terminal where you are executing tcpdump, you should see something like:

14:24:55.078222 IP (tos 0x0, ttl 64, id 32578, offset 0, flags [DF], proto TCP (6), length 967)
    172.17.0.15.33260 > customer-7dcd544ff9-652ds.8080: Flags [P.], cksum 0x5bf5 (incorrect -> 0x595e), seq 2211080917:2211081832, ack 2232186801, win 391, options [nop,nop,TS val 5958433 ecr 5779275], length 915: HTTP
E....B@.@._........
......j...w.....[......
.Z.!.X/K.............w$.?....&T.`n.....UX.C&)Cj....y..{.&..I..	..<.
.....A..q.;...o.9+.4..;...6|".......M.4Wm.:}.....^..v..2..?VW[&s........@}.~B.>D.k..H...r.... .L..i,.
...=..=..y..[.k..g..0..5.f%..vz|..t.....%.`.|...B..%r0.^k.y.....y.@l$O.....?...J..qc&.........Z.^&..F.....w.">7..	...[.......2.&........>......s.....5
.n$X.....l.#...... ..Q..u..jBI.Z.Eb$9.$.._...!.........~"Xx<....);........Z.
.y/E]......K......... .@s.3.\.
.i.v...#.O<..^.F.	...?..:s...).....e......*..F.Kz..i.jk..xx...#....|.U.!.......X.....@......0.....*...l.v..G)T...9...M.....i.H ..=	.a.hp..&8..L..`.s..d_o.~.T ./.......9..	;F81.......S.{.....1rE..o...`..............c+U...}.{7..Y....Q4.#..(.c]Q...[..8..$u.b...=..6.....~..9..H....R
.9x*q....h0......O......q..Fb)..E..m..=.M.....W.Yk>.......;.2eys..E.....=q.;.k ....R.f.(./^F....4.c..*Y.4....es.....TX`nh..L.z.6....(.X.>c.V.0z........GF%.%..l4P.......@.^Q........46.g.#.n...e.k.._..>.T+.S...t}....

Notice that you cannot see any detail about the communication since it happened through TLS.

Now, let’s disable TLS:

kubectl create -f istiofiles/disable-mtls.yml

And execute again:

curl http://istio-ingressgateway-istio-system.$(minishift ip).nip.io/customer
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

And again check tcpdump output:

host: 0192.168.64.70:31380
user-agent: curl/7.54.0
accept: */*
x-forwarded-for: 172.17.0.1
x-forwarded-proto: http
x-envoy-internal: true
x-request-id: e5c0b90f-341b-9edc-ac3e-7dd8b33f0e8b
x-envoy-decorator-operation: customer.tutorial.svc.cluster.local:8080/
x-b3-traceid: ce289e960a639d11
x-b3-spanid: ce289e960a639d11
x-b3-sampled: 1

Now, you can see that since there is no TLS enabled, the information is not shadowed but in clear.

Clean Up

kubectl delete -f istiofiles/disable-mtls.yml

or you can run:

./scripts/clean.sh