Istio canary deployment

Today i will demonstrate the simplest canary deployment with Istio, just to show you how it works.

i am asuming you got istio installed, if not, you can learn how to do that HERE.

What is a canary deployment?

A canary deployment is a progressive rollout of an application that splits traffic between an already-deployed version and a new version, rolling it out to a subset of users before rolling out fully.

1. Gradual rollout: Instead of updating the entire system at once, a small portion of the infrastructure or user base receives the new version.

2. Limited exposure: This “canary” group (named after the historical use of canaries in coal mines to detect danger) tests the new version in a real production environment.

3. Monitoring and analysis: The performance and behavior of the canary deployment are closely monitored and compared to the stable version.

4. Risk mitigation: If issues are detected, the canary can be quickly rolled back, minimizing the impact on the overall system and user base.

5. Full deployment: If the canary performs well, the update is gradually rolled out to the entire system.

This approach allows teams to catch potential problems early, gather real-world feedback, and make data-driven decisions about wider releases.

Install Istio Bookinfo Demo Application.

Now that we have deployed Istio we will deploy the “Bookinfo” demo application that istio providing.

This application consists of several services written in different languages.

Notice that for the reviews service there are 3 pods, one for each version, v1, v2, v3.

Let’s create a namespace, add a label to allow envoy proxy containers to be injected to the namespace pods, and deploy the bookinfo app.

Notice that there are 3 pods for the reviews service, just like in the diagram.

k create ns bookinfo-app
k label ns bookinfo-app istio-injection=enabled
k apply -f istio-1.22.3/samples/bookinfo/platform/kube/bookinfo.yaml -n bookinfo-app

After deploying the app let’s create the istio gateway for the application.

Execute the following yaml, and you will be able to access the application from outside the cluster by using ingress gateway.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  # The selector matches the ingress gateway pod labels.
  # If you installed Istio using Helm following the standard documentation, this would be "istio=ingress"
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 8080
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080

Now get the ingressgateway external ip which is a load balancer

Now you can navigate to apps product page on http://192.168.66.102/productpage

As you see it is a very simple application consists of Summary, Book details, and Book reviews.

Under the Book Reviews section there is a tag that states by what server the reviews are served, currently it’s “v3”.

Implementing Weight based Canary Deployment.

Good, you got your istio, you got your app, every time you hit the F5 button you get the reviews served by different server.
(right bottom corner).

when reviews is served by v3 the rating stars are red, when served by v2 rating stars are black and when served by v1 there are no rating at all, if you go back up on this page to the application diagram you will see that v1 server is not connected to rating service so that’s ok.

Now it’s time to get familiar with Virtual services and Destination Rules.
Those two component will help us to manage traffic in our cluster and implement canary deployment strategy.

Currently our traffic to reviews service is distributed between 3 servers, we will change that and make sure that traffic sent to only v1 service.
First lets create a virtual service with our subsets and traffic distribution weight.

we will create and apply the following VirtualService yaml file after setting the weight for both v2 and v3 subsets to zero, and v1 subset to 100.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 100
    - destination:
        host: reviews
        subset: v2
      weight: 0
    - destination:
        host: reviews
        subset: v3
      weight: 0

k apply -f reviews-virtual-service.yaml -n bookinfo-app

The next step is to create a DestinationRule for our reviews service.

Before deploying open the following file “/home/rke/stmp/istio/istio-1.22.3/samples/bookinfo/networking/destination-rule-reviews.yaml”
and see that there are 3 subsets that are capable of receiving traffic, v1 ,v2 and v3.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  trafficPolicy:
    loadBalancer:
      simple: RANDOM
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3

k apply -f /home/rke/stmp/istio/istio-1.22.3/samples/bookinfo/networking/destination-rule-reviews.yaml -n bookinfo-app

Now go to bookinfo app product page and refresh it a few times, and make sure that from now on you will get only the v1 service.

You can run the following command from your terminal and verify that.

it will curl 20 times the productpage and from the output you can see that it is only directed to v1.

for i in {1..20}; do curl -s http://192.168.66.102/productpage | grep '<u>reviews-v'; done

Now let’s assume that a new version of reviews service is deployed (v2) and you want to start redirecting some traffic to it, how will you do that?

you will change the weight percentage on reviews virtual service and redeploy the yaml.
let’s change it so that 80 percent of the traffic will be sent to v1 service, and 20 percent will be sent to v2 service.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 80
    - destination:
        host: reviews
        subset: v2
      weight: 20
    - destination:
        host: reviews
        subset: v3
      weight: 0

k apply -f reviews-virtual-service.yaml -n bookinfo-app

And now if you refresh the a page you will be redirected to v2 20% of the tries.

and again with the following command you can test it from your terminal and see for yourself.

for i in {1..20}; do curl -s http://192.168.66.102/productpage | grep '<u>reviews-v'; done

Implementing Weight and "user-agent" parameter based Canary Deployment.

We could be more granular about who get’s exposed to v2 subset, for instance we can set a user parameter or browser type.

Let’s try that with a browser type, 20% of the traffic coming from FireFox will be redirected to v2 subset, all other traffic will be still redirected to v1 subset.

Create and apply the following yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - match:
        - headers:
            user-agent:
              regex: '.*Firefox.*'
      route:
        - destination:
            host: reviews
            subset: v2
          weight: 20
        - destination:
            host: reviews
            subset: v1
          weight: 80
    - route:
        - destination:
            host: reviews
            subset: v1

k apply -f /home/rke/stmp/istio/virtualservice-firefox-20-percent-to-v2.yaml -n bookinfo-app

Now you can open two browsers and see the magic for yourself, on a chrome browser 100% of the requests get to v1 subset, and on Firefox browser it is 80% to v1 and 20% to v2.

Alternatively you can stick to CLI, and test it from your terminal,
first let’s send requests from Chrome browser and then from Firefox.
and see for yourself that 20% of the requests coming from Firefox served by v2.

# firefox
for i in {1..20}; do curl -s -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0" http://192.168.66.102/productpage | grep '<u>reviews-v'; done

# chrome
for i in {1..20}; do curl -s -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/92.0.4515.159 Safari/537.36" http://192.168.66.102/productpage | grep '<u>reviews-v'; done

That’s it, you have managed to implement a canary deployment strategy for the bookinfo app reviews service.
cheers :>

From now on 20% of the traffic from Firefox will be sent to v2 service as you can see in the above image.

you can play with the weights and create your own weights, whatever suits you.

i suggest reading about istio traffic management HERE.