Working With Istio: Track Your Services With Kiali

In previous articles, we’ve explained the concept of service meshes, and we’ve also started exploring Istio as an example of the most popular service mesh applications. In this article, we get to start playing with Istio to understand the features and power it provides. We start by visiting one of the tools that ships with Istio: Kiali.

Note: This lab assumes that you have a working Kubernetes cluster, with Istio installed, and the demo configuration profile selected.

Installing The Sample Application

To understand the full potential of Istio, we need a typical microservices application installed on the cluster. Fortunately, Istio already provides one for us. The Bookinfo application is a book rating service. The application is made up of four microservices:

• The Productpage: written in Python, this service displays the book page to the end-user. In doing so, it must display the book information by contacting the details service and the book reviews by contacting the reviews service.
• The Details Service: written in Ruby, it provides book information.
• The Reviews Service: written in Java, it provides book reviews. In doing so, it must contact the rating service to also get the star rating of the book.
• The Ratings Service: written on NodeJS, it provides the star rating count for books.

The client request is fulfilled through communication between all the microservices as depicted in the following diagram:

Now let’s deploy the application to the cluster. If you haven’t already done so, you need to download the latest Istio version from https://github.com/istio/istio/releases/. Uncompress the archive and from inside the directory, issue the following commands:

• kubectl label namespace default istio-injection=enabled. While this command has nothing to do with the Istio directory we’ve just downloaded, it’s a prerequisite for Istio to work correctly. Istio, by default, injects the sidecar container in any namespace that’s labeled

  istio-injection=enabled

• The application deployment can be done by simply applying the definition file:

  kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

• In a few moments, you should see a number of pods and services created in the default namespace. Wait until all the pods are running and ensure that the application is working by sending a simple HTTP request to the productpage service. We can use another pod as the client (for example, ratings):

  kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o ""

• So far the application is working fine, but it’s only accessible from inside the cluster. If you want to reach it through your browser you have two options: either use the kubectl port forwarding feature or deploy an Istio Gateway. An Istio Gateway can be thought of as the traditional Kubernetes Ingress resource except that it offers much more control potential. We won’t delve too much into what Istio Gateways are capable of in this article, we just need to deploy one:

  kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

• Notice that the above step created a Service of the load balancer type. You can double-check by running the following command:

  kubectl -n istio-system get service istio-ingressgateway

• You can quickly get the Ingress host IP address and port by using the following commands:

  export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
  export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')

• Now, the URL can be acquired as follows:

  $ echo http://$INGRESS_HOST:$INGRESS_PORT/productpage
  http://52.146.55.86:80/productpage

Open the above page in your browser, you should see a page similar to the following:

Now that we have our application deployed, let’s take a look at what Kiali can do for us.

Accessing The Kiali Service

Since Kiali is, by default, an internal service, you can access it in either of two ways:

• Using port forwarding:

  kubectl -n istio-system port-forward  svc/kiali  20001:20001

• Convert the service to LoadBalancer:

  kubectl patch service kiali --patch '{"spec":{"type":"LoadBalancer"}}' -n istio-system

  Then get the IP address and port:

  kubectl -n istio-system get service kiali -o jsonpath='{.status.loadBalancer.ingress[0].ip}

   

  kubectl -n istio-system get service kiali -o jsonpath='{.spec.ports[?(@.name=="http-kiali")].port}'

Navigate to the Kiali URL (for example, http://52.150.35.253:20001/kiali/) you should see a page similar to the following:

Use the following credentials to login:

• Username: admin
• Password: admin

The dashboard looks like this:

Traffic Monitoring

Let’s verify that our Bookinfo application is receiving traffic. Execute the following command:

curl -o /dev/null -s -w %{http_code} http://52.146.55.86/productpage

Instead of refreshing the page several times, we use the watch command to continuously send HTTP GET requests to the productpage service using the curl tool. We direct any output to /dev/null, use -s to suppress curl progress status and use -w to show us the HTTP code of the response message. Now, in a few moments, the Kiali dashboard should look as follows:

We see that we have traffic on the default namespace as well as on the istio-system. You can choose which namespaces to show traffic for by typing its name in the filter (Filter by Name).

Viewing The Namespace Graph

You can visualize the services and their connections in a given namespace by clicking on the “Go To Graph” button as shown:

The resulting page should look as follows:

As you can see, the graph that Kiali generated for us is very close to the system diagram that presented when we were explaining the application’s architecture. This is an excellent way to get a general idea of how a given microservices application is working behind the scenes, which services talk to which, etc. Additionally, Kiali gives you even more visibility by showing you the different components of each part. For example, in the above diagram, we have a triangle and a square in the productpage service. The triangle corresponds to a Kubernetes Service while the rectangle represents the application. You can always view a quick reference of the symbols Kiali uses by clicking on the blue “Legend” button at the bottom.

If you have an eye for details, you may be wondering: “in the architecture diagram, only the reviews API versions 2 and 3 had active communication with the ratings API, why isn’t this shown by Kiali?” Fortunately, you can view that detail by selecting the “Versioned app graph” from the dropdown list as shown:

Now, the view should look like:

Changing Load-Balancing Weights

We mentioned before that Istio provides you with more control over how you want to load balance your traffic. So, while Kubernetes offers only the round-robin method of traffic distribution, through Kiali we can configure weighted routing. In weighted routing, you get to choose which backend service gets more traffic than the others. This is useful in scenarios where you need to do Canary Testing, where a small amount of traffic is routed to the app version with new features while the majority is directed to the old version. If no complaints were received and everything goes OK, more traffic is gradually diverted to the new app version. Assuming that v3 of the reviews app contains the new feature that we need to test, we can direct less traffic to it as follows:

1. Verify that you are in the “versioned app graph,” and you’re still generating traffic to the application.
2. Select the “Requests percentage” from the Edge Labels dropdown list as shown below:
   
3. You’ll immediately notice that traffic routing percentages are shown on the graph:
   
4. Click on the “reviews” service (the triangle). Then, on the right panel, click on the service name (reviews) as shown:
   
5. You’ll be directed to the Services page with the reviews already selected. Click on Actions on the right-hand panel and select “Create Weighted Routing.”
   
6. With v1 of the reviews app not sending any traffic to the ratings service, it appears to be deprecated. Let’s send zero traffic to it. V2 of the ratings app should have most of the traffic while v3 (the featured version) needs to have a smaller amount of connections until we finish our Canary test. Adjust the sliders as shown and click Create:
   
7. If you go back to the graph, you should see that Istio is gradually adjusting traffic routing to match your preference:
   

In a few moments, you should see that v1 has no traffic, v2 has 70%, and v3 30%.

Troubleshooting Outages

“The application is acting weird,” “We get random error messages,” and “The UI does not update in response to the user actions.” Those are sample complaints that you’d frequently hear when operating an application that has issues. Troubleshooting the problem is even worse when the application is using the microservices architecture. There are just too many moving parts and you need to examine each one separately to ensure that it’s working as expected. With dozens of microservices, this can quickly turn into a nightmare. Fortunately, Istio has got you covered. The Kiali application can give you great insight into how your services are doing and which ones are failing. Now, to reinforce what we’ve discussed, let’s work through a practical hands-on:

1. Edit the ratings service changing its port from 9080 to 9090.
2. Since the dependent services are programmed to communicate with this service on port 9080, the connection would timeout with no response.
3. Depending on how the application is designed, you may start experiencing some peculiar behavior: ratings are incorrect, or not shown at all.
4. Looking at the Kiali dashboard, you should see something similar to the following:
5. At first glance, you may think that there’s a problem in the reviews service since all the red lines are concentrated there. However, when you look closer, you find that the lines going to the ratings service are gray. Gray lines mean that there’s no communication with the ratings app. The service is either down or isn’t listening at the designated port.
6. Let’s reset the ratings service port back to 9080. Now, everything should get back to normal in a few seconds.
   

TL;DR

• Istio is one of the most well-known service mesh projects. Throughout this article, we installed the sample application that ships with Istio, the BookInfo app. The application is a good example of a typical microservices application with multiple atomic services interconnected.
• Having installed Istio, (and the sample app) we can start to sense the power Istio provides us through one of the services that it ships with: Kiali.
• With Kiali, you have a great deal of observability not only of the application structure (which services talk to wich, the API versions, the ports, etc.) but also which services are down or misbehaving.
• Kiali offers a valuable weighed routing feature, which allows you to make use of advanced deployment techniques like Canary testing.tio