Making Kubernetes Pods Talk to Each Other

I’ve finally been looking into Kubernetes properly since of late. When learning about containers, pods and services, what pretty much every doc, article, video tells is that,

  1. Containers in the same pod can just use localhost and the port to communicate.
  2. If a pod needs to talk to another pod, it can do so if it knows the IP address and the exposed port(s) of that pod.
  3. Pods tend to be transient. So depending on their IP address for communicating can make the whole communication process very brittle/unreliable. To solve this, one can create services which will act as an abstraction for the pods. The service will have a single DNS name and it’ll load balance the requests among the pods.

But how exactly how would such an app with multiple pods talking to each other would look like? Let’s try out a simple app that demonstrates the 3rd scenario described above since that’s the recommended approach to go. Let’s consider a simple app with a single endpoint /quote, which would simply return a quote.

Our app would consist of the following services:

  • A quotes service – this would return a quote with various metadata associated with the quote (e.g., author, category, length etc.). To keep it simple, we’ll just call the https://quotes.rest/qod to get the quote and return it from the /quote endpoint of this service.
  • A proxy service – this would sit in between the quotes API and the users and present the users with a simplified response with all the additional metadata etc. stripped from the payload.

For ease of implementation and clarity let’s use Ballerina to implement these 2 services. In addition to having network abstractions built in to the language, it also provides support for generating Docker and Kubernetes artifacts for our applications. But for the purpose of this exercise, we’ll just stick to generating the Docker images.

Continue reading
Advertisement

Making Prolink DH-5102U WiFi Adapter work with Arch Linux

Recently my laptop’s WiFi stopped working and I had to look for an alternative. Decided to go with a USB WiFi adapter since that’s the easiest and the quickest option. The adapter I went for was the Prolink DH-5102U since it supposedly shipped a Linux driver. I’m using both Arch Linux and Windows. So it was pretty important that the adapter works in both environments. In Windows it wasn’t an issue and I was up and running in no time. But getting it set up in Arch Linux turned out to be tricky since the provided Linux driver wasn’t building on this environment. The provided driver was for an older kernel version and apparently not compatible with 5.18.

Upon further inspection it turned out to be that this adapter was based on the RTL8821CU chipset and came across the following repo which maintains a driver for it: https://github.com/brektrou/rtl8821CU. But attempts to build the driver from this was also unsuccessful. But luckily, it turned out that there was a driver maintained in Arch Linux repository for the RTL8821CU chipset: https://aur.archlinux.org/packages/rtl8821cu-dkms-git. Even this is not yet updated to support 5.18. But apparently a user has tried out a fix for it as pointed out in https://aur.archlinux.org/packages/rtl8821cu-dkms-git#comment-872195. I tried it out as instructed in the comment and the driver built and installed without any issue and it’s working! Posting this as a blog post in case anyone else run into trouble with the Prolink WiFi adapters in Linux.

Matrix Multiplication Optimization

As the final lab assignment for our Concurrent Programming module, we were asked to optimize matrix multiplication. C++ was set as the language and Linux was set as the environment for this task. The tasks for this lab assignment were as follows:

  • Write a naive sequential program to perform matrix multiplication on a n x n matrix containing double precision values. The matrices were to be filled with random double values. Only the time taken for the actual multiplication part was to be recorded.
  • Identify a C++ library for Linux environment which supports parallel for loops and write a parallelized version of the above program.
  • Running the above 2 versions of matrix multiplication for matrix sizes of n = 100 to n = 1000 in increments of 100 and record the time taken for each of these executions.
  • We were then supposed to look in to ways to optimize matrix multiplication taking the processor architecture into account as well and come up with a 3rd version of the matrix multiplication making use of both parallel for loops and various optimization techniques.

The full source code of this project can be found here: https://github.com/pubudu91/mm-optimization

Naive Sequential Version

For this, the standard matrix multiplication algorithm was used. If the matrices are called A, B and C,


for (int i = 0; i < n; i++)
for (int j = 0; j < n; j++)
for (int k = 0; k < n; k++)
C[i][j] += A[i][k] * B[k][j];

Continue reading

Clustering WSO2 Governance Registry 5.1.0

Introduction

Recently, I created a Governance Registry cluster on EC2 using Nginx as a load balancer. The cluster created was a 3 node cluster, fronted with Nginx. The initial plan was to create a cluster with 2 Nginx load balancers, one each for the Store components and the Publisher components, as given below.

greg-original

Originally proposed deployment pattern

As the first step in creating the cluster, I was advised to create a cluster (same deployment pattern as above) of just one G-Reg node. Although I successfully managed to deploy the cluster, there was a major issue in it. The scenario for the G-Reg cluster I was creating was that the artifacts would be deployed by the publishers and as such, the Publisher context should only be accessible by publishers (typically in a company). The artifacts would be consumed by users and they would access these artifacts through the Store. 

Now in the cluster I created, the issue was that  if a user signs in to the Store, he/she would also be able to access the Publisher and vice versa. This is problematic since only the publishers (i.e: the company) should be able to publish artifacts. This is because G-Reg has the Single Sign On (SSO) feature enabled by default. When SSO is used, a user signed into one system can access other, connected systems without the need to sign in again at those particular systems. More on SSO can be found here.

Continue reading

Creating a WSO2 Application Server cluster using AWS EC2 and Nginx as a load balancer

Introduction

The WSO2 Application server is used for hosting, deploying and managing applications, built on the WSO2 Carbon platform. A detailed documentation of the Application Server can be found here.

As is the case with other WSO2 products, multiple instances of the Application Server can be installed in a cluster. When in a cluster, the work is shared between the multiple instances although it appears as if though there is only a single instance of the Application Server (AS).

In this tutorial, we will be using AWS EC2 instances to create a AS cluster. The EC2 instances will be Ubuntu based. We’ll be creating a cluster of 3 AS nodes, using Nginx for load balancing and MySQL for storing the user management and registry data of the AS. More information on load balancing can be found here and here.

Continue reading

Sinhala in Windows 10

I am usually pretty skeptic about Microsoft’s upgrades to its Windows OS. I’ve been using Windows 7 in my laptop ever since I bought it about over a year back. Windows 8 nor 8.1 appeal to me at all! However, this time, with all the hype for Windows 10, I was curious to try it out.

After obtaining a copy few days back, I was still hesitant to install it, mainly because I didn’t wanna go through the pain of having to comb through the internet for drivers and workarounds for softwares I used to use on Windows 7 (the memories from the switch to Vista from XP was haunting me 😛 ).  After asking around a few who had already installed it, and after checking it on VMWare, I finally decided to give it a go.

The installation process was pretty smooth, though I expected it might be troublesome. I used a bootable USB to install a clean copy of Windows 10. For creating the bootable USB, I used the tool given by Microsoft for creating Windows 7 bootable USBs. I decided on this because my laptop is dual boot and when installing Ubuntu Gnome on it before, I ran in to trouble with UEFI vs. BIOS. The bootable Ubuntu USB given by my friend was a UEFI installer and it caused a boot-loader nightmare when installed along Windows 7 (which was BIOS based). So in order to avoid facing those problems again, I used the Windows USB/DVD Download Tool. It allowed me to create a BIOS based bootable Windows 10 USB.

Happily for me, the software I used on Windows 7 runs without problem in Windows 10 as well (well, at least so far! 😛 ). However there was a minor annoyance in Windows 10, in my short experience with the OS. It does not seem to support any other language pack, other than the pack you selected initially, when starting the installation. I found this out when trying to browse websites containing Sinhala text; the pages simply didn’t render the text. To fix this, go to the language settings of the control panel and download the Sinhala language pack. If you were already using any browsers etc. restart them to let the changes take effect.

language

Software Engineering Project (SEP): Introduction

In our 5th semester, we have a module called Software Engineering Project, in which we have to carry out a Software Engineering Project, individually. The purpose of this project is to put the knowledge we gained on Computer Science and Software Engineering in to practice. We could either select a project of our own or select from a list of ideas given to us. For my project, I selected a project given to us. It is a project of LIRNEasia, a regional organization which carries out various researches.

The project is to develop a web-based tool to visualizing propagation of information/disease in networks with the option to overlay such visualizations on a map. Such a tool will give valuable insights to researchers upon visual inspection of the data. There are actually 3 other projects related to this project, which in combination will give researchers and policy makers a valuable tool.

In my project, the aim is to provide a visualization overlaid on a map for researchers studying disease/information propagation through network-based models. When overlaid on a map, such models provide the ability to simulate scenarios and identify vulnerable regions etc.

There are several challenges to be addressed when going ahead with this project. First, the scalability. The tool should be able to handle large volumes of input data, without crashing the browser. Secondly the input file format is generic and not completely specified, which means the tool should be capable of handling input files with data formatted in various ways. Third problem is the extensibility of the tool. The tool needs to be designed in a way that it enables other developers to add features to it or build other components or services on top of it. At the moment, I haven’t really figured out how to tackle these problems. But I feel the right sort of modular architecture will sort out or at least make it easier to tackle these problems.

Using Apache Jena With Netbeans

So it is GSoC time and I came across an interesting project which deals with linked data. In order to apply for it, I had to get Jena, a Java library for handling RDF graphs. The documentation provides info on how to configure Eclipse to use it. But Eclipse is not really my choice for an IDE. I prefer Netbeans. So I wanted to configure Jena to be used in Netbeans. Most of the resources I looked up suggested using Maven. But I wasn’t interested in that. After a bit of tinkering around, I managed to get it configured as a library in Netbeans. It is quite easy actually.

Go to Tools -> Libraries and create a new library called Jena. In the classpath tab, go to ‘Add Jar/Folder’ and navigate to the path where you extracted Jena and go to the ‘lib’ folder. Select all the jar files in the ‘lib’ folder and add them. Then, optionally, you can add the Jena source files and Java docs as well. And that is it.

Screenshot 2015-03-25 02.35.57

To use Jena in a new project, right click the libraries directory in the created project on the ‘Projects’ tab and select ‘Add Library’ and select Jena.

First taste of real world software development

Few weeks back, our Database Systems lecturer, Dr. Shehan, asked for a couple of volunteers to work on developing a volunteer management system. Me being the adventurous programming enthusiast that I am, volunteered at once. Along with me, Yasiru, Amitha, Dinusha, Kasun, Riyafa, Chathura and Lasantha also volunteered. The project was actually a project done by a student of the senior batch (Ridwan) as a part of his Software Engineering course module in the 5th semester. He had wanted to move on from the project and therefore, is handing over the project to us.

The project is to implement a volunteer management system for the newly instated National Volunteering Secretariat (NVS). This is an initiative of the Ministry of Social Services and the United Nations Volunteers (UNV). UNV is handling the project on behalf of the NVS and we had our first meeting with the UNV project team last Monday. This was our first proper meeting with a client and I must say, it was may more interesting than they make it sound in lectures on requirements gathering 😛

The meeting was mainly to introduce us Continue reading

Laravel

So we are just about to take part in a hackathon organized by Koding and my friend asked me to look up this PHP framework called laravel because it might come in handy for the project idea we have in mind.

The supporting documentation for it is overwhelmingly for linux. So had to do some searching in order to figure out how to configure it on windows. I initially followed the instructions given in the official documentation. In it, the first step – installing composer – didn’t give me any issues. You just simply have to run the setup and that is it.

However, the second step – installing laravel – gave me some issues. I was trying to follow the install via installer option of installing laravel. Although the installer Continue reading