Introduction

Recently, I created a Governance Registry cluster on EC2 using Nginx as a load balancer. The cluster created was a 3 node cluster, fronted with Nginx. The initial plan was to create a cluster with 2 Nginx load balancers, one each for the Store components and the Publisher components, as given below.

Originally proposed deployment pattern

As the first step in creating the cluster, I was advised to create a cluster (same deployment pattern as above) of just one G-Reg node. Although I successfully managed to deploy the cluster, there was a major issue in it. The scenario for the G-Reg cluster I was creating was that the artifacts would be deployed by the publishers and as such, the Publisher context should only be accessible by publishers (typically in a company). The artifacts would be consumed by users and they would access these artifacts through the Store. 

Now in the cluster I created, the issue was that  if a user signs in to the Store, he/she would also be able to access the Publisher and vice versa. This is problematic since only the publishers (i.e: the company) should be able to publish artifacts. This is because G-Reg has the Single Sign On (SSO) feature enabled by default. When SSO is used, a user signed into one system can access other, connected systems without the need to sign in again at those particular systems. More on SSO can be found here.

To solve this issue, it was decided to use a G-Reg node either as a Store or a Publisher, but not both. The modified deployment pattern was as given below, with two Store nodes and two Publisher nodes.

Deployment pattern to solve the SSO issue

Once this pattern was finalized, the next step was to deploy this cluster on AWS EC2 instances.

Configuring the Cluster

For the configurations, I followed the instructions given in the WSO2 clustering documentation. However, this particular documentation is from a previous version and is not entirely compatible with G-Reg 5.1.0.

Configuring the databases

For configuring the databases you can follow the given steps here. However, we can omit the second database, WSO2_AM_DB. This database is only required if we are installing WSO2 API Manager features on top of G-Reg (which we won’t be doing in this setup).

When granting permissions for the DB user, use the following commands, instead of the one given in step 2.

mysql> grant all on governancedb.* TO regadmin@"%" identified by "regadmin";
mysql> grant all on WSO2_USER_DB.* TO regadmin@"%" identified by "regadmin";

The % sign tells the MySQL server to allow access for the user regadmin from any IP. For just trying out a cluster, this will be easier than having to give permission to each of the EC2 instances.

Configuring the Datasources

You can configure the datasources for the cluster using step 3. In step 3.a, we only need the configurations for the 2 remote databases we created above. Step 3.b is not exactly required since the local databases are the default in the product. In step 3.c, we configure the server instances to use the remote database for storing user management related data, so that these data can be shared across the cluster.

Step 3.d deals with the configuration of the registry and config mounting. The remoteInstance configuration in the registry.xml file should conform to the following format.

More info on remote instance mount configurations can be found here.

The MySQL JDBC driver mentioned in step 3.e can be found here. If you are using another DB, use the relevant connector.

Enabling Clustering

Steps for enabling clustering can be found here. In step 1.g, the port numbers for the members of the cluster should differ if the G-Reg instances are running on the same server. Configuring the MgtHostName in step 3 is important. Without this, in EC2, the server would try to redirect sign-in requests to the private IP of a server for some reason.

Deployment Synchronization

To setup a basic cluster, we can leave out the configuration of the deployment synchronizer.

Additional Configuration

Apart from the above, you also have to configure the <GREG_HOME>/repository/conf/identity/sso-idp-config.xml file. The sso-idp-config.xml should look like the following for Store nodes, upon completing configuration.

For publisher nodes, it should look like the following:

Configuring Nginx

Configure the Nginx load balancer as per the instructions given here. The final configuration should be similar to the following configuration.

Starting the Servers

Map store.wso2.com and publisher.wso2.com to the IP of your Nginx load balancer in your /etc/hosts files. Now you can start up the servers and access the Store through https://store.wso2.com/store and the Publisher through https://publisher.wso2.com/publisher