Learning from Architecture Assessment

Learning from Architecture Assessment

In the recent past, I got a short assignment to analyze/assess the application architecture of an existing application, where they were planning to increase the users by 3 folds. The client wanted me to understand the current capability of the application and assess it can scale to the expected user load.

As I was doing this first time, I was not sure how to proceed. I consulted some of the senior members and got some tips as well, perhaps most of the things were different when I reached the client place.

So, I thought of summarizing my experience and give away some useful information if some architect gets such request in future.

Pre-conditions: 

This assessment is a Black box assessment which essentially means,  you need to look at high level architecture NOT the code.

Expectations from this assessment :

  •  Scale of the app to the required Users
  •  Identify Performance bottlenecks / Risks
  •  Suggest mitigation plans for identified issues
  •  Rate the risks and prioritize
  •  Suggest a road map with estimations to reach the target state.

Q: Any Standard approach available ?

A: Yes & No.

Yes, because there are some standard approaches available based on the previous experience. For eg: ATAM (Architecture Tradeoff Analysis Method) by SEI (Software Engineering Institute)

No, because the approach might change as per the application and available documentation and resources.

My Custom Approach: 

  • Do interviews with Technical and Business teams.
  • Understand the high level pain points of users
  • Map these pain points to Architecture
  • Collect metrics (memory, CPU usage, requests/sec, etc.)
  • Identify the risks and suggest mitigation plans.

I divided the architecture in 3 different types based on the application high level understanding. Then most importantly understand the business and types of users and their activities on the application. Identify the peak loads, most used functionalities, background jobs & their scheduled times, Master data loads, Any remote data push/pulls via different protocols and any other type of load coming to our application servers. If the application is used across different geo locations/time zones find out the over lap of users and their activities during peak loads (particularly).

Logical Architecture:  This is mostly the high level components involved in the application at each layer and how they communicate to each other and cross cutting concerns like logging, security handled. How the data is transmitted between application and database.

Integration Architecture: If the current application has any communication channels open both inbound or outbound with any external system. Usually all Enterprise applications have integrations with their ERP/CRM/SSO/DW/MDM systems. Apart from these depending on application there might be some systems outside the Enterprise also integrated. As cloud is becoming popular, these days many companies decide to store their content/documents in Box/Drive etc. and many other possibilities are there depending on the business of the application.

Deployment Architecture: If the application is deployed in traditional system, this is a key diagram you need to analyze and figure out the load balancing strategies. What is the hardware configuration of each server, their health reports during the performance testing, will become critical to decide the suggestions of this architecture.

Key Challenges involved: 

Availability of update architecture documentation:  this is very rarely available for most of the applications. In course of development the architecture will be changed, no architect will update the documents prepared at the design stage. So, I faced very similar challenge. They did NOT choose to document most of the stuff due to lack of time 😦

When you reach this state, all you are left with is to interview key players ( Architects, Lead developers, Business Analysts, Project Managers, Domain Experts etc;)

Availability of Resources for these interviews: as I already mentioned , they are key players. They are always busy doing some stuff. If you are not having a push from top, they might choose to ignore you. You somehow has to get time from these guyz to extract the information you need and in time.

Language issues: You might choose to ignore if you are lucky enough to work with all English speaking teams. For me it was  a challenge as half the team speak one language and the other half speaks another language. I am the only person who speak English 🙂

Conclusion:  First understand the business and collect metrics on required parameters. Do interview required people and gather information. Once you have all required information, analyze properly and figure out the key risks in the architectures and suggest best mitigation plans.

Hope this is helpful and happy assessments 🙂

Other Useful references :

Enterprise Architecture Assessment Framework

Assessing technical architecture

Advertisements
Apache Solr -Unboxing

Apache Solr -Unboxing

Overview of Apache Solr:

  • Apache Solr is a standalone full-text search server with Apache Lucene at the back end.
  • Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.
  • In brief Apache Solr exposes Lucene’s JAVA API as REST like API’s which can be called over HTTP from any programming language/platform.
  • Solr is a Schema based indexing.

Why I need Solr ? 

  • Fast Indexing/Searching, Indexes can be merged/optimized (Index compaction).
  • Great admin interface can be used over HTTP.
  • Support for integration with various other products like drupal CMS, etc
  • Caching, Replication, Distributed search.
  • Full support for REST and readily available client API SolrJ
  • Awesome community support too.
  • Full Text Search
  • Faceted navigation
  • More items like this(Recommendation)/ Related searches
  • Spell Suggest/Auto-Complete
  • Custom document ranking/ordering
  • Snippet generation/highlighting
  • And many more…….

Architecture of Solr

SolrArchitecture

How to Install ?

  • Download Solr
    • Minimum Requirements
      • Directory for storing index files
      • Directory for storing configuration files
      • Solr_Homehaving other dependencies
      • A Servlet container(tomcat, jetty) with appropriate configuration

Configuring Solr

  • Solrconfig.xml
    • Contains most of the parameters for configuring Solr itself.
    • Initial Parameters
    • Request Handlers
    • Components
  • Schema.xml
    • Contains all of the details about document structure, index-time, query-time processing.
    • Field Types
    • Filters
    • Copy Fields

How to Start the Solr Server

Once you download the Solr, You can unzip the folder and go navigate to the Solr-Version\bin using command prompt.

Run Command – Solr start -p 8983 ( -p signifies the port, you can give any valid port number after the parameter)  You can see below message if it got started.

Waiting up to 30 to see Solr running on port 8983
Started Solr server on port 8983. Happy searching!

If you want to stop run – Solr stop -p 8983 ( Use the same port on which you have started)

After Start, if you want to see the  admin console of Solr, Please try going to http://localhost:8983/solr/ 

You would be able to see the screen as below.

SolrAdminInterface

Create Core

Once your Server is up and running, you may like to create a Core ( DB/Schema in RDBMS terms).

In your admin Interface, Click on “Core Admin”, then “Add Core“. Type in your schema name and Folder path etc. as shown below.

SolrCoreCreation

Note: schema.xml, solrconfig.xml names are recommended to keep as is.

Now, the Schema is ready, You can start creating you own fields in schema.xml which will be there in the path “solr-5.4.1\server\solr\new_core\conf“.

Fields / Types 

  • Create Field
    • <field name=”author” type=”string” indexed=”true” stored=”true” />
  • Custom Field
    • <fieldTypename=”text_general_ngram” class=”solr.TextField”positionIncrementGap=”100″>
      <analyzer type=”index”>
      </analyzer>
      <analyzer type=”query”>
      </analyzer>
      </fieldType>

Querying Solr 

  • Plain Text Search
    • q = text: “searchString”
  • Expanding Search to More Fields
    • title:name& type:review& price:[* To 500]
  • Add Facets
    • facet.field=product & facet.field=rating

Summary

If you want to add search to your web site, it is easy with Apache Solr, You can use the features of Solr and make your life easy for searching anything. There are many plugins available for Solr if you would like to integrate your search with RDBMS fields as well. Hope it would be useful for you.

Happy Reading 🙂

I need Bower ? or Grunt?

I need Bower ? or Grunt?

When we start learning MEAN stack, we get this question for sure. Even I asked the same question when I initially heard about these two tools. I thought they both does the same thing. To discuss this, First we need to know what these two tools does.

When we started developing every layer in Javascript, we need some tools to make our life easy. The most common one’s you need are, the dependency management and build management.

So, I would like to start with what they are…. then you would see the differences easily.

Assumption: Before you read further, I assume you already know how to use npm command and install packages like Grunt, Bower. (If no, try this link)

Let’s get started with

Bower

Why Bower ?

  • Acts as a Package Manager
  • Dependency Management of the modules
  • Bower works by fetching, installing packages and keeps them up to date.
  • Bower downloads the dependencies for you and installs them on the required path.
  • This installation could be as global module or project module.

Key Features

  • Bower runs on Git
  • Can be integrated with other tools like Grunt, Gulp etc.
  • Bower can manage any packages hosted on NPM platform
  • Can manage HTML,CSS, JS packages

In my view, whoever comes from Java,J2EE background can easily relate this to a Maven and its POM.XML to the bower.json file. In this file, you exactly give the same content what we give in Pom, the dependent libraries and their versions. This is very useful in later phases if you want to change the version of any library you are using just go hear and change the version. You are done. Even you can manually download the js file and keep it. To avoid this manual work and make developer’s life easy we need this tool.

Grunt

Why Grunt ?

  • Grunt is a Task Runner
  • Grunt is a task based command line build tool
  • Helps to automate any step while building you application
  • Can be integrated with many plug-ins
  • Very commonly used for jobs like Minification of CSS, JS & run Unit test cases

Key Features

  • Grunt helps you run tasks based on configurations
  • Provides automation for different environments like Dev, Test, etc
  • Helps you to increase the performance of the application using different plugins
  • Helps you optimize images, minify file sizes, run unit tests

In my view, Grunt acts like our good old Apache Ant, where you used to define the tasks in BUILD.XML and run them  based on the need. Very similarly Grunt helps you create your own tasks and run them based on the your requirement. Here we define Gruntfile.js where you define your tasks.

I think now, you got a better understanding of these two tools.

Now these two tools are meant for two different things. So, don’t get confused. We need both of these tools. Where as there are tools which are very similar to Grunt for eg. Gulp. We need more detailed comparison if we have to choose one of them.

Happy Reading 🙂

Angular JS – How to create a Controller ?

Angular JS – How to create a Controller ?

Here I am going to show you the ways you can create a controller in Angular JS.

I am assuming, you already know the advantages of Angular JS and how to use Angular JS basics like ng-app, ng-controller, $scope.

Just to give you some basic idea on Angular JS, It is a Web Framework which would give you lot of flexibility and de-couple your Model-View-Controller with the client side code. This has been the most popular and critical feature of Angular. Lets try to get into the controller part. The controller is very important component in the MV* pattern which Angular talks about.

This controller can be created in 3 different ways which I would show you here.

Basic Way: (Controller as Global Function)  / app.js

This way is easy to create, perhaps this is not the best way to solve the problem. As this is a global function there could be many complications when we have multiple functions defined in the app. If the app grows bigger there is no way to isolate this function from others. So, this is not the way to go.


var app = angular.module("app", []);
var TestController = function($scope){
 $scope.count = 10;
});


2nd Way (Application level Controller)

This is application level controller defined as we are defining controller on app. Tomorrow we might need many other things at application level which might be included in the same controller. So, this is the angular way of doing it, still not the best practice. There could be modular way of doing it and adding as dependency to the application (app).


var app = angular.module("app", []);
app.controller('TestController', function($scope){
 $scope.count = 10;
});

3rd Way (Module level Controller)

Here we are defining another module which is a controllers and added this module as dependency to the app module. This way tomorrow if we want to add another controller we can simple add it to this module and it will be available in the application easily. If we don’t want the new controller exposed to the app then we can simply define another module. So, here is the best practice to create a controller in Angular.


var app = angular.module("app", ['controllers']);
angular.module("controllers", []).controller('TestController', function($scope){
 $scope.count = 10;
});

Note: To keep the simplicity, I have used a very basic example to show you the differences. You can use the same HTML given below to test all the three controllers. Put the controller code in app.js file.

HTML :


<!DOCTYPE html>
<html ng-app="app">
<head>
 <title>Hello</title>
</head>
<body>

<div ng-controller="TestController"> 
 {{count}}
</div>


</body>
<script type="text/javascript" src="js/angular-min.js"></script>
<script type="text/javascript" src="app.js"></script>
</html>

Why Micro Services ?

Why Micro Services ?

Micro Services are the next big thing in the modern software architecture and development. The distributed applications in the recent past changed the world of software outlook. Now the applications are developed, used, work and built differently. This caused the logical evolution of Micro Services. Some best examples of game changers in the software world are Git ( Distributed source control), Cloud computing (AWS, Azure, …), BitCoin (Distributed virtual currency).

Let’s begin our journey with understanding, What are Micro Services?

The Philosophy

The whole idea of Micro Service architecture is that they are small, focused and should be scaled.  

The Definition

The MicroService architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – Fowler & Lewis, 2014.

This idea looks very similar to SOA. When I first heard about Micro Services, I thought they are just web Services and got many questions related to them. Some of them are as below
Why these services are called Micro Services?
What is the difference between these micro services and web Services ?
How different this world is different from SOA world?

Characteristics of Micro Service:

Do One thing well : Services should hide, how they work and do a single thing and do it well. The encapsulation is based on the business function of the service that it encapsulates the functional requirements.

Business Domain Centric : Each Service should align to the context with in the domain model for this to make sense in the domain and in the service architecture.

Decentralization : Every service should be decentralized and autonomous, so that each service can operate on its own schedule and priorities. Each service can be updated and deployed as fit.

Smart Endpoints and dumb pipes: This characteristic has come based on SOA, in SOA we have many products / ESB where we can define lot of logic in routing and transforming etc. Where as in Micro services the focus on login inside the service and the communication should happen very dumb like a REST / HTTP resource call to get to the service.   

Fail-Safe : Enterprise systems need to be smart, in that they fail in a way that doesn’t break things in unpredictable ways. Anticipate failure in everything, among with  validation of input and validation of data we send downstream.

Automation : Services should use automation which keeps management and operational support efficient. Employing large number of small services can become a mess if you do not embrace automation.

Principles of Micro Services – Encapsulation, Automation, Business Domain Centric, Isolation, Fail-safe and Observable.

Micro Service Vs Web Service:

The regular Web services are definitely different from Micro services. The regular web services support is based on protocols like HTTP / SOAP. These services are invented to solve the integration problems between application when we have polyglot technologies with different vendors/ applications. So, the aim of web services is to integrate or build a channel to communicate between these applications. Where as Micro services aim is NOT to integrate applications, perhaps to reduce the complexity of a monolithic application to micro services. This would give us re-use, maintainability, test-ability, scalability etc in the application.

Micro Services Vs SOA:

SOA started with very similar characteristics mentioned above, but the idea got converted when many vendors started introducing new features to SOA. Then we have a vendor lock-in problem now. We have lot of logic built in around the services NOT inside the services. So, Micro services are the out come of learning of SOA. Still, micro services are evolving. We might end up with very similar problems of SOA, if we are not cautious. Hope we would be able to use micro services the way they are explained.

Technology Support:

Here are the technology

Building Micro services with Spring Boot

Building Micro services with Drop Wizard

References:

Fowler & Lewis on MicroServices

Videos on MicroService Architecture

Java 8 – Re using Lambda Expressions

Java 8 – Re using Lambda Expressions

In my previous Java 8 Lambda expression blog, I tried to explain what is a Lambda expression in general, how to create a Lambda expression in Java 8 etc. Here we are going to look at the re usability perspective of the Lambda expression. It is very much required to re use the code or apply the DRY design principle when we are programming.

Let’s take some example to start with, Here I am trying to create a List of countries for each region, and trying to find the number of countries start with letter say “B” or “N”. Perhaps, to achieve the same result in the conventional way of Java we should run a loop by Iterating the list and putting a if condition and lot of unnecessary code.

Here is how the traditional code looks like


public class OldWay {
     final static List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final static List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

public static void main(String[] args) {
     int count = 0;
     for(String ctry : asia){
     if(ctry.substring(0,1).equals(&quot;N&quot;)){ count ++; }
}
     System.out.println(&quot;Countries name starting with N :&quot; + count);
     }
}

So, we have a much better way to achieve the same thing with lambda expressions. It is easy and simple. We need to tune our Java brains a bit to think in functional way. For me it took long time to tune myself 🙂


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

     public static void main(String[] args) {
          final List&lt;String&gt; asia =
          Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

          final List&lt;String&gt; europe =
          Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

         // re using the Lambda Expression ...
         final Predicate&lt;String&gt; startsWithB = name -&gt; name.startsWith(&quot;B&quot;);

         final long countStartswithBinAsia = asia.stream().filter(startsWithB).count();

         System.out.println(&quot; Asian Countries starts with B :&quot; + countStartswithBinAsia);

         final long countStartswithBinEurope = europe.stream().filter(startsWithB).count();

         System.out.println(&quot; European Countries starts with B :&quot; + countStartswithBinEurope);

       }

}

Output:


Asian Countries starts with B :2
European Countries starts with B :1

Now in the above example we have seen a Predicate which is getting re used as a method in Java. But there is a problem here, as we are not able to reuse this predicate for any other character than B. Below we would look at the Predicate as a full fledged method and could be re usable in any other class with a parameter to it. This is the best way to re use a predicate.


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

// re using the Lambda Expression ... with whatever character you want !!!
public static Predicate&lt;String&gt; startsWithCharacter(final String character) {
return name -&gt; name.startsWith(character);
}

public static void main(String[] args) {
     final List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

    final long countStartswithBinAsia = asia.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

    System.out.println(&quot; Asian Countries starts with N :&quot; + countStartswithBinAsia);

     final long countStartswithBinEurope = europe.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

     System.out.println(&quot; European Countries starts with N :&quot; + countStartswithBinEurope);

     }

}

Output:


Asian Countries starts with N :1
European Countries starts with N :2

Hope this was useful to you and in next article, I would like to discuss the FunctionalInterface in Java 8.
Thanks and Happy Reading…

What is “No” in NoSQL ?

What is “No” in NoSQL ?

We keep hearing about NoSQL these days. Many of us might think what’s the big deal if there is no “SQL”. Some of us might think OMG, there is no SQL ? How the hell the things are going to work, if there is no SQL? So, Let’s try to understand what does it mean when we say, NoSQL Database. The easiest way to confirm is Yes, there is no SQL if you use these databases. Yet, there is no need to worry too much about it. Perhaps SQL has it’s own limitations, though it has many advantages and ruling the industry for many years. The first and foremost of all is Relationship. For Eg; If you want to store company data like company name, address, departments, employees etc. You could already identify the entities and their relationships like one-one, one-many, many-one etc. If you want to aggregate the data? is there any way? Yes, there are joins, unions etc. So, lets try to fetch the data for the below page and see how many tables will be involved when we have to fetch the data for this page.

SQL-Relationships
Fig: 1

In Figure 1, we have seen the number of relationships you need to just define a structure of a company. If the same thing we have to do with a NoSQL format, how would it look like ? I mean the non relational databases.


{
'Company':'XYZ Company',
'Departments':[
'Dept1':
{ 'Name': 'HR',
'Employees':[ {'Name':'A','address':'City A', 'Sport': 'Tennis' },{'Name':'B','address':'City B', 'Sport': 'Cricket' } ]},
'Dept2':
{ 'Name': 'Admin',
'Employees':[ {'Name':'X','address':'City X', 'Phone':12121 },{'Name':'Z','address':'City Z', 'Phone':232323 } ]},
'Dept3':
{ 'Name': 'IT',
'Employees':[ {'Name':'N','address':'City N' },{'Name':'M','address':'City M' } ]}
],
'Addressg':'#55, 10 Avenue, ABC City, US'
}

This is what is called as Aggregation in NoSQL world. This structure would give you a flexibility of retrieving all the details of the company together and fast, instead of storing the data in different tables and different columns. This difference leads to many other differences between these two worlds.

If you closely observe the above JSON object, each employee object is having different attributes For Eg: HR department has  sport, Admin depart has Phone and IT department has non of these attributes. So, this is possible because these databases are scheme less. Each employee and each document might contain completely different format. This could be advantage as well as problem. Usually this is considered as advantages for NoSQL databases because we don’t need to define Schema before hand and this would give the flexibility to add new fields whenever we want in future to the same document.

References:

The Book – No SQL Distilled by Pramod and Martin Fowler would give you a very good understanding of NoSQL.

Blogs you would like to follow – nosql.mypopescu , NoSQL Weekly

Happy Reading 🙂