Learning from Architecture Assessment

Learning from Architecture Assessment

In the recent past, I got a short assignment to analyze/assess the application architecture of an existing application, where they were planning to increase the users by 3 folds. The client wanted me to understand the current capability of the application and assess it can scale to the expected user load.

As I was doing this first time, I was not sure how to proceed. I consulted some of the senior members and got some tips as well, perhaps most of the things were different when I reached the client place.

So, I thought of summarizing my experience and give away some useful information if some architect gets such request in future.

Pre-conditions: 

This assessment is a Black box assessment which essentially means,  you need to look at high level architecture NOT the code.

Expectations from this assessment :

  •  Scale of the app to the required Users
  •  Identify Performance bottlenecks / Risks
  •  Suggest mitigation plans for identified issues
  •  Rate the risks and prioritize
  •  Suggest a road map with estimations to reach the target state.

Q: Any Standard approach available ?

A: Yes & No.

Yes, because there are some standard approaches available based on the previous experience. For eg: ATAM (Architecture Tradeoff Analysis Method) by SEI (Software Engineering Institute)

No, because the approach might change as per the application and available documentation and resources.

My Custom Approach: 

  • Do interviews with Technical and Business teams.
  • Understand the high level pain points of users
  • Map these pain points to Architecture
  • Collect metrics (memory, CPU usage, requests/sec, etc.)
  • Identify the risks and suggest mitigation plans.

I divided the architecture in 3 different types based on the application high level understanding. Then most importantly understand the business and types of users and their activities on the application. Identify the peak loads, most used functionalities, background jobs & their scheduled times, Master data loads, Any remote data push/pulls via different protocols and any other type of load coming to our application servers. If the application is used across different geo locations/time zones find out the over lap of users and their activities during peak loads (particularly).

Logical Architecture:  This is mostly the high level components involved in the application at each layer and how they communicate to each other and cross cutting concerns like logging, security handled. How the data is transmitted between application and database.

Integration Architecture: If the current application has any communication channels open both inbound or outbound with any external system. Usually all Enterprise applications have integrations with their ERP/CRM/SSO/DW/MDM systems. Apart from these depending on application there might be some systems outside the Enterprise also integrated. As cloud is becoming popular, these days many companies decide to store their content/documents in Box/Drive etc. and many other possibilities are there depending on the business of the application.

Deployment Architecture: If the application is deployed in traditional system, this is a key diagram you need to analyze and figure out the load balancing strategies. What is the hardware configuration of each server, their health reports during the performance testing, will become critical to decide the suggestions of this architecture.

Key Challenges involved: 

Availability of update architecture documentation:  this is very rarely available for most of the applications. In course of development the architecture will be changed, no architect will update the documents prepared at the design stage. So, I faced very similar challenge. They did NOT choose to document most of the stuff due to lack of time 😦

When you reach this state, all you are left with is to interview key players ( Architects, Lead developers, Business Analysts, Project Managers, Domain Experts etc;)

Availability of Resources for these interviews: as I already mentioned , they are key players. They are always busy doing some stuff. If you are not having a push from top, they might choose to ignore you. You somehow has to get time from these guyz to extract the information you need and in time.

Language issues: You might choose to ignore if you are lucky enough to work with all English speaking teams. For me it was  a challenge as half the team speak one language and the other half speaks another language. I am the only person who speak English 🙂

Conclusion:  First understand the business and collect metrics on required parameters. Do interview required people and gather information. Once you have all required information, analyze properly and figure out the key risks in the architectures and suggest best mitigation plans.

Hope this is helpful and happy assessments 🙂

Other Useful references :

Enterprise Architecture Assessment Framework

Assessing technical architecture

Advertisements
Apache Solr -Unboxing

Apache Solr -Unboxing

Overview of Apache Solr:

  • Apache Solr is a standalone full-text search server with Apache Lucene at the back end.
  • Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.
  • In brief Apache Solr exposes Lucene’s JAVA API as REST like API’s which can be called over HTTP from any programming language/platform.
  • Solr is a Schema based indexing.

Why I need Solr ? 

  • Fast Indexing/Searching, Indexes can be merged/optimized (Index compaction).
  • Great admin interface can be used over HTTP.
  • Support for integration with various other products like drupal CMS, etc
  • Caching, Replication, Distributed search.
  • Full support for REST and readily available client API SolrJ
  • Awesome community support too.
  • Full Text Search
  • Faceted navigation
  • More items like this(Recommendation)/ Related searches
  • Spell Suggest/Auto-Complete
  • Custom document ranking/ordering
  • Snippet generation/highlighting
  • And many more…….

Architecture of Solr

SolrArchitecture

How to Install ?

  • Download Solr
    • Minimum Requirements
      • Directory for storing index files
      • Directory for storing configuration files
      • Solr_Homehaving other dependencies
      • A Servlet container(tomcat, jetty) with appropriate configuration

Configuring Solr

  • Solrconfig.xml
    • Contains most of the parameters for configuring Solr itself.
    • Initial Parameters
    • Request Handlers
    • Components
  • Schema.xml
    • Contains all of the details about document structure, index-time, query-time processing.
    • Field Types
    • Filters
    • Copy Fields

How to Start the Solr Server

Once you download the Solr, You can unzip the folder and go navigate to the Solr-Version\bin using command prompt.

Run Command – Solr start -p 8983 ( -p signifies the port, you can give any valid port number after the parameter)  You can see below message if it got started.

Waiting up to 30 to see Solr running on port 8983
Started Solr server on port 8983. Happy searching!

If you want to stop run – Solr stop -p 8983 ( Use the same port on which you have started)

After Start, if you want to see the  admin console of Solr, Please try going to http://localhost:8983/solr/ 

You would be able to see the screen as below.

SolrAdminInterface

Create Core

Once your Server is up and running, you may like to create a Core ( DB/Schema in RDBMS terms).

In your admin Interface, Click on “Core Admin”, then “Add Core“. Type in your schema name and Folder path etc. as shown below.

SolrCoreCreation

Note: schema.xml, solrconfig.xml names are recommended to keep as is.

Now, the Schema is ready, You can start creating you own fields in schema.xml which will be there in the path “solr-5.4.1\server\solr\new_core\conf“.

Fields / Types 

  • Create Field
    • <field name=”author” type=”string” indexed=”true” stored=”true” />
  • Custom Field
    • <fieldTypename=”text_general_ngram” class=”solr.TextField”positionIncrementGap=”100″>
      <analyzer type=”index”>
      </analyzer>
      <analyzer type=”query”>
      </analyzer>
      </fieldType>

Querying Solr 

  • Plain Text Search
    • q = text: “searchString”
  • Expanding Search to More Fields
    • title:name& type:review& price:[* To 500]
  • Add Facets
    • facet.field=product & facet.field=rating

Summary

If you want to add search to your web site, it is easy with Apache Solr, You can use the features of Solr and make your life easy for searching anything. There are many plugins available for Solr if you would like to integrate your search with RDBMS fields as well. Hope it would be useful for you.

Happy Reading 🙂

I need Bower ? or Grunt?

I need Bower ? or Grunt?

When we start learning MEAN stack, we get this question for sure. Even I asked the same question when I initially heard about these two tools. I thought they both does the same thing. To discuss this, First we need to know what these two tools does.

When we started developing every layer in Javascript, we need some tools to make our life easy. The most common one’s you need are, the dependency management and build management.

So, I would like to start with what they are…. then you would see the differences easily.

Assumption: Before you read further, I assume you already know how to use npm command and install packages like Grunt, Bower. (If no, try this link)

Let’s get started with

Bower

Why Bower ?

  • Acts as a Package Manager
  • Dependency Management of the modules
  • Bower works by fetching, installing packages and keeps them up to date.
  • Bower downloads the dependencies for you and installs them on the required path.
  • This installation could be as global module or project module.

Key Features

  • Bower runs on Git
  • Can be integrated with other tools like Grunt, Gulp etc.
  • Bower can manage any packages hosted on NPM platform
  • Can manage HTML,CSS, JS packages

In my view, whoever comes from Java,J2EE background can easily relate this to a Maven and its POM.XML to the bower.json file. In this file, you exactly give the same content what we give in Pom, the dependent libraries and their versions. This is very useful in later phases if you want to change the version of any library you are using just go hear and change the version. You are done. Even you can manually download the js file and keep it. To avoid this manual work and make developer’s life easy we need this tool.

Grunt

Why Grunt ?

  • Grunt is a Task Runner
  • Grunt is a task based command line build tool
  • Helps to automate any step while building you application
  • Can be integrated with many plug-ins
  • Very commonly used for jobs like Minification of CSS, JS & run Unit test cases

Key Features

  • Grunt helps you run tasks based on configurations
  • Provides automation for different environments like Dev, Test, etc
  • Helps you to increase the performance of the application using different plugins
  • Helps you optimize images, minify file sizes, run unit tests

In my view, Grunt acts like our good old Apache Ant, where you used to define the tasks in BUILD.XML and run them  based on the need. Very similarly Grunt helps you create your own tasks and run them based on the your requirement. Here we define Gruntfile.js where you define your tasks.

I think now, you got a better understanding of these two tools.

Now these two tools are meant for two different things. So, don’t get confused. We need both of these tools. Where as there are tools which are very similar to Grunt for eg. Gulp. We need more detailed comparison if we have to choose one of them.

Happy Reading 🙂

Why Micro Services ?

Why Micro Services ?

Micro Services are the next big thing in the modern software architecture and development. The distributed applications in the recent past changed the world of software outlook. Now the applications are developed, used, work and built differently. This caused the logical evolution of Micro Services. Some best examples of game changers in the software world are Git ( Distributed source control), Cloud computing (AWS, Azure, …), BitCoin (Distributed virtual currency).

Let’s begin our journey with understanding, What are Micro Services?

The Philosophy

The whole idea of Micro Service architecture is that they are small, focused and should be scaled.  

The Definition

The MicroService architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – Fowler & Lewis, 2014.

This idea looks very similar to SOA. When I first heard about Micro Services, I thought they are just web Services and got many questions related to them. Some of them are as below
Why these services are called Micro Services?
What is the difference between these micro services and web Services ?
How different this world is different from SOA world?

Characteristics of Micro Service:

Do One thing well : Services should hide, how they work and do a single thing and do it well. The encapsulation is based on the business function of the service that it encapsulates the functional requirements.

Business Domain Centric : Each Service should align to the context with in the domain model for this to make sense in the domain and in the service architecture.

Decentralization : Every service should be decentralized and autonomous, so that each service can operate on its own schedule and priorities. Each service can be updated and deployed as fit.

Smart Endpoints and dumb pipes: This characteristic has come based on SOA, in SOA we have many products / ESB where we can define lot of logic in routing and transforming etc. Where as in Micro services the focus on login inside the service and the communication should happen very dumb like a REST / HTTP resource call to get to the service.   

Fail-Safe : Enterprise systems need to be smart, in that they fail in a way that doesn’t break things in unpredictable ways. Anticipate failure in everything, among with  validation of input and validation of data we send downstream.

Automation : Services should use automation which keeps management and operational support efficient. Employing large number of small services can become a mess if you do not embrace automation.

Principles of Micro Services – Encapsulation, Automation, Business Domain Centric, Isolation, Fail-safe and Observable.

Micro Service Vs Web Service:

The regular Web services are definitely different from Micro services. The regular web services support is based on protocols like HTTP / SOAP. These services are invented to solve the integration problems between application when we have polyglot technologies with different vendors/ applications. So, the aim of web services is to integrate or build a channel to communicate between these applications. Where as Micro services aim is NOT to integrate applications, perhaps to reduce the complexity of a monolithic application to micro services. This would give us re-use, maintainability, test-ability, scalability etc in the application.

Micro Services Vs SOA:

SOA started with very similar characteristics mentioned above, but the idea got converted when many vendors started introducing new features to SOA. Then we have a vendor lock-in problem now. We have lot of logic built in around the services NOT inside the services. So, Micro services are the out come of learning of SOA. Still, micro services are evolving. We might end up with very similar problems of SOA, if we are not cautious. Hope we would be able to use micro services the way they are explained.

Technology Support:

Here are the technology

Building Micro services with Spring Boot

Building Micro services with Drop Wizard

References:

Fowler & Lewis on MicroServices

Videos on MicroService Architecture

What is IoT ( Internet of Things ) ?

What is IoT ( Internet of Things ) ?

internet of things
Internet of things

 

We keep hearing about this new thing called IoT and Big Data. Let’s discuss the primer for today. I will try to give my understanding of IoT ( Internet of Things). There is nothing new about Internet. We all know what is internet and how we can use this for our daily life. The simplistic definition would be, we can’t imagine the life with out internet. It is part of our life. I love this image, which says “Internet is our basic need”.

Internet - Basic Human Need
Internet – Basic Human Need

Courtesy: Google ( I really don’t know who created this image)

Now, let’s get to our main point of today – IoT

Now, we are talking about Machines coming into Internet. So, each and every Machine/ Device / Thing will be connected to Internet. Let me try to give some real example I felt in my life. When I switched my internet vendor /Provider 2 years back the guy told me our Wi-Fi router can support 10 devices. I really felt 10 is too many that time as I was having only 1 smart phone at that time and 2 laptops. If I look back the same thing now, 2 smart phones (me and Wife), 1 tablet, 2 laptops, 1 kindle and the list may go up very soon… like Smart TV, smart Fridge, smart Air Conditioner, so on….  what all these devices doing on Internet and Why?

Personal scenarios:

Lets take some examples why you need your Fridge / Air Conditioner / TV on internet.

You have started from your office at 5 in the evening of a summer day. you need to travel 30 mins to reach home, you want your house cozy before you reach there. If your A/C is in internet, you would be able to switch it on when you start through your smart phone.

Your colleagues asked you for a party which is planned suddenly in office today. You want to have ice cubes ready before you get your friends home. If your Fridge is in internet, you would be able to set temperature before you start from office, and your ice cubes will be ready for the party.

Business Scenarios:

Lets think you manufacture a UPS/ Cooling unit/ Fridge / TV etc, if you don’t have this smart feature in your product there could be potential problems as below. Firstly customers might not be interested to buy your product as competitors already provide this feature for little more price. Whoever could effort little more money would buy that. Secondly, you won’t be able to analyse your product correctly. If you have this feature in your product you would be able to receive statistics of your product every second or minute. This will enable you to understand the usage of your product more and the problems in your product as well. So, You would be able to serve your customers what they are looking for.

So, As you know this is not a new thing at all. We already use this for many things in our real life. IoT is the new term tossed by industry to popularize this M2M (Machine to Machine) connectivity.

Reference: Here are some really good articles about IoT you would like to read.

IoT Blog – Schneider Electric

Cisco IoT Blog

Hope this was helpful.

Happy reading 🙂

How to explain Cloud Computing to your Granny ?

How to explain Cloud Computing to your Granny ?

In my previous post I explained the characteristics of cloud computing, perhaps people say if you know something very well you should be able to explain the thing to your granny. If you could do it successfully then you know about this technology well.

I consider this correct, So I gave this attempt. Lets hope this will workout well for you as well. I hope this would be useful for some of you as well in understanding the concepts.

Lets discuss about the Service Models of Cloud computing. IaaS, PaaS, SaaS.

Lets try to explain these models to our Granny 😉

IaaS (Infrastructure as a Service)

If you have to build a house, what is that you need before you start constructing the house ? Foundation – which is the basic need to build in our case this is  If you don’t have sufficient infrastructure at your end, you will hire the Infrastructure from these cloud companies.

If you don’t have sufficient time and energy to build the foundation of the house, you will hire the labor to build it and pay them hourly basis. The same way you will pay for the infrastructure you have used either based on the number of cores / memory utilized.

Eg: Rackspace, VMWare

PaaS (Platform as a Service)

If you don’t want to build house or you don’t have budget to build a house what you would do? Most likely you would rent a house … right ? This is much cheaper option and you will get a ready house where you can live in immediately. still you might have to spend some money on furniture like Sofa, Fridge, TV, etc.

In the same way, the service provider would provide you the platforms like Google App Engine, Microsoft Azure, Red Hat’s OpenShift, provides the platform to you where you can write your own Apps and add-ons using given platform. These platforms offer you the basic building blocks for you to start your work quickly and scale easily.

SaaS (Software as a Service)

If you take the previous example, you might think of taking a fully furnished apartment for rent. So you don’t even need to develop your own software on the platform. Just use the software and pay as per you use. This again has many models based on the application for charge back, some applications charge per size, some per request etc. Like based on the furniture and amenities you have in your house the rent will be charged. Eg SaaS services are  Microsoft’s Office 365, Prezi.com and millions of software.

Hope you can try explaining cloud computing to your grand parents too.

Happy reading 🙂

Is Service Orientation a need for today’s Enterprise ?

Is Service Orientation a need for today’s Enterprise ?

As part of my learning, I started the SOA ( Service Oriented Architecture ) in the recent past. I started reading some blogs, sites and books about SOA to understand the basics and why we need to go towards SOA for today’s enterprises. I would like to share my ideas and knowledge I got in this process. Hope this would give you some pointers to start your journey of SOA.

What is Service Orientation :  Wiki tells us that, this is a paradigm to build software in form of services. We would take it further and can tell – “Think all logically divisible parts of your program as service”. Lets take a real time example for better understanding.

Case: Lets take a house which is having a 1 Hall, 1 Kitchen and 2 bed rooms (of-course couple of bath rooms as well). If you think in terms of services what are the services you could think of to form a house.

  • Electrical Wiring
  • Plumbing
  • Doors
  • Windows
  • Taps
  • Microwave
  • Fridge
  • Shower
  • Etc.

All these things together make a house. So, shall we think all these things as services?

Why not ?

If we have these many services, are we saying our house as SOA implementation ?

Now, what does SOA principles has to say about this –

Very similar to SOLID principles in Object Orientation, SOA has their own SOA principles. We have to look at them and figure out our model is matching with the SOA principles. You might think we already have these services as web services then SOA is the same. Are web service is the only way to achieve SOA ? that is a different question to answer but to be precise the current/contemporary way of implementing SOA is web services (May be in future we might have something different). Anyway, Lets get back to Principles of SOA.

Don’t get puzzled if they look very familiar 🙂

Loose Coupling: The services defined should maintain a relationship that minimizes dependencies, and only requires the awareness of each other.

Service Contract: All services should adhere to a service agreement. This could be a service description of a single or multiple related services. Services should do only whatever is mentioned in the contract.

Autonomy: Services should have a control over the logic they encapsulate. This means that each service should do the specific task and it should not do anything more than that.

Abstraction: Beyond what is described in the service contract, services hide the logic from outside world.

Reusability: Logic is divided into services with the intention of promoting reuse.

Composability: Collection of services can be coordinated and assembled to form composite services.

Statelessness: Services should minimize/zero retaining information specific to an activity / state.

Discoverability: Services are designed to be outwardly descriptive, so that they can be found and accessed via available discovery mechanisms.

Considering these principles, If you design any SOA system that would be considered as better architecture of your application. So, I feel these are the needs of current enterprise applications. And SOA can give a perfect solution if you are able to see the needs through your Service Orientation lenses.

Here are some Good Reads about SOA:

Book: Service-Oriented Architecture: Concepts, Technology, and Designjoi (Prentice Hall Service-Oriented Computing Series from Thomas Erl)

Articles:

SOA in a Nutshell by IBM Developer works

New to SOA and Web Services?

Another Good start would be – ServiceOrientation.com

Hope this read was helpful, Happy learning and reading !!!