Apache Solr -Unboxing

Apache Solr -Unboxing

Overview of Apache Solr:

  • Apache Solr is a standalone full-text search server with Apache Lucene at the back end.
  • Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.
  • In brief Apache Solr exposes Lucene’s JAVA API as REST like API’s which can be called over HTTP from any programming language/platform.
  • Solr is a Schema based indexing.

Why I need Solr ? 

  • Fast Indexing/Searching, Indexes can be merged/optimized (Index compaction).
  • Great admin interface can be used over HTTP.
  • Support for integration with various other products like drupal CMS, etc
  • Caching, Replication, Distributed search.
  • Full support for REST and readily available client API SolrJ
  • Awesome community support too.
  • Full Text Search
  • Faceted navigation
  • More items like this(Recommendation)/ Related searches
  • Spell Suggest/Auto-Complete
  • Custom document ranking/ordering
  • Snippet generation/highlighting
  • And many more…….

Architecture of Solr

SolrArchitecture

How to Install ?

  • Download Solr
    • Minimum Requirements
      • Directory for storing index files
      • Directory for storing configuration files
      • Solr_Homehaving other dependencies
      • A Servlet container(tomcat, jetty) with appropriate configuration

Configuring Solr

  • Solrconfig.xml
    • Contains most of the parameters for configuring Solr itself.
    • Initial Parameters
    • Request Handlers
    • Components
  • Schema.xml
    • Contains all of the details about document structure, index-time, query-time processing.
    • Field Types
    • Filters
    • Copy Fields

How to Start the Solr Server

Once you download the Solr, You can unzip the folder and go navigate to the Solr-Version\bin using command prompt.

Run Command – Solr start -p 8983 ( -p signifies the port, you can give any valid port number after the parameter)  You can see below message if it got started.

Waiting up to 30 to see Solr running on port 8983
Started Solr server on port 8983. Happy searching!

If you want to stop run – Solr stop -p 8983 ( Use the same port on which you have started)

After Start, if you want to see the  admin console of Solr, Please try going to http://localhost:8983/solr/ 

You would be able to see the screen as below.

SolrAdminInterface

Create Core

Once your Server is up and running, you may like to create a Core ( DB/Schema in RDBMS terms).

In your admin Interface, Click on “Core Admin”, then “Add Core“. Type in your schema name and Folder path etc. as shown below.

SolrCoreCreation

Note: schema.xml, solrconfig.xml names are recommended to keep as is.

Now, the Schema is ready, You can start creating you own fields in schema.xml which will be there in the path “solr-5.4.1\server\solr\new_core\conf“.

Fields / Types 

  • Create Field
    • <field name=”author” type=”string” indexed=”true” stored=”true” />
  • Custom Field
    • <fieldTypename=”text_general_ngram” class=”solr.TextField”positionIncrementGap=”100″>
      <analyzer type=”index”>
      </analyzer>
      <analyzer type=”query”>
      </analyzer>
      </fieldType>

Querying Solr 

  • Plain Text Search
    • q = text: “searchString”
  • Expanding Search to More Fields
    • title:name& type:review& price:[* To 500]
  • Add Facets
    • facet.field=product & facet.field=rating

Summary

If you want to add search to your web site, it is easy with Apache Solr, You can use the features of Solr and make your life easy for searching anything. There are many plugins available for Solr if you would like to integrate your search with RDBMS fields as well. Hope it would be useful for you.

Happy Reading 🙂

I need Bower ? or Grunt?

I need Bower ? or Grunt?

When we start learning MEAN stack, we get this question for sure. Even I asked the same question when I initially heard about these two tools. I thought they both does the same thing. To discuss this, First we need to know what these two tools does.

When we started developing every layer in Javascript, we need some tools to make our life easy. The most common one’s you need are, the dependency management and build management.

So, I would like to start with what they are…. then you would see the differences easily.

Assumption: Before you read further, I assume you already know how to use npm command and install packages like Grunt, Bower. (If no, try this link)

Let’s get started with

Bower

Why Bower ?

  • Acts as a Package Manager
  • Dependency Management of the modules
  • Bower works by fetching, installing packages and keeps them up to date.
  • Bower downloads the dependencies for you and installs them on the required path.
  • This installation could be as global module or project module.

Key Features

  • Bower runs on Git
  • Can be integrated with other tools like Grunt, Gulp etc.
  • Bower can manage any packages hosted on NPM platform
  • Can manage HTML,CSS, JS packages

In my view, whoever comes from Java,J2EE background can easily relate this to a Maven and its POM.XML to the bower.json file. In this file, you exactly give the same content what we give in Pom, the dependent libraries and their versions. This is very useful in later phases if you want to change the version of any library you are using just go hear and change the version. You are done. Even you can manually download the js file and keep it. To avoid this manual work and make developer’s life easy we need this tool.

Grunt

Why Grunt ?

  • Grunt is a Task Runner
  • Grunt is a task based command line build tool
  • Helps to automate any step while building you application
  • Can be integrated with many plug-ins
  • Very commonly used for jobs like Minification of CSS, JS & run Unit test cases

Key Features

  • Grunt helps you run tasks based on configurations
  • Provides automation for different environments like Dev, Test, etc
  • Helps you to increase the performance of the application using different plugins
  • Helps you optimize images, minify file sizes, run unit tests

In my view, Grunt acts like our good old Apache Ant, where you used to define the tasks in BUILD.XML and run them  based on the need. Very similarly Grunt helps you create your own tasks and run them based on the your requirement. Here we define Gruntfile.js where you define your tasks.

I think now, you got a better understanding of these two tools.

Now these two tools are meant for two different things. So, don’t get confused. We need both of these tools. Where as there are tools which are very similar to Grunt for eg. Gulp. We need more detailed comparison if we have to choose one of them.

Happy Reading 🙂

Angular JS – How to create a Controller ?

Angular JS – How to create a Controller ?

Here I am going to show you the ways you can create a controller in Angular JS.

I am assuming, you already know the advantages of Angular JS and how to use Angular JS basics like ng-app, ng-controller, $scope.

Just to give you some basic idea on Angular JS, It is a Web Framework which would give you lot of flexibility and de-couple your Model-View-Controller with the client side code. This has been the most popular and critical feature of Angular. Lets try to get into the controller part. The controller is very important component in the MV* pattern which Angular talks about.

This controller can be created in 3 different ways which I would show you here.

Basic Way: (Controller as Global Function)  / app.js

This way is easy to create, perhaps this is not the best way to solve the problem. As this is a global function there could be many complications when we have multiple functions defined in the app. If the app grows bigger there is no way to isolate this function from others. So, this is not the way to go.


var app = angular.module("app", []);
var TestController = function($scope){
 $scope.count = 10;
});


2nd Way (Application level Controller)

This is application level controller defined as we are defining controller on app. Tomorrow we might need many other things at application level which might be included in the same controller. So, this is the angular way of doing it, still not the best practice. There could be modular way of doing it and adding as dependency to the application (app).


var app = angular.module("app", []);
app.controller('TestController', function($scope){
 $scope.count = 10;
});

3rd Way (Module level Controller)

Here we are defining another module which is a controllers and added this module as dependency to the app module. This way tomorrow if we want to add another controller we can simple add it to this module and it will be available in the application easily. If we don’t want the new controller exposed to the app then we can simply define another module. So, here is the best practice to create a controller in Angular.


var app = angular.module("app", ['controllers']);
angular.module("controllers", []).controller('TestController', function($scope){
 $scope.count = 10;
});

Note: To keep the simplicity, I have used a very basic example to show you the differences. You can use the same HTML given below to test all the three controllers. Put the controller code in app.js file.

HTML :


<!DOCTYPE html>
<html ng-app="app">
<head>
 <title>Hello</title>
</head>
<body>

<div ng-controller="TestController"> 
 {{count}}
</div>


</body>
<script type="text/javascript" src="js/angular-min.js"></script>
<script type="text/javascript" src="app.js"></script>
</html>

Why Micro Services ?

Why Micro Services ?

Micro Services are the next big thing in the modern software architecture and development. The distributed applications in the recent past changed the world of software outlook. Now the applications are developed, used, work and built differently. This caused the logical evolution of Micro Services. Some best examples of game changers in the software world are Git ( Distributed source control), Cloud computing (AWS, Azure, …), BitCoin (Distributed virtual currency).

Let’s begin our journey with understanding, What are Micro Services?

The Philosophy

The whole idea of Micro Service architecture is that they are small, focused and should be scaled.  

The Definition

The MicroService architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – Fowler & Lewis, 2014.

This idea looks very similar to SOA. When I first heard about Micro Services, I thought they are just web Services and got many questions related to them. Some of them are as below
Why these services are called Micro Services?
What is the difference between these micro services and web Services ?
How different this world is different from SOA world?

Characteristics of Micro Service:

Do One thing well : Services should hide, how they work and do a single thing and do it well. The encapsulation is based on the business function of the service that it encapsulates the functional requirements.

Business Domain Centric : Each Service should align to the context with in the domain model for this to make sense in the domain and in the service architecture.

Decentralization : Every service should be decentralized and autonomous, so that each service can operate on its own schedule and priorities. Each service can be updated and deployed as fit.

Smart Endpoints and dumb pipes: This characteristic has come based on SOA, in SOA we have many products / ESB where we can define lot of logic in routing and transforming etc. Where as in Micro services the focus on login inside the service and the communication should happen very dumb like a REST / HTTP resource call to get to the service.   

Fail-Safe : Enterprise systems need to be smart, in that they fail in a way that doesn’t break things in unpredictable ways. Anticipate failure in everything, among with  validation of input and validation of data we send downstream.

Automation : Services should use automation which keeps management and operational support efficient. Employing large number of small services can become a mess if you do not embrace automation.

Principles of Micro Services – Encapsulation, Automation, Business Domain Centric, Isolation, Fail-safe and Observable.

Micro Service Vs Web Service:

The regular Web services are definitely different from Micro services. The regular web services support is based on protocols like HTTP / SOAP. These services are invented to solve the integration problems between application when we have polyglot technologies with different vendors/ applications. So, the aim of web services is to integrate or build a channel to communicate between these applications. Where as Micro services aim is NOT to integrate applications, perhaps to reduce the complexity of a monolithic application to micro services. This would give us re-use, maintainability, test-ability, scalability etc in the application.

Micro Services Vs SOA:

SOA started with very similar characteristics mentioned above, but the idea got converted when many vendors started introducing new features to SOA. Then we have a vendor lock-in problem now. We have lot of logic built in around the services NOT inside the services. So, Micro services are the out come of learning of SOA. Still, micro services are evolving. We might end up with very similar problems of SOA, if we are not cautious. Hope we would be able to use micro services the way they are explained.

Technology Support:

Here are the technology

Building Micro services with Spring Boot

Building Micro services with Drop Wizard

References:

Fowler & Lewis on MicroServices

Videos on MicroService Architecture

Is Service Orientation a need for today’s Enterprise ?

Is Service Orientation a need for today’s Enterprise ?

As part of my learning, I started the SOA ( Service Oriented Architecture ) in the recent past. I started reading some blogs, sites and books about SOA to understand the basics and why we need to go towards SOA for today’s enterprises. I would like to share my ideas and knowledge I got in this process. Hope this would give you some pointers to start your journey of SOA.

What is Service Orientation :  Wiki tells us that, this is a paradigm to build software in form of services. We would take it further and can tell – “Think all logically divisible parts of your program as service”. Lets take a real time example for better understanding.

Case: Lets take a house which is having a 1 Hall, 1 Kitchen and 2 bed rooms (of-course couple of bath rooms as well). If you think in terms of services what are the services you could think of to form a house.

  • Electrical Wiring
  • Plumbing
  • Doors
  • Windows
  • Taps
  • Microwave
  • Fridge
  • Shower
  • Etc.

All these things together make a house. So, shall we think all these things as services?

Why not ?

If we have these many services, are we saying our house as SOA implementation ?

Now, what does SOA principles has to say about this –

Very similar to SOLID principles in Object Orientation, SOA has their own SOA principles. We have to look at them and figure out our model is matching with the SOA principles. You might think we already have these services as web services then SOA is the same. Are web service is the only way to achieve SOA ? that is a different question to answer but to be precise the current/contemporary way of implementing SOA is web services (May be in future we might have something different). Anyway, Lets get back to Principles of SOA.

Don’t get puzzled if they look very familiar 🙂

Loose Coupling: The services defined should maintain a relationship that minimizes dependencies, and only requires the awareness of each other.

Service Contract: All services should adhere to a service agreement. This could be a service description of a single or multiple related services. Services should do only whatever is mentioned in the contract.

Autonomy: Services should have a control over the logic they encapsulate. This means that each service should do the specific task and it should not do anything more than that.

Abstraction: Beyond what is described in the service contract, services hide the logic from outside world.

Reusability: Logic is divided into services with the intention of promoting reuse.

Composability: Collection of services can be coordinated and assembled to form composite services.

Statelessness: Services should minimize/zero retaining information specific to an activity / state.

Discoverability: Services are designed to be outwardly descriptive, so that they can be found and accessed via available discovery mechanisms.

Considering these principles, If you design any SOA system that would be considered as better architecture of your application. So, I feel these are the needs of current enterprise applications. And SOA can give a perfect solution if you are able to see the needs through your Service Orientation lenses.

Here are some Good Reads about SOA:

Book: Service-Oriented Architecture: Concepts, Technology, and Designjoi (Prentice Hall Service-Oriented Computing Series from Thomas Erl)

Articles:

SOA in a Nutshell by IBM Developer works

New to SOA and Web Services?

Another Good start would be – ServiceOrientation.com

Hope this read was helpful, Happy learning and reading !!!

Code Review, Is it worth spending time?

Code Review, Is it worth spending time?

Recently, I have been asked to check and deploy a process for Design and Code review in many different projects. All projects already have this process in place either officially or un-officially. When I said process some Engineering managers said, do we really need this as a process? We are already doing it on a need basis, and we have static code analysis tools in place. So, is it worth wasting two developers (Reviewer, Author) for Code review ?

What tools can do? 

  • Check the code follows conventions like lines of code/ per method
  • Variables or method naming conventions 
  • Unit test coverage %
  • Find Duplicate Code
  • Cyclomatic complexity [Some smart tools does this too…]

Then I thought what HUMAN brain can do, NOT any tool, 

  • Have the customer requirements met ? 
  • Possible use of a Design Pattern [ Strategy, Decorator or Observer ]?
  • or a Design principles like SRP, OCP or LSP ? 
  • Is it an Anti pattern ? 
  • Use of existing library [ gson, Jackson, etc]
  • Domain Knowledge
  • Project Experience [ very important to understand the implications and regressions of the code ]                                                                                                                                   last but not least 
  • Knowledge Sharing

Your list can grow much bigger than what I have written. I feel though you have Analysis tools of your code and they definitely help you improve your code, you need somebody who is senior to you in the project and technically sound to have a look at your code and give you a fair way to improve your code. 

Please share your thoughts and experiences. 

Framework Vs Platform

Framework Vs Platform

In day to day life we use many jargons, particularly in technology industry. In the recent past I have observed that there are some words which are used interchangeably. I heard some people using Framework and Platform interchangeably very often. 

Here are my thoughts on the differences. 

Framework:

Framework provides pre-written code which are very common for most of the projects. They provide a
way to extend their code and write your own using their Framework. Eg: Spring, Hibernate are Java based Frameworks. So, It it also tells us that they might have constraints like Java / JDK is required to execute any Spring Framework code. They might or might not work on all Operating systems. Still, it is a framework which provides common functionality implementation for any Java web project. 

Final understanding we might get here, Framework is a template / Base / Generic code given to extend and write your own code for your own purpose. 

Highlights: 

  • One level of Abstraction 
  • Easy to Use (For Developers)
  • Less Complex to Understand 

Platform:

Platform is  a little bigger thing than Framework, Java is a platform as Java provides a JVM(Java Virtual Machine), SDK(Software development Kit), Libraries / APIs (Application Program Interface). Using Java you can write Desktop applications/ Web Applications etc. Java has a stack of Frameworks supporting many applications. Java is the base for many different frameworks. and JVM is the base for many different languages other than Java ( eg: Groovy, JRuby, Jython etc).

 Highlights: 

  • Multiple Levels of Abstractions 
  • Difficult to Use (For Developers)
  • More Complex to Understand

Hope this was helpful, Any new thoughts ?

Happy Reading 🙂