Apache Solr -Unboxing

Apache Solr -Unboxing

Overview of Apache Solr:

  • Apache Solr is a standalone full-text search server with Apache Lucene at the back end.
  • Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.
  • In brief Apache Solr exposes Lucene’s JAVA API as REST like API’s which can be called over HTTP from any programming language/platform.
  • Solr is a Schema based indexing.

Why I need Solr ? 

  • Fast Indexing/Searching, Indexes can be merged/optimized (Index compaction).
  • Great admin interface can be used over HTTP.
  • Support for integration with various other products like drupal CMS, etc
  • Caching, Replication, Distributed search.
  • Full support for REST and readily available client API SolrJ
  • Awesome community support too.
  • Full Text Search
  • Faceted navigation
  • More items like this(Recommendation)/ Related searches
  • Spell Suggest/Auto-Complete
  • Custom document ranking/ordering
  • Snippet generation/highlighting
  • And many more…….

Architecture of Solr

SolrArchitecture

How to Install ?

  • Download Solr
    • Minimum Requirements
      • Directory for storing index files
      • Directory for storing configuration files
      • Solr_Homehaving other dependencies
      • A Servlet container(tomcat, jetty) with appropriate configuration

Configuring Solr

  • Solrconfig.xml
    • Contains most of the parameters for configuring Solr itself.
    • Initial Parameters
    • Request Handlers
    • Components
  • Schema.xml
    • Contains all of the details about document structure, index-time, query-time processing.
    • Field Types
    • Filters
    • Copy Fields

How to Start the Solr Server

Once you download the Solr, You can unzip the folder and go navigate to the Solr-Version\bin using command prompt.

Run Command – Solr start -p 8983 ( -p signifies the port, you can give any valid port number after the parameter)  You can see below message if it got started.

Waiting up to 30 to see Solr running on port 8983
Started Solr server on port 8983. Happy searching!

If you want to stop run – Solr stop -p 8983 ( Use the same port on which you have started)

After Start, if you want to see the  admin console of Solr, Please try going to http://localhost:8983/solr/ 

You would be able to see the screen as below.

SolrAdminInterface

Create Core

Once your Server is up and running, you may like to create a Core ( DB/Schema in RDBMS terms).

In your admin Interface, Click on “Core Admin”, then “Add Core“. Type in your schema name and Folder path etc. as shown below.

SolrCoreCreation

Note: schema.xml, solrconfig.xml names are recommended to keep as is.

Now, the Schema is ready, You can start creating you own fields in schema.xml which will be there in the path “solr-5.4.1\server\solr\new_core\conf“.

Fields / Types 

  • Create Field
    • <field name=”author” type=”string” indexed=”true” stored=”true” />
  • Custom Field
    • <fieldTypename=”text_general_ngram” class=”solr.TextField”positionIncrementGap=”100″>
      <analyzer type=”index”>
      </analyzer>
      <analyzer type=”query”>
      </analyzer>
      </fieldType>

Querying Solr 

  • Plain Text Search
    • q = text: “searchString”
  • Expanding Search to More Fields
    • title:name& type:review& price:[* To 500]
  • Add Facets
    • facet.field=product & facet.field=rating

Summary

If you want to add search to your web site, it is easy with Apache Solr, You can use the features of Solr and make your life easy for searching anything. There are many plugins available for Solr if you would like to integrate your search with RDBMS fields as well. Hope it would be useful for you.

Happy Reading 🙂

I need Bower ? or Grunt?

I need Bower ? or Grunt?

When we start learning MEAN stack, we get this question for sure. Even I asked the same question when I initially heard about these two tools. I thought they both does the same thing. To discuss this, First we need to know what these two tools does.

When we started developing every layer in Javascript, we need some tools to make our life easy. The most common one’s you need are, the dependency management and build management.

So, I would like to start with what they are…. then you would see the differences easily.

Assumption: Before you read further, I assume you already know how to use npm command and install packages like Grunt, Bower. (If no, try this link)

Let’s get started with

Bower

Why Bower ?

  • Acts as a Package Manager
  • Dependency Management of the modules
  • Bower works by fetching, installing packages and keeps them up to date.
  • Bower downloads the dependencies for you and installs them on the required path.
  • This installation could be as global module or project module.

Key Features

  • Bower runs on Git
  • Can be integrated with other tools like Grunt, Gulp etc.
  • Bower can manage any packages hosted on NPM platform
  • Can manage HTML,CSS, JS packages

In my view, whoever comes from Java,J2EE background can easily relate this to a Maven and its POM.XML to the bower.json file. In this file, you exactly give the same content what we give in Pom, the dependent libraries and their versions. This is very useful in later phases if you want to change the version of any library you are using just go hear and change the version. You are done. Even you can manually download the js file and keep it. To avoid this manual work and make developer’s life easy we need this tool.

Grunt

Why Grunt ?

  • Grunt is a Task Runner
  • Grunt is a task based command line build tool
  • Helps to automate any step while building you application
  • Can be integrated with many plug-ins
  • Very commonly used for jobs like Minification of CSS, JS & run Unit test cases

Key Features

  • Grunt helps you run tasks based on configurations
  • Provides automation for different environments like Dev, Test, etc
  • Helps you to increase the performance of the application using different plugins
  • Helps you optimize images, minify file sizes, run unit tests

In my view, Grunt acts like our good old Apache Ant, where you used to define the tasks in BUILD.XML and run them  based on the need. Very similarly Grunt helps you create your own tasks and run them based on the your requirement. Here we define Gruntfile.js where you define your tasks.

I think now, you got a better understanding of these two tools.

Now these two tools are meant for two different things. So, don’t get confused. We need both of these tools. Where as there are tools which are very similar to Grunt for eg. Gulp. We need more detailed comparison if we have to choose one of them.

Happy Reading 🙂

Angular JS – How to create a Controller ?

Angular JS – How to create a Controller ?

Here I am going to show you the ways you can create a controller in Angular JS.

I am assuming, you already know the advantages of Angular JS and how to use Angular JS basics like ng-app, ng-controller, $scope.

Just to give you some basic idea on Angular JS, It is a Web Framework which would give you lot of flexibility and de-couple your Model-View-Controller with the client side code. This has been the most popular and critical feature of Angular. Lets try to get into the controller part. The controller is very important component in the MV* pattern which Angular talks about.

This controller can be created in 3 different ways which I would show you here.

Basic Way: (Controller as Global Function)  / app.js

This way is easy to create, perhaps this is not the best way to solve the problem. As this is a global function there could be many complications when we have multiple functions defined in the app. If the app grows bigger there is no way to isolate this function from others. So, this is not the way to go.


var app = angular.module("app", []);
var TestController = function($scope){
 $scope.count = 10;
});


2nd Way (Application level Controller)

This is application level controller defined as we are defining controller on app. Tomorrow we might need many other things at application level which might be included in the same controller. So, this is the angular way of doing it, still not the best practice. There could be modular way of doing it and adding as dependency to the application (app).


var app = angular.module("app", []);
app.controller('TestController', function($scope){
 $scope.count = 10;
});

3rd Way (Module level Controller)

Here we are defining another module which is a controllers and added this module as dependency to the app module. This way tomorrow if we want to add another controller we can simple add it to this module and it will be available in the application easily. If we don’t want the new controller exposed to the app then we can simply define another module. So, here is the best practice to create a controller in Angular.


var app = angular.module("app", ['controllers']);
angular.module("controllers", []).controller('TestController', function($scope){
 $scope.count = 10;
});

Note: To keep the simplicity, I have used a very basic example to show you the differences. You can use the same HTML given below to test all the three controllers. Put the controller code in app.js file.

HTML :


<!DOCTYPE html>
<html ng-app="app">
<head>
 <title>Hello</title>
</head>
<body>

<div ng-controller="TestController"> 
 {{count}}
</div>


</body>
<script type="text/javascript" src="js/angular-min.js"></script>
<script type="text/javascript" src="app.js"></script>
</html>

Why Micro Services ?

Why Micro Services ?

Micro Services are the next big thing in the modern software architecture and development. The distributed applications in the recent past changed the world of software outlook. Now the applications are developed, used, work and built differently. This caused the logical evolution of Micro Services. Some best examples of game changers in the software world are Git ( Distributed source control), Cloud computing (AWS, Azure, …), BitCoin (Distributed virtual currency).

Let’s begin our journey with understanding, What are Micro Services?

The Philosophy

The whole idea of Micro Service architecture is that they are small, focused and should be scaled.  

The Definition

The MicroService architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – Fowler & Lewis, 2014.

This idea looks very similar to SOA. When I first heard about Micro Services, I thought they are just web Services and got many questions related to them. Some of them are as below
Why these services are called Micro Services?
What is the difference between these micro services and web Services ?
How different this world is different from SOA world?

Characteristics of Micro Service:

Do One thing well : Services should hide, how they work and do a single thing and do it well. The encapsulation is based on the business function of the service that it encapsulates the functional requirements.

Business Domain Centric : Each Service should align to the context with in the domain model for this to make sense in the domain and in the service architecture.

Decentralization : Every service should be decentralized and autonomous, so that each service can operate on its own schedule and priorities. Each service can be updated and deployed as fit.

Smart Endpoints and dumb pipes: This characteristic has come based on SOA, in SOA we have many products / ESB where we can define lot of logic in routing and transforming etc. Where as in Micro services the focus on login inside the service and the communication should happen very dumb like a REST / HTTP resource call to get to the service.   

Fail-Safe : Enterprise systems need to be smart, in that they fail in a way that doesn’t break things in unpredictable ways. Anticipate failure in everything, among with  validation of input and validation of data we send downstream.

Automation : Services should use automation which keeps management and operational support efficient. Employing large number of small services can become a mess if you do not embrace automation.

Principles of Micro Services – Encapsulation, Automation, Business Domain Centric, Isolation, Fail-safe and Observable.

Micro Service Vs Web Service:

The regular Web services are definitely different from Micro services. The regular web services support is based on protocols like HTTP / SOAP. These services are invented to solve the integration problems between application when we have polyglot technologies with different vendors/ applications. So, the aim of web services is to integrate or build a channel to communicate between these applications. Where as Micro services aim is NOT to integrate applications, perhaps to reduce the complexity of a monolithic application to micro services. This would give us re-use, maintainability, test-ability, scalability etc in the application.

Micro Services Vs SOA:

SOA started with very similar characteristics mentioned above, but the idea got converted when many vendors started introducing new features to SOA. Then we have a vendor lock-in problem now. We have lot of logic built in around the services NOT inside the services. So, Micro services are the out come of learning of SOA. Still, micro services are evolving. We might end up with very similar problems of SOA, if we are not cautious. Hope we would be able to use micro services the way they are explained.

Technology Support:

Here are the technology

Building Micro services with Spring Boot

Building Micro services with Drop Wizard

References:

Fowler & Lewis on MicroServices

Videos on MicroService Architecture

Java 8 – Re using Lambda Expressions

Java 8 – Re using Lambda Expressions

In my previous Java 8 Lambda expression blog, I tried to explain what is a Lambda expression in general, how to create a Lambda expression in Java 8 etc. Here we are going to look at the re usability perspective of the Lambda expression. It is very much required to re use the code or apply the DRY design principle when we are programming.

Let’s take some example to start with, Here I am trying to create a List of countries for each region, and trying to find the number of countries start with letter say “B” or “N”. Perhaps, to achieve the same result in the conventional way of Java we should run a loop by Iterating the list and putting a if condition and lot of unnecessary code.

Here is how the traditional code looks like


public class OldWay {
     final static List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final static List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

public static void main(String[] args) {
     int count = 0;
     for(String ctry : asia){
     if(ctry.substring(0,1).equals(&quot;N&quot;)){ count ++; }
}
     System.out.println(&quot;Countries name starting with N :&quot; + count);
     }
}

So, we have a much better way to achieve the same thing with lambda expressions. It is easy and simple. We need to tune our Java brains a bit to think in functional way. For me it took long time to tune myself 🙂


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

     public static void main(String[] args) {
          final List&lt;String&gt; asia =
          Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

          final List&lt;String&gt; europe =
          Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

         // re using the Lambda Expression ...
         final Predicate&lt;String&gt; startsWithB = name -&gt; name.startsWith(&quot;B&quot;);

         final long countStartswithBinAsia = asia.stream().filter(startsWithB).count();

         System.out.println(&quot; Asian Countries starts with B :&quot; + countStartswithBinAsia);

         final long countStartswithBinEurope = europe.stream().filter(startsWithB).count();

         System.out.println(&quot; European Countries starts with B :&quot; + countStartswithBinEurope);

       }

}

Output:


Asian Countries starts with B :2
European Countries starts with B :1

Now in the above example we have seen a Predicate which is getting re used as a method in Java. But there is a problem here, as we are not able to reuse this predicate for any other character than B. Below we would look at the Predicate as a full fledged method and could be re usable in any other class with a parameter to it. This is the best way to re use a predicate.


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

// re using the Lambda Expression ... with whatever character you want !!!
public static Predicate&lt;String&gt; startsWithCharacter(final String character) {
return name -&gt; name.startsWith(character);
}

public static void main(String[] args) {
     final List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

    final long countStartswithBinAsia = asia.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

    System.out.println(&quot; Asian Countries starts with N :&quot; + countStartswithBinAsia);

     final long countStartswithBinEurope = europe.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

     System.out.println(&quot; European Countries starts with N :&quot; + countStartswithBinEurope);

     }

}

Output:


Asian Countries starts with N :1
European Countries starts with N :2

Hope this was useful to you and in next article, I would like to discuss the FunctionalInterface in Java 8.
Thanks and Happy Reading…

What is “No” in NoSQL ?

What is “No” in NoSQL ?

We keep hearing about NoSQL these days. Many of us might think what’s the big deal if there is no “SQL”. Some of us might think OMG, there is no SQL ? How the hell the things are going to work, if there is no SQL? So, Let’s try to understand what does it mean when we say, NoSQL Database. The easiest way to confirm is Yes, there is no SQL if you use these databases. Yet, there is no need to worry too much about it. Perhaps SQL has it’s own limitations, though it has many advantages and ruling the industry for many years. The first and foremost of all is Relationship. For Eg; If you want to store company data like company name, address, departments, employees etc. You could already identify the entities and their relationships like one-one, one-many, many-one etc. If you want to aggregate the data? is there any way? Yes, there are joins, unions etc. So, lets try to fetch the data for the below page and see how many tables will be involved when we have to fetch the data for this page.

SQL-Relationships
Fig: 1

In Figure 1, we have seen the number of relationships you need to just define a structure of a company. If the same thing we have to do with a NoSQL format, how would it look like ? I mean the non relational databases.


{
'Company':'XYZ Company',
'Departments':[
'Dept1':
{ 'Name': 'HR',
'Employees':[ {'Name':'A','address':'City A', 'Sport': 'Tennis' },{'Name':'B','address':'City B', 'Sport': 'Cricket' } ]},
'Dept2':
{ 'Name': 'Admin',
'Employees':[ {'Name':'X','address':'City X', 'Phone':12121 },{'Name':'Z','address':'City Z', 'Phone':232323 } ]},
'Dept3':
{ 'Name': 'IT',
'Employees':[ {'Name':'N','address':'City N' },{'Name':'M','address':'City M' } ]}
],
'Addressg':'#55, 10 Avenue, ABC City, US'
}

This is what is called as Aggregation in NoSQL world. This structure would give you a flexibility of retrieving all the details of the company together and fast, instead of storing the data in different tables and different columns. This difference leads to many other differences between these two worlds.

If you closely observe the above JSON object, each employee object is having different attributes For Eg: HR department has  sport, Admin depart has Phone and IT department has non of these attributes. So, this is possible because these databases are scheme less. Each employee and each document might contain completely different format. This could be advantage as well as problem. Usually this is considered as advantages for NoSQL databases because we don’t need to define Schema before hand and this would give the flexibility to add new fields whenever we want in future to the same document.

References:

The Book – No SQL Distilled by Pramod and Martin Fowler would give you a very good understanding of NoSQL.

Blogs you would like to follow – nosql.mypopescu , NoSQL Weekly

Happy Reading 🙂

How to connect MongoDB using Java

How to connect MongoDB using Java

Here are the steps to connect the MongoDB from the java class.

Obviously this is not all you are looking for, this will provide you a basic idea of connecting a mongo DB using java. The easiest DB connection so far I have seen is this. All you need to do is download the driver class or include it in your pom dependency and create a main class with two method calls.

You are ready to go !!!

Step 1: Download / Add your pom dependency.

pom.xml looks like this


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.apc</groupId>
 <artifactId>m101j</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <dependencies>
 <dependency>
 <groupId>org.mongodb</groupId>
 <artifactId>mongo-java-driver</artifactId>
 <version>2.10.1</version>
 </dependency>
 </dependencies>
</project>

Step2: Java class looks like this


package com.abc.mongo;

import java.net.UnknownHostException;

import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;

public class Main {

/**
* @param args
* @throws UnknownHostException
*/
public static void main(String[] args) throws UnknownHostException {

MongoClient client = new MongoClient( new ServerAddress(&quot;localhost&quot;, 27017));

DB db = client.getDB(&quot;DBNAME&quot;); \\ Usally there is always a DB with name test

DBCollection collection = db.getCollection(&quot;COLLECTIONNAME&quot;); \\ You can easily create a collection by going to Mongo Shell.

DBObject document = collection.findOne();

System.out.println(document);
}

}

Try to run this main program and you would be able to see the output of the Collection results on the console.

Hope this was helpful. Thanks

Other Useful references :

MongoDB Commands Help

MongoDB University Where you can attend MooC