Apache Solr -Unboxing

Apache Solr -Unboxing

Overview of Apache Solr:

  • Apache Solr is a standalone full-text search server with Apache Lucene at the back end.
  • Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.
  • In brief Apache Solr exposes Lucene’s JAVA API as REST like API’s which can be called over HTTP from any programming language/platform.
  • Solr is a Schema based indexing.

Why I need Solr ? 

  • Fast Indexing/Searching, Indexes can be merged/optimized (Index compaction).
  • Great admin interface can be used over HTTP.
  • Support for integration with various other products like drupal CMS, etc
  • Caching, Replication, Distributed search.
  • Full support for REST and readily available client API SolrJ
  • Awesome community support too.
  • Full Text Search
  • Faceted navigation
  • More items like this(Recommendation)/ Related searches
  • Spell Suggest/Auto-Complete
  • Custom document ranking/ordering
  • Snippet generation/highlighting
  • And many more…….

Architecture of Solr

SolrArchitecture

How to Install ?

  • Download Solr
    • Minimum Requirements
      • Directory for storing index files
      • Directory for storing configuration files
      • Solr_Homehaving other dependencies
      • A Servlet container(tomcat, jetty) with appropriate configuration

Configuring Solr

  • Solrconfig.xml
    • Contains most of the parameters for configuring Solr itself.
    • Initial Parameters
    • Request Handlers
    • Components
  • Schema.xml
    • Contains all of the details about document structure, index-time, query-time processing.
    • Field Types
    • Filters
    • Copy Fields

How to Start the Solr Server

Once you download the Solr, You can unzip the folder and go navigate to the Solr-Version\bin using command prompt.

Run Command – Solr start -p 8983 ( -p signifies the port, you can give any valid port number after the parameter)  You can see below message if it got started.

Waiting up to 30 to see Solr running on port 8983
Started Solr server on port 8983. Happy searching!

If you want to stop run – Solr stop -p 8983 ( Use the same port on which you have started)

After Start, if you want to see the  admin console of Solr, Please try going to http://localhost:8983/solr/ 

You would be able to see the screen as below.

SolrAdminInterface

Create Core

Once your Server is up and running, you may like to create a Core ( DB/Schema in RDBMS terms).

In your admin Interface, Click on “Core Admin”, then “Add Core“. Type in your schema name and Folder path etc. as shown below.

SolrCoreCreation

Note: schema.xml, solrconfig.xml names are recommended to keep as is.

Now, the Schema is ready, You can start creating you own fields in schema.xml which will be there in the path “solr-5.4.1\server\solr\new_core\conf“.

Fields / Types 

  • Create Field
    • <field name=”author” type=”string” indexed=”true” stored=”true” />
  • Custom Field
    • <fieldTypename=”text_general_ngram” class=”solr.TextField”positionIncrementGap=”100″>
      <analyzer type=”index”>
      </analyzer>
      <analyzer type=”query”>
      </analyzer>
      </fieldType>

Querying Solr 

  • Plain Text Search
    • q = text: “searchString”
  • Expanding Search to More Fields
    • title:name& type:review& price:[* To 500]
  • Add Facets
    • facet.field=product & facet.field=rating

Summary

If you want to add search to your web site, it is easy with Apache Solr, You can use the features of Solr and make your life easy for searching anything. There are many plugins available for Solr if you would like to integrate your search with RDBMS fields as well. Hope it would be useful for you.

Happy Reading 🙂

Java 8 – Re using Lambda Expressions

Java 8 – Re using Lambda Expressions

In my previous Java 8 Lambda expression blog, I tried to explain what is a Lambda expression in general, how to create a Lambda expression in Java 8 etc. Here we are going to look at the re usability perspective of the Lambda expression. It is very much required to re use the code or apply the DRY design principle when we are programming.

Let’s take some example to start with, Here I am trying to create a List of countries for each region, and trying to find the number of countries start with letter say “B” or “N”. Perhaps, to achieve the same result in the conventional way of Java we should run a loop by Iterating the list and putting a if condition and lot of unnecessary code.

Here is how the traditional code looks like


public class OldWay {
     final static List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final static List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

public static void main(String[] args) {
     int count = 0;
     for(String ctry : asia){
     if(ctry.substring(0,1).equals(&quot;N&quot;)){ count ++; }
}
     System.out.println(&quot;Countries name starting with N :&quot; + count);
     }
}

So, we have a much better way to achieve the same thing with lambda expressions. It is easy and simple. We need to tune our Java brains a bit to think in functional way. For me it took long time to tune myself 🙂


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

     public static void main(String[] args) {
          final List&lt;String&gt; asia =
          Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

          final List&lt;String&gt; europe =
          Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

         // re using the Lambda Expression ...
         final Predicate&lt;String&gt; startsWithB = name -&gt; name.startsWith(&quot;B&quot;);

         final long countStartswithBinAsia = asia.stream().filter(startsWithB).count();

         System.out.println(&quot; Asian Countries starts with B :&quot; + countStartswithBinAsia);

         final long countStartswithBinEurope = europe.stream().filter(startsWithB).count();

         System.out.println(&quot; European Countries starts with B :&quot; + countStartswithBinEurope);

       }

}

Output:


Asian Countries starts with B :2
European Countries starts with B :1

Now in the above example we have seen a Predicate which is getting re used as a method in Java. But there is a problem here, as we are not able to reuse this predicate for any other character than B. Below we would look at the Predicate as a full fledged method and could be re usable in any other class with a parameter to it. This is the best way to re use a predicate.


package com.ravi.lambda.advanced;

import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;

public class ExampleCollections {

// re using the Lambda Expression ... with whatever character you want !!!
public static Predicate&lt;String&gt; startsWithCharacter(final String character) {
return name -&gt; name.startsWith(character);
}

public static void main(String[] args) {
     final List&lt;String&gt; asia =
     Arrays.asList(&quot;India&quot;, &quot;Nepal&quot;, &quot;China&quot;, &quot;Singapore&quot;, &quot;Srilanka&quot;, &quot;Bhutan&quot;, &quot;Bangladesh&quot;);

     final List&lt;String&gt; europe =
     Arrays.asList(&quot;Netherlands&quot;, &quot;Denmark&quot;, &quot;Sweden&quot;, &quot;Norway&quot;, &quot;Ireland&quot;, &quot;Britain&quot;, &quot;Spain&quot;);

    final long countStartswithBinAsia = asia.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

    System.out.println(&quot; Asian Countries starts with N :&quot; + countStartswithBinAsia);

     final long countStartswithBinEurope = europe.stream().filter(startsWithCharacter(&quot;N&quot;)).count();

     System.out.println(&quot; European Countries starts with N :&quot; + countStartswithBinEurope);

     }

}

Output:


Asian Countries starts with N :1
European Countries starts with N :2

Hope this was useful to you and in next article, I would like to discuss the FunctionalInterface in Java 8.
Thanks and Happy Reading…

How to connect MongoDB using Java

How to connect MongoDB using Java

Here are the steps to connect the MongoDB from the java class.

Obviously this is not all you are looking for, this will provide you a basic idea of connecting a mongo DB using java. The easiest DB connection so far I have seen is this. All you need to do is download the driver class or include it in your pom dependency and create a main class with two method calls.

You are ready to go !!!

Step 1: Download / Add your pom dependency.

pom.xml looks like this


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.apc</groupId>
 <artifactId>m101j</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <dependencies>
 <dependency>
 <groupId>org.mongodb</groupId>
 <artifactId>mongo-java-driver</artifactId>
 <version>2.10.1</version>
 </dependency>
 </dependencies>
</project>

Step2: Java class looks like this


package com.abc.mongo;

import java.net.UnknownHostException;

import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;

public class Main {

/**
* @param args
* @throws UnknownHostException
*/
public static void main(String[] args) throws UnknownHostException {

MongoClient client = new MongoClient( new ServerAddress(&quot;localhost&quot;, 27017));

DB db = client.getDB(&quot;DBNAME&quot;); \\ Usally there is always a DB with name test

DBCollection collection = db.getCollection(&quot;COLLECTIONNAME&quot;); \\ You can easily create a collection by going to Mongo Shell.

DBObject document = collection.findOne();

System.out.println(document);
}

}

Try to run this main program and you would be able to see the output of the Collection results on the console.

Hope this was helpful. Thanks

Other Useful references :

MongoDB Commands Help

MongoDB University Where you can attend MooC

How to create a Quartz Job?

How to create a Quartz Job?

The Quartz Jobs can be created with a very simple program by just adding the required libraries to the class path of your project. Here I will try to explain two different ways to configure a Quartz job. This complete post is divided into 3 sections.

Section 1: Common steps for both the approaches

Section 2: Simple Job with out any XML/Properties file involved

Section 3: With XML files and properties files to manage the Jobs.

So, lets put our hands in dirt.

Tools you need before you start this example: Eclipse Juno/Kepler.

Section 1: Common steps to create a Quartz Job/Scheduler

Step 1: Download the Jars for Quartz either from Maven or manual download.

Step 2: Create a Java project  {I am going with older approach}.

Step 3: Refer your downloaded jars in your class path (by Right Click on Project -> Build path -> Add External Archives ) See screen shot below, You could even use jta-1.1.jar instead of jboss-transaction-spi. But don’t miss this jar as this is a mandatory dependency for Quartz 2.2.1.

Quartz_reference_lib

Note: Once these steps are over either you can choose Section 2 or Section 3. [I recommend Section 3]

Section 2:  Simple Job using Java Code

Step 1: Create a class which implements org.quartz.Job interface.

Step 2: Implement execute() method of this interface.

Step 3: Write your business logic inside this execute method i.e. what has to be done when your job gets executed.

See below code as example

package com.ravi.schedulers;

import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

public class ExampleQuartzJob implements Job{

    @Override
    public void execute(JobExecutionContext arg0) throws JobExecutionException    {
       // TODO Auto-generated method stub
       System.out.println("Logic Goes here........");
    }
}

Create another class.

package com.ravi.schedulers.test;

import org.quartz.JobBuilder;
import org.quartz.JobDetail;
import org.quartz.Scheduler;
import org.quartz.SchedulerException;
import org.quartz.SchedulerFactory;
import org.quartz.SimpleScheduleBuilder;
import org.quartz.Trigger;
import org.quartz.TriggerBuilder;
import org.quartz.impl.StdSchedulerFactory;

public class TestBJob {

    public static void main(String[] args) {
        try {
            // Create Job details 
            JobDetail job = JobBuilder.newJob(TestJob.class)
                            .withIdentity("testJob")
                            .build();
           // Create the trigger with running interval of the job
           Trigger trigger = TriggerBuilder.newTrigger()
                          .withSchedule(SimpleScheduleBuilder.simpleSchedule()
                          .withIntervalInSeconds(30)
                          .repeatForever()).build();
          //This is how you start the job
          SchedulerFactory schFactory = new StdSchedulerFactory();
          Scheduler scheduler = schFactory.getScheduler();
          // tricky step, if you don't start the scheduler, it won't run.
          scheduler.start();
          // This is the thing which will run your Job at the triggered time/interval.
          scheduler.scheduleJob(job, trigger);
 
          //Use it to stop the scheduler. 
          //sch.shutdown();
      } catch (SchedulerException e) {
        e.printStackTrace();
      }
   }
}

Here are the next steps ….

You might get some warning as below as I missed the dependency of slf4j –

SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation

But the program will run as it is not a mandatory dependency. If you want to avoid warning please add this jar as well in your references.

Lets move on to other section.

Section 3: Scheduler Jobs with XML, Properties files.

Step1: Create a XML file with name quartz-config.xml (You can even use another name :), just for identification I am using this name.)

<?xml version="1.0" encoding="UTF-8"?>
<job-scheduling-data
xmlns="http://www.quartz-scheduler.org/xml/JobSchedulingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.quartz-scheduler.org/xml/JobSchedulingData

http://www.quartz-scheduler.org/xml/job_scheduling_data_1_8.xsd"

version="1.8">

<schedule>
<job>
<name>FirstJob</name>
<group>Group1</group>
<description>This is Test Job</description>
<job-class>com.ravi.schedulers.ExampleQuartzJob</job-class>
</job>
<trigger>
<cron>
<name>dummyTrigger</name>
<job-name>FirstJob</job-name>
<job-group>Group1</job-group>
<!-- It will run every 2 seconds -->
<cron-expression>0/2 * * * * ?</cron-expression>
</cron>
</trigger>
</schedule>
</job-scheduling-data>

Step2: Create a properties file with name quartz.properties, in the same folder.

org.quartz.scheduler.instanceName = ExampleQuartzJob
org.quartz.threadPool.threadCount = 3
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore
org.quartz.plugin.jobInitializer.class =org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin
org.quartz.plugin.jobInitializer.fileNames = com/ravi/schedulers/quartz-config.xml
org.quartz.plugin.jobInitializer.failOnFileNotFound = true

Once you are done with these two changes, you have to create a main method class to test this job.

package com.ravi.schedulers;

import javax.naming.NamingException;
import org.quartz.Scheduler;
import org.quartz.SchedulerException;
import org.quartz.SchedulerFactory;
import org.quartz.impl.StdSchedulerFactory;

public class QuartzRunner {
     public static void main(String[] args) throws SchedulerException, NamingException {
         // Creating scheduler factory and scheduler
         SchedulerFactory factory = new StdSchedulerFactory(
                     "com/ravi/schedulers/quartz.properties");
         Scheduler scheduler = factory.getScheduler();
         // Start scheduler
         scheduler.start();
     }
}

This is how you can create a scheduler job using Quartz API. Hope this was helpful and useful for your work.

If you are using Spring-Boot, your scheduling job become dam easy….

Here is how you need to configure your scheduling in Spring boot.

Use @EnableScheduling annotation at your application class/ Main program of Spring boot.

Then simply put @Scheduled(fixedDelay=150000) on your method which need to be scheduled.

For Example, I want to update my Log data into DB every 15 seconds. Here is the code for it,


@Scheduled(fixedDelay=150000)

public void updateDBLogs() {

// Read log file

// apply business logic

// insert into DB

}

There are many options than fixed delay for scheduling … you can even try Cron job like this.

@Scheduled(cron = “0 0 0/1 * * ?”)

Happy Scheduling 🙂

Lambda Expression Basic Example – Java 8

Lambda Expression Basic Example – Java 8

Why Java 8 supports functional programming is a big debate we can avoid for now. Lets try to understand what the features Java 8 has given for supporting functional programming. First thing we hear is Java 8 has provided Lambda Expressions, According to Oracle’s Chief Architect Mark Reinhold, The Single Largest Upgrade Ever. Lets start learning the Lambda expression now.

Lambda’s are anonymous functions which doesn’t need a name, input arguments or return type. If I say like that you might think why and how they are useful ?

Shall we really use them? If you could see a very small and basic example given for Prime Numbers, you would be able to understand the simplicity and ease of use of Lambda’s. Explanation of the code is given below the example.

Note: Please look at the methods [ isPrimeNum(Old School of Coding) and isPrime( New School of Functional Coding)].

package com.ravi.lambda;

import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.IntStream;

public class PrimeNumber {

     public static void main(String[] args) {

          // Old Style
          System.out.println(&amp;amp;quot; 1 is Prime ? -&amp;amp;quot; + isPrimeNum(1));
          System.out.println(&amp;amp;quot; 2 is Prime ? -&amp;amp;quot; + isPrimeNum(2));
          System.out.println(&amp;amp;quot; 3 is Prime ? -&amp;amp;quot; + isPrimeNum(3));
          System.out.println(&amp;amp;quot; 4 is Prime ? -&amp;amp;quot; + isPrimeNum(4));

          // New Style
          System.out.println(&amp;amp;quot; 1 is Prime ? -&amp;amp;quot; + isPrime(1));
          System.out.println(&amp;amp;quot; 2 is Prime ? -&amp;amp;quot; + isPrime(2));
          System.out.println(&amp;amp;quot; 3 is Prime ? -&amp;amp;quot; + isPrime(3));
          System.out.println(&amp;amp;quot; 4 is Prime ? -&amp;amp;quot; + isPrime(4));

     }
     /**
     *
     * @param number
     * @return
     */
     static boolean isPrimeNum(Integer number){
          if(number == 1 ){
              return false;
          }
          for(int i=2; i&amp;amp;lt; number; i++){
               if(number % i == 0 ) {
                    return false;
               }
          }
     return true;
     }

     /**
     *
     * @param number
     * @return
     */
     static boolean isPrime(int number){
     // Functional or Declarative Way of doing
     return number &amp;amp;gt; 1 &amp;amp;amp;&amp;amp;amp; IntStream.range(2, number).noneMatch( index -&amp;amp;gt; number % index == 0 );
     }
}

Lets look at the isPrimeNum method. Did you notice anything strange ? I guess no, because we know that Java works that way.

Loop over all numbers which are greater than 2 till n-1, as 1 is not a prime number and any number can be divisible by 1, we would like to start with 2. So, we had to handle the case of 1 separately.

Instead of Looping the code we could do it with Functional way –  isPrime demonstrates that code.

The easiest way to identify a Lambda Expression is “->“. A Lambda can take Optional Parameters, and executes the expressions or statements with in the braces.  Let me explain the above example.

First condition we are checking is number greater than 1 and Stream is the new API given by Java 8 [Which can be used in place of Collections… Read More ] which provides sequence of events. I have used “IntStream” which is defined for Integers, which has this Static Method called Range (Runs the loop from 2 till the given number )and noneMatch (returns a Boolean value, whether no elements of this stream match the provided predicate )instance method which take Predicate as input, this is where we have used a Lambda expression to see the number is divisible by the index.

Lambda’s has too many things, I hope this is a very good starting point for you. I would try to give you how to make it generic and re use a Lambda expression in next blog.

References:

Oracle Blog about Lambda Expressions

Lambda Expression FAQ