Saturday, April 4, 2015

Concurrency and Persistency using Chronicle Map

If you want a persistent, less heap and concurrent maps, then Chronicle Hash Map is the best one to use. Chronicle Map also helps in sharing the data between the JVMs. Earlier version of ChronicleMap is SharedHashMap.

Download

Add the following repository to your pom.xml with the following dependency.
<!-- Repository. Add inside repositories -->
<repository>
    <id>Snapshot Repository</id>
    <name>Snapshot Repository</name>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
</repository>

<!-- Add the following dependency -->
<dependency>
  <groupId>net.openhft</groupId>
  <artifactId>chronicle-map</artifactId>
  <version>2.1.4</version>
</dependency>

Points to be noted

There are few points to be noted about a ChronicleMap.
  • It stores the data onto a file for persistency.
  • In order to implement, the object (which has to be stored) must implement Serializable/Externalizable
  • Two programs can share the same file (Used while creating the map) to share the same data across JVMs.
  • Replication can be done using Chronicle Map using TCP/IP. (not detailed in the current post).
  • No need to depend on JVM Heap Memory as the data is store Off-Heap (on a file).
  • And the last important point is, the performance of the Map is purely dependent on how the Operating System allocates the memory to the file.
If we clearly observe, all these advantages or features are because of storing a data off-heap using a file.

How to create

ChronicleMap is created using ChronicleMapBuilder. See the sample code for creating the map.
        Map<Integer,String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).create();
If you want to persist the data to a file. Use like this
        File f = new File("/tmp/file.map");
        Map<Integer, String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).
                createPersistedTo(f);
Here, we are creating map and persisting to a file /tmp/file.map. As its storing the data onto a file, we can limit the size of the Map by specific number of entries it can store. See below
        Map<Integer,String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).
                entries(2000).
                createPersistedTo(new File("/tmp/file.map"));
There is a difference between Concurrent Map and Chronicle Map in size. It can't be re-sized once we fix the entries size as re-sizing is a costly operation on Map.

ChronicleMap Interface

There are few specials methods which are provided by ChronicleMap. See below

  • V getUsing(K key, V value); getUsing is same as get(key) but getUsing will return the value in value parameter without creating a new object whereas get will create a new object for returning the value with key.
  • V acquireUsing(K key, V value); acquireUsing is again same as getUsing but if there is no value defined with key, it will insert a new entry with key and returns the same value.

ChronicleMap can read and write the data from/to a JSON object. The following methods can be used to do so

  • void getAll(File toFile) throws IOException; To read map from the file which was created by another ChronicleMap using JSON format.
  • void putAll(File fromFile) throws IOException; To dump the entire map into a file using a JSON format.
  • void close(); As the data is stored off-heap, its recommended to close the map to release the heap data and persist the data.

All other methods are same as Map as it implements java.util.Map. Explore more on ChronicleMap

Happy Learning!!!!

Sunday, March 22, 2015

Format XML using Java

I have been looking for various options for format and pretty print XML string, I found few different options to do that with or without external dependencies. Even Java itself provided some APIs to do so. I will try to list down some of the ways to pretty print XML string but I will not explain the code because its simple, easy and more readable.

Java DOM API

No need of any external dependencies to run the below code. 
    public String formatXML(String input)
    {
        try
        {
            final InputSource src = new InputSource(new StringReader(input));
            final Node document = DocumentBuilderFactory.newInstance()
                    .newDocumentBuilder().parse(src).getDocumentElement();

            final DOMImplementationRegistry registry = DOMImplementationRegistry
                    .newInstance();
            final DOMImplementationLS impl = (DOMImplementationLS) registry
                    .getDOMImplementation("LS");
            final LSSerializer writer = impl.createLSSerializer();

            writer.getDomConfig().setParameter("format-pretty-print",
                    Boolean.TRUE);
            writer.getDomConfig().setParameter("xml-declaration", true);

            return writer.writeToString(document);
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

Java XML Transformer

Same as first code. This code doesn't need any external dependencies. 

    public String formatXML(String input)
    {
        try
        {
            Transformer transformer = TransformerFactory.newInstance()
                    .newTransformer();
            transformer.setOutputProperty(OutputKeys.INDENT, "yes");
            transformer.setOutputProperty(
                    "{http://xml.apache.org/xslt}indent-amount", "3");

            StreamResult result = new StreamResult(new StringWriter());
            DOMSource source = new DOMSource(parseXml(input));
            transformer.transform(source, result);
            return result.getWriter().toString();
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

    private Document parseXml(String in)
    {
        try
        {
            DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
            DocumentBuilder db = dbf.newDocumentBuilder();
            InputSource is = new InputSource(new StringReader(in));
            return db.parse(is);
        } catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }

Using Apache xerces impl

The below code needs an external dependency xerces-impl. Add the following dependency in maven.  
<dependency>
     <groupId>xerces</groupId>
     <artifactId>xercesImpl</artifactId>
     <version>2.8.0</version>
</dependency>
    public String formatXML(String input)
    {
        try
        {
            final Document document = parseXml(input);
            OutputFormat format = new OutputFormat(document);
            format.setLineWidth(65);
            format.setIndenting(true);
            format.setIndent(3);
            Writer out = new StringWriter();
            XMLSerializer serializer = new XMLSerializer(out, format);
            serializer.serialize(document);
            return out.toString();
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

    private Document parseXml(String in)
    {
        try
        {
            DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
            DocumentBuilder db = dbf.newDocumentBuilder();
            InputSource is = new InputSource(new StringReader(in));
            return db.parse(is);
        } catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }

Using dom4j API

The below code needs dom4j dependency. Add the following dependency in maven.
<dependency>
     <groupId>dom4j</groupId>
     <artifactId>dom4j</artifactId>
     <version>1.6.1</version>
</dependency>
    public String formatXML(String input)
    {
        try 
        {
            Document doc = DocumentHelper.parseText(input);  
            StringWriter sw = new StringWriter();  
            OutputFormat format = OutputFormat.createPrettyPrint();  
            format.setIndent(true);
            format.setIndentSize(3); 
            XMLWriter xw = new XMLWriter(sw, format);  
            xw.write(doc);  
            
            return sw.toString();
        }
        catch(Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }
Please comment if you know any other ways to pretty print XML using Java. Happy Learning!!!!

Sunday, March 1, 2015

Logging JDBC activities using Log4JDBC

In general terms, logging is an activity of recording what's happening in run-time of the application. Sometimes, we need logging for JDBC activities. Log4jdbc is the best way to log the jdbc activities by a Java Application.

Logging Options

The options available in log4jdbc are
  • jdbc.sqlonly: Logs all the SQLs executed by Java Code.
  • jdbc.sqltiming: Logs time taken by each SQL to execute.
  • jdbc.audit: Logs all the activities of the JDBC (Except Result sets). 
  • jdbc.resultset: Includes the audit and plus Result set activities.
  • jdbc.connection: Connection details like opening and closing the database connections will be logged.
In addition to above, if we use log4j-jdbc-remix, it adds few more
  • jdbc.resultsettable: Prints the result set in format of a table with column names. 
  • We can configure this as a datasource.
Any of these can be added to log appenders as required.

Maven Dependencies

The following are the maven dependencies to be added to configure log4jdbc (I am using Apache log4j, you can use any others implementation - java.util, logback, jakarta logging etc as required ). 
  <!-- SLF4J API -->
  <dependency>
   <groupId>org.slf4j</groupId>
   <artifactId>slf4j-api</artifactId>
   <version>1.6.4</version>
  </dependency>
 
  <!-- LOG4J -->
  <dependency>
   <groupId>org.slf4j</groupId>
   <artifactId>slf4j-log4j12</artifactId>
   <version>1.6.4</version>
  </dependency>

  <!-- LOG4J for JDBC -->
  <dependency>
   <groupId>org.lazyluke</groupId>
   <artifactId>log4jdbc-remix</artifactId>
   <version>0.2.7</version>
  </dependency>

Log4J Configuration

Configure Log4j first and add the log4jdbc loggers to it. Sample log4j.properties file as below.
I am trying to print
  • SQLs
  • SQL Timing
  • Result Set tables
So the log4j.properties file looks like

# Root logger option
log4j.rootLogger=INFO, stdout
 
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

log4j.logger.jdbc.sqlonly=INFO
log4j.logger.jdbc.sqltiming=INFO
log4j.logger.jdbc.audit=OFF
log4j.logger.jdbc.resultset=ERROR
log4j.logger.jdbc.connection=ERROR
log4j.logger.jdbc.resultsettable=ON

Change Driver and JDBC URL

  • Change the driver from <your implementation driver> to net.sf.log4jdbc.DriverSpy 
  • Prefix the JDBC Connection URL with jdbc:log4jdbc instead of jdbc:
Example: I am using MySQL. So, I changed
  • JDBC Driver from com.mysql.jdbc.Driver to net.sf.log4jdbc.DriverSpy 
  • MySQL JDBC Connection string changed from jdbc:mysql://localhost:3306/test to jdbc:log4jdbc:mysql://localhost:3306/test
By default, log4jdbc loads all the drivers. If we want to load only the specified drivers, use the following system properties otherwise all the drivers will be loaded, and the system will pick the required driver based on the implementation.
  • -Dlog4jdbc.drivers=<driverclass>[,<driverclass>...] : To load the specific drivers
  • -Dlog4jdbc.auto.load.popular.drivers=false: To switch off loading all other drivers except mentioned by above

Log4JDBC Log

Sample Log printed with above configuration while accessing the data to/from database as below.

2015-03-01 18:07:04 INFO  Main:19 - Started
2015-03-01 18:07:05 INFO  sqlonly:236 - INSERT INTO Student VALUES(5,'Veeresham',25,'M') 
2015-03-01 18:07:05 INFO  sqltiming:357 - INSERT INTO Student VALUES(5,'Veeresham',25,'M')  {executed in 180 msec}
2015-03-01 18:07:05 INFO  sqlonly:236 - select * from Student 
2015-03-01 18:07:05 INFO  sqltiming:357 - select * from Student  {executed in 1 msec}
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  resultsettable:108 - |id |name      |age |gender |
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  resultsettable:108 - |1  |John      |20  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |2  |Lary      |28  |F      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |3  |Blob      |22  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |5  |Veeresham |25  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  Main:28 - Completed

When we set SQLs to true, log will replace the placeholders with the actual arguments set to it.

Try to set and unset the parameters and see different levels of information.


Happy Learning!!!!

Saturday, February 28, 2015

Java Web-Services and Overview

What is Web-service

Web-service is a piece of software which is available on internet and accessible to client using XML (Standard Inter-operable format). 

Types of Web-services

Web-services are categorized into two types based on the type of implementation. They are "BIG" web-services and "REST"ful web-services.

"BIG" Web-services

This is also called as JAX-WS (Java API for XML Web-Services). Forget about the definition, we will see the terms which denote the BIG Web-services.
  • SOAP: An XML language which is used for communication between Client and Server. Abbreviated to Simple Object Access Protocol
  • WSDL: The operations exposed or available on the Web-service are defined using a machine-readable language called WSDL (Web-Service Definition Language). 
Going forward, we will see more about JAX-WS and example (may be in subsequent posts).

RESTful Web-services

RESTful web-services mostly integrated with HTTP for request and response without depending on WSDL and XML messaging. RESTful (Representational State Transfer (RESTful) web service) functionality is available in JAX-RS.
As the funtionality depends on HTTP, it's stateless and more suited for ad-hoc and simple scenarios.

How it works

Think of a scenario, how we access a web-site. There is no major difference between accessing a web-site or a web-service except for few differences. First is, the response by a web-site is a human-readable where-as the response by web-service is intended for another program (mostly machine readable). Web-site content can have text, multimedia or graphical content etc, where-as the web-service returns content in XML, JSON etc formats. 
We will see contract, request and response formats/samples in JAX-WS (JAX-RS is not in the scope of the post). Just to remind
  • WSDL defines set of contracts that means what are the operations are allowed on the web-service. 
  • SOAP is used for carrying request and sending the response back to the client.
Sample WSDL is as below. When we look at the operation tag in bindings we will find the operations supported by the Web-service

<definitions name="MyService"
   targetNamespace="http://www.examples.com/wsdl/GetDetails.wsdl"
   xmlns="http://schemas.xmlsoap.org/wsdl/"
   xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
   xmlns:tns="http://www.examples.com/wsdl/HelloService.wsdl"
   xmlns:xsd="http://www.w3.org/2001/XMLSchema">
 
   <message name="GetDetailsRequest">
      <part name="firstName" type="xsd:string"/>
   </message>
 
   <message name="GetDetailsResponse">
      <part name="greeting" type="xsd:string"/>
   </message>

   <portType name="GetDetails_PortType">
      <operation name="getDetails">
         <input message="tns:GetDetailsRequest"/>
         <output message="tns:GetDetailsResponse"/>
      </operation>
   </portType>

   <binding name="GetDetails_Binding" type="tns:GetDetails_PortType">
      <soap:binding style="rpc"
         transport="http://schemas.xmlsoap.org/soap/http"/>
      <operation name="getDetails">
         <soap:operation soapAction="getDetails"/>
         <input>
            <soap:body
               encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
               namespace="urn:examples:myservice"
               use="encoded"/>
         </input>
  
         <output>
            <soap:body
               encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
               namespace="urn:examples:myservice"
               use="encoded"/>
         </output>
      </operation>
   </binding>

   <service name="Hello_Service">
      <documentation>WSDL File for MyService</documentation>
      <port binding="tns:GetDetails_Binding" name="GetDetails_Port">
         <soap:address
            location="http://www.examples.com/GetDetails/" />
      </port>
   </service>
</definitions>

SOAP Envelope is used for request and response. It consists of
  • Header (Optional): Header contains very specific information related to the application (like authentication etc).
  • Body: The actual content containing the details of the Request/Response based on the context.
  • Fault Block (Optional): Fault will be in response message if it contains an error. The details of the error, error number and error description will be inside the block.
Sample SOAP envelope, will be

<SOAP-ENV:Envelope
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
  xmlns:xsd="http://www.w3.org/1999/XMLSchema">
   <SOAP-ENV:Header>
     <!-- Header Data -->
   </SOAP-ENV:Header>
   <SOAP-ENV:Body>
     <!-- Body Info -->
     <!-- Optionally fault -->
     <SOAP-ENV:Fault>
     <faultcode xsi:type="xsd:string">SOAP-ENV:Client</faultcode>
     <faultstring xsi:type="xsd:string">Error Message</faultstring>
      </SOAP-ENV:Fault>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

In the next post, we will how to create and call the web-service in Java using JAX-WS.

Happy Learning!!!!

Monday, January 26, 2015

Cloning an Object - Deep and Shallow Copy

In Java, Cloning is a way to create an identical object adhering to some properties. clone() is one of the methods provided by the Object class. When we look at the javadoc of the clone method, it is explained as (The definition is short and modified version).

A Clone object should follow the properties even though these are not forced. ( a is an object of any type )
  • a != a.clone() must be true. 
  • a.getClass() should be equal to a.clone().getClass()
  • a.equals(a.clone()) must be true
In addition to the above properties, to make clone method to work, the class of the object must implement Cloneable interface otherwise, a cached exception CloneNotSupportedException will be thrown. (Even thought the Cloneable interface doesn't hold the method clone() but it should be implemented to clone an object).

Shallow and Deep Copying

Java default implementation of clone method clones only primitive members and copies the references of the other class type variables. This is Shallow Coping. Just call super.clone() inside the clone method. 

Example of Shallow Copy

Rectangle.java
public class Rectangle implements Cloneable
{
    private Long length;
    private Long breadth;
    
    public Rectangle(Long l, Long b)
    {
        this.length = l;
        this.breadth = b;
    }
    //getters and setters are ignored.

    @Override
    protected Object clone() throws CloneNotSupportedException
    {
        return super.clone();
    }
    @Override
    public boolean equals(Object obj)
    {
        if(obj instanceof Rectangle)
        {
            Rectangle other = (Rectangle)obj;
            return (other.length == length && other.breadth == breadth);
        }
        return false;
    }
    @Override
    public int hashCode()
    {
        int hashCode = 0;
        if(length != null)
            hashCode += length.hashCode();
        if(breadth != null)
            hashCode += breadth.hashCode();
        return hashCode;
    }
    @Override
    public String toString()
    {
        StringBuffer buffer = new StringBuffer();
        buffer.append("Length : ");
        buffer.append(length);
        buffer.append("; Breadth : ");
        buffer.append(breadth);
        return buffer.toString();
    }
}

Sample Execution
        Rectangle r = new Rectangle(10L, 12L);
        try
        {
            Rectangle s = (Rectangle)r.clone();
            System.out.println("r is : "+r);
            System.out.println("s is : "+s);
            System.out.println("Properties");
            System.out.println("r == s : "+(r == s));
            System.out.println("r.equals(s) : "+(r.equals(s)));
            System.out.println("r.getClass() == s.getClass() : "+(r.getClass() == s.getClass()));
        } catch (CloneNotSupportedException e)
        {
            e.printStackTrace();
        }

Output looks like:
 
r is : Length : 10; Breadth : 12
s is : Length : 10; Breadth : 12
Properties
r == s : false
r.equals(s) : true
r.getClass() == s.getClass() : true

Points to be noted

  • No need to write any implementation as Java by default does the shallow copying. 
  • clone method always return object of type Object
  • clone method throws CloneNotSupportedException
  • call super.clone() when only shallow copying is required Otherwise we need to copy the remaining objects.
  • Rectangle method implemented Cloneable Interface otherwise clone() cannot be called on the object of type Rectangle. 
  • hashCode and equals methods also to be implemented otherwise equals() doesn't return true when object and it's clone are compared.

Deep Copying

If the class contains non-primitive type members, the default implementation copies the references instead of creating a copy. So, the cloned object won't be a real copy. In order to clone the object with non-primitive members, we should explicitly copy the members. 

Example of Deep Copy

Person.java
public class Person implements Cloneable
{
    private String name;
    private Address address;

    //Getters and setters ignored.

    @Override
    protected Object clone() throws CloneNotSupportedException
    {
        Person p = (Person) super.clone();
        p.setAddress((Address)getAddress().clone());
        return p;
    }
    @Override
    public int hashCode()
    {
        int hashCode = 0;
        if(name != null)
            hashCode += name.hashCode();
        if(address != null)
            hashCode += address.hashCode();
        return hashCode;
    }
    @Override
    public boolean equals(Object obj)
    {
        if(obj instanceof Person)
        {
            Person other = (Person)obj;
            return (other.getName().equals(name) && other.getAddress().equals(getAddress()));
        }
        return false;
    }
    
}

Address.java
public class Address implements Cloneable
{
    private String city;
    private String country;
    //Getters and setters are ignored.

    @Override
    protected Object clone() throws CloneNotSupportedException
    {
        return super.clone();
    }
    @Override
    public int hashCode()
    {
        int hashCode = 0;
        if(city != null)
            hashCode += city.hashCode();
        if(country != null)
            hashCode += country.hashCode();
        return hashCode;
    }
    @Override
    public boolean equals(Object obj)
    {
        if(obj instanceof Address)
        {
            Address other = (Address)obj;
            return (other.getCity().equals(city) && other.getCountry().equalsIgnoreCase(country));        
        }
        return false;
    }
}

Points to be noted

  • Person class contains an member of type Address. If clone is not called on address type, then only name of the person will be copied onto the cloned object.
  • Instead of using the clone method on the Address object, we can copy field by field. (little clumsy though, if we wish we can).
  • The three properties still hold on all the objects which are Cloneable (Person and Address both).
  • To make clone to work, either all the sub-classes need to implement Cloneable or write the logic to copy the members.
And finally, the three properties which are followed by above classes need not to be satisfied, or Java doesn't force to implement but it's always good practice to make the class to follow if it has to be cloned. Otherwise write a simple method copy to create a new copied object, instead of using Clone method. 

Happy Learning!!!!