Debugging MQ Java Client Applications

I find it very difficult to debug the Java client application written for MQ. Most of the error messages are self explanatory but some of the error messages are difficult to diagnose and find the cause. Enable tracing would really help to identify the root of the problem. In this post, we will see how to enable tracing on Java applications.

MQ Trace

Simple MQ Trace enabling can be done by adding a JVM parameter like this

java -Dcom.ibm.mq.commonservices=trace.properties ...

Create trace.properties with the following attributes

Diagnostics.MQ=enabled
Diagnostics.Java=explorer,wmqjavaclasses,all
Diagnostics.Java.Trace.Detail=high
Diagnostics.Java.Trace.Destination.File=enabled
Diagnostics.Java.Trace.Destination.Console=disabled
Diagnostics.Java.Trace.Destination.Pathname=/tmp/trace
Diagnostics.Java.FFDC.Destination.Pathname=/tmp/FFDC
Diagnostics.Java.Errors.Destination.Filename=/tmp/errors/AMQJERR.LOG

Points to be noted

• Add libraries of IBM JRE (From the directory $JRE_HOME/lib) to CLASSPATH
• Create the directories /tmp/FFDC, /tmp/trace and /tmp/errors
• Add read permission to trace.properties
• Keep an eye on the log file size and make sure to disable logging when not required. Otherwise, it will keep filling the disk space. (Logs size will become some gigs in few hours easily)

One you start the Java Client, it will start printing the trace into the mentioned files. If you want to disable the tracing, make the Diagnostics.MQ property value to disabled so that trace will be stopped. 

JSSE Debug

In case of secure connection with MQ, if you want to enable debug for JSSE, then you can use the JVM parameter java.net.debug with different values

-Djavax.net.debug=true

This prints full trace of JSSE, this can be limited to only handshake by changing the value of the parameter to ssl:handshake

-Djavax.net.debug=ssl:handshake

This prints only the trace related to handshake, all other trace will be simply ignored.

Happy Learning!!!!

Concurrency and Persistency using Chronicle Map

If you want a persistent, less heap and concurrent maps, then Chronicle Hash Map is the best one to use. Chronicle Map also helps in sharing the data between the JVMs. Earlier version of ChronicleMap is SharedHashMap.

Download

Add the following repository to your pom.xml with the following dependency.

<!-- Repository. Add inside repositories -->
<repository>
    <id>Snapshot Repository</id>
    <name>Snapshot Repository</name>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
</repository>

<!-- Add the following dependency -->
<dependency>
  <groupId>net.openhft</groupId>
  <artifactId>chronicle-map</artifactId>
  <version>2.1.4</version>
</dependency>

Points to be noted

There are few points to be noted about a ChronicleMap.

• It stores the data onto a file for persistency.
• In order to implement, the object (which has to be stored) must implement Serializable/Externalizable
• Two programs can share the same file (Used while creating the map) to share the same data across JVMs.
• Replication can be done using Chronicle Map using TCP/IP. (not detailed in the current post).
• No need to depend on JVM Heap Memory as the data is store Off-Heap (on a file).
• And the last important point is, the performance of the Map is purely dependent on how the Operating System allocates the memory to the file.

If we clearly observe, all these advantages or features are because of storing a data off-heap using a file.

How to create

ChronicleMap is created using ChronicleMapBuilder. See the sample code for creating the map.

        Map<Integer,String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).create();

If you want to persist the data to a file. Use like this

        File f = new File("/tmp/file.map");
        Map<Integer, String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).
                createPersistedTo(f);

Here, we are creating map and persisting to a file /tmp/file.map. As its storing the data onto a file, we can limit the size of the Map by specific number of entries it can store. See below

        Map<Integer,String> map = ChronicleMapBuilder.of(Integer.class,
                String.class).
                entries(2000).
                createPersistedTo(new File("/tmp/file.map"));

There is a difference between Concurrent Map and Chronicle Map in size. It can't be re-sized once we fix the entries size as re-sizing is a costly operation on Map.

ChronicleMap Interface

There are few specials methods which are provided by ChronicleMap. See below

• V getUsing(K key, V value); getUsing is same as get(key) but getUsing will return the value in value parameter without creating a new object whereas get will create a new object for returning the value with key.
• V acquireUsing(K key, V value); acquireUsing is again same as getUsing but if there is no value defined with key, it will insert a new entry with key and returns the same value.

ChronicleMap can read and write the data from/to a JSON object. The following methods can be used to do so

• void getAll(File toFile) throws IOException; To read map from the file which was created by another ChronicleMap using JSON format.
• void putAll(File fromFile) throws IOException; To dump the entire map into a file using a JSON format.
• void close(); As the data is stored off-heap, its recommended to close the map to release the heap data and persist the data.

All other methods are same as Map as it implements java.util.Map. Explore more on ChronicleMap

Happy Learning!!!!

Format XML using Java

I have been looking for various options for format and pretty print XML string, I found few different options to do that with or without external dependencies. Even Java itself provided some APIs to do so. I will try to list down some of the ways to pretty print XML string but I will not explain the code because its simple, easy and more readable.

Java DOM API

No need of any external dependencies to run the below code. 

    public String formatXML(String input)
    {
        try
        {
            final InputSource src = new InputSource(new StringReader(input));
            final Node document = DocumentBuilderFactory.newInstance()
                    .newDocumentBuilder().parse(src).getDocumentElement();

            final DOMImplementationRegistry registry = DOMImplementationRegistry
                    .newInstance();
            final DOMImplementationLS impl = (DOMImplementationLS) registry
                    .getDOMImplementation("LS");
            final LSSerializer writer = impl.createLSSerializer();

            writer.getDomConfig().setParameter("format-pretty-print",
                    Boolean.TRUE);
            writer.getDomConfig().setParameter("xml-declaration", true);

            return writer.writeToString(document);
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

Java XML Transformer

Same as first code. This code doesn't need any external dependencies. 

    public String formatXML(String input)
    {
        try
        {
            Transformer transformer = TransformerFactory.newInstance()
                    .newTransformer();
            transformer.setOutputProperty(OutputKeys.INDENT, "yes");
            transformer.setOutputProperty(
                    "{http://xml.apache.org/xslt}indent-amount", "3");

            StreamResult result = new StreamResult(new StringWriter());
            DOMSource source = new DOMSource(parseXml(input));
            transformer.transform(source, result);
            return result.getWriter().toString();
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

    private Document parseXml(String in)
    {
        try
        {
            DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
            DocumentBuilder db = dbf.newDocumentBuilder();
            InputSource is = new InputSource(new StringReader(in));
            return db.parse(is);
        } catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }

Using Apache xerces impl

The below code needs an external dependency xerces-impl. Add the following dependency in maven.  

<dependency>
     <groupId>xerces</groupId>
     <artifactId>xercesImpl</artifactId>
     <version>2.8.0</version>
</dependency>

    public String formatXML(String input)
    {
        try
        {
            final Document document = parseXml(input);
            OutputFormat format = new OutputFormat(document);
            format.setLineWidth(65);
            format.setIndenting(true);
            format.setIndent(3);
            Writer out = new StringWriter();
            XMLSerializer serializer = new XMLSerializer(out, format);
            serializer.serialize(document);
            return out.toString();
        } catch (Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

    private Document parseXml(String in)
    {
        try
        {
            DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
            DocumentBuilder db = dbf.newDocumentBuilder();
            InputSource is = new InputSource(new StringReader(in));
            return db.parse(is);
        } catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }

Using dom4j API

The below code needs dom4j dependency. Add the following dependency in maven.

<dependency>
     <groupId>dom4j</groupId>
     <artifactId>dom4j</artifactId>
     <version>1.6.1</version>
</dependency>

    public String formatXML(String input)
    {
        try 
        {
            Document doc = DocumentHelper.parseText(input);  
            StringWriter sw = new StringWriter();  
            OutputFormat format = OutputFormat.createPrettyPrint();  
            format.setIndent(true);
            format.setIndentSize(3); 
            XMLWriter xw = new XMLWriter(sw, format);  
            xw.write(doc);  
            
            return sw.toString();
        }
        catch(Exception e)
        {
            e.printStackTrace();
            return input;
        }
    }

Please comment if you know any other ways to pretty print XML using Java. Happy Learning!!!!

Logging JDBC activities using Log4JDBC

In general terms, logging is an activity of recording what's happening in run-time of the application. Sometimes, we need logging for JDBC activities. Log4jdbc is the best way to log the jdbc activities by a Java Application.

Logging Options

The options available in log4jdbc are

• jdbc.sqlonly: Logs all the SQLs executed by Java Code.
• jdbc.sqltiming: Logs time taken by each SQL to execute.
• jdbc.audit: Logs all the activities of the JDBC (Except Result sets). 
• jdbc.resultset: Includes the audit and plus Result set activities.
• jdbc.connection: Connection details like opening and closing the database connections will be logged.

In addition to above, if we use log4j-jdbc-remix, it adds few more

• jdbc.resultsettable: Prints the result set in format of a table with column names. 
• We can configure this as a datasource.

Any of these can be added to log appenders as required.

Maven Dependencies

The following are the maven dependencies to be added to configure log4jdbc (I am using Apache log4j, you can use any others implementation - java.util, logback, jakarta logging etc as required ). 

  <!-- SLF4J API -->
  <dependency>
   <groupId>org.slf4j</groupId>
   <artifactId>slf4j-api</artifactId>
   <version>1.6.4</version>
  </dependency>
 
  <!-- LOG4J -->
  <dependency>
   <groupId>org.slf4j</groupId>
   <artifactId>slf4j-log4j12</artifactId>
   <version>1.6.4</version>
  </dependency>

  <!-- LOG4J for JDBC -->
  <dependency>
   <groupId>org.lazyluke</groupId>
   <artifactId>log4jdbc-remix</artifactId>
   <version>0.2.7</version>
  </dependency>

Log4J Configuration

Configure Log4j first and add the log4jdbc loggers to it. Sample log4j.properties file as below.
I am trying to print

• SQLs
• SQL Timing
• Result Set tables

So the log4j.properties file looks like

# Root logger option
log4j.rootLogger=INFO, stdout
 
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

log4j.logger.jdbc.sqlonly=INFO
log4j.logger.jdbc.sqltiming=INFO
log4j.logger.jdbc.audit=OFF
log4j.logger.jdbc.resultset=ERROR
log4j.logger.jdbc.connection=ERROR
log4j.logger.jdbc.resultsettable=ON

Change Driver and JDBC URL

• Change the driver from <your implementation driver> to net.sf.log4jdbc.DriverSpy 
• Prefix the JDBC Connection URL with jdbc:log4jdbc instead of jdbc:

Example: I am using MySQL. So, I changed

• JDBC Driver from com.mysql.jdbc.Driver to net.sf.log4jdbc.DriverSpy 
• MySQL JDBC Connection string changed from jdbc:mysql://localhost:3306/test to jdbc:log4jdbc:mysql://localhost:3306/test

By default, log4jdbc loads all the drivers. If we want to load only the specified drivers, use the following system properties otherwise all the drivers will be loaded, and the system will pick the required driver based on the implementation.

• -Dlog4jdbc.drivers=<driverclass>[,<driverclass>...] : To load the specific drivers
• -Dlog4jdbc.auto.load.popular.drivers=false: To switch off loading all other drivers except mentioned by above

Log4JDBC Log

Sample Log printed with above configuration while accessing the data to/from database as below.

2015-03-01 18:07:04 INFO  Main:19 - Started
2015-03-01 18:07:05 INFO  sqlonly:236 - INSERT INTO Student VALUES(5,'Veeresham',25,'M') 
2015-03-01 18:07:05 INFO  sqltiming:357 - INSERT INTO Student VALUES(5,'Veeresham',25,'M')  {executed in 180 msec}
2015-03-01 18:07:05 INFO  sqlonly:236 - select * from Student 
2015-03-01 18:07:05 INFO  sqltiming:357 - select * from Student  {executed in 1 msec}
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  resultsettable:108 - |id |name      |age |gender |
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  resultsettable:108 - |1  |John      |20  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |2  |Lary      |28  |F      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |3  |Blob      |22  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |5  |Veeresham |25  |M      |
2015-03-01 18:07:05 INFO  resultsettable:108 - |---|----------|----|-------|
2015-03-01 18:07:05 INFO  Main:28 - Completed

When we set SQLs to true, log will replace the placeholders with the actual arguments set to it.

Try to set and unset the parameters and see different levels of information.

Happy Learning!!!!

Java Web-Services and Overview

What is Web-service

Web-service is a piece of software which is available on internet and accessible to client using XML (Standard Inter-operable format). 

Types of Web-services

Web-services are categorized into two types based on the type of implementation. They are "BIG" web-services and "REST"ful web-services.

"BIG" Web-services

This is also called as JAX-WS (Java API for XML Web-Services). Forget about the definition, we will see the terms which denote the BIG Web-services.

• SOAP: An XML language which is used for communication between Client and Server. Abbreviated to Simple Object Access Protocol
• WSDL: The operations exposed or available on the Web-service are defined using a machine-readable language called WSDL (Web-Service Definition Language). 

Going forward, we will see more about JAX-WS and example (may be in subsequent posts).

RESTful Web-services

RESTful web-services mostly integrated with HTTP for request and response without depending on WSDL and XML messaging. RESTful (Representational State Transfer (RESTful) web service) functionality is available in JAX-RS.
As the funtionality depends on HTTP, it's stateless and more suited for ad-hoc and simple scenarios.

How it works

Think of a scenario, how we access a web-site. There is no major difference between accessing a web-site or a web-service except for few differences. First is, the response by a web-site is a human-readable where-as the response by web-service is intended for another program (mostly machine readable). Web-site content can have text, multimedia or graphical content etc, where-as the web-service returns content in XML, JSON etc formats. 

We will see contract, request and response formats/samples in JAX-WS (JAX-RS is not in the scope of the post). Just to remind

• WSDL defines set of contracts that means what are the operations are allowed on the web-service. 
• SOAP is used for carrying request and sending the response back to the client.

Sample WSDL is as below. When we look at the operation tag in bindings we will find the operations supported by the Web-service

<definitions name="MyService"
   targetNamespace="http://www.examples.com/wsdl/GetDetails.wsdl"
   xmlns="http://schemas.xmlsoap.org/wsdl/"
   xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
   xmlns:tns="http://www.examples.com/wsdl/HelloService.wsdl"
   xmlns:xsd="http://www.w3.org/2001/XMLSchema">
 
   <message name="GetDetailsRequest">
      <part name="firstName" type="xsd:string"/>
   </message>
 
   <message name="GetDetailsResponse">
      <part name="greeting" type="xsd:string"/>
   </message>

   <portType name="GetDetails_PortType">
      <operation name="getDetails">
         <input message="tns:GetDetailsRequest"/>
         <output message="tns:GetDetailsResponse"/>
      </operation>
   </portType>

   <binding name="GetDetails_Binding" type="tns:GetDetails_PortType">
      <soap:binding style="rpc"
         transport="http://schemas.xmlsoap.org/soap/http"/>
      <operation name="getDetails">
         <soap:operation soapAction="getDetails"/>
         <input>
            <soap:body
               encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
               namespace="urn:examples:myservice"
               use="encoded"/>
         </input>
  
         <output>
            <soap:body
               encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
               namespace="urn:examples:myservice"
               use="encoded"/>
         </output>
      </operation>
   </binding>

   <service name="Hello_Service">
      <documentation>WSDL File for MyService</documentation>
      <port binding="tns:GetDetails_Binding" name="GetDetails_Port">
         <soap:address
            location="http://www.examples.com/GetDetails/" />
      </port>
   </service>
</definitions>

SOAP Envelope is used for request and response. It consists of

• Header (Optional): Header contains very specific information related to the application (like authentication etc).
• Body: The actual content containing the details of the Request/Response based on the context.
• Fault Block (Optional): Fault will be in response message if it contains an error. The details of the error, error number and error description will be inside the block.

Sample SOAP envelope, will be

<SOAP-ENV:Envelope
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
  xmlns:xsd="http://www.w3.org/1999/XMLSchema">
   <SOAP-ENV:Header>
     <!-- Header Data -->
   </SOAP-ENV:Header>
   <SOAP-ENV:Body>
     <!-- Body Info -->
     <!-- Optionally fault -->
     <SOAP-ENV:Fault>
     <faultcode xsi:type="xsd:string">SOAP-ENV:Client</faultcode>
     <faultstring xsi:type="xsd:string">Error Message</faultstring>
      </SOAP-ENV:Fault>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

In the next post, we will how to create and call the web-service in Java using JAX-WS.

Happy Learning!!!!