Wednesday, March 29, 2017

What is Spring Framework

Spring Framework is a Java platform that provides comprehensive infrastructure support for developing Java applications. Spring facilitates the applications building from “plain old Java objects” (POJOs) and to apply enterprise services non-invasively to POJOs. This capability applies to the Java SE programming model and to full and partial Java EE.

Some of the Spring platform advantages are:

* Make a Java method execute in a transaction without having to deal with transaction APIs.
* Make a local Java method a remote procedure without having to deal with remote APIs.
* Make a local Java method a message handler without having to deal with JMS APIs

Design Patterns which are used on Spring project - Part 1

Composite Pattern

In order to manager the nested nature of the tasks we will use the composite pattern, which allows each Task object to have references to other Task objects below/within it. The Major and Minor Task objects will also keep track of other Task related information such as due date, and description (see Class Diagram).

* Task = a Task object is an interface used to describe both MajorTask objects and MinorTask
objects.
* Major Task = A MajorTask object is one that include a reference to 1 to many subtasks.
* MinorTask = a MinorTask object or “Leaf” is a task with no tasks below it.

Memento Pattern

The memento pattern is used when “snapshots” of an object’s current state want to be saved for possible reference. Here, we use the pattern to control document versioning.

* Document = this object holds a reference to the actual file we are currently working with, as
well as other pertinent information like revision number and tags.
* Version = This is the snapshot of the document, which keeps track of the state of the Document
object when it asked to create this version of Memento of itself.
* CareTaker = the CareTaker keeps track of all the different Version objects as they are created
over time. It maintains them all so that any one of them can be referenced or reverted back to
in the future if necessary

Strategy Pattern

We use the Strategy Pattern in relation to roles and permissions of users. Each Role represents a set of
permissions allowable in the system. Each User will hold an instance of a Role object using one or more of the following permissions “strategies”. This will allow for easy adjustment of permissions.
■ GrantWorker
■ GrantOwner
■ Technical Admin
■ ReadOnly/Auditor
■ Grant Manager
■ Financial Analyst

Factory Method
We used the Factory Method pattern for the creation of Workflows to get rid of some overhead when they are created. Certain workflows can be designed with tasks already in place to help with the grant process. When creating a workflow, the WorkFlowFactory will create all of the necessary tasks “behind the scenes”.

* WorkFlow = an interface of the different kinds of workflows
■ Research
■ CFA
■ Reapply
■ Audit
* WorkFlowFactory = creates the specific workflows

Struts 2 Architecture

Spring - Inversion of control and dependency injection


The concept behind Inversion of Control (IoC) is often expressed in the Hollywood Principle: "Don't call me, I'll call you.". IoC moves the responsibility for making things happen into the framework, and away from application code.

Dependency Injection (DI) is a form of IoC that removes explicit dependence on container APIs; ordinary Java methods are used to inject dependencies such as collaborating objects or configuration values into application object instances. Where configuration is concerned this means that while in traditional container architectures such as EJB, a component might call the container to say "where's object X, which I need to do my work", with Dependency Injection the container figures out that the component needs an X object, and provides it to it at runtime. The container does this figuring out based on method signatures (usually JavaBean properties or constructors) and, possibly, configuration data such as XML.

Tuesday, March 28, 2017

Externalization in Java

Before going into what externalization is, you need to have some knowledge on what serialization is because externalization is nothing but serialization but an alternative for it and Externalizable interface extends Serializable interface. Check Serialization article for information on serialization. Just as an overview, Serialization is the process of converting an object's state (including its references) to a sequence of bytes, as well as the process of rebuilding those bytes into a live object at some future time. Serialization can be achieved by an object by implementing Serializable interface or Externalizable interface.
Well, when serialization by implementing Serializable interface is serving your purpose, why should you go for externalization?
Good question! Serializing by implementing Serializable interface has some issues. Lets see one by one what they are.
  • Serialization is a recursive algorithm. What I mean to say here is, apart from the fields that are required, starting from a single object, until all the objects that can be reached from that object by following instance variables, are also serialized. This includes the super class of the object until it reaches the "Object" class and the same way the super class of the instance variables until it reaches the "Object" class of those variables. Basically all the objects that it can read. This leads to lot of overheads. Say for example, you need only car type and licence number but using serialization, you cannot stop there. All the information that includes description of car, its parts, blah blah will be serialized. Obviously this slows down the performance.

  • Both serializing and deserializing require the serialization mechanism to discover information about the instance it is serializing. Using the default serialization mechanism, will use reflection to discover all the field values. Also the information about class description is added to the stream which includes the descption of all the serializable superclasses, the description of the class and the instance data associated with the specific instance of the class. Lots of data and metadata and again performance issue.

  • You know that serialization needs serialVersionUID, a unique Id to identify the information persisted. If you dont explicitly set a serialiVersionUID, serialization will compute the serialiVersionUID by going through all the fields and methods. So based on the size of the class, again serialization mechanism takes respective amount of time to calculate the value. A third performance issue.

  • Above three points confirm serialization has performance issues. Apart from performance issues,
  • When an object that implements Serializable interface, is serialized or de-serialized, no constructor of the object is called and hence any initialization which is done in the constructor cannot be done. Although there is an alternative of writing all initialization logic in a separate method and call it in constructor and readObject methods so that when an object is created or deserialized, the initialization process can happen but it definitely is a messy approach.

The solution for all the above issues is Externalization. Cool. Here enters the actual topic.
So what is externalization?
Externalization is nothing but serialization but by implementing Externalizable interface to persist and restore the object. To externalize your object, you need to implement Externalizable interface that extends Serializable interface. Here only the identity of the class is written in the serialization stream and it is the responsibility of the class to save and restore the contents of its instances which means you will have complete control of what to serialize and what not to serialize. But with serialization the identity of all the classes, its superclasses, instance variables and then the contents for these items is written to the serialization stream. But to externalize an object, you need a default public constructor.
Unlike Serializable interface, Externalizable interface is not a marker interface and it provides two methods - writeExternal and readExternal. These methods are implemented by the class to give the class a complete control over the format and contents of the stream for an object and its supertypes. These methods must explicitly coordinate with the supertype to save its state. These methods supersede customized implementations of writeObject and readObject methods.
How serialization happens? JVM first checks for the Externalizable interface and if object supports Externalizable interface, then serializes the object using writeExternal method. If the object does not support Externalizable but implement Serializable, then the object is saved using ObjectOutputStream. Now when an Externalizable object is reconstructed, an instance is created first using the public no-arg constructor, then the readExternal method is called. Again if the object does not support Externalizable, then Serializable objects are restored by reading them from an ObjectInputStream.
Lets see a simple example.
import java.io.*;

public class Car implements Externalizable {

    String name;
    int year;

    /*
     * mandatory public no-arg constructor
     */
    public Car() { super(); }
    
    Car(String n, int y) {
 name = n;
 year = y;
    }

    /** 
     * Mandatory writeExernal method. 
     */
    public void writeExternal(ObjectOutput out) throws IOException  {
 out.writeObject(name);
 out.writeInt(year);
    }

    /** 
     * Mandatory readExternal method. 
     */
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
 name = (String) in.readObject();
 year = in.readInt();
    }

    /** 
     * Prints out the fields. used for testing!
     */
    public String toString() {
        return("Name: " + name + "\n" + "Year: " + year);
    }
}


import java.io.*;

public class ExternExample {
   
    public static void main(String args[]) {

 // create a Car object 
 Car car = new Car("Mitsubishi", 2009);
 Car newCar = null;
 
 //serialize the car
 try {
     FileOutputStream fo = new FileOutputStream("tmp");
     ObjectOutputStream so = new ObjectOutputStream(fo);
     so.writeObject(car);
     so.flush();
 } catch (Exception e) {
     System.out.println(e);
     System.exit(1);
 }

 // de-serialize the Car
 try {
     FileInputStream fi = new FileInputStream("tmp");
     ObjectInputStream si = new ObjectInputStream(fi);       
     newCar = (Car) si.readObject();
 }
 catch (Exception e) {
     System.out.println(e);
     System.exit(1);
 }

 /* 
  * Print out the original and new car information
  */
 System.out.println("The original car is ");
 System.out.println(car);
 System.out.println("The new car is ");
        System.out.println(newCar);
    }
}

In this example, class Car implements Externalizable interface which means that car object is ready for serialization. This class have two public methods - "writeExternal" and "readExternal". Unlike Serializable interface which will serialize all the variables in the object with just by implementing the interface, here you have to explicitly mention what fields or variables you want to serialize and the same is done in "writeExternal" and "readExternal" methods. So in the "ExternExample" class, when you write the "Car" object to the OutputStream, the "writeExternal" method is called and the data is persisted. The same applies to "readExternal" method in the Car object i.e., when you read the "Car" object from the ObjectInputStream, "readExternal" method is called.

What will happen when an externalizable class extends a non externalizable super class?
Then in this case, you need to persist the super class fields also in the sub class that implements Externalizable interface. Look at this example. 
/**
 * The superclass does not implement externalizable
 */
class Automobile  {
    
    /*
     * Instead of making thse members private and adding setter
     * and getter methods, I am just giving default access specifier.
     * You can make them private members and add setters and getters.
     */
    String regNo;
    String mileage;
    
    /* 
     * A public no-arg constructor 
     */
    public Automobile() {}

    Automobile(String rn, String m) {
 regNo = rn;
 mileage = m;
    }
}

public class Car extends Automobile implements Externalizable {

    String name;
    int year;

    /*
     * mandatory public no-arg constructor
     */
    public Car() { super(); }
    
    Car(String n, int y) {
 name = n;
 year = y;
    }

    /** 
     * Mandatory writeExernal method. 
     */
    public void writeExternal(ObjectOutput out) throws IOException  {
 /* 
  * Since the superclass does not implement the Serializable interface
  * we explicitly do the saving.
  */
 out.writeObject(regNo);
 out.writeObject(mileage);
    
 //Now the subclass fields
 out.writeObject(name);
 out.writeInt(year);
    }

    /** 
     * Mandatory readExternal method. 
     */
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
 /* 
  * Since the superclass does not implement the Serializable interface
  * we explicitly do the restoring
  */
 regNo = (String) in.readObject();
 mileage = (String) in.readObject();
 
 //Now the subclass fields
 name = (String) in.readObject();
 year = in.readInt();
    }

    /** 
     * Prints out the fields. used for testing!
     */
    public String toString() {
        return("Reg No: " + regNo + "\n" + "Mileage: " + mileage +
         "Name: " + name + "\n" + "Year: " + year );
    }
}
 
Here the Automobile class does not implement Externalizable interface. So to persist the fields in the automobile class the writeExternal and readExternal methods of Car class are modified to save/restore the super class fields first and then the sub class fields.
Sounds good! What if the super class implements the Externalizable interface?
Well, in this case the super class will also have the readExternal and writeExternal methods as in Car class and will persist the respective fields in these methods. 
import java.io.*;

/**
 * The superclass implements externalizable
 */
class Automobile implements Externalizable {
    
    /*
     * Instead of making thse members private and adding setter
     * and getter methods, I am just giving default access specifier.
     * You can make them private members and add setters and getters.
     */
    String regNo;
    String mileage;
    
    /* 
     * A public no-arg constructor 
     */
    public Automobile() {}

    Automobile(String rn, String m) {
 regNo = rn;
 mileage = m;
    }

    public void writeExternal(ObjectOutput out) throws IOException {
 out.writeObject(regNo);
 out.writeObject(mileage);
    }

    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        regNo = (String)in.readObject();
        mileage = (String)in.readObject();
    }

}

public class Car extends Automobile implements Externalizable {

    String name;
    int year;

    /*
     * mandatory public no-arg constructor
     */
    public Car() { super(); }
    
    Car(String n, int y) {
 name = n;
 year = y;
    }

    /** 
     * Mandatory writeExernal method. 
     */
    public void writeExternal(ObjectOutput out) throws IOException  {
 // first we call the writeExternal of the superclass as to write
 // all the superclass data fields
 super.writeExternal(out);

 //Now the subclass fields
 out.writeObject(name);
 out.writeInt(year);
    }

    /** 
     * Mandatory readExternal method. 
     */
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
 // first call the superclass external method
 super.readExternal(in);
    
 //Now the subclass fields
 name = (String) in.readObject();
 year = in.readInt();
    }

    /** 
     * Prints out the fields. used for testing!
     */
    public String toString() {
        return("Reg No: " + regNo + "\n" + "Mileage: " + mileage +
         "Name: " + name + "\n" + "Year: " + year );
    }
}

In this example since the Automobile class stores and restores its fields in its own writeExternal and readExternal methods, you dont need to save/restore the superclass fields in sub class but if you observe closely the writeExternal and readExternal methods of Car class closely, you will find that you still need to first call the super.xxxx() methods that confirms the statement the externalizable object must also coordinate with its supertype to save and restore its state.
Lets see the difference in sizes when you serialize using Serializable interface and serialize using Externalizable interface
Let's take a simple case, an object of type SimpleClass with just few fields - firstName, lastName, weight and location, containing data {"Brad", "Pitt", 180.5, {49.345, 67.567}}. When you serialize this object that is about 24 bytes by implementing Serializable interface, it turns into 220 bytes (approx). As it turns out, the basic serialization mechanism stores all kinds of information in the file so that it can deserialize without any other assistance. Look at the format below when the object is serialized and you will understand why it is turned out to 200 bytes.
Length: 220
Magic: ACED
Version: 5
  OBJECT
    CLASSDESC
    Class Name: "SimpleClass"
    Class UID:  -D56EDC726B866EBL
    Class Desc Flags: SERIALIZABLE;
    Field Count: 4
    Field type: object
    Field name: "firstName"
    Class name: "Ljava/lang/String;"
    Field type: object
    Field name: "lastName"
    Class name: "Ljava/lang/String;"
 Field type: float
    Field name: "weight"
    Field type: object
    Field name: "location"
    Class name: "Ljava/awt/Point;"
    Annotation: ENDBLOCKDATA
    Superclass description: NULL
   STRING: "Brad"
   STRING: "Pitt"
   float: 180.5
   OBJECT
      CLASSDESC
      Class Name: "java.awt.Point"
      Class UID:  -654B758DCB8137DAL
      Class Desc Flags: SERIALIZABLE;
      Field Count: 2
      Field type: integer
      Field name: "x"
      Field type: integer
      Field name: "y"
      Annotation: ENDBLOCKDATA
      Superclass description: NULL
     integer: 49.345
     integer: 67.567

Now if you serialize the same by extending Externalizable interface, the size will be reduced drastically and the information saved in the persistant store is also reduced a lot. Here is the result of serializing the same class, modified to be externalizable. Notice that the actual data is not parseable externally any more--only your class knows the meaning of the data!
Length: 54
Magic: ACED
Version: 5
  OBJECT
    CLASSDESC
    Class Name: "SimpleClass"
    Class UID:  5CB3777417A3AB5BL
    Class Desc Flags: EXTERNALIZABLE;
    Field Count: 0
    Annotation
      ENDBLOCKDATA
    Superclass description
      NULL
  EXTERNALIZABLE: 
   [70 00 04 4D 61 72 6B 00 05 44 61 76 69 73 43 3C
    80 00 00 00 00 01 00 00 00 01]

Well, externalization has its own limitations
Externalization efficiency comes at a price. The default serialization mechanism adapts to application changes due to the fact that metadata is automatically extracted from the class definitions (observe the format above and you will see that when the object is serialized by implementing Serializable interface, the class metadata(definitions) are written to the persistent store while when you serialize by implementing Externalizable interface, the class metadata is not written to the persistent store). Externalization on the other hand isn't very flexible and requires you to rewrite your marshalling and demarshalling code whenever you change your class definitions.
As you know a default public no-arg constructor will be called when serializing the objects that implements Externalizable interface. Hence, Externalizable interface can't be implemented by Inner Classes in Java as all the constructors of an inner class in Java will always accept the instance of the enclosing class as a prepended parameter and therefore you can't have a no-arg constructor for an inner class. Inner classes can achieve object serialization by only implementing Serializable interface.
If you are subclassing your externalizable class, you have to invoke your superclass’s implementation. So this causes overhead while you subclass your externalizable class. Observe the examples above where the superclass writeExternal method is explicitly called in the subclass writeExternal method.
Methods in externalizable interface are public. So any malicious program can invoke which results into loosing the prior serialized state.
Once your class is tagged with either Serializable or Externalizable, you can't change any evolved version of your class to the other format. You alone are responsible for maintaining compatibility across versions. That means that if you want the flexibility to add fields in the future, you'd better have your own mechanism so that you can skip over additional information possibly added by those future versions.
So much of it. Here are some final tips for serialization.
You can decide whether to implement Externalizable or Serializable on a class-by-class basis. Within the same application, some of your classes can be Serializable, and some can be Externalizable. This makes it easy to evolve your application in response to actual performance data and shifting requirements. You can do the following thing:
* Make all your classes implement Serializable.
* Then make some of them, the ones you send often and for which serialization is inefficient, implement Externalizable instead.
To reduce memory size:
* Write primitives or Strings directly. For example, instead of writing out a contained object, Point (in SimpleClass, we have a field of type Point), write out each of its integer coordinates separately. When you read them in, create a new Point from the two integers. This can be very significant in terms of size: an array of three Points takes 117 bytes; an array of 6 ints takes 51 bytes.
* Strings are special-cased and don't carry much of the object overhead; you will normally use them as is. However, the serialized representation of a String is UTF, which works great for ASCII characters, is neutral for most European characters, but causes a 50% increase in size for Japanese and other scripts. If you have significant strings of Asian text you better serialize a char array instead.

Serialization in Java

Serialization is the process of converting an object's state (including its references) to a sequence of bytes, as well as the process of rebuilding those bytes into a live object at some future time. Simple......Converting an object to bytes and bytes back to object. So when is serialization used? Serialization is used when you want to persist the object. It is also used by RMI to pass objects between JVMs, either as arguments in a method invocation from a client to a server or as return values from a method invocation. In general, serialization is used when we want the object to exist beyond the lifetime of the JVM.
Lets see couple of different scenarios (examples) where we use serialization.
  • 1. Banking example: When the account holder tries to withdraw money from the server through ATM, the account holder information along with the withdrawl details will be serialized (marshalled/flattened to bytes) and sent to server where the details are deserialized (unmarshalled/rebuilt the bytes)and used to perform operations. This will reduce the network calls as we are serializing the whole object and sending to server and further request for information from client is not needed by the server.
  • 2. Stock example: Lets say an user wants the stock updates immediately when he request for it. To achieve this, everytime we have an update, we can serialize it and save it in a file. When user requests the information, deserialize it from file and provide the information. This way we dont need to make the user wait for the information until we hit the database, perform computations and get the result.
Here are some uses of serialization
  • To persist data for future use.
  • To send data to a remote computer using such client/server Java technologies as RMI or socket programming.
  • To "flatten" an object into array of bytes in memory.
  • To exchange data between applets and servlets.
  • To store user session in Web applications.
  • To activate/passivate enterprise java beans.
  • To send objects between the servers in a cluster.

So far we saw what and when serialization is used.
Lets see now how serialization is performed in java.
Java provides Serialization API, a standard mechanism to handle object serialization. To persist an object in java, the first step is to flatten the object. For that the respective class should implement "java.io.Serializable" interface. Thats it. We dont need to implement any methods as this interface do not have any methods. This is a marker interface/tag interface. Marking a class as Serializable indicates the underlying API that this object can be flattened into bytes and subsequently inflated in the future.
public class SerialClass implements Serializable {
    private Date currentTime;

    public SerialClass() {
        currentTime = Calendar.getInstance().getTime();
    }

    public Date getCurrentTime() {
        return currentTime;
    }
}

Now you marked the object for flattening. Next step is to actually persist the object. To persist an object you need to use node stream to write to file systems or transfer a flattened object across a network wire and have it be rebuilt on the other side. You can use java.io.ObjectOutputStream class, a filter stream which is a wrapper around a lower-level byte stream.
So to write an object you use "writeObject(<<instance>>)" method of "java.io.ObjectOutputStream" class and to read an object you use "readObject()" method of "java.io.ObjectOutputStream" class. "readObject()" can read only serialized object, that means if the class does not implement "java.io.Serializable" interface, "readObject()" cannot read that object.
//Class to persist the time in a flat file time.ser
public class PersistSerialClass {

    public static void main(String [] args) {
        String filename = "time.ser";

        if(args.length > 0){
            filename = args[0];
        }
  
        PersistSerialClass time = new PersistSerialClass();
        FileOutputStream fos = null;
        ObjectOutputStream out = null;

        try{
            fos = new FileOutputStream(filename);
            out = new ObjectOutputStream(fos);
            out.writeObject(time);
            out.close();
        }catch(IOException ex){
            ex.printStackTrace();
        }
     }
 }


//Class to read the time from a flat file time.ser
public class ReadSerialClass {

    public static void main(String [] args) {
        String filename = "time.ser";

        if(args.length > 0){
            filename = args[0];
        }
  
        PersistSerialClass time = null;
        FileInputStream fis = null;
        ObjectInputStream in = null;

        try{
            fis = new FileInputStream(filename);
            in = new ObjectInputStream(fis);
            time = (PersistSerialClass)in.readObject();
            in.close();
        }catch(IOException ex){
            ex.printStackTrace();
        }catch(ClassNotFoundException cnfe){
            cnfe.printStackTrace();
        }

        // print out restored time
        System.out.println("Restored time: " + time.getTime());

        // print out the current time
        System.out.println("Current time: " 
   + Calendar.getInstance().getTime());

     }
 }
When you serialize an object, only the object's state will be saved, not the object's class file or methods.
When you serialize the above example class, the serialized class will look like below. Surprising.. isn't it? Yes, when you serialized a 2 byte object, you see 51 bytes serialized file. How did it convert to 51 bytes file? To know this,
Let's see step by step on how the object is serialized and de-serialized.
So when an object is serailized 
  • First it writes out the serialization stream magic data - What is serialization stream magic data? This is nothing but the STREAM_MAGIC and STREAM_VERSION data (static data) so that JVM can deserialize it when it has to. The STRAM_MAGIC will be "AC ED" and the STREAM_VERSION will be the version of the JVM.
  • Then it writes out the metadata (description) of the class associated with an instance. So in the below example after writing out the magic data, it writes out the description of "SerialClass" class. What does this description include? It includes the length of the class, the name of the class, serialVersionUID (or serial version), various flags and the number of fields in this class.
  • Then it recursively writes out the metadata of the superclass until it finds java.lang.object. Again in the below example, it writes out the description of "SerialParent" and "SerialParentParent" classes.
  • Once it finishes writing the metadata information, it then starts with the actual data associated with the instance. But this time, it starts from the top most superclass. So it starts from "SerialParentParent", then writes out "SerialParent".
  • Finally it writes the data of objects associated with the instance starting from metadata to actual content. So here it starts writing the metadata for the class Contain and then the contents of it as usual recursively.

class SerialParentParent implements Serializable {
    int parentParentVersion = 10;
}

class SerialParent implements Serializable {
    int parentVersion = 11;
}

class Contain implements Serializable{ 
    int containVersion = 20;
}

public class SerialClass extends SerialParent implements Serializable {
    int version = 12; 
    Contain con = new Contain(); 

    public int getVersion() {  
        return version; 
    }
 
    public static void main(String args[]) throws IOException {  
        FileOutputStream fos = new FileOutputStream("temp.ser");  
        ObjectOutputStream oos = new ObjectOutputStream(fos);  
        SerialClass st = new SerialClass();  
        oos.writeObject(st);  
        oos.flush();  
        oos.close(); 
    }
}

How to customize the default protocol?
MMmmmm.. Now its getting more interesting. Lets say, you need to perform some specific operations in the constructor when you are instantiating the class but you cant perform those operations when you deserialize the object because constructor wont be called when an object is de-serialized. Here we are restoring an object but not reconstructing an object. Then how will you call or perform those operations when you desrialize the object? Well, you have a way here and its simple too. You can enhance the normal process by providing two methods inside your serializable class. Those methods are:

private void writeObject(ObjectOutputStream out) throws IOException; 

private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException; 

Notice that both methods are declared private and ofcourse they must be declared private, proving that neither method is inherited and overridden or overloaded. The trick here is that the virtual machine will automatically check to see if either method is declared during the corresponding method call. The virtual machine can call private methods of your class whenever it wants but no other objects can. Thus, the integrity of the class is maintained and the serialization protocol can continue to work as normal.
public class SerialClass implements Serializable {
    private Date currentTime;

    public SerialClass() {
        calculateCurrentTime();
    }

    public Date getCurrentTime() {
        return currentTime;
    }

    private Date calculateCurrentTime(){
        currentTime = Calendar.getInstance().getTime();
    }

    private void writeObject(ObjectOutputStream out) throws IOException {
        out.defaultWriteObject();
    }

    private void readObject(ObjectInputStream in) 
     throws IOException, ClassNotFoundException{

        // our "pseudo-constructor"
        in.defaultReadObject();
        // now perfrom same operation you need to do in constructor
        calculateCurrentTime();
    }
}

Ooops. I mentioned earlier that for a class to be serializable either the class should implement "Serializable" interface or one of its super class should implement "Serializable" interface. Now what if you dont want to serialize one of the sub class of a serializable class? You have a way here tooo. To stop the automatic serialization, you can once again use the private methods to just throw the NotSerializableException in your class.
private void writeObject(ObjectOutputStream out) throws IOException{

    throw new NotSerializableException("Dont Serialize");
}

private void readObject(ObjectInputStream in) throws IOException{

    throw new NotSerializableException("Dont Serialize");
}

Well... One more way to serialize the object - the Externalizable Interface
Again there is one more way to serialize the object - create your own protocol with the Externalizable interface. Instead of implementing the Serializable interface, you can implement Externalizable, which contains two methods:

public void writeExternal(ObjectOutput out) throws IOException;

public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException;

The Externalization is discussed as separate topic. Check it out here or check the menu.
How not to serialize some fields in a serializble object?
Sometimes you dont want to serialize/store all the fields in the object. Say some fields you want to hide to preserve the privacy or some fields you may want to read only from master data, then you dont seriaalize them. To do this, you just need to declare a field as transient field.
transient private int checkPoint;
Also the static fields are not serialized. Actually there is no point in serializing static fields as static fields do not represent object state but represent class state and it can be modified by any other object. Lets assume that you have serialized a static field and its value and before deserialization of the object, the static field value is changed by some other object. Now the static field value that is serialized/stored is no more valid. Hence it make no point in serializing the static field.

Apart from declaring the field as transient, there is another tricky way of controlling what fields can be serialized and what fields cannot be. This is by overriding the writeObject() method while serialization and inside this method you are responsible for writing out the appropriate fields. When you do this, you may have to override readObject() method as well. This sound similar to using Externalizable where you will write writeExternal() and readExternal() methods but anyways lets not take this route as this is not a neat route.
Note that serialization does not care about access modifiers. It serializes all private, public and protected fields.
Nonserializable objects
Earlier we discussed about not serializing certain fields in a serializable object and why it may be needed sometimes. But now lets see why certain objects should not be serialized? As you know, the Object class does not implement Serializable interface and hence any object by default is not serializable. To make an object serializable, the respective class should explicitly implement Serializable interface. However certain system classes defined by java like "Thread", "OutputStream", "Socket" are not serializable. Why so? Lets take a step back - now what is the use of serializing the Thread running in System1 JVM using System1 memory and then deserializing it in System2 and trying to run in System2 JVM. Makes no sense right! Hence these classes are not serializable.
Ok. So far so good. Now what if you want to serialize an object containing an instance of Thread? Simple. Declare the Thread instance as transient.
public class SerialClass implements Serializable, Runnable {
    transient private Thread myThread;

    public PersistentAnimation(int animationSpeed) {
  ...
  ...
    }
}

Versioning issues
One very important item to look at is the versioning issue. Sometimes you wil get "java.io.InvalidClassException" but when you check the class (it will be Serializable class), you will find nothing wrong with it. Then what is causing this exception to be thrown? Ok. Here it is. You create a Serializable class, instantiate it, and write it out to an object stream. That flattened object sits in the file system for some time. Meanwhile, you update the class file by adding a new field. Then try to read the flattened object. InvalidClassException is thrown because all persistent-capable classes are automatically given a unique identifier. If the identifier of the class does not equal the identifier of the flattened object, the exception will be thrown and when you update the class with a new field, a new identifier will be generated.
To fix this issue, manually add the identifier to the class. The identifier that is part of all classes is maintained in a field called serialVersionUID. If you wish to control versioning, you simply have to provide the serialVersionUID field manually and ensure it is always the same, no matter what changes you make to the classfile. More about it is discussed in separate topic. Check it here.
Performance Issues/Improvement with Serialization
The default way of implementing the serialization (by implementing the Serializable interface) has performance glitches. Say you have to write an object 10000 times in a flat file through serialization, this will take much more (alomost double) the time than it takes to write the same object 10000 times to console. To overcome this issue, its always better to write your custom protocol instead of going for default option.
Also always note to close the streams (object output and input streams). The object references are cached in the output stream and if the stream is not closed, the system may not garbage collect the objects written to a stream and this will result in poor performance.
Using Serialization always have performance issues? Nope... Let me give you a situation where it is used for performance improvement. Lets assume you have an application that should display a map and pointing to different areas in the map should highlight those areas in different color. Since all these are images, when you point to each location, loading an image each time will slow the application heavily. To resolve this issue, serialization can be used. So here since the images wont change, you can serialize the image object and the respective points on the map (x and y co-ordinates) and deserialize it as and when necessary. This improves the performance greatly.
What happens to inner classes? We forgot all about it.
Yes, you can serialize inner classes by implementing the Serializable interface but it has some problems. Inner classes (declared in a non-static context) will always contain implicit references to their enclosing classes and these references are always non-transient. So, while object serialization process of inner classes, the enclosing classes will also be serialized. Now the problem is that the synthetic fields generated by Java compilers to implement inner classes are pretty much implementation dependent and hence we may face compatibility issues while deserialization on a different platform having a .class file generated by a different Java compiler. The default serialVerionUID may also be different in such cases. Not only this, the names assigned to the local and anonymous inner classes are also implementation dependent. Thus, we see that object serialization of inner classes may pose some unavoidable compatibility issues and hence the serialization of inner classes is strongly discouraged.

Monday, May 2, 2016

Take thread dumps from a JVM on UNIX or Windows?

How can I take thread dumps from a JVM on UNIX or Windows?

A thread dump is a list of all the Java threads that are currently active in a Java Virtual Machine (JVM).
There are several ways to take thread dumps from a JVM. It is highly recommended to take more than 1 thread dump. A good practice is to take 10 thread dumps at a regular interval (for example, one thread dump every ten seconds).

Step 1: Get the PID of your Java process


The first piece of information you will need to be able to obtain a thread dump is your Java process's PID.
The Java JDK ships with the jps command which lists all Java process ids. You can run this command like this:
jps -l 70660 sun.tools.jps.Jps 70305


Note: In Linux and UNIX, you may have to run this command as sudo -u user jps -l, where "user" is the username of the user that the Java process is running as.
If this doesn't work or you still cannot find your Java process, (path not set, JDK not installed, or older Java version), use
  • UNIX, Linux, and Mac OS X: ps -el | grep java
  • Windows: Press Ctrl+Shift+Esc to open the task manager and find the PID of the Java process

Step 2: Request a thread dump from the JVM  

jstack
If installed/available, we recommend using the jstack tool. It prints thread dumps to the command line console.
To obtain a thread dump using jstack, run the following command:
jstack <pid>
You can output consecutive thread dumps to a file by using the console output redirect/append directive:
jstack <pid> >> threaddumps.log
Notes:
  • The jstack tool is available since JDK 1.5 (for JVM on Windows it's available in some versions of JDK 1.5 and JDK 1.6 only).
  • jstack works even if the -Xrs jvm parameter is enabled
  • It's not possible to use the jstack tool from JDK 1.6 to take threaddumps from a process running on JDK 1.5.
  • In Linux and UNIX, you may need to run this command as sudo -u user jstack <pid> >> threaddumps.log, where "user" is the user that the Java process is running as.
  • In Windows, if you run jstack and get the error "Not enough storage is available to process this command" then you must run jstack as the windows SYSTEM user.  You can do this by using psexec which you can download here. Then you can run jstack like this: psexec -s jstack <pid> >> threaddumps.log

jstack script

Here's a script, taken from eclipse.org that will take a series of thread dumps using jstack.  It also takes the thread level cpu usage using top command as well.
#!/bin/bash
if [ $# -eq 0 ]; then
    echo >&2 "Usage: jstackSeries <pid> <run_user> [ <count> [ <delay> ] ]"
    echo >&2 "    Defaults: count = 10, delay = 0.5 (seconds)"
    exit 1
fi
pid=$1          # required
user=$2         # required
count=${3:-10}  # defaults to 10 times
delay=${4:-0.5} # defaults to 0.5 seconds
while [ $count -gt 0 ]
do
    sudo -u $user top -H -b -n1 -p $pid >top.$pid.$(date +%H%M%S.%N) &
    sudo -u $user jstack -l $pid >jstack.$pid.$(date +%H%M%S.%N)
    sleep $delay
    let count--
    echo -n "."
done

Just run it like this:
sh jstackSeries.sh [pid] [cq5serveruser] [count] [delay]
For example:
sh jstackSeries.sh 1234 cq5serveruser 10 3
  • 1234 is the pid of the Java process
  • cq5serveruser is the Linux or UNIX user that the Java process runs as
  • 10 is how many thread dumps to take
  • 3 is the delay between each dump
Note: The top output has the native thread id in decimal format while the jstack output has the nid in hexadecimal.  You can match the high cpu thread from the top output to the jstack output by converting the thread id to hexadecimal.
Thread Dump Tool for Adobe Experience Manager
If you are using Adobe Experience Manager product then you can install this tool to have a simple UI for generating thread dumps.

Alternative ways to obtain a thread dump

If the jstack tool is not available to you then you can take thread dumps as follows:

Note: Some tools cannot take thread dumps from the JVM if the commandline parameter -Xrs is enabled. If you are having trouble taking thread dumps then please see if this option is enabled.
UNIX, Mac OS X, and Linux (JDK 1.4 or lesser version)
On UNIX, Mac OS X, and Linux, you can send a QUIT signal to the Java process to tell it to output a thread dump to standard output.
  1. Run this command to do this:
    kill -QUIT <pid>
    You may need to run this command as sudo -u user kill -QUIT <pid> where "user" is the user that the Java process is running as.
  2. If you are starting CQSE using the crx-quickstart/server/start script then your thread dumps will be output to crx-quickstart/server/logs/startup.log. If you are using a third-party application server such as JBoss, WebSphere, Tomcat, or other, see the server's documentation to find out which file the standard output is directed to.
Windows:
JDK 1.X
  1. Download javadump.exe (attached below).
  2. Start the JVM with these three arguments (they must be in the right order):
    -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=C: mpjvmoutput.log
  3. Press Ctrl+Shift+Esc to open the Task Manager.
  4. Find the PID of the Java process.
  5. From the command line, run
    javadump.exe [pid]
  6. The thread dump will appear in the jvmoutput.log file mentioned in step 2.
JDK 1.6
Get a thread dump from jconsole tool, by using a plugin: [0]
Here's how you can request a thread dump:
  1. Add the following parameter to the jvm running Communique : -Dcom.sun.management.jmxremote
  2. Download and install JDK 1.6 (if not done yet).
  3. Download and extract the Thread Dump Analyzer utility. [1]
  4. Run jconsole.exe of JDK 1.6:
                                jconsole.exe -pluginpath /path/to/file/tda.jar
  5. Click the Thread dumps tab.
  6. Click the Request Thread Dump link.