Saturday, February 20, 2010

EHCache with Spring AOP

Building on the previous post (Spring AOP with @AspectJ annotations), we now add EHCache functionality to the Aspect class.  We won't be adding any new Java class files, but we'll be modifying the existing class files.  Here's a screenshot of the finished project in Eclipse.  New elements are highlighted in red.




Adding EHCache functionality

Add the EHCache JAR file to the project.  We'll be using ehcache-core-1.7.0.jar.  Grab your copy from http://www.ehcache.org.  You'll also need to create configuration file for EHCache.  Name it ehcache.xml and put it under the com.aop package.  Here's the content of the file:

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="ehcache.xsd" updateCheck="false">

    <diskStore path="java.io.tmpdir"/>

    <defaultCache
            maxElementsInMemory="10000"
            eternal="false"
            timeToIdleSeconds="120"
            timeToLiveSeconds="120"
            overflowToDisk="true"
            diskSpoolBufferSizeMB="30"
            maxElementsOnDisk="10000000"
            diskPersistent="false"
            diskExpiryThreadIntervalSeconds="120"
            memoryStoreEvictionPolicy="LRU"
            />


    <cache name="defaultCache"
           maxElementsInMemory="100"
           eternal="false"
           timeToIdleSeconds="1800"
           timeToLiveSeconds="3600"          
           overflowToDisk="false"
           diskPersistent="false"
           memoryStoreEvictionPolicy="LRU"
            />

</ehcache>
Next, we'll need to configure EHCache to be available in Spring.  For this, the app-ctx.xml needs to be edited.  Add the following lines to the app-ctx.xml file:


<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
    <property name="configLocation" value="classpath:com/aop/ehcache.xml"></property>
</bean>

<bean id="methodCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
    <property name="cacheManager">
        <ref local="cacheManager" />
    </property>
    <property name="cacheName" value="defaultCache"/>
</bean>


The Caching Aspect Class

Now that Spring and EHCache have been configured, we can modify the MyAspect.java class file to include the caching features.  We begin by changing the PointCut in the MyAspect.java.  Instead of intercepting all public methods, we now only intercept public methods beginning with getMap.  In addition, we a private Cache property to hold the method cache as well as some logic to store/retrieve the method cache.  Here's the updated MyAspect.java class:

package com.aop;

import net.sf.ehcache.Cache;
import net.sf.ehcache.Element;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;

@Aspect
public class MyAspect {
   
    private Cache methodCache;
   
    @Around("execution(public * getMap*(..))")
    public Object executor(ProceedingJoinPoint point) throws Throwable {

        System.out.println("Cutting before...");
       
        Object retVal;
       
        String methodName = point.getTarget().getClass().toString() + "," + point.toShortString();

        if (methodCache.isKeyInCache(methodName)) {
            System.out.println("Returning from cache....");
            return methodCache.get(methodName).getObjectValue();
        } else {
            retVal = point.proceed();
           
            if (retVal != null) {
                System.out.println("returned object: " + retVal.toString());
                System.out.println("Storing into cache...");
               
                methodCache.put(new Element(methodName, retVal));
            }
           
        }

        System.out.println("Cutting after...");
       
        return retVal;
    }

    public Cache getMethodCache() {
        return methodCache;
    }

    public void setMethodCache(Cache methodCache) {
        this.methodCache = methodCache;
    }
}
As the cache key, we're using a combination of the class name and the method name.  The rest of the changes are pretty straight forward. 


Spring Managed Bean - MyBean.java

We add a new method called getMap1() in MyBean.java.  Here's the code:


public HashMap<String, String> getMap1() {
   
    System.out.println("getMap1 == init");
   
    HashMap<String, String> map = new HashMap<String, String>();
   
    for (int i = 0; i < 100; i++) {
        map.put(Integer.toString(i), "new value " + i);
    }

    System.out.println("getMap1 == return");
   
    return map;
   
}


Main Runner

In the class which contains the main method, we'll be calling the getMap1() multiple times to determine if the caching is working.  The output should be like below:

Cutting before...
getMap1 == init
getMap1 == return
returned object: {35=new value 35, 36=new value 36, 33=new value 33, 34=new value 34, 39=new value 39, 37=new value 37, 38=new value 38, 43=new value 43, 42=new value 42, 41=new va
lue 41, 40=new value 40, 22=new value 22, 23=new value 23, 24=new value 24, 25=new value 25, 26=new value 26, 27=new value 27, 28=new value 28, 29=new value 29, 3=new value 3, 2=ne
w value 2, 1=new value 1, 0=new value 0, 30=new value 30, 7=new value 7, 6=new value 6, 32=new value 32, 5=new value 5, 31=new value 31, 4=new value 4, 9=new value 9, 8=new value 8
, 19=new value 19, 17=new value 17, 18=new value 18, 15=new value 15, 16=new value 16, 13=new value 13, 14=new value 14, 11=new value 11, 12=new value 12, 21=new value 21, 20=new v
alue 20, 99=new value 99, 98=new value 98, 97=new value 97, 96=new value 96, 95=new value 95, 94=new value 94, 93=new value 93, 92=new value 92, 91=new value 91, 90=new value 90, 1
0=new value 10, 88=new value 88, 89=new value 89, 79=new value 79, 78=new value 78, 77=new value 77, 82=new value 82, 83=new value 83, 80=new value 80, 81=new value 81, 86=new valu
e 86, 87=new value 87, 84=new value 84, 85=new value 85, 67=new value 67, 66=new value 66, 69=new value 69, 68=new value 68, 70=new value 70, 71=new value 71, 72=new value 72, 73=n
ew value 73, 74=new value 74, 75=new value 75, 76=new value 76, 59=new value 59, 58=new value 58, 57=new value 57, 56=new value 56, 55=new value 55, 64=new value 64, 65=new value 6
5, 62=new value 62, 63=new value 63, 60=new value 60, 61=new value 61, 49=new value 49, 48=new value 48, 45=new value 45, 44=new value 44, 47=new value 47, 46=new value 46, 51=new
value 51, 52=new value 52, 53=new value 53, 54=new value 54, 50=new value 50}
Storing into cache...
Cutting after...
Cutting before...
Returning from cache....
Cutting before...
Returning from cache....
Cutting before...
Returning from cache....

Friday, February 19, 2010

Spring AOP with @AspectJ annotations

Here's a quick sample on how to use Spring's AOP feature with @AspectJ annotations.  First off, please read the Spring Reference manual on AOP.  This post is a copy and paste tutorial (hopefully I get it right :P).  We begin by creating a normal Java Project in Eclipse.  I've named it "springtest".  You can name it whatever you like.  Here's a screenshot of the project in Eclipse:

Take note of the JAR files in the project.  Those are the JAR files required by Spring to use AOP features.  They can be found in Spring's lib and dist folders (you'll need the full Spring download with dependencies).
  • spring.jar - main Spring JAR file; obtained from the "dist" folder
  • aspectjrt.jar, aspjectjweaver.jar - AspectJ JAR file; obtained from "lib/aspectj" folder
  • commons-logging.jar - Commons logging JAR file; obtained from "lib/jakarta-commons" folder
  • cglib-nodep-2.1_3.jar - Code generation lib JAR file; obtained from "lib/cglib" folder

Spring Application Context File - app-ctx.xml
Create the Spring app context file in the com.aop package and name it app-ctx.xml.  Here's the contents of the file:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xsi:schemaLocation="
            http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
            http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">

    <aop:aspectj-autoproxy></aop:aspectj-autoproxy>
   
    <bean id="myBean" class="com.aop.MyBean"/>
   
    <bean id="myAspect" class="com.aop.MyAspect"/>
</beans>

The highlighted line is to enable @AspectJ autoproxying of beans declared. 


Aspect Class - MyAspect.java
Next, create a Java class file and name it MyAspect.java.  This file will contain the Aspect as well as the PointCut we'll be using for this simple how to.  File content shown below:

package com.aop;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;

@Aspect
public class MyAspect {
   
    @Around("execution(public * *(..))")
    public Object executor(ProceedingJoinPoint point) throws Throwable {

        System.out.println("Cutting before...");
       
        Object retVal = point.proceed();
       
        if (retVal != null) {
            System.out.println("returned object: " + retVal.toString());
        }
       
        System.out.println("Cutting after...");
       
        return retVal;
    }
}
Remember that aspectj-autoproxy declaration in the app-ctx.xml file?  Once we annotate a class file using the @Aspect, Spring AOP will automatically pickup and process the class files.

Notice that we're using the Around advice.  What this does is prints the line "Cutting before..." right before the method is executed and "Cutting after..." right after the method is executed.  If the method returns a value, print it out.  For more information on other types of advices, please refer to the Spring reference doc :)  What we're doing here is to cut around all public methods. 


The Simple Spring Managed Bean - MyBean.java
This is a simple Spring managed bean which contains methods which will be intercepted by the aspect class defined above.  Here's the file contents:

package com.aop;

public class MyBean {
    public void test() {
        System.out.println("MyBean>>>test");
    }
   
    public void test2() {
        System.out.println("MyBean>>>test2");
    }
   
    public String returnTest() {
        System.out.println("MyBean>>>returnTest");
        return "this is a test";
    }
}

The Main class - MainRunner.java
This will be the class which contains the main method.  Here's the contents:

package com.aop;

import org.springframework.context.support.ClassPathXmlApplicationContext;

public class MainRunner {
    public static void main(String[] args) {
        try {
            ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("com/aop/app-ctx.xml");
           
            MyBean bean = (MyBean) ctx.getBean("myBean");
            bean.test();
            bean.test2();
            bean.returnTest();
           
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
   
}
The first step is to obtain the application context via ClassPathXmlApplicationContext.  Then call get an instance of MyBean and subsequently call its methods (test, test2 and returnTest).  If you've done everything right, run the program and the following output should be displayed:

Cutting before...
MyBean>>>test
Cutting after...
Cutting before...
MyBean>>>test2
Cutting after...
Cutting before...
MyBean>>>returnTest
returned object: this is a test
Cutting after...
There you have it, the simplest way of AOP :)

Thursday, January 21, 2010

Load balancing OpenVPN connections via IPVS (Linux Virtual Server)

First Steps
 
First off, I've performed the setup and configuration on my own local PC using VirtualBox VMs.  Network for all the VMs are set to bridge mode.  Here's how the servers are configured:
director
--------
Fedora 12
VIP=eth0:0 192.168.1.150
RIP=eth0 192.168.1.200

ovpnserver1
-----------
Ubuntu
RIP=eth0 192.168.1.201
VIP=lo:0 192.168.1.150 (no arp)

ovpnserver2
-----------
Ubuntu
RIP=eth0 192.168.1.202
VIP=lo:0 192.168.1.150 (no arp)
Here's how the servers are laid out:
director ------- ovpnserver1
       |
       |-------- ovpnserver2
IPVS will be configured via the direct routing method; as opposed to NAT or Tunneling.


Setting Up The First OpenVPN Server (ovpnserver1)

Begin by configuring the network interface.  Here's how the /etc/network/interfaces file looks like:
# The loopback network interface
auto lo
iface lo inet loopback

# VIP routing
auto lo:0
iface lo:0 inet static
address 192.168.1.150
netmask 255.255.255.255

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.201
network 192.168.1.0
netmask 255.255.255.0
gateway 192.168.1.1
Restart networking service if necessary:
# service networking restart
Next, install OpenVPN package in the server:
# apt-get install openvpn
Now configure OpenVPN:
# cd /etc/openvpn
# cp -r /usr/share/doc/openvpn/examples/easy-rsa/2.0/ .
# mv easy-rsa rsa
# cd rsa
Edit the vars script and edit the appropriate variables at the end of the file (e.g. KEY_COUNTRY, KEY_CITY).  We then generate the necessary keys for the server:
# . ./vars
# ./clean-all
# ./build-dh
# ./pkitool --initca
# ./pkitool --server server
The CA and server keys/certs are now in /etc/openvpn/rsa/keys.  Once the keys and certs have been generated, we'll need to create a server configuration file.  For the purpose of this test, we'll authenticate the user's credentials using the supplied auth-pam.pl script.  This script authenticates OpenVPN against the system users.  Create the configuration file in /etc/openvpn/server.conf with the following contents:
port 1194
proto udp
dev tun

ca /etc/openvpn/rsa/keys/ca.crt
cert /etc/openvpn/rsa/keys/server.crt
key /etc/openvpn/rsa/keys/server.key
dh /etc/openvpn/rsa/keys/dh1024.pem

server 10.128.127.0 255.255.255.0
ifconfig-pool-persist ipp.txt

push "redirect-gateway def1"
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"

keepalive 10 120

comp-lzo

max-clients 50

persist-key
persist-tun

status openvpn-status.log
log-append /var/log/openvpn.log
verb 3
mute 20

client-cert-not-required
username-as-common-name

auth-user-pass-verify auth-pam.pl via-file
Ensure that you have Perl::PAM module installed for the script to work:
# apt-get install libauthen-pam-perl
Enable IP forwarding:
# vim /etc/sysctl.conf
Ensure that the following line is in the file:
net.ipv4.ip_forward=1
Refresh sysctl params from /etc/sysctl.conf:
# sysctl -p
Enable NAT/MASQ rules in iptables:
# iptables -t nat -A POSTROUTING -s 10.128.127.0/24 -o eth0 -j MASQUERADE
# iptables-save
Now start OpenVPN daemon:
# service openvpn restart


Setting Up The Second OpenVPN Server (ovpnserver2)

Like the first server, we begin by configuring the network interface.  Here's how the /etc/network/interfaces file looks like:
# The loopback network interface
auto lo
iface lo inet loopback

# VIP routing
auto lo:0
iface lo:0 inet static
address 192.168.1.150
netmask 255.255.255.255

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.202
network 192.168.1.0
netmask 255.255.255.0
gateway 192.168.1.1
Restart networking service if necessary:
# service networking restart
Next, install OpenVPN package in the server:
# apt-get install openvpn
Instead of configuring this instance, we'll copy the entire /etc/openvpn folder from ovpnserver1 to ovpnserver2.  This is required as the CA certs/keys must match.


Setting Up The Load Balancer (director)

Configure the network interfaces like so:
/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
HWADDR=08:00:27:81:7A:C2
IPADDR=192.168.1.200
BOOTPROTO=none
NETMASK=255.255.255.0
DNS2=208.67.222.222
TYPE=Ethernet
GATEWAY=192.168.1.1
DNS1=208.67.220.220
IPV6INIT=no
ONBOOT=yes
USERCTL=no
PREFIX=24
NAME="System eth0"
UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03

/etc/sysconfig/network-scripts/ifcfg-eth0:0

DEVICE=eth0:0
HWADDR=08:00:27:81:7A:C2
IPADDR=192.168.1.150
BOOTPROTO=none
NETMASK=255.255.255.0
DNS2=208.67.222.222
TYPE=Ethernet
GATEWAY=192.168.1.1
DNS1=208.67.220.220
IPV6INIT=no
ONBOOT=yes
USERCTL=no
PREFIX=24
NAME="System eth0:0"
UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
Turn on ipvsadm daemon:
# chkconfig ipvsadm on
# chkconfig --list ipvsadm
ipvsadm         0:off   1:off   2:on    3:on    4:on    5:on    6:off
The ipvsadm daemon reads its config file from /etc/sysconfig/ipvsadm.  Create the file with the following contents:
-A -u 192.168.1.150:1194 -s rr
-a -u 192.168.1.150:1194 -r 192.168.1.201:1194
-a -u 192.168.1.150:1194 -r 192.168.1.202:1194
ipvsadm uses the direct routing method by default.  The first line in the file instructs ipvsadm to add a virtual service.  In this case, it's a UDP service denoted by the -u parameter.  The -s rr parameter instructs it to use the round-robin scheduling method.  The subsequent two lines adds servers to the virtual service.  Since we have 2 virtual machines, we specify both the IPs.  Restart the ipvsadm service once you're done:
# service ipvsadm restart
Type the following command to verify that you have a working setup:
# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
UDP  192.168.1.150:openvpn rr
  -> 192.168.1.201:openvpn        Route   1      0          0
  -> 192.168.1.202:openvpn        Route   1      0          0




OpenVPN Client Configuration File

The client configuration file is quite straight forward.  Instead of having multiple "remote" options, you'll just have one which will be pointing to the director's IP.  The other thing to note is to add the "float" option.  Below's the configuration file for this purpose:
client
dev tun
float
proto udp
remote 192.168.1.150 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
auth-user-pass
cipher BF-CBC
comp-lzo
verb 4
mute 20
Here's a snippet from openvpn's man page on the "float" option:
--float
    Allow  remote  peer to change its IP address and/or port number, such as due to DHCP (this is the default if --remote is not used).  --float when
    specified with --remote allows an OpenVPN session to initially connect to a peer at a known address, however if packets arrive from a new address
    and  pass  all  authentication  tests,  the new address will take control of the session.  This is useful when you are connecting to a peer which
    holds a dynamic address such as a dial-in user or DHCP client.

    Essentially, --float tells OpenVPN to accept authenticated packets from any address, not only the address which was  specified  in  the  --remote
    option.

Tuesday, January 19, 2010

Mounting an NTFS partition in a folder

To mount an NTFS partition in a folder similar to what's done in Unix systems, do the following:


  • Go to the "Run..." screen and type in compmgmt.msc.
  • Click on "Disk Management".
  • Right click on the volume you want to use and select "Change Drive Letter and Paths..."
  • Click "Add".  Another window will be displayed like below:


  • Click on the second radio box "Mount in the following empty NTFS folder:".  This is where you key in the folder name where you want the partition to be mounted on.
  • Click "OK" once you're done.
Open up Windows Explorer and navigate to C:\.  You should be able to see that the "second-vol" directory has a different icon like the one shown below:



And that's it! :)

Thursday, January 14, 2010

Connecting to MQ using Spring without WAS

Here's how to connect to MQ using spring without WAS.  First, here's the properties value for the connection settings:
# MQ related values
mq.jms.qcf=IBMMQQCF
mq.jms.request.queue=TEST.REQUEST.QUEUE

# Connection details
mq.host.url=192.168.1.200:1415/SYSTEM.DEF.SVRCONN
mq.factoryclass=com.ibm.mq.jms.context.WMQInitialContextFactory

# Authentication details
mq.auth=simple
mq.user=username
mq.password=password
Now the Spring beans configuration:
    <bean id="queueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiName" value="${mq.jms.qcf}" />
        <property name="resourceRef" value="false" />
        <property name="jndiEnvironment">
            <props>
                <prop key="java.naming.factory.initial">${mq.factoryclass}</prop>
                <prop key="java.naming.provider.url">${mq.host.url}</prop>
                <prop key="java.naming.security.authentication">${mq.auth}</prop>
                <prop key="java.naming.security.principal">${mq.user}</prop>
                <prop key="java.naming.security.credentials">${mq.password}</prop>
            </props>
        </property>
    </bean>

    <bean id="requestQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiName" value="${mq.jms.request.queue}" />
        <property name="resourceRef" value="false" />
        <property name="jndiEnvironment">
            <props>
                <prop key="java.naming.factory.initial">${mq.factoryclass}</prop>
                <prop key="java.naming.provider.url">${mq.host.url}</prop>
                <prop key="java.naming.security.credentials">${mq.password}</prop>
                <prop key="java.naming.security.authentication">${mq.auth}</prop>
            </props>
        </property>
    </bean>
Once that's done, you can use the normal JMS calls to send the message to the queue.  Below is a sample of how to send a message:
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueReceiver;
import javax.jms.QueueSender;
import javax.jms.QueueSession;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.log4j.Logger;

public class JMSProcessor {
    private QueueConnectionFactory qcf;
    private Queue reqQ;
    private Queue respQ;

    private long timeout = 0;

    private Logger log = Logger.getLogger(MessageProcessor.class);

    public void sendMessage(final String msg) throws Exception {
       
        String messageOut = null;
       
        QueueConnection connection = null;
        QueueSession queueSession = null;
        QueueSender queueSender = null;

        try {
            connection = qcf.createQueueConnection();
            connection.start();
       
            queueSession = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
       
            queueSender = queueSession.createSender(reqQ);
           
            TextMessage reqMsg = queueSession.createTextMessage(msg);
           
            reqMsg.setJMSDestination(reqQ);

            queueSender.send(reqMsg);

        } catch (Exception e) {
            throw e;

        } finally {
            try {
                connection.close();
                queueSession.close();
                queueSender.close();
            } catch (JMSException e1) {
                log.error(e1.getMessage(), e1);
            }
        }
    }

}


Installing Shorewall in Ubuntu

I started off with the simple command:
# apt-get install shorewall shorewall-doc
Thought everything was installed properly.  But I got the following error instead:
# shorewall start
Compiling...
   No shorewall compiler installed
Seems I'm missing one more package:
# apt-get install shorewall-perl
Once the shorewall-perl package is you should get the following output:
# shorewall start
Compiling...
   ERROR: No firewall zone defined
Now read up on configuring Shorewall! :)

Monday, January 11, 2010

Configuring OpenVPN + FreeRADIUS + MySQL

Here's a quick guide on how to configure OpenVPN to use FreeRADIUS for authentication.  I've only tried it on Ubuntu Karmic.  YMMV on other distros.

Download the radiusplugin here http://www.nongnu.org/radiusplugin/.  You'll have to compile it from source as there's currently no deb package for it.  Compile steps below:
# apt-get install libgcrypt11 libgcrypt11-dev build-essential
# make
The output will be a single radiusplugin.so file.  Now move the .so file and the .cnf file to the proper openvpn directory like so:
# cp radiusplugin.so /etc/openvpn/
# cp radiusplugin.cnf /etc/openvpn/
First off, edit the radiusplugin.cnf file.  Focus on the "server" section and ensure that the details are correct:
server
{
        # The UDP port for radius accounting.
        acctport=1813
        # The UDP port for radius authentication.
        authport=1812
        # The name or ip address of the radius server.
        name=127.0.0.1
        # How many times should the plugin send the if there is no response?
        retry=1
        # How long should the plugin wait for a response?
        wait=1
        # The shared secret.
        sharedsecret=testing123
}
Next up, edit the openvpn server config file and add the following line:
plugin /etc/openvpn/radiusplugin.so /etc/openvpn/radiusplugin.cnf
Restart openvpn service and start freeradius service and login using a username/password pair which is defined in FreeRADIUS. 

You should now have working setup.  More info below: