Installing Open Nebula 3.0 on Ubuntu 10.04

• Downloaded the .deb file from Open Nebula website (http://dev.opennebula.org/packages/opennebula-3.0.0/Ubuntu-10.04/opennebula_3.0.0-1_amd64.deb)

• Installed it but gave some problems hence had to do the following steps:

dpkg -i opennebula_3.0.0-1_amd64.deb
apt-get install libmysqlclient16 libxmlrpc-c3
apt-get -f install 

• Once that's installed, the oned daemon will be started automatically.  

• Login as oneadmin and export the ONE_AUTH variable as follow:

export ONE_AUTH=/var/lib/one/auth

• You should be able to access the one* commands as oneadmin.

EHCache with Spring AOP

Building on the previous post (Spring AOP with @AspectJ annotations), we now add EHCache functionality to the Aspect class.  We won't be adding any new Java class files, but we'll be modifying the existing class files.  Here's a screenshot of the finished project in Eclipse.  New elements are highlighted in red.

Adding EHCache functionality

Add the EHCache JAR file to the project.  We'll be using ehcache-core-1.7.0.jar.  Grab your copy from http://www.ehcache.org.  You'll also need to create configuration file for EHCache.  Name it ehcache.xml and put it under the com.aop package.  Here's the content of the file:

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="ehcache.xsd" updateCheck="false">

    <diskStore path="java.io.tmpdir"/>

    <defaultCache
            maxElementsInMemory="10000"
            eternal="false"
            timeToIdleSeconds="120"
            timeToLiveSeconds="120"
            overflowToDisk="true"
            diskSpoolBufferSizeMB="30"
            maxElementsOnDisk="10000000"
            diskPersistent="false"
            diskExpiryThreadIntervalSeconds="120"
            memoryStoreEvictionPolicy="LRU"
            />


    <cache name="defaultCache"
           maxElementsInMemory="100"
           eternal="false"
           timeToIdleSeconds="1800"
           timeToLiveSeconds="3600"          
           overflowToDisk="false"
           diskPersistent="false"
           memoryStoreEvictionPolicy="LRU"
            />

</ehcache>

Next, we'll need to configure EHCache to be available in Spring.  For this, the app-ctx.xml needs to be edited.  Add the following lines to the app-ctx.xml file:

<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
    <property name="configLocation" value="classpath:com/aop/ehcache.xml"></property>
</bean>

<bean id="methodCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
    <property name="cacheManager">
        <ref local="cacheManager" />
    </property>
    <property name="cacheName" value="defaultCache"/>
</bean>

The Caching Aspect Class

Now that Spring and EHCache have been configured, we can modify the MyAspect.java class file to include the caching features.  We begin by changing the PointCut in the MyAspect.java.  Instead of intercepting all public methods, we now only intercept public methods beginning with getMap.  In addition, we a private Cache property to hold the method cache as well as some logic to store/retrieve the method cache.  Here's the updated MyAspect.java class:

package com.aop;

import net.sf.ehcache.Cache;
import net.sf.ehcache.Element;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;

@Aspect
public class MyAspect {
   
    private Cache methodCache;
   
    @Around("execution(public * getMap*(..))")
    public Object executor(ProceedingJoinPoint point) throws Throwable {

        System.out.println("Cutting before...");
       
        Object retVal;
       
        String methodName = point.getTarget().getClass().toString() + "," + point.toShortString();

        if (methodCache.isKeyInCache(methodName)) {
            System.out.println("Returning from cache....");
            return methodCache.get(methodName).getObjectValue();
        } else {
            retVal = point.proceed();
           
            if (retVal != null) {
                System.out.println("returned object: " + retVal.toString());
                System.out.println("Storing into cache...");
               
                methodCache.put(new Element(methodName, retVal));
            }
           
        }

        System.out.println("Cutting after...");
       
        return retVal;
    }

    public Cache getMethodCache() {
        return methodCache;
    }

    public void setMethodCache(Cache methodCache) {
        this.methodCache = methodCache;
    }
}

As the cache key, we're using a combination of the class name and the method name.  The rest of the changes are pretty straight forward. 


Spring Managed Bean - MyBean.java

We add a new method called getMap1() in MyBean.java.  Here's the code:

public HashMap<String, String> getMap1() {
   
    System.out.println("getMap1 == init");
   
    HashMap<String, String> map = new HashMap<String, String>();
   
    for (int i = 0; i < 100; i++) {
        map.put(Integer.toString(i), "new value " + i);
    }

    System.out.println("getMap1 == return");
   
    return map;
   
}

Main Runner

In the class which contains the main method, we'll be calling the getMap1() multiple times to determine if the caching is working.  The output should be like below:

Cutting before...
getMap1 == init
getMap1 == return
returned object: {35=new value 35, 36=new value 36, 33=new value 33, 34=new value 34, 39=new value 39, 37=new value 37, 38=new value 38, 43=new value 43, 42=new value 42, 41=new va
lue 41, 40=new value 40, 22=new value 22, 23=new value 23, 24=new value 24, 25=new value 25, 26=new value 26, 27=new value 27, 28=new value 28, 29=new value 29, 3=new value 3, 2=ne
w value 2, 1=new value 1, 0=new value 0, 30=new value 30, 7=new value 7, 6=new value 6, 32=new value 32, 5=new value 5, 31=new value 31, 4=new value 4, 9=new value 9, 8=new value 8
, 19=new value 19, 17=new value 17, 18=new value 18, 15=new value 15, 16=new value 16, 13=new value 13, 14=new value 14, 11=new value 11, 12=new value 12, 21=new value 21, 20=new v
alue 20, 99=new value 99, 98=new value 98, 97=new value 97, 96=new value 96, 95=new value 95, 94=new value 94, 93=new value 93, 92=new value 92, 91=new value 91, 90=new value 90, 1
0=new value 10, 88=new value 88, 89=new value 89, 79=new value 79, 78=new value 78, 77=new value 77, 82=new value 82, 83=new value 83, 80=new value 80, 81=new value 81, 86=new valu
e 86, 87=new value 87, 84=new value 84, 85=new value 85, 67=new value 67, 66=new value 66, 69=new value 69, 68=new value 68, 70=new value 70, 71=new value 71, 72=new value 72, 73=n
ew value 73, 74=new value 74, 75=new value 75, 76=new value 76, 59=new value 59, 58=new value 58, 57=new value 57, 56=new value 56, 55=new value 55, 64=new value 64, 65=new value 6
5, 62=new value 62, 63=new value 63, 60=new value 60, 61=new value 61, 49=new value 49, 48=new value 48, 45=new value 45, 44=new value 44, 47=new value 47, 46=new value 46, 51=new
value 51, 52=new value 52, 53=new value 53, 54=new value 54, 50=new value 50}
Storing into cache...
Cutting after...
Cutting before...
Returning from cache....
Cutting before...
Returning from cache....
Cutting before...
Returning from cache....

Spring AOP with @AspectJ annotations

Here's a quick sample on how to use Spring's AOP feature with @AspectJ annotations.  First off, please read the Spring Reference manual on AOP.  This post is a copy and paste tutorial (hopefully I get it right :P).  We begin by creating a normal Java Project in Eclipse.  I've named it "springtest".  You can name it whatever you like.  Here's a screenshot of the project in Eclipse:

Take note of the JAR files in the project.  Those are the JAR files required by Spring to use AOP features.  They can be found in Spring's lib and dist folders (you'll need the full Spring download with dependencies).

• spring.jar - main Spring JAR file; obtained from the "dist" folder
• aspectjrt.jar, aspjectjweaver.jar - AspectJ JAR file; obtained from "lib/aspectj" folder
• commons-logging.jar - Commons logging JAR file; obtained from "lib/jakarta-commons" folder
• cglib-nodep-2.1_3.jar - Code generation lib JAR file; obtained from "lib/cglib" folder

Spring Application Context File - app-ctx.xml
Create the Spring app context file in the com.aop package and name it app-ctx.xml.  Here's the contents of the file:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xsi:schemaLocation="
            http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
            http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">

    <aop:aspectj-autoproxy></aop:aspectj-autoproxy>
   
    <bean id="myBean" class="com.aop.MyBean"/>
   
    <bean id="myAspect" class="com.aop.MyAspect"/>
</beans>

The highlighted line is to enable @AspectJ autoproxying of beans declared. 


Aspect Class - MyAspect.java
Next, create a Java class file and name it MyAspect.java.  This file will contain the Aspect as well as the PointCut we'll be using for this simple how to.  File content shown below:

package com.aop;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;

@Aspect
public class MyAspect {
   
    @Around("execution(public * *(..))")
    public Object executor(ProceedingJoinPoint point) throws Throwable {

        System.out.println("Cutting before...");
       
        Object retVal = point.proceed();
       
        if (retVal != null) {
            System.out.println("returned object: " + retVal.toString());
        }
       
        System.out.println("Cutting after...");
       
        return retVal;
    }
}

Remember that aspectj-autoproxy declaration in the app-ctx.xml file?  Once we annotate a class file using the @Aspect, Spring AOP will automatically pickup and process the class files.

Notice that we're using the Around advice.  What this does is prints the line "Cutting before..." right before the method is executed and "Cutting after..." right after the method is executed.  If the method returns a value, print it out.  For more information on other types of advices, please refer to the Spring reference doc :)  What we're doing here is to cut around all public methods. 


The Simple Spring Managed Bean - MyBean.java
This is a simple Spring managed bean which contains methods which will be intercepted by the aspect class defined above.  Here's the file contents:

package com.aop;

public class MyBean {
    public void test() {
        System.out.println("MyBean>>>test");
    }
   
    public void test2() {
        System.out.println("MyBean>>>test2");
    }
   
    public String returnTest() {
        System.out.println("MyBean>>>returnTest");
        return "this is a test";
    }
}

The Main class - MainRunner.java
This will be the class which contains the main method.  Here's the contents:

package com.aop;

import org.springframework.context.support.ClassPathXmlApplicationContext;

public class MainRunner {
    public static void main(String[] args) {
        try {
            ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("com/aop/app-ctx.xml");
           
            MyBean bean = (MyBean) ctx.getBean("myBean");
            bean.test();
            bean.test2();
            bean.returnTest();
           
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
   
}

The first step is to obtain the application context via ClassPathXmlApplicationContext.  Then call get an instance of MyBean and subsequently call its methods (test, test2 and returnTest).  If you've done everything right, run the program and the following output should be displayed:

Cutting before...
MyBean>>>test
Cutting after...
Cutting before...
MyBean>>>test2
Cutting after...
Cutting before...
MyBean>>>returnTest
returned object: this is a test
Cutting after...

There you have it, the simplest way of AOP :)

Load balancing OpenVPN connections via IPVS (Linux Virtual Server)

First Steps
 
First off, I've performed the setup and configuration on my own local PC using VirtualBox VMs.  Network for all the VMs are set to bridge mode.  Here's how the servers are configured:

director
--------
Fedora 12
VIP=eth0:0 192.168.1.150
RIP=eth0 192.168.1.200

ovpnserver1
-----------
Ubuntu
RIP=eth0 192.168.1.201
VIP=lo:0 192.168.1.150 (no arp)

ovpnserver2
-----------
Ubuntu
RIP=eth0 192.168.1.202
VIP=lo:0 192.168.1.150 (no arp)

Here's how the servers are laid out:

director ------- ovpnserver1
       |
       |-------- ovpnserver2

IPVS will be configured via the direct routing method; as opposed to NAT or Tunneling.


Setting Up The First OpenVPN Server (ovpnserver1)

Begin by configuring the network interface.  Here's how the /etc/network/interfaces file looks like:

# The loopback network interface
auto lo
iface lo inet loopback

# VIP routing
auto lo:0
iface lo:0 inet static
address 192.168.1.150
netmask 255.255.255.255

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.201
network 192.168.1.0
netmask 255.255.255.0
gateway 192.168.1.1

Restart networking service if necessary:

# service networking restart

Next, install OpenVPN package in the server:

# apt-get install openvpn

Now configure OpenVPN:

# cd /etc/openvpn
# cp -r /usr/share/doc/openvpn/examples/easy-rsa/2.0/ .
# mv easy-rsa rsa
# cd rsa

Edit the vars script and edit the appropriate variables at the end of the file (e.g. KEY_COUNTRY, KEY_CITY).  We then generate the necessary keys for the server:

# . ./vars
# ./clean-all
# ./build-dh
# ./pkitool --initca
# ./pkitool --server server

The CA and server keys/certs are now in /etc/openvpn/rsa/keys.  Once the keys and certs have been generated, we'll need to create a server configuration file.  For the purpose of this test, we'll authenticate the user's credentials using the supplied auth-pam.pl script.  This script authenticates OpenVPN against the system users.  Create the configuration file in /etc/openvpn/server.conf with the following contents:

port 1194
proto udp
dev tun

ca /etc/openvpn/rsa/keys/ca.crt
cert /etc/openvpn/rsa/keys/server.crt
key /etc/openvpn/rsa/keys/server.key
dh /etc/openvpn/rsa/keys/dh1024.pem

server 10.128.127.0 255.255.255.0
ifconfig-pool-persist ipp.txt

push "redirect-gateway def1"
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"

keepalive 10 120

comp-lzo

max-clients 50

persist-key
persist-tun

status openvpn-status.log
log-append /var/log/openvpn.log
verb 3
mute 20

client-cert-not-required
username-as-common-name

auth-user-pass-verify auth-pam.pl via-file

Ensure that you have Perl::PAM module installed for the script to work:

# apt-get install libauthen-pam-perl

Enable IP forwarding:

# vim /etc/sysctl.conf

Ensure that the following line is in the file:

net.ipv4.ip_forward=1

Refresh sysctl params from /etc/sysctl.conf:

# sysctl -p

Enable NAT/MASQ rules in iptables:

# iptables -t nat -A POSTROUTING -s 10.128.127.0/24 -o eth0 -j MASQUERADE
# iptables-save

Now start OpenVPN daemon:

# service openvpn restart

Setting Up The Second OpenVPN Server (ovpnserver2)

Like the first server, we begin by configuring the network interface.  Here's how the /etc/network/interfaces file looks like:

# The loopback network interface
auto lo
iface lo inet loopback

# VIP routing
auto lo:0
iface lo:0 inet static
address 192.168.1.150
netmask 255.255.255.255

# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.202
network 192.168.1.0
netmask 255.255.255.0
gateway 192.168.1.1

Restart networking service if necessary:

# service networking restart

Next, install OpenVPN package in the server:

# apt-get install openvpn

Instead of configuring this instance, we'll copy the entire /etc/openvpn folder from ovpnserver1 to ovpnserver2.  This is required as the CA certs/keys must match.


Setting Up The Load Balancer (director)

Configure the network interfaces like so:

/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
HWADDR=08:00:27:81:7A:C2
IPADDR=192.168.1.200
BOOTPROTO=none
NETMASK=255.255.255.0
DNS2=208.67.222.222
TYPE=Ethernet
GATEWAY=192.168.1.1
DNS1=208.67.220.220
IPV6INIT=no
ONBOOT=yes
USERCTL=no
PREFIX=24
NAME="System eth0"
UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03

/etc/sysconfig/network-scripts/ifcfg-eth0:0

DEVICE=eth0:0
HWADDR=08:00:27:81:7A:C2
IPADDR=192.168.1.150
BOOTPROTO=none
NETMASK=255.255.255.0
DNS2=208.67.222.222
TYPE=Ethernet
GATEWAY=192.168.1.1
DNS1=208.67.220.220
IPV6INIT=no
ONBOOT=yes
USERCTL=no
PREFIX=24
NAME="System eth0:0"
UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03

Turn on ipvsadm daemon:

# chkconfig ipvsadm on
# chkconfig --list ipvsadm
ipvsadm         0:off   1:off   2:on    3:on    4:on    5:on    6:off

The ipvsadm daemon reads its config file from /etc/sysconfig/ipvsadm.  Create the file with the following contents:

-A -u 192.168.1.150:1194 -s rr
-a -u 192.168.1.150:1194 -r 192.168.1.201:1194
-a -u 192.168.1.150:1194 -r 192.168.1.202:1194

ipvsadm uses the direct routing method by default.  The first line in the file instructs ipvsadm to add a virtual service.  In this case, it's a UDP service denoted by the -u parameter.  The -s rr parameter instructs it to use the round-robin scheduling method.  The subsequent two lines adds servers to the virtual service.  Since we have 2 virtual machines, we specify both the IPs.  Restart the ipvsadm service once you're done:

# service ipvsadm restart

Type the following command to verify that you have a working setup:

# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
UDP  192.168.1.150:openvpn rr
  -> 192.168.1.201:openvpn        Route   1      0          0
  -> 192.168.1.202:openvpn        Route   1      0          0

OpenVPN Client Configuration File

The client configuration file is quite straight forward.  Instead of having multiple "remote" options, you'll just have one which will be pointing to the director's IP.  The other thing to note is to add the "float" option.  Below's the configuration file for this purpose:

client
dev tun
float
proto udp
remote 192.168.1.150 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
auth-user-pass
cipher BF-CBC
comp-lzo
verb 4
mute 20

Here's a snippet from openvpn's man page on the "float" option:

--float
    Allow  remote  peer to change its IP address and/or port number, such as due to DHCP (this is the default if --remote is not used).  --float when
    specified with --remote allows an OpenVPN session to initially connect to a peer at a known address, however if packets arrive from a new address
    and  pass  all  authentication  tests,  the new address will take control of the session.  This is useful when you are connecting to a peer which
    holds a dynamic address such as a dial-in user or DHCP client.

    Essentially, --float tells OpenVPN to accept authenticated packets from any address, not only the address which was  specified  in  the  --remote
    option.

Mounting an NTFS partition in a folder

To mount an NTFS partition in a folder similar to what's done in Unix systems, do the following:

• Go to the "Run..." screen and type in compmgmt.msc.
• Click on "Disk Management".
• Right click on the volume you want to use and select "Change Drive Letter and Paths..."
• Click "Add".  Another window will be displayed like below:

• Click on the second radio box "Mount in the following empty NTFS folder:".  This is where you key in the folder name where you want the partition to be mounted on.
• Click "OK" once you're done.

Open up Windows Explorer and navigate to C:\.  You should be able to see that the "second-vol" directory has a different icon like the one shown below:

And that's it! :)