OSX Lion JAVA_HOME

Apple’s OS Lion has a pretty neat trick for handling the JAVA_HOME variable


Chill:bin rus$ /usr/libexec/java_home
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

which is super awesome as you can simply do


Chill:bin rus$ export JAVA_HOME=$(/usr/libexec/java_home)
Chill:bin rus$ echo $JAVA_HOME
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

to make sure you have the correct $JAVA_HOME set up, every time!

I’ve added this commend to my ~/.profile file so that whenever I open a new terminal, the $JAVA_HOME variable gets set correctly!

google-apis-mavenized does not compile

I need to write some backup software for our hosted Google Apps, so as we’re a Java shop I’m going to be using the Java API. Also we’re heavy users of Hudson and Maven so want to get the checked out source building with that.

There is a Google Code project called google-apis-mavenized which looks like it has done most of the work for us, but it does not build when it’s checked out.

 

rus@builder:/proxy/rus$ svn checkout http://google-apis-mavenized.googlecode.com/svn/trunk/gdata-mavenized/ gdata-mavenized

 

rus@builder:/proxy/rus/$ cd gdata-mavenized

rus@builder:/proxy/rus/gdata-mavenized$ mvn install

 

/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com/google/gdata/data/extensions/ExtendedProperty.java:[20,31] cannot find symbol

symbol : class CoreErrorDomain

location: package com.google.gdata.client

 

After some investigation the gdata-mavenized project symlinks in files from the original gData project, but presumably due to the API constantly changing, the maintainers of google-apis-mavenized got bored or fed up so didn’t symlink in newer files.

A really quick way to fix this is simply to delete the main google directory that contains all the symlinks, and just symlink that back to the real directory that contains all the source files!

 

rus@builder:/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com$ rm -rf google/

rus@builder:/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com$ ln -s ../../../../../../gdata-external/java/src/com/google/ .

 

Bamo, problem solved!

 

rus@builder:/proxy/rus/gdata-mavenized$ mvn install

[INFO] Scanning for projects…

[INFO] Reactor build order:

[INFO] google-gdata-mavenized-1-SNAPSHOT

[INFO] google-gdata-apis:1-SNAPSHOT

[INFO] gdata-client:1-SNAPSHOT

[INFO] gdata-appsforyourdomain:1-SNAPSHOT

[INFO] gdata-base:1-SNAPSHOT

[INFO] gdata-calendar:1-SNAPSHOT

[INFO] gdata-codesearch-1-SNAPSHOT

[INFO] gdata-photos:1-SNAPSHOT

[INFO] gdata-spreadsheet:1-SNAPSHOT

[INFO] gdata-docs:1-SNAPSHOT

[INFO] samples-1-SNAPSHOT

[INFO] sample-util-1-SNAPSHOT

[INFO] sample-appsforyourdomain-client-1-SNAPSHOT

[INFO] sample-authsub-webapp-1-SNAPSHOT

[INFO] sample-blogger-client-1-SNAPSHOT

[INFO] sample-calendar-client-1-SNAPSHOT

[INFO] sample-codesearch-client-1-SNAPSHOT

[INFO] sample-photos-client-1-SNAPSHOT

[INFO] sample-tester-client-1-SNAPSHOT

 

[INFO] ————————————————————————

[INFO] Reactor Summary:

[INFO] ————————————————————————

[INFO] google-gdata-mavenized-1-SNAPSHOT ………………… SUCCESS [11.711s]

[INFO] google-gdata-apis:1-SNAPSHOT …………………….. SUCCESS [0.152s]

[INFO] gdata-client:1-SNAPSHOT …………………………. SUCCESS [1:02.902s]

[INFO] gdata-appsforyourdomain:1-SNAPSHOT ……………….. SUCCESS [2.881s]

[INFO] gdata-base:1-SNAPSHOT …………………………… SUCCESS [2.021s]

[INFO] gdata-calendar:1-SNAPSHOT ……………………….. SUCCESS [2.365s]

[INFO] gdata-codesearch-1-SNAPSHOT ……………………… SUCCESS [1.397s]

[INFO] gdata-photos:1-SNAPSHOT …………………………. SUCCESS [3.297s]

[INFO] gdata-spreadsheet:1-SNAPSHOT …………………….. SUCCESS [1.912s]

[INFO] gdata-docs:1-SNAPSHOT …………………………… SUCCESS [1.773s]

[INFO] samples-1-SNAPSHOT ……………………………… SUCCESS [0.173s]

[INFO] sample-util-1-SNAPSHOT ………………………….. SUCCESS [1.304s]

[INFO] sample-appsforyourdomain-client-1-SNAPSHOT ………… SUCCESS [1.651s]

[INFO] sample-authsub-webapp-1-SNAPSHOT …………………. SUCCESS [24.340s]

[INFO] sample-blogger-client-1-SNAPSHOT …………………. SUCCESS [1.466s]

[INFO] sample-calendar-client-1-SNAPSHOT ………………… SUCCESS [0.355s]

[INFO] sample-codesearch-client-1-SNAPSHOT ………………. SUCCESS [1.396s]

[INFO] sample-photos-client-1-SNAPSHOT ………………….. SUCCESS [1.192s]

[INFO] sample-tester-client-1-SNAPSHOT ………………….. SUCCESS [1.656s]

[INFO] ————————————————————————

[INFO] ————————————————————————

[INFO] BUILD SUCCESSFUL

[INFO] ————————————————————————

[INFO] Total time: 2 minutes 6 seconds

[INFO] Finished at: Wed Apr 15 12:40:25 BST 2009

[INFO] Final Memory: 19M/45M

[INFO] ————————————————————————

rus@builder:/proxy/rus/gdata-mavenized$ find . -name *.jar

./gdata-samples/codesearch/target/sample-codesearch-client-1-SNAPSHOT.jar

./gdata-samples/authsub/target/authsub_sample/WEB-INF/lib/gdata-calendar-1-SNAPSHOT.jar

./gdata-samples/authsub/target/authsub_sample/WEB-INF/lib/gdata-client-1-SNAPSHOT.jar

./gdata-samples/appsforyourdomain/target/sample-appsforyourdomain-client-1-SNAPSHOT.jar

./gdata-samples/util/target/sample-util-1-SNAPSHOT.jar

./gdata-samples/tester/target/sample-tester-client-1-SNAPSHOT.jar

./gdata-samples/calendar/target/sample-calendar-client-1-SNAPSHOT.jar

./gdata-samples/photos/target/sample-photos-client-1-SNAPSHOT.jar

./gdata-samples/blogger/target/sample-blogger-client-1-SNAPSHOT.jar

./gdata-apis/codesearch/target/gdata-codesearch-1.0.jar

./gdata-apis/appsforyourdomain/target/gdata-appsforyourdomain-1.0.jar

./gdata-apis/base/target/gdata-base-1.0.jar

./gdata-apis/spreadsheet/target/gdata-spreadsheet-1.0.jar

./gdata-apis/docs/target/gdata-docs-1.0.jar

./gdata-apis/calendar/target/gdata-calendar-1.0.jar

./gdata-apis/client/target/gdata-client-1.0.jar

./gdata-apis/photos/target/gdata-photos-1.0.jar

./gdata-external/java/sample/codesearch/lib/CodeSearchClient.jar

./gdata-external/java/sample/appsforyourdomain/lib/AppsForYourDomainClient.jar

./gdata-external/java/sample/util/lib/sample-util.jar

./gdata-external/java/sample/spreadsheet/lib/GUIDemo.jar

./gdata-external/java/sample/spreadsheet/lib/IndexClient.jar

./gdata-external/java/sample/spreadsheet/lib/ImportClient.jar

./gdata-external/java/sample/spreadsheet/lib/ListDemo.jar

./gdata-external/java/sample/spreadsheet/lib/CellDemo.jar

./gdata-external/java/sample/tester/lib/TesterClient.jar

./gdata-external/java/sample/gbase/lib/gdata-base-cmdline.jar

./gdata-external/java/sample/calendar/lib/CalendarClient.jar

./gdata-external/java/sample/photos/lib/PhotosClient.jar

./gdata-external/java/lib/gdata-client-1.0.jar

./gdata-external/java/lib/gdata-appsforyourdomain-1.0.jar

./gdata-external/java/lib/gdata-spreadsheet-1.0.jar

./gdata-external/java/lib/gdata-base-1.0.jar

./gdata-external/java/lib/gdata-calendar-1.0.jar

./gdata-external/java/lib/gdata-codesearch-1.0.jar

./gdata-external/java/lib/gdata-photos-1.0.jar

Woohoo!

Tomcat HelloWorld Servlet with Eclipse

I’m really trying to get in to this whole Java web development frame of mind, as it’s a bit of fun, a bit of a giggle, and it’s massive in this area of the world! So obviously my first port of call was dusting off Eclipse and kicking out a HelloWorld style Java servlet!

I grabbed a copy of O’Rielly’s Java Servlet Programming to get started and found it really invaluable, and definitely recommend this book to anyone starting out in Java servlet programming.

Eclipse on Ubuntu, even Intrepid is well old, so rather than work with the out of date supplied package, I found it best to download the latest version direct from the site. Also the easiest way to hook Tomcat in to Eclipse is also to download that from the site.

Get the latest Eclipse IDE for Java EE Developers and Tomcat 6

Extract them both to somewhere reasonable, I like ~/local :


idimmu@boosh:~/local$ ls -al
total 173212
drwxr-xr-x 4 idimmu idimmu 4096 2009-03-11 15:01 .
drwxr-xr-x 63 idimmu idimmu 4096 2009-03-11 15:01 ..
drwxr-xr-x 9 idimmu idimmu 4096 2009-03-11 15:01 apache-tomcat-6.0.18
-rw-r--r-- 1 idimmu idimmu 6142197 2009-03-11 11:27 apache-tomcat-6.0.18.tar.gz
drwxr-sr-x 9 idimmu idimmu 4096 2009-02-23 19:36 eclipse
-rw-r--r-- 1 idimmu idimmu 171022452 2009-03-11 10:57 eclipse-jee-ganymede-SR2-linux-gtk.tar.gz

Start Eclipse! The first thing that Eclipse will do is ask you to create a new Workspace. Your home directory isn’t a bad choice to put this!

Tomcat HelloWorld Servlet with Eclipse

Go to New->Project

Tomcat HelloWorld Servlet with Eclipse

Select ‘Dynamic Web Project’

Set ‘Project Name’ to ‘helloworld’

Create a ‘New’ ‘Target Runtime’

Select ‘Apache Tomcat v6.0’

Select the Tomcat Installation Directory you extracted Tomcat to earlier.

Then click ‘Finish’ to create the project.
Go to File->New->Servlet

Enter ‘HelloWorld’ as the ‘Class name’
Click ‘Finish’

A new ‘HelloWorld.java’ file will be created with most of the work done for you!

Tomcat HelloWorld Servlet with Eclipse

Look at all the shiny code the IDE has already written for you!

import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
* Servlet implementation class HelloWorld
*/
public class HelloWorld extends HttpServlet {
private static final long serialVersionUID = 1L;

/**
* Default constructor.
*/
public HelloWorld() {
// TODO Auto-generated constructor stub
}

/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
}

/**
* @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
}

}

This code does nothing on it’s own, well, it will generate a completely empty web page if you build it and deploy it to Tomcat, not very exciting .. so ..

Import the following new class:


import java.io.PrintWriter;

Insert the following code in to the doGet function stub:


response.setContentType("text/html");
PrintWriter pw = response.getWriter();
pw.println("");
pw.println("");
pw.println("");
pw.println("

Hello World

");
pw.println("");

The complete source code for the class will now look like this:

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
* Servlet implementation class HelloWorld
*/
public class HelloWorld extends HttpServlet {
private static final long serialVersionUID = 1L;

/**
* @see HttpServlet#HttpServlet()
*/
public HelloWorld() {
super();
// TODO Auto-generated constructor stub
}

/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/html");
PrintWriter pw = response.getWriter();
pw.println("");
pw.println("");
pw.println("");
pw.println("

Hello World

");
pw.println("");
}

/**
* @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
}

}

‘Save All’ using Shift+Ctrl+S or the File menu.
Go to the ‘Run’ menu and select ‘Run’ or press Ctrl+F11 to build the servlet, deploy it to Tomcat and run it!

Tomcat HelloWorld Servlet with Eclipse

Make sure ‘Tomcat v6.0 Server’ is selected and click ‘Always use this server when running this project’ then click ‘Finish’

Tada, Hello World! You might have to run it a few times to get Tomcat to sort itself out, as it’s a bit wonky, but the very mundane site should now be available on http://localhost:8080/helloworld/HelloWorld!!

Eclipse

If you really want to be a rock and roll Java super star and learn how to use Eclipse and Tomcat properly, I thoroughly suggest you get a copy of O’Reilly’s Eclipse book and learn how to use it properly.

Nexus on Tomcat 5.5 on Ubuntu Hardy

I’m trying out this Continuous Integration fun at the moment.

My end game is to get Hudson, Maven and Nexus working together to continuously build and run unit tests against code, which then gets turned in to Deb packages. A new Xen VM will then be created and configured using Puppet which the new Deb package is then deployed to. Finally Selenium will then be run to automate testing of the deployment.

Thats the plan anyway ..

I’ve been deploying everything on Ubuntu Hardy for the time being, and the latest app I am working on is Nexus. I’m deploying it as a War under Tomcat 5.5 and for a while was just getting the following error:


06-Mar-2009 20:29:08 org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive nexus.war
06-Mar-2009 20:29:12 org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
06-Mar-2009 20:29:12 org.apache.catalina.core.StandardContext start
SEVERE: Context [/nexus] startup failed due to previous errors

I hate that error. It’s rubbish. To make it a little more verbose I edited /var/lib/tomcat5.5/webapps/nexus/WEB-INF/log4j.properties and changed the rootLogger to DEBUG and fixed the appender path to go somewhere sensible (/var/log/tomcat55/nexus.log) and restarted Tomcat! This logged the following interesting error:


2009-03-06 20:42:01.066 ERROR [main:] - org.sonatype.nexus.configuration.application.source.ApplicationConfigurationSource:file:
******************************************************************************
* Could not create configuration file [ /usr/share/tomcat5.5/sonatype-work/nexus/conf/nexus.xml]!!!! *
* Nexus cannot start properly until the process has read+write permissions to this folder *
******************************************************************************
2009-03-06 20:42:01.091 ERROR [main:] - org.sonatype.nexus.Nexus:default: Could not start Nexus, bad IO exception!
java.io.FileNotFoundException: /usr/share/tomcat5.5/sonatype-work/nexus/conf/nexus.xml (No such file or directory)

Not exactly rocket science to see what is going on here!! And really easy to fix!


mkdir /usr/share/tomcat5.5/sonatype-work
chown tomcat55: /usr/share/tomcat5.5/sonatype-work

A quick restart of tomcat, after turning the logging back down to INFO and ta da! A working Nexus repo!

Apache2 with SSL and Tomcat5.5 on Ubuntu

One of the newer features to our site is an access control mechanism to force specific paths to only be delivered over SSL when our customers have particularly sensitive data. We already use Apache2 with mod_jk to talk to the Tomcat5.5 instance running our app so the only part left is to enable SSL!

First make sure mod_ssl is enabled:


root@reltest-tcj0:/var/log/apache2# a2enmod
Which module would you like to enable?
Your choices are: actions asis auth_anon auth_dbm auth_digest auth_ldap cache cern_meta cgid cgi dav_fs dav deflate disk_cache expires ext_filter file_cache headers imap include info jk ldap mem_cache mime_magic proxy_connect proxy_ftp proxy_http proxy rewrite speling ssl suexec unique_id userdir usertrack vhost_alias
Module name? ssl
This module is already enabled!

Then we configure mod_jk to pass it’s SSL environment variables to Tomcat by adding the following to apache2.conf


JkExtractSSL On
JkHTTPSIndicator HTTPS
JkSESSIONIndicator SSL_SESSION_ID
JkCIPHERIndicator SSL_CIPHER
JkCERTSIndicator SSL_CLIENT_CERT

Tell Apache2 to listen on the SSL port by editing ports.conf


Listen 443

We want to make sure we have the latest common CA certificates in order to establish a trusted root for our new shiny signed certificate!


apt-get install ca-certificates

If you have a lovely genuinely signed certificate like we do you might need to then add it’s intermediate certificate to the ca-certificates system. Move the certificate to /usr/share/ca-certificates then add it’s location to /etc/ca-certificates.conf

Now run update-ca-certificates to update the system’s certificate store (located in /etc/ssl/certs/ca-certificates.crt).


root@reltest-tcj0:/etc/apache2/sites-enabled# update-ca-certificates
Updating certificates in /etc/ssl/certs....done.

We want the same site to simply be available over SSL I’m going to duplicate the existing VirtualHost for that site specifying the use of port 80 for the original vhost and port 443 for the new one that uses SSL. The only change that needs to be made to the new vhost are the following SSL directives:


SSLEngine On
SSLCertificateFile /etc/apache2/ssl/domain.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/domain.com.key
SSLCACertificateFile /etc/ssl/certs/ca-certificates.crt

Obviously making sure the keys are in the right place!

And lastly make sure that NameVirtualHost settings exist for both port 80 and port 443!


NameVirtualHost *:80
NameVirtualHost *:443

et voila.

PHP Java Bridge in Ubuntu Gutsy with Lucene

The php/java bridge it a pretty awesome little protocol that basically lets us use java classes inside our own PHP applications! This lets you harness the awesome power of all the Java libraries that exist, including the popular Lucene search engine library.

I referenced two excellent blog entries here and here whilst implementing Lucene search for this blog, but I am writing up the experience anyway to compare issues and difficulties and enhance my understanding of the process.

To start with Java, Lucene and the bridge dependancies must be installed (remember to enable multiverse in your apt sources)


apt-get install sun-java6-jre sun-java6-jdk liblucene-java libitext-java
update-java-alternatives -s java-6-sun

Grab the php-java-bridge deb package from sourceforge and install it. The fact it is v4 does not reflect that it is only for PHP version 4! There are RPMs for version 5 which you could turn in to a deb package using alien but at the moment I am feeling lazy so I will see how version 4 works out first.


wget http://downloads.sourceforge.net/php-java-bridge/php-java-bridge_4.3.0-1_i386.deb
dpkg -i php-java-bridge_4.3.0-1_i386.deb

Apache should restart now, if not restart it yourself.

To check that it is working look at the output of phpinfo(), there should be a new shiny java section! Listing the running processes also is interesting!


root 20205 0.0 0.7 664520 15520 ? Sl 17:18 0:00 java -Djava.library.path=/usr/lib/php5/20060613+lfs
-Djava.class.path=/usr/lib/php5/20060613+lfs/JavaBridge.jar -Djava.awt.headless=true
-Dphp.java.bridge.base=/usr/lib/php5/20060613+lfs php.java.bridge.Standalone LOCAL:@java-bridge-4ee9 1

as does netstat


unix 2 [ ACC ] STREAM LISTENING 1913999 @java-bridge-4ee9

I think it gets started when apache starts, as java.so is loaded in to the PHP, I’m still investigating that.

As far as starting the Lucene development goes, this was a pretty good tutorial on how it all works and this site has some good Java example code that I used to work out how the PHP should work.

Below is my PHP Lucene test code, it just creates one document with a description then searches the index description for ‘idi test’ and outputs the match. It’s pretty rad!


java_require('/usr/share/java/lucene.jar');

$analyzer = new Java('org.apache.lucene.analysis.StopAnalyzer');
$writer = new Java('org.apache.lucene.index.IndexWriter', '/path/to/store/lucene/data/in', $analyzer, true);

$doc = new Java('org.apache.lucene.document.Document');
$field = new Java('org.apache.lucene.document.Field','description','idi data test',true, true, true);
$doc->add($field);

$writer->addDocument($doc);

$writer->close();

$indexer = new Java('org.apache.lucene.search.IndexSearcher','/path/to/store/lucene/data/in');
$parser = new Java('org.apache.lucene.queryParser.QueryParser','description',$analyzer);
$query = $parser->parse('rus test');

$hits = $indexer->search($query);

for ($i = 0; $i < $hits->length(); $i++) {
$found = $hits->doc($i);
print $i.".".$found->get('description');
}
?>

Now that it’s working I just have to incorperate it in to the site 🙂

Configuring Tomcat 5.5 and Apache 2 with mod_jk

mod_jk is a conduit between a web server and Tomcat, it supports a variety of web servers including IIS. Using mod_jk to put Apache in front of Tomcat lets you use all the power of Apache (caching, gzip, mod_rewrite, etc) whilst at the same time serving content from Tomcat, also with Ubuntu it’s really easy to set up!

First of all install the software, you will need to enable the backports repository on Dapper for this.


apt-get install sun-java6-bin sun-java6-jdk tomcat5.5 libapache2-mod-jk

The Tomcat 5.5 that comes with Ubuntu already has an AJP connector configured on port 8009 so there is no additional configuration to do to it’s server.xml file.

We then need to configure a worker.properties file for Apache2 which tells it about the Tomcat instance, I make mine in /etc/apache2/worker.properties


worker.list=idimmu
worker.idimmu.type=ajp13
worker.idimmu.host=localhost
worker.idimmu.port=8009

Make sure mod_jk is then enabled with a2enmod jk (it probably already is).

And finally we tell Apache2 about the worker instance in /etc/apache2/apache2.conf


JkWorkersFile /etc/apache2/workers.properties
JkLogFile /var/log/apache2/mod_jk.log
JkLogLevel info
JkMount /* idimmu

This will direct any requests to the Apache2 server to the Tomcat server