idimmu . net

A web based memory replacement service.

google-apis-mavenized does not compile April 15, 2009

Posted by idimmu in java.
I need to write some backup software for our hosted Google Apps, so as we're a Java shop I'm going to be using the Java API. Also we're heavy users of Hudson and Maven so want to get the checked out source building with that.

There is a Google Code project called google-apis-mavenized which looks like it has done most of the work for us, but it does not build when it's checked out.



rus@builder:/proxy/rus$ svn checkout http://google-apis-mavenized.googlecode.com/svn/trunk/gdata-mavenized/ gdata-mavenized

<snip>

rus@builder:/proxy/rus/$ cd gdata-mavenized
rus@builder:/proxy/rus/gdata-mavenized$ mvn install

<snip>

/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com/google/gdata/data/extensions/ExtendedProperty.java:[20,31] cannot find symbol
symbol : class CoreErrorDomain
location: package com.google.gdata.client



After some investigation the gdata-mavenized project symlinks in files from the original gData project, but presumably due to the API constantly changing, the maintainers of google-apis-mavenized got bored or fed up so didn't symlink in newer files.

A really quick way to fix this is simply to delete the main google directory that contains all the symlinks, and just symlink that back to the real directory that contains all the source files!


rus@builder:/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com$ rm -rf google/
rus@builder:/proxy/rus/gdata-mavenized/gdata-apis/client/src/main/java/com$ ln -s ../../../../../../gdata-external/java/src/com/google/ .


Bamo, problem solved!


rus@builder:/proxy/rus/gdata-mavenized$ mvn install
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO] google-gdata-mavenized-1-SNAPSHOT
[INFO] google-gdata-apis:1-SNAPSHOT
[INFO] gdata-client:1-SNAPSHOT
[INFO] gdata-appsforyourdomain:1-SNAPSHOT
[INFO] gdata-base:1-SNAPSHOT
[INFO] gdata-calendar:1-SNAPSHOT
[INFO] gdata-codesearch-1-SNAPSHOT
[INFO] gdata-photos:1-SNAPSHOT
[INFO] gdata-spreadsheet:1-SNAPSHOT
[INFO] gdata-docs:1-SNAPSHOT
[INFO] samples-1-SNAPSHOT
[INFO] sample-util-1-SNAPSHOT
[INFO] sample-appsforyourdomain-client-1-SNAPSHOT
[INFO] sample-authsub-webapp-1-SNAPSHOT
[INFO] sample-blogger-client-1-SNAPSHOT
[INFO] sample-calendar-client-1-SNAPSHOT
[INFO] sample-codesearch-client-1-SNAPSHOT
[INFO] sample-photos-client-1-SNAPSHOT
[INFO] sample-tester-client-1-SNAPSHOT

<snip>

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] google-gdata-mavenized-1-SNAPSHOT ..................... SUCCESS [11.711s]
[INFO] google-gdata-apis:1-SNAPSHOT .......................... SUCCESS [0.152s]
[INFO] gdata-client:1-SNAPSHOT ............................... SUCCESS [1:02.902s]
[INFO] gdata-appsforyourdomain:1-SNAPSHOT .................... SUCCESS [2.881s]
[INFO] gdata-base:1-SNAPSHOT ................................. SUCCESS [2.021s]
[INFO] gdata-calendar:1-SNAPSHOT ............................. SUCCESS [2.365s]
[INFO] gdata-codesearch-1-SNAPSHOT ........................... SUCCESS [1.397s]
[INFO] gdata-photos:1-SNAPSHOT ............................... SUCCESS [3.297s]
[INFO] gdata-spreadsheet:1-SNAPSHOT .......................... SUCCESS [1.912s]
[INFO] gdata-docs:1-SNAPSHOT ................................. SUCCESS [1.773s]
[INFO] samples-1-SNAPSHOT .................................... SUCCESS [0.173s]
[INFO] sample-util-1-SNAPSHOT ................................ SUCCESS [1.304s]
[INFO] sample-appsforyourdomain-client-1-SNAPSHOT ............ SUCCESS [1.651s]
[INFO] sample-authsub-webapp-1-SNAPSHOT ...................... SUCCESS [24.340s]
[INFO] sample-blogger-client-1-SNAPSHOT ...................... SUCCESS [1.466s]
[INFO] sample-calendar-client-1-SNAPSHOT ..................... SUCCESS [0.355s]
[INFO] sample-codesearch-client-1-SNAPSHOT ................... SUCCESS [1.396s]
[INFO] sample-photos-client-1-SNAPSHOT ....................... SUCCESS [1.192s]
[INFO] sample-tester-client-1-SNAPSHOT ....................... SUCCESS [1.656s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 minutes 6 seconds
[INFO] Finished at: Wed Apr 15 12:40:25 BST 2009
[INFO] Final Memory: 19M/45M
[INFO] ------------------------------------------------------------------------

rus@builder:/proxy/rus/gdata-mavenized$ find . -name *.jar
./gdata-samples/codesearch/target/sample-codesearch-client-1-SNAPSHOT.jar
./gdata-samples/authsub/target/authsub_sample/WEB-INF/lib/gdata-calendar-1-SNAPSHOT.jar
./gdata-samples/authsub/target/authsub_sample/WEB-INF/lib/gdata-client-1-SNAPSHOT.jar
./gdata-samples/appsforyourdomain/target/sample-appsforyourdomain-client-1-SNAPSHOT.jar
./gdata-samples/util/target/sample-util-1-SNAPSHOT.jar
./gdata-samples/tester/target/sample-tester-client-1-SNAPSHOT.jar
./gdata-samples/calendar/target/sample-calendar-client-1-SNAPSHOT.jar
./gdata-samples/photos/target/sample-photos-client-1-SNAPSHOT.jar
./gdata-samples/blogger/target/sample-blogger-client-1-SNAPSHOT.jar
./gdata-apis/codesearch/target/gdata-codesearch-1.0.jar
./gdata-apis/appsforyourdomain/target/gdata-appsforyourdomain-1.0.jar
./gdata-apis/base/target/gdata-base-1.0.jar
./gdata-apis/spreadsheet/target/gdata-spreadsheet-1.0.jar
./gdata-apis/docs/target/gdata-docs-1.0.jar
./gdata-apis/calendar/target/gdata-calendar-1.0.jar
./gdata-apis/client/target/gdata-client-1.0.jar
./gdata-apis/photos/target/gdata-photos-1.0.jar
./gdata-external/java/sample/codesearch/lib/CodeSearchClient.jar
./gdata-external/java/sample/appsforyourdomain/lib/AppsForYourDomainClient.jar
./gdata-external/java/sample/util/lib/sample-util.jar
./gdata-external/java/sample/spreadsheet/lib/GUIDemo.jar
./gdata-external/java/sample/spreadsheet/lib/IndexClient.jar
./gdata-external/java/sample/spreadsheet/lib/ImportClient.jar
./gdata-external/java/sample/spreadsheet/lib/ListDemo.jar
./gdata-external/java/sample/spreadsheet/lib/CellDemo.jar
./gdata-external/java/sample/tester/lib/TesterClient.jar
./gdata-external/java/sample/gbase/lib/gdata-base-cmdline.jar
./gdata-external/java/sample/calendar/lib/CalendarClient.jar
./gdata-external/java/sample/photos/lib/PhotosClient.jar
./gdata-external/java/lib/gdata-client-1.0.jar
./gdata-external/java/lib/gdata-appsforyourdomain-1.0.jar
./gdata-external/java/lib/gdata-spreadsheet-1.0.jar
./gdata-external/java/lib/gdata-base-1.0.jar
./gdata-external/java/lib/gdata-calendar-1.0.jar
./gdata-external/java/lib/gdata-codesearch-1.0.jar
./gdata-external/java/lib/gdata-photos-1.0.jar


Woohoo!

New Ubuntu Jaunty Screen April 9, 2009

Posted by idimmu in linux.
At the moment I'm trying out the beta for Ubuntu Jaunty and one the first thing I noticed was the new version of screen available!

Initially you are provided with a menu to choose a theme!



I went with option 3, Ubuntu Dark! You're then presented with an interesting new screen display, with 2 rows of status at the bottom, in lots of colours, displaying information about the CPU and RAM etc!



There's a prompt to press F9 for options which presents a curses based interface to change settings and the information on the bottom status bar.



After smiling a little, I decided my best course of action was to delete the screen settings


rus@boosh:~$ rm -rf .screen*


and restart screen using option 1, plain! It's not that I fear change, but I don't need to know any of that information! It doesn't add value to my terminal experience, where as the extra 2 lines of space will!

Generating Secure Passwords With pwgen April 6, 2009

Posted by idimmu in linux.
At my current place of employment we have a fairly strict password policy for everything, all passwords must be unique (so no single password used everywhere) and we use KeePass and KeePassX to manage them, so we can have fairly long passwords that aren't immediately rememberable full of fun characters.

Generating the passwords is a dull process so we take the sting out of it using the pwgen utility!


rus@boosh:~$ pwgen -y 12
foh8Yieghu-L aeChier/o0Si aiKaRa!ngie9 su$o3ooHe7ub Dov3oong$eis jia2ChieG(ee
ahbee`R9pei^ Di9Ep@o2Iuf: oFae3ahyi!th queix$ai6Sha ha6bem!oh4iG eiphoiCh8vu`
Dee0rohcae}j eyaeH2hah&hi kozo3aeP&ait xee4ee>Ghi9i ahR8iKe2Am`a roh.fei9deiN
Zu"u0xie2ahw ees4Gei<Lohv Ooh4holie'D poox1ooGhae- kai;gar2ieX1 Pe'oj1gae3im
Leip3pahGh{i ak0qua]u"Ghi Qui)wauDah8i Aiw~aish9oL7 sho]sh8Kae8i geipoo1Fei+n
Eengoh<nae1a rie_y#aiGh6L uX0sooch<aex Ahphai+y6aih el;ing3Ook3a Tuzai8voo#gh
kee<seeX8Dij quoL9Iu`p0ee eife7eeQuai^ wi5Ro9jei&S6 AishaiS0noh. roo*Zaic0ohc
AJ3aina`ax5j eegh2xa+a%Vo shia=qu3Thai xo{um9Wo1cha Eshei9WaeGh, eira1eiCh?oo
izie4ohT'oi4 jo.y0UYaeHeo Aich3Teiwe-u iR}e9eeDosoh ohree6iV-uom Eed.eC4Sho%f
choo2Phooyi; lu5ooChoh=Pi eengi1aVei]f ul5Reephae@s aiWa`naa2yai shei'ke1uNoo
gi3oph5Ea;b2 IeQu/oo]Ha6O Ohghoph4ieN} eeF@ee-n7ien Ohth;ei2ahhe otu@Th6phut1
vi"hah1leiSe aeGh<ahpe1ee aeh*ezeZaep6 ohL=eiquei8e AiSee)Fohh0h mohgh1Xa2ee:
cahLe>Ph1gox Dei`jaiGh1wo rei>pie'w4Ai iePhoom5ohr@ Ool1aingie?s aefii/K4tahl
tae5eeH^ei*w pheegha5Joh] Ku4va~Chaeg9 dei0she{i0Ae osha5ohF>aif ou;S3ucoW1Ba
AiK.aise5xub AeJ^ee4leebe rau(L8Bu2ail ir8keB/eik1e Soox*ai9aing Tho8heu/Ye4A
Oc(i3aeth~ae Hais!iey8phi oe4oothu5Av> ahv7ti-k0Cox sei(ch>ieg3B uoR3ieS8ooz@
eeBi@vei4igh Yi=ughi6Ur5y yag8ooYo|ezu wae1roo@Foh1 Aedah4AhG?ae we8aiYahj'ie
oogee3ui~Ng= jaefaD,oh5Ie Oopap9ariT;o ChahDeid=i3T aech)eeGee1a Eegh+oo+G0He
Bo4ahT*iejah Akou#fie7oi5 Giki9voom>og pai0boo"va6A woo0mah2AeX[ eiGh3pu&o7ee
kiequie1miL] oohei'soe2Ee The3yo_Mah2m ahzei:d1eTae Phay9Xoh;zai iu}qu6quiMie


Changing variables in a BASH while loop April 2, 2009

Posted by idimmu in linux.
There is an error that I often run in to when working with files and while loops in BASH. Often I have scripts similar to the following, where I cat a file and read it in using a while loop to process variables in the file:


#!/bin/bash

count=0;

cat testfile | while read line
do
count=$(($count+1));
echo $count;
done

echo "Total $count";


and using the following test data in testfile as:


cake
pie
thongs


I would expect an output like:


1
2
3
Total 3


But instead I get:


idimmu@boosh:~$ ./t.sh
1
2
3
Total 0


What is happening is due to the | (pipe) bash is forking and creating a new process so any variables we are altering and changing ($count) are being manipulated in the child process, and then lost when the subprocess finishes!

A lot of people I've spoken to who have seen this either completely change their code structure to accommodate, or worse, change shell completely to something like KSH!

If we tweak our script ever so slightly, and read the file in at the end of the BASH while loop...


#!/bin/bash

count=0;

while read line
do
count=$(($count+1));
echo $count;
done < testfile

echo "Total $count";


everything changes :)


idimmu@boosh:~$ ./t.sh
1
2
3
Total 3


and we get the output we expect! An excellent book on how to actually use BASH can be found here: Learning the bash Shell: Unix Shell Programming (In a Nutshell (O'Reilly))!

Error: Could not stat() command file '/var/lib/nagios3/rw/nagios.cmd'! March 31, 2009

Posted by idimmu in linux.
I've been doing a lot of Nagios deployments recently, and this error always bites me, on all Ubuntu versions, including Hardy and Intrepid (haven't quite bit the bullet to try the Jaunty beta yet :) )


Error: Could not stat() command file '/var/lib/nagios3/rw/nagios.cmd'!

The external command file may be missing, Nagios may not be running, and/or Nagios may not be checking external commands.

An error occurred while attempting to commit your command for processing.


This can be quite easily fixed with the following command line fu:


sudo /etc/init.d/nagios3 stop
sudo dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios3/rw
sudo dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios3
sudo /etc/init.d/nagios3 start


Now you should be able to send Nagios remote commands and commands via the web interface to your heart's content!

For other tips like this, and useful techniques such as implementing Nagios redundancy and distributed monitoring, I thoroughly recommend this book:


Nagios: System and Network Monitoring