idimmu . net

Bing The Most Altruistic Search Engine

rus — Mon, 15 Apr 2013 19:21:01 +0000

I needed to access Bing’s Webmaster Tools to have a look at the configuration for this site. I couldn’t remember the URL off hand so I decided to ‘Google’ the search term using Bing!

That’s very nice of them … I guess part of Bing’s preference engine might rank pages higher depending on where the search words are in the title of your page or something!

Think Carefully About Your Clever Project Names

rus — Thu, 04 Apr 2013 13:30:09 +0000

We’re building a new exciting cluster at work using Linux HA and stuff to make it work magically.

In the olden days of yore RedHat and co were using Pacemaker with the old crm (Cluster Resource Manager / Cluster Relationship Manager, pick one..) tool for cluster management, which was nice. Now it looks like RedHat have removed the crm command from their repositories and have switched to PCS which stands for either Pacemaker/Corosync Configuration System or is the plural of PC (Personal Computers).

This makes searching for anything useful on the Internet quite difficult.

It would have been better to go with a random pronouncable alpha-character string non word, than the stupid PCS acronym, and probably get you some delicious Web 2.0 VC!

Income Report March 2013

rus — Tue, 02 Apr 2013 13:54:10 +0000

Following on from last month’s report March was quite an interesting month. Changing perspective from using this site as a simple memory dump to one that people, other than myself, would find useful lead to some interesting usage patterns, from both myself and my visitors as well as some changes to the blog!

Due to my change of attitude I actually watched my traffic stats, where I noticed some odd things in my logs, I thought I fixed the problem… It turns out having an URL or page with the phrase ‘apache access logs’ in it is a really bad thing as now that page is plagued with bot hits using Russian domains as a referrer, trying to get a back link for their domain in my web stats. It’s receiving about 1500 unique hits a day at the moment which is skewing my stats quite heavily.

As an experiment I increased the size of the Adsense banners to their larger size, which fit in my new theme. I also attempted to monetize some of the more applicable pages with affiliate links back to O’Reilly products. I think I’m going to have to start to use the number of Adsense views as an indicator of traffic, rather than awstats from now on, due to these pesky Russians. I hope they aren’t mafia!

March 2013 Income Report
Adsense Page Views: 5227
Adsense Clicks: 19
Adsense: £7.50
O’Reilly Page Views: 7022
O’Reilly Clicks: 85
O’Reilly: £0.00
So that’s a really interesting increase, up from £0.33 last month, purely by changing the size of the Adsense banner. Awstats is reporting 8361 unique visitors, up from 6332 in February, but unfortunately I can’t trust that metric any more due to the heavy bot activity skewing the stats. Adsense is reporting 3,889 page views for February so it does look like I am actually getting more traffic.
After speaking to the owner of Lampdocs after a recent post they made about trying to monetize a Linux blog, the main conclusion we came to was more traffic and Adsense. Unfortunately my strength comes from my personal systems administration and development skills, I wonder if I can come up with a killer product worth selling ..

Installing Magento

rus — Tue, 02 Apr 2013 12:37:10 +0000

One of my clients wanted an E-Commerce solution for his website and after a little bit of analysis we opted for the community edition of Magento.

We wanted something based on the usual LAMP stack, that was OpenSource so it could be extended, was free as in beer, had great international support, was fully featured and enterprise ready, could pass PCI DSS compliance, allowed reasonable payment gateway options, could scale and was easy to extend as well as backup. Also we wanted complete control of the deployment, rather than integrating with a 3rd party cloud service provider such as Shopify to keep costs down and retain flexibility.

Installing Magento

With out going in to the specifics of configuring an Apache VirtualHost or installing MySQL here’s a rough guide on how to install Magento.

Download Magento from their download page. Always opt for the latest version as it includes important security fixes. You will need to create an account on their site for this.

Create a new MySQL database for the installation and note it’s credentials for later use.

Extract the Magento archive to your document root.

You will need to set write permissions for the web server to write to the following files and directories.

var
var/.htaccess
app/etc

You can either chmod 777 or get a little cleverer about ownership. e.g.

sudo chmod -R 777 var var/.htaccess app/etc

Assuming Apache is correctly configured you can now use your web browser to visit the install directory at the URL you’ve installed Magento at and follow the online install guide. e.g. http://www.shopidimmu.net/, then the wizard will be located here: http://www.shopidimmu.net/install/.

Choosing A Credit Card Processor

After we’d correctly installed and configured Magento, I left it to the web team to get on with populating it with products and get it to look pretty but we still had to find the best credit card processor to accept payments as we didn’t want to use Paypal. We also wanted someone who offered physical solutions to accept card payments with chip and pin.

After checking out some reviews for Merchant Warehouse and Charge.com we decided to opt with Merchant Warehouse as they offer really easy integration with Magento with their MerchantWare plugin.

Installing the Magento Connect MerchantWare plugin was trivial and just required copying and pasting our extension key in to the Magento Connect Manager and clicking install.

monit: error connecting to the monit daemon

rus — Thu, 28 Mar 2013 11:48:53 +0000

We’re rolling out monit on our new platform at the request of a vendor to manage their new service. I’ve always been dead against these kinds of automated failure recovery tools as they often require human intervention after the fact anyway and all the platforms I’ve managed will have failed the server anyway so why not restart the services after the root cause analysis is done? My tune is slowly changing though and I’m coming to appreciate this method of systems recovery a lot more.

Whilst playing with it though I got the following error

root@newshiny:~# monit summary monit: error connecting to the monit daemon

What what? The daemon’s definitely running, why can’t I pole it’s status?

root@newshiny:~# ps aux | grep monit root 325293 0.0 0.0 16440 1276 ? Sl Mar27 0:10 /usr/bin/monit root 496627 0.0 0.0 105348 832 pts/0 S+ 11:43 0:00 grep monit root@newshiny:~# service monit status monit (pid 325293) is running...

After reading the documentation this monit: error connecting to the monit daemon seemed to be an epic case of rushing in to things, skimming the documentation and PEBCAK!

Solving monit: error connecting to the monit daemon

Monit can present an HTTP interface which I didn’t enable as I thought it was just for me, it turns out it’s also for the command line tools!

It’s really easy to enable, in /etc/monit.conf or wherever your conf file is located just add

set httpd port 2812 and use address localhost allow localhost

and restart monit with

service monit restart

and Bob’s your mother’s brother.

root@newshiny:~# netstat -lpn | grep 2812 tcp 0 0 127.0.0.1:2812 0.0.0.0:* LISTEN 325293/monit

root@newshiny:~# monit summary The Monit daemon 5.2.5 uptime: 19h 18m

Process 'shiny_manager' running Process 'shiny_proxy' running Process 'shiny_server' running System 'system_newshiny' running

Problem solved!

Odd Occurrences In Apache Access Logs

rus — Wed, 27 Mar 2013 23:33:21 +0000

I’ve been watching my Awstats installation this month as idimmu.net is about to peak with over 7000 unique visitors in a month for the first time in it’s history, which is pretty awesome. But there’s been something really weird going on in the results ..

This is kind of ironic as in a recent job interview I was asked

What would you look for to ascertain suspicious activity on an instance of Apache serving static image assets?

Obviously I aced the question, and whilst my server isn’t limited to static assets, it does have the GNU tool chain installed

6487 views for my Elgg CSS Fix page this month. For the record at the time of writing the site has had 6908 unique views and the next highest viewed page is only at 2446 views. Something is up! I wonder what ..

root@holly /var/log/apache2 # grep elgg idimmu.net.access.log | head 91.198.94.225 - - [24/Mar/2013:06:44:52 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.0" 200 37928 "http://www.idimmu.net/2011/11/21/elgg-1-8-tidypics-group-fix/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:44:52 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.0" 200 37928 "http://www.idimmu.net/2011/11/21/elgg-1-8-tidypics-group-fix/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:44:53 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/index.php HTTP/1.0" 301 471 "http://www.idimmu.net/index.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:44:53 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/index.php HTTP/1.0" 301 471 "http://www.idimmu.net/index.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:47:54 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.0" 200 37928 "http://www.idimmu.net/2011/11/21/elgg-1-8-tidypics-group-fix/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:47:54 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.0" 200 37928 "http://www.idimmu.net/2011/11/21/elgg-1-8-tidypics-group-fix/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:47:54 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/index.php HTTP/1.0" 301 471 "http://www.idimmu.net/index.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11" 91.198.94.225 - - [24/Mar/2013:06:47:55 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/index.php HTTP/1.0" 301 471 "http://www.idimmu.net/index.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11" 216.152.249.243 - - [24/Mar/2013:06:49:34 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.1" 200 9931 "http://www.idimmu.net/" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en]" 91.198.94.225 - - [24/Mar/2013:06:50:55 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/ HTTP/1.0" 200 37927 "http://www.idimmu.net/2011/11/21/elgg-1-8-tidypics-group-fix/" "Opera/9.80 (Windows NT 6.2; WOW64) Presto/2.12.388 Version/12.11"

Ok, so my access log rotated out on the 24th March but the data is still meaningful with out having to put together exact logs for the entire month! One IP 91.198.94.225 seems to be retrieving the Elgg page over and over again, several times a minute?!

root@holly /var/log/apache2 # grep 91.198.94.225 idimmu.net.access.log | wc -l 5876 root@holly /var/log/apache2 # grep elgg idimmu.net.access.log | grep 91.198.94.225 | wc -l 5876

And seemingly that same IP address is ONLY requesting my Elgg page, no other pages!

root@holly /var/log/apache2 # grep index.php idimmu.net.access.log | grep 91.198.94.225 | head -n 1 91.198.94.225 - - [24/Mar/2013:06:44:53 +0000] "GET /2011/11/21/elgg-1-8-tidypics-group-fix/index.php HTTP/1.0" 301 471 "http://www.idimmu.net/index.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11" root@holly /var/log/apache2 # grep index.php idimmu.net.access.log | grep 91.198.94.225 | wc -l 2938 root@holly /var/log/apache2 # grep -v index.php idimmu.net.access.log | grep 91.198.94.225 | wc -l 2938

Also half of it’s requests are for _/2011/11/21/elgg-1-8-tidypics-group-fix/index.php_ and the other half are for _/2011/11/21/elgg-1-8-tidypics-group-fix_ which is double weird. The index.php page extension just redirects to the extension-less URL due to the incredible intellectual artificial intelligence running the CMS platform I use!

The Elgg page has maybe 1 or 2 back links out there, none are to the index.php extended URL and looking at the Apache logs the referrer is http://www.idimmu.net/index.php which isn’t a valid URL anyway, which suggests whatever is doing this is spoofing the referrer.

Also also, what kind of browser definition is

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1.0 Safari/537.11

Some random infosec page I’ve never heard of suggests it’s a valid Chrome user agent running on Windows. You can write lots of interesting plugins for Chrome, including scrapers and bots so this is possibly a valid option. Why it also mentions Safari I’ve no idea though!

The 91.198.94.225 IP appears in Google several times for spamming so possibly that’s the point of the bot. Judging by the rest of the search results there appears to be a LOT of comment spam to old PHP guestbook implementations that look to have seemingly trivial captures, so maybe the bot thinks my site is running one of these? It does appear to be stuck in some kind of infinite loop on one page however

Whois says it’s an IP located in Poland! Has anyone else seen any ‘interesting’ impressions from this IP address, or anything similar? Fortunately I’m pretty good with computers.

iptables -A INPUT -s 91.198.94.225 -j DROP

In your face Polish!

For more iptables related nonsense I suggest you try O’Reilly’s Linux iptables Pocket Reference or man iptables if you don’t want to buy anything Or you know, just keep reading my stuff!

Experiments In Bitcoin

rus — Mon, 25 Mar 2013 23:48:02 +0000

Earlier today I got in to discussing Bitcoin Arbitrage with a mate of mine, with the cunning plan of creating a bot to monitor different exchanges and profit on the difference in trading prices. After an analysis this was deemed a silly idea, or conversely we aren’t good enough, as moving real money or bitcoins around the exchanges seems to be an extremely painful process. The first step though was creating a wallet!

Bitcoind

Multibit is a fantastic cross platform GUI Bitcoin wallet which is super easy to use but limits you to only using the workstation you’ve installed your wallet on, unless you get all remote desktopy. There are also numerous places that let you host your own wallets and most exchanges also feature a wallet too, but I’m very DIY! BitcoinD however is a little cross platform daemon that just sits their ticking away, managing your wallet, whislt responding to JSON-RPC over HTTP. Perfect for me!

‘Installation’ on Linux was easy, just download the tar.gz, extract, then run the bitcoind binary of choice! I run mine in a screen but there are init scripts if that’s your thing. It creates a wallet file in ~/.bitcoin. When it starts it suggests you configure a username and password for the RPC interface. You should do this. My ~/.bitcoin/bitcoin.conf looks like this:

rpcuser=rusisawesome rpcpassword=thisismysecretpassword

GUI

Ok, so you’ve a deamon running, now what. Well it logs to ~/.bitcoin/debug.log which is some indicator that it’s working, but a pretty GUI would be nice. I opted for bitcoin-js-remote, a nice javascript web app app, that comes with it’s own daemon or can be served over Apache. I’ve got Apache installed at home so I opted for that.

Extract bitcoin-js-remote somewhere nice, e.g. /var/www/bjr and configure Apache. You’ll want to use HTTP Basic Auth to protect the directory, which we’ll go in to, and ideally use SSL (best practice ftw) which we won’t go in to!

You’ll need to create, or add to, an htpasswd file with the RPC credentials you’ve configured bitcoind to use earlier.

htpasswd -b -c /etc/apache2/htpasswd.pass rusisawesome thisismysecretpassword

BitcoinD by default will listen on port 8332 on localhost, and the bitcoin-js-remote will call back via Apache to talk to it so we need to configure Apache to do a bit of proxying as discussed here.

AuthType Basic
AuthName Restricted
AuthUserFile /etc/apache2/htpasswd.pass
Require valid-user

ProxyPass /lbc/ http://127.0.0.1:8332/

AuthType Basic
AuthName “Bitcoin Access”
AuthUserFile /etc/apache2/htpasswd.pass
Require user rusisawesome
Order deny,allow
Allow from all

Restart Apache and jobs a good’n.

Bitcoin-js-remote also has a settings.json file. I’m not sure if it’s necessary but I added my credentials to that:

{ "RPC": { "url": "/lbc/", "user": "rusisawesome", "password": "thisismysecretpassword", "account": "" }, "labelsmode": 1 }

I don’t think this is necessary to be honest. You can now connect to your webserver, present the basic auth credentials and it should log you straight in to your new shiny web based client!

If you are hosting your own wallet, it’s advisable to back it up, as if you lose it, you lose all the money that’s inside it!

There are some free sources on the Internet that you can use to get some meagre BTC to play around with, I think I’ve got around $0.15 worth, though as you can see I’m still waiting for transactions to come through, so far the fastest has taken 3 hours and the slowest over 12 hours. Moving BTC around different exchanges for arbitrage purposes is kind of hopeless …

My Bitcoin address is 13gcobY9ofgnokCsU436yNVgCqGB3PVW6z, there are even nice bitcoin URL shortners so I don’t have to remember that monstrosity!

The Blockchain site, as well as offering wallet hosting, lets you observe and track all the transactions going on in the Bitcoin network as well as see the state of all the wallets on the network (if you know the address). The details of my wallet are here and the first transaction made to it earlier today is here. I have to say when I saw that transaction I got all excited by the prospect of almost 5 BTC, but it appears that multiple transactions can be done in the same block, mine was the smaller one at 0.002 BTC

I ran a Bitcoin miner on my MacBook at work earlier and it reckoned it would take 10,000 days to generate 1BTC. I think I’ve missed the bubble there

At the moment Bitcoin feels a little like the Esperanto of the financial world, but you can already by drugs and pay for strippers and pizza with it, which is probably more than Esperanto will let you do!

What’s really awesome about bitcoin is the bitcoin community itself. Reading about it feels like there’s a lot of entrepreneurial adventure going on whilst everyone is making land grabs to provide previously taken for granted services in new and exciting ways due to the financial flexibility that bitcoin provides. I think it’s exciting.

Android Create AVD

rus — Sat, 23 Mar 2013 17:55:06 +0000

The second part of this series covers creating our first Android Virtual Device or AVD. The Android Virtual Device (AVD) is basically a configuration for the Android SDK’s emulator that lets you define the hardware and software characteristics of an actual Android advice that you can then test your code on. Whilst not being as good as running your app on real hardware, testing it on an AVD is the next best thing!

You can either watch the Android Create AVD video above or follow the steps below, it’s dead easy!

Start The AVD Manager

First we need to start the AVD Manager by going to Window -> AVD Manager

Create A New AVD

Then click on New to bring up the AVD Configuration Dialog.

Choose AVD Settings

Enter an appropriate name, select your API target, the size of the SD card and skin/resolution you want to use then click Create AVD.

AVD Will Appear In The List

The AVD creation will take about 1 minute to run, but then the new AVD will appear in your list of available AVDs. It’s possible to make many different AVDs to test your application on many different screen sizes and hardware profiles!

If you’re new to the Android mobile operating system, Learning Android is the perfect way to master the fundamentals. This gentle introduction shows you how to use Android’s basic building blocks to develop user interfaces, store data, and more. You’ll build an example application throughout the course of book, adding new features with each chapter. You’ll also build your own toolbox of code patterns that will help you program any type of Android application with ease.

Install Android Eclipse

rus — Sat, 23 Mar 2013 17:50:16 +0000

Hi guys, when telling a story it’s always best to start at the beginning, so the first thing to do on my journey is talk about how are we going to install Android Eclipse. My workstation is a MacBook but the process is the same on both Windows and Linux. I’ve put together a short video of the entire thing, from downloading Eclipse and the Android SDK to installing Eclipse, the ADT (Android Development Tools) plugin and then configuring ADT to use the Android SDK!

The steps are really simple.

Download and install the Eclipse IDE for Java Developers from http://www.eclipse.org/downloads/
Download and extract the Android SDK from http://developer.android.com/sdk/index.html
Install the ADT plugin in Eclipse

After following these 3 easy steps to install Android Eclipse you should be ready to go!

10 Most Used Linux Commands

rus — Thu, 07 Mar 2013 16:26:24 +0000

Whilst skimming my Bash history today looking for an esoteric one liner I’d written earlier I started to ponder what my most used commands were, it’s easy enough to find out! This is actually output from my Mac, not a Linux box, I tricked you

Chill:~ rus$ history | awk {'print $2'} | sort | uniq -c | sort -nr | head -n 10 106 ssh 49 ls 44 cd 42 curl 34 scp 25 vi 15 mv 15 cp 12 grep 10 rm

ssh looks to be number one, which is understandable as I do a lot of work on remote machines.. so what does my usage pattern on remote servers look like?

root@xflipnag2:~# history | awk {'print $2'} | sort | uniq -c | sort -nr | head -n 10 211 vi 160 ls 150 cd 89 svn 39 grep 38 ps 28 cat 24 /usr/lib/nagios/plugins/check_http 24 puppet 20 puppetapply

This time it really is a Linux box, running OEl6 if you must know! Pretty similar, it’s a monitoring node so recently I’ve been using the check_http command to run some HTTP tests. Puppetapply is an alias of ours to run Puppet in a manner that will make changes, by default it runs noop.

It’s nice to see grep rocking, one of my favourite commands really. Though I’m surprised sed hasn’t shown up anywhere!

What are your 10 most used commands?