svn: Aborting commit: remains in conflict February 11, 2009
Posted by idimmu in linux.I got this annoying SVN error today, that I hadn't come across before. Even after resolving the conflict in the file, highlighted by lots of <<<<<<<, I still couldn't get my commit to work!
idimmu@boosh:~/work/systems/trunk/dns$ svn ci idimmu.net -m "new funky domain"
svn: Commit failed (details follow):
svn: Aborting commit: '/home/rus/work/systems/trunk/dns/idimmu.net' remains in conflict
After some reading it was easy to resolve!
idimmu@boosh:~/work/systems/trunk/dns$ svn resolved idimmu.net
Resolved conflicted state of 'idimmu.net'
rus@boosh:~/work/systems/trunk/dns$ svn ci idimmu.net -m "new funky domain"
Sending idimmu.net
Transmitting file data .
Committed revision 14281.
A lesson in regular svn updateing I guess :)
rpmdb: Lock table is out of available locker February 9, 2009
Posted by idimmu in linux.I had a crazy weird bug today whilst running Puppet on one of our CentOS boxes where no packages were being installed. A quick investigation with yum yielded the following scenario!
Id I tried to use yum as root or with sudo it gave the following error ..
[idimmu@server ~]$ sudo yum check-update
rpmdb: Lock table is out of available locker entries
error: db4 error(22) from db->close: Invalid argument
error: cannot open Providename index using db3 - Cannot allocate memory (12)
Repository update is listed more than once in the configuration
Repository base is listed more than once in the configuration
Setting up repositories
http://www.mirrorservice.org/sites/mirror.centos.org/Null/updates/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
Trying other mirror.
Cannot open/read repomd.xml file for repository: update
failure: repodata/repomd.xml from update: [Errno 256] No more mirrors to try.
Error: failure: repodata/repomd.xml from update: [Errno 256] No more mirrors to try.
But worked fine if I did it as a non root user ..
[idimmu@server ~]$ yum check-update
Repository update is listed more than once in the configuration
Repository base is listed more than once in the configuration
Setting up repositories
Reading repository metadata in from local files
acl.i386 2.2.23-5.4.el4 base
apr-util.i386 0.9.4-22.el4 base
at.i386 3.1.8-82.el4 base
audit.i386 1.0.16-3.el4 base
<snip>
I did a little bit of investigating with Google as to how yum works and discovered a few things..
http://www.mirrorservice.org/sites/mirror.centos.org/Null/updates/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
The Null in the URl stood out as an obvious case of "there's something wrong here", so looking at the yum.conf file ..
baseurl=http://www.mirrorservice.org/sites/mirror.centos.org/$releasever/os/$basearch/
You can see that the NULL is coming from the $releasever variable. So, how is that variable set?
At the top of the yum.conf there is
distroverpkg=centos-release
What yum does is check which version of the package specified by distroverpkg is installed, and assign that to $releasever.
Great, but why does yum think the version is NULL? Well, a search for the first error ..
rpmdb: Lock table is out of available locker entries
resulted in this site which explained everything!
Basically RPM uses a Berkley database and due to too many instances of rpm dieing (the server is an old server) it left locks on the database that were never cleared. In order to fix the problem we have to remove the locks, by unfortunately, deleting the database and recreating it!
Backup the database directory /var/lib/rpm first:
tar cvzf rpmdb-backup.tar.gz /var/lib/rpm
Delete the locked Berkeley databases that rpm uses:
rm /var/lib/rpm/__db.00*
Force rpm to rebuild the databases
rpm --rebuilddb
Now, check the database to make sure it has been rebuilt correctly:
rpm -qa | sort
Problem solved, my database is no longer corrupt and Puppet is installing packages again!
File Roller and 7zip August 12, 2008
Posted by idimmu in linux.I recently needed to extract a file with the new, popular 7zip compression, which Ubuntu's File Roller doesn't support out of the box. Enabling 7zip support in File Roller was as simple as:
apt-get install p7zip-full
Saving the RAID May 22, 2008
Posted by idimmu in linux.I managed to screw up my home machine when upgrading from Dapper to Hardy somehow, so decided to flatten and reinstall Hardy from scratch.
This is all well and good but I have a RAID 5 array on my machine which stores all my important documents and stuff (I really hate drive failure). Ubuntu didn't suddenly detect my RAID array and figure everything out, which was not entirely unsurprising but was a little scary.
After some light reading, and a few sighs and heart flutters I worked out the magic runes to tell the system I have a RAID array, where it was and how to use it!
root@server:/root# mdadm -A /dev/md0 -v /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 4.
mdadm: added /dev/sdb to /dev/md0 as 1
mdadm: added /dev/sdc to /dev/md0 as 2
mdadm: added /dev/sdd to /dev/md0 as 3
mdadm: added /dev/sde to /dev/md0 as 4
mdadm: added /dev/sda to /dev/md0 as 0
mdadm: /dev/md0 has been started with 5 drives.
root@server:/root# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sde[4] sdd[3] sdc[2] sdb[1]
976793600 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices: <none>
root@server:/root# mdadm -Es
ARRAY /dev/md0 level=raid5 num-devices=5 UUID=19c77b42:c5e86159:c23e7b08:7f8ceabd
xend refusing to start March 10, 2008
Posted by idimmu in linux.We recently had a few power outages at work, some scheduled, some not, and this played havoc with our xen servers.
One of the problems we had was that xend would not start (and thus xendomains would also not start).
Checking /var/log/xen/xend.log gave us the following snippet:
inst = XendNode()
File "/usr/lib/python2.5/site-packages/xen/xend/XendNode.py", line 164, in __init__
saved_pifs = self.state_store.load_state('pif')
File "/usr/lib/python2.5/site-packages/xen/xend/XendStateStore.py", line 104, in
load_state
dom = minidom.parse(xml_path)
File "xml/dom/minidom.py", line 1913, in parse
File "xml/dom/expatbuilder.py", line 924, in parse
File "xml/dom/expatbuilder.py", line 211, in parseFile
ExpatError: no element found: line 1, column 0
[2008-03-10 21:37:40 18122] INFO (__init__:1094) Xend exited with status 1.
A quick google of that error revealed several people that had come across the same problem, but no actual answer!
It looks like xen is having problems parsing an xml file, so some quick mental inspiration, and the find command, yielded /var/lib/xend/state/pif.xml which was a 0 byte file! A comparison to a working server showed that it should (or atleast could) contain this:
<?xml version="1.0" ?>
<pifs/>
A copy and paste later and we had a working xend! However it refused to create any of the xenlets:
root@xen0:/etc/xen# xm create server0.cfg
Using config file "./server0.cfg".
Error: The privileged domain did not balloon!
Despite their being plenty of RAM!
root@xen0:/var/log/xen# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 7928 8 r----- 832.8
root@xen0:/var/log/xen# free
total used free shared buffers cached
Mem: 8119416 393028 7726388 0 11344 58832
-/+ buffers/cache: 322852 7796564
Swap: 15631224 0 15631224
An strace of the process revealed xen did think it had less memory available than it actually had ..
[2008-03-10 21:47:48 18620] DEBUG (__init__:1094) Balloon: 131064 KiB free; 0 to scrub;
need 524288; retries: 20.
As we had a working xend finally we decided to implement a technique we'd learned from working with Windows machines and rebooted the server. This magically fixed the memory issue, it would have been nice to know what actually caused it and if there was a proper fix though.
