Tag Archive: linux

Every now and then I encounter a server that have multiple external drives attached to it. When the server is rebooted – let’s say we just updated the kernel – it sometimes happens that the external drives don’t get assigned the same device numbers they had before. This, in turn, means that we can’t mount them the old way, i.e. by device number, via /etc/fstab.

Fortunately, there are alternative ways of doing this. Let’s start by finding out the filesytem ID.

For ext2/3/4, do:

# dumpe2fs /dev/sdb1 | grep UUID
dumpe2fs 1.39 (29-May-2006)
Filesystem UUID:          5fbb68f9-86af-4784-bdb3-3ab452a5f263

For reiserfs, do:

# debugreiserfs /dev/sdc1 | grep UUID
debugreiserfs 3.6.21 (2009 www.namesys.com)

UUID: aa3c3244-83c2-491e-bab5-067c5e73ce9c

For xfs, do:

# xfs_admin -u /dev/sdc1 
UUID = 0d4139ef-db18-417c-8dc0-273f94acfa3a

JFS, as far as I can tell, does not automatically generate a UUID when you create the filesystem. You have to do it.

# jfs_tune -U random /dev/sdc1 
jfs_tune version 1.1.12, 24-Aug-2007
UUID updated successfully.

Now, to view the UUID, do:

# jfs_tune -l /dev/sdc1 | grep UUID
File system UUID:       cab1b11d-1f09-4dc6-aec8-b84537ff895f
External log UUID:      b02579b7-0100-0000-0000-000001000000

Now, adjust your /etc/fstab:

#/dev/sdc1                                 /backup   ext3  defaults 0 0 
UUID=b7bcf64a-e172-42ce-8ce7-12365cc72c6c  /backup   ext3  defaults 0 0

This will ensure that each filesystem is mounted in the right place, even if the device names come up in the wrong order.

I encountered this today when trying to query a package on a Red Hat server:

rpmdb: Lock table is out of available locker entries
error: db4 error(22) from db->close: Invalid argument
error: cannot open Name index using db3 – Cannot allocate memory  (12)

This happens when an operation involving the rpm db gets interrupted and the locks not cleared.  The solution is rather simple:

1.   Make a backup of /var/lib/rpm   (because you never know)

2.   rm /var/lib/rpm/__db.00*

3.   rpm --rebuilddb

You should have a working rpm databse now.

Today I got tasked with removing duplicate mails from a mail folder with over 100,000 mails in it.  Doing this from a mail client is so impractical, it’s not even worth giving any thought at all.  Fortunately, the mailbox is on a mail server using Maildir style mailboxes, so I knew this could be done with minimum effort.

I discovered the ‘reformail’ utility, part of courier-imap, and after a few trial runs, I settled on the following:

# cd /path/to/mailbox/Maildir/cur

# for i in `find . -type f`; do reformail -D 10000000 /tmp/duplicates <$i && rm $i; done

-D looks for, and deletes duplicates.

10000000 is the length of the temporary file where a list of message IDs will be written

/tmp/duplicates is the aforementioned temporary file.

The temporary file needs to be big enough to accommodate the message ID of each mail.  In this particular case, I have found the average length to be 54 characters, but I would suggest using around double that to be safe.  So adjust to your needs.

In a big mail folder, and especially on ext3, this will take a long time to complete.

I recently decided to move my enormous mail archive from my trusty Courier-imap/Maildir setup to DBmail. The reason is simple.  I have several mail folders with 100,000+ mails in.  That means several directories with upwards of 100,000 files in.   And that means bad performance.    There is another reason:  I can execute far more powerful searches with an SQL query, than any mail client can allow me to do.

DBmail is a POP3/IMAP server that uses a regular database server (currently MySQL, PostgreSQL or SQLite) for its mail store.  Given the obvious advantages, I’m surprised this isn’t more popular.

There is one gotcha to the setup.  MySQL, being Swedish, has a default collation setting of ‘latin1_swedish_ci’ while DBmail assumes ‘utf8_general_ci’ will be set.  But the DBmail docs, and even the MySQL notes page, does not mention this* at all, and the included create_tables.mysql script does not set the correct collation either.   This results in the following error showing up in the logs:

Sep 14 03:00:01 hermes dbmail/maintenance[16708]: Error: dbmysql.c,db_mysql_check_collations(+138): collation mismatch, your MySQL configuration specifies a different charset than the data currently in your DBMail database.

This is easily fixed.  Assuming your database is called ‘dbmail’ do:

mysql> use information_schema;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> select * from SCHEMATA where SCHEMA_NAME = ‘dbmail’;

The ‘DEFAULT_COLLATION_NAME’ column will most likely show ‘latin1_swedish_ci’ – this is the problem.  Run the following:
mysql> alter database `dbmail` collate `utf8_general_ci`;
Query OK, 1 row affected (0.02 sec)

Run the select query again and check if the right collation is showing.  After doing this, dbmail should connect to MySQL without any problems.