Discriminate against me, please!

I have all the advantages - I'm a healthy able straight young-to-middle-aged white man from an educated 1st world background, etc.

I strongly support "affirmative action" including quota systems to address workplace employment balance and diversity. It's important for all of us, including the highly advantaged and privileged. The recent Google memo kerfuffle has prompted me to explain why I would support things that are seemingly contrary to my own interests. But before you read further, please read this brilliant rebuttal of the above memo. (Tolerance is not a moral precept discusses the philosophy in more depth).

Core 2 Duo CPU whine - workaround for Linux

Do you use Linux on a laptop and fear you're going to be driven totally insane by the high-pitched whining noise (squeal/screech) coming from your Core 2 Duo CPU's badly designed cpu packaging and voltage regulation? Like proximity to your laptop gives you tinea?

Me too. A workaround that'll cost you a little battery life (but probably not more than a few tens of minutes out of four-plus hour runtimes) is to add processor.max_cstate=3 to your grub command line. This turns off the problematic C4 power state of deep CPU sleep at reduced voltage, but otherwise leaves power management unaffected.

( For Ubuntu users, dpkg-reconfigure grub2 will offer to let you edit the kernel command line. )

LVM vs VSS - it's no contest

Richard Jones writes about Microsoft's Virtual Shadow Copy Service (VSS, not to be confused with Visual Source Safe), and laments the lack of any equivalent or good alternative on Linux servers.

I couldn't agree more, but there's more to it than what he focuses on. Application consistency is in fact the least of the problems.

APP CONSISTENCY

App consistency and pausing is missing in Linux, but is in principle not hard to introduce. It is something where D-BUS can be very useful, as it provides another piece of the puzzle in the form of inter-application and application<->system signaling. The trouble is that, like with most things in the Linux-related world, some kind of agreement needs to be reached by interested parties.

All it really takes for app consistency is two d-bus events:

1. "Prepare for snapshot - pause activity, fsync, etc."
2. "Snapshot taken/failed/cancelled, safe to resume"

... though there needs to be some aliveness-checking and time-limiting in place so that a broken application can't cause indefinite outages in other apps by not responding to a snapshot message.

Off the top of my head, products that'd benefit from this include Cyrus IMAPd, PostgreSQL, MySQL, any other database you care to name...

... IS ONLY A SMALL PART OF THE PUZZLE

As compared to VSS, using dm-snapshot (LVM snapshots) on Linux suffers from a number of significant deficiencies and problems:

• Snapshots require you to use LVM. LVM doesn't support write barriers, so file systems must use much slower full flushes instead.
• Snapshots are block-level not file-system-level, so the file system isn't aware of the snapshot being taken.
• Because snapshots are block-level and not filesystem-aware, the snapshot must track even low-level file system activity like file defragmentation, erasing free space, etc. This means they grow fast and have a higher impact on system I/O load.
• Accessing a snapshot requires mounting it manually to another location and faffing around with the changed paths. There's no way for an app to simply request that it sees a consistent point-in-time view of the FS instead of the current state, as in VSS. This is clumsy and painful especially for backups and the like.
• The file system being mounted has to be able to cope with being mounted read-only in crashed state - it can't do journal replay/recovery etc. LVM doesn't even give the *filesystem* a chance to make its self consistent before the snapshot. Some file systems, like XFS, offer manual pause/resume commands that may be issued before and after a snapshot is taken to work around this.
• Snapshots don't age out particularly gracefully. You have to guess how much space you'll need to store changes, as LVM isn't smart enough to just use free space in the VG until it runs out. The snapshot's change tracking space is reserved and can't be used for anything else (even another snapshot) until the snapshot is freed. If you over-estimate you can have fewer snapshots and need to keep more free space around. If you under-estimate your snapshot may die before you finish using it. Argh!

So: even with the ability to pause app activity, there's unfortunately a lot more to be done at the LVM level before anything even half as good as VSS is possible. LVM has been progressing little, if at all, for quite a few years now and some of these problems (such as the namespace issues and lack of FS integration) aren't even practical to solve with a block-driver level approach like LVM's.

At this point, one can only hope that BTRFS can do a better job, so that we can switch to runnning btrfs on mdadm raid volumes and breathe a sigh of relief.

Scripts to add replaygain tags to a flac collection

If, like me, you like to be able to play your music without constantly having to lunge for the volume control, and you store the master copy of your (ripped) CD collection in FLAC format you might be interested in a quick way to (re)calculate all the ReplyGain tags on your music. I'll also go into how to automatically create smaller MP3 copies of the files with the replaygain bias "burned in" so stupid players still get the volume right.
Finally, there's another little script here that can be used to fix up issues like inconsistent artist naming. It only does "The X" -> "X, The" at the moment, but it's easy enough to extend.
Read on if you're happy with shell scripting (and a Linux or advanced Mac OS X user).

The best eBook reader for Linux is currently....

 Microsoft Reader run under WINE.

Sigh. Not only is it the best, it's practically the only one unless you're content with fixed-format PDF. Few eBooks are available in, or reasonably convertible to, HTML, and even if they were there aren't any HTML renderers that can do half-decent H&J. None at all can hyphenate even poorly, and justification support tends to be limited to clumsy expansion-only justification that is ugly and not very nice to read.

So, to get a decent result one would basically have to hand-convert a plain text or HTML format book (possibly after pdf-to-text conversion) to TeX and typeset it for a particular display. That's not exactly a nice, convenient way to sit down for a good read. Even then, unless you use pdftex and read with Adobe Reader it won't even remember your place!

By contrast the Microsoft Reader .lit format is fairly widespread, supports automatic and somewhat decent H&J (though nothing on TeX / InDesign ), remembers your place in each book in the library, tracks and manages the library without forcing a particular on-disk structure on it, supports easy drag-and-drop of a book onto the program even from Nautilus, etc. It's friggin' emulated* Windows software that hasn't been updated or improved since it was practically abandoned by MS in 2003 and it's still better than anything available natively for Linux.

The situation is just as dire for Linux-based ( and Symbian-based ) phones and tablets. Given the spread of Qt to more and more devices, as well as all major platforms, I'm increasingly tempted to start work on a Qt-based reader with decent H&J, library management, place tracking / bookmarking / margin notes, etc. But how can there not be something out there already? Am I just blind, or is there really a gaping hole this big in free software capabilities?

Any suggestions? Anyone interested in working on one?

* I know, I know; I just don't care that W.I.N.E. Kudos to the WINE folks for their amazing work.

Australian Bandwidth Pricing

I recently switched hosting of large files (12Mb or so PDFs) on my employer's website from the existing Australian host to one in the USA. Why? Because it's cheaper to send data from the USA to Australian users than it is to send it from within Australia..

About 100x cheaper, in this case, when comparing Anchor Networks per-Gb pricing to SimpleCDN's.

SimpleCDN doesn't have any Australian node(s). Data gets requested by their US nodes on first request, cached, and sent back to Australia. Yet they're incredibly, vastly cheaper than anything local I can find.

The root of the problem appears to be that Australian hosting providers charge all IP traffic as if it were to go via an international link. There's no provision made for peering or intra-national traffic at the majority of hosts. This may be an issue with the hosting provider its self, or it may be with their upstream bandwidth suppliers, but I don't care. Internet routing is designed to solve this sort of problem - thankyou BGP - and peering points exist for a reason.

It's actually way cheaper to store your data in the UK, Ireland, Germany, France, or pretty much anywhere except Australia even if your users are 100% Australian. Isn't that kind of sad?

Technical Support for Commercial / Proprietary Software

The Anchor Networks head sysadmin has an opinion on commercial support for software that's pretty similar to mine - it's garbage. Both of us have learned this from painful experience.

The post is well worth a read if you're in a sysadmin/tech line of work. It mirrors my experiences with several vendors very closely, except that this particular case doesn't include any inter-vendor buck-passing or blame games. There's a reason more and more of the systems at work run on software I have source code to and can rely on myself to maintain - because that way, things actually get fixed.

If you think Anchor's experience with dedicated commercial support organizations is bad, you should try contacting tech support for incredibly expensive commercial software you've licensed and asking them to support their product! I've had totally disinterested or completely useless support from vendors of ten thousand dollar software packages. After all, just because I paid for it doesn't mean I should expect it to work as advertised or expect them to be interested in fixing bugs, right?

Adobe, Quark, MYOB, Apple. This means you.

Anyway, the downside of doing all the support work in-house is that you need to have the skills to undestand and run the systems you use. You can't run a DNS server if you don't understand DNS, can't run a mail server if you don't understand IMAP,SMTP,POP3 and TLS, etc. However, given that vendor support seems to be totally useless except for problems a retarded monkey could figure out, it's beyond me how people with no understanding of the systems they work with ever get anything done, whether or not they're paying for support.

Maybe they don't? It'd explain a lot about many of the businesses I work with...