Steve Kemp's Blog

By the time this blog entry goes live I'll be running upon my new machine. The migration process was mostly straightfoward and followed my plan:

• Using my existing desktop system as a PXE server to install Lenny over the network.
• Copied over important directories.
• Restored backups.
• Turned off old machine.

Of course it wasn't that simple in practise, as previously mentioned the whole reason I was looking for a new machine was because the software RAID upon my old desktop was failing - One of the two drives was completely dead.

As I'd feared the second drive failed partway through my migration. But thankfully I'd copied off the important stuff before then, and the backups I have off-site mostly covered everything else. (The things I lost were things I can find again such as ~/Music, ~/Videos. On the one hand they're too large to backup, on the other hand I should probably do it next time as they never change.)

Unfortunately the version of X in Lenny refused to work with the GeForce G210 video card I had. To be more correct using the Vesa driver I could get a picture and a smooth desktop, but when watching videos with xine I got maybe two frames a second. Both the open nv driver and the closed nvidia driver failed to support the card - so I swapped hardware, and I'm now running with the GeForce 7300 GS card from my previous desktop. This allows me to watch videos at full-screen with no issues. (Desktop size is 1600x1200 FWIW).

So now it's just a matter of tweaking the system. I've installed enough to be useful:

• miredo - So I have IPv6 connectivity despite Virgion.
• squid - So that I have a decent cache for surfing.
• pdnsd - So I have a caching nameserver and am not at the whim of Virgin.
• kvm - So I can setup scratch machines for play.

I've still got to setup pbuilder, but that'll be done shortly, and I've installed backported packages such that I can watch youtube videos. I'm currently running firefox from lenny but I expect that will change soon enough - not least because that version fails to support "adblockplus", only "adblock".

Two partitions md0 for /boot and md1 used as LVM, from which I've taken /, /home, etc:

Filesystem                      Size    Used    Avail Use% Mounted on
/dev/mapper/birthday--vol-root   9.9G     2.8G   6.6G  30% /
/dev/mapper/birthday--vol-home   22G      4.3G  16G    22% /home
/dev/mapper/birthday--vol-music  127G    43G    78G    36% /mnt/music
/dev/md0                         988M    38M    901M    4% /boot
/dev/mapper/birthday--vol-kvm    22G      8.8G  12G    44% /mnt/kvm
/dev/sdg1                        163G    143G   12G    93% /media/disk
skx@birthday:~/hg/blog/data$

skx@birthday:~/hg/blog/data$ sudo pvs
[sudo] password for skx:
  PV         VG           Fmt  Attr PSize   PFree
  /dev/md1   birthday-vol lvm2 a-   464.82G 274.51G

Update: Three irritations with this machine:

1. As supplied the BIOS was set with "USB Mouse" and "USB Keyboard" set to "disabled". I had to beg the loan of a keyboard from a neighbour.
2. As supplied the BIOS had virtualisation set to "disabled". Not a huge shock, but it caught me out regardless.
3. As supplied the system had only a single SATA power connector. Annoying given that the motherboard is advertised as having "onboard RAID" and I'd purchased it with two hard drives. Happily I had a spare adaptor to hand.

I'd still recommend Novatech, but the last point had me swearing for a few minutes until I realised I did have a spare adaptor in the house.

ObFilm: Chitty Chitty Bang Bang

Tomorrow, all being well, I'll receive a new computer.

I've always run Debian unstable upon my desktop in the past, partly because I wanted to have "new stuff" and partly because I needed a Debian unstable system for building Debian packages with.

However I'm strongly tempted to just install Lenny. I use that upon my work desktop and it does me just fine for surfing, building tools, and similar.

I can use pbuilder, sbuildd, or similar to build packages for upload to Debian, and if I want to experiment with new-hotness I can use a KVM guest or two.

Providing the hardware works with Lenny (and I have no reason to believe it won't) then there's no obvious downside I can think of.

The only potential complication will be restoring my backups, it is possible that my firefox databases, and similar things, might not work on older version. Still we shall see.

I plan to install software RAID, and run the system on LVM because quite frankly it rocks. Unless my current system fails in the next 24 hours I can use that to do the installation (My current desktop acts as a TFTP/DHCP/NFS server so I can use it to PXE-boot).

Anyway now I need to go eat food, tidy my desk, and decide what to call the machine .. At the moment the choice is between "march.my.flat" and birthday.my.flat, as my 34th birthday is on March 10th.

ObFilm: 300

As previously mentioned I've switched my webserving over to a mixture of apache2 & thttpd.

I chose thttpd as it is simple to configure for my needs, and supports the execution of CGI scripts. Some of the other simple webservers available to Debian's current stable release (such as nginx) don't support CGI so they were ruled out.

Of course prior to choosing thttpd I looked at the state of the Debian package. Distressingly the package has no current maintainer and has several bugs open, including some that have been open for several years without comment.

I've just made my second upload fixing a couple of bugs, including ones that I could see affecting myself, but now I'm done with it.

In conclusion:

• I've fixed a few bugs.
• I suspect that many of the open bugs are 100% unreproducable and should be closed after checking with the submitter.
• The package could do with a volunteer to maintain it.

On the one hand it is "just another webserver", on the other hand it is genuinely small, simple to configure, and has a couple of compelling features (CGI + throttling).

So. Go. Adopt. Maintain.

Pretty please...

ObFilm: Red Sonia

Recently I was flirting with the idea of creating an online game, but I got distracted by wondering how to make the back-end more flexible.

To communicate the state of the game to N connected clients I figured I needed some kind of server which would accept "join"/"quit" requests and then make changes available.

To that end I came up with the idea that a client would make requests via HTTP such as:

http://example.com/server/game/chess/join

This would regard the originating client as part of a new chess game, for example, and return a UID identifying the "game channel".

http://example.com/server/changes/1-2-3-4

This will retrieve a list of all events which had occurred in the game which had not already been sent.

(Here 1-2-3-4 is obviously the UID previously allocated.)

http://example.com/server/submit/1-2-3-4/move

This would submit the move "move" to the server.

After mulling this over for a while it seemed like a great reusable solution, I'd make an initial "join" request, then repeated polling with the allocated UID would allow game moves to be observed. All using JSON over HTTP as the transport.

It was only this morning that I realised I'd have saved a lot of time if I'd just proxied requests to a private IRC server, as the functionality is essentially the same.

Still I'm sure this pattern of "join"/"poll"/"quit" could be useful for a lot of dynamic websites, even in the non-gaming world. So although the idea was mostly shelved it was an interesting thing to have experimented with.

D'oh.

ObFilm: Conan The Barbarian

I don't talk about SPAM publicly these days, for reasons that are probably self-explanatory.

However this is just insane:

• Saturday 20th February 2010: Registered a new domain.
• Sunday 21st February 2010: Received first spam.

Currently at 40+ SPAM mails and rising; all mails addressed to "postmaster@", rather than any past users of the domain. (I can see from http://archive.org that the domain was last active in 2008.)

ObSubject: The Goonies

I've been overhauling the way that I am host a number of virtual websites upon my main box. Partly to increase security, and partly for a cleaner separation or roles, ownership, and control. (In general everything on my box is "mine", but some things are "ours"...)

After a fair amount of experimentation I decided that I wasn't willing or able to rewrite all my Apache mod_rewrite rules just yet. So my interim plan was to update each existing virtual host:

• Add a dedicated user & group to run it under.
• Launch it via a minimal server listening upon the loopback adapter.
• Have Apache 2.x proxy through to it.
  • Expanding any mod_rewrite rules prior to the proxying.

To make it clear what the users were for I decided that every hosting-user would have an "s-" prefix. So the virtual host "static.steve.org.uk" was initially going to be served by the s-static user.

The thttpd configuration file would look like this, and would be located in /etc/thttpd/sites/static.steve.org.uk:

host=127.0.0.1
port=1008
dir=/home/www/static.steve.org.uk/htdocs/
chroot
user=s-static
throttles=/etc/thttpd/throttle.conf
logfile=/home/www/static.steve.org.uk/logs/thttpd.log
pidfile=/home/www/static.steve.org.uk/pid/file

(I wrote a trivial script to stop/start all the sites en mass, and removed the default thttpd init script, logrotation job, and similar things.)

How did I decide which port to run this instance under? By taking the UID of the user:

steve@skx:~$ id s-static
uid=1008(s-static) gid=1009(s-static) groups=1009(s-static)

With this in place I could then update the Apache configuration file from serving the site directly to merely proxying to the back-end server:

<VirtualHost *>
    ServerName  static.steve.org.uk

    # Proxy ACL
    <Proxy *>
        Order allow,deny
        Allow from all
    </Proxy>

    # Proxy directives
    ProxyPass          /   http://localhost:1008/
    ProxyPassReverse   /   http://localhost:1008/
    ProxyPreserveHost on
</VirtualHost>

So was that all there is to it? Sadly not. There were a couple of minor issues, some of which were:

cronjobs

I have various cron-jobs in my main steve account which previously updated blog indexes, etc. (I use namazu2 to make my blog searchable.)

I had to change the ownership of the existing indexes, the scripts themselves, and move the cronjob to the new s-blog user.

cross-user dependencies

I run a couple of sites which pull in content from other locations. For example a couple of list summaries, and archives. These are generally fed from a ~/.procmail snippet under my primary login.

Since my primary login no longer owns the web-tree it is no longer able to update things directly. Instead I had to duplicate a couple of subscriptions and move this work under the UID of the site-owner.

I'm no longer running apache

For a day or two I'd forgotten I was using the apache facility to include snippets in my site; such as links to my wishlist.

Since I'm not using Apache in the back-end server-parsed files no longer work. Happily I'm using a simple template-based setup for my main sites, so I updated the template-parser to understand "##include ./path/to/file". For example this source file produces my donation page.

The upshot is my "static" site is even more static, which is a good thing.

uploads are harder

Several of my domains host entirely static content which is generated on my main desktop machine, and then uploaded via rsync post-build.

I had to add some more accounts and configure SSH keys, then update the uploading routines/Makefiles appropriately. Not a major annoyance, but suddenly my sshd_config file has gone from "PermitUser steve,backup" to including many additional accounts.

The single biggest pain was handling my my mercurial repositories - overhauling that took a bit of creativity to ensure that nothing was broken for existing or new checkouts. I wish that a backport of mercurial-server was trivial because I'd love to be using that.

In general though watching the thttpd logs has been sufficient to spot problems. I had to tweak things a little to generate statistics properly, but otherwise all is good.

Why thttpd? Well small, lightweight, and the ability to run CGI scripts. Something missing from nginx for example.

I'm still aiming to remove apache2 from the front-end - it is mostly just a dumb proxy, but it does perform some ACL operations and expand mod_rewrite rules. I could port those to another engine .. but not today.

The most likely candidates are nginx, perlbal, or lighttpd - each of these should be capable of doing simple ACL checks, and performing mod_rewrite-like rules.

ObFilm: Mallrats

As part of some house-keeping I've been checking over my systems and ensuring they're all tickity-boo for the past couple of days.

One thing that I'm getting increasingly tempted by is converting my kvm guest to a 64-bit system.

I've not quite sold myself on the prospect of what will be a fair amount of downtime, but I'm 90% there.

I do think that a lot of my setup needs an overhaul, for example:

• Running all my websites under www-data is beginning to worry me.
• Running services as root is beginning to make me more and more paranoid.

One possible plan is to wipe my system, and then restore data from backups. A perhaps saner approach is divide my guest into two smaller ones, and migrate services over one by one (e.g. website1, website2, .. websiteN, email, etc).

For the moment I've taken a complete dump of my existing guest, and I'm running it with an IP in the 10.0.0.0/24 range on my desktop. That's at least given me a clear idea of the amount of work involved.

I'm still a little unclear on how best to manage running N websites with the intention they'll each run under their own UID. I guess it comes down to having a few instances of nginx/lighttpd/apache and then proxy from *:80 to the actual back-end. Precisely which mixture of services to use is a little overwhelming. Though at some point soon I need to start enabling IPv6 support, and that changes things a little.

(Not least because nginx has no IPv6 support present in the Lenny release - I've got a backported package which I run on the Debian Administration website.)

It's possible I could hack mod_vhost_alias to redirect/proxy to a local port based upon the virtual hostname present in the request - that's pretty trivial and I've already done something similar for work purposes. Though something like that should presumably already exist? I would expect a map of some form:

example.org: 127.0.0.1:8080
example.net: 127.0.0.1:9090

That has to be about the minimum necessary information to make the decision; a pair of vhost name & local destination.

/me googles some..

Update

OK quick update I've added local users for some of my sites, and now have them running under thttpd.

skx:/etc/thttpd# ls -ltr /home/www/ | tail -n 4
drwxr-sr-x  4 s-static   s-static   4096 Jan 15 01:41 static.steve.org.uk
drwxr-sr-x  5 s-openid   s-openid   4096 Feb 16 21:31 openid.steve.org.uk
drwxr-sr-x  6 s-images   s-images   4096 Feb 16 21:52 images.steve.org.uk
drwxr-sr-x  5 s-packages s-packages 4096 Feb 16 22:03 packages.steve.org.uk

That seems to work well, with a small wrapper script to start N instances of thttpd instead of a single one. Minor issues are that I'm using mod_proxy to forward requests to the thtpd instances running upon the loopback - and it was initially logging 127.0.0.1 as the source IP. A quick patch later all is well.

I'll leave it running a couple of the simple sites for the next few days and see if it kills children. If it does I'll convert the rest.

Probably will aim to have nginx in front of thttpd, instead of Apache, but this way I don't have to worry about mod_rewrite rules just yet.

ObFilm: Cruel Intentions

I've been trying to remember to post the pictures I like online for the past few months. So this is a reminder to myself.

This image below didn't turn out quite how I wanted it to:

• I was hoping for a nicer sihouet upon the lady's face.
• The tree-branch on the left irritates me.

But that said I keep on coming back to look at it. I like the lighting, and I love the way that the brick wall on the right hand side angles towards the building on the horizon.

Enjoy. Or not.

A similarly "not perfect" image is this outdoor shot. I have only one irritation with this shot - and that is that the trees are clipped at the top. Meh, such is life.

(I have two styles of photography; semi-random where I snap what is in front of me, and staged where I try to construct a particular picture - the two images above? One of each.)

ObFilm: Bound

I was interested to see Adnan Hodzic discuss life without evolution in the GNOME environment recently.

I too use GNOME as my desktop environment (I sometimes toy with various tiling window managers before getting annoyed at something or other).

My solution to the GNOME problem is to purge the gnome-desktop-environment package and instead my own local package gnome-desktop-minimal. This package is a meta-package which includes a smaller selection of GNOME packages, notably ignoring several that the gnome-core package would pull in such as eog - why install that when I prefer qiv or feh?

If I believed we could agree on precisely which packages to include I would submit a bug to the gnome team "Please provide gnome-desktop-minimal" or similar. Still I suspect individual biases/preferences will make such a suggestion contentious at best and impossible to satisfy at worst.

ObTitle: Léon

Slaughter

I received my first patch to slaughter today, which made me happy.

(I've made a new release including it, and updated the list of primitives to actually document the file-deletion facilities which previously I'd omitted to avoid encouraging mass-breakage.)

Signing Binaries

Andrew Pollock mentions that the days of elfsign might be numbered.

This is a shame because I've always liked the idea of signing binaries. Once upon a time, in the 2.4.x days, I wrote a kernel patch which would refuse to execute non-signed binaries. (This was mostly a waste of time; since it denied the execution of shell scripts. Which meant that the system init scripts mostly failed. My solution in the end was to only modprobe my module once the system was up and running, and hope for the best ...)

Right now, having performed only a quick search, I don't see anything like that at the moment.

• elfsign will let you store a binaries MD5 hash.
• bsign will let you sign a binary with a GPG key.

But where is the kernel patch to only execute such hashed/signed binaries, preventing the execution of random shell scripts and potentially trojaned binaries?

Without that I think signing binaries is a crazyish thing to do. Sure you can test that a file hasn't been modified, but even without those tools you can do the same thing via md5sums.

(ObRandom: Clearly if you mass-modify all your binaries the system md5sums database will be trashed.)

Perl UTF

I've received a bug report against chronicle, my blog compiler.

It seems that some versions of perl fail to run this:

     #
     #  Run the command, reading stdout.
     #
    open( FILTER, "$cmd|;utf8" ) or
       die "Failed to run filter: $!";

Removing the ;utf8 filter allows things to work, but will trash any UTF-8 characters from the output - so that's a nasty solution.

I'm not sure what the sane solution is here, so I'm going to sit on it for a few days and continue to write test scripts.

ObSubject: 300