Peace, Love and C++

I’ve expanded a bit on my idea (that is, the sbuilder wrapper to help aid with managing more then one sbuild chroot), and it got out of hand very quickly.

It’s gone from one small shell script to a set of 8, and growing. Quickly.

Until I spin these off into their own repo, they’ll be in my dot-bin

Before getting too far, please relize this is only how I’m using sbuild. There are many other (more) sane configurations that you can (and should) use.

This was not made to be useful for anyone other then me :)

I’ve added the ability to use unionfs-fuse, set up a way to login interactively and a script to create new chroots without having them register as a global schroot.

Firstly, forgive me for my sins. This is all a huge hack, but it’s use isn’t the worst thing in the world.

As of the time of this writing, I now have a gcc and clang chroot for each of stable, testing and unstable.

Now — the big question — how do you use this. Rather, this is more of a how do I use sbuilder :)

Firstly, I’ve created a file for each schroot set that I’m using. Here’s my /etc/schroot/chroot.d/sid.

[sid]
type=directory
description=Debian sid/amd64 autobuilder
directory=/var/sbuild/sid
groups=root,sbuild
root-groups=root,sbuild
script-config=sbuild/config

It’s very similar to the one that sbuilder-createchroot creates.

If you’re just looking for aufs / unionfs support, keep in mind schroot has some great support. check man 5 schroot.conf.

Next, create a dist.d directory in /var/sbuild — such as /var/sbuild/sid.d. In the future, this might get automagically created by the sb-create script. For now, it just expects that exists.

The arguments for all the sb* commands is always in the order of variant, dist.

variant defaults to “generic”, which I’ve set as an alias to the generic chroot, and the dist defaults to “sid”. At some point, I’ll change the default to be the last UNRELEASED entry in the changelog.

So, to create a new chroot to become your clang chroot, you may choose to invoke it as sb-create clang, or more explicitly sb-create clang sid. To create a new chroot to become your stable / gcc chroot, you may create it with sb-create gcc stable.

After you have a chroot, you can start to use the sb or sbo scripts. sbo is just a wrapper around the sb script that sets an envvar to tell the sb-prehook and sb-posthook to set up the unionfs overlay.

The unionfs overlay routine replaces the dist symbolic link (such as /var/sbuild/testing) with a folder, and mounts the real chroot (such as /var/sbuild/testing.d/testing-gcc) under the unionfs divert point (which is something like /var/sbuild/testing.overlay.d/testing-gcc). This lets us make sure the chroot is never affected by a routine build, even if it’s designed to trash a chroot (unless, of course, it uses any number of ways to get outside the chroot and trash the pristine directory — this is to prevent mistakes, not malicious scripts).

By the way, for those interested, there’s a small issue with what I’m describing it’s very subtle. Basically, there’s a few assumptions made by how schroot operates, and handles “sessions”. Since it’ll keep stuff mounted if the session is still active, it makes purging all this a real … pain.

The magic behind the scripts is in the sb-endsessions routine — which will properly end all the schroot sessions for that dist, and since we have our own high-level locking system, we shouldn’t have a situation where this kills off another build.

The code (which might have issues) looks like:

SESSIONS=`schroot --all-sessions -l | grep "^session:$DIST-"`
for session in $SESSIONS; do
    echo "W: Ending session $session"
    schroot -e -c $session
done

By backing of schroot’s mounts, we can do our mounting / removing without worrying about klobbering schroot.

Now, there’s also the case of wanting to interactively log into a chroot. For that, there’s sb-login and sb-maint, both take the usual variant / dist arguments. sb-login logs in as the current user, and sb-maint logs in as the root user.

Eventually, I’ll be adding some scripts to upgrade all the build chroots for a dist at once.

I’ve recently found myself in need of more then one sbuild chroot, each one set up slightly differently then the next. For a while, providing different behavior with unstable and sid was all well and good, but I figure, like any good computer scientist, I should solve the general problem.

As a result, I’ve written an sbuild wrapper to help maintain sanity. It uses a series of symbolic links to hide everything out of the way, and some bash hackery.

Basically, I’ve set up a /var/sbuild directory, full of a few different files. For each distribution, there is one folder, one symbolic link and up to two other files. There’s the dist.d file (such as /var/sbuild/sid.d) that contains all the “variants”, dist symbolic link (such as /var/sbuild/sid) which points to the currently active chroot, and up to two lock files.

The script will default to sid-default (that is to say, exactly, /var/sbuild/sid.d/sid-default - which is a symbolc link to the “default” chroot), but may take two arguments, in the order of variant, then dist.

Currently, I have two chroots, one for gcc, and one for clang (set up like Sylvestre used for cland.d.n) — but I may add some more with different setups.

Currently the behavior is to have the second wait kindly if you kick off two sbuilds at once, and fail out a third.

There are a bunch of bugs, and I’m not sure that this will work very well, but it’s working.

The next step is to combine this with unionfs overlays to help protect accidentally trashing the chroots.

You can find the script on my GitHub, pasted below.

#!/bin/bash

DIST="sid"
VARIANT="default"
SBUILD=/var/sbuild

if [ "x$1" != "x" ]; then
    VARIANT=$1
fi
if [ "x$2" != "x" ]; then
    DIST=$2
fi

if [ -e $SBUILD/$DIST.lock ]; then
    PID=`cat $SBUILD/$DIST.lock`
    echo "W: We *must* wait for the finish of PID $PID"
    echo "W: Blocking execution until this file is removed."
    if [ -e $SBUILD/$DIST.wait ]; then
        echo "E: Someone else is waiting. Bombing out."
        exit 2
    fi
    echo $$ > $SBUILD/$DIST.wait
    while [ -e $SBUILD/$DIST.lock ]; do
        sleep 1
    done
    echo "I: Dropping out of lock. Purging wait."
    rm $SBUILD/$DIST.wait
fi

echo "I: Aquiring lock"
echo $$ > $SBUILD/$DIST.lock

CURRENT=$(basename `ls -xl $SBUILD/$DIST | awk '{print $11}'`)

# We set up the name like:
#   dist-variant
# such as "sid-clang" is cool.
#echo "I: We're currently $CURRENT."

if [ "x$CURRENT" == "x$DIST-$VARIANT" ]; then
    echo "I: This matches the desired setup. No action."
else
    if [ -e $SBUILD/$DIST.d/$DIST-$VARIANT ]; then
        echo "I: Re-setting $DIST symlink."
        rm $SBUILD/$DIST
        ln -s $SBUILD/$DIST.d/$DIST-$VARIANT $SBUILD/$DIST
    else
        echo "E: No such dist! Looking in $SBUILD/$DIST.d/$DIST-$VARIANT."
        exit 1
    fi
fi

echo "I: "
echo "I: Dist:    $DIST"
echo "I: Variant: $VARIANT"
echo "I: "
# echo "I: Before we go farther, we're updating."
# sbuild-update -d $DIST --upgrade

sbuild -d $DIST

echo "I: Releasing lock"
rm $SBUILD/$DIST.lock

I’ve decided to start putting together packages to rebuild the bare bones install I use for most of my systems, since rebuilding everything by hand is starting to get annoying. I call it hairy candy.

Because my setup relies on some chem interesting chem hacks, none of this is fit — in any way — for the archive.

If the vim team was to catch wind of what I was doing, they’d be none too happy :)

While these packages work great for me, I can’t stress enough how badly they can break something.

Using Zack’s guide, I was able to get an archive up and running, no problem. Taking his advice, I even got signing working. Simple.

mini-dinstall rocks — a nice alternative to a PPA, for sure. At least, for one arch :)

Now, the one spin I took on Zack’s work was to add a “landing” page — a listing of what’s in the archive.

I did this using a bit of Python & Jinja2. The script follows.

#!/usr/bin/env python
# Copyright (c) Paul Tagliamonte, under the terms of the Expat license.

from jinja2 import Template
import sys

def parse_packages(fd):
    ret = []
    cur = {}
    key = None
    for line in fd.readlines():
    if line.strip() == "":
        ret.append(cur)
        cur = {}
        continue

    if line[0] == " ":
        cur[key] += "\n" + line.strip()
    else:
        key, val = line.split(":", 1)
        cur[key] = val.strip()
    return ret

pkgs = parse_packages(open(sys.argv[1], 'r'))
context = {
    "pkgs": pkgs
}

t = Template(open(sys.argv[2], 'r').read())
print t.render(**context)

Invocation looks something like:

render.py $ARCHIVE/debian/wicked/Packages $DATA/index.html

I kick this script off in the singing script, since mini-dinstall doesn’t have a exit hook.

Works great, thanks everyone!

Crazy title, I know.

Over the past $WHILE, I’ve found myself checking for the same things. Some of these are fit for inclusion into lintian it’s self — others, not so much.

Here’s a rundown on what my lintian wrapper (tragically also called “lintian” and just higher up on my $PATH. I know it’s a hack.)

Since I only use lintian on .changes files - it pre-processes for some information that is stuffed in the changes.

The first step is to get all the bugs that the package closes. It then takes these bugs, and queries the BTS to see what package they’re filed against. If the bug is filed against a different bug then the package’s source, it’ll emit a warning. It has a greylist for stuff like “WNPP”. This will fail if something’s filed against it’s binary package rather then it’s source :)

Next, the wrapper will get “other” info from the changes. It checks if the suite is in the blacklist (“UNRELEASED”, and “testing”, (stable for me as well, since I don’t usually deal with SRUs often)).

It’ll also check for the changed-by and maintainer, ensuring they’re the same. This will fail for co-maint’d packages.

If you’ve not guessed by now, this system relies on false-positives :)

Nextly, if there are .debs in the Files, it’ll go through and check the control of each. It’ll print out a stanza with the short-description inside a template, to see if the description “works”.

After that, it’ll attempt to make HTTP requests to the Homepage (and eventually URLs in the description and VCS-Browse)

If the package has “python” in the source name anywhere, or in any of it’s binary packages, it’ll trigger jwilk’s lintian4python over the source as well.

After this, it passes through to regular lintian.

Oh, and it colorizes all the output :)

Here is a (mostly lint-free) package:

As you can see, it’s very (very) verbose, but i’d rather cross stuff off a checklist then worry about something being off.

Anyway, it’s a work in progress. Not even a serious one, at that. It’s a bunch of hacks put together, so I can’t really endorse anyone else using this stuff. If you feel crazy enough to do so, all the tools are found on my dot-bin repo on GitHub

Firstly, please excuse the ugly formatting of this ultra-wide image on my blog, and I’m sure the respective planets that I’m on.

poŝto is my first “real” node project, and I’m planning to get it to a point where it makes email less painful then it otherwise is. The code’s on my github, and I’d like to thank Rick Waldron for his work in helping me, as well as Scott Wells for the name.

For those wondering, poŝto is an Esperanto word meaning (according to wiktionary):

1. a public institution (usually government-run) to deliver mail, post

2. mail distribution in general

3. one’s mail, collectively

The front-end’s name is “poŝtilo” — the suffix denotes “tool” such that poŝtilo means something like “mail tool” — fitting!

More to come…

Another day, s’more patches.

• 667343 (qterm)
• 672007 (pokerth)
• 672010 (cryptkeeper)
• 672052 (qastools)
• 672071 (pxe-kexec)
• 672005 (l2tp-ipsec-vpn)
• 672003 (dibbler)

Consider helping out!

S’more patches today to the grooven’ vibes of Home Movies.

• 667221 (kdesvn)
• 667200 (hexxagon)
• 667190 (gpsim) - not really a patch, updated existing patch
• 667230 (kredentials)
• 667233 (kvpnc)

Consider helping out :)

I’m at Transparency Camp, so if you’re around, feel free to find me and say hi!

I’ve been pretty much off the radar completely, so this is just to say I’m alive and doing well :)

For those who don’t know, Transparency Camp is an unconference on open government, government transparency and all that good stuff.

S’more days, s’more patches:

Hacked out a few over the last few days, this time, to some good eats. I’m making a habit of this tv-hack-time thing.

• patch for 667194 (gsmartcontrol)
• patch for 667177 (gdisk)
• patch for 667178 (gkrellkam)
• patch for 667180 (gkrellmitime)
• patch for 667216 (jigzo)
• patch for 667217 (jstest-gtk)

Consider helping out with the gcc-4.7 ftbfss, there’s a bunch!

Another day, more patches. This time, it was to the rockn’ vibes of Chopped.

Patch for:

• 667365 (s51dude)
• 667331 (prerex)
• 667328 (posterazor)
• 667313 (openmsx-debugger)
• 667308 (oggvideotools)

In the mean-time, I put together a script for GCC 4.7 transition bugs that are owned by the QA team with patches in the BTS.

It’s easy as cake to throw together a QA upload, so I encourage anyone out there to do some easy hacking :)