xyzzy xyzzy...

tunafish, tune a mono

DrScofield — Mon, 30 Jun 2008 18:13:42 +0000

daTwitch and nebadon, both infamous opensim developers did an interesting experiment over the weekend: they patched mono and rebuild it with the --with-large-heap configuration option and then load tested OpenSim — result: you get way more bang for your buck!

so, i’ve updated the mono-build script to include that patch and also use the large heap configuration option. admittedly, they did use mono from its SVN trunk, so i’m not sure how much the script will help, but, have a go and let me know how it works out.

splitting opensim’s chat module

DrScofield — Tue, 27 May 2008 09:34:15 +0000

similar to other virtual worlds opensim supports in-world chat (and instant messaging) so that two avatars can “talk” to one another. the code that implements that functionality is mostly contained in ChatModule, a region module.

OpenSim also supports an IRC bridge: you can tie your OpenSim grid to an IRC channel hosted by an IRC network and everything that’s being said (chatted) in any of the OpenSim regions is reported by an OpenSim IRC bot on the IRC channel — and vice versa.

until recently¹ chat traffic and IRC bridging was done by one single region module, ChatModule. as we are thinking about adding an XMPP bridge that was sub-optimal and we needed to split ChatModule…

…to do that i’ve basically removed the ISimChat interface through which ChatModule was being called and turned the interface into a set of events in EventManager: OpenSim basically has something that we could virtual information bus that allows region modules to register handlers for certain event types. for each event type we usually have a trigger method in EventManager which a system component can invoke and pass data into to have it distributed via the “virtual info bus” to all subscribed region modules. for the chat functionality we now have two new events:

ChatFromWorldEvent — which is used to distribute chat from in-world objects
ChatBroadcastEvent — which is used to send a broadcast chat to all logged in users

plus, of course, the corresponding TriggerOnChatFromWorld and TriggerOnChatBroadcast methods as well.

the old ChatModule now exists as ChatModule and IRCBridgeModule. the new ChatModule now only takes care of chat traffic, nothing else. IRCBridgeModule is subscribed to ChatFromWorldEvent to receive all chat traffic from in-world and is also subscribed to each user’s client object to receive all chat traffic from clients (ChatModule does this also).

using the new ChatBroadcastEvent it was now possible to implement the LSL function llOwnerSay() as normal chat instead of having instant messages pop up.

also, /me is now properly translated in both directions. typing /me sneezes in the IRC window² causes IRC to say:

…which is then correctly rendered in OpenSim:

likewise, typing /me laughs as Dr Scofield in the secondlife™ client shows up as follows in the secondlife client:

and is correctly reported in the IRC channel:

another new thing: as you can see

IRCBridgeModule now reports more (and correct) presence information as well.

roughly before subversion release 4850 or thereabouts. ↩
i’m using pidgin for all my IRC, google talk, and sametime chats. ↩

we have buttons!

DrScofield — Sun, 18 May 2008 18:44:19 +0000

since time immemorial (well, almost) i own a private epson 1260 scanner. a couple of weeks ago i — foolishly, as it turns out now — offered to my in-laws to copy all lake district articles out of our country walking collections. foolishly, because i realized that country walking must have their offices on one of those lonesome, remote fells: there are lots and lots and lots and lots of lake district articles and routes in country walking.

i started off with the gscan2pdf tool — really slow to start up, at that speed summer would be over before i had everything scanned in and converted to PDF. not good.

next try: kooka, KDE 3.5’s own scan tool — faster, but non-intuitive, you apparently have to rename and save each individual file. winter time before i’d be done.

i noticed, once again, that the epson 1260 does have a scan button (and a print and a mail and a web button) — if only that were working under linux. last time i checked for a tool (gazillion years ago), there was nothing, zilch, nix available to utilize that button. googling for plustek and scan button (the scan chipset inside the epson 1260 is a plustek chipset) this time did turn up something: scanbuttond! and even better: it’s available as a ubuntu package in hardy heron:

% apt-get install scanbuttond

installed the beast. following the instructions on the gentoo wiki gave me a working one-button-scan process! i modified the scan script slightly to deposit the fresh scan in my $HOME/tmp/scans/ directory.

converting a bunch of JPG images into a single PDF is also quite easy: you imagemagick’s convert command:¹

% convert *.jpeg allinone.pdf

voila!

if you haven’t installed that yet, it’s a simple
```
% apt-get install imagemagick
```
away. ↩

automatically downloading LindenLab™ account statements

DrScofield — Thu, 15 May 2008 13:14:22 +0000

our lab “owns” an island on the LindenLab grid and we’ve created a “lab” avatar who is the estate owner. as the island is paid for by the lab, we need to have monthly account statements on file for the bean counters. so far, i had to remember to log in to the lab avatar’s account page, copy the account statement and mail it our friendly lady in accounting taking care of these things (yes, we are a small lab)…

…the only problem with that is that i tend to forget these things and our friendly accounting lady has to be rather patient and usually ends up sending reminder notes.

so, after another one of these notes and as it was becoming rather embarrassing that i kept forgetting, i decided to tackle it once and for all and write a python script to do this automatically. the very excellent mechanize python package turned out to be an invaluable tool.

here’s the script in all its glory:

#!/usr/bin/python
# -*- encoding: utf-8 -*-

import re
import smtplib
import time

from email.mime.text import MIMEText
from mechanize import Browser
from optparse import OptionParser
from ConfigParser import ConfigParser

class Ooops(Exception):
    def __init__(self, msg):
        self.value = msg

def __str__(self):
        return msg

if __name__ == ‘__main__’:

try:
        parser = OptionParser()
        parser.add_option(’-c’, ‘–config’, dest = ‘config’, 
                          help = ‘path to configuration file’, metavar = ‘CONFIG-FILE’)
        (options, args) = parser.parse_args()

if not options.config:
            parser.error(’–config option mandatory’)

config = ConfigParser()
        config.readfp(open(options.config))

slURL = config.get(’lindenlab’, ‘url’)
        slAvatarFirstname = config.get(’lindenlab’, ‘firstname’)
        slAvatarLastname = config.get(’lindenlab’, ‘lastname’)
        slAvatarPassword = config.get(’lindenlab’, ‘password’)

browser = Browser()

# open http://www.secondlife.com/ and select the “resident login” link
        browser.open(slURL)
        browser.follow_link(text_regex = r’Resident\s*Login’, nr = 0)

# fill out the login form
        browser.select_form(nr = 0)
        browser['form[username]‘] = slAvatarFirstname
        browser['form[lastname]‘] = slAvatarLastname
        browser['form[password]‘] = slAvatarPassword
        browser.submit()

# page title should read: “Second Life | Your Account: WayLate Binder”
        rePageTitle = re.compile(r’Second\s+Life\s+\|\s+Your Account:’)
        match = rePageTitle.match(browser.title())
        if not match: raise Ooops(’not on account page’)

# select account history link and retrieve the statement
        statement_page = browser.follow_link(text_regex = r’Account\s+History’, nr = 0)
        statement = statement_page.get_data()

reURL = re.compile(r’^(?P<host>https?://[^/]+)’)
        match = reURL.match(browser.geturl())
        if not match: raise Ooops(’no host URL’)
        url = match.groups(’host’)

reHref = re.compile(r’href\s*=\s*”/’)
        statement = reHref.sub(’href=”%s/’ % url, statement)

browser.close()

msgSubject = config.get(’mail’, ’subject’)
        msgSubject = time.strftime(msgSubject)
        
        msgTo = config.get(’mail’, ‘to’)
        msgFrom = config.get(’mail’, ‘from’)
        msgCc = config.get(’mail’, ‘cc’)

msgEnvTo = msgTo.split(’,')
        if msgCc: 
            msgEnvTo += msgCc.split(’,')

msg = MIMEText(statement, ‘html’, ‘UTF-8′)
        msg['Subject'] = msgSubject
        msg['To'] = msgTo
        msg['Cc'] = msgCc
        msg['From'] = msgFrom

smtp = smtplib.SMTP(’localhost’)
        smtp.sendmail(msgFrom, msgEnvTo, msg.as_string())
        smtp.close()
    
    except Ooops, e:
        print ‘oops: failed to retrieve account statement: %s’ % e.value

it reads the avatar name, password and so forth from a configuration file:

[lindenlab]
url = http://www.secondlife.com
firstname = Foo
lastname = Bar
password = secret

[mail]
subject = %Y-%m LindenLab monthly account statement for Foo Bar
to = accounting@foo.bar.com
cc = me@foo.bar.com
from = me@foo.bar.com

adapt the config file to your taste and invoke as follows:

sl-account-statement --config config.cfg

voila!

RESTful regions

DrScofield — Wed, 14 May 2008 11:49:01 +0000

we are currently trying to figure out what the best approach for a REST “API” for regions is and would like to solicit comments

currently the idea is to have a scheme as follows:

http://opensim.foobar.org:9000/admin/regions —
- GET returns an array of (UUID, name, x location, y location, region’s REST URL)
- POST would create a region
http://opensim.foobar.org:9000/admin/regions/4b787c46-1e3c-40ae-9494-2c924428f8e5/
- GET would return detailed information about region
- DELETE would delete region
- PUT would update region information
http://opensim.foobar.org:9000/admin/regions/4b787c46-1e3c-40ae-9494-2c924428f8e5/name
- GET would return name of region
- PUT would update name of region
- (similar for other region attributes if it makes sense)

current planning is to have this as an ApplicationPlugin level set of RestPlugins. an alternative would be to use region modules plus an ApplicationPlugin (for GET/POST on /admin/regions).

this applies to region (meta) data. the next question would be how to structure this for avatar information and also for stuff like inventory: something along the lines below?

http://opensim.foobar.org/admin/avatars
- GET returns list of known avatars?
- POST creates account?
http://opensim.foobar.org/admin/avatars/430f1da7-0e35-4c0f-985d-15046c077967/
- GET returns detailed information about avatar?
- PUT updates?
- DELETE deletes?
http://opensim.foobar.org/admin/avatars/430f1da7-0e35-4c0f-985d-15046c077967/inventory/
- GET returns inventory listing?
- POST adds items to inventory?
- DELETE deletes inventory items?

which serialization format for OpenSim REST services?

DrScofield — Tue, 13 May 2008 22:04:19 +0000

we are currently looking at adding REST support to the RemoteAdmin plugin (other stuff to follow) — the idea being that we can get information about (as well as change state of) a running OpenSim instance not only via XmlRpc but also via REST.

looking at the way we currently deal with serialization we either

use LLSD where mandated by secondlife (e.g., CAPS)
use .NET’s System.Xml.XmlSerializer
use XmlRpc serialization (which goes back to System.Xml)

LLSD is being used by linden lab’s and is proposed by LL/SLAWG as the serialization protocol of choice (IIUC). in my opinion it suffers from being a bit verbose and not really easy to parse.

XmlSerializer is “built in”. on the other hand it’s wordy and more complex to parse; also, it doesn’t distinguish between the number 4711 and the string “4711″ — you have to know that a certain tag contains a string (or a number).

another alternative that has been suggested is JSON. JSON is lean, and easy to use from AJAX apps as well as python scripts (and the likes). it requires an additional DLL. it does however distinguish between strings and numbers. using a library like jsonexserializer — BSD license — makes it easy to use from C#.

personally, i’ve started to like JSON because it’s lean and easy to use — but i’d like to hear from you guys what you think about this. so, give it a vote!

Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.

updated mono build script for hardy heron and mono 1.9.1

DrScofield — Thu, 08 May 2008 12:23:55 +0000

mucking around with trying to get mono to run on hardy heron (and my custom compiled kernel¹) i ended up with a mutilated mono installation. just following my previous recipe didn’t work out too well for the mono document browser…duh…:-(

so, i took sean’s shell script and gave that a spin. that worked mostly — i had to adapt it to mono 1.9.1 for the mono-doc stuff and also install additional mono packages. here’s the revised edition (click here for a downloadable version):

#!/bin/sh

BUILDDIR=/usr/src/local/mono/1.9.1

# This is needed to pick up our built mono for commands
export PATH=/usr/local/bin:$PATH 

apt-get install -y build-essential bison gawk
apt-get install -y libglib2.0-dev
apt-get install -y libpng12-dev libx11-dev libfontconfig1-dev
apt-get install -y libfreetype6-dev libjpeg62-dev libtiff4-dev
apt-get install -y libungif4-dev libexif-dev libcairo2-dev
apt-get install -y libpango1.0-dev libgtk2.0-dev libglade2-dev
apt-get install -y libgnome2-dev libgnomecanvas2-dev libgnomeui-dev
apt-get install -y libgnomeprint2.2-dev libgnomeprintui2.2-dev
apt-get install -y libpanel-applet2-dev libgtksourceview-dev
apt-get install -y libgtkhtml3.14-dev
apt-get install -y intltool

apt-get remove 'mono-*' libgdiplus

mkdir -p $BUILDDIR
cd $BUILDDIR

wget -c http://go-mono.com/sources/libgdiplus/libgdiplus-1.9.tar.bz2
tar xvf libgdiplus-1.9.tar.bz2
cd libgdiplus-1.9
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://go-mono.com/sources/mono/mono-1.9.1.tar.bz2
tar xvf mono-1.9.1.tar.bz2
cd mono-1.9.1
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://switch.dl.sourceforge.net/sourceforge/nant/nant-0.86-beta1-src.tar.gz
tar xvf nant-0.86-beta1-src.tar.gz
cd nant-0.86-beta1
make 
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/gtk-sharp212/gtk-sharp-2.12.0.tar.bz2
tar xvf gtk-sharp-2.12.0.tar.bz2
cd gtk-sharp-2.12.0
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/gecko-sharp2/gecko-sharp-2.0-0.13.tar.bz2
tar xvf gecko-sharp-2.0-0.13.tar.bz2
cd gecko-sharp-2.0-0.13
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://go-mono.com/sources/gnome-sharp2/gnome-sharp-2.16.1.tar.gz
tar xvf gnome-sharp-2.16.1.tar.gz
cd gnome-sharp-2.16.1
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://go-mono.com/sources/gtksourceview-sharp2/gtksourceview-sharp-2.0-0.12.tar.bz2
tar xvf gtksourceview-sharp-2.0-0.12.tar.bz2
cd gtksourceview-sharp-2.0-0.12
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/gnome-desktop-sharp2/gnome-desktop-sharp-2.20.1.tar.bz2
tar xvf gnome-desktop-sharp-2.20.1.tar.bz2
cd gnome-desktop-sharp-2.20.1
./configure --prefix=/usr/local
make
make install
cd ..

cd mono-1.9.1
wget  -c http://go-mono.com/sources/monodoc/monodoc-1.9.zip
unzip monodoc-1.9.zip
cd monodoc-1.9
./configure --prefix=/usr/local
make
make install
cd ../..

wget -c http://go-mono.com/sources/mono-tools/mono-tools-1.9.tar.bz2
tar xvf mono-tools-1.9.tar.bz2
cd mono-tools-1.9
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/mono-debugger/mono-debugger-0.60.tar.bz2
tar xvf mono-debugger-0.60.tar.bz2
cd mono-debugger-0.60
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/heap-buddy/heap-buddy-0.2.tar.gz
tar xvf heap-buddy-0.2.tar.gz
cd heap-buddy-0.2
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/mono-addins/mono-addins-0.3.1.tar.bz2
tar xvf mono-addins-0.3.1.tar.bz2
cd mono-addins-0.3.1
./configure --prefix=/usr/local
make
make install
cd ..

wget -c http://ftp.novell.com/pub/mono/sources/monodevelop/monodevelop-1.0.tar.bz2
tar xvf monodevelop-1.0.tar.bz2
cd monodevelop-1.0
./configure --prefix=/usr/local
make
make install
cd ..

note: you might want to adapt the setting of BUILDDIR!

yes, i’m not using the ubuntu supplied kernel as
1. the suspend–resume stuff is just not working on my thinkpad X60(s), and
2. the iwl3945 WLAN driver just doesn’t grok our companies IEEE80211X setup
↩

fixing absurdly high wordpress CPU loads…

DrScofield — Fri, 02 May 2008 17:32:36 +0000

ever since i switched to wordpress 2.5 (and 2.5.1) i was experiencing absurdly high CPU loads whenever i posted a new blog entry or changed a page — culminating in loads of 124+ yesterday, effectively bringing our server to a standstill! not good. not good at all.

a ps ax | grep apache2 | wc -l revealed a rocking 104 apache2 processes running! yikes! all with about 21MB of memory usage (at least), no wonder the poor box was thrashing for dear life…

looking at what ubuntu had installed i found that (a) we had apache-prefork running and (b) switching to apache-worker¹ would remove mod_php5 — which we need to have all the PHP scripts running (including this blog). hmmm.

some googling later it transpired that switching to thread-based apache was the right thing to do and also, that the fast CGI module mod_fcgid would be the solution to the disappearing php5 apache module.

here’s what i did: first install threaded apache

% apt-get install apache2-mpm-worker

then, install the fcgid module

% apt-get install libapache2-mod-fcgid

then, modify /etc/apache2/mods-available/fcgid.conf to read as follows:


    AddHandler    fcgid-script .php
    FCGIWrapper   /usr/local/sbin/php-fcgi .php

    SocketPath    /var/lib/apache2/fcgid/sock
    SharememPath  /var/lib/apache2/fcgid/shm
    IPCConnectTimeout 60

create /usr/local/sbin/php-fcgi .php with the following content:

#!/bin/sh
PHPRC="/etc/php5/apache2"
export PHPRC
PHP_FCGI_CHILDREN=1
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /usr/bin/php5-cgi

now, all that’s left to do, is ensure that for each virtual host you have ExecCGI enabled:


        Options ExecCGI Indexes FollowSymLinks MultiViews
        AllowOverride All

        Order allow,deny
        Allow from all

restart your apache and we are back in business — on my box the load is now back in sane regions: instead of 124+ we are back at 0.52 for posting a blog entry (at most)!

the thread based apache. ↩

installing libgpod from SVN on ubuntu hardy heron

DrScofield — Fri, 02 May 2008 12:22:03 +0000

having bitten the bullet and upgraded from feisty fawn to hardy heron i had to reinstall my amorok from svn to get proper support for artwork again…which requires an installation of libgpod from svn…which on hardy heron requires gdk-pixbuf…

…just installing libgdk-pixbuf2 and libgdk-pixbuf-dev didn’t satisfy libgpod’s configure script though. what did the trick was

% apt-get install libgtk2.0-dev

compiling amarok now…

going from gutsy gibbon to hardy heron…or: muawhaha, mono is hosed…

DrScofield — Wed, 30 Apr 2008 21:03:38 +0000

just thought i’d let you participate in today’s adventure: triggered by ansgar commenting on ubuntu offering him to upgrade to ubuntu 8.04 “hardy heron” i though, “hmm, good idea, let’s do a quick update”…

…well, a couple of hours later i seemed to be back in business, hardy heron was installed on “my” x60 and i was just about do another OpenSim build — when i noticed that nant was seg faulting! now, wait a moment, it clearly was not supposed to do that. ok, can happen, what with new libs and all that. so, let’s walk on over to /usr/local/src/mono/nant-0.86b and do a re-install. should be easy, peasy…

…except it wasn’t: mono’s compiler mcs kept seg faulting as well! hmmm.

ok. let’s reinstall mono. following my own script i started the re-install — which crashed as well after about 5min of mucking around.

panic stations!

googling around it transpired that this would happen when running hardy heron with a custom compiled kernel of pre-2.6.24 vintage — which i was

i’m happy to report that with 2.6.25 mono is running again