we noticed last week (well, actually a bit longer than that) that OpenSim would not really care to run on PowerPC platforms. the symptoms were that (a) it would die with an array index exception on restart, and (b) the SL client would crash horribly when connecting to the region before the restart with most of the development going on on intel boxes, suspecting endian issues was not really rocket-science.

short detour: when we talk of “endian issues” we refer to the way a CPU organizes data internally. to quote wikipedia:

endianness is the byte (and sometimes bit) ordering used to represent some kind of data. Typical cases are the order in which integer values are stored as bytes in computer memory (relative to a given memory addressing scheme) and the transmission order over a network or other medium. When specifically talking about bytes, endianness is also referred to simply as byte order.

the terms big endian and little endian are much older than computers, much older. they originate with the inhabitants of jonathan swift’s Lilliput and Blefuscu and refer to the way they would crack an egg open: big endians started at the big end.

for reasons shrouded in the fogs of history, intel x86 CPUs are little endian, motorola CPU generally are big endian as are PowerPC CPUs.[^1]

anyhow, to return to our story, we were suspecting endianess issues with OpenSim — and the underlying libsecondlife. following up on clues provided by Neas Bade i started looking at openjpeg-libsl (the native code library responsible for doing most of the JPEG2000 coding). that turned out to be a dead end more or less.

turning my attention to the terrain handling, it transpired that the terrain when generated was looking fine (you can save a generated terrain via OpenSim’s region console — which i did and then examined it using [the gimp][]). loading it back in from the data stores (OpenSim supports a number of data storage adapters) however, returned a corrupted terrain image. that lead to fix #1: serialization of the terrain in the SQLite and MySQL data storage adapters was fine, but reification was borked.[^2]

unfortunately, while OpenSim would now restart fine, we still could not connect as the client would barf violently on the terrain data OpenSim was handing out looking at the code in OpenSim that supplied the client with the ill-digested terrain dope it transpired that this time it was libsecondlife that was cracking the eggs open from the little end only. actually, it was doing so in two places: LLUUIDs were not properly serialized and packed bits were completely oblivious of the fact that one could start cracking the eggs open at the big end. bug fixes #2 and #3 (this time to libsecondlife).[^3][^4]

so, what steps do you need to take to get OpenSim running on PPC? here are the instructions:

1. check out opensim itself:
   • svn co http://opensimulator.org/svn/opensim/trunk opensim
   • build according to instructions
     • ./runprebuild.sh; nant
2. check out opensim-libs:
   • svn co http://opensimulator.org/svn/opensim-libs/libsl1550 opensim-libs
   • build: cd opensim-libs; nant
   • you should end up with a sparkling new libsecondlife.dll in bin
   • copy that one over into the opensim/bin directory.
3. enter opensim-libs/openjpeg-libsl and do
   • a make, and then
   • copy the newly built libopenjpeg-libsl-2.1.2.0.so into opensim/bin as well
4. almost there! enter opensim/bin and edit libsecondlife.dll.config and remove the cpu="x86" tag in the last dllmap line, so that it looks like this:

start OpenSim.exe and enjoy.

SyntaxHighlighter is quite a nice wordpress plugin that provides syntax highlighting for a range of programming languages (among them my favorites python & C#). the problem is, in its original unadulterated form it doesn’t play with my favorite markup plugin php markdown extra at all

after quite a bit of hacking, it transpires that one can make them play nicely with one another. the idea that i pursued was to change the [source:XXX] tag to < source lang=XXX> and have php markdown treat it the same as the < pre> tag.¹

getting markdown extra to recognize the < source> tag is fairly easy², i needed to change # Tags treated as block tags only if the opening tag is alone on it’s line: var $context_block_tags = ’script|noscript|math|ins|del’; to read

next, in the syntax plugin, all occurrences of [source: are replaced by < source lang=.

i’ve created ZIP files of the modified SyntaxHighlighter and Markdown Extra plugins, share and enjoy.

while looking at the 24C3 hack list of the currently ongoing 24C3 hack conf, i found this howto detailing the procedure to create fake fingerprints — all you need are:

• glass with fingerprint of the person you want to impersonate
• screw-top of a bottle (like the one from a coke bottle)
• super-glue
• wood glue (PVA)
• skin friendly glue (theatrical glue)
• digital camera
• PC
• laser printer
• foil

the process itself is rather easy — and, together with german TV station WDR, they demonstrated that you can use that method to fool the fingerprint recognition system used by the European super-market chain EDEKA…

EDEKA seemed non-fazed:

Edeka Südwest teilt uns auf Anfrage schriftlich mit, man sehe „keinen Handlungsbedarf“. Die bestehenden Sicherheitsvorkehrungen seien, „wie unsere Erfahrungen gezeigt haben, vollkommen ausreichend“.[^1]

…and they went on to state that the system they used was being used by the US government and other governments worldwide.

wow. good security relies either on something only i know, or something only i have, or something only i am. the emphasis is on only, your fingerprints? they are all over the place…and that place…and that place over there as well, yep.

the scary thing is, fingerprints are being used (as secondary biometrics) for the new biometric EU passports!

one of the projects i’m currently working on is the VU*gate system that allows us to write scripts or applications to control an avatar in virtual worlds. VU*gate consists of a gateway daemon “talking” on one side the virtual world network protocol towards the virtual world grid (e.g., for linden lab’s secondlife grid we are using the excellent libsecondlife code) and on the other side a network protocol for which we can easily provide specific language bindings — thus, we can use Java to program virtual world applications (or python or JavaScript or Perl or …).

for the first version of VU*gate (which we then called SLPuppeteer, but that name is already used it transpired) we used a line based protocol modeled on SMTP, enhanced with asynchronous extension (virtual worlds, just like real worlds, are inherent asynchronous). that worked (and still does work) quite well for a lot of things: chatting, instant-messaging, moving about are all easily done with a one line command — and also easily parsed. where this starts to become painful is when you have to deal with more complex data (avatar login including not just fullname and password but also grid URI and location, for example, or group membership data). mind you, VUgate mark 1 is currently in full production use and a perl script is updating the Eightbar SL group[^2] via VUgate, using data supplied by our corporate directory Fringe.

clearly, something more structured was what i wanted. preferably with some easy to use libraries taking care of the nitty-gritty details of data serialization and reification. colleagues had suggested switching to REST, but that doesn’t really support asynchronous event reporting (unless you revert to polling, yuck).

the solution i came up with was XMPP, which is the underlying transport protocol for all Jabber based instant messaging services (e.g., googletalk, which is probably the best known of these). after trying the erlang based ejabberd (if you don’t like perl because it looks like an army of flies had reverse bowel movements, you certainly are going to like erlang, yikes) i eventually settled on openfire which’s main merit is that it’s got a rather nice admin web interface and can route messages with custom XML namespaces!

so far, so good — what has all this got to do with perl? well, the current language binding of choice for VUgate is perl (object oriented perl, even), so i wanted to keep the existing perl-based VUgate applications, which meant i needed a perl XMPP stack. turns out there are a couple, the most promising was Net::XMPP. while Net::XMPP played nicely with ejabberd, it definitely did not like Openfire and never managed to even login and, worse, that error (SAX parser flying out of the curve) had been reported and the “fixes” suggested earlier didn’t work anymore. so, chalk one up against good old perl.

next: for asynchronous stuff like VU*gate it would be really nice to use threading in the applications (along with stuff like mutexes, locks, signalling, and so forth). recent perl version claim to support threading…well,

It’s very important to remember when dealing with Perl threads that Perl Threads Are Not X Threads, for all values of X. [perlthrtut]

which is then revealed in the “shared and unshared data” section:

The biggest difference between Perl ithreads and the old 5.005 style threading, or for that matter, to most other threading systems out there, is that by default, no data is shared. [ibid]

hmmmmmm…turns out that you can share some data between threads, but there are some severe restrictions: only simple variables or references to those. no sharing of file descriptors it seems (so, no reader–writer thread separation :-(.

indeed, not what you’d expect.

time to start looking for new stuff…

…time to introduce ourselves to python. it’s has proper threading, and — ta da! — has a proper and non-barfing XMPP stack!