For those that don’t know, the DLNA is a consortium that aims to standardise how various media devices around your house communicate with each other (that is, your home theater, TV, laptop, phone, tablet, …). One piece of this problem is having a standard way of identifying the type of a file, and communicating this between devices. For example, say your laptop (MediaServer in DLNA parlance) is sharing the movies you’ve got with your TV (MediaPlayer), and your TV can play only upto 720p H.264-encoded video. When the MediaServer is sharing files, it needs to provide sufficient information about the file so that the MediaPlayer knows whether it can play it or not, so that it can be intelligent about what files show up in its UI.

How the DLNA specification achieves this is by using “profiles”. For each media format supported by the DLNA specification, a number of profiles are defined, that identify the audio/video codec used, the container, and (in a sense) the complexity of decoding the file. (for multimedia geeks, that translates to things like the codec profile, resolution, framerate/samplerate, bitrate, etc.)

For example, if a file is indicated to be of a DLNA profile named AAC_ISO_320, this indicates that this is an audio file encoded with the AAC codec, contained in an MP4 container (that’s “ISO”), with a bitrate of at most 320 kbps. Similarly, a file with profile AVC_MP4_MP_SD_MPEG1_L3 represents a file with H.264 (a.k.a. AVC) video coded in the H.264 Main Profile at specific resolutions upto 720×576, MP3 audio, in an MP4 container (there are more restrictions, but I don’t want to swamp you with details).

So now we have a problem statement – given a media file, we need to get the corresponding DLNA profile. It’s easiest to break this problem into 3 pieces:

1. Discovery: First we need to get all the metadata that the DLNA specification requires us to check. Using GStreamer and Edward’s gst-convenience library, getting the metadata we needed was reasonably simple. Where the metadata wasn’t available (mostly codec profiles and bitrate), I’ve tried to expose the required data from the corresponding GStreamer plugin.

2. DLNA Profiles: I won’t rant much about the DLNA specification, because that’s a whole series of blog posts in itself, but the spec is sometimes overly restrictive and doesn’t support a number of popular formats (Matroska, AVI, DivX, OGG, Theora). With this in mind, we decided that it would be nice to have a generic way to store the constraints specified by the DLNA specification and use them in our library. We chose to store the profile constraints in XML files. This allows non-programmers to tweak the profile data when their devices resort to non-standard methods to work around the limitations of the DLNA spec.

3. Matching: With 1. and 2. above in place, we just need some glue code to take the metadata from discovery and match it with the profiles loaded from disk. For the GStreamer hackers in the audience, the profile storage format we chose looks suspiciously like serialized GstCaps, so matching allows us to reuse some GStreamer code. Another advantage of this will be revealed soon.

So there you have it folks, this covers the essence of what GUPnP DLNA does. So what’s next?

1. Frankie Says Relax: Since the DLNA spec can often be too strict about what media is supported, we’ve decided to introduce a soon-to-come “relaxed mode” which should make a lot more of your media match some profile.

2. I Can Haz Trancoding: While considering how to store the DLNA profiles loaded from the XML on disk, we chose to use GstEncodingProfiles from the gst-convenience library since the restrictions defined by the DLNA spec closely resemble the kind of restrictions you’d expect to set while encoding a file (codec, bitrate, resolution, etc. again). One nice fallout of this is that (in theory), it should be easy to reuse these to transcode media that doesn’t match any profile (the encodebin plugin from gst-convenience makes this a piece of cake). That is, if GStreamer can play your media, Rygel will be able to stream it.

Apart from this, we’ll be adding support for more profiles, extending the API as more uses arise, adding more automated tests, and on and on. If you’re interested in the code, check out (sic) the repository on Gitorious.

Collabora Multimedians, looking for a canal

Nevertheless, I had a great time meeting all the cool folks at Collabora Multimedia at our company meeting. Managed to trundle out for my Rygel + DLNA lightning talk (more updates on this in a subsequent post). Things did get better subsequently, and I had an amazing week-long vacation in Germany, and now I’m back at home with my ninja skillz fully recharged!

p.s.: This also adds to my count of Gentoo boxen. :)

That said, I still don’t like the idea of running a proprietary operating system on it (as beautiful as OS X is ;)), so I continue to happily use Gentoo. The standard amd64 install works just fine with some minor hiccups (keyboard doesn’t work on the LiveCD, kernel only shows a console with vesafb).

The one thing that did bother me is BIOS-emulation. For those coming from the PC world, Macs don’t have a BIOS. They run something called EFI which is significantly more advanced (though I think the jury’s out on quirkiness issues and Linus certainly doesn’t approve of the added complexity).

Anyway, in order to support booting other OSes (=> Windows) exactly as they would on PCs, Apple has added a BIOS emulation layer. This is how Ubuntu (at least as of 9.10) boots on Macbooks. Given that both the bootloader (be it Grub2 or elilo) and the Linux kernel support booting in an EFI environment, it rubbed me the wrong way to take the easy way out and just boot them in BIOS mode. There is a reasonable technical argument for this – I see no good reason to add one more layer of software (read bugs) when there is no need at all. After a lot of pain, I did manage do make Linux boot in EFI-only mode. There is not enough (accurate, easily-findable) documentation out there, so this is hard-won knowledge. :) I’m putting this up to help others avoid this pain.

Here’s what I did (I might be missing some stuff since this was done almost a month ago). The basic boot steps look something like this:

1. EFI firmware starts on boot
2. Starts rEFIt, a program that extends the default bootloader to provide a nice bootloader menu, shell, etc.
3. Scans FAT/HFS partitions (no ext* support, despite some claims on the Internet) for bootable partitions (i.e. having a /efi/… directory with valid boot images)
4. Runs the Grub2 EFI image from a FAT partition
5. Loads the Linux kernel (and initrd/initramfs if any) from /boot
6. Kernel boots normally with whatever your root partition is

Now you could use elilo instead of Grub2, but I found this it to not work well (or at all) for me, so I just used a Grub2 (1.97.1, with some minor modifications) (just adds an “efi” USE-flag to build with --with-platform=efi). While I could make /boot a FAT partition, this would break the installkernel script (it’s run by make install in your kernel source directory), which makes symlinks for your latest/previous kernel image.

Instructions for installing the Grub2 EFI image are here. Just ignore the “bless” instructions (that’s for OS X), and put the EFI image and other stuff in something like /efi/grub (the /efi is mandatory). You can create a basic config file using grub-mkconfig and then tweak it to taste. The Correct Way™ to do this, though, is to edit the files in /etc/grub.d/.

Of course, you need to enable EFI support in the kernel, but that’s it. With this, you’re all set for the (slightly obsessive-compulsive) satisfaction of not having to enable yet another layer to support yet another proprietary interface, neither of which you have visibility or control over.

We started this event in 2008 because there was a huge gap between the open source world and academia in India. The aim was to expose enthusiastic students to what the F/OSS world has to offer, how they can participate in the community, contribute, and get that warm, happy feeling in the gut. :) And I’ve met enough people who make me believe that we have been successful in this endeavour.

One complaint we always get is that we are not newbie-friendly. From the beginning, we took an active call to channel our limited resources towards encouraging people to just start hacking and contributing (that’s the important part, remember?), which necessarily meant that if this was your first exposure to the F/OSS world, things could be a bit overwhelming.

This time, the organising team is trying something different. FOSSKriti ’10 will have loosely have 2 tracks. One track is like previous editions of the event – it’s meant for people who are comfortable with coding, possibly already F/OSS hackers. The agenda, I am given to understand, is “Come – Sit – Fork – Code – Share LuLZ.” :) And being privy to what some of the FOSSKriti veterans are planning, I am extremely excited about what this track will bring.

The second track is meant for students who are enthusiastic about F/OSS but need a little more guidance with getting started. There will be talks and workshops to help them get bootstrapped, and hopefully provide them with sufficient resources to take the ball and run. The schedule for this track is already up.

This is not complete, so keep an eye out for updates. Unfortunately, again, I will not be able to make it to the event. :( If you’re a student in India, interested in F/OSS, possibly not too far from Kanpur, this is an event you cannot miss!

There’s a nice tutorial for generating Vala bindings that serves as a good starting point. The process basically involves running a tool called vapigen, which examines your headers and libraries, and generates a GIR file from them (it’s an XML file describing your GObject-based API). It then converts this GIR file into a “VAPI” file which describes the API in a format that Vala can understand. Sounds simple, doesn’t it?

Now if only it were that simple :). The introspected file is not perfect, which means you need to manually annotate some bits to make sure the generated VAPI accurately represents the C API. These annotations are specified in a metadata file. You need to include things like “the string returned by this function must be freed by the caller” (that’s a transfer_ownership), or, object type Foo is derived from object type FooDaddy (specified using the base_class directive). Not all these directives are documented, so you might need to grok around the sources (specifically, vapigen/valagidlparser.vala) and ask on IRC (#vala on irc.gnome.org).

All said and done, the process really is quite straightforward. The work is in [my gst-convenience repository][arun-gst-conv-ks.git] right now (should be merged with the main repository soon). I really must thank all the folks on #vala who helped me with all the questions and some of the bugs that I discovered. Saved me a lot of frustration!

I’ve already got Rygel using these bindings, though that’s not been integrated yet. More updates in days to come.

As Srini had announced, Dec 5th was GNOME Day at FOSS.IN this year. We kicked the day off with Shreyas giving a developer’s introduction to GNOME 3.0. This was followed by another well-received talk by Srini on the Mobiln2 UI and Clutter.

By the end of lunch, it turned out our already packed schedule had got some new additions from the other enthusiastic GNOME folks around! The afternoon session was kicked off by Arun ‘vimzard’ Chaganty introducing what newbies need to know to dive into GNOME development. Tobias Mueller followed with a talk about GNOME Bugsquadding. Sayamindu and Dimitris then took the stage for a short L10n talk. Next up was a talk about Anjal by Puthali. Olivier then gave a hackers’ introduction to Empathy/Telepathy, Srinidhi and Bharath did a quick introduction to using the OpenSUSE Build Service.

Wait, I’m not done yet. :) The final session on GNOME Performance was a 4-hit combo with me giving a quick introduction to Sysprof, Lennart introducing mutrace, Krishnan giving a pretty wow introduction to using DTrace to profile GNOME, and Dhaval giving a short introduction to how cgroups could help make GNOME more responsive.

Phew! That was a long and awesome day, with some icing on the cake in the form of stickers and T-shirts. The last were possible thanks to the GNOME Foundation, so a huge thanks to them!

Sponsored by GNOME!

As some of you might already know, my next step, which has had me bouncing off the walls for the last month, is to join the great folks at Collabora Multimedia working on the PulseAudio sound server. I’ll be working from home here, in Bangalore (in your face, 1.5-hour commute!). It is incredibly exciting for me to be working with a talented bunch of folks and actively contributing to open source software as part of my work!

More updates as they happen. :)

The GNOME Foundation has funded a very significant part of my expense for this trip (making it possible at all), so a huge thanks to Travel Committee for giving me this opportunity. :) To summarise …

Sponsored by GNOME!

Shreyas and I reached Gran Canaria early in the morning of Day 1, but were too tired to make it to the first 2 keynotes. We woke up, had breakfast by the beach (the apartment we were in was <100 steps from the beach, and the auditorium was a 20 minute walk down the same beach — photos soon).

We did make it to Richard Stallman’s talk. It was quite generic, not surprisingly about software freedom, and nothing new to most of us. Of note were the great vitriol towards C# and the heathens who use it to create new software and a rather terrible and inappropriate attempt at humour that has been blogged about to death.

I met a huge number of people subsequently, some who’ve been at FOSS.IN before, and many whom I only knew by their online presence. The second half of the day was devoted to a number of Lightning Talks. I was pleasantly surprised to see the amount of work happening on semantic-aware projects. Good stuff.

Way to sleepy to continue making sense. More details on subsequent days, photos and so forth to come soon.

Edit: In the name of avoiding further procrastination, here are the photos.

I don’t know what it is about us — perhaps we are inherently political animals, and have some basic need to take an opposing stand. Or maybe there are just enough of us that this automatically happens. Or maybe it’s not even inherently an Indian problem. It’s bound to happen in any group, and is not, in itself, a problem. What is a problem is respect. In any reasonably large group, there are bound to be people who don’t like each other. Sometimes they might even be able bring themselves to have a shred of respect for each other (an extreme that is unfortunately seen all-too-often). What is unforgivable, though, is failure to respect the community.

When you start arguing with someone, on IRC, a mailing-list, or even in person, always remember that by making the flames personal, you are disrespecting the community. By dragging everyone around you into the mud, you are sowing the seed of trouble and strife. You end up forcing people to take a side or walk away, effectively killing the community.

My post here comes from painfully watching this happen too many times. Even more so from watching it happen now. So here’s my request, nay plea — if you’re in the midst of such a tiff, take a moment to see what it’s doing to the community you’re in (I’m assuming we’re well past the “assume good will” stage here — you did start with “assume good will”, didn’t you?). If nothing else, find a way to keep what’s personal separate.

Addendum: If you’re one of the folks who shake their head and walk away when this happens, here’s my plea to you — don’t walk away. Point out to the people responsible that their personal quarrels are not germane to the community, and ask them to take it off the community’s channels of communication. If enough people did this, maybe some sense would prevail.

Note: I’m leaving comments open for constructive discussion.

Note2: In case I come off sounding like I am innocent of all this, these are lessons that I have learned the hard way.

Some background first. LiveJournal lets you export your blog posts one month at a time. You can feed these files to the WordPress LiveJournal importer. I’ve been blogging there since December 2003, so that was definitely not an option. Some digging around eventually brought me to ljdump. This is a really nifty tool, even if you just want to back up all your posts. It dumps your data into a large set of XML files, which you can collate with the convertdump.py script for uploading to the WordPress LiveJournal importer.

There was one hiccup here — a lot of the XML files corresponding to my earlier posts (at least) had an extraneous ASCII character 4 at the end of some lines. I had to use a simple for i in <lj-user>/*xml; do sed -i -e s:$'\004':: before using convertdump.py, and things were back on track (sed ftw!). I used the script to make one big XML file with all my posts, and fed it to the LJ importer, and all my posts were in.

But my tags, unfortunately, were not. ljdump happily pulls the tags from LiveJournal, but the importer just ignores them. I found a sort-of patch to fix this, but it seems to be quite antiquated. Based on this and the WordPress importer (that’s the importer that allows WordPress to import from another WordPress blog’s exported output), I wrote my own patch to import LJ tags (against WordPress 2.7.1). Just cd into your blog directory and do a patch -p0 < wp-livejournal-import-tags.patch to use it.

That’s it — I dropped all the old posts (requires a plugin to do it all at one shot), and then imported the big XML file again, and voila!

Trivial as it was, it was great to see how easy hacking the WordPress code was. There’s more to come in days ahead. I hope it remains this easy. :D

Update: Just noticed that the imported comments are not threaded. This kind of blows, because there have been some really long threads on some posts. I guess I’ll wait till the new WordPress goes stable and do a re-import. (file under #suckage)

In the unlikely event that you’re still wondering what FOSS.IN is all about, and whether you should come, just head on over and check out the little video we’ve made. It should answer any questions that you have about what the 2008 edition of FOSS.IN is all about.

Time to head to the venue now, see you there!

p.s.: I like this one too :-) …

That’s right — the VTU has done the one thing that will ensure that no student of theirs will ever learn a Microsoft-related technology — a ton of Microsoft software is now part of the official curriculum.

Thank you, VTU!

Aside …

Reminds me of the “Basic Computer Skills” Lab in 3rd semester, where we had to create a document in Word and a presentation in PowerPoint. The external examiner expected you to remember exactly under which menu each random feature lay. It took her about 10 minutes to figure out that I was searching through the menus blindly after every question. :)

Not to mention 5th semester, where our DBMS lecturer tried to strong-arm me into learning Visual Basic for a project on databases. This one I managed to hold out on, and did my work in PHP+MySQL.

Preparation for FOSS.IN/2008 is well on way, and this year is going to be different. The Call for Participation is out. The newest thing in there is that there aren’t going to be nearly as many talks as before. You’ll see the term FOSS WorkOuts rather prominently displayed, and this is where the action is going to be. We’re going to be seeing a lot more doing than in years gone by. Head on over to the CfP to learn more.

Atul’s post on the new format has caused some furore in the community, in addition to some pockets of encouragement (links abound and the topic is hackneyed, so no linky). All I have to add is this — a lot of people who are working on distros and doing packaging seem to be gravely offended. Well, I’m a packager too (erm, did I mention that I am now a Gentoo developer?), and there is no reason to take offense. What we’re trying to say is that we can be achieving more at the event to increase both the number of contributors as well as the depth of contribution, and the latter especially is the focus. I can expound on about this, but there’s been enough talk.

The cool folks over at IndLinux have already started plotting, and we’ve been trying to get some traction on some GNOME performance work. Hope we can get some more folks to run with it. I, for one, am looking eagerly forward to the proposals we get this year.

The intimation about doing this was sent at 2357 hours today (yesterday, to be precise) to all. The notification basically stated that because of “undesirable activities”, Internet will, with immediate effect, be disabled from 0000 to 0600 every day. And that’s it — poof. The hostel network is disconnected from the rest of the Institute, thus making sure that nobody can access the Internet (or even the Institute’s own computing facilities). To compensate, the Computer Center (with a capacity of <200 computers) is to be kept open 24×7.

Of course, this was unacceptable, so a bunch of us decided that something needs to be done. There are 2 issues — the decision, and how it was implemented. While the decision itself needs discussion (more about this later), the implementation is of immediate concern. People were not prepared, and work on several people’s theses were affected. Plus, this has been done just a little after the end-semester exams, when most students are not on campus. This sort of fascism usually rears its head under precisely these circumstances. We decided that what needed to be addressed right now is the implementation — the Internet has to be made available this night.

A couple of our student representatives spoke to the Dean of Student Affairs (the DoSA — the official channel between the students and the administration). The DoSA basically said that they, the various Deans and the Director (and Deputy Director?) have made the decision at nothing would be done about it. More precisely, the Director, as the highest power in the Institute has taken the decision and that’s that. Further discussion may be taken up with him.

About 60-70 of us went to the Director’s house at about 2:30 (the entire process was peaceful — there was not shouting or slogans). We met with the security, who called the Head of the Computer Center (CC) and the DoSA to the place after some attempted dodging.

The CC Head turned up first and started asking what our problem was. He offered such resources as a vehicle to transfer us from hostel to CC as well as as many pen-drives as we require to transfer data from our machines to the CC machines. The DoSA just said that we’ve given you 2 years to think about whether this should be implementing it, and now we will be implementing it, so there.

Our student representatives (who did a pretty good job), after some dialogue, got the connection reinstated for tonight. They will be further taking up the issue later today.

The decision itself is extremely foolish, of course. Moreover, the dictatorial way in which this is being done is just as shocking. Let’s see how things pan out in time. Perhaps sense and sanity will prevail.

In addition, pkgcore 0.4.4 has my patch to support HTTP proxies for rsync. This was a fun patch to write, small as it is. The code is beautiful, and Brian Harring (ferringb) and Patrick Lauer (bonsaikitten) walked me through a lot of it. Good stuff!

I’ve also been working on splitting the gnome-python* ebuilds to make the dependency trees for packages that use these bindings a lot saner. This has been longer and more painstaking that intended. It wouldn’t even have happened if Jim Ramsay (lack) hadn’t made an excellent start with the gnome-python-desktop split, since all subsequent work was based on that. Hope this is useful to someone, though. :-)

As I said, a good week.