Mike Baynton’s Mediawiki Dev Blog

MediaWiki’s Mime detection system, which is used as a core component of upload verification as well as to direct media files to the appropriate media handler (for proper transformations, extraction of metadata, etc) works okay for the limited number of formats that are currently officially supported. Basically we’re talking images, some vector graphics formats, djvu files, and ogg A/V. Unfortunately, reliably identifying this small subset of media types has already required many sprinklings of code specific to a single content-type to tweak incorrect magic mime guesses,  etc.

 Try feeding the existing MimeMagic module a .asf video file or a lossless/uncompressed YUV video stream. (Hint: it no workie.) Since many more media types will be added in the near future through my project and others, I decided something definitive needed to be done. Rather than continuing to add more and more special-case code to the one (already bloated) class every time a new problem comes up, I’m creating a framework for the main module to make use of plugins as necessary that specialize in particular content-types. Without getting too technical about a plugin’s abilities, they will function as self-contained classes capable of supplying the main module with information about the content type(s) they target in general as well as in the context of a specific given file. 

A number of todo’s in the main module highlight other shortcomings that I’m shooting to address, or at least design the plugin framework to make it easier to address all of them as well. 

This wasn’t something I foresaw myself working on per-se, but it is something that really should be done before I get into the heart of my proposed project. I’ll make my case for it on wikitech-l and post my code to my branch on SVN in the coming days. (That’s right, I’m too chicken to ask first, I’m just writing it and hoping it will guilt-trip it’s way into mainline code 🙂 Actually, I think conceptually my idea is perfectly valid, so the only thing to worry about is if people don’t like my implementation…but I’m putting much care into it too. Stay tuned to hear how it goes…

I’ve been unusually inspired to be productive over the past few days…but mostly that’s turned into a discovery of how hard it is to jump into things. There’s just so much content – not only in code but also in ongoing discussions and documentation – to keep track of that I don’t know where to start or if I’m missing key considerations because I haven’t reviewed absolutely everything out there. For example,  in hopes that video contributions become popular, I want to implement this so that it can scale to multiple systems that do recoding. But how to do that most easily, considering the underlying technology/protocols in use on wikimedia’s private networks? The only real trick is preventing two recoding systems from simultaneously grabbing the same job from the queue before it is updated. One solution would be to just let this happen, and detect it by having all recode systems dump their output directly to a central file store using a predictable naming scheme. Then open new files with O_EXCL|O_CREAT, and if it fails then someone else has already claimed that job, so go get another one. But this requires a shared filesystem that supports that…currently afaik wikimedia is using NFS, but does an open call with O_EXCL|O_CREAT  work right under NFS? Heck if I know. And there’s discussion about removing the use of NFS anyway and switching to an API for private server to server file transfer in development by Tim Starling. I’m afraid if that route is taken, I won’t even be able to encode directly to the central file store (instead make locally, then copy, then delete…which takes a bit longer and is more complicated.)

Then there’s this whole concept of “media handlers” (which Tim’s also at work on) – From the looks of older posts to wikitech-l, they’re supposed to be relevant. I haven’t found formal documentation though, or any mention of them in includes/SpecialUpload.php, where uploads are handled. Makes me think they’re for the display side of things only, but wikitech-l messages make it look otherwise. I could scour lots more code to figure out what’s going on, but I’m waiting for a chance to talk with Tim about this stuff right now (hence the blog entry), which hopefully will quickly straighten a lot of things out.

 I have gotten a bit done…on the media compatibility front for example I’ve found at least one common codec in use today that I wan’t able to decode with my previously discussed MPlayer/ffmpeg or VLC combinations. The bad news is that there’s nothing I can do about it: the reason none of these can decode it is that I found there is no open-source decoder at all. The good news is that I am getting pretty convinced that MPlayer will be an easy-to-use tool that will “just work” for just about anything the open-source world can decode. I suspected this all along from personal usage, but wanted to test a bit more extensively and systematically for this project. For those interested, the undecodable codec was AMR, an audio (speech-optimized) codec developed for use on GSM and 3G cell networks. It’s relevant because some phones with cams stick it into their video streams…presumably because they have hardware that optimizes AMR encoding and that’s all they can handle when recording live video. Interestingly, if you feed it into Windows Media Player, it works just fine. Guess Micro$oft licenced it. I’d be curious to know how Facebook, which actively encourages cellphone videos to be uploaded to their video sharing service, got around this. Considering Wiki*’s different usage/audience, I don’t think I’ll continue to persue it, though.

 That’s all for now.

I’m surveying the decoding and rapid file identification capabilities of MPlayer, vlc, ffmpeg, etc to determine which are most worth utilizing and supporting. After the research and testing I’ve already conducted, I’m leaning towards using MPlayer as the decoder of choice. It has extensive codec support via its use of binary codec packages, has a nice (and usually very speedy) wrapper script that outputs parsable information suitable for verifying a file is decodable at upload time, and provides flexible encoding options (either through its own MEncoder or by outputting in YUV.)

Nevertheless, I still plan to make a framework that lends itself to adding multiple additional software suites. Here’s how I imagine it’ll work: the mediawiki administrator will be able to specify in a configuration file an ordered list of the software packages available, for example MPlayer,ffmpeg. Then, when a file is uploaded that appears to be a video, first MPlayer’s inspector will get a chance to analyze the file and declare it decodable. If it cannot, ffmpeg gets a chance to inspect the file (it looks like it would be easy to get this functionality from ffmpeg APIs if there isn’t some little proggy out there that does it already.) The first software suite that claims to be able to decode it will get the job, and this will be stored as part of the job’s info in the recoding queue. If nothing can decode it, the user gets an error and the upload is discarded.

To continue my testing now and as I develop the entire project, it would be helpful to have a diverse collection of video files to test with. The more I can see MPlayer fail myself, the better I can write scripts that detect it, so the more obscure the container format/codecs the better. I’m particularly interested in files made by digicams, cell-phones, and consumer encoding hardware. So, to my oh so many readers, if you have any of the above named devices, please send your masterpieces to me via anonymous ftp to mikeb.servehttp.com. Upload to the pub directory. Thank you for your help.

 Also, I’m excited about the increasing amount of news surfacing about efforts to get Theora files playing natively in various browsers, especially Firefox 3, and also the efforts to achieve this functionality in existing browsers by mapping the new HTML5 stuff to existing browser plugins with javascript. It’s awesome to see Wikimedia is smoothing out the playing of Theora files in-browser too, thanks gmaxwell!

This is my first…well, take your pick. First blog, but also first open-source development experience. Being as I’m such a newbie, I think WordPress’ default first post title is appropriate. I’m here thanks to Google’s recent annual efforts to get students off of their (figurative) a**es to benefit any number of open-source endeavours while gaining personal experience. I hope to do both…though I think in my case the benefit may be more mine than Mediaiki’s – I’ve got much to learn. Not that I feel over my head, I finally started going through MediaWiki code in ernest today, and after you get past all the fancy SVN concepts and buzzwords, its all the same stuff I’ve seen before. Actually, its quite refreshing. My past web-programming experience had all been either me doing everything from scratch (no fun on larger projects), or me cleaning up somebody else’s totally disastrous mess for some low-budget site or other. Seriously folks, there’s a lot of ugly code out there powering most any smaller proprietery site you visit. If you ask me it’s its own minor software crisis. I’d pretty much come to the opinion that Those That Knew What They Were Doing found better uses for their talents than making websites do things.

 I should have gotten involved with MediaWiki a long time ago. I actually considered it once years ago and was scared off by a lack of time, lack of ideas for cool contributions, and lack of confidence. It’s nice to see such quantities of comparatively well-organized code, so far with comments as necessary, in a (both human and computer) language I can read. And I think it will be much more fun to collaborate on a project for once rather than being stuck doing everything myself.

 What exactly am I doing, you may ask? Well, after spending the better part of a weekend some months ago drafting a GSoC proposal to WordPress, I decided on the (already extended) proposal due-date to skip math class and whip off another one based on a project suggested on the project ideas page. The focus of the proposal was to allow users to upload video in as many codec/container formats as possible and have them automatically and asynchronously recoded to Ogg Theora, appropriate for streaming to a Java Applet in-browser media player. “Whip off” might not be the right term – I had already done a lot of thinking and feasibility research about the idea, but had all but run out of time to actually write it up. In hindsight, I’m glad I skipped class. (And, thanks to Leslie/the GSoC program directors for extending the proposal deadline, I wouldn’t be here otherwise.)

For more details on the project, I give you my Original Proposal.