My new music library is all nicely ripped Apple Lossless files in .m4a containers. These contain metadata, but it’s not id3v2 and the universe of mp3 tools mostly don’t work. ffmpeg’s ffprobe sort of works but doesn’t show all tags.

What does work for displaying m4a metadta is mp4v2, installable via Homebrew. Docs here. It seems pretty stable but still gets some updates.

(imac) = mp4file --list test.m4a
BRAND  COMPAT              SIZING  FILE
----------------------------------------------------------------------
M4A    M4A,isom,mp42       32-bit  test.m4a

(imac) = mp4info test.m4a
mp4info version 1.9.1
test.m4a:
Track	Type	Info
1	audio	alac, 260.806 secs, 562 kbps, 44100 Hz
 Name: Pastourelle (E passo de dossai: Ah! viens pres de moi), folksong for voice & orchestra (Chants d'Auvergne, Series 2, No. 1)
 Artist: Dawn Upshaw
 Composer: Joseph Canteloube
 Release Date: 1994
 Album: Canteloube: Songs of the Auvergne
 Track: 1 of 15
 Disk: 1 of 1
 Genre: Vocal Music
 BPM: 0
 Part of Compilation: no
 Cover Art pieces: 1
 Album Artist: Kent Nagano
 Media Type: Movie
 TV Episode: 0
 TV Season: 0

(imac) = mp4track --list test.m4a
track[0] id=1
  type           = audio
  enabled        = true
  inMovie        = true
  inPreview      = true
  layer          = 0
  alternateGroup = 0
  volume         = 1.0000
  width          = 0.00000000
  height         = 0.00000000
  language       = UNDEFINED(0)
  handlerName    =
  userDataName   = <absent>

(imac) = mp4art --list test.m4a
IDX     BYTES  CRC32     TYPE       FILE
----------------------------------------------------------------------
  0     11255  299646cd  jpeg       test.m4a

(imac) = mp4tags
mp4tags: You must specify at least one MP4 file.
usage mp4tags OPTION... FILE...
Adds or modifies iTunes-compatible tags on MP4 files.

      -help            Display this help text and exit
      -version         Display version information and exit
  -A, -album       STR  Set the album title
  -a, -artist      STR  Set the artist information
  -b, -tempo       NUM  Set the tempo (beats per minute)
  -c, -comment     STR  Set a general comment
  -C, -copyright   STR  Set the copyright information
  -d, -disk        NUM  Set the disk number
  -D, -disks       NUM  Set the number of disks
  -e, -encodedby   STR  Set the name of the person or company who encoded the file
  -E, -tool        STR  Set the software used for encoding
  -g, -genre       STR  Set the genre name
  -G, -grouping    STR  Set the grouping name
  -H, -hdvideo     NUM  Set the HD flag (1\0)
  -i, -type        STR  Set the Media Type(tvshow, movie, music, ...)
  -I, -cnid        NUM  Set the cnID
  -l, -longdesc    NUM  Set the short description
  -L, -lyrics      NUM  Set the lyrics
  -m, -description STR  Set the short description
  -M, -episode     NUM  Set the episode number
  -n, -season      NUM  Set the season number
  -N, -network     STR  Set the TV network
  -o, -episodeid   STR  Set the TV episode ID
  -P, -picture     PTH  Set the picture as a .png
  -s, -song        STR  Set the song title
  -S  -show        STR  Set the TV show
  -t, -track       NUM  Set the track number
  -T, -tracks      NUM  Set the number of tracks
  -w, -writer      STR  Set the composer information
  -y, -year        NUM  Set the release date
  -R, -albumartist STR  Set the album artist
  -r, -remove      STR  Remove tags by code (e.g. "-r cs"
                        removes the comment and song tags)

Looking at traffic stats and Google’s mod_pagespeed I thought of a way to optimize my blog a bit: replace some of the small images with inline data: URLs. According to Wikipedia this works in all modern web browsers. IE7 is out of luck, but it (kind of) works in IE8.

For example, this Creative Commons logo is a 1082 byte GIF. Small, but there’s a lot of overhead in doing an HTTP request just for that image used in one place on my page. Fetching with wget takes 11 packets and 3 or 4 round trips. (A little better in a real browser with HTTP keepalives).

As a base64 data: URL it’s only 1467 bytes. That adds 1-2 TCP packets to the request and 0-1 round trips. The downside of inlining is that the image is now served with every single HTML page. No caching and bots will have to pay for it too. In the end I think it’s more work for my server, less work for the client, which is a good tradeoff for small images. (It may be less work for the server in general; fewer client requests per page.)

I arbitrarily decided 3k was the cutoff for converting to data URLs. That results in 5 images:

• http://www.somebits.com/~nelson/weblog-files/atom.png
• http://www.somebits.com/~nelson/img/nelson.png
• http://www.somebits.com/~nelson/weblog-files/pb_blosxom_yellow.png
• http://www.somebits.com/~nelson/weblog-files/masthead16.png
• http://creativecommons.org/images/public/somerights20.gif

I converted the images via this online encoder. I am too lazy to measure the actual impact the change had, too much trouble. The HTML size of my blog went up 10k (46k vs 36k), but I’m loading 5 fewer HTTP requests (16 vs 20) and not loading about 7k worth of image binaries. I’m optimistic.

My electricity bill is just terrible now, $500 / month. Since we moved in we’re using 800-1200 kWh / month, or about 1400W average usage all day every day. We’re both home all day and use computers, so I expect our power usage to be high, but that’s absurd. Where’s it going?

Now that we have a Smart Meter, PG&E has an awesome “My Energy” site where you can see hourly energy usage for your house, per day. Here’s a typical day, Tuesday January 24, 2012. We tend to be awake from 7am to 10pm. (I also have a private graph for the whole month.)

Looking at this, it’s clear turning the lights on in the morning and at night has a big impact. Our house has terrible lighting design, a lot of light wasted in ceiling cans. Also still all incandescent bulbs, although on fancy dimmers that should lower power usage.

What really surprises me is the constant usage of 700W all the time, even when we’re asleep. We turn the lights out at night! And sometimes it’s worse, up to 1000W. Fixing that nighttime usage seems like a good place to start.

Here’s a quick audit of things I know take constant power.

• 150W Wine cooler (450W when active)
• 50W Refrigerator (average)
• 120W My computer (when in use)
• 200W Ken’s computer (constant)
• 50W Linux server (constant)
• 80W A/V rack (when idle)

That’s 650W right there, all the time. We can easily turn off my computer and Ken’s computer, that’d cut about 300W out of the wasteage. My computer already goes down to 30W when idle but I can make that 1W. Ken’s computer is awful; it never goes to low power mode and the monitors constantly wake up in the middle of the night. I don’t know why, it’s a Windows 7 box.

I just unplugged the wine cooler. It’s really not necessary in winter. It’s insane that it runs one third of the time even when it’s cool outside; the manual says something about maintaining humidity. Paying $40 / month to keep my wine at 54F instead of 65F seems stupid. It may be more power than that, btw, the Kill-a-Watt registers 450W but 750VA. The website says it draws 6 amps when running, which is more like 660 kWh/hour, yes?

It’d be nice to shut down the A/V rack. But the Tivo is actively doing something useful, and that’s half the power. The other half is keeping a bunch of Sonos units running at 5W each. Probably not worth the bother to buy a timer or something.

Other ideas

It’d be awfully nice to get the usage data in real time off my meter and not have to wait 2 days for it to show up on the website. The buzzword for this is “Home Area Network” and PG&E, of course, is slow to deploy. No doubt in part because they have to worry about the insane people with imaginary diseases. Anyway, here’s some info on a HAN trial.

It’d be nice to install a solar panel on my roof. SF Solar Map estimates I have 1300 sq ft of space for panels which is good for 5kW. At 4.5 hours of sunlight / day that’s 6324 kWH / year, or about half my electricity usage for the year. A 5kW system costs $32,000, but after incentives the cost to me is $21,000. It’d save me about $2600 / year so I’d break even in roughly 8 years. A 2kW system would cost $7400 and break even in about 7 years. (Cost estimate assume $6.50 / W-dc, which may be optimistic.)

Theory

All this TV setup makes me curious; am I getting proper surround sound? I have 5.1 speakers connected to a Denon AVR-2309CI and everything HDMI, is it actually working?

The Denon receiver can decode the following formats via HDMI. I barely know what most of these are, but here’s a useful reference, also wikipedia.

• 2 channel linear PCM (16, 20, or 24  bit samples, 32-192 kHz), also multi-channel.
• Dolby Digital (5.1), DD Plus (7.1), Dolby TrueHD (7.1 high quality). Also known as AC3.
• DTS (5.1) and DTS-HD (7.1)

When playing back a 2 channel audio source, my Denon supports a bunch of playback modes to fake out surround sound (“matrixed“). I believe the sound has to have been mixed this way for this to actually work.

• Dolby Pro Logic II (fake 5ch) and IIx (fake 7.1)
• DTS Neo:6 (fake 6.1) and “Neural” (XM HD Surround)

Looking at the random video files I have on my hard drive, most have audio in MP3 2.0 or AAC 2.0. One is DTS 5.1 and a couple are Dolby Digital 5.1. I don’t think there’s any way to tell if any of those 2.0 files are matrix encoded fake surround, probably not.

Playback

That’s all the theory, what happens in practice?

The good news is my new AppleTV seems to correctly play back 5.1 sources, via both Netflix and PLEX. In particular the receiver seems to autodetect DTS or Dolby Digital and if I stand next to a rear speaker it does sound differentiated. Yay! It may help I told PLEX my receiver can decode AC3 (aka Dolby Digital).

The confusing news is for all these stereo sources I don’t really know what receiver setting is best to use. I have no idea how common this matrixed audio stuff is or whether it’s worth the effort to decode it. I’ve got a button on my remote labelled “Movie Sound” that cycles the various matrix modes but can’t really hear much difference in a quick test.

Testing

After a bit of looking I found a test suite, the charmingly named “AC3 Dolby digital 5 1ch Sound Test”. It’s available around the net in various places, like filestube. (Or watch it in 2.0 on Youtube.)

This file plays back fine via PLEX on my Apple TV. I also played with using Handbrake to encode it in various ways. Sounds great in true Dolby Digital 5.1. Also sounded surprisingly good in matrixed Dolby PLII; the rear channel separation worked well, although the subwoofer was silent. HandBrake also supports a “6-channel discrete” mode which didn’t really work right on my A/V system.

My A/V rack is controlled by a UniversalRemote MX 900 with a Univeral MRF-350 RF Base Station. It’s not going to win any UI awards but when we were shopping for this stuff three years ago this came in as the most capable and flexible system. So sad the A/V guy I hired to set it up, money well spent: he did an excellent job. My whole system is controlled from a single remote that’s not awful to use.

And as of today, now the new Apple TV is, too. I reprogrammed the remote myself. It turned out to be relatively easy..

1. Get a copy of the programming software MX_900Setup.zip (23,290,684 bytes); the official source requires a password they only give to resellers.
2. Run it on a windows box and plug the remote in via Mini-USB
3. Select “Communication / Upload” to get a copy of the current remote configuration from the remote itself.
4. Save that file away for safekeeping.
5. Edit. The manual is essential because the paradigm is weird. Most things are driven by the Program menu. Macros are recorded by clicking simulated remote buttons on the emulator. Codes can be imported via a database, and there’s a learning mode I fortunately didn’t have to use.
6. Download back to remote. Done!

The hardest part turned out to be digging through boxes to find a spare IR emitter on a cable. The IR Blaster in the front of my base station didn’t seem to be enabled, probably to help prevent crosstalk. It’s 2012 and we still use infrared and little cables to control A/V devices, it’s terrible.

All these new computers have me interested in improving throughput. I’ve suddenly got gigabit ethernet, a huge amount of RAM, fast CPUs with many cores.. What’s missing is fast I/O. The hard drive’s 30 megabyte/second throughput is often a limiting factor for me. What are my options?

• Put my data on an SSD. Appealing, but I don’t trust consumer SSDs not to fail. Also I hate having multiple disks with different capacity; I can never decide what to put on the SSD, what to put on platters. IE: right now I’m doing a lot of work with 300 gigs of music files and would like that to be fast, but it’d be stupid and expensive to leave those on an SSD forever (at 2012 prices).
• Use an SSD as a writeback cache for hard drive. Very appealing option, and there’s three options. My Z68 motherboard includes support for Intel Smart Response Technology but sadly there’s no Linux support. There are two Linux projects that do something similar: Flashcache and Bcache. But both seem pretty bleeding edge and require lots of manual setup. The Bcache release notes a few months ago included a comment about “slow silent data corruption” that put me right off.
• Hardware RAID. This Z68 motherboard has Intel RAID right on the motherboard, perfect! Only it turns out this kind of RAID is what’s known as fakeRAID; the operating system does most of the work. Linux can support fake RAID but if you’re going that route most people suggest just using Linux software RAID. (The advantage of motherboard RAID is the BIOS can help boot your OS).
• Software RAID. It’s been awhile since I checked in on the state of Linux RAID and sadly, it’s in disarray. Docs on the net are all randomly out of date, talking about 2.6 kernels or 3 year old versions of Ubuntu. I think the systems are pretty stable, being based on md, just poorly documentd. These 9.10 docs are the best I’ve found. The Ubuntu installer will create a RAID array for you or you can go low level with mdadm. Apparently GRUB 2 understands Linux software RAID so you can boot right off the RAID disk.

Of course there are many flavors of RAID. I’m mostly interested in speed, not reliability, so RAID 0 seems like the obvious choice except everyone says that’s nuts because you double your chance of a disk failure destroying everything. Fun!

bash-4.2 (default on Ubuntu) has a super annoying bug.

d=/usr/bin
ls $d/ssh-keyg[user presses tab]
ls \$d/ssh-key

In a nutshell, bash “helpfully” adds a backslash to escape the dollar sign because the user must have meant the literal character $. Previous to bash-4.2 it would have expanded the variable, so we would have seen “ls /usr/bin/ssh-keygen” instead. Which is way more useful

A quick search online finds the usual set of incorrect explanations and cargo-cult changes to bash-completion. (A popular fix is to use complete to make it so that the “cd” command works, leaving expansion broken for every other command.)

The actual truth is that this expansion is intended behavior in 4.2. Chet Ramey himself said so. He also recognizes this is awkward and has written a patch to add a new shopt called “direxpand” that will restore the desired, pre-4.2 behavior. Sadly that patch hasn’t made its way into Ubuntu’s Oneiric distribution yet.

One workaround for now is to press ESC C-e before hitting tab; that forces the command line to expand everything first.

I’m trying to unlock a new AppleTV 2 so I can use XBMC or Plex to play videos from my home network on my PC. It’s been seven years since I first ran XBMC on a hacked Xbox and it’s still way too hard to get low power hardware to play your own TV.

AwkwardTV seems like a useful reference.

Some notes on the jailbreak process:

• Apple firmware is signed via some online process. You need a “shsh” signature to bless firmware to run, even on already jailbroken systems. In general Apple only signs the most recent firmware, so it’s very hard to downgrade firmware.
• There’s a program called TinyUmbrella that will generate shsh for firmware you have already. However, it’s mostly not useful except generating a shsh for a current version. My AppleTV out of the box had 4.3 but I was “too late” to get a shsh for it, despite it running. TinyUmbrella is evil software that modifies /etc/hosts and does god knows what else as root. There’s a competitor called iFaith, which may get shsh even for old firmwares, but it’s Windows only.
• Seas0npass is the preferred jailbreaking software. Their guide is quite good. Don’t be fooled that it’s dated a year ago; they keep updating the jailbreak software.
• Jailbreaking really boils down to putting the AppleTV in DFU mode and reflashing the firmware with some funky thing. All the weirdness around Seas0npass is creating that firmware and then scripting iTunes to install it.
• If something goes wrong this Seas0npass troubleshooting guide is great. In particular I had the problem of iTunes saying “This device isn’t eligible for the requested build”. Turns out TinyUmbrella had evilly rewritten my /etc/hosts and was breaking the iTunes update process; telling TinyUmbrella not to do that fixed it.
• It took about seven tries, but I now have an Apple TV 2 running 4.4.4 (3330), but jailbroken. Ie: I can run my own software and the hardware I purchased.

Now what? Well simple thing is log in as root with the default password “alpine”. Hey, look, it’s like a Unix box! It’s even got apt-get, plus a bunch of apt sources of dodgy software.

First thing up is to install Plex, a media player that works by streaming video from a transcoding server. There are many sets of conflicting instructions on how to install Plex. This forum thread seems up to date as of January 2012. Conceptually…

• Install Python via apt
• Download this scary installer(a zip file)
• Change your root password back to the default “alpine”
• Run the scary installer
• It reboots the Apple TV, and now you have Plex!
• Run Plex, go to Settings / Security and disable tracking.

Next thing would be to install XBMC, but I’m already feeling bruised from dealing with all this poorly packaged hackerware. Full props to the folks making media software on unauthorized devices, but man is usability a challenge.

BTW, two crappy things about the Apple TV:

• There’s no HDMI cable in the box
• The remote is IR only

I decided to build a custom Linux kernel on my Ubuntu 10.10 Oneiric box. 3.0.0 feels awfully old, and I’ve been having trouble with a couple of things (well documented here). I was hoping a modern 3.2 kernel from Ubuntu precise would fix both my ethernet driver problem and, by building specifically for a modern Intel CPU, enable sched_mc_power_savings. No such luck.

Building a Linux kernel has gotten remarkably complex. I mostly quit trying myself back in 2.2 or so; once loadable kernel modules got reliable there was little reason for a top end developer like me to muck with the kernel. These Ubuntu-specific instructions got me there, although it’s a long winded and complex thing involving a lot of configuration management.

Then I got it built and installed and rebooted and miracle! It worked. Only with the same old bugs. 10+ seconds to initialize the ethernet, and that power savings sysctl still missing. I gave up and removed my custom kernel. I’ll wait for the Ubuntu release to upgrade.

I bought a Cyberpower UPS for my Linux server. I mostly like it, but the Linux software they include for it (pwrstat and pwrstatd) is comically bad. Network UPS Tools supports this UPS hardware too, if I weren’t so invested in this software already I’d try it instead.

Here are some bugs:

1. There’s no source code available, just two little ELF binaries pwrstat and pwrstatd
2. pwrstatd has to run as root, not a big surprise since it can shutdown the machine, but see bug #1. Also pwrstat (the userspace program) has to run as root, even to get the UPS status! It seems to mmap() itself into pwrstatd :-(
3. The default configuration is to shut down your machine after 60 seconds of power failure, then shut the whole UPS off after 10 minutes. That’s awfully aggressive.
4. Once a Linux box is shutdown, the ACPI system means it won’t reboot itself again when power is restored. Wake-on-LAN is the right way to fix this, I don’t know a quick and dirty way.
5. The various utility shell scripts /etc/pwrstatd-*.sh are very poorly written. The mail program assumes /bin/mail has a -r flag (mine doesn’t). One script has a test for ENABLE_EMAIL being “YES”, the other tests for it being “yes”. None of the test arguments are quoted so missing variables throw shell syntax errors.
6. Did I mention all the real work is done in pwrstatd, which runs as root, and for which there is no source code provided?

The cool thing about the UPS is the status report it gives over the USB includes a bunch of data, including how many watts the UPS is driving right now. I wrote a little Munin script and now I have a fancy graph of my server’s power consumption.

That graph above includes me unplugging the UPS and letting it run to zero. Original estimate was 82 minutes of runtime, actually got about 70. Oddly the power drain went up 10W after unplugged; the inefficiency of the AC generation?

This UPS seems to consume 10W of its own even when fully charged, and about 20W when charging. Not happy about that 10W. Also the “battery capacity” value is totally bogus during charging; it only updates when I force the UPS to run off battery or run a self-test.

Here’s my Munin plugin (first ever!)

#!/usr/bin/env python

"""
pwrstat - Plugin to monitor Cyberpower UPS via pwrstat

by Nelson Minar <nelson@monkey.org>, placed into the public domain

Unfortunately this plugin must run as root, because pwrstat requires
root to even read data from the UPS.
"""

import sys, subprocess

# config
if len(sys.argv) > 1:
 if sys.argv[1] == "config":
 print """
graph_title Cyberpower UPS Status
graph_category UPS
graph_vlabel value

inV.label Utility Voltage
inV.min 0
inV.max 200
inV.warning 100:130

outV.label Output Voltage
outV.min 0
outV.max 200
outV.warning 100:130

battery.label Battery Capacity (Percent)
battery.min 0
battery.max 100
battery.warning 35:105

runtime.label Remaining Runtime (Minutes)
runtime.min 0
runtime.max 200
runtime.warning 10:200

loadWatts.label Load (Watts)
loadWatts.min 0
loadWatts.max 1000
loadWatts.warning 30:150

loadPct.label Load (Percent)
loadPct.min 0
loadPct.max 100
loadPct.warning 0:75
"""
 sys.exit(0)
 else:
 sys.stderr.write("Unknown argument %s\n" % sys.argv[1])
 sys.exit(0)

# fetch

output = subprocess.check_output(("/usr/sbin/pwrstat", "-status"), shell=False)
output = output.split('\n')
data = {}
for datum in output[10:-2]:
 name = datum[:29].strip('.').strip()
 value = datum[31:].strip()
 data[name] = value

print "inV.value", data["Utility Voltage"].split()[0]
print "outV.value", data["Output Voltage"].split()[0]
print "battery.value", data["Battery Capacity"].split()[0]
print "runtime.value", data["Remaining Runtime"].split()[0]
print "loadWatts.value", data["Load"].split()[0]
print "loadPct.value", data["Load"].split()[1].split('(')[1]

sys.exit(0)

# This is the pwrstat format that we parse. Note, it contains tabs.
# Comes from pwrstat version 1.2
exampleOutput="""
The UPS information shows as following:

Properties:
 Model Name................... UPS CP1000AVRLCD
 Firmware Number.............. BFC7101.4K1
 Rating Voltage............... 120 V
 Rating Power................. 600 Watt

Current UPS status:
 State........................ Normal
 Power Supply by.............. Utility Power
 Utility Voltage.............. 125 V
 Output Voltage............... 124 V
 Battery Capacity............. 100 %
 Remaining Runtime............ 77 min.
 Load......................... 54 Watt(9 %)
 Line Interaction............. None
 Test Result.................. Passed at 2012/01/25 01:39:47
 Last Power Event............. None
"""