• svg:text – baseline considered harmful

    SVG has a bunch of properties for laying out text. Two really handy ones are “dominant-baseline” and “baseline-shift”, which together let you move text up and down a bit. “dominant-baseline: central” seems particularly useful because it lets you lay out text where 0, 0 is the middle of the text, not the baseline underneath the letters. Ie: visually center things vertically.

    Unfortunately those properties don’t work in many SVG implementations. They work in Safari and Chrome, but not in Mobile Safari, IE, or Firefox.

    A workaround is to use the dy parameter on the text nodes to move them around. dy: .35em is about like centering. I don’t think this works via CSS, though, only in the DOM.

  • Before and After

    I’ve added some new code to my wind history project to show winds based on month of year. With all 12 months selected the new code should produce similar results to the old code. I don’t really have any proper tests for this stuff, though, so I’m just kinda looking at the pictures and saying “yeah, looks about right”. Here’s a before and after: do the wind roses look the same to you? (Note that I moved the location of one wind rose, San Jose. That’s a bug fix).

     

     

     

    Update: Seth was nice enough to register the images and do a difference calculation for me. Which is awesome and sad all at once :-P

  • Browser testing lab

    Firefox, Chrome, Safari, and the ugly duckling: IE9 in a Windows VM.

  • Geolocation API tests

    I’m adding geolocation to my windhistory project. It’s a very easy API. Some quick notes based on a call to getCurrentPosition() with a few options: { maximumAge: 1000*60*60, timeout: 1000*2, enableHighAccuracy: false}

    Chrome on MacOS seems to use IP address to identify what city I’m in. It puts me at 37.775/-122.419, the middle of San Francisco, whether my WiFi is on or not.

    Safari on MacOS seems to require WiFi location. It won’t geolocate at all if wifi is turned off; the Javascript API returns error -2 (no position available). With WiFi on it seems to use the same airhook style location as the iPhone. It puts me at 37.749/-122.428, which is just about where I’m sitting. MacOS has privacy System Preferences for location services. The geolocation API stops working if you disable location services entirely, but it’s not necessary to have “Safari” checked in order for Safari to geolocate you. That seems like a bug, even if Safari itself is doing some privacy protection.

    Firefox 6 on MacOS also seems to use WiFi location, 37.749/-122.428. Didn’t test what it did without wifi.

    MSIE 9 on Windows 7, running in a VirtualBox VM on my Mac, gives me a slightly different location of 37.748/-122.416. I’ve got no idea where it got that idea, but it’s a pretty good fix. Certainly not wifi! Maybe Microsoft has a geolocation service guessing based on external IP?

    Firefox 4 on Windows 7 (in the VM) puts me at 37.775/-122.419, just like Chrome.

    Bottom line: Chrome seems to use an IP database, Safari and Firefox use the Mac API, and who knows what IE is doing.

  • Quick thoughts on writing Objective C

    Playing with my new Mac, this is my first time writing Objective C code in 15 years, and C or C++ code in nearly 10. Some quick thoughts after finishing Your First Mac Application.

    • Header files? In 2011? That require forward declarations of classes? Ugh.
    • Objective C properties are handy syntactic sugar. I have no idea what “synthesize” really does, but I like the result.
    • Static typing is awkward. I’m prefering dot style for properties
      self.track.volume = sender.floatValue;
      But if the type of “sender” is id that doesn’t work. I have to fall back to
      self.track.volume = [sender floatValue]
      and rely on Objective C magically not type checking method names on id. Objective C has duck typing, sorta, but not everywhere.
    • Manual memory management continues to suck. Apple has papered over it in various ways, with retained properties and autorelease and now ARC. But the fact that the programmer is even aware of reference counts is a serious flaw in the language. Seriously, every other OO language for 20 years has had garbage collection. Hell, even C languages have had automatic garbage collection via a conservative collector like the Boehm collector. WTF is wrong with Apple that their development language requires manual memory management?
    • Xcode 4.1 is a pretty solid IDE. I could drop into it with just a basic tutorial and understand most of what was going on quickly. The Interface Builder stuff so far seems OK, although we’ll see how it works with real code. I’m impressed with how one can do complex user interface constraints entirely graphically.
  • The size of an empty Mac app

    I created my first Macintosh application today, following Apple’s excellent docs. I didn’t write any code yet, just the default Cocoa application template. I’m always fascinated what the size of empty apps is; it often indicates how simple and clean the underlying architecture is.

    I did “Build for Run” and stripped the binary. The compiled app is 44k. The binary is 14k, the NIB is 26k. It takes 10M of memory (80+M if you count shared).

    Xcode created 13 files in my source directory: 14,000 lines of source totalling some 400k. Almost all of that is an XML bloated NIB file I’ll never look at as text. There’s 51 lines of ObjC code in 3 files. Pretty much all that is boilerplate; the important bits are

    int main(int argc, char *argv[]) {
        return NSApplicationMain(argc, (const char **)argv);
    }
    @interface TrackMixAppDelegate : NSObject <NSApplicationDelegate> {
        NSWindow *window;
    }
    @property (assign) IBOutlet NSWindow *window;
    - (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
        // Insert code here to initialize your application
    }
    

    It’s certainly nicer than the 400+ lines of unmaintainable garbage that Visual Studio’s wizard generates doing a similar task. I’m optimistic that the NIB system will work out well. MacApp’s version of this was a fucking disaster in the OS 6 days, but NextStep Interface Builder was always pretty impressive.

  • Whitespace in HTML

    I forget this about once a year. These two kinds of markup do not render equivalently:

    <div>
    <span></span>
    </div>

    <div><span></span></div>

    The problem is the whitespace in the line breaks; those get rendered. In Chrome, they seem to add 1 pixel of spacing.

  • Digital elevation models

    Doing some research on Digital Elevation Models, for making terrain maps.

    SRTM

    (Wikipedia, OSM Wiki) sourced from a space shuttle overflight in 2000. Comes in several flavours and sources.

    Sources:

    • NASA source (download link). Original data. Version 2.1 is their latest, cleaned up by NGA. The NASA has some data voids, particularly at mountains and lakes. Documentation of the dataset.
      License: Data said to be fully open for any use; don’t have a specific reference.
    • CGIAR-CSI Data processed to remove voids. SRTM3 only. They publish it as “version 4.1”, version number reflects their data cleaning. Also available as an Amazon EBS volume.
      License: non-commercial with attribution.
    • HydroSHEDS. SRTM data with voids filled, SRTM3 only. Also vector data on watersheds. Technical documentation.
      License: “free for non-commercial use”
    Details on NASA source:
    SRTM1:
    • Resolution: 1 arc second, roughly 30m, z=12
    • Format: .HGT files, 3601×3601 per one degree square file. Each sample is 16 bits, height in meters (-32768 for no data). GDAL supports this format.
    • Coverage: Continental US, a bit of Alaska, and various US territories. Coverage map.
    • Size: 1117 files of 25 megs each, or 27 GB uncompressed. 2:1 compression with gzip.
    SRTM3:
    • Resolution: 3 arc seconds, roughly 90m, z=10
    • Format: .HGT files as SRTM, 1201×1201 per one degree square file.
    • Coverage: global land mass from 60N to 56S. (Missing Antarctica, significant Northern parts of North America, Europe and Asia).  Coverage map.
    • Size: 14546 files of 2.8 megs each, or 39 GB. 2:1 compression with gzip.
    SRTM30:
    • Resolution: 30 arc seconds, roughly 1km, z=7
    • Coverage: global from 90N to 60S. (Missing Antarctica). Coverage map. See also GTOPO30.

    National Elevation Dataset

    Wikipedia. Also NED FAQ. USGS data from various sources.
    • Resolution: 1 arc-second (z=12), 1/3 arc-second (z=14), and 1/9 arc-second (z=15).
    • Format: accessible via WMS. Data downloads provided in ArcGrid format and “float format”, looks like 32 bits / sample.
    • Coverage: United States and Mexico only. 1/3 for the 48 states, 1/9 in some detailed places. Coverage maps: 1 arcsecond1/31/9
    • Size: files appear to be double the size of equivalent SRTM data. Rough guess: 50GB for 1 arc-second, 450GB for 1/3 arc-second.

    Viewfinder Panoramas

    Eclectic collection of more detailed / patched data than SRTM in certain spots. Unclear licensing.

    ASTER GDEM

    Joint NASA/Japan METI project, produces a 30m DEM. (Wikipedia, OSM Wiki) Has coverage between 83N to 83S and some holes filled compared to SRTM. Some question on quality; the data comes with a “quality assessment file”. May be free to use, but some concern on the OSM Wiki that its license is not consistent.

    OpenDEM

    Well intentioned but small project to provide better DEM data. Some useful notes.

    GMTED2010

    USGS project. Global data: 90S to 83N. 7.5 arc second (30m). Looks to be a combination of SRTM data and other sources. License is presumably public domainish. The metadata page says “Metadata access constraints: No restrictions, All GMTED2010 data products and associated metadata are publically available”.

  • Notes on building a slippy map of the Fisk maps

    I got interested in these Fisk maps of the Mississippi, so I’m making a slippy map. Here’s what I did.

    1. Downloaded the rectified TIFFs from the Army. These have already been rotated and georeferenced and are pretty high resolution. The p22* files are the maps I worked with.
    2. Created pyramidal overviews for the TIFFs so I could explore them in QuantumGIS more quickly. Not strictly necessary, and in fact may be harmful: gdal2tiles seems to pick up the JPG compression.
      gdaladdo -r average –config COMPRESS_OVERVIEW JPEG –config PHOTOMETRIC_OVERVIEW YCBCR –config INTERLEAVE_OVERVIEW PIXEL -ro p22_s12.tif 2 4 8 16 32
    3. Created a virtual raster layer including properly annotated sections of no data
      gdalbuildvrt -srcnodata “255 255 255” fisk.vrt *.tif
    4. Generated slippy map tiles
      gdal2tiles.py -z 4-13 fisk.vrt
    At that point, I was done! gdal2tiles.py includes a Google maps interface that works. Of course I couldn’t leave well enough alone, so I wrote my own Javascript interface using Polymaps.
    Some other caveats and notes:
    Data quickly gets big and slow. The source map is only 32 megs of TIFFs, maybe 10,000 x 150,000 pixels. But all the cutting and repackaging and uploading is going to take hours. I did OK experimenting with a small subset (just 2 TIFFs of 15). I decided to cut the render to z=13 instead of going full resolution to z=14: 4x speedup! (Final result: roughly 6 hours CPU, 230 megs of GIFs for the tiles).
    The source TIFFs include page margins. A proper seamless map would require cutting those off by hand. I didn’t bother for this quick hack.
    Output image format is a problem. The default PNG output of gdal2tiles is enormous. Some details, including file size of the tileset for a sample input of only 2 of the TIFFs
    1. 32 megs: source TIFFs (sheets 12 and 13 only)
    2. 60 megs: gdal2tiles output, 24 bit PNGs with transparency. (was 220 megs at z=14)
    3. 50 megs: optipng. Doesn’t help as much as I need, only 10-20%.
    4. 15 megs: JPG. These would be awesome, but I need transparency because the source maps aren’t rectangular
    5. 24 megs: 8 bit PNGs with transparency. Unfortunately imagemagick’s PNG8 conversion is buggy, particularly with transparency, and I can’t use it.
      convert foo.png PNG8:foo-8.png
    6. 28 megs: GIFs. imagemagick does a good job emitting GIFs with transparency.
  • iPhone consolidated.db location tracking notes

    I’m trying to understand the consolidated.db iPhone location database, which means understanding how iOS stores and managing data. Here’s some notes.

    Resources

    1. @aallan’s original announcement
    2. iPhone Tracker (Objective C code)
    3. Technical instructions from howto.wired
    4. nphonetrack (.NET code)
    5. iPhoneLs.py
    6. iphone-tracker.py
    7. Quick start for sqlite3
    Notes
    • When you make a backup of an iPhone, it creates a big directory full of files with random 160 bit filenames. These are named files in the iPhone that iphonels.py can display. My 3 biggest files are named
      HomeDomain::Library/Caches/com.apple.WebAppCache/ApplicationCache.db
      AppDomain-com.foreflight.ForeFlightMobile::Documents/Procedures.zip
      RootDomain::Library/Caches/locationd/consolidated.db
    • consolidated.db is the file with the location data in it. It’s a sqlite3 database.
    • The juicy tables are (apparently) CellLocation and WifiLocation. The *Counts tables seem to contain just a row count. Presumably the Boxes tables are some sort of spatial index. I can find no record of how often I was at a specific location other than what’s implicit in the timestamps and density of cell/wifi locations.
    • I have 32,013 rows in CellLocation, 171,040 rows in WifiLocation
    • Timestamps are like 309803342: I believe this is NSDate seconds since 1 Jan 2001. Add 978307200 to get it in seconds since Unix 1970 epoch.
    • HorizontalAccuracy varies from 50-300 for WifiLocation, and up to 80,000 for CellLocation.
    • Confidence is 0, 50, 60, 65, 68, or 70
    • Speed and Course are always -1 (no surprise).
    Some quicky sqlite3 shell code
    .mode csv
    .output wifilocation.csv
    select Timestamp, Latitude, Longitude from WifiLocation order by Timestamp;
    .output CellLocation.csv
    select Timestamp, Latitude, Longitude from CellLocation order by Timestamp;
    
    Some pasted info from sqlite3
    
    sqlite> .tables
    CdmaCellLocation                   CellLocationCounts
    CdmaCellLocationBoxes              CellLocationHarvest
    CdmaCellLocationBoxes_node         CellLocationHarvestCounts
    CdmaCellLocationBoxes_parent       CellLocationLocal
    CdmaCellLocationBoxes_rowid        CellLocationLocalBoxes
    CdmaCellLocationCounts             CellLocationLocalBoxes_node
    CdmaCellLocationHarvest            CellLocationLocalBoxes_parent
    CdmaCellLocationHarvestCounts      CellLocationLocalBoxes_rowid
    CdmaCellLocationLocal              CellLocationLocalCounts
    CdmaCellLocationLocalBoxes         CompassCalibration
    CdmaCellLocationLocalBoxes_node    Fences
    CdmaCellLocationLocalBoxes_parent  Location
    CdmaCellLocationLocalBoxes_rowid   LocationHarvest
    CdmaCellLocationLocalCounts        LocationHarvestCounts
    Cell                               TableInfo
    CellLocation                       Wifi
    CellLocationBoxes                  WifiLocation
    CellLocationBoxes_node             WifiLocationCounts
    CellLocationBoxes_parent           WifiLocationHarvest
    CellLocationBoxes_rowid            WifiLocationHarvestCounts
    
    CREATE TABLE CellLocation (
      MCC INTEGER,
      MNC INTEGER,
      LAC INTEGER,
      CI INTEGER,
      Timestamp FLOAT,
      Latitude FLOAT,
      Longitude FLOAT,
      HorizontalAccuracy FLOAT,
      Altitude FLOAT,
      VerticalAccuracy FLOAT,
      Speed FLOAT,
      Course FLOAT,
      Confidence INTEGER,
    PRIMARY KEY (MCC, MNC, LAC, CI));
    
    CREATE TABLE WifiLocation (
      MAC TEXT,
      Timestamp FLOAT,
      Latitude FLOAT,
      Longitude FLOAT,
      HorizontalAccuracy FLOAT,
      Altitude FLOAT,
      VerticalAccuracy FLOAT,
      Speed FLOAT,
      Course FLOAT,
      Confidence INTEGER,
      PRIMARY KEY (MAC));