Brian C. Lane » Programming

Home Media Server Updates

Brian Lane — Sun, 18 Apr 2010 06:26:31 +0000

Tonight I finished adding delete handlers and a delete confirmation dialog to hms. This is probably the state that the code will be in for my LFNW presentation . . . → Read More: Home Media Server Updates

color output from git log -p on OSX

Brian Lane — Sun, 18 Apr 2010 06:20:16 +0000

By default git on OSX wasn’t colorizing its output. Two things needed to be setup – setting the color.ui to auto and setting the pager (less) to allow raw characters. Add this to ~/.gitconfig

[color] ui = auto [core] pager = less -R

Presto! Nice colorized output . . . → Read More: color output from git log -p on OSX

Newseum Page Grabber Script

Brian Lane — Sun, 07 Feb 2010 17:08:20 +0000

Newseum archives the front pages of of over 500 newspapers from all around the world. If you know the ID of the papers you want to see you can use this simple Python program to download the jpg of the papers’ front page to your local system.

Edit the CITIES list to set the IDs of the papers to be grabbed.

#!/usr/bin/env python
"""
    Quick Newseum Frontpage Grabber script
    Copyright 2009 by Brian C. Lane
    Imp Software
    All Rights Reserved

    Modify CITIES list below to add the city designators (as seen in the
    URLS at http://www.newseum.org/todaysfrontpages/default.asp)
"""
import urllib2
import re
import os
import urlparse

# Add more cities here
CITIES = [ "AL_AS", "AL_MA",   ]

NEWSEUM_URL="http://www.newseum.org/todaysfrontpages/hr.asp?fpVname=%s"
NEWSEUM_IMG="http://www.newseum.org"

def fetchNewseumImage(city):
    """
    Fetch the image for a city
    """
    print "Parsing the page for %s" % (city)
    page = urllib2.urlopen(NEWSEUM_URL % city).read()

    # Quick and dirty grep for the image name
    match = re.search('
The source is also hosted here at github.

Simple SQL Schema Migration

Brian Lane — Sun, 10 Jan 2010 20:57:49 +0000

I’m working on an application to manage my streaming media for my Roku player using sqlite3 and Python for everything. One thing I’ve learned over the years is that your SQL schema always changes. Once your code is in production you always have something you need to change about it, whether it’s adding a new column, changing a type or tables to support new features. I wanted a way to automatically update the database schema when a new version of the code is installed. I don’t want to jump into the complexity of using SQLAlchemy and migrate so I came up with this simple method. The database has a table named schema with a single row with the current schema version in it. This class checks the current version and executes any missing commands, bringing it up to the latest version.

class DbSchema(object):
    """
    Database schema creation and modification
    """

    # Schema revisions, rev[0], etc. is a list of SQL operations to run to
    # bring the database up to date.
    sql = ["""  create table users(id INTEGER PRIMARY KEY, username UNIQUE, password, email);
                create table sources(id INTEGER PRIMARY KEY, name, type, path UNIQUE);

                create table schema(version);
                insert into schema(version) values(1);
            """,
            # Add default values
            """ insert into users(username, password) values("admin","badpassword");

                update schema set version=2;
            """,
            # Add media table
            """ create table media(
                    id INTEGER PRIMARY KEY,
                    name,
                    path
                );

                update schema set version=3;
            """,
        ]

    def __init__(self, database):
        self.database = database

    def upgrade(self):
        """
        Upgrade the database to the current schema version
        """
        # Get the current schema version number
        conn = sqlite3.connect(self.database)
        conn.row_factory = sqlite3.Row
        cur = conn.cursor()
        try:
            cur.execute("select version from schema")
            version = cur.fetchone()['version']
        except:
            version = 0

        if len(self.sql) > version:
            for update in self.sql[version:]:
                cur.executescript(update)
        cur.close()
        conn.close()

Initialize it with the path to your database, and then call the upgrade() method on it.

    schema = DbSchema("./library.db")
    schema.upgrade()

Add new tables, changes to old tables, default data, etc. to new strings in the sql list. Make sure to update the schema table’s version as the last step for each revision. As long as you only make database changes via this class you will ensure that the tables are ready for your new code when each new revision is released.

This version is being used with sqlite3, but works just as well with mysql or postgresql.

Streaming Local Video With Your Roku

Brian Lane — Mon, 21 Dec 2009 00:38:42 +0000

Ever since I switched from Dish Network to a Roku player a few months ago I have had a couple of things I wanted my player to be able to do. The ability to categorize my Netflix queue is at the top of my list and it would be very cool to be able to play video from a local server. The SDK won’t help me modify the Netflix application, but streaming from a local source is actually not a huge problem.

On Friday Roku finally released the SDK for their handy little Roku Video player (sometimes called a Netflix box since that was originally the only content you could stream). The SDK is free, all you have to do is signup at the developer page. The zip archive contains everything you need to get started hacking together a new channel for your Roku player.

For the most part development is cross platform. You are restricted to using their language, BrightScript, so there are no compilers or IDE needed. It helps if you have the make utility available on your system, the examples come with makefiles that will zip up the application and install it into your player with a single command. If you want nice images to show up while scanning through a video you will need to capture them from the file and create a BIF file to tell the player when to show the images. The biftool is currently only available on linux and Windows.

To be able to upload software to your player you need to enable developer mode. This is accomplished with a set of remote control keypresses. For some reason it took me about 10 tries before I got it to work. I’m still not sure if it depends on where you start from, but what finally worked for me was being at the top level channel menu with something other than settings selected.

Once that is done the box now has a simple server running on port 80 and a debug port on 8085 where you can view debug output from your app or from the system if the upload failed, or if parsing the script files failed. That’s all there is to it. No cumbersome app signing needed at this point. It makes it very easy to edit, upload, try, debug, repeat. If you want to actually publish your application as a new channel for the world to use the process is more involved and requires Roku to approve your application.

The SDK includes, among other things, a couple of video examples. One simply points to a URL and plays it (the URL is hard-coded into the application). The other is more fully featured and uses TED talk videos. It allows you to setup a hierarchy of categories using a couple of XML files. This is a good place to start for streaming local content.

The first thing I did was setup an Apache 2.2.13 server on my son’s Fedora11 system. I simply aliased the directory of mp4 videos and it was up and running. As a side note, Chrome (at least on OSX) can stream mp4 videos over http without installing anything else.

I then played around with the simple stream example to make sure I could actually watch the videos. They were encoded using Handbrake on either a OSX or Fedora system. Most are 1G in size or less, and I left most of the encode settings alone. I found that about the first hour streams fine, but after that point the player starts to rebuffer every minute or so. It appears to depend on the location in the file, so it may have something to do with how it was encoded.

The documentation included appears to be fairly complete. The have extensive documentation on BrightScript, and on the Roku extensions that you use to access to features of the player. I did find a couple of things that weren’t described in detail, but at this point that hasn’t hindered progress at all. BrightScript is a fairly simple language and anyone with programming experience should have no trouble catching on quickly.

The example video app uses a hard-coded URL for the source of the XML driving it. I added a keyboard page that prompts the user to enter the host or IP address of their video server. It saves this to the registry (persistent storage) so it only needs to be entered once. For the moment, this is really the only change I needed to make to the example application. Everything else happens on the server side.

I slapped together a Python script to take the names of my mp4 videos and generate the feed XML to drive the player. I used mp4info to pull the video’s bitrate and format from the file and insert into the movie’s description page. I don’t have cover art for them, so that got hard-coded to a default image. The script generates a static XML file for the Roku to load when my custom application is run.

At this point I can browse the movies and play them. Mostly. I’ve run into a couple of glitches, and am not sure exactly what the solution for them is yet. One is that it seems that at about the 1 hour point in the video playback it starts to rebuffer every minute or two. If I stop playback, restart and skip to the 1 hour marks it still happens, so it appears to depend on the point in the file, not the time spent playing.

The other is that the aspect ratio of many of the videos isn’t quite right. All of them seem to be squished to some degree or another. This is probably due to the Handbrake parameters I used when encoding them from the original DVDs. They look fine when played back on a PC but the Roku either isn’t properly sizing them, or I’m doing something wrong (more than likely).

Overall I am very happy with the Roku and the SDK. They have done an excellent job of documenting the system, providing example code and especially keeping it from turning into a brick. Several times I have managed to get it to lock up and reset itself. It has always come right back up with no problems. I am looking forward to seeing what other channels people come up with. On the top of the list for me is Hulu streaming, but I’ll leave that up to someone else to implement.

You can look at my code, for what its worth, at my bitbucket homevideo repository.

SharePics Accepted by Apple!

Brian Lane — Fri, 18 Dec 2009 05:15:55 +0000

After many ~~hours~~minutes of redesign the new look of SharePics was ready to go. I re-packaged the app, captured new screenshots, and re-submitted it to the AppStore. 9 days later I received an email notifying me that my app was now available for sale in the App Store!

Thanks to everyone who offered advice and design ideas after the crushing rejection last month. I couldn’t have done it without you! Now you can easily share your pictures with your friends (note, you need an iPhone or iPod Touch with Bluetooth support and iPhone OS v3.0 or later).

iPhone Tech Talk Seattle

Brian Lane — Mon, 02 Nov 2009 15:10:30 +0000

Today is the Seattle iPhone developer talk event. I’m headed for the ferry in a few minutes and figured I would give the Wordpress iPhone app a try. I’ll update this post as I have time today.

Well, it was a long day! I used twitter a bit, but didn’t have a chance for any longer updates here. Here are my impressions and short notes from the sessions I attended.

I attended 4 presentations: UI Design Essentials, Adding In App Purchase, Core Data, Testing and debugging and Networking.

All the talks were packed with good information. The speakers were of varying levels of energy. This means that it was sometimes hard to concentrate on the material because the speaker was a bit boring.

UI Design Essentials – Your icon is your business card and it will make or break the success of your app. Most users buy with their phones (this surprised me, I buy with iTunes) so all they see is your icon and the name of your app in their search results. The brain processes shapes and colors first, then textureand smaller details. This means your icon should have distinctive colors and a clear outline, not a complicated and cluttered design. You should be using sound in your app. People respond instinctually to sounds, they make your app more polished. You should support landscape mode. Studies show that those under 35 prefer two thumb landscape and those over prefer single finger typing. Read the Human Interface Guideline doc. It will tell you how to design an excellent User Interface. Too many people haven’t read it, or have only skimmed it. (This could be said for most of Apple’s documents, they have a huge amount of useful information available if people would only take advantage of it).

In App Purchase – This talk mostly covered the steps needed to enable and test purchases in your app. You can setup test itunes accounts to use to test the buying process. There are 3 types of purchases: Consumables, non-Consumables and subscriptions. It is up to you to track subscription information and expiration. You should give uses the option of restoring everything they have bought, for cases where they are setting up a new (ie. replacement) phone. Consumables are not restored. In App Purchase is a game changer. No more need for supporting 2 versions of an app.

Working With Core Data – This is the one where I almost fell asleep. Being right after lunch (Ham sandwich, potato salad, chips and an Apple) probably didn’t help. Core Data is backed by SQLite. Don’t put blobs in the db, the file system is much better at handling that. Normalized vs. denormalized. If you aren’t careful you can end up with some messy tables. Table schema is private, no direct access, unless you want to break it. Core Data can batch results to save memory. Returns a NSArray looking object that will return more objects as you iterate through it but really only holds X in memory at one time.

Testing and Debugging – Covered how to add Logical and App Unit Testing with Xcode. Support is built in, but setup of the in-app one was a bit convoluted so I’ll have to look it up. Static analysis using clang is awesome. Built into Xcode, uses the UI to show you how it came to its conclusions. Suggested that a new build target be created with it turned on since it does take longer than a normal build (and doesn’t really need to be run every time). I was surprised it didn’t cover Instruments. Covered ad-hoc distribution for beta testers, building and distributing the executable and provisioning profile. Also stressed the importance of saving the executable and dsym someplace safe — when you get a crash report you must have the exact binary and debug symbols the user is running. It isn’t enough to just rebuild them, so zip them up and stuff them into source control.

Networking – Covered ‘normal’ networking first. Use the highest level classes unless absolutely necessary — you will end up writing unneeded code otherwise. Don’t use threaded synchronous functions, use delegates instead. Covered Bonjour, registering a bonjour name, browsing for services on the LAN. Don’t setup the discovered services when you find them, it is a waste of time and can slow things down dramatically. Wait until the user picks the device+service they want to connect to. Covered GameKit. Works with WiFi and BT, unless you are using PeerPicker, which only works with BT right now. Supports peer to peer voice chat, example source is still in the works. Lots of new and updated networking example code is available.

Overall I found the conference to be useful and informative. I would have liked them to have Q&A but the speakers always ran out of time. The environment was different than the open source conferences I have attended (LFNW, OSCON). There was no interaction with the audience, and very little interaction within the audience. At LFNW especially you have alot of feedback in the audience, both with questions asked and questions answered by other attendees.

I was actually expecting the content to be a bit more over my head (I do not consider myself an ObjC expert), but I found it all to be understandable and accessible, but not too simplistic. I think Apple has done a good job of targeting the middle road of iPhone developers. There were a few times when I felt like the speaker was talking down a bit, but it likely was only because of the potential mixture of experience present.