1ft Startup

An interview went up a few days ago on Nokia’s Workshop blog about coming up with and developing our first private title, and the problems we ran into with testing and porting. Check it out!

Thanks, Naveen!

We originally launched a little game called Mitosis nearly a year ago, as a cool experiment in what it would take to get a basic, unbranded game from nothing all the way through the door and as far as we can take it. Like most geeks, we’ve released very few of our own projects to commercial success, spending our time on other peoples’ projects for the actual cash.

If we were going to, say, write up the process of how we got Mitosis launched, it would look something like this:

- Build a cool game. The fun part. When you think you’ve got something good, you should consider the option to…
- Take it to an agent. We happened to know a few good people who could represent us to the carriers, otherwise we’d have languished out on GetJAR (more on that next time). Your agent will entice you with eye-popping (but un-guaranteed, of course) numbers. Your agent will also give you feedback from carriers, middlemen, basically whoever happens to be nearby, and if you’re lucky, you’ll eventually….
- Get reviewed by the carriers for deck placement. It takes MUCH longer than you think. Dealing with carriers is extremely slow; typically they have review processes that aren’t constant, but periodic: monthly, or even quarterly. If you’re one of the lucky, chosen few, you get to spend about 60 days to….
- Port your app. Foolhardy and spendthrift people that we are, we decided to do this in-house with the help of Device Anywhere. This is a possible option, but porting is never enjoyable. Do not underestimate how much work this is. It’s really a tedious and ongoing process, and some problems you simply can’t get around. Definitely take the time to streamline your build tools as much as possible, and you can survive to see the day that you…
- Get your builds certified. Like Verizon’s relationship with NSTL, T-Mobile now has a for-pay certification program with a company called True North. In our experience, their testing has been fair, and their test cases are reasonable; the true downside is simply that it’s up to the developer/publisher (depending on your agreement) to pay for the testing, and re-testing if you fail. That’s life! And now, there’s not much left to do but….
- Wait for it to show up in the catalog. Maybe we got lucky, but we ended up in some sort of deck-placement purgatory for many months while there was some sort of internal reorganization. Recently, finally, we launched on T-Mobile. Yeah!

Our important take-homes from this whole process:
- Publishing through carriers puts you at the mercy of many, many forces beyond your control.
- Decide what you want your company to be good at and focus on improving it.
- Don’t underestimate the pain of porting.
- The last 5% of the process can will drag on for months.

And most importantly, don’t forget why you’re doing what you are doing. We set out to have some fun, take a little break from our day jobs, and make some great games. And to be honest, we started to get burnt out by the actual business process. Once your game is out the door and in the hands of pre-teens spending their parents’ second mortage, though, there’s really no feeling like it.. And we may be just foolish enough to do it again.

I was about to write an amendment to the previous post per Apple’s recent announcements about the upcoming SDK, but found a few people who already have similar thoughts and concerns. Remember that the iPod already has software developers building things for it, but it hasn’t been a huge codefest bonanza the way it was with, say, Palm software, because Apple has quite a tight grip on who they allow to develop for them and who they don’t (not to mention: they control distribution with the iTunes store).

Nokia and Apple are easily the dominant players in the new generation of high-end handsets. They’re both well poised to do something everyone in the industry has been dreaming of for years: loosening the stranglehold that the carriers have on content and open the market to crazier ideas. My money is on Nokia supporting those endeavors and Apple, well, not so much. I’d say that the great unwashed out here in reality are stuck with Java ME or web apps for at least the near future.. But I would be extremely happy to be wrong.

So, you’ve organized a clever build system that can manage a thousand different SKUs of your mobile software with the greatest of ease, and now you have to stop putting off the fun part: testing. How the heck do you really test on dozens of devices?
Your first (and probably most important) stop should be a few things to consider before even loading up a phone:

1. Make sure you have all of the features and behavior you want in the app. If you make functional changes to your game/app halfway through porting, you’ve invalidated all the fun work you’ve done and you get to do it again. Think outside of just your own requirements! Be doubly sure that your app also has all the features required by the carrier/vendor/external test team you’re porting for.
2. Be very careful choosing the phones you want to port to. Each phone will have its own quirks and will be less easy than you think it will, so choose carefully. If you can, pick newer devices, and carefully screen external demands for discontinued models (requirements get old fast, so be sure you’re not just being handed last quarter’s list). While some consumers are likely have the discontinued handsets out in the wild, people are less likely to be buying content than on their brand new Nokia. Also, old devices are going to be a pain to find yourself for testing.

Once you’ve trimmed your handset list and settled all your features, you’ve got basically three levels of quality testing, each with their own tradeoffs:

1. Beta programs run by mobile communities. These programs allow you to upload your code to a select group of community members and allow them to try out your code. You get helpful feedback, they get early access to the coolest new stuff; everyone wins. Here are a couple of them:
   • GetJAR Beta Testing
   • Vodafone Betavine

   I’ve personally used GetJAR’s beta program and had surprisingly good results. One reason for this is that GetJAR is actually very strict about users who sign up, and ban users who download your game/app and don’t give any feedback. Some users were extremely helpful; GetJAR delivers a spreadsheet at the end of the test program with handsets and basic details about what worked and what didn’t. Occasionally, you’ll get the user’s contact info to follow up.
   
   This is a great way to go for free, coarse app testing. This is a terrible way to go for carrier-grade testing, because you won’t get enough detail, enough devices, you’re unlikely to get to repeat tests you need for bugfixes, and you have to entice users to download your app in the first place. But, if you’re a hobbyist with a budget of about $0, and are not taking this to a carrier/deck with serious porting requirements, it’s a very good (and free) first stab.

2. DeviceAnywhere. These guys have saved our low-end lives! Here’s how it works:
   
   MobileComplete (the company behind the operation) has a lab full of handsets that they let users lease remotely, by the hour. That is, they’ve hotwired a few hundred phones to the internet and let you control them remotely, and yes, these are the real devices being controlled over the real net! It’s a clever business and a neat trick — For those of us who can’t change even set our clocks for daylight savings time without electrocuting ourselves, it makes our head spin.
   
   I’ve found the testing process pretty smooth for most cases. Some devices aren’t always available (all their users are pulling from the same pool), but they open up pretty quickly. You will experience some problems with sound on some devices, and rapidly-updating features like video streaming will be laggy, but this will be a very good verification for 98% of what you’re up to.

3. Brute force. This means either:
   • Buying yourself every phone you need to test
   • Paying a porting house to do the work for you.

   Both of those options are things you want to give a lot of thought to before you head toward them. Porting shops will cost you at least a few hundred dollars per phone, and that can stack up after 50-60 handsets. My personal opinion is that if you’re in this business seriously, gaining the porting toolchain and expertise in-house will be very good for you in the long run.

All in all, porting is one thing that really hurts about mobile from an engineering point of view. It’s hard to estimate, resource-intensive, and very hard to manage and maintain. I’ve heard one developer recently estimate this as taking about 100%-150% of the initial development effort per project, which feels about right to me.

One last thing about this: Porting is a process, not a destination! You’ll inevitably be a huge success once you have widespread support for your snazzy new widget, and want to continue support for phones in the future months and years… So in spite of the urge to make spaghetti to get it to work, it’s even more critical than usual to keep the code and build process clean and repeatable!

One of the major differences between mobile code and desktop code is the severe number of SKUs (that is, variations on the build) that you have to support to make a dent in the customer base. Device support is a truly a long tail; after a couple hundred devices you’re getting close to about 80% of the market.

The biggest problem with J2ME in this case is that, unlike the desktop/PC world of Java, the VMs are built by MANY groups with different interpretations of the spec. Even if that weren’t the case, you’re trying to support some very different hardware characteristics: memory, screen resolution, network limitations, storage, etc.

All this means is that if you want your app to be a success, you’re going to adapt your code to support hundreds of phones.
Ugh.

What’s a developer to do? Here are a few basic points that work for me:

• Stay Organized. This should go without saying, but make a concerted effort to keep your builds clean, repeatable, and stowed away well. Someone will be asking you to send them the latest working build for their Motorola C290, and you should always have it handy. Do one of these:
  • Keep a folder heirarchy for your compiled files. Mine looks like:

    ->Builds
       -><app name>
           ->Version (e.g. 0.9.4)
               ->Screen Size (e.g. 176x220)
                    <standard build>
                   ->Custom build 1 (e.g. Nokia Series 60)
                       <build>
                   ->Custom build 2
                    ...
    

    For a solution like this, you’ll need a handy way to look up capabilities to help you dig out the right build for a handset.

  • Build/adapt a simple CMS. If you’ve got some time, this is the best option you’ve got (we’re working on a new one we may be able to share with you, stay tuned). You’ll thank yourself later.
• Write reusable code. Write with the intention of building what you write into a framework. OK, this will probably net you a few more KB in code, but it will save you from hurting yourself over and over again. Write as few new lines as you possibly can in a new app. Every time you fix a handset-specific bug, you’ve fixed it for everything you ever write in the future. In fact, if you’re really pathological, you can extract basically everything out of most apps into your own scripting/markup language and build your apps without writing a single line of real J2ME code. Sounds like a lot of work, not not nearly as much work as supporting two dozen codebases.
• Don’t use the APIs. Really. Do it yourself. The reasons for this tend to fall into two categories:
  • Lack of control. Generally, more advanced apps don’t make use of the standard javax.microedition.lcdui.Form drawing mode for professional apps, and definitely not for work-for-hire (Try explaining to a client that there’s no way you can change the title’s background, see how they react!).
  • Lack of bug-freeness. Think it’s handy that you can change the color of a font and write it to a virtual buffer? The original Nokia Series 60s didn’t think it was useful enough to support. Get in a habit of wincing every time you take advantage of another cool GameCanvas feature to your code. Sorry!
• Use a preprocessor. Java/OO purists are going to hate this, but the Java purists must hate quite a lot about J2ME. Use a preprocessor (like the one included with antenna) to build code that works against different handset sizes and capabilities.
• Use the JAD properties to tweak your code. This is far easier to manage than a total rebuild. Take advantage of the JAD properties for simple differences in builds, such as softkey codes, sound characteristics, timers adjusted to the speed of the handset, etc. Again, you’ll be sacrificing a little code size (and maybe a tiny bit of speed, in some cases), but seriously.
  

Quick — your eyes are rolling into the back of your head in your Introduction to Physics lecture hall, and you want to open up a game on your phone to kill the last 20 minutes of class. What’s the first thing you do when you flip open your phone?

If your answer was “make sure the sound is turned off so I don’t get caught / disturb the class, and sound on cell phone games suck anyway,” you’ll be happy to know that our first in-house game release, Mitosis, comes with that audio feature built in just for you; it has no sound. Here are a couple reasons why:

• Sound compatibility across handsets is more bug-prone than nearly any other porting problem, with the possible exception of networking code. As a small shop, we really only have about a half dozen phones that we can test on, which would severely limit us if we weren’t able to make a few assumptions. So why bother, if:
• As mentioned above, people really don’t care about sound in J2ME games! Of course in a sweeping generalization like that, there are bound to be people who disagree, but if you take a look at the polls on Midlet Review, game reviews anywhere, posts on the J2ME forums, you’ll find that people only ever complain about sound, never consider it positively. It’s dead last on everyone’s list. Anecdotally, nearly all of the people I see playing games on the subway, standing in line, in waiting rooms, do have the sound turned off.

That’s not to say that we’re a shop made of mimes and we’ll never do sound, and we’ll never try to do a really high-quality job on sound effects or music! For one thing, carriers may mandate that we add sound, or we may just miss it or feel like it’s very important to some particular game/app. For the time being, though, mum’s the word.

Caveat: Does not apply to iPod games! Those gadgets are obviously made for music, plus they’ve got no external speaker, so it’s a different, uh, game altogether.

I wanted to preface this entry with where we left off last year: Hope is not lost for the weekend-warrior mobile game developer! Here are some upsides to try to make you feel better after losing all will to carry on in our last episode:

Mobile games are lo-fidelity. The point here is that it doesn’t take a multimillion-dollar studio to make a game that can compete on a level playing field with everyone else in mobile (yet!).

Distribution isn’t as hard as it looks. Installation is automatic for J2ME apps, so no need to hassle with installers, and there are plenty of sites dedicated to getting a very international distribution for your games. (What kind of volume this actually provides, well, I’m figuring that out now. More on it later)

These are still pretty early days in mobile. The small guys haven’t quite been pushed off the playing field. Most mobile game still really suck (even from the big shops), so if you can do something that’s not a painful, crashing eyesore, you’ve got a decent shot.

Mitosis Launched!

So, since the dawn of time (about 7-8 months ago), Theoretic Labs has been about consulting and helping our friends out with their startups. On the “side” however, we have been working on a LOT of other projects. A few of them will come to fruition in the next few weeks, a few more will come in the months following.

I’m really very excited to announce the first of these, which is a simple game called Mitosis. The basic premise of the game is actually hard to articulate concisely in text, but really easy once you play it for a few seconds. You play the colored cell on the bottom of the screen, and shoot down the incoming cells from the top. You can only destroy the ones that match your color, and if you shoot one that does not match, you’ll swap colors with that cell. Think Zoop (which apparently nobody remembers but me), but simplified angles of play and a little more disgusting.

Why Can’t Anyone Play Mobile Games Online?

I was surprised to find that nearly nobody has a “try this game online for free” type feature for their cell phone games. It seems that this is because it’s really hard to find a decent MIDP 2.0 emulator that runs as an applet in the browser. The closest the real world has got is a handful of commercial products, and a broken open-source one called ME4SE. Well, it turns out that ME4SE isn’t too hard to fix, so I did it for the release of this game. Try it here! I’ll find a way to reapply these fixes back to the OS community after I clean it up a bit.

As a total aside: I know that Java has really lost this web-application technology fight to Ajax and Flash by squandering their early lead with the most spectacular array of installation and execution bugs seen since, well, forever. But please, Java, don’t go away on the web, I need you for an emulator! You can still fix the mess you’ve made.

Mobile Distribution

Our first distribution partners are the low-hanging fruit for indies, Greystripe’s super-clever AdWrap service and ClickGamer, for the more traditional purchased format. We have some people working on our behalf to get our products under the noses of the carriers as well, so with any luck that will pan out in short-ish order.

It’s probably too early to comment on how it’s been working out with these partners so I’ll hold my tongue, but rest assured that will come soon!

There are two major real-world reasons that the shoestring development world for mobiles is rather more difficult than traditional startup software development:

1. Broad phone support is nigh-impossible on a low budget.
   
   First of all, any BREW phones are right out the window: Qualcomm’s BREW platform is prohibitive to small fries, because they have a strict, expensive testing process for any app to be ‘approved’ for a particular handset/carrier, and customers with BREW phones can only install apps that their carrier makes available. This means you’ll need to recoup thousands of dollars in testing costs, as well as hire someone to manage carrier relationships (or do it yourself, you poor bugger). This model effectively cuts out everyone who’s working from a garage, or anyone who wants to give their app away for free.
   
   So that leaves us with Java ME as the other major platform. Java ME is, of course, sweetly intended to be a “Write Once, Run Anywhere” platform, but the KVM authors are all have their own eccentric disloyalty to the spec. This is the kind of thing that an outside observer wouldn’t believe to actually be a hindrance these days, I mean, it’s the year 2000 and hey, we’ve got flying cars! Alas, it’s truly amazing how cracked out some implementations can be. Coding errors aside, there are intrinsic differences in the handsets’ actual, physical capabilities anyhow: Screen dimensions, resource constraints, whether the phone has a Clear key, etc. These attributes must be addressed across thousands of devices regardless.
2. People buy new phones more often than they buy new underwear. This is really an amendment to the first statement, because it causes two sub-problems:
   1. Your app may not work on their new device, and therefore:
   2. You must test new devices eternally.

   You can still install a good chunk of shareware from the Windows 95 days on a new Vista install without much trouble. Even if you couldn’t, it seems that the replacement rate of phones far outstrips that of computers (though I don’t have numbers for that one… lazyweb?), so people are likely to be making better use of their older apps and older OSs on their computer than on the phone.
   

In this environment, it’s extremely unlikely that we’ll see something similar to the runaway successes that we’ve seen in shareware, selling apps and games that scarcely need updating in a decade to keep doing well for themselves. While the shareware makers on the desktop are kicking back on their porch sipping lemonade and responding to fanmail, all us mobile indies have got the Sisyphusean task of sneaking into the Cingular store to test our apps on the hottest new Nokia.

I hate to leave off here on a down note, but I will be back soon with the more interesting upsides, and how to get rid of at least some of the pain that plagues us.

I just gave a J2ME Game API workshop to Michael Sharon’s excellent Mobile Application Design class over at NYU’s ITP program. We walked through the best practices behind creating an old-skool side-scroller game. Talk notes and source from the game we put together can be found here.