{"id":98,"date":"2010-11-22T18:51:07","date_gmt":"2010-11-23T02:51:07","guid":{"rendered":"http:\/\/bynomial.com\/blog\/?p=98"},"modified":"2010-11-22T18:51:07","modified_gmt":"2010-11-23T02:51:07","slug":"beta-testing","status":"publish","type":"post","link":"http:\/\/bynomial.com\/blog\/?p=98","title":{"rendered":"Beta testing"},"content":{"rendered":"

This post gives a quick introduction to beta testing for iOS developers. \u00c2\u00a0Specifically, I’ll walk through the major steps needed to legitimately send out a working copy of your app to testers without going through the app store – a process known as ad hoc distribution<\/em>.<\/p>\n

Why beta test?<\/h2>\n

All programmers become attached to their code over time. \u00c2\u00a0It’s just being human. \u00c2\u00a0You think to yourself, “why do I need to spend time sharing this with people who don’t know anything about my app?” \u00c2\u00a0You can just test it yourself, faster and cheaper!<\/p>\n

Right?<\/p>\n

The problem is, it’s really hard<\/em> for you to treat your app like you haven’t built it yourself. \u00c2\u00a0There’s a tendency to only test what you just wrote, and to play with things that you know will work. \u00c2\u00a0I can’t tell you how many times I’ve done this – gotten so used to using the app the way I know it works, that I forget about all the ways it can break.<\/p>\n

So there is a huge value to a fresh pair of eyes. \u00c2\u00a0You may like to pretend that all customers will perceive the app exactly the way you do, but that isn’t reality. \u00c2\u00a0What a new user thinks and does with your app is not obvious, and this is why we test.<\/p>\n

Getting testers’ UDIDs<\/h2>\n

A UDID is a unique identifier string per device (iPhone, iPad, or iPod touch). \u00c2\u00a0Your Apple developer program account will usually give you 100 device slots that you can register. \u00c2\u00a0Basically, by requiring you to register devices, Apple can avoid people selling their apps directly through this system. \u00c2\u00a0You can recycle your slots, but only very slowly, as there is a 1-year waiting period to recycle any given slot (see the last two paragraphs of this page<\/a>).<\/p>\n

A user can find their UDID a couple ways:<\/p>\n

    \n
  1. They can search for “udid” in the app store, and download one of those apps. \u00c2\u00a0These generally give the user the option to email their UDID directly to you. \u00c2\u00a0Although I think option 2 is faster, some users might feel like this is easier, since there seems to be some psychological “work” associated with plugging in an iPad\/iPhone.<\/li>\n
  2. They can plug in their iPads, open iTunes, and click on “Joe’s iPad” (whatever the name is) under Devices in the left-hand panel. \u00c2\u00a0Next click on the serial number of your device, and it switches to display the UDID.<\/li>\n<\/ol>\n

    Here’s a link to instructions on finding a UDID<\/a> – useful for emailing to new testers.<\/p>\n

    Building an ipa file<\/h2>\n

    This is the most fun part.<\/p>\n

    An ipa file is a file testers can install in two steps:<\/p>\n

      \n
    1. Double-click on the ipa file, which loads it into iTunes for them; and then<\/li>\n
    2. Sync their device with iTunes.<\/li>\n<\/ol>\n

      In some cases, they might have to check the box next to your app’s name under the Apps tab amongst the list of their sync settings for their device.<\/p>\n

      Generating that ipa file, however, is a bit of work. \u00c2\u00a0Here’s an overview of what you need to do.<\/p>\n

      These steps are for the Provisioning Portal, available to all Apple developer program members through developer.apple.com. \u00c2\u00a0I’m assuming you’ve already done some usually-previously-required steps like creating your certificates and an app ID. \u00c2\u00a0If not, see the links below.<\/p>\n

        \n
      1. In the Devices section, add the tester’s device using the UDID they sent you.<\/li>\n
      2. In the Provisioning section, click on the Distribution tab, and either add a new profile (if this is your first tester), or click Edit next to the existing profile for your app. \u00c2\u00a0This will be an ad hoc distribution profile, and the other options are straightforward – always select all the devices you want to be able to run the app on.<\/li>\n
      3. Download the provisioning profile from the previous step, also from within the Provisioning section.<\/li>\n
      4. In Xcode, create a build configuration called “Ad Hoc Distribution” (the name does not affect the build, so you can change this string to suit your needs). \u00c2\u00a0Modify the build settings to sign the binary using the new profile (iPhone Distribution, under Code Signing Identity).<\/li>\n
      5. In the build configuration drop-down (always visible in the top-left of the main Xcode window), choose Device, and then Ad Hoc Distribution.<\/li>\n
      6. From the Build menu, choose Build and Archive.<\/li>\n
      7. From Xcode’s Organizer window (ctrl-cmd-O), click on your app’s name under Archived Applications on the left.<\/li>\n
      8. Choose the latest archived version (if this is your first time through, there is no choice here), click on Share…, and then Save to Disk…<\/li>\n
      9. Save the file where-ever you like, and email it to your testers. \u00c2\u00a0Done!<\/li>\n<\/ol>\n

        I’ve actually been relatively terse in the above list. \u00c2\u00a0There are two much more detailed lists you can walk through if this is all new and scary. \u00c2\u00a0The first is from Apple directly. \u00c2\u00a0Once you’re logged in to developer.apple.com\/ios, go to the Provisioning Portal, click on Distribution on the left, and read through each of the items starting with “Obtaining your iOS Distribution Certificate” – each of those items is a link that expands when clicked (link<\/a>). \u00c2\u00a0Those steps are official and comprehensive.<\/p>\n

        There’s also a nice stackoverflow answer with a detailed walkthrough posted by Arne Evertsson<\/a>.<\/p>\n

        Tips for getting feedback<\/h2>\n

        Actually getting useful feedback from testers is harder than you may think. \u00c2\u00a0Expect a significant portion of testers to not follow through. \u00c2\u00a0Those who do send feedback may easily send non-useful tips without careful forethought. \u00c2\u00a0A few things can go wrong<\/strong>:<\/p>\n