Augmented Reality on iPhone: Marker Tracking Demo

I've been working on an augmented reality game app for iPhone recently. The player will be able to interact with the game by placing and moving specially designed marker cards in front of the iPhone's camera. The app analyzes video frames in real time and updates the scene accordingly. Here is demo of what I've got so far - marker tracking on iPhone 3GS.


The marker tracking algorithm is done by a slightedly modified version of NyARtoolkitCPP. Analyzing each frame takes < 55ms in average. The OpenGL ES rendering can achieve 30 frames per second. The current algorithm relies on thresholding so it is very sensitive to changing lighting condition. Also, template matching based identification requires matching against the whole pattern library for each detected marker. I'm seeking for better algorithms that use edge detecting and id recognition. StbTracker, the successor of ARToolkitPlus, looks ideal for devices such as iPhone but unfortunately it's not open source.

The most discussed topic by iPhone AR application developers today is real time video frame grabbing. There isn't an elegant way yet. For now, I'm using private APIs to repeatedly fetch the camera preview view. The good news is that we are going to have full access to still and video camera data in iPhone OS 4. A peek at the related APIs in the iPhone SDK 4 beta 2 further confirms that.

The OpenGL ES overlay content is displayed in a second UIWindow above the one containing the camera preview. This only point of dosing so is to get a clean preview image without the overlaying content. These ugly hacks will be all gone once we have iPhone OS 4.

Bypassing code signing to test your app on the iPhone device

Apple requires apps to be signed with a signing identity before you run them on device or distributing them to App Store. To get a valid signing identity (code signing certificate + private key), you must apply for Apple's iPhone developer program and pay them $99.

It is possible to run and test your app on your jailbroken iPhone without the need to pay (you'll still have to pay if you wish to publish your app on the App Store anyway). This process is often referred to as bypassing code signing.

Open Cydia on your iPhone, in the home page, scroll down and tap Bypassing Code Signature in the Developer section, you'll see instructions to bypass code signing. There are 3 options. Option #1 is what I tried. You can find detailed instructions for the other options on the internet.

Option #1: The verification check in the iPhone OS has been hacked so you can sign your code with any self-signing certificate and iPhone will let you pass. If you want the most integrated and smooth development experience with Xcode (Build and Go and debug on the device), this is the option for you.
You need to do two things.
The first is to bypass the Xcode verification.

1. Create a self signing certificate named iPhone Developer following the instructions by Apple Obtaning a Signing Identity .
   
2. in Xcode - > project - > edit project setting -> Add User - Defined Setting
   PROVISIONING_PROFILE_ALLOWED : NO
   PROVISIONING_PROFILE_REQUIRED : NO
   (there seems to be a way in which you don't have to add these settings on a per project basis but I haven't tried. see Developing Application for iPhone OS)
   
3. add in info.plist - > SignerIdentity : Apple iPhone OS Application Signing
   (from http://forums.macrumors.com/showthread.php?t=640341 )
   

Now you should be able to choose device targets and build your app. Try clean if you see code signing related errors.
You'll probably get a device verification error if your iPhone's Mobile Installation files haven't been patched yet. So the second thing to do is to compromise the verification on the iPhone.

1. add http://cydia.hackulo.us to your Cydia sources
2. install miPatch in Cydia
3. reboot your iPhone (I didn't reboot and it worked IIRC)

Done.

see Developing Application for iPhone OS for more information.

Option #2: Just build your app and scp it to your iPhone and run ldid -S yourapp. ldid (Link Identity Editor ) is a command line package available in cydia. see http://www.ipodtouchfans.com/forums/showthread.php?t=104884

Now I can debug on my device live - tap a button and stop at a breakpoint set in Xcode. I wouldn't want to write any iPhone app without a real Mac!

Legal Notice: I don't live in a country where this kind of notice is favoured. But for completeness, please know that bypassing Apple's restrictions is usually illegal. However, $99 is too much for a curious guy who just wants to test his Hello World.

Private methods in Objective-C

Private method in Objective-C

Objective-C doesn't have the concept of private methods although you can use @private, @public, @protected to specify visibility scope for variables of a class.
One way to simulate private methods is to use category.

// in MyClass.m, before the main @implementation block
@interface MyClass (Private)
- (void)privateMethod;
@end

// you won't get warnings if this block is missing
@implementation MyClass (Private)
- (void)privateMethod {
  //do sth...
}
@end

Note that it's a good practice to write an @implementation block for the category as soon as you finish the category @interface declaration. If the @implementation block for the category is missing, the compiler won't warn you about any missing method implementations.

// in MyClass.m, before the main @implementation block
@interface MyClass ()
- (void)privateMethod;
@end

@implementation MyClass
- (void)privateMethod {
  //do sth...
}
@end

You can also leave the category name blank to use an extension. If you do so, you must implement the methods in the main @implementation block. The compiler will always issue warnings when implementations for the methods are missing.

References

1. http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/Articles/chapter_6_section_1.html#//apple_ref/doc/uid/TP30001163-CH20-SW1
2. http://www.otierney.net/objective-c.html#categories
3. Best way to define private methods for a class in Objective-C