Navigation
« Safari: Keyboard shortcut for opening current page in Google Chrome | Main | Asynchronous, lazy initialization with synchronous accessor »
Monday
May142012

Peer to peer synching with TouchDB

Updated 2012-06-05: Incorporated Jens’s suggestions and corrections.

TouchDB is a lean CouchDB-compatible database framework that can be embedded in iOS applications (or more generally, mobile or desktop applictions but this post is about iOS). Jens Alfke, its author, describes it this way: “If CouchDB is MySQL, then TouchDB is SQLite.” The project is available on github.

TouchDB is CouchDB-compatible with respect to its replication API when initiated on the device against another ‘regular’ CouchDB. You can create push and pull replication tasks on TouchDB. However, out of the box, TouchDB does not offer an HTTP interface for other TouchDB (or CouchDB) instances to connect to. This means that initially, you are limited to a “star” topology with a regular CouchDB at its center and iOS devices with TouchDB connecting to it as a synchronization hub.

However, with a little extra work, it is quite easy to turn this into a peer to peer setup, thanks to the Listener framework Jens has included in TouchDB.

In order to get this to work, you first need to build the listener framework. To do so, clone the git repository, pull the submodules and build the “Listener iOS Framework” target as follows:

git clone https://github.com/couchbaselabs/TouchDB-iOS
cd TouchDB-iOS
git submodule init
git submodule update
xcodebuild -target "Listener iOS Framework"
open build/Release-ios-universal

The open command will open a Finder window with the framework, which you need to add to your existing project.

After you have done that, you need to start the listener. One place where you might want to do that could be application:didFinishLaunchingWithOptions:. Add the following code to start the listener:

CouchTouchDBServer *server = [CouchTouchDBServer sharedInstance];
[server tellTDServer:^(TDServer *tdServer) {
  NSLog(@"Starting listener");
  _listener = [[TDListener alloc] initWithTDServer:tdServer port:59840];
  [_listener start];
}];

NB: Make sure _listener is retained outside the block and lives on, otherwise your listener goes out of scope and stops listening immediately. And as you can tell from the unbalanced alloc message: these code snippets are assuming ARC.

This is basically all you need to do to connect to your TouchDB instance via HTTP. For example, you could use curl to query it for documents. However, peer to peer benefits from advertising and discovering your service via Bonjour and the rest of this article briefly describes how to achieve this.

First off the advertising part. Add the following to a startup section of your application, for example right after creating the listener:

UIDevice *device = [UIDevice currentDevice];
self.netService = [[NSNetService alloc] initWithDomain:@"local" type:@"_myapp._tcp" name:device.name port:59840];
NSData *data = [NSNetService dataFromTXTRecordDictionary:[NSDictionary dictionaryWithObject:conf.localDbname forKey:@"path"]];
[self.netService setTXTRecordData:data];
[self.netService publish];

Replace “myapp” and 59840 with values of your choosing and note that it is advisable to choose a better service name than simply the device name as I have done in this example.

For discovery, you create an NSNetServiceBrowser and search for hosts of your service type:

self.browser = [[NSNetServiceBrowser alloc] init];
self.browser.delegate = self;
[self.browser searchForServicesOfType:@"_myapp._tcp" inDomain:@"local"];

You will be notified of any matches by implementing the following NSNetServiceBrowserDelegate protocol callback:

- (void)netServiceBrowser:(NSNetServiceBrowser *)netServiceBrowser didFindService:(NSNetService *)netService moreComing:(BOOL)moreServicesComing
{
  [self.services addObject:service];
  if (! moreServiceComing) {
    [self.tableView reloadData];
  }
}

In this example, I’ve added the service to an array. This could be an array that is driving a UITableView for example. (There’s a complete bonjour browser example available on the iOS Dev Center that includes a browsing UI and discovery and resolution for bonjour services that these code examples are based on.)

As Jens Alfke correctly points out in the comments, it is important to implement the companion method netServiceBrowser:didRemoveService:moreComing: as well in order to remove a service from the list when it disappears:

- (void)netServiceBrowser:(NSNetServiceBrowser *)netServiceBrowser didRemoveService:(NSNetService *)netService moreComing:(BOOL)moreServicesComing
{
  [self.service removeObject:service];
  if (! moreServiceComing) {
    [self.tableView reloadData];
  }
}

Once a service is selected in this table view, we try to resolve it:

- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath
{
  NSNetService *service = [self.services objectAtIndex:indexPath.row];
  [service setDelegate:self];
  [service resolveWithTimeout:0.0];
}

Finally, we implement the relevant part of the NSNetServiceDelegate protocol to handle the resolved address. This is where we would then update the sync settings for our app, which is encapsulated in the [self updateSync:url] in this example. This would be the same updateSync: present in the TouchDB example apps.

- (void)netServiceDidResolveAddress:(NSNetService *)sender {
  // Construct the URL including the port number
  // Also use the path, username and password fields that can be in the TXT record
  NSDictionary* dict = [NSNetService dictionaryFromTXTRecordData:[service TXTRecordData]];
  NSString *host = [service hostName];
  NSString* user = [self copyStringFromTXTDict:dict which:@"u"];
  NSString* pass = [self copyStringFromTXTDict:dict which:@"p"];
  NSString* portStr = @"";
	
  // Note that [NSNetService port:] returns an NSInteger in host byte order
  NSInteger port = [service port];
  if (port != 0 && port != 80) {
    portStr = [[NSString alloc] initWithFormat:@":%d",port];
  }

  NSString* path = [self copyStringFromTXTDict:dict which:@"path"];
  if (!path || [path length]==0) {
    path = [[NSString alloc] initWithString:@"/"];
  } else if (![[path substringToIndex:1] isEqual:@"/"]) {
    NSString *tempPath = [[NSString alloc] initWithFormat:@"/%@",path];
    path = tempPath;
  }
	
  NSString *ipAddress = nil;
  for (NSData* data in [service addresses]) {
    char addressBuffer[100];
    struct sockaddr_in* socketAddress = (struct sockaddr_in*) [data bytes];
    int sockFamily = socketAddress->sin_family;
    if (sockFamily == AF_INET /* || sockFamily == AF_INET6 */) {
      const char* addressStr = inet_ntop(sockFamily,
                                         &(socketAddress->sin_addr), addressBuffer,
                                         sizeof(addressBuffer));
      int port = ntohs(socketAddress->sin_port);
      if (addressStr && port) {
        NSLog(@"Found service at %s:%d", addressStr, port);
        ipAddress = [NSString stringWithCString:addressStr encoding:NSASCIIStringEncoding];
      }
    }
  }

  NSString* url = [[NSString alloc] initWithFormat:@"http://%@%@%@%@%@%@%@",
                   user?user:@"",
                   pass?@":":@"",
                   pass?pass:@"",
                   (user||pass)?@"@":@"",
                   ipAddress?ipAddress:host,
                   portStr,
                   path];
	
  NSLog(@"service: %@", service);
  NSLog(@"url: %@", url);
  [self updateSyncURL:url];
}

The method above references one simple helper method to access bonjour data from the service:

- (NSString *)copyStringFromTXTDict:(NSDictionary *)dict which:(NSString*)which {
  // Helper for getting information from the TXT data
  NSData* data = [dict objectForKey:which];
  NSString *resultString = nil;
  if (data) {
    resultString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
  }
  return resultString;
}

As mentioned above, this bonjour code is mostly from the Apple example code of BonjourWeb but it required some minor changes. I’ve added the path component to broadcast which database to replicate with. I’ve also commented out the AF_INET6 socket family part, because it did not work with the replication and for the same reason I’m using the IP address for the URL rather than the clear name, because this also did not yield a working connection.

Hopefully this post will help people getting started with TouchDB peer-to-peer replication!

Reader Comments (4)

Great post, Sven! Although at some point I hope to obsolete it by adding Bonjour support to the TouchDBListener :-)

A couple of little issues:

* In addition to didFindService:, the app also needs to implement didRemoveService: as well. I know your example isn't intended to be complete, but I think it's important to point this out because if someone forgets to implement the latter method, the app will _appear_ to work, but it won't ever remove services from the list if they go offline, so after a while the UI will be inaccurate.

* When creating the NSNetService you may want to use a more specific name, typically a user-name, instead of the device's name. In the worst case, the user might not want their real name broadcast on the LAN.

* There's a typo in one of the samples, a few bogus characters right after "searchForServicesOfType:".

* I'm guessing this code assumes ARC. It would be a good idea to point this out in the beginning, otherwise there are a couple of reference leaks that will result if someone uses this in a non-ARC app.

May 15, 2012 | Unregistered CommenterJens Alfke

(Sorry for the late action I only just noticed your comment – I need to double check the notification settings. New site woes ;)

Thanks for your kind words, suggestions, and corrections, Jens. I'll update the post asap. As to your wanting to support Bonjour directly from within TouchDB: if there's interest, maybe I could just pull the basics out of my example and put it into something reusable. (I don't want to promise too much as I'll be very busy soon.)

May 23, 2012 | Registered CommenterSven A. Schmidt

Hi Sven!
Thanx for great article. Can you tell me if TouchDB is compatible with Android OS?

May 25, 2012 | Unregistered CommenterDarren Kelth

Darren, glad you liked the article! There's TouchDB-Android on github but I'm not sure how far along it is in porting TouchDB for iOS but it's definitely the place to start.

May 25, 2012 | Registered CommenterSven A. Schmidt
Comments for this entry have been disabled. Additional comments may not be added to this entry at this time.