Date: Fri, 30 Aug 2002 09:16:12 -0700 Subject: Re: Managing connections From: Jens Alfke On Friday, August 30, 2002, at 03:41 AM, Dustin Mierau wrote: > Does anyone here have a good solution for this? Anyone have a good > idea what iChat or the demoed iTunes does? iChat registers a DNS 'TXT' record for its service, the contents of which contain a number of key/value pairs giving info like your real name, online status, status message, email address, AIM ID, etc. Unfortunately in Jaguar the contents of the record are pretty thoroughly obfuscated, for reasons of history and deadline pressure; I'd like to fix that at some future point when we rev the protocol. The good news is that the natural format of the TXT record is a list of key/value pairs, so there's a natural data format for it already. When iChat learns of a new client on the net, it performs a query on its iChat service's TXT record. It leaves the query open so as to subscribe to any future updates. Since all the DNS queries are multicast over the subnet, the amount of traffic is pretty low. An additional bit of information you might or might not find useful is that iChat stores the buddy picture (JPEG format) in yet another DNS record, a 'NUL' record. The reason is that it's relatively large and doesn't change often. The TXT record contains the MD5 hash of the picture. Then a client can check the hash, see if it already has a cached picture with that hash, and only needs to retrieve the NUL record if it doesn't have the picture already. [Most of this clever stuff is courtesy of Stuart Cheshire and Jeremy Wyld. I just contributed a few ideas here and there.] I have no idea how the iTunes demo works behind the scenes. Nor do I have any idea or when or whether that feature will appear in a released version of iTunes. I know I want it though :) _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Mon, 30 Sep 2002 23:13:28 +0200 From: Pierre Baillet Subject: Re: [OT]Re: Service announcement? Jens, thanks for these details, i already had seen the TXT record and was wondering what nice things i would find inside when i'll get the time to base64 decode this stuff. I'll try and setup a fake iChat client running on linux that will use Rendez-Vous to self-announce. I'll try and use the sample code available (it seems to me that the libinfo doesn't compile out of the box on a linux box, but i haven't had the time to look deeper)... I'm quite interested in RdzVous in general and iChat in particular as this is, in my opinion, the best way to get in other platforms (even if Zeroconf looks nice, i'm not sure i'd be able to convince my sysadmin to drop DHCP for Zeroconf anytime sooner)... Whenever i get something to work, i'll notice you. Cheers, On Mon, Sep 30, 2002, Jens Alfke wrote: > > On Monday, September 30, 2002, at 11:44 AM, Pierre Baillet wrote: > > > > this post is probably a bit out-topic but i was wondering if any of > > you > > know wheter Apple is willing to open a bit its iChat specs. > > > > It's not really off-topic for this list. > > iChat uses a service named _ichat. The presence information (name, > avail/idle/away, email, etc.) is stored in the DNS TXT record > associated with the service, and the picture is stored in JPEG format > in the NUL record. > > The problem is that, for both historical and schedule reasons, the data > in the TXT record isn't stored in a very sensible way. It's > fundamentally a list of key-value string pairs, but those get first > packed into a binary encoding which is then stored as a CDATA element > in an XML plist that goes into the TXT. Yuck. > > (See, before Rendezvous, iChat used its own custom protocol to > multicast presence, and sent the binary data as the contents of a > multicast packet. When we converted to Rendezvous, the fastest thing to > do was just keep the binary data but encode it into a form that could > be stored in a TXT record.) > > It would make much, much more sense to store the key-value pairs > directly in the Pascal-style strings that comprise the TXT record; > that's how TXT records are supposed to be used. Sometime soon I'm going > to rev iChat to do this, but I'm not sure when that will ship. (And > backwards compatibility will be needed for a while.) > > Instant messaging and file transfer are done by opening a TCP socket to > the other machine and using a variant of the Jabber protocol with all > the login steps removed. Only elements are used, since of > course presence is handled via Rendezvous. The port number to connect > to is found in the value of the 'port.p2pj' property in the destination > machine's property list. > > > _____________________________________ > Jens Alfke ? iChat Engineering -- Pierre Baillet ********************************************************************************************* ********************************************************************************************* Date: Mon, 30 Sep 2002 11:12:13 -0700 Subject: Re: Service announcement? From: Jens Alfke On Monday, September 30, 2002, at 10:11 AM, Reed Hedges wrote: > I've read through Apple's documentation and parts of the RFCs, but > haven't seen anything like this yet. Is there any mechanism for a > service to announce its presence (broadcast or multicast) to > interested/"listening" clients? (e.g. when it starts up) Yes. When a client browses for instances of a service, it can "leave the browse open" which means it will continue to receive notifications as other instances of the service on the network register and unregister. That's what iChat does. _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Fri, 18 Oct 2002 09:45:53 -0700 Subject: Re: getting TXT record in the mDNSResponder code From: Jens Alfke On Thursday, October 17, 2002, at 05:00 PM, Reed Hedges wrote: > I've been reading through messing around with the example > mDNSResponder code from developer.apple.com, but can't seem to figure > out how to get the associated TXT record of the service (the example > Responder program). Any hints? In a nutshell, you construct a hostname by concatenating the service's name, type and domain; then do a regular DNS query on that hostname. Here's some sample code. I implemented this in a category on NSNetService, so 'self' refers to the NSNetService instance. - (dns_reply_t*) _queryDNSRecord: (int)recordType // caller must free reply { dns_question_t question; dns_reply_t* reply; NSString* hostName = [NSString stringWithFormat: @"%@.%@%@", [self name],[self type],[self domain]]; question.name = (char*) [hostName UTF8String]; question.class = DNS_CLASS_IN; question.type = recordType; static sdns_handle_t *sDNS; if( ! sDNS ) { sDNS = sdns_open(); if( ! sDNS ) return nil; // Using a static isn't thread-safe, but then there are currently // no thread-safe DNS resolution APIs, anyway. } // Perform the query return sdns_query(sDNS, &question); } To get the TXT record, you pass DNS_TYPE_TXT as the recordType. The number of answers will be in reply->header->ancount. The i'th TXT record is in reply->answer[i]->data.TXT ... this is a struct containing a count and an array of char*s. _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Wed, 20 Nov 2002 11:14:00 -0800 Subject: Re: SetProtocolSpecificInformation ? From: Jens Alfke On Wednesday, November 20, 2002, at 10:34 AM, I wrote: > * setProtocolSpecificInformation does not (I'm told) push anything > down to mDNSResponder, making it basically useless. I'm not sure what > the bug number is for this issue -- I will make sure it's entered and > prioritized. I got this halfway wrong. This method does correctly set the initial contents of the TXT record, provided you call it before you publish your service. It does not, however, update the TXT record after the service has been published. Currently, as I said, there is no way to update the TXT record without using private APIs. The closest approximation you can use today is to delete your service and then publish a new service with an updated TXT record. _____________________________________ Jens Alfke iChat Engineering Apple Computer "Bother," said Pooh. ********************************************************************************************* ********************************************************************************************* Date: Wed, 20 Nov 2002 10:34:54 -0800 Subject: Re: SetProtocolSpecificInformation ? From: Jens Alfke On Tuesday, November 19, 2002, at 08:19 PM, Alec Carlson wrote: > I have a few questions about the NSNetService > setProtocolSpecificInformation > and protocolSpecificInformation methods. The documentation says that > these > methods are legacy functions for supplying additional info when > registering > a service and then retrieving that info when browsing for a service. Man, this is getting really spooky! I just discovered those methods yesterday, and someone else inside Apple started asking about them yesterday, and now this. Must be the meteor shower that set off all this activity. > 1. Is there a relationship between these methods and the DNS TXT > records ? Yes. These methods should really be named "contentsOfTXTRecord" or something like that. At the time these methods were implemented and commented, it was thought that TXT records were not of general interest and would only be needed for a couple of legacy protocols; but then later it turned out that they are actually quite useful for service metadata. > 1. Do these methods work in 10.2.1 ? I can > setProtocolSpecificInformation > and then publish my service but I don't get anything back in the > netServiceBrowser:didFindService: delegate method when I send > protocolSpecificInformation to the NSNetService * argument. Summary: The getter halfway works, the setter doesn't work at all. * protocolSpecificInformation will return the initial contents of the TXT record, but there is unfortunately a bug that causes it to ignore any subsequent updates. This is OK for services that use a fixed TXT record, but not for those that update the TXT record on the fly (like iChat). The internal Radar bug number for this is 3105519. There is a possibility it will be fixed in an upcoming Software Update. * setProtocolSpecificInformation does not (I'm told) push anything down to mDNSResponder, making it basically useless. I'm not sure what the bug number is for this issue -- I will make sure it's entered and prioritized. iChat uses some lower level DNS APIs to get around both these problems, but I believe those are private, so I can't reveal them here. :-( We're definitely going through some growing pains in this area. The ability to use TXT records for dynamically-updated metadata was added very late in the game (partly to ensure iChat would be able to use Rendezvous instead of the custom multicast-based protocol it used to use) and not all of the kinks got worked out by the time Jaguar was shipped. I'm crossing my fingers that the fixes can get pushed out to the world pretty soon. _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Tue, 10 Dec 2002 14:35:00 -0800 Subject: Re: Problem with browse ? From: Jens Alfke On Tuesday, December 10, 2002, at 02:19 PM, Smith Kennedy wrote: > Perhaps I should be doing the setDelegate: and resolve: call somewhere > else? iChat does the setDelegate: immediately but defers the resolve calls. It uses a temporary array "_servicesAdded" to stuff the new services into until it has time to resolve them, then adds them to the main "_services" array: - (void)netServiceBrowser:(NSNetServiceBrowser *)aNetServiceBrowser didFindService:(NSNetService *)aNetService moreComing:(BOOL)moreComing { [_servicesAdded addObject:aNetService]; [aNetService setDelegate:self]; if( ! moreComing ) { [_servicesAdded makeObjectsPerformSelector: @selector(resolve) withObject: nil]; [_services addObjectsFromArray: _servicesAdded]; [_servicesAdded removeAllObjects]; } } It's been a few months so I don't remember exactly, but I think my experience was that, for whatever reason, the resolve calls slowed things down. _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Wed, 11 Dec 2002 09:24:40 -0800 Subject: Re: SetProtocolSpecificInformation ? From: Jens Alfke On Wednesday, December 11, 2002, at 08:32 AM, Smith Kennedy wrote: > Sorry to bring this up again, but I can't seem to figure out how to > get the protocolSpecificInformation: method to actually give me the > TXT information that my device has provided to its responder. It should be available as soon as your netServiceDidResolveAddress: callback is invoked. The current iChat builds do it this way and it seems reliable. Of course you have to call resolve on the service beforehand to get this callback. The bug you mentioned only affects services that update their TXT record after first publishing it; the high level APIs won't ever receive the update. But the initial TXT record from the time the client first resolves the service should be available. _____________________________________ Jens Alfke iChat Engineering Apple Computer ********************************************************************************************* ********************************************************************************************* Date: Thu, 6 Feb 2003 09:20:11 -0800 Subject: Re: Rendezvous trademarks and spec [ was Re: Rendezvous Kitchen Invitation ] From: Jens Alfke On Wednesday, February 5, 2003, at 10:07 PM, Eric Seidel wrote: > I would strongly agree with John's suggestion, that a standard form > for DNS Text records needs to be laid out. It seems as though at > least a suggestion has already been offered for HTTP and the printer > spec is supposedly "on it's way," but I would like to see a grammar > specified for *all* text messages if possible... maybe that's just too > much specification? (maybe it's already been done...) It has: This defines a syntax for encoding a property/value list in the TXT record. You really can't get much more specific than that and remain general the next level of detail would be schema definitions for individual service types like printers, file servers, webcams, doorknobs ... (iChat's TXT record currently does not follow this spec, unfortunately, but the next release will; and I'll write up at least an informal document describing the schema.) _____________________________________ Jens Alfke iChat Engineering Apple Computer