mikeash.com pyblog/friday-qa-2009-01-16.html commentshttp://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsmikeash.com Recent CommentsThu, 28 Mar 2024 18:46:24 GMTPyRSS2Gen-1.0.0http://blogs.law.harvard.edu/tech/rssbil - 2016-03-03 21:07:36http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsComments here sound like xpc is the way to go for IPCe14de6c9639a973d925f96177eb21e72Thu, 03 Mar 2016 21:07:36 GMTmikeash - 2009-01-20 00:48:01http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsThanks for the comment. I'm aware that the comment system falls down with non-ASCII characters, but the python/MySQL combination has defeated my every attempt to fix the problem. Maybe I'll try again soon, in the meantime, sorry, only ASCII accepted here.bab909e0453f851ed620c9eaa9d179f7Tue, 20 Jan 2009 00:48:01 GMTDamien Sorresso - 2009-01-19 17:44:36http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsIt's worth noting that anything dealing with Mach under the covers will invariably have to deal with Mach bootstraps. A naive use of CFMessagePort is okay in most cases, but on Leopard, each user has his own Mach bootstrap, and each console session (be it SSH or the GUI console) has its own Mach bootstrap underneath there. Therefore, you cannot expect, having registered a DO server in the GUI (aka Aqua) bootstrap, that a client in the SSH (aka StandardIO) bootstrap will be able to look up the name. <br /> <br />I'd highly recommend checking out the Daemonomicon for further details. <br /> <br /><a href="http://developer.apple.com/technotes/tn2005/tn2083.html">http://developer.apple.com/technotes/tn2005/tn2083.html</a> <br /> <br />Also, if you're going to use Mach, don't bother using the mach_* APIs. Learn to love MiG. Even XNU exports its interfaces via MiG, and there are very few clients of the mach_* APIs. MiG has evolved over years and years to encompass various corner cases in its generated code that would be very difficult for just about anyone to deal with their first time through. And when using MiG, keep in mind that, to do really interesting stuff, you're going to have to delve into the implementation at some point. MiG doesn't do a very good job of maintaining a wall of separation between interface and implementation, so don't feel too dirty about it. <br /> <br />More on MiG... <br /> <br /><a href="ftp://ftp.cs.cmu.edu/project/mach/doc/unpublished/mig.ps">ftp://ftp.cs.cmu.edu/project/mach/doc/unpublished/mig.ps</a> <br /> <br />Also, for simple, synchronous communication over a socket, you can also check out BetterAuthorizationSample. <br /> <br /><a href="http://developer.apple.com/samplecode/BetterAuthorizationSample/index.html">http://developer.apple.com/samplecode/BetterAuthorizationSample/index.html</a> <br /> <br />It has a nice library for sending commands to a server over the wire as serialized CFDictionary's, making it very easy to create a rich, synchronous protocol for client/server communication. For asynchronous communication, you'd have to look elsewhere though. I had written a library to do what BetterAuthorizationSample's did using asynchronous means by way of CFRunLoop's, but I can't release it for various reasons. <br /> <br />It's also worth pointing out the most basic difference between Mach and Unix domain sockets. Mach is a message-based protocol, while sockets are streams. So when you get a Mach message, it's guaranteed that it's the whole message. With sockets, you have to work out things like dealing with incomplete messages, whether the kernel buffer has been filled, etc. Mach takes care of all that for you. The downside is the the loss of portability, lack of documentation and misunderstandings that come from poor terminology choices on the parts of the original Mach developers. But for local interprocess communication, Mach is (in my opinion) the better option once you have mastered it. <br /> <br />The deal-breaker for Mach is if you need to pass file descriptors between processes. Mac OS X does not currently support passing file descriptors via Mach messages. Nor can it pass Mach port rights over sockets. Those two worlds are separate in this respect. <br /> <br />I'd also like to note that IPC is particularly important on Mac OS X, especially on the daemon side of things, because it's the mechanism for expressing dependencies when writing a launchd daemon or agent. <br /> <br />By the way, Mike, your comment system doesn't handle umlauts very well. :)6461c30a068f961fc306159ba8fad5c6Mon, 19 Jan 2009 17:44:36 GMTJamie Hope - 2009-01-19 13:30:23http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#comments"DO is really cool technology and it's the sort of thing that tends to blow people's minds when they come to Objective-C from lesser languages such as Java or C++." <br /> <br />I've never used DO, but your description of it sounds very much like Java's Remote Method Invocation or Jini. Except that Jini *does* allow you to specify the IPC mechanism to allow for encryption or whatever.c655992f76f33f9265bc66139997ecfcMon, 19 Jan 2009 13:30:23 GMThas - 2009-01-18 09:28:16http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsmikeash: "I strongly disagree and think it's completely fair. AppleScript is so bad that I went as far as writing my own Apple Event builder library (AEVTBuilder) to avoid using it in one application." <br /> <br />And I wrote appscript so that I could control AppleScriptable applications from Python (and later Ruby and ObjC). More precisely, I spent the first year trying to make appscript work <i>better</i> than AppleScript... and then I then spent the next four years just trying to make the damn thing work <i>as well as</i> AppleScript. <br /> <br />Along the way, I learnt a valuable lesson: you cannot out-AppleScript AppleScript, and thinking you can just gets your butt kicked. It's pretty much impossible to create a high-level Apple event bridge that works better than AppleScript does - even if you get the basics right (which early versions of appscript didn't, and most others never have), you still have a ton of problems to deal with: almost every application available today is developed and [hopefully] tested against AppleScript, with the result that many of AppleScript's own unique quirks are relied on by those applications. For a high-level bridge to work really well, it not only needs to speak Apple events fluently, it also needs to replicate AppleScript's own particular way of speaking them. <br /> <br /> <br />"I don't know how you're getting anything to work flawlessly in AppleScript but whatever secret you have, I'd like to know what it is." <br /> <br />No secret, alas (otherwise I'd patent it and be rich). Just lots of hard work. Although it's still less work than is needed to script applications equally well from Python/Ruby/ObjC, since at least with AppleScript you have a very well established community and lots of existing example scripts and documentation to help you. (And believe me, it took me several years and much poking from the likes of Matt Neuburg to suck it up and admit this.) <br /> <br /> <br />"My experience with AppleScript is that it's a frustrating exercise in trial and error for even the most trivial of tasks." <br /> <br />Oh, I quite agree. AppleScript syntax causes real problems: in order to provide high-level readability (i.e. anyone can look at existing AppleScript code and get a fair idea of <i>what</i> it does) it sacrifices low-level semantics (i.e. it's a real bear to understand <i>how</i> it does it). e.g. I recently discussed this at <a href="http://macscripter.net/viewtopic.php?pid=108997">http://macscripter.net/viewtopic.php?pid=108997</a> <br /> <br />However, the majority of problems in figuring out how to get any particular scriptable application to do what you want it to do are due to the schlonky implementations and crappy documentation provided by individual application developers. And these deficiencies affect <i>all</i> application scripters equally; so regardless of what language you use, you'll find it "a frustrating exercise in trial and error". <br /> <br />... <br /> <br />Like I say, I think Apple should replace the whole lot with a conventional, dumb-as-rocks object-oriented IPC system, as that's what most users - and most Apple engineers - seem to want anyway, but realistically I don't expect them to do so any time soon as that would be an outright admission that they screwed up the first time around. <br /> <br />In the meantime, sure, blame AppleScript for mistakes it is responsible for (e.g. syntax soup). However, blaming it for everybody else's screw-ups as well - crappy application scripting interfaces and documentation; faulty Apple event bridges, etc. - only serves to let the real culprits off the hook. And that ultimately doesn't serve anyone's interests. <br />98b548c5818c59ce974ebcc9af517e85Sun, 18 Jan 2009 09:28:16 GMTmikeash - 2009-01-18 07:52:37http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsI see the asshole crowd is out now. Thanks very much, 74.79.232.49 for that delightful security test, but I'm afraid I'm going to have to delete both the attempted script injection and the very long string of nonsense now.e9c24ddf18d0f66b5d83852a52f3f916Sun, 18 Jan 2009 07:52:37 GMTmikeash - 2009-01-18 05:16:42http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#comments<b>Erik:</b> Thanks for the suggestion. To go over it quickly here, DO tends to be unreliable under really heavy use, with complicated objects, or when used remotely. It also tends to deal unreliably with timeouts. It's still worth investigation because if your needs are simple, those problems mostly go away. <br /> <br /><b>arwyn:</b> Yep, MIG (stands for Mach Interface Generator for those who don't know and want to find out more) is definitely worth checking out if you want to look more into the mach side of things. <br /> <br /><b>has:</b> I strongly disagree and think it's completely fair. AppleScript is so bad that I went as far as writing my own Apple Event builder library (AEVTBuilder) to avoid using it in one application. I don't know how you're getting <i>anything</i> to work flawlessly in AppleScript but whatever secret you have, I'd like to know what it is. My experience with AppleScript is that it's a frustrating exercise in trial and error for even the most trivial of tasks. It's useful for banging out quickie scripts to get some apps to do what you want as a user. As a developer, my opinion is that AppleScript the language is 100% useless and should simply be avoided. AppleScript support in your app can still be worthwhile but it hurts pretty hard. <br /> <br /><b>Pau Garcia i Quiles:</b> The reason I didn't mention DBUS is perfectly answered by the beginning of your second question: "It's not included with Mac OS X". To be frank, IPC mechanisms which don't ship with the OS and which require installing daemons do not interest me. <br />0b583194bcb1c649c11e842d79033801Sun, 18 Jan 2009 05:16:42 GMTPau Garcia i Quiles - 2009-01-18 03:03:36http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsNo mention of DBUS? <br /> <br /><a href="http://www.freedesktop.org/wiki/Software/dbus">http://www.freedesktop.org/wiki/Software/dbus</a> <br /><a href="http://dbus.darwinports.com/">http://dbus.darwinports.com/</a> <br /> <br />It's not included with Mac OS X but it's available and it's used extensively by the Mac OS X port of KDE4 ( <a href="http://mac.kde.org/">http://mac.kde.org/</a> )13b6d82c031041eea4f4df72715e0929Sun, 18 Jan 2009 03:03:36 GMTSuits - 2009-01-18 00:32:24http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsWow, I had to write some code on windows and it's great writing a CreateRemoteThread. After writing that code i moved over to Mac. I never looked into the mach API's but now i will give them a lookoverfc779bdc7b63a18e5a649d990bb0d14cSun, 18 Jan 2009 00:32:24 GMTdan - 2009-01-18 00:20:32http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsAnother nice thing about Mach ports is that you can send a memory buffer as part of a message (OOL data). The buffer is remapped into the target process with copy-on-write semantics, so basically you can send over an arbitrarily large chunk of data between processes with virtually no overhead. The Mach APIs will even deal with deallocating the memory for you if you tell it. Amit Singh's book has details. <br /> <br />I'd also be curious to hear what problems you've had with DO.1283a1a9fdafee0d63f3d11e8222bb4eSun, 18 Jan 2009 00:20:32 GMThas - 2009-01-17 23:43:00http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsBTW, while I've never used DO myself, I'm disappointed to hear that it has problems too. IMO Apple should develop a new, conventional object-oriented IPC system - something simple, efficient and reliable. Perhaps something along the lines of the traditional browser DOM, which is what most developers and users seem to expect and prefer anyway. Apple event IPC - which they've never gotten to work completely right and most likely never will - could then be quietly moved to legacy status and eventually retired completely, thereby curing all its ills. Then again, if they can't get DO right either... :/18ca3e3e2b895af91b7bae44da00f206Sat, 17 Jan 2009 23:43:00 GMThas - 2009-01-17 23:26:18http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#comments"Generally it's best to avoid AppleScript and simply send the corresponding raw Apple Events instead, either directly or through a mechanism like Scripting Bridge." <br /> <br />While I completely understand the sentiment, as a long-time application scripter I feel this is still a little unfair. It's true the language itself has many significant faults - unpredictable and ambiguous syntax being the biggest of them (even William Cook, one of the original designers, now admits this was a mistake). However, even after fifteen years the AppleScript language still provides the most elegant and reliable API for creating and sending Apple events to other applications. <br /> <br />(It also provides the only OSA-based language component worth a damn, although IMO that's as much the fault of the OSA API as other language developers. But I digress.) <br /> <br />While the format string-based AEBuild* functions are fine for performing fairly simple tasks from C/C++/ObjC, they become very verbose for more complex tasks. And while Scripting Bridge may provide a higher-level API, it heavily obfuscates the actual Apple events mechanism (far more than AppleScript does), reducing functionality, confusing and misleading users as to what's going on, and causing various incompatibilities with many scriptable applications. Even appscript, which must've been through a few thousand hours of design and testing by now, still occasionally chokes on some application commands that would work flawlessly in AppleScript. <br /> <br />The other thing to remember is that an awful lot of what AppleScript gets blamed for isn't actually its fault at all, but the fault of individual application developers who don't adequately design, test and/or document their Apple event APIs. In turn, some of that blame can be placed on Apple in general for not providing application developers with sufficient tools and guidance in the first place. <br />0360d822f010ca3b186593061c101c9eSat, 17 Jan 2009 23:26:18 GMTChristopher Lloyd - 2009-01-17 22:52:08http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsMIG? might as well cover RPC's with rpcgen too :) <br />975bcd3820e87b94c77a6ac28dcfb42cSat, 17 Jan 2009 22:52:08 GMTJonathan - 2009-01-17 19:33:24http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsAn interesting review of the possibilities. In just over a year of Cocoa coding I have found myself using all of these methods of IPC with the exception of FIFO's and shared memory. There is no getting away from IPC in modern coding. <br /> <br />My first port of call was DO, which looks great on paper put doesn't seem to stack up too well in practice - I would hesitate to use it in anger for large scale network communication. It doesn't win any prizes in the cross platform parade either, but then neither does the .NET remoting technology. <br /> <br />AppleScript et al are undeniably nasty, whether you look at them at the individual event level, at the OSA component level or at the actual script level. It is however pervasive and powerful. 38c9dbd4e88012df46d52b36f8331260Sat, 17 Jan 2009 19:33:24 GMTarwyn - 2009-01-17 13:50:16http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsI think your higher-level IPC section should have included a blurb about MIG. Most of the system uses it as a high level* way to automate the mach minutia, far more than any of the other methods you listed. <br /> <br />* high level being a somewhat relative termbe8813b9f96347bc6c50240675d6c9b7Sat, 17 Jan 2009 13:50:16 GMTErik - 2009-01-17 09:55:51http://www.mikeash.com/?page=pyblog/friday-qa-2009-01-16.html#commentsI'm curious about the unreliable behavior you described regarding DO, and why it's worthy of investigation in spite of this. Perhaps more detail on this could be a topic for a future Friday.c874c2509304d08d6c2d7cf6fcbfca24Sat, 17 Jan 2009 09:55:51 GMT