Friday Q&A 2009-01-02: Pros and Cons of Private APIs

It's a new year, and that means a new Friday Q&A! This week I'm going to take Steven Degutis's suggestion and discuss the ups and downs of using private APIs.

Getting Started

I'm not going to discuss what private APIs are out there or how to figure them out, as that would cause me to badly miss my deadline and make this thing way too long. Instead I just want to address this question: should you use them at all, and if so, when?

There are two pretty obvious extremes to the answer, and a lot of people who believe each end. One extreme is that private APIs should never be used, period, full stop. They're bad, don't want to touch them, don't even acknowledge that they exist. The other extreme is that they're fine and dandy, use them like you'd use anything else.

As with most things, I believe the truth lies somewhere in the middle. But where, exactly, and how do you determine if something is worth using?

First let's review the disadvantages, which you're probably familiar with already. An API is essentially a contract between the creator of the API (generally Apple in the context of this blog) and the user of that API. When an API is public, the creator promises not to change that API in an incompatible fashion. With private APIs no such promise exists, and they can change at any time. This change can cause your application to malfunction, crash, or refuse to start.

And the advantages? Well that one's easy. Private APIs let you do stuff you couldn't otherwise do.

So like most of engineering, it's a tradeoff. You have benefits and disadvantages, and you have to decide which one is more significant.

Elements of the Tradeoff

Using a private API is, ultimately, a maintenance issue. (Except on the App Store, where it's a legal issue, but that's outside the scope of this post.) If you use nothing but public APIs, your app is basically guaranteed to work forever. (Where "forever" really means "until Apple decides not to maintain backwards compatibility anymore". But note that ancient PowerPC-only Carbon apps still run on the latest Mac OS X, and that Classic didn't disappear until 10.4; Apple still keeps old stuff working for a good long time.) If you use a private API, your app is likely to break at some point.

But when? That's one of the big questions you need to answer. There are basically four levels to consider:

  1. Never. Sometimes a private API may be so fundamental and so widely used that it gets essentially fixed in stone despite not being public. A good example of this on Mac OS X is the mach APIs, which are technically private but which underly everything at a very fundamental level.
  2. Major releases. Most private APIs fall into this category, where you can be reasonably (although never 100%) confident that they will continue to work throughout the lifetime of the current major OS release. In other words, it will keep working on 10.5 but is likely to break on 10.6. Typically private APIs end up forming part of a support structure for the public APIs and can't be changed without a major reworking of those public APIs, and that only happens with a new major release.
  3. Minor releases. Occasionally something can't even be relied upon to keep working during the life of a major release. Early versions of LiveDictionary were like this. They relied on fiddly internal details of WebCore's implementation, like C++ method and ivar layout. These offsets were subject to change at pretty much any time, so LiveDictionary generally broke every single time Safari got updated. (Later on, public APIs became available that I was able to use instead, which solved the problem once and for all. For more details of what was going on under the hood in those dark days before the public APIs were available, see Hacking C++ From C.)
  4. Any time. Generally this means that you aren't using the private API right. This is much more common than with public APIs since you don't have any documentation, you have no guarantee as to the API's requirements, constraints, preconditions, postconditions, etc.

Another big question you need to answer is how bad the break, when it comes, is likely to be. Again, there are different levels to consider:

  1. No effect. It's unlikely that you'll get here. If there's no effect from having it break, why are you even using the thing?
  2. Lose a feature. Often you can write your code in such a way that the breakage is likely to be detectable and so you can simply disable whatever feature uses it.
  3. Crash your app. This is pretty common.
  4. Crash other apps. For developers of stuff that loads into other programs this is very common, for self-contained processes not so much. LiveDictionary did this. When LiveDictionary broke, it didn't just crash, it crashed Safari too.

Which category you fall into depends not only on what you're using but how. For example, LiveDictionary would toss up a warning and offer to disable itself for the duration if the WebCore version was higher than what it knew about. A brave user could try to use it anyway (and there was at least one time when that version changed in unexpected ways and stopped this precaution from working) but this helped a lot. Obviously the higher up this list you are, the better off you are.

And lastly you need to figure out how long it will take you to fix the break. This depends greatly on your skill, your availability, what you're using, how critical it is, how it broke, and other such factors.

Coming to a Conclusion

Now you have enough information to run the cost/benefit analysis. The benefit side is pretty easy. The cost side can be determined by looking at how often you're likely to break, how bad it's likely to be, and how much time and effort it will take to fix. If it's a huge feature and will almost never break and will be trivial to fix when it does, then go for it. If it's a minor feature and will cause huge problems when it breaks every three months, pass. For LiveDictionary, the entire app was built around this feature, so it was worth it even though it required frequent difficult fixes.

Remember that the cost is not just to you, but to your users. If you're really unlucky, the break will be so bad that it's not even obvious that it's your fault, and they'll figure it out only after much head-scratching. Once they do figure it out, they will hate you if your fix doesn't come really fast. This means that for a really crucial and breakable feature, you need to stay available and ready to create and release a fix.

Private APIs can be invaluable, but their use must be weighed carefully. Sometimes it pays off very well, and sometimes it's a terrible choice. By carefully examining your app's vulnerability to breakage and your ability to fix it, you can decide whether it's the right move for you.