mikeash.com pyblog/friday-qa-2011-01-04-practical-floating-point.html comments

Joshua - 2011-05-27 00:22:23

Fri, 27 May 2011 00:22:23 GMT

Fantastic post, thank you. I'm and absolute beginner when it comes to programming (sure, I can do a lot of 'things, but I know far to little in depth). Other then the old 'careful for with floats', would you have some other topics you could point at about pitfalls I will encounter? Perhaps a bit to broad a question, so let me rephrase: What's the last one simple thing that you did wrong and made you retrace your code for an hour?

mikeash - 2011-02-24 18:24:29

Thu, 24 Feb 2011 18:24:29 GMT

Cocoa does indeed store the actual float values, but drawing gets complicated.

Primitives (like NSBezierPath) are fully antialiased (unless you turn that off). If you draw them on fractional pixels, you get fuzzy lines. This is a common problem for Cocoa developers, because drawing a line on integer coordinates actually puts the line halfway between two pixels, and you get a two-pixel-wide gray line instead of a single pixel black line like you want.

I'm not sure what happens with images. Could go either way.... Likewise with views.

In any case, a lot of Apple's code has smarts to draw on pixel boundaries even if you have fractional coordinates. Presumably that's what's happening with your buttons. If they were really being drawn at fractional coordinates, they would probably look terrible. Instead, I imagine their coordinates are getting rounded off for display.

I would still recommend rounding the coordinates yourself, rather than relying on Cocoa to do it for you, but it'll usually work at least.

Christopher Pickslay - 2011-02-24 07:01:40

Thu, 24 Feb 2011 07:01:40 GMT

Another great post, Mike. You mention "Using floating point gives Cocoa additional flexibility to produce good approximate results when not working on whole pixels." Does this mean that Cocoa stores the actual float values, but rounds when drawing?

I'm working on a project where I have to lay out a grid of buttons. The size and number of buttons changed several times, and I got sick of recalculating the buffer between them, so I started doing this:

// 5 buttons, 46 pixels tall, in a 256 pixel height box
static const CGFloat kVerticalGap = (256 - (5 * 46))/6.0;

At a glance, the buttons appear to be perfectly spaced, even though that buffer evaluates to 4 1/3. Now that I look closely at the grid, it does appear that the second and fifth gaps are actually 5 pixels.

Sean McBride - 2011-01-26 22:36:11

Wed, 26 Jan 2011 22:36:11 GMT

Great article Mike. Two additional thoughts:

Might have been interesting to mention casting. What happens casting float/double to an integer type? Particularly with nan & inf. :)

CGFloat. It's either float or double. That can make it annoying to deal with, especially with compiler 64 to 32 truncation warnings. To use sin() or sinf()? To use the 'f' suffix with constants or not?

(BTW I'm not actually asking...)

Mike G. - 2011-01-20 19:42:11

Thu, 20 Jan 2011 19:42:11 GMT

Of course, the correct way to get halfpi is
double halfpi = M_PI_2;
:)

mikeash - 2011-01-20 00:38:16

Thu, 20 Jan 2011 00:38:16 GMT

Yes, I guess I didn't word that very well, but I meant that x/INFINITY = 0 for !isinf(x)

James Bell - 2011-01-19 03:22:58

Wed, 19 Jan 2011 03:22:58 GMT

Did you maybe mean to say dividing a finite number by infinity is zero?

James Bell - 2011-01-19 03:19:27

Wed, 19 Jan 2011 03:19:27 GMT

You said that INFINITY/INFINITY results in zero. (The last sentence of the first paragraph about the INFINITY macro.) It's actually NaN. Infinity divided by infinity is indeterminate... but I guess you hadn't introduced that concept yet. :)

mikeash - 2011-01-18 17:36:41

Tue, 18 Jan 2011 17:36:41 GMT

Square roots and pi tend to feature quite a lot in real code, especially when doing graphics, so I wouldn't say "all". However, I certainly agree with the utility of rational numbers.

For "good", GMP provides rational number support. I don't think it qualifies as "simple", but you probably can't have both at the moment....

Simon - 2011-01-18 16:46:38

Tue, 18 Jan 2011 16:46:38 GMT

Some modern languages start (finally) to offer rational data types, which are absolutely precise. (perl does it with bigrat, lisp does it too, python is starting to do it).

Since all and every numbers (excepted square roots, pi and e) used in a program are rational numbers, it should really become a standard asap.

Do you know a good and simple library to do rational representation in C (or in Obj-C)?

mikeash - 2011-01-17 15:03:21

Mon, 17 Jan 2011 15:03:21 GMT

How embarrassing. Fixed, thanks!

Mike - 2011-01-17 14:24:36

Mon, 17 Jan 2011 14:24:36 GMT

Small correction: 998.75 = 9.9875 × 10^2

Pierre Lebeaupin - 2011-01-15 14:25:17

Sat, 15 Jan 2011 14:25:17 GMT

Nice. A few complements:
Back in the day, the Mac would use a 80-bit format for SANE (their software floating-point library), then also the 96-bit format of the 68k FPU; I don't remember whether they were mapped to the native Pascal floating-point types, though. And just for your amusement: the TI-89 (which I've been known to tinker with from time to time) uses a decimal-based floating-point format, probably to have results more accurate in appearance (e.g. finite decimals can always be represented accurately, as long as they don't go over the available mantissa, which is represented, yes, in Binary-Coded Decimal)

Floating-point is fast these days (at least for this audience), with one exception: on the ARMv6 iOS devices (or all iOS devices, if you have an ARMv6-only executable, as opposed to an ARMv6/ARMv7 fat one) if you've kept the default of compiling for thumb, as thumb on ARMv6 does not have access to the floating-point instructions, so every single addition, subtraction, multiplication, etc. has to be done with a system call. If you're using floating-point calculations in any non-trivial fashion on iOS, make sure you disable thumb at least for the ARMv6 half (Jeff Lamarche explains how to do so here: http://iphonedevelopment.blogspot.com/2010/07/thumb.html ). Also, intensive usage of doubles is not recommended on iOS (not that fast and the precision is usually pointless).

Every time you intend to do a floating-point equality comparison, think long and hard about what it is you actually want to do. Normally floating-point variables are purely data, and not used for control. Of course equality will work if the floating-point calculations only ever involve integers, but if you were sure that would always happen, you would be using integers and not floats, wouldn't you? If you want to check that your calculations do give the expected result, then you should figure out using numerical analysis the maximum error that can be introduced by your algorithm, then check that the result is within that error range. But in most cases this is not equality you want to check, but whether two values are close enough not to matter making a difference; how close enough (1%? 0.1%) depends on the purpose. For instance, I worked on a video application on iPad, and like the built-in Video app, we wanted to allow the user to switch between letterboxed and pan and scan. But if the aspect ratios of the screen and the video are close enough, the difference isn't going to matter, so might as well lock to letterboxed and hide the button; I decided this would be the case if and only if the aspect ratios differed by less than 1%.

Unit accuracy may give some surprises, especially when converting to float. For instance, I recently wanted to have a control that would represent a circle rotating around the center with a period of two seconds. So (among other things) I need to take the tv_sec of gettimeofday and divide it by 2, but how to do so? If I convert to float first, it's going to be too big for unit accuracy to be preserved (these days the number of seconds since the epoch is more that one billion, much more than 2^24) so it's not going to behave as expected. The solution is to first do the entirely integer operation of getting the remainder of division by 2, convert that to float, then divide that by 2.0; the integer part is gone, but it wasn't going to matter anyway after multiplying by 2.0*M_PI and getting the sinf and cosf.

J Osborne: the formats used for the vector architectures I know about (AltiVec, SSE, NEON) do match IEEE 754, it's the operations that usually take some liberties or do not implement the full IEEE 754 semantics (e.g. NEON, the vector architecture on ARM, not only flushes denormals to zero on input and output by default, but contrary to AltiVec cannot even be configured to preserve them).

Ross Carter: You should also be reading "What Every Computer Scientist Should Know...". You WILL write problematic floating-point code otherwise.

Steven Wart - 2011-01-14 21:57:35

Fri, 14 Jan 2011 21:57:35 GMT

very nice. if only we could convince the investment banks to stop using floating point numbers for their currency calculations :)

mikeash - 2011-01-14 19:18:12

Fri, 14 Jan 2011 19:18:12 GMT

Thanks for the typo patrol, fixed now.

Chris - 2011-01-14 18:54:06

Fri, 14 Jan 2011 18:54:06 GMT

Minor typo: "Trigonemtric" should be "Trigonometric".

Ross Carter - 2011-01-14 18:50:09

Fri, 14 Jan 2011 18:50:09 GMT

This is exactly what I have been wanting to read for years. "What Every Computer Scientist Should Know..." is over my head. Now I understand! Thanks!

J Osborne - 2011-01-14 18:33:26

Fri, 14 Jan 2011 18:33:26 GMT

"Note that there is nothing in C which requires the floating point types to use IEEE-754 semantics. However, that is what is used on all Apple platforms, and what you're likely to find anywhere else, so everything I discuss here assumes IEEE-754."

Largely true, but some exceptions:

"long double" isn't always a IEEE-754 float on Apple platforms. The G5 uses a head-tail format for long double. The x86 might use the 80-bit Intel stuff which is close but not the same (to Intel's credit IEEE-754 was still being created when Intel made the first FPU for the 8086, and the 80-bit format was strongly influenced by what IEEE was doing, but doesn't match what IEEE-754 eventually became).

The vector FP formats are not always IEEE-754, and while they are frequently "close" the special values (denormals, NaNs, INFs, and -0) are all suspect (as in if you rely on them go read the SSE-X spec, or whatever vector instruction set you are using to make sure they behave the way you expect).

"On heftier processors (like on the Mac), using floating point can actually make your code go faster, due to freeing up integer units for other tasks."

A lot of CPUs have faster FP multiply and/or divide then integer multiply/divide, esp for the "worst case". Some x86's have a worst time integer multiply in the 20 cycle range while the FP multiply is well under 10 cycles. I seem to recall that divide is worse, but don't remember specific numbers.