mikeash.com pyblog/friday-qa-2009-06-19-mac-os-x-process-memory-statistics.html comments

mikeash - 2009-07-10 11:14:18

Fri, 10 Jul 2009 11:14:18 GMT

On my system, at least, the most common cause of IO-based slowdown is swap, which in turn is caused by memory exhaustion. I've seen Time Machine and other such things make the machine slow as well, but not as frequently and not as dramatically. Much will depend on your hardware and your individual usage habits, of course.

Levi Figueira - 2009-07-10 10:57:34

Fri, 10 Jul 2009 10:57:34 GMT

@juancn is SOOO right... Usually when the system is having a major slowdown, it is caused by high disk usage *not* RAM issues...

I keep wondering why a why to monitor per process disk activity is not integrated with the Activity Monitor (just the totals... not very helpful).

From my experience, the cause for major system slowdows are apps like Time Machine (local) or Backblaze (online) and other backup apps, or Spotlight and other search/indexing utilities... Those reaaaally bog down my machine and are really hard to track down/stop.

This article is such an enlightening read, but it comes down to: it's very hard to track *real* memory usage, because... there's no such thing! ;) haha It's soo complicated and "abstract"...

Thanks for the tip on iotop, juancn! :)

juancn - 2009-06-30 21:30:56

Tue, 30 Jun 2009 21:30:56 GMT

One command that can be very useful finding which process is slowing down your machine is "iotop".

You run it like this:

~>sudo iotop -P

And it displays a % of disk IO each process is using :)

CPU usage is easy to see in activity monitor or by using top. But disk I/O is usually hard to track down.

Andrew - 2009-06-22 21:13:10

Mon, 22 Jun 2009 21:13:10 GMT

@scott: in fairness, re Firefox's historical memory leaks/allocation schemes...

http://dotnetperls.com/browser-memory and
http://dotnetperls.com/chrome-memory

This matches my experience, since the 3.0 release. FF has cleaned up its act.

e.g., Firefox running for 13 days, presently about 80 tabs open:

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
460 firefox-bi 10.7% 31:29:37 26 277 6391 338M 50M 490M 939M

(CPU usage hovers between 10-20%, probably Flash and javascript cruft that just keeps churning in unviewed tabs)

I also find it useful to keep separate profiles (run /Applications/Firefox.app/Contents/MacOS/firefox-bin --profilemanager
from Terminal), one profile for regular browsing with just a few plugins enabled, and another for web dev work with firebug, html validators, etc.

Mike Smith - 2009-06-21 00:54:00

Sun, 21 Jun 2009 00:54:00 GMT

@ssp

It's nice to pine for the "good old days", but the Darwin memory manager is nothing like the 9 memory manager, and Mac OS X apps tend to have more complex behaviours than their Classic ancestors, so the situation overall just isn't as straightforward anymore.

@natevw

As Jean-Daniel explains, shared libraries are internally subdivided based on (amongst other things) their writability. It's worth noting that the parts that might be writable but which have not yet been written remain shared using a technique called copy-on-write (often abbreviated COW).

@chuck

The accounting used to maintain those numbers is complicated, and they are not directly derived one from the others. It may help to know that most shared libraries are kept in what is known as the "shared segment", and the system uses a trick to share portions of the pmap and VM pagetable (data structures used by the MMU) that correspond to this segment between processes. This is a healthy performance optimisation, but it means that updates made to these data structures cannot be trivially accounted for across all processes.

In particular, RSIZE is very difficult to account precisely. If one task causes a page to be made present in a shared library, it's reasonable to charge that task for the page. If another task then uses that page, it might be reasonable to charge it for it as well, but there is no event that tells the system the second task has used the page. Further, when the page is later evicted and recycled, there is no list of all the tasks that used the page. Indeed, once it's made present, it's present in all tasks. Do you account a resident page in the shared segment against every task in the system? That would make the number useless, as one task running lots of code in a framework would make it look like every task in the system was blowing out its resident set.

Instead, RSIZE is managed using several tricks that try to make it 'relevant' at the cost of being 'precise'. You're encouraged to read the source code for the details.

@Miles

The act of "inspecting" a process causes a large chunk of its address space to be shared with the tool inspecting it, and so it goes from being accounted as VSIZE to VSHARED. You'll note that if you quit and restart Activity Monitor, the VSIZE accounting pops back up to where it was.

mikeash - 2009-06-20 23:39:45

Sat, 20 Jun 2009 23:39:45 GMT

The Activity Monitor global system memory terms are clearly explained in the help. Please RTFM.

Chuck: Any given chunk of physical RAM could be mapped 0 or more times into the process.

For questions about CPU usage and such, I think you're in the wrong place. This is a programming blog.

Jean-Daniel Dupas - 2009-06-20 18:26:10

Sat, 20 Jun 2009 18:26:10 GMT

If system libraries are shared via shared memory, how is process-specific state kept separate?

Shared library are composed of multiple sections. One with the code (read-only), one with the process specific state (read-write) (global variables) etc.

And only the "read-only" sections are shared.

In useful tool list, it's worth to mention MallocDebug too.

Matthew Kosterman - 2009-06-20 11:33:55

Sat, 20 Jun 2009 11:33:55 GMT

I'd love to know how to figure out which process(es) is/are "stuck" when this is reported in TOP. It seems as though whenever my machine is behaving erratically, with inexplicable pauses in execution, I can run TOP and I will see one or more "stuck" processes in the header info.

I've searched all over and not come up with much info other than what I can summarize as: it doesn't necessarily mean something bad, it just means a program is waiting for something.

It would be nice to know which program and what its waiting for.....

Miles - 2009-06-20 07:39:46

Sat, 20 Jun 2009 07:39:46 GMT

Here's what I've always wondered: In Activity Monitor, why does the "Virtual Memory" number for a process drop by about 568MB for a process when you inspect it?

Chuck - 2009-06-20 06:57:58

Sat, 20 Jun 2009 06:57:58 GMT

I just can't see why, by your definition of RPRVT, RSHRD and RSIZE, RSIZE is not equal to RPRVT+RSHRD.
RSIZE measures actual memory, but you said RPRVT is the amount of address space, local to the process, which corresponds to items currently present in physical RAM.

I'm sure I'm just missing something

natevw - 2009-06-20 06:54:36

Sat, 20 Jun 2009 06:54:36 GMT

If system libraries are shared via shared memory, how is process-specific state kept separate?

Nick - 2009-06-20 06:45:52

Sat, 20 Jun 2009 06:45:52 GMT

Tom Vanderlinden: Nice" in the the CPU tab is also a mystery

"nice" is scheduling priority, try 'man nice' in a terminal or http://en.wikipedia.org/wiki/Nice_(Unix)

Tom Vanderlinden - 2009-06-20 06:06:14

Sat, 20 Jun 2009 06:06:14 GMT

Could you take one more step and translate the Activity Monitor terms used in the System Memory tab:
Wired
Active
Inactive
Used
Free
Vm size:
Page ins/outs

"Nice" in the the CPU tab is also a mystery,
but I suppose that is another subject.

ssp - 2009-06-20 05:52:22

Sat, 20 Jun 2009 05:52:22 GMT

One of the nicest explanations of the mythical numbers I have seen, thanks.

Yet - and I have been wondering about this for years - my main question "which number corresponds to the number displayed by Mac OS 9" still remained unanswered, or rather answered as 'none'.

Which is a shame because the OS 9 numbers were actually useful in the sense that they told me which application would be worth quitting when running into swapping hell. In OS X I have to intuitively quit notorious memory hogs like iPhoto or VMWare but it seems to be impossible to collect reasonable information about which application is most worth quitting from these numbers.

Jon - 2009-06-20 05:46:02

Sat, 20 Jun 2009 05:46:02 GMT

How do these terms relate to the terms used in activity manager. Is private the same as wired?

Marc - 2009-06-20 04:07:37

Sat, 20 Jun 2009 04:07:37 GMT

Scott: nice one :-)

Scott - 2009-06-20 04:05:40

Sat, 20 Jun 2009 04:05:40 GMT

A common question is "which apps are making my machine slow or filling up my RAM?" Of course,

the answer is "Firefox"

:-)

Thanks for the information!

Nick - 2009-06-20 03:00:47

Sat, 20 Jun 2009 03:00:47 GMT

A common question is "which apps are making my machine slow or filling up my RAM?" Of course, this is a difficult question, but I think that RPRVT is the right single number to look at.

Bryce - 2009-06-19 23:31:20

Fri, 19 Jun 2009 23:31:20 GMT

Very clear and helpful. Thanks!