Managing Virtual Memory

Lately there have been a lot of threads on the DAZ Studio Forums about crashes of the program (mostly when attempting to render something). As there can be so many factors involved when it comes to what causes any program to crash, troubleshooting these issues becomes a guessing game as to which one applies to any particular situation. However, one item that is important for 3D rendering of any sort is how memory is used in your computer. I will try to provide some help here for some ideas on tweaking the settings in Windows for how the Operating System manages your RAM.

Notice: This information is provided based on my experience managing my own computer systems over the past 20 or so years. No guarantee or warranty is either expressed or implied in the tips provided here. If you choose to follow any suggestion offered in this information, you are doing so at your own risk!

32 verses 64-bit computing

I won’t go into all the implications of what the difference is between a 32-bit and 64-bit version of Microsoft Windows. For this conversation, it is sufficient to say that in 32-bit versions of the Windows operating system, an application is (by default) limited to no more than 2 Gigabytes of memory at a time. And the system overall cannot address more than a total of 4 Gigabytes of RAM.

In a 64-bit version of Windows, a 32-bit application is allowed to access up to 4 Gigabytes of memory. On the other hand, a 64-bit application running on a 64-bit version of Windows would (theoretically) be permitted to access up to 8 Terabytes (that’s 8096 Gigabytes) of memory!

So, simply running a 64-bit version of Windows will provide more available memory for your 32-bit applications, and it will allow you to run 64-bit applications as well.

What about /3GB?

Ok, you’ve probably seen all these great tips about how in your 32-bit Windows, if you change the startup sequence to include the /3GB switch that your applications will “magically” be allowed to use 3 Gigabytes instead of 2, right? Well … not exactly. See, that switch does ALLOW applications to use up to 3 Gigabytes of RAM. However there are some significant caveats to that. First and foremost, the application has to have been created (compiled) with the ability to take advantage of that OS feature. Most 32-bit applications are NOT compiled that way.

Second, what you are doing by using /3GB is telling Windows that it should use less memory for itself and give more memory to the applications. If you have a lot of services and other things running in your operating system space, you could actually see significantly worse performance of your system by enabling the /3GB switch.

As for DAZ Studio, the executable itself indicates that it does support the large address space option. And evidence from end-user testing suggests that the 32-bit version of DS 4.0 (and later) does benefit in some way from the /3GB switch in Windows. So, it may be worth a try if nothing else seems to help.

Memory Usage

Microsoft provides several tools for monitoring the memory usage of your applications. Task Manager has the Process tab where you can sort by memory usage to see what is consuming that resource. On the performance tab, you can also see how much of your physical and virtual memory is in use. Also from that tab, starting with Windows Vista, they provided a tool called Resource Monitor which can show in greater detail exactly what resources your applications are using.

Physical Memory

Ok, let’s be honest here, there isn’t much you can do to influence the use of your physical RAM other than adding more. So get more if you can. :) You can also use tools like the built-in memory tester in Windows 7 ( link ) to test your RAM chips for potential problems.

Virtual Memory

In discussing memory in Windows / DAZ Studio, I think one part that is often overlooked is the virtual memory (aka page file) portion of memory management. First a quick understanding of what virtual memory is all about.

Pagefile.sys

Let’s say you have Firefox, DAZ Studio, and Skype all running under a 32-bit version of Windows 7. Each of those applications has been told by Windows that it has up to 2 Gigabytes of memory available. However, your system only have 4 Gigabytes of RAM in it. And Windows itself needs quite a bit of RAM just for basic operations. What happens when you fill up your physical RAM?

That’s where the pagefile comes in to play. When physical memory starts to become scarce, Windows looks for pages of memory which haven’t been accessed in a while. Those unused pages are temporarily swapped out of RAM and into a file on your hard drive (pagefile.sys). If the data in that page is requested by an application, the virtual memory manager, loads the page back from the page file into RAM and the data is available again.

Managing Virtual Memory

By default, Windows sets up a pagefile.sys on your system drive (c:) and configures it to be “system managed”. This means that Windows looks at your memory usage over time and adjusts the size of that page file to handle your likely needs. For general computing use, it’s a good way to go. The system balances the need for virtual memory with trying not to take up too much of your system drive with the page file and things are good.

The problem with 3D graphics, though, is that our use of memory doesn’t match the models that Microsoft used when they created the virtual memory manager. We can quickly require large amounts of memory, and the more complicated we get with what we’re doing, the larger the textures we’re working with, etc. the more memory we need. We don’t follow a nice even curve of memory usage, our memory needs spike and plummet often.

This causes the virtual memory manager some issues when it tries to figure out what to do with your page file. In my experience, Windows doesn’t grow the page file quickly enough to keep up with our needs because it is looking at average usage over time, whereas we need memory NOW. So, one thing I do whenever I build (or rebuild) my graphics machines is to take manual control over the page file.

How Much Page File?

Before I get into the mechanics of setting your page file, let’s talk a bit about how much of a page file you may need. Well … it depends. Mostly on a couple of factors. The first of which is how many hard drives you have in your computer. The second is how much space they have available.

How Many Drives?

You have at least one hard drive (C:\). But in many desktops (and some laptops) you may have more than one hard drive installed. Note that I am talking here about physical hard drives. While you can separate a hard drive into multiple partitions, the advice I’m about to give doesn’t help in that case, it only helps if you have separate physical storage devices.

Windows (and applications) perform better when the page file is on a separate physical drive from your system files. Your C:\ drive is already going to be busy with Windows asking for DLLs, program executables, etc. Keeping the pagefile.sys there as well is going to create contention for that resource. You are better off moving it to another physical drive if you can.

There is, however, limited benefit to splitting the page file into multiple drives. The added overhead of needing to query extra devices and re-combining pages tends to balance out the benefit of the extra device IO speed.

A quick note about Solid State Drives (SSDs) … I’m torn about using those for page file. First, since the page file is essentially an extension of your RAM, the faster the storage device the better; so in that way SSDs make sense. However one problem that plagues SSDs is fragmentation over time when you do a lot of reading and writing to them, which is exactly what happens to the page file. So I guess it is up to you.

What is the proper sizing?

This is going to depend on a few factors. The biggest of which is how much space is available on the drives you’re going to use for paging. Now don’t go all crazy and allocate 100 Gigabytes or something just because you have it. You want to keep things reasonable. My rule of thumb is that normally I want somewhere between 1.5 to 2.0 times the physical RAM I have in my PC. So, if I have 8 Gigabytes of RAM, I want a page file between 12 and 16 Gigabytes in size.

If I have 16 Gigabytes available and I’m not overly crunching space on my drive to get it, I’ll use that, otherwise I’ll go smaller. I would hate to go smaller than 1.0 times my physical RAM though. Below that, I’m likely to starve my system for memory.

You’ll see in the mechanics section below in regards to setting the page file size that you can tell Windows “Start at this size and grow to this…” Don’t bother. I always set both the min and max to the same value. Expanding a page file is “expensive” in computing terms. It also opens up to the page file becoming fragmented on the hard drive (thus making it slower) and if you only have the C drive available, you can end up filling up your system drive when the page file expands (which trust me you NEVER want to do!).

There is a drawback to not allowing Windows to ever expand the page file, though. If you were to happen to fill up all 16 Gigabytes of page file and Windows needed more, the next memory allocation call would fail. So, if you think that’s a possibility (and your hard drive space can stand it), you might consider something like a range of 16-18 to give yourself a little breathing room.

As with any “rule” like this, there are going to be exceptions. The 1.5x sizing works fine up to a point. Once you get to where your PC has more than 8 Gbytes of RAM, it starts to be excessive from a pure need basis. You need something (see the Warning section at the bottom), but if you have a significant amount of physical RAM, a smaller page file could easily work for you. I would just keep an eye on your memory usage, especially when using memory intensive programs, and make sure you’re not getting close to maxing out the page file usage.

How Do I set it?

Note that the following screenshots are from Windows 7. If you are using Windows XP or Windows 8, your specific dialog boxes, etc. may vary, however the concepts are the same.

  1. Open Control Panel – I always change Control Panel to the Large Icon view since I find the Categories view annoying
  2. Click on the System Icon

    Control Panel

    Control Panel

  3. On the left panel, click on Advanced System Settings

    System Control Panel

    System Control Panel

  4. Click on the Advanced tab

    System Properties

    System Properties

  5. In the Performance section, click on the Settings button

    Advanced System Properties

    Advanced System Properties

  6. Click on the Advanced tab

    Performance Settings

    Performance Settings

  7. In the Virtual Memory section, click on the Change button

    Advanced Performance Settings

    Advanced Performance Settings

  8. Uncheck the checkbox at the top that says “Automatically manage paging file sizes for all drives”

    Virtual Memory Dialog

    Virtual Memory Dialog

  9. Select the hard drive where you want to have a page file and change the “System managed size” option to “Custom Size”
    Setting Page File Size

    Setting Page File Size

    1. In the Initial Size, type the size of the page file in Megabytes (Gigabytes x 1024)
    2. In the Maximum Size, either type the same number, or some number 1024-2048 Megabytes larger
    3. Click the Set button
  10. If you have more than one hard drive, and you are moving the page file from your system drive to another hard drive, select the C drive, then choose “No paging file” and click the Set button.
    Note: Windows will warn you that without a page file on the system drive you won’t be able to create a dump file in the event of a system crash (BSOD). While this is true, in 20 years of experience as a Microsoft technology consultant, I’ve only actually NEEDED a dump file once.
  11. Click OK to save the settings
  12. Windows will warn you that the new settings will only apply after the next reboot, you can reboot right away if you’re ready to test

If something really goes badly for you with these manual changes, you will still be able to boot Windows, even without a page file anywhere. At that point, you can re-check the system managed option to get back to how things were before you messed with things.

Corrupted Page Files

I have on very rare occasions seen a corrupted page file. When that occurs, it can look like the computer has issues with it’s physical memory even though the problem is with the data stored in the page file. If you suspect this could be the case for you:

  1. Follow the instructions shown above
  2. Set all hard drives to “No page file”
  3. Reboot your computer
  4. Using Windows Explorer, ensure that the pagefile.sys file is gone from the root of all your hard drives (you may need to turn on the ability to see hidden and system files in order to see it)
  5. Reset the virtual memory page file settings to what you want them to be, forcing Windows to create a new file
  6. Reboot the computer again to start using your new page file

WARNING

Do NOT run your computer for any length of time without any page file at all. Even if you believe you have enough physical RAM for any memory needs your applications might require. Many applications (and even Windows itself) expect to be able to store certain information in the pagefile when they know in advance that it will seldom be needed. Running without any page file on your computer for an extended period is likely to cause significant stability problems!

Guidance not Rules

The above information is intended to be guidance for you to work with in exploring if configuring your page file might help performance and stability issues for your PC. The information here should not be taken as a hard and fast rule as to how everyone MUST configure their computer.  Naturally, I can’t guarantee you that changing your page file configuration is going to fix problems you might have with rendering in DAZ Studio. However, taking greater control over how Windows manages your memory allocation may help some and isn’t likely to hurt.

About these ads

1 Comment »

  1. karl3d Said:

    For anyone following this … I’ve added a warning about not running your PC without a page file for any significant length of time.


{ RSS feed for comments on this post} · { TrackBack URI }

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: