It is currently Mon Aug 24, 2026 10:45 am


All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: Memory and Swap space
PostPosted: Sat Aug 16, 2014 4:48 pm  (#1) 
Offline
GimpChat Member

Joined: Jul 19, 2014
Posts: 31
GIMP Version: 2.8.10
Operating System: Windows
GIMP Experience: Intermediate Level



I've 24 Gig of RAM but I find GIMP x64 on Windows to be unreliable with large files. It also uses far more memory for an given file than the 32 bit version.

At the moment I'm using a 32bit portable version that runs from a RAM disc. I thought this would mean GIMP used the RAM disk as swap space, but I'm not sure it is doing. When starting any job (via Python scripting) things run fast, but things slow down as the job progresses. If I run a second job after the first slows down, it too is slow. If I run a second job before GIMP slows down, it will start fast. Is this just because of the way swap space is accessed, even from a RAM disc? What else might be wrong?

Task Manager show very little RAM being used system-wide. If GIMP was using the RAM disk, it should show up as RAM being used (because memory for the RAM disc is dynamically allocated).

I imagine GIMP x64 on Linux is fine, so that is something I'll try, but I still want to get things working better on Windows. GIMP doesn't seem to use the system paging file, which is fine. I only made one to test performance.


Share on Facebook Share on Twitter Share on Orkut Share on Digg Share on MySpace Share on Delicious Share on Technorati
Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 5:23 pm  (#2) 
Offline
Script Coder
User avatar

Joined: Apr 23, 2010
Posts: 1553
Location: not from Guildford after all
Lumo wrote:
At the moment I'm using a 32bit portable version that runs from a RAM disc. I thought this would mean GIMP used the RAM disk as swap space, but I'm not sure it is doing.

GIMP uses the directory specified in Edit->Preferences:Folders for swap. If this directory is on your RAM disk then you should change it. It makes no sense to swap from memory to a RAM disk (you are just moving things around in memory).

What is the size of your tile cache (as specified in Edit->Preferences:Environment)?

_________________
Any sufficiently primitive technology is indistinguishable from a rock.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 6:09 pm  (#3) 
Offline
GimpChat Member

Joined: Jul 19, 2014
Posts: 31
No, that's not quite right. This is very basic stuff but I'll type it out anyway: 32 bit programs are theoretically limited to 4 Gig of memory. In practice they are limited to even less. People put swap space on RAM discs to work around this limitation and it works very well. Especially in Photoshop if you have an older version that doesn't support x64.

You really shouldn't make such general statements. People might think it's true. If you'd quantified your statement as referring to the 64 bit version only, you'd be right, but you'd be entirely off topic and it'd be a RTFOP moment.

I've been looking more closely at the problem I'm having with GIMP. I think the slow down is simply related to processing only. I suppose it makes sense that as the image gets bigger it will take longer to make it bigger still.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 6:46 pm  (#4) 
Offline
GimpChat Member

Joined: Jul 11, 2010
Posts: 349
Maybe you should reread what saulgoode wrote. I think you missed his point.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 6:55 pm  (#5) 
Offline
GimpChat Member
User avatar

Joined: Apr 07, 2010
Posts: 14182
Saulgoode is one of the most qualified members in this forum. I would take his answers to the bank every time. He has helped me more times than I can count.

_________________
Image


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 7:15 pm  (#6) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4812
What Lumo says make sense in his context... He is running a 32-bit Gimp than can only access 3GB of RAM. So it's made to swap to the RAM disk which is still a lot faster than swapping to a true disk.

But that's why 64-bit program were created. They use a bit more memory than 32-bit programs (because all the memory addresses now require eight bytes instead of four)(see for some figures, but they are no longer limited to the (theoretical) 4GB of RAM.

And I don't think that Lumo's arguments about increased memory usage and assumed unreliability of the 64-bit version are a good enough reason to stay with the 32-bit version, especially if you have to jury-rig RAM disks to make it do work that the 64-bit version will do much more easily.

_________________
Image


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 7:26 pm  (#7) 
Offline
Script Coder
User avatar

Joined: Apr 23, 2010
Posts: 1553
Location: not from Guildford after all
Lumo wrote:
No, that's not quite right. This is very basic stuff but I'll type it out anyway: 32 bit programs are theoretically limited to 4 Gig of memory. In practice they are limited to even less. People put swap space on RAM discs to work around this limitation and it works very well. Especially in Photoshop if you have an older version that doesn't support x64.

On a 32-bit system, a particular process can not access more than a 4GB address space. If that single process has been allocated 4GB of physical memory then there is no possible way for that process to trigger a page fault. In other words and more specifically, once the GIMP process has been allocated 4GB of physical memory, it will not cause any swapping. GIMP may fork other processes but then each of those forks will have its own 4GB virtual address space.

If you have a second (third, fourth, ...) process running on your system (whether or not it was forked from GIMP) and it has not yet had a full 4GB of physical RAM mapped into its 4GB virtual address space, and it then attempts to allocate some more memory, one of two things could (successfully) happen: 1 - there is unmapped RAM available and this gets mapped into the process's virtual address space (no swapping) or 2 - there is not enough physical RAM available and so physical memory has to be "dumped" to the swap file. By creating a swap file on a RAM disk (decreasing the amount of available physical RAM), all you are doing is making 2 more likely and incurring a penalty in moving memory from one location to another.

_________________
Any sufficiently primitive technology is indistinguishable from a rock.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 7:56 pm  (#8) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
@Lumo LOL, man. In gimp go to Help>About. If you wait long enough you'll see Saul Goode's name there.
Yeah, he's one of the developers/contributors.

Saulgoode already answered that but checking sometimes application manual helps.
From the official gimp documentation page A low Tile Cache value means that GIMP sends data really quickly to disk, not making real use of the available RAM and making the disks work without real reason.

Especially so on Windows OS with its not very good swapping behavior (compared to that of Linux).


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 8:11 pm  (#9) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4812
Hmm. It was my understanding that Lumo's setup is the Gimp swap (gimpswap.$pid) going to the RAM disk.

_________________
Image


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 8:48 pm  (#10) 
Offline
Script Coder
User avatar

Joined: Apr 23, 2010
Posts: 1553
Location: not from Guildford after all
ofnuts wrote:
Hmm. It was my understanding that Lumo's setup is the Gimp swap (gimpswap.$pid) going to the RAM disk.

Well, if you have your tile cache set to 4GB, this would allow for about a hundred 5000x3000 layers being open before tile swapping should be necessary. In such a situation, using a RAM disk for the GIMP swap directory could prove beneficial; however, GIMP tile swapping is not particularly efficient and it is best to avoid getting into such situations (hence the question I posed earlier). On Linux-based systems, one is probably better off omitting the RAM disk and letting the system use the memory for general disk caching.

If 64-bit GIMP on Windows is that unreliable, I would suggest running GIMP on 64-bit Linux running in a virtual machine. 64-bit processors not only avoid the memory limitations of 32-bit, but also have twice as many registers available and better streamlining of floating point operations.

_________________
Any sufficiently primitive technology is indistinguishable from a rock.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sat Aug 16, 2014 10:02 pm  (#11) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
ofnuts wrote:
Hmm. It was my understanding that Lumo's setup is the Gimp swap (gimpswap.$pid) going to the RAM disk.


The problem is a bit deeper than just the speed with which disks can read/write the data.
Yes, the RAM disk is 10x faster than even SSDs. But it doesn't mean we can make gimp work 10x faster.
Not without graphics acceleration software at the backend, which gimp doesn't support yet.
Even 3D applications that are limited to only OpenCL/GL CPU rendering allowed by your graphic card work faster with tiles set to a bigger size.
Tile Cache size becomes less relevant with GPU rendering but I'm afraid I won't live long enough to see gimp
with CUDA kernel support :hehe
In case of gimp tile cache is important but only to a certain extent beyond which no amount of RAM can accelerate
or even prevent its performance from a slowdown.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Sun Aug 17, 2014 2:05 am  (#12) 
Offline
GimpChat Member
User avatar

Joined: Dec 08, 2013
Posts: 1088
:bigthup
If I may, in the menu HELP>ABOUT resides a list of formidable and experienced people ie; developement team. Mr Goode is listed :roll:
molly wrote:
Saulgoode is one of the most qualified members in this forum. I would take his answers to the bank every time. He has helped me more times than I can count.

_________________
Image


Top
 Post subject: Re: Memory and Swap space
PostPosted: Mon Aug 18, 2014 4:53 pm  (#13) 
Offline
GimpChat Member

Joined: Jul 19, 2014
Posts: 31
Alright, the problem is nothing to do with the RAM disk, swap file, tile cache, or anything else covered so far. It turns out that each time I run the script it runs slower than the previous time, as well as slowing down during processing. Using a fresh copy of GIMP Portable makes everything lightning quick again. What took several hours yesterday takes just a few minutes now.

I did an automated compare of the two GIMP directories and the problem files seem to be:
Quote:
settings\recently-used.xbel
fontconfig\cache\*

I deleted recently-used.xbel first, and that boosted the speed to nearly double. Then I deleted all the font cache files and the speed shot up to match that of the new copy of GIMP Portable. The speed is astonishing considering what it has been.

The idea of running 64bit Linux in a virtual machine is a great suggestion. I forgot that whole 64bit OSs can be virtualised, as all my virtual OSs are 32bit. A lightweight 64bit distro might give this work a real boost, even with the extra overheads of virtualisation.


Top
 Post subject: Re: Memory and Swap space
PostPosted: Mon Aug 18, 2014 4:56 pm  (#14) 
Offline
GimpChat Member
User avatar

Joined: Nov 16, 2011
Posts: 5127
Location: Metro Vancouver, BC
Good going Lumo, thank you for the info.

_________________
Image
Gimp 2.8.18, Linux, median user
Gimp Chat Tutorials Index
Spirit Bear (Kermode)


Top
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 5 hours [ DST ]



* Login  



Powered by phpBB3 © phpBB Group