It is currently Tue Jul 23, 2024 2:28 am


All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: GIMP Memory Management & Crashing (Solved)
PostPosted: Mon Dec 30, 2013 10:45 am  (#1) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
I have a system with 4G of RAM and when I work with an image with lots of 800x600 layers, somewhere around 120, GIMP crashes. The error generated is.

GLib-ERROR **: gmem.c:165: failed to allocate 12400 bytes

Any tips on optimizing GIMP's memory use or do I just need more RAM?

Does anyone know if layers that have been merged/deleted are actually tossed or are they hanging around on a memory heap somewhere?

I should also mention that I'm seeing this when using Script-fu to generate a bunch of layers. The number of layers I can generate/manipulate seems to depend how much other stuff I'm doing within the script.

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Share on Facebook Share on Twitter Share on Orkut Share on Digg Share on MySpace Share on Delicious Share on Technorati
Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 11:39 am  (#2) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
More RAM never hurts if the machine architecture allows RAM expansion.

Deleting or merging layers actually increases the memory use within the same
gimp session (see MB size in information panel on the bottom of gimp window).
It does so to maintain undo operations integrity (hence, the less undo steps you assign in gimp prefs, the less memory is used).
What can be done:
1.Saving gimp document and reopening it dumps the undo history or simply remove all undo operations
in Undo History dockable panel.
2.Increase tile cache memory to at least 80% of RAM in Preferences.
3.Gimp swap folder by default is in .gimp-2.8 folder (Edit>Preferences>Folders).
Moving it to other location with enough disk space (preferably external HDD with fast USB 3.0 or eSATA connection) also helps.
GIMP's tile-based memory allocation scheme populates and flushes swap folder when things go beyond
RAM capacity, besides, most of the image rendering takes place in the swap folder since GIMP doesn't support yet
multi-core, multi-threading processing.

Hope it helps.
P.S. You can also clear gimp Document History. Information about hundreds if not thousands of images with thumbnails
may increase gimp start-up time and, of course, uses up some RAM.
Another trick I learned when I was using old computer with limited RAM and processor speed - if GIMP
crashes while working with particular document, you can temporarily turn off Color Management
Edit>Preferences>Color Management> No color Management


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 12:27 pm  (#3) 
Offline
GimpChat Member

Joined: May 23, 2012
Posts: 150
GnuTux

don't know if this will help you but,
i made an new folder i called "on-or-off"
in my user scripts folder as a test.
i put my Gimp-surfr.scm file in the on-or-off folder.
use Preferences and script-fu refresh
to turn on or off, turn off user stuff you don't use.
can do this for all user resources, brushes, gradients...,
except *user\plugins*
cutting memory requirements


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 12:56 pm  (#4) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4757
Mikechat wrote:
GnuTux

don't know if this will help you but,
i made an new folder i called "on-or-off"
in my user scripts folder as a test.
i put my Gimp-surfr.scm file in the on-or-off folder.
use Preferences and script-fu refresh
to turn on or off, turn off user stuff you don't use.
can do this for all user resources, brushes, gradients...,
except *user\plugins*
cutting memory requirements


I have a script to help you :) Make sure you get the latest (from sourceforge)

_________________
Image


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 1:05 pm  (#5) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4757
K1TesseraEna wrote:
More RAM never hurts if the machine architecture allows RAM expansion.

Deleting or merging layers actually increases the memory use within the same
gimp session (see MB size in information panel on the bottom of gimp window).
It does so to maintain undo operations integrity (hence, the less undo steps you assign in gimp prefs, the less memory is used).
What can be done:
1.Saving gimp document and reopening it dumps the undo history or simply remove all undo operations
in Undo History dockable panel.
2.Increase tile cache memory to at least 80% of RAM in Preferences.
3.Gimp swap folder by default is in .gimp-2.8 folder (Edit>Preferences>Folders).
Moving it to other location with enough disk space (preferably external HDD with fast USB 3.0 or eSATA connection) also helps.
GIMP's tile-based memory allocation scheme populates and flushes swap folder when things go beyond
RAM capacity, besides, most of the image rendering takes place in the swap folder since GIMP doesn't support yet
multi-core, multi-threading processing.

Hope it helps.
P.S. You can also clear gimp Document History. Information about hundreds if not thousands of images with thumbnails
may increase gimp start-up time and, of course, uses up some RAM.
Another trick I learned when I was using old computer with limited RAM and processor speed - if GIMP
crashes while working with particular document, you can temporarily turn off Color Management
Edit>Preferences>Color Management> No color Management


I think you suggestion 2) is counter-productive. The more RAM you use for the tile cache, the less RAM can be used by other apps... or by GIMP for things that don't go in the tile cache. So I would reduce the tile cache size... 3) I don't see why the HDD should be external. Ideally a second internal HDD should be used, or an external HDD only if there is a
very good link (but these HDD are usually removable, so may not be there when you start Gimp). Otherwise the standard location is also a good choice. GnuTux is on Linux and the Linux filesystems are a lot more efficient than those of the "popular platforms".

_________________
Image


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 1:12 pm  (#6) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4757
GnuTux wrote:
I have a system with 4G of RAM and when I work with an image with lots of 800x600 layers, somewhere around 120, GIMP crashes. The error generated is.

GLib-ERROR **: gmem.c:165: failed to allocate 12400 bytes

Any tips on optimizing GIMP's memory use or do I just need more RAM?

Does anyone know if layers that have been merged/deleted are actually tossed or are they hanging around on a memory heap somewhere?

I should also mention that I'm seeing this when using Script-fu to generate a bunch of layers. The number of layers I can generate/manipulate seems to depend how much other stuff I'm doing within the script.


Make sure the Script disables the undo stack while it runs (ideally does most of its work in an invisible window with undo disabled).

Otherwise it's hard to tell what's happening without some system data. But I doubt that the problem is due to RAM exhaustion because there is virtual memory for this so unless you have no swap on your system or the swap is full (easy to check....) this kind of thing never happens. I would believe more in some memory block corruption that makes the glibc allocator trip. The underlying error may have been logged somewhere.

_________________
Image


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 1:29 pm  (#7) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
ofnuts wrote:
I think you suggestion 2) is counter-productive. The more RAM you use for the tile cache, the less RAM can be used by other apps... or by GIMP for things that don't go in the tile cache. So I would reduce the tile cache size... 3) I don't see why the HDD should be external. Ideally a second internal HDD should be used, or an external HDD only if there is a
very good link (but these HDD are usually removable, so may not be there when you start Gimp). Otherwise the standard location is also a good choice. GnuTux is on Linux and the Linux filesystems are a lot more efficient than those of the "popular platforms".


When I answered GnuTux I also kept in mind that this thread is on public forum and
most of the visitors reading it are using 'popular platforms' (>75% to put it in numbers).

You're right, it could be a partition on internal disk, but I did not say it should be external, I said 'preferably external'.
That's what most users have these days, especially users of laptops.

About suggestion 2, this is actually what's recommended by the GIMP team - http://www.gimp.org/unix/howtos/tile_cache.html


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 1:35 pm  (#8) 
Offline
GimpChat Member

Joined: Dec 27, 2012
Posts: 393
Location: England
ofnuts wrote:
Mikechat wrote:
GnuTux

don't know if this will help you but,
i made an new folder i called "on-or-off"
in my user scripts folder as a test.
i put my Gimp-surfr.scm file in the on-or-off folder.
use Preferences and script-fu refresh
to turn on or off, turn off user stuff you don't use.
can do this for all user resources, brushes, gradients...,
except *user\plugins*
cutting memory requirements


I have a script to help you :) Make sure you get the latest (from sourceforge)


I just tried out the link to souceforge and I'm getting a page not found message. Just wanted to check that the script version I'm using is the latest one.

_________________
Image

DeviantArt gallery


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 2:00 pm  (#9) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
Thanks for all the responses. I tried adjusting the suggested options but nothing I do seems to make one bit of difference. It's almost like there is no paging/swapping going on. I disabled undo and it made no difference.

I've did some more testing and it turns out that I can manipulate many more layers using the devel version of GIMP 2.9.1 running in a VM than I can with GIMP 2.8.x running on my main system outside of the VM.

I have more fonts, patterns, gradient & scripts in 2.8 but not that many. I'm not convinced the extra resources are causing the issue. Yes.. 2.9.1 was much slower in the VM but I was able to generate 200 layers with no problem, whereas I always crashed just after 100 layers on GIMP 2.8.x.

I'm gonna try moving 2.9.1 out of the VM and and see if I can run it alongside 2.8.x. Just gotta make sure all the libs are available to 2.9.1.

I'm not sure what's going on here. :puzzled

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 2:35 pm  (#10) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
GnuTux wrote:
Thanks for all the responses. I tried adjusting the suggested options but nothing I do seems to make one bit of difference. It's almost like there is no paging/swapping going on. I disabled undo and it made no difference.

I've did some more testing and it turns out that I can manipulate many more layers using the devel version of GIMP 2.9.1 running in a VM than I can with GIMP 2.8.x running on my main system outside of the VM.

I have more fonts, patterns, gradient & scripts in 2.8 but not that many. I'm not convinced the extra resources are causing the issue. Yes.. 2.9.1 was much slower in the VM but I was able to generate 200 layers with no problem, whereas I always crashed just after 100 layers on GIMP 2.8.x.

I'm gonna try moving 2.9.1 out of the VM and and see if I can run it alongside 2.8.x. Just gotta make sure all the libs are available to 2.9.1.

I'm not sure what's going on here. :puzzled


GIMP 2.8 can handle >200 layers without crashing. In my 'can GIMP multitask?' thread here on GC a year ago
I ran simultaneously a few plugins (C and Python) and a script-fu script on 7 images. One of the plugins was B&P Word Art which created 800+
layers (it uses all fonts in the Font folder by default settings).
[[BTW, I accidentally deleted that video on Youtube, so that thread ought to be removed.
But if you give me some time, GnuTux, I can find the video (or make a new one) and re-upload it.]]

So obviously, it's more of the resource problem than GIMP 2.8 ability to handle 100's of layers


Last edited by K1TesseraEna on Mon Dec 30, 2013 2:42 pm, edited 1 time in total.

Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 2:39 pm  (#11) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
Well, I moved the .gimp-2.8 folder and let it create from scratch and it still crashes so there is not a GIMP resource issue. I couldn't run V2.9.1 that I copied over from the F20 VM. Looks like maybe a GTK version issue.

gimp-2.9: undefined symbol: g_type_add_instance_private

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 2:46 pm  (#12) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
GnuTux wrote:
Well, I moved the .gimp-2.8 folder and let it create from scratch and it still crashes so there is not a GIMP resource issue. I couldn't run V2.9.1 that I copied over from the F20 VM. Looks like maybe a GTK version issue.

gimp-2.9: undefined symbol: g_type_add_instance_private


No, I meant hardware resource - RAM, CPU speed, etc


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 2:54 pm  (#13) 
Offline
Script Coder
User avatar

Joined: Oct 25, 2010
Posts: 4757
K1TesseraEna wrote:
About suggestion 2, this is actually what's recommended by the GIMP team - http://www.gimp.org/unix/howtos/tile_cache.html[/font]

Indeed... this is a setting that covers most uses cases a priori, but in GnuTux' case that's still the wrong way to go IMHO.

_________________
Image


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 3:16 pm  (#14) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
K1TesseraEna wrote:
No, I meant hardware resource - RAM, CPU speed, etc

Yes, I understood what you meant. Certainly, more hardware resources would improve my results. Currently, I have a 3.2ghz Quad core /w 4G Ram and tons of HD space. More RAM would certainly help but that doesn't explain why my results would be so much better using GIMP 2.9.1 in a small VM on the same machine.

I wanted a clean test so I allowed the .gimp-2.8 resource folder to be created from scratch, but it didn't help.

I guess it's time to move to F20 and dump the old F18 install, keeping F19 intact. I usually keep the last two versions of Fedora available so I guess that will be my next PC project. :hehe

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 3:27 pm  (#15) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
K1TesseraEna wrote:
GIMP 2.8 can handle >200 layers without crashing. In my 'can GIMP multitask?' thread here on GC a year ago
I ran simultaneously a few plugins (C and Python) and a script-fu script on 7 images. One of the plugins was B&P Word Art which created 800+
layers (it uses all fonts in the Font folder by default settings).

It's all relative. I can run a simple script and create 500+ large layers on GIMP2.8 before it crashes & probably 1000+ layers on 2.9.1 in a small VM on the same box. The number of layers that I can get just depends on what else the script is doing and how large the layers are.

Considering the results I'm seeing with 2,.9.1 in a small VM on the same box, there must be a problem with 2.8.x on my machine and nothing that has been suggested so far has addressed the issue.

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 3:29 pm  (#16) 
Offline
Script Coder
User avatar

Joined: Jun 22, 2010
Posts: 1171
Location: Here and there
Any chance of a look at your script that's showing the problem?

When I was writing my "test for duplicate scripts" script in script-fu I was sometimes getting what looked like memory corruption within the script (variables containing the wrong contents) and found I had to restructure things a bit to build shorter lists etc.

Kevin


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 3:45 pm  (#17) 
Offline
GimpChat Member

Joined: Apr 12, 2010
Posts: 5870
@Gnu Tux

Did you tried the first suggestion of K1TesseraEna ?
That for me , on my computer does a LOT of differences , especially in case of long session :

After 30 minutes ,max 1 hour , i check the work in progress, just to be sure that i will not need the Undo history, i save everything, and reopen

I think kept record of undo history require a lot of resources because at soon reloaded, with same images but purged from history is quite more responsive on heavy images.

And yes would be possible discard history without reloading...but after using many tools, script, plugin i prefer exit and reload, just in case one of the many filters or tools used during the session didn't released properly all the memory used ...

_________________
My 3D Gallery on Deviantart http://photocomix2.deviantart.com/
Main gallery http://www.flickriver.com/photos/photocomix-mandala/
Mandala and simmetry http://www.flickriver.com/photos/photocomix_mandala/

Image

Mrs Wilbress


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 4:00 pm  (#18) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
Yep, I did all that, PC. GIMP 2.8, for the most part, actually runs fine, until I start generating 100s & 100s of layers. That's when things get wonky. It's not something that I would normally do, anyway. I mean, I wouldn't normally create that large of an extrusion. I'm just testing the limits. ;)

paynekj wrote:
Any chance of a look at your script that's showing the problem?

When I was writing my "test for duplicate scripts" script in script-fu I was sometimes getting what looked like memory corruption within the script (variables containing the wrong contents) and found I had to restructure things a bit to build shorter lists etc.

Kevin

Well, as it turned it out, the problem was not script related so I've came back and edited this post to remove my initial response.

What would be nice tho, would be a good Script-Fu stress script, so we can compare the various GIMP versions and platforms. :bigthup

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 4:17 pm  (#19) 
Offline
GimpChat Member
User avatar

Joined: Nov 15, 2012
Posts: 949
Location: Mesa, AZ
I've had this same issue with gimp since I switched from 2.6 to 2.8. So for quite a while. Any images I work on that are larger, I run the risk of a big fat nope from gimp. I might not mind so much if it told me it couldn't manage somehow and gave me an option to save. But it sort of punches you with it and then runs off to shut down. There's nothing you can do to save your file at all. Once you see the dreaded error of memory allocation, you're already hosed.

I tried all the suggestions given here. more cache, less cache, color management off, etc. I've had gimp 2.8 on an xp machine and now on a vista, it was a windows 7 before. I can't necessarily increase my ram nilly-willy, got bills to pay before I get to buy new toys. I would appreciate nothing more than a revamp of the oops-we-screwed-up into an oops-we-screwed-up-here's-a-save instead.

_________________
Image

I now have a Tumblr, check it out!


Top
 Post subject: Re: GIMP Memory Management
PostPosted: Mon Dec 30, 2013 4:24 pm  (#20) 
Offline
GimpChat Founder
User avatar

Joined: May 22, 2008
Posts: 5242
Location: Gimpville
You might be onto something, ek22. It might be an issue with V2.8, being I'm not seeing this problem in GIMP 2.9.1. I have a older box that's still running 2.6.x, so I should test it there as well. Off to try that right now.. :bigthup

_________________
“If you reach for the stars, you just might land on a decently sized hill.” - Stuart Hill


Top
Post new topic Reply to topic  [ 24 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


   Similar Topics   Replies 
No new posts How to access Image-Color Management - Discard profile with python_fu?

4

No new posts Astronomy plugins / script fu not working crashing

1

No new posts Attachment(s) You Got Something - In Memory Of JJ Cale

6



* Login  



Powered by phpBB3 © phpBB Group