It is currently Thu Jul 25, 2024 11:22 am


All times are UTC - 5 hours [ DST ]



Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: GIMP environment setup - Photoretouching - GMIC usage [solved]
PostPosted: Sat Mar 02, 2013 3:14 pm  (#1) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
GIMP Version: 2.8.4
Operating System: Windows
OS Version: Win 7 - 32 bits
GIMP Experience: Beginner Level

List any relevant plug-ins or scripts:
GMIC - enhancement

List any ERROR messages you received:
failed to allocate memory



After having done in the previous days some tests here and there with sample images of limited size, I wanted to try experiment GMIC possibilities on a real photo, and by real I mean having a size which nowadays is quite usual, 6 megabytes, JPG.
I got many times the error indicated above, so I increased in Preferences - Environment - Maximum new image size (step by step) from the standard Gimp value up to (now) 756 Megabytes. I still get that message on GMIC functions like - for instance - Make Up.
So come my questions:
1-do I have to increase that value again? and to which value?
2-do I have to modify some other parameter in Gimp preferences?
3-is it normal that an input JPG image of 6 MB become a Gimp image of about 40 MB?
4-is it normal that some function of GMIC use in the intermediate steps so much memory?
Thanks a lot

_________________
"Where am I ?"


Last edited by dinasset on Mon Mar 04, 2013 1:23 am, edited 1 time in total.

Share on Facebook Share on Twitter Share on Orkut Share on Digg Share on MySpace Share on Delicious Share on Technorati
Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sat Mar 02, 2013 4:29 pm  (#2) 
Offline
Global Moderator
User avatar

Joined: Nov 16, 2011
Posts: 5128
Location: Metro Vancouver, BC
How much memory do you have on your computer?

_________________
Image
Gimp 2.8.18, Linux, median user
Gimp Chat Tutorials Index
Spirit Bear (Kermode)


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sat Mar 02, 2013 4:36 pm  (#3) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
Some explanation: a jpeg file is normally compressed, so uncompressed in memory it will take up a lot more. And then you have the fact G'MIC works in 32 bit float per channel per pixel, which basically means 4 times as much memory usage compared to standard 8bit RGB. The calculation will be something like: height * width * 3 * 4 (bytes). And that's before any operations... if you use a filter that has to duplicate layers you instantly double the memory usage (and sometimes a lot more layers are needed).


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sat Mar 02, 2013 5:49 pm  (#4) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
1. Since GIMP uses tile based image processing, increasing Tile Cache size would be more useful than increasing max image size.
When GIMP switches to GEGL libraries (v2.10) it is going to use GEGL buffers instead of GIMP tiles + OpenCL graphic acceleration => image render time will be reduced 20-30%.
2. See #1. You can also assign all of the CPU cores to GIMP, but the problem is that multi-threading in GIMP is not always working yet (in fact, more often it doesn't). The dev's need to work on that one. Images get bigger, processing time is crucial. Although, until GIMP uses GPU-based image processing (vs CPU-based) there always be some lagging and plugin crashing, especially on huge size images.
If increasing tile cache size didn't help the last thing you can try is to turn off the Color Management in Preferences. I know it's not the right way to do things in GIMP, but in most similar cases it helps (after you get done with the particular image you can turn it back on).
3. That is absolutely normal, XCF files store a lot of info, like transparency (X KB for every pixel of transparency), layers, paths, selections. In-session file-size can be as big as 1GB, because of all that info plus undo history is also stored in caches but when you save XCF it drops some of the unnecessary info and 1GB in-session file becomes 50-60 MB.
4. I'm not sure about gmic-gimp api interactions, but gmic uses its own programming language. And it seems to be noticeably slower than C plugins (that is plugins written in C language), even 32-bit ones. You won't see a difference on small-size images, but when you try to process 20000 x 10000 px raw images (which is quite normal for nowadays pro digital camera raw images) imported to GIMP with UFRaw plugin, gmic would crash 7 out of 10 times on certain filters. The same with same dimension jpeg's at 7-8 MB in size. Still, I'm not saying it's gmic's fault, it could be looping back to GIMP, after all. Try online gmic and see if there are the same problems. I've never tried online gmic so i can't tell if that is the case.
I'm on 64-bit Win 7 16 GB RAM i7 8-core CPU with all 8 cores and 2 GB for image size and tile cashes size dedicated to GIMP consumption. Still have the same problems as you on big images.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sat Mar 02, 2013 8:09 pm  (#5) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
Interesting K1TesseraEna, a 20000x10000 float image would be over 2GB and over 3GB with alpha channel.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sat Mar 02, 2013 9:35 pm  (#6) 
Offline
GimpChat Member
User avatar

Joined: Sep 22, 2012
Posts: 2089
Location: US
garagecoder wrote:
Interesting K1TesseraEna, a 20000x10000 float image would be over 2GB and over 3GB with alpha channel.


That's right. I've been working on some NASA huge resolution images not long time ago (see my The Earth topic here on GC).
Opening a 6 MB 21600 x 10800 jpg image in GIMP 'weights' 1.1 GB, just adding an alpha channel to it adds another 0.5 GB right away. Adding a transparent layer filled with BG color on top of it increases the floating file size up to 2.5GB. At this stage you can see how gimp is really choking and tiling slowly.

Tried to attach one of the files but even after double compression it is more than 5MB allowed on this site. 'Save for web' and 'Save with RIOT' plugins can't handle this much resolution as well. But there is plenty of Whole-Earth Images on the web some with 30000 x 15000 resolution (NASA website). Anyone curious can download such an image and try it for himself.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 2:15 am  (#7) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
Many thanks for the detailed info given.
Following K1T indications I modified my GIMP environment to these values:
1-Tile cache = 3 gigabytes (maximum allowed)
2-Image size = 1 gigabyte
3-Colour management = off
Still GMIC MakeUp crashes, with the following message:
"*** Error in ./make_up/edges/gradient_norm/*repeat/*local/ *** [instance(0,0,0,0,00000000,non-shared)] gmic<float>::assign(): Failed to allocate memory (182.3 Mio) for image (4608,3456,1,3)."
(additional info: make-up on a smaller image - 350 kb - works with no problem)
My system is as follows:
- IntelCore i5 750 2.67 GHz
- RAM 4 GB Dual Channel DDR3 668 MHz
- hard drives 2 * 1 Terabyte
Is this system insufficient to run on my photos GIMP + GMIC?
Very bad news, if so.....

_________________
"Where am I ?"


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 4:56 am  (#8) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
I've tried using that filter on images larger than you say and it works for me (ram usage hits about 2GB), so can you give us some more info?

- G'MIC version
- exact size of the starting image (height, width, does it have an alpha channel)
- settings of the make_up filter

then I'll test to see if it goes wrong for me, thanks.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 5:58 am  (#9) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
Thanks GC for your support.
Well, first of all, because - based on your questions - I suspected to have some wrong version of GMIC, I removed the gmic files and reinstalled using the installation file provided (gmic_gimp_win32_3.exe), downloaded on January 28, then I updated the definitions once entered into gmis to have the most current ones.
- on the GMIC panel the version shown is 1.5.4.0
- the image size is in pixels 4608 * 3456
- the image is jpg and open as a background layer, thus w/o alpha channel
- setting are the default (space 15, value 4)
I'm hoping you can reproduce the error.
(BTW after reinstalling gmic and updating scripts, the error is still there)

_________________
"Where am I ?"


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 6:21 am  (#10) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
Hmm that's very strange, everything the same as mine except I'm using 64 bit. I'll test with 32 bit command line to see what happens.

*edit*
Good news (well in a way :hehe ), I get the same error with gmic 32 bit version from command line but not 64 bit. So we know two things: it's nothing to do with gimp and it must be to do with assigning enough memory in 32 bit (I think the limit will be 2GB of address space). What we do from here I'm not sure though :?


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 6:39 am  (#11) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
Thanks GC.
In the meanwhile, I have new infos:
- just doing a very little crop (image reduced to 4264 * 3168) MakeUp works
- even saving the result exporting it as png, reopening it, the script works
There is something magic in the original size?....

_________________
"Where am I ?"


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 6:45 am  (#12) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
The image size must just be very close to the 32bit memory limit for that filter, it's possible the only way around it is to use 64 bit (which of course for you means a new OS!). There are tricks to get around this sort of limit in code but it can be slow and I doubt it will be done when 64bit works.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 6:50 am  (#13) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
...also: the reduced (in pixels) image is much bigger than the original (being png, I suppose): 12.6 MB instead of 6.6 MB, which means that the problem is NOT related to MB size but maybe to pixel size ?!?

_________________
"Where am I ?"


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 6:53 am  (#14) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
Yes that's right, the image file size on disk is almost irrelevant because different formats have different compression. All that matters once it's loaded is the dimensions (height, width, channels).


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 7:03 am  (#15) 
Offline
GimpChat Member
User avatar

Joined: Jan 20, 2013
Posts: 14816
Location: roma, italy
OK! Many thanks for your excellent support, now I'm sure it's not my fault.
I'll try to understand the exact limit in pixels for a 32 bits memory.
Hope some other gimp-er will benefit of the infos you gave me.
Thanks a lot again.
(btw, I don't now how to "close" a topic in gimp-chat...)

_________________
"Where am I ?"


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 7:14 am  (#16) 
Offline
GimpChat Member

Joined: Sep 24, 2012
Posts: 275
Location: Scotland
:bigthup No problem, I'll add something to the gmic wiki about this since I've seen a few questions about large images lately.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 4:33 pm  (#17) 
Offline
GimpChat Member

Joined: Apr 12, 2010
Posts: 5870
Quote:
when you try to process 20000 x 10000 px raw images (which is quite normal for nowadays pro digital camera ..raw images ...it crashes)


I think is a more then reasonable result :
it is true that for nowadays pro digital camera size as 20000x10000 are not unusual

But on the other hand is expected a 64bit OS and plenty of Ram to edit image at that size,
for sure can't be done with a 32bit OS limited to 2GB ram max :
of that 2GB about 500MB are needed by the operative system to display itself and what left can't be sufficient to edit at similar sizes

_________________
My 3D Gallery on Deviantart http://photocomix2.deviantart.com/
Main gallery http://www.flickriver.com/photos/photocomix-mandala/
Mandala and simmetry http://www.flickriver.com/photos/photocomix_mandala/

Image

Mrs Wilbress


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 4:36 pm  (#18) 
Offline
Global Moderator
User avatar

Joined: Apr 07, 2010
Posts: 14182
dinasset wrote:
OK! Many thanks for your excellent support, now I'm sure it's not my fault.
I'll try to understand the exact limit in pixels for a 32 bits memory.
Hope some other gimp-er will benefit of the infos you gave me.
Thanks a lot again.
(btw, I don't now how to "close" a topic in gimp-chat...)


Go to the first post in the thread and edit the subject line and put solved next to the title

_________________
Image


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 7:58 pm  (#19) 
Offline
GimpChat Member

Joined: Feb 13, 2013
Posts: 86
The Make Up filter is mine, and I have updated it, to give more options and reduce memory usage.

There are now two options (fast and limit hue range) which are disabled by default, so that the the filter will run with about half the memory requirements. Previously you did not have these options and the were hard coded as enabled, so the filter was using more memory than always necessary.

The main part of the filter is also a bit more memory efficient.

The penalty is that without the fast option, the filter is much slower.


Top
 Post subject: Re: GIMP environment setup - Photoretouching - GMIC usage
PostPosted: Sun Mar 03, 2013 8:22 pm  (#20) 
Offline
GimpChat Member

Joined: Feb 13, 2013
Posts: 86
dinasset wrote:
Still GMIC MakeUp crashes, with the following message:
"*** Error in ./make_up/edges/gradient_norm/*repeat/*local/ *** [instance(0,0,0,0,00000000,non-shared)] gmic<float>::assign(): Failed to allocate memory (182.3 Mio) for image (4608,3456,1,3)."


This tells me that the problems is with the 'limit hue range' section of the filter, so if you disable that in the updated filter, hopefully it will work. The results will be slightly different and I suggest set G'MIC to create a new layer for the output. Then you will probably have to add a layer mask in GIMP to have the effect only on the skin tones.


Top
Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


   Similar Topics   Replies 
No new posts Attachment(s) SETUP MULTIPLE GUIDES

2

No new posts Gmic

2

No new posts Attachment(s) GMIC not installing

2

No new posts Attachment(s) GMIC Challenge

24

No new posts Attachment(s) GMIC in the menu

10



* Login  



Powered by phpBB3 © phpBB Group