Parallelization in next G'MIC [1.5.7.0]

K1TesseraEna · **Joined:** Sep 22, 2012 **Posts:** 2089 **Location:** US

Ronounours wrote:

Updated filters to allow the use of up to 8 cores for parallelized filters.
Anyone able to check the speed gain, if any ?

I tested 1 filter so far
900x900 rgba
Rodilius 1 core 13.72 sec
4 cores 8.32 sec
8 cores 11.43 sec

Edit. My 8 core render time is greater than 4 cores but it is misleading info and can be dismissed [I forgot about two other
programs with dedicated CPU affinity running simultaneously on the background :oops:

].

YAFU · **Joined:** Nov 09, 2011 **Posts:** 726

A slightly different times than the first time. Here they are:

Kubuntu 64bit - i7-3770 with 4 cores (8 threads)
Image 6000x6000px - Smooth [anisotropic]

1- 58sec
2- 34sec
4- 27sec
8- 17sec

:coolthup

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

Ha, thanks.
YAFU : That's a great improvement indeed

.
K1TesseraEna : Well, I'm not surprised at all, the Rodilius effect is probably the one that can be parallelized the less (because of the non thread-safety of the FFTW3 library intensively used in this filter).

samj · **Joined:** May 09, 2012 **Posts:** 914

Bonjour,

Here is a test build of the latest version of G'MIC with a new 64-bit Windows environment.

K1TesseraEna · **Joined:** Sep 22, 2012 **Posts:** 2089 **Location:** US

Tried Anisotropic smoothing on big size image 8192 x 4096.
The longer computation time is - the more accurate comparison, right?
1 core 05:27.58
2 cores 02:02.42
4 cores 01:48.16
8 cores 00:54.06 [that's 5.5x faster than 1 core rendering]

There's practically only 14 sec difference between 2 and 4 cores. I ran the test twice,
the results are pretty much the same, give or take a fraction of a second.

@sam Thanks, your last build with new environment works fine (as usual) and have
improved render times a few seconds for the same image. :bigthup

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

Thanks for your tests K1, a factor x5 sounds really good to me.

K1TesseraEna · **Joined:** Sep 22, 2012 **Posts:** 2089 **Location:** US

Ronounours wrote:

Thanks for your tests K1, a factor x5 sounds really good to me.

that's simply fantastic! Thank you David!
Just wondering how much faster it could get if GMIC had the ability to support OpenGL/Cl accelerators.

lylejk · **Joined:** Sep 24, 2010 **Posts:** 12850

You asketh for too mucheth too K1Te. lol

Still crossing my fingers; jealous of those i7 folk since I still only have a Dual core AMD that's 2008 vintage. lol

lylejk · **Joined:** Sep 24, 2010 **Posts:** 12850

samj wrote:

Bonjour,

Here is a test build of the latest version of G'MIC with a new 64-bit Windows environment.

Thanks again samj; the progress bar is fixed too.

PhotoComix · **Joined:** Apr 12, 2010 **Posts:** 5870

Parallelization works really well , the location of the Parallelization controls make sense now in this experimental phases

But for the future i am convinced would be more logical and ergonomic as "global" gmic option( as now are "Output" and "Input" options ) ,and not a option to add to each filter

Because the user computer will remain the same even when changing filters, so once given the correct setting seems logical apply them automatically to ALL the other filters supporting "Parallelization" ,as for the global I/O preferences

Also the graphic interface of many gmic filters already show a often intimidating numbers of controls and add other 2 would worsen

In my vision this hypothetical global option for Parallelization will be applied only for the filters supporting it , and make no differences for the others

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

Definitely not a good idea

Parallelization of image processing algorithm is actually a complex thing, and two different filters may (and probably will) use different techniques to be parallelized.
Currently, all the parallelized filters in G'MIC use the same technique, that requires two parameters (number of cores and overlap), so you may think that these two options could be extended to all filters in the future (hence making them as global parameters would be right), but this is not the case.
For instance, the 'Warhol' filter could be easily parallelized without needs for an 'overlap' parameter.
The 'number of cores' could be perhaps moved as a "global" option, but this could be interpreted as the fact that all filters can be parallelized which is wrong.

samj · **Joined:** May 09, 2012 **Posts:** 914

Bonjour,

I do not know if it can be interesting in programming but I enclose a small code to determine the number of "cores" on Windows.

To test you can click on the file "test.bat".
Compilations are made under Mingw and Mingw-W64.

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

Thanks samj,
I'm currently adding a new substituting expression in G'MIC that returns the number of available code.
Borrowed this piece of code from stackoverflow : http://stackoverflow.com/questions/1503 ... -a-machine

samj · **Joined:** May 09, 2012 **Posts:** 914

@Ronounours

For Windows, this is the same code (as Microsoft recommends).

samj · **Joined:** May 09, 2012 **Posts:** 914

@Ronounours

On the road to 1.5.7.1, expression '@*'

To compile under Windows I had to change the line 4051, file "Cimg.h" :
return cimg::max(1U,res);
by
return res;

Error :
./CImg.h:4052:5: warning: control reaches end of non-void function [-Wreturn-type]

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

Thanks Samj, this has been corrected.
I'm adding some new options to automatically determine the 'ideal' number of threads when doing parallel processing.
I've also modified the FFT code so that it uses the multi-threaded version of the FFTW3 library call.
Rodilius should be way faster in upcoming 1.5.7.1.

Iain · **Joined:** Feb 13, 2013 **Posts:** 86

Ronounours wrote:

I've also modified the FFT code so that it uses the multi-threaded version of the FFTW3 library call.
.

That is great news. My noise reduction detail recovery method uses FFT and is also pretty slow at the moment.

Ronounours · **Joined:** Jan 03, 2011 **Posts:** 1656

I've just posted new binaries for the 1.5.7.1 beta version of the G'MIC plug-in, at:

https://sourceforge.net/projects/gmic/files/beta/

(available for linux32, linux64 and win32).
If you are interested, you can test and tell me how this new Fourier transform behaves now.

lylejk · **Joined:** Sep 24, 2010 **Posts:** 12850

Question for samj. If I run you version that get's the files online, does it get the files from git, or does the program poll files from your site and then compiles the files from there? Just curios. If the script gets from git, then I can always run the lastest beta. Again, I'm no coder and clueless about this stuff and just know it works. lol

samj · **Joined:** May 09, 2012 **Posts:** 914

Bonjour,

@Lylejk

- Download http://www.aljacom.com/~gmic/Makefile_G ... 64bits.zip
- Copy
\Makefile_GMIC-32bits-64bits\scripts-64bits\beta\compile_gmic_gimp_plugin_beta_64_win.sh
in
C:\petit-msys-mingw-64bits-gimp28
- Start MSYS clicking "start_MSYS.bat".
- Enter the two command lines that follow.
cd /env
./compile_gmic_gimp_plugin_beta_64_win.sh

I'm sorry if the answer is not good but I do not really understand the question.