Hi everyone,

for my very first message I just wanna say how much I enjoy using GIMP and related stuff (GAP / G'MIC etc..), I'm no artist really but I do a lot of "abstract" images, and animation pieces which I sometimes put to music ...
Everything goes smoothly with the animations.. until.. (usually) around the point my xcf file is 2.1GB's in size. After successfully saving the project many times up until that point I eventually get a message telling me of some problem (see attachment) when saving.. if I try to reopen the file sometimes hundreds of layers will just be missing - I have successfully worked around it using different approaches each time including splitting images to frames and re trying, and a couple of other workarounds which can be long-winded and not always the best way to spend my time!! Anyway does anyone here know of anything that might be causing this problem and any solutions to it?
Grateful thanks.

Martin.

Is it a 32-bit Gimp version or a 64-bit one? What type of filesystem is the disk? FAT32? NTFS?

In addition, to get a full picture, some hardware details might assist. In particular installed memory and available drive space.

A 2GB file is a monster, from what I read there is a 4GB xcf limit in gimp, somewhere in the bug reports so still within limits.

Out of interest what size in memory is Gimp showing when one of these files is open?

This is a 20 MB xcf which increases x3 when loaded. Hover mouse over info bar.

Thanks for taking time out to reply guys..

Firstly, it's a 64-bit Gimp version, file system is NTFS. I'm using Windows 10 and my installed memory is 16 GB DDR3, available drive space is approximately 771 GB (out of 932 GB), processor is an Intel Core i5 (quad-core) 3.2Ghz, the file size in memory of one of these huge files is about 4.7 GB, so pretty hefty to say the least.

It may be that I am going about animations wrongly, not sure, I simply add layer after layer to act as frames, which I'm guessing is the normal thing to do.. (?)... I'm not overly familiar with some of the technical (artistic) side of things, can just about grasp the technological aspects though.. but bugs and other problems relating to software I'm not great with so any help is greatly appreciated.

Cheers

Gimp can do animations, but this doesn't mean it has been designed for it. When you talk about hundreds of layers you are obviously stretching it to its limits. And how are you going to share a 2GB animation?

There can be more efficient way to work, like using Gimp to create short sequences, save the sequence as individual PNG files, and then assemble all the files as a video using ffmpeg or equivalent.

Yeah I realise it's probably stretching it, I had a similar idea about creating short sequences first, might give that a go.. I just downloaded Open Shot video editor maybe I should try my luck with that then, as Gimp isn't really designed for such things.
I use the GAP Video Encoder within GIMP and add sound etc to turn it into an .avi file (which ends up being 100-150MB), then post it on facebook or YouTube, straightforward enough to share it (if that's what you meant by "share it"(?))..

I would consider splitting the monster into sections.

This is going to get a bit off Gimp (and GAP) for animation and more to video.

GAP does use ffmpeg, but a very old version. Things change and especially ffmpeg syntax. What used to be simple is now a little more complicated but not impossible.

First ffmpeg for windows, If you go here https://ffmpeg.zeranoe.com/builds/ you can get a 64 bit static latest version. All you really need from the zip is the single file ffmpeg.exe

You can make an animated gif straight into a video

syntax:-r 10 input framerate -i input gif ..... -r 30 output framerate

In a Win7 virtual machine: http://imgur.com/Gchcrqp

Interesting you tried OpenShot, I sometimes use this in linux where it works very well. The Windows version is a beta and a bit flaky but worth a try. Plenty of presets for youtube etc.

It might be worth looking at Avidemux which is well established, plenty of how-to's around. Load the first video and then keep appending further sections, just keep frame size/ frame rate the same for all. It will also add an audio track to the video. A final render to a more compressed format, say mp4 for youtube.

best of luck

Thanks for the reply..

I think that's definitely the way to go, to split it into smaller parts.. I had an idea to do that a while ago but had no idea how to then 'bind' the parts into one whole (without the usual Gimp route,,) but you've given me some great links, any input from someone with better knowledge of the technology than me is cool, I'm still considering the OpenShot option, but having tested it out I can certainly agree with the 'flaky' description you give it.. it seems to want to do all the right things but doesn't manage to pull off the desired effect always.. still it's an option for simply stitching stuff together, definitely an option.

I'll look into the Avidemux, looks interesting.. the other link you gave me for the ffmpeg download.. is that just the bare bones for use with applications that use ffmpeg or is it a stand-alone thing?

I'll post a link to my youtube channel (only a few videos at present on it) if that's allowed.. and if you're interested to see what a novice artist (me) can do with GIMP alone.. would be nice to get feedback (any criticism favourable or otherwise is appreciated!)

Thanks again.

ffmpeg: Very much stand-alone and command line.
If you get the 64 bit static version the only file you need from the zip is in the bin folder and is ffmpeg.exe. That single file contains all the necessary libraries.

I am not too good with Windows these days but I knocked up a batch file to do the ffmpeg work on a series of numbered sequence files. Attached, remember to unzip it.

How to get layers out of Gimp in a numbered sequence? I use anitools, http://registry.gimp.org/node/25029 if you download that remove the .txt at the end and pop the anitools.py file into your plug-ins folder. There are other scripts/plugins that do the same.

I have been known to make a video demo at the drop of a hat, so here it is
https://youtu.be/tgYvvayFAuQ about 3 minutes.

best of luck

Great stuff, checked your YouTube vid out, very clearly explained, had a look at one of your many vids and was pleasantly surprised to see you addressing a query I have had of late, and couldn't get my head round, which is the Paths tool.. finally I can understand what it's used for.. I'm not up on creative tool processes as much as I'd like but that was all very succinctly expounded upon.. I can see it's advantages now
I replied on your ffmpeg / png-avi video on YouTube.
All the best.

Jehan, you say what?

Hmmm… the 2GB size limit really looks like one of these filesystem limits, especially with older Windows OS. But as you say it's Windows 10, 64-bit on NTFS, that would be a weird limit (I don't think they have such silly limits anymore).
Anyway if you can upload one of your XCF somewhere, a good idea could be to open a bug report and give a link: https://bugzilla.gnome.org/enter_bug.cgi?product=GIMP
This way, we can try and check the file. Maybe there is an actual file corruption during saving time. If so, that would be worth checking. Even better, if you have any reproduction steps which always result in a corrupted file on your system, we could check if they also work on ours.

As for GIMP not being suited for big animations, it's mostly true (for now) because there is no efficient way to store an animation in a XCF file, and no efficient GUI to handle the logics behind. This will change soon. That's the kind of things we do: viewtopic.php?f=25&t=15138

The idea behind 1 layer == 1 frame is just terrible as soon as you want to do serious animation (more than a few seconds). Well on the other hand, you say you do mostly abstract animations. So maybe in your case, you don't have any choice (well… not sure what an "abstract" animation entails exactly, but I could imagine that it means there is not much levels in the animation, so you can't really use the power of layers; just guessing).

Also as other noted, you should definitely break your artwork into smaller pieces (usually you would cut it in "sequences", that's what we do). Note that that's what all film-makers will do. You don't edit a single movie into 1 single file (you could imagine it would be terrabytes in the end of a feature-length movie!). So you need to break it into smaller chunks that you can actually process. Then later you edit and "link" your sequences together. An animation artwork is often incremental this way.

For info on any of our projects, we never had XCF files that big as yours. So I would guess that's probably what you have to work on to improve your workflow. :-)

Appreciate the input, thanks Jehan!

Well I've definitely got the message about breaking my work up into smaller parts first, I'm doing that now and that does help, at least my work is intact..

I'm attempting to upload an xcf (one of the corrupted ones) to my Google Drive, my internet connection is not the fastest by any means and as the file is just over 2GB this could take some time (currently it's telling me it will take 4hrs 45mins!), after that I will link it on the bug report site you mentioned. As for the reproduction steps.. what I do is simply make some artwork, then add to it or modify it in some way at each step (layer) often employing plug-ins and scripts to enhance things of course.. sometimes completely changing views from one frame to the next .. all of this is usually problem -free, but like I've said before, once I reach that 2GB mark, things seem to become problematic and the newer saves can be corrupted, missing frames, disordered in some way.. hopefully by my keeeping them smaller in size that won't happen again.

Thanks for the animation tips, I'm not in any way, shape or form a serious artist, more of a doodler lol.. the stuff I produce (which I refer to as 'abstract animation', although I'm not sure that's a precise definition..) is usually just a lot of colourful patterns and weird effects put together with no real vision as such, I often complete a work, then use GAP to animate it, after which I make some adjustments and then put music or other sound to it.. here's an example of one of my latest efforts: https://www.youtube.com/watch?v=edCZ-ZpUgnE (best viewed in 720p)

So, I'm not sure how else to achieve the results I want with animating without simply adding layer upon layer, to me this is similar to one frame following another frame and so on.. but maybe I'm wrong on this (I DO understand the concept of individual layers in general and how they work in relation to single images, and in fact some of the layers that I use as frames are themselves flattened images from multiple layers) - how would you approach it differently using GIMP as a base?

I'll update when the xcf file is uploaded... that reminds me, I'd better create an account on the GNOME bugzilla. site

MANY THANKS!!

Which category should I put the bug report in btw? The choices are: Data / General / Gimp-Python / Jenkins / libgimp / User Interface / Windows Installer (I left a few out that are not relevant)

If anyone is interested the following link is to one of my .xcf files that is large (2GB - although only about 750 layers) and got corrupted somehow: https://drive.google.com/file/d/0B1KF4x ... sp=sharing

Cheers.

I can not immediately reproduce the saving xcf error. Using one of ofnuts clever plug-ins, duplicated existing layers to create a larger file - 994 layers. That saves (2.8 GB) and opens ok here - linux laptop 8 GB memory


That little linux utility xcfinfo lists the layers in a file and reports what is expected,
original file: Version 0, 1218x914 RGB color, 751 layers, compressed RLE
larger file: Version 0, 1218x914 RGB color, 994 layers, compressed RLE
so no 'missing' layers reported.

For video:
You do need a uniform frame (canvas) size and frame rate (fps) for easily joining parts into a complete file.
Not entirely true of your file which has some small layers. The timing though, is uniform throughout 100ms = 1/10th second. So you then have a choice, use a video frame rate of 10 fps and 1 layer per frame, or say 20 fps and double up each frame. That is what that little batch file does, uses ffmpeg to generate 2 or 3 video frames for each image layer.

I read that as meaning that some aren't

Indeed not

Means another one of the total clever plugins available at

http://sourceforge.net/projects/gimp-to ... s/scripts/

Interesting.. thanks again for the input.. why would some layers be duplicated though? Or did I do that originally.. don't quite get it.
Anyway, this is the error I was shown when I opened the file before uploading it onto my Drive:



SO, it was missing some content, not sure how much now but could have been as much as 20% of the original if I remember correctly.

Also, with this file there is a lot of strange pixelation going on that was definitely not there to start with..

Oh yeah, the batch program works fine thanks, but the anitools thing, I'm not sure how to get that to work in GIMP, obviously you gave me a link to anitools.py.txt and told me to remove the .txt part, do I have to use the Python_Fu part of GIMP for that (which I've never used before.. no idea how to.. not being a programmer or anything !) or is there something I'm missing (probably!)

I'll be looking into Ofnuts' Gimp Tools - cheers.

Oh btw.. as you can see - when the file was opened the filename doesn't appear it just says 'Untitled' at the top - this happens every time an xcf file goes bonkers.

Must be some kind of protection. There are several reasons for a file to be corrupted on load, and some of them do not mean that the initial file is damaged (network error if it is on a remote system for instance). So this would prevent you to overwrite the "good" file with the partially loaded copy.