HiRes Image sequence uses up all the RAM in seconds and crashes VPX

Scenestealer wrote on 9/11/2017, 4:32 PM

I wonder what others experiences have been with the above problem which I have been trying to help a member with in the German forum.

https://www.magix.info/us/forum/arbeitsspeicher-voll-32gb--1195354/

I tested a project with 600 hires 14 MegaPixel images from a timelapse sequence produced in my Lumix camera, using (his) method described of allowing VPX to create a second movie in the project and automatically create and place a video of the sequence in the current Movie ( This occurs after saying yes to a prompt that appears when dragging a large number of images eg. a timelapse sequence to the Timeline. It is these many files that are using up all (your) memory and creating an "Access Violation" error in Windows. Just playing the timeline with these images in the project uses up RAM at a rate of Gigabytes every few seconds, as does Exporting that section of the project.

The best you can do is to drag all these images (with a duration "1frame" set in the image import duration in Program Settings) into a new project and then try to export to Magix Video (.mxv) format at your projects "Movie Settings" resolution, hopefully before all your RAM gets used up. Note:- do not playback the timeline before trying this. Then bring this .mxv into your main project, after deleting the sequence and images VPX previously created.

The other better solution is to convert the images within your camera (if it has this capability) to an .mp4 movie and import that to VPX.

As Matthes has suggested, it may help to manually set the Windows Virtual memory size to something huge also......

Peter

Comments

browj2 wrote on 9/11/2017, 10:19 PM

Hi Peter,

I don't have the material to test this out the same way and since I am still out camping, I only have my Surface Pro 4 with 8GB RAM and using the latest patch that came out on Monday for VPX.

I put 640 photos into a folder taken using 3 cameras - DSLR, and cell phones BB and iPhone, with resolutions of 5184x3456 (3:2 but is supposed to be 16:9), 3264x1836 (16:9), 4032x3024 (4:3).

I dragged all onto the timeline. For the record, here is the message that pops up:

I had never tried importing in bulk before, so this was new to me. It creates a nested sequence, which I have used before.

Then I exported the main movie, the one with the nested sequence. It started fine albeit slow with 90-100 CPU usage. At about 35-40% I was getting low CPU usage, but 100% memory and disk drive usage. Then I got the request to close some programs, VPX. I didn't. Then got a memory overload error message, the project did an auto-save, then I clicked the error message off and VPX crashed.

I did not follow your instruction about not playing back the movie. I did it once, then deleted everything, including the second movie, then without closing and reopening VPX, I re-imported the 640 photos, then exported.

Did you export the nested sequence movie, or the one with the photos?

Edit: I just tried exporting the movie with the images and it went to 11s of the 21s movie before crashing.

Re-Edit: Strange, my DSLR is supposed to be set up for 16:9 but the images came in a 3:2 and the movie changed to 3:2, but the main movie with the nested sequence stayed at 16:9.

I exported the images movie as a movie, not to Magix format and imported the movie into a new project. It automatically switched to 3:2 but I changed it to 16:9 NTSC, then exported to mxv and it worked.

Scenestealer wrote on 9/12/2017, 12:14 AM

Hi John

Thanks for testing.

Looks like you have found what I found when exporting via the nested movie after following the VPX prompts.

As I implied in my second paragraph, I could only avoid crashing during export by importing all the still images into the timeline instead of letting VPX create a nested sequence. Even then it just completed by the skin of it's teeth, with maxed out RAM (15GB odd) and some heavy paging to the SSD Page file. The problem seems to be that using the second movie and main movie sequence route, the program can not release, or release fast enough, the memory to the disk Page file (SSD or not).

I have found a Program Setting that stops the gobbling up of the available RAM during preview but unfortunately not export, ie. Video/Audio Tab > Other > tick "Reduce resolution for Preview (increased performance)" . I suggest this scenario is just what that setting is there to help (and is not the same as reducing the resolution with the Lightning Bolt).

Peter

johnebaker wrote on 9/12/2017, 3:49 AM

I have tested this, not having hires images I used 2900 1920 x 1080 images (4 hour time lapse sequence).

VPX loaded all images and created the nested sequence - combined video? - no crashing, CPU 62-63 %, RAM (16GB) usage 31 - 32%.

Exported to MP4 with HWA (Intel HD4600) same resolution 1920 x 1080 - CPU 30-31%, RAM maxed out at 91%, C: drive maxed out at 100%.

I did find the VPX slowed down considerably during export from a rate of approx 50 frames per sec to 6 fps.

Looking at the performance monitor I found the 'offending' service was Memory Compression - a new feature introduced with Windows 10 to speed up paging, reducing page faults by compressing/decompressing data to/from the page file.

@browj2

Did you notice or monitor the processor for slowing down due to the thermal management lowering the clock speed to keep the processor within limits?

HTH

John EB

browj2 wrote on 9/12/2017, 8:23 AM

@johnebaker

The export process did slow down to a crawl, about 1 fps., but I did not monitor the processor for slowing down due to thermal management....

Scenestealer wrote on 9/12/2017, 7:13 PM

Hi John EB

Looking at the performance monitor I found the 'offending' service was Memory Compression - a new feature introduced with Windows 10 to speed up paging, reducing page faults by compressing/decompressing data to/from the page file.

Interesting that John - I could see that band in the graphic of the memory tab in TM if you hover the mouse over it it says something about compression. This band grows then suddenly shrinks during export and I assume it may be corresponding to the memory being compressed before being paged out to virtual.

Thanks for your tests but I think the key is the Hi resolution of the images is creating problems. Actually I read something in the FAQ's at Magix support2 some time ago that they recommend resizing stills to the Res. of your project before importing to "Improve performance".

Using the nested movie method do you see the extreme cumulative increase in RAM usage if you play the same section several times? ( presuming you do not have the "Reduced resolution for preview" selected in Program Settings>Video Audio tab that I mentioned above).