Question VAM Memory (RAM) Issues have gotten very bad for me. Can anyone help?

[starplat1num]

Hey all,

Have been using VAM for a few years now, both in desktop and in VR using my oculus. Previously I was running on 32GB ram, WITHOUT the XMP profile unlocked, as a result, I was running at the base/stock speed. During this period, VAM worked great, and only crashed every so often in VR, and very rarely in desktop, no matter what scene i was using/loading.

Since then, I made a few upgrades to my system including GPU, CPU & RAM. In terms of my RAM, i bought sticks with the same 32gb, but higher clock speeds. I then enabled XMP profile and increased the speed to the maximum my motherboard would allow. I then noticed that when loading the very same scenes, VAM would crash my system, as in the screen would go black, flash for a period, be unresponsive and VAM will crash. I also noticed that the memory would reach a very high percentage before this close (possible leak?). I thought maybe this was an issue with the ram speed, so i lowered my speed piece by piece, to eventually reaching base, but i think would still have the same crashing issues.

I then bought another set of ram. this time with 64gb, thinking this headroom would be more than enough, but i am still crashing when loading scenes, say after loading my 3rd scene, with only one or two characters in it. This is just through using it in desktop mode by the way, when using it in VR mode, it is even worse.

I just don't know why VAM is all of a sudden using more RAM in my system than it ever used before and i am having these crashing issues whilst using the same content on super system specs?

I have attached an image of my system specs and task manager when operating VAM in a single person scene. 32gbGB seems very high? Especially for a first scene load on a fresh VAM startup This can go even further, and then it crashes. Unfortunately I don't have my old sticks of RAM to test as I sold them after upgrading.

Btw, gaming performance for my other titles via steam etc does not experience the same memory issues from what I have been able to tell.

Does anyone have an idea what is going on here?

This is my new RAM:

https://www.amazon.co.uk/gp/product/B07YV9VYYY/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

This is my old RAM:

https://www.amazon.co.uk/Corsair-CMK32GX4M2A2400C14-Vengeance-Performance-Desktop/dp/B0134EU9YA

 

[DJ]

The issue is probably your XMP profile. Try disabling it and see if your stability issues go away. My new system was crashing constantly until i did this.

If you find that was the issue, try upgrading to the latest bios version for your motherboard, then enabling it again. You may get it to work that way.

 

[Case]

The issue is probably your XMP profile. Try disabling it and see if your stability issues go away. My new system was crashing constantly until i did this.

If you find that was the issue, try upgrading to the latest bios version for your motherboard, then enabling it again. You may get it to work that way.

Does the same recommendation apply for the "Too many heap sections" error?

Running Ryzen 7 5800x + 4x 16Gb GSkill Ripjaws (https://www.amazon.de/G-Skill-Arbeitsspeicher-288-polig-CL16-19-39-F4-3600C16D-16GVKC/dp/B07X8DVDZZ), and I'm getting those "pretty often" (= "not often enough to be intolerable, but often enough to be annoying")

P.S.: Any recommendations for the location of the pagefile? I'm running VaM on my nvme 1TB primary SSD drive ("C:"), but I also have a 1TB SATA-SSD - can I move the pagefile to the slower SATA-ssd, or does that slow VaM down somehow? (does VaM even need the fsking pagefile? 64Gb RAM should be plenty?)

 

[DJ]

In user preferences > performance 2 you can see how much physical and paged memory is in use at any given time. This screenshot was taken right after startup on a clean Vam install; it looks like the page file is used even when there's plenty of RAM free.

I've always been told that the page file should be on the fastest drive you have, but I'm no expert.

 

[animetiddyenthusiast]

@starplat1num you need to raise voltage on your ram to get stable xmp most likely. You can also try reducing the overclocked speed as well as match the flck clock in bios if you're on an amd CPU.

Does the same recommendation apply for the "Too many heap sections" error?

Running Ryzen 7 5800x + 4x 16Gb GSkill Ripjaws (https://www.amazon.de/G-Skill-Arbeitsspeicher-288-polig-CL16-19-39-F4-3600C16D-16GVKC/dp/B07X8DVDZZ), and I'm getting those "pretty often" (= "not often enough to be intolerable, but often enough to be annoying")

P.S.: Any recommendations for the location of the pagefile? I'm running VaM on my nvme 1TB primary SSD drive ("C:"), but I also have a 1TB SATA-SSD - can I move the pagefile to the slower SATA-ssd, or does that slow VaM down somehow? (does VaM even need the fsking pagefile? 64Gb RAM should be plenty?)

Similar setup but have half your ram. I think you just have to accept that vam uses a ton of ram. Limit your var count and hard reset or restart the app from time to time. If you disable or reduce pagefile it will just crash when it fills up. I also disable the memory optimizations and just accept there will be crashes. I think pagefile on main disk and cache on sata SSD should be good. To reduce var count you can use a method like I described in my recent post

 

[Case]

@starplat1num you need to raise voltage on your ram to get stable xmp most likely. You can also try reducing the overclocked speed as well as match the flck clock in bios if you're on an amd CPU.

Ooooof - Ram tuning??!!? Scary! I mean ... it seems good advice, but ... I'm hardly able to do CPU-OC, and from what I've heard, Ram-tinkering is another science in itself?

May I badger you for some in-detail advice if I ever try that?

Similar setup but have half your ram. I think you just have to accept that vam uses a ton of ram. Limit your var count and hard reset or restart the app from time to time. If you disable or reduce pagefile it will just crash when it fills up. I also disable the memory optimizations and just accept there will be crashes. I think pagefile on main disk and cache on sata SSD should be good. To reduce var count you can use a method like I described in my recent post

- I've moved the VaM-cache to yet another drive, an old 2Tb HDD - @DJ did a bit of testing a while ago & apparently, the VaM-cache doesn't benefit from higher-than-HDD access speeds (the discussion is somewhere on the official Discord)

- My var repository is split between my 1Tb nvme primary drive and the secondary SATA-SSD drive (the resources on the secondary drive are linked into the repository on the primary via junction links) - mostly I keep scenes in the repository on the primary & everything else on the secondary (My rationale is that (dynamic) scenes are the most "diverse" type of resource, compared to more specialized stuff like looks-packs, texture packs or assets (CUA), and hence, I recon that scenes will benefit most from fast access speeds)

- I leave the Ram-optimization for scenes on (but not the one for presets) - usually, I can run 2h+ sessions with multiple big scene loads. Though I do periodically check the Vam's builtin Ram-diagnostics and do a hard reset every once in a while when I want to start a scene with "virgin" conditions.

- Limiting Var count .... yeah, perennial headache. Currently, I use Feelfar's varmanager to keep my var-count in the lower single-digit thousands, but I'm
looking at switching to BillPrime's varmanager. Bit easier said than done, since a switch in tool also means I have to rebuild my "workflow" from scratch.

But I hear you - lowering var-count is probably the best thing you can do for your VaM's performance.



P.S.: I've found that generally, VaM doesn't draw much more than 32Gb physical RAM, even if you hand it 64 Gb to play with. And it does use the pagefile, even if it has 20+Gb Ram left. No idea why - one would think it'd use up all the available RAM before switching to a slower storage method, but ...

 

[animetiddyenthusiast]

Ooooof - Ram tuning??!!? Scary! I mean ... it seems good advice, but ... I'm hardly able to do CPU-OC, and from what I've heard, Ram-tinkering is another science in itself?

May I badger you for some in-detail advice if I ever try that?

you can but I only really check this account when I'm horny lol so no promises on a timely reply. And I also moved vam fully to my external drive for the time being to take a break while I work on my modded skyrimvr setup so not sure why I'm even here.

Anyway to be clear I don't mean manual OC. I use the default XMP mode on my bios to get my ram to the speed its rated for. But they dont test too thoroughly so its not very stable under heavy/sustained work. All I do is change the DRAM voltage from auto which is around 1.1V usually up to 1.35V. Here's an image (not mine) of where that setting is in my BIOS.

And yeah VAM starts using my pagefile instantly. I'm usually at like 24/32GB physical memory usage and 60GB virtual once I've loaded a couple scenes. And yeah the rest of your setup is good I think and I also only have it optimize on scene load and just do manual optimization if I feel like it.

I've used both var managers you mentioned. Bill Prime's seemed good but gives massive error logs and is annoying to use if you have even a single broken var or if you've manually deduplicated assets like morphs to reduce load time. I've found this vam toolbox app to be useful sometimes as it can do some of the same cleaning stuff and can update assets without having to visit the hub. I also used feelfar's var manager for a while and its very powerful but I could never figure out how tf to use the dependency checker. Its also not good for managing massive amounts of vars. If you keep small amounts and like the profile management its great for that but it was just geting to be more harm than help so I didn't bother. it also still writes the vars to your addonpackages folder which means super long load times. As I described in another post, using the bepinex var browser plugin seems to be the best option as it only moves selected plugins to the addonpackages folder and will automatically retrieve dependencies from your repo. Just use a symlink (not hardlink or junction) from your repo to the allpackages folder. The only real downside I've found is you need to use that plugin to access the hub ingame for updates and new downloads if you want stuff to work properly, and downloading ingame is much slower for me than doing it outside the game.

 

[Case]

you can but I only really check this account when I'm horny lol so no promises on a timely reply. And I also moved vam fully to my external drive for the time being to take a break while I work on my modded skyrimvr setup so not sure why I'm even here.

Anyway to be clear I don't mean manual OC. I use the default XMP mode on my bios to get my ram to the speed its rated for. But they dont test too thoroughly so its not very stable under heavy/sustained work. All I do is change the DRAM voltage from auto which is around 1.1V usually up to 1.35V. Here's an image (not mine) of where that setting is in my BIOS.

And yeah VAM starts using my pagefile instantly. I'm usually at like 24/32GB physical memory usage and 60GB virtual once I've loaded a couple scenes. And yeah the rest of your setup is good I think and I also only have it optimize on scene load and just do manual optimization if I feel like it.

Ok, thx! Haven't gotten the heap sections error in week - maybe the BIOS update & reducing some morphs helped. Good you said it wasn't your rig (2133MHz? What kind of shitty XMP profile is that? The Zen7 3800x needs 3200Mhz?)

Still have some really looks .vars -How can ppl be so stupid and still have "reloaded" folders in their .vars in 2023? Have to holler a bit @SlamT. And some other really ugly old stuff that's referenced by 30 or so people. It's not just UniversenS - KittyMocap & other veteran creators too.

I've used both var managers you mentioned. Bill Prime's seemed good but gives massive error logs and is annoying to use if you have even a single broken var or if you've manually deduplicated assets like morphs to reduce load time. I've found this vam toolbox app to be useful sometimes as it can do some of the same cleaning stuff and can update assets without having to visit the hub. I also used feelfar's var manager for a while and its very powerful but I could never figure out how tf to use the dependency checker. Its also not good for managing massive amounts of vars. If you keep small amounts and like the profile management its great for that but it was just geting to be more harm than help so I didn't bother. it also still writes the vars to your addonpackages folder which means super long load times. As I described in another post, using the bepinex var browser plugin seems to be the best option as it only moves selected plugins to the addonpackages folder and will automatically retrieve dependencies from your repo. Just use a symlink (not hardlink or junction) from your repo to the allpackages folder. The only real downside I've found is you need to use that plugin to access the hub ingame for updates and new downloads if you want stuff to work properly, and downloading ingame is much slower for me than doing it outside the game.

Yeah, not using BillPrime's tool atm, since I found that it doesn't compute some dependencies correctly. FeelFar - what you said. It's superb for tinkering with single .vars (with the new "details"-tab, you get dependencies and dependents, both .var and local), but it lacks some of the categorizing features that BillPrime implemented quite nicely.

It's a bit buggy though - always get a runtime error when it scans dependencies. Looks a bit like an artefact leftover from an old preset, but I can't find it anymore.

VamToolBox was meant to be a softlinker/deduper, but Kruk2 lost interest halfway through. FYI - I'd leave the "dependency eraser" in the VarFixer tab alone. The library it uses to rezip the archives is compatible with VaM, but 7Zip flags a Headers Error, and cotnoyoume's ResourceExtractor ignores them completely.

 

[animetiddyenthusiast]

Ok, thx! Haven't gotten the heap sections error in week - maybe the BIOS update & reducing some morphs helped. Good you said it wasn't your rig (2133MHz? What kind of shitty XMP profile is that? The Zen7 3800x needs 3200Mhz?)

Still have some really looks .vars -How can ppl be so stupid and still have "reloaded" folders in their .vars in 2023? Have to holler a bit @SlamT. And some other really ugly old stuff that's referenced by 30 or so people. It's not just UniversenS - KittyMocap & other veteran creators too.




Yeah, not using BillPrime's tool atm, since I found that it doesn't compute some dependencies correctly. FeelFar - what you said. It's superb for tinkering with single .vars (with the new "details"-tab, you get dependencies and dependents, both .var and local), but it lacks some of the categorizing features that BillPrime implemented quite nicely.

It's a bit buggy though - always get a runtime error when it scans dependencies. Looks a bit like an artefact leftover from an old preset, but I can't find it anymore.

VamToolBox was meant to be a softlinker/deduper, but Kruk2 lost interest halfway through. FYI - I'd leave the "dependency eraser" in the VarFixer tab alone. The library it uses to rezip the archives is compatible with VaM, but 7Zip flags a Headers Error, and cotnoyoume's ResourceExtractor ignores them completely.

Yeah I mainly used toolbox as an updater. As for the runtime errors in feelfar's program those occur anytime you try to run another task when a thread is still active. Sometimes this is just from being too hasty and running another task without waiting for it to finish but the also program doesn't always properly close it's threads. You can use task manager or ideally something like procexp or process lasso to monitor this and kill the entire process tree when necessary but this is annoying and just one of a few good reasons to leave it behind

And yeah honestly the constant issues with badly packed vars and missing or duplicated dependencies were taking too much time so I've been taking a break. Part of me is tempted to just manually unpack and repack some stuff as that would be more reliable than the tools which all seem to have some kind of flaw, but it could also go very poorly.

 

[adamgali]

One word: disable caching. Slower but stable.
You're welcome.

 

[atani]

View attachment 273608
One word: disable caching. Slower but stable.
You're welcome.

What's the reasoning for disabling cache as a solution to crashes?

On my part, disabling XMP reduces considerably the chance of VaM crashing, it just almost never crashes unless I abuse it.

 

[adamgali]

What's the reasoning for disabling cache as a solution to crashes?

On my part, disabling XMP reduces considerably the chance of VaM crashing, it just almost never crashes unless I abuse it.

Caching eats up a lot of memory, heap, once disabled, memory usage drops to almost a half and VAM becomes more stable.

 

[atani]

Ok, something I need to check then.
For the post topic I still think XMP profiles is the problem source, or at least part of the crash problem cause. On my part disabling this did wonders and didn't have to worry about anything.
Pity that the OP didn't come back with an update.


WTF @Sween, what's with the downvote?

 

[Sparowe]

I have a question for those of us here using "Var Manager" by feelfar. After the program does it's thing and creates the "Tidified and Redundant" folders respectively can I empty/delete/move/etc all the files it deemed "redundant"? I'm under the assumption that everything sorted into this folder is everything the program found having multiple copies of the same file or files that were either obsolete or older file versions no longer in use. My redundant folder alone is closing in on 2TB while my Tidified folder is sitting at 900GB. I've been searching for an answer on the discord or by simply googling but haven't had any luck getting a definitive "yes or no".

Thanks and sorry if me asking here in this thread was inappropriate but I was also looking up info regarding the heap crash error I just now got.

 

[Sparowe]

Ok, something I need to check then.
For the post topic I still think XMP profiles is the problem source, or at least part of the crash problem cause. On my part disabling this did wonders and didn't have to worry about anything.
Pity that the OP didn't come back with an update.


WTF @Sween, what's with the downvote?

I brought you back to normalcy

 

[3rdStoryline]

Yup like another has said, You may need to raise your RAM voltage. However, you also don't want to just max out the speed of your RAM to whatever your board can handle. Your CPU prefers a specific speed. The further you are from that speed the higher the chance you'll run into issues. Motherboard manufacturers often allow for faster RAM speeds just in case a CPU comes out that prefers that higher speed and uses the same socket.

I'd check what the preferred RAM speed is for your CPU. If your board has your current RAM on it's supported list, then when you Apply the default XMP(or DOCP(EXPO) if your AMD) profile it would clock the RAM (if the ram can go that fast) to what your CPU prefers. If you are still experiencing issues, then you need to either Update your BIOS(in hopes they have resolved issues with RAM) or manually increase your Voltage on the RAM.

 

[VRmatt]

I had the 'too many heaps error' also.
I raised the RAM voltage yesterday and also ran VamSorb.
Issue solved so far!

**Edit after testing:
VaM runs more stable, & faster, but loading complex scenes still results in the GC/heap error.