Memory Leaks

[Bioxx 796]

I need some specifics from folks. Were you performing a particular task a whole bunch before the memory usage got out of hand? Does it only happen in SSP or can it happen in SMP as well? How much RAM are you allocating to minecraft? How long does it take to get out of hand. Memory doesn't just randomly begin to leak, something somewhere is setting aside memory and not releasing it. The TFC codebase is quite large so it may be tough to find. Our number one priority in trying to solve this is for you to provide us with the fastest way possible to replicate the memory leak. Only then can we begin to track it down.

[Omicron 62]

(And people should remember to do their tests only with the base TFC installation, no additional mods. Not even a texture pack. If you don't get any leaks that way, slowly start enabling extra stuff and check for when it starts happening.)

[Emris 156]

I tested this with allocating 1024MB in SSP.

Opened my system monitor and started the game.

Then in game i did nothing. I was just standing idle in my hut.

There were a few sheep, chicken and cows around.

3 fruit trees and 5 barrels were also near my hut. A few barrels were sealed.

On my system monitor the memory went slowly, but constantly, to 8GB.

The lag starts when the swapping begins.

I was also looking at memory usage with F3

Used memory was cycling between ~30% and ~55%.

Allocated memory was at 100% after a few seconds and was also rising very slowly after every Used memory reset.

My computer physical memory was growing around 100MB every minute or so.

But started growing faster at night after monsters spawned.

I also tested this with Optifine. Same thing except the CPU usage was smaller. That's probably because i set it to 40fps max.

I also tested this in my development environment with the same world and can confirm the same effect.

Then i went far enough so that the chunk where my hut was, gets unloaded.

The memory leak stopped!

Even at night time with all the monsters spawning had no negative effects.

We can rule out animals because at the new spot, they were also there.

So that leaves barrels, fruit trees, a forge, food table, stone anvil, fire pit, planted oak/fir/willow saplings and i also had a small garden with a few veggies growing. I can give a full list if needed.

That would be about all that is in, and around, the hut.

Also good to know is the leaked memory only gets freed after a client restart. Exiting and reentering the world is not enough.

I hope this helps

[Gwtheyrn 70]

I get the exact same thing as the Emris with the exception being that I didn't have a forge or any fruit trees. I had a food table, some veggies, three barrels, a fire pit, some log piles, a stone anvil, and some white cedar saplings growing.

[Omicron 62]

Can't be the fire pit since it didn't leak for me. I played for a couple days and the only thing that consumed memory was the act of loading new chunks.

Here's the rundown:

Total system memory used at start: 1.75 GB / 6 GB

Launched a completely fresh installation (just copied my saved world over) with args:

java -jar TFCLauncher.jar -Xms256M -Xmx768M -Xss2M -XX:MaxPermSize=128m -XX:UseSSE=3 -XX:+AggressiveOpts -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode

Memory assigned to java process: 56 MB

Logging in, launching game to main menu

Memory assigned to java process: up to 775 MB, after a while of sitting idle it gets cleaned to 622 MB

Loading my singleplayer world, which was created with Hotfix 7 and continued with Hotfix 8

"ID 106 is mismatched between world and game"

"ID 31 is mismatched between world and game"

Welp... guess I'm making a new world instead.

But, on selecting "No" to not continue, the game freezes on "shutting down internal server" and I need to kill it via task manager.

Re-launched client, created new world. Rhyolite, ash wood, chicken and cows - lucky! Instantly the world starts throwing native copper and cassiterite at me. I have to cut down trees to have enough logs to mark the positions since I can't possibly get enough cobblestone.

Loading into world:

Memory assigned to java process: 1,575 MB

Total system memory used: 3.40 GB / 6 GB

Spent day 1 punching grass and rocks and leaves, cutting down trees and digging a 3x3x3 hole in the ground in the evening.

Memory assigned to java process: 1,605 MB

Total system memory used: 3.43 GB / 6 GB

Night 1: Firepit GUI still exhibits annoying tendency to throw every item the player touches to the ground. Observation: although you need two rocks to start knapping, you can consume your last rock by using the new 'place fresh rock into the knapping interface' mechanic.

Day 2: God dammit game, stop giving me ore, I need rocks!

Night 2: Sitting around with nothing to do since I'm out of wood and stone. I guess I could count my stacks of cassiterite and native copper...

Memory use isn't changing so far.

Day 3, Night 3: More of the same; finally making some headway on the resource gathering front.

Memory use is up to 1,800 MB on the java process.

Day 4: Ranging a little further, maybe a 5 - 10 chunk circle around my 'base'. Looking for clay and flux stone. No such look, but found sheep. Still hitting rocks, grass and trees and marking ore spots with logs and cobblestone.

Memory assigned to java process: 1,996 MB

Total system memory used: 4.12 GB / 6 GB

Memory use is not going down during night 4, although I spend it all sitting in my little hobbit hole. Which is now 4x3x3 and roofed with logs.

Day 5: Setting out for a bigger expedition. Promptly it starts to rain. Found clay, also pumpkins. Made my way back home.

Memory assigned to java process: 2,361 MB

Total system memory used: 4.43 GB / 6 GB

Night 5: Unfortunately I need to quit now. It seems the main memory consumer is generating/loading new chunks (I have view distance Far set), and it doesn't seem to unload them at all.

[Gwtheyrn 70]

Chunk loading, unfortunately, doesn't explain how the system memory usage rises while sitting idle, though.

[transcengopher 349]

I'm getting lag spikes when I'm turning around. Basically, at some point I can stand in place, turning my viewpoint in the game, and lagg the hell away. -Xms512, -Xmx2G.

But. I have Optifine installed, and those spikes don't look like TFC problem. I shall run some tests now on clean install and see how things'll turn out for me.

[Zerren 75]

Tested with 512 minimum 1024 max, default TFC installation hotfix 9, new world.

Tried as Emris_Morath pointed out; nothing if it was a fresh world. The moment I put down barrels, veggies, and fruit trees.. Gone, was my CPU.

Singleplayer, by the way

EDIT:

Removed them all, and made a 9x9x9 cube of barrels, and watched as my game disintegrated. The frame rate nosedived the moment I finished the construction, and the memory FLEW UP. Locked up after 2 minutes.

Did the same with fruit trees and veggies. Nothing.

It seems barrels are the culprit? They cause MAJOR fps drops as well. ~120 down to ~5 with the cube of barrels.

EDIT 2:

It's DEFINITELY barrels for my install. I tried fullscale.. just to be sure. A 50x50x50 cube of barrels. My game instantly locked up and my memory usage went to 8 million. I tried again in a fresh world to see how far I could get manually placing the barrels and when I got to about the 50th barrel my fps was at 2 and memory was flying up at a rate of 100k per second.

[transcengopher 349]

It seems barrels are the culprit? They cause MAJOR fps drops as well. ~120 down to ~5 with the cube of barrels.

Which, basically, means custom blocks rendering, right? Have you tried ingot piles?

[Zerren 75]

Which, basically, means custom blocks rendering, right? Have you tried ingot piles?

I can try right now. Give me a few minutes. See edit

It doesn't necessarily mean custom rendering though; it could be the TileEntity or Container for the barrel is ticking somewhere in a loop (for the barrel seal time maybe). The FPS drop stays even when not looked at.

EDIT:

Ingot piles are not affected. While they DO reduce framerate SLIGHTLY when there are about 200 full ingot piles (which is to be expected with custom rendering), they do NOT increase my CPU usage.

[Gwtheyrn 70]

Not necessarily. We use a number of other custom blocks. My guess is that Zerren is right, that it has something to do with barrels specifically.

[platanusoccidentalis 35]

Yes, I would agree that it deals with barrels specifically. http://terrafirmacraft.com/f/topic/3745-barrels-cause-lag/

[Menoch 335]

I noticed that when barrels are sealed, and I log out or leave the chunk and then come back, they are unsealed. Could it have something to do with a timer for the barrel seal? Not sure if soehow that is having an effect? But barrels send me into a lagfest when they are on screen or in my general vicinity.

[platanusoccidentalis 35]

I have found that, as you say, the closer you are to them, the worse it gets. If I walk directly alongside them, my FPS has dropped as low as 2, compared to my normal 110 with OptiFine.

[Imarav 1]

Being in vicinity of barrels causes rapid memory leak and drastic drop in FPS over time. Removing barrels fixes the issue.

[Bioxx 796]

I'm testing the barrels now. If this turns out to be the major cause then we can all blame dunk . What is the timeframe that it takes you guys to see the memory hit 6-8GB levels?

Edit: NVM I'm finally able to duplicate this thing. My initial testing shows some basic issues with the code but I can't find what exactly is causing the runaway memory. I should be able to find it before too long though, so hopefully we'll have it today.

[Zerren 75]

I'm testing the barrels now. If this turns out to be the major cause then we can all blame dunk . What is the timeframe that it takes you guys to see the memory hit 6-8GB levels?

For a single barrel, upwards of 30 minutes, but the moment you start adding more it becomes a serious problem. 64 barrels and you don't last 3 minutes before memory is at 8 million for myself.

I tested ingot piles and fruit trees as well, it's only barrels.

Edit: This is on hotfix 9, recommended forge, not using the launcher, only mods installed. The leak happens with and without optifine installed.

[Baulten 1]

I would like to add my input: I'm experiencing no memory leaks with multiple barrels on an SMP server, and none of the people who play with me are either. This appears to be an SSP issue only.

[killster550 401]

bioxx, could it be caused by the barrels somehow unsealing and sealing rapidly within a shorter timespan, with the standard reset state being unsealed? because they also have to constantly update the item inside, calculate the duration, and block access to it nonstop, there could be an issue there

[Bioxx 796]

It's definitely render related with the barrels. I don't know WHY the model code was bad but I completely rewrote the barrel as a standard block and now no memory leak.

[killster550 401]

maybe, due to its infinitly fast updating state, it was rerendering the thing time and time again, o fast it couldnt delete the old one in time, causing a growth in memory usage (yes i am talking off of the top of my head without any form of coding experience, but it seems logical)

[Zerren 75]

If I might ask, are you keeping the full block barrel, or are you rewriting the model code?

[Gwtheyrn 70]

Wow. Bioxx does it again. Thanks a bunch for fixing that, mate.

Don't be too hard on Dunk either. It could have happened to anyone. After all, look at all the Ph.D.s they have working for Microsoft and they STILL made Windows 8.

[Omicron 62]

Windows 8 is a good OS... it's just that you're simply no longer the target demographic unless you buy a tablet.