NOT verified! Crash workaround (for players) and a solution approach (for the devs)

WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
edited December 2016 in Subnautica Bug Reporting
So, all of you who are concerned with the frequent crashes of the game.
The reason is, that the engine can't handle the threads of the code. Confirmed to NOT be related

This is a problem that's commonly known in the Software industry and is related to not multi-core CPU optimized software.

Problem is, that if the code of the software is not delicately adapted to multi-core CPUs, sometimes the threads of the software overlap in their usage of several CPU cores.
The hardware then can't decide on which core the software shall be executed and overflows or simultaneous memory assignments happen (which kills the process) .

What does this mean to you?
As a gamer:
Try the following workaround (if have tested it on my system and the crashs went from every 5 minutes to none for an hour and ongoing).
Start the game -> open the task manager -> go to the processes tab -> right-click "Subnautica.exe" -> choose "Set affinity" -> reduce the amount of cores to which the game is assigned
Start with 1 core, if your hardware is potent (should be enough since the majority of the workload is done by the graphics card anyway), and only increase the amount if you realize that isn't sufficient.

Since my cores each "only" run on 2,2Ghz (but I have 40 of them :-P), I chose to assign the process to 4 cores of each CPU.
https://www.dropbox.com/s/7x22zmv30jdtfhb/subnautica workaround.png?dl=0

I ran the game now for over an hour and the problem has not occured again.

BEWARE: you have to assign the CPU affinity each time you start the game since the process gets a new ID each time it is loaded!

As the developers:
Try to check the thread handler for a clean execution. I dunno Unity that good anymore, but I guess there should be a thread priority handler somewhere. You need to optimize the code regarding the assginment of threats to several CPU cores.

Please give some feedback whether this solution works for you, if not than it's not the real problem and I just had a lucky run.

Kind regards

Comments

  • nelserunelseru va Join Date: 2016-02-25 Member: 213441Members
    Why do you have 40 cores are u rinning a server
  • WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
    edited December 2016
    It's my workstation for music production, the server runs on a 8 core sytem
  • PW1986PW1986 Join Date: 2016-12-20 Member: 225218Members
    I think this actually makes a lot of sense. Back when the game was in a very early phase of development, there wasn't enough going on to make the threading issue either noticeable - as the content was added with each update, more and more was being asked of the CPU. Now that the game has become so much more complex, what was once a minor annoyance now appears to be a major bug.

    This corroborates several crash reports I've reviewed that show memory usage is well within acceptable bounds for my system. Furthermore I doubt my GPU is to blame for frequent crashing as it can handle vastly more demanding tasks, such as VR without any issues.

    I feel like this is going to get worse again in the next update unless some stability issues are addressed as a priority.
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    @WarpZone32 Can you try this out? You've got 8 cores, try dropping it to 4.

    This would also explain why I've not had this problem at all, as my i5 has 4 cores, which are not HyperThreading (meaning they aren't split virtually into 2 cores per core).
  • RileyTheFoxRileyTheFox Join Date: 2016-12-20 Member: 225219Members
    I've only started encountering this crash today and it seems to happen way more often than every 5 minutes. I'm not sure if it's crashing for the same reason as your solution above, but it seems to be the same reason so I went ahead and tried your workaround and no matter how many cores I set my game to, it still seems to crash within a minute or two of loading into my save. Am I possibly doing something wrong here? I'm not entirely sure myself. Though, I can post my crash log if that helps any because again, I'm not entirely sure if it's the same crash reason or not. By the way, the save I have that's crashing has 1 day, 7 hours and 35 minutes of playtime on it. I've notice people having problems with it after a certain amount of time so I'm not too sure if that has anything to do with it.

    Apologies in advance if this should be posted on it's own thread, I just figured this relates to the solution above so I wanted to see if I was messing up somewhere or not.
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    edited December 2016
    @dzedward @KillerGymSox @NoxZetheus @coldkingnowhere @nelseru @Gamerman86 @ThunderHawk2 @Dracobomba @xxthing45xx @Forman @Zafar @cloudsoda @MRHelmond @hurtfuljesse21 @mrcreepisback @aasub @Jawrippa @alex_mech @Drown @Miniwarchief @Gourmet_Guy @Late123 @CottonCorn @hdja18 @TheAdamator @BlackEDGE @Vanroth @allahsiztosbaga @WhiteFireDaemon @skinlessdream @SiaNautica9 @JamieShadez @Skope @nickfatfly @sabin77 @El_Hadschi @Whorthy @djoveryde @Nerkerz @Forgerer @QuasarSuperstar @trafri @Late123 @Switchgear @Legenbaer @Xenoto @Casual_Player @obaboune @Okiraishi


    Have a look at the OP guys and see if this helps you, especially if you have more than 4 cores (look in your dxdiag for that, or in your task manager under the performance tab).

    If this works for all of you consistently, here's a way to automatically have this done (if I understand correctly):

    An easy way to do this would be with Process Lasso, which you can download the free x64 version here. (Link to free is found at the bottom of the page if you don't want to use my direct link to it.)
    Links to some articles on their site: (as I'm not able to open the program up to figure it out at this moment, @ work)
    https://bitsum.com/tips-and-tweaks/should-i-micromanage-cpu-affinities/
    https://bitsum.com/tips-and-tweaks/when-cpu-affinity-matters/
    https://bitsum.com/tips-and-tweaks/spreading-the-load/
  • bigc00pbigc00p Join Date: 2016-12-20 Member: 225220Members
    Tried setting affinity, but no dice. On one core, my FPS stayed below 10, I gave it 4 cores and it was playable. Hopped in seamoth and was headed back to the spawn pod and I get the usual FPS drops a few times, but as soon as I get close to the pod it locks, and crashes. I have about 10+ crash reports with this same behavior. All seem to revolve around the seamoth. Really sucks because I really really enjoy this game. I'm hoping they find a fix soon.

    Also, if it helps:

    I7 920 @ 2.67Ghz
    12 GB RAM
    NVIDIA GTX980 4GB RAM
    Win 10 (10.0.0) 64bit
    8 Logical Processors
    Version: 42313
    Build Date: 12/14/2016
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    Can you all check out my thread here and tell me if you've got other processes besides Subnautica using Subnautica resources?
  • WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
    OK, sad to hear, then it's most likely not linked to thread assigning... Well, it was worth giving it a try.
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    edited December 2016
    @Wustuv & @ThunderHawk2 can I get your thoughts on these known bugs?
    All bases, including wrecked bases, always render (so build simpler bases)
    https://trello.com/c/tyLIB9ZO

    All power generators create an FPS / performance drop (including solar), increasing with each one built
    https://trello.com/c/60XWDIja

    Deconstructing parts of bases to increase performance won't work, as those resources are not released until the entire base is deconstructed in its entirety
    https://trello.com/c/N6kT4rdC
  • ThunderHawk2ThunderHawk2 Join Date: 2016-12-19 Member: 225164Members
    edited December 2016
    @Wustuv - sorry, but it looks like this solution doesn't work.

    I started Subnautica and assigned one core. I'm running an Intel i7-6700K Skylake (8 core). The game stuttered but basically ran fine, so I piloted my Cyclops over to one of the my places I know causes access violation crashes and I got an access violation crash. The problem could still be related to multithreading, but it's not a multicore issue... or at least, not multiple cores in the processor. I'd love to blame my graphics card, but I know it's probably not that either - I feel it's extremely likely that the game is closing a handle or file or something that it then tries to read from, or is trying to read from a file it's already reading from, probably across multiple threads.

    @0x6A7232 - all three of those are the same bug in my opinion, which you probably already knew. I replicated it. Symptoms include being able to see your base (or parts of it, like the solar panels) from really far away when you can't see any of the other terrain.

    If we're still looking for theories on the cause of the access violation crash and assume this bug is related, it could be that because certain graphics assets never unload, the game sometimes crashes when you get near them. The game tries to load their models or textures or some other file a second time, but oops, the file is already open. Crash. This could explain why I sometimes get the access violation crashes when approaching my base. I suppose it might explain why cleaning the cache helps... but does it really? We specifically avoid cleaning the base area during cache cleaning, so I think this bug is not likely to be related.
  • WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
    OK, I really did just have a lucky run yesterday. After coming back to my base from the lost river, the crashes started again.
    But, I also have new information: As coincidences occur, I just finished swapping my old Radeon graphics card with a new GeForce.
    And, BAAAM, problem still occurs. So I'd say we can now definitely cross out everything behind the windows graphics driver access layer.

    I still stick with the assumption that it is the thread handler, but this is where we as users reach the point of "can't help anymore", because we don't have access to the code nor the core files (scrambled).

    It's now the task of the devs to find what crashes the threads. I say the thread handler in conjunction with the CPUs RAM access is the problem point, the root cause has to be identified.
    It seems to be connected to the threads which are responsible for user created content, since if you look at the crash logs, it's most prone to happen around Bases, or in areas crowded with vehicles (just for fun I built some Moths and parked them close to my PRAWN and my Clops, crashed).
    A 5-Why analysis or a Fault-Tree-Analysis should uncover the root of the problem. I just hope they have a experienced member in Quality Management ;-)

    Kind regards
  • mrcreepisbackmrcreepisback France Join Date: 2016-12-19 Member: 225159Members
    Am i the only one to have this bug only since the last Update ? On my mind it looks like wrong textures mapping, but i don't know enougth of Unity or C# to answer correctly .
  • BDelacroixBDelacroix Florida Join Date: 2016-04-08 Member: 215511Members
    Having reduced the processor affinity to 4 of them, I have not crashed.
  • NerkerzNerkerz Austin Join Date: 2016-12-14 Member: 224953Members
    I haven't actually crashed in game in quite a while after @0x6A7232 suggested I clear cache directories (that was the 40062 Oct build). The only problems I have had in game since, are minor bugs. I say in game, because the only time I seem to crash, is when I try to exit the game. This happened in 40062 as well as currently in 42313 Dec. It doesn't happen consistently either.

    So, I have no idea if this would be related to in game crashes that were randomly happening before. I just traveled farther than I've ever gone in the game, loading multiple biomes, to try and reach my life pod which seems to be getting infinitely farther away (I don't even know if that is supposed to happen at all, but I'm guessing not like this, as I get close to it and it drifts away). I traveled to -30 18 17 (traveling for about 5 minutes straight for it in seamoth), or what the info said was the Kush biome and never crashed. I exited to main menu, then clicked quit and it crashed. This always brings up the windows notification "This program may not be working properly..." and I always have to click "close program".

    Is crashing upon exit at all related to these in-game crashes?
  • Gourmet_GuyGourmet_Guy Ohio Join Date: 2016-06-16 Member: 218647Members
    I have 4 cores, wouldn't decreasing the amount allowed to run for the game decrease performance???
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    edited December 2016
    I have 4 cores, wouldn't decreasing the amount allowed to run for the game decrease performance???

    Maybe. You can always put it back (and actually, if you decrease them using Task Manager, it's only effective for as long as the program is running; on quit, when you launch the program again, it will have default behavior).
    Nerkerz wrote: »
    I haven't actually crashed in game in quite a while after @0x6A7232 suggested I clear cache directories (that was the 40062 Oct build). The only problems I have had in game since, are minor bugs. I say in game, because the only time I seem to crash, is when I try to exit the game. This happened in 40062 as well as currently in 42313 Dec. It doesn't happen consistently either.

    So, I have no idea if this would be related to in game crashes that were randomly happening before. I just traveled farther than I've ever gone in the game, loading multiple biomes, to try and reach my life pod which seems to be getting infinitely farther away (I don't even know if that is supposed to happen at all, but I'm guessing not like this, as I get close to it and it drifts away). I traveled to -30 18 17 (traveling for about 5 minutes straight for it in seamoth), or what the info said was the Kush biome and never crashed. I exited to main menu, then clicked quit and it crashed. This always brings up the windows notification "This program may not be working properly..." and I always have to click "close program".

    Is crashing upon exit at all related to these in-game crashes?

    I don't think so. Different cause, I think. But I don't know.
  • WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
    I have 4 cores, wouldn't decreasing the amount allowed to run for the game decrease performance???

    Of course, in theory, because you dedicate less computing time to the application. But effectively, if you have a CPU with a high per-core performance, dunno, let's say 3GHz or similar, you wouldn't necessarily notice it, because the vast majority of work is done by the graphics card. The CPU on the other side is responsible for, amongst others, stuff like RAM assigning, and telling the graphics card WHAT work it has to do. That's why I think the crashes are connected to CPU "behaviour", the threads of the game (meaning all the single process flows which run in the background) are not handled effectively at this time.
  • sgtkneecapssgtkneecaps Albuquerque Join Date: 2016-12-23 Member: 225375Members
    I registered to tell you that it works - sort of. I think there's still a problem with the game, but I dropped down my cores to 4 (from 11) and it was running with some lag but not game-crashing/game ending freezing like before! It only started to get very uneasy in the Grand Reef. I'll have to fiddle with it some more. But at least there was a temporary solution.



  • NerkerzNerkerz Austin Join Date: 2016-12-14 Member: 224953Members
    I spent more time running around, exploring 9 different biomes. Although I thought it was about to crash many times (frame rate dove and then it hung up completely for 1-2 seconds upon loading new areas sometimes), it stabilized afterward. Reducing the number of cores for me just resulted in it running much worse. No crashes, but insane lag.

    I see some of you in here saying you dropped to 4 cores and it was more stable, if not fixed. I'm already at 4 physical cores, with the i7 4790k @4GHz. Although I have HyperThreading, that doesn't seem to be an issue. I actually can't get mine to crash. Does anyone else have the same processor and have been experiencing crashes? If so, maybe I could recreate a crash scenario. Might help to narrow things down. Dunno if that would be helpful, but I'm curious anyway.
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    edited December 2016
    Nerkerz wrote: »
    I spent more time running around, exploring 9 different biomes. Although I thought it was about to crash many times (frame rate dove and then it hung up completely for 1-2 seconds upon loading new areas sometimes), it stabilized afterward. Reducing the number of cores for me just resulted in it running much worse. No crashes, but insane lag.

    I see some of you in here saying you dropped to 4 cores and it was more stable, if not fixed. I'm already at 4 physical cores, with the i7 4790k @4GHz. Although I have HyperThreading, that doesn't seem to be an issue. I actually can't get mine to crash. Does anyone else have the same processor and have been experiencing crashes? If so, maybe I could recreate a crash scenario. Might help to narrow things down. Dunno if that would be helpful, but I'm curious anyway.

    For you, I'm not sure, but I think there's a way to limit a process to the four physical cores, preventing a HyperThreading virtual 'core' from being assigned. See those links I hid in the spoiler, there's an article on that: //EDIT: Looks like it's in the article "Spreading the load"//
    0x6A7232 wrote: »
    @dzedward @KillerGymSox @NoxZetheus @coldkingnowhere @nelseru @Gamerman86 @ThunderHawk2 @Dracobomba @xxthing45xx @Forman @Zafar @cloudsoda @MRHelmond @hurtfuljesse21 @mrcreepisback @aasub @Jawrippa @alex_mech @Drown @Miniwarchief @Gourmet_Guy @Late123 @CottonCorn @hdja18 @TheAdamator @BlackEDGE @Vanroth @allahsiztosbaga @WhiteFireDaemon @skinlessdream @SiaNautica9 @JamieShadez @Skope @nickfatfly @sabin77 @El_Hadschi @Whorthy @djoveryde @Nerkerz @Forgerer @QuasarSuperstar @trafri @Late123 @Switchgear @Legenbaer @Xenoto @Casual_Player @obaboune @Okiraishi


    Have a look at the OP guys and see if this helps you, especially if you have more than 4 cores (look in your dxdiag for that, or in your task manager under the performance tab).

    If this works for all of you consistently, here's a way to automatically have this done (if I understand correctly):

    An easy way to do this would be with Process Lasso, which you can download the free x64 version here. (Link to free is found at the bottom of the page if you don't want to use my direct link to it.)
    Links to some articles on their site: (as I'm not able to open the program up to figure it out at this moment, @ work)
    https://bitsum.com/tips-and-tweaks/should-i-micromanage-cpu-affinities/
    https://bitsum.com/tips-and-tweaks/when-cpu-affinity-matters/
    https://bitsum.com/tips-and-tweaks/spreading-the-load/

  • WustuvWustuv Join Date: 2016-12-19 Member: 225188Members
    Guys, I just had another idea: could someone please post what brand of CPU they are using? If I'm correct, AMD and Intel have implemented Hyperthreading in different ways. Maybe this could be a path to the root cause and the solution. For me, I use Intel Xeons, Nerkerz we already know uses a Core I7. @sgtkneecaps, would you be so kind to tell us your CPU brand?

    Kind regards
  • evolartevolart Join Date: 2016-12-24 Member: 225447Members
    This seemed to help a ton. I think it may have had something to do with it not running on the HT cores for me but not sure! My game was crashing every 5 minutes all of a sudden after 10 hours of gameplay. Reboots, driver updates, closing all background apps....nothing but this helped. Thank you!
  • WarpZone32WarpZone32 Join Date: 2016-12-13 Member: 224911Members
    edited December 2016
    0x6A7232 wrote: »
    @WarpZone32 Can you try this out? You've got 8 cores, try dropping it to 4.

    This would also explain why I've not had this problem at all, as my i5 has 4 cores, which are not HyperThreading (meaning they aren't split virtually into 2 cores per core).

    Interesting... I had actually played the game into an unplayable position, with a sea trench on one side and the Floating Island on the other. Loading the game, turning around, and swimming one meter caused a guaranteed crash.

    After reading this, I dialed it all the way down to one core, and while the overall framerate was worse and I had several 5-second freezes, I did manage to swim back to my lifepod.

    Cranked it back up to 4 cores, hoping to further improve performance and zone in on a happy middle-of-the-road solution. Performance seemed slightly better.

    Game crashed en route to Floating Island again. Details (I.E. my route from lifepod to floating island) in the "Procede to Waypoint" post in the Salvage Wars thread.
  • sgtkneecapssgtkneecaps Albuquerque Join Date: 2016-12-23 Member: 225375Members
    edited December 2016
    @Wustuv

    My specs are
    Intel(R) Core(TM) i7-5820k CPU @ 3.30GHz (11 cores)
    64 GB Ram
    NVIDIA Geforce GTX 950

    I've been playing every day since I made my original post. 5 cores seems to work the best! Except yesterday I went to the Grand Reef to get some aluminum oxide crystals and it lagged and crashed twice, and I am now struggling to get far away from the reef to stop the crashing. I'm sure in time I will be able to do it but I haven't had a problem much with 5 cores, other than occasional lag and a small resolvable freeze. I am 16+ hours into my game now. I will update if I get away from the Grand Reef successfully.

    Also have been running into duplicate doors in some wrecks. Haven't been too much of a problem. Usually its 2-3, one had 9 >.>

    -- Becca
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    @Wustuv

    My specs are
    Intel(R) Core(TM) i7-5820k CPU @ 3.30GHz (11 cores)
    64 GB Ram
    NVIDIA Geforce GTX 950

    I've been playing every day since I made my original post. 5 cores seems to work the best! Except yesterday I went to the Grand Reef to get some aluminum oxide crystals and it lagged and crashed twice, and I am now struggling to get far away from the reef to stop the crashing. I'm sure in time I will be able to do it but I haven't had a problem much with 5 cores, other than occasional lag and a small resolvable freeze. I am 16+ hours into my game now. I will update if I get away from the Grand Reef successfully.

    Also have been running into duplicate doors in some wrecks. Haven't been too much of a problem. Usually its 2-3, one had 9 >.>

    -- Becca

    You actually have 6 cores, but they are all HyperThreading (virtually split into two cores each for slightly better performance) so it appears that there are 12 cores, not 11. ;)

    http://ark.intel.com/products/82932/Intel-Core-i7-5820K-Processor-15M-Cache-up-to-3_60-GHz


    With that RAM, though, holy cow. Have you checked out DIMMDrive?
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    Follow-up: I tried out DIMMDrive. Isn't compatible with something, I guess, as it crashes. So if you get it, give it a shot, but if it doesn't work, I'd ask for a refund and try again later after it's been updated some more (which is what I'm doing).
  • sgtkneecapssgtkneecaps Albuquerque Join Date: 2016-12-23 Member: 225375Members
    @0x6A7232

    Ah yes I miscounted heh. It seems like it works best with 5/12 "checked" So to speak.

    UPDATE: after a 7 minute freeze, outside of the Grand Reef, it unfroze and it was a lag free journey back home. Guess I'm not going back there any time soon.

    I suppose the point is that setting the affinity has worked for me so far. Thanks!
  • 0x6A72320x6A7232 US Join Date: 2016-10-06 Member: 222906Members
    edited December 2016
    @0x6A7232

    Ah yes I miscounted heh. It seems like it works best with 5/12 "checked" So to speak.

    UPDATE: after a 7 minute freeze, outside of the Grand Reef, it unfroze and it was a lag free journey back home. Guess I'm not going back there any time soon.

    I suppose the point is that setting the affinity has worked for me so far. Thanks!

    Read the "spreading the load" part in the spoiler section about setting only non-hyperthreaded cores:
    0x6A7232 wrote: »
    @dzedward @KillerGymSox @NoxZetheus @coldkingnowhere @nelseru @Gamerman86 @ThunderHawk2 @Dracobomba @xxthing45xx @Forman @Zafar @cloudsoda @MRHelmond @hurtfuljesse21 @mrcreepisback @aasub @Jawrippa @alex_mech @Drown @Miniwarchief @Gourmet_Guy @Late123 @CottonCorn @hdja18 @TheAdamator @BlackEDGE @Vanroth @allahsiztosbaga @WhiteFireDaemon @skinlessdream @SiaNautica9 @JamieShadez @Skope @nickfatfly @sabin77 @El_Hadschi @Whorthy @djoveryde @Nerkerz @Forgerer @QuasarSuperstar @trafri @Late123 @Switchgear @Legenbaer @Xenoto @Casual_Player @obaboune @Okiraishi


    Have a look at the OP guys and see if this helps you, especially if you have more than 4 cores (look in your dxdiag for that, or in your task manager under the performance tab).

    If this works for all of you consistently, here's a way to automatically have this done (if I understand correctly):

    An easy way to do this would be with Process Lasso, which you can download the free x64 version here. (Link to free is found at the bottom of the page if you don't want to use my direct link to it.)
    Links to some articles on their site: (as I'm not able to open the program up to figure it out at this moment, @ work)
    https://bitsum.com/tips-and-tweaks/should-i-micromanage-cpu-affinities/
    https://bitsum.com/tips-and-tweaks/when-cpu-affinity-matters/
    https://bitsum.com/tips-and-tweaks/spreading-the-load/

  • WarpZone32WarpZone32 Join Date: 2016-12-13 Member: 224911Members
    0x6A7232 wrote: »
    @WarpZone32 Can you try this out? You've got 8 cores, try dropping it to 4.

    This would also explain why I've not had this problem at all, as my i5 has 4 cores, which are not HyperThreading (meaning they aren't split virtually into 2 cores per core).

    Couldn't tell if it helped or not. Still crashed some. Didn't crash as often as I thought it might. Subjective self-reporting is subjective. Note this was before switching to experimental.
Sign In or Register to comment.