Mod breaking changes in the next patch

Max

Hi everyone,

As most of you probably know, the next patch for Natural Selection 2 is going to add support for Linux. In order to make this happen we had to create an OpenGL backend for the game. Whenever we make changes to the engine, we try really hard to provide backwards compatibility and not break any existing mods; we know from our history developing Natural Selection 1 as a mod how frustrating this can be. For the most part, our changes to accommodate OpenGL and don't affect mods at all, but there are 2 exceptions.

The first is the .fx files which will no longer be used by the engine. FX files are a Direct3D format for specifying shaders. Because they are D3D specific, we had to create our own system which can also work with OpenGL. Our format is similar and generally it's a straight-forward process to convert from one to the other.

The second is the .screenfx files, which have also changed formats. The old screenfx file format was our own invention and we could have preserved it with our OpenGL changes. However, since the fx files that the screenfx files reference would already need to be updated, we decided to improve the format by unifying it with the render_setup files that are used to control the entire rendering pipeline. The render_setup files were essentially a superset of the functionality in the old screen effects, so this makes the screen effects quite a bit more powerful and simplified the code for dealing with them.

So, if your mod contains either .fx or .screenfx files, it will not work properly with the next build we release. It's not done yet, but I plan on making it so that this will be detected automatically and the mod will be de-activated. We'd love to work with the creators of existing mods to get them up-to-date for when the release goes out. If you're interested or have any questions, e-mail me at: max@unknownworlds.com.

ezekel

Not that this is exactly mod related, but if we're running windows will we be able to choose between openGL and d3d? (I'm just curious to test if openGL runs any different)

and question two, how exactly would I get my alienvision.fx to work again? it's the same base file the only thing that's been changed in the colors; so would this no longer work? or would it be unaffected as that's not really a mod

It's Super Effective!

With holding my comment until next patch

swalk

I believe this would affect consistency checking and the solution is to add these new files to the consistency check?
What are the new file format names?

Rio

What about the surface_shader files? Is the format of those going to change too?
Can you maybe provide the NS2 shaders with the new format as a reference to mod authors?

Max

Rio wrote: »

What about the surface_shader files? Is the format of those going to change too?
Can you maybe provide the NS2 shaders with the new format as a reference to mod authors?

Existing surface_shader files should work fine.

We did find one bug with some of our surface_shaders that had no effect with D3D9 but resulted in graphical corruption with OpenGL and D3D11. This was caused by non-finite floating point values being written into the frame buffer. The most common case was in code that looked like this: abs(pow(x, y)). In some cases x would be negative and result would be an undefined value. The fix was simple: pow(abs(x),y). Any mod surface_shaders that copied our code or has similar problems will need to be updated.

Because the surface_shaders are written in HLSL (which is D3D-specific) to make them work with OpenGL we had to create a HLSL parser and a GLSL generator. It is possible that a surface_shader could contain some HLSL code that our parser doesn't handle properly.

The new format FadeBlink screen effect is attached as an example.

ezekel wrote: »

Not that this is exactly mod related, but if we're running windows will we be able to choose between openGL and d3d? (I'm just curious to test if openGL runs any different)

and question two, how exactly would I get my alienvision.fx to work again? it's the same base file the only thing that's been changed in the colors; so would this no longer work? or would it be unaffected as that's not really a mod

Yes, currently you can choose between OpenGL, D3D9 and D3D11 on Windows.

For your alien vision mod, you would need to take the same steps again with the updated file (make a copy and alter the colors). You shouldn't directly modifying the files in the installation directory though, since this may prevent you from joining servers that have consistency checking enabled for that file. Instead you should create a mod and upload it to the workshop.

swalk wrote: »

I believe this would affect consistency checking and the solution is to add these new files to the consistency check?
What are the new file format names?

The new file extensions are *.hlsl and *.shader. The old *.fx extension is obsolete and won't require consistency checking anymore. I've updated the default server config to include these.

swalk

Awesome, thanks for the quick reply!

Racer1

I'd be interested to hear how you made decisions on the .hlsl / .shader models. Did the existence of CgFX tempt you or influence your decisions?

McGlaspie

Thanks Max. Much appreciated.

It'd be really nice to hear from Brain and/or Andi about what Lua related mod-breaking changes are going in as well.

Max

McGlaspie wrote: »

Thanks Max. Much appreciated.

It'd be really nice to hear from Brain and/or Andi about what Lua related mod-breaking changes are going in as well.

There weren't any.

Ghosthree3

Thanks for the warning!

McGlaspie

Is there now a limit to the number of texture that can be defined in a surface shader?

Rio

Seems like no one updated the builder_setup.xml for the new shader files.
This also causes the game to crash when changing a file with hotload enabled.

Remove this (otherwise builder throws errors)

<rule> <!-- Shader files -->
  <match>${src_dir}\*.fx</match>
  <output_file>${dst_dir}\%1.fx</output_file>
  <input_file>
            <execute>"${builder_dir}\CompileShader" -dependencies "${src_dir}\%1.fx"</execute>
        </input_file>			
  <command>"${builder_dir}\CompileShader" -out "${dst_dir}\%1.fx" "${src_dir}\%1.fx"</command>
</rule>

Add this

<rule> <!-- Shader files -->
  <match>${src_dir}\*.hlsl</match>
  <output_file>${dst_dir}\%1.hlsl</output_file>
  <command>copy "${src_dir}\%1.hlsl" "${dst_dir}\%1.hlsl"</command>
</rule>

<rule> <!-- Shader files -->
  <match>${src_dir}\*.shader</match>
  <output_file>${dst_dir}\%1.shader</output_file>
  <command>copy "${src_dir}\%1.shader" "${dst_dir}\%1.shader"</command>
</rule>