Tested with 3D Vision
wulf 21
Join Date: 2011-05-03 Member: 96875Members
Join Date: 2011-05-03 Member: 96875Members
I tested NS2 with Nvidia 3D vision and the first thing I thought was: What a mess! But then I found that the engine is actually working properly with 3D-Vision for the most part. In this topic I am trying to explain the problems I ran over. One word to the "solutions" I'm going to post: I never took part in programming a 3D engine, so I don't know if the problems can be solved like I think or even can be solved at all. These are just my ideas that come from what I know about 3D graphics and I hope they can help.
If you know how to do that you can view the 3D-screenshots on this page cross-eyed, i.e. the left image is the one for the right eye and the other way round.
For those who own 3D-vision, too, I packed the screenshots in a zip file. <a href="http://www.file-upload.net/download-3404350/3D-screenshots.zip.html" target="_blank">You can download them here.</a> If you own another 3D-monitor or TV connected to your computer or maybe anaglyph glasses, you can view them with the <a href="http://stereo.jpn.org/eng/stphmkr/" target="_blank">Stereo Photo Maker</a> (Freeware) or the <a href="http://3dtv.at/Downloads/Index_en.aspx" target="_blank">Stereoscopic Player</a> (Shareware, max 5 minutes video playback with free version). Both applications contain viewing options for multiple monitors/TVs and multiple differently-colored glasses. You will probably have to swap the pictures to view them correctly if you don't use 3D-Vision.
To prove what I said at the beginning here is a screenshot with no major issues visible.
<img src="http://i.imgur.com/juIb7.jpg" border="0" class="linked-image" />
<i>packed filename: turret.jps</i>
The model of the pistol, turret and computer console are actually looking pretty good in 3D. The only thing that doesn't fit on this picture is the different lighting of the wall in the background on the left/right picture, more to this later.
What 3D-Vision basically does is to render two pictures instead of one with slightly shifted and rotated perspectives to the left and right from the original perspective. This may lead to issues if some parts of the engine calculate something independently from the main render. One issue here are shadows, like you can see on the next shot:
<img src="http://i.imgur.com/JoxAT.jpg" border="0" class="linked-image" />
<i>packed filename: shadow.jps</i>
As you can see, the shadow of the power node is positioned wrong on both pictures for the left and right eye. If you view it in 3D and focus on the power node, it looks like the thing has two shadows. (but it looks weird because you see each shadow only on one eye). Comparing the position on the shadow with the crosshair which is at screen depth (but that's only a minor issue for which Nvidia has already included an own solution), you will notice that the position of the shadow actually does NOT change between the left and right picture. If you view the shot in 3D and focus on the crosshair this time, the shadow will appear as if its glued on the screen.
This indicates that the shadows are rendered in a process that is independent from the main render and this process is using the original perspective instead of the shifted and rotated perspective. If it is possible the solution would be that the shadowing process needs to read out the perspective the current frame is actually rendered in rather than using the original input perspective.
This leads to the issue with lights but examining it carefully revealed that it actually isn't separate issue but rather a result from the issue with shadows. Beneath are a 2D-screenshot that shows how the light should look like and a 3D-shot that shows how it looks like with 3D-Vision.
<img src="http://i.imgur.com/sAzDT.jpg" border="0" class="linked-image" />
<i>packed filename: light 2D.jpg</i>
<img src="http://i.imgur.com/F8YtB.jpg" border="0" class="linked-image" />
<i>packed filename: light 3D.jps</i>
What you can see is that the shape (notice the kink that in 2D was round) and position of the light does no longer fit to what it looked like in 2D. Furthermore the position and look of the light in the picture for the left/right eye doesn't fit together (in 3D it should look the same except the switch of perspective). Nor is it at the same position on the screen like in the shadow example. (unfortunately that's not to good to see on this shot because it was taken in the ready-room --> no crosshair). First I thought that this doesn't make any sense but comparing the 2D and one of the 3D pictures shows what is happening:
From the 2D picture to the picture for the right eye (which is the one on the left!) the lamp where the light comes from moved slightly to the right, because the right-eye's perspective it slightly rotated to the left. If you now imagine the light moving the same way this will lead to that the left edge of the light (the sharp one) is exactly where it should be (notice that the position of the left edge <i>relative to the wall</i>, not the screen, is the same as in the 2D shot). The right edge of the light however does not move at all (<i>relative to the wall</i> which has moved to the right, it looks like the right edge has moved to the left.). Exactly the same, only the other way round happens for the picture for the left eye.
This reveals that the light actually consists of 2 components. The lighting itself is moving correctly with the change of perspective but would have a sharp edge if no improvements were made. It is improved by adding a soft shadow at the edge of the light. The shadow, however, is not moving with the change of perspective. So the apparent change of position and shape of the lighting is only a result of the combination of correctly working lighting and not correctly working shadows and should automatically be solved, too, if the shadow problem is solved.
Summing up so far: The shadow issue I described doesn't seem to be a problem if you only look at these the shots here. But if you look of a room full of lights and shadows which are all displayed differently for both eyes this isn't only looking weird, it will cause a weird feeling too which alone makes the game unplayable in stereo 3D. Furthermore due to the above effect, always more light is "cut out" by shadows than in 2D, i.e. all lights become smaller. The farther you are away from the lights, the greater is this effect. Remember I selected the shots here on purpose to describe what I think the problem is, not to show how horrible stereo-3D looks currently.
But now I am going to show an even bigger issue, unless of course, the developers completely change the way infestations are displayed (e.g. like in the dynamic infesation video in which it rather looked like a 3D-object). Take a look at the following screenshot:
<img src="http://i.imgur.com/h65ZU.jpg" border="0" class="linked-image" />
<i>packed filename: infestation.jps</i>
As you can see there seem to be holes in the infestation that are shaped like the weapon and the egg. But therefore parts of the weapon and the egg seem to be covered with infestation. Notice that the hole is not at the same position relative to the crosshair, in fact it seems to have "moved" exactly twice as far as the egg compared to what it would look like in 2D (or plain 3D like some people say). The same applies to the light coming from a tech point, which was a little harder to find:
<img src="http://i.imgur.com/rGt9f.jpg" border="0" class="linked-image" />
<i>packed filename: techpoint.jps</i>
By the way, that was the main reason why my first thought was "What a mess!". I spawned as skulk after turning on 3D vision and the whole thing looks even worse with all the tooth in front of you.
It looks like there is something wrong with the hidden surface removal of transparent stuff like the infestation which seems to be a texture overlay at this time. While the hidden surface removal of 3D objects seems to be part of the main rendering process (which works fine with 3D vision, even trasparent objects like glass), the hidden surface removal for things like infestation seems to be independent of that. After all the other 3D-objects are rendered with the correctly changed perspective the process seems to take that data to remove parts of the infestation that should be hidden (which is still done in 3D) and is giving that data to the next process. Unfortunatly 3D Vision (or rather one of its programmers) is thinking now that it is getting new data from the original perspective and is performing the perspective change AGAIN.
A possible solution would be to use the original perspective for hidden surface removal instead of the perspective that is actually rendered (so to speak, do exactly the opposite of the shadow solution). However this wouldn't be the ideal solution since the shape of the objects is slightly different in the original perspective, so it wouldn't fit perfectly. The ideal solution would be to tell 3D-vision somehow, that the data is already in the correct perspective but I don't know whether this is possible at all.
Some words at the end: I know that 3D-Vision is definitely not one of the dev's top priority. But since I'm a 3D-fan I nevertheless reported my findings. Since the engine is pretty much finished and now the focus is on the actual game content, I hope that an engine programmer has the time to try out some of that but I don't know if that will ever happen.
And if someone actually fixes this, please implement a crosshair that is rendered at correct depth, too. (or maybe simply infinite depth like source games do). Nvidias crosshair I mentioned above is simply too ugly. ;-)
If you know how to do that you can view the 3D-screenshots on this page cross-eyed, i.e. the left image is the one for the right eye and the other way round.
For those who own 3D-vision, too, I packed the screenshots in a zip file. <a href="http://www.file-upload.net/download-3404350/3D-screenshots.zip.html" target="_blank">You can download them here.</a> If you own another 3D-monitor or TV connected to your computer or maybe anaglyph glasses, you can view them with the <a href="http://stereo.jpn.org/eng/stphmkr/" target="_blank">Stereo Photo Maker</a> (Freeware) or the <a href="http://3dtv.at/Downloads/Index_en.aspx" target="_blank">Stereoscopic Player</a> (Shareware, max 5 minutes video playback with free version). Both applications contain viewing options for multiple monitors/TVs and multiple differently-colored glasses. You will probably have to swap the pictures to view them correctly if you don't use 3D-Vision.
To prove what I said at the beginning here is a screenshot with no major issues visible.
<img src="http://i.imgur.com/juIb7.jpg" border="0" class="linked-image" />
<i>packed filename: turret.jps</i>
The model of the pistol, turret and computer console are actually looking pretty good in 3D. The only thing that doesn't fit on this picture is the different lighting of the wall in the background on the left/right picture, more to this later.
What 3D-Vision basically does is to render two pictures instead of one with slightly shifted and rotated perspectives to the left and right from the original perspective. This may lead to issues if some parts of the engine calculate something independently from the main render. One issue here are shadows, like you can see on the next shot:
<img src="http://i.imgur.com/JoxAT.jpg" border="0" class="linked-image" />
<i>packed filename: shadow.jps</i>
As you can see, the shadow of the power node is positioned wrong on both pictures for the left and right eye. If you view it in 3D and focus on the power node, it looks like the thing has two shadows. (but it looks weird because you see each shadow only on one eye). Comparing the position on the shadow with the crosshair which is at screen depth (but that's only a minor issue for which Nvidia has already included an own solution), you will notice that the position of the shadow actually does NOT change between the left and right picture. If you view the shot in 3D and focus on the crosshair this time, the shadow will appear as if its glued on the screen.
This indicates that the shadows are rendered in a process that is independent from the main render and this process is using the original perspective instead of the shifted and rotated perspective. If it is possible the solution would be that the shadowing process needs to read out the perspective the current frame is actually rendered in rather than using the original input perspective.
This leads to the issue with lights but examining it carefully revealed that it actually isn't separate issue but rather a result from the issue with shadows. Beneath are a 2D-screenshot that shows how the light should look like and a 3D-shot that shows how it looks like with 3D-Vision.
<img src="http://i.imgur.com/sAzDT.jpg" border="0" class="linked-image" />
<i>packed filename: light 2D.jpg</i>
<img src="http://i.imgur.com/F8YtB.jpg" border="0" class="linked-image" />
<i>packed filename: light 3D.jps</i>
What you can see is that the shape (notice the kink that in 2D was round) and position of the light does no longer fit to what it looked like in 2D. Furthermore the position and look of the light in the picture for the left/right eye doesn't fit together (in 3D it should look the same except the switch of perspective). Nor is it at the same position on the screen like in the shadow example. (unfortunately that's not to good to see on this shot because it was taken in the ready-room --> no crosshair). First I thought that this doesn't make any sense but comparing the 2D and one of the 3D pictures shows what is happening:
From the 2D picture to the picture for the right eye (which is the one on the left!) the lamp where the light comes from moved slightly to the right, because the right-eye's perspective it slightly rotated to the left. If you now imagine the light moving the same way this will lead to that the left edge of the light (the sharp one) is exactly where it should be (notice that the position of the left edge <i>relative to the wall</i>, not the screen, is the same as in the 2D shot). The right edge of the light however does not move at all (<i>relative to the wall</i> which has moved to the right, it looks like the right edge has moved to the left.). Exactly the same, only the other way round happens for the picture for the left eye.
This reveals that the light actually consists of 2 components. The lighting itself is moving correctly with the change of perspective but would have a sharp edge if no improvements were made. It is improved by adding a soft shadow at the edge of the light. The shadow, however, is not moving with the change of perspective. So the apparent change of position and shape of the lighting is only a result of the combination of correctly working lighting and not correctly working shadows and should automatically be solved, too, if the shadow problem is solved.
Summing up so far: The shadow issue I described doesn't seem to be a problem if you only look at these the shots here. But if you look of a room full of lights and shadows which are all displayed differently for both eyes this isn't only looking weird, it will cause a weird feeling too which alone makes the game unplayable in stereo 3D. Furthermore due to the above effect, always more light is "cut out" by shadows than in 2D, i.e. all lights become smaller. The farther you are away from the lights, the greater is this effect. Remember I selected the shots here on purpose to describe what I think the problem is, not to show how horrible stereo-3D looks currently.
But now I am going to show an even bigger issue, unless of course, the developers completely change the way infestations are displayed (e.g. like in the dynamic infesation video in which it rather looked like a 3D-object). Take a look at the following screenshot:
<img src="http://i.imgur.com/h65ZU.jpg" border="0" class="linked-image" />
<i>packed filename: infestation.jps</i>
As you can see there seem to be holes in the infestation that are shaped like the weapon and the egg. But therefore parts of the weapon and the egg seem to be covered with infestation. Notice that the hole is not at the same position relative to the crosshair, in fact it seems to have "moved" exactly twice as far as the egg compared to what it would look like in 2D (or plain 3D like some people say). The same applies to the light coming from a tech point, which was a little harder to find:
<img src="http://i.imgur.com/rGt9f.jpg" border="0" class="linked-image" />
<i>packed filename: techpoint.jps</i>
By the way, that was the main reason why my first thought was "What a mess!". I spawned as skulk after turning on 3D vision and the whole thing looks even worse with all the tooth in front of you.
It looks like there is something wrong with the hidden surface removal of transparent stuff like the infestation which seems to be a texture overlay at this time. While the hidden surface removal of 3D objects seems to be part of the main rendering process (which works fine with 3D vision, even trasparent objects like glass), the hidden surface removal for things like infestation seems to be independent of that. After all the other 3D-objects are rendered with the correctly changed perspective the process seems to take that data to remove parts of the infestation that should be hidden (which is still done in 3D) and is giving that data to the next process. Unfortunatly 3D Vision (or rather one of its programmers) is thinking now that it is getting new data from the original perspective and is performing the perspective change AGAIN.
A possible solution would be to use the original perspective for hidden surface removal instead of the perspective that is actually rendered (so to speak, do exactly the opposite of the shadow solution). However this wouldn't be the ideal solution since the shape of the objects is slightly different in the original perspective, so it wouldn't fit perfectly. The ideal solution would be to tell 3D-vision somehow, that the data is already in the correct perspective but I don't know whether this is possible at all.
Some words at the end: I know that 3D-Vision is definitely not one of the dev's top priority. But since I'm a 3D-fan I nevertheless reported my findings. Since the engine is pretty much finished and now the focus is on the actual game content, I hope that an engine programmer has the time to try out some of that but I don't know if that will ever happen.
And if someone actually fixes this, please implement a crosshair that is rendered at correct depth, too. (or maybe simply infinite depth like source games do). Nvidias crosshair I mentioned above is simply too ugly. ;-)
Comments
I was actually just going to make a post aobut this, and here you are, doing it for me.
Thanks for all this. And although I doubt this will ever get priortity (and believe me, it does not need to, at all), it would be great to see messed around with some day.
It is great though to see the potential for ns2 in 3d, some examples are looking the marines in 3d. They look absolutely amazing model wise (excluding shadow problem)
Obviously we would be obligated to demand a video of you wearing the 3D glasses :P
Well maybe you didn't really read what I wrote on how to view these screenshots at the top. You either need a 3D-monitor / TV or simply colored (anaglyph) 3D glasses and the appropriate software. Or you have to learn how to view 3D-images cross-eyed. If you google for cross-eyed 3D you will probably find a tutorial somewhere.
Of course these screenshots do not show how it is looking in-game if you do not view them in 3D. :P
Im not really sure. 3D Vision originally uses its own shutter glasses that are controlled by an infrared transmitter that is connected to the computer via USB and that needs a 120 Hz LCD (or a >=100 Hz CRT if you still own one).
However there is a 3D-Vision TV-play software, too, to enable playing on 3D-TVs (I think via HDMI 1.4a). If your screen is capable of calculating the missing 6 perspectives from only 2, that <i>might</i> work. (I think I've read somewhere that someone developed an autostereoscopic screen working like that to view 3D-movies that only contain 2 perspectives).
But probably it will be better for you to use a 3D-driver of another company (like TriDef, IZ3D e.t.c.). You should inform yourself if any of these drivers support an output of more than 2 views.
Thank you for your reply, really. Glad to know I've actually reached a dev, I couldn't hope for more.