Both Nvidia And Ati Cheating On 3dmark Now...

DOOManiacDOOManiac Worst. Critic. Ever. Join Date: 2002-04-17 Member: 462Members, NS1 Playtester
<a href='http://www.shacknews.com/onearticle.x/26637' target='_blank'>Seems like Futuremark doesn't have any friends anymore</a>. :P

I just thought I'd post this before fanboys start a thread like its JUST Nvidia cheating, or JUST ati cheating...

Comments

  • LikuLiku I, am the Somberlain. Join Date: 2003-01-10 Member: 12128Members
    Ouch. DOOM, what was your 3D Mark score?
  • TalesinTalesin Our own little well of hate Join Date: 2002-11-08 Member: 7710NS1 Playtester, Forum Moderators
    Odd that (a) it was a DROP in performance, and not a boost, and (b) they haven't figured out WHAT caused it. For all they know, it could just be something in that one scene triggering one of the ATI dynamic optimizations that didn't work.
    For example, Freelancer had problems because the coders did something funky with the text overlay that no one in their right mind should ever be addressing. ATI's drivers borked on it.. but ONLY where there were massive amounts of text being displayed. Perhaps Game 4 invokes something that simply is a bug in the drivers... which is quite likely, with a drop in performance.

    nVidia, on the other hand, has specific proof against it that they manually added clip-planes SPECIFICALLY for that program. It boosted performance immensely. If they were generated dynamically, it would be an optimization and worthy of commendation. Instead, it was an out-and-out cheat.
  • WheeeeWheeee Join Date: 2003-02-18 Member: 13713Members, Reinforced - Shadow
    I have to agree; it doesn't make any sense that ATi would have included cheats for only one of the game tests, and that it only gave an 8% boost on the one test, a little over 1% overall (as opposed to nVidia's 24% across-the-board). By the way, the margin of error for the test is 3%, and ATi's 1-2% falls within that range admirably. Not to say that ATi wouldn't cheat if it got the chance to, but when nVidia is doing it blatantly - that's a different issue.

    And about the Quake 3 thing - the driver release after the cheats found in ATi's drivers at that time caused a performance gain, after taking out the cheats (kind of ironic, non-cheating drivers beating out the cheating ones). Optimize = win.
  • TalesinTalesin Our own little well of hate Join Date: 2002-11-08 Member: 7710NS1 Playtester, Forum Moderators
    edited May 2003
    Read closer. It didn't boost the score.. it dropped the score. It did *worse* than with the older drivers, which is pretty well-known about Cats above 3.1 at this point. They're unstable, and have unpredictable (as-of-yet) problems. Just roll back to Cat3.1, and you'll be fine.

    Also, I'd be wary of anything Shacknews says about it... they haven't even read about how someone with a copy of 3DMark2003 Full just took the camera and moved it off the predefined track. nVidia did not own a full version, so they did not know that you COULD move it off the track. Shacknews doesn't even know HOW they were cheating, listing it as possibly dropping the image quality.
  • WheeeeWheeee Join Date: 2003-02-18 Member: 13713Members, Reinforced - Shadow
    edited May 2003
  • WheeeeWheeee Join Date: 2003-02-18 Member: 13713Members, Reinforced - Shadow
    No no no

    Futuremark went and audited the performance of nVidia's drivers and ATi's drivers with their old release of 3dmark03, by patching parts of it. They found that nVidia's drivers detect parts of 3dmark03's testing code, and then automatically activate their own scripts that bypass the code path that the 3dmark03 uses, and instead uses a much quicker render (they even said that the nVidia render looks different than with the default DirectX 9 rasterizer by a bit). Apparently this happened across the board with nVidia's Detonator 44.03's. They also do a bit of playing around with the buffers, and ignore some of the 3dmark instructions in favor of their own scripts.
  • WheeeeWheeee Join Date: 2003-02-18 Member: 13713Members, Reinforced - Shadow
    Just to clarify once again, this isn't about Detonator or Catalyst drivers - it's about 3dmark. nVidia included auto-detection of 3dmark03 in their detonator 44.03 driver set, which ignored 3dmark's instructions and rendered each test using their own scripts (giving a higher score). After the audit, Futuremark patched 3dmark and the nVidia cards' scores dropped by 24% overall, whereas the radeon's score dropped 1.9% overall (which only came from a drop of 8% in one test; the rest of the tests didn't really differ). The error margin of the test is around 3%.
  • 84n44n84n44n Join Date: 2003-05-11 Member: 16207Members
    i read about this on tweakers.net a few days ago...

    Some german company or test lab i dunno has a special 3dmark03 version in which you can pause the benching and rotate the camera. when continuing, the bench should go on from the other angle. With these evil drivers, the render was all garbled when the camera was rotated.. i saw a pic somewhere but i cant find it.
  • TalesinTalesin Our own little well of hate Join Date: 2002-11-08 Member: 7710NS1 Playtester, Forum Moderators
    You can actually do that with the full version. As noted, nVidia did not own a copy of the full version, so they did not know that you could alter the camera angles, as well as move it off its predefined movement track. <b>Bus-ted!</b>
  • CForresterCForrester P0rk(h0p Join Date: 2002-10-05 Member: 1439Members, Constellation
    <!--QuoteBegin--Talesin+May 24 2003, 12:38 AM--></span><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td><b>QUOTE</b> (Talesin @ May 24 2003, 12:38 AM)</td></tr><tr><td id='QUOTE'><!--QuoteEBegin--> Read closer. It didn't boost the score.. it dropped the score. It did *worse* than with the older drivers, which is pretty well-known about Cats above 3.1 at this point. They're unstable, and have unpredictable (as-of-yet) problems. Just roll back to Cat3.1, and you'll be fine.

    Also, I'd be wary of anything Shacknews says about it... they haven't even read about how someone with a copy of 3DMark2003 Full just took the camera and moved it off the predefined track. nVidia did not own a full version, so they did not know that you COULD move it off the track. Shacknews doesn't even know HOW they were cheating, listing it as possibly dropping the image quality. <!--QuoteEnd--> </td></tr></table><span class='postcolor'> <!--QuoteEEnd-->
    Thanks for the warning. I was planning on updating my drivers, soon, but I think I'll wait now.
Sign In or Register to comment.