TechSpot

NVIDIA caught cheating?

By Julio Franco
May 15, 2003
  1. NVIDIA Driver bug skews 3D Mark results?

    ExtremeTech has posted some information regarding NVIDIA's latest Detonator FX drivers that supposedly could be inflating 3D Mark 2003 scores a tad by cutting corners on the "game based" tests. NVIDIA has said they are looking into the problem and it may be just a 'driver bug'.

    Seriously, cheating or not, I don't blame NVIDIA for trying to cut corners, if a lesson should be learned from this is not to rely on standard benchmark utilities completely but rather on real world applications/games... all other benchmarks we have seen and read based on games such as UT2003, Serious Sam 2 and even Doom 3 remain valid.
     
  2. acidosmosis

    acidosmosis TechSpot Chancellor Posts: 1,350

    hm

    This "controversy" has been all over the net. To be honest, about ATI vs. Nvidia. It' just an issue I really dont give a flip about. Companies are going to cheat, lie and be deceitful. That is why I rely on my own instincts and knowledge instead of others opinions, and benchmarks that already don't show true ratings for hardware. The rest of you should do the same. We will let computer illiterate's rely on benchmarks. Otherwise they can pay people like me or you for consultation charges to help them buy the right products.
     
  3. TS | Thomas

    TS | Thomas TS Rookie Posts: 1,319

    It'll be most interesting of course to see how long it takes for this bug to be fixed.

    HardOCP has a good post on this matter though, just a few bits here of interest;

    "Two days after Extremetech was not given the opportunity to benchmark DOOM3, they come out swinging heavy charges of NVIDIA intentionally inflating benchmark scores in 3DMark03. What is interesting here is that Extremetech uses tools not at NVIDIA's disposal to uncover the reason behind the score inflations. These tools are not "given" to NVIDIA anymore as the will not pay the tens of thousands of dollars required to be on the "beta program" for 3DMark "membership".

    Conversely, our own Brent Justice found a NVIDIA driver bug last week using our UT2K3 benchmark that slanted the scores heavily towards ATI. Are we to conclude that NVIDIA was unfairly increasing the workload to decrease its UT2K3 score? I have a feeling that Et has some motives of their own that might make a good story."
     
  4. forciano

    forciano TS Rookie Posts: 24

    perhaps you guys should do a comparison when cat 3.5 come out, many people (including me) want to see the performance of both cards with more mature drivers.
     
  5. Julio Franco

    Julio Franco TechSpot Editor Topic Starter Posts: 7,058   +645

    Ironically Hardocp is posting that, the same guys that in the old days made up controversy for everything only for the extra site exposure, not a bad site just that bit of it always bothered me.
     
  6. MrGaribaldi

    MrGaribaldi TechSpot Ambassador Posts: 2,512

    Read ET's article and JUDGE FOR YOURSELF!

    One of the reasons many are up in arms about this isn't because it's just another syntethic benchmark, but because a lot of OEM deals center around the perfomance of a card in these benchmarks...

    So when a company cheats in 3dmark (which is a very well known benchmark for OEM's) it can sway deals worth millions, based on a fake result...


    I, personally, think Nvidia is cheating in 3dmark03 with their latest drivers based on the fact that the "clipping" trick used doesn't just insert itself like this by itself...
    Had it been happening where everyone could see it, yes then it would most likely be a bug, but as it only happens when you "go off the rail" the chances are extremly slim...


    As far as [H] goes, yes his post is valid... Kyle does state the facts known to him, but the way it's stated removes all claims of objectivity! Nor has he tried to find out more about this himself, or substantiated the "ET is pissed since they didn't get to benchmark D3" remark...

    But let's split it up:
    1. Yes, it was released 2 days after the D3 benchmarks... But it was also released two days after the DetFX was released...
    And if you take a look at B3D, you'll find Wavey and Reverend talking about this article one week ago...
    Wavey sent pictures of what was happening to Nvidia asking them for an explanation, but has yet to receive any official explanations.

    2. Nvidia doesn't have access to the beta/debug version of 3dmark03. Quite correct. But they were a beta member for (over) 16 months before they left... So it was their own decision not to have access to the build...
    "They don't want to pay the tens of thousands of dollars to be a beta member"....
    Isn't this the company which has around a billion of cash lying around?!?
    To optimize for 3dmark03 the way they've done also costs them quite a lot of money....

    3. Kyle can post anything he wants on [H], as he owns the site... Salvator (sp?) is an employee of Ziff-Davis and can not just post whatever he fancies...
    ZD is a large IT company, and stand to loose quite a lot if these are baseless charges, but still they decided to publish them.

    4. "Nvidia is doing this do discredit 3dmark/futuremark"
    This could very well be the case, but why don't Nvidia say that openly?!?
    There is no mention of this in the readme, and the drivers are published like regular drivers!
    Thus it does not seem to me like they're doing this to discredit 3dmark, but just to boost their score...

    5. "Only fanATIcs are shouting abuse at this"... Errr, where did he get this idea...
    Take a look at B3D (as the discussion is running very hot there), strip away the obvious fanboys (at both sides), and you'll see that most (if not all) of the "in betweens" are outraged as well...

    This is because it this could be further reaching than most people think about...
    Yes, it's only been noticed in 3dmark so far. But who's to say that Nvidia aren't doing this in any other benchmark which "runs on rails"?
    They could very well be doing the same thing for Q3's Timedemo 1,2,3... Yes, it's an actual game, but with the timedemos it is only a benchmark...
    There is no way to "jump into" the demo and look around...

    Or take the flyby in UT2k3... the FX scores way above the 9800pro, but not in the botmatch... Could this be because of similar culling/clipping?
    And before you tell you me you can fly around by yourself, it's not that hard to do a "listen" for a key being pressed, and stop doing the optimization....

    Or how about RightMark, AquaMark etc?

    NOTE: I'm not saying Nvidia are cheating in all the benchmarks, but that it is a possibility based on what ET discovered!



    IF Nvidia really wanted to stop people using 3dmark as a benchmark, why not add some code omt the drivesr which makes it impossible to run 3dmark?
    That would've been a fair way to tell the world about Nvidias dislike for 3dmark, since "it doesn't test the way we think it should do"...
    Why all this optimizing/cheating, if you don't want people to use the benchmark?
    According to B3D and ET there has been a noticeable increase in bugs with nvidia drivers, that seem to correspond with increased scores...



    I think I'll stop now before I start raving...
    I just got pissed of at Kyle's "fact post" (which i'm posting at the end of this post, so you can see the entire one for yourself...


    One last thing before I shut up (for now), read ET's article for yourself, and think about it...
    Kyle never had a proper rebuttal of what they found, he only talked about the circumstances they posted it...

    .02$ (and thank you for reading so far)


     
  7. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

    great analysis, MrG

    There is only one problem :)

    I don't have any money to buy the GeforceFX and 3DMark2003 and I still would like to see some screenshots that show what's really happening ...

    I've coded some 3D apps so my developer side would like to find out a little bit more, at least some screenshots of these "corners" everybody's talking about ...
     
  8. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

  9. MrGaribaldi

    MrGaribaldi TechSpot Ambassador Posts: 2,512

    Hehe, thanks :)

    Nice to see that someone enjoyed it!
     
  10. MrGaribaldi

    MrGaribaldi TechSpot Ambassador Posts: 2,512

    Hmm...

    Either tech news is very slow at the moment, or I managed to scare off even the staff from this part of the site... :D
     
  11. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

    i dunno, maybe they all had a buffer overflow while reading :)

    hey guys, please upgrade your drivers hehehehe :D :D
     
  12. Mictlantecuhtli

    Mictlantecuhtli TS Evangelist Posts: 4,345   +11

  13. TS | Thomas

    TS | Thomas TS Rookie Posts: 1,319

    Nope, I've just been away a few days :) Yes, I now see Beyond3D's bit at http://www.beyond3d.com/forum/viewtopic.php?topic=5856&forum=9 as well. I dunno to be honest, it's a pretty crappy thing to do if it is true (i.e. it's a purposeful "bug"), though they would have to be aware that it would be noticed if anyone used 3D Mark's freeview mode.

    From NVIDIAs point of view "We improved performance & did not sacrifice image quality" - & to all *but* those beta testers they have indeed done that.

    More importantly though is that there's no real penalty for "cheating" like this. It annoys people like us, same as it did when ATi & Trident (Cough) do it, but is there really any long term harm done to the companies for this fraudulent misrepresentation? Nope.

    Well, this is why I prefer doing Soundcard reviews. There's not much that can be done underhandedly by companies about them :)
     
  14. negroplasty

    negroplasty TS Guru Posts: 516   +12

    If NVIDIA is cheating, very well, I accept this - after seeing valid evidence of course. But those who think that ATi or other companies have never done anything like this are fools to think so. The only difference is, they didn't get caught, or they did and nobody made a big deal about it - either way, I stand beside my point.
     
  15. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

    tim, you speak as if you were NVIDIA's official lawyer or something hehheeh ;)
     
  16. Nvidia is Desparate

    Tim if you recall ATI was hammered when it was said that Radeon 8500 drivers were quake optimized.
    And Nvidia was one of the first to cry foul.
    It seems to me that Nvidia knew its 5900 could only compete with the 9800 pro and not out perform it.
    Nvidia is desperate to show it can regain perfoermance interest from the consumer.
    This will all be mute at the 5900's launch as ATI has already Taped it's R360 out and is said to be releasing on Nvidia's 5900 launch date.
    Seems Nvidia is a day late and a dollar short again.
    Oh BTW whats up with all these artifacts in my latest detonator drivers (seems Nvidia is now the one with the crap drivers).
     
  17. negroplasty

    negroplasty TS Guru Posts: 516   +12

    Well, let's notice that ATi was also caught cheating by Futuremark, not as badly mind you, but it is there. It is in the margin of error so everyone says, but I will believe that when I see/hear it!

    Futuremarks PDF: ***
     
  18. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

  19. negroplasty

    negroplasty TS Guru Posts: 516   +12

    Let me apologize for my invalid accusations against ATi, they were all unfounded and incorrect, apparently. I hope that I didn't offend anyone with my constant ranting about NVIDIA and ATi; my criticisms were purely comparisons between the two excellent chip manufacturing companies. Once again, I apologize, long live ATi and NVIDIA ;) !!!
     
  20. DigitAlex

    DigitAlex TechSpot Paladin Posts: 536

    As you said, the two companies are not that bad :)

    If you remember some events in the past, the competition always results in better achievements.

    USA vs USSR / Russia for space, at the beginning of the space explorations.

    Many archeologist hunting and competing for dinosaur bones ...

    As long as the competition is FAIR, it always leads to positive results !

    ugh
     
Topic Status:
Not open for further replies.

Similar Topics

Add New Comment

You need to be a member to leave a comment. Join thousands of tech enthusiasts and participate.
TechSpot Account You may also...