Commissioning Misleading Core i9-9900K Benchmarks

Well, you can call the company anything you want, but they are at least willing to answer the questions about their testing. Looking at their resume I would say this is a company that does corporate testing, and they were out of their element testing enthusiast gear. I think it is wrong to accuse anyone outright with at least getting their side of the story though.

https://www.principledtechnologies....ng_PC_gaming_processor_study_interim_1018.pdf

When you mess up the basics of benchmarking, something is wrong. They not only failed to isolate variables but they went out of their way to give Intel the advantage. They gave the Intel CPU a better cooler and did not set XMP on the AMD system. That's benchmarking 101 and has nothing to do with just the enthusiast market. In addition, they limited the AMD CPU to only 4 cores. You have to go out of your way to do that.

Their explanation is damage control at this point because there is no scenario where they were doing this by accident. The clearly went to an extent to cripple the AMD processor's performance.

Who's promising 15-25% improvement in IPC? That won't happen any time soon IMO. If it's as big as 5%, I'll be impressed as that'll catch them up to Intel and then only clock speed will be the distinguishing factor.

Oh yeah, and AMD's actually reasonable prices.

I think you are vastly underestimating what a node shrink and new architecture can do. AMD got a 68% IPC improvement going from bulldozer to Zen. 15-25% is very possible on zen 2.
 
Well, you can call the company anything you want, but they are at least willing to answer the questions about their testing. Looking at their resume I would say this is a company that does corporate testing, and they were out of their element testing enthusiast gear. I think it is wrong to accuse anyone outright with at least getting their side of the story though.

https://www.principledtechnologies....ng_PC_gaming_processor_study_interim_1018.pdf

When you mess up the basics of benchmarking, something is wrong. They not only failed to isolate variables but they went out of their way to give Intel the advantage. They gave the Intel CPU a better cooler and did not set XMP on the AMD system. That's benchmarking 101 and has nothing to do with just the enthusiast market. In addition, they limited the AMD CPU to only 4 cores. You have to go out of your way to do that.

Their explanation is damage control at this point because there is no scenario where they were doing this by accident. The clearly went to an extent to cripple the AMD processor's performance.

Who's promising 15-25% improvement in IPC? That won't happen any time soon IMO. If it's as big as 5%, I'll be impressed as that'll catch them up to Intel and then only clock speed will be the distinguishing factor.

Oh yeah, and AMD's actually reasonable prices.

I think you are vastly underestimating what a node shrink and new architecture can do. AMD got a 68% IPC improvement going from bulldozer to Zen. 15-25% is very possible on zen 2.

Did you not read the press release? They did use the AMD version of XMP. They agreed they messed up with AMD "Game Mode" using it on Ryzen when it is for Threadripper... you would think AMD would disable that on Ryzen, eh? Steve here already stated the cooler isn't as big a deal as people are making it out to be since this wasn't an overclocking thing, although Ryzen is more sensitive to thermals. I have a feeling that putting the same cooler on the Ryzen will make very little difference, but ymmv.

Your mind is made up, no amount of explanation will change that, I get it. All the AMD fans are outraged that Intel hired some company to do a hatchet job on them, and ruin their own reputation while they are at it... sure, I buy that. They must have made millions to run their own name down, right?
 
Did you not read the press release? They did use the AMD version of XMP. They agreed they messed up with AMD "Game Mode" using it on Ryzen when it is for Threadripper... you would think AMD would disable that on Ryzen, eh? Steve here already stated the cooler isn't as big a deal as people are making it out to be since this wasn't an overclocking thing, although Ryzen is more sensitive to thermals. I have a feeling that putting the same cooler on the Ryzen will make very little difference, but ymmv.

Your mind is made up, no amount of explanation will change that, I get it. All the AMD fans are outraged that Intel hired some company to do a hatchet job on them, and ruin their own reputation while they are at it... sure, I buy that. They must have made millions to run their own name down, right?

Please educate yourself


By the way, there are techspot staff in the comments section of this video giving it a thumbs up.

AMD fans? So TechSpot, GamersNexus, and pretty much every Tech News and Review website are AMD fanboys? That's not being a fan boy, it's holding a company accountable.

How exactly does one "accidentally" enable game mode and document the whole process in the white paper? It must also be coincidence that Intel funds principle technology and developed their software suite for them.
 
Last edited:
Please educate yourself

Heh. I am familiar enough with the world that I don't need to have someone tell me how to think.

The outrage on the forums over this is fanned by the fact that it started because no one talked to the company who conducted the testing. Steve here at Hardware Unboxed was right to call out the variances. But then you have to contact the folks and give them a chance to answer to their results. Steve at Gamers Nexus was fortunate enough to be close to the company office and had the good common sense to go speak to them. They could have shut the door in his face, they didn't.

The rest of the YouTube tech world just piles on mindlessly after, as far as I am concerned. The company issued a press release answering most of the concerns, as well as stating that they will redo the Ryzen 7 2700X results due to not being knowledgeable about AMD's Game Mode, which they used for Threadripper. I can't say I know their minds, like you do apparently, but I could see how they might mistake Game Mode to be something AMD thinks is a good thing to enable. Call it an honest mistake they are willing to rectify.

Finally, I don't believe a company that has been in business since 2003, and works with a lot of corporations, would risk their reputation just to "shill" for Intel, doesn't make sense.
 
Heh. I am familiar enough with the world that I don't need to have someone tell me how to think.

The outrage on the forums over this is fanned by the fact that it started because no one talked to the company who conducted the testing. Steve here at Hardware Unboxed was right to call out the variances. But then you have to contact the folks and give them a chance to answer to their results. Steve at Gamers Nexus was fortunate enough to be close to the company office and had the good common sense to go speak to them. They could have shut the door in his face, they didn't.

The rest of the YouTube tech world just piles on mindlessly after, as far as I am concerned. The company issued a press release answering most of the concerns, as well as stating that they will redo the Ryzen 7 2700X results due to not being knowledgeable about AMD's Game Mode, which they used for Threadripper. I can't say I know their minds, like you do apparently, but I could see how they might mistake Game Mode to be something AMD thinks is a good thing to enable. Call it an honest mistake they are willing to rectify.

Finally, I don't believe a company that has been in business since 2003, and works with a lot of corporations, would risk their reputation just to "shill" for Intel, doesn't make sense.

Knowledge is not a way of thinking, it's data that one learns. You can stop with "don't tell me how to think!" because no one is doing that.

Steve talked with the company and the result is still the same. If they had shut the door in his face they would only look worse.


Here are some of their responses

"“Only the 2700X came with a cooler. The rest required a cooler to be added. AMD certainly did not seem to think their cooler was inadequate, so it seemed a reasonable choice.”"

This is not a response I'd expect from an expert. This is a snarky comment I'd expect from a teenager. He doesn't apologize for making such a basic mistake and there is no indication that they even feel this was an issue. Here is GN's response:

"If testing in a controlled environment, the single element which matters is equality between all test beds. If that’s an NH-U14S, so be it – AMD gets one, too. In this instance, for whatever reason, it seems the AMD platforms were relegated to objectively weaker stock coolers"

As is made clear, PT completely failed to justify their use of different coolers and in fact GN concludes here that they failed to even create a controlled environment.

In response to using 64GB of memory (which disadvantages AMD) GN stated

"Very few are reasonably purchasing this much memory and, given that this benchmark focuses entirely on gaming tests (and not “production” tasks), we must look at it from a gaming scenario. No meaningful gaming build with a non-HEDT platform is opting for 64GB of memory."

and in response PT only had this to say:

"I don't have a particular response to that"

Their response to game mode?

"Use of "Game Mode" on the AMD Ryzen 7 2700X: Some inquiries we have received concern the use of the Ryzen utility and the number of active cores in the AMD-based systems. Based on AMD's recommendations and our initial testing on the Threadripper processors, we found installing the AMD Ryzen Master utility and enabling Game Mode increased most results. For consistency purposes, we did that for all AMD systems across Threadripper and Ryzen. We are now doing additional testing with the AMD systems in Creator Mode. We will update the report with new results."

BS. AMD only recommends the use of Game Mode with Threadripper CPUs. AMD has NEVER recommended game mode for use on Ryzen processors. Yes, they went out of their way to enable game mode but not install the same cooler for each test bed. No, they threw out any idea of consistency out the window. On top of that, they try to pass the blame off to AMD. You aren't going to make me believe professionals didn't once google what Game Mode does or ask AMD? Yeah, no.

They were also, once again as professionals, completely unaware of the games they used were in GPU bound situations.

Finally, I don't believe a company that has been in business since 2003, and works with a lot of corporations, would risk their reputation just to "shill" for Intel, doesn't make sense.

You didn't watch the Adored video at all did you? PT has deep ties to Intel that go far beyond just paying for this benchmark.

If anything, the interview has only made it more certain. Don't take my word for it, Steve explicitly states at the end of the video that "it is absolutely fair to question the validity of the data".

Near the start of the video the CEO of PT states he's 'been benchmarking for longer then you've been alive". It is impossible that PT made that many mistakes, things even a 1st time benchmarker on youtube wouldn't do, and then go on the record and play incompetent. Their mistakes are intentional and it's why Intel has and continues to pay them.

Take a look at PT's portfolio

https://www.principledtechnologies.com/portfolio-marketing#Intel2004

They have more Intel entries then every other company combined and it's the only one they sort by year. In fact many of their other entries for other companies deal with products containing Intel IP as well.

They are consistent when it benefited Intel and inconsistent when it benefited Intel. This wasn't remotely objective and isn't remotely anything I'd expect out of professionals. Bill Catchings has been benchmarking/marketing since 1998 (you can still find his software projects online). The answers he gave in that interview are completely contrary to the few interviews I've read from him in the past two decades. He knows better, that much is certain.
 
This attempt only shows intel's helplessness and nothing else. This industry giant just put themselves in a ridiculous place
 
I agree with Linus. I think this whole thing has been blown way out of proportion. I mean, we usually ignore sponsored benchmarks and this isn’t the first time we’ve seen a manufacturer pull bs in the face of a launch.

Parts of me feel that this is exactly what Intel wanted. The outraged press have made sure every knows that Intel make the fastest gaming CPU. (Just not by as much as the sponsored document states). Even people who aren’t really into their hardware very much will have now been made aware that Intel is faster because of the noise made over this.

Also I personally feel the prices are because Ryzen fell short of expectations. All Intel needed to do was move the launch date of one or two parts forward 6 months to retain dominance. When you compare core for core Intels chips beat AMD chips at every single test. Now it appears that Intel can segment the premium end of the market by turning off hyperthreading. As AMD might just about be able to match a 9700K in multithreading only. This leaves the 9900K with no competitors and Intel will cash in.

I hope Ryzen 2 can actually beat Intel with core performance and not just shovel more cores onto a die.
 
I agree with Linus. I think this whole thing has been blown way out of proportion. I mean, we usually ignore sponsored benchmarks and this isn’t the first time we’ve seen a manufacturer pull bs in the face of a launch.

Parts of me feel that this is exactly what Intel wanted. The outraged press have made sure every knows that Intel make the fastest gaming CPU. (Just not by as much as the sponsored document states). Even people who aren’t really into their hardware very much will have now been made aware that Intel is faster because of the noise made over this.

Also I personally feel the prices are because Ryzen fell short of expectations. All Intel needed to do was move the launch date of one or two parts forward 6 months to retain dominance. When you compare core for core Intels chips beat AMD chips at every single test. Now it appears that Intel can segment the premium end of the market by turning off hyperthreading. As AMD might just about be able to match a 9700K in multithreading only. This leaves the 9900K with no competitors and Intel will cash in.

I hope Ryzen 2 can actually beat Intel with core performance and not just shovel more cores onto a die.

They would not have as big an issue on their hands had PT and Intel just apologized. Instead we get PT trying to sidestep responsibility and Intel actually doubled down on the benchmarks.

This is only blowing up because Intel refuses to own up to obviously bad numbers. You are right, this would not have been an issue otherwise.
 
9900K pricing was announced today and boy it’s expensive. My prediction is that it reviewers will slam it for its price but despite this it will sell like no tomorrow. Because reviewers don’t understand that not everyone out there are value buyers. A Renault is better value for money than a BMW but driving enthusiasts would pay more for the BMW because it’s a better vehicle. Same thing goes on in the CPU world. Why have AMD when you can have Intel?

Of course regarding this, Intel will exaggerate the results, they have a poor marketing department clearly. It may not be 50% faster but it’s still faster as they have better engineers and that’s what’s important. Far more important than price when it comes to flagship products anyway. Value buyers will go further down the product stack on either manufacturer. Although I’m sure the 2700X will now be considered a budget part in the face of these new prices.

Non value buyers buy 9980XE or 2950X. Why? Because they can. They have 4K monitors and their 2080 Tis are bottlenecking the system. 9900K is just a mainstream product bound by value, and the only customers seem to be competitive gamers with 1080p 200Hz+ monitors.
 
Non value buyers buy 9980XE or 2950X. Why? Because they can. They have 4K monitors and their 2080 Tis are bottlenecking the system. 9900K is just a mainstream product bound by value, and the only customers seem to be competitive gamers with 1080p 200Hz+ monitors.

You are forgetting the epeen crowd, who always buy the best single thread, most expensive consumer processor every year.
 
I agree with Linus. I think this whole thing has been blown way out of proportion. I mean, we usually ignore sponsored benchmarks and this isn’t the first time we’ve seen a manufacturer pull bs in the face of a launch.

Parts of me feel that this is exactly what Intel wanted. The outraged press have made sure every knows that Intel make the fastest gaming CPU. (Just not by as much as the sponsored document states). Even people who aren’t really into their hardware very much will have now been made aware that Intel is faster because of the noise made over this.
+1 That and 1 more point: It seems 9900k (and 9700k obviously) is only marginally better than 8700k (at least in games, according to refreshed paid tests) so they divert the attention/blame on Ryzen.
 
Back