How We Test: CPU Gaming Benchmarks

[Julio Franco]

#ThrowBackThursday This a topic that's often raised when we do our CPU gaming benchmarks. We perform a ton of CPU and GPU benchmarks tests throughout the year, a big portion of which are dedicated to gaming. The goal is to work out which CPU will offer you the most bang for your buck at a given price point, now and hopefully in the future.

Read the full article here.

https://www.techspot.com/article/1637-how-we-test-cpu-gaming-benchmarks/

 

[BSim500]

Good article, but I'm still willing to bet you'll get comments like "y u test at 1080p steve. I own 4K!" a few weeks down the line... ;-)

"Additionally, don’t fall into the trap of assuming everyone uses Ultra quality settings or targets just 60 fps."

^ Finally someone who gets it. Ignoring the fact that people who buy budget rigs like G4560 + GTX 1050's tend to do so for lighter-middle weight / older games, a quick glance at Youtube shows that for AAA's running 50-60fps on Med-High and using common sense optimised tweaking (disabling thirstiest shaders like Ambient Occlusion or Hairworks without turning textures down low, or use more efficient AA like SMAA / FXAA over MSAA) is obviously more desirable and commonplace than forcing it to 22fps Ultra "for the sake of consistency" which never did make any real sense vs top-end given they are two completely different markets. If someone really wants to run a game on low-end hardware, they'll find a way (see Ryzen APU's which halve the horsepower again). And if someone wants to deliberately cripple any hardware using 8x MSAA + DSR/VSR + "Yeti" preset Hairworks, 65,536x tessellation & 50 mile view distances to convince themselves for the need of an upgrade, they'll find a way to do that too...

As for providing more "real-world" results by testing with a lower-end GPU, I’d say unless we tested a range of GPUs at a range of resolutions and quality settings, you’re not going to see the kind of real-world results many claim to deliver. Given the enormous and unrealistic undertaking that kind of testing would become for any more than a few select games, the best option is to test with a high-end GPU. And if you can do so at 2 or 3 resolutions, like we often do, this will mimic GPU scaling performance. Don’t get me wrong, it’s not a dumb suggestion to test with lower end graphics cards, it’s just a different kind of test.

I agree the workload is too high to test a huge number of games (certainly for every game). Personally though, I did find your last year's Kaby Lake vs Ryzen 1 CPU vs GPU scaling article very helpful (link), and I'd definitely like to see that article repeated when the new hardware generation comes out (like Coffee Lake vs Ryzen 2 + Turing, perhaps with a GTX 1150Ti on medium thrown in)?

 

[Thannos]

An extremely good article and very well explained. All of your points are valid and accurate as always Steve.

If anyone doubts the way Techspot does their benchmarking after reading this I'm going to assume they either have incredibly poor reading comprehension or were dropped on their head a few too many times as infants.

 

[fadingfool]

Never going to please all the people all of the time. I do enjoy reading the CPU and GPU (well all hardware) reviews - though one thing seems be common in the Ryzen tests: there seems to be almost a hard limit on fps (a very high one but there all the same) no matter what settings are changed (it can't shift data to the GPU any faster). This is specific with Ryzen and I suspect is due to the infinity fabric bandwidth. This gives an advantage to intel's ring bus but the new mesh architecture doesn't have this innate advantage.
When we see Intel's consumer 8 core it will be interesting if they stick with the ring bus (or like the older xeon's have multiple ring buses) or if mesh will be used (with subsequent performance hit).

 

[dirtyferret]

Steve, allow me to fix your CPU test suite as it is extremely flawed.

Step one - Choose a conclusion for your test making sure it lavishly praises the brand that happens to be in my PC at the time while giving not single ounce of credit to the competition. Further cement your conclusion by stating what a complete waste of money and PC street cred (internet forum cred) anyone would make not purchasing said product.

Step two - create a testing suite that reinforces the conclusion above while disregarding any test that may in any form show competitive results or god forbid showing the preferred product trailing in any test.

Your devoted readers,

The fan boy community.

 

[HardReset]

The goal is to work out which CPU will offer you the most bang for your buck at a given price point, now and hopefully in the future.
Not long ago we compared the evenly matched Core i5-8400 and Ryzen 5 2600. Overall the R5 2600 was faster once fine-tuned, but ended up costing more per frame making the 8400 the cheaper and more practical option for most gamers.

So 6C/6T CPU is more future proof than 6C/12T CPU "(y)"

Does not compute.

Edit: IIRC you previously stopped recommending quad core i5 for gaming and to get i7 instead. Difference between i5 and i7? SMT.

So your logic is pretty flawed here. You talk about future, recommend i7 over i5 because SMT but when AMD has SMT and Intel does not, suddenly SMT is ignored

 

[Strawman]

The goal is to work out which CPU will offer you the most bang for your buck at a given price point, now and hopefully in the future.
Not long ago we compared the evenly matched Core i5-8400 and Ryzen 5 2600. Overall the R5 2600 was faster once fine-tuned, but ended up costing more per frame making the 8400 the cheaper and more practical option for most gamers.

So 6C/6T CPU is more future proof than 6C/12T CPU "(y)"

Does not compute.

Edit: IIRC you previously stopped recommending quad core i5 for gaming and to get i7 instead. Difference between i5 and i7? SMT.

So your logic is pretty flawed here. You talk about future, recommend i7 over i5 because SMT but when AMD has SMT and Intel does not, suddenly SMT is ignored

He didn't say it was more future proof. According to what he said, it's not even nowproof, since he admits that the R5 is faster even today when tuned. The problem is, the i5 8400 without any tuning is cheaper and more plug and play. I would never choose it over an R5 2600, but I can see the appeal for somebody other than me.

 

[HardReset]

He didn't say it was more future proof. According to what he said, it's not even nowproof, since he admits that the R5 is faster even today when tuned. The problem is, the i5 8400 without any tuning is cheaper and more plug and play. I would never choose it over an R5 2600, but I can see the appeal for somebody other than me.

That's another good example of double standards. When talking about this "future proof" things, editors should first decide if future proof is important or not. And if it is, then decide what makes products future proof.

It just seems that almost exactly same thing (SMT) is future proof when Intel has it but when AMD has it, it's just ignored.

 

[dirtyferret]

Lol future proof! I make a comical post about fanboys and the two biggest fanboys on the forums come in to cement the stereo type right on cue, lol!

 

[hahahanoobs]

It's sad Steve had to do this. It's simply common sense guys. Period.

 

[gamerk2]

He didn't say it was more future proof. According to what he said, it's not even nowproof, since he admits that the R5 is faster even today when tuned. The problem is, the i5 8400 without any tuning is cheaper and more plug and play. I would never choose it over an R5 2600, but I can see the appeal for somebody other than me.

That's another good example of double standards. When talking about this "future proof" things, editors should first decide if future proof is important or not. And if it is, then decide what makes products future proof.

It just seems that almost exactly same thing (SMT) is future proof when Intel has it but when AMD has it, it's just ignored.

But future proof = performance. And Intel w/ SMT is still faster then AMD w/ SMT.

 

[HardReset]

But future proof = performance. And Intel w/ SMT is still faster then AMD w/ SMT.

Future proof = performance on future games.

AMD has more efficient SMT than Intel has and i5 does not have SMT at all.

 

[hahahanoobs]

Future proof = performance on future games.

AMD has more efficient SMT than Intel has and i5 does not have SMT at all.

Future proof = acceptable performance in future games.

Not when equal cores, clocks, power consumption and SMT are compared...

 

[HardReset]

Future proof = acceptable performance in future games.

Not when equal cores, clocks, power consumption and SMT are compared...

And previously i7 was more future proof than i5. Difference between i5 and i7? SMT.

 

[dirtyferret]

Best and only future proof way of buying tech is to buy it in the future otherwise its a fools game played by fanboys. The list of "future proof" CPUs is littered with poor IPC cores (Intel Q8200, Phenom I, Phenom II x6, FX series) that never got faster as multi-core was optimized and the same with today's offerings. The only CPUs that had a"future" life for gaming were CPUs with high OC. That is why the intel OC quads routinely beat the ryzen 1600 in gaming and AMD fanboys cry in the wind about "future proofing". Toms posted their best CPUs for gaming list and if you want a future proof CPU get one at the top of the list you can OC...or just buy one in the future.

https://www.tomshardware.com/reviews/cpu-hierarchy,4312.html

 

[HardReset]

Best and only future proof way of buying tech is to buy it in the future otherwise its a fools game played by fanboys. The list of "future proof" CPUs is littered with poor IPC cores (Intel Q8200, Phenom I, Phenom II x6, FX series) that never got faster as multi-core was optimized and the same with today's offerings. The only CPUs that had a"future" life for gaming were CPUs with high OC. That is why the intel OC quads routinely beat the ryzen 1600 in gaming and AMD fanboys cry in the wind about "future proofing". Toms posted their best CPUs for gaming list and if you want a future proof CPU get one at the top of the list you can OC...or just buy one in the future.

https://www.tomshardware.com/reviews/cpu-hierarchy,4312.html

Yeah and year ago Intel fanboys said that nobody needs more than 4 cores for gaming. Today nobody recommends quad core for gaming. As expected.

 

[hahahanoobs]

And previously i7 was more future proof than i5. Difference between i5 and i7? SMT.

What's the difference? About $150.
What sells more? i5's.

Yeah and year ago Intel fanboys said that nobody needs more than 4 cores for gaming. Today nobody recommends quad core for gaming. As expected.

Except dual and quads are the most popular today. Ryzen didn't change that in two years. Also, the 6T 8400 beats the 12T 2600 @ stock in games, so what was that about core count again?

 

[dirtyferret]

Allow me to list the best gaming CPUs in order so AMD fanboys can cry over their "but

Yeah and year ago Intel fanboys said that nobody needs more than 4 cores for gaming. Today nobody recommends quad core for gaming. As expected.

Truth hurts you like a sunburn on pasty white skin

 

[HardReset]

What's the difference? About $150.
What sells more? i5's.

What price difference has to do with future proof?

i5 sells more so it's more future proof, err what?

Except dual and quads are the most popular today. Ryzen didn't change that in two years. Also, the 6T 8400 beats the 12T 2600 @ stock in games, so what was that about core count again?

So you would recommend quad core for custom gaming machine? Ryzen launched March 2017, quite far from two years.

So 6 cores is better for gaming than 4 cores? Year ago 4C 7700K, that was supposed to be future proof and best for gaming.
When only AMD had 6 cores for desktop, 4 cores was better

Allow me to list the best gaming CPUs in order so AMD fanboys can cry over their "but

Truth hurts you like a sunburn on pasty white skin

Intel fanboys constantly change opinions, I don't. That's the difference. When Intel had only quad cores for LGA115X, Intel fanboys touted quad core as future proof and best gaming CPU. When Intel launched 6 cores for LGA115X, suddenly nobody recommends 4 cores for gaming. Surprising? No.

 

[Ira Wechsler]

Using 720p resolution is a bogus metric tondontests on since almost no one uses this resolution to game on these days. Stop the. Nonsense and quit using an archaic metric. Nothing else is to be said on this. Hands down it us bogus.

In addition. The 2600 is definitely more future proof than an Intel product with only 6 threads. As more and more new games come to market with proper design to handle more cores and more threads like SMT enabled or hyperthreading provides, the I5 product will definitely put many gamers in a disadvantageous position. Bogus test design leads to bogus results.and erroneous conclusions.

 

[dirtyferret]

Ah the famous AMD fan boy battle cry of of improper testing, it's the developers fault the chips lack IPC of the competition, and it's not fast now but it will be fast in the future which has never ever happened in the history of tech but yeah it will happen now

 

[hahahanoobs]

What price difference has to do with future proof?

i5 sells more so it's more future proof, err what?



So you would recommend quad core for custom gaming machine? Ryzen launched March 2017, quite far from two years.

So 6 cores is better for gaming than 4 cores? Year ago 4C 7700K, that was supposed to be future proof and best for gaming.
When only AMD had 6 cores for desktop, 4 cores was better



Intel fanboys constantly change opinions, I don't. That's the difference. When Intel had only quad cores for LGA115X, Intel fanboys touted quad core as future proof and best gaming CPU. When Intel launched 6 cores for LGA115X, suddenly nobody recommends 4 cores for gaming. Surprising? No.

Price has everything to do with how future proof you can go.

Just in: 4C 7700K was still gamers choice when 6C Ryzen came out. In fact, Sandy Bridge and up were/are probably the most future proof chips on the planet right now. Ryzen 2 did very little to sway gamers compared to Ryzen 1. There is still little reason to upgrade for those Intel cats.

Future proofing lesson 101 complete.

 

[Burty117]

Lol future proof! I make a comical post about fanboys and the two biggest fanboys on the forums come in to cement the stereo type right on cue, lol!

Who are you on about?

Future proof = performance on future games.

AMD has more efficient SMT than Intel has and i5 does not have SMT at all.

Ay! My Main Squeeze is here! How you doing buddy? It's been a while, glad to see you haven't given up on your quest for all things AMD! How's the future proof GPU and CPU you bought 4 years ago? Running the latest games at 200fps still I assume

 

[HardReset]

Price has everything to do with how future proof you can go.

Just in: 4C 7700K was still gamers choice when 6C Ryzen came out. In fact, Sandy Bridge and up were/are probably the most future proof chips on the planet right now. Ryzen 2 did very little to sway gamers compared to Ryzen 1. There is still little reason to upgrade for those Intel cats.

Future proofing lesson 101 complete.

7700K was supposed to be future proof because games use at most 4 cores. Now when Intel has i5,8400, i7-8700K etc 6-core chips, suddenly games seem to demand more than 4 cores, at least nobody recommends 4 core CPU's for gaming any more.. That's Intel fanboy logic. If Intel does not have more than 4 cores to offer, then 4 cores is best for gaming. When Intel now has 6 core CPU for mainstream market, 6 core CPU is suddenly best for gaming. It's very easy to predict that games run best on 8 cores according to Intel fanboys when Intel finally offers octa core to mainstream market.

Ryzen 2 was just little manufacturing tech upgrade, even architectural upgrades were present on Ryzen 1 but not enabled. Zen 2 will be real upgrade.

Ay! My Main Squeeze is here! How you doing buddy? It's been a while, glad to see you haven't given up on your quest for all things AMD! How's the future proof GPU and CPU you bought 4 years ago? Running the latest games at 200fps still I assume

That depends on games and settings.

 

[Strawman]

But future proof = performance. And Intel w/ SMT is still faster then AMD w/ SMT.

Nope, absolutely not.

Future proof has nothing to do with current performance, that's why there is a "future" there in that phrase. It's completely apparent to anyone with a working brain that modern games require more and more cores to run properly. A recent example is AC origins. While an R5 1600 and a i5 7600k were neck and neck in most games, the SMT of the R5 1600 allowed it to easily surpass the 7600k on this game, due to the lack of threads.

When you see CPU's hitting close to 100% usage on games, that's not a good sign of future proofness. They are already in their limits. When it comes to those AAA titles, the 12 threads of the R5 lineup are going to surpass the 6 of the 8400 sooner or later. There is no doubt about that, I mean, they already do.

Now on that note, both are freaking awesome CPU's for the money, it's just that the R5 give more bang for the buck, and anyone arguing it about it is just wrong.