1. TechSpot is dedicated to computer enthusiasts and power users. Ask a question and give support. Join the community here.
    TechSpot is dedicated to computer enthusiasts and power users.
    Ask a question and give support.
    Join the community here, it only takes a minute.
    Dismiss Notice

UserBenchmark offers explanation for changes to CPU score weights

By onetheycallEric ยท 32 replies
Jul 28, 2019
Post New Reply
  1. This past week a portion of the enthusiast community was lit aflame with controversy over UserBenchmark's decision to adjust its CPU scoring system. While not a popular destination among hardcore users or reviewers, UserBenchmark remains a widely used tool for the general consumer and its comparisons rank highly in Google search results -- Tom's Hardware notes the website serves nearly 10 million visitors each month.

    Enthusiasts were taken back by the website's questionable decision to adjust its scoring algorithm in favor of single-threaded performance, inflating scores in such a way that doesn't appear to be entirely representative of the processor in question. Previously, UserBenchmark apportioned the average score as 30% of the single-threaded performance, 60% of the quad-core performance, and 10% of the multi-core performance. Now, the weighted average is 40% single-core, 58% quad-core, and only 2% multi-core.

    They claim this is to offset "unrealistic overestimating" for the new core-heavy Ryzen 3000 series, and the new score weighing indeed doesn't show AMD's newest chips in quite as favorable a light -- a point of contention among AMD fans, to be sure. UserBenchmark updated its FAQ section to address the outcry.

    Shortly after the Ryzen 3000 release, which we welcomed emphatically, we noticed that our CPU gaming and desktop estimates were unrealistically overestimating all CPUs with core counts beyond 8 so we corrected the estimates. Our underlying data points for single, quad and multi core performance remain unchanged and are clearly visible together with gaming, desktop and workstation scores on each of our product and comparison pages. (...) At present we estimate that our CPU gaming index is accurate to around 8%. By rebalancing the weights in favor of more cores we can probably reduce the error to around 5%. Even after a rebalance, the 4 core i3-9350KF would still, on average, offer a similar gaming experience to the 18 core i9-9980XE, a fact which many of our most vocal critics seem to find hilarious...

    In UserBenchmark's defense, they do acknowledge readers need to look deeper than the top of the page to get the big picture. However, the bizarre choice to weigh the average score so heavily in favor of single core/thread performance is hard to ignore. Especially when multi-core and multi-threaded performance is only going to become more important.

    While I won't take sides here, I will point out professional reviews are the best way to measure a CPU's performance and glean a purchasing decision. If you haven't already, check out TechSpot's Ryzen 3000 coverage.

    Permalink to story.

     
  2. mosu

    mosu TS Evangelist Posts: 508   +122

    When Intel will produce commercial non-server parts with 16 to 32 cores, they'll change back the scoring system for whatever reason they will consider as an explanation.
     
  3. QuantumPhysics

    QuantumPhysics TS Evangelist Posts: 1,639   +1,233

    As if Intel or AMD actually care about benchmarks.

    You really have no other choice.
     
  4. grumblguts

    grumblguts TS Addict Posts: 114   +104

    Manipulation to consumer information.
    surly this comes under false advertising and subject to criminal proceedings
     
    MikitaM likes this.
  5. VitalyT

    VitalyT Russ-Puss Posts: 4,612   +3,222

    So the benchmark relies on results from software that's not optimized for use with that many cores, and suddenly it's the CPU's fault?
     
    MikitaM likes this.
  6. Dharan

    Dharan TS Rookie

    Yeah... Seems very much like on Intel's payroll... Such websites should be called out for what they are doing periodically to create awareness in tech community....
     
  7. Puiu

    Puiu TS Evangelist Posts: 3,573   +2,055

    Even Intel fans agree that this is 100% something decided behind the scenes by Intel. God knows how much they paid them.
     
  8. hahahanoobs

    hahahanoobs TS Evangelist Posts: 2,619   +945

    "Especially when multi-core and multi-threaded performance is only going to become more important."

    Someone remind me what the keywords are here....
     
  9. cliffordcooley

    cliffordcooley TS Guardian Fighter Posts: 11,584   +5,140

    All I have to say is; anyone looking at the scoring system, should understand where the numbers come from. I for one do not believe the numbers should be averaged into one.
     
    Impudicus likes this.
  10. treetops

    treetops TS Evangelist Posts: 2,647   +616

    It's popularity is due to it being a small download, fast test and requiring user to open their webpage to see the results.(which probably upticks the site on google results) UserBenchmarks is a terrible way to compare parts. Even before this update. It's like that misleading 3600 "leak" that showed it above the 9900k on passmark on single threaded performance. That was due to a l2n passmark score. Being that it was the only score of course it would be higher on l2n compared to an average of millions of benchmarks made by l2n non l2n air water stock ovclock etc etc.

    I quoted passmarks single threaded performance chart in the past, but I didn't understand the way it went about it's testing. No graph that goes off of user scores is ever a standard that should be used when comparing components. User benchmarks are a for fun comparison. THEY SHOULD SHOW STOCK SCORES with a clearly shown test system to let people know what a realistic comparison looks like.

    Another reason your average joe will use a test like this is to make sure what they have is working properly. Say you buy a video card and you just want to make sure it's not somehow underperforming, it sounds like a good idea to head over to a site like this. But even with that they need to show a base performance.

    This site is misleading and unrealistic. It's only value is for fun benchmarks against other users. Claiming to be anything else....
     
  11. fps4ever

    fps4ever TS Evangelist Posts: 329   +322

    There are plenty of other good free bench marking tools out there so why bother with an Intel biased one. I can understand adjusting score percentages but all the way down to 2% for multi core smells like Intel influence.
     
    Charles Olson, MikitaM and Impudicus like this.
  12. brucek

    brucek TS Maniac Posts: 201   +224

    There's no reason to do an artificial "blended" number which will be right for no one. They should just have a checkbox for the user to indicate whether they regularly do any of the few (consumer) things that make use of > 4 cores, or just present two sets of numbers.

    If you have a real parallel workload the number of cores is super relevant to you and should not be watered down. For everyone else, any cores past 4 are likely not adding anything at all and I'd argue that core #3 and core #4 might be contributing precious little too.
     
    Charles Olson likes this.
  13. pcnthuziast

    pcnthuziast TS Evangelist Posts: 614   +210

    Nobody should use UserBenchmark as a legitimate source for info or comparison. Nobody!
     
    Charles Olson and MikitaM like this.
  14. Cycloid Torus

    Cycloid Torus Stone age computing - click on the rock below.. Posts: 4,139   +1,218

    Should be 3-D, with weights set by user.
     
    Impudicus likes this.
  15. kmo911

    kmo911 TS Booster Posts: 132   +14

    Hopefully they would find alike info when they collect info from benchmark.
     
  16. Evernessince

    Evernessince TS Evangelist Posts: 4,183   +3,793

    I don't know about Intel paying anyone but the decision is certainly receiving flack from both Intel and AMD users. Going off what we know, I do not believe these metrics represent modern workloads in games or otherwise.
     
    Charles Olson, MikitaM and Impudicus like this.
  17. cliffordcooley

    cliffordcooley TS Guardian Fighter Posts: 11,584   +5,140

    Maybe ten years ago, certainly not with today's applications.
     
  18. Maxiking

    Maxiking TS Booster Posts: 62   +84

    AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.
     
  19. Puiu

    Puiu TS Evangelist Posts: 3,573   +2,055

    This is about not deceiving people, not about cry babies. Otherwise nobody would care about illegal anti-competitive practices. Don't sit by and do nothing.
     
  20. fps4ever

    fps4ever TS Evangelist Posts: 329   +322

    That is an excellent way to look at it, if um you are like a 12 year year old prepubescent.
     
  21. Maxiking

    Maxiking TS Booster Posts: 62   +84


    You mean like the false 4.6 Ghz boost on 3900x?

    Or false boosts on Ryzen 3xxx in general and no one giving a damn about it?

    Good joke, m8.
     
  22. Maxiking

    Maxiking TS Booster Posts: 62   +84

    Speak of the devil...
     
  23. fps4ever

    fps4ever TS Evangelist Posts: 329   +322

    Huh? I like both Intel and AMD. Especially close competition for a win on the consumer side. Your fake injustice is displaced. I sense an Intel bias leaking through.
     
    Last edited: Jul 29, 2019
  24. Evernessince

    Evernessince TS Evangelist Posts: 4,183   +3,793

    You are referring to a BIOS issue that has already been fixed. If the issue was as bad as you make it seem, AMD would already be getting sued. This isn't a GTX 970 3.5GB RAM thing.
     
    Charles Olson and MikitaM like this.
  25. arrowflash

    arrowflash TS Enthusiast Posts: 58   +46

    I fail to see what's wrong here. Single core, single threaded ipc performance IS more important for power users. They didn't change much, only increased the weight from 30% (which I'd say it was indeed too low) to 40% - in fact, if it was up to me, I'd put the weighted average of single core performance at 50% at the very least, unless my benchmarks were targeted at corporate consumers for whom multi-core performance might indeed have more weight.
    Anyways, their timing for this change couldn't be worse. Of course there would be drama especially from AMD fanboys. This coming from someone who's an opportunistic buyer and doesn't favor any companies.
     

Add your comment to this article

You need to be a member to leave a comment. Join thousands of tech enthusiasts and participate.
TechSpot Account You may also...