AMD, VIA, Nvidia quit benchmark group due to 'Intel bias'

Jos

Jos

Posts: 3,073   +97
Staff

AMD has publicly announced it is withdrawing its support from BAPCo, a non-profit consortium which develops and distributes a benchmarking program called SYSmark, refusing to endorse the latest version of the suite because its results are allegedly unrepresentative of the workloads used in everyday computing. The company also believes there's a bias in favor of Intel.

The suite uses a number of application-based benchmarks to recreate usage patterns in the areas of office productivity, data/financial analysis, system management, media creation, 3D modeling and web development. Among the applications used in SYSmark 2012 are Microsoft Office, Adobe Creative Suite, Adobe Acrobat, WinZip, Autodesk AutoCAD and 3ds Max, and others. But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs.

"The heart of our complaint is this: the SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions," AMD's chief marketing officer Nigel Dessau wrote in a blog post.

He went on to explain how a previous complaint in 2010 got them a ban threat from BAPCo and that they nonetheless tried to work within the consortium in order to get the next-generation benchmark, SYSmark 2012, correct. Unfortunately that didn't happen, according to Dessau, who claims that only a small number of the benchmark operations in SYSmark 2012 actually contribute to the final score, and that a large proportion of the score is actually based on system performance during OCR and file compression tasks -- things an average user will rarely ever do.

Interestingly, AMD is not the only defector, as Nvidia and VIA have also decided to leave the consortium. Echoing AMD's sentiment, VIA said it wanted a more open and transparent process for developing fair and objective benchmarks that accurately measure real world performance, while Nvidia declined to comment on their reasons.

It's also not the first time AMD has had issues with BAPCo. In 2002, AMD accused the consortium of removing applications from the benchmark that showed a performance advantage for AMD over Intel.

For its part BAPCo is defending its 2012 benchmark, noting that AMD participated in its two-year development, and lamented that the latter had "once more violated the confidentiality agreement it signed in an attempt to dissuade customers from using SySmark to assess the performance of their systems." Other industry giants still backing BAPCo include Microsoft, Dell, Hewlett-Packard, Lenovo, Samsung, Sony, Toshiba and Intel.

Permalink to story.

https://www.techspot.com/news/44386-amd-via-nvidia-quit-benchmark-group-due-to-intel-bias.html

 
Hmmm. BAPco getting kickbacks from Intel OR AMD CPU's are just not as powerful as Intels? I say both can be true.

But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs.
Click to expand...

How many people have GPU's that can add to the processing of their workloads compared to people who just rely on their CPU? Now that AMD has their Llano system, are they hoping to gain more points with this?

I don't know. I guess there should at least be an option to use the GPU in the tests OR do it automatically if it senses the system has a GPU that can be used for those tasks.
 
Jibberish18 said:
Hmmm. BAPco getting kickbacks from Intel OR AMD CPU's are just not as powerful as Intels? I say both can be true.

But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs.
Click to expand...

How many people have GPU's that can add to the processing of their workloads compared to people who just rely on their CPU? Now that AMD has their Llano system, are they hoping to gain more points with this?

I don't know. I guess there should at least be an option to use the GPU in the tests OR do it automatically if it senses the system has a GPU that can be used for those tasks.
Click to expand...

Any modern GPu can do general purpose computing (nvidia from series 8000 and AMD from 3-4000 - not sure there) so most of the GPU's can do it. And if you take in cosideration that now laptops come with really good IGP's of GPU's then you have a really large portion of users that have the capability.
Also programmers are starting to work towards offloading part of the program onto GPU's (think browsers, photoshop, etc). In a few years this will become the norm.
 
In my experience, the real benchmark is the price and at any price point an AMD processor does a better job than it's Intel equivalent. A minus for Intel is that they put emphasis on speed rather then quality and here I'm speaking about transcoding video content, lacking DirectX11 in games and so on...and about delivering Atom to the public the way they did it.
 
Jibberish, I use allot of SW in the animation and game development industry that use the GPU(s). for doing on the fly renders and multi pass renders.

As mosu says, AMD processors have the cost advantage, ( particularly the 1090/1100T matched with a HD69xx GPU(s) is the best bang for my dollar by a long shot.)
 
Well I kind of felt that benchmarks can be biased and this is why I built one when I bought my AMD 1090T just to be sure.. turns out that AMD is still slower and sometimes in the same price ranges but then again AMD does a great job for lower end options..

http://www.testmycpu.com/ if any1 wants to try it.
 
nickblame said:
Well I kind of felt that benchmarks can be biased and this is why I built one when I bought my AMD 1090T just to be sure.. turns out that AMD is still slower and sometimes in the same price ranges but then again AMD does a great job for lower end options..

http://www.testmycpu.com/ if any1 wants to try it.
Click to expand...

pretty cool,

i clicked run, and it as running, how long does it take to run?
 
Are u sure we are talking about the same Intel and Amd......coz i dont see Amd having the slightest chance in any of the pricebrackets atm. Unless u take into consideration the cost of upgrading in a 3yr period, but then again i have never had the same cpu for more then 2yrs. My needs forces me to have a discrete gpu so i wont ever need the built in gpu and i doubt any gamer will ever need it for the next 3-4yrs until they prove alot stronger then the current generation.
 
Pfft. AMD's crying about intel bias even though everyone knows their chips are slower. Cry some more.

As for Nvidia, well it's quite hard to compare computing using the gpu to computing using the cpu.
 
http://www.testmycpu.com/results.php

I made 2nd rank!

BTW, my cpu clock is not 3.4Ghz... its 5Ghz OCed.
 
It's basically a fact that Intel is faster. :/
Click to expand...

It actually is a fact. A benchmark DOES NOT care if your CPU has more "bang for your buck". All it cares about, is performance. So you might say "Well I got my 6 core AMD for $200 and it threads like a mofo!" That's all fine and dandy but it might still not be able to accomplish as much as a Quad Core Intel Jerusalem chip or whatever.

Although other posters here do make a point I suppose. I don't entirely agree but it is a point.
 
thatguyandrew92 said:
It's basically a fact that Intel is faster. :/
Click to expand...
Well of course it is. When benchmarking programs refuse to add metrics that can indicate otherwise, you have all the "fact" you need.

Regardless of the actual product performance, benchmarking software should be as impartial as possible when determining results. It should also utilize all avenues of optimization for each product to ensure "best case scenario" results as well. If AMD has indeed worked to add their optimization and been excluded, thats about as damning as you can get as a benchmark software.

In any case, I've never once had SysMark sway my opinion on purchasing decisions. When building a machine, it makes more sense to look at actual performance in the applications you will use rather than a generalized machine score. If I want max FPS in Crysis, I'm dont give a damn about Sysmark.
 
PanicX said:
In any case, I've never once had SysMark sway my opinion on purchasing decisions. When building a machine, it makes more sense to look at actual performance in the applications you will use rather than a generalized machine score. If I want max FPS in Crysis, I'm dont give a damn about Sysmark.
Click to expand...
Of course the obvious problem is that the majority of people (review readers) need to be able to boil down all the numbers into a single figure (after skipping straight to the conclusions page). Best number/Highest on the chart = best CPU/GPU/HDD/ whatever.

Offering different benchmarks will give the reader a better rounded overview of what the component is capable of, but I would suggest that for many people, unless that component tops all the charts then they are going to be disappointed with the ambiguity of "multiple winners" and confused that there is no "outright winner" THERE CAN BE ONLY ONE!!!

With APU's (or on-die/cpu graphics) blurring the lines between what constitutes a CPU and GPU, using a CPU compute-only benchmark becomes less valid (esp. one that may favour a particular arch)...as do traditional GPU benchmarks and game fps when the GPU takes on traditional CPU-compute tasks (hardware acceleration, GPGPU/Co-processor etc.)
 
You must log in or register to reply here.

Similar threads

E
Intel appears to have pulled a fast one on benchmark tests for its Xeon processors
Replies
13
Views
256
Mr Majestyk
M
D
Snapdragon X Elite is faster than Intel and AMD flagship CPUs in Geekbench
Replies
15
Views
190
quadibloc
quadibloc
E
Nvidia's Grace Hopper GH200 superchip benchmarked, holds up well against AMD and Intel...
Replies
7
Views
244
Mr Majestyk
M

Latest posts

Back