Home › News › Hardware
AMD, VIA, Nvidia quit benchmark group due to 'Intel bias'
AMD has publicly announced it is withdrawing its support from BAPCo, a non-profit consortium which develops and distributes a benchmarking program called SYSmark, refusing to endorse the latest version of the suite because its results are allegedly unrepresentative of the workloads used in everyday computing. The company also believes there's a bias in favor of Intel.
The suite uses a number of application-based benchmarks to recreate usage patterns in the areas of office productivity, data/financial analysis, system management, media creation, 3D modeling and web development. Among the applications used in SYSmark 2012 are Microsoft Office, Adobe Creative Suite, Adobe Acrobat, WinZip, Autodesk AutoCAD and 3ds Max, and others. But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs.
"The heart of our complaint is this: the SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions," AMD's chief marketing officer Nigel Dessau wrote in a blog post.
He went on to explain how a previous complaint in 2010 got them a ban threat from BAPCo and that they nonetheless tried to work within the consortium in order to get the next-generation benchmark, SYSmark 2012, correct. Unfortunately that didn't happen, according to Dessau, who claims that only a small number of the benchmark operations in SYSmark 2012 actually contribute to the final score, and that a large proportion of the score is actually based on system performance during OCR and file compression tasks -- things an average user will rarely ever do.
Interestingly, AMD is not the only defector, as Nvidia and VIA have also decided to leave the consortium. Echoing AMD's sentiment, VIA said it wanted a more open and transparent process for developing fair and objective benchmarks that accurately measure real world performance, while Nvidia declined to comment on their reasons.
It's also not the first time AMD has had issues with BAPCo. In 2002, AMD accused the consortium of removing applications from the benchmark that showed a performance advantage for AMD over Intel.
For its part BAPCo is defending its 2012 benchmark, noting that AMD participated in its two-year development, and lamented that the latter had "once more violated the confidentiality agreement it signed in an attempt to dissuade customers from using SySmark to assess the performance of their systems." Other industry giants still backing BAPCo include Microsoft, Dell, Hewlett-Packard, Lenovo, Samsung, Sony, Toshiba and Intel.
Related Stories
User Comments (22)
Post a comment|
Jibberish18
on June 23, 2011 9:43 AM |
Hmmm. BAPco getting kickbacks from Intel OR AMD CPU's are just not as powerful as Intels? I say both can be true. But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs. How many people have GPU's that can add to the processing of their workloads compared to people who just rely on their CPU? Now that AMD has their Llano system, are they hoping to gain more points with this? I don't know. I guess there should at least be an option to use the GPU in the tests OR do it automatically if it senses the system has a GPU that can be used for those tasks. |
|
Puiu
on June 23, 2011 10:35 AM |
Jibberish18 said: Hmmm. BAPco getting kickbacks from Intel OR AMD CPU's are just not as powerful as Intels? I say both can be true. But AMD argues that BAPCo is not taking advantage of GPUs for general purpose computing tasks, despite the fact that many applications support it, and instead solely relies on performance of CPUs. How many people have GPU's that can add to the processing of their workloads compared to people who just rely on their CPU? Now that AMD has their Llano system, are they hoping to gain more points with this? I don't know. I guess there should at least be an option to use the GPU in the tests OR do it automatically if it senses the system has a GPU that can be used for those tasks. Any modern GPu can do general purpose computing (nvidia from series 8000 and AMD from 3-4000 - not sure there) so most of the GPU's can do it. And if you take in cosideration that now laptops come with really good IGP's of GPU's then you have a really large portion of users that have the capability. Also programmers are starting to work towards offloading part of the program onto GPU's (think browsers, photoshop, etc). In a few years this will become the norm. |
|
mosu
on June 23, 2011 10:38 AM |
In my experience, the real benchmark is the price and at any price point an AMD processor does a better job than it's Intel equivalent. A minus for Intel is that they put emphasis on speed rather then quality and here I'm speaking about transcoding video content, lacking DirectX11 in games and so on...and about delivering Atom to the public the way they did it. |
|
Guest
on June 23, 2011 10:46 AM |
Jibberish, I use allot of SW in the animation and game development industry that use the GPU(s). for doing on the fly renders and multi pass renders. As mosu says, AMD processors have the cost advantage, ( particularly the 1090/1100T matched with a HD69xx GPU(s) is the best bang for my dollar by a long shot.) |
|
nickblame
on June 23, 2011 10:46 AM |
Well I kind of felt that benchmarks can be biased and this is why I built one when I bought my AMD 1090T just to be sure.. turns out that AMD is still slower and sometimes in the same price ranges but then again AMD does a great job for lower end options.. http://www.testmycpu.com/ if any1 wants to try it. |
|
Guest
on June 23, 2011 11:00 AM |
Balls get in the way to a bad dancer. |
|
Burty117
on June 23, 2011 11:06 AM |
nickblame said: Well I kind of felt that benchmarks can be biased and this is why I built one when I bought my AMD 1090T just to be sure.. turns out that AMD is still slower and sometimes in the same price ranges but then again AMD does a great job for lower end options.. http://www.testmycpu.com/ if any1 wants to try it. pretty cool, i clicked run, and it as running, how long does it take to run? |
|
gwailo247
on June 23, 2011 11:24 AM |
"You guys are mean, I'm taking my CPUs and going home." - AMD |
|
zillion
on June 23, 2011 11:42 AM |
Are u sure we are talking about the same Intel and Amd......coz i dont see Amd having the slightest chance in any of the pricebrackets atm. Unless u take into consideration the cost of upgrading in a 3yr period, but then again i have never had the same cpu for more then 2yrs. My needs forces me to have a discrete gpu so i wont ever need the built in gpu and i doubt any gamer will ever need it for the next 3-4yrs until they prove alot stronger then the current generation. |
|
princeton
on June 23, 2011 12:02 PM |
Pfft. AMD's crying about intel bias even though everyone knows their chips are slower. Cry some more. As for Nvidia, well it's quite hard to compare computing using the gpu to computing using the cpu. |
|
Johnny Utah
on June 23, 2011 12:36 PM |
Princeton said: Pfft. AMD's crying about intel bias even though everyone knows their chips are slower. Cry some more. This |
|
Guest
on June 23, 2011 1:51 PM |
http://www.testmycpu.com/results.php I made 2nd rank! BTW, my cpu clock is not 3.4Ghz... its 5Ghz OCed. |
|
thatguyandrew92
on June 23, 2011 3:06 PM |
It's basically a fact that Intel is faster. :/ |
|
Jibberish18
on June 23, 2011 4:27 PM |
It's basically a fact that Intel is faster. :/ It actually is a fact. A benchmark DOES NOT care if your CPU has more "bang for your buck". All it cares about, is performance. So you might say "Well I got my 6 core AMD for $200 and it threads like a mofo!" That's all fine and dandy but it might still not be able to accomplish as much as a Quad Core Intel Jerusalem chip or whatever. Although other posters here do make a point I suppose. I don't entirely agree but it is a point. |
|
PanicX
on June 23, 2011 4:33 PM |
thatguyandrew92 said: It's basically a fact that Intel is faster. :/ Well of course it is. When benchmarking programs refuse to add metrics that can indicate otherwise, you have all the "fact" you need. Regardless of the actual product performance, benchmarking software should be as impartial as possible when determining results. It should also utilize all avenues of optimization for each product to ensure "best case scenario" results as well. If AMD has indeed worked to add their optimization and been excluded, thats about as damning as you can get as a benchmark software. In any case, I've never once had SysMark sway my opinion on purchasing decisions. When building a machine, it makes more sense to look at actual performance in the applications you will use rather than a generalized machine score. If I want max FPS in Crysis, I'm dont give a damn about Sysmark. |
|
nickblame
on June 23, 2011 4:44 PM |
Guest said: http://www.testmycpu.com/results.php I made 2nd rank! BTW, my cpu clock is not 3.4Ghz... its 5Ghz OCed. the clock read 3.4Ghz in idle, when busy the clock climbs so its natural to detect only 3.4Ghz. I might need to monitor the clock throughout the benchmark and keep the peak.. |
|
PinothyJ
on June 23, 2011 6:15 PM |
Well are not we all experts then :P? |
|
dividebyzero
on June 23, 2011 7:41 PM |
In any case, I've never once had SysMark sway my opinion on purchasing decisions. When building a machine, it makes more sense to look at actual performance in the applications you will use rather than a generalized machine score. If I want max FPS in Crysis, I'm dont give a damn about Sysmark. Of course the obvious problem is that the majority of people (review readers) need to be able to boil down all the numbers into a single figure (after skipping straight to the conclusions page). Best number/Highest on the chart = best CPU/GPU/HDD/ whatever. Offering different benchmarks will give the reader a better rounded overview of what the component is capable of, but I would suggest that for many people, unless that component tops all the charts then they are going to be disappointed with the ambiguity of "multiple winners" and confused that there is no "outright winner" THERE CAN BE ONLY ONE!!! With APU's (or on-die/cpu graphics) blurring the lines between what constitutes a CPU and GPU, using a CPU compute-only benchmark becomes less valid (esp. one that may favour a particular arch)...as do traditional GPU benchmarks and game fps when the GPU takes on traditional CPU-compute tasks (hardware acceleration, GPGPU/Co-processor etc.) |
|
captaincranky
on June 23, 2011 10:28 PM |
" THERE CAN BE ONLY ONE" !!! I see the "Highlander" series has made the third world. With that out of the way, "what's a benchmark"? Somedays, I swear my heart's not into this computer s***..... |
|
dividebyzero
on June 23, 2011 10:49 PM |
I see the "Highlander" series has made the third world. Series??? We've seen just the one. That Christopher Lambert must have a fine future as an actor ahead of him.
Somedays, I swear my heart's not into this computer s***.....  Uh oh, sounds like Jobs has another Apple convert. |
|
captaincranky
on June 23, 2011 11:05 PM |
Series??? We've seen just the one. That Christopher Lambert must have a fine future as an actor ahead of him. Soon you'll have Christopher Lloyd in, "Back to the Future", and you'll forget all about him.
Uh oh, sounds like Jobs has another Apple convert. "It just works, by cracky".....: |
|
Guest
on June 24, 2011 4:11 AM |
Quite true. Most benchmark suites are full of it. |
Most Popular
| Trending | Featured |
-
iOS 5.1.1 untethered jailbreak tool released, supports 4S, iPad 3
-
After five days, Facebook ranks as worst IPO flop of the decade
-
Rumor: Windows 8 RC will launch June 1, will ship with Adobe Flash
-
Rumor: AMD "Piledriver" FX CPU production to begin Q3 2012
-
Is Apple's USB wall adapter really worth $29?
Editors' Keyboard Picks
Subscribe to TechSpot
Get free exclusive content, learn about new features and tech breaking news.
