Introduction
While it might seem that this article is a direct answer to 3DMark03 being launched this week, it is actually something that has been brewing within [H]ard|OCP for a while now. Brent, [H] VidCard Editor, has done a good job previewing and evaluating 3DMark03 and getting to what really needs to be known when 3DMark03 is used as a tool. At this moment in time, I find it hard to place any real-world value on the 3DMark03 score, as it does not represent anything but specific tests that FutureMark deems valuable. I am not going to dwell on that anymore except to say that we will not be using the overall scoring data from 3DMark03 in the evaluation of video cards at this time and that the introduction of 3DMark03 this week seems to only hammer home our points in this editorial.
The Problem
To put it simply, current synthetic benchmarks overall, do a disservice to the hardware community and everyone that will ever buy a 3D video card or a computer that has one installed. Outside of the possibility of being misleading by focusing on performance factors that are not representative of actual game play, there is another dark “secret” that is actually being paid for with your money, and at the same time forces hardware manufacturers to deliver less to you.
As illustrated by the “old” and “new” driver sets from NVIDIA in our 3DMark03 Preview, it is obvious that NVIDIA has put some work into making sure that NVIDIA “looks good” when the final score is displayed on your monitor. Be assured that ATI has done the same exact thing. Both companies would be making bad decisions if they did not optimize for techniques used in this specific benchmark. ATI has had advance copies of the benchmark, where NVIDIA has not, this is because NVIDIA will no longer pay FutureMark a “subscription fee”. This alone spurs several arguments but please stay with me on our line of thought as there is a very important point to be made here that will make all those other discussions meaningless.
If NVIDIA and ATI are devoting resources to optimizing their drivers for a synthetic benchmark that does not equate to real world gaming, what benefit is in it for us? By “us”, I mean the folks that buy the video cards and the games. The short answer to the question is that it does not benefit us at all, but rather harms us. While NVIDIA and ATI are slaving away to make sure that optimizations are built into their drivers so they get better benchmark scores as illustrated above, the gamer's true experience gets ignored.
