Since the release of 3DMark03 the question for reviewers has been whether or not to use synthetic benchmarks for 3D reviews. Beyond3D now asks the same question, however rather than looking for opinions from the usual suspects,
they put it to a few game developers and ask what they think is right.

[Dany Lepage] During the pre-production phase of a video game, you work as hard as possible to include as many features as possible in your engine so artists/level designers can use these features to make the game. Features that are coming late are either not going to be use as much or don't require as much involvement from the assets creator. This explains why there are so few real per pixel games out there (with normal maps), it changes the whole production pipeline and that need to be done from the beginning of production or previous assets are going to be wasted.

Practically, what this means is that we rush features in and we barely think about optimization for a while. Would that be worthwhile to benchmark video cards (or CPUs) with this kind of engine? In my opinion, the answer is no.

Read more