Quake3World.com
https://www.quake3world.com/forum/

Quake 3 Map Compile Benchmark
https://www.quake3world.com/forum/viewtopic.php?f=10&t=4776		 Page 1 of 2
Author:  mjrpes [ 05-10-2005 05:08 PM ]
Post subject:  Quake 3 Map Compile Benchmark
This is my attempt at a standardized benchmark to see which systems compile quake maps the best.

It's been a year or so since I last posted this, but a thread in GD made me think that perhaps some of you have upgraded your systems since then. I'm still curious to see how an Althon64 compares to a P4 system. It looks like a standard P4 is a better system than the Althon XP, but these processors are starting to get old (nevermind how old quake 3 is itself!) with the Althon64 out and dual processors around the corner.

Webpage: http://www.ciole.net/quake_bench/

From the webpage you can download the benchmark tool and post your results. :)
Author:  obsidian [ 05-10-2005 06:05 PM ]
Post subject: 
Maybe it's because my 3.0 GHz processor and 1GB of RAM is too fast for it, but I can't seem to get the benchmark to run correctly (or at least output the results).

I have it installed in C:\Games\Quake III Arena\benchmark\

It gets as far as displaying "Now Benchmarking Map Compile . . . Please Wait . . ." then the Command Prompt window closes. It seems as if the BSP and PRT file are generated, but I doubt that it's getting as far as the light compile, since it only runs for a few seconds.

In the results.txt file is this:
QUAKE3 MAP BENCHMARK 1.1 - RESULTS
==================================

Any ideas?
Author:  mjrpes [ 05-10-2005 06:22 PM ]
Post subject: 
Interesting.

The process goes from map compile, to vis, to bspc, then to lighting.

At the stage you mention, it is definitely getting caught up in the map compile, since no output for the map compile is made into the result.txt file. The actual command it runs at that point is:

"q3map2 -fs_basepath .. -meta -v q3dm1sample.map"

I wonder if something could be wrong here. I ran the batch file where I commented out the actual command, so as to emulate a super fast system that could compile the map instantly, and no problems were encountered.

I don't know what else to think here.
Author:  mjrpes [ 05-10-2005 11:10 PM ]
Post subject: 
obsidian, try downloading this new version:

http://ciole.net/quake_bench/quake3_map ... rk_1.2.zip

This new version has all necessary files and doesn't rely on quake3 anymore; you can run it from anywhere. Perhaps this will solve the problem you were having.
Author:  [acid] [ 05-11-2005 11:41 AM ]
Post subject: 
i need an upgrade :D
Author:  TTK-Bandit [ 05-11-2005 01:10 PM ]
Post subject: 
the problem seems to be that obsidian has no english version of windows.
the date and time variables are diffent on different windows language versions.
so.. I just wrote a little test app, that can do it:
http://mitglied.lycos.de/Q3Coderz/files/

Just copy the 2 files into the "files" directory and run bench.exe
bench.cfg format is pretty simple:
1. line is benchmark version
all lines that follow are benchmark tests in this format:
name|commandline

But I've seen that you build your homepage data out of the output,
so you'd better tell me what output you wanna have, so I can rewrite some lines.
(I can't run your batch files either and I don't wanna fix'em :shrug: )
Have Fun!
Bandit.
Author:  bork[e] [ 05-11-2005 01:35 PM ]
Post subject: 
I didn't use the new version when I just posted my results. Didn't notice that print until I was comparing my results with others.
Author:  mjrpes [ 05-11-2005 03:22 PM ]
Post subject: 
TTK-Bandit wrote:
the problem seems to be that obsidian has no english version of windows.
the date and time variables are diffent on different windows language versions.
so.. I just wrote a little test app, that can do it:
http://mitglied.lycos.de/Q3Coderz/files/

Just copy the 2 files into the "files" directory and run bench.exe
bench.cfg format is pretty simple:
1. line is benchmark version
all lines that follow are benchmark tests in this format:
name|commandline

But I've seen that you build your homepage data out of the output,
so you'd better tell me what output you wanna have, so I can rewrite some lines.
(I can't run your batch files either and I don't wanna fix'em :shrug: )
Have Fun!
Bandit.



Wow, very cool. I'm not that surprised the batch file was the culprit, but I never would have though about different languages being the issue. I don't like batch files anyway, so this is a good reason to get away from it.

I'm at work now but I'll get you the info on the formatting when I get home.
Author:  mjrpes [ 05-11-2005 05:54 PM ]
Post subject: 
Alright here's the format:

Quote:
QUAKE3 MAP BENCHMARK 1.2 - RESULTS
==================================
Map Compile = 00:05
Vis = 01:05
Bspc = 00:22
Lighting = 02:12
Total = 03:44


Line 1 should be exactly as above. I parse to make sure the 24th character is equal to 2, so that results are submitted using the correct version (1.2) of the test.

Line 2 is ignored.

For the next five lines, all that matter is that there is an equal sign somewhere and that there is a time value of the format "mm:ss" to the right of the equal sign. No need to worry about whitespace or text to the left of the equal sign. These lines should be in the order of Map Compile, Vis, Bspc, Lighting, and Total, like above.

There shouldn't be any additional stuff besides these seven lines.
Author:  mjrpes [ 05-11-2005 08:25 PM ]
Post subject: 
bork[e] wrote:
I didn't use the new version when I just posted my results. Didn't notice that print until I was comparing my results with others.


if you post a new score up i'll delete the old one
Author:  obsidian [ 05-11-2005 09:39 PM ]
Post subject: 
If I just compile the map with those compile switches using my own batch file, shouldn't I be able to achieve reliable results and just post the times from the compile log?

And I'm using an English version of WinXP Pro.
Author:  mjrpes [ 05-11-2005 10:05 PM ]
Post subject: 
I'd imagine so. But for consistancy sake I'd run it with the new benchmark tool I created, since there's less chance of your time being off with the rest because the files you have might be different. Then you could post your results here and I'd add them to the webpage.

Or you could wait until TTK-Bandit modifies his progam to output everything correctly, and I'll create a new benchmark tool to download and you can use that.
Author:  TTK-Bandit [ 05-12-2005 09:25 AM ]
Post subject: 
go try it :)
Author:  Survivor [ 05-12-2005 10:05 AM ]
Post subject: 
There ya got my magnificent results. Maybe i'll retry when i do a format, my system is a bit clogged.
Author:  TTK-Bandit [ 05-12-2005 10:29 AM ]
Post subject: 
optimized a few things (will not change results ;)) and updated the package
I'm not looking very often into this forum, so I added the sourcecode to the package if there's need to change something.
Author:  Survivor [ 05-12-2005 10:33 AM ]
Post subject: 
Maybe just an idea for you to look into but some programs check on processor, ram and os version themselves. Maybe try to integrate that into the results for generalized standard.
Author:  mjrpes [ 05-12-2005 11:32 AM ]
Post subject: 
Survivor wrote:
Maybe just an idea for you to look into but some programs check on processor, ram and os version themselves. Maybe try to integrate that into the results for generalized standard.


ya know... that's a really good idea. mebbe when i get some time after school ends in a couple of weeks i'll experiment with that.
Author:  mjrpes [ 05-12-2005 11:39 AM ]
Post subject: 
TTK-Bandit wrote:
optimized a few things (will not change results ;)) and updated the package
I'm not looking very often into this forum, so I added the sourcecode to the package if there's need to change something.


checking now. thanks for your help, i'll add in you in for credit when i update the site with the new benchmark... hopefully sometime tonight.

i'm also glad you including source. i'm thinking of playing around with an auto-submit to website feature, so people don't have to do the copy and paste thing. also the idea of automatically getting cpu/ram info sounds neat, whenever i might get a moment to look into that!
Author:  TTK-Bandit [ 05-12-2005 12:31 PM ]
Post subject: 
writing system info is a good idea, but to submit automatically not!
I don't like programms that access internet without permission.
if you add it, ask the user at least if he wants to..
Author:  TTK-Bandit [ 05-12-2005 01:38 PM ]
Post subject: 
whenever you might get a moment to look into that: ;)

edit: (removed code snippet for os, cpu, ram detection)
never mind.. I added it to the bench.exe and uploaded it on a seperate file to my webspace.
Sry dunno why I'm this generously today :dork:


ah nearly forgot:
you just need to adapt your web script to read the new format:

QUAKE3 MAP BENCHMARK 1.2 - RESULTS
==================================
OS = WinXP
CPU = Intel w/ SSE2 2670 MHz
RAM = 511 MByte
==================================
Map Compile = 00:05
...
Author:  mjrpes [ 05-13-2005 12:45 AM ]
Post subject: 
Thanks for you generousity TTK-Bandit, even if you don't know why :)

I updated the tool with your program, so hopefully the wierd issue that obsidian had will go away, even if it wasn't related to the date thing. Give it another go, obsidian :)

I'm waiting a bit to impliment the bench2 program that you made, giving you system info, as I want to see if it's possible to get more specific cpu info. I notice when I go to system properties in windows I'm told I'm running an Athlon XP processor, but the cpu detail in the benchmark says only its an AMD with 3DNow. I found an article here,

http://www.codeguru.com/Cpp/W-P/system/ ... /c9087__2/

that offer another way to get cpu info, using windows system registry. I wonder if there would be any downsides to doing it this way. I'll have to delay looking at this for a couple of weeks, as I've procrastinated enough as it is with school work :(
Author:  TTK-Bandit [ 05-13-2005 03:16 AM ]
Post subject: 
I've seen that too, but I'm not sure if all operating systems support these values..
winxp does I think win2k will too, but what with nt or 9x ?
well fortunately I have win98se on my laptop so I can tell you, it has only these values:
Identifier
Update Status
VendorIdentifier
and as you may have seen, they are not the values you are looking for.
you need to find another way...
Author:  TTK-Bandit [ 05-13-2005 05:02 AM ]
Post subject: 
ok this time I know why I'm so generously today..
I've learned something new :D
got something for ya:

Output:
Quote:
Type: GenuineIntel
Name: Intel(R) Pentium(R) 4 CPU 2.66GHz
Features: FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV
Extended Features: VME, DE, PSE, MSR, PAE, MCE, APIC, SEP, MTRR, PGE, MCA, PAT,
PSE36, FXSR, DS, SS, TM

just tell me, what how you want them in the results.txt
Type and Extended Features may be irrelevant, so tell me if you want them or not.
(I'll leave the source inside, if you at some time want those, but I'll comment'em out)

edit: maybe this way?:
CPU = Intel(R) Pentium(R) 4 CPU 2.66GHz | FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV
Author:  obsidian [ 05-13-2005 06:47 AM ]
Post subject: 
BTW, the Q3Map2 version you have in the zip is 2.5.11. You may want to update that with 2.5.16 since there are some additional compiler optimizations since then.

The Bench.exe works great. Thanks for looking into this. I'll post some more accurate benchmark results later, since I tried it now with a bunch of other stuff running.
Author:  obsidian [ 05-13-2005 08:38 AM ]
Post subject: 
Actually, looks like 2.5.11 is slightly faster overall:

2.5.11
Map Compile= 30125ms
Vis= 46984ms
Bspc= 35062ms
Lightning= 164438ms
Total Time= 276609ms

2.5.16
Map Compile= 29891ms
Vis= 47516ms
Bspc= 47797ms
Lightning= 174000ms
Total Time= 299204ms
Author:  Fjoggs [ 05-13-2005 09:04 AM ]
Post subject: 
Why's bspc different when it's the same compiler used for that specific progress (?)
Author:  obsidian [ 05-13-2005 09:12 AM ]
Post subject: 
Not sure, but looking through it again, it's safe to say that something was wrong with the above results... 30 seconds for BSP? I just noticed that after posting the results.

Anyway, recompiled with 2.5.11 after restarting my computer and these look like better results. I'll try with 2.5.16 again later:

2.5.11
Map Compile= 4203ms
Vis= 43703ms
Bspc= 24609ms
Lightning= 115641ms

mjrpes: Please remove the slower results that I posted on the website. Thanks.
Author:  obsidian [ 05-13-2005 09:34 AM ]
Post subject: 
Q3Map2 2.5.16 Results:

Map Compile= 4594ms
Vis= 43234ms
Bspc= 24672ms
Lightning= 112078ms
Total Time= 184578ms

Q3Map2 2.5.16 is faster overall by about 4 seconds. BSPC is the same.
Author:  Fjoggs [ 05-13-2005 10:04 AM ]
Post subject: 
Yeah that made more sense. :)
Author:  obsidian [ 05-13-2005 10:07 AM ]
Post subject: 
Noticed in the results.txt it says "lightning" instead of "lighting".
Author:  mjrpes [ 05-13-2005 10:50 AM ]
Post subject: 
TTK-Bandit wrote:
ok this time I know why I'm so generously today..
I've learned something new :D
got something for ya:

Output:
Quote:
Type: GenuineIntel
Name: Intel(R) Pentium(R) 4 CPU 2.66GHz
Features: FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV
Extended Features: VME, DE, PSE, MSR, PAE, MCE, APIC, SEP, MTRR, PGE, MCA, PAT,
PSE36, FXSR, DS, SS, TM

just tell me, what how you want them in the results.txt
Type and Extended Features may be irrelevant, so tell me if you want them or not.
(I'll leave the source inside, if you at some time want those, but I'll comment'em out)

edit: maybe this way?:
CPU = Intel(R) Pentium(R) 4 CPU 2.66GHz | FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV



I think you can leave out the stuff to the right of the pipe symbol. What I'll do is let the user submit what's in the text file, and then I'll allow them to edit the processor info before they finally submit it, so they can include additional info like cache size, multi processors, and Hyper Threading. The stuff like FPU, MMX, SSE, etc, you can already deduce that it exists, based on what the processor is (Pentium 4, AMD Athlon, etc), so I don't think it's necessary. That's my take on it.

The RAM and OS info are also important. So the three additional fields in the Results.txt file should be,

CPU, RAM, OS

You can put it in any order you want, and you can put in a blank/dividing line in the results file to make everything look clearer if you want to. I'll just take what you have done and make my php script match it accordingly.

Thanks again :)
Author:  mjrpes [ 05-13-2005 10:56 AM ]
Post subject: 
obsidian wrote:
BTW, the Q3Map2 version you have in the zip is 2.5.11. You may want to update that with 2.5.16 since there are some additional compiler optimizations since then.



I should have updated q3map2 to the most recent version when I redid the benchmark to version 1.2 and reset all the results. I don't know if I want to do it again, because then everyone would have to redo the test if we want the results to be consistent.

But I'm glad it finally worked for you. :)
Author:  TTK-Bandit [ 05-13-2005 12:26 PM ]
Post subject: 
OK did it.. You can now download the (hopefully) last version: 1.2
(again with sourcecode)
Author:  mjrpes [ 05-14-2005 03:35 PM ]
Post subject: 
updated with system info included in results :up:
Author:  Hr.O [ 05-24-2005 10:11 AM ]
Post subject: 
Silly benchmarking :)

Just a warning for you guys, do not rely too much on these benchmarks. Since you are all running it on your "working" systems, this benchmark is bound to be pretty subjective.

More often then not it's the background software that dictates the speed of your system, and that makes it dangerous to compare this benchmark with anything else then previous results from your own system. To make this long story a bit shorter, the only objective thing you can conclude is "What background programs/services make my computer run slower/faster then before"

It's a pitty we lost a great deal of forum stuff in the past, because this a recurring discussion. We've had it a few times before.
digging into memory I remember once in the time you could still time the BSP fase of q3dm7sample.map in minutes :icon26: think it was round 2k2.

Hr.O
Page 1 of 2 All times are UTC - 8 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/