Welcome to www.sunxperf.org

Xperf -- Measuring, Collecting and Publishing 2D Graphics Performance for Sun Solaris Based Systems.

Jim Balson
balson@comcast.net
Last updated: December 19th 2004

What is X11perf?

X11perf is an X Windows based application designed to test the performance of the 2D X WIndows server operations and by extension, the underlying graphics hardware. It is distributed by http://www.x.org as part of the X Window System. It takes ~4 hours to run on my Ultra-60 2x300 PGX64 based system. It runs an exhaustive set of tests and comes up with a final performance number called an Xmark. The higher the Xmark, the faster the server/hardware.

Overview - What is Xperf?

Xperf itself is a collection of shell scripts. One of the shell scripts, aptly named xperf, is distributed as part of the xperf package. it's job is to collect data about the host computer. The data are things that can affect graphics performance, like CPU speed, screen resolution, Bit-per-Pixel, etc (there are more details below). It then runs x11perf (which s part of the xperf distribution in both source and binary form) to gather the hard core graphics performance numbers. It then rolls this data up into a filename such that the back end scripts at xperf@comcast.net can parse it and create a sorted table from it all.

Back End File Processing

The back end is simply an account at my ISP which is read periodically from my Sun Ultra-60. I use fetchmail to read the incoming mail every 5 minutes, and another script that checks the inbox for new mail 30 minutes. When new mail has arrived, it saves the first message to a file and then looks for the following string in the file:

x11perf-filename

If it does not find this string in the email, it thinks the mail is junk mail and deletes it. Otherwise, it runs yet another script to grab the Xmark from the file and prepends the Xmark to the filename and moves the file to the results directory. It then runs another script to create the sorted HTML table based on the Xmarks from all of the files in the results directory. And then yet another to upload the HTML files to the web server.

Motivation

My motivation for doing this is because I have recently changed my graphics card resolution form 1280x1024 24bpp to 1600x1200 24bpp and find the graphics performance a bit slower. I want to stay at 1600x1200 as I like the added granularity. So I'd like to possibly upgrade the graphics card, but I can't find any x11perf performance charts for the cards I am interested in. Without x11perf output, I can't make an intelligent decision about whether a faster card will really help me or not. So I decided to collect and publish my own performance data.

Also, would faster CPU's actually help me? With 2x300 CPU's in an Ultra 60, I might be better off upgrading the CPU's to 450's. But I don't have any data on that either in terms of how much (if any) of a performance boost to expect and would it be worth the money? Would the money be better spent on a faster graphics card?

So basically, I am dataless. I just don't have any data to go on, and so this xperf package is an attempt to obtain and publish the data one needs to make these types of decisions. I can't be the only person trying to figure this out right? I plan on keeping this page up indefinitely as I think it could be beneficial to the entire Solaris community as a whole.

I consider it only a partial pay back to the Solaris community for all of the assistance that has been given to me throughout the years.

The Xperf Package

This package consists of the following:

The xperf shell script to gather information about your system CPU, Graphics Card, current screen resolution and BPP (Bits per Pixel).
x11perf source code for you to compile
x11perf binaries for Solaris (SPARC and INTEL) Compiled with the Sun Compilers

The Xperf Shell Script

CPU speed can affect graphics performance. The more a particular graphics operation is not optimized in hardware, the more work the CPU must do to complete that particular operation.

This script does the following:

Collects information about the CPU type (SPARC vs INTEL) and generation (UltraSPARC I vs II vs III)
Collects information about the MHZ of the CPU's.
Collects information about the type of Graphics Card
Collects information about the screen resolution and Bits per Pixel.

It rolls all of this information into the name of the file where the x11perf results are stored. For instance, on my system, a Sun Ultra 60 2x300 with a PGX64, the file name is as follows:

PGX64:1600x1200:24bpp:SUNW,Ultra-60-5.8:UltraSPARC-II:2x296MHz:aspen

Note: that each field separated by a colon is a column in the x11perf performance tables.

Once it collects this information, it will then run xdpyinfo and dump the output into this file. Then it will run x11perf and dump that output into this file as well. And then, it will dump the string x11perf-filename into the file followed by the actual filename. Without this string in the file, the backend scripts that read and process the email for xperf@comcast.net will think it's junk email and file under /dev/null.

An example filename is as follows:

x11perf-filename PGX64:1600x1200:24bpp:SUNW,Ultra-60-5.8:UltraSPARC-II:2x296MHz:aspen

The xperf script uses a few different methods to collect this data.

Data about the Graphics Card must be entered manually. Please try to keep card descriptions to under 25 characters as this field is truncated after the 25th character.
Data about the CPU(s) and System type is had from running prtdiag -v
Data about the Operating System is had from uname.
Data about the screen resolution and BPP is had from running the xdpyinfo program.

When the script is complete (after running x11perf), you will be asked to email the file to xperf@comcast.net

It will typically take some number of hours to complete a full run of x11perf. On my system, a 2x300 Sun Ultra 60 with the PGX64 it takes a little under 4 hours to run depending on the screen resolution and BPP. In fact, I believe a faster graphics and faster CPU's will probably run about the same amount of time as I think x11perf synchronizes it's tests so that each test takes around the same amount of time. But we'll see if this is true.

X11perf sources

The x11perf source has been culled from the X11R6.6 source tree which is available from http://www.x.org. They are available in case you feel you need to build your own binaries. Please use Sun compilers when building X11perf as the GCC does not generate very efficient code on SPARC. It's better on Intel, but my experience is, and I have performance numbers to prove it, is that the Sun Compilers generate more efficient code on Solaris INTEL as well. But, your mileage may vary ...

X11perf Binaries

The x11perf binaries included in the distribution were built on Solaris 8 SPARC and Solaris 8 INTEL. They are included for folks who do not have the Sun Compilers. The binaries were compiled with the following Sun compilers:

SPARC: cc: Forte Developer 7 C 5.4 2002/03/09
INTEL: cc: Sun WorkShop 6 update 2 C 5.3 Patch 111680-07 2002/11/12

With the following optimization switches:

$ cc -fast ...

In fact, I have modified the Makefile for the x11perf sources to compile them with this flag(s).

The check sum on the x11perf binaries are as follows:

pine: /u3/users/xperf/bin > sum x11perf*
22053 287 x11perf.i386
13205 303 x11perf.sparc

Running the script.

Do not run the x11perf binary by itself. Run the xperf shell script. One of the most important things the script does is it dumps the output from x11perf to a file that has a name that the scripts on the receiving end can make sense of. The name that it generates is dependent on the hardware in the system it is run on. I do not need, and in fact can't process any x11perf data without it having been run from the xperf script.

Do the following:

$ ./xperf

Then walk away and come back in 4 hours. X11perf can take some time to complete.

Runtime Notes

If you use Gnome as your desktop, then you'll need top disable the screen saver. There is something Gnome does when it goes into screen saver mode that causes the Xserver to 'think' that it doesn't really need to draw anything and the results are skewed in such a way that a PGX64 card gets a fiinal Xmark number of 70 or more (I've gotten an Xmark of 73 and 80-something twice, when an Xmark of 5-10 is more accurate). If you can, either disable the screen saver of run some other window manager during the test.

Platform Support

The xperf script has been tested on the following machines:

Solaris 8/9 and 10 SPARC.
Solaris 8, 9 and 10 x86 (Intel and AMD)

Download

A new release of xperf to fix a problem on some systems of obtaining the correct Megahertz and CPU type.

-rw-r--r-- 1 xperf staff 179780 Dec 19 22:02 xperf-0.20.tar.gz

Results

If you see results for your graphics card in the table and it was run on the same system as yours, you might be tempted to not run x11perf and submit any results. Please don't think this. I want duplicate results as this helps to verify previously submitted results.

When the xperf script is done, please email the resulting data file to the following email address: xperf@comcast.net

When viewing the performance table and seeing the results for the graphics card in your system keep the following in mind:

I need duplicate results of card 'xxx' and system'yyy' to help verify previously submitted results.
I need performance results for card 'xxx' at different screen resolutions and different BBP (Bits per Pixel). So just because you see your card and system in the table doesn't mean that the final Xmark number matches the Xmark that you will get. It depends on screen resolution and BPP as much as CPU speed.
I need data for card 'xxx' on varying machines to see how much CPU speed and system architecture affects graphics performance.

So please, take the time to run the tests. It won't take but a few minutes of your time downloading, starting the script and then mailingthe results file. Remember you can run the tests overnight.

When mailing results to xperf@comcast.net, mail 1 result file per email. Mailing more than 1 results file per email is an undefined operation as to what will happen to any of the results contained within.

Notes:

You should try to always stay on top of the latest patches from Sun that could affect the performance of your graphics hardware. For instance, there exists a patch (114537-17 for Solaris 8) that can increase the performance of the XVR-100 from a Xmark of 14.4020 to 25.3049 on a 2x1200 MHZ CPU Sun Fire 280R.
On this aformentioned Sun Fire 280R 2x1200 US-III+, the xperf script could not obtain the MHZ of the CPU's because xprtdiag did not print it out. I don't know if this is a xprtdiag anomaly on this particular class of machine. I don't have any 280R's here to play with to debug this problem.

The X11perf Performance Table

This table is automatically generated. it will be updated frequently.

Raw X11perf Data

The raw data fiels snt in and also uploaded to the xperf web site. I strip the email headers from them before uploading. You can now see the detailed x11perf output from various graphics cards.

Updating the Performance Table.

This is a completely hands off operation. Everything is automated by shell scripts and cron.

The performance table is updated every time a new set of email messages arrives at xperf@comcast.net. Email is checked every 30 minutes, and if there is valid email that contains x11perf output, then the scripts are run to generate a new performance table. So when you send valid data in to be processed, you should see it in the performance table within 60 minutes or so. If not, see the Problems/Comments Section.

Problems or Comments

Direct all Questions, comments or problems to balson@comcast.net

DO NOT send these type of requests to xperf@comcast.net as email to that account is not read by a human. email to this account other than x11perf results generated by xperf is promptly filed away in /dev/null.