SPEC Benchmarking Workshop

SPEC Benchmarking Workshop, Paderborn, July 16, 1996

Am 16 Juli 1996 fand das turnusmäßige Meeting der "Open Systems Group" von SPEC (Standard Performance Evaluation Corporation) in Paderborn mit Siemens Nixdorf Informationssysteme (SNI) als Gastgeber statt. Am ersten Tag des dreitägigen Treffens gab ein Workshop zum Thema SPEC/Benchmarking den SPEC-Vertretern (fast alle aus den USA) und europäischen Fachleuten im Gebiet Leistungsmessung (z.B. aus der GI/ITG-Interessengruppe "Messung, Modellierung und Bewertung von Rechensystemen") Gelegenheit zu Informations- und Meinungsaustausch.

Der folgende Abschnitt enthält das Workshop-Programm. Ein ausführlicherer Bericht findet sich in der Zeitschrift "iX-Magazin" Nr. 9 / 96 ab Seite 24.

Nähere Informationen:

Dr. Reinhold P. Weicker, Siemens Nixdorf Informationssysteme AG, OEC HES PM4
Address: D-33094 Paderborn, Germany (for postal mail)
Phone: +49-5251-8-12766 Fax: +49-5251-8-15104
E-Mail: weicker.pad@sni.de

9:15   Jeff Garelick, SPEC / IBM, Austin/TX: The SPEC CPU95 tool environment
          For better ease of use and better comparability of results, SPEC has developed a set of tools for compilation and execution of the CPU95 benchmarks; use
          of these tools is required for official result publications. The philosophy of the tool set and experiences with the tools are explained.

10:00 Ralph Hülsenbusch and Thomas Sieverding, iX Magazin, Hannover:
          Experiences with the SPEC95 benchmarks in product reviews
          The German iX magazine regularly published product reviews which include the results of benchmark runs conducted by iX. The talk reports about problems
          discovered and insights gained in such benchmark runs.

10:45 Break

11:15 Michael Ehrig, HP, Böblingen/Germany:
          Experiences with the SPEC SFS Benchmark
          For a series of product reviews to be published in iX Magazin,Michael Ehrig and a colleague (then at Karlsruhe University) made tests of NFS file server
          systems with the SPEC SFS benchmark (LADDIS). Systems from five manufacturers were tested.

11:45 Lunch

12:45 Prof. Werner Dirlewanger and Peter Tietsch, Kassel University:
          Experiences with a framework for the SPEC CPU benchmarks based on the DIN benchmarking standard
          The German DIN standard 66237 (also submitted to ISO) describes ageneral framework for benchmarking. In a project at Kassel University, the SPEC95
          CPU benchmarks are executed in a DIN-conforming way, resulting in a heterogenuous capacity method.

13:30 Carsten Mund, Technical University of Chemnitz:
          Influences of background loads on the SPEC95 benchmarks
          In work for a diplom thesis, the SPEC95 benchmarks were intentionally measured in a "non-benchmarking" way, with the background load that is normally
          present in a university computing center. The talk reports about the experiences gained in this study.

14:00 Break

14:30 Dr. Ran Giladi, Ben-Gurion University of the Negev, Israel:
          SPEC as a Performance Evaluation Measure
          Together with Niv Ahituv, Dr. Giladi has published critical remarks about the SPEC (1992) CPU benchmarks in "Computer" (Aug, 1995). These views
          should be discussed in the light of the question what should be done similarly or differently for the next CPU suite (SPEC98).

15:15 Jens Simon, Paderborn University:
          Accurate Performance Prediction for Massively Parallel Systems and ist Applications
          The Paderborn Center for Parallel Computing is a research institute affiliated with Paderborn University, it concentrates on massively parallel computers.

15:45 Bodo Parady, SPEC / Sun, Mountain View/CA, USA:
          The ISCA/ACM/IEEE/SPEC Benchmark Award Program
          Benchmarks traditionally come from public domain codes. It has been suggested that professional societies, together with SPEC, find newways of soliciting
          good programs as benchmark candidates.

16:30 Peer Überholz, Wuppertal University:
          SU3WU Lattice Gauge Model, a possible SPEC98 benchmark candidate
          A program from the area of quantum chromodynanmics that could be a first candidate for the SPEC98 floating-point benchmarks

17:00 Alex Carlton, SPEC / HP, Cupertino/CA, USA:
          SPEC's new Web server benchmark
          Based on its successful SFS (NFS file server) benchmark, SPEC is currently finishing work on a Web server benchmark. Alex Carlton, the release manager
          of this benchmark, describes its philosophy and implementation.