How to select a long-lived stable version of hep-benchmark-suite


I was running HS basically like this:

git clone
./hep-benchmark-suite/examples/hepscore/ -s "Site" -d $WORK_DIR

However instead of using the master branch I need to ensure the execution is as deterministic and reproducible as possible over time, with only minor bug fixes but no other major changes in behaviour.
Which branch or tag should I use? v2.2 ?


I tried the v2.2 branch but I noticed this:

./hep-benchmark-suite/examples/hepscore/ -s "bench" -d $WORK_DIR/hepscore
Setting site to bench
Setting the working directory to /mnt/bench/work/hepscore
Running script: ./hep-benchmark-suite/examples/hepscore/ - version: 1.2.1
Creating the WORKDIR /mnt/bench/work/hepscore
Latest suite release selected: v2.2.
Requirement already satisfied: pip in ./env_bmk/lib/python3.9/site-packages (21.2.3)
Collecting pip
  Using cached pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.3
    Uninstalling pip-21.2.3:
      Successfully uninstalled pip-21.2.3
Successfully installed pip-24.0
Collecting git+
  Cloning (to revision v1.5) to /mnt/bench/work/tmpdir/pip-req-build-rdokkvu1
  Running command git clone --filter=blob:none --quiet /mnt/bench/work/tmpdir/pip-req-build-rdokkvu1
  Running command git checkout -q f6169255d1ee03a8637366b6a7f1a680205c5da2
  Resolved to commit f6169255d1ee03a8637366b6a7f1a680205c5da2

Seems to work fine and eventually complete, but then the process seems to hang indefinitely at the end, until I Ctrl-C. This would significantly interfere with our automated benchmark system.

2024-05-08 23:31:47, hepscore.hepscore:_run_benchmark [INFO] Executing 3 runs of alice-digi-reco-core-run3-ma-bmk [v2.1_x86_64]
2024-05-08 23:31:47, hepscore.hepscore:_run_benchmark [INFO] Starting run0
2024-05-08 23:53:29, hepscore.hepscore:_run_benchmark [INFO] Starting run1
2024-05-09 00:11:21, hepscore.hepscore:_run_benchmark [INFO] Starting run2
2024-05-09 00:28:17, hepscore.hepscore:_run_benchmark [INFO] 
2024-05-09 00:28:17, hepscore.hepscore:gen_score [INFO] Final result: 239.0693
2024-05-09 00:28:17, hepbenchmarksuite.hepbenchmarksuite:run [INFO] Completed hepscore with return code 0
2024-05-09 00:28:17, hepbenchmarksuite.plugins.extractor:__init__ [INFO] you should run this program as super-user for a complete output.
2024-05-09 00:28:17, hepbenchmarksuite.plugins.extractor:collect_sw [INFO] Collecting SW information.
2024-05-09 00:28:17, hepbenchmarksuite.plugins.extractor:collect_hw [INFO] Collecting HW information.
2024-05-09 00:28:17, hepbenchmarksuite.plugins.extractor:collect_cpu [INFO] Collecting CPU information.

^CTraceback (most recent call last):
  File "/mnt/bench/work/hepscore/env_bmk/bin/bmkrun", line 285, in <module>
  File "/mnt/bench/work/hepscore/env_bmk/bin/bmkrun", line 253, in main
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 57, in start
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 99, in run
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 107, in check_lock
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 68, in run
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 113, in cleanup
    self._result = utils.prepare_metadata(self._config_full, self._extra)
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 288, in prepare_metadata
    'HW': hw_data.collect_hw(),
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/plugins/", line 347, in collect_hw
    "CPU"    : self.collect_cpu(),
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/plugins/", line 111, in collect_cpu
    'Power_Policy': self.exec_cmd(f"cat {scaling_governors} | sort | uniq"),
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/plugins/", line 70, in exec_cmd
    reply, _ = utils.exec_cmd(cmd_str)
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 141, in exec_cmd
    return_code, reply, error = run_piped_commands(cmd_str, env)
  File "/mnt/bench/work/hepscore/env_bmk/lib64/python3.9/site-packages/hepbenchmarksuite/", line 175, in run_piped_commands
    output =, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True, env=env)
  File "/usr/lib64/python3.9/", line 507, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib64/python3.9/", line 1134, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib64/python3.9/", line 1995, in _communicate
    ready =
  File "/usr/lib64/python3.9/", line 416, in select
    fd_event_list = self._selector.poll(timeout)

Dear Ryan,

we will fix and release a new tag.
Meanwhile, may I ask you to open a GGUS ticket using the instructions at How to Run HEPScore23 Benchmark