Dear All,
I’m trying to run HEPscoreSlim20 example with hep-score 1.0.0 from cvmfs unpacked [1], however, the cms-gen-sim-bmk benchmark returns empty result and benchmark reports failure. I tried the same config file on a CERN WN (running within singularity with image [1]), and on a CentOS 8 desktop (pip3 installed hepscore, singularity image [1]).
The run log is e.g. [2].
When a colleague ran the same benchmark on 24th January [3], the results were not empty, with the same config file for cms-gen-sim-bmk.
Would you please help me understand what changed, and how to make default HEPscoreSlim20 benchmark configuration to produce final result?
Thank you, cheers,
Jarka
[1] /cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/hep-benchmarks/hep-benchmark-suite/hep-benchmark-suite-cc7:prior
[2]
# hep-score --version
hep-score 1.0.0
# cat ~/hepscore/hepscore_slim.yml
hepscore_benchmark:
benchmarks:
cms-gen-sim-bmk:
ref_scores:
gen-sim: 0.726
weight: 1.0
version: v2.1
args:
threads: 4
events: 20
settings:
name: HEPscoreSlim20
reference_machine: "CPU Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz"
registry: dir:///cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/hep-benchmarks/hep-workloads
method: geometric_mean
repetitions: 1
retries: 1
scaling: 355
container_exec: singularity
# time hep-score -f ~/hepscore/hepscore_slim.yml ~/hepscore/hsresults
2021-03-20 23:01:48 hepscore [INFO] HEPscoreSlim20 Benchmark
2021-03-20 23:01:48 hepscore [INFO] Config Hash: 53d1dd3d60b6418ef53d764c66b4860751e701d36235146744ae0a925b34d13d
2021-03-20 23:01:48 hepscore [INFO] System: Linux ... 4.18.0-240.15.1.el8_3.x86_64 #1 SMP Mon Mar 1 17:16:16 UTC 2021 x86_64
2021-03-20 23:01:48 hepscore [INFO] Container Execution: singularity
2021-03-20 23:01:48 hepscore [INFO] Registry: dir:///cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/hep-benchmarks/hep-workloads
2021-03-20 23:01:48 hepscore [INFO] Output: /home/$USER/hepscore/hsresults/HEPscore_20Mar2021_230148
2021-03-20 23:01:48 hepscore [INFO] Date: Sat Mar 20 23:01:48 2021
2021-03-20 23:01:48 hepscore [INFO] Executing 1 run of cms-gen-sim-bmk
2021-03-20 23:01:48 hepscore [INFO] Starting run0
2021-03-21 00:29:26 hepscore [INFO]
2021-03-21 00:29:26 hepscore [WARNING] No results: fail
2021-03-21 00:29:26 hepscore [ERROR] BENCHMARK FAILURE
2021-03-21 00:29:26 hepscore [ERROR] Results = [].
active_config {'hepscore_benchmark': {'benchmarks': {'cms-gen-sim-bmk': {'ref_scores': {'gen-sim': 0.726}, 'weight': 1.0, 'version': 'v2.1', 'args': {'threads': 4, 'events': 20}}}, 'settings': {'name': 'HEPscoreSlim20', 'reference_machine': 'CPU Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz', 'registry': 'dir:///cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/hep-benchmarks/hep-workloads', 'method': 'geometric_mean', 'repetitions': 1, 'retries': 1, 'scaling': 355, 'container_exec': 'singularity'}}}
real 87m38.404s
user 50m37.648s
sys 7m46.464s
[3]
2021-01-24 10:42:59, hepbenchmarksuite.benchmarks:run_hepscore [INFO] Starting hepscore
HEPscoreSlim20 Benchmark
Config Hash: 53d1dd3d60b6418ef53d764c66b4860751e701d36235146744ae0a925b34d13d
System: Linux wn1906250.... 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64
Container Execution: singularity
Registry: dir:///cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/hep-benchmarks/hep-workloads
Output: ./suite_results/run_2021-01-24_1042/HEPSCORE
Date: Sun Jan 24 10:42:59 2021
2021-01-24 10:42:59, root:_run_benchmark [INFO] Executing 1 run of cms-gen-sim-bmk
2021-01-24 10:42:59, root:_run_benchmark [INFO] Creating singularity cache ./suite_results/run_2021-01-24_1042/HEPSCORE/scache
2021-01-24 10:42:59, root:_run_benchmark [INFO] Starting run0
2021-01-24 11:18:00, root:_container_rm [INFO] Removing temporary singularity cache ./suite_results/run_2021-01-24_1042/HEPSCORE/scache
2021-01-24 11:18:00, root:gen_score [INFO] Final result: 312.1515
2021-01-24 11:18:00, hepbenchmarksuite.hepbenchmarksuite:run [INFO] Completed hepscore with return code 0.8793
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [INFO] you should run this program as super-user for a complete output.
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] package lshw is not installed
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] b''
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] package ipmitool is not installed
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] b''
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] package dmidecode is not installed
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:__init__ [ERROR] b''
2021-01-24 11:18:00, hepbenchmarksuite.plugins.extractor:collect_sw [INFO] Collecting SW information.
2021-01-24 11:18:02, hepbenchmarksuite.plugins.extractor:collect_hw [INFO] Collecting HW information.
2021-01-24 11:18:02, hepbenchmarksuite.plugins.extractor:collect_cpu [INFO] Collecting CPU information.
2021-01-24 11:18:02, hepbenchmarksuite.plugins.extractor:collect_bios [INFO] Collecting BIOS information.
2021-01-24 11:18:02, hepbenchmarksuite.plugins.extractor:collect_system [INFO] Collecting system information.
2021-01-24 11:18:02, hepbenchmarksuite.plugins.extractor:collect_memory [INFO] Collecting system memory.
2021-01-24 11:18:03, hepbenchmarksuite.plugins.extractor:collect_storage [INFO] Collecting system storage.
2021-01-24 11:18:03, hepbenchmarksuite.hepbenchmarksuite:cleanup [INFO] Reading result file: ./suite_results/run_2021-01-24_1042/HEPSCORE/hepscore_result.json
2021-01-24 11:18:03, hepbenchmarksuite.hepbenchmarksuite:cleanup [INFO] Successfully completed all requested benchmarks
=========================================================
BENCHMARK RESULTS FOR wn1906250.....
=========================================================
Suite start: 2021-01-24T10:42:54Z
Suite end: 2021-01-24T11:18:00Z
Machine CPU Model: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
HEPSCORE Benchmark = 312.15 over benchmarks dict_keys(['cms-gen-sim-bmk'])