Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/77054f3946161ea43b67a623f33d4647e5c850fc (+ 9622 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/91/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0091 (llvmorg-18-init-04967-g77054f394616) 2023-07-27T11:18:01 0:00:00
Previous 0060 (no-change) 2023-06-14T08:48:13 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 10 52
Performance Improvements 44 45
Unchanged Tests 822 779
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
507.cactuBSSN_r 2.50% 11339.000 11623.000 - 2.92% -
505.mcf_r 2.29% 14226.000 14552.000 - 2.86% -
544.nab_r 1.91% 14203.000 14474.000 - 1.10% -
538.imagick_r 1.88% 13863.000 14124.000 - 1.73% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
505.mcf_r 2.90% 69.000 71.000 - -2.74% -
505.mcf_r (avg) 2.90% 69.000 71.000 - -2.74% -
544.nab_r 1.96% 51.000 52.000 - 26.83% -
544.nab_r (avg) 1.96% 51.000 52.000 - 26.83% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -19.05% 21.000 17.000 - -22.73% -
nb_execution_successful -19.05% 21.000 17.000 - -22.73% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
526.blender_r -6.53% 12513.000 11696.000 - -5.92% -
525.x264_r -3.92% 6448.000 6195.000 - -17.29% -
541.leela_r -2.94% 13661.000 13259.000 - -2.30% -
511.povray_r -1.82% 20729.000 20351.000 - -5.55% -
508.namd_r -1.14% 9562.000 9453.000 - -1.27% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
541.leela_r (max) -15.56% 45.000 38.000 - -48.65% -
508.namd_r (max) -13.04% 23.000 20.000 - -81.48% -
541.leela_r (max) (percentage) -12.12% 0.330 0.290 - -47.27% -
508.namd_r (max) (percentage) -12.00% 0.250 0.220 - -80.53% -
525.x264_r (max) -11.11% 27.000 24.000 - 118.18% -
511.povray_r (max) -10.77% 130.000 116.000 - -1.69% -
526.blender_r (max) -10.31% 524.000 470.000 - 968.18% -
511.povray_r (max) (percentage) -9.52% 0.630 0.570 - 3.64% -
525.x264_r -7.14% 14.000 13.000 - 116.67% -
525.x264_r (avg) -7.14% 14.000 13.000 - 116.67% -
525.x264_r (max) (percentage) -7.14% 0.420 0.390 - 160.00% -
526.blender_r -6.54% 107.000 100.000 - 143.90% -
526.blender_r (avg) -6.54% 107.000 100.000 - 143.90% -
531.deepsjeng_r (max) (percentage) -6.35% 0.630 0.590 - -23.38% -
510.parest_r (max) -6.25% 48.000 45.000 - 4.65% -
505.mcf_r (max) (percentage) -6.25% 0.960 0.900 - 32.35% -
548.exchange2_r (max) (percentage) -5.71% 0.350 0.330 - 50.00% -
531.deepsjeng_r (max) -5.56% 54.000 51.000 - -23.88% -
557.xz_r (max) (percentage) -5.26% 0.190 0.180 - -5.26% -
538.imagick_r (max) (percentage) -4.84% 0.620 0.590 - 40.48% -
557.xz_r (max) -4.76% 21.000 20.000 - -4.76% -
520.omnetpp_r (max) (percentage) -4.55% 1.540 1.470 - -41.20% -
520.omnetpp_r (max) -4.42% 226.000 216.000 - -40.98% -
505.mcf_r (max) -4.41% 136.000 130.000 - 35.42% -
523.xalancbmk_r (max) -4.11% 73.000 70.000 - 34.62% -
526.blender_r (max) (percentage) -4.06% 4.190 4.020 - 1016.67% -
523.xalancbmk_r (max) (percentage) -3.70% 0.540 0.520 - 36.84% -
554.roms_r (max) (percentage) -3.57% 0.280 0.270 - 285.71% -
503.bwaves_r (max) (percentage) -3.13% 0.320 0.310 - 244.44% -
519.lbm_r -2.86% 35.000 34.000 - 17.24% -
519.lbm_r (avg) -2.86% 35.000 34.000 - 17.24% -
500.perlbench_r (max) (percentage) -2.78% 2.520 2.450 - 84.21% -
544.nab_r (max) (percentage) -2.38% 0.420 0.410 - 57.69% -
538.imagick_r (max) -2.35% 85.000 83.000 - 43.10% -
541.leela_r -2.33% 43.000 42.000 - -28.81% -
541.leela_r (avg) -2.33% 43.000 42.000 - -28.81% -
500.perlbench_r (max) -1.97% 406.000 398.000 - 86.85% -
511.povray_r -1.35% 74.000 73.000 - -5.19% -
511.povray_r (avg) -1.35% 74.000 73.000 - -5.19% -

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 2.92% 11293.000 11623.000 - 2.50% -
505.mcf_r 2.86% 14148.000 14552.000 - 2.29% -
538.imagick_r 1.73% 13884.000 14124.000 - 1.88% -
500.perlbench_r 1.22% 16051.000 16247.000 - 0.78% -
544.nab_r 1.10% 14316.000 14474.000 - 1.91% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
526.blender_r (max) (percentage) 1016.67% 0.360 4.020 - -4.06% -
526.blender_r (max) 968.18% 44.000 470.000 - -10.31% -
554.roms_r (max) (percentage) 285.71% 0.070 0.270 - -3.57% -
503.bwaves_r (max) (percentage) 244.44% 0.090 0.310 - -3.13% -
526.blender_r (avg) (percentage) 160.61% 0.330 0.860 - 0.00% -
525.x264_r (max) (percentage) 160.00% 0.150 0.390 - -7.14% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
525.x264_r (avg) (percentage) 144.44% 0.090 0.220 - 0.00% -
526.blender_r 143.90% 41.000 100.000 - -6.54% -
526.blender_r (avg) 143.90% 41.000 100.000 - -6.54% -
525.x264_r (max) 118.18% 11.000 24.000 - -11.11% -
525.x264_r 116.67% 6.000 13.000 - -7.14% -
525.x264_r (avg) 116.67% 6.000 13.000 - -7.14% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
500.perlbench_r (max) 86.85% 213.000 398.000 - -1.97% -
500.perlbench_r (max) (percentage) 84.21% 1.330 2.450 - -2.78% -
544.nab_r (max) 59.46% 37.000 59.000 - 0.00% -
544.nab_r (max) (percentage) 57.69% 0.260 0.410 - -2.38% -
548.exchange2_r (max) (percentage) 50.00% 0.220 0.330 - -5.71% -
538.imagick_r (max) 43.10% 58.000 83.000 - -2.35% -
531.deepsjeng_r (avg) (percentage) 41.46% 0.410 0.580 - 0.00% -
538.imagick_r (max) (percentage) 40.48% 0.420 0.590 - -4.84% -
531.deepsjeng_r 38.89% 36.000 50.000 - 0.00% -
531.deepsjeng_r (avg) 38.89% 36.000 50.000 - 0.00% -
523.xalancbmk_r (max) (percentage) 36.84% 0.380 0.520 - -3.70% -
505.mcf_r (max) 35.42% 96.000 130.000 - -4.41% -
523.xalancbmk_r (max) 34.62% 52.000 70.000 - -4.11% -
505.mcf_r (max) (percentage) 32.35% 0.680 0.900 - -6.25% -
557.xz_r (avg) (percentage) 30.00% 0.100 0.130 - 0.00% -
557.xz_r 27.27% 11.000 14.000 - 0.00% -
557.xz_r (avg) 27.27% 11.000 14.000 - 0.00% -
544.nab_r 26.83% 41.000 52.000 - 1.96% -
544.nab_r (avg) 26.83% 41.000 52.000 - 1.96% -
544.nab_r (avg) (percentage) 24.14% 0.290 0.360 - 0.00% -
519.lbm_r (avg) (percentage) 17.65% 0.170 0.200 - 0.00% -
519.lbm_r 17.24% 29.000 34.000 - -2.86% -
519.lbm_r (avg) 17.24% 29.000 34.000 - -2.86% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
510.parest_r (max) (percentage) 5.26% 0.190 0.200 - -4.76% -
510.parest_r (max) 4.65% 43.000 45.000 - -6.25% -
511.povray_r (max) (percentage) 3.64% 0.550 0.570 - -9.52% -
500.perlbench_r 2.05% 146.000 149.000 - 0.68% -
500.perlbench_r (avg) 2.05% 146.000 149.000 - 0.68% -
500.perlbench_r (avg) (percentage) 1.10% 0.910 0.920 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - -19.05% -
nb_execution_successful -22.73% 22.000 17.000 - -19.05% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.16% 624815.000 617567.000 - -0.51% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -17.29% 7490.000 6195.000 - -3.92% -
526.blender_r -5.92% 12432.000 11696.000 - -6.53% -
511.povray_r -5.55% 21547.000 20351.000 - -1.82% -
541.leela_r -2.30% 13571.000 13259.000 - -2.94% -
531.deepsjeng_r -1.60% 8789.000 8648.000 - -0.38% -
523.xalancbmk_r -1.34% 13812.000 13627.000 - -0.49% -
508.namd_r -1.27% 9575.000 9453.000 - -1.14% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
508.namd_r (max) -81.48% 108.000 20.000 - -13.04% -
508.namd_r (max) (percentage) -80.53% 1.130 0.220 - -12.00% -
549.fotonik3d_r (max) (percentage) -60.33% 1.210 0.480 - 0.00% -
508.namd_r -50.82% 61.000 30.000 - 0.00% -
508.namd_r (avg) -50.82% 61.000 30.000 - 0.00% -
508.namd_r (avg) (percentage) -50.00% 0.640 0.320 - 0.00% -
541.leela_r (max) -48.65% 74.000 38.000 - -15.56% -
519.lbm_r (max) -48.28% 29.000 15.000 - 0.00% -
541.leela_r (max) (percentage) -47.27% 0.550 0.290 - -12.12% -
519.lbm_r (max) (percentage) -47.06% 0.170 0.090 - 0.00% -
520.omnetpp_r (max) (percentage) -41.20% 2.500 1.470 - -4.55% -
520.omnetpp_r (max) -40.98% 366.000 216.000 - -4.42% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
541.leela_r -28.81% 59.000 42.000 - -2.33% -
541.leela_r (avg) -28.81% 59.000 42.000 - -2.33% -
541.leela_r (avg) (percentage) -27.27% 0.440 0.320 - 0.00% -
531.deepsjeng_r (max) -23.88% 67.000 51.000 - -5.56% -
531.deepsjeng_r (max) (percentage) -23.38% 0.770 0.590 - -6.35% -
538.imagick_r (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
538.imagick_r -19.51% 41.000 33.000 - 0.00% -
538.imagick_r (avg) -19.51% 41.000 33.000 - 0.00% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
523.xalancbmk_r -6.06% 33.000 31.000 - 0.00% -
523.xalancbmk_r (avg) -6.06% 33.000 31.000 - 0.00% -
510.parest_r -5.88% 34.000 32.000 - 0.00% -
510.parest_r (avg) -5.88% 34.000 32.000 - 0.00% -
505.mcf_r (avg) (percentage) -5.77% 0.520 0.490 - 0.00% -
557.xz_r (max) (percentage) -5.26% 0.190 0.180 - -5.26% -
511.povray_r -5.19% 77.000 73.000 - -1.35% -
511.povray_r (avg) -5.19% 77.000 73.000 - -1.35% -
557.xz_r (max) -4.76% 21.000 20.000 - -4.76% -
505.mcf_r -2.74% 73.000 71.000 - 2.90% -
505.mcf_r (avg) -2.74% 73.000 71.000 - 2.90% -
520.omnetpp_r (avg) (percentage) -2.17% 0.920 0.900 - 0.00% -
511.povray_r (max) -1.69% 118.000 116.000 - -10.77% -
520.omnetpp_r -1.49% 134.000 132.000 - 0.00% -
520.omnetpp_r (avg) -1.49% 134.000 132.000 - 0.00% -

Report Time: 0.36s
Execution (seconds)
Name Current %
500.perlbench_r 16247.000 -
502.gcc_r 10984.000 -
505.mcf_r 14552.000 2.29%
507.cactuBSSN_r 11623.000 2.50%
508.namd_r 9453.000 -1.14%
510.parest_r 22926.000 -
511.povray_r 20351.000 -1.82%
519.lbm_r 17492.000 -
520.omnetpp_r 14750.000 -
523.xalancbmk_r 13627.000 -
525.x264_r 6195.000 -3.92%
526.blender_r 11696.000 -6.53%
531.deepsjeng_r 8648.000 -
538.imagick_r 14124.000 1.88%
541.leela_r 13259.000 -2.94%
544.nab_r 14474.000 1.91%
557.xz_r 11491.000 -
Geometric Mean 13041.731 -
Code Size (bytes)
Name Current %
500.perlbench_r 2145487.000 -
502.gcc_r 8927360.000 -
505.mcf_r 27163.000 -
507.cactuBSSN_r 3682091.000 -
508.namd_r 617567.000 -
510.parest_r 8668264.000 -
511.povray_r 916446.000 -
519.lbm_r 13319.000 -
520.omnetpp_r 2153593.000 -
523.xalancbmk_r 4724615.000 -
525.x264_r 497770.000 -
526.blender_r 13663226.000 -
531.deepsjeng_r 80950.000 -
538.imagick_r 1700649.000 -
541.leela_r 145423.000 -
544.nab_r 190920.000 -
557.xz_r 176888.000 -
Geometric Mean 739068.760 -
Score
Name Current %
nb_compile_successful 17.000 -19.05%
nb_execution_successful 17.000 -19.05%
nb_total_tests 23.000 -
Geometric Mean 18.802 -13.14%
sample variability
Name Current %
500.perlbench_r 149.000 -
500.perlbench_r (avg) 149.000 -
500.perlbench_r (avg) (percentage) 0.920 -
500.perlbench_r (max) 398.000 -1.97%
500.perlbench_r (max) (percentage) 2.450 -2.78%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.310 -3.13%
505.mcf_r 71.000 2.90%
505.mcf_r (avg) 71.000 2.90%
505.mcf_r (avg) (percentage) 0.490 -
505.mcf_r (max) 130.000 -4.41%
505.mcf_r (max) (percentage) 0.900 -6.25%
508.namd_r 30.000 -
508.namd_r (avg) 30.000 -
508.namd_r (avg) (percentage) 0.320 -
508.namd_r (max) 20.000 -13.04%
508.namd_r (max) (percentage) 0.220 -12.00%
510.parest_r 32.000 -
510.parest_r (avg) 32.000 -
510.parest_r (avg) (percentage) 0.140 -
510.parest_r (max) 45.000 -6.25%
510.parest_r (max) (percentage) 0.200 -
511.povray_r 73.000 -1.35%
511.povray_r (avg) 73.000 -1.35%
511.povray_r (avg) (percentage) 0.360 -
511.povray_r (max) 116.000 -10.77%
511.povray_r (max) (percentage) 0.570 -9.52%
519.lbm_r 34.000 -2.86%
519.lbm_r (avg) 34.000 -2.86%
519.lbm_r (avg) (percentage) 0.200 -
519.lbm_r (max) 15.000 -
519.lbm_r (max) (percentage) 0.090 -
520.omnetpp_r 132.000 -
520.omnetpp_r (avg) 132.000 -
520.omnetpp_r (avg) (percentage) 0.900 -
520.omnetpp_r (max) 216.000 -4.42%
520.omnetpp_r (max) (percentage) 1.470 -4.55%
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 31.000 -
523.xalancbmk_r (avg) 31.000 -
523.xalancbmk_r (avg) (percentage) 0.230 -
523.xalancbmk_r (max) 70.000 -4.11%
523.xalancbmk_r (max) (percentage) 0.520 -3.70%
525.x264_r 13.000 -7.14%
525.x264_r (avg) 13.000 -7.14%
525.x264_r (avg) (percentage) 0.220 -
525.x264_r (max) 24.000 -11.11%
525.x264_r (max) (percentage) 0.390 -7.14%
526.blender_r 100.000 -6.54%
526.blender_r (avg) 100.000 -6.54%
526.blender_r (avg) (percentage) 0.860 -
526.blender_r (max) 470.000 -10.31%
526.blender_r (max) (percentage) 4.020 -4.06%
531.deepsjeng_r 50.000 -
531.deepsjeng_r (avg) 50.000 -
531.deepsjeng_r (avg) (percentage) 0.580 -
531.deepsjeng_r (max) 51.000 -5.56%
531.deepsjeng_r (max) (percentage) 0.590 -6.35%
538.imagick_r 33.000 -
538.imagick_r (avg) 33.000 -
538.imagick_r (avg) (percentage) 0.240 -
538.imagick_r (max) 83.000 -2.35%
538.imagick_r (max) (percentage) 0.590 -4.84%
541.leela_r 42.000 -2.33%
541.leela_r (avg) 42.000 -2.33%
541.leela_r (avg) (percentage) 0.320 -
541.leela_r (max) 38.000 -15.56%
541.leela_r (max) (percentage) 0.290 -12.12%
544.nab_r 52.000 1.96%
544.nab_r (avg) 52.000 1.96%
544.nab_r (avg) (percentage) 0.360 -
544.nab_r (max) 59.000 -
544.nab_r (max) (percentage) 0.410 -2.38%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.330 -5.71%
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.480 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.270 -3.57%
557.xz_r 14.000 -
557.xz_r (avg) 14.000 -
557.xz_r (avg) (percentage) 0.130 -
557.xz_r (max) 20.000 -4.76%
557.xz_r (max) (percentage) 0.180 -5.26%
Geometric Mean 5.261 -2.69%