Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/fdb1a891b64c27522a2386a8025f8ad5c7e02bfb (+ 395 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/52/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0052 (llvmorg-17-init-13800-gfdb1a891b64c) 2023-06-01T20:44:58 0:00:00
Previous 0051 (llvmorg-17-init-13405-g0a168131b4f4) 2023-05-27T17:52:18 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 17 51
Performance Improvements 38 61
Added Tests 4 0
Unchanged Tests 817 764
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
511.povray_r 3.00% 20145.000 20750.000 - -3.70% -
505.mcf_r 2.63% 13964.000 14331.000 - 1.29% -
541.leela_r 1.52% 13393.000 13597.000 - 0.19% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) (percentage) 58.18% 0.550 0.870 - 163.64% -
520.omnetpp_r (unexplained diff vs previous) 56.79% 81.000 127.000 - 164.58% -
508.namd_r (max) 17.39% 23.000 27.000 - -75.00% -
508.namd_r (max) (percentage) 16.00% 0.250 0.290 - -74.34% -
520.omnetpp_r (max) (percentage) 7.50% 0.800 0.860 - -65.60% -
520.omnetpp_r (max) 5.93% 118.000 125.000 - -65.85% -
511.povray_r 3.66% 82.000 85.000 - 10.39% -
511.povray_r (avg) 3.66% 82.000 85.000 - 10.39% -
541.leela_r 2.44% 41.000 42.000 - -28.81% -
541.leela_r (avg) 2.44% 41.000 42.000 - -28.81% -
531.deepsjeng_r 2.00% 50.000 51.000 - 41.67% -
531.deepsjeng_r (avg) 2.00% 50.000 51.000 - 41.67% -
505.mcf_r 1.67% 60.000 61.000 - -16.44% -
505.mcf_r (avg) 1.67% 60.000 61.000 - -16.44% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
557.xz_r -1.64% 11587.000 11397.000 - -1.38% -
544.nab_r -1.15% 14294.000 14130.000 - -1.30% -
521.wrf_r -1.10% 14593.000 14432.000 - -2.13% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (unexplained diff vs previous) -36.67% 30.000 19.000 - -20.83% -
519.lbm_r (unexplained diff vs previous) (percentage) -35.29% 0.170 0.110 - -21.43% -
525.x264_r (max) (percentage) -18.97% 0.580 0.470 - 213.33% -
525.x264_r (max) -18.92% 37.000 30.000 - 172.73% -
519.lbm_r (max) (percentage) -10.53% 0.570 0.510 - 200.00% -
519.lbm_r (max) -10.10% 99.000 89.000 - 206.90% -
541.leela_r (max) (percentage) -7.14% 0.280 0.260 - -52.73% -
525.x264_r -6.67% 15.000 14.000 - 133.33% -
525.x264_r (avg) -6.67% 15.000 14.000 - 133.33% -
505.mcf_r (max) (percentage) -6.00% 0.500 0.470 - -30.88% -
541.leela_r (max) -5.41% 37.000 35.000 - -52.70% -
511.povray_r (max) (percentage) -5.17% 0.580 0.550 - 0.00% -
519.lbm_r -4.76% 42.000 40.000 - 37.93% -
519.lbm_r (avg) -4.76% 42.000 40.000 - 37.93% -
523.xalancbmk_r (max) -4.23% 71.000 68.000 - 30.77% -
521.wrf_r (max) -4.16% 481.000 461.000 - 322.94% -
531.deepsjeng_r (max) (percentage) -4.08% 0.980 0.940 - 22.08% -
544.nab_r (max) -4.00% 50.000 48.000 - 29.73% -
523.xalancbmk_r (max) (percentage) -3.85% 0.520 0.500 - 31.58% -
531.deepsjeng_r (max) -3.57% 84.000 81.000 - 20.90% -
521.wrf_r (max) (percentage) -3.03% 3.300 3.200 - 332.43% -
505.mcf_r (max) -2.90% 69.000 67.000 - -30.21% -
554.roms_r (max) (percentage) -2.56% 0.390 0.380 - 442.86% -
538.imagick_r (max) (percentage) -2.33% 0.430 0.420 - 0.00% -
554.roms_r (max) -2.25% 89.000 87.000 - 443.75% -
500.perlbench_r (max) (percentage) -2.04% 1.470 1.440 - 8.27% -
544.nab_r -2.00% 50.000 49.000 - 19.51% -
544.nab_r (avg) -2.00% 50.000 49.000 - 19.51% -
549.fotonik3d_r (max) (percentage) -1.85% 0.540 0.530 - -56.20% -
511.povray_r (max) -1.72% 116.000 114.000 - -3.39% -
538.imagick_r (max) -1.69% 59.000 58.000 - 0.00% -
500.perlbench_r (max) -1.69% 237.000 233.000 - 9.39% -
510.parest_r (max) -1.61% 62.000 61.000 - 41.86% -
521.wrf_r -1.02% 294.000 291.000 - 153.04% -
521.wrf_r (avg) -1.02% 294.000 291.000 - 153.04% -

Added Tests - sample variability
508.namd_r (unexplained diff vs previous)
508.namd_r (unexplained diff vs previous) (percentage)
525.x264_r (unexplained diff vs previous)
525.x264_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 1.47% 11293.000 11459.000 - 0.61% -
505.mcf_r 1.29% 14148.000 14331.000 - 2.63% -
500.perlbench_r 1.11% 16051.000 16229.000 - 0.46% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
554.roms_r (max) 443.75% 16.000 87.000 - -2.25% -
554.roms_r (max) (percentage) 442.86% 0.070 0.380 - -2.56% -
521.wrf_r (max) (percentage) 332.43% 0.740 3.200 - -3.03% -
521.wrf_r (max) 322.94% 109.000 461.000 - -4.16% -
554.roms_r 230.77% 13.000 43.000 - 0.00% -
554.roms_r (avg) 230.77% 13.000 43.000 - 0.00% -
554.roms_r (avg) (percentage) 216.67% 0.060 0.190 - 0.00% -
525.x264_r (max) (percentage) 213.33% 0.150 0.470 - -18.97% -
519.lbm_r (max) 206.90% 29.000 89.000 - -10.10% -
519.lbm_r (max) (percentage) 200.00% 0.170 0.510 - -10.53% -
525.x264_r (max) 172.73% 11.000 30.000 - -18.92% -
520.omnetpp_r (unexplained diff vs previous) 164.58% 48.000 127.000 - 56.79% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 163.64% 0.330 0.870 - 58.18% -
521.wrf_r (avg) (percentage) 158.97% 0.780 2.020 - 0.00% -
525.x264_r (avg) (percentage) 155.56% 0.090 0.230 - -4.17% -
521.wrf_r 153.04% 115.000 291.000 - -1.02% -
521.wrf_r (avg) 153.04% 115.000 291.000 - -1.02% -
525.x264_r 133.33% 6.000 14.000 - -6.67% -
525.x264_r (avg) 133.33% 6.000 14.000 - -6.67% -
503.bwaves_r (max) (percentage) 44.44% 0.090 0.130 - 0.00% -
531.deepsjeng_r (avg) (percentage) 43.90% 0.410 0.590 - 0.00% -
510.parest_r (max) (percentage) 42.11% 0.190 0.270 - 0.00% -
503.bwaves_r (max) 42.11% 19.000 27.000 - 0.00% -
510.parest_r (max) 41.86% 43.000 61.000 - -1.61% -
531.deepsjeng_r 41.67% 36.000 51.000 - 2.00% -
531.deepsjeng_r (avg) 41.67% 36.000 51.000 - 2.00% -
519.lbm_r 37.93% 29.000 40.000 - -4.76% -
519.lbm_r (avg) 37.93% 29.000 40.000 - -4.76% -
519.lbm_r (avg) (percentage) 35.29% 0.170 0.230 - -4.17% -
523.xalancbmk_r (max) (percentage) 31.58% 0.380 0.500 - -3.85% -
544.nab_r (max) (percentage) 30.77% 0.260 0.340 - -2.86% -
523.xalancbmk_r (max) 30.77% 52.000 68.000 - -4.23% -
544.nab_r (max) 29.73% 37.000 48.000 - -4.00% -
531.deepsjeng_r (max) (percentage) 22.08% 0.770 0.940 - -4.08% -
531.deepsjeng_r (max) 20.90% 67.000 81.000 - -3.57% -
544.nab_r (avg) (percentage) 20.69% 0.290 0.350 - 0.00% -
557.xz_r (avg) (percentage) 20.00% 0.100 0.120 - 0.00% -
544.nab_r 19.51% 41.000 49.000 - -2.00% -
544.nab_r (avg) 19.51% 41.000 49.000 - -2.00% -
557.xz_r 18.18% 11.000 13.000 - 0.00% -
557.xz_r (avg) 18.18% 11.000 13.000 - 0.00% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
511.povray_r 10.39% 77.000 85.000 - 3.66% -
511.povray_r (avg) 10.39% 77.000 85.000 - 3.66% -
500.perlbench_r (max) 9.39% 213.000 233.000 - -1.69% -
500.perlbench_r (max) (percentage) 8.27% 1.330 1.440 - -2.04% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -4.55% 22.000 21.000 - 0.00% -
nb_execution_successful -4.55% 22.000 21.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
521.wrf_r -2.06% 23893420.000 23401808.000 - -0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -13.83% 7490.000 6454.000 - 0.14% -
526.blender_r -3.85% 12432.000 11953.000 - -0.82% -
511.povray_r -3.70% 21547.000 20750.000 - 3.00% -
521.wrf_r -2.13% 14746.000 14432.000 - -1.10% -
557.xz_r -1.38% 11556.000 11397.000 - -1.64% -
531.deepsjeng_r -1.35% 8789.000 8670.000 - 0.44% -
544.nab_r -1.30% 14316.000 14130.000 - -1.15% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
508.namd_r (max) -75.00% 108.000 27.000 - 17.39% -
508.namd_r (max) (percentage) -74.34% 1.130 0.290 - 16.00% -
508.namd_r (unexplained diff vs previous) (percentage) -73.64% 1.100 0.290 - 0.00% -
508.namd_r (unexplained diff vs previous) -73.33% 105.000 28.000 - 0.00% -
520.omnetpp_r (max) -65.85% 366.000 125.000 - 5.93% -
520.omnetpp_r (max) (percentage) -65.60% 2.500 0.860 - 7.50% -
549.fotonik3d_r (max) (percentage) -56.20% 1.210 0.530 - -1.85% -
541.leela_r (max) (percentage) -52.73% 0.550 0.260 - -7.14% -
541.leela_r (max) -52.70% 74.000 35.000 - -5.41% -
548.exchange2_r (max) -50.00% 40.000 20.000 - 0.00% -
548.exchange2_r (max) (percentage) -50.00% 0.220 0.110 - 0.00% -
508.namd_r -42.62% 61.000 35.000 - 0.00% -
508.namd_r (avg) -42.62% 61.000 35.000 - 0.00% -
508.namd_r (avg) (percentage) -42.19% 0.640 0.370 - 0.00% -
557.xz_r (max) -38.10% 21.000 13.000 - 0.00% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
557.xz_r (max) (percentage) -36.84% 0.190 0.120 - 0.00% -
503.bwaves_r -33.33% 21.000 14.000 - 0.00% -
503.bwaves_r (avg) -33.33% 21.000 14.000 - 0.00% -
526.blender_r -31.71% 41.000 28.000 - 0.00% -
526.blender_r (avg) -31.71% 41.000 28.000 - 0.00% -
505.mcf_r (max) (percentage) -30.88% 0.680 0.470 - -6.00% -
505.mcf_r (max) -30.21% 96.000 67.000 - -2.90% -
503.bwaves_r (avg) (percentage) -30.00% 0.100 0.070 - 0.00% -
541.leela_r (avg) (percentage) -29.55% 0.440 0.310 - 0.00% -
541.leela_r -28.81% 59.000 42.000 - 2.44% -
541.leela_r (avg) -28.81% 59.000 42.000 - 2.44% -
526.blender_r (avg) (percentage) -27.27% 0.330 0.240 - 0.00% -
526.blender_r (max) -27.27% 44.000 32.000 - 0.00% -
548.exchange2_r (avg) (percentage) -26.67% 0.150 0.110 - 0.00% -
548.exchange2_r -25.93% 27.000 20.000 - 0.00% -
548.exchange2_r (avg) -25.93% 27.000 20.000 - 0.00% -
526.blender_r (max) (percentage) -25.00% 0.360 0.270 - 0.00% -
538.imagick_r -24.39% 41.000 31.000 - 0.00% -
538.imagick_r (avg) -24.39% 41.000 31.000 - 0.00% -
538.imagick_r (avg) (percentage) -23.33% 0.300 0.230 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) -21.43% 0.140 0.110 - -35.29% -
519.lbm_r (unexplained diff vs previous) -20.83% 24.000 19.000 - -36.67% -
525.x264_r (unexplained diff vs previous) -18.18% 11.000 9.000 - 0.00% -
505.mcf_r (avg) (percentage) -17.31% 0.520 0.430 - 0.00% -
505.mcf_r -16.44% 73.000 61.000 - 1.67% -
505.mcf_r (avg) -16.44% 73.000 61.000 - 1.67% -
500.perlbench_r (avg) (percentage) -9.89% 0.910 0.820 - 0.00% -
520.omnetpp_r -8.96% 134.000 122.000 - -0.81% -
520.omnetpp_r (avg) -8.96% 134.000 122.000 - -0.81% -
500.perlbench_r -8.90% 146.000 133.000 - 0.76% -
500.perlbench_r (avg) -8.90% 146.000 133.000 - 0.76% -
520.omnetpp_r (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
523.xalancbmk_r -6.06% 33.000 31.000 - 0.00% -
523.xalancbmk_r (avg) -6.06% 33.000 31.000 - 0.00% -
510.parest_r -5.88% 34.000 32.000 - 0.00% -
510.parest_r (avg) -5.88% 34.000 32.000 - 0.00% -
511.povray_r (max) -3.39% 118.000 114.000 - -1.72% -

Report Time: 0.33s
Execution (seconds)
Name Current %
500.perlbench_r 16229.000 -
502.gcc_r 11049.000 -
503.bwaves_r 21254.000 -
505.mcf_r 14331.000 2.63%
507.cactuBSSN_r 11459.000 -
508.namd_r 9528.000 -
510.parest_r 22945.000 -
511.povray_r 20750.000 3.00%
519.lbm_r 17544.000 -
520.omnetpp_r 14634.000 -
521.wrf_r 14432.000 -1.10%
523.xalancbmk_r 13696.000 -
525.x264_r 6454.000 -
526.blender_r 11953.000 -
531.deepsjeng_r 8670.000 -
538.imagick_r 13813.000 -
541.leela_r 13597.000 1.52%
544.nab_r 14130.000 -1.15%
548.exchange2_r 18542.000 -
554.roms_r 23016.000 -
557.xz_r 11397.000 -1.64%
Geometric Mean 14040.099 -
Code Size (bytes)
Name Current %
500.perlbench_r 2154343.000 -
502.gcc_r 8935600.000 -
503.bwaves_r 777018.000 -
505.mcf_r 27203.000 -
507.cactuBSSN_r 3676226.000 -
508.namd_r 620707.000 -
510.parest_r 8611292.000 -
511.povray_r 915834.000 -
519.lbm_r 13339.000 -
520.omnetpp_r 2153601.000 -
521.wrf_r 23401808.000 -
523.xalancbmk_r 4726075.000 -
525.x264_r 498942.000 -
526.blender_r 13664258.000 -
531.deepsjeng_r 80874.000 -
538.imagick_r 1700929.000 -
541.leela_r 144975.000 -
544.nab_r 190988.000 -
548.exchange2_r 2551287.000 -
554.roms_r 3034357.000 -
557.xz_r 177248.000 -
Geometric Mean 991107.572 -
sample variability
Name Current %
500.perlbench_r 133.000 -
500.perlbench_r (avg) 133.000 -
500.perlbench_r (avg) (percentage) 0.820 -
500.perlbench_r (max) 233.000 -1.69%
500.perlbench_r (max) (percentage) 1.440 -2.04%
503.bwaves_r 14.000 -
503.bwaves_r (avg) 14.000 -
503.bwaves_r (avg) (percentage) 0.070 -
503.bwaves_r (max) 27.000 -
503.bwaves_r (max) (percentage) 0.130 -
505.mcf_r 61.000 1.67%
505.mcf_r (avg) 61.000 1.67%
505.mcf_r (avg) (percentage) 0.430 -
505.mcf_r (max) 67.000 -2.90%
505.mcf_r (max) (percentage) 0.470 -6.00%
508.namd_r 35.000 -
508.namd_r (avg) 35.000 -
508.namd_r (avg) (percentage) 0.370 -
508.namd_r (max) 27.000 17.39%
508.namd_r (max) (percentage) 0.290 16.00%
508.namd_r (unexplained diff vs previous) 28.000 -
508.namd_r (unexplained diff vs previous) (percentage) 0.290 -
510.parest_r 32.000 -
510.parest_r (avg) 32.000 -
510.parest_r (avg) (percentage) 0.140 -
510.parest_r (max) 61.000 -1.61%
510.parest_r (max) (percentage) 0.270 -
511.povray_r 85.000 3.66%
511.povray_r (avg) 85.000 3.66%
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 114.000 -1.72%
511.povray_r (max) (percentage) 0.550 -5.17%
519.lbm_r 40.000 -4.76%
519.lbm_r (avg) 40.000 -4.76%
519.lbm_r (avg) (percentage) 0.230 -
519.lbm_r (max) 89.000 -10.10%
519.lbm_r (max) (percentage) 0.510 -10.53%
519.lbm_r (unexplained diff vs previous) 19.000 -36.67%
519.lbm_r (unexplained diff vs previous) (percentage) 0.110 -35.29%
520.omnetpp_r 122.000 -
520.omnetpp_r (avg) 122.000 -
520.omnetpp_r (avg) (percentage) 0.840 -
520.omnetpp_r (max) 125.000 5.93%
520.omnetpp_r (max) (percentage) 0.860 7.50%
520.omnetpp_r (unexplained diff vs previous) 127.000 56.79%
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.870 58.18%
521.wrf_r 291.000 -1.02%
521.wrf_r (avg) 291.000 -1.02%
521.wrf_r (avg) (percentage) 2.020 -
521.wrf_r (max) 461.000 -4.16%
521.wrf_r (max) (percentage) 3.200 -3.03%
523.xalancbmk_r 31.000 -
523.xalancbmk_r (avg) 31.000 -
523.xalancbmk_r (avg) (percentage) 0.230 -
523.xalancbmk_r (max) 68.000 -4.23%
523.xalancbmk_r (max) (percentage) 0.500 -3.85%
525.x264_r 14.000 -6.67%
525.x264_r (avg) 14.000 -6.67%
525.x264_r (avg) (percentage) 0.230 -
525.x264_r (max) 30.000 -18.92%
525.x264_r (max) (percentage) 0.470 -18.97%
525.x264_r (unexplained diff vs previous) 9.000 -
525.x264_r (unexplained diff vs previous) (percentage) 0.140 -
526.blender_r 28.000 -
526.blender_r (avg) 28.000 -
526.blender_r (avg) (percentage) 0.240 -
526.blender_r (max) 32.000 -
526.blender_r (max) (percentage) 0.270 -
531.deepsjeng_r 51.000 2.00%
531.deepsjeng_r (avg) 51.000 2.00%
531.deepsjeng_r (avg) (percentage) 0.590 -
531.deepsjeng_r (max) 81.000 -3.57%
531.deepsjeng_r (max) (percentage) 0.940 -4.08%
538.imagick_r 31.000 -
538.imagick_r (avg) 31.000 -
538.imagick_r (avg) (percentage) 0.230 -
538.imagick_r (max) 58.000 -1.69%
538.imagick_r (max) (percentage) 0.420 -2.33%
541.leela_r 42.000 2.44%
541.leela_r (avg) 42.000 2.44%
541.leela_r (avg) (percentage) 0.310 -
541.leela_r (max) 35.000 -5.41%
541.leela_r (max) (percentage) 0.260 -7.14%
544.nab_r 49.000 -2.00%
544.nab_r (avg) 49.000 -2.00%
544.nab_r (avg) (percentage) 0.350 -
544.nab_r (max) 48.000 -4.00%
544.nab_r (max) (percentage) 0.340 -
548.exchange2_r 20.000 -
548.exchange2_r (avg) 20.000 -
548.exchange2_r (avg) (percentage) 0.110 -
548.exchange2_r (max) 20.000 -
548.exchange2_r (max) (percentage) 0.110 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.530 -1.85%
554.roms_r 43.000 -
554.roms_r (avg) 43.000 -
554.roms_r (avg) (percentage) 0.190 -
554.roms_r (max) 87.000 -2.25%
554.roms_r (max) (percentage) 0.380 -2.56%
557.xz_r 13.000 -
557.xz_r (avg) 13.000 -
557.xz_r (avg) (percentage) 0.120 -
557.xz_r (max) 13.000 -
557.xz_r (max) (percentage) 0.120 -
Geometric Mean 5.893 -5.67%