Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/e91123fb1b8146b087571d2052a5adbbc9400c9f (+ 472 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/44/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0044 (llvmorg-17-init-11613-ge91123fb1b81) 2023-05-14T21:27:36 0:00:00
Previous 0043 (llvmorg-17-init-11141-gda3c06a4823b) 2023-05-11T23:52:25 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 5 53
Performance Improvements 47 53
Added Tests 2 2
Unchanged Tests 822 768
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
507.cactuBSSN_r 1.09% 11312.000 11435.000 - 1.26% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
531.deepsjeng_r 2.63% 38.000 39.000 - 8.33% -
531.deepsjeng_r (avg) 2.63% 38.000 39.000 - 8.33% -
538.imagick_r (max) 2.17% 46.000 47.000 - -18.97% -
508.namd_r (max) 1.96% 51.000 52.000 - -51.85% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
521.wrf_r -1.90% 14844.000 14562.000 - -1.25% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) (percentage) -83.65% 1.040 0.170 - -48.48% -
520.omnetpp_r (unexplained diff vs previous) -83.55% 152.000 25.000 - -47.92% -
525.x264_r (unexplained diff vs previous) -66.67% 9.000 3.000 - -72.73% -
525.x264_r (unexplained diff vs previous) (percentage) -66.67% 0.120 0.040 - -73.33% -
520.omnetpp_r (max) (percentage) -49.68% 1.570 0.790 - -68.40% -
520.omnetpp_r (max) -49.57% 230.000 116.000 - -68.31% -
525.x264_r (max) (percentage) -11.76% 0.170 0.150 - 0.00% -
519.lbm_r (max) (percentage) -11.54% 0.260 0.230 - 35.29% -
519.lbm_r (max) -11.11% 45.000 40.000 - 37.93% -
525.x264_r (max) -8.33% 12.000 11.000 - 0.00% -
544.nab_r (max) (percentage) -7.69% 0.390 0.360 - 38.46% -
544.nab_r (max) -7.27% 55.000 51.000 - 37.84% -
549.fotonik3d_r (max) -6.32% 253.000 237.000 - -37.47% -
549.fotonik3d_r (max) (percentage) -6.17% 0.810 0.760 - -37.19% -
520.omnetpp_r (avg) (percentage) -6.00% 1.000 0.940 - 2.17% -
519.lbm_r -5.71% 35.000 33.000 - 13.79% -
519.lbm_r (avg) -5.71% 35.000 33.000 - 13.79% -
510.parest_r (max) -5.62% 89.000 84.000 - 95.35% -
520.omnetpp_r -5.48% 146.000 138.000 - 2.99% -
520.omnetpp_r (avg) -5.48% 146.000 138.000 - 2.99% -
531.deepsjeng_r (max) (percentage) -5.33% 0.750 0.710 - -7.79% -
525.x264_r -5.26% 19.000 18.000 - 200.00% -
525.x264_r (avg) -5.26% 19.000 18.000 - 200.00% -
510.parest_r (max) (percentage) -5.13% 0.390 0.370 - 94.74% -
519.lbm_r (avg) (percentage) -5.00% 0.200 0.190 - 11.76% -
531.deepsjeng_r (max) -4.69% 64.000 61.000 - -8.96% -
541.leela_r (max) (percentage) -4.26% 0.470 0.450 - -18.18% -
525.x264_r (avg) (percentage) -4.00% 0.250 0.240 - 166.67% -
523.xalancbmk_r (max) -3.77% 53.000 51.000 - -1.92% -
505.mcf_r (max) (percentage) -3.53% 0.850 0.820 - 20.59% -
521.wrf_r (max) -3.52% 369.000 356.000 - 226.61% -
500.perlbench_r (max) (percentage) -3.51% 1.710 1.650 - 24.06% -
505.mcf_r (max) -3.39% 118.000 114.000 - 18.75% -
500.perlbench_r (max) -3.31% 272.000 263.000 - 23.47% -
557.xz_r (max) (percentage) -3.23% 0.310 0.300 - 57.89% -
541.leela_r (max) -3.23% 62.000 60.000 - -18.92% -
548.exchange2_r (max) -3.17% 63.000 61.000 - 52.50% -
548.exchange2_r (max) (percentage) -2.94% 0.340 0.330 - 50.00% -
557.xz_r (max) -2.86% 35.000 34.000 - 61.90% -
523.xalancbmk_r (max) (percentage) -2.56% 0.390 0.380 - 0.00% -
511.povray_r (max) (percentage) -1.96% 0.510 0.500 - -9.09% -
554.roms_r (max) (percentage) -1.96% 0.510 0.500 - 614.29% -
554.roms_r (max) -1.72% 116.000 114.000 - 612.50% -
521.wrf_r -1.62% 185.000 182.000 - 58.26% -
521.wrf_r (avg) -1.62% 185.000 182.000 - 58.26% -
521.wrf_r (max) (percentage) -1.61% 2.490 2.450 - 231.08% -

Added Tests - sample variability
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r 2.07% 7490.000 7645.000 - 0.04% -
507.cactuBSSN_r 1.26% 11293.000 11435.000 - 1.09% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
554.roms_r (max) (percentage) 614.29% 0.070 0.500 - -1.96% -
554.roms_r (max) 612.50% 16.000 114.000 - -1.72% -
521.wrf_r (max) (percentage) 231.08% 0.740 2.450 - -1.61% -
521.wrf_r (max) 226.61% 109.000 356.000 - -3.52% -
554.roms_r 215.38% 13.000 41.000 - 0.00% -
554.roms_r (avg) 215.38% 13.000 41.000 - 0.00% -
525.x264_r 200.00% 6.000 18.000 - -5.26% -
525.x264_r (avg) 200.00% 6.000 18.000 - -5.26% -
554.roms_r (avg) (percentage) 200.00% 0.060 0.180 - 0.00% -
525.x264_r (avg) (percentage) 166.67% 0.090 0.240 - -4.00% -
510.parest_r (max) 95.35% 43.000 84.000 - -5.62% -
510.parest_r (max) (percentage) 94.74% 0.190 0.370 - -5.13% -
557.xz_r 63.64% 11.000 18.000 - 0.00% -
557.xz_r (avg) 63.64% 11.000 18.000 - 0.00% -
557.xz_r (max) 61.90% 21.000 34.000 - -2.86% -
521.wrf_r (avg) (percentage) 60.26% 0.780 1.250 - 0.00% -
557.xz_r (avg) (percentage) 60.00% 0.100 0.160 - 0.00% -
521.wrf_r 58.26% 115.000 182.000 - -1.62% -
521.wrf_r (avg) 58.26% 115.000 182.000 - -1.62% -
557.xz_r (max) (percentage) 57.89% 0.190 0.300 - -3.23% -
548.exchange2_r (max) 52.50% 40.000 61.000 - -3.17% -
548.exchange2_r (max) (percentage) 50.00% 0.220 0.330 - -2.94% -
544.nab_r (max) (percentage) 38.46% 0.260 0.360 - -7.69% -
519.lbm_r (max) 37.93% 29.000 40.000 - -11.11% -
544.nab_r (max) 37.84% 37.000 51.000 - -7.27% -
519.lbm_r (max) (percentage) 35.29% 0.170 0.230 - -11.54% -
500.perlbench_r (max) (percentage) 24.06% 1.330 1.650 - -3.51% -
500.perlbench_r (max) 23.47% 213.000 263.000 - -3.31% -
544.nab_r (avg) (percentage) 20.69% 0.290 0.350 - 0.00% -
505.mcf_r (max) (percentage) 20.59% 0.680 0.820 - -3.53% -
544.nab_r 19.51% 41.000 49.000 - 0.00% -
544.nab_r (avg) 19.51% 41.000 49.000 - 0.00% -
505.mcf_r (max) 18.75% 96.000 114.000 - -3.39% -
511.povray_r (avg) (percentage) 16.67% 0.360 0.420 - 0.00% -
500.perlbench_r (avg) (percentage) 16.48% 0.910 1.060 - 0.00% -
500.perlbench_r 15.75% 146.000 169.000 - 0.00% -
500.perlbench_r (avg) 15.75% 146.000 169.000 - 0.00% -
511.povray_r 15.58% 77.000 89.000 - 0.00% -
511.povray_r (avg) 15.58% 77.000 89.000 - 0.00% -
519.lbm_r 13.79% 29.000 33.000 - -5.71% -
519.lbm_r (avg) 13.79% 29.000 33.000 - -5.71% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
519.lbm_r (avg) (percentage) 11.76% 0.170 0.190 - -5.00% -
531.deepsjeng_r (avg) (percentage) 9.76% 0.410 0.450 - 0.00% -
531.deepsjeng_r 8.33% 36.000 39.000 - 2.63% -
531.deepsjeng_r (avg) 8.33% 36.000 39.000 - 2.63% -
520.omnetpp_r 2.99% 134.000 138.000 - -5.48% -
520.omnetpp_r (avg) 2.99% 134.000 138.000 - -5.48% -
520.omnetpp_r (avg) (percentage) 2.17% 0.920 0.940 - -6.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
521.wrf_r -2.20% 23893420.000 23368952.000 - 0.02% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
526.blender_r -3.74% 12432.000 11967.000 - -0.04% -
505.mcf_r -1.73% 14148.000 13903.000 - -0.37% -
523.xalancbmk_r -1.43% 13812.000 13614.000 - -0.94% -
521.wrf_r -1.25% 14746.000 14562.000 - -1.90% -
531.deepsjeng_r -1.22% 8789.000 8682.000 - 0.28% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
525.x264_r (unexplained diff vs previous) (percentage) -73.33% 0.150 0.040 - -66.67% -
525.x264_r (unexplained diff vs previous) -72.73% 11.000 3.000 - -66.67% -
520.omnetpp_r (max) (percentage) -68.40% 2.500 0.790 - -49.68% -
520.omnetpp_r (max) -68.31% 366.000 116.000 - -49.57% -
508.namd_r (max) (percentage) -52.21% 1.130 0.540 - 0.00% -
508.namd_r (max) -51.85% 108.000 52.000 - 1.96% -
508.namd_r -50.82% 61.000 30.000 - 0.00% -
508.namd_r (avg) -50.82% 61.000 30.000 - 0.00% -
508.namd_r (avg) (percentage) -50.00% 0.640 0.320 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) (percentage) -48.48% 0.330 0.170 - -83.65% -
520.omnetpp_r (unexplained diff vs previous) -47.92% 48.000 25.000 - -83.55% -
503.bwaves_r -42.86% 21.000 12.000 - 0.00% -
503.bwaves_r (avg) -42.86% 21.000 12.000 - 0.00% -
503.bwaves_r (avg) (percentage) -40.00% 0.100 0.060 - 0.00% -
549.fotonik3d_r (max) -37.47% 379.000 237.000 - -6.32% -
549.fotonik3d_r (max) (percentage) -37.19% 1.210 0.760 - -6.17% -
526.blender_r -31.71% 41.000 28.000 - 0.00% -
526.blender_r (avg) -31.71% 41.000 28.000 - 0.00% -
526.blender_r (avg) (percentage) -27.27% 0.330 0.240 - 0.00% -
538.imagick_r -26.83% 41.000 30.000 - 0.00% -
538.imagick_r (avg) -26.83% 41.000 30.000 - 0.00% -
538.imagick_r (avg) (percentage) -26.67% 0.300 0.220 - 0.00% -
503.bwaves_r (max) -26.32% 19.000 14.000 - 0.00% -
541.leela_r -25.42% 59.000 44.000 - 0.00% -
541.leela_r (avg) -25.42% 59.000 44.000 - 0.00% -
541.leela_r (avg) (percentage) -25.00% 0.440 0.330 - 0.00% -
549.fotonik3d_r -24.59% 244.000 184.000 - 0.00% -
549.fotonik3d_r (avg) -24.59% 244.000 184.000 - 0.00% -
549.fotonik3d_r (avg) (percentage) -24.36% 0.780 0.590 - 0.00% -
526.blender_r (max) -22.73% 44.000 34.000 - 0.00% -
503.bwaves_r (max) (percentage) -22.22% 0.090 0.070 - 0.00% -
526.blender_r (max) (percentage) -19.44% 0.360 0.290 - 0.00% -
538.imagick_r (max) (percentage) -19.05% 0.420 0.340 - 0.00% -
538.imagick_r (max) -18.97% 58.000 47.000 - 2.17% -
541.leela_r (max) -18.92% 74.000 60.000 - -3.23% -
541.leela_r (max) (percentage) -18.18% 0.550 0.450 - -4.26% -
523.xalancbmk_r -15.15% 33.000 28.000 - 0.00% -
523.xalancbmk_r (avg) -15.15% 33.000 28.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) -12.50% 0.240 0.210 - 0.00% -
511.povray_r (max) -9.32% 118.000 107.000 - -0.93% -
511.povray_r (max) (percentage) -9.09% 0.550 0.500 - -1.96% -
531.deepsjeng_r (max) -8.96% 67.000 61.000 - -4.69% -
531.deepsjeng_r (max) (percentage) -7.79% 0.770 0.710 - -5.33% -
505.mcf_r -5.48% 73.000 69.000 - 0.00% -
505.mcf_r (avg) -5.48% 73.000 69.000 - 0.00% -
505.mcf_r (avg) (percentage) -3.85% 0.520 0.500 - 0.00% -
523.xalancbmk_r (max) -1.92% 52.000 51.000 - -3.77% -

Added Tests - sample variability
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)

Report Time: 0.34s
Execution (seconds)
Name Current %
500.perlbench_r 15954.000 -
502.gcc_r 10988.000 -
503.bwaves_r 21147.000 -
505.mcf_r 13903.000 -
507.cactuBSSN_r 11435.000 1.09%
508.namd_r 9636.000 -
510.parest_r 22931.000 -
511.povray_r 21425.000 -
519.lbm_r 17567.000 -
520.omnetpp_r 14690.000 -
521.wrf_r 14562.000 -1.90%
523.xalancbmk_r 13614.000 -
525.x264_r 7645.000 -
526.blender_r 11967.000 -
531.deepsjeng_r 8682.000 -
538.imagick_r 13824.000 -
541.leela_r 13439.000 -
544.nab_r 14284.000 -
548.exchange2_r 18609.000 -
549.fotonik3d_r 31250.000 -
554.roms_r 22889.000 -
557.xz_r 11474.000 -
Geometric Mean 14671.280 -
Code Size (bytes)
Name Current %
500.perlbench_r 2153031.000 -
502.gcc_r 8926772.000 -
503.bwaves_r 775906.000 -
505.mcf_r 27207.000 -
507.cactuBSSN_r 3676914.000 -
508.namd_r 621419.000 -
510.parest_r 8610108.000 -
511.povray_r 914774.000 -
519.lbm_r 13339.000 -
520.omnetpp_r 2152091.000 -
521.wrf_r 23368952.000 -
523.xalancbmk_r 4724495.000 -
525.x264_r 499610.000 -
526.blender_r 13660926.000 -
531.deepsjeng_r 81086.000 -
538.imagick_r 1701469.000 -
541.leela_r 144811.000 -
544.nab_r 191016.000 -
548.exchange2_r 2555903.000 -
549.fotonik3d_r 1245130.000 -
554.roms_r 3032517.000 -
557.xz_r 177048.000 -
Geometric Mean 1001341.554 -
sample variability
Name Current %
500.perlbench_r 169.000 -
500.perlbench_r (avg) 169.000 -
500.perlbench_r (avg) (percentage) 1.060 -
500.perlbench_r (max) 263.000 -3.31%
500.perlbench_r (max) (percentage) 1.650 -3.51%
503.bwaves_r 12.000 -
503.bwaves_r (avg) 12.000 -
503.bwaves_r (avg) (percentage) 0.060 -
503.bwaves_r (max) 14.000 -
503.bwaves_r (max) (percentage) 0.070 -
505.mcf_r 69.000 -
505.mcf_r (avg) 69.000 -
505.mcf_r (avg) (percentage) 0.500 -
505.mcf_r (max) 114.000 -3.39%
505.mcf_r (max) (percentage) 0.820 -3.53%
508.namd_r 30.000 -
508.namd_r (avg) 30.000 -
508.namd_r (avg) (percentage) 0.320 -
508.namd_r (max) 52.000 1.96%
508.namd_r (max) (percentage) 0.540 -
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 84.000 -5.62%
510.parest_r (max) (percentage) 0.370 -5.13%
511.povray_r 89.000 -
511.povray_r (avg) 89.000 -
511.povray_r (avg) (percentage) 0.420 -
511.povray_r (max) 107.000 -
511.povray_r (max) (percentage) 0.500 -1.96%
519.lbm_r 33.000 -5.71%
519.lbm_r (avg) 33.000 -5.71%
519.lbm_r (avg) (percentage) 0.190 -5.00%
519.lbm_r (max) 40.000 -11.11%
519.lbm_r (max) (percentage) 0.230 -11.54%
519.lbm_r (unexplained diff vs previous) 0.000 -
519.lbm_r (unexplained diff vs previous) (percentage) 0.000 -
520.omnetpp_r 138.000 -5.48%
520.omnetpp_r (avg) 138.000 -5.48%
520.omnetpp_r (avg) (percentage) 0.940 -6.00%
520.omnetpp_r (max) 116.000 -49.57%
520.omnetpp_r (max) (percentage) 0.790 -49.68%
520.omnetpp_r (unexplained diff vs previous) 25.000 -83.55%
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.170 -83.65%
521.wrf_r 182.000 -1.62%
521.wrf_r (avg) 182.000 -1.62%
521.wrf_r (avg) (percentage) 1.250 -
521.wrf_r (max) 356.000 -3.52%
521.wrf_r (max) (percentage) 2.450 -1.61%
523.xalancbmk_r 28.000 -
523.xalancbmk_r (avg) 28.000 -
523.xalancbmk_r (avg) (percentage) 0.210 -
523.xalancbmk_r (max) 51.000 -3.77%
523.xalancbmk_r (max) (percentage) 0.380 -2.56%
525.x264_r 18.000 -5.26%
525.x264_r (avg) 18.000 -5.26%
525.x264_r (avg) (percentage) 0.240 -4.00%
525.x264_r (max) 11.000 -8.33%
525.x264_r (max) (percentage) 0.150 -11.76%
525.x264_r (unexplained diff vs previous) 3.000 -66.67%
525.x264_r (unexplained diff vs previous) (percentage) 0.040 -66.67%
526.blender_r 28.000 -
526.blender_r (avg) 28.000 -
526.blender_r (avg) (percentage) 0.240 -
526.blender_r (max) 34.000 -
526.blender_r (max) (percentage) 0.290 -
531.deepsjeng_r 39.000 2.63%
531.deepsjeng_r (avg) 39.000 2.63%
531.deepsjeng_r (avg) (percentage) 0.450 -
531.deepsjeng_r (max) 61.000 -4.69%
531.deepsjeng_r (max) (percentage) 0.710 -5.33%
538.imagick_r 30.000 -
538.imagick_r (avg) 30.000 -
538.imagick_r (avg) (percentage) 0.220 -
538.imagick_r (max) 47.000 2.17%
538.imagick_r (max) (percentage) 0.340 -
541.leela_r 44.000 -
541.leela_r (avg) 44.000 -
541.leela_r (avg) (percentage) 0.330 -
541.leela_r (max) 60.000 -3.23%
541.leela_r (max) (percentage) 0.450 -4.26%
544.nab_r 49.000 -
544.nab_r (avg) 49.000 -
544.nab_r (avg) (percentage) 0.350 -
544.nab_r (max) 51.000 -7.27%
544.nab_r (max) (percentage) 0.360 -7.69%
548.exchange2_r 27.000 -
548.exchange2_r (avg) 27.000 -
548.exchange2_r (avg) (percentage) 0.150 -
548.exchange2_r (max) 61.000 -3.17%
548.exchange2_r (max) (percentage) 0.330 -2.94%
549.fotonik3d_r 184.000 -
549.fotonik3d_r (avg) 184.000 -
549.fotonik3d_r (avg) (percentage) 0.590 -
549.fotonik3d_r (max) 237.000 -6.32%
549.fotonik3d_r (max) (percentage) 0.760 -6.17%
554.roms_r 41.000 -
554.roms_r (avg) 41.000 -
554.roms_r (avg) (percentage) 0.180 -
554.roms_r (max) 114.000 -1.72%
554.roms_r (max) (percentage) 0.500 -1.96%
557.xz_r 18.000 -
557.xz_r (avg) 18.000 -
557.xz_r (avg) (percentage) 0.160 -
557.xz_r (max) 34.000 -2.86%
557.xz_r (max) (percentage) 0.300 -3.23%
Geometric Mean 5.365 -25.68%