Fields

hardware None
os None

Parameters

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/f4c781174b4b8fd958d9074c25eb06b6e276cb1c (+ 177 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/230/
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0230 (llvmorg-19-init-16703-gf4c781174b4b) 2024-06-27T13:23:30 0:00:00
Previous 0229 (llvmorg-19-init-16526-gf58930f70588) 2024-06-18T01:00:22 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00


Tests Summary

Status Group # # (B)
Performance Regressions 19 63
Performance Improvements 37 59
Added Tests 0 12
Unchanged Tests 868 790
Total Tests 924

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
521.wrf_r 1.46% 13263.000 13457.000 - -8.74% -
541.leela_r 1.13% 13415.000 13567.000 - -0.03% -
500.perlbench_r 1.11% 15914.000 16090.000 - 0.24% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) (percentage) 152.50% 0.400 1.010 0.000 206.06% 0.000
520.omnetpp_r (unexplained diff vs previous) 150.85% 59.000 148.000 0.000 208.33% 0.000
519.lbm_r (unexplained diff vs previous) (percentage) 3.13% 0.320 0.330 0.000 135.71% 0.000
554.roms_r (max) 3.03% 33.000 34.000 - 112.50% -
554.roms_r 2.56% 39.000 40.000 - 207.69% -
554.roms_r (avg) 2.56% 39.000 40.000 - 207.69% -
519.lbm_r (unexplained diff vs previous) 2.00% 50.000 51.000 0.000 112.50% 0.000
541.leela_r (max) 2.00% 50.000 51.000 - -31.08% -
505.mcf_r 1.92% 52.000 53.000 - -27.40% -
505.mcf_r (avg) 1.92% 52.000 53.000 - -27.40% -
500.perlbench_r 1.41% 71.000 72.000 - -50.68% -
500.perlbench_r (avg) 1.41% 71.000 72.000 - -50.68% -
511.povray_r 1.15% 87.000 88.000 - 14.29% -
511.povray_r (avg) 1.15% 87.000 88.000 - 14.29% -
521.wrf_r 1.14% 176.000 178.000 - 54.78% -
521.wrf_r (avg) 1.14% 176.000 178.000 - 54.78% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
549.fotonik3d_r -1.56% 19250.000 18950.000 - -39.58% -
520.omnetpp_r -1.00% 14761.000 14613.000 - -0.33% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (max) -9.39% 245.000 222.000 - -39.34% -
520.omnetpp_r (max) (percentage) -8.43% 1.660 1.520 - -39.20% -
538.imagick_r (max) -7.50% 40.000 37.000 - -36.21% -
549.fotonik3d_r (max) -7.12% 379.000 352.000 - -7.12% -
538.imagick_r (max) (percentage) -6.90% 0.290 0.270 - -35.71% -
508.namd_r (max) (percentage) -6.67% 0.300 0.280 - -75.22% -
500.perlbench_r (max) (percentage) -5.71% 0.350 0.330 - -75.19% -
549.fotonik3d_r (max) (percentage) -5.58% 1.970 1.860 - 53.72% -
557.xz_r (max) (percentage) -5.56% 0.720 0.680 - 257.89% -
510.parest_r (max) -5.26% 57.000 54.000 - 25.58% -
544.nab_r (max) (percentage) -5.26% 0.570 0.540 - 107.69% -
557.xz_r (max) -5.00% 80.000 76.000 - 261.90% -
544.nab_r (max) -4.94% 81.000 77.000 - 108.11% -
531.deepsjeng_r (max) -4.65% 43.000 41.000 - -38.81% -
510.parest_r (max) (percentage) -4.00% 0.250 0.240 - 26.32% -
531.deepsjeng_r (max) (percentage) -4.00% 0.500 0.480 - -37.66% -
505.mcf_r (max) (percentage) -3.85% 0.780 0.750 - 10.29% -
500.perlbench_r (max) -3.64% 55.000 53.000 - -75.12% -
505.mcf_r (max) -3.60% 111.000 107.000 - 11.46% -
548.exchange2_r (max) (percentage) -3.57% 0.280 0.270 - 22.73% -
508.namd_r (max) -3.57% 28.000 27.000 - -75.00% -
523.xalancbmk_r (max) (percentage) -3.23% 0.310 0.300 - -21.05% -
519.lbm_r -3.17% 63.000 61.000 - 110.34% -
519.lbm_r (avg) -3.17% 63.000 61.000 - 110.34% -
526.blender_r (max) -3.05% 394.000 382.000 - 768.18% -
511.povray_r (max) (percentage) -2.60% 0.770 0.750 - 36.36% -
526.blender_r (max) (percentage) -2.44% 3.280 3.200 - 788.89% -
527.cam4_r (max) -2.07% 193.000 189.000 - 0.00% -
511.povray_r (max) -1.96% 153.000 150.000 - 27.12% -
549.fotonik3d_r -1.69% 177.000 174.000 - -28.69% -
549.fotonik3d_r (avg) -1.69% 177.000 174.000 - -28.69% -
521.wrf_r (max) (percentage) -1.52% 1.970 1.940 - 162.16% -
527.cam4_r (max) (percentage) -1.41% 1.420 1.400 - 0.00% -
527.cam4_r -1.27% 79.000 78.000 - 0.00% -
527.cam4_r (avg) -1.27% 79.000 78.000 - 0.00% -

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
503.bwaves_r 8.63% 775742.000 842676.000 - 0.00% -
554.roms_r 4.57% 3060081.000 3199872.000 - 0.00% -
520.omnetpp_r 1.04% 2149519.000 2171961.000 - -0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 1.57% 11293.000 11470.000 - -0.58% -
508.namd_r 1.30% 9575.000 9699.000 - 0.58% -
505.mcf_r 1.26% 14148.000 14326.000 - 0.63% -
538.imagick_r 1.12% 13884.000 14039.000 - -0.36% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
526.blender_r (max) (percentage) 788.89% 0.360 3.200 - -2.44% -
526.blender_r (max) 768.18% 44.000 382.000 - -3.05% -
557.xz_r (max) 261.90% 21.000 76.000 - -5.00% -
557.xz_r (max) (percentage) 257.89% 0.190 0.680 - -5.56% -
554.roms_r (avg) (percentage) 233.33% 0.060 0.200 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) 208.33% 48.000 148.000 0.000 150.85% 0.000
554.roms_r 207.69% 13.000 40.000 - 2.56% -
554.roms_r (avg) 207.69% 13.000 40.000 - 2.56% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 206.06% 0.330 1.010 0.000 152.50% 0.000
526.blender_r (avg) (percentage) 200.00% 0.330 0.990 - 0.00% -
526.blender_r 187.80% 41.000 118.000 - -0.84% -
526.blender_r (avg) 187.80% 41.000 118.000 - -0.84% -
521.wrf_r (max) (percentage) 162.16% 0.740 1.940 - -1.52% -
554.roms_r (max) (percentage) 142.86% 0.070 0.170 - 0.00% -
521.wrf_r (max) 139.45% 109.000 261.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) 135.71% 0.140 0.330 0.000 3.13% 0.000
519.lbm_r (avg) (percentage) 135.29% 0.170 0.400 - -2.44% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) 112.50% 24.000 51.000 0.000 2.00% 0.000
554.roms_r (max) 112.50% 16.000 34.000 - 3.03% -
519.lbm_r 110.34% 29.000 61.000 - -3.17% -
519.lbm_r (avg) 110.34% 29.000 61.000 - -3.17% -
544.nab_r (max) 108.11% 37.000 77.000 - -4.94% -
544.nab_r (max) (percentage) 107.69% 0.260 0.540 - -5.26% -
510.parest_r 94.12% 34.000 66.000 - 0.00% -
510.parest_r (avg) 94.12% 34.000 66.000 - 0.00% -
519.lbm_r (max) (percentage) 94.12% 0.170 0.330 - 0.00% -
510.parest_r (avg) (percentage) 93.33% 0.150 0.290 - 0.00% -
557.xz_r 81.82% 11.000 20.000 - 0.00% -
557.xz_r (avg) 81.82% 11.000 20.000 - 0.00% -
557.xz_r (avg) (percentage) 80.00% 0.100 0.180 - 0.00% -
519.lbm_r (max) 72.41% 29.000 50.000 - 0.00% -
521.wrf_r (avg) (percentage) 70.51% 0.780 1.330 - 0.00% -
521.wrf_r 54.78% 115.000 178.000 - 1.14% -
521.wrf_r (avg) 54.78% 115.000 178.000 - 1.14% -
549.fotonik3d_r (max) (percentage) 53.72% 1.210 1.860 - -5.58% -
511.povray_r (max) (percentage) 36.36% 0.550 0.750 - -2.60% -
511.povray_r (max) 27.12% 118.000 150.000 - -1.96% -
510.parest_r (max) (percentage) 26.32% 0.190 0.240 - -4.00% -
510.parest_r (max) 25.58% 43.000 54.000 - -5.26% -
548.exchange2_r (max) (percentage) 22.73% 0.220 0.270 - -3.57% -
511.povray_r (avg) (percentage) 22.22% 0.360 0.440 - 0.00% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
520.omnetpp_r 18.66% 134.000 159.000 - -0.62% -
520.omnetpp_r (avg) 18.66% 134.000 159.000 - -0.62% -
520.omnetpp_r (avg) (percentage) 18.48% 0.920 1.090 - 0.00% -
549.fotonik3d_r (avg) (percentage) 17.95% 0.780 0.920 - 0.00% -
531.deepsjeng_r 16.67% 36.000 42.000 - 0.00% -
531.deepsjeng_r (avg) 16.67% 36.000 42.000 - 0.00% -
511.povray_r 14.29% 77.000 88.000 - 1.15% -
511.povray_r (avg) 14.29% 77.000 88.000 - 1.15% -
505.mcf_r (max) 11.46% 96.000 107.000 - -3.60% -
505.mcf_r (max) (percentage) 10.29% 0.680 0.750 - -3.85% -
523.xalancbmk_r (avg) (percentage) 4.17% 0.240 0.250 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
521.wrf_r -7.53% 23893420.000 22095159.000 - -0.00% -
508.namd_r -1.85% 624815.000 613249.000 - -0.03% -
519.lbm_r -1.52% 13451.000 13247.000 - 0.00% -
507.cactuBSSN_r -1.23% 3707734.000 3661994.000 - -0.02% -
549.fotonik3d_r -1.14% 1256502.000 1242171.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
549.fotonik3d_r -39.58% 31362.000 18950.000 - -1.56% -
525.x264_r -16.68% 7490.000 6241.000 - 0.10% -
554.roms_r -12.23% 22927.000 20123.000 - 0.99% -
519.lbm_r -11.86% 17526.000 15448.000 - 0.33% -
521.wrf_r -8.74% 14746.000 13457.000 - 1.46% -
511.povray_r -7.11% 21547.000 20015.000 - 0.72% -
526.blender_r -3.80% 12432.000 11960.000 - -0.53% -
557.xz_r -2.95% 11556.000 11215.000 - 0.10% -
503.bwaves_r -1.69% 21135.000 20778.000 - 0.03% -
531.deepsjeng_r -1.60% 8789.000 8648.000 - -0.07% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
508.namd_r (max) (percentage) -75.22% 1.130 0.280 - -6.67% -
500.perlbench_r (max) (percentage) -75.19% 1.330 0.330 - -5.71% -
500.perlbench_r (max) -75.12% 213.000 53.000 - -3.64% -
508.namd_r (max) -75.00% 108.000 27.000 - -3.57% -
525.x264_r (max) -63.64% 11.000 4.000 - 0.00% -
508.namd_r (avg) (percentage) -60.94% 0.640 0.250 - 0.00% -
508.namd_r -60.66% 61.000 24.000 - 0.00% -
508.namd_r (avg) -60.66% 61.000 24.000 - 0.00% -
503.bwaves_r (max) -57.89% 19.000 8.000 - 0.00% -
503.bwaves_r (unexplained diff vs previous) (percentage) -57.14% 0.070 0.030 0.000 0.00% 0.000
503.bwaves_r (unexplained diff vs previous) -57.14% 14.000 6.000 0.000 0.00% 0.000
503.bwaves_r (max) (percentage) -55.56% 0.090 0.040 - 0.00% -
500.perlbench_r -50.68% 146.000 72.000 - 1.41% -
500.perlbench_r (avg) -50.68% 146.000 72.000 - 1.41% -
500.perlbench_r (avg) (percentage) -50.55% 0.910 0.450 - 0.00% -
525.x264_r (max) (percentage) -46.67% 0.150 0.080 - 0.00% -
538.imagick_r -43.90% 41.000 23.000 - 0.00% -
538.imagick_r (avg) -43.90% 41.000 23.000 - 0.00% -
538.imagick_r (avg) (percentage) -43.33% 0.300 0.170 - 0.00% -
503.bwaves_r -42.86% 21.000 12.000 - 0.00% -
503.bwaves_r (avg) -42.86% 21.000 12.000 - 0.00% -
503.bwaves_r (avg) (percentage) -40.00% 0.100 0.060 - 0.00% -
520.omnetpp_r (max) -39.34% 366.000 222.000 - -9.39% -
520.omnetpp_r (max) (percentage) -39.20% 2.500 1.520 - -8.43% -
531.deepsjeng_r (max) -38.81% 67.000 41.000 - -4.65% -
531.deepsjeng_r (max) (percentage) -37.66% 0.770 0.480 - -4.00% -
538.imagick_r (max) -36.21% 58.000 37.000 - -7.50% -
538.imagick_r (max) (percentage) -35.71% 0.420 0.270 - -6.90% -
541.leela_r (max) -31.08% 74.000 51.000 - 2.00% -
541.leela_r (max) (percentage) -30.91% 0.550 0.380 - 0.00% -
505.mcf_r (avg) (percentage) -28.85% 0.520 0.370 - 0.00% -
549.fotonik3d_r -28.69% 244.000 174.000 - -1.69% -
549.fotonik3d_r (avg) -28.69% 244.000 174.000 - -1.69% -
505.mcf_r -27.40% 73.000 53.000 - 1.92% -
505.mcf_r (avg) -27.40% 73.000 53.000 - 1.92% -
523.xalancbmk_r (max) (percentage) -21.05% 0.380 0.300 - -3.23% -
548.exchange2_r (avg) (percentage) -13.33% 0.150 0.130 - 0.00% -
549.fotonik3d_r (max) -7.12% 379.000 352.000 - -7.12% -
541.leela_r -5.08% 59.000 56.000 - 0.00% -
541.leela_r (avg) -5.08% 59.000 56.000 - 0.00% -
541.leela_r (avg) (percentage) -4.55% 0.440 0.420 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 100.00% 23.000 46.000 - 0.00% -
nb_compile_successful 90.91% 22.000 42.000 - 0.00% -
nb_execution_successful 90.91% 22.000 42.000 - 0.00% -

Added Tests - Code Size
527.cam4_r

Added Tests - Execution
527.cam4_r

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)
527.cam4_r
527.cam4_r (avg)
527.cam4_r (avg) (percentage)
527.cam4_r (max)
527.cam4_r (max) (percentage)


Report Time: 0.34s
Name Current %
500.perlbench_r 16090.000 1.11%
502.gcc_r 10941.000 -
503.bwaves_r 20778.000 -
505.mcf_r 14326.000 -
507.cactuBSSN_r 11470.000 -
508.namd_r 9699.000 -
510.parest_r 22827.000 -
511.povray_r 20015.000 -
519.lbm_r 15448.000 -
520.omnetpp_r 14613.000 -1.00%
521.wrf_r 13457.000 1.46%
525.x264_r 6241.000 -
526.blender_r 11960.000 -
527.cam4_r 13571.000 -
531.deepsjeng_r 8648.000 -
538.imagick_r 14039.000 -
541.leela_r 13567.000 1.13%
544.nab_r 14325.000 -
549.fotonik3d_r 18950.000 -1.56%
554.roms_r 20123.000 -
557.xz_r 11215.000 -
Geometric Mean 13769.868 -
Name Current %
500.perlbench_r 2142381.000 -
502.gcc_r 8908872.000 -
503.bwaves_r 842676.000 -
505.mcf_r 27063.000 -
507.cactuBSSN_r 3661994.000 -
508.namd_r 613249.000 -
510.parest_r 8687416.000 -
511.povray_r 910709.000 -
519.lbm_r 13247.000 -
520.omnetpp_r 2171961.000 -
521.wrf_r 22095159.000 -
525.x264_r 496138.000 -
526.blender_r 13663261.000 -
527.cam4_r 9070069.000 -
531.deepsjeng_r 80922.000 -
538.imagick_r 1692345.000 -
541.leela_r 143473.000 -
544.nab_r 190832.000 -
549.fotonik3d_r 1242171.000 -
554.roms_r 3199872.000 -
557.xz_r 176880.000 -
Geometric Mean 989245.361 -
Name Current %
500.perlbench_r 72.000 1.41%
500.perlbench_r (avg) 72.000 1.41%
500.perlbench_r (avg) (percentage) 0.450 -
500.perlbench_r (max) 53.000 -3.64%
500.perlbench_r (max) (percentage) 0.330 -5.71%
503.bwaves_r 12.000 -
503.bwaves_r (avg) 12.000 -
503.bwaves_r (avg) (percentage) 0.060 -
503.bwaves_r (max) 8.000 -
503.bwaves_r (max) (percentage) 0.040 -
503.bwaves_r (unexplained diff vs previous) 6.000 -
503.bwaves_r (unexplained diff vs previous) (percentage) 0.030 -
505.mcf_r 53.000 1.92%
505.mcf_r (avg) 53.000 1.92%
505.mcf_r (avg) (percentage) 0.370 -
505.mcf_r (max) 107.000 -3.60%
505.mcf_r (max) (percentage) 0.750 -3.85%
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 24.000 -
508.namd_r (avg) 24.000 -
508.namd_r (avg) (percentage) 0.250 -
508.namd_r (max) 27.000 -3.57%
508.namd_r (max) (percentage) 0.280 -6.67%
510.parest_r 66.000 -
510.parest_r (avg) 66.000 -
510.parest_r (avg) (percentage) 0.290 -
510.parest_r (max) 54.000 -5.26%
510.parest_r (max) (percentage) 0.240 -4.00%
511.povray_r 88.000 1.15%
511.povray_r (avg) 88.000 1.15%
511.povray_r (avg) (percentage) 0.440 -
511.povray_r (max) 150.000 -1.96%
511.povray_r (max) (percentage) 0.750 -2.60%
519.lbm_r 61.000 -3.17%
519.lbm_r (avg) 61.000 -3.17%
519.lbm_r (avg) (percentage) 0.400 -
519.lbm_r (max) 50.000 -
519.lbm_r (max) (percentage) 0.330 -
519.lbm_r (unexplained diff vs previous) 51.000 2.00%
519.lbm_r (unexplained diff vs previous) (percentage) 0.330 3.13%
520.omnetpp_r 159.000 -
520.omnetpp_r (avg) 159.000 -
520.omnetpp_r (avg) (percentage) 1.090 -
520.omnetpp_r (max) 222.000 -9.39%
520.omnetpp_r (max) (percentage) 1.520 -8.43%
520.omnetpp_r (unexplained diff vs previous) 148.000 150.85%
520.omnetpp_r (unexplained diff vs previous) (percentage) 1.010 152.50%
521.wrf_r 178.000 1.14%
521.wrf_r (avg) 178.000 1.14%
521.wrf_r (avg) (percentage) 1.330 -
521.wrf_r (max) 261.000 -
521.wrf_r (max) (percentage) 1.940 -1.52%
523.xalancbmk_r (avg) (percentage) 0.250 -
523.xalancbmk_r (max) (percentage) 0.300 -3.23%
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 4.000 -
525.x264_r (max) (percentage) 0.080 -
526.blender_r 118.000 -
526.blender_r (avg) 118.000 -
526.blender_r (avg) (percentage) 0.990 -
526.blender_r (max) 382.000 -3.05%
526.blender_r (max) (percentage) 3.200 -2.44%
527.cam4_r 78.000 -1.27%
527.cam4_r (avg) 78.000 -1.27%
527.cam4_r (avg) (percentage) 0.580 -
527.cam4_r (max) 189.000 -2.07%
527.cam4_r (max) (percentage) 1.400 -1.41%
531.deepsjeng_r 42.000 -
531.deepsjeng_r (avg) 42.000 -
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 41.000 -4.65%
531.deepsjeng_r (max) (percentage) 0.480 -4.00%
538.imagick_r 23.000 -
538.imagick_r (avg) 23.000 -
538.imagick_r (avg) (percentage) 0.170 -
538.imagick_r (max) 37.000 -7.50%
538.imagick_r (max) (percentage) 0.270 -6.90%
541.leela_r 56.000 -
541.leela_r (avg) 56.000 -
541.leela_r (avg) (percentage) 0.420 -
541.leela_r (max) 51.000 2.00%
541.leela_r (max) (percentage) 0.380 -
544.nab_r 41.000 -
544.nab_r (avg) 41.000 -
544.nab_r (avg) (percentage) 0.290 -
544.nab_r (max) 77.000 -4.94%
544.nab_r (max) (percentage) 0.540 -5.26%
548.exchange2_r (avg) (percentage) 0.130 -
548.exchange2_r (max) (percentage) 0.270 -3.57%
549.fotonik3d_r 174.000 -1.69%
549.fotonik3d_r (avg) 174.000 -1.69%
549.fotonik3d_r (avg) (percentage) 0.920 -
549.fotonik3d_r (max) 352.000 -7.12%
549.fotonik3d_r (max) (percentage) 1.860 -5.58%
554.roms_r 40.000 2.56%
554.roms_r (avg) 40.000 2.56%
554.roms_r (avg) (percentage) 0.200 -
554.roms_r (max) 34.000 3.03%
554.roms_r (max) (percentage) 0.170 -
557.xz_r 20.000 -
557.xz_r (avg) 20.000 -
557.xz_r (avg) (percentage) 0.180 -
557.xz_r (max) 76.000 -5.00%
557.xz_r (max) (percentage) 0.680 -5.56%
Geometric Mean 5.546 17.01%