Fields

hardware None
os None

Parameters

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/2438f41bc45fc5e86bd4e9f77fae198719eed313 (+ 2343 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/242/
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0242 (llvmorg-20-init-02065-g2438f41bc45f) 2024-07-30T14:45:52 0:00:00
Previous 0239 (no-change) 2024-07-18T10:01:44 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00


Tests Summary

Status Group # # (B)
Performance Regressions 18 61
Performance Improvements 39 61
Added Tests 0 12
Unchanged Tests 867 790
Total Tests 924

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
525.x264_r 2.83% 6212.000 6388.000 - -14.71% -
511.povray_r 2.22% 19549.000 19983.000 - -7.26% -
549.fotonik3d_r 1.12% 18892.000 19103.000 - -39.09% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (unexplained diff vs previous) 45.45% 44.000 64.000 0.000 166.67% 0.000
519.lbm_r (unexplained diff vs previous) (percentage) 44.83% 0.290 0.420 0.000 200.00% 0.000
519.lbm_r (max) 8.47% 59.000 64.000 - 120.69% -
519.lbm_r (max) (percentage) 7.69% 0.390 0.420 - 147.06% -
519.lbm_r 3.39% 59.000 61.000 - 110.34% -
519.lbm_r (avg) 3.39% 59.000 61.000 - 110.34% -
519.lbm_r (avg) (percentage) 2.56% 0.390 0.400 - 135.29% -
554.roms_r 2.44% 41.000 42.000 - 223.08% -
554.roms_r (avg) 2.44% 41.000 42.000 - 223.08% -
500.perlbench_r 1.49% 67.000 68.000 - -53.42% -
500.perlbench_r (avg) 1.49% 67.000 68.000 - -53.42% -
511.povray_r 1.16% 86.000 87.000 - 12.99% -
511.povray_r (avg) 1.16% 86.000 87.000 - 12.99% -
549.fotonik3d_r 1.08% 185.000 187.000 - -23.36% -
549.fotonik3d_r (avg) 1.08% 185.000 187.000 - -23.36% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
505.mcf_r -4.07% 14552.000 13960.000 - -1.33% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) (percentage) -60.34% 0.580 0.230 0.000 -30.30% 0.000
520.omnetpp_r (unexplained diff vs previous) -59.52% 84.000 34.000 0.000 -29.17% 0.000
520.omnetpp_r (max) -10.00% 200.000 180.000 - -50.82% -
520.omnetpp_r (max) (percentage) -9.42% 1.380 1.250 - -50.00% -
505.mcf_r (max) -8.16% 282.000 259.000 - 169.79% -
557.xz_r (max) -7.69% 13.000 12.000 - -42.86% -
541.leela_r (max) -7.14% 56.000 52.000 - -29.73% -
541.leela_r (max) (percentage) -6.98% 0.430 0.400 - -27.27% -
531.deepsjeng_r (max) (percentage) -5.66% 1.060 1.000 - 29.87% -
508.namd_r (max) -5.61% 196.000 185.000 - 71.30% -
531.deepsjeng_r (max) -5.56% 90.000 85.000 - 26.87% -
508.namd_r (max) (percentage) -5.39% 2.040 1.930 - 70.80% -
549.fotonik3d_r (max) (percentage) -4.15% 1.930 1.850 - 52.89% -
505.mcf_r (max) (percentage) -4.12% 1.940 1.860 - 173.53% -
548.exchange2_r (max) (percentage) -3.85% 0.260 0.250 - 13.64% -
523.xalancbmk_r (max) (percentage) -3.70% 0.270 0.260 - -31.58% -
510.parest_r (max) (percentage) -3.57% 0.280 0.270 - 42.11% -
505.mcf_r -3.39% 59.000 57.000 - -21.92% -
505.mcf_r (avg) -3.39% 59.000 57.000 - -21.92% -
500.perlbench_r (max) (percentage) -3.33% 0.300 0.290 - -78.20% -
510.parest_r (max) -3.17% 63.000 61.000 - 41.86% -
549.fotonik3d_r (max) -3.02% 364.000 353.000 - -6.86% -
503.bwaves_r (max) (percentage) -2.94% 0.340 0.330 - 266.67% -
503.bwaves_r (max) -2.86% 70.000 68.000 - 257.89% -
554.roms_r (max) (percentage) -2.27% 0.440 0.430 - 514.29% -
544.nab_r (max) (percentage) -2.22% 0.450 0.440 - 69.23% -
500.perlbench_r (max) -2.13% 47.000 46.000 - -78.40% -
520.omnetpp_r -1.92% 156.000 153.000 - 14.18% -
520.omnetpp_r (avg) -1.92% 156.000 153.000 - 14.18% -
520.omnetpp_r (avg) (percentage) -1.85% 1.080 1.060 - 15.22% -
527.cam4_r (max) -1.67% 180.000 177.000 - 0.00% -
544.nab_r (max) -1.59% 63.000 62.000 - 67.57% -
527.cam4_r (max) (percentage) -1.50% 1.330 1.310 - 0.00% -
511.povray_r (max) (percentage) -1.43% 0.700 0.690 - 25.45% -
526.blender_r (max) -1.40% 358.000 353.000 - 702.27% -
526.blender_r (max) (percentage) -1.34% 2.990 2.950 - 719.44% -
554.roms_r (max) -1.15% 87.000 86.000 - 437.50% -
521.wrf_r (max) (percentage) -1.08% 1.850 1.830 - 147.30% -

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r 1.01% 2149519.000 2171257.000 - -0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 1.97% 11293.000 11516.000 - 0.10% -
538.imagick_r 1.24% 13884.000 14056.000 - -0.02% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
526.blender_r (max) (percentage) 719.44% 0.360 2.950 - -1.34% -
526.blender_r (max) 702.27% 44.000 353.000 - -1.40% -
554.roms_r (max) (percentage) 514.29% 0.070 0.430 - -2.27% -
554.roms_r (max) 437.50% 16.000 86.000 - -1.15% -
503.bwaves_r (max) (percentage) 266.67% 0.090 0.330 - -2.94% -
503.bwaves_r (max) 257.89% 19.000 68.000 - -2.86% -
554.roms_r (avg) (percentage) 250.00% 0.060 0.210 - 0.00% -
554.roms_r 223.08% 13.000 42.000 - 2.44% -
554.roms_r (avg) 223.08% 13.000 42.000 - 2.44% -
519.lbm_r (unexplained diff vs previous) (percentage) 200.00% 0.140 0.420 0.000 44.83% 0.000
526.blender_r (avg) (percentage) 187.88% 0.330 0.950 - 0.00% -
526.blender_r 175.61% 41.000 113.000 - 0.00% -
526.blender_r (avg) 175.61% 41.000 113.000 - 0.00% -
505.mcf_r (max) (percentage) 173.53% 0.680 1.860 - -4.12% -
505.mcf_r (max) 169.79% 96.000 259.000 - -8.16% -
519.lbm_r (unexplained diff vs previous) 166.67% 24.000 64.000 0.000 45.45% 0.000
521.wrf_r (max) (percentage) 147.30% 0.740 1.830 - -1.08% -
519.lbm_r (max) (percentage) 147.06% 0.170 0.420 - 7.69% -
519.lbm_r (avg) (percentage) 135.29% 0.170 0.400 - 2.56% -
521.wrf_r (max) 123.85% 109.000 244.000 - -0.41% -
525.x264_r (avg) (percentage) 122.22% 0.090 0.200 - 0.00% -
519.lbm_r (max) 120.69% 29.000 64.000 - 8.47% -
519.lbm_r 110.34% 29.000 61.000 - 3.39% -
519.lbm_r (avg) 110.34% 29.000 61.000 - 3.39% -
525.x264_r 100.00% 6.000 12.000 - 0.00% -
525.x264_r (avg) 100.00% 6.000 12.000 - 0.00% -
510.parest_r (avg) (percentage) 86.67% 0.150 0.280 - 0.00% -
510.parest_r 85.29% 34.000 63.000 - 0.00% -
510.parest_r (avg) 85.29% 34.000 63.000 - 0.00% -
557.xz_r 72.73% 11.000 19.000 - 0.00% -
557.xz_r (avg) 72.73% 11.000 19.000 - 0.00% -
508.namd_r (max) 71.30% 108.000 185.000 - -5.61% -
508.namd_r (max) (percentage) 70.80% 1.130 1.930 - -5.39% -
521.wrf_r (avg) (percentage) 70.51% 0.780 1.330 - 0.00% -
557.xz_r (avg) (percentage) 70.00% 0.100 0.170 - 0.00% -
544.nab_r (max) (percentage) 69.23% 0.260 0.440 - -2.22% -
544.nab_r (max) 67.57% 37.000 62.000 - -1.59% -
521.wrf_r 53.91% 115.000 177.000 - 0.57% -
521.wrf_r (avg) 53.91% 115.000 177.000 - 0.57% -
549.fotonik3d_r (max) (percentage) 52.89% 1.210 1.850 - -4.15% -
510.parest_r (max) (percentage) 42.11% 0.190 0.270 - -3.57% -
510.parest_r (max) 41.86% 43.000 61.000 - -3.17% -
531.deepsjeng_r (max) (percentage) 29.87% 0.770 1.000 - -5.66% -
531.deepsjeng_r (max) 26.87% 67.000 85.000 - -5.56% -
549.fotonik3d_r (avg) (percentage) 25.64% 0.780 0.980 - 0.00% -
511.povray_r (max) (percentage) 25.45% 0.550 0.690 - -1.43% -
511.povray_r (avg) (percentage) 22.22% 0.360 0.440 - 0.00% -
531.deepsjeng_r (avg) (percentage) 21.95% 0.410 0.500 - 0.00% -
531.deepsjeng_r 16.67% 36.000 42.000 - 0.00% -
531.deepsjeng_r (avg) 16.67% 36.000 42.000 - 0.00% -
511.povray_r (max) 16.10% 118.000 137.000 - 0.74% -
520.omnetpp_r (avg) (percentage) 15.22% 0.920 1.060 - -1.85% -
520.omnetpp_r 14.18% 134.000 153.000 - -1.92% -
520.omnetpp_r (avg) 14.18% 134.000 153.000 - -1.92% -
548.exchange2_r (max) (percentage) 13.64% 0.220 0.250 - -3.85% -
511.povray_r 12.99% 77.000 87.000 - 1.16% -
511.povray_r (avg) 12.99% 77.000 87.000 - 1.16% -
523.xalancbmk_r (avg) (percentage) 4.17% 0.240 0.250 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
549.fotonik3d_r -10.71% 1256502.000 1121872.000 - 0.21% -
521.wrf_r -8.21% 23893420.000 21930679.000 - -0.04% -
503.bwaves_r -5.47% 775742.000 733301.000 - -0.03% -
554.roms_r -3.59% 3060081.000 2950309.000 - -0.62% -
508.namd_r -1.88% 624815.000 613081.000 - 0.01% -
519.lbm_r -1.52% 13451.000 13247.000 - 0.00% -
507.cactuBSSN_r -1.24% 3707734.000 3661654.000 - -0.01% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
549.fotonik3d_r -39.09% 31362.000 19103.000 - 1.12% -
525.x264_r -14.71% 7490.000 6388.000 - 2.83% -
554.roms_r -12.64% 22927.000 20028.000 - 0.22% -
519.lbm_r -12.05% 17526.000 15414.000 - 0.42% -
521.wrf_r -9.43% 14746.000 13355.000 - 0.45% -
511.povray_r -7.26% 21547.000 19983.000 - 2.22% -
526.blender_r -3.73% 12432.000 11968.000 - -0.17% -
541.leela_r -3.63% 13571.000 13078.000 - -0.61% -
531.deepsjeng_r -2.32% 8789.000 8585.000 - 0.36% -
503.bwaves_r -1.55% 21135.000 20808.000 - 0.17% -
502.gcc_r -1.49% 11030.000 10866.000 - -0.10% -
505.mcf_r -1.33% 14148.000 13960.000 - -4.07% -
520.omnetpp_r -1.30% 14662.000 14472.000 - -0.23% -
557.xz_r -1.13% 11556.000 11425.000 - -0.25% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
500.perlbench_r (max) -78.40% 213.000 46.000 - -2.13% -
500.perlbench_r (max) (percentage) -78.20% 1.330 0.290 - -3.33% -
500.perlbench_r -53.42% 146.000 68.000 - 1.49% -
500.perlbench_r (avg) -53.42% 146.000 68.000 - 1.49% -
500.perlbench_r (avg) (percentage) -52.75% 0.910 0.430 - 0.00% -
508.namd_r -52.46% 61.000 29.000 - 0.00% -
508.namd_r (avg) -52.46% 61.000 29.000 - 0.00% -
508.namd_r (avg) (percentage) -51.56% 0.640 0.310 - 0.00% -
520.omnetpp_r (max) -50.82% 366.000 180.000 - -10.00% -
520.omnetpp_r (max) (percentage) -50.00% 2.500 1.250 - -9.42% -
538.imagick_r (avg) (percentage) -46.67% 0.300 0.160 - 0.00% -
538.imagick_r -46.34% 41.000 22.000 - 0.00% -
538.imagick_r (avg) -46.34% 41.000 22.000 - 0.00% -
538.imagick_r (max) (percentage) -45.24% 0.420 0.230 - 0.00% -
538.imagick_r (max) -44.83% 58.000 32.000 - 0.00% -
557.xz_r (max) -42.86% 21.000 12.000 - -7.69% -
557.xz_r (max) (percentage) -42.11% 0.190 0.110 - -8.33% -
525.x264_r (max) -36.36% 11.000 7.000 - 0.00% -
503.bwaves_r -33.33% 21.000 14.000 - 0.00% -
503.bwaves_r (avg) -33.33% 21.000 14.000 - 0.00% -
523.xalancbmk_r (max) (percentage) -31.58% 0.380 0.260 - -3.70% -
520.omnetpp_r (unexplained diff vs previous) (percentage) -30.30% 0.330 0.230 0.000 -60.34% 0.000
503.bwaves_r (avg) (percentage) -30.00% 0.100 0.070 - 0.00% -
541.leela_r (max) -29.73% 74.000 52.000 - -7.14% -
520.omnetpp_r (unexplained diff vs previous) -29.17% 48.000 34.000 0.000 -59.52% 0.000
541.leela_r (max) (percentage) -27.27% 0.550 0.400 - -6.98% -
549.fotonik3d_r -23.36% 244.000 187.000 - 1.08% -
549.fotonik3d_r (avg) -23.36% 244.000 187.000 - 1.08% -
505.mcf_r -21.92% 73.000 57.000 - -3.39% -
505.mcf_r (avg) -21.92% 73.000 57.000 - -3.39% -
505.mcf_r (avg) (percentage) -21.15% 0.520 0.410 - 0.00% -
525.x264_r (max) (percentage) -20.00% 0.150 0.120 - 0.00% -
548.exchange2_r (avg) (percentage) -13.33% 0.150 0.130 - 0.00% -
541.leela_r -10.17% 59.000 53.000 - 0.00% -
541.leela_r (avg) -10.17% 59.000 53.000 - 0.00% -
549.fotonik3d_r (max) -6.86% 379.000 353.000 - -3.02% -
541.leela_r (avg) (percentage) -6.82% 0.440 0.410 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 100.00% 23.000 46.000 - 0.00% -
nb_compile_successful 90.91% 22.000 42.000 - 0.00% -
nb_execution_successful 90.91% 22.000 42.000 - 0.00% -

Added Tests - Code Size
527.cam4_r

Added Tests - Execution
527.cam4_r

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)
527.cam4_r
527.cam4_r (avg)
527.cam4_r (avg) (percentage)
527.cam4_r (max)
527.cam4_r (max) (percentage)


Report Time: 0.36s
Name Current %
500.perlbench_r 15913.000 -
502.gcc_r 10866.000 -
503.bwaves_r 20808.000 -
505.mcf_r 13960.000 -4.07%
507.cactuBSSN_r 11516.000 -
508.namd_r 9636.000 -
510.parest_r 22826.000 -
511.povray_r 19983.000 2.22%
519.lbm_r 15414.000 -
520.omnetpp_r 14472.000 -
521.wrf_r 13355.000 -
525.x264_r 6388.000 2.83%
526.blender_r 11968.000 -
527.cam4_r 13558.000 -
531.deepsjeng_r 8585.000 -
538.imagick_r 14056.000 -
541.leela_r 13078.000 -
544.nab_r 14231.000 -
549.fotonik3d_r 19103.000 1.12%
554.roms_r 20028.000 -
557.xz_r 11425.000 -
Geometric Mean 13723.662 -
Name Current %
500.perlbench_r 2143901.000 -
502.gcc_r 8913924.000 -
503.bwaves_r 733301.000 -
505.mcf_r 27031.000 -
507.cactuBSSN_r 3661654.000 -
508.namd_r 613081.000 -
510.parest_r 8694496.000 -
511.povray_r 911385.000 -
519.lbm_r 13247.000 -
520.omnetpp_r 2171257.000 -
521.wrf_r 21930679.000 -
525.x264_r 495866.000 -
526.blender_r 13661669.000 -
527.cam4_r 8888002.000 -
531.deepsjeng_r 80802.000 -
538.imagick_r 1692045.000 -
541.leela_r 143421.000 -
544.nab_r 190696.000 -
549.fotonik3d_r 1121872.000 -
554.roms_r 2950309.000 -
557.xz_r 176904.000 -
Geometric Mean 972795.157 -
Name Current %
500.perlbench_r 68.000 1.49%
500.perlbench_r (avg) 68.000 1.49%
500.perlbench_r (avg) (percentage) 0.430 -
500.perlbench_r (max) 46.000 -2.13%
500.perlbench_r (max) (percentage) 0.290 -3.33%
503.bwaves_r 14.000 -
503.bwaves_r (avg) 14.000 -
503.bwaves_r (avg) (percentage) 0.070 -
503.bwaves_r (max) 68.000 -2.86%
503.bwaves_r (max) (percentage) 0.330 -2.94%
505.mcf_r 57.000 -3.39%
505.mcf_r (avg) 57.000 -3.39%
505.mcf_r (avg) (percentage) 0.410 -
505.mcf_r (max) 259.000 -8.16%
505.mcf_r (max) (percentage) 1.860 -4.12%
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 29.000 -
508.namd_r (avg) 29.000 -
508.namd_r (avg) (percentage) 0.310 -
508.namd_r (max) 185.000 -5.61%
508.namd_r (max) (percentage) 1.930 -5.39%
510.parest_r 63.000 -
510.parest_r (avg) 63.000 -
510.parest_r (avg) (percentage) 0.280 -
510.parest_r (max) 61.000 -3.17%
510.parest_r (max) (percentage) 0.270 -3.57%
511.povray_r 87.000 1.16%
511.povray_r (avg) 87.000 1.16%
511.povray_r (avg) (percentage) 0.440 -
511.povray_r (max) 137.000 -
511.povray_r (max) (percentage) 0.690 -1.43%
519.lbm_r 61.000 3.39%
519.lbm_r (avg) 61.000 3.39%
519.lbm_r (avg) (percentage) 0.400 2.56%
519.lbm_r (max) 64.000 8.47%
519.lbm_r (max) (percentage) 0.420 7.69%
519.lbm_r (unexplained diff vs previous) 64.000 45.45%
519.lbm_r (unexplained diff vs previous) (percentage) 0.420 44.83%
520.omnetpp_r 153.000 -1.92%
520.omnetpp_r (avg) 153.000 -1.92%
520.omnetpp_r (avg) (percentage) 1.060 -1.85%
520.omnetpp_r (max) 180.000 -10.00%
520.omnetpp_r (max) (percentage) 1.250 -9.42%
520.omnetpp_r (unexplained diff vs previous) 34.000 -59.52%
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.230 -60.34%
521.wrf_r 177.000 -
521.wrf_r (avg) 177.000 -
521.wrf_r (avg) (percentage) 1.330 -
521.wrf_r (max) 244.000 -
521.wrf_r (max) (percentage) 1.830 -1.08%
523.xalancbmk_r (avg) (percentage) 0.250 -
523.xalancbmk_r (max) (percentage) 0.260 -3.70%
525.x264_r 12.000 -
525.x264_r (avg) 12.000 -
525.x264_r (avg) (percentage) 0.200 -
525.x264_r (max) 7.000 -
525.x264_r (max) (percentage) 0.120 -
526.blender_r 113.000 -
526.blender_r (avg) 113.000 -
526.blender_r (avg) (percentage) 0.950 -
526.blender_r (max) 353.000 -1.40%
526.blender_r (max) (percentage) 2.950 -1.34%
527.cam4_r 78.000 -
527.cam4_r (avg) 78.000 -
527.cam4_r (avg) (percentage) 0.580 -
527.cam4_r (max) 177.000 -1.67%
527.cam4_r (max) (percentage) 1.310 -1.50%
531.deepsjeng_r 42.000 -
531.deepsjeng_r (avg) 42.000 -
531.deepsjeng_r (avg) (percentage) 0.500 -
531.deepsjeng_r (max) 85.000 -5.56%
531.deepsjeng_r (max) (percentage) 1.000 -5.66%
538.imagick_r 22.000 -
538.imagick_r (avg) 22.000 -
538.imagick_r (avg) (percentage) 0.160 -
538.imagick_r (max) 32.000 -
538.imagick_r (max) (percentage) 0.230 -
541.leela_r 53.000 -
541.leela_r (avg) 53.000 -
541.leela_r (avg) (percentage) 0.410 -
541.leela_r (max) 52.000 -7.14%
541.leela_r (max) (percentage) 0.400 -6.98%
544.nab_r 41.000 -
544.nab_r (avg) 41.000 -
544.nab_r (avg) (percentage) 0.290 -
544.nab_r (max) 62.000 -1.59%
544.nab_r (max) (percentage) 0.440 -2.22%
548.exchange2_r (avg) (percentage) 0.130 -
548.exchange2_r (max) (percentage) 0.250 -3.85%
549.fotonik3d_r 187.000 1.08%
549.fotonik3d_r (avg) 187.000 1.08%
549.fotonik3d_r (avg) (percentage) 0.980 -
549.fotonik3d_r (max) 353.000 -3.02%
549.fotonik3d_r (max) (percentage) 1.850 -4.15%
554.roms_r 42.000 2.44%
554.roms_r (avg) 42.000 2.44%
554.roms_r (avg) (percentage) 0.210 -
554.roms_r (max) 86.000 -1.15%
554.roms_r (max) (percentage) 0.430 -2.27%
557.xz_r 19.000 -
557.xz_r (avg) 19.000 -
557.xz_r (avg) (percentage) 0.170 -
557.xz_r (max) 12.000 -7.69%
557.xz_r (max) (percentage) 0.110 -
Geometric Mean 6.210 -2.01%