Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/4b5366c9512aa273a5272af1d833961e1ed156e7 (+ 648 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/96/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0096 (llvmorg-18-init-05928-g4b5366c9512a) 2023-09-11T16:27:40 0:00:00
Previous 0094 (llvmorg-18-init-05280-gc154ba8abeb6) 2023-09-05T13:54:23 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 15 54
Performance Improvements 40 50
Added Tests 2 2
Unchanged Tests 819 770
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
541.leela_r 2.79% 13275.000 13646.000 - 0.55% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) (percentage) 451.22% 0.410 2.260 - 584.85% -
520.omnetpp_r (unexplained diff vs previous) 443.33% 60.000 326.000 - 579.17% -
508.namd_r (unexplained diff vs previous) 160.00% 10.000 26.000 - -75.24% -
508.namd_r (unexplained diff vs previous) (percentage) 145.45% 0.110 0.270 - -75.45% -
520.omnetpp_r (max) (percentage) 90.52% 1.160 2.210 - -11.60% -
520.omnetpp_r (max) 87.06% 170.000 318.000 - -13.11% -
508.namd_r (max) 38.89% 18.000 25.000 - -76.85% -
508.namd_r (max) (percentage) 35.00% 0.200 0.270 - -76.11% -
520.omnetpp_r (avg) (percentage) 8.24% 0.850 0.920 - 0.00% -
541.leela_r (max) 6.67% 15.000 16.000 - -78.38% -
520.omnetpp_r 5.60% 125.000 132.000 - -1.49% -
520.omnetpp_r (avg) 5.60% 125.000 132.000 - -1.49% -
541.leela_r 2.70% 37.000 38.000 - -35.59% -
541.leela_r (avg) 2.70% 37.000 38.000 - -35.59% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
505.mcf_r -3.94% 14558.000 13984.000 - -1.16% -
511.povray_r -2.23% 20846.000 20382.000 - -5.41% -
520.omnetpp_r -2.21% 14739.000 14413.000 - -1.70% -
544.nab_r -1.30% 12351.000 12191.000 - -14.84% -
500.perlbench_r -1.19% 16282.000 16088.000 - 0.23% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
525.x264_r (unexplained diff vs previous) -27.78% 18.000 13.000 - 18.18% -
525.x264_r (unexplained diff vs previous) (percentage) -27.59% 0.290 0.210 - 40.00% -
505.mcf_r (max) -12.62% 103.000 90.000 - -6.25% -
557.xz_r (max) -12.50% 16.000 14.000 - -33.33% -
519.lbm_r -9.52% 42.000 38.000 - 31.03% -
519.lbm_r (avg) -9.52% 42.000 38.000 - 31.03% -
519.lbm_r (max) -9.38% 96.000 87.000 - 200.00% -
519.lbm_r (max) (percentage) -9.09% 0.550 0.500 - 194.12% -
505.mcf_r (max) (percentage) -8.45% 0.710 0.650 - -4.41% -
519.lbm_r (avg) (percentage) -8.33% 0.240 0.220 - 29.41% -
511.povray_r (max) -8.15% 135.000 124.000 - 5.08% -
557.xz_r (max) (percentage) -7.14% 0.140 0.130 - -31.58% -
544.nab_r (max) -6.67% 60.000 56.000 - 51.35% -
511.povray_r (max) (percentage) -6.15% 0.650 0.610 - 10.91% -
544.nab_r (max) (percentage) -6.12% 0.490 0.460 - 76.92% -
525.x264_r (max) -4.76% 21.000 20.000 - 81.82% -
505.mcf_r -4.17% 72.000 69.000 - -5.48% -
505.mcf_r (avg) -4.17% 72.000 69.000 - -5.48% -
538.imagick_r (max) -4.17% 72.000 69.000 - 18.97% -
538.imagick_r (max) (percentage) -3.85% 0.520 0.500 - 19.05% -
548.exchange2_r (max) (percentage) -3.23% 0.310 0.300 - 36.36% -
500.perlbench_r (max) -3.23% 372.000 360.000 - 69.01% -
523.xalancbmk_r (max) (percentage) -3.13% 0.320 0.310 - -18.42% -
510.parest_r (max) (percentage) -2.56% 0.390 0.380 - 100.00% -
511.povray_r -2.47% 81.000 79.000 - 2.60% -
511.povray_r (avg) -2.47% 81.000 79.000 - 2.60% -
526.blender_r (max) (percentage) -2.46% 3.660 3.570 - 891.67% -
523.xalancbmk_r (max) -2.33% 43.000 42.000 - -19.23% -
544.nab_r -2.33% 43.000 42.000 - 2.44% -
544.nab_r (avg) -2.33% 43.000 42.000 - 2.44% -
510.parest_r (max) -2.25% 89.000 87.000 - 102.33% -
500.perlbench_r (max) (percentage) -2.18% 2.290 2.240 - 68.42% -
531.deepsjeng_r (max) -2.17% 46.000 45.000 - -32.84% -
526.blender_r (max) -2.11% 427.000 418.000 - 850.00% -
531.deepsjeng_r (max) (percentage) -1.85% 0.540 0.530 - -31.17% -

Added Tests - sample variability
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 2.00% 11293.000 11519.000 - 0.25% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
526.blender_r (max) (percentage) 891.67% 0.360 3.570 - -2.46% -
526.blender_r (max) 850.00% 44.000 418.000 - -2.11% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 584.85% 0.330 2.260 - 451.22% -
520.omnetpp_r (unexplained diff vs previous) 579.17% 48.000 326.000 - 443.33% -
554.roms_r (max) (percentage) 228.57% 0.070 0.230 - -4.17% -
503.bwaves_r (max) (percentage) 211.11% 0.090 0.280 - -3.45% -
519.lbm_r (max) 200.00% 29.000 87.000 - -9.38% -
519.lbm_r (max) (percentage) 194.12% 0.170 0.500 - -9.09% -
525.x264_r (avg) (percentage) 155.56% 0.090 0.230 - 0.00% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
526.blender_r (avg) (percentage) 139.39% 0.330 0.790 - 0.00% -
525.x264_r 133.33% 6.000 14.000 - 0.00% -
525.x264_r (avg) 133.33% 6.000 14.000 - 0.00% -
525.x264_r (max) (percentage) 126.67% 0.150 0.340 - -2.86% -
526.blender_r 124.39% 41.000 92.000 - 0.00% -
526.blender_r (avg) 124.39% 41.000 92.000 - 0.00% -
510.parest_r (max) 102.33% 43.000 87.000 - -2.25% -
510.parest_r (max) (percentage) 100.00% 0.190 0.380 - -2.56% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
525.x264_r (max) 81.82% 11.000 20.000 - -4.76% -
544.nab_r (max) (percentage) 76.92% 0.260 0.460 - -6.12% -
500.perlbench_r (max) 69.01% 213.000 360.000 - -3.23% -
500.perlbench_r (max) (percentage) 68.42% 1.330 2.240 - -2.18% -
544.nab_r (max) 51.35% 37.000 56.000 - -6.67% -
525.x264_r (unexplained diff vs previous) (percentage) 40.00% 0.150 0.210 - -27.59% -
548.exchange2_r (max) (percentage) 36.36% 0.220 0.300 - -3.23% -
519.lbm_r 31.03% 29.000 38.000 - -9.52% -
519.lbm_r (avg) 31.03% 29.000 38.000 - -9.52% -
557.xz_r (avg) (percentage) 30.00% 0.100 0.130 - 0.00% -
519.lbm_r (avg) (percentage) 29.41% 0.170 0.220 - -8.33% -
557.xz_r 27.27% 11.000 14.000 - 0.00% -
557.xz_r (avg) 27.27% 11.000 14.000 - 0.00% -
531.deepsjeng_r (avg) (percentage) 24.39% 0.410 0.510 - 0.00% -
531.deepsjeng_r 22.22% 36.000 44.000 - 0.00% -
531.deepsjeng_r (avg) 22.22% 36.000 44.000 - 0.00% -
544.nab_r (avg) (percentage) 20.69% 0.290 0.350 - 0.00% -
538.imagick_r (max) (percentage) 19.05% 0.420 0.500 - -3.85% -
538.imagick_r (max) 18.97% 58.000 69.000 - -4.17% -
525.x264_r (unexplained diff vs previous) 18.18% 11.000 13.000 - -27.78% -
511.povray_r (max) (percentage) 10.91% 0.550 0.610 - -6.15% -
511.povray_r (avg) (percentage) 8.33% 0.360 0.390 - 0.00% -
510.parest_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
510.parest_r 5.88% 34.000 36.000 - 0.00% -
510.parest_r (avg) 5.88% 34.000 36.000 - 0.00% -
511.povray_r (max) 5.08% 118.000 124.000 - -8.15% -
511.povray_r 2.60% 77.000 79.000 - -2.47% -
511.povray_r (avg) 2.60% 77.000 79.000 - -2.47% -
544.nab_r 2.44% 41.000 42.000 - -2.33% -
544.nab_r (avg) 2.44% 41.000 42.000 - -2.33% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
519.lbm_r -3.00% 13451.000 13048.000 - 0.00% -
508.namd_r -1.27% 624815.000 616856.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -17.69% 7490.000 6165.000 - -0.21% -
544.nab_r -14.84% 14316.000 12191.000 - -1.30% -
526.blender_r -5.63% 12432.000 11732.000 - 0.34% -
511.povray_r -5.41% 21547.000 20382.000 - -2.23% -
520.omnetpp_r -1.70% 14662.000 14413.000 - -2.21% -
531.deepsjeng_r -1.27% 8789.000 8677.000 - 0.50% -
508.namd_r -1.21% 9575.000 9459.000 - -0.27% -
505.mcf_r -1.16% 14148.000 13984.000 - -3.94% -
523.xalancbmk_r -1.11% 13812.000 13659.000 - 0.66% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
541.leela_r (max) -78.38% 74.000 16.000 - 6.67% -
541.leela_r (max) (percentage) -78.18% 0.550 0.120 - 0.00% -
508.namd_r (max) -76.85% 108.000 25.000 - 38.89% -
508.namd_r (max) (percentage) -76.11% 1.130 0.270 - 35.00% -
508.namd_r (unexplained diff vs previous) (percentage) -75.45% 1.100 0.270 - 145.45% -
508.namd_r (unexplained diff vs previous) -75.24% 105.000 26.000 - 160.00% -
549.fotonik3d_r (max) (percentage) -61.98% 1.210 0.460 - -2.13% -
508.namd_r -54.10% 61.000 28.000 - 0.00% -
508.namd_r (avg) -54.10% 61.000 28.000 - 0.00% -
508.namd_r (avg) (percentage) -53.12% 0.640 0.300 - 0.00% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
541.leela_r (avg) (percentage) -36.36% 0.440 0.280 - 0.00% -
541.leela_r -35.59% 59.000 38.000 - 2.70% -
541.leela_r (avg) -35.59% 59.000 38.000 - 2.70% -
557.xz_r (max) -33.33% 21.000 14.000 - -12.50% -
531.deepsjeng_r (max) -32.84% 67.000 45.000 - -2.17% -
557.xz_r (max) (percentage) -31.58% 0.190 0.130 - -7.14% -
531.deepsjeng_r (max) (percentage) -31.17% 0.770 0.530 - -1.85% -
538.imagick_r -26.83% 41.000 30.000 - 0.00% -
538.imagick_r (avg) -26.83% 41.000 30.000 - 0.00% -
538.imagick_r (avg) (percentage) -26.67% 0.300 0.220 - 0.00% -
523.xalancbmk_r (max) -19.23% 52.000 42.000 - -2.33% -
523.xalancbmk_r (max) (percentage) -18.42% 0.380 0.310 - -3.13% -
523.xalancbmk_r -15.15% 33.000 28.000 - 0.00% -
523.xalancbmk_r (avg) -15.15% 33.000 28.000 - 0.00% -
520.omnetpp_r (max) -13.11% 366.000 318.000 - 87.06% -
523.xalancbmk_r (avg) (percentage) -12.50% 0.240 0.210 - 0.00% -
520.omnetpp_r (max) (percentage) -11.60% 2.500 2.210 - 90.52% -
500.perlbench_r (avg) (percentage) -10.99% 0.910 0.810 - 0.00% -
500.perlbench_r -10.96% 146.000 130.000 - -0.76% -
500.perlbench_r (avg) -10.96% 146.000 130.000 - -0.76% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
505.mcf_r (max) -6.25% 96.000 90.000 - -12.62% -
505.mcf_r -5.48% 73.000 69.000 - -4.17% -
505.mcf_r (avg) -5.48% 73.000 69.000 - -4.17% -
505.mcf_r (max) (percentage) -4.41% 0.680 0.650 - -8.45% -
505.mcf_r (avg) (percentage) -3.85% 0.520 0.500 - 0.00% -
520.omnetpp_r -1.49% 134.000 132.000 - 5.60% -
520.omnetpp_r (avg) -1.49% 134.000 132.000 - 5.60% -

Added Tests - sample variability
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
500.perlbench_r 16088.000 -1.19%
502.gcc_r 10939.000 -
505.mcf_r 13984.000 -3.94%
507.cactuBSSN_r 11519.000 -
508.namd_r 9459.000 -
510.parest_r 22981.000 -
511.povray_r 20382.000 -2.23%
519.lbm_r 17548.000 -
520.omnetpp_r 14413.000 -2.21%
523.xalancbmk_r 13659.000 -
525.x264_r 6165.000 -
526.blender_r 11732.000 -
531.deepsjeng_r 8677.000 -
538.imagick_r 13966.000 -
541.leela_r 13646.000 2.79%
544.nab_r 12191.000 -1.30%
557.xz_r 11493.000 -
Geometric Mean 12867.869 -
Code Size (bytes)
Name Current %
500.perlbench_r 2150028.000 -
502.gcc_r 8952980.000 -
505.mcf_r 27179.000 -
507.cactuBSSN_r 3675585.000 -
508.namd_r 616856.000 -
510.parest_r 8690810.000 -
511.povray_r 911927.000 -
519.lbm_r 13048.000 -
520.omnetpp_r 2154681.000 -
523.xalancbmk_r 4725804.000 -
525.x264_r 497659.000 -
526.blender_r 13668063.000 -
531.deepsjeng_r 81506.000 -
538.imagick_r 1701515.000 -
541.leela_r 145391.000 -
544.nab_r 189924.000 -
557.xz_r 177600.000 -
Geometric Mean 738484.443 -
sample variability
Name Current %
500.perlbench_r 130.000 -
500.perlbench_r (avg) 130.000 -
500.perlbench_r (avg) (percentage) 0.810 -
500.perlbench_r (max) 360.000 -3.23%
500.perlbench_r (max) (percentage) 2.240 -2.18%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.280 -
505.mcf_r 69.000 -4.17%
505.mcf_r (avg) 69.000 -4.17%
505.mcf_r (avg) (percentage) 0.500 -
505.mcf_r (max) 90.000 -12.62%
505.mcf_r (max) (percentage) 0.650 -8.45%
508.namd_r 28.000 -
508.namd_r (avg) 28.000 -
508.namd_r (avg) (percentage) 0.300 -
508.namd_r (max) 25.000 38.89%
508.namd_r (max) (percentage) 0.270 35.00%
508.namd_r (unexplained diff vs previous) 26.000 160.00%
508.namd_r (unexplained diff vs previous) (percentage) 0.270 145.45%
510.parest_r 36.000 -
510.parest_r (avg) 36.000 -
510.parest_r (avg) (percentage) 0.160 -
510.parest_r (max) 87.000 -2.25%
510.parest_r (max) (percentage) 0.380 -2.56%
511.povray_r 79.000 -2.47%
511.povray_r (avg) 79.000 -2.47%
511.povray_r (avg) (percentage) 0.390 -
511.povray_r (max) 124.000 -8.15%
511.povray_r (max) (percentage) 0.610 -6.15%
519.lbm_r 38.000 -9.52%
519.lbm_r (avg) 38.000 -9.52%
519.lbm_r (avg) (percentage) 0.220 -8.33%
519.lbm_r (max) 87.000 -9.38%
519.lbm_r (max) (percentage) 0.500 -9.09%
519.lbm_r (unexplained diff vs previous) 0.000 -
519.lbm_r (unexplained diff vs previous) (percentage) 0.000 -
520.omnetpp_r 132.000 5.60%
520.omnetpp_r (avg) 132.000 5.60%
520.omnetpp_r (avg) (percentage) 0.920 8.24%
520.omnetpp_r (max) 318.000 87.06%
520.omnetpp_r (max) (percentage) 2.210 90.52%
520.omnetpp_r (unexplained diff vs previous) 326.000 443.33%
520.omnetpp_r (unexplained diff vs previous) (percentage) 2.260 451.22%
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 28.000 -
523.xalancbmk_r (avg) 28.000 -
523.xalancbmk_r (avg) (percentage) 0.210 -
523.xalancbmk_r (max) 42.000 -2.33%
523.xalancbmk_r (max) (percentage) 0.310 -3.13%
525.x264_r 14.000 -
525.x264_r (avg) 14.000 -
525.x264_r (avg) (percentage) 0.230 -
525.x264_r (max) 20.000 -4.76%
525.x264_r (max) (percentage) 0.340 -
525.x264_r (unexplained diff vs previous) 13.000 -27.78%
525.x264_r (unexplained diff vs previous) (percentage) 0.210 -27.59%
526.blender_r 92.000 -
526.blender_r (avg) 92.000 -
526.blender_r (avg) (percentage) 0.790 -
526.blender_r (max) 418.000 -2.11%
526.blender_r (max) (percentage) 3.570 -2.46%
531.deepsjeng_r 44.000 -
531.deepsjeng_r (avg) 44.000 -
531.deepsjeng_r (avg) (percentage) 0.510 -
531.deepsjeng_r (max) 45.000 -2.17%
531.deepsjeng_r (max) (percentage) 0.530 -1.85%
538.imagick_r 30.000 -
538.imagick_r (avg) 30.000 -
538.imagick_r (avg) (percentage) 0.220 -
538.imagick_r (max) 69.000 -4.17%
538.imagick_r (max) (percentage) 0.500 -3.85%
541.leela_r 38.000 2.70%
541.leela_r (avg) 38.000 2.70%
541.leela_r (avg) (percentage) 0.280 -
541.leela_r (max) 16.000 6.67%
541.leela_r (max) (percentage) 0.120 -
544.nab_r 42.000 -2.33%
544.nab_r (avg) 42.000 -2.33%
544.nab_r (avg) (percentage) 0.350 -
544.nab_r (max) 56.000 -6.67%
544.nab_r (max) (percentage) 0.460 -6.12%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.300 -3.23%
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.460 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.230 -
557.xz_r 14.000 -
557.xz_r (avg) 14.000 -
557.xz_r (avg) (percentage) 0.130 -
557.xz_r (max) 14.000 -12.50%
557.xz_r (max) (percentage) 0.130 -7.14%
Geometric Mean 4.094 -16.45%