Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/e13bed4c5f3544c076ce57e36d9a11eefa5a7815 (+ 1 commits)
regression LLVM-1001
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/113/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0113 (llvmorg-18-init-07933-ge13bed4c5f35) 2023-10-04T13:57:10 0:00:00
Previous 0112 (llvmorg-18-init-07932-ga16f6462d756) 2023-10-02T16:25:15 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 7 55
Performance Improvements 31 45
Added Tests 6 5
Unchanged Tests 856 795
Total Tests 900

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
525.x264_r 3.22% 6204.000 6404.000 - -14.50% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
541.leela_r (max) 163.89% 36.000 95.000 - 28.38% -
541.leela_r (max) (percentage) 159.26% 0.270 0.700 - 27.27% -
541.leela_r 8.11% 37.000 40.000 - -32.20% -
541.leela_r (avg) 8.11% 37.000 40.000 - -32.20% -
541.leela_r (avg) (percentage) 7.14% 0.280 0.300 - -31.82% -
538.imagick_r (max) 1.72% 58.000 59.000 - 1.72% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
500.perlbench_r -2.63% 16423.000 15991.000 - -0.37% -
505.mcf_r -2.36% 14410.000 14070.000 - -0.55% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
508.namd_r (max) -12.90% 31.000 27.000 - -75.00% -
508.namd_r (max) (percentage) -12.12% 0.330 0.290 - -74.34% -
519.lbm_r (max) (percentage) -8.33% 0.240 0.220 - 29.41% -
520.omnetpp_r (max) (percentage) -7.66% 2.090 1.930 - -22.80% -
519.lbm_r (max) -7.50% 40.000 37.000 - 27.59% -
520.omnetpp_r (max) -6.91% 304.000 283.000 - -22.68% -
525.x264_r (max) (percentage) -5.88% 0.170 0.160 - 6.67% -
500.perlbench_r (max) -4.78% 356.000 339.000 - 59.15% -
531.deepsjeng_r (max) -4.55% 44.000 42.000 - -37.31% -
511.povray_r (max) -4.51% 133.000 127.000 - 7.63% -
505.mcf_r (max) -4.17% 24.000 23.000 - -76.04% -
531.deepsjeng_r (max) (percentage) -3.92% 0.510 0.490 - -36.36% -
503.bwaves_r (max) (percentage) -3.85% 0.260 0.250 - 177.78% -
508.namd_r -3.70% 27.000 26.000 - -57.38% -
508.namd_r (avg) -3.70% 27.000 26.000 - -57.38% -
505.mcf_r -3.57% 56.000 54.000 - -26.03% -
505.mcf_r (avg) -3.57% 56.000 54.000 - -26.03% -
526.blender_r (max) (percentage) -3.33% 0.600 0.580 - 61.11% -
523.xalancbmk_r (max) (percentage) -3.13% 0.320 0.310 - -18.42% -
511.povray_r (max) (percentage) -3.03% 0.660 0.640 - 16.36% -
526.blender_r (max) -2.82% 71.000 69.000 - 56.82% -
544.nab_r (max) (percentage) -2.63% 0.380 0.370 - 42.31% -
523.xalancbmk_r (max) -2.33% 43.000 42.000 - -19.23% -
500.perlbench_r (max) (percentage) -2.30% 2.170 2.120 - 59.40% -
500.perlbench_r -2.19% 137.000 134.000 - -8.22% -
500.perlbench_r (avg) -2.19% 137.000 134.000 - -8.22% -
544.nab_r (max) -1.85% 54.000 53.000 - 43.24% -
511.povray_r -1.22% 82.000 81.000 - 5.19% -
511.povray_r (avg) -1.22% 82.000 81.000 - 5.19% -

Added Tests - sample variability
508.namd_r (unexplained diff vs previous)
508.namd_r (unexplained diff vs previous) (percentage)
520.omnetpp_r (unexplained diff vs previous)
520.omnetpp_r (unexplained diff vs previous) (percentage)
541.leela_r (unexplained diff vs previous)
541.leela_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 1.90% 11293.000 11508.000 - 0.58% -
538.imagick_r 1.74% 13884.000 14126.000 - 0.83% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
541.leela_r (unexplained diff vs previous) 313.04% 23.000 95.000 - 0.00% -
541.leela_r (unexplained diff vs previous) (percentage) 311.76% 0.170 0.700 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) 189.58% 48.000 139.000 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 187.88% 0.330 0.950 - 0.00% -
503.bwaves_r (max) (percentage) 177.78% 0.090 0.250 - -3.85% -
554.roms_r (max) (percentage) 171.43% 0.070 0.190 - 0.00% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
526.blender_r (avg) (percentage) 118.18% 0.330 0.720 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
526.blender_r 109.76% 41.000 86.000 - 0.00% -
526.blender_r (avg) 109.76% 41.000 86.000 - 0.00% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
526.blender_r (max) (percentage) 61.11% 0.360 0.580 - -3.33% -
500.perlbench_r (max) (percentage) 59.40% 1.330 2.120 - -2.30% -
500.perlbench_r (max) 59.15% 213.000 339.000 - -4.78% -
526.blender_r (max) 56.82% 44.000 69.000 - -2.82% -
544.nab_r (max) 43.24% 37.000 53.000 - -1.85% -
544.nab_r (max) (percentage) 42.31% 0.260 0.370 - -2.63% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 0.00% -
519.lbm_r 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (avg) 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (max) (percentage) 29.41% 0.170 0.220 - -8.33% -
541.leela_r (max) 28.38% 74.000 95.000 - 163.89% -
519.lbm_r (max) 27.59% 29.000 37.000 - -7.50% -
541.leela_r (max) (percentage) 27.27% 0.550 0.700 - 159.26% -
520.omnetpp_r 26.87% 134.000 170.000 - 0.00% -
520.omnetpp_r (avg) 26.87% 134.000 170.000 - 0.00% -
520.omnetpp_r (avg) (percentage) 26.09% 0.920 1.160 - -0.85% -
548.exchange2_r (max) (percentage) 22.73% 0.220 0.270 - 0.00% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
531.deepsjeng_r 16.67% 36.000 42.000 - 0.00% -
531.deepsjeng_r (avg) 16.67% 36.000 42.000 - 0.00% -
511.povray_r (max) (percentage) 16.36% 0.550 0.640 - -3.03% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
557.xz_r 9.09% 11.000 12.000 - 0.00% -
557.xz_r (avg) 9.09% 11.000 12.000 - 0.00% -
511.povray_r (max) 7.63% 118.000 127.000 - -4.51% -
525.x264_r (max) (percentage) 6.67% 0.150 0.160 - -5.88% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r 5.19% 77.000 81.000 - -1.22% -
511.povray_r (avg) 5.19% 77.000 81.000 - -1.22% -
544.nab_r 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -
538.imagick_r (max) 1.72% 58.000 59.000 - 1.72% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.56% 624815.000 615051.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -14.50% 7490.000 6404.000 - 3.22% -
511.povray_r -7.30% 21547.000 19975.000 - -0.95% -
526.blender_r -3.88% 12432.000 11950.000 - -0.01% -
519.lbm_r -3.62% 17526.000 16892.000 - -0.18% -
531.deepsjeng_r -1.58% 8789.000 8650.000 - -0.03% -
502.gcc_r -1.02% 11030.000 10918.000 - -0.11% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
508.namd_r (unexplained diff vs previous) -87.62% 105.000 13.000 - 0.00% -
508.namd_r (unexplained diff vs previous) (percentage) -87.27% 1.100 0.140 - 0.00% -
557.xz_r (max) -76.19% 21.000 5.000 - 0.00% -
505.mcf_r (max) -76.04% 96.000 23.000 - -4.17% -
505.mcf_r (max) (percentage) -75.00% 0.680 0.170 - 0.00% -
508.namd_r (max) -75.00% 108.000 27.000 - -12.90% -
508.namd_r (max) (percentage) -74.34% 1.130 0.290 - -12.12% -
557.xz_r (max) (percentage) -73.68% 0.190 0.050 - 0.00% -
549.fotonik3d_r (max) (percentage) -63.64% 1.210 0.440 - 0.00% -
508.namd_r -57.38% 61.000 26.000 - -3.70% -
508.namd_r (avg) -57.38% 61.000 26.000 - -3.70% -
508.namd_r (avg) (percentage) -56.25% 0.640 0.280 - -3.45% -
538.imagick_r (avg) (percentage) -40.00% 0.300 0.180 - 0.00% -
538.imagick_r -39.02% 41.000 25.000 - 0.00% -
538.imagick_r (avg) -39.02% 41.000 25.000 - 0.00% -
531.deepsjeng_r (max) -37.31% 67.000 42.000 - -4.55% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
531.deepsjeng_r (max) (percentage) -36.36% 0.770 0.490 - -3.92% -
541.leela_r -32.20% 59.000 40.000 - 8.11% -
541.leela_r (avg) -32.20% 59.000 40.000 - 8.11% -
541.leela_r (avg) (percentage) -31.82% 0.440 0.300 - 7.14% -
510.parest_r (max) (percentage) -26.32% 0.190 0.140 - 0.00% -
505.mcf_r -26.03% 73.000 54.000 - -3.57% -
505.mcf_r (avg) -26.03% 73.000 54.000 - -3.57% -
510.parest_r (max) -25.58% 43.000 32.000 - 0.00% -
505.mcf_r (avg) (percentage) -25.00% 0.520 0.390 - 0.00% -
520.omnetpp_r (max) (percentage) -22.80% 2.500 1.930 - -7.66% -
520.omnetpp_r (max) -22.68% 366.000 283.000 - -6.91% -
523.xalancbmk_r (max) -19.23% 52.000 42.000 - -2.33% -
523.xalancbmk_r (max) (percentage) -18.42% 0.380 0.310 - -3.13% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
523.xalancbmk_r -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) -9.09% 33.000 30.000 - 0.00% -
525.x264_r (max) -9.09% 11.000 10.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) -8.33% 0.240 0.220 - 0.00% -
500.perlbench_r -8.22% 146.000 134.000 - -2.19% -
500.perlbench_r (avg) -8.22% 146.000 134.000 - -2.19% -
500.perlbench_r (avg) (percentage) -7.69% 0.910 0.840 - 0.00% -

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
500.perlbench_r 15991.000 -2.63%
502.gcc_r 10918.000 -
505.mcf_r 14070.000 -2.36%
507.cactuBSSN_r 11508.000 -
508.namd_r 9521.000 -
510.parest_r 22929.000 -
511.povray_r 19975.000 -
519.lbm_r 16892.000 -
520.omnetpp_r 14694.000 -
523.xalancbmk_r 13710.000 -
525.x264_r 6404.000 3.22%
526.blender_r 11950.000 -
531.deepsjeng_r 8650.000 -
538.imagick_r 14126.000 -
541.leela_r 13637.000 -
544.nab_r 14432.000 -
557.xz_r 11442.000 -
Geometric Mean 13016.017 -
Code Size (bytes)
Name Current %
500.perlbench_r 2149183.000 -
502.gcc_r 8945660.000 -
505.mcf_r 27255.000 -
507.cactuBSSN_r 3680247.000 -
508.namd_r 615051.000 -
510.parest_r 8649760.000 -
511.povray_r 916014.000 -
519.lbm_r 13343.000 -
520.omnetpp_r 2150201.000 -
523.xalancbmk_r 4713967.000 -
525.x264_r 497022.000 -
526.blender_r 13641826.000 -
531.deepsjeng_r 81370.000 -
538.imagick_r 1695789.000 -
541.leela_r 144595.000 -
544.nab_r 191256.000 -
557.xz_r 176220.000 -
Geometric Mean 738609.500 -
sample variability
Name Current %
500.perlbench_r 134.000 -2.19%
500.perlbench_r (avg) 134.000 -2.19%
500.perlbench_r (avg) (percentage) 0.840 -
500.perlbench_r (max) 339.000 -4.78%
500.perlbench_r (max) (percentage) 2.120 -2.30%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.250 -3.85%
505.mcf_r 54.000 -3.57%
505.mcf_r (avg) 54.000 -3.57%
505.mcf_r (avg) (percentage) 0.390 -
505.mcf_r (max) 23.000 -4.17%
505.mcf_r (max) (percentage) 0.170 -
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 26.000 -3.70%
508.namd_r (avg) 26.000 -3.70%
508.namd_r (avg) (percentage) 0.280 -
508.namd_r (max) 27.000 -12.90%
508.namd_r (max) (percentage) 0.290 -12.12%
508.namd_r (unexplained diff vs previous) 13.000 -
508.namd_r (unexplained diff vs previous) (percentage) 0.140 -
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 32.000 -
510.parest_r (max) (percentage) 0.140 -
511.povray_r 81.000 -1.22%
511.povray_r (avg) 81.000 -1.22%
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 127.000 -4.51%
511.povray_r (max) (percentage) 0.640 -3.03%
519.lbm_r 40.000 -
519.lbm_r (avg) 40.000 -
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 37.000 -7.50%
519.lbm_r (max) (percentage) 0.220 -8.33%
520.omnetpp_r 170.000 -
520.omnetpp_r (avg) 170.000 -
520.omnetpp_r (avg) (percentage) 1.160 -
520.omnetpp_r (max) 283.000 -6.91%
520.omnetpp_r (max) (percentage) 1.930 -7.66%
520.omnetpp_r (unexplained diff vs previous) 139.000 -
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.950 -
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 30.000 -
523.xalancbmk_r (avg) 30.000 -
523.xalancbmk_r (avg) (percentage) 0.220 -
523.xalancbmk_r (max) 42.000 -2.33%
523.xalancbmk_r (max) (percentage) 0.310 -3.13%
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 10.000 -
525.x264_r (max) (percentage) 0.160 -5.88%
526.blender_r 86.000 -
526.blender_r (avg) 86.000 -
526.blender_r (avg) (percentage) 0.720 -
526.blender_r (max) 69.000 -2.82%
526.blender_r (max) (percentage) 0.580 -3.33%
531.deepsjeng_r 42.000 -
531.deepsjeng_r (avg) 42.000 -
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 42.000 -4.55%
531.deepsjeng_r (max) (percentage) 0.490 -3.92%
538.imagick_r 25.000 -
538.imagick_r (avg) 25.000 -
538.imagick_r (avg) (percentage) 0.180 -
538.imagick_r (max) 59.000 1.72%
538.imagick_r (max) (percentage) 0.420 -
541.leela_r 40.000 8.11%
541.leela_r (avg) 40.000 8.11%
541.leela_r (avg) (percentage) 0.300 7.14%
541.leela_r (max) 95.000 163.89%
541.leela_r (max) (percentage) 0.700 159.26%
541.leela_r (unexplained diff vs previous) 95.000 -
541.leela_r (unexplained diff vs previous) (percentage) 0.700 -
544.nab_r 43.000 -
544.nab_r (avg) 43.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 53.000 -1.85%
544.nab_r (max) (percentage) 0.370 -2.63%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.270 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.440 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.190 -
557.xz_r 12.000 -
557.xz_r (avg) 12.000 -
557.xz_r (avg) (percentage) 0.110 -
557.xz_r (max) 5.000 -
557.xz_r (max) (percentage) 0.050 -
Geometric Mean 3.799 2.76%