Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/8abb2ace888bdd04a1bdb4ac2f2fc25d57a5760a (+ 87 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/118/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0118 (llvmorg-18-init-08020-g8abb2ace888b) 2023-10-05T20:54:29 0:00:00
Previous 0113 (llvmorg-18-init-07933-ge13bed4c5f35) 2023-10-04T13:57:10 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 6 51
Performance Improvements 24 43
Added Tests 2 5
Unchanged Tests 868 801
Total Tests 900

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
526.blender_r 4.68% 11950.000 12509.000 - 0.62% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
526.blender_r 4.65% 86.000 90.000 - 119.51% -
526.blender_r (avg) 4.65% 86.000 90.000 - 119.51% -
526.blender_r (max) 2.90% 69.000 71.000 - 61.36% -
511.povray_r 1.23% 81.000 82.000 - 6.49% -
511.povray_r (avg) 1.23% 81.000 82.000 - 6.49% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
531.deepsjeng_r -1.49% 8650.000 8521.000 - -3.05% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
525.x264_r (max) -10.00% 10.000 9.000 - -18.18% -
510.parest_r (max) -9.38% 32.000 29.000 - -32.56% -
510.parest_r (max) (percentage) -7.14% 0.140 0.130 - -31.58% -
525.x264_r (max) (percentage) -6.25% 0.160 0.150 - 0.00% -
520.omnetpp_r (max) (percentage) -5.70% 1.930 1.820 - -27.20% -
520.omnetpp_r (max) -5.65% 283.000 267.000 - -27.05% -
531.deepsjeng_r (max) -4.76% 42.000 40.000 - -40.30% -
541.leela_r (max) (percentage) -4.29% 0.700 0.670 - 21.82% -
541.leela_r (max) -4.21% 95.000 91.000 - 22.97% -
531.deepsjeng_r (max) (percentage) -4.08% 0.490 0.470 - -38.96% -
544.nab_r (max) -3.77% 53.000 51.000 - 37.84% -
508.namd_r (max) -3.70% 27.000 26.000 - -75.93% -
523.xalancbmk_r (max) (percentage) -3.23% 0.310 0.300 - -21.05% -
511.povray_r (max) (percentage) -3.13% 0.640 0.620 - 12.73% -
544.nab_r (max) (percentage) -2.70% 0.370 0.360 - 38.46% -
523.xalancbmk_r (max) -2.38% 42.000 41.000 - -21.15% -
531.deepsjeng_r -2.38% 42.000 41.000 - 13.89% -
531.deepsjeng_r (avg) -2.38% 42.000 41.000 - 13.89% -
511.povray_r (max) -2.36% 127.000 124.000 - 5.08% -
500.perlbench_r (max) (percentage) -2.36% 2.120 2.070 - 55.64% -
526.blender_r (max) (percentage) -1.72% 0.580 0.570 - 58.33% -
538.imagick_r (max) -1.69% 59.000 58.000 - 0.00% -
500.perlbench_r (max) -1.47% 339.000 334.000 - 56.81% -

Added Tests - sample variability
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
507.cactuBSSN_r 1.36% 11293.000 11447.000 - -0.53% -
538.imagick_r 1.09% 13884.000 14035.000 - -0.64% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
503.bwaves_r (max) (percentage) 177.78% 0.090 0.250 - 0.00% -
554.roms_r (max) (percentage) 171.43% 0.070 0.190 - 0.00% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
526.blender_r 119.51% 41.000 90.000 - 4.65% -
526.blender_r (avg) 119.51% 41.000 90.000 - 4.65% -
526.blender_r (avg) (percentage) 118.18% 0.330 0.720 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
526.blender_r (max) 61.36% 44.000 71.000 - 2.90% -
526.blender_r (max) (percentage) 58.33% 0.360 0.570 - -1.72% -
519.lbm_r (unexplained diff vs previous) (percentage) 57.14% 0.140 0.220 - 0.00% -
500.perlbench_r (max) 56.81% 213.000 334.000 - -1.47% -
500.perlbench_r (max) (percentage) 55.64% 1.330 2.070 - -2.36% -
519.lbm_r (unexplained diff vs previous) 54.17% 24.000 37.000 - 0.00% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 0.00% -
544.nab_r (max) (percentage) 38.46% 0.260 0.360 - -2.70% -
519.lbm_r 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (avg) 37.93% 29.000 40.000 - 0.00% -
544.nab_r (max) 37.84% 37.000 51.000 - -3.77% -
519.lbm_r (max) (percentage) 29.41% 0.170 0.220 - 0.00% -
519.lbm_r (max) 27.59% 29.000 37.000 - 0.00% -
520.omnetpp_r 26.87% 134.000 170.000 - 0.00% -
520.omnetpp_r (avg) 26.87% 134.000 170.000 - 0.00% -
520.omnetpp_r (avg) (percentage) 26.09% 0.920 1.160 - 0.00% -
541.leela_r (max) 22.97% 74.000 91.000 - -4.21% -
548.exchange2_r (max) (percentage) 22.73% 0.220 0.270 - 0.00% -
541.leela_r (max) (percentage) 21.82% 0.550 0.670 - -4.29% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
531.deepsjeng_r 13.89% 36.000 41.000 - -2.38% -
531.deepsjeng_r (avg) 13.89% 36.000 41.000 - -2.38% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
511.povray_r (max) (percentage) 12.73% 0.550 0.620 - -3.13% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
557.xz_r 9.09% 11.000 12.000 - 0.00% -
557.xz_r (avg) 9.09% 11.000 12.000 - 0.00% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r 6.49% 77.000 82.000 - 1.23% -
511.povray_r (avg) 6.49% 77.000 82.000 - 1.23% -
511.povray_r (max) 5.08% 118.000 124.000 - -2.36% -
544.nab_r 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.56% 624815.000 615095.000 - 0.01% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -14.42% 7490.000 6410.000 - 0.09% -
511.povray_r -6.95% 21547.000 20049.000 - 0.37% -
519.lbm_r -3.41% 17526.000 16929.000 - 0.22% -
531.deepsjeng_r -3.05% 8789.000 8521.000 - -1.49% -
505.mcf_r -1.11% 14148.000 13991.000 - -0.56% -
502.gcc_r -1.03% 11030.000 10916.000 - -0.02% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
557.xz_r (max) -76.19% 21.000 5.000 - 0.00% -
505.mcf_r (max) -76.04% 96.000 23.000 - 0.00% -
508.namd_r (max) -75.93% 108.000 26.000 - -3.70% -
508.namd_r (max) (percentage) -75.22% 1.130 0.280 - -3.45% -
505.mcf_r (max) (percentage) -75.00% 0.680 0.170 - 0.00% -
557.xz_r (max) (percentage) -73.68% 0.190 0.050 - 0.00% -
549.fotonik3d_r (max) (percentage) -63.64% 1.210 0.440 - 0.00% -
508.namd_r -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) (percentage) -56.25% 0.640 0.280 - 0.00% -
531.deepsjeng_r (max) -40.30% 67.000 40.000 - -4.76% -
538.imagick_r (avg) (percentage) -40.00% 0.300 0.180 - 0.00% -
538.imagick_r -39.02% 41.000 25.000 - 0.00% -
538.imagick_r (avg) -39.02% 41.000 25.000 - 0.00% -
531.deepsjeng_r (max) (percentage) -38.96% 0.770 0.470 - -4.08% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
510.parest_r (max) -32.56% 43.000 29.000 - -9.38% -
541.leela_r -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) (percentage) -31.82% 0.440 0.300 - 0.00% -
510.parest_r (max) (percentage) -31.58% 0.190 0.130 - -7.14% -
520.omnetpp_r (max) (percentage) -27.20% 2.500 1.820 - -5.70% -
520.omnetpp_r (max) -27.05% 366.000 267.000 - -5.65% -
505.mcf_r -26.03% 73.000 54.000 - 0.00% -
505.mcf_r (avg) -26.03% 73.000 54.000 - 0.00% -
505.mcf_r (avg) (percentage) -25.00% 0.520 0.390 - 0.00% -
523.xalancbmk_r (max) -21.15% 52.000 41.000 - -2.38% -
523.xalancbmk_r (max) (percentage) -21.05% 0.380 0.300 - -3.23% -
525.x264_r (max) -18.18% 11.000 9.000 - -10.00% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
523.xalancbmk_r -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) -8.33% 0.240 0.220 - 0.00% -
500.perlbench_r (avg) (percentage) -7.69% 0.910 0.840 - 0.00% -
500.perlbench_r -7.53% 146.000 135.000 - 0.75% -
500.perlbench_r (avg) -7.53% 146.000 135.000 - 0.75% -

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
500.perlbench_r 16147.000 -
502.gcc_r 10916.000 -
505.mcf_r 13991.000 -
507.cactuBSSN_r 11447.000 -
508.namd_r 9520.000 -
510.parest_r 22912.000 -
511.povray_r 20049.000 -
519.lbm_r 16929.000 -
520.omnetpp_r 14686.000 -
523.xalancbmk_r 13724.000 -
525.x264_r 6410.000 -
526.blender_r 12509.000 4.68%
531.deepsjeng_r 8521.000 -1.49%
538.imagick_r 14035.000 -
541.leela_r 13615.000 -
544.nab_r 14399.000 -
557.xz_r 11481.000 -
Geometric Mean 13038.054 -
Code Size (bytes)
Name Current %
500.perlbench_r 2149367.000 -
502.gcc_r 8952252.000 -
505.mcf_r 27255.000 -
507.cactuBSSN_r 3679771.000 -
508.namd_r 615095.000 -
510.parest_r 8670308.000 -
511.povray_r 917074.000 -
519.lbm_r 13343.000 -
520.omnetpp_r 2153409.000 -
523.xalancbmk_r 4724899.000 -
525.x264_r 498478.000 -
526.blender_r 13666786.000 -
531.deepsjeng_r 81246.000 -
538.imagick_r 1698925.000 -
541.leela_r 145171.000 -
544.nab_r 191524.000 -
557.xz_r 177036.000 -
Geometric Mean 739616.971 -
sample variability
Name Current %
500.perlbench_r 135.000 -
500.perlbench_r (avg) 135.000 -
500.perlbench_r (avg) (percentage) 0.840 -
500.perlbench_r (max) 334.000 -1.47%
500.perlbench_r (max) (percentage) 2.070 -2.36%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.250 -
505.mcf_r 54.000 -
505.mcf_r (avg) 54.000 -
505.mcf_r (avg) (percentage) 0.390 -
505.mcf_r (max) 23.000 -
505.mcf_r (max) (percentage) 0.170 -
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 26.000 -
508.namd_r (avg) 26.000 -
508.namd_r (avg) (percentage) 0.280 -
508.namd_r (max) 26.000 -3.70%
508.namd_r (max) (percentage) 0.280 -
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 29.000 -9.38%
510.parest_r (max) (percentage) 0.130 -7.14%
511.povray_r 82.000 1.23%
511.povray_r (avg) 82.000 1.23%
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 124.000 -2.36%
511.povray_r (max) (percentage) 0.620 -3.13%
519.lbm_r 40.000 -
519.lbm_r (avg) 40.000 -
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 37.000 -
519.lbm_r (max) (percentage) 0.220 -
519.lbm_r (unexplained diff vs previous) 37.000 -
519.lbm_r (unexplained diff vs previous) (percentage) 0.220 -
520.omnetpp_r 170.000 -
520.omnetpp_r (avg) 170.000 -
520.omnetpp_r (avg) (percentage) 1.160 -
520.omnetpp_r (max) 267.000 -5.65%
520.omnetpp_r (max) (percentage) 1.820 -5.70%
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 30.000 -
523.xalancbmk_r (avg) 30.000 -
523.xalancbmk_r (avg) (percentage) 0.220 -
523.xalancbmk_r (max) 41.000 -2.38%
523.xalancbmk_r (max) (percentage) 0.300 -3.23%
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 9.000 -10.00%
525.x264_r (max) (percentage) 0.150 -6.25%
526.blender_r 90.000 4.65%
526.blender_r (avg) 90.000 4.65%
526.blender_r (avg) (percentage) 0.720 -
526.blender_r (max) 71.000 2.90%
526.blender_r (max) (percentage) 0.570 -1.72%
531.deepsjeng_r 41.000 -2.38%
531.deepsjeng_r (avg) 41.000 -2.38%
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 40.000 -4.76%
531.deepsjeng_r (max) (percentage) 0.470 -4.08%
538.imagick_r 25.000 -
538.imagick_r (avg) 25.000 -
538.imagick_r (avg) (percentage) 0.180 -
538.imagick_r (max) 58.000 -1.69%
538.imagick_r (max) (percentage) 0.420 -
541.leela_r 40.000 -
541.leela_r (avg) 40.000 -
541.leela_r (avg) (percentage) 0.300 -
541.leela_r (max) 91.000 -4.21%
541.leela_r (max) (percentage) 0.670 -4.29%
544.nab_r 43.000 -
544.nab_r (avg) 43.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 51.000 -3.77%
544.nab_r (max) (percentage) 0.360 -2.70%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.270 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.440 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.190 -
557.xz_r 12.000 -
557.xz_r (avg) 12.000 -
557.xz_r (avg) (percentage) 0.110 -
557.xz_r (max) 5.000 -
557.xz_r (max) (percentage) 0.050 -
Geometric Mean 3.674 -1.49%