Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/4a4b8570f7c16346094c59fab1bd8debf9b177e1 (+ 1411 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/125/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0125 (llvmorg-18-init-10748-g4a4b8570f7c1) 2023-10-27T21:30:53 0:00:00
Previous 0122 (llvmorg-18-init-09337-g159614a52f0b) 2023-10-19T00:17:42 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 12 49
Performance Improvements 22 48
Added Tests 0 5
Unchanged Tests 866 798
Total Tests 900

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r 2.92% 14575.000 15000.000 - 2.31% -
531.deepsjeng_r 1.34% 8581.000 8696.000 - -1.06% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (unexplained diff vs previous) (percentage) 330.00% 0.100 0.430 - 207.14% -
519.lbm_r (unexplained diff vs previous) 329.41% 17.000 73.000 - 204.17% -
519.lbm_r (max) 213.04% 23.000 72.000 - 148.28% -
519.lbm_r (max) (percentage) 207.14% 0.140 0.430 - 152.94% -
525.x264_r (max) 12.50% 8.000 9.000 - -18.18% -
519.lbm_r 5.26% 38.000 40.000 - 37.93% -
519.lbm_r (avg) 5.26% 38.000 40.000 - 37.93% -
520.omnetpp_r 2.96% 169.000 174.000 - 29.85% -
520.omnetpp_r (avg) 2.96% 169.000 174.000 - 29.85% -
520.omnetpp_r (max) 2.93% 239.000 246.000 - -32.79% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
507.cactuBSSN_r -1.11% 11493.000 11365.000 - 0.64% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
557.xz_r (max) (percentage) -20.00% 0.050 0.040 - -78.95% -
557.xz_r (max) -20.00% 5.000 4.000 - -80.95% -
541.leela_r (max) -4.94% 81.000 77.000 - 4.05% -
508.namd_r (max) -4.17% 24.000 23.000 - -78.70% -
508.namd_r (max) (percentage) -3.85% 0.260 0.250 - -77.88% -
538.imagick_r (max) -3.51% 57.000 55.000 - -5.17% -
541.leela_r (max) (percentage) -3.33% 0.600 0.580 - 5.45% -
544.nab_r (max) (percentage) -2.94% 0.340 0.330 - 26.92% -
531.deepsjeng_r (max) -2.78% 36.000 35.000 - -47.76% -
511.povray_r (max) -2.54% 118.000 115.000 - -2.54% -
531.deepsjeng_r (max) (percentage) -2.38% 0.420 0.410 - -46.75% -
544.nab_r (max) -2.08% 48.000 47.000 - 27.03% -
500.perlbench_r (max) (percentage) -2.05% 1.950 1.910 - 43.61% -
526.blender_r (max) (percentage) -1.82% 0.550 0.540 - 50.00% -
505.mcf_r -1.82% 55.000 54.000 - -26.03% -
505.mcf_r (avg) -1.82% 55.000 54.000 - -26.03% -
511.povray_r (max) (percentage) -1.72% 0.580 0.570 - 3.64% -
526.blender_r (max) -1.54% 65.000 64.000 - 45.45% -
500.perlbench_r (max) -1.28% 312.000 308.000 - 44.60% -
511.povray_r -1.20% 83.000 82.000 - 6.49% -
511.povray_r (avg) -1.20% 83.000 82.000 - 6.49% -

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r 2.31% 14662.000 15000.000 - 2.92% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
519.lbm_r (unexplained diff vs previous) (percentage) 207.14% 0.140 0.430 - 330.00% -
519.lbm_r (unexplained diff vs previous) 204.17% 24.000 73.000 - 329.41% -
503.bwaves_r (max) (percentage) 177.78% 0.090 0.250 - 0.00% -
554.roms_r (max) (percentage) 171.43% 0.070 0.190 - 0.00% -
519.lbm_r (max) (percentage) 152.94% 0.170 0.430 - 207.14% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
519.lbm_r (max) 148.28% 29.000 72.000 - 213.04% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
526.blender_r (avg) (percentage) 118.18% 0.330 0.720 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
526.blender_r 107.32% 41.000 85.000 - 0.00% -
526.blender_r (avg) 107.32% 41.000 85.000 - 0.00% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
526.blender_r (max) (percentage) 50.00% 0.360 0.540 - -1.82% -
526.blender_r (max) 45.45% 44.000 64.000 - -1.54% -
500.perlbench_r (max) 44.60% 213.000 308.000 - -1.28% -
500.perlbench_r (max) (percentage) 43.61% 1.330 1.910 - -2.05% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 4.35% -
519.lbm_r 37.93% 29.000 40.000 - 5.26% -
519.lbm_r (avg) 37.93% 29.000 40.000 - 5.26% -
520.omnetpp_r 29.85% 134.000 174.000 - 2.96% -
520.omnetpp_r (avg) 29.85% 134.000 174.000 - 2.96% -
544.nab_r (max) 27.03% 37.000 47.000 - -2.08% -
544.nab_r (max) (percentage) 26.92% 0.260 0.330 - -2.94% -
520.omnetpp_r (avg) (percentage) 26.09% 0.920 1.160 - 0.00% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
548.exchange2_r (max) (percentage) 18.18% 0.220 0.260 - 0.00% -
531.deepsjeng_r 16.67% 36.000 42.000 - 0.00% -
531.deepsjeng_r (avg) 16.67% 36.000 42.000 - 0.00% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
557.xz_r 9.09% 11.000 12.000 - 0.00% -
557.xz_r (avg) 9.09% 11.000 12.000 - 0.00% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r 6.49% 77.000 82.000 - -1.20% -
511.povray_r (avg) 6.49% 77.000 82.000 - -1.20% -
541.leela_r (max) (percentage) 5.45% 0.550 0.580 - -3.33% -
544.nab_r 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) 4.88% 41.000 43.000 - 0.00% -
541.leela_r (max) 4.05% 74.000 77.000 - -4.94% -
511.povray_r (max) (percentage) 3.64% 0.550 0.570 - -1.72% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.68% 624815.000 614295.000 - -0.11% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -13.90% 7490.000 6449.000 - 0.67% -
511.povray_r -6.07% 21547.000 20240.000 - -0.72% -
526.blender_r -4.35% 12432.000 11891.000 - 0.19% -
519.lbm_r -3.74% 17526.000 16870.000 - -0.43% -
541.leela_r -1.26% 13571.000 13400.000 - -0.98% -
523.xalancbmk_r -1.22% 13812.000 13643.000 - -0.49% -
505.mcf_r -1.07% 14148.000 13997.000 - -0.86% -
531.deepsjeng_r -1.06% 8789.000 8696.000 - 1.34% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
557.xz_r (max) -80.95% 21.000 4.000 - -20.00% -
557.xz_r (max) (percentage) -78.95% 0.190 0.040 - -20.00% -
508.namd_r (max) -78.70% 108.000 23.000 - -4.17% -
508.namd_r (max) (percentage) -77.88% 1.130 0.250 - -3.85% -
505.mcf_r (max) -77.08% 96.000 22.000 - 0.00% -
505.mcf_r (max) (percentage) -76.47% 0.680 0.160 - 0.00% -
549.fotonik3d_r (max) (percentage) -63.64% 1.210 0.440 - 0.00% -
508.namd_r -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) (percentage) -56.25% 0.640 0.280 - 0.00% -
531.deepsjeng_r (max) -47.76% 67.000 35.000 - -2.78% -
531.deepsjeng_r (max) (percentage) -46.75% 0.770 0.410 - -2.38% -
538.imagick_r (avg) (percentage) -40.00% 0.300 0.180 - 0.00% -
538.imagick_r -39.02% 41.000 25.000 - 0.00% -
538.imagick_r (avg) -39.02% 41.000 25.000 - 0.00% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
520.omnetpp_r (max) (percentage) -34.40% 2.500 1.640 - 0.00% -
520.omnetpp_r (max) -32.79% 366.000 246.000 - 2.93% -
510.parest_r (max) -32.56% 43.000 29.000 - 0.00% -
541.leela_r -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) (percentage) -31.82% 0.440 0.300 - 0.00% -
510.parest_r (max) (percentage) -31.58% 0.190 0.130 - 0.00% -
505.mcf_r -26.03% 73.000 54.000 - -1.82% -
505.mcf_r (avg) -26.03% 73.000 54.000 - -1.82% -
505.mcf_r (avg) (percentage) -25.00% 0.520 0.390 - 0.00% -
523.xalancbmk_r (max) -25.00% 52.000 39.000 - 0.00% -
523.xalancbmk_r (max) (percentage) -23.68% 0.380 0.290 - 0.00% -
525.x264_r (max) -18.18% 11.000 9.000 - 12.50% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
523.xalancbmk_r -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) -8.33% 0.240 0.220 - 0.00% -
500.perlbench_r (avg) (percentage) -7.69% 0.910 0.840 - 0.00% -
500.perlbench_r -7.53% 146.000 135.000 - 0.75% -
500.perlbench_r (avg) -7.53% 146.000 135.000 - 0.75% -
538.imagick_r (max) -5.17% 58.000 55.000 - -3.51% -
538.imagick_r (max) (percentage) -4.76% 0.420 0.400 - -2.44% -
511.povray_r (max) -2.54% 118.000 115.000 - -2.54% -

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)

Report Time: 0.34s
Execution (seconds)
Name Current %
500.perlbench_r 16168.000 -
502.gcc_r 10961.000 -
505.mcf_r 13997.000 -
507.cactuBSSN_r 11365.000 -1.11%
508.namd_r 9541.000 -
510.parest_r 22887.000 -
511.povray_r 20240.000 -
519.lbm_r 16870.000 -
520.omnetpp_r 15000.000 2.92%
523.xalancbmk_r 13643.000 -
525.x264_r 6449.000 -
526.blender_r 11891.000 -
531.deepsjeng_r 8696.000 1.34%
538.imagick_r 13945.000 -
541.leela_r 13400.000 -
544.nab_r 14421.000 -
557.xz_r 11650.000 -
Geometric Mean 13030.777 -
Code Size (bytes)
Name Current %
500.perlbench_r 2149507.000 -
502.gcc_r 8950576.000 -
505.mcf_r 27175.000 -
507.cactuBSSN_r 3678323.000 -
508.namd_r 614295.000 -
510.parest_r 8654420.000 -
511.povray_r 916748.000 -
519.lbm_r 13343.000 -
520.omnetpp_r 2152201.000 -
523.xalancbmk_r 4721839.000 -
525.x264_r 498558.000 -
526.blender_r 13703222.000 -
531.deepsjeng_r 81162.000 -
538.imagick_r 1697669.000 -
541.leela_r 145083.000 -
544.nab_r 191788.000 -
557.xz_r 177176.000 -
Geometric Mean 739375.773 -
sample variability
Name Current %
500.perlbench_r 135.000 -
500.perlbench_r (avg) 135.000 -
500.perlbench_r (avg) (percentage) 0.840 -
500.perlbench_r (max) 308.000 -1.28%
500.perlbench_r (max) (percentage) 1.910 -2.05%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.250 -
505.mcf_r 54.000 -1.82%
505.mcf_r (avg) 54.000 -1.82%
505.mcf_r (avg) (percentage) 0.390 -
505.mcf_r (max) 22.000 -
505.mcf_r (max) (percentage) 0.160 -
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 26.000 -
508.namd_r (avg) 26.000 -
508.namd_r (avg) (percentage) 0.280 -
508.namd_r (max) 23.000 -4.17%
508.namd_r (max) (percentage) 0.250 -3.85%
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 29.000 -
510.parest_r (max) (percentage) 0.130 -
511.povray_r 82.000 -1.20%
511.povray_r (avg) 82.000 -1.20%
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 115.000 -2.54%
511.povray_r (max) (percentage) 0.570 -1.72%
519.lbm_r 40.000 5.26%
519.lbm_r (avg) 40.000 5.26%
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 72.000 213.04%
519.lbm_r (max) (percentage) 0.430 207.14%
519.lbm_r (unexplained diff vs previous) 73.000 329.41%
519.lbm_r (unexplained diff vs previous) (percentage) 0.430 330.00%
520.omnetpp_r 174.000 2.96%
520.omnetpp_r (avg) 174.000 2.96%
520.omnetpp_r (avg) (percentage) 1.160 -
520.omnetpp_r (max) 246.000 2.93%
520.omnetpp_r (max) (percentage) 1.640 -
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 30.000 -
523.xalancbmk_r (avg) 30.000 -
523.xalancbmk_r (avg) (percentage) 0.220 -
523.xalancbmk_r (max) 39.000 -
523.xalancbmk_r (max) (percentage) 0.290 -
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 9.000 12.50%
525.x264_r (max) (percentage) 0.140 -
526.blender_r 85.000 -
526.blender_r (avg) 85.000 -
526.blender_r (avg) (percentage) 0.720 -
526.blender_r (max) 64.000 -1.54%
526.blender_r (max) (percentage) 0.540 -1.82%
531.deepsjeng_r 42.000 -
531.deepsjeng_r (avg) 42.000 -
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 35.000 -2.78%
531.deepsjeng_r (max) (percentage) 0.410 -2.38%
538.imagick_r 25.000 -
538.imagick_r (avg) 25.000 -
538.imagick_r (avg) (percentage) 0.180 -
538.imagick_r (max) 55.000 -3.51%
538.imagick_r (max) (percentage) 0.400 -
541.leela_r 40.000 -
541.leela_r (avg) 40.000 -
541.leela_r (avg) (percentage) 0.300 -
541.leela_r (max) 77.000 -4.94%
541.leela_r (max) (percentage) 0.580 -3.33%
544.nab_r 43.000 -
544.nab_r (avg) 43.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 47.000 -2.08%
544.nab_r (max) (percentage) 0.330 -2.94%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.260 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.440 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.190 -
557.xz_r 12.000 -
557.xz_r (avg) 12.000 -
557.xz_r (avg) (percentage) 0.110 -
557.xz_r (max) 4.000 -20.00%
557.xz_r (max) (percentage) 0.040 -20.00%
Geometric Mean 3.680 5.12%