Fields

hardware None
os None

Parameters

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/678cdd67b957cb49e7fe2cce2610cd720cbdf130 (+ 423 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-cpu2017rate--llvm-arm-master-Oz-build/254/
Filter

llvm-arm-master-Oz test results

Run Order Start Time Duration
Current 0254 (llvmorg-21-init-14517-g678cdd67b957) 2025-06-01T03:28:35 0:00:00
Previous 0253 (llvmorg-21-init-14094-g972ecc31f72e) 2025-05-27T00:58:16 0:00:00
Baseline 0002 (llvmorg-17-init-04570-g24416d22bb28) 2023-03-12T06:27:23 0:00:00


Tests Summary

Status Group # # (B)
Performance Regressions 41 3
Performance Improvements 21 8
Added Tests 2 76
Unchanged Tests 566 543
Total Tests 630

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r 3.79% 3849.000 3995.000 - 4.77% -
557.xz_r 1.10% 2537.000 2565.000 - -0.27% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
531.deepsjeng_r (unexplained diff vs previous) (percentage) 620.00% 0.050 0.360 - 0.00% -
505.mcf_r (unexplained diff vs previous) (percentage) 610.00% 0.100 0.710 - 0.00% -
505.mcf_r (unexplained diff vs previous) 600.00% 6.000 42.000 - 0.00% -
531.deepsjeng_r (unexplained diff vs previous) 566.67% 3.000 20.000 - 0.00% -
557.xz_r (unexplained diff vs previous) 460.00% 5.000 28.000 - 0.00% -
557.xz_r (unexplained diff vs previous) (percentage) 445.00% 0.200 1.090 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) 217.39% 46.000 146.000 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 204.17% 1.200 3.650 - 0.00% -
505.mcf_r (max) (percentage) 195.83% 0.240 0.710 - 0.00% -
505.mcf_r (max) 192.86% 14.000 41.000 - 0.00% -
525.x264_r (unexplained diff vs previous) (percentage) 179.41% 0.340 0.950 - 0.00% -
525.x264_r (unexplained diff vs previous) 178.57% 14.000 39.000 - 0.00% -
520.omnetpp_r (max) 164.91% 57.000 151.000 - 0.00% -
557.xz_r (max) 154.55% 11.000 28.000 - 0.00% -
520.omnetpp_r (max) (percentage) 152.67% 1.500 3.790 - 0.00% -
557.xz_r (max) (percentage) 134.04% 0.470 1.100 - 0.00% -
531.deepsjeng_r (max) 81.82% 11.000 20.000 - 0.00% -
508.namd_r (unexplained diff vs previous) (percentage) 75.00% 0.080 0.140 - 0.00% -
531.deepsjeng_r (max) (percentage) 71.43% 0.210 0.360 - 0.00% -
508.namd_r (unexplained diff vs previous) 66.67% 3.000 5.000 - 0.00% -
519.lbm_r (max) 25.00% 8.000 10.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) 25.00% 8.000 10.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) 23.81% 0.210 0.260 - 0.00% -
519.lbm_r (max) (percentage) 18.18% 0.220 0.260 - 0.00% -
541.leela_r (unexplained diff vs previous) 18.18% 33.000 39.000 - 0.00% -
541.leela_r (unexplained diff vs previous) (percentage) 17.74% 0.620 0.730 - 0.00% -
510.parest_r (max) (percentage) 12.50% 0.240 0.270 - 0.00% -
544.nab_r (unexplained diff vs previous) (percentage) 12.50% 0.320 0.360 - 0.00% -
544.nab_r (unexplained diff vs previous) 11.11% 27.000 30.000 - 0.00% -
520.omnetpp_r 6.67% 45.000 48.000 - 0.00% -
520.omnetpp_r (avg) 6.67% 45.000 48.000 - 0.00% -
510.parest_r (max) 6.25% 16.000 17.000 - 0.00% -
541.leela_r 4.55% 22.000 23.000 - 0.00% -
541.leela_r (avg) 4.55% 22.000 23.000 - 0.00% -
520.omnetpp_r (avg) (percentage) 4.27% 1.170 1.220 - 0.00% -
531.deepsjeng_r (avg) (percentage) 3.03% 0.330 0.340 - 0.00% -
525.x264_r (avg) (percentage) 2.56% 0.390 0.400 - 0.00% -
541.leela_r (avg) (percentage) 2.33% 0.430 0.440 - 0.00% -
505.mcf_r (avg) (percentage) 1.54% 0.650 0.660 - 0.00% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
511.povray_r (unexplained diff vs previous) -50.00% 4.000 2.000 - 0.00% -
511.povray_r (unexplained diff vs previous) (percentage) -49.35% 0.770 0.390 - 0.00% -
511.povray_r -20.00% 5.000 4.000 - 0.00% -
511.povray_r (avg) -20.00% 5.000 4.000 - 0.00% -
526.blender_r (unexplained diff vs previous) -15.38% 52.000 44.000 - 0.00% -
526.blender_r (unexplained diff vs previous) (percentage) -15.00% 0.400 0.340 - 0.00% -
508.namd_r (max) -9.52% 21.000 19.000 - 0.00% -
526.blender_r (max) (percentage) -8.86% 0.790 0.720 - 0.00% -
526.blender_r (max) -8.74% 103.000 94.000 - 0.00% -
523.xalancbmk_r (max) -6.90% 29.000 27.000 - 0.00% -
508.namd_r (max) (percentage) -6.90% 0.580 0.540 - 0.00% -
511.povray_r (max) (percentage) -6.11% 1.310 1.230 - 0.00% -
510.parest_r -5.56% 18.000 17.000 - 0.00% -
510.parest_r (avg) -5.56% 18.000 17.000 - 0.00% -
541.leela_r (max) (percentage) -5.13% 0.780 0.740 - 0.00% -
525.x264_r (max) -5.00% 40.000 38.000 - 0.00% -
541.leela_r (max) -4.88% 41.000 39.000 - 0.00% -
523.xalancbmk_r (max) (percentage) -4.76% 0.630 0.600 - 0.00% -
525.x264_r (max) (percentage) -3.09% 0.970 0.940 - 0.00% -
508.namd_r (avg) (percentage) -2.56% 0.390 0.380 - 0.00% -
511.povray_r (avg) (percentage) -2.06% 0.970 0.950 - 0.00% -

Added Tests - sample variability
510.parest_r (unexplained diff vs previous)
510.parest_r (unexplained diff vs previous) (percentage)

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r 4.77% 3813.000 3995.000 - 3.79% -
505.mcf_r 4.33% 5634.000 5878.000 - -0.71% -
544.nab_r 3.97% 8092.000 8413.000 - 0.36% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
538.imagick_r -1.68% 1079012.000 1060867.000 - 0.00% -
511.povray_r -1.41% 551640.000 543867.000 - 0.00% -
505.mcf_r -1.39% 15863.000 15643.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
538.imagick_r -1.38% 4424.000 4363.000 - 0.02% -
523.xalancbmk_r -1.01% 4674.000 4627.000 - -0.11% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful 14.29% 14.000 16.000 - 0.00% -
nb_execution_successful 7.14% 14.000 15.000 - 0.00% -
nb_total_tests 6.67% 15.000 16.000 - 0.00% -

Added Tests - Code Size
510.parest_r

Added Tests - Execution
510.parest_r

Added Tests - sample variability
500.perlbench_r (avg) (percentage)
500.perlbench_r (max) (percentage)
505.mcf_r
505.mcf_r (avg)
505.mcf_r (avg) (percentage)
505.mcf_r (max)
505.mcf_r (max) (percentage)
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
510.parest_r
510.parest_r (avg)
510.parest_r (avg) (percentage)
510.parest_r (max)
510.parest_r (max) (percentage)
510.parest_r (unexplained diff vs previous)
510.parest_r (unexplained diff vs previous) (percentage)
511.povray_r
511.povray_r (avg)
511.povray_r (avg) (percentage)
511.povray_r (max)
511.povray_r (max) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
523.xalancbmk_r
523.xalancbmk_r (avg)
523.xalancbmk_r (avg) (percentage)
523.xalancbmk_r (max)
523.xalancbmk_r (max) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
526.blender_r
526.blender_r (avg)
526.blender_r (avg) (percentage)
526.blender_r (max)
526.blender_r (max) (percentage)
531.deepsjeng_r
531.deepsjeng_r (avg)
531.deepsjeng_r (avg) (percentage)
531.deepsjeng_r (max)
531.deepsjeng_r (max) (percentage)
538.imagick_r
538.imagick_r (avg)
538.imagick_r (avg) (percentage)
538.imagick_r (max)
538.imagick_r (max) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
544.nab_r
544.nab_r (avg)
544.nab_r (avg) (percentage)
544.nab_r (max)
544.nab_r (max) (percentage)
557.xz_r
557.xz_r (avg)
557.xz_r (avg) (percentage)
557.xz_r (max)
557.xz_r (max) (percentage)


Report Time: 0.22s
Name Current %
502.gcc_r 4115.000 -
505.mcf_r 5878.000 -
508.namd_r 3695.000 -
510.parest_r 6650.000 -
511.povray_r 516.000 -
519.lbm_r 3876.000 -
520.omnetpp_r 3995.000 3.79%
523.xalancbmk_r 4627.000 -
525.x264_r 4089.000 -
526.blender_r 13133.000 -
531.deepsjeng_r 5578.000 -
538.imagick_r 4363.000 -
541.leela_r 5349.000 -
544.nab_r 8413.000 -
557.xz_r 2565.000 1.10%
Geometric Mean 4312.172 -
Name Current %
502.gcc_r 4830527.000 -
505.mcf_r 15643.000 -
508.namd_r 352203.000 -
510.parest_r 3359547.000 -
511.povray_r 543867.000 -
519.lbm_r 11434.000 -
520.omnetpp_r 994337.000 -
523.xalancbmk_r 2259566.000 -
525.x264_r 274596.000 -
526.blender_r 6532585.000 -
531.deepsjeng_r 58967.000 -
538.imagick_r 1060867.000 -
541.leela_r 71987.000 -
544.nab_r 95257.000 -
557.xz_r 115528.000 -
Geometric Mean 343849.468 -
Name Current %
500.perlbench_r (avg) (percentage) 0.800 -
500.perlbench_r (max) (percentage) 0.690 -
505.mcf_r 38.000 -
505.mcf_r (avg) 38.000 -
505.mcf_r (avg) (percentage) 0.660 1.54%
505.mcf_r (max) 41.000 192.86%
505.mcf_r (max) (percentage) 0.710 195.83%
505.mcf_r (unexplained diff vs previous) 42.000 600.00%
505.mcf_r (unexplained diff vs previous) (percentage) 0.710 610.00%
508.namd_r 14.000 -
508.namd_r (avg) 14.000 -
508.namd_r (avg) (percentage) 0.380 -2.56%
508.namd_r (max) 19.000 -9.52%
508.namd_r (max) (percentage) 0.540 -6.90%
508.namd_r (unexplained diff vs previous) 5.000 66.67%
508.namd_r (unexplained diff vs previous) (percentage) 0.140 75.00%
510.parest_r 17.000 -5.56%
510.parest_r (avg) 17.000 -5.56%
510.parest_r (avg) (percentage) 0.270 -
510.parest_r (max) 17.000 6.25%
510.parest_r (max) (percentage) 0.270 12.50%
510.parest_r (unexplained diff vs previous) 18.000 -
510.parest_r (unexplained diff vs previous) (percentage) 0.270 -
511.povray_r 4.000 -20.00%
511.povray_r (avg) 4.000 -20.00%
511.povray_r (avg) (percentage) 0.950 -2.06%
511.povray_r (max) 6.000 -
511.povray_r (max) (percentage) 1.230 -6.11%
511.povray_r (unexplained diff vs previous) 2.000 -50.00%
511.povray_r (unexplained diff vs previous) (percentage) 0.390 -49.35%
519.lbm_r 10.000 -
519.lbm_r (avg) 10.000 -
519.lbm_r (avg) (percentage) 0.260 -
519.lbm_r (max) 10.000 25.00%
519.lbm_r (max) (percentage) 0.260 18.18%
519.lbm_r (unexplained diff vs previous) 10.000 25.00%
519.lbm_r (unexplained diff vs previous) (percentage) 0.260 23.81%
520.omnetpp_r 48.000 6.67%
520.omnetpp_r (avg) 48.000 6.67%
520.omnetpp_r (avg) (percentage) 1.220 4.27%
520.omnetpp_r (max) 151.000 164.91%
520.omnetpp_r (max) (percentage) 3.790 152.67%
520.omnetpp_r (unexplained diff vs previous) 146.000 217.39%
520.omnetpp_r (unexplained diff vs previous) (percentage) 3.650 204.17%
523.xalancbmk_r 26.000 -
523.xalancbmk_r (avg) 26.000 -
523.xalancbmk_r (avg) (percentage) 0.580 -
523.xalancbmk_r (max) 27.000 -6.90%
523.xalancbmk_r (max) (percentage) 0.600 -4.76%
525.x264_r 16.000 -
525.x264_r (avg) 16.000 -
525.x264_r (avg) (percentage) 0.400 2.56%
525.x264_r (max) 38.000 -5.00%
525.x264_r (max) (percentage) 0.940 -3.09%
525.x264_r (unexplained diff vs previous) 39.000 178.57%
525.x264_r (unexplained diff vs previous) (percentage) 0.950 179.41%
526.blender_r 52.000 -
526.blender_r (avg) 52.000 -
526.blender_r (avg) (percentage) 0.400 -
526.blender_r (max) 94.000 -8.74%
526.blender_r (max) (percentage) 0.720 -8.86%
526.blender_r (unexplained diff vs previous) 44.000 -15.38%
526.blender_r (unexplained diff vs previous) (percentage) 0.340 -15.00%
531.deepsjeng_r 18.000 -
531.deepsjeng_r (avg) 18.000 -
531.deepsjeng_r (avg) (percentage) 0.340 3.03%
531.deepsjeng_r (max) 20.000 81.82%
531.deepsjeng_r (max) (percentage) 0.360 71.43%
531.deepsjeng_r (unexplained diff vs previous) 20.000 566.67%
531.deepsjeng_r (unexplained diff vs previous) (percentage) 0.360 620.00%
538.imagick_r 5.000 -
538.imagick_r (avg) 5.000 -
538.imagick_r (avg) (percentage) 0.130 -
538.imagick_r (max) 6.000 -
538.imagick_r (max) (percentage) 0.140 -
541.leela_r 23.000 4.55%
541.leela_r (avg) 23.000 4.55%
541.leela_r (avg) (percentage) 0.440 2.33%
541.leela_r (max) 39.000 -4.88%
541.leela_r (max) (percentage) 0.740 -5.13%
541.leela_r (unexplained diff vs previous) 39.000 18.18%
541.leela_r (unexplained diff vs previous) (percentage) 0.730 17.74%
544.nab_r 30.000 -
544.nab_r (avg) 30.000 -
544.nab_r (avg) (percentage) 0.360 -
544.nab_r (max) 30.000 -
544.nab_r (max) (percentage) 0.360 -
544.nab_r (unexplained diff vs previous) 30.000 11.11%
544.nab_r (unexplained diff vs previous) (percentage) 0.360 12.50%
557.xz_r 22.000 -
557.xz_r (avg) 22.000 -
557.xz_r (avg) (percentage) 0.870 -
557.xz_r (max) 28.000 154.55%
557.xz_r (max) (percentage) 1.100 134.04%
557.xz_r (unexplained diff vs previous) 28.000 460.00%
557.xz_r (unexplained diff vs previous) (percentage) 1.090 445.00%
Geometric Mean 4.154 25.17%