Fields

hardware None
os None

Parameters

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/9743b99cd1d1775f9f367e5f1c6d40ba09ec523b (+ 844 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-cpu2017rate--llvm-arm-master-Os-build/235/
Filter

llvm-arm-master-Os test results

Run Order Start Time Duration
Current 0235 (llvmorg-21-init-02307-g9743b99cd1d1) 2025-02-13T20:00:24 0:00:00
Previous 0233 (llvmorg-21-init-01463-g9d7177a2d7e6) 2025-02-07T23:34:25 0:00:00
Baseline 0002 (llvmorg-17-init-04550-g890e6c871d31) 2023-03-11T10:55:27 0:00:00


Tests Summary

Status Group # # (B)
Performance Regressions 10 5
Performance Improvements 44 7
Added Tests 0 72
Unchanged Tests 558 528
Total Tests 612

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (unexplained diff vs previous) 116.67% 6.000 13.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) 111.76% 0.170 0.360 - 0.00% -
523.xalancbmk_r (unexplained diff vs previous) 75.00% 20.000 35.000 - 0.00% -
523.xalancbmk_r (unexplained diff vs previous) (percentage) 71.70% 0.530 0.910 - 0.00% -
505.mcf_r (unexplained diff vs previous) 24.32% 37.000 46.000 - 0.00% -
505.mcf_r (unexplained diff vs previous) (percentage) 22.22% 0.720 0.880 - 0.00% -
523.xalancbmk_r 3.45% 29.000 30.000 - 0.00% -
523.xalancbmk_r (avg) 3.45% 29.000 30.000 - 0.00% -
525.x264_r (avg) (percentage) 2.38% 0.420 0.430 - 0.00% -
523.xalancbmk_r (avg) (percentage) 1.28% 0.780 0.790 - 0.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
502.gcc_r -1.09% 3678.000 3638.000 - -1.33% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
531.deepsjeng_r (unexplained diff vs previous) -96.00% 25.000 1.000 - 0.00% -
531.deepsjeng_r (unexplained diff vs previous) (percentage) -95.83% 0.480 0.020 - 0.00% -
557.xz_r (unexplained diff vs previous) (percentage) -88.06% 0.670 0.080 - 0.00% -
557.xz_r (unexplained diff vs previous) -87.50% 16.000 2.000 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) -83.33% 36.000 6.000 - 0.00% -
520.omnetpp_r (unexplained diff vs previous) (percentage) -83.18% 1.070 0.180 - 0.00% -
508.namd_r (unexplained diff vs previous) -70.37% 27.000 8.000 - 0.00% -
508.namd_r (unexplained diff vs previous) (percentage) -70.11% 0.870 0.260 - 0.00% -
541.leela_r (unexplained diff vs previous) (percentage) -53.85% 0.260 0.120 - 0.00% -
541.leela_r (unexplained diff vs previous) -53.85% 13.000 6.000 - 0.00% -
544.nab_r (max) -50.00% 58.000 29.000 - 0.00% -
544.nab_r (max) (percentage) -48.65% 0.740 0.380 - 0.00% -
526.blender_r (unexplained diff vs previous) -36.73% 49.000 31.000 - 0.00% -
526.blender_r (unexplained diff vs previous) (percentage) -36.36% 0.440 0.280 - 0.00% -
525.x264_r (unexplained diff vs previous) -33.33% 39.000 26.000 - 0.00% -
525.x264_r (unexplained diff vs previous) (percentage) -32.82% 1.310 0.880 - 0.00% -
520.omnetpp_r (max) -25.00% 36.000 27.000 - 0.00% -
520.omnetpp_r (max) (percentage) -25.00% 1.080 0.810 - 0.00% -
541.leela_r (max) -15.38% 13.000 11.000 - 0.00% -
526.blender_r (max) -14.58% 48.000 41.000 - 0.00% -
526.blender_r (max) (percentage) -13.64% 0.440 0.380 - 0.00% -
508.namd_r (max) (percentage) -12.79% 0.860 0.750 - 0.00% -
544.nab_r (unexplained diff vs previous) (percentage) -12.12% 0.330 0.290 - 0.00% -
508.namd_r (max) -11.54% 26.000 23.000 - 0.00% -
544.nab_r (unexplained diff vs previous) -11.54% 26.000 23.000 - 0.00% -
541.leela_r (max) (percentage) -11.54% 0.260 0.230 - 0.00% -
500.perlbench_r (max) -10.00% 10.000 9.000 - 0.00% -
505.mcf_r (max) (percentage) -8.62% 1.160 1.060 - 0.00% -
531.deepsjeng_r (max) (percentage) -7.89% 0.760 0.700 - 0.00% -
525.x264_r (max) -7.69% 39.000 36.000 - 0.00% -
531.deepsjeng_r (max) -7.69% 39.000 36.000 - 0.00% -
505.mcf_r (max) -6.78% 59.000 55.000 - 0.00% -
525.x264_r (max) (percentage) -6.06% 1.320 1.240 - 0.00% -
557.xz_r (max) (percentage) -5.61% 1.960 1.850 - 0.00% -
523.xalancbmk_r (max) (percentage) -4.74% 1.900 1.810 - 0.00% -
557.xz_r (max) -4.35% 46.000 44.000 - 0.00% -
523.xalancbmk_r (max) -4.17% 72.000 69.000 - 0.00% -
538.imagick_r (max) (percentage) -3.70% 0.270 0.260 - 0.00% -
511.povray_r (max) (percentage) -3.26% 0.920 0.890 - 0.00% -
526.blender_r (avg) (percentage) -2.38% 0.420 0.410 - 0.00% -
526.blender_r -2.17% 46.000 45.000 - 0.00% -
526.blender_r (avg) -2.17% 46.000 45.000 - 0.00% -
511.povray_r (avg) (percentage) -1.25% 0.800 0.790 - 0.00% -

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
505.mcf_r 4.86% 17534.000 18386.000 - 0.00% -
557.xz_r 1.41% 120580.000 122280.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r 3.32% 3253.000 3361.000 - -0.18% -
544.nab_r 2.95% 7637.000 7862.000 - 0.29% -
508.namd_r 1.01% 3079.000 3110.000 - 0.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
511.povray_r -1.83% 591352.000 580549.000 - 0.00% -
525.x264_r -1.12% 325233.000 321593.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -6.97% 3184.000 2962.000 - -0.87% -
541.leela_r -1.50% 5137.000 5060.000 - 0.12% -
502.gcc_r -1.33% 3687.000 3638.000 - -1.09% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful 7.14% 14.000 15.000 - 0.00% -
nb_execution_successful 7.14% 14.000 15.000 - 0.00% -

Added Tests - Code Size
500.perlbench_r

Added Tests - Execution
500.perlbench_r

Added Tests - sample variability
500.perlbench_r
500.perlbench_r (avg)
500.perlbench_r (avg) (percentage)
500.perlbench_r (max)
500.perlbench_r (max) (percentage)
505.mcf_r
505.mcf_r (avg)
505.mcf_r (avg) (percentage)
505.mcf_r (max)
505.mcf_r (max) (percentage)
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
511.povray_r
511.povray_r (avg)
511.povray_r (avg) (percentage)
511.povray_r (max)
511.povray_r (max) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
523.xalancbmk_r
523.xalancbmk_r (avg)
523.xalancbmk_r (avg) (percentage)
523.xalancbmk_r (max)
523.xalancbmk_r (max) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
526.blender_r
526.blender_r (avg)
526.blender_r (avg) (percentage)
526.blender_r (max)
526.blender_r (max) (percentage)
531.deepsjeng_r
531.deepsjeng_r (avg)
531.deepsjeng_r (avg) (percentage)
531.deepsjeng_r (max)
531.deepsjeng_r (max) (percentage)
538.imagick_r
538.imagick_r (avg)
538.imagick_r (avg) (percentage)
538.imagick_r (max)
538.imagick_r (max) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
544.nab_r
544.nab_r (avg)
544.nab_r (avg) (percentage)
544.nab_r (max)
544.nab_r (max) (percentage)
557.xz_r
557.xz_r (avg)
557.xz_r (avg) (percentage)
557.xz_r (max)
557.xz_r (max) (percentage)


Report Time: 0.22s
Name Current %
500.perlbench_r 2372.000 -
502.gcc_r 3638.000 -1.09%
505.mcf_r 5204.000 -
508.namd_r 3110.000 -
511.povray_r 464.000 -
519.lbm_r 3578.000 -
520.omnetpp_r 3361.000 -
523.xalancbmk_r 3837.000 -
525.x264_r 2962.000 -
526.blender_r 11046.000 -
531.deepsjeng_r 5189.000 -
538.imagick_r 4049.000 -
541.leela_r 5060.000 -
544.nab_r 7862.000 -
557.xz_r 2396.000 -
Geometric Mean 3571.130 -
Name Current %
500.perlbench_r 1273102.000 -
502.gcc_r 5341988.000 -
505.mcf_r 18386.000 -
508.namd_r 395410.000 -
511.povray_r 580549.000 -
519.lbm_r 11490.000 -
520.omnetpp_r 1099266.000 -
523.xalancbmk_r 2464429.000 -
525.x264_r 321593.000 -
526.blender_r 6880215.000 -
531.deepsjeng_r 61140.000 -
538.imagick_r 1248956.000 -
541.leela_r 81308.000 -
544.nab_r 104317.000 -
557.xz_r 122280.000 -
Geometric Mean 351814.391 -
Name Current %
500.perlbench_r 16.000 -
500.perlbench_r (avg) 16.000 -
500.perlbench_r (avg) (percentage) 0.710 -
500.perlbench_r (max) 9.000 -10.00%
500.perlbench_r (max) (percentage) 0.420 -
505.mcf_r 35.000 -
505.mcf_r (avg) 35.000 -
505.mcf_r (avg) (percentage) 0.690 -
505.mcf_r (max) 55.000 -6.78%
505.mcf_r (max) (percentage) 1.060 -8.62%
505.mcf_r (unexplained diff vs previous) 46.000 24.32%
505.mcf_r (unexplained diff vs previous) (percentage) 0.880 22.22%
508.namd_r 11.000 -
508.namd_r (avg) 11.000 -
508.namd_r (avg) (percentage) 0.360 -
508.namd_r (max) 23.000 -11.54%
508.namd_r (max) (percentage) 0.750 -12.79%
508.namd_r (unexplained diff vs previous) 8.000 -70.37%
508.namd_r (unexplained diff vs previous) (percentage) 0.260 -70.11%
511.povray_r 3.000 -
511.povray_r (avg) 3.000 -
511.povray_r (avg) (percentage) 0.790 -1.25%
511.povray_r (max) 4.000 -
511.povray_r (max) (percentage) 0.890 -3.26%
511.povray_r (unexplained diff vs previous) 2.000 -
511.povray_r (unexplained diff vs previous) (percentage) 0.430 -
519.lbm_r 11.000 -
519.lbm_r (avg) 11.000 -
519.lbm_r (avg) (percentage) 0.310 -
519.lbm_r (max) 12.000 -
519.lbm_r (max) (percentage) 0.360 -
519.lbm_r (unexplained diff vs previous) 13.000 116.67%
519.lbm_r (unexplained diff vs previous) (percentage) 0.360 111.76%
520.omnetpp_r 38.000 -
520.omnetpp_r (avg) 38.000 -
520.omnetpp_r (avg) (percentage) 1.140 -
520.omnetpp_r (max) 27.000 -25.00%
520.omnetpp_r (max) (percentage) 0.810 -25.00%
520.omnetpp_r (unexplained diff vs previous) 6.000 -83.33%
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.180 -83.18%
523.xalancbmk_r 30.000 3.45%
523.xalancbmk_r (avg) 30.000 3.45%
523.xalancbmk_r (avg) (percentage) 0.790 1.28%
523.xalancbmk_r (max) 69.000 -4.17%
523.xalancbmk_r (max) (percentage) 1.810 -4.74%
523.xalancbmk_r (unexplained diff vs previous) 35.000 75.00%
523.xalancbmk_r (unexplained diff vs previous) (percentage) 0.910 71.70%
525.x264_r 12.000 -
525.x264_r (avg) 12.000 -
525.x264_r (avg) (percentage) 0.430 2.38%
525.x264_r (max) 36.000 -7.69%
525.x264_r (max) (percentage) 1.240 -6.06%
525.x264_r (unexplained diff vs previous) 26.000 -33.33%
525.x264_r (unexplained diff vs previous) (percentage) 0.880 -32.82%
526.blender_r 45.000 -2.17%
526.blender_r (avg) 45.000 -2.17%
526.blender_r (avg) (percentage) 0.410 -2.38%
526.blender_r (max) 41.000 -14.58%
526.blender_r (max) (percentage) 0.380 -13.64%
526.blender_r (unexplained diff vs previous) 31.000 -36.73%
526.blender_r (unexplained diff vs previous) (percentage) 0.280 -36.36%
531.deepsjeng_r 16.000 -
531.deepsjeng_r (avg) 16.000 -
531.deepsjeng_r (avg) (percentage) 0.310 -
531.deepsjeng_r (max) 36.000 -7.69%
531.deepsjeng_r (max) (percentage) 0.700 -7.89%
531.deepsjeng_r (unexplained diff vs previous) 1.000 -96.00%
531.deepsjeng_r (unexplained diff vs previous) (percentage) 0.020 -95.83%
538.imagick_r 8.000 -
538.imagick_r (avg) 8.000 -
538.imagick_r (avg) (percentage) 0.200 -
538.imagick_r (max) 10.000 -
538.imagick_r (max) (percentage) 0.260 -3.70%
541.leela_r 20.000 -
541.leela_r (avg) 20.000 -
541.leela_r (avg) (percentage) 0.410 -
541.leela_r (max) 11.000 -15.38%
541.leela_r (max) (percentage) 0.230 -11.54%
541.leela_r (unexplained diff vs previous) 6.000 -53.85%
541.leela_r (unexplained diff vs previous) (percentage) 0.120 -53.85%
544.nab_r 35.000 -
544.nab_r (avg) 35.000 -
544.nab_r (avg) (percentage) 0.450 -
544.nab_r (max) 29.000 -50.00%
544.nab_r (max) (percentage) 0.380 -48.65%
544.nab_r (unexplained diff vs previous) 23.000 -11.54%
544.nab_r (unexplained diff vs previous) (percentage) 0.290 -12.12%
557.xz_r 19.000 -
557.xz_r (avg) 19.000 -
557.xz_r (avg) (percentage) 0.810 -
557.xz_r (max) 44.000 -4.35%
557.xz_r (max) (percentage) 1.850 -5.61%
557.xz_r (unexplained diff vs previous) 2.000 -87.50%
557.xz_r (unexplained diff vs previous) (percentage) 0.080 -88.06%
Geometric Mean 3.508 -19.75%