Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/10217b9d9b4814eb086cffd7f008cb72967cd739 (+ 435 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/103/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0103 (llvmorg-18-init-06564-g10217b9d9b48) 2023-09-20T17:24:21 0:00:00
Previous 0102 (llvmorg-18-init-06129-ga292e7edf8b2) 2023-09-19T00:50:10 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 14 65
Performance Improvements 25 40
Added Tests 6 0
Unchanged Tests 831 771
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r 4.24% 14524.000 15140.000 - 3.26% -
511.povray_r 2.13% 19954.000 20380.000 - -5.42% -
500.perlbench_r 1.49% 15960.000 16198.000 - 0.92% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
520.omnetpp_r (unexplained diff vs previous) 1521.05% 38.000 616.000 - 1183.33% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 1465.38% 0.260 4.070 - 1133.33% -
520.omnetpp_r (max) 367.88% 137.000 641.000 - 75.14% -
520.omnetpp_r (max) (percentage) 346.32% 0.950 4.240 - 69.60% -
520.omnetpp_r 22.05% 127.000 155.000 - 15.67% -
520.omnetpp_r (avg) 22.05% 127.000 155.000 - 15.67% -
520.omnetpp_r (avg) (percentage) 17.05% 0.880 1.030 - 11.96% -
538.imagick_r 3.70% 27.000 28.000 - -31.71% -
538.imagick_r (avg) 3.70% 27.000 28.000 - -31.71% -
511.povray_r 2.67% 75.000 77.000 - 0.00% -
511.povray_r (avg) 2.67% 75.000 77.000 - 0.00% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
525.x264_r (max) -26.32% 19.000 14.000 - 27.27% -
525.x264_r (max) (percentage) -25.81% 0.310 0.230 - 53.33% -
525.x264_r -7.69% 13.000 12.000 - 100.00% -
525.x264_r (avg) -7.69% 13.000 12.000 - 100.00% -
508.namd_r (max) (percentage) -5.77% 0.520 0.490 - -56.64% -
541.leela_r (max) -5.71% 70.000 66.000 - -10.81% -
557.xz_r (max) -5.00% 20.000 19.000 - -9.52% -
523.xalancbmk_r (max) (percentage) -4.88% 0.410 0.390 - 2.63% -
525.x264_r (avg) (percentage) -4.55% 0.220 0.210 - 133.33% -
505.mcf_r (max) (percentage) -4.35% 0.460 0.440 - -35.29% -
510.parest_r (max) -4.11% 73.000 70.000 - 62.79% -
508.namd_r (max) -4.08% 49.000 47.000 - -56.48% -
541.leela_r (max) (percentage) -3.85% 0.520 0.500 - -9.09% -
523.xalancbmk_r (max) -3.64% 55.000 53.000 - 1.92% -
510.parest_r (max) (percentage) -3.13% 0.320 0.310 - 63.16% -
531.deepsjeng_r (max) (percentage) -3.13% 1.280 1.240 - 61.04% -
505.mcf_r (max) -3.08% 65.000 63.000 - -34.38% -
531.deepsjeng_r (max) -2.73% 110.000 107.000 - 59.70% -
544.nab_r (max) (percentage) -2.63% 0.380 0.370 - 42.31% -
510.parest_r -2.56% 39.000 38.000 - 11.76% -
510.parest_r (avg) -2.56% 39.000 38.000 - 11.76% -
538.imagick_r (max) (percentage) -2.17% 0.460 0.450 - 7.14% -
544.nab_r (max) -1.85% 54.000 53.000 - 43.24% -
538.imagick_r (max) -1.56% 64.000 63.000 - 8.62% -
526.blender_r (max) (percentage) -1.51% 3.310 3.260 - 805.56% -

Added Tests - sample variability
510.parest_r (unexplained diff vs previous)
510.parest_r (unexplained diff vs previous) (percentage)
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)
525.x264_r (unexplained diff vs previous)
525.x264_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r 3.26% 14662.000 15140.000 - 4.24% -
505.mcf_r 1.65% 14148.000 14381.000 - 0.32% -
507.cactuBSSN_r 1.17% 11293.000 11425.000 - -0.56% -
538.imagick_r 1.17% 13884.000 14046.000 - 0.49% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
520.omnetpp_r (unexplained diff vs previous) 1183.33% 48.000 616.000 - 1521.05% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 1133.33% 0.330 4.070 - 1465.38% -
510.parest_r (unexplained diff vs previous) 1050.00% 4.000 46.000 - 0.00% -
510.parest_r (unexplained diff vs previous) (percentage) 900.00% 0.020 0.200 - 0.00% -
526.blender_r (max) (percentage) 805.56% 0.360 3.260 - -1.51% -
526.blender_r (max) 788.64% 44.000 391.000 - -0.76% -
503.bwaves_r (max) (percentage) 200.00% 0.090 0.270 - 0.00% -
554.roms_r (max) (percentage) 200.00% 0.070 0.210 - 0.00% -
519.lbm_r (unexplained diff vs previous) 179.17% 24.000 67.000 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) 178.57% 0.140 0.390 - 0.00% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
526.blender_r (avg) (percentage) 139.39% 0.330 0.790 - 0.00% -
519.lbm_r (max) (percentage) 135.29% 0.170 0.400 - 0.00% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - -4.55% -
519.lbm_r (max) 131.03% 29.000 67.000 - 0.00% -
526.blender_r 129.27% 41.000 94.000 - 0.00% -
526.blender_r (avg) 129.27% 41.000 94.000 - 0.00% -
525.x264_r 100.00% 6.000 12.000 - -7.69% -
525.x264_r (avg) 100.00% 6.000 12.000 - -7.69% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
520.omnetpp_r (max) 75.14% 366.000 641.000 - 367.88% -
520.omnetpp_r (max) (percentage) 69.60% 2.500 4.240 - 346.32% -
510.parest_r (max) (percentage) 63.16% 0.190 0.310 - -3.13% -
510.parest_r (max) 62.79% 43.000 70.000 - -4.11% -
531.deepsjeng_r (max) (percentage) 61.04% 0.770 1.240 - -3.13% -
531.deepsjeng_r (max) 59.70% 67.000 107.000 - -2.73% -
525.x264_r (max) (percentage) 53.33% 0.150 0.230 - -25.81% -
544.nab_r (max) 43.24% 37.000 53.000 - -1.85% -
544.nab_r (max) (percentage) 42.31% 0.260 0.370 - -2.63% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 0.00% -
519.lbm_r 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (avg) 37.93% 29.000 40.000 - 0.00% -
531.deepsjeng_r (avg) (percentage) 31.71% 0.410 0.540 - 0.00% -
531.deepsjeng_r 27.78% 36.000 46.000 - 0.00% -
531.deepsjeng_r (avg) 27.78% 36.000 46.000 - 0.00% -
548.exchange2_r (max) (percentage) 27.27% 0.220 0.280 - 0.00% -
525.x264_r (max) 27.27% 11.000 14.000 - -26.32% -
557.xz_r (avg) (percentage) 20.00% 0.100 0.120 - 0.00% -
557.xz_r 18.18% 11.000 13.000 - 0.00% -
557.xz_r (avg) 18.18% 11.000 13.000 - 0.00% -
520.omnetpp_r 15.67% 134.000 155.000 - 22.05% -
520.omnetpp_r (avg) 15.67% 134.000 155.000 - 22.05% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
520.omnetpp_r (avg) (percentage) 11.96% 0.920 1.030 - 17.05% -
510.parest_r 11.76% 34.000 38.000 - -2.56% -
510.parest_r (avg) 11.76% 34.000 38.000 - -2.56% -
538.imagick_r (max) 8.62% 58.000 63.000 - -1.56% -
538.imagick_r (max) (percentage) 7.14% 0.420 0.450 - -2.17% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r (avg) (percentage) 5.56% 0.360 0.380 - 0.00% -
544.nab_r 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) 4.88% 41.000 43.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) 4.17% 0.240 0.250 - 0.00% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -
523.xalancbmk_r 3.03% 33.000 34.000 - 0.00% -
523.xalancbmk_r (avg) 3.03% 33.000 34.000 - 0.00% -
523.xalancbmk_r (max) (percentage) 2.63% 0.380 0.390 - -4.88% -
523.xalancbmk_r (max) 1.92% 52.000 53.000 - -3.64% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.41% 624815.000 615983.000 - -0.20% -
519.lbm_r -1.07% 13451.000 13307.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -17.37% 7490.000 6189.000 - 0.03% -
511.povray_r -5.42% 21547.000 20380.000 - 2.13% -
526.blender_r -3.47% 12432.000 12001.000 - 0.66% -
519.lbm_r -3.08% 17526.000 16986.000 - 0.40% -
541.leela_r -1.50% 13571.000 13367.000 - -0.92% -
531.deepsjeng_r -1.42% 8789.000 8664.000 - 0.51% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
500.perlbench_r (max) -92.02% 213.000 17.000 - 0.00% -
500.perlbench_r (max) (percentage) -91.73% 1.330 0.110 - 0.00% -
525.x264_r (unexplained diff vs previous) -81.82% 11.000 2.000 - 0.00% -
525.x264_r (unexplained diff vs previous) (percentage) -80.00% 0.150 0.030 - 0.00% -
549.fotonik3d_r (max) (percentage) -62.81% 1.210 0.450 - 0.00% -
508.namd_r (max) (percentage) -56.64% 1.130 0.490 - -5.77% -
508.namd_r (max) -56.48% 108.000 47.000 - -4.08% -
508.namd_r -54.10% 61.000 28.000 - 0.00% -
508.namd_r (avg) -54.10% 61.000 28.000 - 0.00% -
508.namd_r (avg) (percentage) -53.12% 0.640 0.300 - 0.00% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
505.mcf_r (max) (percentage) -35.29% 0.680 0.440 - -4.35% -
505.mcf_r (max) -34.38% 96.000 63.000 - -3.08% -
538.imagick_r (avg) (percentage) -33.33% 0.300 0.200 - 0.00% -
541.leela_r -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) -32.20% 59.000 40.000 - 0.00% -
541.leela_r (avg) (percentage) -31.82% 0.440 0.300 - 0.00% -
538.imagick_r -31.71% 41.000 28.000 - 3.70% -
538.imagick_r (avg) -31.71% 41.000 28.000 - 3.70% -
500.perlbench_r (avg) (percentage) -27.47% 0.910 0.660 - 0.00% -
500.perlbench_r -27.40% 146.000 106.000 - 0.95% -
500.perlbench_r (avg) -27.40% 146.000 106.000 - 0.95% -
511.povray_r (max) -21.19% 118.000 93.000 - 0.00% -
511.povray_r (max) (percentage) -16.36% 0.550 0.460 - -2.13% -
541.leela_r (max) -10.81% 74.000 66.000 - -5.71% -
557.xz_r (max) (percentage) -10.53% 0.190 0.170 - -5.56% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
505.mcf_r (avg) (percentage) -9.62% 0.520 0.470 - 0.00% -
557.xz_r (max) -9.52% 21.000 19.000 - -5.00% -
541.leela_r (max) (percentage) -9.09% 0.550 0.500 - -3.85% -
505.mcf_r -8.22% 73.000 67.000 - 0.00% -
505.mcf_r (avg) -8.22% 73.000 67.000 - 0.00% -

Report Time: 0.33s
Execution (seconds)
Name Current %
500.perlbench_r 16198.000 1.49%
502.gcc_r 10946.000 -
505.mcf_r 14381.000 -
507.cactuBSSN_r 11425.000 -
508.namd_r 9608.000 -
510.parest_r 22898.000 -
511.povray_r 20380.000 2.13%
519.lbm_r 16986.000 -
520.omnetpp_r 15140.000 4.24%
523.xalancbmk_r 13709.000 -
525.x264_r 6189.000 -
526.blender_r 12001.000 -
531.deepsjeng_r 8664.000 -
538.imagick_r 14046.000 -
541.leela_r 13367.000 -
544.nab_r 14403.000 -
557.xz_r 11461.000 -
Geometric Mean 13045.863 -
Code Size (bytes)
Name Current %
500.perlbench_r 2149235.000 -
502.gcc_r 8952888.000 -
505.mcf_r 27183.000 -
507.cactuBSSN_r 3680047.000 -
508.namd_r 615983.000 -
510.parest_r 8673728.000 -
511.povray_r 917614.000 -
519.lbm_r 13307.000 -
520.omnetpp_r 2153545.000 -
523.xalancbmk_r 4724595.000 -
525.x264_r 497870.000 -
526.blender_r 13668890.000 -
531.deepsjeng_r 81270.000 -
538.imagick_r 1699641.000 -
541.leela_r 145159.000 -
544.nab_r 191572.000 -
557.xz_r 177644.000 -
Geometric Mean 739634.755 -
sample variability
Name Current %
500.perlbench_r 106.000 -
500.perlbench_r (avg) 106.000 -
500.perlbench_r (avg) (percentage) 0.660 -
500.perlbench_r (max) 17.000 -
500.perlbench_r (max) (percentage) 0.110 -
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.270 -
505.mcf_r 67.000 -
505.mcf_r (avg) 67.000 -
505.mcf_r (avg) (percentage) 0.470 -
505.mcf_r (max) 63.000 -3.08%
505.mcf_r (max) (percentage) 0.440 -4.35%
508.namd_r 28.000 -
508.namd_r (avg) 28.000 -
508.namd_r (avg) (percentage) 0.300 -
508.namd_r (max) 47.000 -4.08%
508.namd_r (max) (percentage) 0.490 -5.77%
510.parest_r 38.000 -2.56%
510.parest_r (avg) 38.000 -2.56%
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 70.000 -4.11%
510.parest_r (max) (percentage) 0.310 -3.13%
510.parest_r (unexplained diff vs previous) 46.000 -
510.parest_r (unexplained diff vs previous) (percentage) 0.200 -
511.povray_r 77.000 2.67%
511.povray_r (avg) 77.000 2.67%
511.povray_r (avg) (percentage) 0.380 -
511.povray_r (max) 93.000 -
511.povray_r (max) (percentage) 0.460 -
519.lbm_r 40.000 -
519.lbm_r (avg) 40.000 -
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 67.000 -
519.lbm_r (max) (percentage) 0.400 -
519.lbm_r (unexplained diff vs previous) 67.000 -
519.lbm_r (unexplained diff vs previous) (percentage) 0.390 -
520.omnetpp_r 155.000 22.05%
520.omnetpp_r (avg) 155.000 22.05%
520.omnetpp_r (avg) (percentage) 1.030 17.05%
520.omnetpp_r (max) 641.000 367.88%
520.omnetpp_r (max) (percentage) 4.240 346.32%
520.omnetpp_r (unexplained diff vs previous) 616.000 1521.05%
520.omnetpp_r (unexplained diff vs previous) (percentage) 4.070 1465.38%
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 34.000 -
523.xalancbmk_r (avg) 34.000 -
523.xalancbmk_r (avg) (percentage) 0.250 -
523.xalancbmk_r (max) 53.000 -3.64%
523.xalancbmk_r (max) (percentage) 0.390 -4.88%
525.x264_r 12.000 -7.69%
525.x264_r (avg) 12.000 -7.69%
525.x264_r (avg) (percentage) 0.210 -4.55%
525.x264_r (max) 14.000 -26.32%
525.x264_r (max) (percentage) 0.230 -25.81%
525.x264_r (unexplained diff vs previous) 2.000 -
525.x264_r (unexplained diff vs previous) (percentage) 0.030 -
526.blender_r 94.000 -
526.blender_r (avg) 94.000 -
526.blender_r (avg) (percentage) 0.790 -
526.blender_r (max) 391.000 -
526.blender_r (max) (percentage) 3.260 -1.51%
531.deepsjeng_r 46.000 -
531.deepsjeng_r (avg) 46.000 -
531.deepsjeng_r (avg) (percentage) 0.540 -
531.deepsjeng_r (max) 107.000 -2.73%
531.deepsjeng_r (max) (percentage) 1.240 -3.13%
538.imagick_r 28.000 3.70%
538.imagick_r (avg) 28.000 3.70%
538.imagick_r (avg) (percentage) 0.200 -
538.imagick_r (max) 63.000 -1.56%
538.imagick_r (max) (percentage) 0.450 -2.17%
541.leela_r 40.000 -
541.leela_r (avg) 40.000 -
541.leela_r (avg) (percentage) 0.300 -
541.leela_r (max) 66.000 -5.71%
541.leela_r (max) (percentage) 0.500 -3.85%
544.nab_r 43.000 -
544.nab_r (avg) 43.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 53.000 -1.85%
544.nab_r (max) (percentage) 0.370 -2.63%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.280 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.450 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.210 -
557.xz_r 13.000 -
557.xz_r (avg) 13.000 -
557.xz_r (avg) (percentage) 0.120 -
557.xz_r (max) 19.000 -5.00%
557.xz_r (max) (percentage) 0.170 -
Geometric Mean 4.946 -