Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/0a168131b4f420d9f561926c643c143c84c97535 (+ 1180 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/51/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0051 (llvmorg-17-init-13405-g0a168131b4f4) 2023-05-27T17:52:18 0:00:00
Previous 0053 (llvmorg-17-init-14156-g10e83005367c) 2023-06-05T22:13:23 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 49 53
Performance Improvements 28 56
Unchanged Tests 799 767
Total Tests 876

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
557.xz_r 1.40% 11427.000 11587.000 - 0.27% -
507.cactuBSSN_r 1.15% 11260.000 11390.000 - 0.86% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
505.mcf_r (max) (percentage) 42.86% 0.350 0.500 - -26.47% -
505.mcf_r (max) 38.00% 50.000 69.000 - -28.12% -
525.x264_r (max) 32.14% 28.000 37.000 - 236.36% -
525.x264_r (max) (percentage) 31.82% 0.440 0.580 - 286.67% -
538.imagick_r (max) 22.92% 48.000 59.000 - 1.72% -
538.imagick_r (max) (percentage) 22.86% 0.350 0.430 - 2.38% -
519.lbm_r (max) (percentage) 18.75% 0.480 0.570 - 235.29% -
519.lbm_r (max) 17.86% 84.000 99.000 - 241.38% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 13.89% -
511.povray_r (max) (percentage) 11.54% 0.520 0.580 - 5.45% -
548.exchange2_r 11.11% 18.000 20.000 - -25.93% -
548.exchange2_r (avg) 11.11% 18.000 20.000 - -25.93% -
511.povray_r 10.81% 74.000 82.000 - 6.49% -
511.povray_r (avg) 10.81% 74.000 82.000 - 6.49% -
541.leela_r (avg) (percentage) 10.71% 0.280 0.310 - -29.55% -
525.x264_r (avg) (percentage) 9.09% 0.220 0.240 - 166.67% -
531.deepsjeng_r (max) (percentage) 8.89% 0.900 0.980 - 27.27% -
511.povray_r (max) 8.41% 107.000 116.000 - -1.69% -
541.leela_r 7.89% 38.000 41.000 - -30.51% -
541.leela_r (avg) 7.89% 38.000 41.000 - -30.51% -
541.leela_r (max) (percentage) 7.69% 0.260 0.280 - -49.09% -
531.deepsjeng_r (max) 7.69% 78.000 84.000 - 25.37% -
525.x264_r 7.14% 14.000 15.000 - 150.00% -
525.x264_r (avg) 7.14% 14.000 15.000 - 150.00% -
523.xalancbmk_r (max) (percentage) 6.12% 0.490 0.520 - 36.84% -
521.wrf_r (max) (percentage) 6.11% 3.110 3.300 - 345.95% -
523.xalancbmk_r (max) 5.97% 67.000 71.000 - 36.54% -
554.roms_r (max) 5.95% 84.000 89.000 - 456.25% -
521.wrf_r (max) 5.95% 454.000 481.000 - 341.28% -
541.leela_r (max) 5.71% 35.000 37.000 - -50.00% -
554.roms_r (max) (percentage) 5.41% 0.370 0.390 - 457.14% -
531.deepsjeng_r (avg) (percentage) 5.36% 0.560 0.590 - 43.90% -
510.parest_r (max) 5.08% 59.000 62.000 - 44.19% -
519.lbm_r 5.00% 40.000 42.000 - 44.83% -
519.lbm_r (avg) 5.00% 40.000 42.000 - 44.83% -
538.imagick_r (avg) (percentage) 4.55% 0.220 0.230 - -23.33% -
531.deepsjeng_r 4.17% 48.000 50.000 - 38.89% -
531.deepsjeng_r (avg) 4.17% 48.000 50.000 - 38.89% -
500.perlbench_r (max) 3.95% 228.000 237.000 - 11.27% -
510.parest_r (max) (percentage) 3.85% 0.260 0.270 - 42.11% -
526.blender_r (max) (percentage) 3.85% 0.260 0.270 - -25.00% -
549.fotonik3d_r (max) (percentage) 3.85% 0.520 0.540 - -55.37% -
500.perlbench_r (max) (percentage) 3.52% 1.420 1.470 - 10.53% -
538.imagick_r 3.33% 30.000 31.000 - -24.39% -
538.imagick_r (avg) 3.33% 30.000 31.000 - -24.39% -
526.blender_r (max) 3.23% 31.000 32.000 - -27.27% -
505.mcf_r (avg) (percentage) 2.38% 0.420 0.430 - -17.31% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
511.povray_r -2.85% 20736.000 20145.000 - -6.51% -
505.mcf_r -2.28% 14290.000 13964.000 - -1.30% -
541.leela_r -1.59% 13609.000 13393.000 - -1.31% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
503.bwaves_r (max) -70.97% 93.000 27.000 - 42.11% -
503.bwaves_r (max) (percentage) -70.45% 0.440 0.130 - 44.44% -
544.nab_r (max) (percentage) -66.98% 1.060 0.350 - 34.62% -
544.nab_r (max) -66.89% 151.000 50.000 - 35.14% -
520.omnetpp_r (unexplained diff vs previous) -66.67% 243.000 81.000 - 68.75% -
520.omnetpp_r (unexplained diff vs previous) (percentage) -66.26% 1.630 0.550 - 66.67% -
557.xz_r (max) -55.17% 29.000 13.000 - -38.10% -
557.xz_r (max) (percentage) -53.85% 0.260 0.120 - -36.84% -
520.omnetpp_r (max) -52.03% 246.000 118.000 - -67.76% -
520.omnetpp_r (max) (percentage) -51.81% 1.660 0.800 - -68.00% -
508.namd_r (max) -47.73% 44.000 23.000 - -78.70% -
508.namd_r (max) (percentage) -45.65% 0.460 0.250 - -77.88% -
503.bwaves_r -33.33% 21.000 14.000 - -33.33% -
503.bwaves_r (avg) -33.33% 21.000 14.000 - -33.33% -
503.bwaves_r (avg) (percentage) -30.00% 0.100 0.070 - -30.00% -
544.nab_r (avg) (percentage) -22.22% 0.450 0.350 - 20.69% -
544.nab_r -21.88% 64.000 50.000 - 21.95% -
544.nab_r (avg) -21.88% 64.000 50.000 - 21.95% -
557.xz_r (avg) (percentage) -14.29% 0.140 0.120 - 20.00% -
557.xz_r -13.33% 15.000 13.000 - 18.18% -
557.xz_r (avg) -13.33% 15.000 13.000 - 18.18% -
520.omnetpp_r -6.82% 132.000 123.000 - -8.21% -
520.omnetpp_r (avg) -6.82% 132.000 123.000 - -8.21% -
520.omnetpp_r (avg) (percentage) -5.62% 0.890 0.840 - -8.70% -
519.lbm_r (unexplained diff vs previous) -3.23% 31.000 30.000 - 25.00% -

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
554.roms_r (max) (percentage) 457.14% 0.070 0.390 - 5.41% -
554.roms_r (max) 456.25% 16.000 89.000 - 5.95% -
521.wrf_r (max) (percentage) 345.95% 0.740 3.300 - 6.11% -
521.wrf_r (max) 341.28% 109.000 481.000 - 5.95% -
525.x264_r (max) (percentage) 286.67% 0.150 0.580 - 31.82% -
519.lbm_r (max) 241.38% 29.000 99.000 - 17.86% -
525.x264_r (max) 236.36% 11.000 37.000 - 32.14% -
519.lbm_r (max) (percentage) 235.29% 0.170 0.570 - 18.75% -
554.roms_r 230.77% 13.000 43.000 - 0.00% -
554.roms_r (avg) 230.77% 13.000 43.000 - 0.00% -
554.roms_r (avg) (percentage) 216.67% 0.060 0.190 - 0.00% -
525.x264_r (avg) (percentage) 166.67% 0.090 0.240 - 9.09% -
521.wrf_r (avg) (percentage) 158.97% 0.780 2.020 - 0.00% -
521.wrf_r 155.65% 115.000 294.000 - -0.34% -
521.wrf_r (avg) 155.65% 115.000 294.000 - -0.34% -
525.x264_r 150.00% 6.000 15.000 - 7.14% -
525.x264_r (avg) 150.00% 6.000 15.000 - 7.14% -
520.omnetpp_r (unexplained diff vs previous) 68.75% 48.000 81.000 - -66.67% -
520.omnetpp_r (unexplained diff vs previous) (percentage) 66.67% 0.330 0.550 - -66.26% -
519.lbm_r 44.83% 29.000 42.000 - 5.00% -
519.lbm_r (avg) 44.83% 29.000 42.000 - 5.00% -
503.bwaves_r (max) (percentage) 44.44% 0.090 0.130 - -70.45% -
510.parest_r (max) 44.19% 43.000 62.000 - 5.08% -
531.deepsjeng_r (avg) (percentage) 43.90% 0.410 0.590 - 5.36% -
510.parest_r (max) (percentage) 42.11% 0.190 0.270 - 3.85% -
503.bwaves_r (max) 42.11% 19.000 27.000 - -70.97% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 4.35% -
531.deepsjeng_r 38.89% 36.000 50.000 - 4.17% -
531.deepsjeng_r (avg) 38.89% 36.000 50.000 - 4.17% -
523.xalancbmk_r (max) (percentage) 36.84% 0.380 0.520 - 6.12% -
523.xalancbmk_r (max) 36.54% 52.000 71.000 - 5.97% -
544.nab_r (max) 35.14% 37.000 50.000 - -66.89% -
544.nab_r (max) (percentage) 34.62% 0.260 0.350 - -66.98% -
531.deepsjeng_r (max) (percentage) 27.27% 0.770 0.980 - 8.89% -
531.deepsjeng_r (max) 25.37% 67.000 84.000 - 7.69% -
519.lbm_r (unexplained diff vs previous) 25.00% 24.000 30.000 - -3.23% -
544.nab_r 21.95% 41.000 50.000 - -21.88% -
544.nab_r (avg) 21.95% 41.000 50.000 - -21.88% -
519.lbm_r (unexplained diff vs previous) (percentage) 21.43% 0.140 0.170 - -5.56% -
544.nab_r (avg) (percentage) 20.69% 0.290 0.350 - -22.22% -
557.xz_r (avg) (percentage) 20.00% 0.100 0.120 - -14.29% -
557.xz_r 18.18% 11.000 13.000 - -13.33% -
557.xz_r (avg) 18.18% 11.000 13.000 - -13.33% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 13.89% -
500.perlbench_r (max) 11.27% 213.000 237.000 - 3.95% -
500.perlbench_r (max) (percentage) 10.53% 1.330 1.470 - 3.52% -
511.povray_r 6.49% 77.000 82.000 - 10.81% -
511.povray_r (avg) 6.49% 77.000 82.000 - 10.81% -
511.povray_r (max) (percentage) 5.45% 0.550 0.580 - 11.54% -
538.imagick_r (max) (percentage) 2.38% 0.420 0.430 - 22.86% -
538.imagick_r (max) 1.72% 58.000 59.000 - 22.92% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -4.55% 22.000 21.000 - 0.00% -
nb_execution_successful -4.55% 22.000 21.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
521.wrf_r -2.06% 23893420.000 23401816.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -13.95% 7490.000 6445.000 - -0.23% -
511.povray_r -6.51% 21547.000 20145.000 - -2.85% -
526.blender_r -3.06% 12432.000 12052.000 - 0.56% -
531.deepsjeng_r -1.79% 8789.000 8632.000 - -0.75% -
541.leela_r -1.31% 13571.000 13393.000 - -1.59% -
505.mcf_r -1.30% 14148.000 13964.000 - -2.28% -
521.wrf_r -1.04% 14746.000 14593.000 - -0.25% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
508.namd_r (max) -78.70% 108.000 23.000 - -47.73% -
508.namd_r (max) (percentage) -77.88% 1.130 0.250 - -45.65% -
520.omnetpp_r (max) (percentage) -68.00% 2.500 0.800 - -51.81% -
520.omnetpp_r (max) -67.76% 366.000 118.000 - -52.03% -
549.fotonik3d_r (max) (percentage) -55.37% 1.210 0.540 - 3.85% -
541.leela_r (max) -50.00% 74.000 37.000 - 5.71% -
548.exchange2_r (max) -50.00% 40.000 20.000 - 0.00% -
548.exchange2_r (max) (percentage) -50.00% 0.220 0.110 - 0.00% -
541.leela_r (max) (percentage) -49.09% 0.550 0.280 - 7.69% -
508.namd_r -42.62% 61.000 35.000 - 0.00% -
508.namd_r (avg) -42.62% 61.000 35.000 - 0.00% -
508.namd_r (avg) (percentage) -42.19% 0.640 0.370 - 0.00% -
557.xz_r (max) -38.10% 21.000 13.000 - -55.17% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
557.xz_r (max) (percentage) -36.84% 0.190 0.120 - -53.85% -
503.bwaves_r -33.33% 21.000 14.000 - -33.33% -
503.bwaves_r (avg) -33.33% 21.000 14.000 - -33.33% -
526.blender_r -31.71% 41.000 28.000 - 0.00% -
526.blender_r (avg) -31.71% 41.000 28.000 - 0.00% -
541.leela_r -30.51% 59.000 41.000 - 7.89% -
541.leela_r (avg) -30.51% 59.000 41.000 - 7.89% -
503.bwaves_r (avg) (percentage) -30.00% 0.100 0.070 - -30.00% -
541.leela_r (avg) (percentage) -29.55% 0.440 0.310 - 10.71% -
505.mcf_r (max) -28.12% 96.000 69.000 - 38.00% -
526.blender_r (avg) (percentage) -27.27% 0.330 0.240 - 0.00% -
526.blender_r (max) -27.27% 44.000 32.000 - 3.23% -
548.exchange2_r (avg) (percentage) -26.67% 0.150 0.110 - 10.00% -
505.mcf_r (max) (percentage) -26.47% 0.680 0.500 - 42.86% -
548.exchange2_r -25.93% 27.000 20.000 - 11.11% -
548.exchange2_r (avg) -25.93% 27.000 20.000 - 11.11% -
526.blender_r (max) (percentage) -25.00% 0.360 0.270 - 3.85% -
538.imagick_r -24.39% 41.000 31.000 - 3.33% -
538.imagick_r (avg) -24.39% 41.000 31.000 - 3.33% -
538.imagick_r (avg) (percentage) -23.33% 0.300 0.230 - 4.55% -
505.mcf_r -17.81% 73.000 60.000 - 0.00% -
505.mcf_r (avg) -17.81% 73.000 60.000 - 0.00% -
505.mcf_r (avg) (percentage) -17.31% 0.520 0.430 - 2.38% -
500.perlbench_r (avg) (percentage) -9.89% 0.910 0.820 - 0.00% -
500.perlbench_r -9.59% 146.000 132.000 - 0.76% -
500.perlbench_r (avg) -9.59% 146.000 132.000 - 0.76% -
520.omnetpp_r (avg) (percentage) -8.70% 0.920 0.840 - -5.62% -
520.omnetpp_r -8.21% 134.000 123.000 - -6.82% -
520.omnetpp_r (avg) -8.21% 134.000 123.000 - -6.82% -
523.xalancbmk_r -6.06% 33.000 31.000 - 0.00% -
523.xalancbmk_r (avg) -6.06% 33.000 31.000 - 0.00% -
510.parest_r -5.88% 34.000 32.000 - 0.00% -
510.parest_r (avg) -5.88% 34.000 32.000 - 0.00% -
511.povray_r (max) -1.69% 118.000 116.000 - 8.41% -

Report Time: 0.36s
Execution (seconds)
Name Current %
500.perlbench_r 16154.000 -
502.gcc_r 10975.000 -
503.bwaves_r 21141.000 -
505.mcf_r 13964.000 -2.28%
507.cactuBSSN_r 11390.000 1.15%
508.namd_r 9556.000 -
510.parest_r 22970.000 -
511.povray_r 20145.000 -2.85%
519.lbm_r 17525.000 -
520.omnetpp_r 14761.000 -
521.wrf_r 14593.000 -
523.xalancbmk_r 13690.000 -
525.x264_r 6445.000 -
526.blender_r 12052.000 -
531.deepsjeng_r 8632.000 -
538.imagick_r 13874.000 -
541.leela_r 13393.000 -1.59%
544.nab_r 14294.000 -
548.exchange2_r 18618.000 -
554.roms_r 22931.000 -
557.xz_r 11587.000 1.40%
Geometric Mean 14016.181 -
Code Size (bytes)
Name Current %
500.perlbench_r 2153455.000 -
502.gcc_r 8929796.000 -
503.bwaves_r 777058.000 -
505.mcf_r 27207.000 -
507.cactuBSSN_r 3676370.000 -
508.namd_r 620707.000 -
510.parest_r 8611096.000 -
511.povray_r 916114.000 -
519.lbm_r 13339.000 -
520.omnetpp_r 2153009.000 -
521.wrf_r 23401816.000 -
523.xalancbmk_r 4725899.000 -
525.x264_r 498962.000 -
526.blender_r 13664614.000 -
531.deepsjeng_r 81026.000 -
538.imagick_r 1700893.000 -
541.leela_r 144983.000 -
544.nab_r 190956.000 -
548.exchange2_r 2551327.000 -
554.roms_r 3034157.000 -
557.xz_r 177268.000 -
Geometric Mean 991155.696 -
sample variability
Name Current %
500.perlbench_r 132.000 -
500.perlbench_r (avg) 132.000 -
500.perlbench_r (avg) (percentage) 0.820 -
500.perlbench_r (max) 237.000 3.95%
500.perlbench_r (max) (percentage) 1.470 3.52%
503.bwaves_r 14.000 -33.33%
503.bwaves_r (avg) 14.000 -33.33%
503.bwaves_r (avg) (percentage) 0.070 -30.00%
503.bwaves_r (max) 27.000 -70.97%
503.bwaves_r (max) (percentage) 0.130 -70.45%
505.mcf_r 60.000 -
505.mcf_r (avg) 60.000 -
505.mcf_r (avg) (percentage) 0.430 2.38%
505.mcf_r (max) 69.000 38.00%
505.mcf_r (max) (percentage) 0.500 42.86%
508.namd_r 35.000 -
508.namd_r (avg) 35.000 -
508.namd_r (avg) (percentage) 0.370 -
508.namd_r (max) 23.000 -47.73%
508.namd_r (max) (percentage) 0.250 -45.65%
510.parest_r 32.000 -
510.parest_r (avg) 32.000 -
510.parest_r (avg) (percentage) 0.140 -
510.parest_r (max) 62.000 5.08%
510.parest_r (max) (percentage) 0.270 3.85%
511.povray_r 82.000 10.81%
511.povray_r (avg) 82.000 10.81%
511.povray_r (avg) (percentage) 0.410 13.89%
511.povray_r (max) 116.000 8.41%
511.povray_r (max) (percentage) 0.580 11.54%
519.lbm_r 42.000 5.00%
519.lbm_r (avg) 42.000 5.00%
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 99.000 17.86%
519.lbm_r (max) (percentage) 0.570 18.75%
519.lbm_r (unexplained diff vs previous) 30.000 -3.23%
519.lbm_r (unexplained diff vs previous) (percentage) 0.170 -
520.omnetpp_r 123.000 -6.82%
520.omnetpp_r (avg) 123.000 -6.82%
520.omnetpp_r (avg) (percentage) 0.840 -5.62%
520.omnetpp_r (max) 118.000 -52.03%
520.omnetpp_r (max) (percentage) 0.800 -51.81%
520.omnetpp_r (unexplained diff vs previous) 81.000 -66.67%
520.omnetpp_r (unexplained diff vs previous) (percentage) 0.550 -66.26%
521.wrf_r 294.000 -
521.wrf_r (avg) 294.000 -
521.wrf_r (avg) (percentage) 2.020 -
521.wrf_r (max) 481.000 5.95%
521.wrf_r (max) (percentage) 3.300 6.11%
523.xalancbmk_r 31.000 -
523.xalancbmk_r (avg) 31.000 -
523.xalancbmk_r (avg) (percentage) 0.230 -
523.xalancbmk_r (max) 71.000 5.97%
523.xalancbmk_r (max) (percentage) 0.520 6.12%
525.x264_r 15.000 7.14%
525.x264_r (avg) 15.000 7.14%
525.x264_r (avg) (percentage) 0.240 9.09%
525.x264_r (max) 37.000 32.14%
525.x264_r (max) (percentage) 0.580 31.82%
526.blender_r 28.000 -
526.blender_r (avg) 28.000 -
526.blender_r (avg) (percentage) 0.240 -
526.blender_r (max) 32.000 3.23%
526.blender_r (max) (percentage) 0.270 3.85%
531.deepsjeng_r 50.000 4.17%
531.deepsjeng_r (avg) 50.000 4.17%
531.deepsjeng_r (avg) (percentage) 0.590 5.36%
531.deepsjeng_r (max) 84.000 7.69%
531.deepsjeng_r (max) (percentage) 0.980 8.89%
538.imagick_r 31.000 3.33%
538.imagick_r (avg) 31.000 3.33%
538.imagick_r (avg) (percentage) 0.230 4.55%
538.imagick_r (max) 59.000 22.92%
538.imagick_r (max) (percentage) 0.430 22.86%
541.leela_r 41.000 7.89%
541.leela_r (avg) 41.000 7.89%
541.leela_r (avg) (percentage) 0.310 10.71%
541.leela_r (max) 37.000 5.71%
541.leela_r (max) (percentage) 0.280 7.69%
544.nab_r 50.000 -21.88%
544.nab_r (avg) 50.000 -21.88%
544.nab_r (avg) (percentage) 0.350 -22.22%
544.nab_r (max) 50.000 -66.89%
544.nab_r (max) (percentage) 0.350 -66.98%
548.exchange2_r 20.000 11.11%
548.exchange2_r (avg) 20.000 11.11%
548.exchange2_r (avg) (percentage) 0.110 -
548.exchange2_r (max) 20.000 -
548.exchange2_r (max) (percentage) 0.110 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.540 3.85%
554.roms_r 43.000 -
554.roms_r (avg) 43.000 -
554.roms_r (avg) (percentage) 0.190 -
554.roms_r (max) 89.000 5.95%
554.roms_r (max) (percentage) 0.390 5.41%
557.xz_r 13.000 -13.33%
557.xz_r (avg) 13.000 -13.33%
557.xz_r (avg) (percentage) 0.120 -14.29%
557.xz_r (max) 13.000 -55.17%
557.xz_r (max) (percentage) 0.120 -53.85%
Geometric Mean 6.247 -8.69%