Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/a16f6462d756804276d4b39267b3c19bcd6949fe (+ 408 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/112/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0112 (llvmorg-18-init-07932-ga16f6462d756) 2023-10-02T16:25:15 0:00:00
Previous 0109 (llvmorg-18-init-07524-g0b07b06effe5) 2023-09-28T21:50:39 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 9 51
Performance Improvements 28 46
Added Tests 0 5
Unchanged Tests 863 798
Total Tests 900

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
500.perlbench_r 1.63% 16159.000 16423.000 - 2.32% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (unexplained diff vs previous) 73.91% 23.000 40.000 - 66.67% -
519.lbm_r (unexplained diff vs previous) (percentage) 71.43% 0.140 0.240 - 71.43% -
519.lbm_r (max) 14.29% 35.000 40.000 - 37.93% -
519.lbm_r (max) (percentage) 14.29% 0.210 0.240 - 41.18% -
505.mcf_r 1.82% 55.000 56.000 - -23.29% -
505.mcf_r (avg) 1.82% 55.000 56.000 - -23.29% -
500.perlbench_r 1.48% 135.000 137.000 - -6.16% -
500.perlbench_r (avg) 1.48% 135.000 137.000 - -6.16% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
526.blender_r -3.95% 12442.000 11951.000 - -3.87% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
525.x264_r (max) (percentage) -10.53% 0.190 0.170 - 13.33% -
520.omnetpp_r (max) -10.06% 338.000 304.000 - -16.94% -
520.omnetpp_r (max) (percentage) -9.91% 2.320 2.090 - -16.40% -
525.x264_r (max) -9.09% 11.000 10.000 - -9.09% -
541.leela_r (max) -7.69% 39.000 36.000 - -51.35% -
541.leela_r (max) (percentage) -6.90% 0.290 0.270 - -50.91% -
508.namd_r (max) -6.06% 33.000 31.000 - -71.30% -
510.parest_r (max) -5.88% 34.000 32.000 - -25.58% -
508.namd_r (max) (percentage) -5.71% 0.350 0.330 - -70.80% -
531.deepsjeng_r (max) (percentage) -5.56% 0.540 0.510 - -33.77% -
526.blender_r (max) -5.33% 75.000 71.000 - 61.36% -
554.roms_r (max) (percentage) -5.00% 0.200 0.190 - 171.43% -
531.deepsjeng_r (max) -4.35% 46.000 44.000 - -34.33% -
511.povray_r (max) (percentage) -4.35% 0.690 0.660 - 20.00% -
511.povray_r (max) -4.32% 139.000 133.000 - 12.71% -
505.mcf_r (max) -4.00% 25.000 24.000 - -75.00% -
544.nab_r (max) -3.57% 56.000 54.000 - 45.95% -
526.blender_r -3.37% 89.000 86.000 - 109.76% -
526.blender_r (avg) -3.37% 89.000 86.000 - 109.76% -
538.imagick_r (max) -3.33% 60.000 58.000 - 0.00% -
500.perlbench_r (max) (percentage) -2.69% 2.230 2.170 - 63.16% -
541.leela_r -2.63% 38.000 37.000 - -37.29% -
541.leela_r (avg) -2.63% 38.000 37.000 - -37.29% -
544.nab_r (max) (percentage) -2.56% 0.390 0.380 - 46.15% -
538.imagick_r (max) (percentage) -2.33% 0.430 0.420 - 0.00% -
526.blender_r (max) (percentage) -1.64% 0.610 0.600 - 66.67% -
500.perlbench_r (max) -1.11% 360.000 356.000 - 67.14% -

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
500.perlbench_r 2.32% 16051.000 16423.000 - 1.63% -
505.mcf_r 1.85% 14148.000 14410.000 - 0.46% -
507.cactuBSSN_r 1.32% 11293.000 11442.000 - -0.13% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
503.bwaves_r (max) (percentage) 188.89% 0.090 0.260 - 0.00% -
554.roms_r (max) (percentage) 171.43% 0.070 0.190 - -5.00% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
526.blender_r (avg) (percentage) 118.18% 0.330 0.720 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
526.blender_r 109.76% 41.000 86.000 - -3.37% -
526.blender_r (avg) 109.76% 41.000 86.000 - -3.37% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
519.lbm_r (unexplained diff vs previous) (percentage) 71.43% 0.140 0.240 - 71.43% -
500.perlbench_r (max) 67.14% 213.000 356.000 - -1.11% -
519.lbm_r (unexplained diff vs previous) 66.67% 24.000 40.000 - 73.91% -
526.blender_r (max) (percentage) 66.67% 0.360 0.600 - -1.64% -
500.perlbench_r (max) (percentage) 63.16% 1.330 2.170 - -2.69% -
526.blender_r (max) 61.36% 44.000 71.000 - -5.33% -
544.nab_r (max) (percentage) 46.15% 0.260 0.380 - -2.56% -
544.nab_r (max) 45.95% 37.000 54.000 - -3.57% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 0.00% -
519.lbm_r (max) (percentage) 41.18% 0.170 0.240 - 14.29% -
519.lbm_r 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (avg) 37.93% 29.000 40.000 - 0.00% -
519.lbm_r (max) 37.93% 29.000 40.000 - 14.29% -
520.omnetpp_r (avg) (percentage) 27.17% 0.920 1.170 - 0.00% -
520.omnetpp_r 26.87% 134.000 170.000 - 0.00% -
520.omnetpp_r (avg) 26.87% 134.000 170.000 - 0.00% -
548.exchange2_r (max) (percentage) 22.73% 0.220 0.270 - 0.00% -
511.povray_r (max) (percentage) 20.00% 0.550 0.660 - -4.35% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
531.deepsjeng_r 16.67% 36.000 42.000 - 0.00% -
531.deepsjeng_r (avg) 16.67% 36.000 42.000 - 0.00% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
525.x264_r (max) (percentage) 13.33% 0.150 0.170 - -10.53% -
511.povray_r (max) 12.71% 118.000 133.000 - -4.32% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
557.xz_r 9.09% 11.000 12.000 - 0.00% -
557.xz_r (avg) 9.09% 11.000 12.000 - 0.00% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r 6.49% 77.000 82.000 - 0.00% -
511.povray_r (avg) 6.49% 77.000 82.000 - 0.00% -
544.nab_r 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) 4.88% 41.000 43.000 - 0.00% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -22.73% 22.000 17.000 - 0.00% -
nb_execution_successful -22.73% 22.000 17.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
508.namd_r -1.56% 624815.000 615051.000 - -0.01% -
519.lbm_r -1.07% 13451.000 13307.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -17.17% 7490.000 6204.000 - 0.13% -
511.povray_r -6.41% 21547.000 20166.000 - 0.01% -
526.blender_r -3.87% 12432.000 11951.000 - -3.95% -
519.lbm_r -3.45% 17526.000 16922.000 - 0.24% -
531.deepsjeng_r -1.55% 8789.000 8653.000 - 0.22% -
557.xz_r -1.03% 11556.000 11437.000 - -0.33% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
557.xz_r (max) -76.19% 21.000 5.000 - 0.00% -
505.mcf_r (max) -75.00% 96.000 24.000 - -4.00% -
505.mcf_r (max) (percentage) -75.00% 0.680 0.170 - -5.56% -
557.xz_r (max) (percentage) -73.68% 0.190 0.050 - 0.00% -
508.namd_r (max) -71.30% 108.000 31.000 - -6.06% -
508.namd_r (max) (percentage) -70.80% 1.130 0.330 - -5.71% -
549.fotonik3d_r (max) (percentage) -63.64% 1.210 0.440 - 0.00% -
508.namd_r -55.74% 61.000 27.000 - 0.00% -
508.namd_r (avg) -55.74% 61.000 27.000 - 0.00% -
508.namd_r (avg) (percentage) -54.69% 0.640 0.290 - 0.00% -
541.leela_r (max) -51.35% 74.000 36.000 - -7.69% -
541.leela_r (max) (percentage) -50.91% 0.550 0.270 - -6.90% -
538.imagick_r (avg) (percentage) -40.00% 0.300 0.180 - 0.00% -
538.imagick_r -39.02% 41.000 25.000 - 0.00% -
538.imagick_r (avg) -39.02% 41.000 25.000 - 0.00% -
541.leela_r -37.29% 59.000 37.000 - -2.63% -
541.leela_r (avg) -37.29% 59.000 37.000 - -2.63% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
541.leela_r (avg) (percentage) -36.36% 0.440 0.280 - 0.00% -
531.deepsjeng_r (max) -34.33% 67.000 44.000 - -4.35% -
531.deepsjeng_r (max) (percentage) -33.77% 0.770 0.510 - -5.56% -
510.parest_r (max) (percentage) -26.32% 0.190 0.140 - -6.67% -
510.parest_r (max) -25.58% 43.000 32.000 - -5.88% -
505.mcf_r (avg) (percentage) -25.00% 0.520 0.390 - 0.00% -
505.mcf_r -23.29% 73.000 56.000 - 1.82% -
505.mcf_r (avg) -23.29% 73.000 56.000 - 1.82% -
523.xalancbmk_r (max) -17.31% 52.000 43.000 - 0.00% -
520.omnetpp_r (max) -16.94% 366.000 304.000 - -10.06% -
520.omnetpp_r (max) (percentage) -16.40% 2.500 2.090 - -9.91% -
523.xalancbmk_r (max) (percentage) -15.79% 0.380 0.320 - 0.00% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
523.xalancbmk_r -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) -9.09% 33.000 30.000 - 0.00% -
525.x264_r (max) -9.09% 11.000 10.000 - -9.09% -
523.xalancbmk_r (avg) (percentage) -8.33% 0.240 0.220 - 0.00% -
500.perlbench_r (avg) (percentage) -7.69% 0.910 0.840 - 0.00% -
500.perlbench_r -6.16% 146.000 137.000 - 1.48% -
500.perlbench_r (avg) -6.16% 146.000 137.000 - 1.48% -

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)

Report Time: 0.39s
Execution (seconds)
Name Current %
500.perlbench_r 16423.000 1.63%
502.gcc_r 10930.000 -
505.mcf_r 14410.000 -
507.cactuBSSN_r 11442.000 -
508.namd_r 9534.000 -
510.parest_r 22893.000 -
511.povray_r 20166.000 -
519.lbm_r 16922.000 -
520.omnetpp_r 14555.000 -
523.xalancbmk_r 13725.000 -
525.x264_r 6204.000 -
526.blender_r 11951.000 -3.95%
531.deepsjeng_r 8653.000 -
538.imagick_r 14010.000 -
541.leela_r 13542.000 -
544.nab_r 14389.000 -
557.xz_r 11437.000 -
Geometric Mean 13014.943 -
Code Size (bytes)
Name Current %
500.perlbench_r 2149167.000 -
502.gcc_r 8945868.000 -
505.mcf_r 27183.000 -
507.cactuBSSN_r 3680247.000 -
508.namd_r 615051.000 -
510.parest_r 8650988.000 -
511.povray_r 915970.000 -
519.lbm_r 13307.000 -
520.omnetpp_r 2150293.000 -
523.xalancbmk_r 4713475.000 -
525.x264_r 496398.000 -
526.blender_r 13641138.000 -
531.deepsjeng_r 81346.000 -
538.imagick_r 1695773.000 -
541.leela_r 144595.000 -
544.nab_r 191276.000 -
557.xz_r 176304.000 -
Geometric Mean 738334.581 -
sample variability
Name Current %
500.perlbench_r 137.000 1.48%
500.perlbench_r (avg) 137.000 1.48%
500.perlbench_r (avg) (percentage) 0.840 -
500.perlbench_r (max) 356.000 -1.11%
500.perlbench_r (max) (percentage) 2.170 -2.69%
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) (percentage) 0.260 -
505.mcf_r 56.000 1.82%
505.mcf_r (avg) 56.000 1.82%
505.mcf_r (avg) (percentage) 0.390 -
505.mcf_r (max) 24.000 -4.00%
505.mcf_r (max) (percentage) 0.170 -
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 27.000 -
508.namd_r (avg) 27.000 -
508.namd_r (avg) (percentage) 0.290 -
508.namd_r (max) 31.000 -6.06%
508.namd_r (max) (percentage) 0.330 -5.71%
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 32.000 -5.88%
510.parest_r (max) (percentage) 0.140 -
511.povray_r 82.000 -
511.povray_r (avg) 82.000 -
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 133.000 -4.32%
511.povray_r (max) (percentage) 0.660 -4.35%
519.lbm_r 40.000 -
519.lbm_r (avg) 40.000 -
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 40.000 14.29%
519.lbm_r (max) (percentage) 0.240 14.29%
519.lbm_r (unexplained diff vs previous) 40.000 73.91%
519.lbm_r (unexplained diff vs previous) (percentage) 0.240 71.43%
520.omnetpp_r 170.000 -
520.omnetpp_r (avg) 170.000 -
520.omnetpp_r (avg) (percentage) 1.170 -
520.omnetpp_r (max) 304.000 -10.06%
520.omnetpp_r (max) (percentage) 2.090 -9.91%
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 30.000 -
523.xalancbmk_r (avg) 30.000 -
523.xalancbmk_r (avg) (percentage) 0.220 -
523.xalancbmk_r (max) 43.000 -
523.xalancbmk_r (max) (percentage) 0.320 -
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 10.000 -9.09%
525.x264_r (max) (percentage) 0.170 -10.53%
526.blender_r 86.000 -3.37%
526.blender_r (avg) 86.000 -3.37%
526.blender_r (avg) (percentage) 0.720 -
526.blender_r (max) 71.000 -5.33%
526.blender_r (max) (percentage) 0.600 -1.64%
531.deepsjeng_r 42.000 -
531.deepsjeng_r (avg) 42.000 -
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 44.000 -4.35%
531.deepsjeng_r (max) (percentage) 0.510 -5.56%
538.imagick_r 25.000 -
538.imagick_r (avg) 25.000 -
538.imagick_r (avg) (percentage) 0.180 -
538.imagick_r (max) 58.000 -3.33%
538.imagick_r (max) (percentage) 0.420 -2.33%
541.leela_r 37.000 -2.63%
541.leela_r (avg) 37.000 -2.63%
541.leela_r (avg) (percentage) 0.280 -
541.leela_r (max) 36.000 -7.69%
541.leela_r (max) (percentage) 0.270 -6.90%
544.nab_r 43.000 -
544.nab_r (avg) 43.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 54.000 -3.57%
544.nab_r (max) (percentage) 0.380 -2.56%
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) (percentage) 0.270 -
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) (percentage) 0.440 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) (percentage) 0.190 -5.00%
557.xz_r 12.000 -
557.xz_r (avg) 12.000 -
557.xz_r (avg) (percentage) 0.110 -
557.xz_r (max) 5.000 -
557.xz_r (max) (percentage) 0.050 -
Geometric Mean 3.683 -