Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/bd382032f6df9529197a391ed80e9ab8c59915f1 (+ 827 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-aarch64-master-O2-build/134/
View Options
Report
Filter

llvm-aarch64-master-O2 test results

Run Order Start Time Duration
Current 0134 (llvmorg-18-init-13527-gbd382032f6df) 2023-11-26T05:58:30 0:00:00
Previous 0133 (llvmorg-18-init-12700-gbc191ac376fb) 2023-11-19T16:23:17 0:00:00
Baseline 0027 (llvmorg-17-init-08118-g0b911a3dc34a) 2023-04-13T21:53:13 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 6 61
Performance Improvements 23 61
Added Tests 7 14
Unchanged Tests 900 800
Total Tests 936

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
511.povray_r 1.73% 19820.000 20163.000 - -6.42% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
549.fotonik3d_r (max) 1.52% 132.000 134.000 - -64.64% -
549.fotonik3d_r 1.36% 147.000 149.000 - -38.93% -
549.fotonik3d_r (avg) 1.36% 147.000 149.000 - -38.93% -
511.povray_r 1.23% 81.000 82.000 - 6.49% -
511.povray_r (avg) 1.23% 81.000 82.000 - 6.49% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
500.perlbench_r -3.33% 15982.000 15450.000 - -3.74% -
519.lbm_r -1.60% 15620.000 15370.000 - -12.30% -
521.wrf_r -1.08% 14628.000 14470.000 - -1.87% -
527.cam4_r -1.03% 14389.000 14241.000 - 0.00% -
526.blender_r -1.01% 11941.000 11820.000 - -4.92% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
519.lbm_r (max) -11.94% 67.000 59.000 - 103.45% -
519.lbm_r (max) (percentage) -9.30% 0.430 0.390 - 129.41% -
554.roms_r (max) (percentage) -5.26% 0.190 0.180 - 157.14% -
554.roms_r (max) -5.13% 39.000 37.000 - 131.25% -
500.perlbench_r (max) -4.92% 305.000 290.000 - 36.15% -
500.perlbench_r -3.73% 134.000 129.000 - -11.64% -
500.perlbench_r (avg) -3.73% 134.000 129.000 - -11.64% -
531.deepsjeng_r (max) -2.86% 35.000 34.000 - -49.25% -
519.lbm_r -2.70% 37.000 36.000 - 24.14% -
519.lbm_r (avg) -2.70% 37.000 36.000 - 24.14% -
541.leela_r (max) -2.60% 77.000 75.000 - 1.35% -
523.xalancbmk_r (max) -2.56% 39.000 38.000 - -26.92% -
541.leela_r -2.50% 40.000 39.000 - -33.90% -
541.leela_r (avg) -2.50% 40.000 39.000 - -33.90% -
503.bwaves_r (max) -1.89% 53.000 52.000 - 173.68% -
541.leela_r (max) (percentage) -1.72% 0.580 0.570 - 3.64% -
500.perlbench_r (max) (percentage) -1.57% 1.910 1.880 - 41.35% -
526.blender_r (max) -1.56% 64.000 63.000 - 43.18% -

Added Tests - sample variability
527.cam4_r
527.cam4_r (avg)
527.cam4_r (avg) (percentage)
527.cam4_r (max)
527.cam4_r (max) (percentage)
527.cam4_r (unexplained diff vs previous)
527.cam4_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
503.bwaves_r 2.71% 775742.000 796734.000 - -0.02% -
554.roms_r 1.65% 3060081.000 3110724.000 - 0.02% -
548.exchange2_r 1.62% 2555603.000 2597110.000 - -0.00% -
549.fotonik3d_r 1.12% 1256502.000 1270551.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
548.exchange2_r 3.23% 18630.000 19232.000 - -0.04% -
507.cactuBSSN_r 1.62% 11293.000 11476.000 - -0.13% -
538.imagick_r 1.38% 13884.000 14075.000 - -0.43% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
503.bwaves_r (max) (percentage) 177.78% 0.090 0.250 - 0.00% -
503.bwaves_r (max) 173.68% 19.000 52.000 - -1.89% -
554.roms_r (max) (percentage) 157.14% 0.070 0.180 - -5.26% -
554.roms_r (avg) (percentage) 150.00% 0.060 0.150 - 0.00% -
554.roms_r 138.46% 13.000 31.000 - 0.00% -
554.roms_r (avg) 138.46% 13.000 31.000 - 0.00% -
525.x264_r (avg) (percentage) 133.33% 0.090 0.210 - 0.00% -
554.roms_r (max) 131.25% 16.000 37.000 - -5.13% -
519.lbm_r (max) (percentage) 129.41% 0.170 0.390 - -9.30% -
526.blender_r (avg) (percentage) 118.18% 0.330 0.720 - 0.00% -
525.x264_r 116.67% 6.000 13.000 - 0.00% -
525.x264_r (avg) 116.67% 6.000 13.000 - 0.00% -
526.blender_r 107.32% 41.000 85.000 - 0.00% -
526.blender_r (avg) 107.32% 41.000 85.000 - 0.00% -
519.lbm_r (max) 103.45% 29.000 59.000 - -11.94% -
521.wrf_r (max) (percentage) 97.30% 0.740 1.460 - 0.00% -
521.wrf_r (max) 93.58% 109.000 211.000 - -0.94% -
521.wrf_r (avg) (percentage) 89.74% 0.780 1.480 - 0.00% -
521.wrf_r 86.09% 115.000 214.000 - -0.93% -
521.wrf_r (avg) 86.09% 115.000 214.000 - -0.93% -
526.blender_r (max) (percentage) 50.00% 0.360 0.540 - 0.00% -
526.blender_r (max) 43.18% 44.000 63.000 - -1.56% -
500.perlbench_r (max) (percentage) 41.35% 1.330 1.880 - -1.57% -
519.lbm_r (avg) (percentage) 41.18% 0.170 0.240 - 0.00% -
500.perlbench_r (max) 36.15% 213.000 290.000 - -4.92% -
544.nab_r (max) 27.03% 37.000 47.000 - 0.00% -
544.nab_r (max) (percentage) 26.92% 0.260 0.330 - 0.00% -
520.omnetpp_r (avg) (percentage) 26.09% 0.920 1.160 - 0.00% -
520.omnetpp_r 25.37% 134.000 168.000 - 0.60% -
520.omnetpp_r (avg) 25.37% 134.000 168.000 - 0.60% -
548.exchange2_r (max) 25.00% 40.000 50.000 - 0.00% -
519.lbm_r 24.14% 29.000 36.000 - -2.70% -
519.lbm_r (avg) 24.14% 29.000 36.000 - -2.70% -
531.deepsjeng_r (avg) (percentage) 19.51% 0.410 0.490 - 0.00% -
548.exchange2_r (max) (percentage) 18.18% 0.220 0.260 - 0.00% -
531.deepsjeng_r 13.89% 36.000 41.000 - 0.00% -
531.deepsjeng_r (avg) 13.89% 36.000 41.000 - 0.00% -
511.povray_r (avg) (percentage) 13.89% 0.360 0.410 - 0.00% -
510.parest_r (avg) (percentage) 13.33% 0.150 0.170 - 0.00% -
510.parest_r 11.76% 34.000 38.000 - 0.00% -
510.parest_r (avg) 11.76% 34.000 38.000 - 0.00% -
548.exchange2_r 11.11% 27.000 30.000 - 0.00% -
548.exchange2_r (avg) 11.11% 27.000 30.000 - 0.00% -
557.xz_r 9.09% 11.000 12.000 - 0.00% -
557.xz_r (avg) 9.09% 11.000 12.000 - 0.00% -
548.exchange2_r (avg) (percentage) 6.67% 0.150 0.160 - 0.00% -
511.povray_r 6.49% 77.000 82.000 - 1.23% -
511.povray_r (avg) 6.49% 77.000 82.000 - 1.23% -
541.leela_r (max) (percentage) 3.64% 0.550 0.570 - -1.72% -
544.nab_r (avg) (percentage) 3.45% 0.290 0.300 - 0.00% -
544.nab_r 2.44% 41.000 42.000 - 0.00% -
544.nab_r (avg) 2.44% 41.000 42.000 - 0.00% -
511.povray_r (max) (percentage) 1.82% 0.550 0.560 - -1.75% -
541.leela_r (max) 1.35% 74.000 75.000 - -2.60% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
521.wrf_r -4.79% 23893420.000 22747745.000 - -0.55% -
508.namd_r -2.13% 624815.000 611521.000 - -0.31% -
507.cactuBSSN_r -1.15% 3707734.000 3665279.000 - -0.48% -
519.lbm_r -1.10% 13451.000 13303.000 - -0.27% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
525.x264_r -17.32% 7490.000 6193.000 - -0.29% -
519.lbm_r -12.30% 17526.000 15370.000 - -1.60% -
554.roms_r -8.77% 22927.000 20917.000 - 0.19% -
511.povray_r -6.42% 21547.000 20163.000 - 1.73% -
526.blender_r -4.92% 12432.000 11820.000 - -1.01% -
500.perlbench_r -3.74% 16051.000 15450.000 - -3.33% -
531.deepsjeng_r -2.83% 8789.000 8540.000 - -0.36% -
549.fotonik3d_r -2.82% 31362.000 30478.000 - 0.98% -
541.leela_r -2.00% 13571.000 13300.000 - -0.88% -
521.wrf_r -1.87% 14746.000 14470.000 - -1.08% -
520.omnetpp_r -1.22% 14662.000 14483.000 - 0.46% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
557.xz_r (max) -80.95% 21.000 4.000 - 0.00% -
557.xz_r (max) (percentage) -78.95% 0.190 0.040 - 0.00% -
508.namd_r (max) -78.70% 108.000 23.000 - 0.00% -
508.namd_r (max) (percentage) -77.88% 1.130 0.250 - 0.00% -
505.mcf_r (max) -77.08% 96.000 22.000 - 0.00% -
505.mcf_r (max) (percentage) -76.47% 0.680 0.160 - 0.00% -
549.fotonik3d_r (max) -64.64% 379.000 134.000 - 1.52% -
549.fotonik3d_r (max) (percentage) -63.64% 1.210 0.440 - 0.00% -
508.namd_r -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) -57.38% 61.000 26.000 - 0.00% -
508.namd_r (avg) (percentage) -56.25% 0.640 0.280 - 0.00% -
531.deepsjeng_r (max) -49.25% 67.000 34.000 - -2.86% -
531.deepsjeng_r (max) (percentage) -48.05% 0.770 0.400 - -2.44% -
538.imagick_r (avg) (percentage) -40.00% 0.300 0.180 - 0.00% -
538.imagick_r -39.02% 41.000 25.000 - 0.00% -
538.imagick_r (avg) -39.02% 41.000 25.000 - 0.00% -
549.fotonik3d_r -38.93% 244.000 149.000 - 1.36% -
549.fotonik3d_r (avg) -38.93% 244.000 149.000 - 1.36% -
549.fotonik3d_r (avg) (percentage) -37.18% 0.780 0.490 - 0.00% -
520.omnetpp_r (max) -35.25% 366.000 237.000 - 0.42% -
520.omnetpp_r (max) (percentage) -34.40% 2.500 1.640 - 0.00% -
541.leela_r -33.90% 59.000 39.000 - -2.50% -
541.leela_r (avg) -33.90% 59.000 39.000 - -2.50% -
510.parest_r (max) -32.56% 43.000 29.000 - 0.00% -
541.leela_r (avg) (percentage) -31.82% 0.440 0.300 - 0.00% -
510.parest_r (max) (percentage) -31.58% 0.190 0.130 - 0.00% -
525.x264_r (max) -27.27% 11.000 8.000 - 0.00% -
523.xalancbmk_r (max) -26.92% 52.000 38.000 - -2.56% -
523.xalancbmk_r (max) (percentage) -26.32% 0.380 0.280 - -3.45% -
505.mcf_r (avg) (percentage) -25.00% 0.520 0.390 - 0.00% -
505.mcf_r -24.66% 73.000 55.000 - 0.00% -
505.mcf_r (avg) -24.66% 73.000 55.000 - 0.00% -
500.perlbench_r -11.64% 146.000 129.000 - -3.73% -
500.perlbench_r (avg) -11.64% 146.000 129.000 - -3.73% -
503.bwaves_r (avg) (percentage) -10.00% 0.100 0.090 - 0.00% -
503.bwaves_r -9.52% 21.000 19.000 - 0.00% -
503.bwaves_r (avg) -9.52% 21.000 19.000 - 0.00% -
523.xalancbmk_r -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) -9.09% 33.000 30.000 - 0.00% -
523.xalancbmk_r (avg) (percentage) -8.33% 0.240 0.220 - 0.00% -
500.perlbench_r (avg) (percentage) -7.69% 0.910 0.840 - 0.00% -
511.povray_r (max) -5.08% 118.000 112.000 - 0.00% -
538.imagick_r (max) (percentage) -4.76% 0.420 0.400 - 0.00% -
538.imagick_r (max) -3.45% 58.000 56.000 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful 4.55% 22.000 23.000 - 0.00% -
nb_execution_successful 4.55% 22.000 23.000 - 0.00% -

Added Tests - Code Size
527.cam4_r

Added Tests - Execution
527.cam4_r

Added Tests - sample variability
507.cactuBSSN_r
507.cactuBSSN_r (avg)
507.cactuBSSN_r (avg) (percentage)
507.cactuBSSN_r (max)
507.cactuBSSN_r (max) (percentage)
527.cam4_r
527.cam4_r (avg)
527.cam4_r (avg) (percentage)
527.cam4_r (max)
527.cam4_r (max) (percentage)
527.cam4_r (unexplained diff vs previous)
527.cam4_r (unexplained diff vs previous) (percentage)

Report Time: 0.33s
Execution (seconds)
Name Current %
500.perlbench_r 15450.000 -3.33%
502.gcc_r 10924.000 -
503.bwaves_r 21141.000 -
505.mcf_r 14244.000 -
507.cactuBSSN_r 11476.000 -
508.namd_r 9510.000 -
510.parest_r 22921.000 -
511.povray_r 20163.000 1.73%
519.lbm_r 15370.000 -1.60%
520.omnetpp_r 14483.000 -
521.wrf_r 14470.000 -1.08%
523.xalancbmk_r 13699.000 -
525.x264_r 6193.000 -
526.blender_r 11820.000 -1.01%
527.cam4_r 14241.000 -1.03%
531.deepsjeng_r 8540.000 -
538.imagick_r 14075.000 -
541.leela_r 13300.000 -
544.nab_r 14247.000 -
548.exchange2_r 19232.000 -
549.fotonik3d_r 30478.000 -
554.roms_r 20917.000 -
557.xz_r 11459.000 -
Geometric Mean 14305.707 -
Code Size (bytes)
Name Current %
500.perlbench_r 2146441.000 -
502.gcc_r 8928812.000 -
503.bwaves_r 796734.000 -
505.mcf_r 27151.000 -
507.cactuBSSN_r 3665279.000 -
508.namd_r 611521.000 -
510.parest_r 8622536.000 -
511.povray_r 913055.000 -
519.lbm_r 13303.000 -
520.omnetpp_r 2151309.000 -
521.wrf_r 22747745.000 -
523.xalancbmk_r 4710929.000 -
525.x264_r 496950.000 -
526.blender_r 13678291.000 -
527.cam4_r 9215097.000 -
531.deepsjeng_r 81434.000 -
538.imagick_r 1694029.000 -
541.leela_r 144399.000 -
544.nab_r 191092.000 -
548.exchange2_r 2597110.000 -
549.fotonik3d_r 1270551.000 -
554.roms_r 3110724.000 -
557.xz_r 176728.000 -
Geometric Mean 1103856.088 -
sample variability
Name Current %
500.perlbench_r 129.000 -3.73%
500.perlbench_r (avg) 129.000 -3.73%
500.perlbench_r (avg) (percentage) 0.840 -
500.perlbench_r (max) 290.000 -4.92%
500.perlbench_r (max) (percentage) 1.880 -1.57%
503.bwaves_r 19.000 -
503.bwaves_r (avg) 19.000 -
503.bwaves_r (avg) (percentage) 0.090 -
503.bwaves_r (max) 52.000 -1.89%
503.bwaves_r (max) (percentage) 0.250 -
505.mcf_r 55.000 -
505.mcf_r (avg) 55.000 -
505.mcf_r (avg) (percentage) 0.390 -
505.mcf_r (max) 22.000 -
505.mcf_r (max) (percentage) 0.160 -
507.cactuBSSN_r 1.000 -
507.cactuBSSN_r (avg) 1.000 -
507.cactuBSSN_r (avg) (percentage) 0.010 -
507.cactuBSSN_r (max) 1.000 -
507.cactuBSSN_r (max) (percentage) 0.010 -
508.namd_r 26.000 -
508.namd_r (avg) 26.000 -
508.namd_r (avg) (percentage) 0.280 -
508.namd_r (max) 23.000 -
508.namd_r (max) (percentage) 0.250 -
510.parest_r 38.000 -
510.parest_r (avg) 38.000 -
510.parest_r (avg) (percentage) 0.170 -
510.parest_r (max) 29.000 -
510.parest_r (max) (percentage) 0.130 -
511.povray_r 82.000 1.23%
511.povray_r (avg) 82.000 1.23%
511.povray_r (avg) (percentage) 0.410 -
511.povray_r (max) 112.000 -
511.povray_r (max) (percentage) 0.560 -
519.lbm_r 36.000 -2.70%
519.lbm_r (avg) 36.000 -2.70%
519.lbm_r (avg) (percentage) 0.240 -
519.lbm_r (max) 59.000 -11.94%
519.lbm_r (max) (percentage) 0.390 -9.30%
520.omnetpp_r 168.000 -
520.omnetpp_r (avg) 168.000 -
520.omnetpp_r (avg) (percentage) 1.160 -
520.omnetpp_r (max) 237.000 -
520.omnetpp_r (max) (percentage) 1.640 -
521.wrf_r 214.000 -
521.wrf_r (avg) 214.000 -
521.wrf_r (avg) (percentage) 1.480 -
521.wrf_r (max) 211.000 -
521.wrf_r (max) (percentage) 1.460 -
523.xalancbmk_r 30.000 -
523.xalancbmk_r (avg) 30.000 -
523.xalancbmk_r (avg) (percentage) 0.220 -
523.xalancbmk_r (max) 38.000 -2.56%
523.xalancbmk_r (max) (percentage) 0.280 -
525.x264_r 13.000 -
525.x264_r (avg) 13.000 -
525.x264_r (avg) (percentage) 0.210 -
525.x264_r (max) 8.000 -
525.x264_r (max) (percentage) 0.140 -
526.blender_r 85.000 -
526.blender_r (avg) 85.000 -
526.blender_r (avg) (percentage) 0.720 -
526.blender_r (max) 63.000 -1.56%
526.blender_r (max) (percentage) 0.540 -
527.cam4_r 146.000 -
527.cam4_r (avg) 146.000 -
527.cam4_r (avg) (percentage) 1.030 -
527.cam4_r (max) 146.000 -
527.cam4_r (max) (percentage) 1.030 -
527.cam4_r (unexplained diff vs previous) 148.000 -
527.cam4_r (unexplained diff vs previous) (percentage) 1.040 -
531.deepsjeng_r 41.000 -
531.deepsjeng_r (avg) 41.000 -
531.deepsjeng_r (avg) (percentage) 0.490 -
531.deepsjeng_r (max) 34.000 -2.86%
531.deepsjeng_r (max) (percentage) 0.400 -
538.imagick_r 25.000 -
538.imagick_r (avg) 25.000 -
538.imagick_r (avg) (percentage) 0.180 -
538.imagick_r (max) 56.000 -
538.imagick_r (max) (percentage) 0.400 -
541.leela_r 39.000 -2.50%
541.leela_r (avg) 39.000 -2.50%
541.leela_r (avg) (percentage) 0.300 -
541.leela_r (max) 75.000 -2.60%
541.leela_r (max) (percentage) 0.570 -1.72%
544.nab_r 42.000 -
544.nab_r (avg) 42.000 -
544.nab_r (avg) (percentage) 0.300 -
544.nab_r (max) 47.000 -
544.nab_r (max) (percentage) 0.330 -
548.exchange2_r 30.000 -
548.exchange2_r (avg) 30.000 -
548.exchange2_r (avg) (percentage) 0.160 -
548.exchange2_r (max) 50.000 -
548.exchange2_r (max) (percentage) 0.260 -
549.fotonik3d_r 149.000 1.36%
549.fotonik3d_r (avg) 149.000 1.36%
549.fotonik3d_r (avg) (percentage) 0.490 -
549.fotonik3d_r (max) 134.000 1.52%
549.fotonik3d_r (max) (percentage) 0.440 -
554.roms_r 31.000 -
554.roms_r (avg) 31.000 -
554.roms_r (avg) (percentage) 0.150 -
554.roms_r (max) 37.000 -5.13%
554.roms_r (max) (percentage) 0.180 -5.26%
557.xz_r 12.000 -
557.xz_r (avg) 12.000 -
557.xz_r (avg) (percentage) 0.110 -
557.xz_r (max) 4.000 -
557.xz_r (max) (percentage) 0.040 -
Geometric Mean 5.819 6.88%