Fields
hardware | None |
os | None |
Parameters
Fields
Name | Value |
---|
Parameters
Name | Value |
---|---|
git_llvm | https://github.com/llvm/llvm-project/commit/e61d6066e2678b1244b1bcbd59a154a82acf1eb8 (+ 86 commits) |
test_url | https://ci.linaro.org/job/tcwg_bmk-code_size-cpu2017fast--llvm-aarch64-master-Oz-build/194/ |
Filter
llvm-aarch64-master-Oz test results
Run | Order | Start Time | Duration |
---|---|---|---|
Current | 0194 (llvmorg-20-init-03909-ge61d6066e267) | 2024-08-26T22:15:54 | 0:00:00 |
Previous | 0192 (llvmorg-20-init-03823-gfac7e87d6634) | 2024-08-25T00:41:05 | 0:00:00 |
Baseline | 0140 (llvmorg-19-init-07219-g72e2e4f7dc68) | 2024-04-03T16:00:23 | 0:00:00 |
Tests Summary
Status Group | # | # (B) |
---|---|---|
Performance Regressions | 131 | 4 |
Performance Improvements | 7 | 3 |
Added Tests | 12 | 156 |
Unchanged Tests | 684 | 671 |
Total Tests | 834 |
Performance Regressions - Execution | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
619.lbm_s | 59.71% | 30012.000 | 47933.000 | - | 0.00% | - |
657.xz_s | 22.06% | 2289.000 | 2794.000 | - | 0.00% | - |
638.imagick_s | 17.48% | 3135.000 | 3683.000 | - | 0.00% | - |
620.omnetpp_s | 11.23% | 4364.000 | 4854.000 | - | 0.00% | - |
603.bwaves_s | 8.98% | 37064.000 | 40391.000 | - | 0.00% | - |
600.perlbench_s | 8.88% | 2848.000 | 3101.000 | - | 0.00% | - |
605.mcf_s | 7.75% | 6933.000 | 7470.000 | - | 0.00% | - |
602.gcc_s | 7.58% | 5172.000 | 5564.000 | - | 0.00% | - |
510.parest_r | 7.27% | 6242.000 | 6696.000 | - | 30.30% | - |
508.namd_r | 5.78% | 2697.000 | 2853.000 | - | 10.45% | - |
649.fotonik3d_s | 5.04% | 40474.000 | 42514.000 | - | 0.00% | - |
644.nab_s | 4.09% | 11665.000 | 12142.000 | - | 0.00% | - |
631.deepsjeng_s | 4.00% | 4449.000 | 4627.000 | - | 0.00% | - |
511.povray_r | 3.13% | 479.000 | 494.000 | - | 3.13% | - |
627.cam4_s | 2.85% | 1932.000 | 1987.000 | - | 0.00% | - |
625.x264_s | 2.43% | 4858.000 | 4976.000 | - | 0.00% | - |
654.roms_s | 2.32% | 137227.000 | 140409.000 | - | 0.00% | - |
526.blender_r | 2.20% | 10208.000 | 10433.000 | - | 10.61% | - |
641.leela_s | 1.77% | 5367.000 | 5462.000 | - | 0.00% | - |
623.xalancbmk_s | 1.53% | 5761.000 | 5849.000 | - | 0.00% | - |
Performance Regressions - sample variability | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
649.fotonik3d_s (max) | 1419.15% | 141.000 | 2142.000 | - | 0.00% | - |
603.bwaves_s (max) | 1362.50% | 248.000 | 3627.000 | - | 0.00% | - |
649.fotonik3d_s (unexplained diff vs previous) | 1357.14% | 140.000 | 2040.000 | 0.000 | 0.00% | 0.000 |
649.fotonik3d_s (max) (percentage) | 1340.00% | 0.350 | 5.040 | - | 0.00% | - |
619.lbm_s (unexplained diff vs previous) | 1325.70% | 1257.000 | 17921.000 | 0.000 | 0.00% | 0.000 |
603.bwaves_s (unexplained diff vs previous) | 1309.75% | 236.000 | 3327.000 | 0.000 | 0.00% | 0.000 |
649.fotonik3d_s (unexplained diff vs previous) (percentage) | 1271.43% | 0.350 | 4.800 | 0.000 | 0.00% | 0.000 |
603.bwaves_s (max) (percentage) | 1240.30% | 0.670 | 8.980 | - | 0.00% | - |
510.parest_r (max) | 1213.51% | 37.000 | 486.000 | - | 0.00% | - |
603.bwaves_s (unexplained diff vs previous) (percentage) | 1187.50% | 0.640 | 8.240 | 0.000 | 0.00% | 0.000 |
510.parest_r (unexplained diff vs previous) | 1127.03% | 37.000 | 454.000 | 0.000 | 0.00% | 0.000 |
510.parest_r (max) (percentage) | 1111.67% | 0.600 | 7.270 | - | 0.00% | - |
620.omnetpp_s (max) | 1111.11% | 45.000 | 545.000 | - | 0.00% | - |
510.parest_r (unexplained diff vs previous) (percentage) | 1049.15% | 0.590 | 6.780 | 0.000 | 0.00% | 0.000 |
631.deepsjeng_s (unexplained diff vs previous) | 1012.50% | 16.000 | 178.000 | 0.000 | 0.00% | 0.000 |
605.mcf_s (max) | 1011.54% | 52.000 | 578.000 | - | 0.00% | - |
620.omnetpp_s (unexplained diff vs previous) | 988.89% | 45.000 | 490.000 | 0.000 | 0.00% | 0.000 |
620.omnetpp_s (max) (percentage) | 979.81% | 1.040 | 11.230 | - | 0.00% | - |
631.deepsjeng_s (unexplained diff vs previous) (percentage) | 969.44% | 0.360 | 3.850 | 0.000 | 0.00% | 0.000 |
605.mcf_s (unexplained diff vs previous) | 932.69% | 52.000 | 537.000 | 0.000 | 0.00% | 0.000 |
605.mcf_s (max) (percentage) | 919.74% | 0.760 | 7.750 | - | 0.00% | - |
619.lbm_s (max) | 901.75% | 2857.000 | 28620.000 | - | 0.00% | - |
620.omnetpp_s (unexplained diff vs previous) (percentage) | 879.61% | 1.030 | 10.090 | 0.000 | 0.00% | 0.000 |
508.namd_r (unexplained diff vs previous) | 875.00% | 16.000 | 156.000 | 0.000 | 0.00% | 0.000 |
605.mcf_s (unexplained diff vs previous) (percentage) | 858.67% | 0.750 | 7.190 | 0.000 | 0.00% | 0.000 |
654.roms_s (max) | 849.56% | 343.000 | 3257.000 | - | 0.00% | - |
654.roms_s (max) (percentage) | 828.00% | 0.250 | 2.320 | - | 0.00% | - |
508.namd_r (unexplained diff vs previous) (percentage) | 827.12% | 0.590 | 5.470 | 0.000 | 0.00% | 0.000 |
508.namd_r (max) | 811.11% | 18.000 | 164.000 | - | 0.00% | - |
619.lbm_s (unexplained diff vs previous) (percentage) | 792.36% | 4.190 | 37.390 | 0.000 | 0.00% | 0.000 |
508.namd_r (max) (percentage) | 725.71% | 0.700 | 5.780 | - | 0.00% | - |
631.deepsjeng_s (max) | 670.83% | 24.000 | 185.000 | - | 0.00% | - |
631.deepsjeng_s (max) (percentage) | 614.29% | 0.560 | 4.000 | - | 0.00% | - |
625.x264_s (max) | 605.88% | 17.000 | 120.000 | - | 0.00% | - |
625.x264_s (max) (percentage) | 556.76% | 0.370 | 2.430 | - | 0.00% | - |
625.x264_s (unexplained diff vs previous) | 555.56% | 18.000 | 118.000 | 0.000 | 0.00% | 0.000 |
625.x264_s (unexplained diff vs previous) (percentage) | 540.54% | 0.370 | 2.370 | 0.000 | 0.00% | 0.000 |
619.lbm_s (max) (percentage) | 527.21% | 9.520 | 59.710 | - | 0.00% | - |
526.blender_r (max) | 332.08% | 53.000 | 229.000 | - | 0.00% | - |
526.blender_r (max) (percentage) | 323.08% | 0.520 | 2.200 | - | 0.00% | - |
638.imagick_s (max) | 278.24% | 170.000 | 643.000 | - | 0.00% | - |
649.fotonik3d_s | 273.68% | 76.000 | 284.000 | - | 0.00% | - |
649.fotonik3d_s (avg) | 273.68% | 76.000 | 284.000 | - | 0.00% | - |
649.fotonik3d_s (avg) (percentage) | 252.63% | 0.190 | 0.670 | - | 0.00% | - |
638.imagick_s (max) (percentage) | 220.73% | 5.450 | 17.480 | - | 0.00% | - |
628.pop2_s (max) | 180.73% | 192.000 | 539.000 | - | 0.00% | - |
628.pop2_s (max) (percentage) | 178.26% | 0.230 | 0.640 | - | 0.00% | - |
625.x264_s | 175.00% | 8.000 | 22.000 | - | 0.00% | - |
625.x264_s (avg) | 175.00% | 8.000 | 22.000 | - | 0.00% | - |
603.bwaves_s | 165.95% | 185.000 | 492.000 | - | 0.00% | - |
603.bwaves_s (avg) | 165.95% | 185.000 | 492.000 | - | 0.00% | - |
625.x264_s (avg) (percentage) | 164.71% | 0.170 | 0.450 | - | 0.00% | - |
638.imagick_s | 163.64% | 55.000 | 145.000 | - | 0.00% | - |
638.imagick_s (avg) | 163.64% | 55.000 | 145.000 | - | 0.00% | - |
619.lbm_s | 147.74% | 1749.000 | 4333.000 | - | 0.00% | - |
619.lbm_s (avg) | 147.74% | 1749.000 | 4333.000 | - | 0.00% | - |
603.bwaves_s (avg) (percentage) | 144.00% | 0.500 | 1.220 | - | 0.00% | - |
654.roms_s | 138.46% | 247.000 | 589.000 | - | 0.00% | - |
654.roms_s (avg) | 138.46% | 247.000 | 589.000 | - | 0.00% | - |
641.leela_s (max) | 134.15% | 41.000 | 96.000 | - | 0.00% | - |
654.roms_s (avg) (percentage) | 133.33% | 0.180 | 0.420 | - | 0.00% | - |
641.leela_s (unexplained diff vs previous) | 131.71% | 41.000 | 95.000 | 0.000 | 0.00% | 0.000 |
641.leela_s (max) (percentage) | 129.87% | 0.770 | 1.770 | - | 0.00% | - |
641.leela_s (unexplained diff vs previous) (percentage) | 128.95% | 0.760 | 1.740 | 0.000 | 0.00% | 0.000 |
638.imagick_s (avg) (percentage) | 125.00% | 1.760 | 3.960 | - | 0.00% | - |
605.mcf_s | 106.45% | 31.000 | 64.000 | - | 0.00% | - |
605.mcf_s (avg) | 106.45% | 31.000 | 64.000 | - | 0.00% | - |
605.mcf_s (avg) (percentage) | 93.33% | 0.450 | 0.870 | - | 0.00% | - |
508.namd_r | 92.86% | 14.000 | 27.000 | - | 0.00% | - |
508.namd_r (avg) | 92.86% | 14.000 | 27.000 | - | 0.00% | - |
508.namd_r (avg) (percentage) | 88.46% | 0.520 | 0.980 | - | 0.00% | - |
511.povray_r (max) | 87.50% | 8.000 | 15.000 | - | 0.00% | - |
526.blender_r | 86.84% | 38.000 | 71.000 | - | 0.00% | - |
526.blender_r (avg) | 86.84% | 38.000 | 71.000 | - | 0.00% | - |
526.blender_r (avg) (percentage) | 81.58% | 0.380 | 0.690 | - | 0.00% | - |
511.povray_r (max) (percentage) | 79.89% | 1.740 | 3.130 | - | 0.00% | - |
620.omnetpp_s | 78.05% | 41.000 | 73.000 | - | 0.00% | - |
620.omnetpp_s (avg) | 78.05% | 41.000 | 73.000 | - | 0.00% | - |
644.nab_s (max) | 71.63% | 289.000 | 496.000 | - | 0.00% | - |
628.pop2_s | 71.30% | 108.000 | 185.000 | - | 0.00% | - |
628.pop2_s (avg) | 71.30% | 108.000 | 185.000 | - | 0.00% | - |
628.pop2_s (avg) (percentage) | 69.23% | 0.130 | 0.220 | - | 0.00% | - |
644.nab_s (max) (percentage) | 64.92% | 2.480 | 4.090 | - | 0.00% | - |
510.parest_r | 64.58% | 48.000 | 79.000 | - | 0.00% | - |
510.parest_r (avg) | 64.58% | 48.000 | 79.000 | - | 0.00% | - |
631.deepsjeng_s | 63.64% | 22.000 | 36.000 | - | 0.00% | - |
631.deepsjeng_s (avg) | 63.64% | 22.000 | 36.000 | - | 0.00% | - |
644.nab_s (unexplained diff vs previous) | 60.61% | 297.000 | 477.000 | 0.000 | 0.00% | 0.000 |
631.deepsjeng_s (avg) (percentage) | 58.00% | 0.500 | 0.790 | - | 0.00% | - |
620.omnetpp_s (avg) (percentage) | 57.29% | 0.960 | 1.510 | - | 0.00% | - |
619.lbm_s (avg) (percentage) | 55.06% | 5.830 | 9.040 | - | 0.00% | - |
510.parest_r (avg) (percentage) | 54.55% | 0.770 | 1.190 | - | 0.00% | - |
644.nab_s (unexplained diff vs previous) (percentage) | 54.12% | 2.550 | 3.930 | 0.000 | 0.00% | 0.000 |
644.nab_s | 41.58% | 101.000 | 143.000 | - | 0.00% | - |
644.nab_s (avg) | 41.58% | 101.000 | 143.000 | - | 0.00% | - |
644.nab_s (avg) (percentage) | 35.63% | 0.870 | 1.180 | - | 0.00% | - |
641.leela_s | 33.33% | 18.000 | 24.000 | - | 0.00% | - |
641.leela_s (avg) | 33.33% | 18.000 | 24.000 | - | 0.00% | - |
641.leela_s (avg) (percentage) | 32.35% | 0.340 | 0.450 | - | 0.00% | - |
511.povray_r (avg) (percentage) | 20.31% | 1.280 | 1.540 | - | 0.00% | - |
657.xz_s | 18.52% | 27.000 | 32.000 | - | 0.00% | - |
657.xz_s (avg) | 18.52% | 27.000 | 32.000 | - | 0.00% | - |
657.xz_s (max) | 18.18% | 22.000 | 26.000 | - | 0.00% | - |
511.povray_r | 16.67% | 6.000 | 7.000 | - | 0.00% | - |
511.povray_r (avg) | 16.67% | 6.000 | 7.000 | - | 0.00% | - |
600.perlbench_s | 15.38% | 13.000 | 15.000 | - | 0.00% | - |
600.perlbench_s (avg) | 15.38% | 13.000 | 15.000 | - | 0.00% | - |
600.perlbench_s (max) | 8.33% | 12.000 | 13.000 | - | 0.00% | - |
Performance Regressions - Score | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
nb_execution_successful | -4.55% | 44.000 | 42.000 | - | 950.00% | - |
nb_compile_successful | -4.35% | 46.000 | 44.000 | - | 100.00% | - |
nb_total_tests | -4.00% | 50.000 | 48.000 | - | 100.00% | - |
Performance Improvements - sample variability | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
648.exchange2_s (max) (percentage) | -7.69% | 0.130 | 0.120 | - | 0.00% | - |
648.exchange2_s (avg) (percentage) | -7.14% | 0.280 | 0.260 | - | 0.00% | - |
648.exchange2_s | -7.05% | 156.000 | 145.000 | - | 0.00% | - |
648.exchange2_s (avg) | -7.05% | 156.000 | 145.000 | - | 0.00% | - |
648.exchange2_s (max) | -6.94% | 72.000 | 67.000 | - | 0.00% | - |
657.xz_s (max) (percentage) | -3.06% | 0.980 | 0.950 | - | 0.00% | - |
600.perlbench_s (max) (percentage) | -2.27% | 0.440 | 0.430 | - | 0.00% | - |
Added Tests - sample variability |
---|
511.povray_r (unexplained diff vs previous) |
511.povray_r (unexplained diff vs previous) (percentage) |
526.blender_r (unexplained diff vs previous) |
526.blender_r (unexplained diff vs previous) (percentage) |
628.pop2_s (unexplained diff vs previous) |
628.pop2_s (unexplained diff vs previous) (percentage) |
638.imagick_s (unexplained diff vs previous) |
638.imagick_s (unexplained diff vs previous) (percentage) |
648.exchange2_s (unexplained diff vs previous) |
648.exchange2_s (unexplained diff vs previous) (percentage) |
654.roms_s (unexplained diff vs previous) |
654.roms_s (unexplained diff vs previous) (percentage) |
Performance Regressions - Execution | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
510.parest_r | 30.30% | 5139.000 | 6696.000 | - | 7.27% | - |
526.blender_r | 10.61% | 9432.000 | 10433.000 | - | 2.20% | - |
508.namd_r | 10.45% | 2583.000 | 2853.000 | - | 5.78% | - |
511.povray_r | 3.13% | 479.000 | 494.000 | - | 3.13% | - |
Performance Improvements - Score | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
nb_execution_successful | 950.00% | 4.000 | 42.000 | - | -4.55% | - |
nb_compile_successful | 100.00% | 22.000 | 44.000 | - | -4.35% | - |
nb_total_tests | 100.00% | 24.000 | 48.000 | - | -4.00% | - |
Added Tests - Code Size |
---|
600.perlbench_s |
602.gcc_s |
603.bwaves_s |
605.mcf_s |
619.lbm_s |
620.omnetpp_s |
623.xalancbmk_s |
625.x264_s |
628.pop2_s |
631.deepsjeng_s |
638.imagick_s |
641.leela_s |
644.nab_s |
648.exchange2_s |
649.fotonik3d_s |
654.roms_s |
657.xz_s |
Added Tests - Execution |
---|
600.perlbench_s |
602.gcc_s |
603.bwaves_s |
605.mcf_s |
619.lbm_s |
620.omnetpp_s |
623.xalancbmk_s |
625.x264_s |
627.cam4_s |
628.pop2_s |
631.deepsjeng_s |
638.imagick_s |
641.leela_s |
644.nab_s |
648.exchange2_s |
649.fotonik3d_s |
654.roms_s |
657.xz_s |
Added Tests - sample variability |
---|
508.namd_r |
508.namd_r (avg) |
508.namd_r (avg) (percentage) |
508.namd_r (max) |
508.namd_r (max) (percentage) |
510.parest_r |
510.parest_r (avg) |
510.parest_r (avg) (percentage) |
510.parest_r (max) |
510.parest_r (max) (percentage) |
511.povray_r |
511.povray_r (avg) |
511.povray_r (avg) (percentage) |
511.povray_r (max) |
511.povray_r (max) (percentage) |
526.blender_r |
526.blender_r (avg) |
526.blender_r (avg) (percentage) |
526.blender_r (max) |
526.blender_r (max) (percentage) |
600.perlbench_s |
600.perlbench_s (avg) |
600.perlbench_s (avg) (percentage) |
600.perlbench_s (max) |
600.perlbench_s (max) (percentage) |
603.bwaves_s |
603.bwaves_s (avg) |
603.bwaves_s (avg) (percentage) |
603.bwaves_s (max) |
603.bwaves_s (max) (percentage) |
603.bwaves_s (unexplained diff vs previous) |
603.bwaves_s (unexplained diff vs previous) (percentage) |
605.mcf_s |
605.mcf_s (avg) |
605.mcf_s (avg) (percentage) |
605.mcf_s (max) |
605.mcf_s (max) (percentage) |
605.mcf_s (unexplained diff vs previous) |
605.mcf_s (unexplained diff vs previous) (percentage) |
619.lbm_s |
619.lbm_s (avg) |
619.lbm_s (avg) (percentage) |
619.lbm_s (max) |
619.lbm_s (max) (percentage) |
619.lbm_s (unexplained diff vs previous) |
619.lbm_s (unexplained diff vs previous) (percentage) |
620.omnetpp_s |
620.omnetpp_s (avg) |
620.omnetpp_s (avg) (percentage) |
620.omnetpp_s (max) |
620.omnetpp_s (max) (percentage) |
620.omnetpp_s (unexplained diff vs previous) |
620.omnetpp_s (unexplained diff vs previous) (percentage) |
625.x264_s |
625.x264_s (avg) |
625.x264_s (avg) (percentage) |
625.x264_s (max) |
625.x264_s (max) (percentage) |
625.x264_s (unexplained diff vs previous) |
625.x264_s (unexplained diff vs previous) (percentage) |
628.pop2_s |
628.pop2_s (avg) |
628.pop2_s (avg) (percentage) |
628.pop2_s (max) |
628.pop2_s (max) (percentage) |
628.pop2_s (unexplained diff vs previous) |
628.pop2_s (unexplained diff vs previous) (percentage) |
631.deepsjeng_s |
631.deepsjeng_s (avg) |
631.deepsjeng_s (avg) (percentage) |
631.deepsjeng_s (max) |
631.deepsjeng_s (max) (percentage) |
631.deepsjeng_s (unexplained diff vs previous) |
631.deepsjeng_s (unexplained diff vs previous) (percentage) |
638.imagick_s |
638.imagick_s (avg) |
638.imagick_s (avg) (percentage) |
638.imagick_s (max) |
638.imagick_s (max) (percentage) |
638.imagick_s (unexplained diff vs previous) |
638.imagick_s (unexplained diff vs previous) (percentage) |
641.leela_s |
641.leela_s (avg) |
641.leela_s (avg) (percentage) |
641.leela_s (max) |
641.leela_s (max) (percentage) |
641.leela_s (unexplained diff vs previous) |
641.leela_s (unexplained diff vs previous) (percentage) |
644.nab_s |
644.nab_s (avg) |
644.nab_s (avg) (percentage) |
644.nab_s (max) |
644.nab_s (max) (percentage) |
644.nab_s (unexplained diff vs previous) |
644.nab_s (unexplained diff vs previous) (percentage) |
648.exchange2_s |
648.exchange2_s (avg) |
648.exchange2_s (avg) (percentage) |
648.exchange2_s (max) |
648.exchange2_s (max) (percentage) |
648.exchange2_s (unexplained diff vs previous) |
648.exchange2_s (unexplained diff vs previous) (percentage) |
649.fotonik3d_s |
649.fotonik3d_s (avg) |
649.fotonik3d_s (avg) (percentage) |
649.fotonik3d_s (max) |
649.fotonik3d_s (max) (percentage) |
649.fotonik3d_s (unexplained diff vs previous) |
649.fotonik3d_s (unexplained diff vs previous) (percentage) |
654.roms_s |
654.roms_s (avg) |
654.roms_s (avg) (percentage) |
654.roms_s (max) |
654.roms_s (max) (percentage) |
654.roms_s (unexplained diff vs previous) |
654.roms_s (unexplained diff vs previous) (percentage) |
657.xz_s |
657.xz_s (avg) |
657.xz_s (avg) (percentage) |
657.xz_s (max) |
657.xz_s (max) (percentage) |
Report Time: 0.36s