Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c839a44c39161c9932d58c28c2949ab3bb94ea6f (+ 44 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/c726a6643125a59e2ba6f992924a2d0098104578 (+ 34 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=e82a7cb1622bff08d8e3a144d7c5516a088f1cbc (+ 8 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/6f0a8fd3aaa73ffc8e87ab9da86f27331820085c (+ 185 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/422/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0422 (4-changed-components) 2024-08-27T23:35:49 0:00:00
Previous 0421 (4-changed-components) 2024-08-14T18:21:26 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 75 77
Performance Improvements 71 138
Added Tests 8 2
Unchanged Tests 1082 1019
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
436.cactusADM 1.80% 2107.000 2145.000 - -87.77% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
445.gobmk (unexplained diff vs previous) 2000.00% 3.000 63.000 0.000 -66.13% 0.000
445.gobmk (unexplained diff vs previous) (percentage) 1950.00% 0.060 1.230 0.000 -20.65% 0.000
473.astar (unexplained diff vs previous) (percentage) 1876.19% 0.210 4.150 0.000 20650.00% 0.000
473.astar (unexplained diff vs previous) 1825.00% 12.000 231.000 0.000 7600.00% 0.000
429.mcf (unexplained diff vs previous) (percentage) 1494.83% 0.580 9.250 0.000 1612.96% 0.000
429.mcf (unexplained diff vs previous) 1350.00% 12.000 174.000 0.000 87.10% 0.000
458.sjeng (unexplained diff vs previous) (percentage) 830.77% 0.130 1.210 0.000 1916.67% 0.000
458.sjeng (unexplained diff vs previous) 820.00% 10.000 92.000 0.000 922.22% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) 539.53% 0.430 2.750 0.000 343.55% 0.000
482.sphinx3 (unexplained diff vs previous) 525.00% 4.000 25.000 0.000 -86.56% 0.000
450.soplex (unexplained diff vs previous) (percentage) 447.89% 0.710 3.890 0.000 93.53% 0.000
470.lbm (unexplained diff vs previous) 428.57% 7.000 37.000 0.000 164.29% 0.000
470.lbm (unexplained diff vs previous) (percentage) 426.32% 0.190 1.000 0.000 809.09% 0.000
450.soplex (unexplained diff vs previous) 425.00% 4.000 21.000 0.000 -93.25% 0.000
470.lbm (max) (percentage) 421.05% 0.190 0.990 - -80.59% -
470.lbm (max) 414.29% 7.000 36.000 - -94.49% -
482.sphinx3 (max) 380.00% 5.000 24.000 - -94.53% -
433.milc (unexplained diff vs previous) (percentage) 363.46% 0.520 2.410 0.000 -26.52% 0.000
433.milc (unexplained diff vs previous) 350.00% 6.000 27.000 0.000 -93.65% 0.000
482.sphinx3 (max) (percentage) 318.75% 0.640 2.680 - 82.31% -
473.astar (max) (percentage) 287.38% 1.030 3.990 - 2750.00% -
473.astar (max) 274.58% 59.000 221.000 - 1005.00% -
465.tonto (unexplained diff vs previous) (percentage) 271.70% 0.530 1.970 0.000 155.84% 0.000
465.tonto (unexplained diff vs previous) 263.33% 60.000 218.000 0.000 246.03% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) 239.47% 0.760 2.580 0.000 1128.57% 0.000
450.soplex (max) 233.33% 6.000 20.000 - -93.42% -
459.GemsFDTD (unexplained diff vs previous) 230.77% 39.000 129.000 0.000 168.75% 0.000
450.soplex (max) (percentage) 216.95% 1.180 3.740 - 89.85% -
433.milc (max) 188.89% 9.000 26.000 - -93.67% -
433.milc (max) (percentage) 179.76% 0.840 2.350 - -25.87% -
435.gromacs (unexplained diff vs previous) (percentage) 134.96% 1.230 2.890 0.000 40.29% 0.000
435.gromacs (unexplained diff vs previous) 128.74% 87.000 199.000 0.000 -37.42% 0.000
456.hmmer (unexplained diff vs previous) 116.67% 6.000 13.000 0.000 -84.34% 0.000
456.hmmer (unexplained diff vs previous) (percentage) 114.29% 0.210 0.450 0.000 -40.00% 0.000
444.namd (unexplained diff vs previous) (percentage) 93.16% 1.170 2.260 0.000 -70.76% 0.000
444.namd (unexplained diff vs previous) 88.89% 9.000 17.000 0.000 -97.57% 0.000
458.sjeng (max) (percentage) 87.50% 0.640 1.200 - 445.45% -
444.namd (max) (percentage) 87.29% 1.180 2.210 - -69.22% -
458.sjeng (max) 85.71% 49.000 91.000 - 193.55% -
410.bwaves (unexplained diff vs previous) (percentage) 84.29% 0.700 1.290 0.000 1190.00% 0.000
410.bwaves (unexplained diff vs previous) 81.40% 43.000 78.000 0.000 420.00% 0.000
444.namd (max) 77.78% 9.000 16.000 - -97.53% -
453.povray (unexplained diff vs previous) (percentage) 77.50% 0.800 1.420 0.000 7000.00% 0.000
453.povray (unexplained diff vs previous) 75.00% 4.000 7.000 0.000 600.00% 0.000
436.cactusADM (max) 65.22% 23.000 38.000 - -93.12% -
436.cactusADM (max) (percentage) 59.29% 1.130 1.800 - -42.86% -
436.cactusADM (unexplained diff vs previous) 58.33% 24.000 38.000 0.000 -93.33% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) 55.26% 1.140 1.770 0.000 -45.54% 0.000
434.zeusmp (max) (percentage) 54.03% 1.240 1.910 - -16.23% -
434.zeusmp (max) 51.61% 31.000 47.000 - -84.39% -
453.povray (max) 50.00% 4.000 6.000 - -96.36% -
453.povray (max) (percentage) 48.94% 0.940 1.400 - -44.66% -
400.perlbench (unexplained diff vs previous) (percentage) 45.53% 1.230 1.790 0.000 -46.57% 0.000
400.perlbench (unexplained diff vs previous) 42.11% 19.000 27.000 0.000 -92.44% 0.000
429.mcf (max) (percentage) 40.77% 6.010 8.460 - 1466.67% -
459.GemsFDTD (max) (percentage) 29.38% 1.940 2.510 - 318.33% -
429.mcf (max) 29.27% 123.000 159.000 - 72.83% -
464.h264ref (max) 29.03% 31.000 40.000 - -35.48% -
464.h264ref (max) (percentage) 27.78% 0.540 0.690 - 60.47% -
459.GemsFDTD (max) 26.26% 99.000 125.000 - -10.71% -
465.tonto (max) (percentage) 24.36% 1.560 1.940 - -45.81% -
465.tonto (max) 22.29% 175.000 214.000 - -26.96% -
410.bwaves (max) (percentage) 20.75% 1.060 1.280 - 509.52% -
410.bwaves (max) 20.31% 64.000 77.000 - 156.67% -
401.bzip2 (unexplained diff vs previous) (percentage) 16.77% 1.670 1.950 0.000 124.14% 0.000
401.bzip2 (unexplained diff vs previous) 14.00% 50.000 57.000 0.000 -56.82% 0.000
437.leslie3d (max) (percentage) 8.51% 3.760 4.080 - 74.36% -
437.leslie3d (max) 4.01% 424.000 441.000 - 25.64% -
473.astar (avg) (percentage) 1.78% 1.690 1.720 - 805.26% -
400.perlbench (max) (percentage) 1.74% 1.720 1.750 - -45.99% -
459.GemsFDTD (avg) (percentage) 1.40% 1.430 1.450 - 61.11% -
458.sjeng (avg) (percentage) 1.30% 0.770 0.780 - 136.36% -
482.sphinx3 (avg) (percentage) 1.06% 1.880 1.900 - 30.14% -
435.gromacs (avg) (percentage) 1.05% 1.910 1.930 - 11.56% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf -8.46% 2056.000 1882.000 - -89.03% -
437.leslie3d -4.08% 11281.000 10821.000 - -27.88% -
473.astar -3.99% 5791.000 5560.000 - -62.76% -
450.soplex -3.74% 561.000 540.000 - -96.51% -
435.gromacs -2.81% 7082.000 6883.000 - -55.36% -
482.sphinx3 -2.68% 933.000 908.000 - -96.97% -
471.omnetpp -2.57% 3000.000 2923.000 - -72.78% -
459.GemsFDTD -2.51% 5137.000 5008.000 - -78.56% -
433.milc -2.35% 1149.000 1122.000 - -91.35% -
444.namd -2.21% 769.000 752.000 - -91.69% -
465.tonto -1.94% 11258.000 11040.000 - 34.55% -
401.bzip2 -1.91% 2987.000 2930.000 - -80.70% -
434.zeusmp -1.91% 2518.000 2470.000 - -81.30% -
400.perlbench -1.75% 1539.000 1512.000 - -85.80% -
453.povray -1.40% 499.000 492.000 - -92.48% -
410.bwaves -1.28% 6114.000 6036.000 - -59.09% -
445.gobmk -1.22% 5169.000 5106.000 - -57.43% -
458.sjeng -1.20% 7678.000 7586.000 - -46.86% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (max) -16.67% 90.000 75.000 - -88.67% -
471.omnetpp (max) (percentage) -14.90% 3.020 2.570 - -58.35% -
471.omnetpp (unexplained diff vs previous) -12.50% 88.000 77.000 0.000 196.15% 0.000
456.hmmer (max) -12.00% 25.000 22.000 - -84.29% -
456.hmmer (max) (percentage) -11.36% 0.880 0.780 - -38.58% -
471.omnetpp (unexplained diff vs previous) (percentage) -10.24% 2.930 2.630 0.000 995.83% 0.000
462.libquantum (max) (percentage) -8.53% 1.290 1.180 - 84.37% -
429.mcf -7.76% 116.000 107.000 - 18.89% -
429.mcf (avg) -7.76% 116.000 107.000 - 18.89% -
447.dealII (max) -7.69% 13.000 12.000 - -88.79% -
401.bzip2 (max) -7.35% 68.000 63.000 - -52.63% -
447.dealII (max) (percentage) -7.25% 0.690 0.640 - -41.82% -
450.soplex -6.25% 16.000 15.000 - -92.15% -
450.soplex (avg) -6.25% 16.000 15.000 - -92.15% -
435.gromacs (max) -4.93% 203.000 193.000 - -37.94% -
401.bzip2 (max) (percentage) -4.80% 2.290 2.180 - 147.73% -
445.gobmk (max) -4.41% 68.000 65.000 - -64.48% -
456.hmmer -4.35% 23.000 22.000 - -70.67% -
456.hmmer (avg) -4.35% 23.000 22.000 - -70.67% -
437.leslie3d -4.12% 485.000 465.000 - 30.25% -
437.leslie3d (avg) -4.12% 485.000 465.000 - 30.25% -
445.gobmk (max) (percentage) -3.76% 1.330 1.280 - -16.34% -
400.perlbench -3.23% 31.000 30.000 - -80.77% -
400.perlbench (avg) -3.23% 31.000 30.000 - -80.77% -
401.bzip2 -3.23% 62.000 60.000 - -9.09% -
401.bzip2 (avg) -3.23% 62.000 60.000 - -9.09% -
403.gcc (max) (percentage) -2.83% 7.070 6.870 - 854.17% -
471.omnetpp -2.82% 71.000 69.000 - -59.41% -
471.omnetpp (avg) -2.82% 71.000 69.000 - -59.41% -
434.zeusmp -2.78% 36.000 35.000 - -82.05% -
434.zeusmp (avg) -2.78% 36.000 35.000 - -82.05% -
470.lbm -2.38% 42.000 41.000 - -86.47% -
470.lbm (avg) -2.38% 42.000 41.000 - -86.47% -
454.calculix (max) (percentage) -2.24% 1.340 1.310 - -72.13% -
435.gromacs -2.22% 135.000 132.000 - -50.38% -
435.gromacs (avg) -2.22% 135.000 132.000 - -50.38% -
435.gromacs (max) (percentage) -2.09% 2.870 2.810 - 39.11% -
473.astar -2.06% 97.000 95.000 - 239.29% -
473.astar (avg) -2.06% 97.000 95.000 - 239.29% -
464.h264ref -1.92% 52.000 51.000 - -34.62% -
464.h264ref (avg) -1.92% 52.000 51.000 - -34.62% -
445.gobmk -1.79% 56.000 55.000 - -40.22% -
445.gobmk (avg) -1.79% 56.000 55.000 - -40.22% -
410.bwaves -1.41% 71.000 70.000 - -56.25% -
410.bwaves (avg) -1.41% 71.000 70.000 - -56.25% -
465.tonto -1.38% 145.000 143.000 - 4.38% -
465.tonto (avg) -1.38% 145.000 143.000 - 4.38% -
459.GemsFDTD -1.37% 73.000 72.000 - -65.71% -
459.GemsFDTD (avg) -1.37% 73.000 72.000 - -65.71% -
456.hmmer (avg) (percentage) -1.23% 0.810 0.800 - 17.65% -
462.libquantum (avg) (percentage) -1.18% 1.690 1.670 - 156.92% -
481.wrf (max) (percentage) -1.15% 1.740 1.720 - -45.40% -
464.h264ref (avg) (percentage) -1.14% 0.880 0.870 - 61.11% -

Added Tests - sample variability
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)
437.leslie3d (unexplained diff vs previous)
437.leslie3d (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 34.55% 8205.000 11040.000 - -1.94% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
473.astar (unexplained diff vs previous) (percentage) 20650.00% 0.020 4.150 0.000 1876.19% 0.000
473.astar (unexplained diff vs previous) 7600.00% 3.000 231.000 0.000 1825.00% 0.000
453.povray (unexplained diff vs previous) (percentage) 7000.00% 0.020 1.420 0.000 77.50% 0.000
473.astar (max) (percentage) 2750.00% 0.140 3.990 - 287.38% -
458.sjeng (unexplained diff vs previous) (percentage) 1916.67% 0.060 1.210 0.000 830.77% 0.000
429.mcf (unexplained diff vs previous) (percentage) 1612.96% 0.540 9.250 0.000 1494.83% 0.000
429.mcf (max) (percentage) 1466.67% 0.540 8.460 - 40.77% -
483.xalancbmk (max) (percentage) 1292.68% 0.410 5.710 - -0.17% -
410.bwaves (unexplained diff vs previous) (percentage) 1190.00% 0.100 1.290 0.000 84.29% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) 1128.57% 0.210 2.580 0.000 239.47% 0.000
473.astar (max) 1005.00% 20.000 221.000 - 274.58% -
471.omnetpp (unexplained diff vs previous) (percentage) 995.83% 0.240 2.630 0.000 -10.24% 0.000
429.mcf (avg) (percentage) 973.58% 0.530 5.690 - 0.71% -
403.gcc (avg) (percentage) 945.90% 0.610 6.380 - 0.00% -
458.sjeng (unexplained diff vs previous) 922.22% 9.000 92.000 0.000 820.00% 0.000
403.gcc (max) (percentage) 854.17% 0.720 6.870 - -2.83% -
470.lbm (unexplained diff vs previous) (percentage) 809.09% 0.110 1.000 0.000 426.32% 0.000
473.astar (avg) (percentage) 805.26% 0.190 1.720 - 1.78% -
453.povray (unexplained diff vs previous) 600.00% 1.000 7.000 0.000 75.00% 0.000
464.h264ref (unexplained diff vs previous) (percentage) 600.00% 0.100 0.700 0.000 0.00% 0.000
410.bwaves (max) (percentage) 509.52% 0.210 1.280 - 20.75% -
458.sjeng (max) (percentage) 445.45% 0.220 1.200 - 87.50% -
410.bwaves (unexplained diff vs previous) 420.00% 15.000 78.000 0.000 81.40% 0.000
401.bzip2 (avg) (percentage) 372.73% 0.440 2.080 - 0.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) 343.55% 0.620 2.750 0.000 539.53% 0.000
459.GemsFDTD (max) (percentage) 318.33% 0.600 2.510 - 29.38% -
465.tonto (unexplained diff vs previous) 246.03% 63.000 218.000 0.000 263.33% 0.000
473.astar 239.29% 28.000 95.000 - -2.06% -
473.astar (avg) 239.29% 28.000 95.000 - -2.06% -
471.omnetpp (unexplained diff vs previous) 196.15% 26.000 77.000 0.000 -12.50% 0.000
458.sjeng (max) 193.55% 31.000 91.000 - 85.71% -
464.h264ref (unexplained diff vs previous) 173.33% 15.000 41.000 0.000 0.00% 0.000
459.GemsFDTD (unexplained diff vs previous) 168.75% 48.000 129.000 0.000 230.77% 0.000
470.lbm (unexplained diff vs previous) 164.29% 14.000 37.000 0.000 428.57% 0.000
462.libquantum (avg) (percentage) 156.92% 0.650 1.670 - -1.18% -
410.bwaves (max) 156.67% 30.000 77.000 - 20.31% -
465.tonto (unexplained diff vs previous) (percentage) 155.84% 0.770 1.970 0.000 271.70% 0.000
401.bzip2 (max) (percentage) 147.73% 0.880 2.180 - -4.80% -
458.sjeng (avg) (percentage) 136.36% 0.330 0.780 - 1.30% -
450.soplex (avg) (percentage) 131.45% 1.240 2.870 - 0.35% -
401.bzip2 (unexplained diff vs previous) (percentage) 124.14% 0.870 1.950 0.000 16.77% 0.000
450.soplex (unexplained diff vs previous) (percentage) 93.53% 2.010 3.890 0.000 447.89% 0.000
450.soplex (max) (percentage) 89.85% 1.970 3.740 - 216.95% -
429.mcf (unexplained diff vs previous) 87.10% 93.000 174.000 0.000 1350.00% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) 85.59% 2.290 4.250 0.000 0.00% 0.000
462.libquantum (max) (percentage) 84.37% 0.640 1.180 - -8.53% -
482.sphinx3 (max) (percentage) 82.31% 1.470 2.680 - 318.75% -
437.leslie3d (avg) (percentage) 80.67% 2.380 4.300 - 0.00% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
437.leslie3d (max) (percentage) 74.36% 2.340 4.080 - 8.51% -
429.mcf (max) 72.83% 92.000 159.000 - 29.27% -
459.GemsFDTD (avg) (percentage) 61.11% 0.900 1.450 - 1.40% -
464.h264ref (avg) (percentage) 61.11% 0.540 0.870 - -1.14% -
464.h264ref (max) (percentage) 60.47% 0.430 0.690 - 27.78% -
454.calculix (avg) (percentage) 52.24% 2.010 3.060 - 0.00% -
471.omnetpp (avg) (percentage) 50.31% 1.590 2.390 - 0.00% -
453.povray (avg) (percentage) 44.00% 1.250 1.800 - 0.00% -
445.gobmk (avg) (percentage) 41.56% 0.770 1.090 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) 40.29% 2.060 2.890 0.000 134.96% 0.000
400.perlbench (avg) (percentage) 39.46% 1.470 2.050 - 0.00% -
435.gromacs (max) (percentage) 39.11% 2.020 2.810 - -2.09% -
437.leslie3d (unexplained diff vs previous) 34.11% 343.000 460.000 0.000 0.00% 0.000
447.dealII (avg) (percentage) 30.84% 1.070 1.400 - 0.00% -
437.leslie3d 30.25% 357.000 465.000 - -4.12% -
437.leslie3d (avg) 30.25% 357.000 465.000 - -4.12% -
482.sphinx3 (avg) (percentage) 30.14% 1.460 1.900 - 1.06% -
437.leslie3d (max) 25.64% 351.000 441.000 - 4.01% -
458.sjeng 25.53% 47.000 59.000 - 0.00% -
458.sjeng (avg) 25.53% 47.000 59.000 - 0.00% -
429.mcf 18.89% 90.000 107.000 - -7.76% -
429.mcf (avg) 18.89% 90.000 107.000 - -7.76% -
456.hmmer (avg) (percentage) 17.65% 0.680 0.800 - -1.23% -
435.gromacs (avg) (percentage) 11.56% 1.730 1.930 - 1.05% -
410.bwaves (avg) (percentage) 7.34% 1.090 1.170 - 0.00% -
465.tonto 4.38% 137.000 143.000 - -1.38% -
465.tonto (avg) 4.38% 137.000 143.000 - -1.38% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -8.10% 120221.000 110489.000 - 0.00% -
437.leslie3d -7.59% 65177.000 60228.000 - 0.00% -
459.GemsFDTD -4.56% 253716.000 242157.000 - 0.00% -
434.zeusmp -3.91% 165699.000 159221.000 - 0.00% -
464.h264ref -3.09% 428364.000 415120.000 - 0.00% -
410.bwaves -2.91% 17880.000 17360.000 - 0.00% -
453.povray -1.83% 709140.000 696190.000 - 0.00% -
435.gromacs -1.78% 679165.000 667109.000 - 0.00% -
433.milc -1.69% 95873.000 94253.000 - -0.04% -
444.namd -1.52% 191059.000 188162.000 - 0.00% -
400.perlbench -1.43% 930652.000 917322.000 - 0.00% -
470.lbm -1.43% 10908.000 10752.000 - 0.00% -
436.cactusADM -1.34% 554386.000 546946.000 - 0.00% -
450.soplex -1.20% 260647.000 257521.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.85% 11614.000 133.000 - 0.00% -
482.sphinx3 -96.97% 29928.000 908.000 - -2.68% -
450.soplex -96.51% 15466.000 540.000 - -3.74% -
453.povray -92.48% 6544.000 492.000 - -1.40% -
444.namd -91.69% 9050.000 752.000 - -2.21% -
433.milc -91.35% 12976.000 1122.000 - -2.35% -
429.mcf -89.03% 17158.000 1882.000 - -8.46% -
436.cactusADM -87.77% 17544.000 2145.000 - 1.80% -
400.perlbench -85.80% 10650.000 1512.000 - -1.75% -
434.zeusmp -81.30% 13211.000 2470.000 - -1.91% -
401.bzip2 -80.70% 15185.000 2930.000 - -1.91% -
447.dealII -80.28% 9751.000 1923.000 - 0.10% -
459.GemsFDTD -78.56% 23356.000 5008.000 - -2.51% -
456.hmmer -73.99% 11046.000 2873.000 - -0.45% -
471.omnetpp -72.78% 10739.000 2923.000 - -2.57% -
470.lbm -71.16% 12819.000 3697.000 - -0.99% -
473.astar -62.76% 14932.000 5560.000 - -3.99% -
464.h264ref -59.67% 14556.000 5871.000 - -0.69% -
410.bwaves -59.09% 14755.000 6036.000 - -1.28% -
445.gobmk -57.43% 11995.000 5106.000 - -1.22% -
435.gromacs -55.36% 15419.000 6883.000 - -2.81% -
458.sjeng -46.86% 14275.000 7586.000 - -1.20% -
437.leslie3d -27.88% 15005.000 10821.000 - -4.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) -98.65% 74.000 1.000 - 0.00% -
444.namd (unexplained diff vs previous) -97.57% 700.000 17.000 0.000 88.89% 0.000
444.namd (max) -97.53% 649.000 16.000 - 77.78% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
444.namd -96.62% 355.000 12.000 - 0.00% -
444.namd (avg) -96.62% 355.000 12.000 - 0.00% -
453.povray (max) -96.36% 165.000 6.000 - 50.00% -
482.sphinx3 -96.10% 436.000 17.000 - 0.00% -
482.sphinx3 (avg) -96.10% 436.000 17.000 - 0.00% -
416.gamess (max) (percentage) -95.50% 2.890 0.130 - 0.00% -
482.sphinx3 (max) -94.53% 439.000 24.000 - 380.00% -
470.lbm (max) -94.49% 653.000 36.000 - 414.29% -
433.milc (max) -93.67% 411.000 26.000 - 188.89% -
433.milc (unexplained diff vs previous) -93.65% 425.000 27.000 0.000 350.00% 0.000
450.soplex (max) -93.42% 304.000 20.000 - 233.33% -
436.cactusADM (unexplained diff vs previous) -93.33% 570.000 38.000 0.000 58.33% 0.000
450.soplex (unexplained diff vs previous) -93.25% 311.000 21.000 0.000 425.00% 0.000
436.cactusADM (max) -93.12% 552.000 38.000 - 65.22% -
400.perlbench (max) -92.46% 345.000 26.000 - 0.00% -
400.perlbench (unexplained diff vs previous) -92.44% 357.000 27.000 0.000 42.11% 0.000
433.milc -92.27% 220.000 17.000 - 0.00% -
433.milc (avg) -92.27% 220.000 17.000 - 0.00% -
450.soplex -92.15% 191.000 15.000 - -6.25% -
450.soplex (avg) -92.15% 191.000 15.000 - -6.25% -
436.cactusADM -90.38% 291.000 28.000 - 0.00% -
436.cactusADM (avg) -90.38% 291.000 28.000 - 0.00% -
453.povray -90.12% 81.000 8.000 - 0.00% -
453.povray (avg) -90.12% 81.000 8.000 - 0.00% -
447.dealII (max) -88.79% 107.000 12.000 - -7.69% -
471.omnetpp (max) -88.67% 662.000 75.000 - -16.67% -
482.sphinx3 (unexplained diff vs previous) -86.56% 186.000 25.000 0.000 525.00% 0.000
470.lbm -86.47% 303.000 41.000 - -2.38% -
470.lbm (avg) -86.47% 303.000 41.000 - -2.38% -
434.zeusmp (unexplained diff vs previous) -84.42% 308.000 48.000 0.000 0.00% 0.000
434.zeusmp (max) -84.39% 301.000 47.000 - 51.61% -
456.hmmer (unexplained diff vs previous) -84.34% 83.000 13.000 0.000 116.67% 0.000
456.hmmer (max) -84.29% 140.000 22.000 - -12.00% -
434.zeusmp -82.05% 195.000 35.000 - -2.78% -
434.zeusmp (avg) -82.05% 195.000 35.000 - -2.78% -
400.perlbench -80.77% 156.000 30.000 - -3.23% -
400.perlbench (avg) -80.77% 156.000 30.000 - -3.23% -
470.lbm (max) (percentage) -80.59% 5.100 0.990 - 421.05% -
447.dealII -75.00% 104.000 26.000 - 0.00% -
447.dealII (avg) -75.00% 104.000 26.000 - 0.00% -
454.calculix (max) (percentage) -72.13% 4.700 1.310 - -2.24% -
444.namd (unexplained diff vs previous) (percentage) -70.76% 7.730 2.260 0.000 93.16% 0.000
456.hmmer -70.67% 75.000 22.000 - -4.35% -
456.hmmer (avg) -70.67% 75.000 22.000 - -4.35% -
444.namd (max) (percentage) -69.22% 7.180 2.210 - 87.29% -
445.gobmk (unexplained diff vs previous) -66.13% 186.000 63.000 0.000 2000.00% 0.000
459.GemsFDTD -65.71% 210.000 72.000 - -1.37% -
459.GemsFDTD (avg) -65.71% 210.000 72.000 - -1.37% -
445.gobmk (max) -64.48% 183.000 65.000 - -4.41% -
416.gamess (avg) (percentage) -61.29% 2.170 0.840 - 0.00% -
471.omnetpp -59.41% 170.000 69.000 - -2.82% -
471.omnetpp (avg) -59.41% 170.000 69.000 - -2.82% -
444.namd (avg) (percentage) -58.78% 3.930 1.620 - 0.62% -
471.omnetpp (max) (percentage) -58.35% 6.170 2.570 - -14.90% -
401.bzip2 (unexplained diff vs previous) -56.82% 132.000 57.000 0.000 14.00% 0.000
410.bwaves -56.25% 160.000 70.000 - -1.41% -
410.bwaves (avg) -56.25% 160.000 70.000 - -1.41% -
401.bzip2 (max) -52.63% 133.000 63.000 - -7.35% -
470.lbm (avg) (percentage) -52.32% 2.370 1.130 - -0.88% -
435.gromacs -50.38% 266.000 132.000 - -2.22% -
435.gromacs (avg) -50.38% 266.000 132.000 - -2.22% -
400.perlbench (unexplained diff vs previous) (percentage) -46.57% 3.350 1.790 0.000 45.53% 0.000
400.perlbench (max) (percentage) -45.99% 3.240 1.750 - 1.74% -
465.tonto (max) (percentage) -45.81% 3.580 1.940 - 24.36% -
436.cactusADM (unexplained diff vs previous) (percentage) -45.54% 3.250 1.770 0.000 55.26% 0.000
481.wrf (max) (percentage) -45.40% 3.150 1.720 - -1.15% -
453.povray (max) (percentage) -44.66% 2.530 1.400 - 48.94% -
436.cactusADM (max) (percentage) -42.86% 3.150 1.800 - 59.29% -
447.dealII (max) (percentage) -41.82% 1.100 0.640 - -7.25% -
445.gobmk -40.22% 92.000 55.000 - -1.79% -
445.gobmk (avg) -40.22% 92.000 55.000 - -1.79% -
456.hmmer (unexplained diff vs previous) (percentage) -40.00% 0.750 0.450 0.000 114.29% 0.000
456.hmmer (max) (percentage) -38.58% 1.270 0.780 - -11.36% -
435.gromacs (max) -37.94% 311.000 193.000 - -4.93% -
435.gromacs (unexplained diff vs previous) -37.42% 318.000 199.000 0.000 128.74% 0.000
464.h264ref (max) -35.48% 62.000 40.000 - 29.03% -
464.h264ref -34.62% 78.000 51.000 - -1.92% -
464.h264ref (avg) -34.62% 78.000 51.000 - -1.92% -
465.tonto (max) -26.96% 293.000 214.000 - 22.29% -
433.milc (unexplained diff vs previous) (percentage) -26.52% 3.280 2.410 0.000 363.46% 0.000
433.milc (max) (percentage) -25.87% 3.170 2.350 - 179.76% -
465.tonto (avg) (percentage) -22.16% 1.670 1.300 - 0.78% -
445.gobmk (unexplained diff vs previous) (percentage) -20.65% 1.550 1.230 0.000 1950.00% 0.000
436.cactusADM (avg) (percentage) -19.28% 1.660 1.340 - 0.75% -
434.zeusmp (unexplained diff vs previous) (percentage) -16.74% 2.330 1.940 0.000 0.00% 0.000
445.gobmk (max) (percentage) -16.34% 1.530 1.280 - -3.76% -
434.zeusmp (max) (percentage) -16.23% 2.280 1.910 - 54.03% -
481.wrf (avg) (percentage) -12.17% 1.890 1.660 - 0.00% -
459.GemsFDTD (max) -10.71% 140.000 125.000 - 26.26% -
433.milc (avg) (percentage) -9.41% 1.700 1.540 - 0.65% -
401.bzip2 -9.09% 66.000 60.000 - -3.23% -
401.bzip2 (avg) -9.09% 66.000 60.000 - -3.23% -
434.zeusmp (avg) (percentage) -2.03% 1.480 1.450 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 0.00% -
nb_compile_successful 65.52% 29.000 48.000 - 0.00% -
nb_execution_successful 65.52% 29.000 48.000 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.54s
Execution (seconds)
Name Current %
400.perlbench 1512.000 -1.75%
401.bzip2 2930.000 -1.91%
410.bwaves 6036.000 -1.28%
429.mcf 1882.000 -8.46%
433.milc 1122.000 -2.35%
434.zeusmp 2470.000 -1.91%
435.gromacs 6883.000 -2.81%
436.cactusADM 2145.000 1.80%
437.leslie3d 10821.000 -4.08%
444.namd 752.000 -2.21%
445.gobmk 5106.000 -1.22%
447.dealII 1923.000 -
450.soplex 540.000 -3.74%
453.povray 492.000 -1.40%
456.hmmer 2873.000 -
458.sjeng 7586.000 -1.20%
459.GemsFDTD 5008.000 -2.51%
462.libquantum 133.000 -
464.h264ref 5871.000 -
465.tonto 11040.000 -1.94%
470.lbm 3697.000 -
471.omnetpp 2923.000 -2.57%
473.astar 5560.000 -3.99%
482.sphinx3 908.000 -2.68%
Geometric Mean 2452.584 -2.03%
Code Size (bytes)
Name Current %
400.perlbench 917322.000 -
401.bzip2 49414.000 -
410.bwaves 17360.000 -
429.mcf 10911.000 -
433.milc 94253.000 -
434.zeusmp 159221.000 -
435.gromacs 667109.000 -
436.cactusADM 546946.000 -
437.leslie3d 60228.000 -
444.namd 188162.000 -
445.gobmk 1397629.000 -
447.dealII 2099712.000 -
450.soplex 257521.000 -
453.povray 696190.000 -
456.hmmer 229084.000 -
458.sjeng 110489.000 -
459.GemsFDTD 242157.000 -
462.libquantum 31313.000 -
464.h264ref 415120.000 -
465.tonto 2874773.000 -
470.lbm 10752.000 -
471.omnetpp 565455.000 -
473.astar 31888.000 -
482.sphinx3 151844.000 -
Geometric Mean 182607.913 -
sample variability
Name Current %
400.perlbench 30.000 -3.23%
400.perlbench (avg) 30.000 -3.23%
400.perlbench (avg) (percentage) 2.050 -
400.perlbench (max) 26.000 -
400.perlbench (max) (percentage) 1.750 1.74%
400.perlbench (unexplained diff vs previous) 27.000 42.11%
400.perlbench (unexplained diff vs previous) (percentage) 1.790 45.53%
401.bzip2 60.000 -3.23%
401.bzip2 (avg) 60.000 -3.23%
401.bzip2 (avg) (percentage) 2.080 -
401.bzip2 (max) 63.000 -7.35%
401.bzip2 (max) (percentage) 2.180 -4.80%
401.bzip2 (unexplained diff vs previous) 57.000 14.00%
401.bzip2 (unexplained diff vs previous) (percentage) 1.950 16.77%
403.gcc (avg) (percentage) 6.380 -
403.gcc (max) (percentage) 6.870 -2.83%
410.bwaves 70.000 -1.41%
410.bwaves (avg) 70.000 -1.41%
410.bwaves (avg) (percentage) 1.170 -
410.bwaves (max) 77.000 20.31%
410.bwaves (max) (percentage) 1.280 20.75%
410.bwaves (unexplained diff vs previous) 78.000 81.40%
410.bwaves (unexplained diff vs previous) (percentage) 1.290 84.29%
416.gamess (avg) (percentage) 0.840 -
416.gamess (max) (percentage) 0.130 -
429.mcf 107.000 -7.76%
429.mcf (avg) 107.000 -7.76%
429.mcf (avg) (percentage) 5.690 -
429.mcf (max) 159.000 29.27%
429.mcf (max) (percentage) 8.460 40.77%
429.mcf (unexplained diff vs previous) 174.000 1350.00%
429.mcf (unexplained diff vs previous) (percentage) 9.250 1494.83%
433.milc 17.000 -
433.milc (avg) 17.000 -
433.milc (avg) (percentage) 1.540 -
433.milc (max) 26.000 188.89%
433.milc (max) (percentage) 2.350 179.76%
433.milc (unexplained diff vs previous) 27.000 350.00%
433.milc (unexplained diff vs previous) (percentage) 2.410 363.46%
434.zeusmp 35.000 -2.78%
434.zeusmp (avg) 35.000 -2.78%
434.zeusmp (avg) (percentage) 1.450 -
434.zeusmp (max) 47.000 51.61%
434.zeusmp (max) (percentage) 1.910 54.03%
434.zeusmp (unexplained diff vs previous) 48.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 1.940 -
435.gromacs 132.000 -2.22%
435.gromacs (avg) 132.000 -2.22%
435.gromacs (avg) (percentage) 1.930 1.05%
435.gromacs (max) 193.000 -4.93%
435.gromacs (max) (percentage) 2.810 -2.09%
435.gromacs (unexplained diff vs previous) 199.000 128.74%
435.gromacs (unexplained diff vs previous) (percentage) 2.890 134.96%
436.cactusADM 28.000 -
436.cactusADM (avg) 28.000 -
436.cactusADM (avg) (percentage) 1.340 -
436.cactusADM (max) 38.000 65.22%
436.cactusADM (max) (percentage) 1.800 59.29%
436.cactusADM (unexplained diff vs previous) 38.000 58.33%
436.cactusADM (unexplained diff vs previous) (percentage) 1.770 55.26%
437.leslie3d 465.000 -4.12%
437.leslie3d (avg) 465.000 -4.12%
437.leslie3d (avg) (percentage) 4.300 -
437.leslie3d (max) 441.000 4.01%
437.leslie3d (max) (percentage) 4.080 8.51%
437.leslie3d (unexplained diff vs previous) 460.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 4.250 -
444.namd 12.000 -
444.namd (avg) 12.000 -
444.namd (avg) (percentage) 1.620 -
444.namd (max) 16.000 77.78%
444.namd (max) (percentage) 2.210 87.29%
444.namd (unexplained diff vs previous) 17.000 88.89%
444.namd (unexplained diff vs previous) (percentage) 2.260 93.16%
445.gobmk 55.000 -1.79%
445.gobmk (avg) 55.000 -1.79%
445.gobmk (avg) (percentage) 1.090 -
445.gobmk (max) 65.000 -4.41%
445.gobmk (max) (percentage) 1.280 -3.76%
445.gobmk (unexplained diff vs previous) 63.000 2000.00%
445.gobmk (unexplained diff vs previous) (percentage) 1.230 1950.00%
447.dealII 26.000 -
447.dealII (avg) 26.000 -
447.dealII (avg) (percentage) 1.400 -
447.dealII (max) 12.000 -7.69%
447.dealII (max) (percentage) 0.640 -7.25%
450.soplex 15.000 -6.25%
450.soplex (avg) 15.000 -6.25%
450.soplex (avg) (percentage) 2.870 -
450.soplex (max) 20.000 233.33%
450.soplex (max) (percentage) 3.740 216.95%
450.soplex (unexplained diff vs previous) 21.000 425.00%
450.soplex (unexplained diff vs previous) (percentage) 3.890 447.89%
453.povray 8.000 -
453.povray (avg) 8.000 -
453.povray (avg) (percentage) 1.800 -
453.povray (max) 6.000 50.00%
453.povray (max) (percentage) 1.400 48.94%
453.povray (unexplained diff vs previous) 7.000 75.00%
453.povray (unexplained diff vs previous) (percentage) 1.420 77.50%
454.calculix (avg) (percentage) 3.060 -
454.calculix (max) (percentage) 1.310 -2.24%
456.hmmer 22.000 -4.35%
456.hmmer (avg) 22.000 -4.35%
456.hmmer (avg) (percentage) 0.800 -1.23%
456.hmmer (max) 22.000 -12.00%
456.hmmer (max) (percentage) 0.780 -11.36%
456.hmmer (unexplained diff vs previous) 13.000 116.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.450 114.29%
458.sjeng 59.000 -
458.sjeng (avg) 59.000 -
458.sjeng (avg) (percentage) 0.780 1.30%
458.sjeng (max) 91.000 85.71%
458.sjeng (max) (percentage) 1.200 87.50%
458.sjeng (unexplained diff vs previous) 92.000 820.00%
458.sjeng (unexplained diff vs previous) (percentage) 1.210 830.77%
459.GemsFDTD 72.000 -1.37%
459.GemsFDTD (avg) 72.000 -1.37%
459.GemsFDTD (avg) (percentage) 1.450 1.40%
459.GemsFDTD (max) 125.000 26.26%
459.GemsFDTD (max) (percentage) 2.510 29.38%
459.GemsFDTD (unexplained diff vs previous) 129.000 230.77%
459.GemsFDTD (unexplained diff vs previous) (percentage) 2.580 239.47%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.670 -1.18%
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.180 -8.53%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 51.000 -1.92%
464.h264ref (avg) 51.000 -1.92%
464.h264ref (avg) (percentage) 0.870 -1.14%
464.h264ref (max) 40.000 29.03%
464.h264ref (max) (percentage) 0.690 27.78%
464.h264ref (unexplained diff vs previous) 41.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.700 -
465.tonto 143.000 -1.38%
465.tonto (avg) 143.000 -1.38%
465.tonto (avg) (percentage) 1.300 -
465.tonto (max) 214.000 22.29%
465.tonto (max) (percentage) 1.940 24.36%
465.tonto (unexplained diff vs previous) 218.000 263.33%
465.tonto (unexplained diff vs previous) (percentage) 1.970 271.70%
470.lbm 41.000 -2.38%
470.lbm (avg) 41.000 -2.38%
470.lbm (avg) (percentage) 1.130 -
470.lbm (max) 36.000 414.29%
470.lbm (max) (percentage) 0.990 421.05%
470.lbm (unexplained diff vs previous) 37.000 428.57%
470.lbm (unexplained diff vs previous) (percentage) 1.000 426.32%
471.omnetpp 69.000 -2.82%
471.omnetpp (avg) 69.000 -2.82%
471.omnetpp (avg) (percentage) 2.390 -
471.omnetpp (max) 75.000 -16.67%
471.omnetpp (max) (percentage) 2.570 -14.90%
471.omnetpp (unexplained diff vs previous) 77.000 -12.50%
471.omnetpp (unexplained diff vs previous) (percentage) 2.630 -10.24%
473.astar 95.000 -2.06%
473.astar (avg) 95.000 -2.06%
473.astar (avg) (percentage) 1.720 1.78%
473.astar (max) 221.000 274.58%
473.astar (max) (percentage) 3.990 287.38%
473.astar (unexplained diff vs previous) 231.000 1825.00%
473.astar (unexplained diff vs previous) (percentage) 4.150 1876.19%
481.wrf (avg) (percentage) 1.660 -
481.wrf (max) (percentage) 1.720 -1.15%
482.sphinx3 17.000 -
482.sphinx3 (avg) 17.000 -
482.sphinx3 (avg) (percentage) 1.900 1.06%
482.sphinx3 (max) 24.000 380.00%
482.sphinx3 (max) (percentage) 2.680 318.75%
482.sphinx3 (unexplained diff vs previous) 25.000 525.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 2.750 539.53%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.710 -
Geometric Mean 8.939 32.48%