Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=932e5949a9f366d149b1b9093c49225502dee890 (+ 212 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/061a82fa2b751b42d0d8ddfcd45367c848d3ee64 (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=71648e80042658f23965924616fb7db9c2fccff9 (+ 65 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=0be619bcb134b82abef6beaaee9db9582c7015a7 (+ 445 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/193/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0193 (3-changed-components) 2024-02-11T16:58:19 0:00:00
Previous 0186 (3-changed-components) 2024-01-24T09:43:41 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 60 75
Performance Improvements 98 154
Added Tests 2 2
Unchanged Tests 1076 1005
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 5.08% 59.000 62.000 - -99.31% -
437.leslie3d 4.68% 10593.000 11089.000 - -26.10% -
450.soplex 3.67% 572.000 593.000 - -96.17% -
429.mcf 2.49% 1485.000 1522.000 - -91.13% -
435.gromacs 1.72% 7166.000 7289.000 - -52.73% -
454.calculix 1.64% 61.000 62.000 - -99.73% -
434.zeusmp 1.10% 2545.000 2573.000 - -80.52% -
433.milc 1.03% 1167.000 1179.000 - -90.91% -
481.wrf 1.02% 5487.000 5543.000 - -67.07% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (unexplained diff vs previous) 600.00% 3.000 21.000 - -93.25% -
450.soplex (unexplained diff vs previous) (percentage) 580.77% 0.520 3.540 - 76.12% -
465.tonto (unexplained diff vs previous) 558.82% 17.000 112.000 - 77.78% -
465.tonto (unexplained diff vs previous) (percentage) 553.33% 0.150 0.980 - 27.27% -
437.leslie3d (unexplained diff vs previous) 350.91% 110.000 496.000 - 44.61% -
437.leslie3d (unexplained diff vs previous) (percentage) 329.81% 1.040 4.470 - 95.20% -
435.gromacs (unexplained diff vs previous) 324.14% 29.000 123.000 - -61.32% -
435.gromacs (unexplained diff vs previous) (percentage) 322.50% 0.400 1.690 - -17.96% -
433.milc (max) 200.00% 4.000 12.000 - -97.08% -
433.milc (max) (percentage) 164.10% 0.390 1.030 - -67.51% -
434.zeusmp (unexplained diff vs previous) 154.55% 11.000 28.000 - -90.91% -
434.zeusmp (unexplained diff vs previous) (percentage) 153.49% 0.430 1.090 - -53.22% -
436.cactusADM (unexplained diff vs previous) 140.00% 5.000 12.000 - -97.89% -
436.cactusADM (unexplained diff vs previous) (percentage) 137.50% 0.240 0.570 - -82.46% -
464.h264ref (unexplained diff vs previous) 133.33% 9.000 21.000 - 40.00% -
464.h264ref (unexplained diff vs previous) (percentage) 133.33% 0.150 0.350 - 250.00% -
437.leslie3d (max) 105.56% 252.000 518.000 - 47.58% -
470.lbm (unexplained diff vs previous) 100.00% 3.000 6.000 - -57.14% -
470.lbm (unexplained diff vs previous) (percentage) 100.00% 0.080 0.160 - 45.45% -
437.leslie3d (max) (percentage) 96.64% 2.380 4.680 - 100.00% -
450.soplex (max) 75.00% 12.000 21.000 - -93.09% -
450.soplex (max) (percentage) 69.91% 2.160 3.670 - 86.29% -
434.zeusmp (max) 64.71% 17.000 28.000 - -90.70% -
434.zeusmp (max) (percentage) 61.76% 0.680 1.100 - -51.75% -
400.perlbench (unexplained diff vs previous) 60.00% 5.000 8.000 - -97.76% -
400.perlbench (unexplained diff vs previous) (percentage) 59.38% 0.320 0.510 - -84.78% -
471.omnetpp (unexplained diff vs previous) (percentage) 44.41% 3.490 5.040 - 2000.00% -
445.gobmk (unexplained diff vs previous) 41.67% 24.000 34.000 - -81.72% -
445.gobmk (unexplained diff vs previous) (percentage) 41.30% 0.460 0.650 - -58.06% -
471.omnetpp (unexplained diff vs previous) 37.74% 106.000 146.000 - 461.54% -
436.cactusADM (max) 33.33% 9.000 12.000 - -97.83% -
471.omnetpp (max) (percentage) 32.96% 3.610 4.800 - -22.20% -
436.cactusADM (max) (percentage) 29.55% 0.440 0.570 - -81.90% -
473.astar (unexplained diff vs previous) (percentage) 28.12% 0.320 0.410 - 1950.00% -
458.sjeng (unexplained diff vs previous) (percentage) 27.50% 0.400 0.510 - 750.00% -
471.omnetpp (max) 26.61% 109.000 138.000 - -79.15% -
473.astar (unexplained diff vs previous) 26.32% 19.000 24.000 - 700.00% -
458.sjeng (unexplained diff vs previous) 25.81% 31.000 39.000 - 333.33% -
435.gromacs (max) 15.74% 108.000 125.000 - -59.81% -
435.gromacs (max) (percentage) 13.91% 1.510 1.720 - -14.85% -
429.mcf (unexplained diff vs previous) 12.12% 33.000 37.000 - -60.22% -
429.mcf (unexplained diff vs previous) (percentage) 9.46% 2.220 2.430 - 350.00% -
450.soplex 6.25% 16.000 17.000 - -91.10% -
450.soplex (avg) 6.25% 16.000 17.000 - -91.10% -
437.leslie3d 5.00% 460.000 483.000 - 35.29% -
437.leslie3d (avg) 5.00% 460.000 483.000 - 35.29% -
471.omnetpp (avg) (percentage) 1.65% 2.420 2.460 - 54.72% -
435.gromacs 1.42% 141.000 143.000 - -46.24% -
435.gromacs (avg) 1.42% 141.000 143.000 - -46.24% -
429.mcf 1.15% 87.000 88.000 - -2.22% -
429.mcf (avg) 1.15% 87.000 88.000 - -2.22% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp -4.80% 3040.000 2894.000 - -73.05% -
462.libquantum -1.45% 138.000 136.000 - -98.83% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (max) (percentage) -81.68% 8.190 1.500 - 70.45% -
401.bzip2 (max) -81.63% 245.000 45.000 - -66.17% -
444.namd (unexplained diff vs previous) (percentage) -75.30% 1.660 0.410 - -94.70% -
444.namd (unexplained diff vs previous) -75.00% 12.000 3.000 - -99.57% -
481.wrf (unexplained diff vs previous) (percentage) -56.09% 2.300 1.010 - 50.75% -
481.wrf (unexplained diff vs previous) -55.56% 126.000 56.000 - -50.00% -
482.sphinx3 (unexplained diff vs previous) -50.00% 2.000 1.000 - -99.46% -
401.bzip2 (unexplained diff vs previous) (percentage) -49.18% 1.830 0.930 - 6.90% -
401.bzip2 (unexplained diff vs previous) -49.09% 55.000 28.000 - -78.79% -
482.sphinx3 (unexplained diff vs previous) (percentage) -47.62% 0.210 0.110 - -82.26% -
453.povray (unexplained diff vs previous) (percentage) -40.59% 1.010 0.600 - 2900.00% -
453.povray (unexplained diff vs previous) -40.00% 5.000 3.000 - 200.00% -
447.dealII (unexplained diff vs previous) (percentage) -39.56% 0.910 0.550 - -29.49% -
447.dealII (unexplained diff vs previous) -38.89% 18.000 11.000 - -85.53% -
410.bwaves (unexplained diff vs previous) (percentage) -33.33% 0.450 0.300 - 200.00% -
410.bwaves (unexplained diff vs previous) -33.33% 27.000 18.000 - 20.00% -
462.libquantum (max) -33.33% 3.000 2.000 - -97.30% -
462.libquantum (unexplained diff vs previous) -33.33% 3.000 2.000 - 100.00% -
462.libquantum (unexplained diff vs previous) (percentage) -32.26% 2.170 1.470 - 14600.00% -
447.dealII (max) -31.58% 19.000 13.000 - -87.85% -
447.dealII (max) (percentage) -31.31% 0.990 0.680 - -38.18% -
462.libquantum (max) (percentage) -29.12% 2.610 1.850 - 189.06% -
444.namd (max) -27.27% 11.000 8.000 - -98.77% -
459.GemsFDTD (max) -27.27% 22.000 16.000 - -88.57% -
444.namd (max) (percentage) -25.15% 1.630 1.220 - -83.01% -
453.povray (max) -25.00% 4.000 3.000 - -98.18% -
453.povray (max) (percentage) -25.00% 1.000 0.750 - -70.36% -
456.hmmer (unexplained diff vs previous) -25.00% 12.000 9.000 - -89.16% -
459.GemsFDTD (max) (percentage) -25.00% 0.360 0.270 - -55.00% -
456.hmmer (unexplained diff vs previous) (percentage) -24.39% 0.410 0.310 - -58.67% -
416.gamess (unexplained diff vs previous) -10.34% 29.000 26.000 - -96.15% -
470.lbm (max) (percentage) -10.00% 0.300 0.270 - -94.71% -
470.lbm (max) -9.09% 11.000 10.000 - -98.47% -
416.gamess (unexplained diff vs previous) (percentage) -8.70% 0.230 0.210 - -92.95% -
400.perlbench (max) -8.33% 24.000 22.000 - -93.62% -
400.perlbench (max) (percentage) -7.59% 1.580 1.460 - -54.94% -
456.hmmer (max) -6.67% 15.000 14.000 - -90.00% -
482.sphinx3 -5.26% 19.000 18.000 - -95.87% -
482.sphinx3 (avg) -5.26% 19.000 18.000 - -95.87% -
464.h264ref (max) (percentage) -4.88% 0.820 0.780 - 81.40% -
458.sjeng (max) -4.76% 84.000 80.000 - 158.06% -
403.gcc (unexplained diff vs previous) (percentage) -4.72% 5.080 4.840 - 572.22% -
416.gamess (max) (percentage) -4.35% 0.230 0.220 - -92.39% -
403.gcc (max) (percentage) -4.30% 12.090 11.570 - 1506.94% -
464.h264ref (max) -4.17% 48.000 46.000 - -25.81% -
456.hmmer (max) (percentage) -3.85% 0.520 0.500 - -60.63% -
410.bwaves (max) (percentage) -3.70% 0.540 0.520 - 147.62% -
458.sjeng (max) (percentage) -3.67% 1.090 1.050 - 377.27% -
416.gamess (max) -3.57% 28.000 27.000 - -95.88% -
429.mcf (max) (percentage) -3.43% 20.100 19.410 - 3494.44% -
445.gobmk (max) (percentage) -3.39% 1.180 1.140 - -25.49% -
445.gobmk (max) -3.28% 61.000 59.000 - -67.76% -
481.wrf (max) (percentage) -3.13% 2.240 2.170 - -31.11% -
410.bwaves (max) -3.12% 32.000 31.000 - 3.33% -
473.astar (max) -3.06% 229.000 222.000 - 1010.00% -
400.perlbench -2.94% 34.000 33.000 - -78.85% -
400.perlbench (avg) -2.94% 34.000 33.000 - -78.85% -
482.sphinx3 (max) (percentage) -2.91% 1.030 1.000 - -31.97% -
473.astar (max) (percentage) -2.84% 3.880 3.770 - 2592.86% -
471.omnetpp -2.74% 73.000 71.000 - -58.24% -
471.omnetpp (avg) -2.74% 73.000 71.000 - -58.24% -
465.tonto (max) (percentage) -2.52% 1.190 1.160 - -67.60% -
459.GemsFDTD -2.13% 94.000 92.000 - -56.19% -
459.GemsFDTD (avg) -2.13% 94.000 92.000 - -56.19% -
470.lbm -2.13% 47.000 46.000 - -84.82% -
470.lbm (avg) -2.13% 47.000 46.000 - -84.82% -
416.gamess (avg) (percentage) -2.13% 0.940 0.920 - -57.60% -
459.GemsFDTD (avg) (percentage) -1.95% 1.540 1.510 - 67.78% -
473.astar -1.92% 104.000 102.000 - 264.29% -
473.astar (avg) -1.92% 104.000 102.000 - 264.29% -
400.perlbench (avg) (percentage) -1.83% 2.180 2.140 - 45.58% -
454.calculix (unexplained diff vs previous) (percentage) -1.83% 1.640 1.610 - 7950.00% -
464.h264ref -1.72% 58.000 57.000 - -26.92% -
464.h264ref (avg) -1.72% 58.000 57.000 - -26.92% -
416.gamess -1.71% 117.000 115.000 - -76.63% -
416.gamess (avg) -1.71% 117.000 115.000 - -76.63% -
481.wrf (max) -1.64% 122.000 120.000 - -77.36% -
444.namd (avg) (percentage) -1.63% 1.840 1.810 - -53.94% -
470.lbm (avg) (percentage) -1.57% 1.270 1.250 - -47.26% -
458.sjeng -1.56% 64.000 63.000 - 34.04% -
458.sjeng (avg) -1.56% 64.000 63.000 - 34.04% -
454.calculix (avg) (percentage) -1.53% 3.260 3.210 - 59.70% -
453.povray (avg) (percentage) -1.53% 1.960 1.930 - 54.40% -
465.tonto (max) -1.49% 134.000 132.000 - -54.95% -
482.sphinx3 (avg) (percentage) -1.49% 2.020 1.990 - 36.30% -
436.cactusADM (avg) (percentage) -1.36% 1.470 1.450 - -12.65% -
410.bwaves -1.35% 74.000 73.000 - -54.37% -
410.bwaves (avg) -1.35% 74.000 73.000 - -54.37% -
447.dealII (avg) (percentage) -1.34% 1.490 1.470 - 37.38% -
458.sjeng (avg) (percentage) -1.20% 0.830 0.820 - 148.48% -
473.astar (avg) (percentage) -1.14% 1.760 1.740 - 815.79% -
481.wrf (avg) (percentage) -1.14% 1.760 1.740 - -7.94% -
456.hmmer (avg) (percentage) -1.09% 0.920 0.910 - 33.82% -
464.h264ref (avg) (percentage) -1.02% 0.980 0.970 - 79.63% -
429.mcf (avg) (percentage) -1.02% 5.880 5.820 - 998.11% -
429.mcf (max) -1.01% 298.000 295.000 - 220.65% -

Added Tests - sample variability
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
456.hmmer 1.42% 231209.000 234488.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 39.27% 8205.000 11427.000 - 0.99% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 14600.00% 0.010 1.470 - -32.26% -
454.calculix (unexplained diff vs previous) (percentage) 7950.00% 0.020 1.610 - -1.83% -
429.mcf (max) (percentage) 3494.44% 0.540 19.410 - -3.43% -
453.povray (unexplained diff vs previous) (percentage) 2900.00% 0.020 0.600 - -40.59% -
473.astar (max) (percentage) 2592.86% 0.140 3.770 - -2.84% -
471.omnetpp (unexplained diff vs previous) (percentage) 2000.00% 0.240 5.040 - 44.41% -
473.astar (unexplained diff vs previous) (percentage) 1950.00% 0.020 0.410 - 28.12% -
403.gcc (max) (percentage) 1506.94% 0.720 11.570 - -4.30% -
483.xalancbmk (max) (percentage) 1304.88% 0.410 5.760 - -0.17% -
473.astar (max) 1010.00% 20.000 222.000 - -3.06% -
429.mcf (avg) (percentage) 998.11% 0.530 5.820 - -1.02% -
403.gcc (avg) (percentage) 913.11% 0.610 6.180 - -0.80% -
473.astar (avg) (percentage) 815.79% 0.190 1.740 - -1.14% -
458.sjeng (unexplained diff vs previous) (percentage) 750.00% 0.060 0.510 - 27.50% -
473.astar (unexplained diff vs previous) 700.00% 3.000 24.000 - 26.32% -
403.gcc (unexplained diff vs previous) (percentage) 572.22% 0.720 4.840 - -4.72% -
471.omnetpp (unexplained diff vs previous) 461.54% 26.000 146.000 - 37.74% -
401.bzip2 (avg) (percentage) 400.00% 0.440 2.200 - -0.90% -
458.sjeng (max) (percentage) 377.27% 0.220 1.050 - -3.67% -
429.mcf (unexplained diff vs previous) (percentage) 350.00% 0.540 2.430 - 9.46% -
458.sjeng (unexplained diff vs previous) 333.33% 9.000 39.000 - 25.81% -
473.astar 264.29% 28.000 102.000 - -1.92% -
473.astar (avg) 264.29% 28.000 102.000 - -1.92% -
464.h264ref (unexplained diff vs previous) (percentage) 250.00% 0.100 0.350 - 133.33% -
429.mcf (max) 220.65% 92.000 295.000 - -1.01% -
453.povray (unexplained diff vs previous) 200.00% 1.000 3.000 - -40.00% -
410.bwaves (unexplained diff vs previous) (percentage) 200.00% 0.100 0.300 - -33.33% -
462.libquantum (max) (percentage) 189.06% 0.640 1.850 - -29.12% -
462.libquantum (avg) (percentage) 181.54% 0.650 1.830 - -0.54% -
458.sjeng (max) 158.06% 31.000 80.000 - -4.76% -
458.sjeng (avg) (percentage) 148.48% 0.330 0.820 - -1.20% -
410.bwaves (max) (percentage) 147.62% 0.210 0.520 - -3.70% -
450.soplex (avg) (percentage) 141.13% 1.240 2.990 - 0.67% -
437.leslie3d (max) (percentage) 100.00% 2.340 4.680 - 96.64% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - -33.33% -
437.leslie3d (unexplained diff vs previous) (percentage) 95.20% 2.290 4.470 - 329.81% -
450.soplex (max) (percentage) 86.29% 1.970 3.670 - 69.91% -
437.leslie3d (avg) (percentage) 83.19% 2.380 4.360 - 0.23% -
464.h264ref (max) (percentage) 81.40% 0.430 0.780 - -4.88% -
464.h264ref (avg) (percentage) 79.63% 0.540 0.970 - -1.02% -
465.tonto (unexplained diff vs previous) 77.78% 63.000 112.000 - 558.82% -
450.soplex (unexplained diff vs previous) (percentage) 76.12% 2.010 3.540 - 580.77% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
401.bzip2 (max) (percentage) 70.45% 0.880 1.500 - -81.68% -
459.GemsFDTD (avg) (percentage) 67.78% 0.900 1.510 - -1.95% -
454.calculix (avg) (percentage) 59.70% 2.010 3.210 - -1.53% -
471.omnetpp (avg) (percentage) 54.72% 1.590 2.460 - 1.65% -
453.povray (avg) (percentage) 54.40% 1.250 1.930 - -1.53% -
445.gobmk (avg) (percentage) 53.25% 0.770 1.180 - -0.84% -
481.wrf (unexplained diff vs previous) (percentage) 50.75% 0.670 1.010 - -56.09% -
437.leslie3d (max) 47.58% 351.000 518.000 - 105.56% -
400.perlbench (avg) (percentage) 45.58% 1.470 2.140 - -1.83% -
470.lbm (unexplained diff vs previous) (percentage) 45.45% 0.110 0.160 - 100.00% -
437.leslie3d (unexplained diff vs previous) 44.61% 343.000 496.000 - 350.91% -
464.h264ref (unexplained diff vs previous) 40.00% 15.000 21.000 - 133.33% -
447.dealII (avg) (percentage) 37.38% 1.070 1.470 - -1.34% -
482.sphinx3 (avg) (percentage) 36.30% 1.460 1.990 - -1.49% -
437.leslie3d 35.29% 357.000 483.000 - 5.00% -
437.leslie3d (avg) 35.29% 357.000 483.000 - 5.00% -
458.sjeng 34.04% 47.000 63.000 - -1.56% -
458.sjeng (avg) 34.04% 47.000 63.000 - -1.56% -
456.hmmer (avg) (percentage) 33.82% 0.680 0.910 - -1.09% -
465.tonto (unexplained diff vs previous) (percentage) 27.27% 0.770 0.980 - 553.33% -
410.bwaves (unexplained diff vs previous) 20.00% 15.000 18.000 - -33.33% -
435.gromacs (avg) (percentage) 13.87% 1.730 1.970 - 0.00% -
410.bwaves (avg) (percentage) 11.93% 1.090 1.220 - -0.81% -
434.zeusmp (avg) (percentage) 9.46% 1.480 1.620 - -0.61% -
465.tonto 7.30% 137.000 147.000 - 0.00% -
465.tonto (avg) 7.30% 137.000 147.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 6.90% 0.870 0.930 - -49.18% -
410.bwaves (max) 3.33% 30.000 31.000 - -3.12% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
464.h264ref -2.43% 428364.000 417940.000 - 0.00% -
434.zeusmp -1.73% 165699.000 162829.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.73% 22860.000 62.000 - 1.64% -
403.gcc -99.31% 9010.000 62.000 - 5.08% -
462.libquantum -98.83% 11614.000 136.000 - -1.45% -
482.sphinx3 -96.86% 29928.000 940.000 - -0.11% -
450.soplex -96.17% 15466.000 593.000 - 3.67% -
453.povray -92.37% 6544.000 499.000 - 0.60% -
444.namd -91.96% 9050.000 728.000 - 0.41% -
429.mcf -91.13% 17158.000 1522.000 - 2.49% -
433.milc -90.91% 12976.000 1179.000 - 1.03% -
436.cactusADM -87.97% 17544.000 2111.000 - 0.57% -
400.perlbench -85.25% 10650.000 1571.000 - -0.51% -
434.zeusmp -80.52% 13211.000 2573.000 - 1.10% -
401.bzip2 -80.07% 15185.000 3026.000 - 0.93% -
447.dealII -79.67% 9751.000 1982.000 - 0.56% -
459.GemsFDTD -73.70% 23356.000 6142.000 - 0.00% -
456.hmmer -73.48% 11046.000 2929.000 - 0.31% -
471.omnetpp -73.05% 10739.000 2894.000 - -4.80% -
470.lbm -70.82% 12819.000 3740.000 - -0.16% -
481.wrf -67.07% 16834.000 5543.000 - 1.02% -
473.astar -60.51% 14932.000 5897.000 - -0.41% -
464.h264ref -59.36% 14556.000 5916.000 - -0.35% -
410.bwaves -59.30% 14755.000 6006.000 - -0.30% -
445.gobmk -56.15% 11995.000 5260.000 - 0.65% -
435.gromacs -52.73% 15419.000 7289.000 - 1.72% -
458.sjeng -46.14% 14275.000 7689.000 - -0.50% -
416.gamess -44.67% 22704.000 12562.000 - 0.21% -
437.leslie3d -26.10% 15005.000 11089.000 - 4.68% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.91% 1074.000 1.000 - 0.00% -
454.calculix -99.78% 459.000 1.000 - 0.00% -
454.calculix (avg) -99.78% 459.000 1.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.57% 700.000 3.000 - -75.00% -
482.sphinx3 (unexplained diff vs previous) -99.46% 186.000 1.000 - -50.00% -
444.namd (max) -98.77% 649.000 8.000 - -27.27% -
470.lbm (max) -98.47% 653.000 10.000 - -9.09% -
453.povray (max) -98.18% 165.000 3.000 - -25.00% -
482.sphinx3 (max) -97.95% 439.000 9.000 - 0.00% -
436.cactusADM (unexplained diff vs previous) -97.89% 570.000 12.000 - 140.00% -
436.cactusADM (max) -97.83% 552.000 12.000 - 33.33% -
400.perlbench (unexplained diff vs previous) -97.76% 357.000 8.000 - 60.00% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
462.libquantum (max) -97.30% 74.000 2.000 - -33.33% -
433.milc (unexplained diff vs previous) -97.18% 425.000 12.000 - 0.00% -
433.milc (max) -97.08% 411.000 12.000 - 200.00% -
444.namd -96.34% 355.000 13.000 - 0.00% -
444.namd (avg) -96.34% 355.000 13.000 - 0.00% -
416.gamess (unexplained diff vs previous) -96.15% 676.000 26.000 - -10.34% -
416.gamess (max) -95.88% 656.000 27.000 - -3.57% -
482.sphinx3 -95.87% 436.000 18.000 - -5.26% -
482.sphinx3 (avg) -95.87% 436.000 18.000 - -5.26% -
403.gcc (unexplained diff vs previous) -95.38% 65.000 3.000 - 0.00% -
470.lbm (max) (percentage) -94.71% 5.100 0.270 - -10.00% -
444.namd (unexplained diff vs previous) (percentage) -94.70% 7.730 0.410 - -75.30% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
400.perlbench (max) -93.62% 345.000 22.000 - -8.33% -
450.soplex (unexplained diff vs previous) -93.25% 311.000 21.000 - 600.00% -
450.soplex (max) -93.09% 304.000 21.000 - 75.00% -
416.gamess (unexplained diff vs previous) (percentage) -92.95% 2.980 0.210 - -8.70% -
416.gamess (max) (percentage) -92.39% 2.890 0.220 - -4.35% -
433.milc -91.36% 220.000 19.000 - 0.00% -
433.milc (avg) -91.36% 220.000 19.000 - 0.00% -
450.soplex -91.10% 191.000 17.000 - 6.25% -
450.soplex (avg) -91.10% 191.000 17.000 - 6.25% -
434.zeusmp (unexplained diff vs previous) -90.91% 308.000 28.000 - 154.55% -
434.zeusmp (max) -90.70% 301.000 28.000 - 64.71% -
456.hmmer (max) -90.00% 140.000 14.000 - -6.67% -
436.cactusADM -89.69% 291.000 30.000 - 0.00% -
436.cactusADM (avg) -89.69% 291.000 30.000 - 0.00% -
456.hmmer (unexplained diff vs previous) -89.16% 83.000 9.000 - -25.00% -
403.gcc (max) -89.06% 64.000 7.000 - 0.00% -
453.povray -88.89% 81.000 9.000 - 0.00% -
453.povray (avg) -88.89% 81.000 9.000 - 0.00% -
459.GemsFDTD (max) -88.57% 140.000 16.000 - -27.27% -
447.dealII (max) -87.85% 107.000 13.000 - -31.58% -
447.dealII (unexplained diff vs previous) -85.53% 76.000 11.000 - -38.89% -
470.lbm -84.82% 303.000 46.000 - -2.13% -
470.lbm (avg) -84.82% 303.000 46.000 - -2.13% -
400.perlbench (unexplained diff vs previous) (percentage) -84.78% 3.350 0.510 - 59.38% -
444.namd (max) (percentage) -83.01% 7.180 1.220 - -25.15% -
436.cactusADM (unexplained diff vs previous) (percentage) -82.46% 3.250 0.570 - 137.50% -
482.sphinx3 (unexplained diff vs previous) (percentage) -82.26% 0.620 0.110 - -47.62% -
436.cactusADM (max) (percentage) -81.90% 3.150 0.570 - 29.55% -
445.gobmk (unexplained diff vs previous) -81.72% 186.000 34.000 - 41.67% -
454.calculix (unexplained diff vs previous) -80.00% 5.000 1.000 - 0.00% -
471.omnetpp (max) -79.15% 662.000 138.000 - 26.61% -
434.zeusmp -78.97% 195.000 41.000 - 0.00% -
434.zeusmp (avg) -78.97% 195.000 41.000 - 0.00% -
400.perlbench -78.85% 156.000 33.000 - -2.94% -
400.perlbench (avg) -78.85% 156.000 33.000 - -2.94% -
401.bzip2 (unexplained diff vs previous) -78.79% 132.000 28.000 - -49.09% -
481.wrf (max) -77.36% 530.000 120.000 - -1.64% -
416.gamess -76.63% 492.000 115.000 - -1.71% -
416.gamess (avg) -76.63% 492.000 115.000 - -1.71% -
447.dealII -72.12% 104.000 29.000 - 0.00% -
447.dealII (avg) -72.12% 104.000 29.000 - 0.00% -
453.povray (max) (percentage) -70.36% 2.530 0.750 - -25.00% -
481.wrf -69.81% 318.000 96.000 - 0.00% -
481.wrf (avg) -69.81% 318.000 96.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -68.90% 3.280 1.020 - 0.00% -
445.gobmk (max) -67.76% 183.000 59.000 - -3.28% -
465.tonto (max) (percentage) -67.60% 3.580 1.160 - -2.52% -
433.milc (max) (percentage) -67.51% 3.170 1.030 - 164.10% -
401.bzip2 (max) -66.17% 133.000 45.000 - -81.63% -
456.hmmer -65.33% 75.000 26.000 - 0.00% -
456.hmmer (avg) -65.33% 75.000 26.000 - 0.00% -
435.gromacs (unexplained diff vs previous) -61.32% 318.000 123.000 - 324.14% -
456.hmmer (max) (percentage) -60.63% 1.270 0.500 - -3.85% -
429.mcf (unexplained diff vs previous) -60.22% 93.000 37.000 - 12.12% -
435.gromacs (max) -59.81% 311.000 125.000 - 15.74% -
456.hmmer (unexplained diff vs previous) (percentage) -58.67% 0.750 0.310 - -24.39% -
471.omnetpp -58.24% 170.000 71.000 - -2.74% -
471.omnetpp (avg) -58.24% 170.000 71.000 - -2.74% -
445.gobmk (unexplained diff vs previous) (percentage) -58.06% 1.550 0.650 - 41.30% -
416.gamess (avg) (percentage) -57.60% 2.170 0.920 - -2.13% -
470.lbm (unexplained diff vs previous) -57.14% 14.000 6.000 - 100.00% -
459.GemsFDTD -56.19% 210.000 92.000 - -2.13% -
459.GemsFDTD (avg) -56.19% 210.000 92.000 - -2.13% -
459.GemsFDTD (max) (percentage) -55.00% 0.600 0.270 - -25.00% -
465.tonto (max) -54.95% 293.000 132.000 - -1.49% -
400.perlbench (max) (percentage) -54.94% 3.240 1.460 - -7.59% -
410.bwaves -54.37% 160.000 73.000 - -1.35% -
410.bwaves (avg) -54.37% 160.000 73.000 - -1.35% -
444.namd (avg) (percentage) -53.94% 3.930 1.810 - -1.63% -
434.zeusmp (unexplained diff vs previous) (percentage) -53.22% 2.330 1.090 - 153.49% -
434.zeusmp (max) (percentage) -51.75% 2.280 1.100 - 61.76% -
481.wrf (unexplained diff vs previous) -50.00% 112.000 56.000 - -55.56% -
470.lbm (avg) (percentage) -47.26% 2.370 1.250 - -1.57% -
435.gromacs -46.24% 266.000 143.000 - 1.42% -
435.gromacs (avg) -46.24% 266.000 143.000 - 1.42% -
447.dealII (max) (percentage) -38.18% 1.100 0.680 - -31.31% -
454.calculix (max) (percentage) -33.62% 4.700 3.120 - 0.00% -
445.gobmk -32.61% 92.000 62.000 - 0.00% -
445.gobmk (avg) -32.61% 92.000 62.000 - 0.00% -
482.sphinx3 (max) (percentage) -31.97% 1.470 1.000 - -2.91% -
481.wrf (max) (percentage) -31.11% 3.150 2.170 - -3.13% -
447.dealII (unexplained diff vs previous) (percentage) -29.49% 0.780 0.550 - -39.56% -
464.h264ref -26.92% 78.000 57.000 - -1.72% -
464.h264ref (avg) -26.92% 78.000 57.000 - -1.72% -
464.h264ref (max) -25.81% 62.000 46.000 - -4.17% -
445.gobmk (max) (percentage) -25.49% 1.530 1.140 - -3.39% -
465.tonto (avg) (percentage) -22.75% 1.670 1.290 - -0.77% -
471.omnetpp (max) (percentage) -22.20% 6.170 4.800 - 32.96% -
435.gromacs (unexplained diff vs previous) (percentage) -17.96% 2.060 1.690 - 322.50% -
435.gromacs (max) (percentage) -14.85% 2.020 1.720 - 13.91% -
436.cactusADM (avg) (percentage) -12.65% 1.660 1.450 - -1.36% -
481.wrf (avg) (percentage) -7.94% 1.890 1.740 - -1.14% -
433.milc (avg) (percentage) -2.94% 1.700 1.650 - -0.60% -
429.mcf -2.22% 90.000 88.000 - 1.15% -
429.mcf (avg) -2.22% 90.000 88.000 - 1.15% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 3.45% 29.000 30.000 - 0.00% -

Added Tests - sample variability
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1571.000 -
401.bzip2 3026.000 -
403.gcc 62.000 5.08%
410.bwaves 6006.000 -
416.gamess 12562.000 -
429.mcf 1522.000 2.49%
433.milc 1179.000 1.03%
434.zeusmp 2573.000 1.10%
435.gromacs 7289.000 1.72%
436.cactusADM 2111.000 -
437.leslie3d 11089.000 4.68%
444.namd 728.000 -
445.gobmk 5260.000 -
447.dealII 1982.000 -
450.soplex 593.000 3.67%
453.povray 499.000 -
454.calculix 62.000 1.64%
456.hmmer 2929.000 -
458.sjeng 7689.000 -
459.GemsFDTD 6142.000 -
462.libquantum 136.000 -1.45%
464.h264ref 5916.000 -
465.tonto 11427.000 -
470.lbm 3740.000 -
471.omnetpp 2894.000 -4.80%
473.astar 5897.000 -
481.wrf 5543.000 1.02%
482.sphinx3 940.000 -
Geometric Mean 2098.515 -
Code Size (bytes)
Name Current %
400.perlbench 929250.000 -
401.bzip2 49690.000 -
403.gcc 2683621.000 -
410.bwaves 17640.000 -
416.gamess 6251075.000 -
429.mcf 10979.000 -
433.milc 95617.000 -
434.zeusmp 162829.000 -
435.gromacs 677869.000 -
436.cactusADM 555604.000 -
437.leslie3d 64776.000 -
444.namd 190766.000 -
445.gobmk 1399885.000 -
447.dealII 2118491.000 -
450.soplex 259941.000 -
453.povray 706478.000 -
454.calculix 1270147.000 -
456.hmmer 234488.000 -
458.sjeng 120025.000 -
459.GemsFDTD 254373.000 -
462.libquantum 31353.000 -
464.h264ref 417940.000 -
465.tonto 2873397.000 -
470.lbm 10916.000 -
471.omnetpp 566460.000 -
473.astar 32228.000 -
481.wrf 2623960.000 -
482.sphinx3 152612.000 -
Geometric Mean 272916.684 -
sample variability
Name Current %
400.perlbench 33.000 -2.94%
400.perlbench (avg) 33.000 -2.94%
400.perlbench (avg) (percentage) 2.140 -1.83%
400.perlbench (max) 22.000 -8.33%
400.perlbench (max) (percentage) 1.460 -7.59%
400.perlbench (unexplained diff vs previous) 8.000 60.00%
400.perlbench (unexplained diff vs previous) (percentage) 0.510 59.38%
401.bzip2 66.000 -
401.bzip2 (avg) 66.000 -
401.bzip2 (avg) (percentage) 2.200 -
401.bzip2 (max) 45.000 -81.63%
401.bzip2 (max) (percentage) 1.500 -81.68%
401.bzip2 (unexplained diff vs previous) 28.000 -49.09%
401.bzip2 (unexplained diff vs previous) (percentage) 0.930 -49.18%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 6.180 -
403.gcc (max) 7.000 -
403.gcc (max) (percentage) 11.570 -4.30%
403.gcc (unexplained diff vs previous) 3.000 -
403.gcc (unexplained diff vs previous) (percentage) 4.840 -4.72%
410.bwaves 73.000 -1.35%
410.bwaves (avg) 73.000 -1.35%
410.bwaves (avg) (percentage) 1.220 -
410.bwaves (max) 31.000 -3.12%
410.bwaves (max) (percentage) 0.520 -3.70%
410.bwaves (unexplained diff vs previous) 18.000 -33.33%
410.bwaves (unexplained diff vs previous) (percentage) 0.300 -33.33%
416.gamess 115.000 -1.71%
416.gamess (avg) 115.000 -1.71%
416.gamess (avg) (percentage) 0.920 -2.13%
416.gamess (max) 27.000 -3.57%
416.gamess (max) (percentage) 0.220 -4.35%
416.gamess (unexplained diff vs previous) 26.000 -10.34%
416.gamess (unexplained diff vs previous) (percentage) 0.210 -8.70%
429.mcf 88.000 1.15%
429.mcf (avg) 88.000 1.15%
429.mcf (avg) (percentage) 5.820 -1.02%
429.mcf (max) 295.000 -1.01%
429.mcf (max) (percentage) 19.410 -3.43%
429.mcf (unexplained diff vs previous) 37.000 12.12%
429.mcf (unexplained diff vs previous) (percentage) 2.430 9.46%
433.milc 19.000 -
433.milc (avg) 19.000 -
433.milc (avg) (percentage) 1.650 -
433.milc (max) 12.000 200.00%
433.milc (max) (percentage) 1.030 164.10%
433.milc (unexplained diff vs previous) 12.000 -
433.milc (unexplained diff vs previous) (percentage) 1.020 -
434.zeusmp 41.000 -
434.zeusmp (avg) 41.000 -
434.zeusmp (avg) (percentage) 1.620 -
434.zeusmp (max) 28.000 64.71%
434.zeusmp (max) (percentage) 1.100 61.76%
434.zeusmp (unexplained diff vs previous) 28.000 154.55%
434.zeusmp (unexplained diff vs previous) (percentage) 1.090 153.49%
435.gromacs 143.000 1.42%
435.gromacs (avg) 143.000 1.42%
435.gromacs (avg) (percentage) 1.970 -
435.gromacs (max) 125.000 15.74%
435.gromacs (max) (percentage) 1.720 13.91%
435.gromacs (unexplained diff vs previous) 123.000 324.14%
435.gromacs (unexplained diff vs previous) (percentage) 1.690 322.50%
436.cactusADM 30.000 -
436.cactusADM (avg) 30.000 -
436.cactusADM (avg) (percentage) 1.450 -1.36%
436.cactusADM (max) 12.000 33.33%
436.cactusADM (max) (percentage) 0.570 29.55%
436.cactusADM (unexplained diff vs previous) 12.000 140.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.570 137.50%
437.leslie3d 483.000 5.00%
437.leslie3d (avg) 483.000 5.00%
437.leslie3d (avg) (percentage) 4.360 -
437.leslie3d (max) 518.000 105.56%
437.leslie3d (max) (percentage) 4.680 96.64%
437.leslie3d (unexplained diff vs previous) 496.000 350.91%
437.leslie3d (unexplained diff vs previous) (percentage) 4.470 329.81%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.810 -1.63%
444.namd (max) 8.000 -27.27%
444.namd (max) (percentage) 1.220 -25.15%
444.namd (unexplained diff vs previous) 3.000 -75.00%
444.namd (unexplained diff vs previous) (percentage) 0.410 -75.30%
445.gobmk 62.000 -
445.gobmk (avg) 62.000 -
445.gobmk (avg) (percentage) 1.180 -
445.gobmk (max) 59.000 -3.28%
445.gobmk (max) (percentage) 1.140 -3.39%
445.gobmk (unexplained diff vs previous) 34.000 41.67%
445.gobmk (unexplained diff vs previous) (percentage) 0.650 41.30%
447.dealII 29.000 -
447.dealII (avg) 29.000 -
447.dealII (avg) (percentage) 1.470 -1.34%
447.dealII (max) 13.000 -31.58%
447.dealII (max) (percentage) 0.680 -31.31%
447.dealII (unexplained diff vs previous) 11.000 -38.89%
447.dealII (unexplained diff vs previous) (percentage) 0.550 -39.56%
450.soplex 17.000 6.25%
450.soplex (avg) 17.000 6.25%
450.soplex (avg) (percentage) 2.990 -
450.soplex (max) 21.000 75.00%
450.soplex (max) (percentage) 3.670 69.91%
450.soplex (unexplained diff vs previous) 21.000 600.00%
450.soplex (unexplained diff vs previous) (percentage) 3.540 580.77%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.930 -1.53%
453.povray (max) 3.000 -25.00%
453.povray (max) (percentage) 0.750 -25.00%
453.povray (unexplained diff vs previous) 3.000 -40.00%
453.povray (unexplained diff vs previous) (percentage) 0.600 -40.59%
454.calculix 1.000 -
454.calculix (avg) 1.000 -
454.calculix (avg) (percentage) 3.210 -1.53%
454.calculix (max) 1.000 -
454.calculix (max) (percentage) 3.120 -
454.calculix (unexplained diff vs previous) 1.000 -
454.calculix (unexplained diff vs previous) (percentage) 1.610 -1.83%
456.hmmer 26.000 -
456.hmmer (avg) 26.000 -
456.hmmer (avg) (percentage) 0.910 -1.09%
456.hmmer (max) 14.000 -6.67%
456.hmmer (max) (percentage) 0.500 -3.85%
456.hmmer (unexplained diff vs previous) 9.000 -25.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.310 -24.39%
458.sjeng 63.000 -1.56%
458.sjeng (avg) 63.000 -1.56%
458.sjeng (avg) (percentage) 0.820 -1.20%
458.sjeng (max) 80.000 -4.76%
458.sjeng (max) (percentage) 1.050 -3.67%
458.sjeng (unexplained diff vs previous) 39.000 25.81%
458.sjeng (unexplained diff vs previous) (percentage) 0.510 27.50%
459.GemsFDTD 92.000 -2.13%
459.GemsFDTD (avg) 92.000 -2.13%
459.GemsFDTD (avg) (percentage) 1.510 -1.95%
459.GemsFDTD (max) 16.000 -27.27%
459.GemsFDTD (max) (percentage) 0.270 -25.00%
459.GemsFDTD (unexplained diff vs previous) 0.000 -
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.000 -
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.830 -
462.libquantum (max) 2.000 -33.33%
462.libquantum (max) (percentage) 1.850 -29.12%
462.libquantum (unexplained diff vs previous) 2.000 -33.33%
462.libquantum (unexplained diff vs previous) (percentage) 1.470 -32.26%
464.h264ref 57.000 -1.72%
464.h264ref (avg) 57.000 -1.72%
464.h264ref (avg) (percentage) 0.970 -1.02%
464.h264ref (max) 46.000 -4.17%
464.h264ref (max) (percentage) 0.780 -4.88%
464.h264ref (unexplained diff vs previous) 21.000 133.33%
464.h264ref (unexplained diff vs previous) (percentage) 0.350 133.33%
465.tonto 147.000 -
465.tonto (avg) 147.000 -
465.tonto (avg) (percentage) 1.290 -
465.tonto (max) 132.000 -1.49%
465.tonto (max) (percentage) 1.160 -2.52%
465.tonto (unexplained diff vs previous) 112.000 558.82%
465.tonto (unexplained diff vs previous) (percentage) 0.980 553.33%
470.lbm 46.000 -2.13%
470.lbm (avg) 46.000 -2.13%
470.lbm (avg) (percentage) 1.250 -1.57%
470.lbm (max) 10.000 -9.09%
470.lbm (max) (percentage) 0.270 -10.00%
470.lbm (unexplained diff vs previous) 6.000 100.00%
470.lbm (unexplained diff vs previous) (percentage) 0.160 100.00%
471.omnetpp 71.000 -2.74%
471.omnetpp (avg) 71.000 -2.74%
471.omnetpp (avg) (percentage) 2.460 1.65%
471.omnetpp (max) 138.000 26.61%
471.omnetpp (max) (percentage) 4.800 32.96%
471.omnetpp (unexplained diff vs previous) 146.000 37.74%
471.omnetpp (unexplained diff vs previous) (percentage) 5.040 44.41%
473.astar 102.000 -1.92%
473.astar (avg) 102.000 -1.92%
473.astar (avg) (percentage) 1.740 -1.14%
473.astar (max) 222.000 -3.06%
473.astar (max) (percentage) 3.770 -2.84%
473.astar (unexplained diff vs previous) 24.000 26.32%
473.astar (unexplained diff vs previous) (percentage) 0.410 28.12%
481.wrf 96.000 -
481.wrf (avg) 96.000 -
481.wrf (avg) (percentage) 1.740 -1.14%
481.wrf (max) 120.000 -1.64%
481.wrf (max) (percentage) 2.170 -3.13%
481.wrf (unexplained diff vs previous) 56.000 -55.56%
481.wrf (unexplained diff vs previous) (percentage) 1.010 -56.09%
482.sphinx3 18.000 -5.26%
482.sphinx3 (avg) 18.000 -5.26%
482.sphinx3 (avg) (percentage) 1.990 -1.49%
482.sphinx3 (max) 9.000 -
482.sphinx3 (max) (percentage) 1.000 -2.91%
482.sphinx3 (unexplained diff vs previous) 1.000 -50.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.110 -47.62%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.760 -
Geometric Mean 6.515 4.91%