Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6179272353d3b4384a2ac306fe11aa024f344d47 (+ 16 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/3ff1b91e772925caadea2c4a2e40f8cbfb6989d7 (+ 82 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=82b5340ebdb8f00589d548e6e2dc8c998f07d0c5 (+ 6 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/3b6df5ec957d4f6d871ecd63a14fdc98bd24dc2d (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/233/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0233 (3-changed-components) 2024-08-20T22:01:27 0:00:00
Previous 0232 (4-changed-components) 2024-08-17T19:44:41 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 29 62
Performance Improvements 104 145
Unchanged Tests 1103 1029
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 3.23% 2752.000 2841.000 - -69.34% -
482.sphinx3 1.22% 898.000 909.000 - -97.52% -
435.gromacs 1.00% 7567.000 7643.000 - -54.65% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (unexplained diff vs previous) (percentage) 1050.00% 0.020 0.230 0.000 -55.77% 0.000
456.hmmer (unexplained diff vs previous) 900.00% 1.000 10.000 0.000 -89.80% 0.000
471.omnetpp (unexplained diff vs previous) 641.67% 12.000 89.000 0.000 43.55% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 611.36% 0.440 3.130 0.000 367.16% 0.000
429.mcf (unexplained diff vs previous) (percentage) 465.08% 0.630 3.560 0.000 3855.56% 0.000
429.mcf (unexplained diff vs previous) 450.00% 8.000 44.000 0.000 340.00% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) 184.97% 1.930 5.500 0.000 433.98% 0.000
437.leslie3d (unexplained diff vs previous) 169.66% 234.000 631.000 0.000 237.43% 0.000
470.lbm (unexplained diff vs previous) (percentage) 137.50% 0.160 0.380 0.000 -92.46% 0.000
470.lbm (unexplained diff vs previous) 133.33% 6.000 14.000 0.000 -98.30% 0.000
471.omnetpp (max) 102.22% 45.000 91.000 - -47.09% -
471.omnetpp (max) (percentage) 95.76% 1.650 3.230 - 73.66% -
453.povray (unexplained diff vs previous) (percentage) 52.05% 0.730 1.110 0.000 -51.53% 0.000
453.povray (unexplained diff vs previous) 50.00% 4.000 6.000 0.000 -96.45% 0.000
433.milc (unexplained diff vs previous) 25.00% 4.000 5.000 0.000 -94.79% 0.000
433.milc (unexplained diff vs previous) (percentage) 24.32% 0.370 0.460 0.000 -34.29% 0.000
437.leslie3d (max) (percentage) 22.01% 4.270 5.210 - 68.06% -
435.gromacs (unexplained diff vs previous) 16.92% 65.000 76.000 0.000 -73.97% 0.000
437.leslie3d (max) 15.47% 517.000 597.000 - 6.61% -
435.gromacs (unexplained diff vs previous) (percentage) 15.12% 0.860 0.990 0.000 -42.77% 0.000
482.sphinx3 7.14% 14.000 15.000 - -98.25% -
482.sphinx3 (avg) 7.14% 14.000 15.000 - -98.25% -
471.omnetpp 4.00% 50.000 52.000 - -52.29% -
471.omnetpp (avg) 4.00% 50.000 52.000 - -52.29% -
450.soplex (unexplained diff vs previous) (percentage) 1.33% 1.500 1.520 0.000 245.45% 0.000
471.omnetpp (avg) (percentage) 1.10% 1.820 1.840 - 55.93% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -5.21% 12108.000 11477.000 - -36.49% -
429.mcf -3.44% 1279.000 1235.000 - -89.47% -
450.soplex -1.50% 600.000 591.000 - -96.24% -
453.povray -1.09% 548.000 542.000 - -92.67% -
400.perlbench -1.04% 1542.000 1526.000 - -86.39% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
410.bwaves (unexplained diff vs previous) -93.62% 47.000 3.000 0.000 -98.83% 0.000
410.bwaves (unexplained diff vs previous) (percentage) -93.59% 0.780 0.050 0.000 -97.13% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) -81.48% 0.810 0.150 0.000 -96.88% 0.000
436.cactusADM (unexplained diff vs previous) -80.95% 21.000 4.000 0.000 -99.66% 0.000
447.dealII (max) -76.67% 30.000 7.000 - -95.33% -
447.dealII (max) (percentage) -76.58% 1.580 0.370 - -77.58% -
445.gobmk (unexplained diff vs previous) -73.91% 46.000 12.000 0.000 -33.33% 0.000
445.gobmk (unexplained diff vs previous) (percentage) -73.33% 0.900 0.240 0.000 60.00% 0.000
473.astar (max) -69.30% 114.000 35.000 - -42.62% -
473.astar (max) (percentage) -69.20% 2.370 0.730 - 55.32% -
473.astar (unexplained diff vs previous) -65.79% 38.000 13.000 0.000 -78.69% 0.000
473.astar (unexplained diff vs previous) (percentage) -65.38% 0.780 0.270 0.000 -42.55% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) -58.67% 0.750 0.310 0.000 -86.52% 0.000
459.GemsFDTD (unexplained diff vs previous) -58.49% 53.000 22.000 0.000 -96.23% 0.000
459.GemsFDTD (max) -58.26% 115.000 48.000 - -91.70% -
459.GemsFDTD (max) (percentage) -57.41% 1.620 0.690 - -69.74% -
401.bzip2 (max) -55.67% 97.000 43.000 - -35.82% -
434.zeusmp (unexplained diff vs previous) -55.56% 18.000 8.000 0.000 -98.30% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) -55.38% 0.650 0.290 0.000 -87.11% 0.000
401.bzip2 (max) (percentage) -55.09% 3.340 1.500 - 275.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -51.75% 1.140 0.550 0.000 34.15% 0.000
401.bzip2 (unexplained diff vs previous) -51.52% 33.000 16.000 0.000 -76.81% 0.000
400.perlbench (max) -48.28% 29.000 15.000 - -83.15% -
400.perlbench (max) (percentage) -46.39% 1.940 1.040 - 30.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -42.92% 2.120 1.210 0.000 -60.46% 0.000
482.sphinx3 (unexplained diff vs previous) -42.11% 19.000 11.000 0.000 -99.02% 0.000
482.sphinx3 (max) (percentage) -40.48% 2.890 1.720 - -45.40% -
482.sphinx3 (max) -40.00% 25.000 15.000 - -98.70% -
434.zeusmp (max) -39.13% 23.000 14.000 - -97.09% -
434.zeusmp (max) (percentage) -34.94% 0.830 0.540 - -76.52% -
462.libquantum (unexplained diff vs previous) (percentage) -34.27% 1.430 0.940 0.000 1075.00% 0.000
462.libquantum (unexplained diff vs previous) -33.33% 3.000 2.000 0.000 -85.71% 0.000
444.namd (max) -30.77% 13.000 9.000 - -98.98% -
444.namd (max) (percentage) -30.56% 1.440 1.000 - -81.85% -
436.cactusADM (max) (percentage) -25.00% 0.800 0.600 - -88.12% -
410.bwaves (max) -25.00% 52.000 39.000 - -85.00% -
436.cactusADM (max) -25.00% 20.000 15.000 - -98.78% -
444.namd (unexplained diff vs previous) -25.00% 12.000 9.000 0.000 -98.92% 0.000
410.bwaves (max) (percentage) -25.00% 0.880 0.660 - -62.71% -
444.namd (unexplained diff vs previous) (percentage) -24.43% 1.310 0.990 0.000 -81.07% 0.000
464.h264ref (unexplained diff vs previous) (percentage) -22.58% 0.310 0.240 0.000 -36.84% 0.000
464.h264ref (unexplained diff vs previous) -21.05% 19.000 15.000 0.000 -74.14% 0.000
462.libquantum (max) (percentage) -17.02% 1.410 1.170 - 1362.50% -
458.sjeng (unexplained diff vs previous) -14.29% 7.000 6.000 0.000 -87.50% 0.000
447.dealII (unexplained diff vs previous) -12.50% 8.000 7.000 0.000 -89.71% 0.000
447.dealII (unexplained diff vs previous) (percentage) -11.90% 0.420 0.370 0.000 -50.67% 0.000
450.soplex (max) -11.11% 9.000 8.000 - -88.41% -
453.povray (max) -11.11% 9.000 8.000 - -95.81% -
429.mcf (max) -10.00% 80.000 72.000 - 94.59% -
453.povray (max) (percentage) -8.99% 1.780 1.620 - -37.45% -
445.gobmk (max) (percentage) -6.59% 0.910 0.850 - 214.81% -
445.gobmk (max) -6.52% 46.000 43.000 - 30.30% -
465.tonto (max) (percentage) -6.32% 0.950 0.890 - -47.95% -
458.sjeng (max) (percentage) -6.06% 0.660 0.620 - 106.67% -
458.sjeng (max) -6.00% 50.000 47.000 - -2.08% -
400.perlbench (unexplained diff vs previous) -5.88% 17.000 16.000 0.000 -54.29% 0.000
429.mcf (max) (percentage) -5.74% 6.270 5.910 - 1746.88% -
465.tonto (max) -5.74% 122.000 115.000 - -48.43% -
470.lbm (max) -5.71% 35.000 33.000 - -96.30% -
433.milc (max) (percentage) -5.59% 1.790 1.690 - 138.03% -
433.milc (max) -5.26% 19.000 18.000 - -81.44% -
470.lbm (max) (percentage) -5.21% 0.960 0.910 - -83.39% -
437.leslie3d -5.07% 552.000 524.000 - 74.09% -
437.leslie3d (avg) -5.07% 552.000 524.000 - 74.09% -
464.h264ref (max) (percentage) -4.76% 0.420 0.400 - 2.56% -
400.perlbench (unexplained diff vs previous) (percentage) -4.55% 1.100 1.050 0.000 238.71% 0.000
456.hmmer -4.00% 25.000 24.000 - -57.14% -
456.hmmer (avg) -4.00% 25.000 24.000 - -57.14% -
464.h264ref (max) -4.00% 25.000 24.000 - -59.32% -
447.dealII -3.45% 29.000 28.000 - -64.56% -
447.dealII (avg) -3.45% 29.000 28.000 - -64.56% -
429.mcf -3.39% 59.000 57.000 - 1.79% -
429.mcf (avg) -3.39% 59.000 57.000 - 1.79% -
456.hmmer (max) (percentage) -2.33% 0.430 0.420 - -19.23% -
459.GemsFDTD -2.22% 90.000 88.000 - -75.00% -
459.GemsFDTD (avg) -2.22% 90.000 88.000 - -75.00% -
464.h264ref -2.04% 49.000 48.000 - -18.64% -
464.h264ref (avg) -2.04% 49.000 48.000 - -18.64% -
410.bwaves (avg) (percentage) -1.92% 1.040 1.020 - -47.96% -
435.gromacs (max) (percentage) -1.92% 2.080 2.040 - 15.91% -
458.sjeng -1.89% 53.000 52.000 - 44.44% -
458.sjeng (avg) -1.89% 53.000 52.000 - 44.44% -
401.bzip2 -1.79% 56.000 55.000 - 22.22% -
401.bzip2 (avg) -1.79% 56.000 55.000 - 22.22% -
456.hmmer (avg) (percentage) -1.69% 0.590 0.580 - 93.33% -
470.lbm (avg) (percentage) -1.68% 1.190 1.170 - -68.63% -
410.bwaves -1.61% 62.000 61.000 - -78.89% -
410.bwaves (avg) -1.61% 62.000 61.000 - -78.89% -
459.GemsFDTD (avg) (percentage) -1.57% 1.270 1.250 - -10.07% -
458.sjeng (avg) (percentage) -1.43% 0.700 0.690 - 200.00% -
473.astar -1.33% 75.000 74.000 - 45.10% -
473.astar (avg) -1.33% 75.000 74.000 - 45.10% -
450.soplex (max) (percentage) -1.32% 1.520 1.500 - 240.91% -
447.dealII (avg) (percentage) -1.30% 1.540 1.520 - 74.71% -
473.astar (avg) (percentage) -1.29% 1.550 1.530 - 282.50% -
435.gromacs (max) -1.27% 157.000 155.000 - -47.64% -
464.h264ref (avg) (percentage) -1.25% 0.800 0.790 - 102.56% -
401.bzip2 (avg) (percentage) -1.04% 1.930 1.910 - 607.41% -
436.cactusADM (avg) (percentage) -1.00% 1.000 0.990 - -58.75% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.86% 1437761.000 1464529.000 - -0.00% -
473.astar 1.65% 15549.000 15805.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (unexplained diff vs previous) (percentage) 3855.56% 0.090 3.560 0.000 465.08% 0.000
429.mcf (max) (percentage) 1746.88% 0.320 5.910 - -5.74% -
462.libquantum (max) (percentage) 1362.50% 0.080 1.170 - -17.02% -
403.gcc (max) (percentage) 1164.41% 0.590 7.460 - -0.27% -
462.libquantum (unexplained diff vs previous) (percentage) 1075.00% 0.080 0.940 0.000 -34.27% 0.000
450.soplex (avg) (percentage) 1041.67% 0.240 2.740 - -0.72% -
429.mcf (avg) (percentage) 866.67% 0.480 4.640 - -0.43% -
462.libquantum (avg) (percentage) 678.95% 0.190 1.480 - -0.67% -
401.bzip2 (avg) (percentage) 607.41% 0.270 1.910 - -1.04% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
437.leslie3d (unexplained diff vs previous) (percentage) 433.98% 1.030 5.500 0.000 184.97% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 367.16% 0.670 3.130 0.000 611.36% 0.000
429.mcf (unexplained diff vs previous) 340.00% 10.000 44.000 0.000 450.00% 0.000
445.gobmk (avg) (percentage) 322.73% 0.220 0.930 - -1.06% -
454.calculix (max) (percentage) 314.29% 0.840 3.480 - 0.00% -
400.perlbench (avg) (percentage) 312.82% 0.390 1.610 - -0.62% -
473.astar (avg) (percentage) 282.50% 0.400 1.530 - -1.29% -
401.bzip2 (max) (percentage) 275.00% 0.400 1.500 - -55.09% -
450.soplex (unexplained diff vs previous) (percentage) 245.45% 0.440 1.520 0.000 1.33% 0.000
450.soplex (max) (percentage) 240.91% 0.440 1.500 - -1.32% -
400.perlbench (unexplained diff vs previous) (percentage) 238.71% 0.310 1.050 0.000 -4.55% 0.000
437.leslie3d (unexplained diff vs previous) 237.43% 187.000 631.000 0.000 169.66% 0.000
445.gobmk (max) (percentage) 214.81% 0.270 0.850 - -6.59% -
458.sjeng (avg) (percentage) 200.00% 0.230 0.690 - -1.43% -
437.leslie3d (avg) (percentage) 173.65% 1.670 4.570 - 0.22% -
433.milc (max) (percentage) 138.03% 0.710 1.690 - -5.59% -
458.sjeng (max) (percentage) 106.67% 0.300 0.620 - -6.06% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
464.h264ref (avg) (percentage) 102.56% 0.390 0.790 - -1.25% -
429.mcf (max) 94.59% 37.000 72.000 - -10.00% -
456.hmmer (avg) (percentage) 93.33% 0.300 0.580 - -1.69% -
435.gromacs (avg) (percentage) 82.11% 0.950 1.730 - -0.57% -
433.milc (avg) (percentage) 79.71% 0.690 1.240 - -0.80% -
447.dealII (avg) (percentage) 74.71% 0.870 1.520 - -1.30% -
437.leslie3d 74.09% 301.000 524.000 - -5.07% -
437.leslie3d (avg) 74.09% 301.000 524.000 - -5.07% -
445.gobmk 74.07% 27.000 47.000 - 0.00% -
445.gobmk (avg) 74.07% 27.000 47.000 - 0.00% -
471.omnetpp (max) (percentage) 73.66% 1.860 3.230 - 95.76% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
437.leslie3d (max) (percentage) 68.06% 3.100 5.210 - 22.01% -
483.xalancbmk (max) (percentage) 64.13% 0.920 1.510 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) 60.00% 0.150 0.240 0.000 -73.33% 0.000
471.omnetpp (avg) (percentage) 55.93% 1.180 1.840 - 1.10% -
473.astar (max) (percentage) 55.32% 0.470 0.730 - -69.20% -
473.astar 45.10% 51.000 74.000 - -1.33% -
473.astar (avg) 45.10% 51.000 74.000 - -1.33% -
458.sjeng 44.44% 36.000 52.000 - -1.89% -
458.sjeng (avg) 44.44% 36.000 52.000 - -1.89% -
471.omnetpp (unexplained diff vs previous) 43.55% 62.000 89.000 0.000 641.67% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) 34.15% 0.410 0.550 0.000 -51.75% 0.000
445.gobmk (max) 30.30% 33.000 43.000 - -6.52% -
400.perlbench (max) (percentage) 30.00% 0.800 1.040 - -46.39% -
401.bzip2 22.22% 45.000 55.000 - -1.79% -
401.bzip2 (avg) 22.22% 45.000 55.000 - -1.79% -
435.gromacs (max) (percentage) 15.91% 1.760 2.040 - -1.92% -
437.leslie3d (max) 6.61% 560.000 597.000 - 15.47% -
464.h264ref (max) (percentage) 2.56% 0.390 0.400 - -4.76% -
429.mcf 1.79% 56.000 57.000 - -3.39% -
429.mcf (avg) 1.79% 56.000 57.000 - -3.39% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.83% 10845.000 10213.000 - 0.00% -
437.leslie3d -3.71% 48001.000 46220.000 - 0.00% -
434.zeusmp -1.82% 141060.000 138498.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.75% 17005.000 212.000 - 0.95% -
482.sphinx3 -97.52% 36586.000 909.000 - 1.22% -
450.soplex -96.24% 15709.000 591.000 - -1.50% -
444.namd -94.33% 15957.000 905.000 - -0.98% -
453.povray -92.67% 7395.000 542.000 - -1.09% -
433.milc -92.00% 13668.000 1093.000 - 0.46% -
429.mcf -89.47% 11727.000 1235.000 - -3.44% -
436.cactusADM -89.33% 24396.000 2602.000 - 0.15% -
434.zeusmp -86.77% 20932.000 2769.000 - -0.29% -
400.perlbench -86.39% 11215.000 1526.000 - -1.04% -
401.bzip2 -82.97% 16971.000 2890.000 - -0.55% -
447.dealII -79.11% 9101.000 1901.000 - -0.37% -
456.hmmer -77.37% 18995.000 4299.000 - -0.23% -
470.lbm -77.35% 16312.000 3694.000 - 0.38% -
459.GemsFDTD -72.07% 25364.000 7083.000 - -0.31% -
471.omnetpp -69.34% 9265.000 2841.000 - 3.23% -
473.astar -62.55% 12983.000 4862.000 - 0.27% -
464.h264ref -59.72% 15208.000 6126.000 - -0.24% -
410.bwaves -59.29% 14745.000 6003.000 - -0.05% -
445.gobmk -58.98% 12396.000 5085.000 - -0.24% -
435.gromacs -54.65% 16853.000 7643.000 - 1.00% -
458.sjeng -52.25% 16018.000 7649.000 - -0.08% -
437.leslie3d -36.49% 18070.000 11477.000 - -5.21% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (unexplained diff vs previous) -99.66% 1173.000 4.000 0.000 -80.95% 0.000
482.sphinx3 (unexplained diff vs previous) -99.02% 1118.000 11.000 0.000 -42.11% 0.000
444.namd (max) -98.98% 879.000 9.000 - -30.77% -
444.namd (unexplained diff vs previous) -98.92% 834.000 9.000 0.000 -25.00% 0.000
410.bwaves (unexplained diff vs previous) -98.83% 257.000 3.000 0.000 -93.62% 0.000
436.cactusADM (max) -98.78% 1231.000 15.000 - -25.00% -
482.sphinx3 (max) -98.70% 1152.000 15.000 - -40.00% -
434.zeusmp (unexplained diff vs previous) -98.30% 470.000 8.000 0.000 -55.56% 0.000
470.lbm (unexplained diff vs previous) -98.30% 822.000 14.000 0.000 133.33% 0.000
482.sphinx3 -98.25% 859.000 15.000 - 7.14% -
482.sphinx3 (avg) -98.25% 859.000 15.000 - 7.14% -
444.namd -97.58% 414.000 10.000 - 0.00% -
444.namd (avg) -97.58% 414.000 10.000 - 0.00% -
410.bwaves (unexplained diff vs previous) (percentage) -97.13% 1.740 0.050 0.000 -93.59% 0.000
434.zeusmp (max) -97.09% 481.000 14.000 - -39.13% -
436.cactusADM (unexplained diff vs previous) (percentage) -96.88% 4.810 0.150 0.000 -81.48% 0.000
453.povray (unexplained diff vs previous) -96.45% 169.000 6.000 0.000 50.00% 0.000
470.lbm (max) -96.30% 893.000 33.000 - -5.71% -
459.GemsFDTD (unexplained diff vs previous) -96.23% 584.000 22.000 0.000 -58.49% 0.000
416.gamess (max) (percentage) -95.86% 3.140 0.130 - 0.00% -
453.povray (max) -95.81% 191.000 8.000 - -11.11% -
436.cactusADM -95.73% 585.000 25.000 - 0.00% -
436.cactusADM (avg) -95.73% 585.000 25.000 - 0.00% -
447.dealII (max) -95.33% 150.000 7.000 - -76.67% -
433.milc (unexplained diff vs previous) -94.79% 96.000 5.000 0.000 25.00% 0.000
453.povray -93.81% 113.000 7.000 - 0.00% -
453.povray (avg) -93.81% 113.000 7.000 - 0.00% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
470.lbm -92.93% 608.000 43.000 - 0.00% -
470.lbm (avg) -92.93% 608.000 43.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -92.46% 5.040 0.380 0.000 137.50% 0.000
459.GemsFDTD (max) -91.70% 578.000 48.000 - -58.26% -
462.libquantum -90.62% 32.000 3.000 - 0.00% -
462.libquantum (avg) -90.62% 32.000 3.000 - 0.00% -
456.hmmer (unexplained diff vs previous) -89.80% 98.000 10.000 0.000 900.00% 0.000
447.dealII (unexplained diff vs previous) -89.71% 68.000 7.000 0.000 -12.50% 0.000
434.zeusmp -88.63% 255.000 29.000 - 0.00% -
434.zeusmp (avg) -88.63% 255.000 29.000 - 0.00% -
450.soplex (max) -88.41% 69.000 8.000 - -11.11% -
436.cactusADM (max) (percentage) -88.12% 5.050 0.600 - -25.00% -
458.sjeng (unexplained diff vs previous) -87.50% 48.000 6.000 0.000 -14.29% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) -87.11% 2.250 0.290 0.000 -55.38% 0.000
450.soplex (unexplained diff vs previous) -86.96% 69.000 9.000 0.000 0.00% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) -86.52% 2.300 0.310 0.000 -58.67% 0.000
433.milc -86.17% 94.000 13.000 - 0.00% -
433.milc (avg) -86.17% 94.000 13.000 - 0.00% -
462.libquantum (unexplained diff vs previous) -85.71% 14.000 2.000 0.000 -33.33% 0.000
410.bwaves (max) -85.00% 260.000 39.000 - -25.00% -
462.libquantum (max) -84.62% 13.000 2.000 - 0.00% -
470.lbm (max) (percentage) -83.39% 5.480 0.910 - -5.21% -
400.perlbench (max) -83.15% 89.000 15.000 - -48.28% -
444.namd (max) (percentage) -81.85% 5.510 1.000 - -30.56% -
456.hmmer (max) -81.63% 98.000 18.000 - 0.00% -
433.milc (max) -81.44% 97.000 18.000 - -5.26% -
444.namd (unexplained diff vs previous) (percentage) -81.07% 5.230 0.990 0.000 -24.43% 0.000
410.bwaves -78.89% 289.000 61.000 - -1.61% -
410.bwaves (avg) -78.89% 289.000 61.000 - -1.61% -
473.astar (unexplained diff vs previous) -78.69% 61.000 13.000 0.000 -65.79% 0.000
447.dealII (max) (percentage) -77.58% 1.650 0.370 - -76.58% -
401.bzip2 (unexplained diff vs previous) -76.81% 69.000 16.000 0.000 -51.52% 0.000
434.zeusmp (max) (percentage) -76.52% 2.300 0.540 - -34.94% -
459.GemsFDTD -75.00% 352.000 88.000 - -2.22% -
459.GemsFDTD (avg) -75.00% 352.000 88.000 - -2.22% -
464.h264ref (unexplained diff vs previous) -74.14% 58.000 15.000 0.000 -21.05% 0.000
435.gromacs (unexplained diff vs previous) -73.97% 292.000 76.000 0.000 16.92% 0.000
458.sjeng (unexplained diff vs previous) (percentage) -73.33% 0.300 0.080 0.000 -11.11% 0.000
459.GemsFDTD (max) (percentage) -69.74% 2.280 0.690 - -57.41% -
470.lbm (avg) (percentage) -68.63% 3.730 1.170 - -1.68% -
447.dealII -64.56% 79.000 28.000 - -3.45% -
447.dealII (avg) -64.56% 79.000 28.000 - -3.45% -
410.bwaves (max) (percentage) -62.71% 1.770 0.660 - -25.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -60.46% 3.060 1.210 0.000 -42.92% 0.000
464.h264ref (max) -59.32% 59.000 24.000 - -4.00% -
436.cactusADM (avg) (percentage) -58.75% 2.400 0.990 - -1.00% -
456.hmmer -57.14% 56.000 24.000 - -4.00% -
456.hmmer (avg) -57.14% 56.000 24.000 - -4.00% -
450.soplex -56.76% 37.000 16.000 - 0.00% -
450.soplex (avg) -56.76% 37.000 16.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -55.77% 0.520 0.230 0.000 1050.00% 0.000
400.perlbench (unexplained diff vs previous) -54.29% 35.000 16.000 0.000 -5.88% 0.000
444.namd (avg) (percentage) -53.85% 2.600 1.200 - 0.00% -
471.omnetpp -52.29% 109.000 52.000 - 4.00% -
471.omnetpp (avg) -52.29% 109.000 52.000 - 4.00% -
453.povray (unexplained diff vs previous) (percentage) -51.53% 2.290 1.110 0.000 52.05% 0.000
447.dealII (unexplained diff vs previous) (percentage) -50.67% 0.750 0.370 0.000 -11.90% 0.000
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
465.tonto (max) -48.43% 223.000 115.000 - -5.74% -
410.bwaves (avg) (percentage) -47.96% 1.960 1.020 - -1.92% -
465.tonto (max) (percentage) -47.95% 1.710 0.890 - -6.32% -
435.gromacs (max) -47.64% 296.000 155.000 - -1.27% -
471.omnetpp (max) -47.09% 172.000 91.000 - 102.22% -
482.sphinx3 (max) (percentage) -45.40% 3.150 1.720 - -40.48% -
400.perlbench -44.19% 43.000 24.000 - 0.00% -
400.perlbench (avg) -44.19% 43.000 24.000 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) -42.77% 1.730 0.990 0.000 15.12% 0.000
473.astar (max) -42.62% 61.000 35.000 - -69.30% -
473.astar (unexplained diff vs previous) (percentage) -42.55% 0.470 0.270 0.000 -65.38% 0.000
453.povray (max) (percentage) -37.45% 2.590 1.620 - -8.99% -
464.h264ref (unexplained diff vs previous) (percentage) -36.84% 0.380 0.240 0.000 -22.58% 0.000
401.bzip2 (max) -35.82% 67.000 43.000 - -55.67% -
433.milc (unexplained diff vs previous) (percentage) -34.29% 0.700 0.460 0.000 24.32% 0.000
445.gobmk (unexplained diff vs previous) -33.33% 18.000 12.000 0.000 -73.91% 0.000
482.sphinx3 (avg) (percentage) -29.36% 2.350 1.660 - -0.60% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
465.tonto -25.68% 148.000 110.000 - 0.92% -
465.tonto (avg) -25.68% 148.000 110.000 - 0.92% -
465.tonto (avg) (percentage) -25.44% 1.140 0.850 - 0.00% -
456.hmmer (max) (percentage) -19.23% 0.520 0.420 - -2.33% -
464.h264ref -18.64% 59.000 48.000 - -2.04% -
464.h264ref (avg) -18.64% 59.000 48.000 - -2.04% -
435.gromacs -17.50% 160.000 132.000 - 0.76% -
435.gromacs (avg) -17.50% 160.000 132.000 - 0.76% -
434.zeusmp (avg) (percentage) -12.30% 1.220 1.070 - -0.93% -
459.GemsFDTD (avg) (percentage) -10.07% 1.390 1.250 - -1.57% -
453.povray (avg) (percentage) -5.23% 1.530 1.450 - 0.00% -
458.sjeng (max) -2.08% 48.000 47.000 - -6.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 0.00% -
nb_compile_successful 65.52% 29.000 48.000 - 0.00% -
nb_execution_successful 65.52% 29.000 48.000 - 0.00% -

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1526.000 -1.04%
401.bzip2 2890.000 -
410.bwaves 6003.000 -
429.mcf 1235.000 -3.44%
433.milc 1093.000 -
434.zeusmp 2769.000 -
435.gromacs 7643.000 1.00%
436.cactusADM 2602.000 -
437.leslie3d 11477.000 -5.21%
444.namd 905.000 -
445.gobmk 5085.000 -
447.dealII 1901.000 -
450.soplex 591.000 -1.50%
453.povray 542.000 -1.09%
456.hmmer 4299.000 -
458.sjeng 7649.000 -
459.GemsFDTD 7083.000 -
462.libquantum 212.000 -
464.h264ref 6126.000 -
465.tonto 12983.000 -
470.lbm 3694.000 -
471.omnetpp 2841.000 3.23%
473.astar 4862.000 -
482.sphinx3 909.000 1.22%
Geometric Mean 2625.670 -
Code Size (bytes)
Name Current %
400.perlbench 562286.000 -
401.bzip2 33454.000 -
410.bwaves 14723.000 -
429.mcf 7717.000 -
433.milc 47741.000 -
434.zeusmp 138498.000 -
435.gromacs 392978.000 -
436.cactusADM 256546.000 -
437.leslie3d 46220.000 -
444.namd 163075.000 -
445.gobmk 1087368.000 -
447.dealII 190403.000 -
450.soplex 135756.000 -
453.povray 493593.000 -
456.hmmer 74413.000 -
458.sjeng 81341.000 -
459.GemsFDTD 194155.000 -
462.libquantum 13134.000 -
464.h264ref 301274.000 -
465.tonto 1464529.000 -
470.lbm 10213.000 -
471.omnetpp 296045.000 -
473.astar 15805.000 -
482.sphinx3 79566.000 -
Geometric Mean 105786.348 -
sample variability
Name Current %
400.perlbench 24.000 -
400.perlbench (avg) 24.000 -
400.perlbench (avg) (percentage) 1.610 -
400.perlbench (max) 15.000 -48.28%
400.perlbench (max) (percentage) 1.040 -46.39%
400.perlbench (unexplained diff vs previous) 16.000 -5.88%
400.perlbench (unexplained diff vs previous) (percentage) 1.050 -4.55%
401.bzip2 55.000 -1.79%
401.bzip2 (avg) 55.000 -1.79%
401.bzip2 (avg) (percentage) 1.910 -1.04%
401.bzip2 (max) 43.000 -55.67%
401.bzip2 (max) (percentage) 1.500 -55.09%
401.bzip2 (unexplained diff vs previous) 16.000 -51.52%
401.bzip2 (unexplained diff vs previous) (percentage) 0.550 -51.75%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 7.460 -
410.bwaves 61.000 -1.61%
410.bwaves (avg) 61.000 -1.61%
410.bwaves (avg) (percentage) 1.020 -1.92%
410.bwaves (max) 39.000 -25.00%
410.bwaves (max) (percentage) 0.660 -25.00%
410.bwaves (unexplained diff vs previous) 3.000 -93.62%
410.bwaves (unexplained diff vs previous) (percentage) 0.050 -93.59%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.130 -
429.mcf 57.000 -3.39%
429.mcf (avg) 57.000 -3.39%
429.mcf (avg) (percentage) 4.640 -
429.mcf (max) 72.000 -10.00%
429.mcf (max) (percentage) 5.910 -5.74%
429.mcf (unexplained diff vs previous) 44.000 450.00%
429.mcf (unexplained diff vs previous) (percentage) 3.560 465.08%
433.milc 13.000 -
433.milc (avg) 13.000 -
433.milc (avg) (percentage) 1.240 -
433.milc (max) 18.000 -5.26%
433.milc (max) (percentage) 1.690 -5.59%
433.milc (unexplained diff vs previous) 5.000 25.00%
433.milc (unexplained diff vs previous) (percentage) 0.460 24.32%
434.zeusmp 29.000 -
434.zeusmp (avg) 29.000 -
434.zeusmp (avg) (percentage) 1.070 -
434.zeusmp (max) 14.000 -39.13%
434.zeusmp (max) (percentage) 0.540 -34.94%
434.zeusmp (unexplained diff vs previous) 8.000 -55.56%
434.zeusmp (unexplained diff vs previous) (percentage) 0.290 -55.38%
435.gromacs 132.000 -
435.gromacs (avg) 132.000 -
435.gromacs (avg) (percentage) 1.730 -
435.gromacs (max) 155.000 -1.27%
435.gromacs (max) (percentage) 2.040 -1.92%
435.gromacs (unexplained diff vs previous) 76.000 16.92%
435.gromacs (unexplained diff vs previous) (percentage) 0.990 15.12%
436.cactusADM 25.000 -
436.cactusADM (avg) 25.000 -
436.cactusADM (avg) (percentage) 0.990 -1.00%
436.cactusADM (max) 15.000 -25.00%
436.cactusADM (max) (percentage) 0.600 -25.00%
436.cactusADM (unexplained diff vs previous) 4.000 -80.95%
436.cactusADM (unexplained diff vs previous) (percentage) 0.150 -81.48%
437.leslie3d 524.000 -5.07%
437.leslie3d (avg) 524.000 -5.07%
437.leslie3d (avg) (percentage) 4.570 -
437.leslie3d (max) 597.000 15.47%
437.leslie3d (max) (percentage) 5.210 22.01%
437.leslie3d (unexplained diff vs previous) 631.000 169.66%
437.leslie3d (unexplained diff vs previous) (percentage) 5.500 184.97%
444.namd 10.000 -
444.namd (avg) 10.000 -
444.namd (avg) (percentage) 1.200 -
444.namd (max) 9.000 -30.77%
444.namd (max) (percentage) 1.000 -30.56%
444.namd (unexplained diff vs previous) 9.000 -25.00%
444.namd (unexplained diff vs previous) (percentage) 0.990 -24.43%
445.gobmk 47.000 -
445.gobmk (avg) 47.000 -
445.gobmk (avg) (percentage) 0.930 -
445.gobmk (max) 43.000 -6.52%
445.gobmk (max) (percentage) 0.850 -6.59%
445.gobmk (unexplained diff vs previous) 12.000 -73.91%
445.gobmk (unexplained diff vs previous) (percentage) 0.240 -73.33%
447.dealII 28.000 -3.45%
447.dealII (avg) 28.000 -3.45%
447.dealII (avg) (percentage) 1.520 -1.30%
447.dealII (max) 7.000 -76.67%
447.dealII (max) (percentage) 0.370 -76.58%
447.dealII (unexplained diff vs previous) 7.000 -12.50%
447.dealII (unexplained diff vs previous) (percentage) 0.370 -11.90%
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 2.740 -
450.soplex (max) 8.000 -11.11%
450.soplex (max) (percentage) 1.500 -1.32%
450.soplex (unexplained diff vs previous) 9.000 -
450.soplex (unexplained diff vs previous) (percentage) 1.520 1.33%
453.povray 7.000 -
453.povray (avg) 7.000 -
453.povray (avg) (percentage) 1.450 -
453.povray (max) 8.000 -11.11%
453.povray (max) (percentage) 1.620 -8.99%
453.povray (unexplained diff vs previous) 6.000 50.00%
453.povray (unexplained diff vs previous) (percentage) 1.110 52.05%
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.480 -
456.hmmer 24.000 -4.00%
456.hmmer (avg) 24.000 -4.00%
456.hmmer (avg) (percentage) 0.580 -1.69%
456.hmmer (max) 18.000 -
456.hmmer (max) (percentage) 0.420 -2.33%
456.hmmer (unexplained diff vs previous) 10.000 900.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.230 1050.00%
458.sjeng 52.000 -1.89%
458.sjeng (avg) 52.000 -1.89%
458.sjeng (avg) (percentage) 0.690 -1.43%
458.sjeng (max) 47.000 -6.00%
458.sjeng (max) (percentage) 0.620 -6.06%
458.sjeng (unexplained diff vs previous) 6.000 -14.29%
458.sjeng (unexplained diff vs previous) (percentage) 0.080 -
459.GemsFDTD 88.000 -2.22%
459.GemsFDTD (avg) 88.000 -2.22%
459.GemsFDTD (avg) (percentage) 1.250 -1.57%
459.GemsFDTD (max) 48.000 -58.26%
459.GemsFDTD (max) (percentage) 0.690 -57.41%
459.GemsFDTD (unexplained diff vs previous) 22.000 -58.49%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.310 -58.67%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 1.480 -
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.170 -17.02%
462.libquantum (unexplained diff vs previous) 2.000 -33.33%
462.libquantum (unexplained diff vs previous) (percentage) 0.940 -34.27%
464.h264ref 48.000 -2.04%
464.h264ref (avg) 48.000 -2.04%
464.h264ref (avg) (percentage) 0.790 -1.25%
464.h264ref (max) 24.000 -4.00%
464.h264ref (max) (percentage) 0.400 -4.76%
464.h264ref (unexplained diff vs previous) 15.000 -21.05%
464.h264ref (unexplained diff vs previous) (percentage) 0.240 -22.58%
465.tonto 110.000 -
465.tonto (avg) 110.000 -
465.tonto (avg) (percentage) 0.850 -
465.tonto (max) 115.000 -5.74%
465.tonto (max) (percentage) 0.890 -6.32%
470.lbm 43.000 -
470.lbm (avg) 43.000 -
470.lbm (avg) (percentage) 1.170 -1.68%
470.lbm (max) 33.000 -5.71%
470.lbm (max) (percentage) 0.910 -5.21%
470.lbm (unexplained diff vs previous) 14.000 133.33%
470.lbm (unexplained diff vs previous) (percentage) 0.380 137.50%
471.omnetpp 52.000 4.00%
471.omnetpp (avg) 52.000 4.00%
471.omnetpp (avg) (percentage) 1.840 1.10%
471.omnetpp (max) 91.000 102.22%
471.omnetpp (max) (percentage) 3.230 95.76%
471.omnetpp (unexplained diff vs previous) 89.000 641.67%
471.omnetpp (unexplained diff vs previous) (percentage) 3.130 611.36%
473.astar 74.000 -1.33%
473.astar (avg) 74.000 -1.33%
473.astar (avg) (percentage) 1.530 -1.29%
473.astar (max) 35.000 -69.30%
473.astar (max) (percentage) 0.730 -69.20%
473.astar (unexplained diff vs previous) 13.000 -65.79%
473.astar (unexplained diff vs previous) (percentage) 0.270 -65.38%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 15.000 7.14%
482.sphinx3 (avg) 15.000 7.14%
482.sphinx3 (avg) (percentage) 1.660 -
482.sphinx3 (max) 15.000 -40.00%
482.sphinx3 (max) (percentage) 1.720 -40.48%
482.sphinx3 (unexplained diff vs previous) 11.000 -42.11%
482.sphinx3 (unexplained diff vs previous) (percentage) 1.210 -42.92%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.510 -
Geometric Mean 5.814 -10.02%