Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=91a84aaf9a1c72d3b56c8cbc55d32327cdcfb8ce (+ 17 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/9817d29cd66762893782a52b2c304c5083bc0023 (+ 60 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=b0fbcb7d0051a68baf26b2aed51a8a31c34d68e5 (+ 9 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/d3af75d288b8a0b36df2d7e6b91e4946c03b3473 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/224/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0224 (3-changed-components) 2024-07-22T02:17:56 0:00:00
Previous 0223 (4-changed-components) 2024-07-19T21:09:25 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 51 64
Performance Improvements 74 134
Unchanged Tests 1111 1038
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
434.zeusmp (unexplained diff vs previous) (percentage) 1885.71% 0.070 1.390 0.000 -38.22% 0.000
434.zeusmp (unexplained diff vs previous) 1800.00% 2.000 38.000 0.000 -91.91% 0.000
445.gobmk (unexplained diff vs previous) 1150.00% 4.000 50.000 0.000 177.78% 0.000
445.gobmk (unexplained diff vs previous) (percentage) 1150.00% 0.080 1.000 0.000 566.67% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 1127.78% 0.180 2.210 0.000 229.85% 0.000
471.omnetpp (unexplained diff vs previous) 1100.00% 5.000 60.000 0.000 -3.23% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) 593.10% 0.580 4.020 0.000 290.29% 0.000
437.leslie3d (unexplained diff vs previous) 567.16% 67.000 447.000 0.000 139.04% 0.000
473.astar (unexplained diff vs previous) (percentage) 565.15% 0.660 4.390 0.000 834.04% 0.000
473.astar (unexplained diff vs previous) 540.62% 32.000 205.000 0.000 236.07% 0.000
473.astar (max) (percentage) 528.36% 0.670 4.210 - 795.74% -
473.astar (max) 512.50% 32.000 196.000 - 221.31% -
435.gromacs (unexplained diff vs previous) (percentage) 467.74% 0.620 3.520 0.000 103.47% 0.000
435.gromacs (unexplained diff vs previous) 453.19% 47.000 260.000 0.000 -10.96% 0.000
433.milc (unexplained diff vs previous) (percentage) 218.75% 0.640 2.040 0.000 191.43% 0.000
433.milc (unexplained diff vs previous) 214.29% 7.000 22.000 0.000 -77.08% 0.000
434.zeusmp (max) 146.67% 15.000 37.000 - -92.31% -
434.zeusmp (max) (percentage) 140.35% 0.570 1.370 - -40.43% -
450.soplex (unexplained diff vs previous) (percentage) 134.53% 2.230 5.230 0.000 1088.64% 0.000
429.mcf (unexplained diff vs previous) (percentage) 131.02% 3.740 8.640 0.000 9500.00% 0.000
450.soplex (unexplained diff vs previous) 123.08% 13.000 29.000 0.000 -57.97% 0.000
429.mcf (unexplained diff vs previous) 112.50% 48.000 102.000 0.000 920.00% 0.000
444.namd (unexplained diff vs previous) (percentage) 102.27% 0.440 0.890 0.000 -82.98% 0.000
444.namd (unexplained diff vs previous) 100.00% 4.000 8.000 0.000 -99.04% 0.000
470.lbm (unexplained diff vs previous) (percentage) 85.48% 0.620 1.150 0.000 -77.18% 0.000
470.lbm (unexplained diff vs previous) 82.61% 23.000 42.000 0.000 -94.89% 0.000
435.gromacs (max) (percentage) 80.85% 1.880 3.400 - 93.18% -
435.gromacs (max) 74.83% 143.000 250.000 - -15.54% -
433.milc (max) (percentage) 63.93% 1.220 2.000 - 181.69% -
433.milc (max) 61.54% 13.000 21.000 - -78.35% -
429.mcf (max) (percentage) 32.28% 6.010 7.950 - 2384.38% -
482.sphinx3 (max) (percentage) 25.99% 2.270 2.860 - -9.21% -
482.sphinx3 (max) 25.00% 20.000 25.000 - -97.83% -
429.mcf (max) 20.78% 77.000 93.000 - 151.35% -
444.namd (max) (percentage) 18.92% 0.740 0.880 - -84.03% -
444.namd (max) 16.67% 6.000 7.000 - -99.20% -
458.sjeng (unexplained diff vs previous) (percentage) 12.00% 0.250 0.280 0.000 -6.67% 0.000
458.sjeng (unexplained diff vs previous) 10.53% 19.000 21.000 0.000 -56.25% 0.000
447.dealII (unexplained diff vs previous) (percentage) 6.73% 1.040 1.110 0.000 48.00% 0.000
447.dealII (unexplained diff vs previous) 5.00% 20.000 21.000 0.000 -69.12% 0.000
401.bzip2 (max) (percentage) 3.85% 2.080 2.160 - 440.00% -
447.dealII (max) (percentage) 3.81% 1.050 1.090 - -33.94% -
401.bzip2 (unexplained diff vs previous) (percentage) 3.77% 2.120 2.200 0.000 436.59% 0.000
473.astar (avg) (percentage) 2.52% 1.590 1.630 - 307.50% -
445.gobmk (max) (percentage) 2.06% 0.970 0.990 - 266.67% -
401.bzip2 (max) 1.69% 59.000 60.000 - -10.45% -
435.gromacs (avg) (percentage) 1.68% 1.790 1.820 - 91.58% -
401.bzip2 (unexplained diff vs previous) 1.64% 61.000 62.000 0.000 -10.14% 0.000
450.soplex (avg) (percentage) 1.44% 2.770 2.810 - 1070.83% -
482.sphinx3 (avg) (percentage) 1.23% 1.620 1.640 - -30.21% -
429.mcf (avg) (percentage) 1.07% 4.660 4.710 - 881.25% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf -7.95% 1283.000 1181.000 - -89.93% -
450.soplex -4.97% 584.000 555.000 - -96.47% -
473.astar -4.21% 4875.000 4670.000 - -64.03% -
437.leslie3d -3.86% 11576.000 11129.000 - -38.41% -
435.gromacs -3.40% 7642.000 7382.000 - -56.20% -
482.sphinx3 -2.86% 908.000 882.000 - -97.59% -
471.omnetpp -2.16% 2777.000 2717.000 - -70.67% -
401.bzip2 -2.16% 2875.000 2813.000 - -83.42% -
433.milc -2.00% 1098.000 1076.000 - -92.13% -
410.bwaves -1.80% 5987.000 5879.000 - -60.13% -
465.tonto -1.52% 12912.000 12716.000 - -2.51% -
459.GemsFDTD -1.48% 7049.000 6945.000 - -72.62% -
434.zeusmp -1.37% 2776.000 2738.000 - -86.92% -
470.lbm -1.14% 3690.000 3648.000 - -77.64% -
453.povray -1.11% 542.000 536.000 - -92.75% -
447.dealII -1.09% 1921.000 1900.000 - -79.12% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) -66.67% 3.000 1.000 - -92.31% -
453.povray (max) -58.33% 12.000 5.000 - -97.38% -
453.povray (max) (percentage) -50.67% 2.250 1.110 - -57.14% -
462.libquantum (unexplained diff vs previous) -50.00% 2.000 1.000 0.000 -92.86% 0.000
462.libquantum (unexplained diff vs previous) (percentage) -50.00% 0.940 0.470 0.000 487.50% 0.000
462.libquantum (max) (percentage) -46.98% 1.490 0.790 - 887.50% -
400.perlbench (max) -14.29% 14.000 12.000 - -86.52% -
465.tonto (max) -10.19% 108.000 97.000 - -56.50% -
400.perlbench (max) (percentage) -9.57% 0.940 0.850 - 6.25% -
450.soplex (max) -9.09% 33.000 30.000 - -56.52% -
471.omnetpp (max) -8.60% 93.000 85.000 - -50.58% -
458.sjeng (max) (percentage) -8.57% 0.350 0.320 - 6.67% -
465.tonto (max) (percentage) -8.33% 0.840 0.770 - -54.97% -
458.sjeng (max) -7.69% 26.000 24.000 - -50.00% -
433.milc -7.14% 14.000 13.000 - -86.17% -
433.milc (avg) -7.14% 14.000 13.000 - -86.17% -
429.mcf -6.78% 59.000 55.000 - -1.79% -
429.mcf (avg) -6.78% 59.000 55.000 - -1.79% -
450.soplex -6.25% 16.000 15.000 - -59.46% -
450.soplex (avg) -6.25% 16.000 15.000 - -59.46% -
471.omnetpp (max) (percentage) -6.25% 3.360 3.150 - 69.35% -
464.h264ref (max) -5.13% 39.000 37.000 - -37.29% -
456.hmmer (max) -5.00% 20.000 19.000 - -80.61% -
464.h264ref (max) (percentage) -4.69% 0.640 0.610 - 56.41% -
437.leslie3d -4.41% 544.000 520.000 - 72.76% -
437.leslie3d (avg) -4.41% 544.000 520.000 - 72.76% -
410.bwaves (max) -4.29% 70.000 67.000 - -74.23% -
456.hmmer (max) (percentage) -4.17% 0.480 0.460 - -11.54% -
459.GemsFDTD (max) -4.12% 97.000 93.000 - -83.91% -
400.perlbench -4.00% 25.000 24.000 - -44.19% -
400.perlbench (avg) -4.00% 25.000 24.000 - -44.19% -
470.lbm (max) -4.00% 50.000 48.000 - -94.62% -
437.leslie3d (max) -3.71% 512.000 493.000 - -11.96% -
450.soplex (max) (percentage) -3.63% 5.790 5.580 - 1168.18% -
459.GemsFDTD (max) (percentage) -3.60% 1.390 1.340 - -41.23% -
436.cactusADM (max) (percentage) -3.57% 0.280 0.270 - -94.65% -
401.bzip2 -3.57% 56.000 54.000 - 20.00% -
401.bzip2 (avg) -3.57% 56.000 54.000 - 20.00% -
410.bwaves (max) (percentage) -3.39% 1.180 1.140 - -35.59% -
410.bwaves -3.12% 64.000 62.000 - -78.55% -
410.bwaves (avg) -3.12% 64.000 62.000 - -78.55% -
470.lbm (max) (percentage) -2.90% 1.380 1.340 - -75.55% -
459.GemsFDTD -2.17% 92.000 90.000 - -74.43% -
459.GemsFDTD (avg) -2.17% 92.000 90.000 - -74.43% -
470.lbm -2.17% 46.000 45.000 - -92.60% -
470.lbm (avg) -2.17% 46.000 45.000 - -92.60% -
445.gobmk -2.08% 48.000 47.000 - 74.07% -
445.gobmk (avg) -2.08% 48.000 47.000 - 74.07% -
464.h264ref -1.96% 51.000 50.000 - -15.25% -
464.h264ref (avg) -1.96% 51.000 50.000 - -15.25% -
471.omnetpp -1.92% 52.000 51.000 - -53.21% -
471.omnetpp (avg) -1.92% 52.000 51.000 - -53.21% -
456.hmmer (avg) (percentage) -1.61% 0.620 0.610 - 103.33% -
435.gromacs -1.47% 136.000 134.000 - -16.25% -
435.gromacs (avg) -1.47% 136.000 134.000 - -16.25% -
473.astar -1.30% 77.000 76.000 - 49.02% -
473.astar (avg) -1.30% 77.000 76.000 - 49.02% -
462.libquantum (avg) (percentage) -1.29% 1.550 1.530 - 705.26% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.86% 1437761.000 1464473.000 - 0.00% -
473.astar 1.59% 15549.000 15797.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (unexplained diff vs previous) (percentage) 9500.00% 0.090 8.640 0.000 131.02% 0.000
429.mcf (max) (percentage) 2384.38% 0.320 7.950 - 32.28% -
403.gcc (max) (percentage) 1203.39% 0.590 7.690 - -0.39% -
450.soplex (max) (percentage) 1168.18% 0.440 5.580 - -3.63% -
450.soplex (unexplained diff vs previous) (percentage) 1088.64% 0.440 5.230 0.000 134.53% 0.000
450.soplex (avg) (percentage) 1070.83% 0.240 2.810 - 1.44% -
429.mcf (unexplained diff vs previous) 920.00% 10.000 102.000 0.000 112.50% 0.000
462.libquantum (max) (percentage) 887.50% 0.080 0.790 - -46.98% -
429.mcf (avg) (percentage) 881.25% 0.480 4.710 - 1.07% -
473.astar (unexplained diff vs previous) (percentage) 834.04% 0.470 4.390 0.000 565.15% 0.000
473.astar (max) (percentage) 795.74% 0.470 4.210 - 528.36% -
462.libquantum (avg) (percentage) 705.26% 0.190 1.530 - -1.29% -
401.bzip2 (avg) (percentage) 622.22% 0.270 1.950 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) 566.67% 0.150 1.000 0.000 1150.00% 0.000
462.libquantum (unexplained diff vs previous) (percentage) 487.50% 0.080 0.470 0.000 -50.00% 0.000
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
401.bzip2 (max) (percentage) 440.00% 0.400 2.160 - 3.85% -
401.bzip2 (unexplained diff vs previous) (percentage) 436.59% 0.410 2.200 0.000 3.77% 0.000
445.gobmk (avg) (percentage) 331.82% 0.220 0.950 - 0.00% -
400.perlbench (avg) (percentage) 323.08% 0.390 1.650 - 0.00% -
454.calculix (max) (percentage) 316.67% 0.840 3.500 - 0.00% -
473.astar (avg) (percentage) 307.50% 0.400 1.630 - 2.52% -
437.leslie3d (unexplained diff vs previous) (percentage) 290.29% 1.030 4.020 0.000 593.10% 0.000
445.gobmk (max) (percentage) 266.67% 0.270 0.990 - 2.06% -
473.astar (unexplained diff vs previous) 236.07% 61.000 205.000 0.000 540.62% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 229.85% 0.670 2.210 0.000 1127.78% 0.000
458.sjeng (avg) (percentage) 221.74% 0.230 0.740 - 0.00% -
473.astar (max) 221.31% 61.000 196.000 - 512.50% -
433.milc (unexplained diff vs previous) (percentage) 191.43% 0.700 2.040 0.000 218.75% 0.000
433.milc (max) (percentage) 181.69% 0.710 2.000 - 63.93% -
437.leslie3d (avg) (percentage) 180.24% 1.670 4.680 - -0.43% -
445.gobmk (unexplained diff vs previous) 177.78% 18.000 50.000 0.000 1150.00% 0.000
429.mcf (max) 151.35% 37.000 93.000 - 20.78% -
437.leslie3d (unexplained diff vs previous) 139.04% 187.000 447.000 0.000 567.16% 0.000
464.h264ref (avg) (percentage) 115.38% 0.390 0.840 - 0.00% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) 103.47% 1.730 3.520 0.000 467.74% 0.000
456.hmmer (avg) (percentage) 103.33% 0.300 0.610 - -1.61% -
435.gromacs (max) (percentage) 93.18% 1.760 3.400 - 80.85% -
435.gromacs (avg) (percentage) 91.58% 0.950 1.820 - 1.68% -
433.milc (avg) (percentage) 86.96% 0.690 1.290 - 0.78% -
447.dealII (avg) (percentage) 81.61% 0.870 1.580 - -0.63% -
445.gobmk 74.07% 27.000 47.000 - -2.08% -
445.gobmk (avg) 74.07% 27.000 47.000 - -2.08% -
437.leslie3d 72.76% 301.000 520.000 - -4.41% -
437.leslie3d (avg) 72.76% 301.000 520.000 - -4.41% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
471.omnetpp (max) (percentage) 69.35% 1.860 3.150 - -6.25% -
483.xalancbmk (max) (percentage) 65.22% 0.920 1.520 - 0.00% -
471.omnetpp (avg) (percentage) 59.32% 1.180 1.880 - 0.00% -
464.h264ref (max) (percentage) 56.41% 0.390 0.610 - -4.69% -
458.sjeng 55.56% 36.000 56.000 - 0.00% -
458.sjeng (avg) 55.56% 36.000 56.000 - 0.00% -
473.astar 49.02% 51.000 76.000 - -1.30% -
473.astar (avg) 49.02% 51.000 76.000 - -1.30% -
445.gobmk (max) 48.48% 33.000 49.000 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) 48.00% 0.750 1.110 0.000 6.73% 0.000
437.leslie3d (max) (percentage) 42.90% 3.100 4.430 - 0.00% -
401.bzip2 20.00% 45.000 54.000 - -3.57% -
401.bzip2 (avg) 20.00% 45.000 54.000 - -3.57% -
458.sjeng (max) (percentage) 6.67% 0.300 0.320 - -8.57% -
400.perlbench (max) (percentage) 6.25% 0.800 0.850 - -9.57% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.83% 10845.000 10213.000 - 0.00% -
437.leslie3d -3.71% 48001.000 46220.000 - 0.00% -
434.zeusmp -1.82% 141060.000 138490.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.74% 17005.000 214.000 - 0.47% -
482.sphinx3 -97.59% 36586.000 882.000 - -2.86% -
450.soplex -96.47% 15709.000 555.000 - -4.97% -
444.namd -94.37% 15957.000 898.000 - -0.88% -
453.povray -92.75% 7395.000 536.000 - -1.11% -
433.milc -92.13% 13668.000 1076.000 - -2.00% -
429.mcf -89.93% 11727.000 1181.000 - -7.95% -
436.cactusADM -89.23% 24396.000 2627.000 - 0.54% -
434.zeusmp -86.92% 20932.000 2738.000 - -1.37% -
400.perlbench -86.51% 11215.000 1513.000 - -0.59% -
401.bzip2 -83.42% 16971.000 2813.000 - -2.16% -
447.dealII -79.12% 9101.000 1900.000 - -1.09% -
470.lbm -77.64% 16312.000 3648.000 - -1.14% -
456.hmmer -77.36% 18995.000 4301.000 - -0.14% -
459.GemsFDTD -72.62% 25364.000 6945.000 - -1.48% -
471.omnetpp -70.67% 9265.000 2717.000 - -2.16% -
473.astar -64.03% 12983.000 4670.000 - -4.21% -
410.bwaves -60.13% 14745.000 5879.000 - -1.80% -
464.h264ref -60.11% 15208.000 6066.000 - -0.83% -
445.gobmk -59.49% 12396.000 5021.000 - -0.99% -
435.gromacs -56.20% 16853.000 7382.000 - -3.40% -
458.sjeng -52.60% 16018.000 7593.000 - -0.28% -
437.leslie3d -38.41% 18070.000 11129.000 - -3.86% -
465.tonto -2.51% 13044.000 12716.000 - -1.52% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (max) -99.43% 1231.000 7.000 - 0.00% -
444.namd (max) -99.20% 879.000 7.000 - 16.67% -
444.namd (unexplained diff vs previous) -99.04% 834.000 8.000 0.000 100.00% 0.000
482.sphinx3 -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (avg) -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (max) -97.83% 1152.000 25.000 - 25.00% -
482.sphinx3 (unexplained diff vs previous) -97.67% 1118.000 26.000 0.000 0.00% 0.000
453.povray (max) -97.38% 191.000 5.000 - -58.33% -
444.namd -97.34% 414.000 11.000 - 0.00% -
444.namd (avg) -97.34% 414.000 11.000 - 0.00% -
453.povray (unexplained diff vs previous) -96.45% 169.000 6.000 0.000 0.00% 0.000
416.gamess (max) (percentage) -95.86% 3.140 0.130 - 0.00% -
436.cactusADM -95.38% 585.000 27.000 - 0.00% -
436.cactusADM (avg) -95.38% 585.000 27.000 - 0.00% -
470.lbm (unexplained diff vs previous) -94.89% 822.000 42.000 0.000 82.61% 0.000
436.cactusADM (max) (percentage) -94.65% 5.050 0.270 - -3.57% -
470.lbm (max) -94.62% 893.000 48.000 - -4.00% -
456.hmmer (unexplained diff vs previous) -93.88% 98.000 6.000 0.000 0.00% 0.000
453.povray -93.81% 113.000 7.000 - 0.00% -
453.povray (avg) -93.81% 113.000 7.000 - 0.00% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
462.libquantum (unexplained diff vs previous) -92.86% 14.000 1.000 0.000 -50.00% 0.000
470.lbm -92.60% 608.000 45.000 - -2.17% -
470.lbm (avg) -92.60% 608.000 45.000 - -2.17% -
434.zeusmp (max) -92.31% 481.000 37.000 - 146.67% -
462.libquantum (max) -92.31% 13.000 1.000 - -66.67% -
434.zeusmp (unexplained diff vs previous) -91.91% 470.000 38.000 0.000 1800.00% 0.000
462.libquantum -90.62% 32.000 3.000 - 0.00% -
462.libquantum (avg) -90.62% 32.000 3.000 - 0.00% -
434.zeusmp -88.24% 255.000 30.000 - 0.00% -
434.zeusmp (avg) -88.24% 255.000 30.000 - 0.00% -
447.dealII (max) -86.67% 150.000 20.000 - 0.00% -
400.perlbench (max) -86.52% 89.000 12.000 - -14.29% -
433.milc -86.17% 94.000 13.000 - -7.14% -
433.milc (avg) -86.17% 94.000 13.000 - -7.14% -
444.namd (max) (percentage) -84.03% 5.510 0.880 - 18.92% -
459.GemsFDTD (max) -83.91% 578.000 93.000 - -4.12% -
444.namd (unexplained diff vs previous) (percentage) -82.98% 5.230 0.890 0.000 102.27% 0.000
456.hmmer (max) -80.61% 98.000 19.000 - -5.00% -
410.bwaves -78.55% 289.000 62.000 - -3.12% -
410.bwaves (avg) -78.55% 289.000 62.000 - -3.12% -
433.milc (max) -78.35% 97.000 21.000 - 61.54% -
470.lbm (unexplained diff vs previous) (percentage) -77.18% 5.040 1.150 0.000 85.48% 0.000
433.milc (unexplained diff vs previous) -77.08% 96.000 22.000 0.000 214.29% 0.000
470.lbm (max) (percentage) -75.55% 5.480 1.340 - -2.90% -
459.GemsFDTD -74.43% 352.000 90.000 - -2.17% -
459.GemsFDTD (avg) -74.43% 352.000 90.000 - -2.17% -
410.bwaves (max) -74.23% 260.000 67.000 - -4.29% -
456.hmmer (unexplained diff vs previous) (percentage) -73.08% 0.520 0.140 0.000 0.00% 0.000
447.dealII (unexplained diff vs previous) -69.12% 68.000 21.000 0.000 5.00% 0.000
470.lbm (avg) (percentage) -66.49% 3.730 1.250 - 0.00% -
447.dealII -62.03% 79.000 30.000 - 0.00% -
447.dealII (avg) -62.03% 79.000 30.000 - 0.00% -
450.soplex -59.46% 37.000 15.000 - -6.25% -
450.soplex (avg) -59.46% 37.000 15.000 - -6.25% -
450.soplex (unexplained diff vs previous) -57.97% 69.000 29.000 0.000 123.08% 0.000
453.povray (max) (percentage) -57.14% 2.590 1.110 - -50.67% -
450.soplex (max) -56.52% 69.000 30.000 - -9.09% -
465.tonto (max) -56.50% 223.000 97.000 - -10.19% -
458.sjeng (unexplained diff vs previous) -56.25% 48.000 21.000 0.000 10.53% 0.000
436.cactusADM (avg) (percentage) -55.83% 2.400 1.060 - 0.00% -
465.tonto (max) (percentage) -54.97% 1.710 0.770 - -8.33% -
456.hmmer -53.57% 56.000 26.000 - 0.00% -
456.hmmer (avg) -53.57% 56.000 26.000 - 0.00% -
471.omnetpp -53.21% 109.000 51.000 - -1.92% -
471.omnetpp (avg) -53.21% 109.000 51.000 - -1.92% -
444.namd (avg) (percentage) -52.69% 2.600 1.230 - -0.81% -
453.povray (unexplained diff vs previous) (percentage) -51.09% 2.290 1.120 0.000 0.90% 0.000
471.omnetpp (max) -50.58% 172.000 85.000 - -8.60% -
458.sjeng (max) -50.00% 48.000 24.000 - -7.69% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
410.bwaves (avg) (percentage) -45.41% 1.960 1.070 - 0.00% -
400.perlbench -44.19% 43.000 24.000 - -4.00% -
400.perlbench (avg) -44.19% 43.000 24.000 - -4.00% -
459.GemsFDTD (max) (percentage) -41.23% 2.280 1.340 - -3.60% -
434.zeusmp (max) (percentage) -40.43% 2.300 1.370 - 140.35% -
434.zeusmp (unexplained diff vs previous) (percentage) -38.22% 2.250 1.390 0.000 1885.71% 0.000
464.h264ref (max) -37.29% 59.000 37.000 - -5.13% -
410.bwaves (max) (percentage) -35.59% 1.770 1.140 - -3.39% -
447.dealII (max) (percentage) -33.94% 1.650 1.090 - 3.81% -
482.sphinx3 (avg) (percentage) -30.21% 2.350 1.640 - 1.23% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
465.tonto -23.65% 148.000 113.000 - -0.88% -
465.tonto (avg) -23.65% 148.000 113.000 - -0.88% -
465.tonto (avg) (percentage) -21.93% 1.140 0.890 - 0.00% -
435.gromacs -16.25% 160.000 134.000 - -1.47% -
435.gromacs (avg) -16.25% 160.000 134.000 - -1.47% -
435.gromacs (max) -15.54% 296.000 250.000 - 74.83% -
464.h264ref -15.25% 59.000 50.000 - -1.96% -
464.h264ref (avg) -15.25% 59.000 50.000 - -1.96% -
437.leslie3d (max) -11.96% 560.000 493.000 - -3.71% -
456.hmmer (max) (percentage) -11.54% 0.520 0.460 - -4.17% -
435.gromacs (unexplained diff vs previous) -10.96% 292.000 260.000 0.000 453.19% 0.000
401.bzip2 (max) -10.45% 67.000 60.000 - 1.69% -
401.bzip2 (unexplained diff vs previous) -10.14% 69.000 62.000 0.000 1.64% 0.000
434.zeusmp (avg) (percentage) -9.84% 1.220 1.100 - 0.92% -
482.sphinx3 (max) (percentage) -9.21% 3.150 2.860 - 25.99% -
458.sjeng (unexplained diff vs previous) (percentage) -6.67% 0.300 0.280 0.000 12.00% 0.000
459.GemsFDTD (avg) (percentage) -5.76% 1.390 1.310 - 0.00% -
453.povray (avg) (percentage) -4.58% 1.530 1.460 - -0.68% -
482.sphinx3 (unexplained diff vs previous) (percentage) -3.59% 3.060 2.950 0.000 0.00% 0.000
471.omnetpp (unexplained diff vs previous) -3.23% 62.000 60.000 0.000 1100.00% 0.000
429.mcf -1.79% 56.000 55.000 - -6.78% -
429.mcf (avg) -1.79% 56.000 55.000 - -6.78% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 0.00% -
nb_compile_successful 65.52% 29.000 48.000 - 0.00% -
nb_execution_successful 65.52% 29.000 48.000 - 0.00% -

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1513.000 -
401.bzip2 2813.000 -2.16%
410.bwaves 5879.000 -1.80%
429.mcf 1181.000 -7.95%
433.milc 1076.000 -2.00%
434.zeusmp 2738.000 -1.37%
435.gromacs 7382.000 -3.40%
436.cactusADM 2627.000 -
437.leslie3d 11129.000 -3.86%
444.namd 898.000 -
445.gobmk 5021.000 -
447.dealII 1900.000 -1.09%
450.soplex 555.000 -4.97%
453.povray 536.000 -1.11%
456.hmmer 4301.000 -
458.sjeng 7593.000 -
459.GemsFDTD 6945.000 -1.48%
462.libquantum 214.000 -
464.h264ref 6066.000 -
465.tonto 12716.000 -1.52%
470.lbm 3648.000 -1.14%
471.omnetpp 2717.000 -2.16%
473.astar 4670.000 -4.21%
482.sphinx3 882.000 -2.86%
Geometric Mean 2576.427 -1.93%
Code Size (bytes)
Name Current %
400.perlbench 561966.000 -
401.bzip2 33454.000 -
410.bwaves 14723.000 -
429.mcf 7693.000 -
433.milc 47741.000 -
434.zeusmp 138490.000 -
435.gromacs 392922.000 -
436.cactusADM 256498.000 -
437.leslie3d 46220.000 -
444.namd 163075.000 -
445.gobmk 1087328.000 -
447.dealII 190779.000 -
450.soplex 135756.000 -
453.povray 493593.000 -
456.hmmer 74389.000 -
458.sjeng 81341.000 -
459.GemsFDTD 194147.000 -
462.libquantum 13126.000 -
464.h264ref 300626.000 -
465.tonto 1464473.000 -
470.lbm 10213.000 -
471.omnetpp 296045.000 -
473.astar 15797.000 -
482.sphinx3 79550.000 -
Geometric Mean 105759.872 -
sample variability
Name Current %
400.perlbench 24.000 -4.00%
400.perlbench (avg) 24.000 -4.00%
400.perlbench (avg) (percentage) 1.650 -
400.perlbench (max) 12.000 -14.29%
400.perlbench (max) (percentage) 0.850 -9.57%
401.bzip2 54.000 -3.57%
401.bzip2 (avg) 54.000 -3.57%
401.bzip2 (avg) (percentage) 1.950 -
401.bzip2 (max) 60.000 1.69%
401.bzip2 (max) (percentage) 2.160 3.85%
401.bzip2 (unexplained diff vs previous) 62.000 1.64%
401.bzip2 (unexplained diff vs previous) (percentage) 2.200 3.77%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 7.690 -
410.bwaves 62.000 -3.12%
410.bwaves (avg) 62.000 -3.12%
410.bwaves (avg) (percentage) 1.070 -
410.bwaves (max) 67.000 -4.29%
410.bwaves (max) (percentage) 1.140 -3.39%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.130 -
429.mcf 55.000 -6.78%
429.mcf (avg) 55.000 -6.78%
429.mcf (avg) (percentage) 4.710 1.07%
429.mcf (max) 93.000 20.78%
429.mcf (max) (percentage) 7.950 32.28%
429.mcf (unexplained diff vs previous) 102.000 112.50%
429.mcf (unexplained diff vs previous) (percentage) 8.640 131.02%
433.milc 13.000 -7.14%
433.milc (avg) 13.000 -7.14%
433.milc (avg) (percentage) 1.290 -
433.milc (max) 21.000 61.54%
433.milc (max) (percentage) 2.000 63.93%
433.milc (unexplained diff vs previous) 22.000 214.29%
433.milc (unexplained diff vs previous) (percentage) 2.040 218.75%
434.zeusmp 30.000 -
434.zeusmp (avg) 30.000 -
434.zeusmp (avg) (percentage) 1.100 -
434.zeusmp (max) 37.000 146.67%
434.zeusmp (max) (percentage) 1.370 140.35%
434.zeusmp (unexplained diff vs previous) 38.000 1800.00%
434.zeusmp (unexplained diff vs previous) (percentage) 1.390 1885.71%
435.gromacs 134.000 -1.47%
435.gromacs (avg) 134.000 -1.47%
435.gromacs (avg) (percentage) 1.820 1.68%
435.gromacs (max) 250.000 74.83%
435.gromacs (max) (percentage) 3.400 80.85%
435.gromacs (unexplained diff vs previous) 260.000 453.19%
435.gromacs (unexplained diff vs previous) (percentage) 3.520 467.74%
436.cactusADM 27.000 -
436.cactusADM (avg) 27.000 -
436.cactusADM (avg) (percentage) 1.060 -
436.cactusADM (max) 7.000 -
436.cactusADM (max) (percentage) 0.270 -3.57%
437.leslie3d 520.000 -4.41%
437.leslie3d (avg) 520.000 -4.41%
437.leslie3d (avg) (percentage) 4.680 -
437.leslie3d (max) 493.000 -3.71%
437.leslie3d (max) (percentage) 4.430 -
437.leslie3d (unexplained diff vs previous) 447.000 567.16%
437.leslie3d (unexplained diff vs previous) (percentage) 4.020 593.10%
444.namd 11.000 -
444.namd (avg) 11.000 -
444.namd (avg) (percentage) 1.230 -
444.namd (max) 7.000 16.67%
444.namd (max) (percentage) 0.880 18.92%
444.namd (unexplained diff vs previous) 8.000 100.00%
444.namd (unexplained diff vs previous) (percentage) 0.890 102.27%
445.gobmk 47.000 -2.08%
445.gobmk (avg) 47.000 -2.08%
445.gobmk (avg) (percentage) 0.950 -
445.gobmk (max) 49.000 -
445.gobmk (max) (percentage) 0.990 2.06%
445.gobmk (unexplained diff vs previous) 50.000 1150.00%
445.gobmk (unexplained diff vs previous) (percentage) 1.000 1150.00%
447.dealII 30.000 -
447.dealII (avg) 30.000 -
447.dealII (avg) (percentage) 1.580 -
447.dealII (max) 20.000 -
447.dealII (max) (percentage) 1.090 3.81%
447.dealII (unexplained diff vs previous) 21.000 5.00%
447.dealII (unexplained diff vs previous) (percentage) 1.110 6.73%
450.soplex 15.000 -6.25%
450.soplex (avg) 15.000 -6.25%
450.soplex (avg) (percentage) 2.810 1.44%
450.soplex (max) 30.000 -9.09%
450.soplex (max) (percentage) 5.580 -3.63%
450.soplex (unexplained diff vs previous) 29.000 123.08%
450.soplex (unexplained diff vs previous) (percentage) 5.230 134.53%
453.povray 7.000 -
453.povray (avg) 7.000 -
453.povray (avg) (percentage) 1.460 -
453.povray (max) 5.000 -58.33%
453.povray (max) (percentage) 1.110 -50.67%
453.povray (unexplained diff vs previous) 6.000 -
453.povray (unexplained diff vs previous) (percentage) 1.120 -
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.500 -
456.hmmer 26.000 -
456.hmmer (avg) 26.000 -
456.hmmer (avg) (percentage) 0.610 -1.61%
456.hmmer (max) 19.000 -5.00%
456.hmmer (max) (percentage) 0.460 -4.17%
456.hmmer (unexplained diff vs previous) 6.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.140 -
458.sjeng 56.000 -
458.sjeng (avg) 56.000 -
458.sjeng (avg) (percentage) 0.740 -
458.sjeng (max) 24.000 -7.69%
458.sjeng (max) (percentage) 0.320 -8.57%
458.sjeng (unexplained diff vs previous) 21.000 10.53%
458.sjeng (unexplained diff vs previous) (percentage) 0.280 12.00%
459.GemsFDTD 90.000 -2.17%
459.GemsFDTD (avg) 90.000 -2.17%
459.GemsFDTD (avg) (percentage) 1.310 -
459.GemsFDTD (max) 93.000 -4.12%
459.GemsFDTD (max) (percentage) 1.340 -3.60%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 1.530 -1.29%
462.libquantum (max) 1.000 -66.67%
462.libquantum (max) (percentage) 0.790 -46.98%
462.libquantum (unexplained diff vs previous) 1.000 -50.00%
462.libquantum (unexplained diff vs previous) (percentage) 0.470 -50.00%
464.h264ref 50.000 -1.96%
464.h264ref (avg) 50.000 -1.96%
464.h264ref (avg) (percentage) 0.840 -
464.h264ref (max) 37.000 -5.13%
464.h264ref (max) (percentage) 0.610 -4.69%
465.tonto 113.000 -
465.tonto (avg) 113.000 -
465.tonto (avg) (percentage) 0.890 -
465.tonto (max) 97.000 -10.19%
465.tonto (max) (percentage) 0.770 -8.33%
470.lbm 45.000 -2.17%
470.lbm (avg) 45.000 -2.17%
470.lbm (avg) (percentage) 1.250 -
470.lbm (max) 48.000 -4.00%
470.lbm (max) (percentage) 1.340 -2.90%
470.lbm (unexplained diff vs previous) 42.000 82.61%
470.lbm (unexplained diff vs previous) (percentage) 1.150 85.48%
471.omnetpp 51.000 -1.92%
471.omnetpp (avg) 51.000 -1.92%
471.omnetpp (avg) (percentage) 1.880 -
471.omnetpp (max) 85.000 -8.60%
471.omnetpp (max) (percentage) 3.150 -6.25%
471.omnetpp (unexplained diff vs previous) 60.000 1100.00%
471.omnetpp (unexplained diff vs previous) (percentage) 2.210 1127.78%
473.astar 76.000 -1.30%
473.astar (avg) 76.000 -1.30%
473.astar (avg) (percentage) 1.630 2.52%
473.astar (max) 196.000 512.50%
473.astar (max) (percentage) 4.210 528.36%
473.astar (unexplained diff vs previous) 205.000 540.62%
473.astar (unexplained diff vs previous) (percentage) 4.390 565.15%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.640 1.23%
482.sphinx3 (max) 25.000 25.00%
482.sphinx3 (max) (percentage) 2.860 25.99%
482.sphinx3 (unexplained diff vs previous) 26.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 2.950 -
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.520 -
Geometric Mean 8.149 60.38%