Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=f2869f51b2ee7fc9357c2030add11435d7f1beef (+ 44 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/471fb09260179fd59d98de9f3d54b29c17232fb6 (+ 152 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=5b54a33435e5533653a9956728f2de9d16a3b4ee (+ 8 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=aaecd681f60fa02c7697198b308336a13b231f30 (+ 372 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/205/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0205 (4-changed-components) 2024-06-12T06:03:10 0:00:00
Previous 0204 (3-changed-components) 2024-06-09T11:49:51 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 24 60
Performance Improvements 106 148
Added Tests 12 0
Unchanged Tests 1094 1028
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
400.perlbench 2.06% 1504.000 1535.000 - -86.31% -
462.libquantum 1.85% 216.000 220.000 - -98.71% -
429.mcf 1.50% 1267.000 1286.000 - -89.03% -
435.gromacs 1.06% 7456.000 7535.000 - -55.29% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (unexplained diff vs previous) 350.00% 2.000 9.000 0.000 -86.96% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) 342.86% 0.070 0.310 0.000 -24.39% 0.000
471.omnetpp (unexplained diff vs previous) 216.67% 6.000 19.000 0.000 -69.35% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 213.64% 0.220 0.690 0.000 2.99% 0.000
433.milc (unexplained diff vs previous) (percentage) 200.00% 0.090 0.270 0.000 -61.43% 0.000
444.namd (unexplained diff vs previous) (percentage) 200.00% 0.110 0.330 0.000 -93.69% 0.000
433.milc (unexplained diff vs previous) 200.00% 1.000 3.000 0.000 -96.88% 0.000
444.namd (unexplained diff vs previous) 200.00% 1.000 3.000 0.000 -99.64% 0.000
434.zeusmp (unexplained diff vs previous) 166.67% 3.000 8.000 0.000 -98.30% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) 163.64% 0.110 0.290 0.000 -87.11% 0.000
447.dealII (unexplained diff vs previous) 90.00% 10.000 19.000 0.000 -72.06% 0.000
447.dealII (unexplained diff vs previous) (percentage) 90.00% 0.500 0.950 0.000 26.67% 0.000
436.cactusADM (unexplained diff vs previous) 37.50% 8.000 11.000 0.000 -99.06% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) 35.48% 0.310 0.420 0.000 -91.27% 0.000
462.libquantum (unexplained diff vs previous) 33.33% 3.000 4.000 0.000 -71.43% 0.000
462.libquantum (unexplained diff vs previous) (percentage) 30.94% 1.390 1.820 0.000 2175.00% 0.000
400.perlbench 3.70% 27.000 28.000 - -34.88% -
400.perlbench (avg) 3.70% 27.000 28.000 - -34.88% -
435.gromacs (unexplained diff vs previous) 2.60% 77.000 79.000 0.000 -72.95% 0.000
435.gromacs (unexplained diff vs previous) (percentage) 1.94% 1.030 1.050 0.000 -39.31% 0.000

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -2.68% 11921.000 11602.000 - -35.79% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
459.GemsFDTD (max) -84.14% 145.000 23.000 - -96.02% -
458.sjeng (max) -84.09% 88.000 14.000 - -70.83% -
459.GemsFDTD (max) (percentage) -83.73% 2.090 0.340 - -85.09% -
458.sjeng (max) (percentage) -83.48% 1.150 0.190 - -36.67% -
437.leslie3d (max) -76.33% 1411.000 334.000 - -40.36% -
437.leslie3d (max) (percentage) -75.68% 11.840 2.880 - -7.10% -
473.astar (max) (percentage) -68.55% 2.830 0.890 - 89.36% -
473.astar (max) -68.38% 136.000 43.000 - -29.51% -
456.hmmer (max) (percentage) -62.16% 0.370 0.140 - -73.08% -
456.hmmer (max) -60.00% 15.000 6.000 - -93.88% -
410.bwaves (max) (percentage) -57.69% 1.040 0.440 - -75.14% -
410.bwaves (max) -57.38% 61.000 26.000 - -90.00% -
482.sphinx3 (max) -45.45% 11.000 6.000 - -99.48% -
482.sphinx3 (max) (percentage) -43.85% 1.300 0.730 - -76.83% -
456.hmmer (unexplained diff vs previous) (percentage) -43.75% 0.160 0.090 0.000 -82.69% 0.000
456.hmmer (unexplained diff vs previous) -42.86% 7.000 4.000 0.000 -95.92% 0.000
473.astar (unexplained diff vs previous) (percentage) -38.00% 1.000 0.620 0.000 31.91% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) -37.93% 0.870 0.540 0.000 -82.35% 0.000
473.astar (unexplained diff vs previous) -37.50% 48.000 30.000 0.000 -50.82% 0.000
482.sphinx3 (unexplained diff vs previous) -37.50% 8.000 5.000 0.000 -99.55% 0.000
450.soplex (max) -34.78% 23.000 15.000 - -78.26% -
437.leslie3d (unexplained diff vs previous) -32.98% 476.000 319.000 0.000 70.59% 0.000
450.soplex (max) (percentage) -32.90% 3.890 2.610 - 493.18% -
437.leslie3d (unexplained diff vs previous) (percentage) -31.08% 3.990 2.750 0.000 166.99% 0.000
445.gobmk (max) -30.19% 53.000 37.000 - 12.12% -
410.bwaves (unexplained diff vs previous) (percentage) -28.57% 0.490 0.350 0.000 -79.89% 0.000
445.gobmk (max) (percentage) -28.57% 1.050 0.750 - 177.78% -
410.bwaves (unexplained diff vs previous) -27.59% 29.000 21.000 0.000 -91.83% 0.000
450.soplex (unexplained diff vs previous) (percentage) -25.37% 0.670 0.500 0.000 13.64% 0.000
450.soplex (unexplained diff vs previous) -25.00% 4.000 3.000 0.000 -95.65% 0.000
429.mcf (max) (percentage) -22.32% 2.240 1.740 - 443.75% -
429.mcf (unexplained diff vs previous) (percentage) -21.69% 1.890 1.480 0.000 1544.44% 0.000
429.mcf (max) -21.43% 28.000 22.000 - -40.54% -
429.mcf (unexplained diff vs previous) -20.83% 24.000 19.000 0.000 90.00% 0.000
453.povray (max) (percentage) -8.13% 1.230 1.130 - -56.37% -
464.h264ref (max) -8.00% 25.000 23.000 - -61.02% -
400.perlbench (unexplained diff vs previous) (percentage) -7.76% 2.190 2.020 0.000 551.61% 0.000
462.libquantum (max) (percentage) -7.50% 2.000 1.850 - 2212.50% -
464.h264ref (max) (percentage) -7.32% 0.410 0.380 - -2.56% -
470.lbm (unexplained diff vs previous) (percentage) -6.98% 0.430 0.400 0.000 -92.06% 0.000
470.lbm (max) -6.67% 15.000 14.000 - -98.43% -
471.omnetpp (max) -6.52% 92.000 86.000 - -50.00% -
470.lbm (unexplained diff vs previous) -6.25% 16.000 15.000 0.000 -98.18% 0.000
471.omnetpp (max) (percentage) -6.23% 3.370 3.160 - 69.89% -
400.perlbench (unexplained diff vs previous) -6.06% 33.000 31.000 0.000 -11.43% 0.000
401.bzip2 (max) -5.41% 37.000 35.000 - -47.76% -
447.dealII (max) -5.26% 19.000 18.000 - -88.00% -
470.lbm (max) (percentage) -4.76% 0.420 0.400 - -92.70% -
401.bzip2 (max) (percentage) -4.62% 1.300 1.240 - 210.00% -
435.gromacs (max) (percentage) -4.43% 1.580 1.510 - -14.20% -
465.tonto (max) -4.23% 71.000 68.000 - -69.51% -
400.perlbench (max) (percentage) -4.19% 2.150 2.060 - 157.50% -
465.tonto (max) (percentage) -3.64% 0.550 0.530 - -69.01% -
437.leslie3d -3.49% 573.000 553.000 - 83.72% -
437.leslie3d (avg) -3.49% 573.000 553.000 - 83.72% -
435.gromacs (max) -3.42% 117.000 113.000 - -61.82% -
436.cactusADM -3.23% 31.000 30.000 - -94.87% -
436.cactusADM (avg) -3.23% 31.000 30.000 - -94.87% -
400.perlbench (max) -3.12% 32.000 31.000 - -65.17% -
447.dealII (max) (percentage) -3.09% 0.970 0.940 - -43.03% -
434.zeusmp -2.94% 34.000 33.000 - -87.06% -
434.zeusmp (avg) -2.94% 34.000 33.000 - -87.06% -
447.dealII -2.86% 35.000 34.000 - -56.96% -
447.dealII (avg) -2.86% 35.000 34.000 - -56.96% -
465.tonto -2.59% 116.000 113.000 - -23.65% -
465.tonto (avg) -2.59% 116.000 113.000 - -23.65% -
458.sjeng (avg) (percentage) -2.30% 0.870 0.850 - 269.57% -
465.tonto (avg) (percentage) -2.22% 0.900 0.880 - -22.81% -
459.GemsFDTD (avg) (percentage) -2.04% 1.470 1.440 - 3.60% -
459.GemsFDTD -1.96% 102.000 100.000 - -71.59% -
459.GemsFDTD (avg) -1.96% 102.000 100.000 - -71.59% -
470.lbm -1.92% 52.000 51.000 - -91.61% -
470.lbm (avg) -1.92% 52.000 51.000 - -91.61% -
471.omnetpp -1.92% 52.000 51.000 - -53.21% -
471.omnetpp (avg) -1.92% 52.000 51.000 - -53.21% -
445.gobmk -1.89% 53.000 52.000 - 92.59% -
445.gobmk (avg) -1.89% 53.000 52.000 - 92.59% -
453.povray (avg) (percentage) -1.86% 1.610 1.580 - 3.27% -
434.zeusmp (max) (percentage) -1.80% 1.110 1.090 - -52.61% -
450.soplex (avg) (percentage) -1.73% 2.890 2.840 - 1083.33% -
433.milc (max) (percentage) -1.69% 0.590 0.580 - -18.31% -
401.bzip2 -1.69% 59.000 58.000 - 28.89% -
401.bzip2 (avg) -1.69% 59.000 58.000 - 28.89% -
482.sphinx3 (avg) (percentage) -1.69% 1.780 1.750 - -25.53% -
410.bwaves (avg) (percentage) -1.68% 1.190 1.170 - -40.31% -
436.cactusADM (avg) (percentage) -1.67% 1.200 1.180 - -50.83% -
434.zeusmp (avg) (percentage) -1.61% 1.240 1.220 - 0.00% -
436.cactusADM (max) (percentage) -1.56% 0.640 0.630 - -87.52% -
458.sjeng -1.52% 66.000 65.000 - 80.56% -
458.sjeng (avg) -1.52% 66.000 65.000 - 80.56% -
444.namd (avg) (percentage) -1.48% 1.350 1.330 - -48.85% -
401.bzip2 (avg) (percentage) -1.47% 2.040 2.010 - 644.44% -
433.milc (avg) (percentage) -1.43% 1.400 1.380 - 100.00% -
470.lbm (avg) (percentage) -1.43% 1.400 1.380 - -63.00% -
410.bwaves -1.43% 70.000 69.000 - -76.12% -
410.bwaves (avg) -1.43% 70.000 69.000 - -76.12% -
456.hmmer (avg) (percentage) -1.37% 0.730 0.720 - 140.00% -
429.mcf (avg) (percentage) -1.35% 5.170 5.100 - 962.50% -
473.astar -1.25% 80.000 79.000 - 54.90% -
473.astar (avg) -1.25% 80.000 79.000 - 54.90% -
473.astar (avg) (percentage) -1.20% 1.660 1.640 - 310.00% -
447.dealII (avg) (percentage) -1.15% 1.740 1.720 - 97.70% -
435.gromacs (avg) (percentage) -1.07% 1.870 1.850 - 94.74% -
471.omnetpp (avg) (percentage) -1.05% 1.910 1.890 - 60.17% -
464.h264ref (avg) (percentage) -1.03% 0.970 0.960 - 146.15% -

Added Tests - sample variability
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d 2.39% 48001.000 49148.000 - 0.00% -
465.tonto 2.18% 1437761.000 1469161.000 - 0.00% -
473.astar 1.44% 15549.000 15773.000 - 0.00% -
453.povray 1.12% 489013.000 494483.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 2212.50% 0.080 1.850 - -7.50% -
462.libquantum (unexplained diff vs previous) (percentage) 2175.00% 0.080 1.820 0.000 30.94% 0.000
429.mcf (unexplained diff vs previous) (percentage) 1544.44% 0.090 1.480 0.000 -21.69% 0.000
403.gcc (max) (percentage) 1288.14% 0.590 8.190 - -0.61% -
450.soplex (avg) (percentage) 1083.33% 0.240 2.840 - -1.73% -
429.mcf (avg) (percentage) 962.50% 0.480 5.100 - -1.35% -
462.libquantum (avg) (percentage) 747.37% 0.190 1.610 - 0.63% -
401.bzip2 (avg) (percentage) 644.44% 0.270 2.010 - -1.47% -
400.perlbench (unexplained diff vs previous) (percentage) 551.61% 0.310 2.020 0.000 -7.76% 0.000
450.soplex (max) (percentage) 493.18% 0.440 2.610 - -32.90% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
429.mcf (max) (percentage) 443.75% 0.320 1.740 - -22.32% -
400.perlbench (avg) (percentage) 374.36% 0.390 1.850 - 0.00% -
445.gobmk (avg) (percentage) 372.73% 0.220 1.040 - -0.95% -
454.calculix (max) (percentage) 326.19% 0.840 3.580 - -0.28% -
473.astar (avg) (percentage) 310.00% 0.400 1.640 - -1.20% -
445.gobmk (unexplained diff vs previous) (percentage) 300.00% 0.150 0.600 0.000 0.00% 0.000
458.sjeng (avg) (percentage) 269.57% 0.230 0.850 - -2.30% -
401.bzip2 (max) (percentage) 210.00% 0.400 1.240 - -4.62% -
437.leslie3d (avg) (percentage) 185.63% 1.670 4.770 - -0.83% -
445.gobmk (max) (percentage) 177.78% 0.270 0.750 - -28.57% -
437.leslie3d (unexplained diff vs previous) (percentage) 166.99% 1.030 2.750 0.000 -31.08% 0.000
400.perlbench (max) (percentage) 157.50% 0.800 2.060 - -4.19% -
464.h264ref (avg) (percentage) 146.15% 0.390 0.960 - -1.03% -
456.hmmer (avg) (percentage) 140.00% 0.300 0.720 - -1.37% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
433.milc (avg) (percentage) 100.00% 0.690 1.380 - -1.43% -
447.dealII (avg) (percentage) 97.70% 0.870 1.720 - -1.15% -
435.gromacs (avg) (percentage) 94.74% 0.950 1.850 - -1.07% -
445.gobmk 92.59% 27.000 52.000 - -1.89% -
445.gobmk (avg) 92.59% 27.000 52.000 - -1.89% -
429.mcf (unexplained diff vs previous) 90.00% 10.000 19.000 0.000 -20.83% 0.000
473.astar (max) (percentage) 89.36% 0.470 0.890 - -68.55% -
437.leslie3d 83.72% 301.000 553.000 - -3.49% -
437.leslie3d (avg) 83.72% 301.000 553.000 - -3.49% -
458.sjeng 80.56% 36.000 65.000 - -1.52% -
458.sjeng (avg) 80.56% 36.000 65.000 - -1.52% -
437.leslie3d (unexplained diff vs previous) 70.59% 187.000 319.000 0.000 -32.98% 0.000
471.omnetpp (max) (percentage) 69.89% 1.860 3.160 - -6.23% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
445.gobmk (unexplained diff vs previous) 66.67% 18.000 30.000 0.000 0.00% 0.000
483.xalancbmk (max) (percentage) 66.30% 0.920 1.530 - 0.00% -
471.omnetpp (avg) (percentage) 60.17% 1.180 1.890 - -1.05% -
473.astar 54.90% 51.000 79.000 - -1.25% -
473.astar (avg) 54.90% 51.000 79.000 - -1.25% -
473.astar (unexplained diff vs previous) (percentage) 31.91% 0.470 0.620 0.000 -38.00% 0.000
401.bzip2 28.89% 45.000 58.000 - -1.69% -
401.bzip2 (avg) 28.89% 45.000 58.000 - -1.69% -
447.dealII (unexplained diff vs previous) (percentage) 26.67% 0.750 0.950 0.000 90.00% 0.000
429.mcf 16.07% 56.000 65.000 - 0.00% -
429.mcf (avg) 16.07% 56.000 65.000 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 13.64% 0.440 0.500 0.000 -25.37% 0.000
445.gobmk (max) 12.12% 33.000 37.000 - -30.19% -
459.GemsFDTD (avg) (percentage) 3.60% 1.390 1.440 - -2.04% -
453.povray (avg) (percentage) 3.27% 1.530 1.580 - -1.86% -
471.omnetpp (unexplained diff vs previous) (percentage) 2.99% 0.670 0.690 0.000 213.64% 0.000

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.90% 10845.000 10205.000 - 0.00% -
434.zeusmp -1.44% 141060.000 139034.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.71% 17005.000 220.000 - 1.85% -
482.sphinx3 -97.48% 36586.000 923.000 - 0.54% -
450.soplex -96.18% 15709.000 600.000 - 0.50% -
444.namd -94.32% 15957.000 906.000 - 0.33% -
453.povray -92.63% 7395.000 545.000 - -0.18% -
433.milc -91.94% 13668.000 1101.000 - 0.27% -
436.cactusADM -89.34% 24396.000 2600.000 - -0.42% -
429.mcf -89.03% 11727.000 1286.000 - 1.50% -
434.zeusmp -86.75% 20932.000 2774.000 - -0.29% -
400.perlbench -86.31% 11215.000 1535.000 - 2.06% -
401.bzip2 -82.99% 16971.000 2886.000 - -0.31% -
447.dealII -78.10% 9101.000 1993.000 - -0.94% -
456.hmmer -77.32% 18995.000 4309.000 - 0.09% -
470.lbm -77.04% 16312.000 3746.000 - -0.40% -
459.GemsFDTD -72.52% 25364.000 6971.000 - 0.10% -
471.omnetpp -70.43% 9265.000 2740.000 - -0.69% -
473.astar -62.61% 12983.000 4854.000 - 0.62% -
410.bwaves -59.50% 14745.000 5972.000 - 0.35% -
464.h264ref -59.40% 15208.000 6174.000 - 0.16% -
445.gobmk -59.34% 12396.000 5040.000 - -0.59% -
435.gromacs -55.29% 16853.000 7535.000 - 1.06% -
458.sjeng -51.87% 16018.000 7709.000 - 0.10% -
437.leslie3d -35.79% 18070.000 11602.000 - -2.68% -
465.tonto -1.03% 13044.000 12910.000 - -0.22% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.77% 879.000 2.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.64% 834.000 3.000 0.000 200.00% 0.000
482.sphinx3 (unexplained diff vs previous) -99.55% 1118.000 5.000 0.000 -37.50% 0.000
482.sphinx3 (max) -99.48% 1152.000 6.000 - -45.45% -
453.povray (unexplained diff vs previous) -99.41% 169.000 1.000 0.000 0.00% 0.000
436.cactusADM (unexplained diff vs previous) -99.06% 1173.000 11.000 0.000 37.50% 0.000
459.GemsFDTD (unexplained diff vs previous) -98.80% 584.000 7.000 0.000 0.00% 0.000
436.cactusADM (max) -98.70% 1231.000 16.000 - 0.00% -
470.lbm (max) -98.43% 893.000 14.000 - -6.67% -
434.zeusmp (unexplained diff vs previous) -98.30% 470.000 8.000 0.000 166.67% 0.000
470.lbm (unexplained diff vs previous) -98.18% 822.000 15.000 0.000 -6.25% 0.000
482.sphinx3 -98.14% 859.000 16.000 - 0.00% -
482.sphinx3 (avg) -98.14% 859.000 16.000 - 0.00% -
444.namd -97.10% 414.000 12.000 - 0.00% -
444.namd (avg) -97.10% 414.000 12.000 - 0.00% -
433.milc (unexplained diff vs previous) -96.88% 96.000 3.000 0.000 200.00% 0.000
453.povray (max) -96.86% 191.000 6.000 - 0.00% -
459.GemsFDTD (max) -96.02% 578.000 23.000 - -84.14% -
456.hmmer (unexplained diff vs previous) -95.92% 98.000 4.000 0.000 -42.86% 0.000
416.gamess (max) (percentage) -95.86% 3.140 0.130 - 0.00% -
450.soplex (unexplained diff vs previous) -95.65% 69.000 3.000 0.000 -25.00% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) -95.65% 2.300 0.100 0.000 0.00% 0.000
436.cactusADM -94.87% 585.000 30.000 - -3.23% -
436.cactusADM (avg) -94.87% 585.000 30.000 - -3.23% -
444.namd (max) (percentage) -94.01% 5.510 0.330 - 0.00% -
456.hmmer (max) -93.88% 98.000 6.000 - -60.00% -
433.milc (max) -93.81% 97.000 6.000 - 0.00% -
434.zeusmp (max) -93.76% 481.000 30.000 - 0.00% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) -93.69% 5.230 0.330 0.000 200.00% 0.000
453.povray -92.92% 113.000 8.000 - 0.00% -
453.povray (avg) -92.92% 113.000 8.000 - 0.00% -
470.lbm (max) (percentage) -92.70% 5.480 0.400 - -4.76% -
453.povray (unexplained diff vs previous) (percentage) -92.14% 2.290 0.180 0.000 0.00% 0.000
470.lbm (unexplained diff vs previous) (percentage) -92.06% 5.040 0.400 0.000 -6.98% 0.000
410.bwaves (unexplained diff vs previous) -91.83% 257.000 21.000 0.000 -27.59% 0.000
470.lbm -91.61% 608.000 51.000 - -1.92% -
470.lbm (avg) -91.61% 608.000 51.000 - -1.92% -
436.cactusADM (unexplained diff vs previous) (percentage) -91.27% 4.810 0.420 0.000 35.48% 0.000
462.libquantum -90.62% 32.000 3.000 - 0.00% -
462.libquantum (avg) -90.62% 32.000 3.000 - 0.00% -
410.bwaves (max) -90.00% 260.000 26.000 - -57.38% -
447.dealII (max) -88.00% 150.000 18.000 - -5.26% -
436.cactusADM (max) (percentage) -87.52% 5.050 0.630 - -1.56% -
434.zeusmp (unexplained diff vs previous) (percentage) -87.11% 2.250 0.290 0.000 163.64% 0.000
434.zeusmp -87.06% 255.000 33.000 - -2.94% -
434.zeusmp (avg) -87.06% 255.000 33.000 - -2.94% -
401.bzip2 (unexplained diff vs previous) -86.96% 69.000 9.000 0.000 350.00% 0.000
459.GemsFDTD (max) (percentage) -85.09% 2.280 0.340 - -83.73% -
433.milc -84.04% 94.000 15.000 - 0.00% -
433.milc (avg) -84.04% 94.000 15.000 - 0.00% -
458.sjeng (unexplained diff vs previous) -83.33% 48.000 8.000 0.000 0.00% 0.000
464.h264ref (unexplained diff vs previous) -82.76% 58.000 10.000 0.000 0.00% 0.000
456.hmmer (unexplained diff vs previous) (percentage) -82.69% 0.520 0.090 0.000 -43.75% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) -82.35% 3.060 0.540 0.000 -37.93% 0.000
465.tonto (unexplained diff vs previous) (percentage) -80.53% 1.130 0.220 0.000 0.00% 0.000
465.tonto (unexplained diff vs previous) -80.41% 148.000 29.000 0.000 0.00% 0.000
410.bwaves (unexplained diff vs previous) (percentage) -79.89% 1.740 0.350 0.000 -28.57% 0.000
450.soplex (max) -78.26% 69.000 15.000 - -34.78% -
482.sphinx3 (max) (percentage) -76.83% 3.150 0.730 - -43.85% -
410.bwaves -76.12% 289.000 69.000 - -1.43% -
410.bwaves (avg) -76.12% 289.000 69.000 - -1.43% -
410.bwaves (max) (percentage) -75.14% 1.770 0.440 - -57.69% -
456.hmmer (max) (percentage) -73.08% 0.520 0.140 - -62.16% -
435.gromacs (unexplained diff vs previous) -72.95% 292.000 79.000 0.000 2.60% 0.000
447.dealII (unexplained diff vs previous) -72.06% 68.000 19.000 0.000 90.00% 0.000
459.GemsFDTD -71.59% 352.000 100.000 - -1.96% -
459.GemsFDTD (avg) -71.59% 352.000 100.000 - -1.96% -
462.libquantum (unexplained diff vs previous) -71.43% 14.000 4.000 0.000 33.33% 0.000
458.sjeng (max) -70.83% 48.000 14.000 - -84.09% -
465.tonto (max) -69.51% 223.000 68.000 - -4.23% -
471.omnetpp (unexplained diff vs previous) -69.35% 62.000 19.000 0.000 216.67% 0.000
462.libquantum (max) -69.23% 13.000 4.000 - 0.00% -
465.tonto (max) (percentage) -69.01% 1.710 0.530 - -3.64% -
458.sjeng (unexplained diff vs previous) (percentage) -66.67% 0.300 0.100 0.000 0.00% 0.000
400.perlbench (max) -65.17% 89.000 31.000 - -3.12% -
470.lbm (avg) (percentage) -63.00% 3.730 1.380 - -1.43% -
435.gromacs (max) -61.82% 296.000 113.000 - -3.42% -
433.milc (unexplained diff vs previous) (percentage) -61.43% 0.700 0.270 0.000 200.00% 0.000
464.h264ref (max) -61.02% 59.000 23.000 - -8.00% -
464.h264ref (unexplained diff vs previous) (percentage) -57.89% 0.380 0.160 0.000 0.00% 0.000
447.dealII -56.96% 79.000 34.000 - -2.86% -
447.dealII (avg) -56.96% 79.000 34.000 - -2.86% -
453.povray (max) (percentage) -56.37% 2.590 1.130 - -8.13% -
450.soplex -54.05% 37.000 17.000 - 0.00% -
450.soplex (avg) -54.05% 37.000 17.000 - 0.00% -
471.omnetpp -53.21% 109.000 51.000 - -1.92% -
471.omnetpp (avg) -53.21% 109.000 51.000 - -1.92% -
434.zeusmp (max) (percentage) -52.61% 2.300 1.090 - -1.80% -
436.cactusADM (avg) (percentage) -50.83% 2.400 1.180 - -1.67% -
473.astar (unexplained diff vs previous) -50.82% 61.000 30.000 0.000 -37.50% 0.000
471.omnetpp (max) -50.00% 172.000 86.000 - -6.52% -
444.namd (avg) (percentage) -48.85% 2.600 1.330 - -1.48% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
401.bzip2 (max) -47.76% 67.000 35.000 - -5.41% -
456.hmmer -44.64% 56.000 31.000 - 0.00% -
456.hmmer (avg) -44.64% 56.000 31.000 - 0.00% -
447.dealII (max) (percentage) -43.03% 1.650 0.940 - -3.09% -
429.mcf (max) -40.54% 37.000 22.000 - -21.43% -
437.leslie3d (max) -40.36% 560.000 334.000 - -76.33% -
410.bwaves (avg) (percentage) -40.31% 1.960 1.170 - -1.68% -
435.gromacs (unexplained diff vs previous) (percentage) -39.31% 1.730 1.050 0.000 1.94% 0.000
458.sjeng (max) (percentage) -36.67% 0.300 0.190 - -83.48% -
400.perlbench -34.88% 43.000 28.000 - 3.70% -
400.perlbench (avg) -34.88% 43.000 28.000 - 3.70% -
473.astar (max) -29.51% 61.000 43.000 - -68.38% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
482.sphinx3 (avg) (percentage) -25.53% 2.350 1.750 - -1.69% -
401.bzip2 (unexplained diff vs previous) (percentage) -24.39% 0.410 0.310 0.000 342.86% 0.000
465.tonto -23.65% 148.000 113.000 - -2.59% -
465.tonto (avg) -23.65% 148.000 113.000 - -2.59% -
465.tonto (avg) (percentage) -22.81% 1.140 0.880 - -2.22% -
433.milc (max) (percentage) -18.31% 0.710 0.580 - -1.69% -
435.gromacs (max) (percentage) -14.20% 1.760 1.510 - -4.43% -
435.gromacs -13.12% 160.000 139.000 - 0.00% -
435.gromacs (avg) -13.12% 160.000 139.000 - 0.00% -
400.perlbench (unexplained diff vs previous) -11.43% 35.000 31.000 0.000 -6.06% 0.000
437.leslie3d (max) (percentage) -7.10% 3.100 2.880 - -75.68% -
464.h264ref (max) (percentage) -2.56% 0.390 0.380 - -7.32% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 0.00% -
nb_compile_successful 65.52% 29.000 48.000 - 0.00% -
nb_execution_successful 65.52% 29.000 48.000 - 0.00% -

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1535.000 2.06%
401.bzip2 2886.000 -
410.bwaves 5972.000 -
429.mcf 1286.000 1.50%
433.milc 1101.000 -
434.zeusmp 2774.000 -
435.gromacs 7535.000 1.06%
436.cactusADM 2600.000 -
437.leslie3d 11602.000 -2.68%
444.namd 906.000 -
445.gobmk 5040.000 -
447.dealII 1993.000 -
450.soplex 600.000 -
453.povray 545.000 -
456.hmmer 4309.000 -
458.sjeng 7709.000 -
459.GemsFDTD 6971.000 -
462.libquantum 220.000 1.85%
464.h264ref 6174.000 -
465.tonto 12910.000 -
470.lbm 3746.000 -
471.omnetpp 2740.000 -
473.astar 4854.000 -
482.sphinx3 923.000 -
Geometric Mean 2639.892 -
Code Size (bytes)
Name Current %
400.perlbench 562934.000 -
401.bzip2 33422.000 -
410.bwaves 14683.000 -
429.mcf 7701.000 -
433.milc 47797.000 -
434.zeusmp 139034.000 -
435.gromacs 393206.000 -
436.cactusADM 257162.000 -
437.leslie3d 49148.000 -
444.namd 163203.000 -
445.gobmk 1088648.000 -
447.dealII 192266.000 -
450.soplex 135488.000 -
453.povray 494483.000 -
456.hmmer 74477.000 -
458.sjeng 81269.000 -
459.GemsFDTD 197147.000 -
462.libquantum 13086.000 -
464.h264ref 301282.000 -
465.tonto 1469161.000 -
470.lbm 10205.000 -
471.omnetpp 296793.000 -
473.astar 15773.000 -
482.sphinx3 79750.000 -
Geometric Mean 106197.826 -
sample variability
Name Current %
400.perlbench 28.000 3.70%
400.perlbench (avg) 28.000 3.70%
400.perlbench (avg) (percentage) 1.850 -
400.perlbench (max) 31.000 -3.12%
400.perlbench (max) (percentage) 2.060 -4.19%
400.perlbench (unexplained diff vs previous) 31.000 -6.06%
400.perlbench (unexplained diff vs previous) (percentage) 2.020 -7.76%
401.bzip2 58.000 -1.69%
401.bzip2 (avg) 58.000 -1.69%
401.bzip2 (avg) (percentage) 2.010 -1.47%
401.bzip2 (max) 35.000 -5.41%
401.bzip2 (max) (percentage) 1.240 -4.62%
401.bzip2 (unexplained diff vs previous) 9.000 350.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.310 342.86%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 8.190 -
410.bwaves 69.000 -1.43%
410.bwaves (avg) 69.000 -1.43%
410.bwaves (avg) (percentage) 1.170 -1.68%
410.bwaves (max) 26.000 -57.38%
410.bwaves (max) (percentage) 0.440 -57.69%
410.bwaves (unexplained diff vs previous) 21.000 -27.59%
410.bwaves (unexplained diff vs previous) (percentage) 0.350 -28.57%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.130 -
429.mcf 65.000 -
429.mcf (avg) 65.000 -
429.mcf (avg) (percentage) 5.100 -1.35%
429.mcf (max) 22.000 -21.43%
429.mcf (max) (percentage) 1.740 -22.32%
429.mcf (unexplained diff vs previous) 19.000 -20.83%
429.mcf (unexplained diff vs previous) (percentage) 1.480 -21.69%
433.milc 15.000 -
433.milc (avg) 15.000 -
433.milc (avg) (percentage) 1.380 -1.43%
433.milc (max) 6.000 -
433.milc (max) (percentage) 0.580 -1.69%
433.milc (unexplained diff vs previous) 3.000 200.00%
433.milc (unexplained diff vs previous) (percentage) 0.270 200.00%
434.zeusmp 33.000 -2.94%
434.zeusmp (avg) 33.000 -2.94%
434.zeusmp (avg) (percentage) 1.220 -1.61%
434.zeusmp (max) 30.000 -
434.zeusmp (max) (percentage) 1.090 -1.80%
434.zeusmp (unexplained diff vs previous) 8.000 166.67%
434.zeusmp (unexplained diff vs previous) (percentage) 0.290 163.64%
435.gromacs 139.000 -
435.gromacs (avg) 139.000 -
435.gromacs (avg) (percentage) 1.850 -1.07%
435.gromacs (max) 113.000 -3.42%
435.gromacs (max) (percentage) 1.510 -4.43%
435.gromacs (unexplained diff vs previous) 79.000 2.60%
435.gromacs (unexplained diff vs previous) (percentage) 1.050 1.94%
436.cactusADM 30.000 -3.23%
436.cactusADM (avg) 30.000 -3.23%
436.cactusADM (avg) (percentage) 1.180 -1.67%
436.cactusADM (max) 16.000 -
436.cactusADM (max) (percentage) 0.630 -1.56%
436.cactusADM (unexplained diff vs previous) 11.000 37.50%
436.cactusADM (unexplained diff vs previous) (percentage) 0.420 35.48%
437.leslie3d 553.000 -3.49%
437.leslie3d (avg) 553.000 -3.49%
437.leslie3d (avg) (percentage) 4.770 -
437.leslie3d (max) 334.000 -76.33%
437.leslie3d (max) (percentage) 2.880 -75.68%
437.leslie3d (unexplained diff vs previous) 319.000 -32.98%
437.leslie3d (unexplained diff vs previous) (percentage) 2.750 -31.08%
444.namd 12.000 -
444.namd (avg) 12.000 -
444.namd (avg) (percentage) 1.330 -1.48%
444.namd (max) 2.000 -
444.namd (max) (percentage) 0.330 -
444.namd (unexplained diff vs previous) 3.000 200.00%
444.namd (unexplained diff vs previous) (percentage) 0.330 200.00%
445.gobmk 52.000 -1.89%
445.gobmk (avg) 52.000 -1.89%
445.gobmk (avg) (percentage) 1.040 -
445.gobmk (max) 37.000 -30.19%
445.gobmk (max) (percentage) 0.750 -28.57%
445.gobmk (unexplained diff vs previous) 30.000 -
445.gobmk (unexplained diff vs previous) (percentage) 0.600 -
447.dealII 34.000 -2.86%
447.dealII (avg) 34.000 -2.86%
447.dealII (avg) (percentage) 1.720 -1.15%
447.dealII (max) 18.000 -5.26%
447.dealII (max) (percentage) 0.940 -3.09%
447.dealII (unexplained diff vs previous) 19.000 90.00%
447.dealII (unexplained diff vs previous) (percentage) 0.950 90.00%
450.soplex 17.000 -
450.soplex (avg) 17.000 -
450.soplex (avg) (percentage) 2.840 -1.73%
450.soplex (max) 15.000 -34.78%
450.soplex (max) (percentage) 2.610 -32.90%
450.soplex (unexplained diff vs previous) 3.000 -25.00%
450.soplex (unexplained diff vs previous) (percentage) 0.500 -25.37%
453.povray 8.000 -
453.povray (avg) 8.000 -
453.povray (avg) (percentage) 1.580 -1.86%
453.povray (max) 6.000 -
453.povray (max) (percentage) 1.130 -8.13%
453.povray (unexplained diff vs previous) 1.000 -
453.povray (unexplained diff vs previous) (percentage) 0.180 -
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.580 -
456.hmmer 31.000 -
456.hmmer (avg) 31.000 -
456.hmmer (avg) (percentage) 0.720 -1.37%
456.hmmer (max) 6.000 -60.00%
456.hmmer (max) (percentage) 0.140 -62.16%
456.hmmer (unexplained diff vs previous) 4.000 -42.86%
456.hmmer (unexplained diff vs previous) (percentage) 0.090 -43.75%
458.sjeng 65.000 -1.52%
458.sjeng (avg) 65.000 -1.52%
458.sjeng (avg) (percentage) 0.850 -2.30%
458.sjeng (max) 14.000 -84.09%
458.sjeng (max) (percentage) 0.190 -83.48%
458.sjeng (unexplained diff vs previous) 8.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.100 -
459.GemsFDTD 100.000 -1.96%
459.GemsFDTD (avg) 100.000 -1.96%
459.GemsFDTD (avg) (percentage) 1.440 -2.04%
459.GemsFDTD (max) 23.000 -84.14%
459.GemsFDTD (max) (percentage) 0.340 -83.73%
459.GemsFDTD (unexplained diff vs previous) 7.000 -
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.100 -
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 1.610 -
462.libquantum (max) 4.000 -
462.libquantum (max) (percentage) 1.850 -7.50%
462.libquantum (unexplained diff vs previous) 4.000 33.33%
462.libquantum (unexplained diff vs previous) (percentage) 1.820 30.94%
464.h264ref 59.000 -
464.h264ref (avg) 59.000 -
464.h264ref (avg) (percentage) 0.960 -1.03%
464.h264ref (max) 23.000 -8.00%
464.h264ref (max) (percentage) 0.380 -7.32%
464.h264ref (unexplained diff vs previous) 10.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.160 -
465.tonto 113.000 -2.59%
465.tonto (avg) 113.000 -2.59%
465.tonto (avg) (percentage) 0.880 -2.22%
465.tonto (max) 68.000 -4.23%
465.tonto (max) (percentage) 0.530 -3.64%
465.tonto (unexplained diff vs previous) 29.000 -
465.tonto (unexplained diff vs previous) (percentage) 0.220 -
470.lbm 51.000 -1.92%
470.lbm (avg) 51.000 -1.92%
470.lbm (avg) (percentage) 1.380 -1.43%
470.lbm (max) 14.000 -6.67%
470.lbm (max) (percentage) 0.400 -4.76%
470.lbm (unexplained diff vs previous) 15.000 -6.25%
470.lbm (unexplained diff vs previous) (percentage) 0.400 -6.98%
471.omnetpp 51.000 -1.92%
471.omnetpp (avg) 51.000 -1.92%
471.omnetpp (avg) (percentage) 1.890 -1.05%
471.omnetpp (max) 86.000 -6.52%
471.omnetpp (max) (percentage) 3.160 -6.23%
471.omnetpp (unexplained diff vs previous) 19.000 216.67%
471.omnetpp (unexplained diff vs previous) (percentage) 0.690 213.64%
473.astar 79.000 -1.25%
473.astar (avg) 79.000 -1.25%
473.astar (avg) (percentage) 1.640 -1.20%
473.astar (max) 43.000 -68.38%
473.astar (max) (percentage) 0.890 -68.55%
473.astar (unexplained diff vs previous) 30.000 -37.50%
473.astar (unexplained diff vs previous) (percentage) 0.620 -38.00%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 16.000 -
482.sphinx3 (avg) 16.000 -
482.sphinx3 (avg) (percentage) 1.750 -1.69%
482.sphinx3 (max) 6.000 -45.45%
482.sphinx3 (max) (percentage) 0.730 -43.85%
482.sphinx3 (unexplained diff vs previous) 5.000 -37.50%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.540 -37.93%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.530 -
Geometric Mean 5.325 -16.51%