Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c33568d2c108478c25fe9959b5ce9d25b97efa6c (unchanged)
git_gcc https://gcc.gnu.org/git/gcc.git (6aba48a8cc128e54ee243d451ac9a843ff41c4f9) (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=77c7c4417410f4bf15ea246dd52603dc7be8de45 (unchanged)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/b0b45b440b71e370b6f3d55512d712b428d523a9 (+ 16236 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/281/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0281 (v6.11.11-89856-gb0b45b440b71) 2024-12-01T16:12:34 0:00:00
Previous 0280 (glibc-2.40.9000-00422-g77c7c441741) 2024-11-26T22:43:02 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 55 67
Performance Improvements 62 141
Added Tests 2 2
Unchanged Tests 1117 1026
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
435.gromacs 2.20% 7396.000 7559.000 - -55.15% -
482.sphinx3 2.11% 899.000 918.000 - -97.49% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (unexplained diff vs previous) (percentage) 2169.23% 0.130 2.950 - 851.61% -
400.perlbench (unexplained diff vs previous) 2150.00% 2.000 45.000 - 28.57% -
437.leslie3d (unexplained diff vs previous) (percentage) 900.00% 0.830 8.300 - 705.83% -
437.leslie3d (unexplained diff vs previous) 826.47% 102.000 945.000 - 405.35% -
410.bwaves (unexplained diff vs previous) 475.00% 4.000 23.000 - -91.05% -
410.bwaves (unexplained diff vs previous) (percentage) 457.14% 0.070 0.390 - -77.59% -
429.mcf (unexplained diff vs previous) 450.00% 2.000 11.000 - 10.00% -
429.mcf (unexplained diff vs previous) (percentage) 443.75% 0.160 0.870 - 866.67% -
433.milc (unexplained diff vs previous) (percentage) 361.11% 0.180 0.830 - 18.57% -
433.milc (unexplained diff vs previous) 350.00% 2.000 9.000 - -90.62% -
433.milc (max) 300.00% 2.000 8.000 - -91.75% -
464.h264ref (max) 283.33% 12.000 46.000 - -22.03% -
464.h264ref (unexplained diff vs previous) (percentage) 266.67% 0.210 0.770 - 102.63% -
464.h264ref (max) (percentage) 261.90% 0.210 0.760 - 94.87% -
464.h264ref (unexplained diff vs previous) 261.54% 13.000 47.000 - -18.97% -
458.sjeng (unexplained diff vs previous) 255.56% 9.000 32.000 - -33.33% -
458.sjeng (unexplained diff vs previous) (percentage) 250.00% 0.120 0.420 - 40.00% -
433.milc (max) (percentage) 241.67% 0.240 0.820 - 15.49% -
445.gobmk (unexplained diff vs previous) 144.44% 18.000 44.000 - 144.44% -
445.gobmk (unexplained diff vs previous) (percentage) 138.89% 0.360 0.860 - 473.33% -
462.libquantum (unexplained diff vs previous) (percentage) 104.44% 0.900 1.840 - 2200.00% -
462.libquantum (unexplained diff vs previous) 100.00% 2.000 4.000 - -71.43% -
444.namd (unexplained diff vs previous) (percentage) 69.39% 0.980 1.660 - -68.26% -
444.namd (unexplained diff vs previous) 66.67% 9.000 15.000 - -98.20% -
444.namd (max) (percentage) 64.65% 0.990 1.630 - -70.42% -
462.libquantum (max) (percentage) 63.06% 1.110 1.810 - 2162.50% -
445.gobmk (max) 57.14% 28.000 44.000 - 33.33% -
435.gromacs (unexplained diff vs previous) 56.73% 104.000 163.000 - -44.18% -
401.bzip2 (unexplained diff vs previous) 55.56% 9.000 14.000 - -79.71% -
444.namd (max) 55.56% 9.000 14.000 - -98.41% -
401.bzip2 (unexplained diff vs previous) (percentage) 54.84% 0.310 0.480 - 17.07% -
435.gromacs (unexplained diff vs previous) (percentage) 53.19% 1.410 2.160 - 24.86% -
445.gobmk (max) (percentage) 52.63% 0.570 0.870 - 222.22% -
462.libquantum (max) 50.00% 2.000 3.000 - -76.92% -
482.sphinx3 (max) 35.71% 14.000 19.000 - -98.35% -
400.perlbench (max) (percentage) 33.49% 2.150 2.870 - 258.75% -
434.zeusmp (unexplained diff vs previous) (percentage) 30.56% 0.720 0.940 - -58.22% -
400.perlbench (max) 30.30% 33.000 43.000 - -51.69% -
434.zeusmp (unexplained diff vs previous) 30.00% 20.000 26.000 - -94.47% -
434.zeusmp (max) (percentage) 28.77% 0.730 0.940 - -59.13% -
482.sphinx3 (max) (percentage) 28.66% 1.640 2.110 - -33.02% -
482.sphinx3 (unexplained diff vs previous) 26.67% 15.000 19.000 - -98.30% -
434.zeusmp (max) 25.00% 20.000 25.000 - -94.80% -
482.sphinx3 (unexplained diff vs previous) (percentage) 23.95% 1.670 2.070 - -32.35% -
410.bwaves (max) 15.00% 20.000 23.000 - -91.15% -
437.leslie3d (max) (percentage) 13.29% 6.770 7.670 - 147.42% -
410.bwaves (max) (percentage) 11.43% 0.350 0.390 - -77.97% -
429.mcf (max) 10.00% 10.000 11.000 - -70.27% -
437.leslie3d (max) 4.68% 834.000 873.000 - 55.89% -
435.gromacs 3.20% 125.000 129.000 - -19.38% -
435.gromacs (avg) 3.20% 125.000 129.000 - -19.38% -
400.perlbench (avg) (percentage) 1.31% 1.530 1.550 - 297.44% -
429.mcf (max) (percentage) 1.15% 0.870 0.880 - 175.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -7.67% 12328.000 11383.000 - -37.01% -
400.perlbench -2.87% 1568.000 1523.000 - -86.42% -
471.omnetpp -2.27% 2770.000 2707.000 - -70.78% -
462.libquantum -1.81% 221.000 217.000 - -98.72% -
444.namd -1.63% 918.000 903.000 - -94.34% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
465.tonto (unexplained diff vs previous) -92.31% 65.000 5.000 - -96.62% -
465.tonto (unexplained diff vs previous) (percentage) -92.16% 0.510 0.040 - -96.46% -
436.cactusADM (unexplained diff vs previous) -75.00% 4.000 1.000 - -99.91% -
436.cactusADM (unexplained diff vs previous) (percentage) -73.33% 0.150 0.040 - -99.17% -
453.povray (unexplained diff vs previous) (percentage) -70.21% 1.880 0.560 - -75.55% -
453.povray (unexplained diff vs previous) -70.00% 10.000 3.000 - -98.22% -
473.astar (unexplained diff vs previous) (percentage) -69.40% 1.340 0.410 - -12.77% -
473.astar (unexplained diff vs previous) -69.23% 65.000 20.000 - -67.21% -
459.GemsFDTD (unexplained diff vs previous) -61.54% 13.000 5.000 - -99.14% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -61.11% 0.180 0.070 - -96.96% -
447.dealII (unexplained diff vs previous) -58.33% 12.000 5.000 - -92.65% -
447.dealII (unexplained diff vs previous) (percentage) -58.06% 0.620 0.260 - -65.33% -
450.soplex (unexplained diff vs previous) -55.56% 9.000 4.000 - -94.20% -
450.soplex (unexplained diff vs previous) (percentage) -55.26% 1.520 0.680 - 54.55% -
459.GemsFDTD (max) -42.11% 19.000 11.000 - -98.10% -
459.GemsFDTD (max) (percentage) -40.74% 0.270 0.160 - -92.98% -
462.libquantum -33.33% 3.000 2.000 - -93.75% -
462.libquantum (avg) -33.33% 3.000 2.000 - -93.75% -
471.omnetpp (max) -32.22% 90.000 61.000 - -64.53% -
471.omnetpp (max) (percentage) -30.58% 3.270 2.270 - 22.04% -
450.soplex (max) -20.00% 10.000 8.000 - -88.41% -
473.astar (max) -17.19% 64.000 53.000 - -13.11% -
436.cactusADM (max) -16.67% 6.000 5.000 - -99.59% -
473.astar (max) (percentage) -16.54% 1.330 1.110 - 136.17% -
471.omnetpp (unexplained diff vs previous) -14.86% 74.000 63.000 - 1.61% -
471.omnetpp (unexplained diff vs previous) (percentage) -12.73% 2.670 2.330 - 247.76% -
450.soplex (max) (percentage) -10.06% 1.690 1.520 - 245.45% -
458.sjeng (max) -9.76% 41.000 37.000 - -22.92% -
458.sjeng (max) (percentage) -9.26% 0.540 0.490 - 63.33% -
436.cactusADM (max) (percentage) -8.70% 0.230 0.210 - -95.84% -
447.dealII (max) -7.69% 13.000 12.000 - -92.00% -
470.lbm (max) (percentage) -7.41% 0.270 0.250 - -95.44% -
437.leslie3d -7.01% 599.000 557.000 - 85.05% -
437.leslie3d (avg) -7.01% 599.000 557.000 - 85.05% -
447.dealII (max) (percentage) -5.88% 0.680 0.640 - -61.21% -
435.gromacs (max) (percentage) -4.96% 2.820 2.680 - 52.27% -
453.povray (max) (percentage) -4.86% 1.850 1.760 - -32.05% -
456.hmmer (max) (percentage) -4.00% 0.250 0.240 - -53.85% -
465.tonto (max) -3.92% 204.000 196.000 - -12.11% -
465.tonto (max) (percentage) -3.77% 1.590 1.530 - -10.53% -
435.gromacs (max) -2.88% 208.000 202.000 - -31.76% -
470.lbm -2.70% 37.000 36.000 - -94.08% -
470.lbm (avg) -2.70% 37.000 36.000 - -94.08% -
401.bzip2 -2.00% 50.000 49.000 - 8.89% -
401.bzip2 (avg) -2.00% 50.000 49.000 - 8.89% -
410.bwaves -1.89% 53.000 52.000 - -82.01% -
410.bwaves (avg) -1.89% 53.000 52.000 - -82.01% -
459.GemsFDTD (avg) (percentage) -1.82% 1.100 1.080 - -22.30% -
447.dealII (avg) (percentage) -1.48% 1.350 1.330 - 52.87% -
473.astar (avg) (percentage) -1.41% 1.420 1.400 - 250.00% -
459.GemsFDTD -1.30% 77.000 76.000 - -78.41% -
459.GemsFDTD (avg) -1.30% 77.000 76.000 - -78.41% -
429.mcf (avg) (percentage) -1.21% 4.130 4.080 - 750.00% -
436.cactusADM (avg) (percentage) -1.19% 0.840 0.830 - -65.42% -
450.soplex (avg) (percentage) -1.19% 2.520 2.490 - 937.50% -
401.bzip2 (avg) (percentage) -1.16% 1.720 1.700 - 529.63% -
410.bwaves (avg) (percentage) -1.12% 0.890 0.880 - -55.10% -

Added Tests - sample variability
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.86% 1437761.000 1464510.000 - 0.00% -
473.astar 1.78% 15549.000 15825.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 2200.00% 0.080 1.840 - 104.44% -
462.libquantum (max) (percentage) 2162.50% 0.080 1.810 - 63.06% -
403.gcc (max) (percentage) 1105.08% 0.590 7.110 - -0.28% -
450.soplex (avg) (percentage) 937.50% 0.240 2.490 - -1.19% -
429.mcf (unexplained diff vs previous) (percentage) 866.67% 0.090 0.870 - 443.75% -
400.perlbench (unexplained diff vs previous) (percentage) 851.61% 0.310 2.950 - 2169.23% -
429.mcf (avg) (percentage) 750.00% 0.480 4.080 - -1.21% -
437.leslie3d (unexplained diff vs previous) (percentage) 705.83% 1.030 8.300 - 900.00% -
462.libquantum (avg) (percentage) 621.05% 0.190 1.370 - 0.74% -
401.bzip2 (avg) (percentage) 529.63% 0.270 1.700 - -1.16% -
445.gobmk (unexplained diff vs previous) (percentage) 473.33% 0.150 0.860 - 138.89% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
437.leslie3d (unexplained diff vs previous) 405.35% 187.000 945.000 - 826.47% -
454.calculix (max) (percentage) 310.71% 0.840 3.450 - 0.00% -
400.perlbench (avg) (percentage) 297.44% 0.390 1.550 - 1.31% -
445.gobmk (avg) (percentage) 290.91% 0.220 0.860 - 0.00% -
400.perlbench (max) (percentage) 258.75% 0.800 2.870 - 33.49% -
473.astar (avg) (percentage) 250.00% 0.400 1.400 - -1.41% -
471.omnetpp (unexplained diff vs previous) (percentage) 247.76% 0.670 2.330 - -12.73% -
450.soplex (max) (percentage) 245.45% 0.440 1.520 - -10.06% -
445.gobmk (max) (percentage) 222.22% 0.270 0.870 - 52.63% -
437.leslie3d (avg) (percentage) 193.41% 1.670 4.900 - 0.82% -
429.mcf (max) (percentage) 175.00% 0.320 0.880 - 1.15% -
458.sjeng (avg) (percentage) 169.57% 0.230 0.620 - 0.00% -
437.leslie3d (max) (percentage) 147.42% 3.100 7.670 - 13.29% -
445.gobmk (unexplained diff vs previous) 144.44% 18.000 44.000 - 144.44% -
473.astar (max) (percentage) 136.17% 0.470 1.110 - -16.54% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) 102.63% 0.380 0.770 - 266.67% -
464.h264ref (max) (percentage) 94.87% 0.390 0.760 - 261.90% -
437.leslie3d 85.05% 301.000 557.000 - -7.01% -
437.leslie3d (avg) 85.05% 301.000 557.000 - -7.01% -
464.h264ref (avg) (percentage) 82.05% 0.390 0.710 - 0.00% -
435.gromacs (avg) (percentage) 80.00% 0.950 1.710 - 0.59% -
456.hmmer (avg) (percentage) 73.33% 0.300 0.520 - 0.00% -
401.bzip2 (max) (percentage) 70.00% 0.400 0.680 - 0.00% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
458.sjeng (max) (percentage) 63.33% 0.300 0.490 - -9.26% -
483.xalancbmk (max) (percentage) 63.04% 0.920 1.500 - 0.00% -
445.gobmk 59.26% 27.000 43.000 - 0.00% -
445.gobmk (avg) 59.26% 27.000 43.000 - 0.00% -
471.omnetpp (avg) (percentage) 56.78% 1.180 1.850 - 0.54% -
433.milc (avg) (percentage) 56.52% 0.690 1.080 - 0.00% -
437.leslie3d (max) 55.89% 560.000 873.000 - 4.68% -
450.soplex (unexplained diff vs previous) (percentage) 54.55% 0.440 0.680 - -55.26% -
447.dealII (avg) (percentage) 52.87% 0.870 1.330 - -1.48% -
435.gromacs (max) (percentage) 52.27% 1.760 2.680 - -4.96% -
458.sjeng (unexplained diff vs previous) (percentage) 40.00% 0.300 0.420 - 250.00% -
445.gobmk (max) 33.33% 33.000 44.000 - 57.14% -
473.astar 33.33% 51.000 68.000 - 0.00% -
473.astar (avg) 33.33% 51.000 68.000 - 0.00% -
458.sjeng 30.56% 36.000 47.000 - 0.00% -
458.sjeng (avg) 30.56% 36.000 47.000 - 0.00% -
400.perlbench (unexplained diff vs previous) 28.57% 35.000 45.000 - 2150.00% -
435.gromacs (unexplained diff vs previous) (percentage) 24.86% 1.730 2.160 - 53.19% -
471.omnetpp (max) (percentage) 22.04% 1.860 2.270 - -30.58% -
433.milc (unexplained diff vs previous) (percentage) 18.57% 0.700 0.830 - 361.11% -
401.bzip2 (unexplained diff vs previous) (percentage) 17.07% 0.410 0.480 - 54.84% -
433.milc (max) (percentage) 15.49% 0.710 0.820 - 241.67% -
429.mcf (unexplained diff vs previous) 10.00% 10.000 11.000 - 450.00% -
401.bzip2 8.89% 45.000 49.000 - -2.00% -
401.bzip2 (avg) 8.89% 45.000 49.000 - -2.00% -
471.omnetpp (unexplained diff vs previous) 1.61% 62.000 63.000 - -14.86% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - 0.00% -
nb_execution_successful -17.24% 29.000 24.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.83% 10845.000 10213.000 - 0.00% -
437.leslie3d -3.79% 48001.000 46184.000 - 0.00% -
434.zeusmp -1.66% 141060.000 138714.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.72% 17005.000 217.000 - -1.81% -
482.sphinx3 -97.49% 36586.000 918.000 - 2.11% -
450.soplex -96.25% 15709.000 589.000 - -0.67% -
444.namd -94.34% 15957.000 903.000 - -1.63% -
453.povray -92.77% 7395.000 535.000 - 0.56% -
433.milc -92.08% 13668.000 1082.000 - -0.82% -
436.cactusADM -89.30% 24396.000 2611.000 - 0.04% -
429.mcf -89.25% 11727.000 1261.000 - 0.88% -
434.zeusmp -86.85% 20932.000 2752.000 - -0.94% -
400.perlbench -86.42% 11215.000 1523.000 - -2.87% -
401.bzip2 -82.93% 16971.000 2897.000 - -0.48% -
447.dealII -78.83% 9101.000 1927.000 - 0.26% -
456.hmmer -77.99% 18995.000 4180.000 - -0.17% -
470.lbm -77.42% 16312.000 3683.000 - 0.00% -
459.GemsFDTD -72.09% 25364.000 7080.000 - 0.07% -
471.omnetpp -70.78% 9265.000 2707.000 - -2.27% -
473.astar -62.57% 12983.000 4859.000 - 0.41% -
464.h264ref -59.69% 15208.000 6130.000 - -0.76% -
410.bwaves -59.64% 14745.000 5951.000 - -0.39% -
445.gobmk -58.85% 12396.000 5101.000 - 0.87% -
435.gromacs -55.15% 16853.000 7559.000 - 2.20% -
458.sjeng -51.99% 16018.000 7690.000 - 0.42% -
437.leslie3d -37.01% 18070.000 11383.000 - -7.67% -
465.tonto -1.45% 13044.000 12855.000 - -0.04% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (unexplained diff vs previous) -99.91% 1173.000 1.000 - -75.00% -
436.cactusADM (max) -99.59% 1231.000 5.000 - -16.67% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.17% 4.810 0.040 - -73.33% -
459.GemsFDTD (unexplained diff vs previous) -99.14% 584.000 5.000 - -61.54% -
470.lbm (max) -98.99% 893.000 9.000 - 0.00% -
444.namd (max) -98.41% 879.000 14.000 - 55.56% -
482.sphinx3 -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (avg) -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (max) -98.35% 1152.000 19.000 - 35.71% -
482.sphinx3 (unexplained diff vs previous) -98.30% 1118.000 19.000 - 26.67% -
453.povray (unexplained diff vs previous) -98.22% 169.000 3.000 - -70.00% -
444.namd (unexplained diff vs previous) -98.20% 834.000 15.000 - 66.67% -
459.GemsFDTD (max) -98.10% 578.000 11.000 - -42.11% -
444.namd -97.83% 414.000 9.000 - 0.00% -
444.namd (avg) -97.83% 414.000 9.000 - 0.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -96.96% 2.300 0.070 - -61.11% -
465.tonto (unexplained diff vs previous) -96.62% 148.000 5.000 - -92.31% -
465.tonto (unexplained diff vs previous) (percentage) -96.46% 1.130 0.040 - -92.16% -
436.cactusADM -96.41% 585.000 21.000 - 0.00% -
436.cactusADM (avg) -96.41% 585.000 21.000 - 0.00% -
416.gamess (max) (percentage) -96.18% 3.140 0.120 - 0.00% -
436.cactusADM (max) (percentage) -95.84% 5.050 0.210 - -8.70% -
470.lbm (max) (percentage) -95.44% 5.480 0.250 - -7.41% -
453.povray (max) -95.29% 191.000 9.000 - 0.00% -
434.zeusmp (max) -94.80% 481.000 25.000 - 25.00% -
434.zeusmp (unexplained diff vs previous) -94.47% 470.000 26.000 - 30.00% -
450.soplex (unexplained diff vs previous) -94.20% 69.000 4.000 - -55.56% -
470.lbm -94.08% 608.000 36.000 - -2.70% -
470.lbm (avg) -94.08% 608.000 36.000 - -2.70% -
453.povray -93.81% 113.000 7.000 - 0.00% -
453.povray (avg) -93.81% 113.000 7.000 - 0.00% -
462.libquantum -93.75% 32.000 2.000 - -33.33% -
462.libquantum (avg) -93.75% 32.000 2.000 - -33.33% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
459.GemsFDTD (max) (percentage) -92.98% 2.280 0.160 - -40.74% -
456.hmmer (unexplained diff vs previous) -92.86% 98.000 7.000 - 0.00% -
447.dealII (unexplained diff vs previous) -92.65% 68.000 5.000 - -58.33% -
447.dealII (max) -92.00% 150.000 12.000 - -7.69% -
433.milc (max) -91.75% 97.000 8.000 - 300.00% -
410.bwaves (max) -91.15% 260.000 23.000 - 15.00% -
410.bwaves (unexplained diff vs previous) -91.05% 257.000 23.000 - 475.00% -
433.milc (unexplained diff vs previous) -90.62% 96.000 9.000 - 350.00% -
434.zeusmp -89.80% 255.000 26.000 - 0.00% -
434.zeusmp (avg) -89.80% 255.000 26.000 - 0.00% -
456.hmmer (max) -89.80% 98.000 10.000 - 0.00% -
450.soplex (max) -88.41% 69.000 8.000 - -20.00% -
433.milc -88.30% 94.000 11.000 - 0.00% -
433.milc (avg) -88.30% 94.000 11.000 - 0.00% -
410.bwaves -82.01% 289.000 52.000 - -1.89% -
410.bwaves (avg) -82.01% 289.000 52.000 - -1.89% -
401.bzip2 (unexplained diff vs previous) -79.71% 69.000 14.000 - 55.56% -
459.GemsFDTD -78.41% 352.000 76.000 - -1.30% -
459.GemsFDTD (avg) -78.41% 352.000 76.000 - -1.30% -
410.bwaves (max) (percentage) -77.97% 1.770 0.390 - 11.43% -
410.bwaves (unexplained diff vs previous) (percentage) -77.59% 1.740 0.390 - 457.14% -
462.libquantum (max) -76.92% 13.000 3.000 - 50.00% -
453.povray (unexplained diff vs previous) (percentage) -75.55% 2.290 0.560 - -70.21% -
470.lbm (avg) (percentage) -73.19% 3.730 1.000 - -0.99% -
401.bzip2 (max) -71.64% 67.000 19.000 - 0.00% -
462.libquantum (unexplained diff vs previous) -71.43% 14.000 4.000 - 100.00% -
444.namd (max) (percentage) -70.42% 5.510 1.630 - 64.65% -
429.mcf (max) -70.27% 37.000 11.000 - 10.00% -
447.dealII -68.35% 79.000 25.000 - 0.00% -
447.dealII (avg) -68.35% 79.000 25.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) -68.26% 5.230 1.660 - 69.39% -
456.hmmer (unexplained diff vs previous) (percentage) -67.31% 0.520 0.170 - 0.00% -
473.astar (unexplained diff vs previous) -67.21% 61.000 20.000 - -69.23% -
436.cactusADM (avg) (percentage) -65.42% 2.400 0.830 - -1.19% -
447.dealII (unexplained diff vs previous) (percentage) -65.33% 0.750 0.260 - -58.06% -
471.omnetpp (max) -64.53% 172.000 61.000 - -32.22% -
456.hmmer -62.50% 56.000 21.000 - 0.00% -
456.hmmer (avg) -62.50% 56.000 21.000 - 0.00% -
450.soplex -62.16% 37.000 14.000 - 0.00% -
450.soplex (avg) -62.16% 37.000 14.000 - 0.00% -
447.dealII (max) (percentage) -61.21% 1.650 0.640 - -5.88% -
434.zeusmp (max) (percentage) -59.13% 2.300 0.940 - 28.77% -
444.namd (avg) (percentage) -58.46% 2.600 1.080 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -58.22% 2.250 0.940 - 30.56% -
410.bwaves (avg) (percentage) -55.10% 1.960 0.880 - -1.12% -
471.omnetpp -54.13% 109.000 50.000 - 0.00% -
471.omnetpp (avg) -54.13% 109.000 50.000 - 0.00% -
456.hmmer (max) (percentage) -53.85% 0.520 0.240 - -4.00% -
400.perlbench (max) -51.69% 89.000 43.000 - 30.30% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
400.perlbench -46.51% 43.000 23.000 - 0.00% -
400.perlbench (avg) -46.51% 43.000 23.000 - 0.00% -
435.gromacs (unexplained diff vs previous) -44.18% 292.000 163.000 - 56.73% -
458.sjeng (unexplained diff vs previous) -33.33% 48.000 32.000 - 255.56% -
482.sphinx3 (avg) (percentage) -33.19% 2.350 1.570 - 0.00% -
482.sphinx3 (max) (percentage) -33.02% 3.150 2.110 - 28.66% -
482.sphinx3 (unexplained diff vs previous) (percentage) -32.35% 3.060 2.070 - 23.95% -
453.povray (max) (percentage) -32.05% 2.590 1.760 - -4.86% -
435.gromacs (max) -31.76% 296.000 202.000 - -2.88% -
465.tonto -29.73% 148.000 104.000 - -0.95% -
465.tonto (avg) -29.73% 148.000 104.000 - -0.95% -
465.tonto (avg) (percentage) -28.95% 1.140 0.810 - -1.22% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
464.h264ref -27.12% 59.000 43.000 - 0.00% -
464.h264ref (avg) -27.12% 59.000 43.000 - 0.00% -
458.sjeng (max) -22.92% 48.000 37.000 - -9.76% -
459.GemsFDTD (avg) (percentage) -22.30% 1.390 1.080 - -1.82% -
434.zeusmp (avg) (percentage) -22.13% 1.220 0.950 - 0.00% -
464.h264ref (max) -22.03% 59.000 46.000 - 283.33% -
435.gromacs -19.38% 160.000 129.000 - 3.20% -
435.gromacs (avg) -19.38% 160.000 129.000 - 3.20% -
464.h264ref (unexplained diff vs previous) -18.97% 58.000 47.000 - 261.54% -
473.astar (max) -13.11% 61.000 53.000 - -17.19% -
473.astar (unexplained diff vs previous) (percentage) -12.77% 0.470 0.410 - -69.40% -
465.tonto (max) -12.11% 223.000 196.000 - -3.92% -
465.tonto (max) (percentage) -10.53% 1.710 1.530 - -3.77% -
453.povray (avg) (percentage) -9.80% 1.530 1.380 - -0.72% -
429.mcf -8.93% 56.000 51.000 - 0.00% -
429.mcf (avg) -8.93% 56.000 51.000 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 0.00% -

Added Tests - sample variability
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1523.000 -2.87%
401.bzip2 2897.000 -
410.bwaves 5951.000 -
429.mcf 1261.000 -
433.milc 1082.000 -
434.zeusmp 2752.000 -
435.gromacs 7559.000 2.20%
436.cactusADM 2611.000 -
437.leslie3d 11383.000 -7.67%
444.namd 903.000 -1.63%
445.gobmk 5101.000 -
447.dealII 1927.000 -
450.soplex 589.000 -
453.povray 535.000 -
456.hmmer 4180.000 -
458.sjeng 7690.000 -
459.GemsFDTD 7080.000 -
462.libquantum 217.000 -1.81%
464.h264ref 6130.000 -
465.tonto 12855.000 -
470.lbm 3683.000 -
471.omnetpp 2707.000 -2.27%
473.astar 4859.000 -
482.sphinx3 918.000 2.11%
Geometric Mean 2617.746 -
Code Size (bytes)
Name Current %
400.perlbench 562078.000 -
401.bzip2 33398.000 -
410.bwaves 14723.000 -
429.mcf 7717.000 -
433.milc 47733.000 -
434.zeusmp 138714.000 -
435.gromacs 393002.000 -
436.cactusADM 256626.000 -
437.leslie3d 46184.000 -
444.namd 163131.000 -
445.gobmk 1087552.000 -
447.dealII 189727.000 -
450.soplex 135852.000 -
453.povray 493733.000 -
456.hmmer 74389.000 -
458.sjeng 81349.000 -
459.GemsFDTD 194087.000 -
462.libquantum 13118.000 -
464.h264ref 301482.000 -
465.tonto 1464510.000 -
470.lbm 10213.000 -
471.omnetpp 296077.000 -
473.astar 15825.000 -
482.sphinx3 79598.000 -
Geometric Mean 105775.523 -
sample variability
Name Current %
400.perlbench 23.000 -
400.perlbench (avg) 23.000 -
400.perlbench (avg) (percentage) 1.550 1.31%
400.perlbench (max) 43.000 30.30%
400.perlbench (max) (percentage) 2.870 33.49%
400.perlbench (unexplained diff vs previous) 45.000 2150.00%
400.perlbench (unexplained diff vs previous) (percentage) 2.950 2169.23%
401.bzip2 49.000 -2.00%
401.bzip2 (avg) 49.000 -2.00%
401.bzip2 (avg) (percentage) 1.700 -1.16%
401.bzip2 (max) 19.000 -
401.bzip2 (max) (percentage) 0.680 -
401.bzip2 (unexplained diff vs previous) 14.000 55.56%
401.bzip2 (unexplained diff vs previous) (percentage) 0.480 54.84%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 7.110 -
410.bwaves 52.000 -1.89%
410.bwaves (avg) 52.000 -1.89%
410.bwaves (avg) (percentage) 0.880 -1.12%
410.bwaves (max) 23.000 15.00%
410.bwaves (max) (percentage) 0.390 11.43%
410.bwaves (unexplained diff vs previous) 23.000 475.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.390 457.14%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.120 -
429.mcf 51.000 -
429.mcf (avg) 51.000 -
429.mcf (avg) (percentage) 4.080 -1.21%
429.mcf (max) 11.000 10.00%
429.mcf (max) (percentage) 0.880 1.15%
429.mcf (unexplained diff vs previous) 11.000 450.00%
429.mcf (unexplained diff vs previous) (percentage) 0.870 443.75%
433.milc 11.000 -
433.milc (avg) 11.000 -
433.milc (avg) (percentage) 1.080 -
433.milc (max) 8.000 300.00%
433.milc (max) (percentage) 0.820 241.67%
433.milc (unexplained diff vs previous) 9.000 350.00%
433.milc (unexplained diff vs previous) (percentage) 0.830 361.11%
434.zeusmp 26.000 -
434.zeusmp (avg) 26.000 -
434.zeusmp (avg) (percentage) 0.950 -
434.zeusmp (max) 25.000 25.00%
434.zeusmp (max) (percentage) 0.940 28.77%
434.zeusmp (unexplained diff vs previous) 26.000 30.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.940 30.56%
435.gromacs 129.000 3.20%
435.gromacs (avg) 129.000 3.20%
435.gromacs (avg) (percentage) 1.710 -
435.gromacs (max) 202.000 -2.88%
435.gromacs (max) (percentage) 2.680 -4.96%
435.gromacs (unexplained diff vs previous) 163.000 56.73%
435.gromacs (unexplained diff vs previous) (percentage) 2.160 53.19%
436.cactusADM 21.000 -
436.cactusADM (avg) 21.000 -
436.cactusADM (avg) (percentage) 0.830 -1.19%
436.cactusADM (max) 5.000 -16.67%
436.cactusADM (max) (percentage) 0.210 -8.70%
436.cactusADM (unexplained diff vs previous) 1.000 -75.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.040 -73.33%
437.leslie3d 557.000 -7.01%
437.leslie3d (avg) 557.000 -7.01%
437.leslie3d (avg) (percentage) 4.900 -
437.leslie3d (max) 873.000 4.68%
437.leslie3d (max) (percentage) 7.670 13.29%
437.leslie3d (unexplained diff vs previous) 945.000 826.47%
437.leslie3d (unexplained diff vs previous) (percentage) 8.300 900.00%
444.namd 9.000 -
444.namd (avg) 9.000 -
444.namd (avg) (percentage) 1.080 -
444.namd (max) 14.000 55.56%
444.namd (max) (percentage) 1.630 64.65%
444.namd (unexplained diff vs previous) 15.000 66.67%
444.namd (unexplained diff vs previous) (percentage) 1.660 69.39%
445.gobmk 43.000 -
445.gobmk (avg) 43.000 -
445.gobmk (avg) (percentage) 0.860 -
445.gobmk (max) 44.000 57.14%
445.gobmk (max) (percentage) 0.870 52.63%
445.gobmk (unexplained diff vs previous) 44.000 144.44%
445.gobmk (unexplained diff vs previous) (percentage) 0.860 138.89%
447.dealII 25.000 -
447.dealII (avg) 25.000 -
447.dealII (avg) (percentage) 1.330 -1.48%
447.dealII (max) 12.000 -7.69%
447.dealII (max) (percentage) 0.640 -5.88%
447.dealII (unexplained diff vs previous) 5.000 -58.33%
447.dealII (unexplained diff vs previous) (percentage) 0.260 -58.06%
450.soplex 14.000 -
450.soplex (avg) 14.000 -
450.soplex (avg) (percentage) 2.490 -1.19%
450.soplex (max) 8.000 -20.00%
450.soplex (max) (percentage) 1.520 -10.06%
450.soplex (unexplained diff vs previous) 4.000 -55.56%
450.soplex (unexplained diff vs previous) (percentage) 0.680 -55.26%
453.povray 7.000 -
453.povray (avg) 7.000 -
453.povray (avg) (percentage) 1.380 -
453.povray (max) 9.000 -
453.povray (max) (percentage) 1.760 -4.86%
453.povray (unexplained diff vs previous) 3.000 -70.00%
453.povray (unexplained diff vs previous) (percentage) 0.560 -70.21%
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.450 -
456.hmmer 21.000 -
456.hmmer (avg) 21.000 -
456.hmmer (avg) (percentage) 0.520 -
456.hmmer (max) 10.000 -
456.hmmer (max) (percentage) 0.240 -4.00%
456.hmmer (unexplained diff vs previous) 7.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.170 -
458.sjeng 47.000 -
458.sjeng (avg) 47.000 -
458.sjeng (avg) (percentage) 0.620 -
458.sjeng (max) 37.000 -9.76%
458.sjeng (max) (percentage) 0.490 -9.26%
458.sjeng (unexplained diff vs previous) 32.000 255.56%
458.sjeng (unexplained diff vs previous) (percentage) 0.420 250.00%
459.GemsFDTD 76.000 -1.30%
459.GemsFDTD (avg) 76.000 -1.30%
459.GemsFDTD (avg) (percentage) 1.080 -1.82%
459.GemsFDTD (max) 11.000 -42.11%
459.GemsFDTD (max) (percentage) 0.160 -40.74%
459.GemsFDTD (unexplained diff vs previous) 5.000 -61.54%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.070 -61.11%
462.libquantum 2.000 -33.33%
462.libquantum (avg) 2.000 -33.33%
462.libquantum (avg) (percentage) 1.370 -
462.libquantum (max) 3.000 50.00%
462.libquantum (max) (percentage) 1.810 63.06%
462.libquantum (unexplained diff vs previous) 4.000 100.00%
462.libquantum (unexplained diff vs previous) (percentage) 1.840 104.44%
464.h264ref 43.000 -
464.h264ref (avg) 43.000 -
464.h264ref (avg) (percentage) 0.710 -
464.h264ref (max) 46.000 283.33%
464.h264ref (max) (percentage) 0.760 261.90%
464.h264ref (unexplained diff vs previous) 47.000 261.54%
464.h264ref (unexplained diff vs previous) (percentage) 0.770 266.67%
465.tonto 104.000 -
465.tonto (avg) 104.000 -
465.tonto (avg) (percentage) 0.810 -
465.tonto (max) 196.000 -3.92%
465.tonto (max) (percentage) 1.530 -3.77%
465.tonto (unexplained diff vs previous) 5.000 -92.31%
465.tonto (unexplained diff vs previous) (percentage) 0.040 -92.16%
470.lbm 36.000 -2.70%
470.lbm (avg) 36.000 -2.70%
470.lbm (avg) (percentage) 1.000 -
470.lbm (max) 9.000 -
470.lbm (max) (percentage) 0.250 -7.41%
470.lbm (unexplained diff vs previous) 0.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.000 -
471.omnetpp 50.000 -
471.omnetpp (avg) 50.000 -
471.omnetpp (avg) (percentage) 1.850 -
471.omnetpp (max) 61.000 -32.22%
471.omnetpp (max) (percentage) 2.270 -30.58%
471.omnetpp (unexplained diff vs previous) 63.000 -14.86%
471.omnetpp (unexplained diff vs previous) (percentage) 2.330 -12.73%
473.astar 68.000 -
473.astar (avg) 68.000 -
473.astar (avg) (percentage) 1.400 -1.41%
473.astar (max) 53.000 -17.19%
473.astar (max) (percentage) 1.110 -16.54%
473.astar (unexplained diff vs previous) 20.000 -69.23%
473.astar (unexplained diff vs previous) (percentage) 0.410 -69.40%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.570 -
482.sphinx3 (max) 19.000 35.71%
482.sphinx3 (max) (percentage) 2.110 28.66%
482.sphinx3 (unexplained diff vs previous) 19.000 26.67%
482.sphinx3 (unexplained diff vs previous) (percentage) 2.070 23.95%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.500 -
Geometric Mean 4.950 10.12%