Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=ff921dfb5fe6076bb6a2bd99e8e9f8abe5e5bd41 (+ 44 commits)
git_gcc https://gcc.gnu.org/git/gcc.git (56f1863ade1bf416e09da305615ecb2ae04602a8) (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=0852c4aab7870adbd188f7d27985f1631c8596df (+ 100 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/cc2fbfb820fc476698d05a3b39f083124e82b2a3 (+ 337 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/344/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0344 (3-changed-components) 2024-12-22T14:15:04 0:00:00
Previous 0344 (glibc-2.36.9000-00340-g8fb923ddc3) 2022-11-29T00:48:43 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 73 12
Performance Improvements 120 176
Added Tests 2 2
Unchanged Tests 1041 1046
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
465.tonto 1.19% 1948611.000 1971787.000 - 1.32% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 34.14% 8303.000 11138.000 - 1.80% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) 7375.00% 0.040 2.990 - -99.97% -
470.lbm (unexplained diff vs previous) (percentage) 3300.00% 0.010 0.340 - -99.87% -
437.leslie3d (unexplained diff vs previous) (percentage) 2594.74% 0.190 5.120 - -74.25% -
465.tonto (unexplained diff vs previous) 2300.00% 1.000 24.000 - -98.64% -
465.tonto (unexplained diff vs previous) (percentage) 2100.00% 0.010 0.220 - -98.64% -
437.leslie3d (unexplained diff vs previous) 2003.57% 28.000 589.000 - -74.55% -
429.mcf (avg) (percentage) 1963.64% 0.220 4.540 - -40.81% -
453.povray (max) (percentage) 1846.67% 0.150 2.920 - -96.80% -
403.gcc (avg) (percentage) 1442.42% 0.330 5.090 - -30.46% -
444.namd (unexplained diff vs previous) (percentage) 1250.00% 0.040 0.540 - -99.95% -
470.lbm (unexplained diff vs previous) 1200.00% 1.000 13.000 - -99.87% -
401.bzip2 (unexplained diff vs previous) (percentage) 1000.00% 0.090 0.990 - -99.73% -
473.astar (avg) (percentage) 850.00% 0.160 1.520 - -63.46% -
462.libquantum (max) (percentage) 835.48% 0.310 2.900 - -97.07% -
403.gcc (max) (percentage) 788.10% 0.420 3.730 - -96.25% -
483.xalancbmk (avg) (percentage) 678.95% 0.380 2.960 - -54.46% -
462.libquantum (avg) (percentage) 567.86% 0.280 1.870 - -75.59% -
454.calculix (max) (percentage) 485.71% 0.140 0.820 - -99.18% -
458.sjeng (avg) (percentage) 470.00% 0.100 0.570 - -82.13% -
458.sjeng (unexplained diff vs previous) (percentage) 450.00% 0.040 0.220 - -99.76% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 375.00% 0.240 1.140 - -99.54% -
401.bzip2 (avg) (percentage) 346.67% 0.300 1.340 - -75.55% -
429.mcf (unexplained diff vs previous) (percentage) 333.33% 0.090 0.390 - -99.97% -
471.omnetpp (unexplained diff vs previous) (percentage) 333.33% 0.060 0.260 - -99.93% -
473.astar 250.00% 22.000 77.000 - -63.85% -
473.astar (avg) 250.00% 22.000 77.000 - -63.85% -
458.sjeng 192.86% 14.000 41.000 - -82.99% -
458.sjeng (avg) 192.86% 14.000 41.000 - -82.99% -
429.mcf (max) (percentage) 178.57% 0.280 0.780 - -99.16% -
435.gromacs (unexplained diff vs previous) (percentage) 166.67% 0.300 0.800 - -99.17% -
458.sjeng (unexplained diff vs previous) 166.67% 6.000 16.000 - -99.77% -
454.calculix (avg) (percentage) 166.67% 1.350 3.600 - -55.11% -
471.omnetpp (avg) (percentage) 165.79% 0.760 2.020 - -75.37% -
429.mcf 148.65% 37.000 92.000 - -15.60% -
429.mcf (avg) 148.65% 37.000 92.000 - -15.60% -
401.bzip2 (max) (percentage) 138.10% 0.420 1.000 - -98.73% -
473.astar (max) (percentage) 137.14% 0.350 0.830 - -98.75% -
433.milc (unexplained diff vs previous) (percentage) 136.36% 0.110 0.260 - -99.97% -
401.bzip2 (unexplained diff vs previous) 130.77% 13.000 30.000 - -99.72% -
465.tonto 125.00% 60.000 135.000 - -63.01% -
465.tonto (avg) 125.00% 60.000 135.000 - -63.01% -
483.xalancbmk (max) (percentage) 118.52% 0.810 1.770 - -97.42% -
450.soplex (avg) (percentage) 110.71% 1.120 2.360 - -63.30% -
447.dealII (unexplained diff vs previous) (percentage) 100.00% 0.030 0.060 - -99.99% -
473.astar (unexplained diff vs previous) (percentage) 100.00% 0.030 0.060 - -99.97% -
464.h264ref (avg) (percentage) 97.22% 0.360 0.710 - -86.05% -
445.gobmk (unexplained diff vs previous) (percentage) 86.67% 0.150 0.280 - -99.75% -
459.GemsFDTD (avg) (percentage) 79.71% 0.690 1.240 - -78.77% -
453.povray (max) 77.78% 9.000 16.000 - -96.79% -
437.leslie3d (max) (percentage) 69.28% 3.190 5.400 - -67.43% -
465.tonto (avg) (percentage) 67.12% 0.730 1.220 - -63.47% -
471.omnetpp (max) (percentage) 57.35% 0.680 1.070 - -98.66% -
437.leslie3d (avg) (percentage) 49.66% 2.940 4.400 - 44.26% -
465.tonto (max) 48.28% 116.000 172.000 - -91.86% -
400.perlbench (max) (percentage) 34.86% 1.750 2.360 - -97.19% -
437.leslie3d (max) 32.48% 468.000 620.000 - -67.86% -
453.povray (avg) (percentage) 28.10% 1.210 1.550 - -80.15% -
435.gromacs (unexplained diff vs previous) 23.40% 47.000 58.000 - -99.17% -
445.gobmk (avg) (percentage) 22.06% 0.680 0.830 - -78.77% -
481.wrf (avg) (percentage) 20.00% 1.500 1.800 - -71.15% -
400.perlbench (avg) (percentage) 19.75% 1.620 1.940 - -69.45% -
447.dealII (avg) (percentage) 19.15% 0.940 1.120 - -80.59% -
450.soplex (max) (percentage) 18.38% 1.360 1.610 - -98.33% -
435.gromacs (avg) (percentage) 17.69% 1.470 1.730 - -65.81% -
437.leslie3d 16.90% 432.000 505.000 - 42.25% -
437.leslie3d (avg) 16.90% 432.000 505.000 - 42.25% -
471.omnetpp (unexplained diff vs previous) 16.67% 6.000 7.000 - -99.93% -
465.tonto (max) (percentage) 10.71% 1.400 1.550 - -91.97% -
459.GemsFDTD (unexplained diff vs previous) 7.02% 57.000 61.000 - -99.61% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -8.61% 59491.000 54370.000 - -8.59% -
459.GemsFDTD -5.06% 245040.000 232644.000 - -4.94% -
434.zeusmp -4.44% 153713.000 146883.000 - -3.39% -
444.namd -3.26% 188365.000 182224.000 - -3.27% -
447.dealII -2.02% 312518.000 306198.000 - -1.73% -
462.libquantum -1.99% 16117.000 15797.000 - -1.79% -
458.sjeng -1.63% 104004.000 102311.000 - -1.67% -
410.bwaves -1.62% 16748.000 16476.000 - -1.69% -
470.lbm -1.45% 9687.000 9547.000 - -1.49% -
435.gromacs -1.30% 500327.000 493840.000 - -1.09% -
450.soplex -1.29% 205068.000 202418.000 - -1.30% -
433.milc -1.21% 61222.000 60484.000 - -1.46% -
401.bzip2 -1.20% 44241.000 43711.000 - -1.18% -
471.omnetpp -1.03% 480176.000 475220.000 - -0.59% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.84% 11534.000 134.000 - -4.29% -
482.sphinx3 -97.21% 30157.000 841.000 - -11.01% -
450.soplex -96.35% 16093.000 588.000 - 3.70% -
444.namd -92.36% 9744.000 744.000 - -0.40% -
453.povray -91.50% 6541.000 556.000 - 1.65% -
433.milc -91.28% 13278.000 1158.000 - 0.61% -
429.mcf -88.19% 17210.000 2032.000 - 42.70% -
434.zeusmp -83.27% 13794.000 2308.000 - -8.19% -
447.dealII -82.09% 9170.000 1642.000 - -4.76% -
401.bzip2 -79.62% 14922.000 3041.000 - 3.40% -
459.GemsFDTD -77.32% 23650.000 5364.000 - -13.11% -
471.omnetpp -74.86% 10751.000 2703.000 - -0.77% -
470.lbm -70.18% 12815.000 3821.000 - 0.87% -
473.astar -63.96% 14193.000 5115.000 - -0.25% -
445.gobmk -57.68% 11659.000 4934.000 - -1.77% -
410.bwaves -57.44% 13871.000 5903.000 - 4.44% -
435.gromacs -53.61% 15631.000 7251.000 - -0.25% -
458.sjeng -48.52% 14233.000 7327.000 - -3.17% -
437.leslie3d -21.79% 14700.000 11497.000 - -1.23% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (unexplained diff vs previous) -99.21% 761.000 6.000 - -99.98% -
482.sphinx3 (max) -99.11% 898.000 8.000 - -99.12% -
444.namd (max) -98.78% 739.000 9.000 - -98.69% -
444.namd -98.18% 439.000 8.000 - -88.24% -
444.namd (avg) -98.18% 439.000 8.000 - -88.24% -
470.lbm (max) -97.99% 598.000 12.000 - -99.56% -
482.sphinx3 -97.92% 624.000 13.000 - -82.43% -
482.sphinx3 (avg) -97.92% 624.000 13.000 - -82.43% -
434.zeusmp (max) -97.79% 271.000 6.000 - -99.71% -
450.soplex (unexplained diff vs previous) -97.37% 38.000 1.000 - -99.99% -
433.milc (max) -96.61% 177.000 6.000 - -99.43% -
450.soplex (max) -95.87% 218.000 9.000 - -98.35% -
410.bwaves (max) -95.37% 518.000 24.000 - -99.27% -
433.milc -94.49% 236.000 13.000 - -81.16% -
433.milc (avg) -94.49% 236.000 13.000 - -81.16% -
462.libquantum -93.75% 32.000 2.000 - -80.00% -
462.libquantum (avg) -93.75% 32.000 2.000 - -80.00% -
450.soplex -92.78% 180.000 13.000 - -63.89% -
450.soplex (avg) -92.78% 180.000 13.000 - -63.89% -
470.lbm (max) (percentage) -92.72% 4.670 0.340 - -99.53% -
462.libquantum (max) -91.43% 35.000 3.000 - -97.83% -
434.zeusmp -90.78% 217.000 20.000 - -86.39% -
434.zeusmp (avg) -90.78% 217.000 20.000 - -86.39% -
453.povray -89.87% 79.000 8.000 - -80.95% -
453.povray (avg) -89.87% 79.000 8.000 - -80.95% -
410.bwaves (max) (percentage) -88.77% 3.740 0.420 - -99.28% -
416.gamess (max) (percentage) -87.00% 2.770 0.360 - -99.07% -
434.zeusmp (max) (percentage) -86.80% 1.970 0.260 - -99.68% -
445.gobmk (max) -86.79% 106.000 14.000 - -99.47% -
470.lbm -85.78% 211.000 30.000 - -87.45% -
470.lbm (avg) -85.78% 211.000 30.000 - -87.45% -
434.zeusmp (unexplained diff vs previous) -84.21% 38.000 6.000 - -99.95% -
447.dealII (max) -84.07% 113.000 18.000 - -98.70% -
444.namd (max) (percentage) -83.53% 7.590 1.250 - -98.64% -
459.GemsFDTD (max) -82.06% 340.000 61.000 - -98.62% -
433.milc (unexplained diff vs previous) -80.00% 15.000 3.000 - -99.97% -
410.bwaves -79.83% 238.000 48.000 - -80.72% -
410.bwaves (avg) -79.83% 238.000 48.000 - -80.72% -
481.wrf (max) (percentage) -79.54% 3.910 0.800 - -98.84% -
447.dealII -79.07% 86.000 18.000 - -81.82% -
447.dealII (avg) -79.07% 86.000 18.000 - -81.82% -
410.bwaves (unexplained diff vs previous) -76.64% 107.000 25.000 - -99.68% -
444.namd (avg) (percentage) -74.50% 4.510 1.150 - -87.53% -
435.gromacs (max) -72.48% 367.000 101.000 - -97.17% -
458.sjeng (max) -72.41% 58.000 16.000 - -99.56% -
482.sphinx3 (unexplained diff vs previous) (percentage) -71.83% 2.520 0.710 - -99.97% -
429.mcf (max) -68.75% 48.000 15.000 - -98.87% -
445.gobmk (max) (percentage) -68.13% 0.910 0.290 - -99.45% -
447.dealII (unexplained diff vs previous) -66.67% 3.000 1.000 - -99.99% -
482.sphinx3 (max) (percentage) -64.77% 2.980 1.050 - -98.91% -
471.omnetpp (max) -61.64% 73.000 28.000 - -98.71% -
459.GemsFDTD -59.51% 163.000 66.000 - -81.67% -
459.GemsFDTD (avg) -59.51% 163.000 66.000 - -81.67% -
436.cactusADM (max) (percentage) -58.99% 2.170 0.890 - -99.00% -
416.gamess (avg) (percentage) -56.07% 1.730 0.760 - -82.81% -
433.milc (max) (percentage) -55.97% 1.340 0.590 - -99.35% -
456.hmmer (max) (percentage) -54.65% 0.860 0.390 - -99.47% -
410.bwaves (avg) (percentage) -52.33% 1.720 0.820 - -81.41% -
470.lbm (avg) (percentage) -52.12% 1.650 0.790 - -87.52% -
401.bzip2 (max) -51.61% 62.000 30.000 - -98.70% -
429.mcf (unexplained diff vs previous) -50.00% 16.000 8.000 - -99.96% -
445.gobmk -49.37% 79.000 40.000 - -79.59% -
445.gobmk (avg) -49.37% 79.000 40.000 - -79.59% -
458.sjeng (max) (percentage) -46.34% 0.410 0.220 - -99.54% -
410.bwaves (unexplained diff vs previous) (percentage) -45.45% 0.770 0.420 - -99.70% -
435.gromacs -45.41% 229.000 125.000 - -65.94% -
435.gromacs (avg) -45.41% 229.000 125.000 - -65.94% -
434.zeusmp (avg) (percentage) -44.30% 1.580 0.880 - -84.96% -
435.gromacs (max) (percentage) -40.43% 2.350 1.400 - -97.15% -
436.cactusADM (avg) (percentage) -40.00% 1.750 1.050 - -82.44% -
464.h264ref (max) (percentage) -35.00% 0.600 0.390 - -99.28% -
471.omnetpp -33.33% 81.000 54.000 - -75.78% -
471.omnetpp (avg) -33.33% 81.000 54.000 - -75.78% -
433.milc (avg) (percentage) -33.15% 1.780 1.190 - -80.33% -
450.soplex (unexplained diff vs previous) (percentage) -29.17% 0.240 0.170 - -99.99% -
473.astar (unexplained diff vs previous) -25.00% 4.000 3.000 - -99.97% -
445.gobmk (unexplained diff vs previous) -22.22% 18.000 14.000 - -99.75% -
459.GemsFDTD (max) (percentage) -20.14% 1.440 1.150 - -98.39% -
462.libquantum (unexplained diff vs previous) -20.00% 5.000 4.000 - -99.97% -
482.sphinx3 (avg) (percentage) -19.81% 2.070 1.660 - -79.07% -
473.astar (max) -14.29% 49.000 42.000 - -98.77% -
401.bzip2 -9.09% 44.000 40.000 - -75.16% -
401.bzip2 (avg) -9.09% 44.000 40.000 - -75.16% -
447.dealII (max) (percentage) -8.06% 1.240 1.140 - -98.58% -
434.zeusmp (unexplained diff vs previous) (percentage) -7.14% 0.280 0.260 - -99.94% -
456.hmmer (avg) (percentage) -5.45% 0.550 0.520 - -88.21% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.32% 1946163.000 1971787.000 - 1.19% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 42.70% 1424.000 2032.000 - -88.19% -
410.bwaves 4.44% 5652.000 5903.000 - -57.44% -
450.soplex 3.70% 567.000 588.000 - -96.35% -
401.bzip2 3.40% 2941.000 3041.000 - -79.62% -
465.tonto 1.80% 10941.000 11138.000 - 34.14% -
453.povray 1.65% 547.000 556.000 - -91.50% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 44.26% 3.050 4.400 - 49.66% -
437.leslie3d 42.25% 355.000 505.000 - 16.90% -
437.leslie3d (avg) 42.25% 355.000 505.000 - 16.90% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -8.59% 59479.000 54370.000 - -8.61% -
459.GemsFDTD -4.94% 244728.000 232644.000 - -5.06% -
434.zeusmp -3.39% 152037.000 146883.000 - -4.44% -
444.namd -3.27% 188385.000 182224.000 - -3.26% -
462.libquantum -1.79% 16085.000 15797.000 - -1.99% -
447.dealII -1.73% 311591.000 306198.000 - -2.02% -
410.bwaves -1.69% 16760.000 16476.000 - -1.62% -
458.sjeng -1.67% 104048.000 102311.000 - -1.63% -
470.lbm -1.49% 9691.000 9547.000 - -1.45% -
433.milc -1.46% 61382.000 60484.000 - -1.21% -
450.soplex -1.30% 205081.000 202418.000 - -1.29% -
401.bzip2 -1.18% 44233.000 43711.000 - -1.20% -
435.gromacs -1.09% 499261.000 493840.000 - -1.30% -
482.sphinx3 -1.01% 112859.000 111718.000 - -0.99% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -13.11% 6173.000 5364.000 - -77.32% -
482.sphinx3 -11.01% 945.000 841.000 - -97.21% -
434.zeusmp -8.19% 2514.000 2308.000 - -83.27% -
447.dealII -4.76% 1724.000 1642.000 - -82.09% -
462.libquantum -4.29% 140.000 134.000 - -98.84% -
458.sjeng -3.17% 7567.000 7327.000 - -48.52% -
445.gobmk -1.77% 5023.000 4934.000 - -57.68% -
437.leslie3d -1.23% 11640.000 11497.000 - -21.79% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
450.soplex (unexplained diff vs previous) (percentage) -99.99% 2557.320 0.170 - -29.17% -
450.soplex (unexplained diff vs previous) -99.99% 14500.000 1.000 - -97.37% -
447.dealII (unexplained diff vs previous) -99.99% 7017.000 1.000 - -66.67% -
447.dealII (unexplained diff vs previous) (percentage) -99.99% 407.020 0.060 - 100.00% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 6.000 - -99.21% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.97% 2759.050 0.710 - -71.83% -
433.milc (unexplained diff vs previous) (percentage) -99.97% 980.800 0.260 - 136.36% -
433.milc (unexplained diff vs previous) -99.97% 11289.000 3.000 - -80.00% -
473.astar (unexplained diff vs previous) -99.97% 10146.000 3.000 - -25.00% -
462.libquantum (unexplained diff vs previous) -99.97% 13518.000 4.000 - -20.00% -
429.mcf (unexplained diff vs previous) (percentage) -99.97% 1305.690 0.390 - 333.33% -
473.astar (unexplained diff vs previous) (percentage) -99.97% 197.850 0.060 - 100.00% -
462.libquantum (unexplained diff vs previous) (percentage) -99.97% 9655.710 2.990 - 7375.00% -
429.mcf (unexplained diff vs previous) -99.96% 18593.000 8.000 - -50.00% -
444.namd (unexplained diff vs previous) -99.95% 8356.000 4.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) -99.95% 1118.610 0.540 - 1250.00% -
434.zeusmp (unexplained diff vs previous) -99.95% 11280.000 6.000 - -84.21% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.94% 448.690 0.260 - -7.14% -
471.omnetpp (unexplained diff vs previous) -99.93% 10659.000 7.000 - 16.67% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.93% 391.300 0.260 - 333.33% -
470.lbm (unexplained diff vs previous) (percentage) -99.87% 263.730 0.340 - 3300.00% -
470.lbm (unexplained diff vs previous) -99.87% 9990.000 13.000 - 1200.00% -
458.sjeng (unexplained diff vs previous) -99.77% 6876.000 16.000 - 166.67% -
458.sjeng (unexplained diff vs previous) (percentage) -99.76% 90.870 0.220 - 450.00% -
445.gobmk (unexplained diff vs previous) -99.75% 5581.000 14.000 - -22.22% -
445.gobmk (unexplained diff vs previous) (percentage) -99.75% 111.110 0.280 - 86.67% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.73% 368.340 0.990 - 1000.00% -
401.bzip2 (unexplained diff vs previous) -99.72% 10833.000 30.000 - 130.77% -
434.zeusmp (max) -99.71% 2055.000 6.000 - -97.79% -
410.bwaves (unexplained diff vs previous) (percentage) -99.70% 137.900 0.420 - -45.45% -
434.zeusmp (max) (percentage) -99.68% 81.770 0.260 - -86.80% -
410.bwaves (unexplained diff vs previous) -99.68% 7794.000 25.000 - -76.64% -
459.GemsFDTD (unexplained diff vs previous) -99.61% 15458.000 61.000 - 7.02% -
470.lbm (max) -99.56% 2746.000 12.000 - -97.99% -
458.sjeng (max) -99.56% 3602.000 16.000 - -72.41% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.54% 250.410 1.140 - 375.00% -
458.sjeng (max) (percentage) -99.54% 47.610 0.220 - -46.34% -
470.lbm (max) (percentage) -99.53% 72.510 0.340 - -92.72% -
445.gobmk (max) -99.47% 2643.000 14.000 - -86.79% -
456.hmmer (max) (percentage) -99.47% 73.590 0.390 - -54.65% -
445.gobmk (max) (percentage) -99.45% 52.630 0.290 - -68.13% -
433.milc (max) -99.43% 1044.000 6.000 - -96.61% -
433.milc (max) (percentage) -99.35% 90.750 0.590 - -55.97% -
464.h264ref (max) (percentage) -99.28% 54.290 0.390 - -35.00% -
410.bwaves (max) (percentage) -99.28% 57.970 0.420 - -88.77% -
410.bwaves (max) -99.27% 3276.000 24.000 - -95.37% -
454.calculix (max) (percentage) -99.18% 99.730 0.820 - 485.71% -
435.gromacs (unexplained diff vs previous) -99.17% 7028.000 58.000 - 23.40% -
435.gromacs (unexplained diff vs previous) (percentage) -99.17% 96.680 0.800 - 166.67% -
429.mcf (max) (percentage) -99.16% 92.890 0.780 - 178.57% -
482.sphinx3 (max) -99.12% 911.000 8.000 - -99.11% -
416.gamess (max) (percentage) -99.07% 38.550 0.360 - -87.00% -
436.cactusADM (max) (percentage) -99.00% 88.730 0.890 - -58.99% -
482.sphinx3 (max) (percentage) -98.91% 96.500 1.050 - -64.77% -
429.mcf (max) -98.87% 1322.000 15.000 - -68.75% -
481.wrf (max) (percentage) -98.84% 68.700 0.800 - -79.54% -
473.astar (max) -98.77% 3406.000 42.000 - -14.29% -
473.astar (max) (percentage) -98.75% 66.430 0.830 - 137.14% -
401.bzip2 (max) (percentage) -98.73% 78.650 1.000 - 138.10% -
471.omnetpp (max) -98.71% 2169.000 28.000 - -61.64% -
401.bzip2 (max) -98.70% 2313.000 30.000 - -51.61% -
447.dealII (max) -98.70% 1384.000 18.000 - -84.07% -
444.namd (max) -98.69% 685.000 9.000 - -98.78% -
471.omnetpp (max) (percentage) -98.66% 79.650 1.070 - 57.35% -
465.tonto (unexplained diff vs previous) -98.64% 1771.000 24.000 - 2300.00% -
465.tonto (unexplained diff vs previous) (percentage) -98.64% 16.190 0.220 - 2100.00% -
444.namd (max) (percentage) -98.64% 91.790 1.250 - -83.53% -
459.GemsFDTD (max) -98.62% 4411.000 61.000 - -82.06% -
447.dealII (max) (percentage) -98.58% 80.280 1.140 - -8.06% -
459.GemsFDTD (max) (percentage) -98.39% 71.460 1.150 - -20.14% -
450.soplex (max) -98.35% 545.000 9.000 - -95.87% -
450.soplex (max) (percentage) -98.33% 96.240 1.610 - 18.38% -
462.libquantum (max) -97.83% 138.000 3.000 - -91.43% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 118.52% -
400.perlbench (max) (percentage) -97.19% 84.060 2.360 - 34.86% -
435.gromacs (max) -97.17% 3573.000 101.000 - -72.48% -
435.gromacs (max) (percentage) -97.15% 49.160 1.400 - -40.43% -
462.libquantum (max) (percentage) -97.07% 98.970 2.900 - 835.48% -
453.povray (max) (percentage) -96.80% 91.250 2.920 - 1846.67% -
453.povray (max) -96.79% 499.000 16.000 - 77.78% -
403.gcc (max) (percentage) -96.25% 99.360 3.730 - 788.10% -
465.tonto (max) (percentage) -91.97% 19.310 1.550 - 10.71% -
465.tonto (max) -91.86% 2112.000 172.000 - 48.28% -
444.namd -88.24% 68.000 8.000 - -98.18% -
444.namd (avg) -88.24% 68.000 8.000 - -98.18% -
456.hmmer (avg) (percentage) -88.21% 4.410 0.520 - -5.45% -
444.namd (avg) (percentage) -87.53% 9.220 1.150 - -74.50% -
470.lbm (avg) (percentage) -87.52% 6.330 0.790 - -52.12% -
470.lbm -87.45% 239.000 30.000 - -85.78% -
470.lbm (avg) -87.45% 239.000 30.000 - -85.78% -
434.zeusmp -86.39% 147.000 20.000 - -90.78% -
434.zeusmp (avg) -86.39% 147.000 20.000 - -90.78% -
464.h264ref (avg) (percentage) -86.05% 5.090 0.710 - 97.22% -
434.zeusmp (avg) (percentage) -84.96% 5.850 0.880 - -44.30% -
458.sjeng -82.99% 241.000 41.000 - 192.86% -
458.sjeng (avg) -82.99% 241.000 41.000 - 192.86% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - -56.07% -
436.cactusADM (avg) (percentage) -82.44% 5.980 1.050 - -40.00% -
482.sphinx3 -82.43% 74.000 13.000 - -97.92% -
482.sphinx3 (avg) -82.43% 74.000 13.000 - -97.92% -
458.sjeng (avg) (percentage) -82.13% 3.190 0.570 - 470.00% -
447.dealII -81.82% 99.000 18.000 - -79.07% -
447.dealII (avg) -81.82% 99.000 18.000 - -79.07% -
459.GemsFDTD -81.67% 360.000 66.000 - -59.51% -
459.GemsFDTD (avg) -81.67% 360.000 66.000 - -59.51% -
410.bwaves (avg) (percentage) -81.41% 4.410 0.820 - -52.33% -
433.milc -81.16% 69.000 13.000 - -94.49% -
433.milc (avg) -81.16% 69.000 13.000 - -94.49% -
453.povray -80.95% 42.000 8.000 - -89.87% -
453.povray (avg) -80.95% 42.000 8.000 - -89.87% -
410.bwaves -80.72% 249.000 48.000 - -79.83% -
410.bwaves (avg) -80.72% 249.000 48.000 - -79.83% -
447.dealII (avg) (percentage) -80.59% 5.770 1.120 - 19.15% -
433.milc (avg) (percentage) -80.33% 6.050 1.190 - -33.15% -
453.povray (avg) (percentage) -80.15% 7.810 1.550 - 28.10% -
462.libquantum -80.00% 10.000 2.000 - -93.75% -
462.libquantum (avg) -80.00% 10.000 2.000 - -93.75% -
445.gobmk -79.59% 196.000 40.000 - -49.37% -
445.gobmk (avg) -79.59% 196.000 40.000 - -49.37% -
482.sphinx3 (avg) (percentage) -79.07% 7.930 1.660 - -19.81% -
445.gobmk (avg) (percentage) -78.77% 3.910 0.830 - 22.06% -
459.GemsFDTD (avg) (percentage) -78.77% 5.840 1.240 - 79.71% -
471.omnetpp -75.78% 223.000 54.000 - -33.33% -
471.omnetpp (avg) -75.78% 223.000 54.000 - -33.33% -
462.libquantum (avg) (percentage) -75.59% 7.660 1.870 - 567.86% -
401.bzip2 (avg) (percentage) -75.55% 5.480 1.340 - 346.67% -
471.omnetpp (avg) (percentage) -75.37% 8.200 2.020 - 165.79% -
401.bzip2 -75.16% 161.000 40.000 - -9.09% -
401.bzip2 (avg) -75.16% 161.000 40.000 - -9.09% -
437.leslie3d (unexplained diff vs previous) -74.55% 2314.000 589.000 - 2003.57% -
437.leslie3d (unexplained diff vs previous) (percentage) -74.25% 19.880 5.120 - 2594.74% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 20.00% -
400.perlbench (avg) (percentage) -69.45% 6.350 1.940 - 19.75% -
437.leslie3d (max) -67.86% 1929.000 620.000 - 32.48% -
437.leslie3d (max) (percentage) -67.43% 16.580 5.400 - 69.28% -
435.gromacs -65.94% 367.000 125.000 - -45.41% -
435.gromacs (avg) -65.94% 367.000 125.000 - -45.41% -
435.gromacs (avg) (percentage) -65.81% 5.060 1.730 - 17.69% -
450.soplex -63.89% 36.000 13.000 - -92.78% -
450.soplex (avg) -63.89% 36.000 13.000 - -92.78% -
473.astar -63.85% 213.000 77.000 - 250.00% -
473.astar (avg) -63.85% 213.000 77.000 - 250.00% -
465.tonto (avg) (percentage) -63.47% 3.340 1.220 - 67.12% -
473.astar (avg) (percentage) -63.46% 4.160 1.520 - 850.00% -
450.soplex (avg) (percentage) -63.30% 6.430 2.360 - 110.71% -
465.tonto -63.01% 365.000 135.000 - 125.00% -
465.tonto (avg) -63.01% 365.000 135.000 - 125.00% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 166.67% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 678.95% -
429.mcf (avg) (percentage) -40.81% 7.670 4.540 - 1963.64% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 1442.42% -
429.mcf -15.60% 109.000 92.000 - 148.65% -
429.mcf (avg) -15.60% 109.000 92.000 - 148.65% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)

Report Time: 0.39s
Execution (seconds)
Name Current %
401.bzip2 3041.000 -79.62%
410.bwaves 5903.000 -57.44%
429.mcf 2032.000 -88.19%
433.milc 1158.000 -91.28%
434.zeusmp 2308.000 -83.27%
435.gromacs 7251.000 -53.61%
437.leslie3d 11497.000 -21.79%
444.namd 744.000 -92.36%
445.gobmk 4934.000 -57.68%
447.dealII 1642.000 -82.09%
450.soplex 588.000 -96.35%
453.povray 556.000 -91.50%
458.sjeng 7327.000 -48.52%
459.GemsFDTD 5364.000 -77.32%
462.libquantum 134.000 -98.84%
465.tonto 11138.000 34.14%
470.lbm 3821.000 -70.18%
471.omnetpp 2703.000 -74.86%
473.astar 5115.000 -63.96%
482.sphinx3 841.000 -97.21%
Geometric Mean 2404.509 -81.96%
Code Size (bytes)
Name Current %
401.bzip2 43711.000 -1.20%
410.bwaves 16476.000 -1.62%
429.mcf 9146.000 -
433.milc 60484.000 -1.21%
434.zeusmp 146883.000 -4.44%
435.gromacs 493840.000 -1.30%
437.leslie3d 54370.000 -8.61%
444.namd 182224.000 -3.26%
445.gobmk 1375090.000 -
447.dealII 306198.000 -2.02%
450.soplex 202418.000 -1.29%
453.povray 658001.000 -
458.sjeng 102311.000 -1.63%
459.GemsFDTD 232644.000 -5.06%
462.libquantum 15797.000 -1.99%
465.tonto 1971787.000 1.19%
470.lbm 9547.000 -1.45%
471.omnetpp 475220.000 -1.03%
473.astar 22752.000 -
482.sphinx3 111718.000 -
Geometric Mean 113993.467 -1.85%
Score
Name Current %
nb_compile_successful 20.000 -31.03%
nb_execution_successful 20.000 -31.03%
nb_total_tests 38.000 31.03%
Geometric Mean 24.771 -14.58%
sample variability
Name Current %
400.perlbench (avg) (percentage) 1.940 19.75%
400.perlbench (max) (percentage) 2.360 34.86%
401.bzip2 40.000 -9.09%
401.bzip2 (avg) 40.000 -9.09%
401.bzip2 (avg) (percentage) 1.340 346.67%
401.bzip2 (max) 30.000 -51.61%
401.bzip2 (max) (percentage) 1.000 138.10%
401.bzip2 (unexplained diff vs previous) 30.000 130.77%
401.bzip2 (unexplained diff vs previous) (percentage) 0.990 1000.00%
403.gcc (avg) (percentage) 5.090 1442.42%
403.gcc (max) (percentage) 3.730 788.10%
410.bwaves 48.000 -79.83%
410.bwaves (avg) 48.000 -79.83%
410.bwaves (avg) (percentage) 0.820 -52.33%
410.bwaves (max) 24.000 -95.37%
410.bwaves (max) (percentage) 0.420 -88.77%
410.bwaves (unexplained diff vs previous) 25.000 -76.64%
410.bwaves (unexplained diff vs previous) (percentage) 0.420 -45.45%
416.gamess (avg) (percentage) 0.760 -56.07%
416.gamess (max) (percentage) 0.360 -87.00%
429.mcf 92.000 148.65%
429.mcf (avg) 92.000 148.65%
429.mcf (avg) (percentage) 4.540 1963.64%
429.mcf (max) 15.000 -68.75%
429.mcf (max) (percentage) 0.780 178.57%
429.mcf (unexplained diff vs previous) 8.000 -50.00%
429.mcf (unexplained diff vs previous) (percentage) 0.390 333.33%
433.milc 13.000 -94.49%
433.milc (avg) 13.000 -94.49%
433.milc (avg) (percentage) 1.190 -33.15%
433.milc (max) 6.000 -96.61%
433.milc (max) (percentage) 0.590 -55.97%
433.milc (unexplained diff vs previous) 3.000 -80.00%
433.milc (unexplained diff vs previous) (percentage) 0.260 136.36%
434.zeusmp 20.000 -90.78%
434.zeusmp (avg) 20.000 -90.78%
434.zeusmp (avg) (percentage) 0.880 -44.30%
434.zeusmp (max) 6.000 -97.79%
434.zeusmp (max) (percentage) 0.260 -86.80%
434.zeusmp (unexplained diff vs previous) 6.000 -84.21%
434.zeusmp (unexplained diff vs previous) (percentage) 0.260 -7.14%
435.gromacs 125.000 -45.41%
435.gromacs (avg) 125.000 -45.41%
435.gromacs (avg) (percentage) 1.730 17.69%
435.gromacs (max) 101.000 -72.48%
435.gromacs (max) (percentage) 1.400 -40.43%
435.gromacs (unexplained diff vs previous) 58.000 23.40%
435.gromacs (unexplained diff vs previous) (percentage) 0.800 166.67%
436.cactusADM (avg) (percentage) 1.050 -40.00%
436.cactusADM (max) (percentage) 0.890 -58.99%
437.leslie3d 505.000 16.90%
437.leslie3d (avg) 505.000 16.90%
437.leslie3d (avg) (percentage) 4.400 49.66%
437.leslie3d (max) 620.000 32.48%
437.leslie3d (max) (percentage) 5.400 69.28%
437.leslie3d (unexplained diff vs previous) 589.000 2003.57%
437.leslie3d (unexplained diff vs previous) (percentage) 5.120 2594.74%
444.namd 8.000 -98.18%
444.namd (avg) 8.000 -98.18%
444.namd (avg) (percentage) 1.150 -74.50%
444.namd (max) 9.000 -98.78%
444.namd (max) (percentage) 1.250 -83.53%
444.namd (unexplained diff vs previous) 4.000 -
444.namd (unexplained diff vs previous) (percentage) 0.540 1250.00%
445.gobmk 40.000 -49.37%
445.gobmk (avg) 40.000 -49.37%
445.gobmk (avg) (percentage) 0.830 22.06%
445.gobmk (max) 14.000 -86.79%
445.gobmk (max) (percentage) 0.290 -68.13%
445.gobmk (unexplained diff vs previous) 14.000 -22.22%
445.gobmk (unexplained diff vs previous) (percentage) 0.280 86.67%
447.dealII 18.000 -79.07%
447.dealII (avg) 18.000 -79.07%
447.dealII (avg) (percentage) 1.120 19.15%
447.dealII (max) 18.000 -84.07%
447.dealII (max) (percentage) 1.140 -8.06%
447.dealII (unexplained diff vs previous) 1.000 -66.67%
447.dealII (unexplained diff vs previous) (percentage) 0.060 100.00%
450.soplex 13.000 -92.78%
450.soplex (avg) 13.000 -92.78%
450.soplex (avg) (percentage) 2.360 110.71%
450.soplex (max) 9.000 -95.87%
450.soplex (max) (percentage) 1.610 18.38%
450.soplex (unexplained diff vs previous) 1.000 -97.37%
450.soplex (unexplained diff vs previous) (percentage) 0.170 -29.17%
453.povray 8.000 -89.87%
453.povray (avg) 8.000 -89.87%
453.povray (avg) (percentage) 1.550 28.10%
453.povray (max) 16.000 77.78%
453.povray (max) (percentage) 2.920 1846.67%
453.povray (unexplained diff vs previous) 0.000 -
453.povray (unexplained diff vs previous) (percentage) 0.000 -
454.calculix (avg) (percentage) 3.600 166.67%
454.calculix (max) (percentage) 0.820 485.71%
456.hmmer (avg) (percentage) 0.520 -5.45%
456.hmmer (max) (percentage) 0.390 -54.65%
458.sjeng 41.000 192.86%
458.sjeng (avg) 41.000 192.86%
458.sjeng (avg) (percentage) 0.570 470.00%
458.sjeng (max) 16.000 -72.41%
458.sjeng (max) (percentage) 0.220 -46.34%
458.sjeng (unexplained diff vs previous) 16.000 166.67%
458.sjeng (unexplained diff vs previous) (percentage) 0.220 450.00%
459.GemsFDTD 66.000 -59.51%
459.GemsFDTD (avg) 66.000 -59.51%
459.GemsFDTD (avg) (percentage) 1.240 79.71%
459.GemsFDTD (max) 61.000 -82.06%
459.GemsFDTD (max) (percentage) 1.150 -20.14%
459.GemsFDTD (unexplained diff vs previous) 61.000 7.02%
459.GemsFDTD (unexplained diff vs previous) (percentage) 1.140 375.00%
462.libquantum 2.000 -93.75%
462.libquantum (avg) 2.000 -93.75%
462.libquantum (avg) (percentage) 1.870 567.86%
462.libquantum (max) 3.000 -91.43%
462.libquantum (max) (percentage) 2.900 835.48%
462.libquantum (unexplained diff vs previous) 4.000 -20.00%
462.libquantum (unexplained diff vs previous) (percentage) 2.990 7375.00%
464.h264ref (avg) (percentage) 0.710 97.22%
464.h264ref (max) (percentage) 0.390 -35.00%
465.tonto 135.000 125.00%
465.tonto (avg) 135.000 125.00%
465.tonto (avg) (percentage) 1.220 67.12%
465.tonto (max) 172.000 48.28%
465.tonto (max) (percentage) 1.550 10.71%
465.tonto (unexplained diff vs previous) 24.000 2300.00%
465.tonto (unexplained diff vs previous) (percentage) 0.220 2100.00%
470.lbm 30.000 -85.78%
470.lbm (avg) 30.000 -85.78%
470.lbm (avg) (percentage) 0.790 -52.12%
470.lbm (max) 12.000 -97.99%
470.lbm (max) (percentage) 0.340 -92.72%
470.lbm (unexplained diff vs previous) 13.000 1200.00%
470.lbm (unexplained diff vs previous) (percentage) 0.340 3300.00%
471.omnetpp 54.000 -33.33%
471.omnetpp (avg) 54.000 -33.33%
471.omnetpp (avg) (percentage) 2.020 165.79%
471.omnetpp (max) 28.000 -61.64%
471.omnetpp (max) (percentage) 1.070 57.35%
471.omnetpp (unexplained diff vs previous) 7.000 16.67%
471.omnetpp (unexplained diff vs previous) (percentage) 0.260 333.33%
473.astar 77.000 250.00%
473.astar (avg) 77.000 250.00%
473.astar (avg) (percentage) 1.520 850.00%
473.astar (max) 42.000 -14.29%
473.astar (max) (percentage) 0.830 137.14%
473.astar (unexplained diff vs previous) 3.000 -25.00%
473.astar (unexplained diff vs previous) (percentage) 0.060 100.00%
481.wrf (avg) (percentage) 1.800 20.00%
481.wrf (max) (percentage) 0.800 -79.54%
482.sphinx3 13.000 -97.92%
482.sphinx3 (avg) 13.000 -97.92%
482.sphinx3 (avg) (percentage) 1.660 -19.81%
482.sphinx3 (max) 8.000 -99.11%
482.sphinx3 (max) (percentage) 1.050 -64.77%
482.sphinx3 (unexplained diff vs previous) 6.000 -99.21%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.710 -71.83%
483.xalancbmk (avg) (percentage) 2.960 678.95%
483.xalancbmk (max) (percentage) 1.770 118.52%
Geometric Mean 4.010 -38.45%