Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/8d373ceaec1f1b27c9e682cfaf71aae19ea48d98 (+ 344 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/228/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0228 (llvmorg-21-init-01235-g8d373ceaec1f) 2025-02-05T14:53:54 0:00:00
Previous 0228 (llvmorg-16-init-15922-ga0b470c9848c) 2022-12-24T12:32:16 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 41 16
Performance Improvements 101 128
Added Tests 10 6
Unchanged Tests 616 618
Total Tests 768

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
450.soplex 2.22% 146765.000 150028.000 - 2.22% -
471.omnetpp 2.08% 316676.000 323250.000 - 2.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) (percentage) 3055.56% 0.090 2.840 - -55.49% -
444.namd (unexplained diff vs previous) (percentage) 3000.00% 0.020 0.620 - 933.33% -
403.gcc (max) (percentage) 1495.45% 0.440 7.020 - 201.29% -
482.sphinx3 (max) (percentage) 1431.58% 0.190 2.910 - -12.08% -
462.libquantum (avg) (percentage) 1115.38% 0.130 1.580 - 105.19% -
403.gcc (avg) (percentage) 988.24% 0.340 3.700 - 572.73% -
473.astar (max) (percentage) 928.57% 0.070 0.720 - -89.25% -
462.libquantum (unexplained diff vs previous) (percentage) 587.50% 0.080 0.550 - 5400.00% -
444.namd (max) (percentage) 463.64% 0.110 0.620 - -92.34% -
447.dealII (unexplained diff vs previous) (percentage) 413.64% 0.220 1.130 - 94.83% -
473.astar (avg) (percentage) 390.00% 0.100 0.490 - -46.74% -
483.xalancbmk (avg) (percentage) 350.00% 0.320 1.440 - -32.08% -
450.soplex (avg) (percentage) 318.18% 0.330 1.380 - 176.00% -
445.gobmk (max) (percentage) 266.67% 0.090 0.330 - -91.15% -
473.astar (unexplained diff vs previous) (percentage) 266.67% 0.060 0.220 - 10.00% -
473.astar (max) 266.67% 9.000 33.000 - -95.99% -
429.mcf (avg) (percentage) 211.90% 0.420 1.310 - -38.21% -
450.soplex (max) (percentage) 134.09% 0.440 1.030 - -37.58% -
401.bzip2 (avg) (percentage) 119.05% 0.210 0.460 - -29.23% -
482.sphinx3 (unexplained diff vs previous) (percentage) 110.53% 0.190 0.400 - -88.34% -
458.sjeng (max) (percentage) 100.00% 0.140 0.280 - -94.55% -
456.hmmer (avg) (percentage) 79.41% 0.340 0.610 - -56.12% -
473.astar 69.23% 13.000 22.000 - -80.36% -
473.astar (avg) 69.23% 13.000 22.000 - -80.36% -
444.namd (unexplained diff vs previous) 66.67% 3.000 5.000 - -37.50% -
445.gobmk (max) 60.00% 10.000 16.000 - -96.13% -
471.omnetpp (avg) (percentage) 55.42% 0.830 1.290 - -19.38% -
458.sjeng (avg) (percentage) 55.00% 0.200 0.310 - -61.25% -
471.omnetpp (unexplained diff vs previous) (percentage) 52.50% 0.800 1.220 - 24.49% -
453.povray (unexplained diff vs previous) (percentage) 51.85% 0.270 0.410 - 0.00% -
429.mcf (max) (percentage) 34.92% 0.630 0.850 - -94.79% -
447.dealII (max) (percentage) 30.68% 0.880 1.150 - -51.68% -
473.astar (unexplained diff vs previous) 25.00% 8.000 10.000 - -60.00% -
445.gobmk (avg) (percentage) 25.00% 0.200 0.250 - -62.69% -
453.povray (avg) (percentage) 11.11% 0.900 1.000 - 25.00% -
401.bzip2 (max) (percentage) 8.82% 0.340 0.370 - -90.21% -
447.dealII (unexplained diff vs previous) 4.76% 21.000 22.000 - -59.26% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
429.mcf -4.31% 7239.000 6927.000 - -4.31% -
462.libquantum -3.59% 12504.000 12055.000 - -3.59% -
453.povray -2.38% 524461.000 511996.000 - -2.38% -
433.milc -2.29% 47025.000 45949.000 - -2.24% -
464.h264ref -2.04% 303991.000 297797.000 - -1.97% -
444.namd -1.85% 124963.000 122654.000 - -1.85% -
458.sjeng -1.71% 80385.000 79010.000 - -1.60% -
401.bzip2 -1.68% 33698.000 33133.000 - -1.68% -
456.hmmer -1.23% 80861.000 79865.000 - -1.23% -
470.lbm -1.22% 9338.000 9224.000 - -1.22% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.37% 9758.000 61.000 - -99.36% -
462.libquantum -98.83% 15507.000 181.000 - -98.74% -
482.sphinx3 -96.54% 28622.000 991.000 - -96.29% -
450.soplex -96.44% 15381.000 548.000 - -96.37% -
444.namd -94.08% 13673.000 810.000 - -93.47% -
453.povray -93.12% 7146.000 492.000 - -92.96% -
433.milc -91.65% 13984.000 1168.000 - -91.41% -
429.mcf -90.21% 11680.000 1144.000 - -91.82% -
401.bzip2 -83.30% 16959.000 2832.000 - -82.45% -
447.dealII -79.83% 9631.000 1943.000 - -79.29% -
471.omnetpp -75.91% 11566.000 2786.000 - -73.41% -
470.lbm -72.70% 14614.000 3989.000 - -69.91% -
456.hmmer -71.83% 13520.000 3808.000 - -67.88% -
473.astar -65.84% 13450.000 4595.000 - -62.56% -
464.h264ref -62.57% 15889.000 5947.000 - -58.98% -
445.gobmk -57.92% 11530.000 4852.000 - -56.26% -
458.sjeng -49.42% 16165.000 8177.000 - -52.06% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (unexplained diff vs previous) -98.36% 61.000 1.000 - -92.31% -
444.namd -98.26% 287.000 5.000 - -98.43% -
444.namd (avg) -98.26% 287.000 5.000 - -98.43% -
482.sphinx3 -97.86% 374.000 8.000 - -98.28% -
482.sphinx3 (avg) -97.86% 374.000 8.000 - -98.28% -
429.mcf (unexplained diff vs previous) -97.83% 46.000 1.000 - -95.45% -
470.lbm (max) -96.06% 432.000 17.000 - -97.76% -
433.milc (max) -95.93% 123.000 5.000 - -95.37% -
456.hmmer (unexplained diff vs previous) -95.79% 95.000 4.000 - -91.30% -
453.povray (max) -95.12% 82.000 4.000 - -84.00% -
470.lbm -94.80% 442.000 23.000 - -94.78% -
470.lbm (avg) -94.80% 442.000 23.000 - -94.78% -
401.bzip2 (unexplained diff vs previous) -94.74% 57.000 3.000 - -90.91% -
470.lbm (unexplained diff vs previous) -94.26% 296.000 17.000 - -97.89% -
403.gcc -93.94% 33.000 2.000 - -96.15% -
403.gcc (avg) -93.94% 33.000 2.000 - -96.15% -
433.milc -93.94% 132.000 8.000 - -93.10% -
433.milc (avg) -93.94% 132.000 8.000 - -93.10% -
453.povray -93.75% 64.000 4.000 - -92.73% -
453.povray (avg) -93.75% 64.000 4.000 - -92.73% -
482.sphinx3 (unexplained diff vs previous) -92.73% 55.000 4.000 - -99.56% -
483.xalancbmk (max) (percentage) -92.65% 0.680 0.050 - -99.61% -
450.soplex (max) -92.54% 67.000 5.000 - -97.99% -
462.libquantum (unexplained diff vs previous) -91.67% 12.000 1.000 - 0.00% -
464.h264ref (max) -91.30% 69.000 6.000 - -90.00% -
403.gcc (max) -90.48% 42.000 4.000 - -98.19% -
462.libquantum -90.00% 20.000 2.000 - -98.18% -
462.libquantum (avg) -90.00% 20.000 2.000 - -98.18% -
453.povray (unexplained diff vs previous) -89.47% 19.000 2.000 - 0.00% -
429.mcf (max) -87.67% 73.000 9.000 - -99.61% -
450.soplex -86.00% 50.000 7.000 - -90.67% -
450.soplex (avg) -86.00% 50.000 7.000 - -90.67% -
456.hmmer (max) -85.26% 95.000 14.000 - -98.87% -
470.lbm (max) (percentage) -84.80% 2.960 0.450 - -92.13% -
456.hmmer (unexplained diff vs previous) (percentage) -84.29% 0.700 0.110 - -71.79% -
447.dealII -82.72% 81.000 14.000 - -85.71% -
447.dealII (avg) -82.72% 81.000 14.000 - -85.71% -
401.bzip2 (max) -82.46% 57.000 10.000 - -98.36% -
470.lbm (avg) (percentage) -80.53% 3.030 0.590 - -82.28% -
433.milc (unexplained diff vs previous) (percentage) -79.55% 0.440 0.090 - -10.00% -
470.lbm (unexplained diff vs previous) (percentage) -78.82% 2.030 0.430 - -92.90% -
471.omnetpp (max) -77.70% 148.000 33.000 - -96.60% -
429.mcf (unexplained diff vs previous) (percentage) -76.92% 0.390 0.090 - -43.75% -
464.h264ref (max) (percentage) -75.00% 0.440 0.110 - -73.81% -
464.h264ref -73.91% 46.000 12.000 - -71.43% -
464.h264ref (avg) -73.91% 46.000 12.000 - -71.43% -
447.dealII (max) -73.81% 84.000 22.000 - -90.13% -
429.mcf -71.43% 49.000 14.000 - -95.27% -
429.mcf (avg) -71.43% 49.000 14.000 - -95.27% -
401.bzip2 (unexplained diff vs previous) (percentage) -67.65% 0.340 0.110 - -45.00% -
444.namd (avg) (percentage) -67.14% 2.100 0.690 - -73.15% -
444.namd (max) -66.67% 15.000 5.000 - -99.50% -
471.omnetpp -63.16% 95.000 35.000 - -79.04% -
471.omnetpp (avg) -63.16% 95.000 35.000 - -79.04% -
471.omnetpp (unexplained diff vs previous) -63.04% 92.000 34.000 - -66.99% -
401.bzip2 -62.86% 35.000 13.000 - -87.50% -
401.bzip2 (avg) -62.86% 35.000 13.000 - -87.50% -
462.libquantum (max) -61.54% 13.000 5.000 - -99.46% -
456.hmmer -48.89% 45.000 23.000 - -85.98% -
456.hmmer (avg) -48.89% 45.000 23.000 - -85.98% -
482.sphinx3 (max) -48.15% 54.000 28.000 - -96.83% -
445.gobmk -47.83% 23.000 12.000 - -83.78% -
445.gobmk (avg) -47.83% 23.000 12.000 - -83.78% -
456.hmmer (max) (percentage) -46.48% 0.710 0.380 - -96.37% -
433.milc (max) (percentage) -42.05% 0.880 0.510 - -36.25% -
482.sphinx3 (avg) (percentage) -37.40% 1.310 0.820 - -52.87% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - -27.59% -
458.sjeng -21.88% 32.000 25.000 - -81.62% -
458.sjeng (avg) -21.88% 32.000 25.000 - -81.62% -
433.milc (avg) (percentage) -18.95% 0.950 0.770 - -10.47% -
453.povray (max) (percentage) -18.10% 1.160 0.950 - 163.89% -
447.dealII (avg) (percentage) -9.41% 0.850 0.770 - -26.67% -
471.omnetpp (max) (percentage) -5.47% 1.280 1.210 - -86.96% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
403.gcc (unexplained diff vs previous)
403.gcc (unexplained diff vs previous) (percentage)
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 2.22% 146765.000 150028.000 - 2.22% -
471.omnetpp 2.08% 316676.000 323250.000 - 2.08% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 5400.00% 0.010 0.550 - 587.50% -
403.gcc (unexplained diff vs previous) (percentage) 5366.67% 0.120 6.560 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) 933.33% 0.060 0.620 - 3000.00% -
403.gcc (avg) (percentage) 572.73% 0.550 3.700 - 988.24% -
403.gcc (max) (percentage) 201.29% 2.330 7.020 - 1495.45% -
450.soplex (avg) (percentage) 176.00% 0.500 1.380 - 318.18% -
453.povray (max) (percentage) 163.89% 0.360 0.950 - -18.10% -
462.libquantum (avg) (percentage) 105.19% 0.770 1.580 - 1115.38% -
447.dealII (unexplained diff vs previous) (percentage) 94.83% 0.580 1.130 - 413.64% -
453.povray (avg) (percentage) 25.00% 0.800 1.000 - 11.11% -
471.omnetpp (unexplained diff vs previous) (percentage) 24.49% 0.980 1.220 - 52.50% -
473.astar (unexplained diff vs previous) (percentage) 10.00% 0.200 0.220 - 266.67% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.31% 7239.000 6927.000 - -4.31% -
462.libquantum -3.59% 12504.000 12055.000 - -3.59% -
453.povray -2.38% 524493.000 511996.000 - -2.38% -
433.milc -2.24% 47001.000 45949.000 - -2.29% -
464.h264ref -1.97% 303767.000 297797.000 - -2.04% -
444.namd -1.85% 124963.000 122654.000 - -1.85% -
401.bzip2 -1.68% 33698.000 33133.000 - -1.68% -
458.sjeng -1.60% 80297.000 79010.000 - -1.71% -
456.hmmer -1.23% 80861.000 79865.000 - -1.23% -
470.lbm -1.22% 9338.000 9224.000 - -1.22% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.36% 9509.000 61.000 - -99.37% -
462.libquantum -98.74% 14409.000 181.000 - -98.83% -
450.soplex -96.37% 15112.000 548.000 - -96.44% -
482.sphinx3 -96.29% 26682.000 991.000 - -96.54% -
444.namd -93.47% 12400.000 810.000 - -94.08% -
453.povray -92.96% 6987.000 492.000 - -93.12% -
429.mcf -91.82% 13988.000 1144.000 - -90.21% -
433.milc -91.41% 13601.000 1168.000 - -91.65% -
401.bzip2 -82.45% 16137.000 2832.000 - -83.30% -
447.dealII -79.29% 9380.000 1943.000 - -79.83% -
471.omnetpp -73.41% 10477.000 2786.000 - -75.91% -
470.lbm -69.91% 13259.000 3989.000 - -72.70% -
456.hmmer -67.88% 11854.000 3808.000 - -71.83% -
473.astar -62.56% 12272.000 4595.000 - -65.84% -
464.h264ref -58.98% 14498.000 5947.000 - -62.57% -
445.gobmk -56.26% 11093.000 4852.000 - -57.92% -
458.sjeng -52.06% 17055.000 8177.000 - -49.42% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - -92.65% -
429.mcf (max) -99.61% 2281.000 9.000 - -87.67% -
482.sphinx3 (unexplained diff vs previous) -99.56% 914.000 4.000 - -92.73% -
444.namd (max) -99.50% 1003.000 5.000 - -66.67% -
462.libquantum (max) -99.46% 919.000 5.000 - -61.54% -
456.hmmer (max) -98.87% 1242.000 14.000 - -85.26% -
444.namd -98.43% 318.000 5.000 - -98.26% -
444.namd (avg) -98.43% 318.000 5.000 - -98.26% -
401.bzip2 (max) -98.36% 609.000 10.000 - -82.46% -
482.sphinx3 -98.28% 464.000 8.000 - -97.86% -
482.sphinx3 (avg) -98.28% 464.000 8.000 - -97.86% -
403.gcc (max) -98.19% 221.000 4.000 - -90.48% -
462.libquantum -98.18% 110.000 2.000 - -90.00% -
462.libquantum (avg) -98.18% 110.000 2.000 - -90.00% -
450.soplex (max) -97.99% 249.000 5.000 - -92.54% -
470.lbm (unexplained diff vs previous) -97.89% 804.000 17.000 - -94.26% -
470.lbm (max) -97.76% 758.000 17.000 - -96.06% -
445.gobmk (unexplained diff vs previous) -97.50% 40.000 1.000 - 0.00% -
458.sjeng (max) -97.49% 876.000 22.000 - 0.00% -
482.sphinx3 (max) -96.83% 883.000 28.000 - -48.15% -
471.omnetpp (max) -96.60% 972.000 33.000 - -77.70% -
456.hmmer (max) (percentage) -96.37% 10.480 0.380 - -46.48% -
403.gcc -96.15% 52.000 2.000 - -93.94% -
403.gcc (avg) -96.15% 52.000 2.000 - -93.94% -
445.gobmk (max) -96.13% 413.000 16.000 - 60.00% -
473.astar (max) -95.99% 822.000 33.000 - 266.67% -
429.mcf (unexplained diff vs previous) -95.45% 22.000 1.000 - -97.83% -
433.milc (max) -95.37% 108.000 5.000 - -95.93% -
429.mcf -95.27% 296.000 14.000 - -71.43% -
429.mcf (avg) -95.27% 296.000 14.000 - -71.43% -
429.mcf (max) (percentage) -94.79% 16.310 0.850 - 34.92% -
470.lbm -94.78% 441.000 23.000 - -94.80% -
470.lbm (avg) -94.78% 441.000 23.000 - -94.80% -
458.sjeng (max) (percentage) -94.55% 5.140 0.280 - 100.00% -
445.gobmk (unexplained diff vs previous) (percentage) -94.44% 0.360 0.020 - 0.00% -
433.milc -93.10% 116.000 8.000 - -93.94% -
433.milc (avg) -93.10% 116.000 8.000 - -93.94% -
470.lbm (unexplained diff vs previous) (percentage) -92.90% 6.060 0.430 - -78.82% -
453.povray -92.73% 55.000 4.000 - -93.75% -
453.povray (avg) -92.73% 55.000 4.000 - -93.75% -
444.namd (max) (percentage) -92.34% 8.090 0.620 - 463.64% -
433.milc (unexplained diff vs previous) -92.31% 13.000 1.000 - -98.36% -
470.lbm (max) (percentage) -92.13% 5.720 0.450 - -84.80% -
456.hmmer (unexplained diff vs previous) -91.30% 46.000 4.000 - -95.79% -
445.gobmk (max) (percentage) -91.15% 3.730 0.330 - 266.67% -
401.bzip2 (unexplained diff vs previous) -90.91% 33.000 3.000 - -94.74% -
450.soplex -90.67% 75.000 7.000 - -86.00% -
450.soplex (avg) -90.67% 75.000 7.000 - -86.00% -
401.bzip2 (max) (percentage) -90.21% 3.780 0.370 - 8.82% -
447.dealII (max) -90.13% 223.000 22.000 - -73.81% -
464.h264ref (max) -90.00% 60.000 6.000 - -91.30% -
473.astar (max) (percentage) -89.25% 6.700 0.720 - 928.57% -
482.sphinx3 (unexplained diff vs previous) (percentage) -88.34% 3.430 0.400 - 110.53% -
401.bzip2 -87.50% 104.000 13.000 - -62.86% -
401.bzip2 (avg) -87.50% 104.000 13.000 - -62.86% -
471.omnetpp (max) (percentage) -86.96% 9.280 1.210 - -5.47% -
456.hmmer -85.98% 164.000 23.000 - -48.89% -
456.hmmer (avg) -85.98% 164.000 23.000 - -48.89% -
447.dealII -85.71% 98.000 14.000 - -82.72% -
447.dealII (avg) -85.71% 98.000 14.000 - -82.72% -
453.povray (max) -84.00% 25.000 4.000 - -95.12% -
445.gobmk -83.78% 74.000 12.000 - -47.83% -
445.gobmk (avg) -83.78% 74.000 12.000 - -47.83% -
470.lbm (avg) (percentage) -82.28% 3.330 0.590 - -80.53% -
458.sjeng -81.62% 136.000 25.000 - -21.88% -
458.sjeng (avg) -81.62% 136.000 25.000 - -21.88% -
473.astar -80.36% 112.000 22.000 - 69.23% -
473.astar (avg) -80.36% 112.000 22.000 - 69.23% -
471.omnetpp -79.04% 167.000 35.000 - -63.16% -
471.omnetpp (avg) -79.04% 167.000 35.000 - -63.16% -
464.h264ref (max) (percentage) -73.81% 0.420 0.110 - -75.00% -
444.namd (avg) (percentage) -73.15% 2.570 0.690 - -67.14% -
456.hmmer (unexplained diff vs previous) (percentage) -71.79% 0.390 0.110 - -84.29% -
464.h264ref -71.43% 42.000 12.000 - -73.91% -
464.h264ref (avg) -71.43% 42.000 12.000 - -73.91% -
471.omnetpp (unexplained diff vs previous) -66.99% 103.000 34.000 - -63.04% -
403.gcc (unexplained diff vs previous) -63.64% 11.000 4.000 - 0.00% -
445.gobmk (avg) (percentage) -62.69% 0.670 0.250 - 25.00% -
458.sjeng (avg) (percentage) -61.25% 0.800 0.310 - 55.00% -
473.astar (unexplained diff vs previous) -60.00% 25.000 10.000 - 25.00% -
447.dealII (unexplained diff vs previous) -59.26% 54.000 22.000 - 4.76% -
456.hmmer (avg) (percentage) -56.12% 1.390 0.610 - 79.41% -
462.libquantum (max) (percentage) -55.49% 6.380 2.840 - 3055.56% -
482.sphinx3 (avg) (percentage) -52.87% 1.740 0.820 - -37.40% -
447.dealII (max) (percentage) -51.68% 2.380 1.150 - 30.68% -
473.astar (avg) (percentage) -46.74% 0.920 0.490 - 390.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -45.00% 0.200 0.110 - -67.65% -
429.mcf (unexplained diff vs previous) (percentage) -43.75% 0.160 0.090 - -76.92% -
429.mcf (avg) (percentage) -38.21% 2.120 1.310 - 211.90% -
450.soplex (max) (percentage) -37.58% 1.650 1.030 - 134.09% -
444.namd (unexplained diff vs previous) -37.50% 8.000 5.000 - 66.67% -
433.milc (max) (percentage) -36.25% 0.800 0.510 - -42.05% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 350.00% -
401.bzip2 (avg) (percentage) -29.23% 0.650 0.460 - 119.05% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - -27.59% -
447.dealII (avg) (percentage) -26.67% 1.050 0.770 - -9.41% -
471.omnetpp (avg) (percentage) -19.38% 1.600 1.290 - 55.42% -
482.sphinx3 (max) (percentage) -12.08% 3.310 2.910 - 1431.58% -
433.milc (avg) (percentage) -10.47% 0.860 0.770 - -18.95% -
433.milc (unexplained diff vs previous) (percentage) -10.00% 0.100 0.090 - -79.55% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Report Time: 0.35s
Execution (seconds)
Name Current %
401.bzip2 2832.000 -83.30%
403.gcc 61.000 -99.37%
429.mcf 1144.000 -90.21%
433.milc 1168.000 -91.65%
444.namd 810.000 -94.08%
445.gobmk 4852.000 -57.92%
447.dealII 1943.000 -79.83%
450.soplex 548.000 -96.44%
453.povray 492.000 -93.12%
456.hmmer 3808.000 -71.83%
458.sjeng 8177.000 -49.42%
462.libquantum 181.000 -98.83%
464.h264ref 5947.000 -62.57%
470.lbm 3989.000 -72.70%
471.omnetpp 2786.000 -75.91%
473.astar 4595.000 -65.84%
482.sphinx3 991.000 -96.54%
Geometric Mean 1482.395 -88.99%
Code Size (bytes)
Name Current %
401.bzip2 33133.000 -1.68%
403.gcc 1653179.000 -
429.mcf 6927.000 -4.31%
433.milc 45949.000 -2.29%
444.namd 122654.000 -1.85%
445.gobmk 1255453.000 -
447.dealII 193875.000 -
450.soplex 150028.000 2.22%
453.povray 511996.000 -2.38%
456.hmmer 79865.000 -1.23%
458.sjeng 79010.000 -1.71%
462.libquantum 12055.000 -3.59%
464.h264ref 297797.000 -2.04%
470.lbm 9224.000 -1.22%
471.omnetpp 323250.000 2.08%
473.astar 15583.000 -
482.sphinx3 87424.000 -
Geometric Mean 94287.377 -1.20%
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 13.000 -62.86%
401.bzip2 (avg) 13.000 -62.86%
401.bzip2 (avg) (percentage) 0.460 119.05%
401.bzip2 (max) 10.000 -82.46%
401.bzip2 (max) (percentage) 0.370 8.82%
401.bzip2 (unexplained diff vs previous) 3.000 -94.74%
401.bzip2 (unexplained diff vs previous) (percentage) 0.110 -67.65%
403.gcc 2.000 -93.94%
403.gcc (avg) 2.000 -93.94%
403.gcc (avg) (percentage) 3.700 988.24%
403.gcc (max) 4.000 -90.48%
403.gcc (max) (percentage) 7.020 1495.45%
403.gcc (unexplained diff vs previous) 4.000 -
403.gcc (unexplained diff vs previous) (percentage) 6.560 -
429.mcf 14.000 -71.43%
429.mcf (avg) 14.000 -71.43%
429.mcf (avg) (percentage) 1.310 211.90%
429.mcf (max) 9.000 -87.67%
429.mcf (max) (percentage) 0.850 34.92%
429.mcf (unexplained diff vs previous) 1.000 -97.83%
429.mcf (unexplained diff vs previous) (percentage) 0.090 -76.92%
433.milc 8.000 -93.94%
433.milc (avg) 8.000 -93.94%
433.milc (avg) (percentage) 0.770 -18.95%
433.milc (max) 5.000 -95.93%
433.milc (max) (percentage) 0.510 -42.05%
433.milc (unexplained diff vs previous) 1.000 -98.36%
433.milc (unexplained diff vs previous) (percentage) 0.090 -79.55%
444.namd 5.000 -98.26%
444.namd (avg) 5.000 -98.26%
444.namd (avg) (percentage) 0.690 -67.14%
444.namd (max) 5.000 -66.67%
444.namd (max) (percentage) 0.620 463.64%
444.namd (unexplained diff vs previous) 5.000 66.67%
444.namd (unexplained diff vs previous) (percentage) 0.620 3000.00%
445.gobmk 12.000 -47.83%
445.gobmk (avg) 12.000 -47.83%
445.gobmk (avg) (percentage) 0.250 25.00%
445.gobmk (max) 16.000 60.00%
445.gobmk (max) (percentage) 0.330 266.67%
445.gobmk (unexplained diff vs previous) 1.000 -
445.gobmk (unexplained diff vs previous) (percentage) 0.020 -
447.dealII 14.000 -82.72%
447.dealII (avg) 14.000 -82.72%
447.dealII (avg) (percentage) 0.770 -9.41%
447.dealII (max) 22.000 -73.81%
447.dealII (max) (percentage) 1.150 30.68%
447.dealII (unexplained diff vs previous) 22.000 4.76%
447.dealII (unexplained diff vs previous) (percentage) 1.130 413.64%
450.soplex 7.000 -86.00%
450.soplex (avg) 7.000 -86.00%
450.soplex (avg) (percentage) 1.380 318.18%
450.soplex (max) 5.000 -92.54%
450.soplex (max) (percentage) 1.030 134.09%
450.soplex (unexplained diff vs previous) 0.000 -
450.soplex (unexplained diff vs previous) (percentage) 0.000 -
453.povray 4.000 -93.75%
453.povray (avg) 4.000 -93.75%
453.povray (avg) (percentage) 1.000 11.11%
453.povray (max) 4.000 -95.12%
453.povray (max) (percentage) 0.950 -18.10%
453.povray (unexplained diff vs previous) 2.000 -89.47%
453.povray (unexplained diff vs previous) (percentage) 0.410 51.85%
456.hmmer 23.000 -48.89%
456.hmmer (avg) 23.000 -48.89%
456.hmmer (avg) (percentage) 0.610 79.41%
456.hmmer (max) 14.000 -85.26%
456.hmmer (max) (percentage) 0.380 -46.48%
456.hmmer (unexplained diff vs previous) 4.000 -95.79%
456.hmmer (unexplained diff vs previous) (percentage) 0.110 -84.29%
458.sjeng 25.000 -21.88%
458.sjeng (avg) 25.000 -21.88%
458.sjeng (avg) (percentage) 0.310 55.00%
458.sjeng (max) 22.000 -
458.sjeng (max) (percentage) 0.280 100.00%
458.sjeng (unexplained diff vs previous) 23.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.280 -
462.libquantum 2.000 -90.00%
462.libquantum (avg) 2.000 -90.00%
462.libquantum (avg) (percentage) 1.580 1115.38%
462.libquantum (max) 5.000 -61.54%
462.libquantum (max) (percentage) 2.840 3055.56%
462.libquantum (unexplained diff vs previous) 1.000 -91.67%
462.libquantum (unexplained diff vs previous) (percentage) 0.550 587.50%
464.h264ref 12.000 -73.91%
464.h264ref (avg) 12.000 -73.91%
464.h264ref (avg) (percentage) 0.210 -27.59%
464.h264ref (max) 6.000 -91.30%
464.h264ref (max) (percentage) 0.110 -75.00%
464.h264ref (unexplained diff vs previous) 5.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.080 -
470.lbm 23.000 -94.80%
470.lbm (avg) 23.000 -94.80%
470.lbm (avg) (percentage) 0.590 -80.53%
470.lbm (max) 17.000 -96.06%
470.lbm (max) (percentage) 0.450 -84.80%
470.lbm (unexplained diff vs previous) 17.000 -94.26%
470.lbm (unexplained diff vs previous) (percentage) 0.430 -78.82%
471.omnetpp 35.000 -63.16%
471.omnetpp (avg) 35.000 -63.16%
471.omnetpp (avg) (percentage) 1.290 55.42%
471.omnetpp (max) 33.000 -77.70%
471.omnetpp (max) (percentage) 1.210 -5.47%
471.omnetpp (unexplained diff vs previous) 34.000 -63.04%
471.omnetpp (unexplained diff vs previous) (percentage) 1.220 52.50%
473.astar 22.000 69.23%
473.astar (avg) 22.000 69.23%
473.astar (avg) (percentage) 0.490 390.00%
473.astar (max) 33.000 266.67%
473.astar (max) (percentage) 0.720 928.57%
473.astar (unexplained diff vs previous) 10.000 25.00%
473.astar (unexplained diff vs previous) (percentage) 0.220 266.67%
482.sphinx3 8.000 -97.86%
482.sphinx3 (avg) 8.000 -97.86%
482.sphinx3 (avg) (percentage) 0.820 -37.40%
482.sphinx3 (max) 28.000 -48.15%
482.sphinx3 (max) (percentage) 2.910 1431.58%
482.sphinx3 (unexplained diff vs previous) 4.000 -92.73%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.400 110.53%
483.xalancbmk (avg) (percentage) 1.440 350.00%
483.xalancbmk (max) (percentage) 0.050 -92.65%
Geometric Mean 2.147 -65.34%