Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/37ce3c2208167b01bad29761a4e0b0a368cb1f36 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/191/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0191 (no-change) 2024-10-02T19:58:46 0:00:00
Previous 0190 (llvmorg-16-init-07360-g7b45dfc6811a) 2022-10-07T14:48:28 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 36 23
Performance Improvements 102 121
Added Tests 10 6
Unchanged Tests 620 618
Total Tests 768

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) (percentage) 3414.29% 0.140 4.920 - -22.88% -
403.gcc (max) (percentage) 2438.46% 0.390 9.900 - 324.89% -
470.lbm (unexplained diff vs previous) (percentage) 2300.00% 0.020 0.480 - -92.08% -
450.soplex (unexplained diff vs previous) (percentage) 2140.00% 0.050 1.120 - 143.48% -
462.libquantum (avg) (percentage) 1146.15% 0.130 1.620 - 110.39% -
444.namd (unexplained diff vs previous) (percentage) 1133.33% 0.030 0.370 - 516.67% -
403.gcc (unexplained diff vs previous) (percentage) 1092.86% 0.140 1.670 - 1291.67% -
447.dealII (unexplained diff vs previous) (percentage) 1085.71% 0.070 0.830 - 43.10% -
403.gcc (avg) (percentage) 887.50% 0.320 3.160 - 474.55% -
429.mcf (unexplained diff vs previous) (percentage) 800.00% 0.030 0.270 - 68.75% -
450.soplex (avg) (percentage) 658.82% 0.170 1.290 - 158.00% -
470.lbm (unexplained diff vs previous) 533.33% 3.000 19.000 - -97.64% -
429.mcf (avg) (percentage) 443.48% 0.230 1.250 - -41.04% -
473.astar (avg) (percentage) 363.64% 0.110 0.510 - -44.57% -
483.xalancbmk (avg) (percentage) 336.36% 0.330 1.440 - -32.08% -
433.milc (unexplained diff vs previous) (percentage) 333.33% 0.300 1.300 - 1200.00% -
433.milc (max) (percentage) 326.67% 0.300 1.280 - 60.00% -
433.milc (avg) (percentage) 238.10% 0.210 0.710 - -17.44% -
473.astar (max) (percentage) 200.00% 0.100 0.300 - -95.52% -
456.hmmer (avg) (percentage) 153.33% 0.300 0.760 - -45.32% -
447.dealII (unexplained diff vs previous) 128.57% 7.000 16.000 - -70.37% -
401.bzip2 (avg) (percentage) 86.36% 0.220 0.410 - -36.92% -
458.sjeng (avg) (percentage) 77.78% 0.180 0.320 - -60.00% -
473.astar 64.29% 14.000 23.000 - -79.46% -
473.astar (avg) 64.29% 14.000 23.000 - -79.46% -
464.h264ref (avg) (percentage) 57.14% 0.140 0.220 - -24.14% -
447.dealII (avg) (percentage) 53.06% 0.490 0.750 - -28.57% -
471.omnetpp (unexplained diff vs previous) (percentage) 41.25% 0.800 1.130 - 15.31% -
471.omnetpp (avg) (percentage) 40.63% 0.960 1.350 - -15.62% -
456.hmmer (max) (percentage) 28.57% 0.490 0.630 - -93.99% -
458.sjeng (max) (percentage) 27.78% 0.180 0.230 - -95.53% -
482.sphinx3 (unexplained diff vs previous) (percentage) 20.00% 0.250 0.300 - -91.25% -
464.h264ref (max) (percentage) 13.33% 0.150 0.170 - -59.52% -
482.sphinx3 (avg) (percentage) 1.28% 0.780 0.790 - -54.60% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
429.mcf -4.20% 7239.000 6935.000 - -4.20% -
462.libquantum -3.28% 12464.000 12055.000 - -3.59% -
433.milc -2.26% 47153.000 46089.000 - -1.94% -
458.sjeng -2.25% 80805.000 78986.000 - -1.63% -
444.namd -2.15% 125319.000 122622.000 - -1.87% -
453.povray -2.07% 523137.000 512312.000 - -2.32% -
401.bzip2 -2.01% 33866.000 33185.000 - -1.52% -
464.h264ref -1.83% 304327.000 298765.000 - -1.65% -
447.dealII -1.56% 198002.000 194907.000 - -0.45% -
470.lbm -1.01% 9350.000 9256.000 - -0.88% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.39% 9801.000 60.000 - -99.37% -
462.libquantum -98.88% 15524.000 174.000 - -98.79% -
482.sphinx3 -96.64% 29446.000 988.000 - -96.30% -
450.soplex -96.50% 15313.000 536.000 - -96.45% -
444.namd -93.70% 12897.000 813.000 - -93.44% -
453.povray -92.95% 6996.000 493.000 - -92.94% -
433.milc -91.71% 13953.000 1157.000 - -91.49% -
429.mcf -90.21% 11565.000 1132.000 - -91.91% -
401.bzip2 -83.02% 16673.000 2831.000 - -82.46% -
447.dealII -81.59% 10466.000 1927.000 - -79.46% -
471.omnetpp -74.78% 11549.000 2913.000 - -72.20% -
470.lbm -74.03% 15318.000 3978.000 - -70.00% -
456.hmmer -71.85% 13549.000 3814.000 - -67.83% -
473.astar -65.80% 13455.000 4601.000 - -62.51% -
464.h264ref -62.68% 15908.000 5937.000 - -59.05% -
445.gobmk -57.43% 11457.000 4877.000 - -56.04% -
458.sjeng -49.08% 16079.000 8188.000 - -51.99% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
444.namd (max) -99.47% 566.000 3.000 - -99.70% -
429.mcf (max) -99.26% 813.000 6.000 - -99.74% -
482.sphinx3 (max) -98.87% 621.000 7.000 - -99.21% -
453.povray (max) -98.45% 129.000 2.000 - -92.00% -
450.soplex (max) -97.64% 254.000 6.000 - -97.59% -
470.lbm (max) -97.23% 830.000 23.000 - -96.97% -
482.sphinx3 -96.94% 229.000 7.000 - -98.49% -
482.sphinx3 (avg) -96.94% 229.000 7.000 - -98.49% -
403.gcc -96.77% 31.000 1.000 - -98.08% -
403.gcc (avg) -96.77% 31.000 1.000 - -98.08% -
444.namd -96.18% 157.000 6.000 - -98.11% -
444.namd (avg) -96.18% 157.000 6.000 - -98.11% -
482.sphinx3 (unexplained diff vs previous) -96.00% 75.000 3.000 - -99.67% -
453.povray -95.18% 83.000 4.000 - -92.73% -
453.povray (avg) -95.18% 83.000 4.000 - -92.73% -
403.gcc (unexplained diff vs previous) -92.86% 14.000 1.000 - -90.91% -
471.omnetpp (max) -92.50% 480.000 36.000 - -96.30% -
470.lbm -91.63% 251.000 21.000 - -95.24% -
470.lbm (avg) -91.63% 251.000 21.000 - -95.24% -
444.namd (max) (percentage) -91.57% 4.390 0.370 - -95.43% -
429.mcf (max) (percentage) -91.32% 7.030 0.610 - -96.26% -
462.libquantum -90.00% 20.000 2.000 - -98.18% -
462.libquantum (avg) -90.00% 20.000 2.000 - -98.18% -
470.lbm (max) (percentage) -88.93% 5.420 0.600 - -89.51% -
401.bzip2 (max) -88.37% 43.000 5.000 - -99.18% -
403.gcc (max) -86.84% 38.000 5.000 - -97.74% -
483.xalancbmk (max) (percentage) -84.85% 0.330 0.050 - -99.61% -
447.dealII (max) -84.11% 107.000 17.000 - -92.38% -
401.bzip2 (unexplained diff vs previous) -82.76% 29.000 5.000 - -84.85% -
445.gobmk (unexplained diff vs previous) -79.10% 67.000 14.000 - -65.00% -
445.gobmk (max) -78.79% 66.000 14.000 - -96.61% -
450.soplex -76.92% 26.000 6.000 - -92.00% -
450.soplex (avg) -76.92% 26.000 6.000 - -92.00% -
464.h264ref (unexplained diff vs previous) -75.00% 24.000 6.000 - 0.00% -
447.dealII -72.55% 51.000 14.000 - -85.71% -
447.dealII (avg) -72.55% 51.000 14.000 - -85.71% -
433.milc -72.41% 29.000 8.000 - -93.10% -
433.milc (avg) -72.41% 29.000 8.000 - -93.10% -
471.omnetpp (max) (percentage) -69.95% 4.160 1.250 - -86.53% -
445.gobmk -69.77% 43.000 13.000 - -82.43% -
445.gobmk (avg) -69.77% 43.000 13.000 - -82.43% -
401.bzip2 -69.44% 36.000 11.000 - -89.42% -
401.bzip2 (avg) -69.44% 36.000 11.000 - -89.42% -
453.povray (max) (percentage) -67.57% 1.850 0.600 - 66.67% -
470.lbm (avg) (percentage) -66.46% 1.640 0.550 - -83.48% -
433.milc (max) -65.85% 41.000 14.000 - -87.04% -
482.sphinx3 (max) (percentage) -64.93% 2.110 0.740 - -77.64% -
471.omnetpp -64.55% 110.000 39.000 - -76.65% -
471.omnetpp (avg) -64.55% 110.000 39.000 - -76.65% -
433.milc (unexplained diff vs previous) -64.29% 42.000 15.000 - 15.38% -
471.omnetpp (unexplained diff vs previous) -64.13% 92.000 33.000 - -67.96% -
456.hmmer (max) -63.64% 66.000 24.000 - -98.07% -
462.libquantum (max) -61.90% 21.000 8.000 - -99.13% -
464.h264ref (max) -56.52% 23.000 10.000 - -83.33% -
445.gobmk (max) (percentage) -50.00% 0.580 0.290 - -92.23% -
445.gobmk (unexplained diff vs previous) (percentage) -50.00% 0.580 0.290 - -19.44% -
429.mcf -46.15% 26.000 14.000 - -95.27% -
429.mcf (avg) -46.15% 26.000 14.000 - -95.27% -
464.h264ref -40.91% 22.000 13.000 - -69.05% -
464.h264ref (avg) -40.91% 22.000 13.000 - -69.05% -
444.namd (avg) (percentage) -39.34% 1.220 0.740 - -71.21% -
458.sjeng (max) -35.71% 28.000 18.000 - -97.95% -
464.h264ref (unexplained diff vs previous) (percentage) -33.33% 0.150 0.100 - 0.00% -
456.hmmer -30.00% 40.000 28.000 - -82.93% -
456.hmmer (avg) -30.00% 40.000 28.000 - -82.93% -
450.soplex (max) (percentage) -29.52% 1.660 1.170 - -29.09% -
401.bzip2 (max) (percentage) -26.92% 0.260 0.190 - -94.97% -
445.gobmk (avg) (percentage) -26.32% 0.380 0.280 - -58.21% -
444.namd (unexplained diff vs previous) -25.00% 4.000 3.000 - -62.50% -
453.povray (avg) (percentage) -24.37% 1.190 0.900 - 12.50% -
450.soplex (unexplained diff vs previous) -14.29% 7.000 6.000 - -91.30% -
447.dealII (max) (percentage) -13.59% 1.030 0.890 - -62.61% -
458.sjeng -7.14% 28.000 26.000 - -80.88% -
458.sjeng (avg) -7.14% 28.000 26.000 - -80.88% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
473.astar (unexplained diff vs previous)
473.astar (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.92% 146765.000 149580.000 - -0.79% -
471.omnetpp 1.67% 316676.000 321958.000 - 0.06% -
473.astar 1.04% 15470.000 15631.000 - 0.73% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 51600.00% 0.010 5.170 - 0.00% -
403.gcc (unexplained diff vs previous) (percentage) 1291.67% 0.120 1.670 - 1092.86% -
433.milc (unexplained diff vs previous) (percentage) 1200.00% 0.100 1.300 - 333.33% -
462.libquantum (unexplained diff vs previous) 800.00% 1.000 9.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) 516.67% 0.060 0.370 - 1133.33% -
403.gcc (avg) (percentage) 474.55% 0.550 3.160 - 887.50% -
403.gcc (max) (percentage) 324.89% 2.330 9.900 - 2438.46% -
450.soplex (avg) (percentage) 158.00% 0.500 1.290 - 658.82% -
450.soplex (unexplained diff vs previous) (percentage) 143.48% 0.460 1.120 - 2140.00% -
462.libquantum (avg) (percentage) 110.39% 0.770 1.620 - 1146.15% -
429.mcf (unexplained diff vs previous) (percentage) 68.75% 0.160 0.270 - 800.00% -
453.povray (max) (percentage) 66.67% 0.360 0.600 - -67.57% -
433.milc (max) (percentage) 60.00% 0.800 1.280 - 326.67% -
473.astar (unexplained diff vs previous) (percentage) 50.00% 0.200 0.300 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) 43.10% 0.580 0.830 - 1085.71% -
433.milc (unexplained diff vs previous) 15.38% 13.000 15.000 - -64.29% -
471.omnetpp (unexplained diff vs previous) (percentage) 15.31% 0.980 1.130 - 41.25% -
453.povray (avg) (percentage) 12.50% 0.800 0.900 - -24.37% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.20% 7239.000 6935.000 - -4.20% -
462.libquantum -3.59% 12504.000 12055.000 - -3.28% -
453.povray -2.32% 524493.000 512312.000 - -2.07% -
433.milc -1.94% 47001.000 46089.000 - -2.26% -
444.namd -1.87% 124963.000 122622.000 - -2.15% -
464.h264ref -1.65% 303767.000 298765.000 - -1.83% -
458.sjeng -1.63% 80297.000 78986.000 - -2.25% -
401.bzip2 -1.52% 33698.000 33185.000 - -2.01% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.37% 9509.000 60.000 - -99.39% -
462.libquantum -98.79% 14409.000 174.000 - -98.88% -
450.soplex -96.45% 15112.000 536.000 - -96.50% -
482.sphinx3 -96.30% 26682.000 988.000 - -96.64% -
444.namd -93.44% 12400.000 813.000 - -93.70% -
453.povray -92.94% 6987.000 493.000 - -92.95% -
429.mcf -91.91% 13988.000 1132.000 - -90.21% -
433.milc -91.49% 13601.000 1157.000 - -91.71% -
401.bzip2 -82.46% 16137.000 2831.000 - -83.02% -
447.dealII -79.46% 9380.000 1927.000 - -81.59% -
471.omnetpp -72.20% 10477.000 2913.000 - -74.78% -
470.lbm -70.00% 13259.000 3978.000 - -74.03% -
456.hmmer -67.83% 11854.000 3814.000 - -71.85% -
473.astar -62.51% 12272.000 4601.000 - -65.80% -
464.h264ref -59.05% 14498.000 5937.000 - -62.68% -
445.gobmk -56.04% 11093.000 4877.000 - -57.43% -
458.sjeng -51.99% 17055.000 8188.000 - -49.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (max) -99.74% 2281.000 6.000 - -99.26% -
444.namd (max) -99.70% 1003.000 3.000 - -99.47% -
482.sphinx3 (unexplained diff vs previous) -99.67% 914.000 3.000 - -96.00% -
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - -84.85% -
482.sphinx3 (max) -99.21% 883.000 7.000 - -98.87% -
401.bzip2 (max) -99.18% 609.000 5.000 - -88.37% -
462.libquantum (max) -99.13% 919.000 8.000 - -61.90% -
482.sphinx3 -98.49% 464.000 7.000 - -96.94% -
482.sphinx3 (avg) -98.49% 464.000 7.000 - -96.94% -
473.astar (max) -98.42% 822.000 13.000 - 0.00% -
462.libquantum -98.18% 110.000 2.000 - -90.00% -
462.libquantum (avg) -98.18% 110.000 2.000 - -90.00% -
444.namd -98.11% 318.000 6.000 - -96.18% -
444.namd (avg) -98.11% 318.000 6.000 - -96.18% -
403.gcc -98.08% 52.000 1.000 - -96.77% -
403.gcc (avg) -98.08% 52.000 1.000 - -96.77% -
456.hmmer (max) -98.07% 1242.000 24.000 - -63.64% -
458.sjeng (max) -97.95% 876.000 18.000 - -35.71% -
403.gcc (max) -97.74% 221.000 5.000 - -86.84% -
470.lbm (unexplained diff vs previous) -97.64% 804.000 19.000 - 533.33% -
450.soplex (max) -97.59% 249.000 6.000 - -97.64% -
470.lbm (max) -96.97% 758.000 23.000 - -97.23% -
445.gobmk (max) -96.61% 413.000 14.000 - -78.79% -
471.omnetpp (max) -96.30% 972.000 36.000 - -92.50% -
429.mcf (max) (percentage) -96.26% 16.310 0.610 - -91.32% -
458.sjeng (max) (percentage) -95.53% 5.140 0.230 - 27.78% -
473.astar (max) (percentage) -95.52% 6.700 0.300 - 200.00% -
444.namd (max) (percentage) -95.43% 8.090 0.370 - -91.57% -
429.mcf -95.27% 296.000 14.000 - -46.15% -
429.mcf (avg) -95.27% 296.000 14.000 - -46.15% -
470.lbm -95.24% 441.000 21.000 - -91.63% -
470.lbm (avg) -95.24% 441.000 21.000 - -91.63% -
401.bzip2 (max) (percentage) -94.97% 3.780 0.190 - -26.92% -
456.hmmer (max) (percentage) -93.99% 10.480 0.630 - 28.57% -
433.milc -93.10% 116.000 8.000 - -72.41% -
433.milc (avg) -93.10% 116.000 8.000 - -72.41% -
453.povray -92.73% 55.000 4.000 - -95.18% -
453.povray (avg) -92.73% 55.000 4.000 - -95.18% -
447.dealII (max) -92.38% 223.000 17.000 - -84.11% -
445.gobmk (max) (percentage) -92.23% 3.730 0.290 - -50.00% -
470.lbm (unexplained diff vs previous) (percentage) -92.08% 6.060 0.480 - 2300.00% -
450.soplex -92.00% 75.000 6.000 - -76.92% -
450.soplex (avg) -92.00% 75.000 6.000 - -76.92% -
453.povray (max) -92.00% 25.000 2.000 - -98.45% -
450.soplex (unexplained diff vs previous) -91.30% 69.000 6.000 - -14.29% -
482.sphinx3 (unexplained diff vs previous) (percentage) -91.25% 3.430 0.300 - 20.00% -
403.gcc (unexplained diff vs previous) -90.91% 11.000 1.000 - -92.86% -
470.lbm (max) (percentage) -89.51% 5.720 0.600 - -88.93% -
401.bzip2 -89.42% 104.000 11.000 - -69.44% -
401.bzip2 (avg) -89.42% 104.000 11.000 - -69.44% -
433.milc (max) -87.04% 108.000 14.000 - -65.85% -
471.omnetpp (max) (percentage) -86.53% 9.280 1.250 - -69.95% -
429.mcf (unexplained diff vs previous) -86.36% 22.000 3.000 - 0.00% -
447.dealII -85.71% 98.000 14.000 - -72.55% -
447.dealII (avg) -85.71% 98.000 14.000 - -72.55% -
401.bzip2 (unexplained diff vs previous) -84.85% 33.000 5.000 - -82.76% -
470.lbm (avg) (percentage) -83.48% 3.330 0.550 - -66.46% -
464.h264ref (max) -83.33% 60.000 10.000 - -56.52% -
456.hmmer -82.93% 164.000 28.000 - -30.00% -
456.hmmer (avg) -82.93% 164.000 28.000 - -30.00% -
445.gobmk -82.43% 74.000 13.000 - -69.77% -
445.gobmk (avg) -82.43% 74.000 13.000 - -69.77% -
458.sjeng -80.88% 136.000 26.000 - -7.14% -
458.sjeng (avg) -80.88% 136.000 26.000 - -7.14% -
473.astar -79.46% 112.000 23.000 - 64.29% -
473.astar (avg) -79.46% 112.000 23.000 - 64.29% -
482.sphinx3 (max) (percentage) -77.64% 3.310 0.740 - -64.93% -
471.omnetpp -76.65% 167.000 39.000 - -64.55% -
471.omnetpp (avg) -76.65% 167.000 39.000 - -64.55% -
444.namd (avg) (percentage) -71.21% 2.570 0.740 - -39.34% -
447.dealII (unexplained diff vs previous) -70.37% 54.000 16.000 - 128.57% -
464.h264ref -69.05% 42.000 13.000 - -40.91% -
464.h264ref (avg) -69.05% 42.000 13.000 - -40.91% -
471.omnetpp (unexplained diff vs previous) -67.96% 103.000 33.000 - -64.13% -
445.gobmk (unexplained diff vs previous) -65.00% 40.000 14.000 - -79.10% -
447.dealII (max) (percentage) -62.61% 2.380 0.890 - -13.59% -
444.namd (unexplained diff vs previous) -62.50% 8.000 3.000 - -25.00% -
458.sjeng (avg) (percentage) -60.00% 0.800 0.320 - 77.78% -
464.h264ref (max) (percentage) -59.52% 0.420 0.170 - 13.33% -
445.gobmk (avg) (percentage) -58.21% 0.670 0.280 - -26.32% -
482.sphinx3 (avg) (percentage) -54.60% 1.740 0.790 - 1.28% -
456.hmmer (avg) (percentage) -45.32% 1.390 0.760 - 153.33% -
473.astar (avg) (percentage) -44.57% 0.920 0.510 - 363.64% -
473.astar (unexplained diff vs previous) -44.00% 25.000 14.000 - 0.00% -
429.mcf (avg) (percentage) -41.04% 2.120 1.250 - 443.48% -
401.bzip2 (avg) (percentage) -36.92% 0.650 0.410 - 86.36% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 336.36% -
450.soplex (max) (percentage) -29.09% 1.650 1.170 - -29.52% -
447.dealII (avg) (percentage) -28.57% 1.050 0.750 - 53.06% -
464.h264ref (avg) (percentage) -24.14% 0.290 0.220 - 57.14% -
462.libquantum (max) (percentage) -22.88% 6.380 4.920 - 3414.29% -
445.gobmk (unexplained diff vs previous) (percentage) -19.44% 0.360 0.290 - -50.00% -
433.milc (avg) (percentage) -17.44% 0.860 0.710 - 238.10% -
471.omnetpp (avg) (percentage) -15.62% 1.600 1.350 - 40.63% -
401.bzip2 (unexplained diff vs previous) (percentage) -10.00% 0.200 0.180 - 5.88% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
401.bzip2 2831.000 -83.02%
403.gcc 60.000 -99.39%
429.mcf 1132.000 -90.21%
433.milc 1157.000 -91.71%
444.namd 813.000 -93.70%
445.gobmk 4877.000 -57.43%
447.dealII 1927.000 -81.59%
450.soplex 536.000 -96.50%
453.povray 493.000 -92.95%
456.hmmer 3814.000 -71.85%
458.sjeng 8188.000 -49.08%
462.libquantum 174.000 -98.88%
464.h264ref 5937.000 -62.68%
470.lbm 3978.000 -74.03%
471.omnetpp 2913.000 -74.78%
473.astar 4601.000 -65.80%
482.sphinx3 988.000 -96.64%
Geometric Mean 1477.646 -89.06%
Code Size (bytes)
Name Current %
401.bzip2 33185.000 -2.01%
403.gcc 1655187.000 -
429.mcf 6935.000 -4.20%
433.milc 46089.000 -2.26%
444.namd 122622.000 -2.15%
445.gobmk 1255425.000 -
447.dealII 194907.000 -1.56%
450.soplex 149580.000 -
453.povray 512312.000 -2.07%
456.hmmer 80241.000 -
458.sjeng 78986.000 -2.25%
462.libquantum 12055.000 -3.28%
464.h264ref 298765.000 -1.83%
470.lbm 9256.000 -1.01%
471.omnetpp 321958.000 -
473.astar 15631.000 -
482.sphinx3 87628.000 -
Geometric Mean 94410.217 -1.50%
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 11.000 -69.44%
401.bzip2 (avg) 11.000 -69.44%
401.bzip2 (avg) (percentage) 0.410 86.36%
401.bzip2 (max) 5.000 -88.37%
401.bzip2 (max) (percentage) 0.190 -26.92%
401.bzip2 (unexplained diff vs previous) 5.000 -82.76%
401.bzip2 (unexplained diff vs previous) (percentage) 0.180 -
403.gcc 1.000 -96.77%
403.gcc (avg) 1.000 -96.77%
403.gcc (avg) (percentage) 3.160 887.50%
403.gcc (max) 5.000 -86.84%
403.gcc (max) (percentage) 9.900 2438.46%
403.gcc (unexplained diff vs previous) 1.000 -92.86%
403.gcc (unexplained diff vs previous) (percentage) 1.670 1092.86%
429.mcf 14.000 -46.15%
429.mcf (avg) 14.000 -46.15%
429.mcf (avg) (percentage) 1.250 443.48%
429.mcf (max) 6.000 -99.26%
429.mcf (max) (percentage) 0.610 -91.32%
429.mcf (unexplained diff vs previous) 3.000 -
429.mcf (unexplained diff vs previous) (percentage) 0.270 800.00%
433.milc 8.000 -72.41%
433.milc (avg) 8.000 -72.41%
433.milc (avg) (percentage) 0.710 238.10%
433.milc (max) 14.000 -65.85%
433.milc (max) (percentage) 1.280 326.67%
433.milc (unexplained diff vs previous) 15.000 -64.29%
433.milc (unexplained diff vs previous) (percentage) 1.300 333.33%
444.namd 6.000 -96.18%
444.namd (avg) 6.000 -96.18%
444.namd (avg) (percentage) 0.740 -39.34%
444.namd (max) 3.000 -99.47%
444.namd (max) (percentage) 0.370 -91.57%
444.namd (unexplained diff vs previous) 3.000 -25.00%
444.namd (unexplained diff vs previous) (percentage) 0.370 1133.33%
445.gobmk 13.000 -69.77%
445.gobmk (avg) 13.000 -69.77%
445.gobmk (avg) (percentage) 0.280 -26.32%
445.gobmk (max) 14.000 -78.79%
445.gobmk (max) (percentage) 0.290 -50.00%
445.gobmk (unexplained diff vs previous) 14.000 -79.10%
445.gobmk (unexplained diff vs previous) (percentage) 0.290 -50.00%
447.dealII 14.000 -72.55%
447.dealII (avg) 14.000 -72.55%
447.dealII (avg) (percentage) 0.750 53.06%
447.dealII (max) 17.000 -84.11%
447.dealII (max) (percentage) 0.890 -13.59%
447.dealII (unexplained diff vs previous) 16.000 128.57%
447.dealII (unexplained diff vs previous) (percentage) 0.830 1085.71%
450.soplex 6.000 -76.92%
450.soplex (avg) 6.000 -76.92%
450.soplex (avg) (percentage) 1.290 658.82%
450.soplex (max) 6.000 -97.64%
450.soplex (max) (percentage) 1.170 -29.52%
450.soplex (unexplained diff vs previous) 6.000 -14.29%
450.soplex (unexplained diff vs previous) (percentage) 1.120 2140.00%
453.povray 4.000 -95.18%
453.povray (avg) 4.000 -95.18%
453.povray (avg) (percentage) 0.900 -24.37%
453.povray (max) 2.000 -98.45%
453.povray (max) (percentage) 0.600 -67.57%
453.povray (unexplained diff vs previous) 3.000 -
453.povray (unexplained diff vs previous) (percentage) 0.610 -
456.hmmer 28.000 -30.00%
456.hmmer (avg) 28.000 -30.00%
456.hmmer (avg) (percentage) 0.760 153.33%
456.hmmer (max) 24.000 -63.64%
456.hmmer (max) (percentage) 0.630 28.57%
456.hmmer (unexplained diff vs previous) 0.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.000 -
458.sjeng 26.000 -7.14%
458.sjeng (avg) 26.000 -7.14%
458.sjeng (avg) (percentage) 0.320 77.78%
458.sjeng (max) 18.000 -35.71%
458.sjeng (max) (percentage) 0.230 27.78%
458.sjeng (unexplained diff vs previous) 1.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.010 -
462.libquantum 2.000 -90.00%
462.libquantum (avg) 2.000 -90.00%
462.libquantum (avg) (percentage) 1.620 1146.15%
462.libquantum (max) 8.000 -61.90%
462.libquantum (max) (percentage) 4.920 3414.29%
462.libquantum (unexplained diff vs previous) 9.000 -
462.libquantum (unexplained diff vs previous) (percentage) 5.170 -
464.h264ref 13.000 -40.91%
464.h264ref (avg) 13.000 -40.91%
464.h264ref (avg) (percentage) 0.220 57.14%
464.h264ref (max) 10.000 -56.52%
464.h264ref (max) (percentage) 0.170 13.33%
464.h264ref (unexplained diff vs previous) 6.000 -75.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.100 -33.33%
470.lbm 21.000 -91.63%
470.lbm (avg) 21.000 -91.63%
470.lbm (avg) (percentage) 0.550 -66.46%
470.lbm (max) 23.000 -97.23%
470.lbm (max) (percentage) 0.600 -88.93%
470.lbm (unexplained diff vs previous) 19.000 533.33%
470.lbm (unexplained diff vs previous) (percentage) 0.480 2300.00%
471.omnetpp 39.000 -64.55%
471.omnetpp (avg) 39.000 -64.55%
471.omnetpp (avg) (percentage) 1.350 40.63%
471.omnetpp (max) 36.000 -92.50%
471.omnetpp (max) (percentage) 1.250 -69.95%
471.omnetpp (unexplained diff vs previous) 33.000 -64.13%
471.omnetpp (unexplained diff vs previous) (percentage) 1.130 41.25%
473.astar 23.000 64.29%
473.astar (avg) 23.000 64.29%
473.astar (avg) (percentage) 0.510 363.64%
473.astar (max) 13.000 -
473.astar (max) (percentage) 0.300 200.00%
473.astar (unexplained diff vs previous) 14.000 -
473.astar (unexplained diff vs previous) (percentage) 0.300 -
482.sphinx3 7.000 -96.94%
482.sphinx3 (avg) 7.000 -96.94%
482.sphinx3 (avg) (percentage) 0.790 1.28%
482.sphinx3 (max) 7.000 -98.87%
482.sphinx3 (max) (percentage) 0.740 -64.93%
482.sphinx3 (unexplained diff vs previous) 3.000 -96.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.300 20.00%
483.xalancbmk (avg) (percentage) 1.440 336.36%
483.xalancbmk (max) (percentage) 0.050 -84.85%
Geometric Mean 2.274 -60.31%