Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/3792b36234b6c87d728f0a905543e284bf961460 (+ 222 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/218/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0218 (llvmorg-20-init-16615-g3792b36234b6) 2024-12-28T22:20:49 0:00:00
Previous 0218 (no-change) 2022-11-08T19:41:27 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 54 30
Performance Improvements 95 118
Added Tests 0 4
Unchanged Tests 631 628
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
450.soplex 2.15% 146877.000 150036.000 - 2.23% -
471.omnetpp 2.07% 316696.000 323250.000 - 2.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (unexplained diff vs previous) (percentage) 45300.00% 0.010 4.540 - 886.96% -
429.mcf (unexplained diff vs previous) (percentage) 20300.00% 0.040 8.160 - 5000.00% -
470.lbm (unexplained diff vs previous) (percentage) 9250.00% 0.040 3.740 - -38.28% -
462.libquantum (unexplained diff vs previous) (percentage) 7066.67% 0.060 4.300 - 42900.00% -
429.mcf (max) (percentage) 6242.86% 0.140 8.880 - -45.55% -
450.soplex (max) (percentage) 4218.18% 0.110 4.750 - 187.88% -
462.libquantum (max) (percentage) 3981.82% 0.110 4.490 - -29.62% -
401.bzip2 (unexplained diff vs previous) (percentage) 3622.22% 0.090 3.350 - 1575.00% -
470.lbm (unexplained diff vs previous) 2483.33% 6.000 155.000 - -80.72% -
473.astar (max) (percentage) 2114.29% 0.070 1.550 - -76.87% -
429.mcf (unexplained diff vs previous) 1900.00% 5.000 100.000 - 354.55% -
482.sphinx3 (unexplained diff vs previous) (percentage) 1805.00% 0.200 3.810 - 11.08% -
401.bzip2 (max) (percentage) 1635.00% 0.200 3.470 - -8.20% -
403.gcc (max) (percentage) 1349.15% 0.590 8.550 - 266.95% -
450.soplex (unexplained diff vs previous) 1200.00% 2.000 26.000 - -62.32% -
462.libquantum (avg) (percentage) 946.67% 0.150 1.570 - 103.90% -
403.gcc (avg) (percentage) 900.00% 0.330 3.300 - 500.00% -
445.gobmk (unexplained diff vs previous) (percentage) 825.00% 0.040 0.370 - 2.78% -
403.gcc (unexplained diff vs previous) (percentage) 706.78% 0.590 4.760 - 3866.67% -
473.astar (max) 688.89% 9.000 71.000 - -91.36% -
429.mcf (max) 575.00% 16.000 108.000 - -95.27% -
401.bzip2 (unexplained diff vs previous) 553.33% 15.000 98.000 - 196.97% -
458.sjeng (max) (percentage) 550.00% 0.040 0.260 - -94.94% -
433.milc (unexplained diff vs previous) (percentage) 468.55% 1.240 7.050 - 6950.00% -
444.namd (unexplained diff vs previous) (percentage) 437.50% 0.160 0.860 - 1333.33% -
433.milc (max) (percentage) 409.40% 1.490 7.590 - 848.75% -
445.gobmk (unexplained diff vs previous) 350.00% 4.000 18.000 - -55.00% -
483.xalancbmk (avg) (percentage) 350.00% 0.320 1.440 - -32.08% -
473.astar (avg) (percentage) 345.45% 0.110 0.490 - -46.74% -
450.soplex (avg) (percentage) 337.50% 0.320 1.400 - 180.00% -
429.mcf (avg) (percentage) 275.00% 0.360 1.350 - -36.32% -
458.sjeng (max) 250.00% 6.000 21.000 - -97.60% -
445.gobmk (max) (percentage) 236.36% 0.110 0.370 - -90.08% -
401.bzip2 (max) 206.06% 33.000 101.000 - -83.42% -
401.bzip2 (avg) (percentage) 150.00% 0.180 0.450 - -30.77% -
464.h264ref (unexplained diff vs previous) (percentage) 150.00% 0.300 0.750 - 0.00% -
456.hmmer (avg) (percentage) 94.12% 0.340 0.660 - -52.52% -
471.omnetpp (max) (percentage) 80.33% 1.830 3.300 - -64.44% -
482.sphinx3 (max) (percentage) 73.68% 2.280 3.960 - 19.64% -
450.soplex (max) 68.75% 16.000 27.000 - -89.16% -
458.sjeng (avg) (percentage) 60.00% 0.200 0.320 - -60.00% -
473.astar 57.14% 14.000 22.000 - -80.36% -
473.astar (avg) 57.14% 14.000 22.000 - -80.36% -
471.omnetpp (avg) (percentage) 53.33% 0.900 1.380 - -13.75% -
445.gobmk (max) 41.67% 12.000 17.000 - -95.88% -
464.h264ref (max) (percentage) 35.71% 0.560 0.760 - 80.95% -
458.sjeng (unexplained diff vs previous) (percentage) 33.33% 0.030 0.040 - 0.00% -
445.gobmk (avg) (percentage) 30.00% 0.200 0.260 - -61.19% -
453.povray (max) (percentage) 28.10% 1.210 1.550 - 330.56% -
447.dealII (unexplained diff vs previous) (percentage) 14.63% 0.410 0.470 - -18.97% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
429.mcf -4.36% 7243.000 6927.000 - -4.31% -
462.libquantum -3.53% 12496.000 12055.000 - -3.59% -
401.bzip2 -2.52% 33998.000 33141.000 - -1.65% -
453.povray -2.47% 525021.000 512076.000 - -2.37% -
433.milc -2.45% 47105.000 45949.000 - -2.24% -
464.h264ref -2.16% 304295.000 297717.000 - -1.99% -
444.namd -1.93% 125071.000 122654.000 - -1.85% -
458.sjeng -1.93% 80589.000 79034.000 - -1.57% -
470.lbm -1.35% 9350.000 9224.000 - -1.22% -
456.hmmer -1.29% 80893.000 79849.000 - -1.25% -
447.dealII -1.08% 195856.000 193739.000 - -1.05% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.36% 9844.000 63.000 - -99.34% -
462.libquantum -98.80% 15511.000 186.000 - -98.71% -
482.sphinx3 -96.52% 29405.000 1024.000 - -96.16% -
450.soplex -96.28% 15410.000 573.000 - -96.21% -
444.namd -94.05% 13692.000 814.000 - -93.44% -
453.povray -93.02% 7164.000 500.000 - -92.84% -
433.milc -91.07% 14128.000 1262.000 - -90.72% -
429.mcf -89.52% 11695.000 1226.000 - -91.24% -
401.bzip2 -82.71% 16900.000 2922.000 - -81.89% -
447.dealII -81.82% 10621.000 1931.000 - -79.41% -
471.omnetpp -75.34% 11531.000 2843.000 - -72.86% -
470.lbm -72.99% 15325.000 4140.000 - -68.78% -
456.hmmer -71.80% 13470.000 3799.000 - -67.95% -
473.astar -65.51% 13454.000 4640.000 - -62.19% -
464.h264ref -62.24% 15814.000 5972.000 - -58.81% -
445.gobmk -57.43% 11420.000 4862.000 - -56.17% -
458.sjeng -49.17% 16103.000 8185.000 - -52.01% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
444.namd (max) -98.78% 573.000 7.000 - -99.30% -
444.namd -98.45% 323.000 5.000 - -98.43% -
444.namd (avg) -98.45% 323.000 5.000 - -98.43% -
482.sphinx3 -98.14% 429.000 8.000 - -98.28% -
482.sphinx3 (avg) -98.14% 429.000 8.000 - -98.28% -
456.hmmer (unexplained diff vs previous) -95.92% 49.000 2.000 - -95.65% -
470.lbm -95.19% 478.000 23.000 - -94.78% -
470.lbm (avg) -95.19% 478.000 23.000 - -94.78% -
403.gcc (unexplained diff vs previous) -94.83% 58.000 3.000 - -72.73% -
453.povray -94.29% 70.000 4.000 - -92.73% -
453.povray (avg) -94.29% 70.000 4.000 - -92.73% -
447.dealII (max) -94.27% 192.000 11.000 - -95.07% -
482.sphinx3 (max) -94.03% 670.000 40.000 - -95.47% -
433.milc -93.79% 145.000 9.000 - -92.24% -
433.milc (avg) -93.79% 145.000 9.000 - -92.24% -
403.gcc -93.75% 32.000 2.000 - -96.15% -
403.gcc (avg) -93.75% 32.000 2.000 - -96.15% -
483.xalancbmk (max) (percentage) -93.51% 0.770 0.050 - -99.61% -
453.povray (max) -91.86% 86.000 7.000 - -72.00% -
403.gcc (max) -91.38% 58.000 5.000 - -97.74% -
462.libquantum -91.30% 23.000 2.000 - -98.18% -
462.libquantum (avg) -91.30% 23.000 2.000 - -98.18% -
456.hmmer (max) -89.80% 49.000 5.000 - -99.60% -
456.hmmer (unexplained diff vs previous) (percentage) -86.11% 0.360 0.050 - -87.18% -
447.dealII -85.71% 98.000 14.000 - -85.71% -
447.dealII (avg) -85.71% 98.000 14.000 - -85.71% -
450.soplex -83.67% 49.000 8.000 - -89.33% -
450.soplex (avg) -83.67% 49.000 8.000 - -89.33% -
470.lbm (avg) (percentage) -81.73% 3.120 0.570 - -82.88% -
471.omnetpp (unexplained diff vs previous) -81.40% 215.000 40.000 - -61.17% -
470.lbm (max) -80.25% 815.000 161.000 - -78.76% -
447.dealII (unexplained diff vs previous) -79.55% 44.000 9.000 - -83.33% -
444.namd (max) (percentage) -79.24% 4.190 0.870 - -89.25% -
464.h264ref -71.11% 45.000 13.000 - -69.05% -
464.h264ref (avg) -71.11% 45.000 13.000 - -69.05% -
444.namd (avg) (percentage) -69.07% 2.360 0.730 - -71.60% -
444.namd (unexplained diff vs previous) -68.18% 22.000 7.000 - -12.50% -
447.dealII (max) (percentage) -66.85% 1.810 0.600 - -74.79% -
471.omnetpp -62.14% 103.000 39.000 - -76.65% -
471.omnetpp (avg) -62.14% 103.000 39.000 - -76.65% -
429.mcf -61.90% 42.000 16.000 - -94.59% -
429.mcf (avg) -61.90% 42.000 16.000 - -94.59% -
456.hmmer (max) (percentage) -59.46% 0.370 0.150 - -98.57% -
401.bzip2 -56.67% 30.000 13.000 - -87.50% -
401.bzip2 (avg) -56.67% 30.000 13.000 - -87.50% -
471.omnetpp (max) -55.92% 211.000 93.000 - -90.43% -
433.milc (max) -54.76% 210.000 95.000 - -12.04% -
462.libquantum (max) -52.94% 17.000 8.000 - -99.13% -
433.milc (unexplained diff vs previous) -49.14% 175.000 89.000 - 584.62% -
464.h264ref (max) -48.86% 88.000 45.000 - -25.00% -
482.sphinx3 (avg) (percentage) -45.89% 1.460 0.790 - -54.60% -
445.gobmk -45.45% 22.000 12.000 - -83.78% -
445.gobmk (avg) -45.45% 22.000 12.000 - -83.78% -
456.hmmer -44.44% 45.000 25.000 - -84.76% -
456.hmmer (avg) -44.44% 45.000 25.000 - -84.76% -
458.sjeng (unexplained diff vs previous) -40.00% 5.000 3.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -33.90% 59.000 39.000 - -95.73% -
433.milc (avg) (percentage) -28.16% 1.030 0.740 - -13.95% -
470.lbm (max) (percentage) -26.88% 5.320 3.890 - -31.99% -
471.omnetpp (unexplained diff vs previous) (percentage) -24.19% 1.860 1.410 - 43.88% -
464.h264ref (avg) (percentage) -24.14% 0.290 0.220 - -24.14% -
447.dealII (avg) (percentage) -21.51% 0.930 0.730 - -30.48% -
458.sjeng -18.75% 32.000 26.000 - -80.88% -
458.sjeng (avg) -18.75% 32.000 26.000 - -80.88% -
462.libquantum (unexplained diff vs previous) -11.11% 9.000 8.000 - 700.00% -
464.h264ref (unexplained diff vs previous) -4.26% 47.000 45.000 - 0.00% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 2.23% 146765.000 150036.000 - 2.15% -
471.omnetpp 2.08% 316676.000 323250.000 - 2.07% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 42900.00% 0.010 4.300 - 7066.67% -
433.milc (unexplained diff vs previous) (percentage) 6950.00% 0.100 7.050 - 468.55% -
429.mcf (unexplained diff vs previous) (percentage) 5000.00% 0.160 8.160 - 20300.00% -
403.gcc (unexplained diff vs previous) (percentage) 3866.67% 0.120 4.760 - 706.78% -
401.bzip2 (unexplained diff vs previous) (percentage) 1575.00% 0.200 3.350 - 3622.22% -
444.namd (unexplained diff vs previous) (percentage) 1333.33% 0.060 0.860 - 437.50% -
450.soplex (unexplained diff vs previous) (percentage) 886.96% 0.460 4.540 - 45300.00% -
433.milc (max) (percentage) 848.75% 0.800 7.590 - 409.40% -
462.libquantum (unexplained diff vs previous) 700.00% 1.000 8.000 - -11.11% -
473.astar (unexplained diff vs previous) (percentage) 665.00% 0.200 1.530 - 0.00% -
433.milc (unexplained diff vs previous) 584.62% 13.000 89.000 - -49.14% -
403.gcc (avg) (percentage) 500.00% 0.550 3.300 - 900.00% -
429.mcf (unexplained diff vs previous) 354.55% 22.000 100.000 - 1900.00% -
453.povray (max) (percentage) 330.56% 0.360 1.550 - 28.10% -
403.gcc (max) (percentage) 266.95% 2.330 8.550 - 1349.15% -
401.bzip2 (unexplained diff vs previous) 196.97% 33.000 98.000 - 553.33% -
450.soplex (max) (percentage) 187.88% 1.650 4.750 - 4218.18% -
473.astar (unexplained diff vs previous) 184.00% 25.000 71.000 - 0.00% -
450.soplex (avg) (percentage) 180.00% 0.500 1.400 - 337.50% -
462.libquantum (avg) (percentage) 103.90% 0.770 1.570 - 946.67% -
464.h264ref (max) (percentage) 80.95% 0.420 0.760 - 35.71% -
471.omnetpp (unexplained diff vs previous) (percentage) 43.88% 0.980 1.410 - -24.19% -
453.povray (avg) (percentage) 23.75% 0.800 0.990 - 0.00% -
482.sphinx3 (max) (percentage) 19.64% 3.310 3.960 - 73.68% -
482.sphinx3 (unexplained diff vs previous) (percentage) 11.08% 3.430 3.810 - 1805.00% -
445.gobmk (unexplained diff vs previous) (percentage) 2.78% 0.360 0.370 - 825.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.31% 7239.000 6927.000 - -4.36% -
462.libquantum -3.59% 12504.000 12055.000 - -3.53% -
453.povray -2.37% 524493.000 512076.000 - -2.47% -
433.milc -2.24% 47001.000 45949.000 - -2.45% -
464.h264ref -1.99% 303767.000 297717.000 - -2.16% -
444.namd -1.85% 124963.000 122654.000 - -1.93% -
401.bzip2 -1.65% 33698.000 33141.000 - -2.52% -
458.sjeng -1.57% 80297.000 79034.000 - -1.93% -
456.hmmer -1.25% 80861.000 79849.000 - -1.29% -
470.lbm -1.22% 9338.000 9224.000 - -1.35% -
447.dealII -1.05% 195792.000 193739.000 - -1.08% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.34% 9509.000 63.000 - -99.36% -
462.libquantum -98.71% 14409.000 186.000 - -98.80% -
450.soplex -96.21% 15112.000 573.000 - -96.28% -
482.sphinx3 -96.16% 26682.000 1024.000 - -96.52% -
444.namd -93.44% 12400.000 814.000 - -94.05% -
453.povray -92.84% 6987.000 500.000 - -93.02% -
429.mcf -91.24% 13988.000 1226.000 - -89.52% -
433.milc -90.72% 13601.000 1262.000 - -91.07% -
401.bzip2 -81.89% 16137.000 2922.000 - -82.71% -
447.dealII -79.41% 9380.000 1931.000 - -81.82% -
471.omnetpp -72.86% 10477.000 2843.000 - -75.34% -
470.lbm -68.78% 13259.000 4140.000 - -72.99% -
456.hmmer -67.95% 11854.000 3799.000 - -71.80% -
473.astar -62.19% 12272.000 4640.000 - -65.51% -
464.h264ref -58.81% 14498.000 5972.000 - -62.24% -
445.gobmk -56.17% 11093.000 4862.000 - -57.43% -
458.sjeng -52.01% 17055.000 8185.000 - -49.17% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - -93.51% -
456.hmmer (max) -99.60% 1242.000 5.000 - -89.80% -
444.namd (max) -99.30% 1003.000 7.000 - -98.78% -
462.libquantum (max) -99.13% 919.000 8.000 - -52.94% -
456.hmmer (max) (percentage) -98.57% 10.480 0.150 - -59.46% -
444.namd -98.43% 318.000 5.000 - -98.45% -
444.namd (avg) -98.43% 318.000 5.000 - -98.45% -
482.sphinx3 -98.28% 464.000 8.000 - -98.14% -
482.sphinx3 (avg) -98.28% 464.000 8.000 - -98.14% -
462.libquantum -98.18% 110.000 2.000 - -91.30% -
462.libquantum (avg) -98.18% 110.000 2.000 - -91.30% -
403.gcc (max) -97.74% 221.000 5.000 - -91.38% -
458.sjeng (max) -97.60% 876.000 21.000 - 250.00% -
403.gcc -96.15% 52.000 2.000 - -93.75% -
403.gcc (avg) -96.15% 52.000 2.000 - -93.75% -
445.gobmk (max) -95.88% 413.000 17.000 - 41.67% -
482.sphinx3 (unexplained diff vs previous) -95.73% 914.000 39.000 - -33.90% -
456.hmmer (unexplained diff vs previous) -95.65% 46.000 2.000 - -95.92% -
482.sphinx3 (max) -95.47% 883.000 40.000 - -94.03% -
429.mcf (max) -95.27% 2281.000 108.000 - 575.00% -
447.dealII (max) -95.07% 223.000 11.000 - -94.27% -
458.sjeng (max) (percentage) -94.94% 5.140 0.260 - 550.00% -
470.lbm -94.78% 441.000 23.000 - -95.19% -
470.lbm (avg) -94.78% 441.000 23.000 - -95.19% -
429.mcf -94.59% 296.000 16.000 - -61.90% -
429.mcf (avg) -94.59% 296.000 16.000 - -61.90% -
453.povray -92.73% 55.000 4.000 - -94.29% -
453.povray (avg) -92.73% 55.000 4.000 - -94.29% -
433.milc -92.24% 116.000 9.000 - -93.79% -
433.milc (avg) -92.24% 116.000 9.000 - -93.79% -
473.astar (max) -91.36% 822.000 71.000 - 688.89% -
471.omnetpp (max) -90.43% 972.000 93.000 - -55.92% -
445.gobmk (max) (percentage) -90.08% 3.730 0.370 - 236.36% -
450.soplex -89.33% 75.000 8.000 - -83.67% -
450.soplex (avg) -89.33% 75.000 8.000 - -83.67% -
444.namd (max) (percentage) -89.25% 8.090 0.870 - -79.24% -
450.soplex (max) -89.16% 249.000 27.000 - 68.75% -
401.bzip2 -87.50% 104.000 13.000 - -56.67% -
401.bzip2 (avg) -87.50% 104.000 13.000 - -56.67% -
456.hmmer (unexplained diff vs previous) (percentage) -87.18% 0.390 0.050 - -86.11% -
447.dealII -85.71% 98.000 14.000 - -85.71% -
447.dealII (avg) -85.71% 98.000 14.000 - -85.71% -
456.hmmer -84.76% 164.000 25.000 - -44.44% -
456.hmmer (avg) -84.76% 164.000 25.000 - -44.44% -
445.gobmk -83.78% 74.000 12.000 - -45.45% -
445.gobmk (avg) -83.78% 74.000 12.000 - -45.45% -
401.bzip2 (max) -83.42% 609.000 101.000 - 206.06% -
447.dealII (unexplained diff vs previous) -83.33% 54.000 9.000 - -79.55% -
470.lbm (avg) (percentage) -82.88% 3.330 0.570 - -81.73% -
458.sjeng -80.88% 136.000 26.000 - -18.75% -
458.sjeng (avg) -80.88% 136.000 26.000 - -18.75% -
470.lbm (unexplained diff vs previous) -80.72% 804.000 155.000 - 2483.33% -
473.astar -80.36% 112.000 22.000 - 57.14% -
473.astar (avg) -80.36% 112.000 22.000 - 57.14% -
470.lbm (max) -78.76% 758.000 161.000 - -80.25% -
473.astar (max) (percentage) -76.87% 6.700 1.550 - 2114.29% -
471.omnetpp -76.65% 167.000 39.000 - -62.14% -
471.omnetpp (avg) -76.65% 167.000 39.000 - -62.14% -
447.dealII (max) (percentage) -74.79% 2.380 0.600 - -66.85% -
403.gcc (unexplained diff vs previous) -72.73% 11.000 3.000 - -94.83% -
453.povray (max) -72.00% 25.000 7.000 - -91.86% -
444.namd (avg) (percentage) -71.60% 2.570 0.730 - -69.07% -
464.h264ref -69.05% 42.000 13.000 - -71.11% -
464.h264ref (avg) -69.05% 42.000 13.000 - -71.11% -
471.omnetpp (max) (percentage) -64.44% 9.280 3.300 - 80.33% -
450.soplex (unexplained diff vs previous) -62.32% 69.000 26.000 - 1200.00% -
445.gobmk (avg) (percentage) -61.19% 0.670 0.260 - 30.00% -
471.omnetpp (unexplained diff vs previous) -61.17% 103.000 40.000 - -81.40% -
458.sjeng (avg) (percentage) -60.00% 0.800 0.320 - 60.00% -
445.gobmk (unexplained diff vs previous) -55.00% 40.000 18.000 - 350.00% -
482.sphinx3 (avg) (percentage) -54.60% 1.740 0.790 - -45.89% -
456.hmmer (avg) (percentage) -52.52% 1.390 0.660 - 94.12% -
473.astar (avg) (percentage) -46.74% 0.920 0.490 - 345.45% -
429.mcf (max) (percentage) -45.55% 16.310 8.880 - 6242.86% -
470.lbm (unexplained diff vs previous) (percentage) -38.28% 6.060 3.740 - 9250.00% -
429.mcf (avg) (percentage) -36.32% 2.120 1.350 - 275.00% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 350.00% -
470.lbm (max) (percentage) -31.99% 5.720 3.890 - -26.88% -
401.bzip2 (avg) (percentage) -30.77% 0.650 0.450 - 150.00% -
447.dealII (avg) (percentage) -30.48% 1.050 0.730 - -21.51% -
462.libquantum (max) (percentage) -29.62% 6.380 4.490 - 3981.82% -
464.h264ref (max) -25.00% 60.000 45.000 - -48.86% -
464.h264ref (avg) (percentage) -24.14% 0.290 0.220 - -24.14% -
447.dealII (unexplained diff vs previous) (percentage) -18.97% 0.580 0.470 - 14.63% -
433.milc (avg) (percentage) -13.95% 0.860 0.740 - -28.16% -
471.omnetpp (avg) (percentage) -13.75% 1.600 1.380 - 53.33% -
444.namd (unexplained diff vs previous) -12.50% 8.000 7.000 - -68.18% -
433.milc (max) -12.04% 108.000 95.000 - -54.76% -
401.bzip2 (max) (percentage) -8.20% 3.780 3.470 - 1635.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
401.bzip2 2922.000 -82.71%
403.gcc 63.000 -99.36%
429.mcf 1226.000 -89.52%
433.milc 1262.000 -91.07%
444.namd 814.000 -94.05%
445.gobmk 4862.000 -57.43%
447.dealII 1931.000 -81.82%
450.soplex 573.000 -96.28%
453.povray 500.000 -93.02%
456.hmmer 3799.000 -71.80%
458.sjeng 8185.000 -49.17%
462.libquantum 186.000 -98.80%
464.h264ref 5972.000 -62.24%
470.lbm 4140.000 -72.99%
471.omnetpp 2843.000 -75.34%
473.astar 4640.000 -65.51%
482.sphinx3 1024.000 -96.52%
Geometric Mean 1517.838 -88.84%
Code Size (bytes)
Name Current %
401.bzip2 33141.000 -2.52%
403.gcc 1653391.000 -
429.mcf 6927.000 -4.36%
433.milc 45949.000 -2.45%
444.namd 122654.000 -1.93%
445.gobmk 1255485.000 -
447.dealII 193739.000 -1.08%
450.soplex 150036.000 2.15%
453.povray 512076.000 -2.47%
456.hmmer 79849.000 -1.29%
458.sjeng 79034.000 -1.93%
462.libquantum 12055.000 -3.53%
464.h264ref 297717.000 -2.16%
470.lbm 9224.000 -1.35%
471.omnetpp 323250.000 2.07%
473.astar 15607.000 -
482.sphinx3 87440.000 -
Geometric Mean 94295.473 -1.33%
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 13.000 -56.67%
401.bzip2 (avg) 13.000 -56.67%
401.bzip2 (avg) (percentage) 0.450 150.00%
401.bzip2 (max) 101.000 206.06%
401.bzip2 (max) (percentage) 3.470 1635.00%
401.bzip2 (unexplained diff vs previous) 98.000 553.33%
401.bzip2 (unexplained diff vs previous) (percentage) 3.350 3622.22%
403.gcc 2.000 -93.75%
403.gcc (avg) 2.000 -93.75%
403.gcc (avg) (percentage) 3.300 900.00%
403.gcc (max) 5.000 -91.38%
403.gcc (max) (percentage) 8.550 1349.15%
403.gcc (unexplained diff vs previous) 3.000 -94.83%
403.gcc (unexplained diff vs previous) (percentage) 4.760 706.78%
429.mcf 16.000 -61.90%
429.mcf (avg) 16.000 -61.90%
429.mcf (avg) (percentage) 1.350 275.00%
429.mcf (max) 108.000 575.00%
429.mcf (max) (percentage) 8.880 6242.86%
429.mcf (unexplained diff vs previous) 100.000 1900.00%
429.mcf (unexplained diff vs previous) (percentage) 8.160 20300.00%
433.milc 9.000 -93.79%
433.milc (avg) 9.000 -93.79%
433.milc (avg) (percentage) 0.740 -28.16%
433.milc (max) 95.000 -54.76%
433.milc (max) (percentage) 7.590 409.40%
433.milc (unexplained diff vs previous) 89.000 -49.14%
433.milc (unexplained diff vs previous) (percentage) 7.050 468.55%
444.namd 5.000 -98.45%
444.namd (avg) 5.000 -98.45%
444.namd (avg) (percentage) 0.730 -69.07%
444.namd (max) 7.000 -98.78%
444.namd (max) (percentage) 0.870 -79.24%
444.namd (unexplained diff vs previous) 7.000 -68.18%
444.namd (unexplained diff vs previous) (percentage) 0.860 437.50%
445.gobmk 12.000 -45.45%
445.gobmk (avg) 12.000 -45.45%
445.gobmk (avg) (percentage) 0.260 30.00%
445.gobmk (max) 17.000 41.67%
445.gobmk (max) (percentage) 0.370 236.36%
445.gobmk (unexplained diff vs previous) 18.000 350.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.370 825.00%
447.dealII 14.000 -85.71%
447.dealII (avg) 14.000 -85.71%
447.dealII (avg) (percentage) 0.730 -21.51%
447.dealII (max) 11.000 -94.27%
447.dealII (max) (percentage) 0.600 -66.85%
447.dealII (unexplained diff vs previous) 9.000 -79.55%
447.dealII (unexplained diff vs previous) (percentage) 0.470 14.63%
450.soplex 8.000 -83.67%
450.soplex (avg) 8.000 -83.67%
450.soplex (avg) (percentage) 1.400 337.50%
450.soplex (max) 27.000 68.75%
450.soplex (max) (percentage) 4.750 4218.18%
450.soplex (unexplained diff vs previous) 26.000 1200.00%
450.soplex (unexplained diff vs previous) (percentage) 4.540 45300.00%
453.povray 4.000 -94.29%
453.povray (avg) 4.000 -94.29%
453.povray (avg) (percentage) 0.990 -
453.povray (max) 7.000 -91.86%
453.povray (max) (percentage) 1.550 28.10%
456.hmmer 25.000 -44.44%
456.hmmer (avg) 25.000 -44.44%
456.hmmer (avg) (percentage) 0.660 94.12%
456.hmmer (max) 5.000 -89.80%
456.hmmer (max) (percentage) 0.150 -59.46%
456.hmmer (unexplained diff vs previous) 2.000 -95.92%
456.hmmer (unexplained diff vs previous) (percentage) 0.050 -86.11%
458.sjeng 26.000 -18.75%
458.sjeng (avg) 26.000 -18.75%
458.sjeng (avg) (percentage) 0.320 60.00%
458.sjeng (max) 21.000 250.00%
458.sjeng (max) (percentage) 0.260 550.00%
458.sjeng (unexplained diff vs previous) 3.000 -40.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.040 33.33%
462.libquantum 2.000 -91.30%
462.libquantum (avg) 2.000 -91.30%
462.libquantum (avg) (percentage) 1.570 946.67%
462.libquantum (max) 8.000 -52.94%
462.libquantum (max) (percentage) 4.490 3981.82%
462.libquantum (unexplained diff vs previous) 8.000 -11.11%
462.libquantum (unexplained diff vs previous) (percentage) 4.300 7066.67%
464.h264ref 13.000 -71.11%
464.h264ref (avg) 13.000 -71.11%
464.h264ref (avg) (percentage) 0.220 -24.14%
464.h264ref (max) 45.000 -48.86%
464.h264ref (max) (percentage) 0.760 35.71%
464.h264ref (unexplained diff vs previous) 45.000 -4.26%
464.h264ref (unexplained diff vs previous) (percentage) 0.750 150.00%
470.lbm 23.000 -95.19%
470.lbm (avg) 23.000 -95.19%
470.lbm (avg) (percentage) 0.570 -81.73%
470.lbm (max) 161.000 -80.25%
470.lbm (max) (percentage) 3.890 -26.88%
470.lbm (unexplained diff vs previous) 155.000 2483.33%
470.lbm (unexplained diff vs previous) (percentage) 3.740 9250.00%
471.omnetpp 39.000 -62.14%
471.omnetpp (avg) 39.000 -62.14%
471.omnetpp (avg) (percentage) 1.380 53.33%
471.omnetpp (max) 93.000 -55.92%
471.omnetpp (max) (percentage) 3.300 80.33%
471.omnetpp (unexplained diff vs previous) 40.000 -81.40%
471.omnetpp (unexplained diff vs previous) (percentage) 1.410 -24.19%
473.astar 22.000 57.14%
473.astar (avg) 22.000 57.14%
473.astar (avg) (percentage) 0.490 345.45%
473.astar (max) 71.000 688.89%
473.astar (max) (percentage) 1.550 2114.29%
473.astar (unexplained diff vs previous) 71.000 -
473.astar (unexplained diff vs previous) (percentage) 1.530 -
482.sphinx3 8.000 -98.14%
482.sphinx3 (avg) 8.000 -98.14%
482.sphinx3 (avg) (percentage) 0.790 -45.89%
482.sphinx3 (max) 40.000 -94.03%
482.sphinx3 (max) (percentage) 3.960 73.68%
482.sphinx3 (unexplained diff vs previous) 39.000 -33.90%
482.sphinx3 (unexplained diff vs previous) (percentage) 3.810 1805.00%
483.xalancbmk (avg) (percentage) 1.440 350.00%
483.xalancbmk (max) (percentage) 0.050 -93.51%
Geometric Mean 5.066 8.00%