Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/44514316bd5ef656076b6baaf6bccb298d98f0ea (+ 610 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/215/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0215 (llvmorg-20-init-16160-g44514316bd5e) 2024-12-17T03:32:04 0:00:00
Previous 0215 (no-change) 2022-11-08T15:21:46 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 39 19
Performance Improvements 106 122
Added Tests 4 6
Unchanged Tests 631 633
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
450.soplex 2.15% 146877.000 150036.000 - 2.23% -
471.omnetpp 2.07% 316696.000 323250.000 - 2.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) (percentage) 3075.00% 0.040 1.270 - -80.09% -
462.libquantum (unexplained diff vs previous) (percentage) 2700.00% 0.020 0.560 - 5500.00% -
458.sjeng (unexplained diff vs previous) (percentage) 1900.00% 0.010 0.200 - 0.00% -
403.gcc (max) (percentage) 1552.83% 0.530 8.760 - 275.97% -
450.soplex (unexplained diff vs previous) (percentage) 1085.71% 0.140 1.660 - 260.87% -
403.gcc (avg) (percentage) 989.66% 0.290 3.160 - 474.55% -
462.libquantum (avg) (percentage) 850.00% 0.160 1.520 - 97.40% -
473.astar (unexplained diff vs previous) (percentage) 744.44% 0.090 0.760 - 280.00% -
458.sjeng (unexplained diff vs previous) 700.00% 2.000 16.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) 520.00% 0.100 0.620 - 287.50% -
433.milc (unexplained diff vs previous) (percentage) 514.29% 0.070 0.430 - 330.00% -
483.xalancbmk (avg) (percentage) 453.85% 0.260 1.440 - -32.08% -
473.astar (max) (percentage) 352.63% 0.190 0.860 - -87.16% -
473.astar (avg) (percentage) 276.92% 0.130 0.490 - -46.74% -
444.namd (unexplained diff vs previous) (percentage) 257.14% 0.070 0.250 - 316.67% -
450.soplex (avg) (percentage) 235.90% 0.390 1.310 - 162.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 211.11% 0.090 0.280 - 40.00% -
473.astar (unexplained diff vs previous) 191.67% 12.000 35.000 - 40.00% -
429.mcf (avg) (percentage) 179.07% 0.430 1.200 - -43.40% -
450.soplex (max) (percentage) 172.58% 0.620 1.690 - 2.42% -
471.omnetpp (max) (percentage) 143.82% 0.890 2.170 - -76.62% -
453.povray (unexplained diff vs previous) (percentage) 129.17% 0.720 1.650 - 0.00% -
401.bzip2 (avg) (percentage) 116.67% 0.180 0.390 - -40.00% -
471.omnetpp (avg) (percentage) 112.50% 0.640 1.360 - -15.00% -
401.bzip2 (max) (percentage) 100.00% 0.140 0.280 - -92.59% -
456.hmmer (avg) (percentage) 88.89% 0.360 0.680 - -51.08% -
429.mcf (max) (percentage) 86.27% 0.510 0.950 - -94.18% -
458.sjeng (max) (percentage) 60.00% 0.200 0.320 - -93.77% -
471.omnetpp (unexplained diff vs previous) (percentage) 58.54% 0.410 0.650 - -33.67% -
473.astar (max) 56.00% 25.000 39.000 - -95.26% -
458.sjeng (avg) (percentage) 32.00% 0.250 0.330 - -58.75% -
473.astar 29.41% 17.000 22.000 - -80.36% -
473.astar (avg) 29.41% 17.000 22.000 - -80.36% -
453.povray (max) (percentage) 19.12% 1.360 1.620 - 350.00% -
445.gobmk (avg) (percentage) 13.04% 0.230 0.260 - -61.19% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
429.mcf -4.36% 7243.000 6927.000 - -4.31% -
462.libquantum -3.53% 12496.000 12055.000 - -3.59% -
401.bzip2 -2.52% 33998.000 33141.000 - -1.65% -
453.povray -2.48% 525021.000 512012.000 - -2.38% -
433.milc -2.45% 47105.000 45949.000 - -2.24% -
464.h264ref -1.98% 304295.000 298269.000 - -1.81% -
444.namd -1.93% 125071.000 122654.000 - -1.85% -
458.sjeng -1.93% 80589.000 79034.000 - -1.57% -
470.lbm -1.35% 9350.000 9224.000 - -1.22% -
456.hmmer -1.29% 80893.000 79849.000 - -1.25% -
447.dealII -1.08% 195856.000 193739.000 - -1.05% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.41% 9755.000 58.000 - -99.39% -
462.libquantum -98.84% 15526.000 180.000 - -98.75% -
482.sphinx3 -96.64% 29362.000 988.000 - -96.30% -
450.soplex -96.49% 15431.000 542.000 - -96.41% -
444.namd -94.09% 13714.000 810.000 - -93.47% -
453.povray -93.25% 7190.000 485.000 - -93.06% -
433.milc -91.66% 13976.000 1165.000 - -91.43% -
429.mcf -90.32% 11669.000 1129.000 - -91.93% -
401.bzip2 -83.32% 16875.000 2815.000 - -82.56% -
447.dealII -81.72% 10571.000 1932.000 - -79.40% -
471.omnetpp -75.99% 11512.000 2764.000 - -73.62% -
470.lbm -74.00% 15334.000 3987.000 - -69.93% -
456.hmmer -71.56% 13399.000 3811.000 - -67.85% -
473.astar -65.99% 13460.000 4578.000 - -62.70% -
464.h264ref -62.18% 15745.000 5954.000 - -58.93% -
445.gobmk -57.70% 11441.000 4840.000 - -56.37% -
458.sjeng -49.42% 16100.000 8143.000 - -52.25% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (max) -99.36% 781.000 5.000 - -99.43% -
482.sphinx3 -98.71% 543.000 7.000 - -98.49% -
482.sphinx3 (avg) -98.71% 543.000 7.000 - -98.49% -
444.namd (max) -98.68% 684.000 9.000 - -99.10% -
444.namd -98.53% 407.000 6.000 - -98.11% -
444.namd (avg) -98.53% 407.000 6.000 - -98.11% -
470.lbm (max) -98.44% 1087.000 17.000 - -97.76% -
470.lbm (unexplained diff vs previous) -98.33% 1015.000 17.000 - -97.89% -
433.milc (max) -97.90% 238.000 5.000 - -95.37% -
456.hmmer (unexplained diff vs previous) -97.73% 44.000 1.000 - -97.83% -
470.lbm -96.66% 599.000 20.000 - -95.46% -
470.lbm (avg) -96.66% 599.000 20.000 - -95.46% -
403.gcc -96.43% 28.000 1.000 - -98.08% -
403.gcc (avg) -96.43% 28.000 1.000 - -98.08% -
433.milc -95.51% 156.000 7.000 - -93.97% -
433.milc (avg) -95.51% 156.000 7.000 - -93.97% -
453.povray -95.29% 85.000 4.000 - -92.73% -
453.povray (avg) -95.29% 85.000 4.000 - -92.73% -
483.xalancbmk (max) (percentage) -94.32% 0.880 0.050 - -99.61% -
470.lbm (max) (percentage) -93.94% 7.090 0.430 - -92.48% -
470.lbm (unexplained diff vs previous) (percentage) -93.50% 6.620 0.430 - -92.90% -
453.povray (max) -92.78% 97.000 7.000 - -72.00% -
462.libquantum -91.67% 24.000 2.000 - -98.18% -
462.libquantum (avg) -91.67% 24.000 2.000 - -98.18% -
456.hmmer (unexplained diff vs previous) (percentage) -90.91% 0.330 0.030 - -92.31% -
456.hmmer (max) -90.91% 44.000 4.000 - -99.68% -
450.soplex (max) -90.53% 95.000 9.000 - -96.39% -
447.dealII (max) -90.41% 219.000 21.000 - -90.58% -
403.gcc (max) -90.20% 51.000 5.000 - -97.74% -
447.dealII -88.52% 122.000 14.000 - -85.71% -
447.dealII (avg) -88.52% 122.000 14.000 - -85.71% -
450.soplex -88.33% 60.000 7.000 - -90.67% -
450.soplex (avg) -88.33% 60.000 7.000 - -90.67% -
470.lbm (avg) (percentage) -86.70% 3.910 0.520 - -84.38% -
453.povray (unexplained diff vs previous) -84.62% 52.000 8.000 - 0.00% -
429.mcf (max) -83.05% 59.000 10.000 - -99.56% -
445.gobmk (max) -80.95% 42.000 8.000 - -98.06% -
464.h264ref (max) -79.25% 53.000 11.000 - -81.67% -
482.sphinx3 (max) (percentage) -78.57% 2.660 0.570 - -82.78% -
444.namd (unexplained diff vs previous) -77.78% 9.000 2.000 - -75.00% -
444.namd (max) (percentage) -76.55% 4.990 1.170 - -85.54% -
433.milc (max) (percentage) -74.85% 1.710 0.430 - -46.25% -
444.namd (avg) (percentage) -74.75% 2.970 0.750 - -70.82% -
429.mcf -74.00% 50.000 13.000 - -95.61% -
429.mcf (avg) -74.00% 50.000 13.000 - -95.61% -
401.bzip2 (max) -69.57% 23.000 7.000 - -98.85% -
401.bzip2 -66.67% 30.000 10.000 - -90.38% -
401.bzip2 (avg) -66.67% 30.000 10.000 - -90.38% -
462.libquantum (max) -66.67% 6.000 2.000 - -99.78% -
462.libquantum (unexplained diff vs previous) -66.67% 3.000 1.000 - 0.00% -
464.h264ref -64.71% 34.000 12.000 - -71.43% -
464.h264ref (avg) -64.71% 34.000 12.000 - -71.43% -
456.hmmer (max) (percentage) -63.64% 0.330 0.120 - -98.85% -
471.omnetpp (unexplained diff vs previous) -61.70% 47.000 18.000 - -82.52% -
445.gobmk (unexplained diff vs previous) -60.00% 5.000 2.000 - -95.00% -
482.sphinx3 (avg) (percentage) -60.00% 1.850 0.740 - -57.47% -
450.soplex (unexplained diff vs previous) -59.09% 22.000 9.000 - -86.96% -
445.gobmk -53.85% 26.000 12.000 - -83.78% -
445.gobmk (avg) -53.85% 26.000 12.000 - -83.78% -
445.gobmk (max) (percentage) -51.35% 0.370 0.180 - -95.17% -
433.milc (unexplained diff vs previous) -50.00% 10.000 5.000 - -61.54% -
471.omnetpp -49.32% 73.000 37.000 - -77.84% -
471.omnetpp (avg) -49.32% 73.000 37.000 - -77.84% -
456.hmmer -47.92% 48.000 25.000 - -84.76% -
456.hmmer (avg) -47.92% 48.000 25.000 - -84.76% -
447.dealII (max) (percentage) -47.12% 2.080 1.100 - -53.78% -
401.bzip2 (unexplained diff vs previous) -46.67% 15.000 8.000 - -75.76% -
464.h264ref (max) (percentage) -44.12% 0.340 0.190 - -54.76% -
433.milc (avg) (percentage) -43.75% 1.120 0.630 - -26.74% -
471.omnetpp (max) -42.16% 102.000 59.000 - -93.93% -
429.mcf (unexplained diff vs previous) -41.67% 12.000 7.000 - -68.18% -
447.dealII (avg) (percentage) -35.34% 1.160 0.750 - -28.57% -
458.sjeng -35.00% 40.000 26.000 - -80.88% -
458.sjeng (avg) -35.00% 40.000 26.000 - -80.88% -
458.sjeng (max) -18.75% 32.000 26.000 - -97.03% -
453.povray (avg) (percentage) -18.49% 1.190 0.970 - 21.25% -
464.h264ref (avg) (percentage) -4.55% 0.220 0.210 - -27.59% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 2.23% 146765.000 150036.000 - 2.15% -
471.omnetpp 2.08% 316676.000 323250.000 - 2.07% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 5500.00% 0.010 0.560 - 2700.00% -
403.gcc (avg) (percentage) 474.55% 0.550 3.160 - 989.66% -
453.povray (max) (percentage) 350.00% 0.360 1.620 - 19.12% -
433.milc (unexplained diff vs previous) (percentage) 330.00% 0.100 0.430 - 514.29% -
444.namd (unexplained diff vs previous) (percentage) 316.67% 0.060 0.250 - 257.14% -
429.mcf (unexplained diff vs previous) (percentage) 287.50% 0.160 0.620 - 520.00% -
473.astar (unexplained diff vs previous) (percentage) 280.00% 0.200 0.760 - 744.44% -
403.gcc (max) (percentage) 275.97% 2.330 8.760 - 1552.83% -
450.soplex (unexplained diff vs previous) (percentage) 260.87% 0.460 1.660 - 1085.71% -
450.soplex (avg) (percentage) 162.00% 0.500 1.310 - 235.90% -
462.libquantum (avg) (percentage) 97.40% 0.770 1.520 - 850.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 40.00% 0.200 0.280 - 211.11% -
473.astar (unexplained diff vs previous) 40.00% 25.000 35.000 - 191.67% -
453.povray (avg) (percentage) 21.25% 0.800 0.970 - -18.49% -
450.soplex (max) (percentage) 2.42% 1.650 1.690 - 172.58% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.31% 7239.000 6927.000 - -4.36% -
462.libquantum -3.59% 12504.000 12055.000 - -3.53% -
453.povray -2.38% 524493.000 512012.000 - -2.48% -
433.milc -2.24% 47001.000 45949.000 - -2.45% -
444.namd -1.85% 124963.000 122654.000 - -1.93% -
464.h264ref -1.81% 303767.000 298269.000 - -1.98% -
401.bzip2 -1.65% 33698.000 33141.000 - -2.52% -
458.sjeng -1.57% 80297.000 79034.000 - -1.93% -
456.hmmer -1.25% 80861.000 79849.000 - -1.29% -
470.lbm -1.22% 9338.000 9224.000 - -1.35% -
447.dealII -1.05% 195792.000 193739.000 - -1.08% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.39% 9509.000 58.000 - -99.41% -
462.libquantum -98.75% 14409.000 180.000 - -98.84% -
450.soplex -96.41% 15112.000 542.000 - -96.49% -
482.sphinx3 -96.30% 26682.000 988.000 - -96.64% -
444.namd -93.47% 12400.000 810.000 - -94.09% -
453.povray -93.06% 6987.000 485.000 - -93.25% -
429.mcf -91.93% 13988.000 1129.000 - -90.32% -
433.milc -91.43% 13601.000 1165.000 - -91.66% -
401.bzip2 -82.56% 16137.000 2815.000 - -83.32% -
447.dealII -79.40% 9380.000 1932.000 - -81.72% -
471.omnetpp -73.62% 10477.000 2764.000 - -75.99% -
470.lbm -69.93% 13259.000 3987.000 - -74.00% -
456.hmmer -67.85% 11854.000 3811.000 - -71.56% -
473.astar -62.70% 12272.000 4578.000 - -65.99% -
464.h264ref -58.93% 14498.000 5954.000 - -62.18% -
445.gobmk -56.37% 11093.000 4840.000 - -57.70% -
458.sjeng -52.25% 17055.000 8143.000 - -49.42% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) -99.78% 919.000 2.000 - -66.67% -
456.hmmer (max) -99.68% 1242.000 4.000 - -90.91% -
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - -94.32% -
429.mcf (max) -99.56% 2281.000 10.000 - -83.05% -
482.sphinx3 (max) -99.43% 883.000 5.000 - -99.36% -
444.namd (max) -99.10% 1003.000 9.000 - -98.68% -
456.hmmer (max) (percentage) -98.85% 10.480 0.120 - -63.64% -
401.bzip2 (max) -98.85% 609.000 7.000 - -69.57% -
482.sphinx3 -98.49% 464.000 7.000 - -98.71% -
482.sphinx3 (avg) -98.49% 464.000 7.000 - -98.71% -
462.libquantum -98.18% 110.000 2.000 - -91.67% -
462.libquantum (avg) -98.18% 110.000 2.000 - -91.67% -
444.namd -98.11% 318.000 6.000 - -98.53% -
444.namd (avg) -98.11% 318.000 6.000 - -98.53% -
403.gcc -98.08% 52.000 1.000 - -96.43% -
403.gcc (avg) -98.08% 52.000 1.000 - -96.43% -
445.gobmk (max) -98.06% 413.000 8.000 - -80.95% -
470.lbm (unexplained diff vs previous) -97.89% 804.000 17.000 - -98.33% -
456.hmmer (unexplained diff vs previous) -97.83% 46.000 1.000 - -97.73% -
470.lbm (max) -97.76% 758.000 17.000 - -98.44% -
403.gcc (max) -97.74% 221.000 5.000 - -90.20% -
458.sjeng (max) -97.03% 876.000 26.000 - -18.75% -
450.soplex (max) -96.39% 249.000 9.000 - -90.53% -
429.mcf -95.61% 296.000 13.000 - -74.00% -
429.mcf (avg) -95.61% 296.000 13.000 - -74.00% -
470.lbm -95.46% 441.000 20.000 - -96.66% -
470.lbm (avg) -95.46% 441.000 20.000 - -96.66% -
433.milc (max) -95.37% 108.000 5.000 - -97.90% -
473.astar (max) -95.26% 822.000 39.000 - 56.00% -
445.gobmk (max) (percentage) -95.17% 3.730 0.180 - -51.35% -
445.gobmk (unexplained diff vs previous) -95.00% 40.000 2.000 - -60.00% -
429.mcf (max) (percentage) -94.18% 16.310 0.950 - 86.27% -
433.milc -93.97% 116.000 7.000 - -95.51% -
433.milc (avg) -93.97% 116.000 7.000 - -95.51% -
471.omnetpp (max) -93.93% 972.000 59.000 - -42.16% -
458.sjeng (max) (percentage) -93.77% 5.140 0.320 - 60.00% -
470.lbm (unexplained diff vs previous) (percentage) -92.90% 6.060 0.430 - -93.50% -
453.povray -92.73% 55.000 4.000 - -95.29% -
453.povray (avg) -92.73% 55.000 4.000 - -95.29% -
401.bzip2 (max) (percentage) -92.59% 3.780 0.280 - 100.00% -
470.lbm (max) (percentage) -92.48% 5.720 0.430 - -93.94% -
456.hmmer (unexplained diff vs previous) (percentage) -92.31% 0.390 0.030 - -90.91% -
450.soplex -90.67% 75.000 7.000 - -88.33% -
450.soplex (avg) -90.67% 75.000 7.000 - -88.33% -
447.dealII (max) -90.58% 223.000 21.000 - -90.41% -
401.bzip2 -90.38% 104.000 10.000 - -66.67% -
401.bzip2 (avg) -90.38% 104.000 10.000 - -66.67% -
445.gobmk (unexplained diff vs previous) (percentage) -88.89% 0.360 0.040 - 0.00% -
473.astar (max) (percentage) -87.16% 6.700 0.860 - 352.63% -
450.soplex (unexplained diff vs previous) -86.96% 69.000 9.000 - -59.09% -
447.dealII -85.71% 98.000 14.000 - -88.52% -
447.dealII (avg) -85.71% 98.000 14.000 - -88.52% -
444.namd (max) (percentage) -85.54% 8.090 1.170 - -76.55% -
456.hmmer -84.76% 164.000 25.000 - -47.92% -
456.hmmer (avg) -84.76% 164.000 25.000 - -47.92% -
470.lbm (avg) (percentage) -84.38% 3.330 0.520 - -86.70% -
445.gobmk -83.78% 74.000 12.000 - -53.85% -
445.gobmk (avg) -83.78% 74.000 12.000 - -53.85% -
482.sphinx3 (max) (percentage) -82.78% 3.310 0.570 - -78.57% -
471.omnetpp (unexplained diff vs previous) -82.52% 103.000 18.000 - -61.70% -
464.h264ref (max) -81.67% 60.000 11.000 - -79.25% -
458.sjeng -80.88% 136.000 26.000 - -35.00% -
458.sjeng (avg) -80.88% 136.000 26.000 - -35.00% -
473.astar -80.36% 112.000 22.000 - 29.41% -
473.astar (avg) -80.36% 112.000 22.000 - 29.41% -
462.libquantum (max) (percentage) -80.09% 6.380 1.270 - 3075.00% -
471.omnetpp -77.84% 167.000 37.000 - -49.32% -
471.omnetpp (avg) -77.84% 167.000 37.000 - -49.32% -
471.omnetpp (max) (percentage) -76.62% 9.280 2.170 - 143.82% -
401.bzip2 (unexplained diff vs previous) -75.76% 33.000 8.000 - -46.67% -
444.namd (unexplained diff vs previous) -75.00% 8.000 2.000 - -77.78% -
453.povray (max) -72.00% 25.000 7.000 - -92.78% -
464.h264ref -71.43% 42.000 12.000 - -64.71% -
464.h264ref (avg) -71.43% 42.000 12.000 - -64.71% -
444.namd (avg) (percentage) -70.82% 2.570 0.750 - -74.75% -
429.mcf (unexplained diff vs previous) -68.18% 22.000 7.000 - -41.67% -
433.milc (unexplained diff vs previous) -61.54% 13.000 5.000 - -50.00% -
445.gobmk (avg) (percentage) -61.19% 0.670 0.260 - 13.04% -
458.sjeng (avg) (percentage) -58.75% 0.800 0.330 - 32.00% -
482.sphinx3 (avg) (percentage) -57.47% 1.740 0.740 - -60.00% -
464.h264ref (max) (percentage) -54.76% 0.420 0.190 - -44.12% -
447.dealII (max) (percentage) -53.78% 2.380 1.100 - -47.12% -
456.hmmer (avg) (percentage) -51.08% 1.390 0.680 - 88.89% -
473.astar (avg) (percentage) -46.74% 0.920 0.490 - 276.92% -
433.milc (max) (percentage) -46.25% 0.800 0.430 - -74.85% -
429.mcf (avg) (percentage) -43.40% 2.120 1.200 - 179.07% -
401.bzip2 (avg) (percentage) -40.00% 0.650 0.390 - 116.67% -
471.omnetpp (unexplained diff vs previous) (percentage) -33.67% 0.980 0.650 - 58.54% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 453.85% -
447.dealII (avg) (percentage) -28.57% 1.050 0.750 - -35.34% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - -4.55% -
433.milc (avg) (percentage) -26.74% 0.860 0.630 - -43.75% -
471.omnetpp (avg) (percentage) -15.00% 1.600 1.360 - 112.50% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Report Time: 0.37s
Execution (seconds)
Name Current %
401.bzip2 2815.000 -83.32%
403.gcc 58.000 -99.41%
429.mcf 1129.000 -90.32%
433.milc 1165.000 -91.66%
444.namd 810.000 -94.09%
445.gobmk 4840.000 -57.70%
447.dealII 1932.000 -81.72%
450.soplex 542.000 -96.49%
453.povray 485.000 -93.25%
456.hmmer 3811.000 -71.56%
458.sjeng 8143.000 -49.42%
462.libquantum 180.000 -98.84%
464.h264ref 5954.000 -62.18%
470.lbm 3987.000 -74.00%
471.omnetpp 2764.000 -75.99%
473.astar 4578.000 -65.99%
482.sphinx3 988.000 -96.64%
Geometric Mean 1471.222 -89.17%
Code Size (bytes)
Name Current %
401.bzip2 33141.000 -2.52%
403.gcc 1653599.000 -
429.mcf 6927.000 -4.36%
433.milc 45949.000 -2.45%
444.namd 122654.000 -1.93%
445.gobmk 1255485.000 -
447.dealII 193739.000 -1.08%
450.soplex 150036.000 2.15%
453.povray 512012.000 -2.48%
456.hmmer 79849.000 -1.29%
458.sjeng 79034.000 -1.93%
462.libquantum 12055.000 -3.53%
464.h264ref 298269.000 -1.98%
470.lbm 9224.000 -1.35%
471.omnetpp 323250.000 2.07%
473.astar 15607.000 -
482.sphinx3 87440.000 -
Geometric Mean 94305.753 -1.32%
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 10.000 -66.67%
401.bzip2 (avg) 10.000 -66.67%
401.bzip2 (avg) (percentage) 0.390 116.67%
401.bzip2 (max) 7.000 -69.57%
401.bzip2 (max) (percentage) 0.280 100.00%
401.bzip2 (unexplained diff vs previous) 8.000 -46.67%
401.bzip2 (unexplained diff vs previous) (percentage) 0.280 211.11%
403.gcc 1.000 -96.43%
403.gcc (avg) 1.000 -96.43%
403.gcc (avg) (percentage) 3.160 989.66%
403.gcc (max) 5.000 -90.20%
403.gcc (max) (percentage) 8.760 1552.83%
429.mcf 13.000 -74.00%
429.mcf (avg) 13.000 -74.00%
429.mcf (avg) (percentage) 1.200 179.07%
429.mcf (max) 10.000 -83.05%
429.mcf (max) (percentage) 0.950 86.27%
429.mcf (unexplained diff vs previous) 7.000 -41.67%
429.mcf (unexplained diff vs previous) (percentage) 0.620 520.00%
433.milc 7.000 -95.51%
433.milc (avg) 7.000 -95.51%
433.milc (avg) (percentage) 0.630 -43.75%
433.milc (max) 5.000 -97.90%
433.milc (max) (percentage) 0.430 -74.85%
433.milc (unexplained diff vs previous) 5.000 -50.00%
433.milc (unexplained diff vs previous) (percentage) 0.430 514.29%
444.namd 6.000 -98.53%
444.namd (avg) 6.000 -98.53%
444.namd (avg) (percentage) 0.750 -74.75%
444.namd (max) 9.000 -98.68%
444.namd (max) (percentage) 1.170 -76.55%
444.namd (unexplained diff vs previous) 2.000 -77.78%
444.namd (unexplained diff vs previous) (percentage) 0.250 257.14%
445.gobmk 12.000 -53.85%
445.gobmk (avg) 12.000 -53.85%
445.gobmk (avg) (percentage) 0.260 13.04%
445.gobmk (max) 8.000 -80.95%
445.gobmk (max) (percentage) 0.180 -51.35%
445.gobmk (unexplained diff vs previous) 2.000 -60.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.040 -
447.dealII 14.000 -88.52%
447.dealII (avg) 14.000 -88.52%
447.dealII (avg) (percentage) 0.750 -35.34%
447.dealII (max) 21.000 -90.41%
447.dealII (max) (percentage) 1.100 -47.12%
447.dealII (unexplained diff vs previous) 0.000 -
447.dealII (unexplained diff vs previous) (percentage) 0.000 -
450.soplex 7.000 -88.33%
450.soplex (avg) 7.000 -88.33%
450.soplex (avg) (percentage) 1.310 235.90%
450.soplex (max) 9.000 -90.53%
450.soplex (max) (percentage) 1.690 172.58%
450.soplex (unexplained diff vs previous) 9.000 -59.09%
450.soplex (unexplained diff vs previous) (percentage) 1.660 1085.71%
453.povray 4.000 -95.29%
453.povray (avg) 4.000 -95.29%
453.povray (avg) (percentage) 0.970 -18.49%
453.povray (max) 7.000 -92.78%
453.povray (max) (percentage) 1.620 19.12%
453.povray (unexplained diff vs previous) 8.000 -84.62%
453.povray (unexplained diff vs previous) (percentage) 1.650 129.17%
456.hmmer 25.000 -47.92%
456.hmmer (avg) 25.000 -47.92%
456.hmmer (avg) (percentage) 0.680 88.89%
456.hmmer (max) 4.000 -90.91%
456.hmmer (max) (percentage) 0.120 -63.64%
456.hmmer (unexplained diff vs previous) 1.000 -97.73%
456.hmmer (unexplained diff vs previous) (percentage) 0.030 -90.91%
458.sjeng 26.000 -35.00%
458.sjeng (avg) 26.000 -35.00%
458.sjeng (avg) (percentage) 0.330 32.00%
458.sjeng (max) 26.000 -18.75%
458.sjeng (max) (percentage) 0.320 60.00%
458.sjeng (unexplained diff vs previous) 16.000 700.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.200 1900.00%
462.libquantum 2.000 -91.67%
462.libquantum (avg) 2.000 -91.67%
462.libquantum (avg) (percentage) 1.520 850.00%
462.libquantum (max) 2.000 -66.67%
462.libquantum (max) (percentage) 1.270 3075.00%
462.libquantum (unexplained diff vs previous) 1.000 -66.67%
462.libquantum (unexplained diff vs previous) (percentage) 0.560 2700.00%
464.h264ref 12.000 -64.71%
464.h264ref (avg) 12.000 -64.71%
464.h264ref (avg) (percentage) 0.210 -4.55%
464.h264ref (max) 11.000 -79.25%
464.h264ref (max) (percentage) 0.190 -44.12%
470.lbm 20.000 -96.66%
470.lbm (avg) 20.000 -96.66%
470.lbm (avg) (percentage) 0.520 -86.70%
470.lbm (max) 17.000 -98.44%
470.lbm (max) (percentage) 0.430 -93.94%
470.lbm (unexplained diff vs previous) 17.000 -98.33%
470.lbm (unexplained diff vs previous) (percentage) 0.430 -93.50%
471.omnetpp 37.000 -49.32%
471.omnetpp (avg) 37.000 -49.32%
471.omnetpp (avg) (percentage) 1.360 112.50%
471.omnetpp (max) 59.000 -42.16%
471.omnetpp (max) (percentage) 2.170 143.82%
471.omnetpp (unexplained diff vs previous) 18.000 -61.70%
471.omnetpp (unexplained diff vs previous) (percentage) 0.650 58.54%
473.astar 22.000 29.41%
473.astar (avg) 22.000 29.41%
473.astar (avg) (percentage) 0.490 276.92%
473.astar (max) 39.000 56.00%
473.astar (max) (percentage) 0.860 352.63%
473.astar (unexplained diff vs previous) 35.000 191.67%
473.astar (unexplained diff vs previous) (percentage) 0.760 744.44%
482.sphinx3 7.000 -98.71%
482.sphinx3 (avg) 7.000 -98.71%
482.sphinx3 (avg) (percentage) 0.740 -60.00%
482.sphinx3 (max) 5.000 -99.36%
482.sphinx3 (max) (percentage) 0.570 -78.57%
482.sphinx3 (unexplained diff vs previous) 0.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.000 -
483.xalancbmk (avg) (percentage) 1.440 453.85%
483.xalancbmk (max) (percentage) 0.050 -94.32%
Geometric Mean 1.888 -70.94%