Machine Info

Fields

hardware	None
os	None

Parameters

Run Info

Fields

Name	Value

Parameters

Name	Value
git_llvm	https://github.com/llvm/llvm-project/commit/f061a395ffb78215a23e0f503e8ea121ee3b13ad (+ 20370 commits)
test_url	https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-arm-master-O2_LTO-build/173/

Report

Filter

llvm-arm-master-O2_LTO test results

Run	Order	Start Time	Duration
Current	0173 (llvmorg-19-init-09943-gf061a395ffb7)	2024-01-15T08:10:10	0:00:00
Previous	0046 (llvmorg-17-init-15504-gf146763e0788)	2023-06-13T07:01:29	0:00:00
Baseline	0015 (llvmorg-17-init-07395-g3283a6fb0df3)	2023-04-07T14:40:57	0:00:00

Tests Summary

Status Group	#	# (B)
Performance Regressions	6	1
Performance Improvements	14	4
Added Tests	71	86
Unchanged Tests	347	347
Total Tests	438

Run-Over-Run Changes Detail

Performance Regressions - Code Size	Δ	Previous	Current	σ	Δ (B)	σ (B)
505.mcf_r	2.20%	25776.000	26344.000	-	2.17%	-

Performance Regressions - sample variability	Δ	Previous	Current	σ	Δ (B)	σ (B)
557.xz_r (avg) (percentage)	37.50%	0.080	0.110	-	0.00%	-
557.xz_r	25.00%	8.000	10.000	-	0.00%	-
557.xz_r (avg)	25.00%	8.000	10.000	-	0.00%	-
557.xz_r (max) (percentage)	16.67%	0.120	0.140	-	0.00%	-
557.xz_r (max)	8.33%	12.000	13.000	-	0.00%	-

Performance Improvements - Execution	Δ	Previous	Current	σ	Δ (B)	σ (B)
557.xz_r	-2.95%	10025.000	9729.000	-	-3.11%	-

Performance Improvements - sample variability	Δ	Previous	Current	σ	Δ (B)	σ (B)
531.deepsjeng_r (max)	-68.92%	74.000	23.000	-	0.00%	-
531.deepsjeng_r (max) (percentage)	-67.74%	0.620	0.200	-	0.00%	-
531.deepsjeng_r	-30.19%	53.000	37.000	-	0.00%	-
531.deepsjeng_r (avg)	-30.19%	53.000	37.000	-	0.00%	-
531.deepsjeng_r (avg) (percentage)	-29.55%	0.440	0.310	-	0.00%	-
505.mcf_r (max)	-20.00%	45.000	36.000	-	0.00%	-
505.mcf_r (max) (percentage)	-18.42%	0.380	0.310	-	0.00%	-
505.mcf_r	-14.81%	27.000	23.000	-	0.00%	-
505.mcf_r (avg)	-14.81%	27.000	23.000	-	0.00%	-
505.mcf_r (avg) (percentage)	-13.04%	0.230	0.200	-	0.00%	-

Performance Improvements - Score	Δ	Previous	Current	σ	Δ (B)	σ (B)
nb_execution_successful	357.14%	7.000	32.000	-	966.67%	-
nb_compile_successful	100.00%	16.000	32.000	-	100.00%	-
nb_total_tests	100.00%	16.000	32.000	-	100.00%	-

Added Tests - Code Size
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - Execution
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - sample variability
500.perlbench_r
500.perlbench_r (avg)
500.perlbench_r (avg) (percentage)
500.perlbench_r (max)
500.perlbench_r (max) (percentage)
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
511.povray_r
511.povray_r (avg)
511.povray_r (avg) (percentage)
511.povray_r (max)
511.povray_r (max) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
526.blender_r
526.blender_r (avg)
526.blender_r (avg) (percentage)
526.blender_r (max)
526.blender_r (max) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
544.nab_r
544.nab_r (avg)
544.nab_r (avg) (percentage)
544.nab_r (max)
544.nab_r (max) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size	Δ (B)	Baseline	Current	σ (B)	Δ	σ
505.mcf_r	2.17%	25784.000	26344.000	-	2.20%	-

Performance Improvements - Execution	Δ (B)	Baseline	Current	σ (B)	Δ	σ
557.xz_r	-3.11%	10041.000	9729.000	-	-2.95%	-

Performance Improvements - Score	Δ (B)	Baseline	Current	σ (B)	Δ	σ
nb_execution_successful	966.67%	3.000	32.000	-	357.14%	-
nb_compile_successful	100.00%	16.000	32.000	-	100.00%	-
nb_total_tests	100.00%	16.000	32.000	-	100.00%	-

Added Tests - Code Size
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - Execution
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - sample variability
500.perlbench_r
500.perlbench_r (avg)
500.perlbench_r (avg) (percentage)
500.perlbench_r (max)
500.perlbench_r (max) (percentage)
505.mcf_r
505.mcf_r (avg)
505.mcf_r (avg) (percentage)
505.mcf_r (max)
505.mcf_r (max) (percentage)
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
511.povray_r
511.povray_r (avg)
511.povray_r (avg) (percentage)
511.povray_r (max)
511.povray_r (max) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
526.blender_r
526.blender_r (avg)
526.blender_r (avg) (percentage)
526.blender_r (max)
526.blender_r (max) (percentage)
531.deepsjeng_r
531.deepsjeng_r (avg)
531.deepsjeng_r (avg) (percentage)
531.deepsjeng_r (max)
531.deepsjeng_r (max) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
544.nab_r
544.nab_r (avg)
544.nab_r (avg) (percentage)
544.nab_r (max)
544.nab_r (max) (percentage)
557.xz_r
557.xz_r (avg)
557.xz_r (avg) (percentage)
557.xz_r (max)
557.xz_r (max) (percentage)

Report Time: 0.21s

Execution (seconds)

Name	Current	%
500.perlbench_r	15405.000	-
502.gcc_r	9054.000	-
505.mcf_r	11873.000	-
508.namd_r	9124.000	-
510.parest_r	20717.000	-
511.povray_r	16792.000	-
519.lbm_r	14776.000	-
520.omnetpp_r	11350.000	-
523.xalancbmk_r	8963.000	-
525.x264_r	9602.000	-
526.blender_r	10631.000	-
531.deepsjeng_r	11981.000	-
538.imagick_r	13986.000	-
541.leela_r	9795.000	-
544.nab_r	11118.000	-
557.xz_r	9729.000	-2.95%
Geometric Mean	11809.841	4.55%

Code Size (bytes)

Name	Current	%
500.perlbench_r	2284056.000	-
502.gcc_r	10175739.000	-
505.mcf_r	26344.000	2.20%
508.namd_r	506671.000	-
510.parest_r	1709360.000	-
511.povray_r	1141452.000	-
519.lbm_r	11038.000	-
520.omnetpp_r	1566949.000	-
523.xalancbmk_r	3504961.000	-
525.x264_r	527790.000	-
526.blender_r	11873266.000	-
531.deepsjeng_r	92131.000	-
538.imagick_r	1255819.000	-
541.leela_r	86292.000	-
544.nab_r	82184.000	-
557.xz_r	131735.000	-
Geometric Mean	515482.535	661.43%

Score

Name	Current	%
nb_compile_successful	32.000	100.00%
nb_execution_successful	32.000	357.14%
nb_total_tests	32.000	100.00%
Geometric Mean	32.000	163.45%

sample variability

Name	Current	%
500.perlbench_r	20.000	-
500.perlbench_r (avg)	20.000	-
500.perlbench_r (avg) (percentage)	0.130	-
500.perlbench_r (max)	18.000	-
500.perlbench_r (max) (percentage)	0.120	-
505.mcf_r	23.000	-14.81%
505.mcf_r (avg)	23.000	-14.81%
505.mcf_r (avg) (percentage)	0.200	-13.04%
505.mcf_r (max)	36.000	-20.00%
505.mcf_r (max) (percentage)	0.310	-18.42%
508.namd_r	9.000	-
508.namd_r (avg)	9.000	-
508.namd_r (avg) (percentage)	0.100	-
508.namd_r (max)	8.000	-
508.namd_r (max) (percentage)	0.090	-
511.povray_r	48.000	-
511.povray_r (avg)	48.000	-
511.povray_r (avg) (percentage)	0.290	-
511.povray_r (max)	43.000	-
511.povray_r (max) (percentage)	0.260	-
519.lbm_r	5.000	-
519.lbm_r (avg)	5.000	-
519.lbm_r (avg) (percentage)	0.040	-
519.lbm_r (max)	10.000	-
519.lbm_r (max) (percentage)	0.070	-
520.omnetpp_r	99.000	-
520.omnetpp_r (avg)	99.000	-
520.omnetpp_r (avg) (percentage)	0.880	-
520.omnetpp_r (max)	101.000	-
520.omnetpp_r (max) (percentage)	0.890	-
525.x264_r	30.000	-
525.x264_r (avg)	30.000	-
525.x264_r (avg) (percentage)	0.320	-
525.x264_r (max)	28.000	-
525.x264_r (max) (percentage)	0.300	-
526.blender_r	13.000	-
526.blender_r (avg)	13.000	-
526.blender_r (avg) (percentage)	0.130	-
526.blender_r (max)	23.000	-
526.blender_r (max) (percentage)	0.220	-
531.deepsjeng_r	37.000	-30.19%
531.deepsjeng_r (avg)	37.000	-30.19%
531.deepsjeng_r (avg) (percentage)	0.310	-29.55%
531.deepsjeng_r (max)	23.000	-68.92%
531.deepsjeng_r (max) (percentage)	0.200	-67.74%
541.leela_r	16.000	-
541.leela_r (avg)	16.000	-
541.leela_r (avg) (percentage)	0.170	-
541.leela_r (max)	14.000	-
541.leela_r (max) (percentage)	0.150	-
544.nab_r	41.000	-
544.nab_r (avg)	41.000	-
544.nab_r (avg) (percentage)	0.370	-
544.nab_r (max)	45.000	-
544.nab_r (max) (percentage)	0.410	-
557.xz_r	10.000	25.00%
557.xz_r (avg)	10.000	25.00%
557.xz_r (avg) (percentage)	0.110	37.50%
557.xz_r (max)	13.000	8.33%
557.xz_r (max) (percentage)	0.140	16.67%
Geometric Mean	3.355	-16.74%

Show Delta
Show Previous Value
Show Standard Deviation
Show Median Absolute Deviation
Show All Values
Show All Samples
Show Sample Counts
Show Small Differences
Number of Comparison Runs
Show Report Graphs
Show Data Table
Hide Report By Default
Test Filter (regexp)
Test Min. Value Filter
Aggregation Function
Mann-Whitney test confidence level

Runs:

Compare To:

Fields

Parameters

Fields

Parameters

llvm-arm-master-O2_LTO test results

Tests Summary

Run-Over-Run Changes Detail

Run-Over-Baseline Changes Detail