Machine Info

Fields

hardware	None
os	None

Parameters

Run Info

Fields

Name	Value

Parameters

Name	Value
git_binutils	https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=328e01595436e937a0cc00cc8451f17beda26799 (+ 28 commits)
git_gcc	https://github.com/gcc-mirror/gcc/commit/2d44ab221f64f01fc676be0da1a6774740d713c6 (+ 46 commits)
git_gdb	https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=a0a97b8cac56ea498bfcfa0afbbe943bf79b7894 (unchanged)
git_newlib	https://sourceware.org/git/?p=newlib-cygwin.git;a=commit;h=7863c07a9282b7aed358c5783976296adfa63505 (unchanged)
git_qemu	https://gitlab.com/qemu-project/qemu/commit/a3c3aaa846ad61b801e7196482dcf4afb8ba34e4 (unchanged)
test_url	https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-arm_eabi-build/453/

Report

Filter

master-arm_eabi test results

Run	Order	Start Time	Duration
Current	0453 (2-changed-components)	2023-11-09T10:28:23	0:00:00
Previous	0362 (basepoints/gcc-14-03631-gd3dd69706af)	2023-09-01T06:35:24	0:00:00
Baseline	0011 (5-changed-components)	2023-06-05T17:27:31	0:00:00

Tests Summary

Status Group	#	# (B)
Performance Regressions	11	12
Performance Improvements	18	17
Added Tests	19	19
Unchanged Tests	897	897
Total Tests	945

Run-Over-Run Changes Detail

Performance Regressions - FAIL	Δ	Previous	Current	σ	Δ (B)	σ (B)
g++.FAIL	164.86%	37.000	98.000	-	-48.15%	-

Performance Regressions - UNSUPPORTED	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.UNSUPPORTED	5.40%	1000.000	1054.000	-	61.41%	-
gcc.UNSUPPORTED	4.93%	8268.000	8676.000	-	5.09%	-
TOTAL.UNSUPPORTED	2.74%	20485.000	21047.000	-	0.36%	-

Performance Regressions - XFAIL	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.XFAIL	773.33%	15.000	131.000	-	670.59%	-
g++.XFAIL	27.28%	1195.000	1521.000	-	44.17%	-
TOTAL.XFAIL	22.40%	2129.000	2606.000	-	29.46%	-
gcc.XFAIL	3.81%	919.000	954.000	-	3.81%	-

Performance Regressions - XPASS	Δ	Previous	Current	σ	Δ (B)	σ (B)
g++.XPASS	130.00%	10.000	23.000	-	130.00%	-
TOTAL.XPASS	72.22%	18.000	31.000	-	72.22%	-

Performance Regressions - bad	Δ	Previous	Current	σ	Δ (B)	σ (B)
g++.bad	133.96%	53.000	124.000	-	-64.37%	-

Performance Improvements - FAIL	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.FAIL	-93.25%	163.000	11.000	-	-97.61%	-
TOTAL.FAIL	-37.25%	247.000	155.000	-	-99.17%	-
gcc.FAIL	-2.13%	47.000	46.000	-	-40.26%	-

Performance Improvements - PASS	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.PASS	15.43%	13321.000	15377.000	-	12.78%	-
g++.PASS	2.91%	202313.000	208197.000	-	4.89%	-
TOTAL.PASS	2.64%	366556.000	376221.000	-	-1.60%	-
gcc.PASS	1.14%	150922.000	152647.000	-	2.43%	-

Performance Improvements - UNRESOLVED	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.UNRESOLVED	-98.16%	163.000	3.000	-	-99.98%	-
g++.UNRESOLVED	-50.00%	6.000	3.000	-	-97.99%	-

Performance Improvements - all	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.all	2.69%	389598.000	400063.000	-	-9.56%	-

Performance Improvements - bad	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.bad	-95.94%	320.000	13.000	-	-97.95%	-
TOTAL.bad	-55.84%	428.000	189.000	-	-99.49%	-
gcc.bad	-5.45%	55.000	52.000	-	-74.26%	-

Performance Improvements - fails	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.fails	-55.63%	426.000	189.000	-	-99.46%	-

Performance Improvements - good	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.good	15.53%	14336.000	16562.000	-	15.79%	-
g++.good	2.94%	214725.000	221035.000	-	4.88%	-
TOTAL.good	2.75%	389170.000	399874.000	-	-1.34%	-
gcc.good	1.35%	160109.000	162277.000	-	2.58%	-

Added Tests - ERROR
TOTAL.ERROR
g++.ERROR
gcc.ERROR
libstdc++.ERROR

Added Tests - KFAIL
TOTAL.KFAIL
g++.KFAIL
gcc.KFAIL
libstdc++.KFAIL

Added Tests - OTHER
TOTAL.OTHER
g++.OTHER
gcc.OTHER
libstdc++.OTHER

Added Tests - UNRESOLVED
gcc.UNRESOLVED
libstdc++.UNRESOLVED

Added Tests - UNTESTED
TOTAL.UNTESTED
g++.UNTESTED
gcc.UNTESTED
libstdc++.UNTESTED

Added Tests - flaky
TOTAL.flaky

Run-Over-Baseline Changes Detail

Performance Regressions - PASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.PASS	-1.60%	382336.000	376221.000	-	2.64%	-

Performance Regressions - UNSUPPORTED	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.UNSUPPORTED	61.41%	653.000	1054.000	-	5.40%	-
gcc.UNSUPPORTED	5.09%	8256.000	8676.000	-	4.93%	-
g++.UNSUPPORTED	1.12%	11192.000	11317.000	-	0.89%	-

Performance Regressions - XFAIL	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.XFAIL	670.59%	17.000	131.000	-	773.33%	-
g++.XFAIL	44.17%	1055.000	1521.000	-	27.28%	-
TOTAL.XFAIL	29.46%	2013.000	2606.000	-	22.40%	-
gcc.XFAIL	3.81%	919.000	954.000	-	3.81%	-

Performance Regressions - XPASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
g++.XPASS	130.00%	10.000	23.000	-	130.00%	-
TOTAL.XPASS	72.22%	18.000	31.000	-	72.22%	-

Performance Regressions - all	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.all	-9.56%	442374.000	400063.000	-	2.69%	-

Performance Regressions - good	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.good	-1.34%	405320.000	399874.000	-	2.75%	-

Performance Improvements - FAIL	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.FAIL	-99.17%	18705.000	155.000	-	-37.25%	-
libstdc++.FAIL	-97.61%	460.000	11.000	-	-93.25%	-
g++.FAIL	-48.15%	189.000	98.000	-	164.86%	-
gcc.FAIL	-40.26%	77.000	46.000	-	-2.13%	-

Performance Improvements - PASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.PASS	12.78%	13634.000	15377.000	-	15.43%	-
g++.PASS	4.89%	198496.000	208197.000	-	2.91%	-
gcc.PASS	2.43%	149024.000	152647.000	-	1.14%	-

Performance Improvements - UNRESOLVED	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.UNRESOLVED	-99.98%	16578.000	3.000	-	-98.16%	-
g++.UNRESOLVED	-97.99%	149.000	3.000	-	-50.00%	-

Performance Improvements - bad	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.bad	-99.49%	37054.000	189.000	-	-55.84%	-
libstdc++.bad	-97.95%	634.000	13.000	-	-95.94%	-
gcc.bad	-74.26%	202.000	52.000	-	-5.45%	-
g++.bad	-64.37%	348.000	124.000	-	133.96%	-

Performance Improvements - fails	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.fails	-99.46%	35217.000	189.000	-	-55.63%	-

Performance Improvements - good	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.good	15.79%	14304.000	16562.000	-	15.53%	-
g++.good	4.88%	210743.000	221035.000	-	2.94%	-
gcc.good	2.58%	158199.000	162277.000	-	1.35%	-

Added Tests - ERROR
TOTAL.ERROR
g++.ERROR
gcc.ERROR
libstdc++.ERROR

Added Tests - KFAIL
TOTAL.KFAIL
g++.KFAIL
gcc.KFAIL
libstdc++.KFAIL

Added Tests - OTHER
TOTAL.OTHER
g++.OTHER
gcc.OTHER
libstdc++.OTHER

Added Tests - UNRESOLVED
gcc.UNRESOLVED
libstdc++.UNRESOLVED

Added Tests - UNTESTED
TOTAL.UNTESTED
g++.UNTESTED
gcc.UNTESTED
libstdc++.UNTESTED

Added Tests - flaky
TOTAL.flaky

Report Time: 0.21s

PASS

	Name	Current	%
	TOTAL.PASS	376221.000	2.64%
	g++.PASS	208197.000	2.91%
	gcc.PASS	152647.000	1.14%
	libstdc++.PASS	15377.000	15.43%
	Geometric Mean	116444.598	5.38%

XPASS

	Name	Current	%
	TOTAL.XPASS	31.000	72.22%
	g++.XPASS	23.000	130.00%
	gcc.XPASS	6.000	-
	libstdc++.XPASS	2.000	-
	Geometric Mean	9.618	41.08%

FAIL

	Name	Current	%
	TOTAL.FAIL	155.000	-37.25%
	g++.FAIL	98.000	164.86%
	gcc.FAIL	46.000	-2.13%
	libstdc++.FAIL	11.000	-93.25%
	Geometric Mean	52.654	-42.44%

XFAIL

	Name	Current	%
	TOTAL.XFAIL	2606.000	22.40%
	g++.XFAIL	1521.000	27.28%
	gcc.XFAIL	954.000	3.81%
	libstdc++.XFAIL	131.000	773.33%
	Geometric Mean	838.940	93.86%

KFAIL

	Name	Current	%
	TOTAL.KFAIL	0.000	-
	g++.KFAIL	0.000	-
	gcc.KFAIL	0.000	-
	libstdc++.KFAIL	0.000	-
	Geometric Mean	0.000	-

UNSUPPORTED

	Name	Current	%
	TOTAL.UNSUPPORTED	21047.000	2.74%
	g++.UNSUPPORTED	11317.000	-
	gcc.UNSUPPORTED	8676.000	4.93%
	libstdc++.UNSUPPORTED	1054.000	5.40%
	Geometric Mean	6831.568	3.48%

UNRESOLVED

	Name	Current	%
	TOTAL.UNRESOLVED	3.000	-98.16%
	g++.UNRESOLVED	3.000	-50.00%
	gcc.UNRESOLVED	0.000	-
	libstdc++.UNRESOLVED	0.000	-
	Geometric Mean	0.017	-99.94%

UNTESTED

	Name	Current	%
	TOTAL.UNTESTED	0.000	-
	g++.UNTESTED	0.000	-
	gcc.UNTESTED	0.000	-
	libstdc++.UNTESTED	0.000	-
	Geometric Mean	0.000	-

ERROR

	Name	Current	%
	TOTAL.ERROR	0.000	-
	g++.ERROR	0.000	-
	gcc.ERROR	0.000	-
	libstdc++.ERROR	0.000	-
	Geometric Mean	0.000	-

OTHER

	Name	Current	%
	TOTAL.OTHER	0.000	-
	g++.OTHER	0.000	-
	gcc.OTHER	0.000	-
	libstdc++.OTHER	0.000	-
	Geometric Mean	0.000	-

good

	Name	Current	%
	TOTAL.good	399874.000	2.75%
	g++.good	221035.000	2.94%
	gcc.good	162277.000	1.35%
	libstdc++.good	16562.000	15.53%
	Geometric Mean	124147.647	5.49%

bad

	Name	Current	%
	TOTAL.bad	189.000	-55.84%
	g++.bad	124.000	133.96%
	gcc.bad	52.000	-5.45%
	libstdc++.bad	13.000	-95.94%
	Geometric Mean	63.090	-55.37%

fails

	Name	Current	%
	TOTAL.fails	189.000	-55.63%
	Geometric Mean	189.000	-55.63%

flaky

	Name	Current	%
	TOTAL.flaky	0.000	-
	Geometric Mean	0.000	-

all

	Name	Current	%
	TOTAL.all	400063.000	2.69%
	Geometric Mean	400063.000	2.69%