From patchwork Mon Jan 12 22:54:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1490 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258479; bh=j20/Y/k4Esx45h7474Zm750ovzjjLopJxhRJcJj9a3s=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=DqR5yXx7J7By5K0LZiZ2Gc+EGThBMZK5F39YZOQGKarNjpQ3TcF2qOiGYNuoaTTtf H4rmrw12q3PlRWNwg9VvKY03+YnAJoeVF9P3SQbyXNdxty+psgz7uf2rd6+8i0/KTg 3m+Gm2nmdTgj6rO4PeX5nrOSvXu6P+9dtBfeZfIU3Ea0KyVZ1BSJ5nqGXHST/EuYeo zEfPvVs7gX4f+F8Sqpsx9A3mxEcjBqnGwnUKN+1Xp50chUvCom3OCWGIi4kpQLqh8E p8U0YIK6ukKo0qnIjdkLGTfMUOwz6Fyp/qlnZ6A4i00vbDQQOFRiLqe2QIRDZUmSrm t9x99qBJtHigg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2FAB1692FE for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id IgAyEBDykehA for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258479; bh=j20/Y/k4Esx45h7474Zm750ovzjjLopJxhRJcJj9a3s=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=DqR5yXx7J7By5K0LZiZ2Gc+EGThBMZK5F39YZOQGKarNjpQ3TcF2qOiGYNuoaTTtf H4rmrw12q3PlRWNwg9VvKY03+YnAJoeVF9P3SQbyXNdxty+psgz7uf2rd6+8i0/KTg 3m+Gm2nmdTgj6rO4PeX5nrOSvXu6P+9dtBfeZfIU3Ea0KyVZ1BSJ5nqGXHST/EuYeo zEfPvVs7gX4f+F8Sqpsx9A3mxEcjBqnGwnUKN+1Xp50chUvCom3OCWGIi4kpQLqh8E p8U0YIK6ukKo0qnIjdkLGTfMUOwz6Fyp/qlnZ6A4i00vbDQQOFRiLqe2QIRDZUmSrm t9x99qBJtHigg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1F900692F5 for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258477; bh=WpLy+FtRQeNrtVxqZ+vOQh9gnw1u+ShPXGyRZInGAko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k6cnKOM5KU8kVHJHvXsSzcDSGEYfVWecoI0YfdCd5nNe9ljCpWCNtAHbKw/HLntUk MFuQfEfSPuN0iw8sKSxkCAbP3Jz5VpESiqNIzOxZfELY3n8PkvJqvu3T6phEXSVypG 6lS51kKLPjIILd6yD9BR7M0vu38ZrXocZSiD+GyYPDdYMYJBrdhdhx6ozb89MnduXR uovC5vSTcPA5W0kajY5BaZZF51zpWT6Wa+2PbuxkJWWYCbyyQ0LpOMtDlVOhZ29Hv+ 27FQlAymDDIUp81Ov/bpwQihE9OaYxq1xU9wqcnunQMSODIixOpq1yM4e5ONoYz+78 P9uUr6jlOZKNQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id D71F36913F; Mon, 12 Jan 2026 15:54:37 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id vDdsuKvdj-U4; Mon, 12 Jan 2026 15:54:37 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258473; bh=q3Y0wfaiSR01Wg7UdI1TosGAHgHiEyo7nzGRMR6nMww=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CglvBIDLxPCw47q945hbIDhPq7GtstA+URdVrocMD4ycx8I2MbOzYqtA7saZnfScc tl7pe6qAeahqSaWShoFIu4w22AJqkeeoCuBOwGXPTOJLvFgRK5FzHimr+EKfQJbZNi S/vE4AeZHzFTldMjeg7rL7/RKz5ech8eMZPJVIi/6M3CB0nOkq8FxUEuG9DGoQmdPZ afSi8VJMSZPJkdHwSyuYr3E8lu2P/BRQD7cOYIX90ro1hCzNIQK14PJgTxEmbRt8t3 xEC5CJ8d7AM705gL7kDt5ThRXNCAmCXIW7/vaFUenJYSvUXtqnxkvyNHZqKe5MmfrC puG3hYkFA30sw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 60C1569237; Mon, 12 Jan 2026 15:54:33 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:54:02 -0700 Message-ID: <20260112225406.3274105-5-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: RFGCHVJALAZ7V656KTR7GVCLVRROAS7F X-Message-ID-Hash: RFGCHVJALAZ7V656KTR7GVCLVRROAS7F X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 4/5] buildman: Prioritise downloaded toolchains over system ones List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass When a toolchain is downloaded via --fetch-arch for an architecture that matches the host (e.g. aarch64 on aarch64), the system-installed gcc from a distribution package may be selected instead of the downloaded one. This happens because both toolchains have the same calculated priority and the system one is scanned first. Add a new PRIORITY_DOWNLOADED level (priority 3) that sits between PRIORITY_PREFIX_GCC_PATH (2) and PRIORITY_CALC (4+). Track which paths come from the 'download' key in the [toolchain] config section and use the higher priority when scanning those paths. The priority hierarchy is now: 0: Explicit [toolchain-prefix] path exists as a file 1: [toolchain-prefix] path + 'gcc' exists as a file 2: [toolchain-prefix] path + 'gcc' found in PATH 3: Downloaded toolchains (from --fetch-arch) 4+: Toolchains from [toolchain] paths (calculated from filename) Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/buildman.rst | 29 +++++++++++++++++++++---- tools/buildman/test.py | 43 +++++++++++++++++++++++++++++++++++++ tools/buildman/toolchain.py | 24 +++++++++++++++++++-- 3 files changed, 90 insertions(+), 6 deletions(-) diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index 379a6ab48ce..603b019540b 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -474,7 +474,10 @@ Setting up sudo mkdir -p /toolchains sudo mv ~/.buildman-toolchains/*/* /toolchains/ - Buildman should now be set up to use your new toolchain. + Buildman should now be set up to use your new toolchain. Downloaded + toolchains are given priority over system-installed toolchains, so if you + have both a downloaded toolchain and one installed via your + distribution's package manager, the downloaded one will be used. At the time of writing, U-Boot has these architectures: @@ -938,13 +941,31 @@ a set of (tag, value) pairs. '[toolchain-prefix]' section This can be used to provide the full toolchain-prefix for one or more - architectures. The full CROSS_COMPILE prefix must be provided. These - typically have a higher priority than matches in the '[toolchain]', due to - this prefix. + architectures. The full CROSS_COMPILE prefix must be provided. The tilde character ``~`` is supported in paths, to represent the home directory. +Toolchain priority + When multiple toolchains are available for an architecture, buildman + selects the one with the highest priority (lowest priority number). + + Note: Lower numbers indicate higher priority, so a toolchain with + priority 3 is preferred over one with priority 6. + + The priority levels are: + + - 0: Full prefix path from '[toolchain-prefix]' that exists as a file + - 1: Prefix from '[toolchain-prefix]' with 'gcc' appended that exists + - 2: Prefix from '[toolchain-prefix]' found in PATH + - 3: Downloaded toolchains (from ``--fetch-arch``) + - 4+: Toolchains found by scanning '[toolchain]' paths (priority + calculated from filename, e.g. '-linux' variants get priority 6) + + This means that downloaded toolchains are preferred over system-installed + toolchains (e.g. from a distribution package), but explicit + '[toolchain-prefix]' entries take the highest priority. + '[toolchain-alias]' section This converts toolchain architecture names to U-Boot names. For example, if an x86 toolchains is called i386-linux-gcc it will not normally be diff --git a/tools/buildman/test.py b/tools/buildman/test.py index b217b907176..da6df1f173c 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -701,6 +701,49 @@ class TestBuild(TestBuildBase): 'crosstool/files/bin/x86_64/.*/' 'x86_64-gcc-.*-nolibc[-_]arm-.*linux-gnueabi.tar.xz') + def test_toolchain_download_priority(self): + """Test that downloaded toolchains have priority over system ones""" + # Create a temp directory structure with two toolchains for same arch + with tempfile.TemporaryDirectory() as tmpdir: + # Create 'system' toolchain path (simulating /usr/bin) + system_path = os.path.join(tmpdir, 'system') + os.makedirs(os.path.join(system_path, 'bin')) + system_gcc = os.path.join(system_path, 'bin', 'aarch64-linux-gcc') + tools.write_file(system_gcc, b'#!/bin/sh\necho gcc') + os.chmod(system_gcc, 0o755) + + # Create 'download' toolchain path + download_path = os.path.join(tmpdir, 'download') + os.makedirs(os.path.join(download_path, 'bin')) + download_gcc = os.path.join(download_path, 'bin', + 'aarch64-linux-gcc') + tools.write_file(download_gcc, b'#!/bin/sh\necho gcc') + os.chmod(download_gcc, 0o755) + + # Check system toolchain priority (not in download_paths) + sys_tc = toolchain.Toolchain(system_gcc, test=False) + self.assertEqual(toolchain.PRIORITY_CALC + 2, sys_tc.priority) + + # Set up toolchains with download path tracked + tcs = toolchain.Toolchains() + tcs.paths = [system_path, download_path] + tcs.download_paths = {download_path} + + # Scan and check which toolchain is selected + with terminal.capture(): + tcs.scan(False, raise_on_error=False) + + # The downloaded toolchain should be selected + tc = tcs.toolchains.get('aarch64') + self.assertIsNotNone(tc) + self.assertTrue(tc.gcc.startswith(download_path), + f"Expected downloaded toolchain from {download_path}, " + f"got {tc.gcc}") + self.assertEqual(toolchain.PRIORITY_DOWNLOADED, tc.priority) + + # Verify downloaded priority beats system priority + self.assertLess(toolchain.PRIORITY_DOWNLOADED, sys_tc.priority) + def test_get_env_args(self): """Test the GetEnvArgs() function""" tc = self.toolchains.select('arm') diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 8ec1dbdebba..8f3d3ab3b0c 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -17,9 +17,17 @@ from u_boot_pylib import command from u_boot_pylib import terminal from u_boot_pylib import tools +# Toolchain priority levels (lower number = higher priority): +# PRIORITY_FULL_PREFIX: Explicit [toolchain-prefix] path exists as a file +# PRIORITY_PREFIX_GCC: [toolchain-prefix] path + 'gcc' exists as a file +# PRIORITY_PREFIX_GCC_PATH: [toolchain-prefix] path + 'gcc' found in PATH +# PRIORITY_DOWNLOADED: Toolchain downloaded via --fetch-arch +# PRIORITY_CALC: Toolchain found by scanning [toolchain] paths; actual +# priority is PRIORITY_CALC + offset based on toolchain name (PRIORITY_FULL_PREFIX, PRIORITY_PREFIX_GCC, PRIORITY_PREFIX_GCC_PATH, - PRIORITY_CALC) = list(range(4)) + PRIORITY_DOWNLOADED, PRIORITY_CALC) = list(range(5)) +# Environment variable / argument types for get_env_args() (VAR_CROSS_COMPILE, VAR_PATH, VAR_ARCH, VAR_MAKE_ARGS) = range(4) class MyHTMLParser(HTMLParser): @@ -290,6 +298,7 @@ class Toolchains: self.toolchains = {} self.prefixes = {} self.paths = [] + self.download_paths = set() self.override_toolchain = override_toolchain self._make_flags = dict(bsettings.get_items('make-flags')) @@ -330,6 +339,15 @@ class Toolchains: self.prefixes = bsettings.get_items('toolchain-prefix') self.paths += self.get_path_list(show_warning) + # Track which paths are from downloaded toolchains + for name, value in bsettings.get_items('toolchain'): + if name == 'download': + fname = os.path.expanduser(value) + if '*' in value: + self.download_paths.update(glob.glob(fname)) + else: + self.download_paths.add(fname) + # pylint: disable=too-many-arguments,too-many-positional-arguments def add(self, fname, test=True, verbose=False, priority=PRIORITY_CALC, arch=None): @@ -435,8 +453,10 @@ class Toolchains: if verbose: print(f" - scanning path '{path}'") fnames = self.scan_path(path, verbose) + priority = (PRIORITY_DOWNLOADED if path in self.download_paths + else PRIORITY_CALC) for fname in fnames: - self.add(fname, True, verbose) + self.add(fname, True, verbose, priority) def list(self): """List out the selected toolchains for each architecture"""