From patchwork Mon Jan 12 22:53:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1487 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258466; bh=4+gPYgEoF3dCqipvCulVuxFSBMp1EIxKpDiXlRY8qb8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=j7qd/aj/No+arg6EAyQMCp68YsUmtZpxJSiokQtGjc8QZLOYJ21wSPAJw4SyBqmRP 3WSrfaito2HUjyd/m3s/MxZ2LT7nsraO5VgeZB9ngujM1mgsnWroiHamfcNr03dD67 1XV6Nekpgak09d5VwGMB3CGKtUB82wyoANXKXhFwS0lxZED+Z9g6gKJ/AaUHR8F73O D+otAjIF0bS5sV7noWy7Eu6m1KOC56eGiGNQhJYb8slydO+vPGevNqKbM0zqqwnQt4 vC8SHCSIBlP+Mr2Iqg+IYFzFsBfGeJENuGLvRk63VEtWLGV+iVHL4BxHNalzXN3rxP myWFQy4W0antA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id C40AF692F6 for ; Mon, 12 Jan 2026 15:54:26 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id jdFLUBvlP0rX for ; Mon, 12 Jan 2026 15:54:26 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258466; bh=4+gPYgEoF3dCqipvCulVuxFSBMp1EIxKpDiXlRY8qb8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=j7qd/aj/No+arg6EAyQMCp68YsUmtZpxJSiokQtGjc8QZLOYJ21wSPAJw4SyBqmRP 3WSrfaito2HUjyd/m3s/MxZ2LT7nsraO5VgeZB9ngujM1mgsnWroiHamfcNr03dD67 1XV6Nekpgak09d5VwGMB3CGKtUB82wyoANXKXhFwS0lxZED+Z9g6gKJ/AaUHR8F73O D+otAjIF0bS5sV7noWy7Eu6m1KOC56eGiGNQhJYb8slydO+vPGevNqKbM0zqqwnQt4 vC8SHCSIBlP+Mr2Iqg+IYFzFsBfGeJENuGLvRk63VEtWLGV+iVHL4BxHNalzXN3rxP myWFQy4W0antA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id B275F69237 for ; Mon, 12 Jan 2026 15:54:26 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258463; bh=14bbgHozudvc6p98ARZX1QVR04eKkWzAPyimunQpHTM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=apTArMvTXxNv+03wk+WVbJh8PBqcsF8OKPv2Qci6V9hTFLVvwqmPtGZ/W0XDdzL80 ClRHgrGPHVAPOYEMzcxNZSpHzFg8+wyi7uDSK3mDhhsAvoq+0NE3ch/reAt/b8SAeK whkcAUM9t4V927JtHiSdsz99ONrmiVXVR8+D/AR5W05tWNQIuXLP4b1e6ppK9jRekm GJYWVg/MXnqrfYDiy6EktZHRD/ddr2FHReBEibaG3bAfVTFlzfSRP8hF5ff8XdxuMA 659GORkpO/MOmuTna8Cvu+mmwJYWmEig8wP3VDosrnZ89TtatyrSSZtF4ktgQIxdbj 29jqoK2mPHCmQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A496769237; Mon, 12 Jan 2026 15:54:23 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id l1T3U0UEN630; Mon, 12 Jan 2026 15:54:23 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258459; bh=2eIKNgV+ZCUjysGk8VeKK03Kz+gbbXehoAYkhFFG8vE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lxSEereEssfXR3EVHY1u61aZiYSLjhc/cAgmRD0WzFT4MpC/sauNyvz3SXMugObOS gQ8WjwFn3nnXD8F4gLo3MDpyLFqaq7fOkYo21E7OOC6nQHgmZ1xekBb/65WWWBT1Gz VWSqV6QnWE0KWo8Q3rqkR7xC2RCi+ICP6zqnWMSpsn4rWV5LG4PiB6AgRZTYpEscPE cTrGvpQYbQj+bhaPtT/eZfjhSvPR+lSk0FXJlDzFjb0OYxbgMiKYfK3VWRsfogU5DA PopicHm5+RCgrff5Sc2yfK7pnM1H5nLZKasASU93J1qQRzh4vrFmCYB4vZ/Ac3rDab 4/yF9XUmxRIrQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 55ABD6913F; Mon, 12 Jan 2026 15:54:19 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:53:59 -0700 Message-ID: <20260112225406.3274105-2-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: W5BXKVMVQRGVSISW7T76VBDQLWNT7KMC X-Message-ID-Hash: W5BXKVMVQRGVSISW7T76VBDQLWNT7KMC X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 1/5] buildman: Update test_reproducible for real Kconfig resolution List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass We plan to change how config adjustments are handled in buildman, so update this test to use a use single board. This will avoid a race condition with parallel builds using merge_config.sh race conditions when multiple builds try to compile Kconfig tools simultaneously. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/func_test.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 4a207bfb00c..ec2b2efc3b6 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -1100,7 +1100,8 @@ Idx Name Size VMA LMA File off Algn def test_reproducible(self): """Test that the -r flag works""" - lines, cfg_data = self.check_command('-r') + # Use single board to avoid parallel merge_config.sh race conditions + lines, cfg_data = self.check_command('board0', '-r') self.assertIn(b'SOURCE_DATE_EPOCH=0', lines[0]) # We should see CONFIG_LOCALVERSION_AUTO unset From patchwork Mon Jan 12 22:54:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1488 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258470; bh=KpcY4BBiZZyQSYnK8jVcMgyTPwuVIsjVP4eJ/OAYyrQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=toosMIxAMLu0G/tsCp8MDWVn3DNxqpm0s+IiW/A5O4hxGH4za9/Cv3qoivPrD++Rt 8r8B8GcvqvX3yihf/4AqgxMvmXOMH3f+7t0fDySQU/wCVB+elm9QNILjclHt9vafbx rTKtQVWle+t7KrQURWbJUXjTNwHkH0lrayD7WRZYbndFnz41Dc+Oj8yMy/s4KEGYBt 90lcy1qhvWoeVloQ7pLc90iEzRDNxzYKQQr+tOzIdJufQTDFiqolpunUjaAXCPYEcr iFIKWkR0j1nhANSpZLWJ5K2dzUdovIKDKaOBOSqWVkloyINcf8oKcvHJK1DK2Kllue piOzThGlMm1RA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4BAE9692FE for ; Mon, 12 Jan 2026 15:54:30 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id RLfrP1nu3U1a for ; Mon, 12 Jan 2026 15:54:30 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258470; bh=KpcY4BBiZZyQSYnK8jVcMgyTPwuVIsjVP4eJ/OAYyrQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=toosMIxAMLu0G/tsCp8MDWVn3DNxqpm0s+IiW/A5O4hxGH4za9/Cv3qoivPrD++Rt 8r8B8GcvqvX3yihf/4AqgxMvmXOMH3f+7t0fDySQU/wCVB+elm9QNILjclHt9vafbx rTKtQVWle+t7KrQURWbJUXjTNwHkH0lrayD7WRZYbndFnz41Dc+Oj8yMy/s4KEGYBt 90lcy1qhvWoeVloQ7pLc90iEzRDNxzYKQQr+tOzIdJufQTDFiqolpunUjaAXCPYEcr iFIKWkR0j1nhANSpZLWJ5K2dzUdovIKDKaOBOSqWVkloyINcf8oKcvHJK1DK2Kllue piOzThGlMm1RA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3B16A692F5 for ; Mon, 12 Jan 2026 15:54:30 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258469; bh=Gp7QWkzvb1oC6MrZkbYboyslyX2VI0F0kUE9G+Jwqfo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DN9B2Rssd/JTQwcI/40YZbqkO4iZTlU9ohuhy6lQY7X/QuwX7BXMYOZTFWhccTQp+ 0IvwShLqxIJyNgOreNEwb2m4+2v91iesabE3rVAUvUrB4yXGGO3um4NOcQQgnPPuDI rJou7qsSZg1uc8uh9vxhRJZlike6bAq5AZanOKUjo3SEXWQQukCDJyoqSicwCDpEWs EM24olh8I353hW5/H+DmMAygvtupjkP+94DV1Z1CsMtPBpyGykvG4Qp82ec7Kx2bo/ vsXlle45JeIF9U+E7YvRQjx+sgHGK0FrB62nWMSflRyTdYLiburhpYcOEBk6++Lnum XhSWpSkjQSxjA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 34F53692F5; Mon, 12 Jan 2026 15:54:29 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id rx2QCIjuX7G1; Mon, 12 Jan 2026 15:54:29 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258464; bh=g+L+l5+iWfhzNK2Q1ZnN9xsFjPAYlphJ7r+PcaoWVAU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g/5UoQC5Mrc2FnYspa6GqAcS3+H4/7+Hc3KgfxUDZRbTJ09z4GfMUWnLUWWcPlugy tA+1CsBURzM1I1QoV7dgYg6mxiUQIwnLlJ4zTRdP7JCaoaQ8TwlFv+G6yqfgXufbQ2 7f9UJ60Kd4UwXl/ZyJnlauFHz0ZfCsm0B7WYs92O7ZaPUIkDqS7JWENbJ2UEh0FHNi QXye8xEfZtKgVFcvEqZjVar8H3NbjM1DqSFkxo4CohTSvJLkru5nl8EfSD70tNT/Sy pVrwW8IoHxpvmVcWnDTqG/EafOUwuTJPGMy/1NdCl32Crxx73zSeVdsk2wK19WKu7M xTysYHRLiknoQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id E9F1C6913F; Mon, 12 Jan 2026 15:54:23 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:54:00 -0700 Message-ID: <20260112225406.3274105-3-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: CCVEWT6LOUYCWZ6DTSEWEVM2L67CHQ67 X-Message-ID-Hash: CCVEWT6LOUYCWZ6DTSEWEVM2L67CHQ67 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 2/5] buildman: Add a board parameter to check_command() List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The check_command() helper function hardcodes 'board0' for the output directory path. Add a board parameter to make the function more explicit about which board is being tested. Update all callers to pass 'board0' as the first argument. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/func_test.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index ec2b2efc3b6..7f4c88701bb 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -1048,23 +1048,26 @@ Idx Name Size VMA LMA File off Algn self.assertEqual(False, control.get_allow_missing(False, True, 2, True)) - def check_command(self, *extra_args): + def check_command(self, brd, *extra_args): """Run a command with the extra arguments and return the commands used Args: + brd (str): Board name to build extra_args (list of str): List of extra arguments Returns: - list of str: Lines returned in the out-cmd file + tuple: + list of str: Lines returned in the out-cmd file + bytes: Contents of .config file """ - self._run_control('-o', self._output_dir, *extra_args) - board0_dir = os.path.join(self._output_dir, 'current', 'board0') - self.assertTrue(os.path.exists(os.path.join(board0_dir, 'done'))) - cmd_fname = os.path.join(board0_dir, 'out-cmd') + self._run_control('-o', self._output_dir, brd, *extra_args) + board_dir = os.path.join(self._output_dir, 'current', brd) + self.assertTrue(os.path.exists(os.path.join(board_dir, 'done'))) + cmd_fname = os.path.join(board_dir, 'out-cmd') self.assertTrue(os.path.exists(cmd_fname)) data = tools.read_file(cmd_fname) - config_fname = os.path.join(board0_dir, '.config') + config_fname = os.path.join(board_dir, '.config') self.assertTrue(os.path.exists(config_fname)) cfg_data = tools.read_file(config_fname) @@ -1072,14 +1075,14 @@ Idx Name Size VMA LMA File off Algn def test_cmd_file(self): """Test that the -cmd-out file is produced""" - lines = self.check_command()[0] + lines = self.check_command('board0')[0] self.assertEqual(2, len(lines)) self.assertRegex(lines[0], b'make O=/.*board0_defconfig') self.assertRegex(lines[0], b'make O=/.*-s.*') def test_no_lto(self): """Test that the --no-lto flag works""" - lines = self.check_command('-L')[0] + lines = self.check_command('board0', '-L')[0] self.assertIn(b'NO_LTO=1', lines[0]) def test_fragments(self): @@ -1110,7 +1113,8 @@ Idx Name Size VMA LMA File off Algn ''', cfg_data) with terminal.capture() as (stdout, _stderr): - lines, cfg_data = self.check_command('-r', '-a', 'LOCALVERSION') + lines, cfg_data = self.check_command('board0', '-r', '-a', + 'LOCALVERSION') self.assertIn(b'SOURCE_DATE_EPOCH=0', lines[0]) # We should see CONFIG_LOCALVERSION_AUTO unset @@ -1456,7 +1460,7 @@ CONFIG_SOC="fred" def test_target(self): """Test that the --target flag works""" - lines = self.check_command('--target', 'u-boot.dtb')[0] + lines = self.check_command('board0', '--target', 'u-boot.dtb')[0] # It should not affect the defconfig line self.assertNotIn(b'u-boot.dtb', lines[0]) From patchwork Mon Jan 12 22:54:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1489 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258474; bh=FXXXH0o5MU1Q315MNdq8mdGtnjBcqpYmoH6hkTIhIGY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=KpsmikgKfABewWiQOpKYLP+6NNuv1MrA4t7bEm1vkh1jvKtWqZeUGI2U7rmSc+8/A JVyTZkyRErHB0uDy2nuy5u6ZjEXUpwzEf2taPRgnxjTBwp/iNoxph2oaZyTBKw9w+a a2dLCnLPn/pDBIBiCZ6R7MnHLgyjixWLJgVPLGHbBCTTjVoAkSTrADerOwgxI37M6w bq6PtxsJ9vIlWMCbQlBx9Xhplpwowfey3xBRIjVzLkd2Siaj/4l3sWvNvhDLxOl/au QtobpHL0vMMMJxs+vEXb0HsJRZ07d0BbxljHkrrqnIyxR1QuBZs7nGTmL+RC+qO3La NnEq8HnOdtrpA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id B9C286913F for ; Mon, 12 Jan 2026 15:54:34 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id OxXXIf_59f9k for ; Mon, 12 Jan 2026 15:54:34 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258474; bh=FXXXH0o5MU1Q315MNdq8mdGtnjBcqpYmoH6hkTIhIGY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=KpsmikgKfABewWiQOpKYLP+6NNuv1MrA4t7bEm1vkh1jvKtWqZeUGI2U7rmSc+8/A JVyTZkyRErHB0uDy2nuy5u6ZjEXUpwzEf2taPRgnxjTBwp/iNoxph2oaZyTBKw9w+a a2dLCnLPn/pDBIBiCZ6R7MnHLgyjixWLJgVPLGHbBCTTjVoAkSTrADerOwgxI37M6w bq6PtxsJ9vIlWMCbQlBx9Xhplpwowfey3xBRIjVzLkd2Siaj/4l3sWvNvhDLxOl/au QtobpHL0vMMMJxs+vEXb0HsJRZ07d0BbxljHkrrqnIyxR1QuBZs7nGTmL+RC+qO3La NnEq8HnOdtrpA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A7C15692F5 for ; Mon, 12 Jan 2026 15:54:34 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258473; bh=L8DiI9g2wqIwrdU24AG79urA0aZL9m0pMEEu5Uq21nI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wj7T913jg6mNPZDzk54WFkcK6VG1AMJvgDFyOgmUdZuo4qyhBeH/gUdYg0WupxPEB ox2BO3xiBE9ntnfEfs4B3/MEs8igjSefOCmLlwI++QBUwLMWOxRge4Sqq20J54XFnF E0QNHlztXNEEdyxmIbnBTl+RyVXmuhsN+KQcM8IfKCpcUZrh6NwfmY4C0xJgOWDYLv xBYmqXbcj2U73YF00LOP2U5j60jrhAupUJPqWJX0WMgefWT76zRF0WY0IT5L6qLo0/ 2MS/Wj+gzzA9hY4Mrg6ol6FCXl1/bn9ShS/2lYrXHYlI+mjZW/aTPmOFrYq8XScuxC 4VQPnynF/saKQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1AA4B6913F; Mon, 12 Jan 2026 15:54:33 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id SCoRbCHABURm; Mon, 12 Jan 2026 15:54:33 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258468; bh=jWUPUPj0C61+LmEbCgpLn+ICED3Asy6HBsejp8oQT+Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rSk/EfkpK7qW5aJtUag+hLqAc/Iwm0DmSf/a+UZwizzp4aNTtqsxovoeT3q80P1c5 5Cxcj7+P8ExazZaPGD3xfjAb6pqA9VALaNpGOA09iKTvhKZebL+ABbiNefP3N/j3OR 9YuDqIM5Soh9IOTDVi0XT3zErXN744JIxgtWz8Q6qxfcG8S1lSTbOsTIX8PUHAUwO3 BX335cjhJuXgey4MEfIDZfLw8XGiAO5rSuXHCMEun9Cmp3pYo7TVJejABMBOQTnlQq 1A9FUm2xvuZoBHlt7Z2VNXyR7Nsw1E6Sz6NDSOAjbairmL1YHdRLGN6pAnnekB9w3K vXbljc6xd6pTQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 82BF569237; Mon, 12 Jan 2026 15:54:28 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:54:01 -0700 Message-ID: <20260112225406.3274105-4-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: KHOHEX3VTOQB23EJSM4AJCNNX6TVVUS7 X-Message-ID-Hash: KHOHEX3VTOQB23EJSM4AJCNNX6TVVUS7 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 3/5] buildman: Use merge_config.sh for --adjust-cfg List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The --adjust-cfg option currently modifies .config directly via cfgutil.adjust_cfg_file(), which does not resolve Kconfig dependencies. This means 'imply' and 'select' relationships are not honoured. For example, enabling CONFIG_UNIT_TEST via '-a UNIT_TEST' does not enable CONFIG_CONSOLE_RECORD even though UNIT_TEST implies it. Fix this by using scripts/kconfig/merge_config.sh to apply config changes. This script merges config fragments into .config and runs 'make alldefconfig' to resolve all Kconfig dependencies. To properly resolve 'imply' relationships, use 'make savedefconfig' to create a minimal defconfig as the base for merge_config.sh. The full .config contains '# CONFIG_xxx is not set' lines which Kconfig treats as "specified", preventing imply from taking effect. Using the minimal defconfig ensures only explicitly set options are 'locked', allowing imply to work correctly. Add adjust_cfg_to_fragment() and run_merge_config() functions to cfgutil to convert the adjust_cfg dictionary into a config-fragment format and handle the merge process. Add BUILDMAN_TEST_A and BUILDMAN_TEST_B options to test/Kconfig with an imply relationship for testing. Update func_test.py to test that the imply mechanism works correctly with --adjust-cfg. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- test/Kconfig | 14 ++++ tools/buildman/builderthread.py | 12 ++- tools/buildman/cfgutil.py | 88 +++++++++++++++++++++ tools/buildman/func_test.py | 136 ++++++++++++++++++++++++++++++-- tools/buildman/test_cfgutil.py | 31 ++++++++ 5 files changed, 266 insertions(+), 15 deletions(-) diff --git a/test/Kconfig b/test/Kconfig index 96723940bac..cd1eec169e8 100644 --- a/test/Kconfig +++ b/test/Kconfig @@ -122,6 +122,20 @@ source "test/lib/Kconfig" source "test/optee/Kconfig" source "test/fdt_overlay/Kconfig" +config BUILDMAN_TEST_A + bool "Buildman test option A" + imply BUILDMAN_TEST_B + help + Test option used by buildman to verify that --adjust-cfg correctly + resolves Kconfig 'imply' dependencies via merge_config.sh. + +config BUILDMAN_TEST_B + bool "Buildman test option B" + help + Test option implied by BUILDMAN_TEST_A. Used by buildman tests to + verify that 'imply' dependencies are resolved when using + merge_config.sh. + endif # UNIT_TEST config POST diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index dee31972e4b..dcf2d8f9ac5 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -516,7 +516,11 @@ class BuilderThread(threading.Thread): setup.config_args, config_out, cmd_list, mrproper) do_config = False # No need to configure next time if req.adjust_cfg: - cfgutil.adjust_cfg_file(cfg_file, req.adjust_cfg) + merge_result = cfgutil.run_merge_config( + setup.cwd, out_dir, cfg_file, req.adjust_cfg, setup.env) + config_out.write(merge_result.combined) + if merge_result.return_code: + result = merge_result return result, do_config, cfg_file def _build_and_get_result(self, req, setup, commit, cmd_list, config_out, @@ -537,12 +541,6 @@ class BuilderThread(threading.Thread): CommandResult: Result of the build """ if result.return_code == 0: - if req.adjust_cfg: - oldc_args = list(setup.args) + ['oldconfig'] - oldc_result = self.make(commit, req.brd, 'oldconfig', setup.cwd, - *oldc_args, env=setup.env) - if oldc_result.return_code: - return oldc_result result = self._build(commit, req.brd, setup.cwd, setup.args, setup.env, cmd_list, config_only) if req.adjust_cfg: diff --git a/tools/buildman/cfgutil.py b/tools/buildman/cfgutil.py index 2d4c6799b5c..cec33e1e62b 100644 --- a/tools/buildman/cfgutil.py +++ b/tools/buildman/cfgutil.py @@ -7,7 +7,9 @@ import os import re +import tempfile +from u_boot_pylib import command from u_boot_pylib import tools @@ -306,3 +308,89 @@ def process_config(fname, squash_config_y): value = '1' config[key] = value return config + + +def adjust_cfg_to_fragment(adjust_cfg): + """Convert adjust_cfg dict to config fragment content + + Args: + adjust_cfg (dict): Changes to make to .config file. Keys are config + names (without CONFIG_ prefix), values are the setting. Format + matches make_cfg_line(): + ~... - disable the option + ...=val - set the option to val (val contains full assignment) + other - enable the option with =y + + Returns: + str: Config fragment content suitable for merge_config.sh + """ + lines = [] + for opt, val in adjust_cfg.items(): + if val.startswith('~'): + lines.append(f'# CONFIG_{opt} is not set') + elif '=' in val: + lines.append(f'CONFIG_{val}') + else: + lines.append(f'CONFIG_{opt}=y') + return '\n'.join(lines) + '\n' if lines else '' + + +def run_merge_config(src_dir, out_dir, cfg_file, adjust_cfg, env): + """Run merge_config.sh to apply config changes with Kconfig resolution + + This uses scripts/kconfig/merge_config.sh to merge config fragments + into the .config file, then runs 'make alldefconfig' to resolve all + Kconfig dependencies including 'imply' and 'select'. + + To properly resolve 'imply' relationships, we must use a minimal + defconfig as the base (not the full .config). The full .config contains + '# CONFIG_xxx is not set' lines which count as "specified" and prevent + imply from taking effect. Using savedefconfig output ensures only + explicitly set options are in the base, allowing imply to work. + + Args: + src_dir (str): Source directory (containing scripts/kconfig) + out_dir (str): Output directory containing .config + cfg_file (str): Path to the .config file + adjust_cfg (dict): Config changes to apply + env (dict): Environment variables + + Returns: + CommandResult: Result of the merge_config.sh operation + """ + # Create a temporary fragment file with the config changes + fragment_content = adjust_cfg_to_fragment(adjust_cfg) + with tempfile.NamedTemporaryFile(mode='w', suffix='.config', + delete=False) as frag: + frag.write(fragment_content) + frag_path = frag.name + + # Create a minimal defconfig from the current .config + # This is necessary for 'imply' to work - the full .config has + # '# CONFIG_xxx is not set' lines that prevent imply from taking effect + defconfig_path = os.path.join(out_dir or '.', 'defconfig') + make_cmd = ['make', f'O={out_dir}' if out_dir else None, + f'KCONFIG_CONFIG={cfg_file}', 'savedefconfig'] + make_cmd = [x for x in make_cmd if x] # Remove None elements + result = command.run_one(*make_cmd, cwd=src_dir, env=env, capture=True, + capture_stderr=True) + if result.return_code: + if os.path.exists(frag_path): + os.unlink(frag_path) + return result + + try: + # Run merge_config.sh with the minimal defconfig as base + # -O sets output dir; defconfig is the base, fragment is merged + merge_script = os.path.join(src_dir or '.', 'scripts', 'kconfig', + 'merge_config.sh') + out = out_dir or '.' + cmd = [merge_script, '-O', out, defconfig_path, frag_path] + result = command.run_one(*cmd, cwd=src_dir, env=env, capture=True, + capture_stderr=True) + finally: + # Clean up temporary files + if os.path.exists(frag_path): + os.unlink(frag_path) + + return result diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 7f4c88701bb..7db4c086207 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -25,6 +25,7 @@ from buildman import boards from buildman.boards import Extended from buildman import bsettings from buildman import builderthread +from buildman import cfgutil from buildman import cmdline from buildman import control from buildman import toolchain @@ -446,6 +447,73 @@ Idx Name Size VMA LMA File off Algn print(line, file=buf) return command.CommandResult(stdout=buf.getvalue(), return_code=0) + def _run_merge_config(self, cmd_list, _kwargs): + """Run the real merge_config.sh script + + Runs merge_config.sh from the real U-Boot source tree with real + Kconfig files for actual dependency resolution. + + Args: + cmd_list (list): Original command and arguments + + Returns: + CommandResult: Result of running the script + """ + # Run from the real U-Boot source tree (not the test's fake git dir) + src_root = os.path.dirname(os.path.dirname(self._buildman_dir)) + merge_script = os.path.join(src_root, 'scripts', 'kconfig', + 'merge_config.sh') + + # Build command with real script path, keeping original arguments + new_cmd = [merge_script] + list(cmd_list[1:]) + + # Use a clean host environment with CROSS_COMPILE cleared so Kconfig + # uses the host gcc + env = dict(os.environ) + env['CROSS_COMPILE'] = '' + + # Temporarily disable TEST_RESULT to run the real command + old_test_result = command.TEST_RESULT + command.TEST_RESULT = None + try: + result = command.run_one(*new_cmd, cwd=src_root, env=env, + capture=True, capture_stderr=True) + finally: + command.TEST_RESULT = old_test_result + return result + + def _handle_make_savedefconfig(self, args, _kwargs): + """Handle make savedefconfig command + + This runs the real make savedefconfig to create a minimal defconfig + from the current .config file. + + Args: + args (list): Arguments to make (after 'make') + + Returns: + CommandResult: Result of running the command + """ + # Run from the U-Boot source tree + src_root = os.path.dirname(os.path.dirname(self._buildman_dir)) + + # Build the full command + new_cmd = ['make'] + list(args) + + # Use a clean host environment with CROSS_COMPILE cleared + env = dict(os.environ) + env['CROSS_COMPILE'] = '' + + # Temporarily disable TEST_RESULT to run the real command + old_test_result = command.TEST_RESULT + command.TEST_RESULT = None + try: + result = command.run_one(*new_cmd, cwd=src_root, env=env, + capture=True, capture_stderr=True) + finally: + command.TEST_RESULT = old_test_result + return result + def _handle_command(self, **kwargs): # pylint: disable=too-many-branches """Handle a command execution. @@ -482,6 +550,12 @@ Idx Name Size VMA LMA File off Algn result = self._handle_command_cpp(args) elif cmd == 'gcc' and args[0] == '-E': result = self._handle_command_cpp(args[1:]) + elif cmd.endswith('merge_config.sh'): + # Run the real merge_config.sh using command.run_one() + result = self._run_merge_config(pipe_list[0], kwargs) + elif cmd == 'make' and 'savedefconfig' in args: + # Handle make savedefconfig - create minimal defconfig from .config + result = self._handle_make_savedefconfig(args, kwargs) else: # Not handled, so abort print('unknown command', kwargs) @@ -1107,22 +1181,20 @@ Idx Name Size VMA LMA File off Algn lines, cfg_data = self.check_command('board0', '-r') self.assertIn(b'SOURCE_DATE_EPOCH=0', lines[0]) - # We should see CONFIG_LOCALVERSION_AUTO unset - self.assertEqual(b'''CONFIG_SOMETHING=1 -# CONFIG_LOCALVERSION_AUTO is not set -''', cfg_data) + # We should see CONFIG_LOCALVERSION_AUTO unset (uses real Kconfig) + self.assertIn(b'# CONFIG_LOCALVERSION_AUTO is not set', cfg_data) with terminal.capture() as (stdout, _stderr): lines, cfg_data = self.check_command('board0', '-r', '-a', 'LOCALVERSION') self.assertIn(b'SOURCE_DATE_EPOCH=0', lines[0]) - # We should see CONFIG_LOCALVERSION_AUTO unset - self.assertEqual(b'''CONFIG_SOMETHING=1 -CONFIG_LOCALVERSION=y -''', cfg_data) + # When user explicitly sets LOCALVERSION, the warning appears self.assertIn('Not dropping LOCALVERSION_AUTO', stdout.getvalue()) + # LOCALVERSION should be present in .config (it's a string config) + self.assertIn(b'CONFIG_LOCALVERSION=', cfg_data) + def test_scan_defconfigs(self): """Test scanning the defconfigs to obtain all the boards""" src = self._git_dir @@ -1619,3 +1691,51 @@ something: me # No reconfigs should be triggered self.assertEqual(0, self._builder.kconfig_reconfig) + + def test_adjust_cfg_no_imply(self): + """Test that direct .config modification does not resolve imply + + Modifying .config directly with cfgutil.adjust_cfg_file() does not + run Kconfig, so 'imply' dependencies are not resolved. This test + demonstrates the limitation that merge_config.sh fixes. + """ + # Create a temporary .config file + cfg_file = os.path.join(self._output_dir, '.config') + tools.write_file(cfg_file, b'# Empty config\n') + + # Use cfgutil to directly modify .config (old approach) + adjust_cfg = {'BUILDMAN_TEST_A': 'BUILDMAN_TEST_A'} + cfgutil.adjust_cfg_file(cfg_file, adjust_cfg) + + # Read the result + cfg_data = tools.read_file(cfg_file) + + # BUILDMAN_TEST_A should be enabled + self.assertIn(b'CONFIG_BUILDMAN_TEST_A=y', cfg_data) + + # But BUILDMAN_TEST_B should NOT be enabled - imply is not resolved + # because we didn't run Kconfig + self.assertNotIn(b'CONFIG_BUILDMAN_TEST_B=y', cfg_data, + 'Direct .config modification should not resolve imply') + + def test_adjust_cfg_imply(self): + """Test that merge_config.sh resolves Kconfig 'imply' dependencies + + The --adjust-cfg option uses merge_config.sh to apply config changes, + which runs 'make alldefconfig' to resolve all Kconfig dependencies + including 'imply'. CONFIG_BUILDMAN_TEST_A implies + CONFIG_BUILDMAN_TEST_B, so enabling BUILDMAN_TEST_A should also enable + BUILDMAN_TEST_B. + """ + # Use single board to avoid parallel merge_config.sh race conditions + # Enable UNIT_TEST since BUILDMAN_TEST_A depends on it + _lines, cfg_data = self.check_command( + 'board0', '-a', 'UNIT_TEST,BUILDMAN_TEST_A') + + # Verify BUILDMAN_TEST_A was enabled + self.assertIn(b'CONFIG_BUILDMAN_TEST_A=y', cfg_data) + + # merge_config.sh resolves imply dependencies, so enabling + # BUILDMAN_TEST_A should also enable BUILDMAN_TEST_B + self.assertIn(b'CONFIG_BUILDMAN_TEST_B=y', cfg_data, + '--adjust-cfg should resolve imply dependencies') diff --git a/tools/buildman/test_cfgutil.py b/tools/buildman/test_cfgutil.py index ba3f0468570..47e522d3d6c 100644 --- a/tools/buildman/test_cfgutil.py +++ b/tools/buildman/test_cfgutil.py @@ -115,6 +115,37 @@ class TestAdjustCfg(unittest.TestCase): '~CONFIG_ABE,CONFIG_MARK=0x456', 'CONFIG_ANNA="anna"']) self.assertEqual(expect, actual) + def test_adjust_cfg_to_fragment(self): + """Test adjust_cfg_to_fragment creates correct fragment content""" + # Empty dict returns empty string + self.assertEqual('', cfgutil.adjust_cfg_to_fragment({})) + + # Enable option + self.assertEqual('CONFIG_FRED=y\n', + cfgutil.adjust_cfg_to_fragment({'FRED': 'FRED'})) + + # Disable option + self.assertEqual('# CONFIG_FRED is not set\n', + cfgutil.adjust_cfg_to_fragment({'FRED': '~FRED'})) + + # Set value + self.assertEqual('CONFIG_FRED=0x123\n', + cfgutil.adjust_cfg_to_fragment({'FRED': 'FRED=0x123'})) + + # Set string value + self.assertEqual('CONFIG_FRED="fred"\n', + cfgutil.adjust_cfg_to_fragment({'FRED': 'FRED="fred"'})) + + # Multiple options (note: dict order is preserved in Python 3.7+) + result = cfgutil.adjust_cfg_to_fragment({ + 'FRED': 'FRED', + 'MARY': '~MARY', + 'JOHN': 'JOHN=42' + }) + self.assertIn('CONFIG_FRED=y', result) + self.assertIn('# CONFIG_MARY is not set', result) + self.assertIn('CONFIG_JOHN=42', result) + def test_check_cfg_file(self): """Test check_cfg_file detects conflicts as expected""" # Check failure to disable CONFIG From patchwork Mon Jan 12 22:54:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1490 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258479; bh=j20/Y/k4Esx45h7474Zm750ovzjjLopJxhRJcJj9a3s=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=DqR5yXx7J7By5K0LZiZ2Gc+EGThBMZK5F39YZOQGKarNjpQ3TcF2qOiGYNuoaTTtf H4rmrw12q3PlRWNwg9VvKY03+YnAJoeVF9P3SQbyXNdxty+psgz7uf2rd6+8i0/KTg 3m+Gm2nmdTgj6rO4PeX5nrOSvXu6P+9dtBfeZfIU3Ea0KyVZ1BSJ5nqGXHST/EuYeo zEfPvVs7gX4f+F8Sqpsx9A3mxEcjBqnGwnUKN+1Xp50chUvCom3OCWGIi4kpQLqh8E p8U0YIK6ukKo0qnIjdkLGTfMUOwz6Fyp/qlnZ6A4i00vbDQQOFRiLqe2QIRDZUmSrm t9x99qBJtHigg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2FAB1692FE for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id IgAyEBDykehA for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258479; bh=j20/Y/k4Esx45h7474Zm750ovzjjLopJxhRJcJj9a3s=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=DqR5yXx7J7By5K0LZiZ2Gc+EGThBMZK5F39YZOQGKarNjpQ3TcF2qOiGYNuoaTTtf H4rmrw12q3PlRWNwg9VvKY03+YnAJoeVF9P3SQbyXNdxty+psgz7uf2rd6+8i0/KTg 3m+Gm2nmdTgj6rO4PeX5nrOSvXu6P+9dtBfeZfIU3Ea0KyVZ1BSJ5nqGXHST/EuYeo zEfPvVs7gX4f+F8Sqpsx9A3mxEcjBqnGwnUKN+1Xp50chUvCom3OCWGIi4kpQLqh8E p8U0YIK6ukKo0qnIjdkLGTfMUOwz6Fyp/qlnZ6A4i00vbDQQOFRiLqe2QIRDZUmSrm t9x99qBJtHigg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1F900692F5 for ; Mon, 12 Jan 2026 15:54:39 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258477; bh=WpLy+FtRQeNrtVxqZ+vOQh9gnw1u+ShPXGyRZInGAko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k6cnKOM5KU8kVHJHvXsSzcDSGEYfVWecoI0YfdCd5nNe9ljCpWCNtAHbKw/HLntUk MFuQfEfSPuN0iw8sKSxkCAbP3Jz5VpESiqNIzOxZfELY3n8PkvJqvu3T6phEXSVypG 6lS51kKLPjIILd6yD9BR7M0vu38ZrXocZSiD+GyYPDdYMYJBrdhdhx6ozb89MnduXR uovC5vSTcPA5W0kajY5BaZZF51zpWT6Wa+2PbuxkJWWYCbyyQ0LpOMtDlVOhZ29Hv+ 27FQlAymDDIUp81Ov/bpwQihE9OaYxq1xU9wqcnunQMSODIixOpq1yM4e5ONoYz+78 P9uUr6jlOZKNQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id D71F36913F; Mon, 12 Jan 2026 15:54:37 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id vDdsuKvdj-U4; Mon, 12 Jan 2026 15:54:37 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258473; bh=q3Y0wfaiSR01Wg7UdI1TosGAHgHiEyo7nzGRMR6nMww=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CglvBIDLxPCw47q945hbIDhPq7GtstA+URdVrocMD4ycx8I2MbOzYqtA7saZnfScc tl7pe6qAeahqSaWShoFIu4w22AJqkeeoCuBOwGXPTOJLvFgRK5FzHimr+EKfQJbZNi S/vE4AeZHzFTldMjeg7rL7/RKz5ech8eMZPJVIi/6M3CB0nOkq8FxUEuG9DGoQmdPZ afSi8VJMSZPJkdHwSyuYr3E8lu2P/BRQD7cOYIX90ro1hCzNIQK14PJgTxEmbRt8t3 xEC5CJ8d7AM705gL7kDt5ThRXNCAmCXIW7/vaFUenJYSvUXtqnxkvyNHZqKe5MmfrC puG3hYkFA30sw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 60C1569237; Mon, 12 Jan 2026 15:54:33 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:54:02 -0700 Message-ID: <20260112225406.3274105-5-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: RFGCHVJALAZ7V656KTR7GVCLVRROAS7F X-Message-ID-Hash: RFGCHVJALAZ7V656KTR7GVCLVRROAS7F X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 4/5] buildman: Prioritise downloaded toolchains over system ones List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass When a toolchain is downloaded via --fetch-arch for an architecture that matches the host (e.g. aarch64 on aarch64), the system-installed gcc from a distribution package may be selected instead of the downloaded one. This happens because both toolchains have the same calculated priority and the system one is scanned first. Add a new PRIORITY_DOWNLOADED level (priority 3) that sits between PRIORITY_PREFIX_GCC_PATH (2) and PRIORITY_CALC (4+). Track which paths come from the 'download' key in the [toolchain] config section and use the higher priority when scanning those paths. The priority hierarchy is now: 0: Explicit [toolchain-prefix] path exists as a file 1: [toolchain-prefix] path + 'gcc' exists as a file 2: [toolchain-prefix] path + 'gcc' found in PATH 3: Downloaded toolchains (from --fetch-arch) 4+: Toolchains from [toolchain] paths (calculated from filename) Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/buildman.rst | 29 +++++++++++++++++++++---- tools/buildman/test.py | 43 +++++++++++++++++++++++++++++++++++++ tools/buildman/toolchain.py | 24 +++++++++++++++++++-- 3 files changed, 90 insertions(+), 6 deletions(-) diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index 379a6ab48ce..603b019540b 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -474,7 +474,10 @@ Setting up sudo mkdir -p /toolchains sudo mv ~/.buildman-toolchains/*/* /toolchains/ - Buildman should now be set up to use your new toolchain. + Buildman should now be set up to use your new toolchain. Downloaded + toolchains are given priority over system-installed toolchains, so if you + have both a downloaded toolchain and one installed via your + distribution's package manager, the downloaded one will be used. At the time of writing, U-Boot has these architectures: @@ -938,13 +941,31 @@ a set of (tag, value) pairs. '[toolchain-prefix]' section This can be used to provide the full toolchain-prefix for one or more - architectures. The full CROSS_COMPILE prefix must be provided. These - typically have a higher priority than matches in the '[toolchain]', due to - this prefix. + architectures. The full CROSS_COMPILE prefix must be provided. The tilde character ``~`` is supported in paths, to represent the home directory. +Toolchain priority + When multiple toolchains are available for an architecture, buildman + selects the one with the highest priority (lowest priority number). + + Note: Lower numbers indicate higher priority, so a toolchain with + priority 3 is preferred over one with priority 6. + + The priority levels are: + + - 0: Full prefix path from '[toolchain-prefix]' that exists as a file + - 1: Prefix from '[toolchain-prefix]' with 'gcc' appended that exists + - 2: Prefix from '[toolchain-prefix]' found in PATH + - 3: Downloaded toolchains (from ``--fetch-arch``) + - 4+: Toolchains found by scanning '[toolchain]' paths (priority + calculated from filename, e.g. '-linux' variants get priority 6) + + This means that downloaded toolchains are preferred over system-installed + toolchains (e.g. from a distribution package), but explicit + '[toolchain-prefix]' entries take the highest priority. + '[toolchain-alias]' section This converts toolchain architecture names to U-Boot names. For example, if an x86 toolchains is called i386-linux-gcc it will not normally be diff --git a/tools/buildman/test.py b/tools/buildman/test.py index b217b907176..da6df1f173c 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -701,6 +701,49 @@ class TestBuild(TestBuildBase): 'crosstool/files/bin/x86_64/.*/' 'x86_64-gcc-.*-nolibc[-_]arm-.*linux-gnueabi.tar.xz') + def test_toolchain_download_priority(self): + """Test that downloaded toolchains have priority over system ones""" + # Create a temp directory structure with two toolchains for same arch + with tempfile.TemporaryDirectory() as tmpdir: + # Create 'system' toolchain path (simulating /usr/bin) + system_path = os.path.join(tmpdir, 'system') + os.makedirs(os.path.join(system_path, 'bin')) + system_gcc = os.path.join(system_path, 'bin', 'aarch64-linux-gcc') + tools.write_file(system_gcc, b'#!/bin/sh\necho gcc') + os.chmod(system_gcc, 0o755) + + # Create 'download' toolchain path + download_path = os.path.join(tmpdir, 'download') + os.makedirs(os.path.join(download_path, 'bin')) + download_gcc = os.path.join(download_path, 'bin', + 'aarch64-linux-gcc') + tools.write_file(download_gcc, b'#!/bin/sh\necho gcc') + os.chmod(download_gcc, 0o755) + + # Check system toolchain priority (not in download_paths) + sys_tc = toolchain.Toolchain(system_gcc, test=False) + self.assertEqual(toolchain.PRIORITY_CALC + 2, sys_tc.priority) + + # Set up toolchains with download path tracked + tcs = toolchain.Toolchains() + tcs.paths = [system_path, download_path] + tcs.download_paths = {download_path} + + # Scan and check which toolchain is selected + with terminal.capture(): + tcs.scan(False, raise_on_error=False) + + # The downloaded toolchain should be selected + tc = tcs.toolchains.get('aarch64') + self.assertIsNotNone(tc) + self.assertTrue(tc.gcc.startswith(download_path), + f"Expected downloaded toolchain from {download_path}, " + f"got {tc.gcc}") + self.assertEqual(toolchain.PRIORITY_DOWNLOADED, tc.priority) + + # Verify downloaded priority beats system priority + self.assertLess(toolchain.PRIORITY_DOWNLOADED, sys_tc.priority) + def test_get_env_args(self): """Test the GetEnvArgs() function""" tc = self.toolchains.select('arm') diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 8ec1dbdebba..8f3d3ab3b0c 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -17,9 +17,17 @@ from u_boot_pylib import command from u_boot_pylib import terminal from u_boot_pylib import tools +# Toolchain priority levels (lower number = higher priority): +# PRIORITY_FULL_PREFIX: Explicit [toolchain-prefix] path exists as a file +# PRIORITY_PREFIX_GCC: [toolchain-prefix] path + 'gcc' exists as a file +# PRIORITY_PREFIX_GCC_PATH: [toolchain-prefix] path + 'gcc' found in PATH +# PRIORITY_DOWNLOADED: Toolchain downloaded via --fetch-arch +# PRIORITY_CALC: Toolchain found by scanning [toolchain] paths; actual +# priority is PRIORITY_CALC + offset based on toolchain name (PRIORITY_FULL_PREFIX, PRIORITY_PREFIX_GCC, PRIORITY_PREFIX_GCC_PATH, - PRIORITY_CALC) = list(range(4)) + PRIORITY_DOWNLOADED, PRIORITY_CALC) = list(range(5)) +# Environment variable / argument types for get_env_args() (VAR_CROSS_COMPILE, VAR_PATH, VAR_ARCH, VAR_MAKE_ARGS) = range(4) class MyHTMLParser(HTMLParser): @@ -290,6 +298,7 @@ class Toolchains: self.toolchains = {} self.prefixes = {} self.paths = [] + self.download_paths = set() self.override_toolchain = override_toolchain self._make_flags = dict(bsettings.get_items('make-flags')) @@ -330,6 +339,15 @@ class Toolchains: self.prefixes = bsettings.get_items('toolchain-prefix') self.paths += self.get_path_list(show_warning) + # Track which paths are from downloaded toolchains + for name, value in bsettings.get_items('toolchain'): + if name == 'download': + fname = os.path.expanduser(value) + if '*' in value: + self.download_paths.update(glob.glob(fname)) + else: + self.download_paths.add(fname) + # pylint: disable=too-many-arguments,too-many-positional-arguments def add(self, fname, test=True, verbose=False, priority=PRIORITY_CALC, arch=None): @@ -435,8 +453,10 @@ class Toolchains: if verbose: print(f" - scanning path '{path}'") fnames = self.scan_path(path, verbose) + priority = (PRIORITY_DOWNLOADED if path in self.download_paths + else PRIORITY_CALC) for fname in fnames: - self.add(fname, True, verbose) + self.add(fname, True, verbose, priority) def list(self): """List out the selected toolchains for each architecture""" From patchwork Mon Jan 12 22:54:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1491 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258484; bh=Hs0xOZb2GottTBOhFVPfOdlN7r4B6OCE3LQnkLB9QoY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=bmz294T4Mvf034bU1DFgOBSHX5WGaFK57X9Ti2RaQDBDDCY/z8wn3c2oCuOnp5qf1 kX/K+0yzJuntY7hHTsTsexr1yvbh1I2Zs55bwsSyTLRwAMyL+S+WLGL7+ngUGe1ugL /bRzEfGo1/ZJlAtOYCmsxWVXc65rzVq7S9vF0PiLgwHhWTJC5nlcFEynR4eDmFNYw2 0BwhrYgwmbNSZ/h6vodnVJi6P+ZINjBY8OE1YjnR1JBA1EWY9uX+AgQqLyY+4EPF9t 0VPFbaM6rTARI7eG5CLupWklGhtJn5CGoutMJ9x9spmvUxYw3YTC6dk4dB4MsPphJm 6ZwdpRTUXSXOg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DF1AF692F6 for ; Mon, 12 Jan 2026 15:54:44 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id GYxeIBiLXqy5 for ; Mon, 12 Jan 2026 15:54:44 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258484; bh=Hs0xOZb2GottTBOhFVPfOdlN7r4B6OCE3LQnkLB9QoY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=bmz294T4Mvf034bU1DFgOBSHX5WGaFK57X9Ti2RaQDBDDCY/z8wn3c2oCuOnp5qf1 kX/K+0yzJuntY7hHTsTsexr1yvbh1I2Zs55bwsSyTLRwAMyL+S+WLGL7+ngUGe1ugL /bRzEfGo1/ZJlAtOYCmsxWVXc65rzVq7S9vF0PiLgwHhWTJC5nlcFEynR4eDmFNYw2 0BwhrYgwmbNSZ/h6vodnVJi6P+ZINjBY8OE1YjnR1JBA1EWY9uX+AgQqLyY+4EPF9t 0VPFbaM6rTARI7eG5CLupWklGhtJn5CGoutMJ9x9spmvUxYw3YTC6dk4dB4MsPphJm 6ZwdpRTUXSXOg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CB09469237 for ; Mon, 12 Jan 2026 15:54:44 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258482; bh=s66eiR1aGNbav54h937OQdWmcl/x972m/+glP+MEtoY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o/45x4qucu0qzIV5d4p2mjZf4mFWjT7MVfVP/8jtehXJlPVwM4k3kKEwvY8J0WwCu W9LtAeDFJ9lmy4KJU35T35xW7aLy4PXDwdPAV7+YFpfi/VOUZFKhfxGfJwB42OTrAu 8mkoT9r2lGD6RmdDFr8aCU99gM0TSv8SyW1GBCeKOgW1EiTdpJ8Jpefc3RIDRmEFkB BBtfW3jKwlLODjvzaYpfh7X45lLD02fPu5O/RKXgxWmRcVFKI8DtFYN1orLP2APoWu 3pDQO01TXLxBDfPOmyk5OsaEcX1XpIRgHstT3uWJ5otmJlUPxcVilZzSxvy04kpb8m 53ek6Ezcqjqhw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 9AF7A6913F; Mon, 12 Jan 2026 15:54:42 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 46LPVMyGUdKu; Mon, 12 Jan 2026 15:54:42 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1768258478; bh=RM18GXjvKWRonkZb1W8uV5YXhrc+TNnZO3tj/wQFbV4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WhPyG/kcd82fBIanqq6V7t+tnEcOa6zQnGDVq2UXoalmUbm0WvA7Hhoxd5hxuJeWw 6vxhX0kTgmVlhp3j+6kl4ItfLcy++FQ1Cs/coK0uXSCyxIJ+zrUsvJ1XvecHfrf0fI y97duPxZX1D+3B99Xhh2CC1osdIa2B7Gp2oxyYO2iZIb53vQohaqE0AA0tpwfW25qB LSxCIBVmy2TPxkS9fNGrtQ1a7EvUQZgaOraU/PP6YGzZsiQgepCQz5294lx74vW9rh YDyU/Z80egiQHL2mmRYvJIfNIgtgDki49F1XwgEbwPnURyTVncdzVZDBLCBxKmpjXN nIXjPkvD9Jshg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 2872269237; Mon, 12 Jan 2026 15:54:38 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 12 Jan 2026 15:54:03 -0700 Message-ID: <20260112225406.3274105-6-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260112225406.3274105-1-sjg@u-boot.org> References: <20260112225406.3274105-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: ZPX42QNPARKAMIXLH7QL3DUTF3SVQD6J X-Message-ID-Hash: ZPX42QNPARKAMIXLH7QL3DUTF3SVQD6J X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 5/5] buildman: Filter out doubled-prefix toolchain binaries List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Some toolchain tarballs from kernel.org contain symlinks with a doubled cross-compile prefix, e.g. 'x86_64-linux-x86_64-linux-gcc' alongside the correct 'x86_64-linux-gcc'. This causes buildman to print a warning about ambiguous toolchains when downloading. Add a regex-based check to detect and filter out these malformed binaries during toolchain scanning. When verbose output is enabled, these files are shown as "ignoring ... (doubled prefix)" rather than "found ..." Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/test.py | 24 ++++++++++++++++++++++++ tools/buildman/toolchain.py | 25 +++++++++++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index da6df1f173c..0f4a5b9e543 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -744,6 +744,30 @@ class TestBuild(TestBuildBase): # Verify downloaded priority beats system priority self.assertLess(toolchain.PRIORITY_DOWNLOADED, sys_tc.priority) + def test_is_doubled_prefix(self): + """Test detection of doubled toolchain prefixes""" + # Valid toolchain names (not doubled) + self.assertFalse( + toolchain.Toolchains.is_doubled_prefix('aarch64-linux-gcc')) + self.assertFalse( + toolchain.Toolchains.is_doubled_prefix('x86_64-linux-gcc')) + self.assertFalse( + toolchain.Toolchains.is_doubled_prefix('arm-linux-gnueabi-gcc')) + self.assertFalse( + toolchain.Toolchains.is_doubled_prefix('gcc')) + + # Doubled prefixes (should be filtered out) + self.assertTrue( + toolchain.Toolchains.is_doubled_prefix( + 'aarch64-linux-aarch64-linux-gcc')) + self.assertTrue( + toolchain.Toolchains.is_doubled_prefix( + 'x86_64-linux-x86_64-linux-gcc')) + + # Not a gcc file + self.assertFalse( + toolchain.Toolchains.is_doubled_prefix('aarch64-linux-ld')) + def test_get_env_args(self): """Test the GetEnvArgs() function""" tc = self.toolchains.select('arm') diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 8f3d3ab3b0c..27302f20d42 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -30,6 +30,10 @@ from u_boot_pylib import tools # Environment variable / argument types for get_env_args() (VAR_CROSS_COMPILE, VAR_PATH, VAR_ARCH, VAR_MAKE_ARGS) = range(4) +# Matches a repeated prefix, e.g. 'aarch64-linux-aarch64-linux-gcc' +RE_DOUBLED_PREFIX = re.compile(r'^(.+)\1gcc$') + + class MyHTMLParser(HTMLParser): """Simple class to collect links from a page @@ -378,6 +382,22 @@ class Toolchains: f"toolchain for arch '{toolchain.arch}' has priority " f"{self.toolchains[toolchain.arch].priority}") + @staticmethod + def is_doubled_prefix(fname): + """Check if a gcc filename has a doubled prefix + + Some toolchain tarballs contain symlinks with the cross-compile prefix + repeated, e.g. 'x86_64-linux-x86_64-linux-gcc'. These are not valid + toolchains and should be ignored. + + Args: + fname (str): Filename to check (basename, not full path) + + Returns: + bool: True if the prefix is doubled, False otherwise + """ + return bool(RE_DOUBLED_PREFIX.match(fname)) + def scan_path(self, path, verbose): """Scan a path for a valid toolchain @@ -394,6 +414,11 @@ class Toolchains: if verbose: print(f" - looking in '{dirname}'") for fname in glob.glob(dirname + '/*gcc'): + basename = os.path.basename(fname) + if self.is_doubled_prefix(basename): + if verbose: + print(f" - ignoring '{fname}' (doubled prefix)") + continue if verbose: print(f" - found '{fname}'") fnames.append(fname)