From patchwork Tue Apr 7 12:26:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2152 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564841; bh=WWvqbUiEjaAJeDVynKI84GP+KgialskbGPwRewwIqvY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=wh7zY9ROW6J2ia7aHSuZPkDTw/2WJT/uIVeeArWWrzz/ZdBZNd5QcavU22EcBa/HG ptbYTiXg/sHK8P4z+o3Oz4k8ugAORV+gwUlPqpNJK3Dka1wVy3xx7S1DQT8IRJ5mp6 tLmB3wdzSy3VrgJZcdSVyA3p0QdclYQJcSHsZaU2d3VKvIXCC1uzbGffZ9DT4jT2x2 H/zn2f540OJOIAtXIqctq294NpAo+YLaobdurbbyJLYwbr7D1cPw18MeeL/fQpRyOw CeWY7a+bd36XjNet7UlhFPpIlPpLJ1oeyZi+6xA4G9JzSJjyg/hARD3xFs0VXu/rPm GQXjRHBaKojXQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 87BB06A39B for ; Tue, 7 Apr 2026 06:27:21 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id EbvnNpouXItS for ; Tue, 7 Apr 2026 06:27:21 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564840; bh=WWvqbUiEjaAJeDVynKI84GP+KgialskbGPwRewwIqvY=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=rNs/uX0xW0q+lFNuhgfDt2Ywsnjxu4wT3TMmG0gsOCDJ6ZW3hdpsX2dhXwDh41Hkg ZA+J1YCjxv0IZD0gIyKvHl7kjGBbq81iqSaBPlaYp4sSjCDA/HhZA3tWK/7tfF0Znq 18ZJQi5kGV5btFzBZMaDQGXGsjlTKMioqMwT4ES23iHvUiVc05wcWsrAfh5d3IM5yu yiUZAIAgViGBDtoUcFonsMLNbYFM855yYj65luzWJWXJ/uSp4o1j30STvJFuLafhqz oTKoabd/EYhmATyAQdO6CjE6i4B053tGYrn2aWxCVCx3nMG5FfqCxkOZPlhqJ9xRaS tW+9o8b8kWfFw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3FE0F6A39E for ; Tue, 7 Apr 2026 06:27:20 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564838; bh=J8U8OEA1RAq3CmlW4XC8R/7K3yri1fvx0og615zyu8Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HeffMdIFeOKBIkboE8i8TqM2iNJGqHHG2rDO17uAtDgpfIRmMnPYLjLH+fhLuDW1E 6og5VJv5eaw7BjL5ptpCxrHmlGcSvWeZUcpXMUSuwiSrQeNcuKwelxr6USILKGitfW b6743SBXFS7YJej0wsOHkoyFQxV/3DDEF1aIDdeEn4eG9t6G7PMq1eLsQWFT5wB0pL 8/eX2mClx0Eo8qZPpWIwyL/Wdf7FlxjNsI2hTb3/NjeL50A17Ksh+JSELvbc/5yESy fNiErPY5Paf43iuFTDApaxvJnkjXHbPm5nH3liAtSFxjdWc4V2z2pfI3FuwlzKGuAc MIkQeUuqdPPVA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5D34C6A39B; Tue, 7 Apr 2026 06:27:18 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id OgbrQjr_esOh; Tue, 7 Apr 2026 06:27:18 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564838; bh=NMmEVoT9XXuo4GwJauiyshDLTUhhnP9y0Xwg+GVHzos=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sLWOo1rHildf6s1MzP9zdiPtoLcmTVA/Gj/zF9xIZ/hNMdmwMYtNxVOEW3Mq37OcI fu46wz++Q7pcsZ2m4xNoetDhaTwevX/i8bU2zeqrGkzlfMbRG/vvd/WKTFFH8sw0zS dV0+cpa5b3pzkB93TrvorHE29o2314bsP18pq+jWYpH+hbtFxu9vCA8+4yutogsFHY YWstZRCXdtuHvTbwt1wi0Y53r19jzZbJ8HSOfz76LGBGSJa25OjzwJ5cYck4v+BDMR Wu+zzA1LC2bkZLjW1C+72kkCPZ8OdwM3oUxcMRYVUTkwmoVOlGfdmiZaD3FR04Z0sw eEudm4qICV8Ww== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id C960C6A386; Tue, 7 Apr 2026 06:27:17 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Tue, 7 Apr 2026 06:26:48 -0600 Message-ID: <20260407122656.3462730-2-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260407122656.3462730-1-sjg@u-boot.org> References: <20260407122656.3462730-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: AFRHTOQ4T5R43YVK2N4ZODGZWHRMGCAW X-Message-ID-Hash: AFRHTOQ4T5R43YVK2N4ZODGZWHRMGCAW X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 1/4] qconfig: Add helper functions for #include defconfig sync List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add three helper functions in preparation for fixing the #include defconfig sync: - _get_defconfig_entries(): parses a preprocessed defconfig to get a dict of config entries by name, for textual comparison against include file content - _build_include_defconfig(): assembles defconfig content from include lines and an overlay delta - _format_sym_value(): formats a kconfiglib symbol's current value as a defconfig line, handling bool/tristate/string/int/hex types No functional change. Signed-off-by: Simon Glass --- tools/qconfig.py | 84 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/tools/qconfig.py b/tools/qconfig.py index 86b2bed9283..8976b863dab 100755 --- a/tools/qconfig.py +++ b/tools/qconfig.py @@ -423,6 +423,43 @@ def _get_min_config_lines(kconf, fname): return lines +def _config_name(line): + """Extract config name (e.g. 'CONFIG_ARM') from a defconfig line + + Args: + line (str): A defconfig line + + Returns: + str or None: The config name, or None if not a config line + """ + stripped = line.strip() + if stripped.startswith('CONFIG_'): + return stripped.split('=', 1)[0] + if stripped.startswith('# CONFIG_'): + return stripped.split()[1] + return None + + +def _get_defconfig_entries(fname): + """Parse a preprocessed defconfig file to get config entries by name + + Args: + fname (str): Path to the preprocessed defconfig file + + Returns: + dict: Mapping of config name (e.g. 'CONFIG_ARM') to the full line + including the value (e.g. 'CONFIG_ARM=y') + """ + entries = {} + with open(fname, encoding='utf-8') as inf: + for line in inf: + line = line.strip() + name = _config_name(line) + if name: + entries[name] = line + return entries + + def _sync_plain_defconfig(kconf, orig, dry_run): """Sync a plain defconfig (no #include) @@ -450,6 +487,53 @@ def _sync_plain_defconfig(kconf, orig, dry_run): return updated +def _build_include_defconfig(include_lines, delta, sep): + """Build defconfig content from include lines and overlay delta + + Assembles a defconfig that uses #include directives by concatenating + the original #include lines with the overlay delta (the CONFIG lines + that are needed on top of what the includes provide). The separator + preserves the blank-line convention from the original file. + + Args: + include_lines (list of bytes): The #include lines + delta (list of str): Sorted overlay config lines + sep (bytes): Separator between includes and delta (b'\\n' or b'') + + Returns: + bytes: The defconfig content + """ + out = b'' + for line in include_lines: + out += line + if delta: + out += sep + for line in delta: + out += line.encode() if isinstance(line, str) else line + return out + + +def _format_sym_value(sym, value=None): + """Format a symbol value as a defconfig line + + Args: + sym (kconfiglib.Symbol): The symbol + value (str or None): Value to format; uses sym.str_value if None + + Returns: + str: The defconfig line + """ + if value is None: + value = sym.str_value + if sym.orig_type in (kconfiglib.BOOL, kconfiglib.TRISTATE): + if value == 'n': + return f'# CONFIG_{sym.name} is not set' + return f'CONFIG_{sym.name}={value}' + if sym.orig_type == kconfiglib.STRING: + return f'CONFIG_{sym.name}="{kconfiglib.escape(value)}"' + return f'CONFIG_{sym.name}={value}' + + def _sync_include_defconfig(kconf, srcdir, orig, dry_run): """Sync a defconfig that uses #include directives From patchwork Tue Apr 7 12:26:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2153 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564843; bh=AFm0SL33nyc/KNar6LoPC2YSxJzc0GJx02TLpqhodBk=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=AqbHLtJil9whirkzUnD/Sa8IcGvmuBQXPuANKOQw+MWsiQwhu+zUVyhUev655fb16 VmYWuYbOg382ezxNFGXp68JmxNknW+bsu0v7O3K0AvSoqUvoKv2YbNjvQIkYWuei3g M6g3w3Egzh1Cryu/z+Ojt2muq5IgPHvWbIy4z0AIHb/n5Zpb0fxVKYCKgY/1ymMlUh B757wHvEx7oyFW2ACW7rywwR4xj4OPVOS0kHLDO87mmStQiu49a8MuhBBirr2cI644 3QlKXGTOO/I4J4aS8GaQYfSVsxVMK3egX4wsBW1K/KB5N+KbM90a9EkDVO/Qdxl93N dMy/nim81hyug== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id F1BD36A386 for ; Tue, 7 Apr 2026 06:27:23 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id APSvHFfuFfK2 for ; Tue, 7 Apr 2026 06:27:23 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564842; bh=AFm0SL33nyc/KNar6LoPC2YSxJzc0GJx02TLpqhodBk=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=kM0hgGUppwTDyXswBTWCip9zeyBZF5tPuvhaHepBsugTAfmzgvr9pqNdzNETACaBo 6/OhtiVjon7TS58Ptu4INaQFEXP8GD/W9Ovcpf8DsSD3jUcwYQEcr/bBzLv0L6asiw Af+wNsehz3HF/zUR5F8FVY0LCwbbJa68ffdfUDCi8l8aUtYTeTysyTlyzd1AZ/fXHo +sFkHHTXrSmGDoazr9jfdoLYT0BZUTSqNj+ZOj09zAD2w67DcGY6bcESSTKQNlblyE 0RnwFJmzf5qdYViOU7eloDKJmGOhoDSHf9xXeXcsLBIPzlIYdBSTij4eNUFcUZwjsI m8LOQ5EooXuRg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id BF47E6A39B for ; Tue, 7 Apr 2026 06:27:22 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564841; bh=0TuntjR2VD5sRO/IWvY+A7/ELtb+Q9OAdqZRglQcBrM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jqzbsHXEpAmw+m8+MOf8NX0P+pZkCnS21/5+HyiSZpArkb/wHY0YXSjr2W7rBAW0P GoPfObQENZFwEGcx83sFw8F7O520fXLzKlGG2islusZSxKwTAMzBOP2nvspStKzE7W 9vrx6UZxWHzaPEY5ir9N+YMbbkvZQHCLpu0rCZRdz+dtBJun1Dezv0bharsgjiCnDt d4H3rOcH6agfw+4yaCCfNMjn8hhEIjwysdKidfrDOn5J+fd2a+oKtV0vhgWcWWPRPc 4E2C8ZZiu7PC4JOmzCSCHOuGnZQfw0lTUDbrsq3oNyGSwlc7pNe1eCfYj/tQxQQMCd n7rURw9U36Hgg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7592E6A3A4; Tue, 7 Apr 2026 06:27:21 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id XoF--WZWwkb7; Tue, 7 Apr 2026 06:27:21 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564839; bh=ceIZCENa67Ic1UIbsRkfut8YoyzTr1ucfzK3FsOTK4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IUFfIHvaH3yDTihLTfBVfWc29tcRxdZUIA220KAa21P8z/3HBPtTO7gPGb6AHPwK3 5Zx/jHS0xg4ATR6+JTUCJTKLNNeqpkcHLh6s45IQ3uX12wtqxzRW9TulNgpjxXJqJg CKhNi+HfGptwRclW2A+FaQienJTuFjJ448Avrc22eROT7hZm3tQarr7dzt6AE1b2uK UsRqr6AVPtwtzqiMIJIXJcD8Ax30YjBRzawIG8dHlblA9lfADHp2qxfX/mHnFkFqsz cjGs4gIgenDfHaLQqORplEhMvEpyOTs6KrDsvNlHLc9LhI3aCDZNcbynW9Dcoto0SA BF+vzbO4j17og== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id EFBCE6A39B; Tue, 7 Apr 2026 06:27:18 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Tue, 7 Apr 2026 06:26:49 -0600 Message-ID: <20260407122656.3462730-3-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260407122656.3462730-1-sjg@u-boot.org> References: <20260407122656.3462730-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: WPUWKXMV6JC4FUV6WO4VM6XRCVLUHGAQ X-Message-ID-Hash: WPUWKXMV6JC4FUV6WO4VM6XRCVLUHGAQ X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 2/4] qconfig: Fix #include defconfig sync delta computation List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The _sync_include_defconfig() function uses set subtraction of two write_min_config() outputs (full_min - base_min) to compute the overlay delta. This is incorrect because write_min_config() only emits entries that differ from Kconfig defaults, and which entries those are depends on what other symbols are set. The base and full configs set different symbols, so their min_configs are not directly comparable via set subtraction: an entry may appear in full_min but not base_min even though the include already provides the same value. Additionally, the include may set values (e.g. CONFIG_SPL_MMC=y) that the target does not want. Since write_min_config() assumes a fresh Kconfig state, it does not emit explicit resets for these. Fix this by: - Filtering full_min against the include files' textual entries rather than the base's min_config - Preserving the original defconfig ordering: keep existing overlay lines in place, only adding new entries and removing obsolete ones - Verifying the result by loading through kconfiglib and comparing against the target; iteratively adding corrections for any remaining differences caused by transitive Kconfig dependencies Signed-off-by: Simon Glass --- tools/qconfig.py | 186 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 155 insertions(+), 31 deletions(-) diff --git a/tools/qconfig.py b/tools/qconfig.py index 8976b863dab..490f68e4ecc 100755 --- a/tools/qconfig.py +++ b/tools/qconfig.py @@ -534,11 +534,108 @@ def _format_sym_value(sym, value=None): return f'CONFIG_{sym.name}={value}' +def _rebuild_overlay(orig, needed): + """Rebuild a #include defconfig preserving the original line ordering + + Keeps existing overlay lines that are still needed (updating values + that changed), drops lines that are now redundant, and appends + genuinely new entries at the end. + + Args: + orig (str): Path to the original defconfig file + needed (dict): Mapping of config name to defconfig line value + + Returns: + bytes: The rebuilt defconfig content + """ + orig_lines = tools.read_file(orig, binary=False).splitlines(keepends=True) + keep = set() + lines = [] + in_overlay = False + for line in orig_lines: + if line.startswith('#include'): + lines.append(line) + continue + name = _config_name(line) + if not name: + # Blank lines or comments - keep them (marks start of overlay) + if not in_overlay: + lines.append(line) + in_overlay = True + continue + in_overlay = True + if name in needed: + # Keep this line (possibly with updated value) + lines.append(needed[name] + '\n') + keep.add(name) + # else: drop the line (no longer needed in overlay) + + # Append new entries that weren't in the original overlay + new_entries = [] + for name, line in needed.items(): + if name not in keep: + new_entries.append(line + '\n') + if new_entries: + lines.extend(sorted(new_entries)) + + return ''.join(lines).encode() + + +def _verify_defconfig(kconf, srcdir, confdir, target, needed, new_content): + """Verify an #include defconfig and fix remaining config differences + + Loads the candidate defconfig through kconfiglib, compares against the + target config, and iteratively adds corrections for any remaining + differences (from include entries needing explicit resets or transitive + Kconfig dependency effects). + + Args: + kconf (kconfiglib.Kconfig): Kconfig instance + srcdir (str): Source-tree directory + confdir (str): Directory for temp file placement + target (dict): Target config {sym_name: str_value} + needed (dict): Overlay entries {config_name: line}, updated in place + new_content (bytes): Current defconfig content to verify + + Returns: + bytes: The verified (possibly updated) defconfig content + """ + for _ in range(3): + with tempfile.NamedTemporaryFile(suffix='_defconfig', + dir=confdir) as tmp: + tmp.write(new_content) + tmp.flush() + verify_pp = _cpp_preprocess(srcdir, tmp.name) + kconf.load_config(verify_pp) + os.unlink(verify_pp) + + # Find configs that differ from target and add corrections + extra_lines = [] + for sym in kconf.unique_defined_syms: + if sym.name not in target or sym.str_value == target[sym.name]: + continue + line = _format_sym_value(sym, target[sym.name]) + name = _config_name(line) + if name not in needed: + needed[name] = line + extra_lines.append(line + '\n') + + if not extra_lines: + break + new_content = new_content.rstrip(b'\n') + b'\n' + for line in sorted(extra_lines): + new_content += line.encode() + + return new_content + + def _sync_include_defconfig(kconf, srcdir, orig, dry_run): """Sync a defconfig that uses #include directives - Computes the minimal delta between the full config and the base config - provided by the included files, preserving the #include structure. + Computes the overlay delta needed on top of the include files to produce + the target config, preserving the original line ordering. The result is + verified against the target config; if any differences remain, the + missing entries are added iteratively. Args: kconf (kconfiglib.Kconfig): Kconfig instance @@ -549,51 +646,44 @@ def _sync_include_defconfig(kconf, srcdir, orig, dry_run): Returns: bool: True if the defconfig was (or would be) updated """ - # Get the full min_config (base + overlay) + # Get the full min_config and target config values full_tmp = _cpp_preprocess(srcdir, orig) full_lines = _get_min_config_lines(kconf, full_tmp) os.unlink(full_tmp) + # Save the target config for verification + target = {sym.name: sym.str_value + for sym in kconf.unique_defined_syms} + # Build a temp file with just the #include lines (no overlay CONFIGs) - # to get the base min_config include_lines = [] with open(orig, 'rb') as inf: for line in inf: if line.startswith(b'#include'): include_lines.append(line) - base_tmp = tempfile.NamedTemporaryFile(prefix='qconfig-base-', - suffix='_defconfig', - dir=os.path.dirname(orig), - delete=False) - base_tmp.writelines(include_lines) - base_tmp.close() + with tempfile.NamedTemporaryFile(suffix='_defconfig', + dir=os.path.dirname(orig)) as tmp: + tmp.writelines(include_lines) + tmp.flush() + base_pp = _cpp_preprocess(srcdir, tmp.name) - base_pp = _cpp_preprocess(srcdir, base_tmp.name) - os.unlink(base_tmp.name) - base_lines = _get_min_config_lines(kconf, base_pp) + base_entries = _get_defconfig_entries(base_pp) os.unlink(base_pp) - # Delta = full - base - delta = sorted(full_lines - base_lines) + # Build the set of configs needed in the overlay: full_min entries not + # already provided by the include files + needed = {} + for line in full_lines: + name = _config_name(line) + if not name or base_entries.get(name) != line.strip(): + needed[name] = line.strip() - # Build the new defconfig: #include lines + delta - # Preserve the separator (blank line or not) from the original - orig_text = tools.read_file(orig, binary=False) - last_include_idx = orig_text.rfind('#include') - after_include = orig_text[orig_text.index('\n', last_include_idx) + 1:] - sep = b'\n' if after_include.startswith('\n') else b'' + new_content = _rebuild_overlay(orig, needed) + new_content = _verify_defconfig(kconf, srcdir, os.path.dirname(orig), + target, needed, new_content) - new_content = b'' - for line in include_lines: - new_content += line - if delta: - new_content += sep - for line in delta: - new_content += line.encode() if isinstance(line, str) else line - - orig_content = tools.read_file(orig) - updated = new_content != orig_content + updated = new_content != tools.read_file(orig) if updated and not dry_run: tools.write_file(orig, new_content) return updated @@ -1811,6 +1901,40 @@ class SyncTests(unittest.TestCase): finally: os.unlink(tmp_name) + def test_sync_include_effective_config(self): + """Syncing a #include defconfig must not change the effective config""" + orig = 'configs/alt_defconfig' + if not os.path.exists(orig): + self.skipTest(f'{orig} not found') + + # Load the original to get the target config + full_pp = _cpp_preprocess(self.srcdir, orig) + self.kconf.load_config(full_pp) + os.unlink(full_pp) + target = {sym.name: sym.str_value + for sym in self.kconf.unique_defined_syms} + + # Sync into a temp copy + tmp_name = orig + '.test_tmp' + shutil.copy2(orig, tmp_name) + try: + _sync_include_defconfig(self.kconf, self.srcdir, tmp_name, + dry_run=False) + + # Load synced defconfig and compare + synced_pp = _cpp_preprocess(self.srcdir, tmp_name) + self.kconf.load_config(synced_pp) + os.unlink(synced_pp) + result = {sym.name: sym.str_value + for sym in self.kconf.unique_defined_syms} + + diffs = {name for name in set(target) | set(result) + if target.get(name) != result.get(name)} + self.assertEqual(diffs, set(), + 'Synced defconfig changed effective config') + finally: + os.unlink(tmp_name) + def do_tests(): """Run doctests and unit tests""" From patchwork Tue Apr 7 12:26:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2154 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564846; bh=h+EhUV8M3/gsFBtMD/zAudzvDGcgOpx8vVikaVBj5KI=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=D/AYD0rIjpWZuCoRz7DJldvGFvyNGh6sD7ZlRXny1Xx7NQ7HfCaieyWp9gAXAhfrW X1NsVbsr9wk1H9ytYJ4NU+dFUR0FKdkz5RVlCz3yPIfyX+Ng/SD3P2gyooDO31d81a 7aCplAqDd9dpJeqPqaHIpsT8mU1fberUdcdjq1bi/Cs3XHR1HXk4yhnbeODAK6mGvF yH0Mxu/nIxMtVl3jzFy1Fds3f7nJsl4LpytS+NnU2Gq160VhGBxMFTTpsYD1pOyGlE yaU/P3xoFSz1H0rfxy9odcu5nO0E79UhwMga9w1ZilXu3hSB43VqjaszQEe//jG0Iw 6Na2iLIKob33w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id E27686A3A1 for ; Tue, 7 Apr 2026 06:27:26 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id wVlWzoAwN9Nn for ; Tue, 7 Apr 2026 06:27:26 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564846; bh=h+EhUV8M3/gsFBtMD/zAudzvDGcgOpx8vVikaVBj5KI=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=D/AYD0rIjpWZuCoRz7DJldvGFvyNGh6sD7ZlRXny1Xx7NQ7HfCaieyWp9gAXAhfrW X1NsVbsr9wk1H9ytYJ4NU+dFUR0FKdkz5RVlCz3yPIfyX+Ng/SD3P2gyooDO31d81a 7aCplAqDd9dpJeqPqaHIpsT8mU1fberUdcdjq1bi/Cs3XHR1HXk4yhnbeODAK6mGvF yH0Mxu/nIxMtVl3jzFy1Fds3f7nJsl4LpytS+NnU2Gq160VhGBxMFTTpsYD1pOyGlE yaU/P3xoFSz1H0rfxy9odcu5nO0E79UhwMga9w1ZilXu3hSB43VqjaszQEe//jG0Iw 6Na2iLIKob33w== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CBB756A39E for ; Tue, 7 Apr 2026 06:27:26 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564843; bh=37As97Jv5OMlmoPoL86CO2MxGcJ0gF634DLaXt+kREg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O4Nt92s5Xrf3H87P66OEjd9OELdcTpbNmbC/dDtjF/AD71+eths27AbiTvRamffoV 0jqtuS3kgYQcHIXoUG5xkeBx9sxZnblCjbAP7CRaE5Tq/MJ1pS99NUguznLWjSDlx6 zW0kHHJYtM2OfTNxG2d0Z7Ub5euL+kvZYPPhjRY2EZ///jYq717x8auiILU8+D9alK GWI0Yfk3KDYWPXZLNPxSKXe2m89xwT+Cr8dUygYugVY29SeY1TSdMnZ2f2qUGyu6/Z tzekp5DwFMzC2gv9XqpfiFJNdiJdOfhrhWpU3luPUqtWJ3BUMAYdJ5xAPQ175k6MGl nSnLfts8aVDTA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id D51CA6A3A4; Tue, 7 Apr 2026 06:27:23 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id aLCRqyDKcEF0; Tue, 7 Apr 2026 06:27:23 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564840; bh=9GdrA22BooWg0VLoDW6du1ZttNK752Tw/6W+dqQI4Ks=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S8dUABmiOQs1LElPIgQB7O+/SU0nFsroLEQFqP72wuNr/0qoxmXrNzFRW274iXyiJ XKezBUddAcBGLPYiPCH1K4OTp47NPz5bNtu+XTVlDW39NjyojozkYPi/3WywHGVOjk CJc1zzFKYgFp4mb3QrjjQZt9abWWVy5o1PE2kTfUYWxuHQ21x1XO43C5j7VAPeGT0M RxZKTzTb6AdXu5EJpPkBJYM4H0tCjJCwoWG2GDMGvwZp/fPvm3LRJsXxOQZZqJSZdr W55DHogYtG3LV1SZkrSzua+v7u6M0hHDenDkFiG9FEBwDMnmsyIkzHBV19PLO/kEN6 37EFnTAqew0uA== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 8A45D6A3AA; Tue, 7 Apr 2026 06:27:20 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Tue, 7 Apr 2026 06:26:50 -0600 Message-ID: <20260407122656.3462730-4-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260407122656.3462730-1-sjg@u-boot.org> References: <20260407122656.3462730-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: DFIBBIMBNYEEGUMJLMUXXLL353N6UZHN X-Message-ID-Hash: DFIBBIMBNYEEGUMJLMUXXLL353N6UZHN X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 3/4] qconfig: Skip cosmetic-only #include defconfig changes List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The #include defconfig sync currently rewrites the file whenever any overlay entries are redundant (already provided by the include). This is cosmetic and creates unnecessary churn in resync commits. Only write changes when the effective config (what kconfiglib resolves after loading) actually differs from the target. Redundant overlay entries are harmless and preserving them keeps the resync commit minimal. Update the test to match: syncing a defconfig with a redundant CONFIG should leave the file unchanged. Also add a test that verifies syncing an #include defconfig does not change the effective config. Signed-off-by: Simon Glass --- tools/qconfig.py | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/tools/qconfig.py b/tools/qconfig.py index 490f68e4ecc..864b42e2808 100755 --- a/tools/qconfig.py +++ b/tools/qconfig.py @@ -683,7 +683,19 @@ def _sync_include_defconfig(kconf, srcdir, orig, dry_run): new_content = _verify_defconfig(kconf, srcdir, os.path.dirname(orig), target, needed, new_content) - updated = new_content != tools.read_file(orig) + # Only update the file if the effective config actually changes. + # Removing redundant overlay entries is cosmetic and would create + # unnecessary churn in the commit. + if new_content == tools.read_file(orig): + return False + + verify_pp = _cpp_preprocess(srcdir, orig) + kconf.load_config(verify_pp) + os.unlink(verify_pp) + orig_effective = {sym.name: sym.str_value + for sym in kconf.unique_defined_syms} + updated = orig_effective != target + if updated and not dry_run: tools.write_file(orig, new_content) return updated @@ -1858,8 +1870,8 @@ class SyncTests(unittest.TestCase): # The output should still start with #include self.assertIn(b'#include', content_after) - def test_sync_include_removes_redundant(self): - """Syncing a #include defconfig removes CONFIGs from the base""" + def test_sync_include_skips_redundant(self): + """Syncing a #include defconfig skips cosmetic-only changes""" # Create a temp defconfig that includes sandbox and redundantly # sets a CONFIG that sandbox already sets with tempfile.NamedTemporaryFile( @@ -1871,12 +1883,13 @@ class SyncTests(unittest.TestCase): try: updated = _sync_include_defconfig(self.kconf, self.srcdir, tmp_name, dry_run=False) - self.assertTrue(updated) + # Redundant CONFIG doesn't change the effective config, + # so the file should not be updated + self.assertFalse(updated) with open(tmp_name) as inf: result = inf.read() - # CONFIG_CMDLINE=y should be gone (it's in the base) - self.assertNotIn('CONFIG_CMDLINE=y', result) - # #include should still be there + # File should be unchanged + self.assertIn('CONFIG_CMDLINE=y', result) self.assertIn('#include "sandbox_defconfig"', result) finally: os.unlink(tmp_name) From patchwork Tue Apr 7 12:26:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2155 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564849; bh=TRu1LzHgNAlGh5BpPsTvMP6/SXc36mmW5K6FGbCdC78=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ccJy+0pe1qXCJLa4433ACYtR9sTyeN1Meodlxn8u49u98hpiEx5GJFlfISy1Qd96S YWp3adRfigbaVHAYH8no5ePJbX+bQgAvvqa+48z9s19sc7kNfWZ4vW+2y6mhnjUn8/ /hjalF0e+Fz1ENIFwVBw+QW8DE+dUnK+n8XYVZg8UQfMq1fiiPCz+FLD9r+sMirK60 FIt+gNcRHLFPN+dzZS3TwWYUEdd936v0lb+1UY+LnUyX6QdWaiMyFS6Oy6QOZBADew 1XmfNmAOkd0QRGTFLrfxJ4G7eqgTPXhMqo/ZKS4hBoXaAilHvpmoP7mf6wZGKNub3M LYfysQuxVRddQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 62FEF6A39B for ; Tue, 7 Apr 2026 06:27:29 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 6omDXCOeYNjz for ; Tue, 7 Apr 2026 06:27:29 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564848; bh=TRu1LzHgNAlGh5BpPsTvMP6/SXc36mmW5K6FGbCdC78=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=LM1D2yCr06t8b6A2BlIUwSxOKs46UuxzSR+1rTiSJwDYGTszAaFQPEqFGlqx2sKnj qPONFWPb+ZTSGI/hs0OvZk7LDcupwtul3KRmmr4B3aN5cWtx6k4fSAa5zq5k7kmNYT zRobVv3Y9R8MFXFYRwSX+SkUlFalktT8HbGVPvzdQa7Zuhjq4Ri0yOAEyipQMo0ik9 lk57zcQIB5u5H/wYlpppTjVtOUziihyp40phRf2pogeWuvirVBZuGk7lvEjQ2565jO z0BSZtjzVoHsoCOCaO00JB1/VhT3e3gVhN3HjqhFVZ1TEMSBgCuLdxerOgPF5RrjoU NTo+DgODpoGBg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5B6676A39E for ; Tue, 7 Apr 2026 06:27:28 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564846; bh=MmBmgOmRloQAgnkENAcuQ30JyULGU5G/6j4nY0csBGw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b24STksxODy8dQ1KPb3fCIYxcaG2rqu63YraWrXMLkW/wLHmhvExGfEJFkf/Q6W6N RAYIC+ayGbsCGeIvlrCIgOssxRw33huCunvS2fQLh7IVNjXvac66kqsQ5LpNelSh6E W8Qzy/oS37+YcEoFi2W/ldbo4kemvryl6099N4q0TY26lT4odrssPO4FRCauUSmitX Vnh9ujM/5Yak45gqg73C9dtDDteOGgsjnyYp7rFPCXYdzFzbvZ9SprUCBL0MuBcPx/ oOyp9aGXrGxXfIzgzMCnS8evJVm0mkvNGB/ODcrh6M0BhWoqtexqwzn2g4GbHE0kTZ Pz5wQe5v99hSQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 8C2C36A39B; Tue, 7 Apr 2026 06:27:26 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id m81wHjhKQVEB; Tue, 7 Apr 2026 06:27:26 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1775564841; bh=MCF8HhiznRDGmLTVwu4tXpyeKR2g9Zjh8Obqmb6Iuak=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=luXFcLfX31jX3WxDZjWZcxRsH7BdVSObp8GEq2OTP6kfnbxNbjx9fBpH5uJS99seu 0bYc7L8LmjqOI2D7MNbzsQ/Ni+AZWhGjzgroQpM6uoDz/NeFVBfSEoxmqouDzBbaW7 kjth884c0oRjtNtmEHAZvLs3/xFNdmE2uoAB3seTKHXDSupoWw344tw4HcXHTrzD14 /64y2VLwk3NrQvz2yVjev3iwCLdAo7Y4nO5KqO8P7iEe/GUzsxS4uds6LqK9LJqmsc TaU7Kcr4WzeR84Q+DWU7HTkUBPyRbZdJlHPXqFRo0cH3W4CfAuN+pfE1Bw0BCzNe+J VD4CChX/CaptQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id A89626A3A6; Tue, 7 Apr 2026 06:27:21 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Tue, 7 Apr 2026 06:26:51 -0600 Message-ID: <20260407122656.3462730-5-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260407122656.3462730-1-sjg@u-boot.org> References: <20260407122656.3462730-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: TTI5AH2JVCGTLAOHUJWOUQIY5FQ23XRJ X-Message-ID-Hash: TTI5AH2JVCGTLAOHUJWOUQIY5FQ23XRJ X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 4/4] test: Add qconfig tests to the test runner List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The qconfig tool has unit tests (qconfig.py -t) but they are not run by the test runner. Add them alongside the other tool tests so they run as part of 'make tcheck'. Signed-off-by: Simon Glass --- .azure-pipelines.yml | 1 + .gitlab-ci.yml | 1 + test/run | 1 + 3 files changed, 3 insertions(+) diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml index 09642e3295e..dad616efa9e 100644 --- a/.azure-pipelines.yml +++ b/.azure-pipelines.yml @@ -149,6 +149,7 @@ stages: ./tools/binman/binman --toolpath ${UBOOT_TRAVIS_BUILD_DIR}/tools tool -f missing ./tools/binman/binman --toolpath ${UBOOT_TRAVIS_BUILD_DIR}/tools test ./tools/buildman/buildman -t + ./tools/qconfig.py -t ./tools/dtoc/dtoc -t ./tools/u_boot_pylib/u_boot_pylib test ./tools/patman/patman test diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7b5097abbdf..a21d54c6c4a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -324,6 +324,7 @@ Run binman, buildman, dtoc, hwids_to_dtsi, Kconfig, patman and pickman suites: ./tools/binman/binman ${TOOLPATH} test; ./tools/binman/binman ${TOOLPATH} test -T; ./tools/buildman/buildman -t; + ./tools/qconfig.py -t; ./tools/dtoc/dtoc -t; ./tools/u_boot_pylib/u_boot_pylib test; ./tools/patman/patman test; diff --git a/test/run b/test/run index 20bad7bc5b5..89b3af05109 100755 --- a/test/run +++ b/test/run @@ -108,6 +108,7 @@ run_test "patman" ./tools/patman/patman test run_test "u_boot_pylib" ./tools/u_boot_pylib/u_boot_pylib test run_test "buildman" ./tools/buildman/buildman -t ${skip} +run_test "qconfig" ./tools/qconfig.py -t run_test "fdt" ./tools/dtoc/test_fdt -t run_test "dtoc" ./tools/dtoc/dtoc -t run_test "hwids_to_dtsi" python3 ./test/scripts/test_hwids_to_dtsi.py