From patchwork Sun Mar 29 11:10:28 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2066 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1774782705; bh=Tzr/CXv5iQif/SsFKUHpLBbvEDygtkPeDMT0wKdDt68=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=HdGH6/Gc5492g22Ny4Y/fwPdowZ7N9RWYxzF8jFvzf5gjf2axtGJ+SOAMJyL713/M TmnaVXfAV/2DYAPJephZZeUVAFvGAjXao0Z3lCmw0b1qhyjGngmgkahRzYvFFUaKzF AaYsJeNPiCAizyUS98BugQ5ZWa7db9WyeQ+VsV1klLMA55plWxk6+iOjMO22kXhGc5 FnL3RtkMf0ntz2YWtDrLgB3/qYKy+Z+6sGWfxVTyofWy6cG1TdKg/AIHcyAhabmIWO zrznanAe5VPH2Uwp9wzrx3kdT73kja4ouV9FgpHIjgSaEw16x5ZlfoOwhLyW9y2YkY 3lw6iDNviXmDQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DB76C6A2BE for ; Sun, 29 Mar 2026 05:11:45 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id aimC_VFRNSdz for ; Sun, 29 Mar 2026 05:11:45 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1774782704; bh=Tzr/CXv5iQif/SsFKUHpLBbvEDygtkPeDMT0wKdDt68=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=jO0tPqZncIlwTQnP3SJAhikHzGCJx1jZf2i+OduhgG6EQC1AtxG3UZ1H2o7KJyR02 pHutwlVdonVoFseIB0iG5uZzNaNSnegk9iVMPTvXSXWqgxAC9xnjylTzI8+qRpebrY M3wWRokSszOYDhkgudQwgA7xcJ+IJQ2n8GOsC0DK3Bo+4XXCWxPNCtdQKpcEYeY6X5 xHt81nu5kvymXJ1lpU0LDI69FIm2wV1imHpp55d0oumsZlrYFlUG7XbGD0/7/O/3fi A9Y7ty3/kaUU/RSJfzpw56dvfeABWeMhcTpMGEKKsf16Qjq7EeCVBH+UZUraY/ctnv LyeFy3L3O3Cdw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CDD5E6A2C1 for ; Sun, 29 Mar 2026 05:11:44 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1774782702; bh=NBm9qeJHlrScxZ1gk1spdp+I3R5bA3WkCzvdyjKHOb8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=REj8nQkY9+Vv+EAfTWvEY+1yHm7+SCc8C3ucTmALKjX/1tG1o5eiDrNTz67Cjp79l et5k7BcPVdd7UaHWkmaMNp6Fy0u2yyYi0gY7cPPa/qV8p4ujXDeqVHWIczHsox70PY twXYe0VONcHrPDKhl4tCQUmW4Jg/NnwDQun26rxPTJXbuaURXpUMtcwjJFTSZAEtxI TfPlj4e7pa7YYtVVqXkc8sdebWcDDkWo/9l/haR6UI7zF9pPSRdNfj4xWp+l5Hns/W hJRDZ7/R7LybwNFQ5qxYuYcQJikQVgSULO/lX2eevVlxr2Zew9FZCZVnje1/z7LXC5 gtG0qDTwaoFwA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A4B016A2BC; Sun, 29 Mar 2026 05:11:42 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id HBx6jcCzApR1; Sun, 29 Mar 2026 05:11:42 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1774782698; bh=c+Cd/kM9xl6YjE31tiVEwk41q09YtVS3BHCE2OGdW8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NKSmUIJhw0IQbTpIGcHXaNSnXU48reNVLIIKpeZaw4yyzOnKyhdEMzFZH119m/eLi edRx6CD8U6DkfIJOL5AkKWemrUxyx9V25njMU7rpw1ScSeXQIgP231MeghlxfpNV8A VEHOazDSgo9sSNtR9XQ+KroPgRpjuX4TIVFNTzcLiL8H4M7u5WfxLUKeUjFi6oRhE7 3KUWC/OvzB4+qtAGcl2lEVTCS52OreAgvzEfZvu9rjW2YJ/RNzci4VzqoFgQ6qVRo7 1tZUf02fG85vTmaQvgf8HU/reMmNYXGtL+gDFfLJV8BuPcgxmyY8vY18k27JvcRLdo 9PWp/hO5s4IAw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id B181B6A2C1; Sun, 29 Mar 2026 05:11:38 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Sun, 29 Mar 2026 05:10:28 -0600 Message-ID: <20260329111037.1352652-4-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260329111037.1352652-1-sjg@u-boot.org> References: <20260329111037.1352652-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 4VLRVPHPPMF3MWC5PB2XTTV74JYGGWHG X-Message-ID-Hash: 4VLRVPHPPMF3MWC5PB2XTTV74JYGGWHG X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 3/7] qconfig: Use kconfiglib for defconfig sync List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The -s option currently spawns two make subprocesses per board (make defconfig + make savedefconfig), requiring cross-compiler toolchains for every architecture. Replace this with kconfiglib's load_config() and write_min_config(), which produces output matching 'make savedefconfig'. This uses the same multiprocessing pattern as the -b optimisation, completing a full sync of ~1500 boards in under two seconds with no toolchain requirement. The -r (git-ref) option still uses the old make-based path, since it needs to build against a different source tree. Signed-off-by: Simon Glass --- tools/qconfig.py | 138 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) -- 2.43.0 diff --git a/tools/qconfig.py b/tools/qconfig.py index 1e1f74a0c85..133fea41212 100755 --- a/tools/qconfig.py +++ b/tools/qconfig.py @@ -406,6 +406,138 @@ def do_build_db(args): return config_db, progress +def _sync_defconfigs_worker(srcdir, defconfigs, result_queue, error_queue, + dry_run): + """Worker process that syncs defconfigs using kconfiglib + + For each defconfig, loads it via kconfiglib and writes a minimal config + (equivalent to 'make savedefconfig'), then compares with the original. + + Args: + srcdir (str): Source-tree directory + defconfigs (list of str): Defconfig filenames to process + result_queue (multiprocessing.Queue): Output queue for + (defconfig, updated) tuples + error_queue (multiprocessing.Queue): Output queue for failed defconfigs + dry_run (bool): If True, do not update defconfig files + """ + os.environ['srctree'] = srcdir + os.environ['UBOOTVERSION'] = 'dummy' + os.environ['KCONFIG_OBJDIR'] = '' + os.environ['CC'] = 'gcc' + kconf = kconfiglib.Kconfig(warn=False) + + for defconfig in defconfigs: + orig = os.path.join(srcdir, 'configs', defconfig) + try: + # Skip defconfigs with #include — savedefconfig mangles them + if b'#include' in tools.read_file(orig): + result_queue.put((defconfig, False, 'has #include')) + continue + + kconf.load_config(orig) + + tmp = tempfile.NamedTemporaryFile( + mode='w', prefix='qconfig-', suffix='_defconfig', + dir=os.path.join(srcdir, 'configs'), delete=False) + tmp.close() + kconf.write_min_config(tmp.name) + + updated = not filecmp.cmp(orig, tmp.name) + if updated and not dry_run: + shutil.move(tmp.name, orig) + else: + os.unlink(tmp.name) + result_queue.put((defconfig, updated, None)) + except Exception as exc: + error_queue.put((defconfig, str(exc))) + + +def do_sync_defconfigs(args): + """Sync defconfig files using kconfiglib instead of make + + Evaluates each defconfig through kconfiglib and writes a minimal config + (equivalent to 'make savedefconfig'), updating the original if it differs. + + Args: + args (Namespace): Program arguments (uses jobs, defconfigs, + defconfiglist, nocolour, dry_run, force_sync) + + Returns: + Progress: progress indicator + """ + srcdir = os.getcwd() + + if args.defconfigs: + defconfigs = [os.path.basename(d) + for d in get_matched_defconfigs(args.defconfigs)] + elif args.defconfiglist: + defconfigs = [os.path.basename(d) + for d in get_matched_defconfigs(args.defconfiglist)] + else: + defconfigs = get_all_defconfigs() + + col = terminal.Color(terminal.COLOR_NEVER if args.nocolour + else terminal.COLOR_IF_TERMINAL) + progress = Progress(col, len(defconfigs)) + + jobs = args.jobs + total = len(defconfigs) + result_queue = multiprocessing.Queue() + error_queue = multiprocessing.Queue() + processes = [] + for i in range(jobs): + chunk = defconfigs[total * i // jobs:total * (i + 1) // jobs] + if not chunk: + continue + proc = multiprocessing.Process( + target=_sync_defconfigs_worker, + args=(srcdir, chunk, result_queue, error_queue, args.dry_run)) + proc.start() + processes.append(proc) + + remaining = total + updated_count = 0 + while remaining: + found = False + while not result_queue.empty(): + defconfig, updated, msg = result_queue.get() + if updated: + updated_count += 1 + name = defconfig[:-len('_defconfig')] + log = col.build(col.BLUE, 'defconfig updated', bright=True) + if args.dry_run: + log = col.build(col.YELLOW, 'would update', bright=True) + print(f'{name.ljust(20)} {log}') + elif msg: + name = defconfig[:-len('_defconfig')] + log = col.build(col.RED, f'ignored: {msg}', bright=True) + print(f'{name.ljust(20)} {log}') + progress.inc(True) + progress.show() + remaining -= 1 + found = True + while not error_queue.empty(): + defconfig, msg = error_queue.get() + print(col.build(col.RED, f'{defconfig}: {msg}', bright=True), + file=sys.stderr) + progress.inc(False) + progress.show() + remaining -= 1 + found = True + if not found: + time.sleep(SLEEP_TIME) + + for proc in processes: + proc.join() + + progress.completed() + if updated_count: + print(col.build(col.BLUE, + f'{updated_count} defconfig(s) updated', bright=True)) + return progress + + # pylint: disable=R0903 class KconfigParser: """A parser of .config and include/autoconf.mk.""" @@ -1928,6 +2060,12 @@ def main(): config_db, progress = do_build_db(args) return write_db(config_db, progress) + if args.force_sync and not args.git_ref: + progress = do_sync_defconfigs(args) + if args.commit: + add_commit(args.configs) + return move_done(progress) + config_db, progress = move_config(args) if args.commit: