From patchwork Mon Mar 16 15:47:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 2003 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1773676135; bh=ZLQyHzkIICe8vdrPeYFphW2euWAVPOVS5BQ8kJe6EtE=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=p3LNCTMOMuygLoYbMIpNGZ1RX5B6nA6064hjnHn3ohMeRKwPG92U9G9PeBNeLuuNC Px9idzkMhp5TgzYaeUjG7buWa5k6myDwCrRHQnasmAb6me1uWH3i8tTxzWqt7qUe0g y999OC0X8nnaUsPehA3+YJr41SV61FY+Jv8FAWlcu+IUXs751UpgcwjqM1MiVcTBNa IrmcxMHb5HXUKt5L5vlyUa73MMWQqDe2yp7ct3V9nAu/Y0wdUUnlWRlSmufyg0CXIZ V0fQ98u+cjWL9VtgnzTrMKf28MRvJlf527gXoOk0reAOar2U2rqlAiguCqdwVHeAuK sVFh4uI8jjIPQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4D6F26A08A for ; Mon, 16 Mar 2026 09:48:55 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id qNFmTj2vU7VY for ; Mon, 16 Mar 2026 09:48:55 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1773676134; bh=ZLQyHzkIICe8vdrPeYFphW2euWAVPOVS5BQ8kJe6EtE=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=PKgMRnDsYpq8t8nwNq6UXIM6XT08LaPAISnCqB0RrPsdif8fuKadcYvTeuDj38vVT xBbo8oMjYvzNUGnZe+ZlrGLN/saZIJ8E96wRaktHErAW2mQ3V4b0cMWga/i9MfhYRd OApLKJGjOC3mCvQ+W9i2ZC6e/bxUyPsF526l+CPTS0xEfp/sErFMRtvLpNiDZEDFJ5 kQWCF8YNr/D1lza6XbEPt0K0He0W7dVktxuoY5xa4yX7gwF24gZ/5heulVFMfmAFk3 85eLwSL1ZLsUxq90Klw9WCykPubkZM8cMDPax60AjUJ5l8n1fr90ymqHdwGbTrASiQ orG3LuV5G+kdA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7C1BF6A08D for ; Mon, 16 Mar 2026 09:48:54 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1773676132; bh=QhjLxDnbQ5AJxzDRlXE23YOcYG4EI1p7sYr+GOv9u1g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=w66rZXRSN1lvM9+rJweolegFhZGRLBml5m/6qb90jvX1jwYaGRc7POA08f9Dxv0JG DTL+NcPJKcB5NUWPDkKq1foe8NpsKSRuRDBu+c++xTnNd0WzaS45EpQB5zt5d3OlBt EX7MCCm0w2M/FkPuBuqR/gesLQQxjLzDvVe6KqqUborpgVFEWXpLGcLJynS9zD6/3x f2Cl/O0Fo42HQIZDd7mphWUkDqc+T55PTMMMrNllzR+Q8QFO70Qa2pMk2yyZAeQ/49 Uc2FoSjdNXorAfwPRIIWgvjySIOBGA2+wGARk3csXG17rxgYkTAJTx+qn6w8loV2IR Rx8XrHY/LlZDA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 296386A084; Mon, 16 Mar 2026 09:48:52 -0600 (MDT) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id XqaL9xWkX2hw; Mon, 16 Mar 2026 09:48:52 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1773676129; bh=vpdECALNjuGPUE89c0wJOFIkMCRZhKtYTVLJGeKN5SM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hrw4A+t0RiCiFhEoYeAaXC9nsmPss1i1VCXpExhXKgulwMWW4R+MIKWXAuXtkOMiu JldtEM9GjhzlkyQ1WDjUR9N2Tq10JmwGUBiZU5nrn8BZTx1bm6XGPI0n+vEjrFrEih csD15UsItIi6MCA1OSchMY6nQ8g74vnUfZKdC7sNY1spwrEvf/p7QoardL/U8g/gEX bFS7VHf3rOIYK32WTAExuC84wcbbi5zmp/RN1Fgq+eQw3PWCwP+uD5/3oIjhxiqgDm YmiTyJObLUQMSolGGAUfGMwrzLCjOvPLKoSg3WlTc8LkADZiIMSSJgPfPe2znuv7ha Y25OfhmyvZC+Q== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 70A146A07C; Mon, 16 Mar 2026 09:48:49 -0600 (MDT) From: Simon Glass To: U-Boot Concept Date: Mon, 16 Mar 2026 09:47:16 -0600 Message-ID: <20260316154733.1587261-12-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260316154733.1587261-1-sjg@u-boot.org> References: <20260316154733.1587261-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: DQJFUWMTZXGFKKWSYMEPSR5VRUX7ZOOU X-Message-ID-Hash: DQJFUWMTZXGFKKWSYMEPSR5VRUX7ZOOU X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 11/20] buildman: Add distributed-build attributes to Builder List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add active_boards and max_boards counters (with lock) for use by the dynamic -j calculation in BuilderThread, so each make invocation gets an appropriate number of jobs based on how many boards are currently in flight. Add an extra_count parameter to init_build() and build_boards() so the progress total can include builds running on remote workers. Add a delay_summary parameter to run_build() and build_boards() so the boss can defer the summary until remote results have been collected. Signed-off-by: Simon Glass --- tools/buildman/builder.py | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index 716bf43a66f..de895d43cfe 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -306,6 +306,9 @@ class Builder: self.checkout = checkout self._num_threads = num_threads self.num_jobs = num_jobs + self.active_boards = 0 + self.max_boards = 0 + self._active_lock = threading.Lock() self._already_done = 0 self.kconfig_reconfig = 0 self.force_build = False @@ -1200,7 +1203,7 @@ class Builder: terminal.print_clear() def init_build(self, commits, board_selected, keep_outputs, verbose, - fragments): + fragments, extra_count=0): """Initialise a build: prepare working space and create jobs This sets up the working directory, output space and job queue @@ -1214,6 +1217,9 @@ class Builder: keep_outputs (bool): True to save build output files verbose (bool): Display build results as they are completed fragments (str): config fragments added to defconfig + extra_count (int): Additional builds expected from external + sources (e.g. distributed workers) to include in the + progress total """ self.commit_count = len(commits) if commits else 1 self.commits = commits @@ -1228,6 +1234,7 @@ class Builder: tprint('\rStarting build...', newline=False) self._start_time = datetime.now() self._setup_build(board_selected, commits) + self.count += extra_count self.process_result(None) self.thread_exceptions = [] # Create jobs to build all commits for each board @@ -1245,12 +1252,16 @@ class Builder: else: self._single_builder.run_job(job) - def run_build(self): + def run_build(self, delay_summary=False): """Run the build to completion - Waits for all jobs to finish and prints a summary. + Waits for all jobs to finish and optionally prints a summary. Call init_build() first to set up the jobs. + Args: + delay_summary (bool): True to skip printing the build + summary at the end (caller will print it later) + Returns: tuple: Tuple containing: - number of boards that failed to build @@ -1266,13 +1277,13 @@ class Builder: # Wait until we have processed all output self.out_queue.join() - if not self._opts.ide: + if not self._opts.ide and not delay_summary: self.print_summary() return (self.fail, self._warned, self.thread_exceptions) def build_boards(self, commits, board_selected, keep_outputs, verbose, - fragments): + fragments, extra_count=0, delay_summary=False): """Build all commits for a list of boards Convenience method that calls init_build() then run_build(). @@ -1284,6 +1295,11 @@ class Builder: keep_outputs (bool): True to save build output files verbose (bool): Display build results as they are completed fragments (str): config fragments added to defconfig + extra_count (int): Additional builds expected from external + sources (e.g. distributed workers) to include in the + progress total + delay_summary (bool): True to skip printing the build + summary at the end (caller will print it later) Returns: tuple: Tuple containing: @@ -1292,8 +1308,8 @@ class Builder: - list of thread exceptions raised """ self.init_build(commits, board_selected, keep_outputs, verbose, - fragments) - return self.run_build() + fragments, extra_count) + return self.run_build(delay_summary) def print_summary(self): """Print the build summary line