From patchwork Wed Dec 17 02:27:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 944 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938528; bh=XAE7Gcvbq1ZKVlq17yh3U0KN7hUWEhpAeu/EnR6g1sE=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ibx6AoImL9A/Hsmn1c2tEViypf7AxOjUr1UBS43gRHk0wcCWulcjK9+WhTaOG0/op bHLm9XuarLijoSeA4pC9uEnffEdPH9Ul7ugIz7UYmX9H014RtM1Hlbbs3s6WTfW3ZM NAuYE25IQh0s+N4kqehBWP03ogIhE81Znr637XUOA6J76bk0DE3WrRuU+dAvhnI8Pp Q3iNHHfC3zyHYPi8rDdl4YRD5goa5Ho2xlvyhXmU5qJ/C4xcm9dD9fXCz35Kd18bYi s/HNyPkr6WMS2HGvq0CF4b47H6PughQGKOzVGdLLE7dv6qW6Nb2c5pcQeK5bC0ve/m 9KAGFWwL6ikOA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7A83A68BB6 for ; Tue, 16 Dec 2025 19:28:48 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id GWdfJfQWyvd5 for ; Tue, 16 Dec 2025 19:28:48 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938528; bh=XAE7Gcvbq1ZKVlq17yh3U0KN7hUWEhpAeu/EnR6g1sE=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ibx6AoImL9A/Hsmn1c2tEViypf7AxOjUr1UBS43gRHk0wcCWulcjK9+WhTaOG0/op bHLm9XuarLijoSeA4pC9uEnffEdPH9Ul7ugIz7UYmX9H014RtM1Hlbbs3s6WTfW3ZM NAuYE25IQh0s+N4kqehBWP03ogIhE81Znr637XUOA6J76bk0DE3WrRuU+dAvhnI8Pp Q3iNHHfC3zyHYPi8rDdl4YRD5goa5Ho2xlvyhXmU5qJ/C4xcm9dD9fXCz35Kd18bYi s/HNyPkr6WMS2HGvq0CF4b47H6PughQGKOzVGdLLE7dv6qW6Nb2c5pcQeK5bC0ve/m 9KAGFWwL6ikOA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 68B2868BAC for ; Tue, 16 Dec 2025 19:28:48 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938527; bh=hpD8GXq2fGF+9H3HdXGzEroWBqOS/GNm4H20yVORW6g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jLwpTa4JEiZLtYkfPfkiGbXke1VNapiU3EwOzJq9LMa5SZV7ZOIp0mOC5Ykp2uiwV PuD4R8YooRG5XXKewxPBhWPef33+ESSKj4nVMT5rAG2EEQXSf31df0VWvuEX0B7l9S gmhOq/mceMic8E+PBYki9aZUuLzF/L7ZZHqo1bWCEbl/nzqgV21oKj5cIRIBBetQZI BU+dZJ/8Y1G0zOJ7otwyx0Lb6uzFX38x6ROLm257f1Lkk/yDSHGddIxKu08iRgbhGV 35MGxoQTREOpNrHkNEgOciPm68aVCvdMQU8QsTse0fx5DOeO/IwPScvvJ9Ao7I6ErC ty38QxCGYYpcA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 60B4968AFF; Tue, 16 Dec 2025 19:28:47 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id ponWsSNy8xDw; Tue, 16 Dec 2025 19:28:47 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938523; bh=cK8ZdjRlhmdrzz13CjmkqeerVfCpYb+m+JY4+pZLmxU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TsAuOMrb6Fm8YgvUc0QaloV2zjNvH+kZWR5r8XWIAIhtKzbmwzc0kyt4H+bOzJBeA 5ARQ+lzSbeIUvqqpmrgwJiQ90QlgtpR/3h/B900TNYqfhab4k4q61fGzJ4+tWORD1I AGSZJIvSNV9c/hQxtVALiwZemv8oUeROh9mqX7BunFTgg4bzVWfloYliKIpCG9aN5G KadGIopocWY9ec8iRcOIMlkAKfaisl/px+Hb7JGvp0gagJS+0xe82cRQpNLfxkgqBg 0lS35qteK24B+iiC2fjbut728CTkzFGmDRnljCVwbraXAWJ7LI0+KwRf1h3sAzGI+d 4MeIW2MFrXsOQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id E288C6884F; Tue, 16 Dec 2025 19:28:42 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:50 -0700 Message-ID: <20251217022823.392557-2-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: GEUNGUUI5ZBTWSE7QOXLUKWLTKVBI7GM X-Message-ID-Hash: GEUNGUUI5ZBTWSE7QOXLUKWLTKVBI7GM X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 01/24] pickman: Add terminal.capture() to tests for silent output List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add terminal.capture() context manager to tests that may produce terminal output, ensuring tests run silently unless explicitly checking output content. Fix a few pyline warnings while here. Signed-off-by: Simon Glass Co-developed-by: Claude Opus 4.5 --- tools/pickman/ftest.py | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index e5ed187bdf6..8e3b91df750 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -51,7 +51,8 @@ class TestRunGit(unittest.TestCase): result = command.CommandResult(stdout=' output with spaces \n') command.TEST_RESULT = result try: - out = control.run_git(['status']) + with terminal.capture(): + out = control.run_git(['status']) self.assertEqual(out, 'output with spaces') finally: command.TEST_RESULT = None @@ -73,7 +74,8 @@ class TestCompareBranches(unittest.TestCase): command.TEST_RESULT = handle_command try: - count, commit = control.compare_branches('master', 'source') + with terminal.capture(): + count, commit = control.compare_branches('master', 'source') self.assertEqual(count, 42) self.assertEqual(commit.hash, 'abc123def456789') @@ -96,7 +98,8 @@ class TestCompareBranches(unittest.TestCase): command.TEST_RESULT = handle_command try: - count, commit = control.compare_branches('branch1', 'branch2') + with terminal.capture(): + count, commit = control.compare_branches('branch1', 'branch2') self.assertEqual(count, 0) self.assertEqual(commit.short_hash, 'def456a') @@ -1071,13 +1074,15 @@ class TestCheckAvailable(unittest.TestCase): def test_check_available_false(self): """Test check_available returns False when gitlab not installed.""" with mock.patch.object(gitlab_api, 'AVAILABLE', False): - result = gitlab_api.check_available() + with terminal.capture(): + result = gitlab_api.check_available() self.assertFalse(result) def test_check_available_true(self): """Test check_available returns True when gitlab is installed.""" with mock.patch.object(gitlab_api, 'AVAILABLE', True): - result = gitlab_api.check_available() + with terminal.capture(): + result = gitlab_api.check_available() self.assertTrue(result) @@ -1190,7 +1195,8 @@ class TestStep(unittest.TestCase): return_value=[mock_mr]): args = argparse.Namespace(cmd='step', source='us/next', remote='ci', target='master') - ret = control.do_step(args, None) + with terminal.capture(): + ret = control.do_step(args, None) self.assertEqual(ret, 0) @@ -1200,7 +1206,8 @@ class TestStep(unittest.TestCase): return_value=None): args = argparse.Namespace(cmd='step', source='us/next', remote='ci', target='master') - ret = control.do_step(args, None) + with terminal.capture(): + ret = control.do_step(args, None) self.assertEqual(ret, 1) @@ -1212,7 +1219,8 @@ class TestStep(unittest.TestCase): return_value=None): args = argparse.Namespace(cmd='step', source='us/next', remote='ci', target='master') - ret = control.do_step(args, None) + with terminal.capture(): + ret = control.do_step(args, None) self.assertEqual(ret, 1) @@ -1241,7 +1249,8 @@ class TestReview(unittest.TestCase): with mock.patch.object(gitlab_api, 'get_open_pickman_mrs', return_value=[]): args = argparse.Namespace(cmd='review', remote='ci') - ret = control.do_review(args, None) + with terminal.capture(): + ret = control.do_review(args, None) self.assertEqual(ret, 0) @@ -1250,7 +1259,8 @@ class TestReview(unittest.TestCase): with mock.patch.object(gitlab_api, 'get_open_pickman_mrs', return_value=None): args = argparse.Namespace(cmd='review', remote='ci') - ret = control.do_review(args, None) + with terminal.capture(): + ret = control.do_review(args, None) self.assertEqual(ret, 1) @@ -1287,7 +1297,7 @@ class TestPoll(unittest.TestCase): """Test poll stops gracefully on KeyboardInterrupt.""" call_count = [0] - def mock_step(args, dbs): + def mock_step(_args, _dbs): call_count[0] += 1 if call_count[0] >= 2: raise KeyboardInterrupt @@ -1299,7 +1309,8 @@ class TestPoll(unittest.TestCase): cmd='poll', source='us/next', interval=1, remote='ci', target='master' ) - ret = control.do_poll(args, None) + with terminal.capture(): + ret = control.do_poll(args, None) self.assertEqual(ret, 0) self.assertEqual(call_count[0], 2) From patchwork Wed Dec 17 02:27:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 945 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938534; bh=NA/V5skG8kkfNuOGeDGGC2Yu5/MjM46iEaBUS/wCmn4=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=IZSYiF4LxPg3KkWMrXZziG3ZISfJGj7wN7GQP/k7ovRdg8OC61D611xJxqEOnXXQM O1BVoOues1qbUhEdr4GmWCMFga1GkpmOcuJpFXfxVroWmwu6m2KCtrw6asjcJo8xkH mEeZK/dwhQT63FVYWQAM0aXj5oWjVhivdcOQ6zd/yXovJr/nRC/vorGIvX3m7ZXlfs 7w3cNYgPJqjsmBI9J3K2Z7fCi+wlonf1jvo04XB2IeLnA/5IoMDmFhxODUv5v6TMTD V2XYK2lzd/VnNyGU3BJdhBn4xx3z54R/UkT7rABTBTSYBenc/itiAd216hRPpkqBl9 ZSUXSGEsz9tGA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4F99468BB8 for ; Tue, 16 Dec 2025 19:28:54 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id UuWqrow2GbdN for ; Tue, 16 Dec 2025 19:28:54 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938534; bh=NA/V5skG8kkfNuOGeDGGC2Yu5/MjM46iEaBUS/wCmn4=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=IZSYiF4LxPg3KkWMrXZziG3ZISfJGj7wN7GQP/k7ovRdg8OC61D611xJxqEOnXXQM O1BVoOues1qbUhEdr4GmWCMFga1GkpmOcuJpFXfxVroWmwu6m2KCtrw6asjcJo8xkH mEeZK/dwhQT63FVYWQAM0aXj5oWjVhivdcOQ6zd/yXovJr/nRC/vorGIvX3m7ZXlfs 7w3cNYgPJqjsmBI9J3K2Z7fCi+wlonf1jvo04XB2IeLnA/5IoMDmFhxODUv5v6TMTD V2XYK2lzd/VnNyGU3BJdhBn4xx3z54R/UkT7rABTBTSYBenc/itiAd216hRPpkqBl9 ZSUXSGEsz9tGA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3E67768AEE for ; Tue, 16 Dec 2025 19:28:54 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938532; bh=kk7BNNqU+zrrL/LVX6+rZKnWliFINwuCm/kyRAxmoKo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=azOv/GZAd4pzsR3axZMhYsChFqD9irSY4x4sEfIFC3QW2z9fZR9I3RrTqactmLW5n TH0ulC8IVyv02TGRvK+hGYksDHeHzsQiog3g7xoNEhMeNKOHNpAtkXmsuD0EO1VO+n bykpc4riCJWNTECO6Bog8HK6a+8TMFvH7GcaF8nB9V0eB/J9x+BR6xNcTeuz+9P2QK uuj7hOb5tcR1kGsKZ3m1np/3dBlWmdoxoqDpJWWmZhW19kF6dGbWuzLGyg1fGf9utm VZ6Z7kpyB1yFiyWY0Yjg/pfHGt6MvLXCTSlHXBwmIcishWsYcCRimjuST6uTqVBaiR j9XF2YY8WcVYQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2470468AFF; Tue, 16 Dec 2025 19:28:52 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id hekp4Q08mDSd; Tue, 16 Dec 2025 19:28:52 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938527; bh=z35H+3PqpD63SShK1qcjxJ9+lmDtuV5k6fwbHzAodow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H7QIdvoBfMjtFXv0TrbZAxbmXDB1KYG6Zg5omB8RMvXDM08lzxYoTLYB8TOQ309Kh NaECQhNUPEhJWOvoHeH5lQ7GkBMrcfVh5qJwSGLN2yMznr+jOtq3QpPrAeHG5l+BJd gikJ51t5TBo1s85eAD8KgkATYFLMIN1lj3yPXwvXXXRatkxl9+diw5qwI06U9ZxMDW LKU3V4/mTcXIKS2JhV2bOJNu1WWneIG405KhVIyrBL2tpQfyq+D+t6I3gB3Zg4+XDC uR8IJ2ozuhK7PXYic60bXPPFkLaf4IDZa3wgV/na4zMdop4un4CyCtVjvLC6xyQMQ3 PMhVFWKBcSZxQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id A62896884F; Tue, 16 Dec 2025 19:28:47 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:51 -0700 Message-ID: <20251217022823.392557-3-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: SR2KDERIVVT3BDWOO3ZHXBX6UFR3W6ID X-Message-ID-Hash: SR2KDERIVVT3BDWOO3ZHXBX6UFR3W6ID X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 02/24] pickman: Move imports to top of control.py List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Move datetime, re, and time imports from inside functions to the top of the file, following Python style guidelines. Also fix a few pylint warnings while here. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 96b014dfeca..0353bae9350 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -6,8 +6,11 @@ """Control module for pickman - handles the main logic.""" from collections import namedtuple +from datetime import date import os +import re import sys +import time import unittest # Allow 'from pickman import xxx' to work via symlink @@ -61,9 +64,9 @@ def compare_branches(master, source): # Get details about the merge-base commit info = run_git(['log', '-1', '--format=%H%n%h%n%s%n%ci', base]) - full_hash, short_hash, subject, date = info.split('\n') + full_hash, short_hash, subject, commit_date = info.split('\n') - return count, Commit(full_hash, short_hash, subject, date) + return count, Commit(full_hash, short_hash, subject, commit_date) def do_add_source(args, dbs): @@ -242,8 +245,6 @@ def format_history_summary(source, commits, branch_name): Returns: str: Formatted summary text """ - from datetime import date - commit_list = '\n'.join( f'- {c.short_hash} {c.subject}' for c in commits @@ -266,9 +267,6 @@ def write_history(source, commits, branch_name, conversation_log): branch_name (str): Name of the cherry-pick branch conversation_log (str): The agent's conversation output """ - import os - import re - summary = format_history_summary(source, commits, branch_name) entry = f"""{summary} @@ -522,8 +520,6 @@ def parse_mr_description(description): Returns: tuple: (source_branch, last_commit_hash) or (None, None) if not parseable """ - import re - # Extract source branch from "## date: source_branch" line source_match = re.search(r'^## [^:]+: (.+)$', description, re.MULTILINE) if not source_match: @@ -664,8 +660,6 @@ def do_poll(args, dbs): Returns: int: 0 on success (never returns unless interrupted) """ - import time - interval = args.interval tout.info(f'Polling every {interval} seconds (Ctrl+C to stop)...') tout.info('') From patchwork Wed Dec 17 02:27:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 946 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938538; bh=+JaDiWLAfgDCOYUKD+319pd5GyKYJ6wUm7mpx9MhdTI=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=vodRs7CFtmToMm0i4NFCCQmTg8ZaUffjt5R7iHJNMnDwmnfZfwzU8iAKDtTOKA+F1 IDXj0Y3e9cthqpXPFglQX+r5cF6H08cWZMAsScSCwNG7+s9JOudrKYnWtRIOFn4AMO KqIZvkYTiYP2uBmRPHz1osYBxBK7smzGrGfeHD5S1V5ze6Vx0nheMlN3enaZ1JWkyb Elxk+54oBQZ9BIsn+lzw9HUpVv1pCPDZB+85mCFK/KSAJBs69jd9uHIG9OzF2LXJuu +pVTVzcykgRPIcrOLjNO/INaAhXXhPFG1mlZNnjn71jNmT2TZLt5g30Q4/8b3nxlbM e0IJbKqu1CsSA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id B7C0268BB8 for ; Tue, 16 Dec 2025 19:28:58 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id RxVSuiZoyiQ7 for ; Tue, 16 Dec 2025 19:28:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938538; bh=+JaDiWLAfgDCOYUKD+319pd5GyKYJ6wUm7mpx9MhdTI=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=vodRs7CFtmToMm0i4NFCCQmTg8ZaUffjt5R7iHJNMnDwmnfZfwzU8iAKDtTOKA+F1 IDXj0Y3e9cthqpXPFglQX+r5cF6H08cWZMAsScSCwNG7+s9JOudrKYnWtRIOFn4AMO KqIZvkYTiYP2uBmRPHz1osYBxBK7smzGrGfeHD5S1V5ze6Vx0nheMlN3enaZ1JWkyb Elxk+54oBQZ9BIsn+lzw9HUpVv1pCPDZB+85mCFK/KSAJBs69jd9uHIG9OzF2LXJuu +pVTVzcykgRPIcrOLjNO/INaAhXXhPFG1mlZNnjn71jNmT2TZLt5g30Q4/8b3nxlbM e0IJbKqu1CsSA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A5EE668AFD for ; Tue, 16 Dec 2025 19:28:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938536; bh=bKSqNxju5bbMmQ6kIMMePTDttlbU8AeZCZwSxf+1dcE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FP5BDhn3bHPXwpvJmvhsBE8LCCHezmxvu+QXI5csGrs4rBXIAJF6uj8K9EH8+IYHT QjMpGchcO11S9q5F+cyy2QJ8A1UlGOOuUdTXa+GHWmjef2+huOvRLybxvImeHNbGU7 PEHkTMMMbjiW+aBi3JXeUtCOkbqox/xN9jP+jzeZ1POu6574sedl1yIQXxiHH9SvGW yswtCKIqDqkmin0qiqeQmMaHgeg9o10JJ0bfwNI7Mbe4WnQRcE56BSuX+6d7SEX2MU lbpnDwLPwsiO613lulexUAah+5rPjrtQ2hiG8LIS/QcPQilTOfNFPjDLXcdhOq3Ypp lsHAtzFCdrkFA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id D1D1268ABD; Tue, 16 Dec 2025 19:28:56 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id PI7W4ObSX1cs; Tue, 16 Dec 2025 19:28:56 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938532; bh=IExaNey6SHa9qAqgKEKmePaVHXVmrezbLUtPOpMXur8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VRILfiptMYeDsxnSDf1JZ5ma9znUxrnSfKjI3PWcLeIb0xoXrbXM7XVP54ggtOgJr 0iL4FCur61Q7T5+P5mY+Ri8Vt781VoimAM1EDBuWtppC7MwlrtwkwURmAmTGjws8Za mll3n35oakZdFf9uzmFqb75dzXGzgHKRY7lArRXGbba4FGzzyLxaUxZzzRrXclRDg0 PRUymffUnLQFFL7CPEkN3Y26EWQETtWlTDAcJ0Hf09xKeX1Os80eD2cz02LS2KutCk T3sTNaaGKw2myUW29fAl/XEjuVMb011zobmbVo255ofAXfP49gOCZwrbWJPRZXXFWw Ra6GbvmwMSqEQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 672946884F; Tue, 16 Dec 2025 19:28:52 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:52 -0700 Message-ID: <20251217022823.392557-4-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: LO6IDOGCXZQ7TZP5EVOSLM7GO6X7NN3L X-Message-ID-Hash: LO6IDOGCXZQ7TZP5EVOSLM7GO6X7NN3L X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 03/24] pickman: Fix pylint warnings List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass - Remove unused commit_hashes variable in agent.py - Add pylint disable for too-many-public-methods in database.py - Add pylint disable for too-many-arguments in database.py functions - Add pylint disable for too-many-branches in control.py do_apply() - Add pylint disable for too-many-lines in ftest.py - Prefix unused mock arguments with underscore in ftest.py This brings the pylint score to 10.00/10. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/agent.py | 1 - tools/pickman/control.py | 2 +- tools/pickman/database.py | 4 +++- tools/pickman/ftest.py | 1 + 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/tools/pickman/agent.py b/tools/pickman/agent.py index 76cf8f98ec3..176cf9773a0 100644 --- a/tools/pickman/agent.py +++ b/tools/pickman/agent.py @@ -60,7 +60,6 @@ async def run(commits, source, branch_name, repo_path=None): f' - {short_hash}: {subject}' for _, short_hash, subject in commits ) - commit_hashes = ' '.join(hash for hash, _, _ in commits) prompt = f"""Cherry-pick the following commits from {source} branch: diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 0353bae9350..474315f7db0 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -299,7 +299,7 @@ def write_history(source, commits, branch_name, conversation_log): tout.info(f'Updated {HISTORY_FILE}') -def do_apply(args, dbs): # pylint: disable=too-many-locals +def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches """Apply the next set of commits using Claude agent Args: diff --git a/tools/pickman/database.py b/tools/pickman/database.py index 118ac5536fa..c8ed8a6df09 100644 --- a/tools/pickman/database.py +++ b/tools/pickman/database.py @@ -24,7 +24,7 @@ LATEST = 2 DB_FNAME = '.pickman.db' -class Database: +class Database: # pylint: disable=too-many-public-methods """Database of cherry-pick state used by pickman""" # dict of databases: @@ -248,6 +248,7 @@ class Database: # commit functions + # pylint: disable-next=too-many-arguments def commit_add(self, chash, source_id, subject, author, status='pending', mergereq_id=None): """Add a commit to the database @@ -348,6 +349,7 @@ class Database: # mergereq functions + # pylint: disable-next=too-many-arguments def mergereq_add(self, source_id, branch_name, mr_id, status, url, created_at): """Add a merge request to the database diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 8e3b91df750..b479f2e0ecd 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -3,6 +3,7 @@ # Copyright 2025 Canonical Ltd. # Written by Simon Glass # +# pylint: disable=too-many-lines """Tests for pickman.""" import argparse From patchwork Wed Dec 17 02:27:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 947 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938543; bh=264U/O3iKDxXZnbPez2HnLVZhfAHzzQ+GpWRT6aG1fQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=hSA14bRLzbIWYfgTruiqune29KJZ2+eY518VQD0jnqj6gfqEQ6WrpbGK4m7M7/fVj uJVGeRTHojE7c4JraFf2lR0MYND+mfmyc80msk2LLpBg6PeXsycHW6+u58XLyfpxMt QsSEwvv/Zp3nIFdfDTHh8MtKXxUOz5mGRtASctEqpplb/9M/bw27riAX4EPtsLyVhz po/MB+/mYoJreEfZle1PFNuOTX/ig/tCAjMwWUjnciDStml7RG5OaDwHkxbydukyl4 nQEVczhcRbDLIB/DO+LEaJ7t3eQUPH9GRa7eebjxgq5H4CVr6AyWzGfPiCKAeZ3V3k qnRjkL5OCL6YQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3020B68ABD for ; Tue, 16 Dec 2025 19:29:03 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id tvJzynN6CXES for ; Tue, 16 Dec 2025 19:29:03 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938543; bh=264U/O3iKDxXZnbPez2HnLVZhfAHzzQ+GpWRT6aG1fQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=hSA14bRLzbIWYfgTruiqune29KJZ2+eY518VQD0jnqj6gfqEQ6WrpbGK4m7M7/fVj uJVGeRTHojE7c4JraFf2lR0MYND+mfmyc80msk2LLpBg6PeXsycHW6+u58XLyfpxMt QsSEwvv/Zp3nIFdfDTHh8MtKXxUOz5mGRtASctEqpplb/9M/bw27riAX4EPtsLyVhz po/MB+/mYoJreEfZle1PFNuOTX/ig/tCAjMwWUjnciDStml7RG5OaDwHkxbydukyl4 nQEVczhcRbDLIB/DO+LEaJ7t3eQUPH9GRa7eebjxgq5H4CVr6AyWzGfPiCKAeZ3V3k qnRjkL5OCL6YQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2078C68AFF for ; Tue, 16 Dec 2025 19:29:03 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938541; bh=m1ZtOK1hUo7cymAgbtWqr3eL5ZJbozDA+5AYMyoBmqo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BgEW8pkyYr3oEWnnSMC3ix2f/2GUrq/mJ75/kmoGjQOdCqBp07M0+VAN1Pab+ryhB 1BGeO58WRG05oFCXjGZkPKWStyCpyIseCv/HExwy7ldrQL/bH15Y/22vcqwszJq6kS TDaU/sB/qxdLF/bg0GhxSAQbK6/wKTkHKW2C0ILtIBCzSXya7G8Jl34JHdXYQiErMW brUfp4LSwLT2A65pdvZpKrSqqYHAfT7a+VfENjY5BkrF+yEOPNurtuDPeXwGglnMYn Yulx9NzSXn6Rphfsgc1mqboK9sN5gzJKFCEl5cYj+GSetRZwnlxQ4hFN9SkZDaBfTS o/CWsxke4M8EQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 89AFC68ABD; Tue, 16 Dec 2025 19:29:01 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id KGWJiF_rTvjC; Tue, 16 Dec 2025 19:29:01 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938537; bh=D6lYX8N6TgiCnPuMzMQMl9i9wrw4B86NVkFFBUteINw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MPkqCB1nOLrUn0CgmVue6eOj7BwethqFkWF2w/M0tUbkO9Kcq6fg6M6fFO+6SYUXs QdejVnlsd0nukH1BNyVK71jOFxTUk5D/Vxxkgyx/PPq3EFd5KLGxzI/bMUDstFgmHu cu6rZzAKMaJvWJQIjz40OXsYt55Bv9TEQYDe5WYb0rQqgekN/ZE2UiyyinTOrnGJ87 RBqYmtRutmaYIXPzg1+1ECPQuG5T388sh0pNwpXjIEBnpCnpfaRMBrM9BEcv50X164 uLXfTv2mDHy7ZZnw4tkBqYWir6/LAYztuyNzOEI822BPh8pStOzD1RPeuwh5qMgOkt kTOT/YWbYN3Eg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 209CA6884F; Tue, 16 Dec 2025 19:28:57 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:53 -0700 Message-ID: <20251217022823.392557-5-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: MC6Y6JIC4XNYDH46QDDYO6MJWZ6ZLK2N X-Message-ID-Hash: MC6Y6JIC4XNYDH46QDDYO6MJWZ6ZLK2N X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 04/24] pickman: Complete database.py coverage List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add tests for the remaining uncovered database.py code paths: - Duplicate database creation error - Already open error when opening twice - Already closed error when closing twice - rollback() method This brings database.py to 100% test coverage. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/ftest.py | 47 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index b479f2e0ecd..cc4a5727ac4 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -314,6 +314,53 @@ class TestDatabase(unittest.TestCase): self.assertIs(dbs1, dbs2) dbs1.close() + def test_duplicate_database_error(self): + """Test creating duplicate database raises error.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + with self.assertRaises(ValueError) as ctx: + database.Database(self.db_path) + self.assertIn('already a database', str(ctx.exception)) + dbs.close() + + def test_open_already_open_error(self): + """Test opening already open database raises error.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + with self.assertRaises(ValueError) as ctx: + dbs.open_it() + self.assertIn('Already open', str(ctx.exception)) + dbs.close() + + def test_close_already_closed_error(self): + """Test closing already closed database raises error.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.close() + with self.assertRaises(ValueError) as ctx: + dbs.close() + self.assertIn('Already closed', str(ctx.exception)) + + def test_rollback(self): + """Test rollback discards uncommitted changes.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + # Make a change but don't commit + dbs.source_set('us/next', 'def456') + # Rollback should discard the change + dbs.rollback() + + result = dbs.source_get('us/next') + self.assertEqual(result, 'abc123') + dbs.close() + def test_source_get_all(self): """Test getting all sources.""" with terminal.capture(): From patchwork Wed Dec 17 02:27:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 948 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938547; bh=QWxIqUF/40dYM++oilT3wdnPLKOkBrXFLK7grB6LSX8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gifHg0yqmEKhgy0YQv5yeOicdWSvdSmmjeHohyONmHtMGCXxd7jdy6USJB9IOQ8+r nmoEoefqdloCcmRhtczMwmbzkZeu1g26qiuwA1bTK5CnesT+JfsK7PvZ0OTPTozAOF i2e0RGqZlcO/ccZuxskSSksJbfamdL69rAKnGAdtrWhUB4ct4XPnZs/SHx3J+Dv6S1 jza7abBli8g5uc0fJqUTrilGtWOeLZot/G2SVsCW+Kdfec9E+NDFOHJSxF/WrkX4FI PrchlUM9XK5E2ygJlowvrMDeWuBWwZJkRi0Da9wlxSOoGFv2/vJjxr4Y0cgwJlzh+m bJ1F/CPKoZ4oQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A7F0E68ABD for ; Tue, 16 Dec 2025 19:29:07 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id IUTKgiQu6c51 for ; Tue, 16 Dec 2025 19:29:07 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938547; bh=QWxIqUF/40dYM++oilT3wdnPLKOkBrXFLK7grB6LSX8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gifHg0yqmEKhgy0YQv5yeOicdWSvdSmmjeHohyONmHtMGCXxd7jdy6USJB9IOQ8+r nmoEoefqdloCcmRhtczMwmbzkZeu1g26qiuwA1bTK5CnesT+JfsK7PvZ0OTPTozAOF i2e0RGqZlcO/ccZuxskSSksJbfamdL69rAKnGAdtrWhUB4ct4XPnZs/SHx3J+Dv6S1 jza7abBli8g5uc0fJqUTrilGtWOeLZot/G2SVsCW+Kdfec9E+NDFOHJSxF/WrkX4FI PrchlUM9XK5E2ygJlowvrMDeWuBWwZJkRi0Da9wlxSOoGFv2/vJjxr4Y0cgwJlzh+m bJ1F/CPKoZ4oQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 94E9C68AFD for ; Tue, 16 Dec 2025 19:29:07 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938546; bh=2fk594I7H/0xARxiIa22A4PHK5nt+LQM86RA3O8/2RA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZBQ0XPPbqUnNZKhrzeymON6jBwPmoHiHU2qunEVISdlw5u1cLZ6fn7e96bfWiIh97 7IuZc4P0V0XWuyKn2uBKcM2eRM+Q48R/jv4uaJuoGRC984ejZeCajdI77lb5DVbYBP TV/5ogq+/O2Yfxs4yepkxE9C4+KtE6umc+c9B/pDWYOaEBrInm7B7srwNSoyPCX+t2 la62Tf/umwNN4rvtaAr3exmnN5yQiCfbgGyTjRLgYk7APzagiKnVwQOZYPShsh6XLj +l4ieK3OZVRk3m5NgLyaaW+E7wv0jKJYg0drcOlsszfo9kTg03jCluATPJlL3hlcEp LCiBrnsIhck3A== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 481E668ABD; Tue, 16 Dec 2025 19:29:06 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id N8gA1_yiyThn; Tue, 16 Dec 2025 19:29:06 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938542; bh=RjoAWzxGANKy0Fn5gF7QLDShn+w1g1HFbg8ozbs2hW4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BIvHPuA31g031U3N2vlxgERjEJ3+LmQ8z0PT7Z3i0wxurxKXMgpb27Etq3JRIA9jX lqGhROP7fZQrYnpcvgYFtzv7F0Go3cYJe+MQLbiORPZSNZeItFZAnn57LzmQy+8iLm LB+/VT4EIxkEsQ4j2GW2pLi9p0g/jSnQ8GKqgjtaR98Ktg3qDhDBAO4pJHPeMlmcjm sHQZPFZU5nNIBzUUFlT7+HbdOciJ3j90XHIr49PXcC/wZryVDJm0vM3GlGKYOWOvnp VOYOFnbQYBJKeBDofI1BwGjb04cJBrNuKixOxrZJc4L+AUgkRHX+69A6215lWWSizA LnU+W9Q2vgq/A== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id C8C686884F; Tue, 16 Dec 2025 19:29:01 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:54 -0700 Message-ID: <20251217022823.392557-6-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: UQ7GNRXA6MBUQS5SCDCTQHMNWZE34KXU X-Message-ID-Hash: UQ7GNRXA6MBUQS5SCDCTQHMNWZE34KXU X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 05/24] pickman: Add test coverage support List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add support for running pickman tests with code coverage checking, similar to binman. Use test_util.run_test_suites() for running tests and test_util.run_test_coverage() for coverage checking. Options added to the 'test' command: -P: Number of processes for parallel test execution -T: Run with coverage checking -v: Verbosity level (0-4) The coverage check allows failures for modules that require external services (agent.py, gitlab_api.py, control.py, database.py) since these cannot easily achieve 100% coverage in unit tests. Also update test_util.py to recognize 'pickman' as using the 'test' subcommand like binman and patman. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/__main__.py | 67 ++++++++++++++++++++++++++++++++- tools/u_boot_pylib/test_util.py | 8 ++-- 2 files changed, 70 insertions(+), 5 deletions(-) diff --git a/tools/pickman/__main__.py b/tools/pickman/__main__.py index 3d311264f06..2d2366c1b80 100755 --- a/tools/pickman/__main__.py +++ b/tools/pickman/__main__.py @@ -9,6 +9,7 @@ import argparse import os import sys +import unittest # Allow 'from pickman import xxx' to work via symlink our_path = os.path.dirname(os.path.realpath(__file__)) @@ -16,6 +17,8 @@ sys.path.insert(0, os.path.join(our_path, '..')) # pylint: disable=wrong-import-position,import-error from pickman import control +from pickman import ftest +from u_boot_pylib import test_util def parse_args(argv): @@ -80,11 +83,65 @@ def parse_args(argv): poll_cmd.add_argument('-t', '--target', default='master', help='Target branch for MR (default: master)') - subparsers.add_parser('test', help='Run tests') + test_cmd = subparsers.add_parser('test', help='Run tests') + test_cmd.add_argument('-P', '--processes', type=int, + help='Number of processes to run tests (default: all)') + test_cmd.add_argument('-T', '--test-coverage', action='store_true', + help='Run tests and check for 100%% coverage') + test_cmd.add_argument('-v', '--verbosity', type=int, default=1, + help='Verbosity level (0-4, default: 1)') + test_cmd.add_argument('tests', nargs='*', help='Specific tests to run') return parser.parse_args(argv) +def get_test_classes(): + """Get all test classes from the ftest module. + + Returns: + list: List of test class objects + """ + return [getattr(ftest, name) for name in dir(ftest) + if name.startswith('Test') and + isinstance(getattr(ftest, name), type) and + issubclass(getattr(ftest, name), unittest.TestCase)] + + +def run_tests(processes, verbosity, test_name): + """Run the pickman test suite. + + Args: + processes (int): Number of processes for concurrent tests + verbosity (int): Verbosity level (0-4) + test_name (str): Specific test to run, or None for all + + Returns: + int: 0 if tests passed, 1 otherwise + """ + result = test_util.run_test_suites( + 'pickman', False, verbosity, False, False, processes, + test_name, None, get_test_classes()) + + return 0 if result.wasSuccessful() else 1 + + +def run_test_coverage(args): + """Run tests with coverage checking. + + Args: + args (list): Specific tests to run, or None for all + """ + # agent.py and gitlab_api.py require external services (Claude, GitLab) + # so they can't achieve 100% coverage in unit tests + test_util.run_test_coverage( + 'tools/pickman/pickman', None, + ['*test*', '*__main__.py', 'tools/u_boot_pylib/*'], + None, extra_args=None, args=args, + allow_failures=['tools/pickman/agent.py', + 'tools/pickman/gitlab_api.py', + 'tools/pickman/control.py']) + + def main(argv=None): """Main function to parse args and run commands. @@ -92,6 +149,14 @@ def main(argv=None): argv (list): Command line arguments (None for sys.argv[1:]) """ args = parse_args(argv) + + if args.cmd == 'test': + if args.test_coverage: + run_test_coverage(args.tests or None) + return 0 + test_name = args.tests[0] if args.tests else None + return run_tests(args.processes, args.verbosity, test_name) + return control.do_pickman(args) diff --git a/tools/u_boot_pylib/test_util.py b/tools/u_boot_pylib/test_util.py index 7bd12705557..b1c8740d883 100644 --- a/tools/u_boot_pylib/test_util.py +++ b/tools/u_boot_pylib/test_util.py @@ -57,7 +57,8 @@ def run_test_coverage(prog, filter_fname, exclude_list, build_dir, glob_list += exclude_list glob_list += ['*libfdt.py', '*/site-packages/*', '*/dist-packages/*'] glob_list += ['*concurrencytest*'] - test_cmd = 'test' if 'binman' in prog or 'patman' in prog else '-t' + use_test = 'binman' in prog or 'patman' in prog or 'pickman' in prog + test_cmd = 'test' if use_test else '-t' prefix = '' if build_dir: prefix = 'PYTHONPATH=$PYTHONPATH:%s/sandbox_spl/tools ' % build_dir @@ -91,13 +92,12 @@ def run_test_coverage(prog, filter_fname, exclude_list, build_dir, print(coverage) if coverage != '100%': print(stdout) - print("To get a report in 'htmlcov/index.html', type: python3-coverage html") + print("To get a report in 'htmlcov/index.html', type: " + "python3-coverage html") print('Coverage error: %s, but should be 100%%' % coverage) ok = False if not ok: if allow_failures: - # for line in lines: - # print('.', line, re.match(r'^(tools/.*py) *\d+ *(\d+) *(\d+)%$', line)) lines = [re.match(r'^(tools/.*py) *\d+ *(\d+) *\d+%$', line) for line in stdout.splitlines()] bad = [] From patchwork Wed Dec 17 02:27:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 949 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938552; bh=MouFL+Yo/GBQvwRxP3L8qCJ/JPlNdH6VKjwcLFb4Azg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=aaJ0RXWK7XJy/gXNUpLRqrJMTm61+rNfozc5B88tH7a2dXRJL3yQ5Z7mlDay3rKe9 0yzxFX99KF+L7lxoLfsq80uoamqXzPR1fvzj97Hu1i4xN2ynxwN1+MaCcyCHtOSnWn XX3f9XLIhJvXveAc8/ThsAoh8bNaOKR0+oNVZiDmX1/BXOO3UrN+LTw3tIMCHf1a0w jo9PQhadtwgSECH+iJa+gILtlZa4r/8oXE1dKZRo9pPWCTOcbCP6yLxbOCxrqtvIOa 3l3HP9FPShqvfzqioNEHrc7z8ALpWqfl0JwKEknqDFc9hCOghTf+e6pwQJ9xfZqWLa +fXBokTgh7PBA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7C66B68AFF for ; Tue, 16 Dec 2025 19:29:12 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id jJgH9X4LVyh4 for ; Tue, 16 Dec 2025 19:29:12 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938552; bh=MouFL+Yo/GBQvwRxP3L8qCJ/JPlNdH6VKjwcLFb4Azg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=aaJ0RXWK7XJy/gXNUpLRqrJMTm61+rNfozc5B88tH7a2dXRJL3yQ5Z7mlDay3rKe9 0yzxFX99KF+L7lxoLfsq80uoamqXzPR1fvzj97Hu1i4xN2ynxwN1+MaCcyCHtOSnWn XX3f9XLIhJvXveAc8/ThsAoh8bNaOKR0+oNVZiDmX1/BXOO3UrN+LTw3tIMCHf1a0w jo9PQhadtwgSECH+iJa+gILtlZa4r/8oXE1dKZRo9pPWCTOcbCP6yLxbOCxrqtvIOa 3l3HP9FPShqvfzqioNEHrc7z8ALpWqfl0JwKEknqDFc9hCOghTf+e6pwQJ9xfZqWLa +fXBokTgh7PBA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6435768BA0 for ; Tue, 16 Dec 2025 19:29:12 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938550; bh=b+ude6wIEC+F7dm6nRZxnaoQ2PlqI/km93/Lg2dgAGQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u7xXjiT7b+ZBX8WLkgsufDaFEB2KvCqGBnyyyw+C5nR6CMCNdGSgHl1ucR47MuN3e pRo2hL+k5DMU00e21YM4rrHnMvLxXsfoTk2/T7FGb0qikzEPgXQcWlitryyVhkqY8t SZlIlQLxBsOxXHUBdW6kmWfiO/nCiN64IZSscLdvWyorpqtHof1UBVfdHEYTJdewQe mFWoj0Go6MfB4cFH6Eihi+AITA9XPzxKnnZa7hZH+iwTbEi9cqTkZNt5gtL4KwQLBr Xk/5YiR/cYveMJZ1+fridHwsXOekK5ay/RBklBmSDpypU8GRIFuAXSvm6nbECo/IV8 CdYz5OKVWGsVw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7C32B68ABD; Tue, 16 Dec 2025 19:29:10 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 2TFOU0KVj2yB; Tue, 16 Dec 2025 19:29:10 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938546; bh=JQZY6jpnrTdV1QHW+SwoCxqH6eMbZWrVOoEs4xV0HC4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M4lAI8nkPbFmsIxS+o2Pbl40rRCVKJ2qI/ee4xSjDkMNbWKbD8snAiNJ7M/Me0z9N qF/7aR/cwnBg+HqiqjMP8t/AvROtZ3EcYNS3llmDflqociEcn9CMmb4oX1S5ks9PoB 7FkIymoAheKgkB8YGx/RZbLHXmfpr6pN814/a2ooFx47OLMqRrj1DMA5ncq/rh3jLw IxKpMpUA+mg2pKquK1Cr3ozAOO4Q4D8yIzJg9YugRSqLsbiqeWuwjACuNC48Bv58qx 9OC4M8qd9sD9GGfmejvgS/+/9i1PFYWy/BGm3YyOwBy4FIn9ZdjbiQOOOE63QJhklY cBus2L1I+RxsA== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 8F46D6884F; Tue, 16 Dec 2025 19:29:06 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:55 -0700 Message-ID: <20251217022823.392557-7-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: MU3SE6MOIL4BJQMJXR5DH6BNMDYUP3JS X-Message-ID-Hash: MU3SE6MOIL4BJQMJXR5DH6BNMDYUP3JS X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 06/24] pickman: Add tests to improve control.py coverage List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add tests to improve test coverage for control.py: - test_poll_continues_on_step_error: Test poll warning on step error - test_format_history_summary: Test history summary formatting - test_get_next_commits_with_empty_lines: Test empty line handling - test_commit_source_resolve_error: Test commit resolution failure - test_unknown_command: Test unknown command handling - test_review_with_mrs_no_comments: Test review with open MRs This improves control.py coverage from 63% to 67%. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/ftest.py | 179 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index cc4a5727ac4..8865296ff11 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -20,6 +20,7 @@ sys.path.insert(0, os.path.join(our_path, '..')) # pylint: disable=wrong-import-position,import-error,cyclic-import from u_boot_pylib import command from u_boot_pylib import terminal +from u_boot_pylib import tout from pickman import __main__ as pickman from pickman import control @@ -1363,6 +1364,184 @@ class TestPoll(unittest.TestCase): self.assertEqual(ret, 0) self.assertEqual(call_count[0], 2) + def test_poll_continues_on_step_error(self): + """Test poll continues when step returns non-zero.""" + call_count = [0] + + def mock_step(_args, _dbs): + call_count[0] += 1 + if call_count[0] >= 2: + raise KeyboardInterrupt + return 1 # Return error + + with mock.patch.object(control, 'do_step', mock_step): + with mock.patch('time.sleep'): + args = argparse.Namespace( + cmd='poll', source='us/next', interval=1, + remote='ci', target='master' + ) + with terminal.capture() as (_, stderr): + ret = control.do_poll(args, None) + + self.assertEqual(ret, 0) + self.assertIn('returned 1', stderr.getvalue()) + + +class TestFormatHistorySummary(unittest.TestCase): + """Tests for format_history_summary function.""" + + def test_format_history_summary(self): + """Test formatting history summary.""" + commits = [ + control.CommitInfo('aaa111', 'aaa111a', 'First commit', 'Author 1'), + control.CommitInfo('bbb222', 'bbb222b', 'Second commit', 'Author 2'), + ] + result = control.format_history_summary('us/next', commits, 'cherry-abc') + + self.assertIn('us/next', result) + self.assertIn('Branch: cherry-abc', result) + self.assertIn('- aaa111a First commit', result) + self.assertIn('- bbb222b Second commit', result) + + +class TestGetNextCommitsEmptyLine(unittest.TestCase): + """Tests for get_next_commits with empty lines.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_get_next_commits_with_empty_lines(self): + """Test get_next_commits handles empty lines in output.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + # Log output with empty lines + log_output = ( + 'aaa111|aaa111a|Author 1|First commit|abc123\n' + '\n' # Empty line + 'bbb222|bbb222b|Author 2|Second commit|aaa111\n' + '\n' # Another empty line + ) + command.TEST_RESULT = command.CommandResult(stdout=log_output) + + commits, merge_found, error = control.get_next_commits(dbs, + 'us/next') + self.assertIsNone(error) + self.assertFalse(merge_found) + self.assertEqual(len(commits), 2) + dbs.close() + + +class TestDoCommitSourceResolveError(unittest.TestCase): + """Tests for do_commit_source error handling.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_commit_source_resolve_error(self): + """Test commit-source fails when commit can't be resolved.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'oldcommit12345') + dbs.commit() + + database.Database.instances.clear() + + def mock_git_fail(**_kwargs): + raise command.CommandExc('git error', command.CommandResult()) + + command.TEST_RESULT = mock_git_fail + + args = argparse.Namespace(cmd='commit-source', source='us/next', + commit='badcommit') + with terminal.capture() as (_, stderr): + ret = control.do_commit_source(args, None) + self.assertEqual(ret, 1) + self.assertIn('Could not resolve', stderr.getvalue()) + + +class TestDoPickmanUnknownCommand(unittest.TestCase): + """Tests for do_pickman with unknown command.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + + def test_unknown_command(self): + """Test do_pickman returns 1 for unknown command.""" + args = argparse.Namespace(cmd='unknown-command') + with terminal.capture(): + ret = control.do_pickman(args) + self.assertEqual(ret, 1) + + +class TestDoReviewWithMrs(unittest.TestCase): + """Tests for do_review with open MRs.""" + + def test_review_with_mrs_no_comments(self): + """Test review with open MRs but no comments.""" + tout.init(tout.INFO) + + mock_mr = { + 'iid': 123, + 'title': '[pickman] Test MR', + 'web_url': 'https://gitlab.com/mr/123', + } + with mock.patch.object(gitlab_api, 'get_open_pickman_mrs', + return_value=[mock_mr]): + with mock.patch.object(gitlab_api, 'get_mr_comments', + return_value=[]): + args = argparse.Namespace(cmd='review', remote='ci') + with terminal.capture() as (stdout, _): + ret = control.do_review(args, None) + + self.assertEqual(ret, 0) + self.assertIn('Found 1 open pickman MR', stdout.getvalue()) + if __name__ == '__main__': unittest.main() From patchwork Wed Dec 17 02:27:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 950 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938557; bh=/vq55TQQEX6y2TTJemQSG/VPqyYi70HBQUWIsjpFhtw=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=RZgksTbXsLqPrFKPON4wQzNA5buY2t+MEroDolw/aKfIK2mY7HsiwElRNOAv6enXL RlzJnGMZDsLo/7yFnXbso/5NwFt7EU1gIGMT0sdpvBoY8YKybvWQV8qnx4MzxAT6e9 Hc+P4l630nLdLLeHeEcxni+xmV5uEG5dyvsUv0Zszc6kO4VQBZ+aS9yRSV2MvEAlhA SNhQilP/VAogwUHSJH5QBBqI0zn3VW7PCe9WI5XNvBt2exMAtpc+tBQU0lX34S7Tr2 LYCOzzhhSwzLSspzKGYbQc62TfxTTLUUAv7h1RY8cPaNEQxwuXn+Po8esxXrSh6nM7 hAQ24fil8ZIVQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1CE0D68BBD for ; Tue, 16 Dec 2025 19:29:17 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 3sdsNbEuv_6H for ; Tue, 16 Dec 2025 19:29:17 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938557; bh=/vq55TQQEX6y2TTJemQSG/VPqyYi70HBQUWIsjpFhtw=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=RZgksTbXsLqPrFKPON4wQzNA5buY2t+MEroDolw/aKfIK2mY7HsiwElRNOAv6enXL RlzJnGMZDsLo/7yFnXbso/5NwFt7EU1gIGMT0sdpvBoY8YKybvWQV8qnx4MzxAT6e9 Hc+P4l630nLdLLeHeEcxni+xmV5uEG5dyvsUv0Zszc6kO4VQBZ+aS9yRSV2MvEAlhA SNhQilP/VAogwUHSJH5QBBqI0zn3VW7PCe9WI5XNvBt2exMAtpc+tBQU0lX34S7Tr2 LYCOzzhhSwzLSspzKGYbQc62TfxTTLUUAv7h1RY8cPaNEQxwuXn+Po8esxXrSh6nM7 hAQ24fil8ZIVQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0BD9568BA0 for ; Tue, 16 Dec 2025 19:29:17 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938555; bh=I22Jx3PuTDhRf2d0pBVERz3cYhi4nZK4jRxzgGIfDNo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bRFN16l0uGAvdcKx8ghN4EFfz5yjJQTbVsuyltOKnAeG3mbZ6vMIc5WOirfMEBtLV MWGK8dORkyVIZ0R6hf0HnYtZlB81Fq0FiFBeZw/Pqj25j2wbEUQMrhLiwj4ANUGHDt /SI/d0+zgh6xYEtY4sv2gNeg+XdqJzb7mmTH+zxbEvnt9AXlgHdYJQqQPqiAVzz3Co hOQ/k0z8KAVaqz0INpxXxrEvP4DKVVdZNZov6mavtN6GX3lGLZxFF2yZFeePF8gZkO BuonNF8x1JcDqf2vB2gtwgwcWo7PhxjaaXI354StthfgGMKcwfftMHa3xt3vbsNwd6 3aeqsGX9loL/w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3D0D268BB6; Tue, 16 Dec 2025 19:29:15 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id vSorHocJeVu4; Tue, 16 Dec 2025 19:29:15 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938551; bh=V2Vv+k5UwnQLOLQOFKNzMHfAWO0hIQoOkzmtdnTBwto=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u6TiV4VmVbsBk3BjfgQYyWPkM1uLHtR7FEV0hGHlSFtKqB3y4qZuYJxVn9qHsk2/7 mcsmqTUVAkoi1tV3Yyo16SuzIB6gfKJXBn+wBDjxFnbMWV9olHfbJ2j7XMmOTHuaAg XkWH8ZAAswOVjI+eY7iR0qEwEQUJ4o165UCxHOJ4vkuLI+/wRH5mBmTNgORoainYUN H8rFvC9Syc19JBXj02p8Sh4mUnvpLapVRfT+jm/tjQ1ltAvWGJG2WwapVgFzw0hZLX K8sE2npvNGdmP9Qti3ccJx2mvtWDsdfo6+cJTbTViTP01styUvZbQZZtY/lZXn5Ff9 0BRw2Op2x+hUQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id BF3FF6884F; Tue, 16 Dec 2025 19:29:10 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:56 -0700 Message-ID: <20251217022823.392557-8-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: GKE7XKBI3W6S5MKHUVEIS33F2NYSFGYZ X-Message-ID-Hash: GKE7XKBI3W6S5MKHUVEIS33F2NYSFGYZ X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 07/24] pickman: Improve testing with write_history() List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Extract get_history() from write_history() to separate the history content generation and file I/O from the git operations. The function now: - Takes a filename parameter - Reads existing content from the file - Writes updated content to the file - Returns both the content and the git commit message This makes the history generation logic fully testable without needing to mock git operations. Add tests for get_history(): - test_get_history_empty: Test with no existing file - test_get_history_with_existing: Test appending to existing content - test_get_history_replaces_existing_branch: Test replacing entry - test_get_history_multiple_commits: Test commit message with multiple commits Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 44 +++++++++++---- tools/pickman/ftest.py | 114 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 148 insertions(+), 10 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 474315f7db0..190f92fc57a 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -258,14 +258,19 @@ Commits: {commit_list}""" -def write_history(source, commits, branch_name, conversation_log): - """Write an entry to the pickman history file +def get_history(fname, source, commits, branch_name, conversation_log): + """Read, update and write history file for a cherry-pick operation Args: + fname (str): History filename to read/write source (str): Source branch name commits (list): list of CommitInfo tuples branch_name (str): Name of the cherry-pick branch conversation_log (str): The agent's conversation output + + Returns: + tuple: (content, commit_msg) where content is the updated history + and commit_msg is the git commit message """ summary = format_history_summary(source, commits, branch_name) entry = f"""{summary} @@ -277,24 +282,43 @@ def write_history(source, commits, branch_name, conversation_log): """ - # Read existing content and remove any entry for this branch + # Read existing content existing = '' - if os.path.exists(HISTORY_FILE): - with open(HISTORY_FILE, 'r', encoding='utf-8') as fhandle: + if os.path.exists(fname): + with open(fname, 'r', encoding='utf-8') as fhandle: existing = fhandle.read() # Remove existing entry for this branch (from ## header to ---) pattern = rf'## [^\n]+\n\nBranch: {re.escape(branch_name)}\n.*?---\n\n' existing = re.sub(pattern, '', existing, flags=re.DOTALL) + content = existing + entry + # Write updated history file - with open(HISTORY_FILE, 'w', encoding='utf-8') as fhandle: - fhandle.write(existing + entry) + with open(fname, 'w', encoding='utf-8') as fhandle: + fhandle.write(content) + + # Generate commit message + commit_msg = f'pickman: Record cherry-pick of {len(commits)} commits from {source}\n\n' + commit_msg += '\n'.join(f'- {c.short_hash} {c.subject}' for c in commits) + + return content, commit_msg + + +def write_history(source, commits, branch_name, conversation_log): + """Write an entry to the pickman history file and commit it + + Args: + source (str): Source branch name + commits (list): list of CommitInfo tuples + branch_name (str): Name of the cherry-pick branch + conversation_log (str): The agent's conversation output + """ + _, commit_msg = get_history(HISTORY_FILE, source, commits, branch_name, + conversation_log) # Commit the history file (use -f in case .gitignore patterns match) run_git(['add', '-f', HISTORY_FILE]) - msg = f'pickman: Record cherry-pick of {len(commits)} commits from {source}\n\n' - msg += '\n'.join(f'- {c.short_hash} {c.subject}' for c in commits) - run_git(['commit', '-m', msg]) + run_git(['commit', '-m', commit_msg]) tout.info(f'Updated {HISTORY_FILE}') diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 8865296ff11..d50be16fea7 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1404,6 +1404,120 @@ class TestFormatHistorySummary(unittest.TestCase): self.assertIn('- bbb222b Second commit', result) +class TestGetHistory(unittest.TestCase): + """Tests for get_history function.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.history_file = tempfile.mkstemp(suffix='.history') + os.close(fd) + os.unlink(self.history_file) # Remove so we start fresh + + def tearDown(self): + """Clean up test fixtures.""" + if os.path.exists(self.history_file): + os.unlink(self.history_file) + + def test_get_history_empty(self): + """Test get_history with no existing file.""" + commits = [ + control.CommitInfo('aaa111', 'aaa111a', 'First commit', 'Author 1'), + ] + content, commit_msg = control.get_history( + self.history_file, 'us/next', commits, 'cherry-abc', + 'Conversation output') + + self.assertIn('us/next', content) + self.assertIn('Branch: cherry-abc', content) + self.assertIn('- aaa111a First commit', content) + self.assertIn('### Conversation log', content) + self.assertIn('Conversation output', content) + self.assertIn('---', content) + + # Verify commit message + self.assertIn('pickman: Record cherry-pick of 1 commits', commit_msg) + self.assertIn('- aaa111a First commit', commit_msg) + + # Verify file was written + with open(self.history_file, 'r', encoding='utf-8') as fhandle: + file_content = fhandle.read() + self.assertEqual(file_content, content) + + def test_get_history_with_existing(self): + """Test get_history appends to existing content.""" + # Create existing file + with open(self.history_file, 'w', encoding='utf-8') as fhandle: + fhandle.write('Previous history content\n') + + commits = [ + control.CommitInfo('bbb222', 'bbb222b', 'New commit', 'Author 2'), + ] + content, commit_msg = control.get_history( + self.history_file, 'us/next', commits, 'cherry-new', + 'New conversation') + + self.assertIn('Previous history content', content) + self.assertIn('cherry-new', content) + self.assertIn('New conversation', content) + self.assertIn('- bbb222b New commit', commit_msg) + + def test_get_history_replaces_existing_branch(self): + """Test get_history removes existing entry for same branch.""" + # Create existing file with an entry for cherry-abc + existing = """## 2025-01-01: us/next + +Branch: cherry-abc + +Commits: +- aaa111a Old commit + +### Conversation log +Old conversation + +--- + +Other content +""" + with open(self.history_file, 'w', encoding='utf-8') as fhandle: + fhandle.write(existing) + + commits = [ + control.CommitInfo('ccc333', 'ccc333c', 'Updated commit', 'Author'), + ] + content, _ = control.get_history(self.history_file, 'us/next', commits, + 'cherry-abc', 'New conversation') + + # Old entry should be removed + self.assertNotIn('Old commit', content) + self.assertNotIn('Old conversation', content) + # New entry should be present + self.assertIn('Updated commit', content) + self.assertIn('New conversation', content) + # Other content should be preserved + self.assertIn('Other content', content) + + def test_get_history_multiple_commits(self): + """Test get_history with multiple commits.""" + commits = [ + control.CommitInfo('aaa111', 'aaa111a', 'First commit', 'Author 1'), + control.CommitInfo('bbb222', 'bbb222b', 'Second commit', 'Author 2'), + control.CommitInfo('ccc333', 'ccc333c', 'Third commit', 'Author 3'), + ] + content, commit_msg = control.get_history( + self.history_file, 'us/next', commits, 'cherry-abc', 'Log') + + # Verify all commits in content + self.assertIn('- aaa111a First commit', content) + self.assertIn('- bbb222b Second commit', content) + self.assertIn('- ccc333c Third commit', content) + + # Verify commit message + self.assertIn('pickman: Record cherry-pick of 3 commits', commit_msg) + self.assertIn('- aaa111a First commit', commit_msg) + self.assertIn('- bbb222b Second commit', commit_msg) + self.assertIn('- ccc333c Third commit', commit_msg) + + class TestGetNextCommitsEmptyLine(unittest.TestCase): """Tests for get_next_commits with empty lines.""" From patchwork Wed Dec 17 02:27:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 951 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938562; bh=4Bi/EZTK9dI2Qc1n3vOwDJwgzprYFV2ae4qZxAIAQTM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=JgX3tCM6I4ILwPfVbkhip1xlPNOHVOorwIPUIoGWFhNj5CWMYj8TseT45livM6ybm 6zXUfjyI4PVdpOV3HN4mI07p7xuJrNSOU7w3gZq25NAF0GJRRfwQpW9n2OsqT4FmTk Cf7kzeqOfn+HG8sptiI4AzAxuEHuQihIJ4Dc6RBRydAgYSSQbx1rFtF1Qke0WRN4b6 ktLlNhKNgW3Ydd5NiISCaYtfkr0mZWFb/3r9sc0pAjjaOylTTSO1vmFG5y81v7Y6Bi RwZ6LQlMgR+kaI7xXp7PlhGKhpHFJvzgTVP0BD9/dI17J7OzfXkDwopSooIqjfbJGJ 0YzUGtBcWV2MQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CB96268BBD for ; Tue, 16 Dec 2025 19:29:22 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id sWPET484DIhf for ; Tue, 16 Dec 2025 19:29:22 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938562; bh=4Bi/EZTK9dI2Qc1n3vOwDJwgzprYFV2ae4qZxAIAQTM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=JgX3tCM6I4ILwPfVbkhip1xlPNOHVOorwIPUIoGWFhNj5CWMYj8TseT45livM6ybm 6zXUfjyI4PVdpOV3HN4mI07p7xuJrNSOU7w3gZq25NAF0GJRRfwQpW9n2OsqT4FmTk Cf7kzeqOfn+HG8sptiI4AzAxuEHuQihIJ4Dc6RBRydAgYSSQbx1rFtF1Qke0WRN4b6 ktLlNhKNgW3Ydd5NiISCaYtfkr0mZWFb/3r9sc0pAjjaOylTTSO1vmFG5y81v7Y6Bi RwZ6LQlMgR+kaI7xXp7PlhGKhpHFJvzgTVP0BD9/dI17J7OzfXkDwopSooIqjfbJGJ 0YzUGtBcWV2MQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id BA39168BA0 for ; Tue, 16 Dec 2025 19:29:22 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938560; bh=VVnlYhTnsPpq7EaaVJQ9Bw/2EPuMvyM7zp/4zOZeobM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tVvmBD7bgN6B+1iwBBQT2/vZ8KSGBLaqw3WLxeWPUVFFIk2j8Agf53vWQg9F3Uhcz 44zcVPMEhHYZije5Of5glkt+OVrLoN29QnPnJEqnRKrL14Jqv1ETSPBy4b2P1rpasT TvdfnrMPsaZkPHyf+Qztn4tAmraAduGFHiL+qNvYpMEpZV5Sz+tKaZTp9R45sAq9p/ /zPTodMuV3I7YKPRLR2tm8/6CbcCx4qTt/RCuDw1Lzm0FLClkNcpWGW2/rN00LFTw8 jWiR9YQhwJvPqrAFq2cyHgkImloqGuAEHGnwdURM1buwdUgh16F1M5XvEEonEsRCUE nTVtETwce2EIw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0AFA368AFF; Tue, 16 Dec 2025 19:29:20 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id R0ZT89k4Tr6S; Tue, 16 Dec 2025 19:29:19 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938555; bh=5NCmHYLULdlaK1npxnTY/07e0Hyn+qlU/hH1gTvGATE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=suuZmJpwUeeHs2M4sPgwE1nHJ83oh3BFnD7Zbp7f2Z+j8EhFsk92RQvlSbVNu0ALL t/b1So0e/XPvyPuDayf0DN/VkFfDg/edpSdNFPSFdoVzleZu4Q0QGCiUIH5XI5WUL6 UyGo4gTjnWfuvw5NfAIK9vvk5BCt8etgOHTWq2kb10pYl1ofQlfbyERUevtCICE7vr 2oP4VULOkajw4SnF3R2lr4qjJtUq9lZFLPY+3D8bx+Ke6WX8DqweHdND5IwJ7ZcJIw ysxeES7XRVae0vUcaiitKOyJcnjBaeu7p5MEOgIPSMOuNqXQbFo9lJLSQqImQMTNto RcJkpQjyfET1Q== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 83AEC6884F; Tue, 16 Dec 2025 19:29:15 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:57 -0700 Message-ID: <20251217022823.392557-9-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: LI4OWPHX55SUP4NRUZB3VPAJADNQT53V X-Message-ID-Hash: LI4OWPHX55SUP4NRUZB3VPAJADNQT53V X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 08/24] pickman: Extract prepare_apply() from do_apply() List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Extract the setup logic from do_apply() into prepare_apply() which: - Gets the next commits from the source branch - Validates the source exists and has commits - Saves the original branch - Generates or uses provided branch name - Deletes existing branch if needed - Prints info about what will be applied Returns (ApplyInfo, return_code) tuple where ApplyInfo contains commits, branch_name, original_branch, and merge_found. This makes the setup logic testable independently from the agent and git operations. Add tests for prepare_apply(): - test_prepare_apply_error: Test error handling for unknown source - test_prepare_apply_no_commits: Test when no commits to apply - test_prepare_apply_with_commits: Test successful preparation - test_prepare_apply_custom_branch: Test custom branch name Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 48 ++++++++++++++--- tools/pickman/ftest.py | 108 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 146 insertions(+), 10 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 190f92fc57a..aa3fd65ec8e 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -39,6 +39,11 @@ Commit = namedtuple('Commit', ['hash', 'short_hash', 'subject', 'date']) CommitInfo = namedtuple('CommitInfo', ['hash', 'short_hash', 'subject', 'author']) +# Named tuple for prepare_apply result +ApplyInfo = namedtuple('ApplyInfo', + ['commits', 'branch_name', 'original_branch', + 'merge_found']) + def run_git(args): """Run a git command and return output.""" @@ -323,32 +328,37 @@ def write_history(source, commits, branch_name, conversation_log): tout.info(f'Updated {HISTORY_FILE}') -def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches - """Apply the next set of commits using Claude agent +def prepare_apply(dbs, source, branch): + """Prepare for applying commits from a source branch + + Gets the next commits, sets up the branch name, and prints info about + what will be applied. Args: - args (Namespace): Parsed arguments with 'source' and 'branch' attributes dbs (Database): Database instance + source (str): Source branch name + branch (str): Branch name to use, or None to auto-generate Returns: - int: 0 on success, 1 on failure + tuple: (ApplyInfo, return_code) where ApplyInfo is set if there are + commits to apply, or None with return_code indicating the result + (0 for no commits, 1 for error) """ - source = args.source commits, merge_found, error = get_next_commits(dbs, source) if error: tout.error(error) - return 1 + return None, 1 if not commits: tout.info('No new commits to cherry-pick') - return 0 + return None, 0 # Save current branch to return to later original_branch = run_git(['rev-parse', '--abbrev-ref', 'HEAD']) # Generate branch name if not provided - branch_name = args.branch + branch_name = branch if not branch_name: # Use first commit's short hash as part of branch name branch_name = f'cherry-{commits[0].short_hash}' @@ -372,6 +382,28 @@ def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches tout.info(f' {commit.short_hash} {commit.subject}') tout.info('') + return ApplyInfo(commits, branch_name, original_branch, merge_found), 0 + + +def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches + """Apply the next set of commits using Claude agent + + Args: + args (Namespace): Parsed arguments with 'source' and 'branch' attributes + dbs (Database): Database instance + + Returns: + int: 0 on success, 1 on failure + """ + source = args.source + info, ret = prepare_apply(dbs, source, args.branch) + if not info: + return ret + + commits = info.commits + branch_name = info.branch_name + original_branch = info.original_branch + # Add commits to database with 'pending' status source_id = dbs.source_get_id(source) for commit in commits: diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index d50be16fea7..1418211d4ae 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -976,7 +976,7 @@ class TestApply(unittest.TestCase): database.Database.instances.clear() - args = argparse.Namespace(cmd='apply', source='unknown') + args = argparse.Namespace(cmd='apply', source='unknown', branch=None) with terminal.capture() as (_, stderr): ret = control.do_pickman(args) self.assertEqual(ret, 1) @@ -994,7 +994,7 @@ class TestApply(unittest.TestCase): database.Database.instances.clear() command.TEST_RESULT = command.CommandResult(stdout='') - args = argparse.Namespace(cmd='apply', source='us/next') + args = argparse.Namespace(cmd='apply', source='us/next', branch=None) with terminal.capture() as (stdout, _): ret = control.do_pickman(args) self.assertEqual(ret, 0) @@ -1518,6 +1518,110 @@ Other content self.assertIn('- ccc333c Third commit', commit_msg) +class TestPrepareApply(unittest.TestCase): + """Tests for prepare_apply function.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_prepare_apply_error(self): + """Test prepare_apply returns error code 1 on source not found.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + + info, ret = control.prepare_apply(dbs, 'unknown', None) + + self.assertIsNone(info) + self.assertEqual(ret, 1) + dbs.close() + + def test_prepare_apply_no_commits(self): + """Test prepare_apply returns code 0 when no commits.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + command.TEST_RESULT = command.CommandResult(stdout='') + + info, ret = control.prepare_apply(dbs, 'us/next', None) + + self.assertIsNone(info) + self.assertEqual(ret, 0) + dbs.close() + + def test_prepare_apply_with_commits(self): + """Test prepare_apply returns ApplyInfo with commits.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + log_output = 'aaa111|aaa111a|Author 1|First commit|abc123\n' + + def mock_git(pipe_list): + cmd = pipe_list[0] if pipe_list else [] + if 'log' in cmd: + return command.CommandResult(stdout=log_output) + if 'rev-parse' in cmd: + return command.CommandResult(stdout='master') + return command.CommandResult(stdout='') + + command.TEST_RESULT = mock_git + + info, ret = control.prepare_apply(dbs, 'us/next', None) + + self.assertIsNotNone(info) + self.assertEqual(ret, 0) + self.assertEqual(len(info.commits), 1) + self.assertEqual(info.branch_name, 'cherry-aaa111a') + self.assertEqual(info.original_branch, 'master') + dbs.close() + + def test_prepare_apply_custom_branch(self): + """Test prepare_apply uses custom branch name.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + log_output = 'aaa111|aaa111a|Author 1|First commit|abc123\n' + + def mock_git(pipe_list): + cmd = pipe_list[0] if pipe_list else [] + if 'log' in cmd: + return command.CommandResult(stdout=log_output) + if 'rev-parse' in cmd: + return command.CommandResult(stdout='master') + return command.CommandResult(stdout='') + + command.TEST_RESULT = mock_git + + info, _ = control.prepare_apply(dbs, 'us/next', 'my-branch') + + self.assertIsNotNone(info) + self.assertEqual(info.branch_name, 'my-branch') + dbs.close() + + class TestGetNextCommitsEmptyLine(unittest.TestCase): """Tests for get_next_commits with empty lines.""" From patchwork Wed Dec 17 02:27:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 952 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938567; bh=TfHuNORe+hVT9KlXTtBoulrl9NKytmVH0kiqULkf+vM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=V+ylsU9MphYNxw8W065vQ+e7NSAdM+QvLo+8LfmhoIT7mzvFquAXxGEjpGaM4giIX 64doa5FlnFZKV2x2Rv9agDpsFkC06UCN4dROFMgNjoCGAdkAhKlikBipQpJp29tnEy lfWnqjFVuVI9975BIzoHSPF9AMhmtJFu2vUeESCN+aLsniGzEP5TyBBkSZit+gt4YT iteM/aW2WP0eIynEIKrA1Pm58Jh1QSGMvM4i1MEd+uOEUHAilv3Q7134BLlU+7BITq hnxrxUBYMVAAf7DzpptMJbFNlBDwPN2a8jDxuLWvB2SvvAowYqHDJPm7ywAHJr8wnD kfuJg93zxrOhQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4E3F968BC1 for ; Tue, 16 Dec 2025 19:29:27 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id WGm6l8pHhY63 for ; Tue, 16 Dec 2025 19:29:27 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938567; bh=TfHuNORe+hVT9KlXTtBoulrl9NKytmVH0kiqULkf+vM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=V+ylsU9MphYNxw8W065vQ+e7NSAdM+QvLo+8LfmhoIT7mzvFquAXxGEjpGaM4giIX 64doa5FlnFZKV2x2Rv9agDpsFkC06UCN4dROFMgNjoCGAdkAhKlikBipQpJp29tnEy lfWnqjFVuVI9975BIzoHSPF9AMhmtJFu2vUeESCN+aLsniGzEP5TyBBkSZit+gt4YT iteM/aW2WP0eIynEIKrA1Pm58Jh1QSGMvM4i1MEd+uOEUHAilv3Q7134BLlU+7BITq hnxrxUBYMVAAf7DzpptMJbFNlBDwPN2a8jDxuLWvB2SvvAowYqHDJPm7ywAHJr8wnD kfuJg93zxrOhQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3D20C68AEE for ; Tue, 16 Dec 2025 19:29:27 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938565; bh=7Lgp7Ym0Y1r9icJKoDqYKUansLwq6zP65g0NMpkuwqE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t1Oz5lB4X2X+VeP3e19/bkG2dNRNCfJL6MLaoHrUU7UGZR5iyVZZSaaW8nBcxbyNn WYI/w6paFEknkikNkA/mq+5rLhlMEQDFw3o3oy6Amc319yCKhD7a04UijJf8o1IPGh 58QINdL6NNYkSB2ZzP2YQH3XMp8ZyZq2YJ9u4eARC0XHNxbhRNpChrNP5Ln/JKOaRO CP02+bKpHX5eSp7FMC2KtVJ2sUH7it0XW6+OVOO1glMIcXZaEtPLp6xrqAy5/VY33O R3SoLgM2HcpRGe/YaClk91sJp3Wjfc7/mpYcm1h8kNLyabdHdT9eJzHDlLIBkNw0Jj I/3UF5hoqk53w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4C52968AFF; Tue, 16 Dec 2025 19:29:25 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id l6JwWqhVqtib; Tue, 16 Dec 2025 19:29:25 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938560; bh=67D4a4D8HVspCkGpvvHibA6bnrt7uBt4Zobfjl8EXP0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dvqhxt9LE2flVyBQ09StJ87fKNZM46CxDKUAMzjc04l06BECn6+bGJRM2jdpSS1cb +IdBxul1ixztRCw3SQl2+9BFSC4rJemHiAKRWQQ/cydBkGinY4Dkc5f9Vs6Cx6zUiU IYsAgkgw+MCjUDOpAm02dH+p0OiW1qvpSCZZb99KQ/VF61d/RN5Qpd+/sdILUf4cNz sBuWYQ+CjXcp77imLfwqFQRCWK8+w0xc/28BHDmoAO3S0sD3Q6xxlf8ThsDxX7cZ7J AmpKyTEQMdcSsiqeuE4sbcUZ/lQ2TVVbei6HULvCwQFL573+idwe4hqq13YfNiwUk0 6/nGfgdmhEDLQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 4D8286884F; Tue, 16 Dec 2025 19:29:20 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:58 -0700 Message-ID: <20251217022823.392557-10-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: JBO6O7YM2DAD47F5G64G6JSRLPVOKAZO X-Message-ID-Hash: JBO6O7YM2DAD47F5G64G6JSRLPVOKAZO X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 09/24] pickman: Extract execute_apply() from do_apply() List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Extract the core 'apply' logic into execute_apply() which handles database operations, agent invocation, and MR creation. This separates the non-git logic from do_apply() which now handles setup, history writing, and branch restoration. Also move branch restoration to the end of do_apply() using a ret variable to ensure we always restore the branch even if MR creation fails. Add tests for execute_apply() covering success, failure, push, and push failure cases. Rename conversation_log to conv_log for brevity. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 86 +++++++++++++++++----------- tools/pickman/ftest.py | 120 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 174 insertions(+), 32 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index aa3fd65ec8e..973db22e106 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -263,7 +263,7 @@ Commits: {commit_list}""" -def get_history(fname, source, commits, branch_name, conversation_log): +def get_history(fname, source, commits, branch_name, conv_log): """Read, update and write history file for a cherry-pick operation Args: @@ -271,7 +271,7 @@ def get_history(fname, source, commits, branch_name, conversation_log): source (str): Source branch name commits (list): list of CommitInfo tuples branch_name (str): Name of the cherry-pick branch - conversation_log (str): The agent's conversation output + conv_log (str): The agent's conversation output Returns: tuple: (content, commit_msg) where content is the updated history @@ -281,7 +281,7 @@ def get_history(fname, source, commits, branch_name, conversation_log): entry = f"""{summary} ### Conversation log -{conversation_log} +{conv_log} --- @@ -309,17 +309,17 @@ def get_history(fname, source, commits, branch_name, conversation_log): return content, commit_msg -def write_history(source, commits, branch_name, conversation_log): +def write_history(source, commits, branch_name, conv_log): """Write an entry to the pickman history file and commit it Args: source (str): Source branch name commits (list): list of CommitInfo tuples branch_name (str): Name of the cherry-pick branch - conversation_log (str): The agent's conversation output + conv_log (str): The agent's conversation output """ _, commit_msg = get_history(HISTORY_FILE, source, commits, branch_name, - conversation_log) + conv_log) # Commit the history file (use -f in case .gitignore patterns match) run_git(['add', '-f', HISTORY_FILE]) @@ -385,25 +385,20 @@ def prepare_apply(dbs, source, branch): return ApplyInfo(commits, branch_name, original_branch, merge_found), 0 -def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches - """Apply the next set of commits using Claude agent +def execute_apply(dbs, source, commits, branch_name, args): # pylint: disable=too-many-locals + """Execute the apply operation: run agent, update database, push MR Args: - args (Namespace): Parsed arguments with 'source' and 'branch' attributes dbs (Database): Database instance + source (str): Source branch name + commits (list): List of CommitInfo namedtuples + branch_name (str): Branch name for cherry-picks + args (Namespace): Parsed arguments with 'push', 'remote', 'target' Returns: - int: 0 on success, 1 on failure + tuple: (ret, success, conv_log) where ret is 0 on success, + 1 on failure """ - source = args.source - info, ret = prepare_apply(dbs, source, args.branch) - if not info: - return ret - - commits = info.commits - branch_name = info.branch_name - original_branch = info.original_branch - # Add commits to database with 'pending' status source_id = dbs.source_get_id(source) for commit in commits: @@ -413,7 +408,7 @@ def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches # Convert CommitInfo to tuple format expected by agent commit_tuples = [(c.hash, c.short_hash, c.subject) for c in commits] - success, conversation_log = agent.cherry_pick_commits(commit_tuples, source, + success, conv_log = agent.cherry_pick_commits(commit_tuples, source, branch_name) # Update commit status based on result @@ -422,15 +417,7 @@ def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches dbs.commit_set_status(commit.hash, status) dbs.commit() - # Write history file if successful - if success: - write_history(source, commits, branch_name, conversation_log) - - # Return to original branch - current_branch = run_git(['rev-parse', '--abbrev-ref', 'HEAD']) - if current_branch != original_branch: - tout.info(f'Returning to {original_branch}') - run_git(['checkout', original_branch]) + ret = 0 if success else 1 if success: # Push and create MR if requested @@ -441,18 +428,53 @@ def do_apply(args, dbs): # pylint: disable=too-many-locals,too-many-branches title = f'[pickman] {commits[-1].subject}' # Description matches .pickman-history entry (summary + conversation) summary = format_history_summary(source, commits, branch_name) - description = f'{summary}\n\n### Conversation log\n{conversation_log}' + description = f'{summary}\n\n### Conversation log\n{conv_log}' mr_url = gitlab_api.push_and_create_mr( remote, branch_name, target, title, description ) if not mr_url: - return 1 + ret = 1 else: tout.info(f"Use 'pickman commit-source {source} " f"{commits[-1].short_hash}' to update the database") - return 0 if success else 1 + return ret, success, conv_log + + +def do_apply(args, dbs): + """Apply the next set of commits using Claude agent + + Args: + args (Namespace): Parsed arguments with 'source' and 'branch' attributes + dbs (Database): Database instance + + Returns: + int: 0 on success, 1 on failure + """ + source = args.source + info, ret = prepare_apply(dbs, source, args.branch) + if not info: + return ret + + commits = info.commits + branch_name = info.branch_name + original_branch = info.original_branch + + ret, success, conv_log = execute_apply(dbs, source, commits, + branch_name, args) + + # Write history file if successful + if success: + write_history(source, commits, branch_name, conv_log) + + # Return to original branch + current_branch = run_git(['rev-parse', '--abbrev-ref', 'HEAD']) + if current_branch != original_branch: + tout.info(f'Returning to {original_branch}') + run_git(['checkout', original_branch]) + + return ret def do_commit_source(args, dbs): diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 1418211d4ae..9c65652f27a 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1622,6 +1622,126 @@ class TestPrepareApply(unittest.TestCase): dbs.close() +class TestExecuteApply(unittest.TestCase): + """Tests for execute_apply function.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + + def test_execute_apply_success(self): + """Test execute_apply with successful cherry-pick.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + commits = [control.CommitInfo('aaa111', 'aaa111a', 'Test commit', + 'Author')] + args = argparse.Namespace(push=False) + + with mock.patch.object(control.agent, 'cherry_pick_commits', + return_value=(True, 'conversation log')): + ret, success, conv_log = control.execute_apply( + dbs, 'us/next', commits, 'cherry-branch', args) + + self.assertEqual(ret, 0) + self.assertTrue(success) + self.assertEqual(conv_log, 'conversation log') + + # Check commit was added to database + commit_rec = dbs.commit_get('aaa111') + self.assertIsNotNone(commit_rec) + self.assertEqual(commit_rec[6], 'applied') # status field + dbs.close() + + def test_execute_apply_failure(self): + """Test execute_apply with failed cherry-pick.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + commits = [control.CommitInfo('bbb222', 'bbb222b', 'Test commit', + 'Author')] + args = argparse.Namespace(push=False) + + with mock.patch.object(control.agent, 'cherry_pick_commits', + return_value=(False, 'error log')): + ret, success, _ = control.execute_apply( + dbs, 'us/next', commits, 'cherry-branch', args) + + self.assertEqual(ret, 1) + self.assertFalse(success) + + # Check commit status is conflict + commit_rec = dbs.commit_get('bbb222') + self.assertEqual(commit_rec[6], 'conflict') + dbs.close() + + def test_execute_apply_with_push(self): + """Test execute_apply with push enabled.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + commits = [control.CommitInfo('ccc333', 'ccc333c', 'Test commit', + 'Author')] + args = argparse.Namespace(push=True, remote='origin', + target='main') + + with mock.patch.object(control.agent, 'cherry_pick_commits', + return_value=(True, 'log')): + with mock.patch.object(gitlab_api, 'push_and_create_mr', + return_value='https://mr/url'): + ret, success, _ = control.execute_apply( + dbs, 'us/next', commits, 'cherry-branch', args) + + self.assertEqual(ret, 0) + self.assertTrue(success) + dbs.close() + + def test_execute_apply_push_fails(self): + """Test execute_apply when MR creation fails.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + + commits = [control.CommitInfo('ddd444', 'ddd444d', 'Test commit', + 'Author')] + args = argparse.Namespace(push=True, remote='origin', + target='main') + + with mock.patch.object(control.agent, 'cherry_pick_commits', + return_value=(True, 'log')): + with mock.patch.object(gitlab_api, 'push_and_create_mr', + return_value=None): + ret, success, _ = control.execute_apply( + dbs, 'us/next', commits, 'cherry-branch', args) + + self.assertEqual(ret, 1) + self.assertTrue(success) # cherry-pick succeeded, MR failed + dbs.close() + + class TestGetNextCommitsEmptyLine(unittest.TestCase): """Tests for get_next_commits with empty lines.""" From patchwork Wed Dec 17 02:27:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 953 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938571; bh=ySek2ehoEN3jKTFKNaoKfOEy2h1iRbel4hnae5F0doM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=UWF3f3KWJBcML1W75OS4YB83GsG8LHJaPDB1bCbEhEyYG644AOsH//ejwHbMIorz6 5ZsWWy2lV+ti/kj1W4BeMuFsbwASwfzJVTtleYHOu+MoNGVLFiNhGtInZD2a5mp/rt eMeSwxBPdXdhyWeVZ3JMF8+HaMOuze0qIJNwRgMfVbUIsGeJmj3VnuCx6xdEc/b58V f5FCg7Jxj8iouIETqOQaMyM381efNSd1/l30AU/wIFLdNesluM5DptbxwB+ozlnQ82 L0McLYnD5dh3Iv+jx6A3h4sZD6jCTaeuI+luJ9ua9nhHtqUNY82heybWD0Hj1nrydw QevhwPxR3lWkg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id BCE8B68BC1 for ; Tue, 16 Dec 2025 19:29:31 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id a8qij1WUMHuL for ; Tue, 16 Dec 2025 19:29:31 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938571; bh=ySek2ehoEN3jKTFKNaoKfOEy2h1iRbel4hnae5F0doM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=UWF3f3KWJBcML1W75OS4YB83GsG8LHJaPDB1bCbEhEyYG644AOsH//ejwHbMIorz6 5ZsWWy2lV+ti/kj1W4BeMuFsbwASwfzJVTtleYHOu+MoNGVLFiNhGtInZD2a5mp/rt eMeSwxBPdXdhyWeVZ3JMF8+HaMOuze0qIJNwRgMfVbUIsGeJmj3VnuCx6xdEc/b58V f5FCg7Jxj8iouIETqOQaMyM381efNSd1/l30AU/wIFLdNesluM5DptbxwB+ozlnQ82 L0McLYnD5dh3Iv+jx6A3h4sZD6jCTaeuI+luJ9ua9nhHtqUNY82heybWD0Hj1nrydw QevhwPxR3lWkg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A9AED68AFD for ; Tue, 16 Dec 2025 19:29:31 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938569; bh=y95tHadaT6I3sL8pyzAIt+WIx6WkeJRfViWLYA6Ukss=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sklOF9i+ui8dS7+4lRyEmWJt1wuM894IAQcXMu1JLBRLjlZAXmJZXRO2YsmrX2PQq qCGV6BUQO61z9/2LyHssQ8lk2Yer5km5+VOBasgxJhRyaLJ8Q3R/s6EzKlpaIyk70I IJi6rA5+NspNh7oWQRcOQaOfL/eKdL85S3fOchmhaKfKWyLM3Z+h2lSYdToE/Oo6WB RzvAw6mpHL3QU2ZNVGrDKNFnvzxJQc+4w2jV04JrJSwdBf88JYNehrI7Raww8HgSSn imeoNKsWSSM0gURC8LZN6bGsY5eulcxdIijd7bZKzTOJHRIkv5EuVRnCJ9aqq98z+V udrTBRapKWqdw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id AE7A76884F; Tue, 16 Dec 2025 19:29:29 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id iz3Bh64yQQb8; Tue, 16 Dec 2025 19:29:29 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938565; bh=pmv33VMeeNSvm9jhFZY/nwOFxWEjKslRYM4qxZ7pdN8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U1zhJ2dZZUi/pKU+tpgNRph4+xvcbBe3MmwxvQbB9/PyqRFJFhudzQO7FjbgItwmg 3D5n4HTlkComcWD35hyKkMuQTAyXasZ64/w4L965pHshBeQ/xuGyb5LM1UCDr+XAYI 34XHkNaP9Cw9wx6VkNkipjBcpOB5TuZcxLTy5YAzyP9vU7l9itfvTCcdOLJQkTRfio s0fHnISNuyosqeiTtsM1AUycrVlKO8WC+wHiHJsR2Eo0D7Gag088xdValowrj8oMIV uAuRQSgdMpXqJ6mBDqttXMUQoXkWbFPfpuJxOh/KBKQY7TuV/bJ1o5J1g79xVrtEuj WKBCQp4V4mq4g== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 19FB468AFD; Tue, 16 Dec 2025 19:29:25 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:27:59 -0700 Message-ID: <20251217022823.392557-11-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 6BS2WT4UTSQVBIQYDYHAQP6NAE3NCEP3 X-Message-ID-Hash: 6BS2WT4UTSQVBIQYDYHAQP6NAE3NCEP3 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 10/24] pickman: Use named tuples for MR data List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Change get_pickman_mrs() and get_mr_comments() to return lists of named tuples instead of dicts. This provides cleaner attribute access (e.g. merge_req.iid instead of merge_req['iid']) and better code documentation. Add PickmanMr and MrComment named tuples to gitlab_api.py. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/agent.py | 2 +- tools/pickman/control.py | 24 +++++++++---------- tools/pickman/ftest.py | 24 +++++++++++-------- tools/pickman/gitlab_api.py | 48 ++++++++++++++++++++++--------------- 4 files changed, 56 insertions(+), 42 deletions(-) diff --git a/tools/pickman/agent.py b/tools/pickman/agent.py index 176cf9773a0..932d61be4d7 100644 --- a/tools/pickman/agent.py +++ b/tools/pickman/agent.py @@ -154,7 +154,7 @@ async def run_review_agent(mr_iid, branch_name, comments, remote, repo_path=None # Format comments for the prompt comment_text = '\n'.join( - f"- [{c['author']}]: {c['body']}" + f'- [{c.author}]: {c.body}' for c in comments ) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 973db22e106..09ffa4a3da3 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -528,29 +528,29 @@ def process_mr_reviews(remote, mrs): processed = 0 for merge_req in mrs: - comments = gitlab_api.get_mr_comments(remote, merge_req['iid']) + comments = gitlab_api.get_mr_comments(remote, merge_req.iid) if comments is None: continue # Filter to unresolved comments - unresolved = [c for c in comments if not c.get('resolved', True)] + unresolved = [c for c in comments if not c.resolved] if not unresolved: continue tout.info('') - tout.info(f"MR !{merge_req['iid']} has {len(unresolved)} comment(s):") + tout.info(f"MR !{merge_req.iid} has {len(unresolved)} comment(s):") for comment in unresolved: - tout.info(f" [{comment['author']}]: {comment['body'][:80]}...") + tout.info(f' [{comment.author}]: {comment.body[:80]}...') # Run agent to handle comments success, _ = agent.handle_mr_comments( - merge_req['iid'], - merge_req['source_branch'], + merge_req.iid, + merge_req.source_branch, unresolved, remote, ) if not success: - tout.error(f"Failed to handle comments for MR !{merge_req['iid']}") + tout.error(f"Failed to handle comments for MR !{merge_req.iid}") processed += 1 return processed @@ -582,7 +582,7 @@ def do_review(args, dbs): # pylint: disable=unused-argument tout.info(f'Found {len(mrs)} open pickman MR(s):') for merge_req in mrs: - tout.info(f" !{merge_req['iid']}: {merge_req['title']}") + tout.info(f" !{merge_req.iid}: {merge_req.title}") process_mr_reviews(remote, mrs) @@ -632,7 +632,7 @@ def process_merged_mrs(remote, source, dbs): processed = 0 for merge_req in merged_mrs: - mr_source, last_hash = parse_mr_description(merge_req['description']) + mr_source, last_hash = parse_mr_description(merge_req.description) # Only process MRs for the requested source branch if mr_source != source: @@ -648,7 +648,7 @@ def process_merged_mrs(remote, source, dbs): full_hash = run_git(['rev-parse', last_hash]) except Exception: # pylint: disable=broad-except tout.warning(f"Could not resolve commit '{last_hash}' from " - f"MR !{merge_req['iid']}") + f"MR !{merge_req.iid}") continue # Check if this commit is an ancestor of source but not of current @@ -669,7 +669,7 @@ def process_merged_mrs(remote, source, dbs): # Update database short_old = current[:12] short_new = full_hash[:12] - tout.info(f"MR !{merge_req['iid']} merged, updating '{source}': " + tout.info(f"MR !{merge_req.iid} merged, updating '{source}': " f'{short_old} -> {short_new}') dbs.source_set(source, full_hash) dbs.commit() @@ -708,7 +708,7 @@ def do_step(args, dbs): if mrs: tout.info(f'Found {len(mrs)} open pickman MR(s):') for merge_req in mrs: - tout.info(f" !{merge_req['iid']}: {merge_req['title']}") + tout.info(f" !{merge_req.iid}: {merge_req.title}") # Process any review comments on open MRs process_mr_reviews(remote, mrs) diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 9c65652f27a..4459f108496 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1233,11 +1233,13 @@ class TestStep(unittest.TestCase): def test_step_with_pending_mr(self): """Test step does nothing when MR is pending.""" - mock_mr = { - 'iid': 123, - 'title': '[pickman] Test MR', - 'web_url': 'https://gitlab.com/mr/123', - } + mock_mr = gitlab_api.PickmanMr( + iid=123, + title='[pickman] Test MR', + web_url='https://gitlab.com/mr/123', + source_branch='cherry-test', + description='Test', + ) with mock.patch.object(gitlab_api, 'get_merged_pickman_mrs', return_value=[]): with mock.patch.object(gitlab_api, 'get_open_pickman_mrs', @@ -1864,11 +1866,13 @@ class TestDoReviewWithMrs(unittest.TestCase): """Test review with open MRs but no comments.""" tout.init(tout.INFO) - mock_mr = { - 'iid': 123, - 'title': '[pickman] Test MR', - 'web_url': 'https://gitlab.com/mr/123', - } + mock_mr = gitlab_api.PickmanMr( + iid=123, + title='[pickman] Test MR', + web_url='https://gitlab.com/mr/123', + source_branch='cherry-test', + description='Test', + ) with mock.patch.object(gitlab_api, 'get_open_pickman_mrs', return_value=[mock_mr]): with mock.patch.object(gitlab_api, 'get_mr_comments', diff --git a/tools/pickman/gitlab_api.py b/tools/pickman/gitlab_api.py index 6720e0a1526..d0128e13f80 100644 --- a/tools/pickman/gitlab_api.py +++ b/tools/pickman/gitlab_api.py @@ -5,6 +5,7 @@ # """GitLab integration for pickman - push branches and create merge requests.""" +from collections import namedtuple import os import re import sys @@ -25,6 +26,17 @@ except ImportError: AVAILABLE = False +# Merge request info returned by get_pickman_mrs() +PickmanMr = namedtuple('PickmanMr', [ + 'iid', 'title', 'web_url', 'source_branch', 'description' +]) + +# Comment info returned by get_mr_comments() +MrComment = namedtuple('MrComment', [ + 'id', 'author', 'body', 'created_at', 'resolvable', 'resolved' +]) + + def check_available(): """Check if the python-gitlab module is available @@ -160,8 +172,7 @@ def get_pickman_mrs(remote, state='opened'): state (str): MR state ('opened', 'merged', 'closed', 'all') Returns: - list: List of dicts with 'iid', 'title', 'web_url', 'source_branch', - 'description' keys, or None on failure + list: List of PickmanMr tuples, or None on failure """ if not check_available(): return None @@ -186,13 +197,13 @@ def get_pickman_mrs(remote, state='opened'): pickman_mrs = [] for merge_req in mrs: if '[pickman]' in merge_req.title: - pickman_mrs.append({ - 'iid': merge_req.iid, - 'title': merge_req.title, - 'web_url': merge_req.web_url, - 'source_branch': merge_req.source_branch, - 'description': merge_req.description or '', - }) + pickman_mrs.append(PickmanMr( + iid=merge_req.iid, + title=merge_req.title, + web_url=merge_req.web_url, + source_branch=merge_req.source_branch, + description=merge_req.description or '', + )) return pickman_mrs except gitlab.exceptions.GitlabError as exc: tout.error(f'GitLab API error: {exc}') @@ -233,8 +244,7 @@ def get_mr_comments(remote, mr_iid): mr_iid (int): Merge request IID Returns: - list: List of dicts with 'id', 'author', 'body', 'created_at', - 'resolvable', 'resolved' keys, or None on failure + list: List of MrComment tuples, or None on failure """ if not check_available(): return None @@ -260,14 +270,14 @@ def get_mr_comments(remote, mr_iid): # Skip system notes (merge status, etc.) if note.system: continue - comments.append({ - 'id': note.id, - 'author': note.author['username'], - 'body': note.body, - 'created_at': note.created_at, - 'resolvable': getattr(note, 'resolvable', False), - 'resolved': getattr(note, 'resolved', False), - }) + comments.append(MrComment( + id=note.id, + author=note.author['username'], + body=note.body, + created_at=note.created_at, + resolvable=getattr(note, 'resolvable', False), + resolved=getattr(note, 'resolved', False), + )) return comments except gitlab.exceptions.GitlabError as exc: tout.error(f'GitLab API error: {exc}') From patchwork Wed Dec 17 02:28:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 954 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938576; bh=GN+h34ksvr9NqVov9Hm8nDhRl1Fmf2CPhpOqRoLL//M=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Vy6ogHLPHnfWW6Zi+Ov5eKuS68e3+4GvfYKwJk3dgDM3SLF/4f8ofyQF840sKq2HT Ilk0XcYw0QIxs8aydhiRcX6Gjo8312QWPUYzgJx4WfAT8r+YfJ7vEO3Ktn9KECnRmH AIDRtzgSnftvwySfgftbADGw5yzqQSrKRvrqs2mrL6/KyiD+G7ieAct1Kxoddmnp+2 h5DCIuy5quO/Xq25hXJRAfh/l3bLYZ7cdx7YKD9e2zD3SR0rAHIbqKIFJy2Kli/K20 3UIgVri7IIIbHDWMpsMAkPYWZzdAp1Pw2XzJq48k949axcGOqg+vyv48lhBwJsapEf 0res8EbYOUR2w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7FE4268AEE for ; Tue, 16 Dec 2025 19:29:36 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 4nYHBa89gPw9 for ; Tue, 16 Dec 2025 19:29:36 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938576; bh=GN+h34ksvr9NqVov9Hm8nDhRl1Fmf2CPhpOqRoLL//M=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Vy6ogHLPHnfWW6Zi+Ov5eKuS68e3+4GvfYKwJk3dgDM3SLF/4f8ofyQF840sKq2HT Ilk0XcYw0QIxs8aydhiRcX6Gjo8312QWPUYzgJx4WfAT8r+YfJ7vEO3Ktn9KECnRmH AIDRtzgSnftvwySfgftbADGw5yzqQSrKRvrqs2mrL6/KyiD+G7ieAct1Kxoddmnp+2 h5DCIuy5quO/Xq25hXJRAfh/l3bLYZ7cdx7YKD9e2zD3SR0rAHIbqKIFJy2Kli/K20 3UIgVri7IIIbHDWMpsMAkPYWZzdAp1Pw2XzJq48k949axcGOqg+vyv48lhBwJsapEf 0res8EbYOUR2w== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6ADAE68AFD for ; Tue, 16 Dec 2025 19:29:36 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938574; bh=m+OY0Kwj1CTcq9DzUWig2WAis3+NakUz6dTIB7g+lT0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KTZ7KtqZ6I6CWC5lzZMBgkosxQt2aTsX0TS4JPE+8pikL19ZWunKNGoHNmuAoMw41 MDr50aE+uo9brngu4b2VPFs66naT4x8p4OA++T1tH05l6/gtaTGOFJj7xwgXU5gGiZ Zfsp+YzpZyY6ABQVw0VTe0i3jkTNvEQPim4bMOit5J/LLWM5OKkn1XRLuK+RnY9XJs 2LKuw8fphEleMUmieOwI1VKBP0iunTfs5v3ZIriuyD/vtULDvUf4Saa6AtpseNE3VU xLJRs56Rhtvi827BOySYl4bPqWXiMycGmFcNAwhBALEW39bDZEzoHrCwWiJJV9FDBd 8rxu6i90fNsfw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1954968BBD; Tue, 16 Dec 2025 19:29:34 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 0GUir5ivSFhF; Tue, 16 Dec 2025 19:29:34 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938570; bh=XrlsPmo3AiSbBAiwhc8/2CndGryByURFoQZ4RJPoyaw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W4U0gGm2cLCZpJ2QRCCiQlfD6lovJo2fqk1qGuv05sI/QDCFb9RIdKgXHdu5glUJ7 dBJZljDTW4D0YjUQlG/ZX3QgArJ6vXQSUK65Td8WOm/KJ3//1eGyYL2pTeJADn5SXG VGDynVwAGxOAe/gINT+ACZUHY8RN+/35O5O22SBch86IMjT82RQ7OshLUjR/p1rlvq o9Yw+MdyibcRdmPg/fWw+rsqV5aLx3KDr91U4PtlmOlcTS/nFc1MS1MRHBFg8zWJqg H6DCr+JNg9uDOS//IkAVtK+Zz7/pqIQbfqCnLKTKm6rwEqMk0dFrN7Hmqjea+6OKzx Rwj0QLdGDCsvg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id D65C268AEE; Tue, 16 Dec 2025 19:29:29 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:00 -0700 Message-ID: <20251217022823.392557-12-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 6LZQKS2SFX3V5FE3OZZFGWMCRF63V23P X-Message-ID-Hash: 6LZQKS2SFX3V5FE3OZZFGWMCRF63V23P X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 11/24] pickman: Fix ancestor check in process_merged_mrs List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Fix the logic for determining if an MR's last commit is newer than the current database position. The check should verify that current is an ancestor of the MR's commit (meaning the MR's commit is newer), not the other way around. Add a test to cover this. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 16 ++----- tools/pickman/ftest.py | 91 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 12 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 09ffa4a3da3..e6b7539ea34 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -651,20 +651,12 @@ def process_merged_mrs(remote, source, dbs): f"MR !{merge_req.iid}") continue - # Check if this commit is an ancestor of source but not of current - # (meaning it's newer than what we have) + # Check if this commit is newer than current (current is ancestor of it) try: - # Is last_hash reachable from source? - run_git(['merge-base', '--is-ancestor', full_hash, source]) + # Is current an ancestor of last_hash? (meaning last_hash is newer) + run_git(['merge-base', '--is-ancestor', current, full_hash]) except Exception: # pylint: disable=broad-except - continue # Not reachable, skip - - try: - # Is last_hash already at or before current? - run_git(['merge-base', '--is-ancestor', full_hash, current]) - continue # Already processed - except Exception: # pylint: disable=broad-except - pass # Not an ancestor of current, so it's newer + continue # current is not an ancestor, so last_hash is not newer # Update database short_old = current[:12] diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 4459f108496..80aa73f8a5b 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1885,5 +1885,96 @@ class TestDoReviewWithMrs(unittest.TestCase): self.assertIn('Found 1 open pickman MR', stdout.getvalue()) +class TestProcessMergedMrs(unittest.TestCase): + """Tests for process_merged_mrs function.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_process_merged_mrs_updates_newer(self): + """Test that newer commits update the database.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'aaa111aaa111aaa111aaa111aaa111aaa111aaa1') + dbs.commit() + + merged_mrs = [gitlab_api.PickmanMr( + iid=100, + title='[pickman] Test MR', + description='## 2025-01-01: us/next\n\n- bbb222b Subject', + source_branch='cherry-test', + web_url='https://gitlab.com/mr/100', + )] + + def mock_git(args): + if args[0] == 'rev-parse': + return 'bbb222bbb222bbb222bbb222bbb222bbb222bbb2' + if args[0] == 'merge-base': + # current is ancestor of last_hash (newer) + return '' + return '' + + with mock.patch.object(gitlab_api, 'get_merged_pickman_mrs', + return_value=merged_mrs): + with mock.patch.object(control, 'run_git', mock_git): + processed = control.process_merged_mrs('ci', 'us/next', dbs) + + self.assertEqual(processed, 1) + new_commit = dbs.source_get('us/next') + self.assertEqual(new_commit, + 'bbb222bbb222bbb222bbb222bbb222bbb222bbb2') + + dbs.close() + + def test_process_merged_mrs_skips_older(self): + """Test that older commits don't update the database.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'bbb222bbb222bbb222bbb222bbb222bbb222bbb2') + dbs.commit() + + merged_mrs = [gitlab_api.PickmanMr( + iid=100, + title='[pickman] Test MR', + description='## 2025-01-01: us/next\n\n- aaa111a Subject', + source_branch='cherry-test', + web_url='https://gitlab.com/mr/100', + )] + + def mock_git(args): + if args[0] == 'rev-parse': + return 'aaa111aaa111aaa111aaa111aaa111aaa111aaa1' + if args[0] == 'merge-base': + # current is NOT ancestor of last_hash (older) + raise RuntimeError('Not an ancestor') + return '' + + with mock.patch.object(gitlab_api, 'get_merged_pickman_mrs', + return_value=merged_mrs): + with mock.patch.object(control, 'run_git', mock_git): + processed = control.process_merged_mrs('ci', 'us/next', dbs) + + self.assertEqual(processed, 0) + # Should remain unchanged + current = dbs.source_get('us/next') + self.assertEqual(current, + 'bbb222bbb222bbb222bbb222bbb222bbb222bbb2') + + dbs.close() + + if __name__ == '__main__': unittest.main() From patchwork Wed Dec 17 02:28:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 955 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938581; bh=eQ6ClwPUAK3DC5f/C2agOxlqe5//SJ2AgYKT+EJhZ1M=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ritSBD+QIEJFKJSsjgZIFxLdA4Oh4pSattvhj9/D6A0znfsHJWI/JrnrB4VrYZFQX aEoCy0nfhLMnmPmrCUMyLuywLSTFMw97BQgl6nWySbWpyw3pX5k2B4nj3sgVka9k7/ wcGx5gfFGH8t0hNT7cu+DpTQds5IUzfmfK5cXMK3nwbt+BR0xco4L42hPf3rM2ocyK DAaz6UplLAEoravhxuKBMbyRQIsx+Ionf3akJm09EmDAd/fV4RSgo8MKybCh/ml8m9 H+x2wqsfZebR7mfnKpXU434fD8r6loCsXdiwcnG7gFHFTNo983tTKLwowZjfbRuNW6 VsXQ3b00qQWZg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0EAB668BC2 for ; Tue, 16 Dec 2025 19:29:41 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id QzNBR7582RXk for ; Tue, 16 Dec 2025 19:29:41 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938580; bh=eQ6ClwPUAK3DC5f/C2agOxlqe5//SJ2AgYKT+EJhZ1M=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=lEdbBscrL2cKvieVGpFpa53r/f1u9u6ZmS/vfN9JaNpnRicM21vShE713ZwFxhZgB l0Y1h1uVcGayufGBmV1CUsgYoDztEIqtrH+HExpeTDKnh5IDBlJY6g9OsA4NhUgKQP laUfC/rFCjM6u9Ym+naHoqB2+juu8y25ytxVPgkFpMjxYMs+nEPT4ZIn7GqpNUf/mt VJr3N5ecr7ESVB7Sr0CQfc1/7flqU9O9Tqj4//iWfjvGLNRu6EXGamc6qySuwhg3+l kd+9bvit3eDVCmBL2bt5VhGIUmyjyp1VKbkmY+YKXe9uegormPZZfE8O5vFtXSKE6P bA4eAmGKbm3Iw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id F0FD76897B for ; Tue, 16 Dec 2025 19:29:40 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938578; bh=zgd3KuoOYD0b/gGu0stp35uyxZ5nQIu0tMi5AXz9Jvg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H0QPV6FlIR2CPN8DiC+53qZcrIs2tisGMkW374ffD80vSnBOtC4ASWXLqO8W6eDr3 do9bdULnMuHxtAzaDprDoX3z8dfFrwIhmtElD5xI2WobiW/c3Ld9UAVDF/d2MQdGD3 z15h/PLTGmKPNCHRttyehb6yrH7iavUz+dd74H5rAn+eiSheEIqccn9HemkMfrmja7 lIZIXGq0cxQFtqz7Fuxb7tGqMMqLp5qVs+yos3pa2w3tz32tmOMVEEoXMxOX/k+9jV /2N6/8C4J1V/fLhBUTGE9yQZer71NAxmHKwSay2hzhQ8EWPUIbg4DbakUKtr0FERvU EjvN5VHx3J/HQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id F02EA68B96; Tue, 16 Dec 2025 19:29:38 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id GuHSWTmxlkVl; Tue, 16 Dec 2025 19:29:38 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938574; bh=Wvnv26SI+rtlF3ogFXPX+fyzKMe+gH7r2paZuvZwmk4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=whsKJ5Eoc+tCPMzcS8K4/yMzzHep+VzEtyc5uXEnDK2c/ncMojtxwl7Rd/d33+pKA jTmnTishlfGxsFqkgXfYHhtOgB4ft7b0kcj60MbuLIDkB8mg1JYoIDxsYpkC9DiKni 99oSez5weQZy/8vzHpfXwjcp7Z/uAYM8CsQFHkp19/fbND/X5FN47g6FznKysSM31j MnUzVTqZwu177kCK7EZqwDhVhl4HYN6e4WI5WU7XdK63/bQHfJW5hhnaNaHHC4gU3T eOLrbDtw5a1HEZbq+3QD4E0VUKtaqn9Pvyp+U7p+r6nqIH4Cl6HtpIb1nlgRZNYYV6 P+fPduxjj88og== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 0C4AB6884F; Tue, 16 Dec 2025 19:29:33 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:01 -0700 Message-ID: <20251217022823.392557-13-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: B5WUKAK3HIMNPBS64OUNPOGZZ6VOUEVU X-Message-ID-Hash: B5WUKAK3HIMNPBS64OUNPOGZZ6VOUEVU X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 12/24] pickman: Fix merge selection to follow first-parent chain List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Fix get_next_commits() to use --first-parent when finding the next merge commit. Without this, git log returns commits in topological order which can find merges from different subsystems that were merged later, rather than following the mainline merge order. The fix uses a two-step approach: 1. Use --first-parent to find the next merge on the mainline 2. Get all commits up to that merge (without --first-parent to include the merged branch's commits) Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 40 ++++++++++++++++++++++++++-------------- tools/pickman/ftest.py | 37 ++++++++++++++++++++++++++++++++----- 2 files changed, 58 insertions(+), 19 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index e6b7539ea34..892100f3479 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -151,7 +151,7 @@ def get_next_commits(dbs, source): """Get the next set of commits to cherry-pick from a source Finds commits between the last cherry-picked commit and the next merge - commit in the source branch. + commit on the first-parent (mainline) chain of the source branch. Args: dbs (Database): Database instance @@ -169,20 +169,38 @@ def get_next_commits(dbs, source): if not last_commit: return None, False, f"Source '{source}' not found in database" - # Get commits between last_commit and source HEAD (oldest first) - # Format: hash|short_hash|author|subject|parents - # Using | as separator since subject may contain colons + # First, find the next merge commit on the first-parent chain + # This ensures we follow the mainline and find merges in order + fp_output = run_git([ + 'log', '--reverse', '--first-parent', '--format=%H|%h|%an|%s|%P', + f'{last_commit}..{source}' + ]) + + if not fp_output: + return [], False, None + + # Find the first merge on the first-parent chain + merge_hash = None + for line in fp_output.split('\n'): + if not line: + continue + parts = line.split('|') + parents = parts[-1].split() + if len(parents) > 1: + merge_hash = parts[0] + break + + # Now get all commits from last_commit to the merge (or end of branch) + # Without --first-parent to include commits from merged branches log_output = run_git([ 'log', '--reverse', '--format=%H|%h|%an|%s|%P', - f'{last_commit}..{source}' + f'{last_commit}..{merge_hash or source}' ]) if not log_output: return [], False, None commits = [] - merge_found = False - for line in log_output.split('\n'): if not line: continue @@ -191,16 +209,10 @@ def get_next_commits(dbs, source): short_hash = parts[1] author = parts[2] subject = '|'.join(parts[3:-1]) # Subject may contain separator - parents = parts[-1].split() commits.append(CommitInfo(commit_hash, short_hash, subject, author)) - # Check if this is a merge commit (has multiple parents) - if len(parents) > 1: - merge_found = True - break - - return commits, merge_found, None + return commits, bool(merge_hash), None def do_next_set(args, dbs): diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 80aa73f8a5b..784e8dd5d1b 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -840,14 +840,27 @@ class TestNextSet(unittest.TestCase): database.Database.instances.clear() - # Mock git log with commits including a merge - log_output = ( + # First-parent log (to find next merge on mainline) + fp_log_output = ( 'aaa111|aaa111a|Author 1|First commit|abc123\n' 'bbb222|bbb222b|Author 2|Second commit|aaa111\n' 'ccc333|ccc333c|Author 3|Merge branch feature|bbb222 ddd444\n' 'eee555|eee555e|Author 4|After merge|ccc333\n' ) - command.TEST_RESULT = command.CommandResult(stdout=log_output) + # Full log (to get all commits up to the merge) + full_log_output = ( + 'aaa111|aaa111a|Author 1|First commit|abc123\n' + 'bbb222|bbb222b|Author 2|Second commit|aaa111\n' + 'ccc333|ccc333c|Author 3|Merge branch feature|bbb222 ddd444\n' + ) + + def mock_git(pipe_list): + cmd = pipe_list[0] if pipe_list else [] + if '--first-parent' in cmd: + return command.CommandResult(stdout=fp_log_output) + return command.CommandResult(stdout=full_log_output) + + command.TEST_RESULT = mock_git args = argparse.Namespace(cmd='next-set', source='us/next') with terminal.capture() as (stdout, _): @@ -931,11 +944,25 @@ class TestGetNextCommits(unittest.TestCase): dbs.source_set('us/next', 'abc123') dbs.commit() - log_output = ( + # First-parent log (to find next merge on mainline) + fp_log_output = ( 'aaa111|aaa111a|Author 1|First commit|abc123\n' 'bbb222|bbb222b|Author 2|Merge branch|aaa111 ccc333\n' + 'ddd444|ddd444d|Author 3|After merge|bbb222\n' ) - command.TEST_RESULT = command.CommandResult(stdout=log_output) + # Full log (to get all commits up to the merge) + full_log_output = ( + 'aaa111|aaa111a|Author 1|First commit|abc123\n' + 'bbb222|bbb222b|Author 2|Merge branch|aaa111 ccc333\n' + ) + + def mock_git(pipe_list): + cmd = pipe_list[0] if pipe_list else [] + if '--first-parent' in cmd: + return command.CommandResult(stdout=fp_log_output) + return command.CommandResult(stdout=full_log_output) + + command.TEST_RESULT = mock_git commits, merge_found, error = control.get_next_commits(dbs, 'us/next') From patchwork Wed Dec 17 02:28:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 956 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938583; bh=ZZA0V/6TuG6RaDZpLtxJY5/CRr3IU+Qyle7pSytxOzA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=pIqxg0h6ord9Xu3BEf2L3gsbAuNSj4dPByaQWL+XpdM1PCcmXWqezgPATysl16xmj M/3UA+fCcknFtdVTCygsGRKoLUzawdKKPk5640f2iSik2U+8sqdnJ44d1vRu4yVYst x+5Qcf+DSKWmAfPWCSCn0qSxrfv/Fzv8m/T0jUiVHQtFM5NDtcsk3QwPFulm7n/8aP hGODV5pufCnT1pbSyOlncLH8kz+ak8FIWe+G9dlPhKpLlxKoA6cseqkmkPYFEw1nwZ xEMALF4tqHO3XVaHZWnIC+UV4/PR8eiTrKPKC16cyCwSansn34l86jvfpx5pCgCvDc l3bHCohTOJB2w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 73A8168AEE for ; Tue, 16 Dec 2025 19:29:43 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id zMOzzkJIc9BK for ; Tue, 16 Dec 2025 19:29:43 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938581; bh=ZZA0V/6TuG6RaDZpLtxJY5/CRr3IU+Qyle7pSytxOzA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=REkIN0Xx7tC3boJVLBjUW4JlMDFpiooIuRYUbEtbuGtPOI6JUYwWQCL+OVYrGdA9Z 2CYlAxoZKUCqV3TSVUYyLMqK8xkE0W9jnQIzTKYUJlFXAx0V0+GXe43GxZtdKZKz1h mo4GNrXpu3sKtUmndzYgAgdxH99Rm7c9j2408XabcofFS/2K5Bu3kQqw0svT4ZUzL0 xHLWROqBP9XrARd5txezADEnYjfLPAJFY5tC3T2o9/t4SkU7YjoGQFyB7jC/jYj7n2 JaWDCXe7SkRxxovNmEaSlHHWmt6OwbJmVoC4mwdqURhNs912n4ucXpH+Xt6Ajic477 fwloRReTq1i5w== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6FE026897B for ; Tue, 16 Dec 2025 19:29:41 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938579; bh=0hk6n4YQ5msKsgKfM/WjfruhDBtJyfQTCV8LGsCHNhQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFDxqIoBIo/QWEy7vzGZhsSJX9z3wea4dZxuUT7pJBe1gOMFHNGYGYgktfPGRjnPV vMganXmC/5H1AYpHgCUL4Ln/EMtZd0hZQiu0Flj50uuPN9t+hy2aG2BnkhQGosIXi0 PlyY/Nwn1/NkcZcCLGjByN7Ly11zOJiXE1Y3xnNiCYCaYLf2oX+Igo308ctPb7IjdJ S7RZWU94DfnhKYLz5UOPICnKuhEIBcBWSfWyuoHXkuQjmHJf03gfLbNwFMlW2+xlKi ayr9L1MfdSbh78nwUAetuxbfto+E9Q8SAiebMCJxTgp3sSRp8tc2nBE9L/t3Ky/FCo FsDuUlV00rWiw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6A87F6884F; Tue, 16 Dec 2025 19:29:39 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 7jNYOVMD3vUs; Tue, 16 Dec 2025 19:29:39 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938579; bh=dCAESc2Oe9RE5lbBWpZ5jyEaIWbQ9+lZ427yCKIgZew=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ByCz8ztwTy1oWgRV2aaufiAuLMysrROM7NQgyjX46a3x3RXgf6MIVCBbwHp2NPuj4 lz+pU2BCcDmcSKiSOwXXmbrlMLi8m2g7TRg6JV6bPHn5XcNEH9w28pHVS3uA0u2GV8 Ih14xufgxiCFdOwrz57CrcolrD7w3Wp3/XNCjHvYyNUX37Vte8LPqISmw9EXjlmxJ3 9LgMDzU+XbXHT0hDM5i3CCVl8mkhDVSrchgzH9xdoTYl/9hPBB1zNR8VyIUvw3lCFZ C8YkzXVcy12SMX/d4TX2Rf14OMMd1NhlEp3+8k1zfevs7dHiriC1AZ6k4X3ixQAfOB aIm7stWFIbe5g== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id C26E068AFF; Tue, 16 Dec 2025 19:29:38 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:02 -0700 Message-ID: <20251217022823.392557-14-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: POWLR5A3Y4QGQNXZWUAU6FMBQEEKFOZW X-Message-ID-Hash: POWLR5A3Y4QGQNXZWUAU6FMBQEEKFOZW X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 13/24] pickman: Add next-merges command List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add a new next-merges command that shows the next N merges to be applied from a source branch, useful for seeing what's coming up. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 7 +++ tools/pickman/__main__.py | 6 +++ tools/pickman/control.py | 51 +++++++++++++++++++ tools/pickman/ftest.py | 101 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 165 insertions(+) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index b469470138b..a382c98eac0 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -68,6 +68,13 @@ This finds commits between the last cherry-picked commit and the next merge commit in the source branch. It stops at the merge commit since that typically represents a logical grouping of commits (e.g., a pull request). +To show the next N merges that will be applied:: + + ./tools/pickman/pickman next-merges us/next + +This shows the upcoming merge commits on the first-parent chain, useful for +seeing what's coming up. Use ``-c`` to specify the count (default 10). + To apply the next set of commits using a Claude agent:: ./tools/pickman/pickman apply us/next diff --git a/tools/pickman/__main__.py b/tools/pickman/__main__.py index 2d2366c1b80..4a60ed7eedc 100755 --- a/tools/pickman/__main__.py +++ b/tools/pickman/__main__.py @@ -56,6 +56,12 @@ def parse_args(argv): subparsers.add_parser('compare', help='Compare branches') subparsers.add_parser('list-sources', help='List tracked source branches') + next_merges = subparsers.add_parser('next-merges', + help='Show next N merges to be applied') + next_merges.add_argument('source', help='Source branch name') + next_merges.add_argument('-c', '--count', type=int, default=10, + help='Number of merges to show (default: 10)') + next_set = subparsers.add_parser('next-set', help='Show next set of commits to cherry-pick') next_set.add_argument('source', help='Source branch name') diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 892100f3479..7ac0fffc67a 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -248,6 +248,56 @@ def do_next_set(args, dbs): return 0 +def do_next_merges(args, dbs): + """Show the next N merges to be applied from a source + + Args: + args (Namespace): Parsed arguments with 'source' and 'count' attributes + dbs (Database): Database instance + + Returns: + int: 0 on success, 1 if source not found + """ + source = args.source + count = args.count + + # Get the last cherry-picked commit from database + last_commit = dbs.source_get(source) + + if not last_commit: + tout.error(f"Source '{source}' not found in database") + return 1 + + # Find merge commits on the first-parent chain + out = run_git([ + 'log', '--reverse', '--first-parent', '--merges', + '--format=%H|%h|%s', + f'{last_commit}..{source}' + ]) + + if not out: + tout.info('No merges remaining') + return 0 + + merges = [] + for line in out.split('\n'): + if not line: + continue + parts = line.split('|', 2) + commit_hash = parts[0] + short_hash = parts[1] + subject = parts[2] if len(parts) > 2 else '' + merges.append((commit_hash, short_hash, subject)) + if len(merges) >= count: + break + + tout.info(f'Next {len(merges)} merges from {source}:') + for i, (_, short_hash, subject) in enumerate(merges, 1): + tout.info(f' {i}. {short_hash} {subject}') + + return 0 + + HISTORY_FILE = '.pickman-history' @@ -786,6 +836,7 @@ COMMANDS = { 'commit-source': do_commit_source, 'compare': do_compare, 'list-sources': do_list_sources, + 'next-merges': do_next_merges, 'next-set': do_next_set, 'poll': do_poll, 'review': do_review, diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 784e8dd5d1b..d452fc823ca 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -904,6 +904,107 @@ class TestNextSet(unittest.TestCase): self.assertIn('bbb222b Second commit', output) +class TestNextMerges(unittest.TestCase): + """Tests for next-merges command.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_next_merges(self): + """Test next-merges shows upcoming merges""" + # Add source to database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + dbs.close() + + database.Database.instances.clear() + + # Mock git log with merge commits + log_output = ( + 'aaa111|aaa111a|Merge branch feature-1\n' + 'bbb222|bbb222b|Merge branch feature-2\n' + 'ccc333|ccc333c|Merge branch feature-3\n' + ) + command.TEST_RESULT = command.CommandResult(stdout=log_output) + + args = argparse.Namespace(cmd='next-merges', source='us/next', count=10) + with terminal.capture() as (stdout, _): + ret = control.do_pickman(args) + self.assertEqual(ret, 0) + output = stdout.getvalue() + self.assertIn('Next 3 merges from us/next:', output) + self.assertIn('1. aaa111a Merge branch feature-1', output) + self.assertIn('2. bbb222b Merge branch feature-2', output) + self.assertIn('3. ccc333c Merge branch feature-3', output) + + def test_next_merges_with_count(self): + """Test next-merges respects count parameter""" + # Add source to database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + dbs.close() + + database.Database.instances.clear() + + # Mock git log with merge commits + log_output = ( + 'aaa111|aaa111a|Merge branch feature-1\n' + 'bbb222|bbb222b|Merge branch feature-2\n' + 'ccc333|ccc333c|Merge branch feature-3\n' + ) + command.TEST_RESULT = command.CommandResult(stdout=log_output) + + args = argparse.Namespace(cmd='next-merges', source='us/next', count=2) + with terminal.capture() as (stdout, _): + ret = control.do_pickman(args) + self.assertEqual(ret, 0) + output = stdout.getvalue() + self.assertIn('Next 2 merges from us/next:', output) + self.assertIn('1. aaa111a', output) + self.assertIn('2. bbb222b', output) + self.assertNotIn('3. ccc333c', output) + + def test_next_merges_no_merges(self): + """Test next-merges with no merges remaining""" + # Add source to database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + dbs.close() + + database.Database.instances.clear() + + command.TEST_RESULT = command.CommandResult(stdout='') + + args = argparse.Namespace(cmd='next-merges', source='us/next', count=10) + with terminal.capture() as (stdout, _): + ret = control.do_pickman(args) + self.assertEqual(ret, 0) + self.assertIn('No merges remaining', stdout.getvalue()) + + class TestGetNextCommits(unittest.TestCase): """Tests for get_next_commits function.""" From patchwork Wed Dec 17 02:28:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 957 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938587; bh=2a0ZQ5X0RQy8VTUoi83cINn0jZnNu7ND9Xgs1FSIVpQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=iqzoIU14W9nhtboCKH4ssRyJLTGnOZkbWHfpyAljlariNmvjaJTleV/tv2FwhsOwL g5qDty2SYXL7s5Bi5WymjMGZteK/+XtMN1J4fDinjxkGZFDhl8shH+jiH6qH4yODzj Eh5nY4PCu4hrTstVzXTi/7kNsoEPMaymwIQX5MtQQji/2gis9FQgLgGQGsma+V1JtA 1oqWSpFewPiKc2v8beUH9V3l/jYlxiJxzFRXIptqbzJoUSTWiSLM22j3/X+A0Xz+Vt SWakzknXMtdRl9s8e5VLw7H8gQIBxoqgkwB4Vn9r2V5Ff0d4F2sUY8VIkIJoHUGFS7 QdSCcZAO/7i7g== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3FFC168BA0 for ; Tue, 16 Dec 2025 19:29:47 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id kzs0X9BcTqCb for ; Tue, 16 Dec 2025 19:29:47 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938587; bh=2a0ZQ5X0RQy8VTUoi83cINn0jZnNu7ND9Xgs1FSIVpQ=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=iqzoIU14W9nhtboCKH4ssRyJLTGnOZkbWHfpyAljlariNmvjaJTleV/tv2FwhsOwL g5qDty2SYXL7s5Bi5WymjMGZteK/+XtMN1J4fDinjxkGZFDhl8shH+jiH6qH4yODzj Eh5nY4PCu4hrTstVzXTi/7kNsoEPMaymwIQX5MtQQji/2gis9FQgLgGQGsma+V1JtA 1oqWSpFewPiKc2v8beUH9V3l/jYlxiJxzFRXIptqbzJoUSTWiSLM22j3/X+A0Xz+Vt SWakzknXMtdRl9s8e5VLw7H8gQIBxoqgkwB4Vn9r2V5Ff0d4F2sUY8VIkIJoHUGFS7 QdSCcZAO/7i7g== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2CC656897B for ; Tue, 16 Dec 2025 19:29:47 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938585; bh=tW0C0iceCPOFsmt3HYpmgq1pezV4rlvi7wcOYm72tfY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p123VFT08w/OCps2HpPFkL29zH5hVkAEcbpi+41gRqezPmM90AiE++SmC3V3z5mWM rRFxiLQyf56IpzxjqjsaIG4znZhriLfmVEvyFJn4IS8PjM3LaMaMjPjam3bpZ6hZZV WIE/BiatmC+2aUoO8lMqrCJfRDlbq/K9jg29tz2grKqqxT1sR0HvOjh9DCSjF2M3vy cSPaQdu3HwHxPejVX7G9h791QKVQojAIqshLS04u4JE0Uyne5bpvJC0qZSiHsmt/k/ QVu/MG/u4sqHApvCd4TQb9elHHiULtzBGeH+RQVuGcT0BQLo4qzAD3k0yjYsKRAWXs PGSPSv2UkcqSg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id BFCE16897B; Tue, 16 Dec 2025 19:29:45 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id aSClAMffi3DG; Tue, 16 Dec 2025 19:29:45 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938579; bh=LtdgVhaApIxylUImIU+m9PsqvwNb2rAnw8LFMdi0luA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=abrhy9KGiKrMz7LdjRfeMNSyRW4bdcoHrfcPX80J/urtkDVoUXVfkCkQpadiQ6HY2 Ubz0bpQL17Cs4N8A4tgihSMtV3HxMontmgfWTZfEM6A5z7IzL6DUsYNdTiDWwdttW1 CErGGbcaDTwWRUDYAD9Ht2e6+yqJkdRmUZj0uk7YqcZdTFSYpSP90dsuigj0rHm+UR vNceSqNIR5eJpgh47+ubVmOWBEIH6Uw4VuTYZJwHLdGkoJXIhuX4nwFzCCsvLIp+4I YYhxrqBq26lJQ8hWSM6GKPS7ZyYCjLXg7L1RPX9ApeXVg84naULo5kTbebtP4vk3Yy f63KMlPd3FHag== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id A9E2E68AFF; Tue, 16 Dec 2025 19:29:39 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:03 -0700 Message-ID: <20251217022823.392557-15-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: SIXRNMW6MS4C2A46MZCVKKX5O7B5N6YO X-Message-ID-Hash: SIXRNMW6MS4C2A46MZCVKKX5O7B5N6YO X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 14/24] pickman: Fix parse_mr_description to ignore short numbers List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The regex for extracting commit hashes from MR descriptions was matching short numbers like "1" from the conversation log (e.g., "- 1 board built"). Fix by requiring commit hashes to be at least 7 characters, which is the minimum length for git short hashes. Shorten the argument name to 'desc' while we are here. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/control.py | 10 +++++----- tools/pickman/ftest.py | 18 ++++++++++++++++++ 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 7ac0fffc67a..acb3b80a4a7 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -651,23 +651,23 @@ def do_review(args, dbs): # pylint: disable=unused-argument return 0 -def parse_mr_description(description): +def parse_mr_description(desc): """Parse a pickman MR description to extract source and last commit Args: - description (str): MR description text + desc (str): MR description text Returns: tuple: (source_branch, last_commit_hash) or (None, None) if not parseable """ # Extract source branch from "## date: source_branch" line - source_match = re.search(r'^## [^:]+: (.+)$', description, re.MULTILINE) + source_match = re.search(r'^## [^:]+: (.+)$', desc, re.MULTILINE) if not source_match: return None, None source = source_match.group(1) - # Extract commits from "- hash subject" lines - commit_matches = re.findall(r'^- ([a-f0-9]+) ', description, re.MULTILINE) + # Extract commits from '- hash subject' lines (must be at least 7 chars) + commit_matches = re.findall(r'^- ([a-f0-9]{7,}) ', desc, re.MULTILINE) if not commit_matches: return None, None diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index d452fc823ca..ab4cceccac7 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1355,6 +1355,24 @@ Branch: cherry-abc""" self.assertIsNone(source) self.assertIsNone(last_hash) + def test_parse_mr_description_ignores_short_hashes(self): + """Test that short numbers in conversation log are not matched.""" + description = """## 2025-01-15: us/next + +Branch: cherry-abc123 + +Commits: +- abc123a First commit +- def456b Second commit + +### Conversation log +- 1 board built (sandbox) +- 2 tests passed""" + source, last_hash = control.parse_mr_description(description) + self.assertEqual(source, 'us/next') + # Should match def456b, not "1" or "2" from conversation log + self.assertEqual(last_hash, 'def456b') + class TestStep(unittest.TestCase): """Tests for step command.""" From patchwork Wed Dec 17 02:28:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 958 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938589; bh=Nb1Am1zGE43Wz+hpZdEleyAyKAFJjGHEXx0++pbfs6k=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=UT1tpAyvX8sJRUaSXUzABrlFH8LDeUKZSGji2kpHgNvJbrT3iWttDFzvdTN3YgjKo xoKZeUkjH8mImYBIPhX0HV3XxwiDpcRarTpFjH18cnVJhWBH+dqaSkQYUfj5YSoO1e GKHM5CpqVhS07LIYwXY7w//8v8D+TMMDEdLY7QdIV6NUTUxtOmOwTp1r82oDtCrEey QrLu5wSh9NivOQ1lVBR6hmIziCQBo+sWt3bbs/AUimXFJkifZDpiCdR0hxTzyrjfvw iW1ZRBaD1WSGHvRw4wQRanYlYxp9kNA9mGitFePCX3t3nohsOKpLFQJ37cv/UP9BtN 9F2Q7Xh4U+bDA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id D1F5E68BC7 for ; Tue, 16 Dec 2025 19:29:49 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id w6zlCZB1p6Of for ; Tue, 16 Dec 2025 19:29:49 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938589; bh=Nb1Am1zGE43Wz+hpZdEleyAyKAFJjGHEXx0++pbfs6k=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=UT1tpAyvX8sJRUaSXUzABrlFH8LDeUKZSGji2kpHgNvJbrT3iWttDFzvdTN3YgjKo xoKZeUkjH8mImYBIPhX0HV3XxwiDpcRarTpFjH18cnVJhWBH+dqaSkQYUfj5YSoO1e GKHM5CpqVhS07LIYwXY7w//8v8D+TMMDEdLY7QdIV6NUTUxtOmOwTp1r82oDtCrEey QrLu5wSh9NivOQ1lVBR6hmIziCQBo+sWt3bbs/AUimXFJkifZDpiCdR0hxTzyrjfvw iW1ZRBaD1WSGHvRw4wQRanYlYxp9kNA9mGitFePCX3t3nohsOKpLFQJ37cv/UP9BtN 9F2Q7Xh4U+bDA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id BE25368AEE for ; Tue, 16 Dec 2025 19:29:49 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938588; bh=OvAOCAJiY1sV03JAaRsePysbUqcBBOcDR/PHV0iyTlg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qGTwnZfbsfONe5uJijcUPyCO/u4ma+hp0o8FuqdNkuI5TyODyY/N3uKTL8g4ujUmC VZWjvp0CO+pDDl19DnPzcJItFroO+Z8UDHN5ABuIk8E01c+HAN/FMIzcANsf+AMQJ+ K/slpEChap22pAS/khWney/6pM/Ujw/2VUdbmpMA9VmrfDwSf8qw0LT2SFhznTmeO+ npWJ/BlOjuIwWtqduaQyp0LaXcvwnHwtqWiw8lpeV1RgmfmVvvtZaKkDCph6WdCsFx OqQzp2rmDe8U6zzQuv3kvuax3DM2fdsiYR/+qjSYIVkdMhuY0mZNVNNnndDSbkSAN8 pTMM1jXeZzvNw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DEF5F6897B; Tue, 16 Dec 2025 19:29:48 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id YbYNgnOhVihl; Tue, 16 Dec 2025 19:29:48 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938584; bh=QAFkC9amMove4mrLtgwOGimXCWDiwfa9BtASeRNXfqU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CdkjjgEI/wgMbLXRYm041Ood9Coxkcz6JKX2GD2FwKo4QT/AISX56cCdkYfcsxtGz xJM2FfrKeBW1S9OAUreyyQpVk9WJW4LB/OkURoYVuFCfr27TbVFxKSIhK4lAsC0eSs sm/84b31OD25g1kdRH+0zkWOldxsCJesr6nZm//mwU+gHgCCWyZ8r1IrVPSBZxknFi ObL/YyySzdB9uUcZ+tl4F+6TeqUP8YP60HePn7UcJtuW4jn5mGmqgUDkRSRzXbdaV+ aNi/8ZCkIGJf0nGAobl2puNw5zG8n55+gkiaaTC9E0Nnwds61OjVqjgYvN5qEdIocO ycwW+1wNY1W5w== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 685F46884F; Tue, 16 Dec 2025 19:29:44 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:04 -0700 Message-ID: <20251217022823.392557-16-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: MZJUVDUWJTSAJKUK7BQWN76HSLXXJ22X X-Message-ID-Hash: MZJUVDUWJTSAJKUK7BQWN76HSLXXJ22X X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 15/24] pickman: Add database tracking for comment processing List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add schema v3 with a processed_comment table to track which MR comments have been addressed by the review agent. This prevents re-processing the same comments when running review or poll commands. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 11 ++++++ tools/pickman/database.py | 59 +++++++++++++++++++++++++++- tools/pickman/ftest.py | 82 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+), 1 deletion(-) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index a382c98eac0..a9d6809d074 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -222,6 +222,17 @@ Tables - ``url``: URL to the merge request - ``created_at``: Timestamp when the MR was created +**comment** + Tracks MR comments that have been processed by the review agent. + + - ``id``: Primary key + - ``mr_iid``: GitLab merge request IID + - ``comment_id``: GitLab comment/note ID + - ``processed_at``: Timestamp when the comment was processed + + This table prevents the same comment from being addressed multiple times + when running ``review`` or ``poll`` commands. + Configuration ------------- diff --git a/tools/pickman/database.py b/tools/pickman/database.py index c8ed8a6df09..b8da21caf58 100644 --- a/tools/pickman/database.py +++ b/tools/pickman/database.py @@ -11,6 +11,7 @@ To adjust the schema, increment LATEST, create a _migrate_to_v() function and add code in migrate_to() to call it. """ +from datetime import datetime import os import sqlite3 @@ -18,7 +19,7 @@ from u_boot_pylib import tools from u_boot_pylib import tout # Schema version (version 0 means there is no database yet) -LATEST = 2 +LATEST = 3 # Default database filename DB_FNAME = '.pickman.db' @@ -129,6 +130,17 @@ class Database: # pylint: disable=too-many-public-methods 'created_at TEXT, ' 'FOREIGN KEY (source_id) REFERENCES source(id))') + def _create_v3(self): + """Migrate database to v3 schema - add comment table""" + # Table for tracking processed MR comments + self.cur.execute( + 'CREATE TABLE comment (' + 'id INTEGER PRIMARY KEY AUTOINCREMENT, ' + 'mr_iid INTEGER, ' + 'comment_id INTEGER, ' + 'processed_at TEXT, ' + 'UNIQUE(mr_iid, comment_id))') + def migrate_to(self, dest_version): """Migrate the database to the selected version @@ -151,6 +163,8 @@ class Database: # pylint: disable=too-many-public-methods self._create_v1() elif version == 2: self._create_v2() + elif version == 3: + self._create_v3() self.cur.execute('DELETE FROM schema_version') self.cur.execute( @@ -413,3 +427,46 @@ class Database: # pylint: disable=too-many-public-methods """ self.execute( 'UPDATE mergereq SET status = ? WHERE mr_id = ?', (status, mr_id)) + + # comment functions + + def comment_is_processed(self, mr_iid, comment_id): + """Check if a comment has been processed + + Args: + mr_iid (int): Merge request IID + comment_id (int): Comment ID + + Return: + bool: True if already processed + """ + res = self.execute( + 'SELECT id FROM comment WHERE mr_iid = ? AND comment_id = ?', + (mr_iid, comment_id)) + return res.fetchone() is not None + + def comment_mark_processed(self, mr_iid, comment_id): + """Mark a comment as processed + + Args: + mr_iid (int): Merge request IID + comment_id (int): Comment ID + """ + self.execute( + 'INSERT OR IGNORE INTO comment ' + '(mr_iid, comment_id, processed_at) VALUES (?, ?, ?)', + (mr_iid, comment_id, datetime.now().isoformat())) + + def comment_get_processed(self, mr_iid): + """Get all processed comment IDs for an MR + + Args: + mr_iid (int): Merge request IID + + Return: + list: List of comment IDs + """ + res = self.execute( + 'SELECT comment_id FROM comment WHERE mr_iid = ?', + (mr_iid,)) + return [row[0] for row in res.fetchall()] diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index ab4cceccac7..ae4ceaceaaa 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -721,6 +721,88 @@ class TestDatabaseCommitMergereq(unittest.TestCase): dbs.close() +class TestDatabaseComment(unittest.TestCase): + """Tests for Database comment functions.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + + def tearDown(self): + """Clean up test fixtures.""" + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + + def test_comment_mark_and_check_processed(self): + """Test marking and checking processed comments""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + + # Comment should not be processed initially + self.assertFalse(dbs.comment_is_processed(123, 456)) + + # Mark as processed + dbs.comment_mark_processed(123, 456) + dbs.commit() + + # Now should be processed + self.assertTrue(dbs.comment_is_processed(123, 456)) + + # Different comment should not be processed + self.assertFalse(dbs.comment_is_processed(123, 789)) + self.assertFalse(dbs.comment_is_processed(999, 456)) + + dbs.close() + + def test_comment_get_processed(self): + """Test getting all processed comments for an MR""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + + # Mark several comments as processed + dbs.comment_mark_processed(100, 1) + dbs.comment_mark_processed(100, 2) + dbs.comment_mark_processed(100, 3) + dbs.comment_mark_processed(200, 10) # Different MR + dbs.commit() + + # Get processed for MR 100 + processed = dbs.comment_get_processed(100) + self.assertEqual(len(processed), 3) + self.assertIn(1, processed) + self.assertIn(2, processed) + self.assertIn(3, processed) + self.assertNotIn(10, processed) + + # Get processed for MR 200 + processed = dbs.comment_get_processed(200) + self.assertEqual(len(processed), 1) + self.assertIn(10, processed) + + dbs.close() + + def test_comment_mark_processed_idempotent(self): + """Test that marking same comment twice doesn't fail""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + + # Mark same comment twice (should not raise) + dbs.comment_mark_processed(123, 456) + dbs.comment_mark_processed(123, 456) + dbs.commit() + + # Should still be processed + self.assertTrue(dbs.comment_is_processed(123, 456)) + + dbs.close() + + class TestListSources(unittest.TestCase): """Tests for list-sources command.""" From patchwork Wed Dec 17 02:28:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 959 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938595; bh=ymn+vsJG8h0ShD6SQ19qqdKlwPTWX5AsDNbRxSVKCKo=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=d8gaPNGyzOQLqg/6bH0CKIoI1DQPEgAnGmbHa4vKuSKkibHKDW2uZQriqweFpmviX nWzqgxjosgSiw42/UzlTeRepwS/WqNFylWuG6Mx2iY18nFTvb/f3ct51XWT9ktRfAo tdqeCf0ty5Ks749XOA2KSqBDXaHTBEpjuTlT144ni9CbGQm4VF6pheyfX+K6vStEk7 cuUFTYzLO/XeSft9NZglyHph0ewES8r5ksBsS5tP2w+5KW+00Q5h04zOi8OzzIwRzf NH8zYio34Xhb9ceyOniPI/u94Mxt475a2B65AeABuXNwyKdhSKeR9f3L3N7R7Di/gW lWNOMpGCWo9GA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 480DA68BC7 for ; Tue, 16 Dec 2025 19:29:55 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id nt0__LUzLa1x for ; Tue, 16 Dec 2025 19:29:55 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938595; bh=ymn+vsJG8h0ShD6SQ19qqdKlwPTWX5AsDNbRxSVKCKo=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=d8gaPNGyzOQLqg/6bH0CKIoI1DQPEgAnGmbHa4vKuSKkibHKDW2uZQriqweFpmviX nWzqgxjosgSiw42/UzlTeRepwS/WqNFylWuG6Mx2iY18nFTvb/f3ct51XWT9ktRfAo tdqeCf0ty5Ks749XOA2KSqBDXaHTBEpjuTlT144ni9CbGQm4VF6pheyfX+K6vStEk7 cuUFTYzLO/XeSft9NZglyHph0ewES8r5ksBsS5tP2w+5KW+00Q5h04zOi8OzzIwRzf NH8zYio34Xhb9ceyOniPI/u94Mxt475a2B65AeABuXNwyKdhSKeR9f3L3N7R7Di/gW lWNOMpGCWo9GA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3867A68BBD for ; Tue, 16 Dec 2025 19:29:55 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938593; bh=2EXqJQgOhPPmNx8IUBLKw1pQg/CbLhRfQVmVU4mv8Pc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EskOPdM7HAPesMP1/e8PCIez3n7r6QfTh16Huimhgcfpda/Ak98S/IP6XDKuXc6cg hwLQuejuRQNE0DiiPlf6FRqWnCGV1ntQsDFTpGqfqPX4/pBNhcLvl+t23Pc2vXhAUW Hq5NcbZ/k6zSQ+KmwOsIaf5W7ByrNdrtHjodsTnIvdR6Nq2S5MtJpKU9CkM/uLXSId QC+7wakPALfXykrtXmoG6IJmaNM8VybMRDqkrWXrOiXIUbZbEdkt7HCQ6MIK/2Ssuq RpFncwR9r9IXbUJvXgiUZLm6pWvDGDtYGS09WbBXNNNcx64KvEXvYyiwPxhuN/jzuz ts2vKCKp1l+4Q== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4F7C968AFD; Tue, 16 Dec 2025 19:29:53 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id Aa_mtwPpVGzs; Tue, 16 Dec 2025 19:29:53 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938589; bh=FU3m76pmNVDU7O5thuzqCEO689t593OUy1p1ERZBMg0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LUv+099rrH174Yaefs326/D1U/JFdS1FgDr5ZiHZZKemSejEouP0/hLKMrZEYmdta lCb9XNeeNMgNbheAK5cxcF2SRVx6bbKDXrQy0JCi9pIxwaO0ppHPcxFR2sB1Kzud4R pUjRpJ7T6XnGcwExhDVbNrnhuZDI75e7amt0mr6AatRGlBYBOXdbMM23YR8XceLQ/y WwyzyLtPV2OzggJIHIomeUn0OXP6E8wt/Nhqjgbtu6Q3vEJnjDnEybwdTs3tWuM0uh WBUqIKJx3Q68ml26hcBBx80BWep1Pzi4xedqhqJAkrmnbl7oS2xpt7ag/LwE0QlG0w wi/lNImpmAkLA== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 2B26B6884F; Tue, 16 Dec 2025 19:29:49 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:05 -0700 Message-ID: <20251217022823.392557-17-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 2QER7L7JN7WWKV4H3KU7PETLHS2EIA52 X-Message-ID-Hash: 2QER7L7JN7WWKV4H3KU7PETLHS2EIA52 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 16/24] pickman: Add a way to update a gitlab merge-request List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add function to update a merge request's description via the GitLab API. This is used to update the MR with the agent's conversation log after processing review comments. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/ftest.py | 42 +++++++++++++++++++++++++++++++++++++ tools/pickman/gitlab_api.py | 37 ++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index ae4ceaceaaa..ec9e7e91d65 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1345,6 +1345,48 @@ class TestCheckAvailable(unittest.TestCase): self.assertTrue(result) +class TestUpdateMrDescription(unittest.TestCase): + """Tests for update_mr_description function.""" + + @mock.patch.object(gitlab_api, 'get_remote_url') + @mock.patch.object(gitlab_api, 'get_token') + @mock.patch.object(gitlab_api, 'AVAILABLE', True) + def test_update_mr_description_success(self, mock_token, mock_url): + """Test successful MR description update.""" + mock_token.return_value = 'test-token' + mock_url.return_value = 'git@gitlab.com:group/project.git' + + mock_mr = mock.MagicMock() + mock_project = mock.MagicMock() + mock_project.mergerequests.get.return_value = mock_mr + + with mock.patch('gitlab.Gitlab') as mock_gitlab: + mock_gitlab.return_value.projects.get.return_value = mock_project + + result = gitlab_api.update_mr_description('origin', 123, + 'New description') + + self.assertTrue(result) + self.assertEqual(mock_mr.description, 'New description') + mock_mr.save.assert_called_once() + + @mock.patch.object(gitlab_api, 'AVAILABLE', False) + def test_update_mr_description_not_available(self): + """Test update_mr_description when gitlab not available.""" + with terminal.capture(): + result = gitlab_api.update_mr_description('origin', 123, 'desc') + self.assertFalse(result) + + @mock.patch.object(gitlab_api, 'get_token') + @mock.patch.object(gitlab_api, 'AVAILABLE', True) + def test_update_mr_description_no_token(self, mock_token): + """Test update_mr_description when no token set.""" + mock_token.return_value = None + with terminal.capture(): + result = gitlab_api.update_mr_description('origin', 123, 'desc') + self.assertFalse(result) + + class TestParseApplyWithPush(unittest.TestCase): """Tests for apply command with push options.""" diff --git a/tools/pickman/gitlab_api.py b/tools/pickman/gitlab_api.py index d0128e13f80..508168aa75c 100644 --- a/tools/pickman/gitlab_api.py +++ b/tools/pickman/gitlab_api.py @@ -320,6 +320,43 @@ def reply_to_mr(remote, mr_iid, message): return False +def update_mr_description(remote, mr_iid, desc): + """Update a merge request's description + + Args: + remote (str): Remote name + mr_iid (int): Merge request IID + desc (str): New description + + Returns: + bool: True on success + """ + if not check_available(): + return False + + token = get_token() + if not token: + tout.error('GITLAB_TOKEN environment variable not set') + return False + + remote_url = get_remote_url(remote) + host, proj_path = parse_url(remote_url) + + if not host or not proj_path: + return False + + try: + glab = gitlab.Gitlab(f'https://{host}', private_token=token) + project = glab.projects.get(proj_path) + merge_req = project.mergerequests.get(mr_iid) + merge_req.description = desc + merge_req.save() + return True + except gitlab.exceptions.GitlabError as exc: + tout.error(f'GitLab API error: {exc}') + return False + + def push_and_create_mr(remote, branch, target, title, desc=''): """Push a branch and create a merge request From patchwork Wed Dec 17 02:28:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 960 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938598; bh=ZBI3snsZDRcM5+TXuC9cjfoW28GQG5VGPxs2ckbsKI8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=oKhGNmRolOWYRxmVrTQPGzcYZFKNGU0YkU5091ln3jxlwIYxz7hv/fObQtsvzXAcW 4NHfvfvAlBTQG58xjs9Nu05QFT1oWOgZVtOFRdh0etelzfT9opLN0TbuDjSdMoD17y nTuO55S3LF4L27iBF6GgDwdJJ5HpQYjauawBJq/0Zk2q3pKE84bpV365tM8vL2VCdK b8x857CVH398zQyw1qdLyUr/HCPqEVFmyyyuASyZLTUxqzwD2DdbUKA34CGjO/14n9 33XALxnEE8ZlAimrImx2LFiVZHLo6lBGb9TMDKu3C3vW355c+cYvvFODhGxYma3DIx CdxmK235hKwiQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id EF34A68AFD for ; Tue, 16 Dec 2025 19:29:58 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id DzdX4iSyfsLE for ; Tue, 16 Dec 2025 19:29:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938598; bh=ZBI3snsZDRcM5+TXuC9cjfoW28GQG5VGPxs2ckbsKI8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=oKhGNmRolOWYRxmVrTQPGzcYZFKNGU0YkU5091ln3jxlwIYxz7hv/fObQtsvzXAcW 4NHfvfvAlBTQG58xjs9Nu05QFT1oWOgZVtOFRdh0etelzfT9opLN0TbuDjSdMoD17y nTuO55S3LF4L27iBF6GgDwdJJ5HpQYjauawBJq/0Zk2q3pKE84bpV365tM8vL2VCdK b8x857CVH398zQyw1qdLyUr/HCPqEVFmyyyuASyZLTUxqzwD2DdbUKA34CGjO/14n9 33XALxnEE8ZlAimrImx2LFiVZHLo6lBGb9TMDKu3C3vW355c+cYvvFODhGxYma3DIx CdxmK235hKwiQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DA31868BA0 for ; Tue, 16 Dec 2025 19:29:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938598; bh=3DTej5Kek2JdFAlBoRocTFG7QNfBXDZiheQxn8Blsp4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IjG8MGW84LAMBqQLwhEF/p72vsAR+pygqB4G3DNWHAUP/boou+bpi+QXMpgZowkuO 1BIr3tAAfL6je5BoHP4DifN7V8rQAUvouM1uVk4Zoah/OIEafOM+q1N44Czq4k85yZ d/8iuTsXhluYkOAZbe9ULpe/Ytd7SzmkVjTebyw6Zw9HrxeMS+bS/8NXls3sGU0Eji 5RwysAbhuYc+bdsHjC0kd/3FD0mBXUuKzVP8Bcv4CUO4AxrgxGxQQW8oZBwim2j9Ud Ndf62y/1fyQY8Wx3s+5gT9sgctzr38P/i64bzv0wFMqTiTnACXapxrztDhWiH+rbHI Qk6pT1FHUhmeQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 210596897B; Tue, 16 Dec 2025 19:29:58 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id vSfcQejwjSRy; Tue, 16 Dec 2025 19:29:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938593; bh=SDn0jHvkyBMfxSJ52O1CpEKHOGLp7kpwfzqdkBNO28k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Esp4B+nlOWNFnythHAbIrHWqta1TLPoz7JXtb/ESwe3py4sbLm01N728LBpq4yS1X zT0ZkGgwew5yFxgPYLIQyHqg3LvAL7hU1Y3XZ1SsqZ7zMvx1eyHI5Vh9k491tefR9x I+i4nNThO3f7BOrcbc/AXVIOvahH/ww8kVVu86GRe/Kztnz1sPO9IQNV95SvD/lbYA CMQYLXaayXWRS02HtAVM0TU5fmtxF9rroHyY0wktBnVoujUp8pAwiKi5abtHqvSMvA mlnG07z3Ue02RnIXiDRn/ktZPp4xpLPfQT/Mn7ZgUPPsG1TLllxFa6/kAjRu4MfEvl 2ws+f+tb6/Uqw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 92DF36884F; Tue, 16 Dec 2025 19:29:53 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:06 -0700 Message-ID: <20251217022823.392557-18-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 77YTKPPF66254PVYJE2JTL4CRG3AE7CN X-Message-ID-Hash: 77YTKPPF66254PVYJE2JTL4CRG3AE7CN X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 17/24] pickman: Improve review handling with comment tracking List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Update the review agent to: - Create local branch with version suffix (-v2, -v3, etc.) - Force push to original remote branch to update existing MR - Use --keep-empty when rebasing to preserve empty merge commits After processing comments: - Mark them as processed in database to avoid reprocessing - Update MR description with agent conversation log - Append review handling to .pickman-history Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 14 +++- tools/pickman/agent.py | 13 ++-- tools/pickman/control.py | 98 ++++++++++++++++++++++---- tools/pickman/ftest.py | 146 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 251 insertions(+), 20 deletions(-) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index a9d6809d074..d94d399ab4d 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -119,8 +119,18 @@ To check open MRs for comments and address them:: ./tools/pickman/pickman review This lists open pickman MRs (those with ``[pickman]`` in the title), checks each -for unresolved comments, and uses a Claude agent to address them. The agent will -make code changes based on the feedback and push an updated branch. +for unresolved comments, and uses a Claude agent to address them. The agent will: + +- Make code changes based on the feedback +- Create a local branch with version suffix (e.g., ``cherry-abc123-v2``) +- Force push to the original remote branch to update the existing MR +- Use ``--keep-empty`` when rebasing to preserve empty merge commits + +After processing, pickman: + +- Marks comments as processed in the database (to avoid reprocessing) +- Updates the MR description with the agent's conversation log +- Appends the review handling to ``.pickman-history`` Options for the review command: diff --git a/tools/pickman/agent.py b/tools/pickman/agent.py index 932d61be4d7..e63248a1150 100644 --- a/tools/pickman/agent.py +++ b/tools/pickman/agent.py @@ -168,16 +168,19 @@ Steps to follow: 3. For each actionable comment: - Make the requested changes to the code - Amend the relevant commit or create a fixup commit -4. Run 'buildman -L --board sandbox -w -o /tmp/pickman' to verify the build -5. Create a new branch with suffix '-v2' (or increment existing version) -6. Push the new branch: git push {remote} +4. Run 'crosfw sandbox -L' to verify the build +5. Create a local branch with suffix '-v2' (or increment: -v3, -v4, etc.) +6. Force push to the ORIGINAL remote branch to update the MR: + git push --force-with-lease {remote} HEAD:{branch_name} 7. Report what changes were made and what reply should be posted to the MR Important: - Keep changes minimal and focused on addressing the comments - If a comment is unclear or cannot be addressed, note this in your report -- Do not force push to the original branch -- The new branch name should be: {branch_name}-v2 (or -v3, -v4 etc if needed) +- Local branch: {branch_name}-v2 (or -v3, -v4 etc.) +- Remote push: always to '{branch_name}' to update the existing MR +- If rebasing is requested, use: git rebase --keep-empty + This preserves empty merge commits which are important for tracking """ options = ClaudeAgentOptions( diff --git a/tools/pickman/control.py b/tools/pickman/control.py index acb3b80a4a7..a6c371f0500 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -574,15 +574,16 @@ def do_commit_source(args, dbs): return 0 -def process_mr_reviews(remote, mrs): +def process_mr_reviews(remote, mrs, dbs): """Process review comments on open MRs Checks each MR for unresolved comments and uses Claude agent to address - them. + them. Updates MR description and .pickman-history with conversation log. Args: remote (str): Remote name mrs (list): List of MR dicts from get_open_pickman_mrs() + dbs (Database): Database instance for tracking processed comments Returns: int: Number of MRs with comments processed @@ -590,35 +591,106 @@ def process_mr_reviews(remote, mrs): processed = 0 for merge_req in mrs: - comments = gitlab_api.get_mr_comments(remote, merge_req.iid) + mr_iid = merge_req.iid + comments = gitlab_api.get_mr_comments(remote, mr_iid) if comments is None: continue - # Filter to unresolved comments - unresolved = [c for c in comments if not c.resolved] + # Filter to unresolved comments that haven't been processed + unresolved = [] + for com in comments: + if com.resolved: + continue + if dbs.comment_is_processed(mr_iid, com.id): + continue + unresolved.append(com) if not unresolved: continue tout.info('') - tout.info(f"MR !{merge_req.iid} has {len(unresolved)} comment(s):") + tout.info(f"MR !{mr_iid} has {len(unresolved)} new comment(s):") for comment in unresolved: tout.info(f' [{comment.author}]: {comment.body[:80]}...') # Run agent to handle comments - success, _ = agent.handle_mr_comments( - merge_req.iid, + success, conversation_log = agent.handle_mr_comments( + mr_iid, merge_req.source_branch, unresolved, remote, ) - if not success: - tout.error(f"Failed to handle comments for MR !{merge_req.iid}") + + if success: + # Mark comments as processed + for comment in unresolved: + dbs.comment_mark_processed(mr_iid, comment.id) + dbs.commit() + + # Update MR description with comments and conversation log + old_desc = merge_req.description + comment_summary = '\n'.join( + f"- [{c.author}]: {c.body}" + for c in unresolved + ) + new_desc = (f"{old_desc}\n\n### Review response\n\n" + f"**Comments addressed:**\n{comment_summary}\n\n" + f"**Response:**\n{conversation_log}") + gitlab_api.update_mr_description(remote, mr_iid, new_desc) + + # Update .pickman-history + update_history_with_review(merge_req.source_branch, + unresolved, conversation_log) + + tout.info(f'Updated MR !{mr_iid} description and history') + else: + tout.error(f"Failed to handle comments for MR !{mr_iid}") processed += 1 return processed -def do_review(args, dbs): # pylint: disable=unused-argument +def update_history_with_review(branch_name, comments, conversation_log): + """Append review handling to .pickman-history + + Args: + branch_name (str): Branch name for the MR + comments (list): List of comments that were addressed + conversation_log (str): Agent conversation log + """ + comment_summary = '\n'.join( + f"- [{c.author}]: {c.body[:100]}..." + for c in comments + ) + + entry = f"""### Review: {date.today()} + +Branch: {branch_name} + +Comments addressed: +{comment_summary} + +### Conversation log +{conversation_log} + +--- + +""" + + # Append to history file + existing = '' + if os.path.exists(HISTORY_FILE): + with open(HISTORY_FILE, 'r', encoding='utf-8') as fhandle: + existing = fhandle.read() + + with open(HISTORY_FILE, 'w', encoding='utf-8') as fhandle: + fhandle.write(existing + entry) + + # Commit the history file + run_git(['add', '-f', HISTORY_FILE]) + run_git(['commit', '-m', f'pickman: Record review handling for {branch_name}']) + + +def do_review(args, dbs): """Check open pickman MRs and handle comments Lists open MRs created by pickman, checks for human comments, and uses @@ -646,7 +718,7 @@ def do_review(args, dbs): # pylint: disable=unused-argument for merge_req in mrs: tout.info(f" !{merge_req.iid}: {merge_req.title}") - process_mr_reviews(remote, mrs) + process_mr_reviews(remote, mrs, dbs) return 0 @@ -765,7 +837,7 @@ def do_step(args, dbs): tout.info(f" !{merge_req.iid}: {merge_req.title}") # Process any review comments on open MRs - process_mr_reviews(remote, mrs) + process_mr_reviews(remote, mrs, dbs) tout.info('') tout.info('Not creating new MR while others are pending') diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index ec9e7e91d65..3fbb4e20dd7 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -8,6 +8,8 @@ import argparse import os +import shutil +import subprocess import sys import tempfile import unittest @@ -23,6 +25,7 @@ from u_boot_pylib import terminal from u_boot_pylib import tout from pickman import __main__ as pickman +from pickman import agent from pickman import control from pickman import database from pickman import gitlab_api @@ -1586,6 +1589,149 @@ class TestReview(unittest.TestCase): self.assertEqual(ret, 1) +class TestUpdateHistoryWithReview(unittest.TestCase): + """Tests for update_history_with_review function.""" + + def setUp(self): + """Set up test fixtures.""" + self.test_dir = tempfile.mkdtemp() + self.orig_dir = os.getcwd() + os.chdir(self.test_dir) + + # Initialize git repo + subprocess.run(['git', 'init'], check=True, capture_output=True) + subprocess.run(['git', 'config', 'user.email', 'test@test.com'], + check=True, capture_output=True) + subprocess.run(['git', 'config', 'user.name', 'Test'], + check=True, capture_output=True) + + def tearDown(self): + """Clean up test fixtures.""" + os.chdir(self.orig_dir) + shutil.rmtree(self.test_dir) + + def test_update_history_with_review(self): + """Test that review handling is appended to history.""" + comments = [ + gitlab_api.MrComment(id=1, author='reviewer1', + body='Please fix the indentation here', + created_at='2025-01-01', resolvable=True, + resolved=False), + gitlab_api.MrComment(id=2, author='reviewer2', body='Add a docstring', + created_at='2025-01-01', resolvable=True, + resolved=False), + ] + conversation_log = 'Fixed indentation and added docstring.' + + control.update_history_with_review('cherry-abc123', comments, + conversation_log) + + # Check history file was created + self.assertTrue(os.path.exists(control.HISTORY_FILE)) + + with open(control.HISTORY_FILE, 'r', encoding='utf-8') as fhandle: + content = fhandle.read() + + self.assertIn('### Review:', content) + self.assertIn('Branch: cherry-abc123', content) + self.assertIn('reviewer1', content) + self.assertIn('reviewer2', content) + self.assertIn('Fixed indentation', content) + + def test_update_history_appends(self): + """Test that review handling appends to existing history.""" + # Create existing history + with open(control.HISTORY_FILE, 'w', encoding='utf-8') as fhandle: + fhandle.write('Existing history content\n') + subprocess.run(['git', 'add', control.HISTORY_FILE], + check=True, capture_output=True) + subprocess.run(['git', 'commit', '-m', 'Initial'], + check=True, capture_output=True) + + comments = [gitlab_api.MrComment(id=1, author='reviewer', body='Fix this', + created_at='2025-01-01', resolvable=True, + resolved=False)] + control.update_history_with_review('cherry-xyz', comments, 'Fixed it') + + with open(control.HISTORY_FILE, 'r', encoding='utf-8') as fhandle: + content = fhandle.read() + + self.assertIn('Existing history content', content) + self.assertIn('### Review:', content) + + +class TestProcessMrReviewsCommentTracking(unittest.TestCase): + """Tests for comment tracking in process_mr_reviews.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.test_dir = tempfile.mkdtemp() + self.orig_dir = os.getcwd() + os.chdir(self.test_dir) + + # Initialize git repo + subprocess.run(['git', 'init'], check=True, capture_output=True) + subprocess.run(['git', 'config', 'user.email', 'test@test.com'], + check=True, capture_output=True) + subprocess.run(['git', 'config', 'user.name', 'Test'], + check=True, capture_output=True) + + def tearDown(self): + """Clean up test fixtures.""" + os.chdir(self.orig_dir) + shutil.rmtree(self.test_dir) + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + + def test_skips_processed_comments(self): + """Test that already-processed comments are skipped.""" + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + + # Mark comment as processed + dbs.comment_mark_processed(100, 1) + dbs.commit() + + mrs = [gitlab_api.PickmanMr( + iid=100, + title='[pickman] Test MR', + source_branch='cherry-test', + description='Test', + web_url='https://gitlab.com/mr/100', + )] + + # Comment 1 is processed, comment 2 is new + comments = [ + gitlab_api.MrComment(id=1, author='reviewer', body='Old comment', + created_at='2025-01-01', resolvable=True, + resolved=False), + gitlab_api.MrComment(id=2, author='reviewer', body='New comment', + created_at='2025-01-01', resolvable=True, + resolved=False), + ] + + with mock.patch.object(gitlab_api, 'get_mr_comments', + return_value=comments): + with mock.patch.object(agent, 'handle_mr_comments', + return_value=(True, 'Done')) as mock_agent: + with mock.patch.object(gitlab_api, 'update_mr_description'): + with mock.patch.object(control, 'update_history_with_review'): + control.process_mr_reviews('ci', mrs, dbs) + + # Agent should only receive the new comment + call_args = mock_agent.call_args + passed_comments = call_args[0][2] + self.assertEqual(len(passed_comments), 1) + self.assertEqual(passed_comments[0].id, 2) + + dbs.close() + + class TestParsePoll(unittest.TestCase): """Tests for poll command argument parsing.""" From patchwork Wed Dec 17 02:28:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 961 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938604; bh=dS2mLe169DDlcPUUs7bx0cg39IYhwHIH5pltFeESjho=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ncVB7OSoq5CwI+B0YKwlU+5vAdSybkCbqKkVdPPoRbCSpVxtuiKQcIOUvOBL+FwWd afXovmopqomZqKsvflDdRCAQjki/2QjtCPAMvJbEE/tVb6ktP/0mG+wa7J2JDUaHxS yS6alM7FzXtAWlVFAtYImOtcbnTB8vxNj/865ukhnrOWGxfUHwqSHt0diE1wC2H7Tk HQNpAHh1l0IiRAqjqCaoGuBAWZub9Mu8ayh0al90TMOOwK2vC+kkVFeTfzb/s7KAkP Q3AMXcYUL2Vt5yElSy9rvvL63Sv9oWFAFuumc7YVlZwS6dOo5QB+lkcTskuKNsKx1N ClOn6nzplCs2g== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6A8A068BCB for ; Tue, 16 Dec 2025 19:30:04 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id YXG0SOfswUxS for ; Tue, 16 Dec 2025 19:30:04 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938604; bh=dS2mLe169DDlcPUUs7bx0cg39IYhwHIH5pltFeESjho=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ncVB7OSoq5CwI+B0YKwlU+5vAdSybkCbqKkVdPPoRbCSpVxtuiKQcIOUvOBL+FwWd afXovmopqomZqKsvflDdRCAQjki/2QjtCPAMvJbEE/tVb6ktP/0mG+wa7J2JDUaHxS yS6alM7FzXtAWlVFAtYImOtcbnTB8vxNj/865ukhnrOWGxfUHwqSHt0diE1wC2H7Tk HQNpAHh1l0IiRAqjqCaoGuBAWZub9Mu8ayh0al90TMOOwK2vC+kkVFeTfzb/s7KAkP Q3AMXcYUL2Vt5yElSy9rvvL63Sv9oWFAFuumc7YVlZwS6dOo5QB+lkcTskuKNsKx1N ClOn6nzplCs2g== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5952B68AFD for ; Tue, 16 Dec 2025 19:30:04 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938602; bh=F1k7xn2+bZsJ81QydnMO3SyDh3O3fXze0qgZ/HnZJvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hu8KMrtwJuvHr2+am9TLCTg35dUHW/7jvz+SN0tfjGRbqckFd+IP7xj+6tkcMY5Yg zG8MOAGy311gPwn4jaMfuzI2oEGyFRY3dydvj55LuXlKuGckFHrNROQbp4r3ArOs0m 8OaeGZhRJZeMzeh+LhcRX92hIBgJQ4Q8NosfqIo9YijaN2cRfJj6VhMSJcowpmsOD6 KQvT5wWAUj4obXRsIviDygmcBJp4vfjYLHE5yjWBPhlig/6FCUjsqTkJj9ThYZygDE ZmqP9n3h2Zwc4ZZj59D3VFEEVdlcZZ2JyeQV2/8/WFV7y0FGSxoibt9nWMD1PFG0cz bDwfCTVpfAj6w== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DEB8768AFD; Tue, 16 Dec 2025 19:30:02 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 67gOqovxYfSb; Tue, 16 Dec 2025 19:30:02 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938598; bh=maiyQk/pRtCpzH7bFcmFig83N+C7qTlscrp9qhOumq0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dOug0mozbU3ga82YG4QZtSDcuuaLIrkdL+2CVfCQ+eg4A+9Cec0LB4j6hH9iqEu92 s8ejY8IDFN74EFET7uMDPfwq4avHmwafF/AEO7rTZrz86SdQRqp119iM+ZkW0U1xAc UGM8zr3VbF5Bye93KH06xQFzYeGUB9xGG8GHAQnd5Q+gkkdVbBnmfdoCATXOEGuW1b g7TIRPZUMsGPBV3lmMQSMZY6SUqu7vRVEwuqupwZ0fE5qZ25PJo+RsdF6+NIeS/9H1 E7n0SCQfUP2OTxzp0qy3RyHyWberRqowUMJHUr3WbpYmRbDiUtWMApB+Gtv9y8h3Qb H/Z7sus1OsYxg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 62E6B6884F; Tue, 16 Dec 2025 19:29:58 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:07 -0700 Message-ID: <20251217022823.392557-19-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 37EQC5JENEIEWAUQAXI62LEYZP2H4WCD X-Message-ID-Hash: 37EQC5JENEIEWAUQAXI62LEYZP2H4WCD X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 18/24] pickman: Add count-merges command List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add command to show the total number of merge commits remaining to be processed from a source branch. This helps track progress through a large backlog of merges. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 7 ++++ tools/pickman/__main__.py | 5 +++ tools/pickman/control.py | 36 +++++++++++++++++ tools/pickman/ftest.py | 83 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index d94d399ab4d..4b99de552b2 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -68,6 +68,13 @@ This finds commits between the last cherry-picked commit and the next merge commit in the source branch. It stops at the merge commit since that typically represents a logical grouping of commits (e.g., a pull request). +To count the total remaining merges to process:: + + ./tools/pickman/pickman count-merges us/next + +This shows how many merge commits remain on the first-parent chain between the +last cherry-picked commit and the source branch tip. + To show the next N merges that will be applied:: ./tools/pickman/pickman next-merges us/next diff --git a/tools/pickman/__main__.py b/tools/pickman/__main__.py index 4a60ed7eedc..1258a0835c2 100755 --- a/tools/pickman/__main__.py +++ b/tools/pickman/__main__.py @@ -54,6 +54,11 @@ def parse_args(argv): commit_src.add_argument('commit', help='Commit hash to record') subparsers.add_parser('compare', help='Compare branches') + + count_merges = subparsers.add_parser('count-merges', + help='Count remaining merges to process') + count_merges.add_argument('source', help='Source branch name') + subparsers.add_parser('list-sources', help='List tracked source branches') next_merges = subparsers.add_parser('next-merges', diff --git a/tools/pickman/control.py b/tools/pickman/control.py index a6c371f0500..446697a0d5a 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -298,6 +298,41 @@ def do_next_merges(args, dbs): return 0 +def do_count_merges(args, dbs): + """Count total remaining merges to be applied from a source + + Args: + args (Namespace): Parsed arguments with 'source' attribute + dbs (Database): Database instance + + Returns: + int: 0 on success, 1 if source not found + """ + source = args.source + + # Get the last cherry-picked commit from database + last_commit = dbs.source_get(source) + + if not last_commit: + tout.error(f"Source '{source}' not found in database") + return 1 + + # Count merge commits on the first-parent chain + fp_output = run_git([ + 'log', '--first-parent', '--merges', '--oneline', + f'{last_commit}..{source}' + ]) + + if not fp_output: + tout.info('0 merges remaining') + return 0 + + count = len([line for line in fp_output.split('\n') if line]) + tout.info(f'{count} merges remaining from {source}') + + return 0 + + HISTORY_FILE = '.pickman-history' @@ -907,6 +942,7 @@ COMMANDS = { 'apply': do_apply, 'commit-source': do_commit_source, 'compare': do_compare, + 'count-merges': do_count_merges, 'list-sources': do_list_sources, 'next-merges': do_next_merges, 'next-set': do_next_set, diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 3fbb4e20dd7..3e898eff188 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1090,6 +1090,89 @@ class TestNextMerges(unittest.TestCase): self.assertIn('No merges remaining', stdout.getvalue()) +class TestCountMerges(unittest.TestCase): + """Tests for count-merges command.""" + + def setUp(self): + """Set up test fixtures.""" + fd, self.db_path = tempfile.mkstemp(suffix='.db') + os.close(fd) + os.unlink(self.db_path) + self.old_db_fname = control.DB_FNAME + control.DB_FNAME = self.db_path + database.Database.instances.clear() + + def tearDown(self): + """Clean up test fixtures.""" + control.DB_FNAME = self.old_db_fname + if os.path.exists(self.db_path): + os.unlink(self.db_path) + database.Database.instances.clear() + command.TEST_RESULT = None + + def test_count_merges(self): + """Test count-merges shows total remaining""" + # Add source to database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + dbs.close() + + database.Database.instances.clear() + + # Mock git log with merge commits (oneline format) + log_output = ( + 'aaa111a Merge branch feature-1\n' + 'bbb222b Merge branch feature-2\n' + 'ccc333c Merge branch feature-3\n' + ) + command.TEST_RESULT = command.CommandResult(stdout=log_output) + + args = argparse.Namespace(cmd='count-merges', source='us/next') + with terminal.capture() as (stdout, _): + ret = control.do_pickman(args) + self.assertEqual(ret, 0) + self.assertIn('3 merges remaining from us/next', stdout.getvalue()) + + def test_count_merges_none(self): + """Test count-merges with no merges remaining""" + # Add source to database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.source_set('us/next', 'abc123') + dbs.commit() + dbs.close() + + database.Database.instances.clear() + + command.TEST_RESULT = command.CommandResult(stdout='') + + args = argparse.Namespace(cmd='count-merges', source='us/next') + with terminal.capture() as (stdout, _): + ret = control.do_pickman(args) + self.assertEqual(ret, 0) + self.assertIn('0 merges remaining', stdout.getvalue()) + + def test_count_merges_source_not_found(self): + """Test count-merges with unknown source""" + # Create empty database + with terminal.capture(): + dbs = database.Database(self.db_path) + dbs.start() + dbs.close() + + database.Database.instances.clear() + + args = argparse.Namespace(cmd='count-merges', source='unknown') + with terminal.capture() as (_, stderr): + ret = control.do_pickman(args) + self.assertEqual(ret, 1) + self.assertIn("Source 'unknown' not found", stderr.getvalue()) + + class TestGetNextCommits(unittest.TestCase): """Tests for get_next_commits function.""" From patchwork Wed Dec 17 02:28:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 962 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938609; bh=kzKv7Xdsx7m/14J7R77Yz5P+/Jywg1bQ8zzHJVbM+kw=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gdaWnQlVW61kOTzDJgDqnw4p02PQuWgyMdtjBEq1WED183PF0w8jn8xXk3OiMm2kG YEL8Cm5Sc/E1VA9lAUMYbK2ftQ773bX+Swn+0VgPyVfmupM4sEQr5ojLNsCqv5aJ3I 0GNdff2chaXSHmJO2mn/+0Xdmc418+irhDt2rUueFo++8jAPX04RT3myPVaJq0C2tt iBa8NFOy1xfd0oe1o0Az/O1JWMLaTwaV3o8dzUOh44Y+FTZv1Cfh0SvDNUC3D9AhOO 3ZpXpQCV8MKCUuslwjymijQVixGSNL6FnXfcoc2ViwsA3exeUiWY7sdFdAt0ZzE/tG EorBmTo452BWg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DB4506897B for ; Tue, 16 Dec 2025 19:30:09 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Mv2CCwSQq773 for ; Tue, 16 Dec 2025 19:30:09 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938609; bh=kzKv7Xdsx7m/14J7R77Yz5P+/Jywg1bQ8zzHJVbM+kw=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gdaWnQlVW61kOTzDJgDqnw4p02PQuWgyMdtjBEq1WED183PF0w8jn8xXk3OiMm2kG YEL8Cm5Sc/E1VA9lAUMYbK2ftQ773bX+Swn+0VgPyVfmupM4sEQr5ojLNsCqv5aJ3I 0GNdff2chaXSHmJO2mn/+0Xdmc418+irhDt2rUueFo++8jAPX04RT3myPVaJq0C2tt iBa8NFOy1xfd0oe1o0Az/O1JWMLaTwaV3o8dzUOh44Y+FTZv1Cfh0SvDNUC3D9AhOO 3ZpXpQCV8MKCUuslwjymijQVixGSNL6FnXfcoc2ViwsA3exeUiWY7sdFdAt0ZzE/tG EorBmTo452BWg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id C99C768AEE for ; Tue, 16 Dec 2025 19:30:09 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938607; bh=Cue/WorZaH4eJ/2jJcmbJWqtAEdugjJm87gLpH81yL4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dRoJjPr7eB23HKJblmkxs62fRBsaP8yVHilx0po5WM0VRhnvAfMMVw6HhP0Luj4oG ry7ZhEFGGpUd4Ov4LEfkkDKjOM2z6GNddev8ZM2PVvbH6Sb/sl9nM84ARfK0BTLs7m c3DnzoCMJmwIT8MdHK8oOcJt6FdCwpa/IsMayT0514Ith1vZmtAgsIikZH2YPY5l4m MOhhxRO5f1gsZzVjbqKXykPyOEVuns1ygSnUm3iRdvGciCOhLgRDh7KImh0PXfoNjE q0A9TvvMZspmtXvKgvnmw42Hg1ByYSJr1N4gxbn290fQ1StRb5GfvOHMNdFyPdNvVe YFc2kfiZoynAQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 99ECB6897B; Tue, 16 Dec 2025 19:30:07 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id gYz03W1xasGr; Tue, 16 Dec 2025 19:30:07 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938603; bh=40bKEgZAk5JSPmQUpirluif4PvFLOkrBsoyOM/ATmDM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qq+2cOC1ROdifs+0yPtiO/59HuAFzUkPwapA2ee2vX6F5gZLMz2IiWCtrXjo9xM7N nkGWt+apzs4F05WL09CYVSJppGAt3IHVL9csqDHvXa7SVatP28Csf0c36IKrpkf877 1CNUemolwCjQuZcuvyB/oI0brubKhU8tMTftEvyUzOtVOPhvg932cpTP6jmq5bcFnC Jdw60O9avXNus6kYv9LvJbAJUSagRG7qu89e40LfmZw6LcqIrGKEvQyD9JVh4ehq/7 JiV5yHMNeNpnHttcgKj3lFk4DtasRO5U1Y95//+kwPeZFdHHGD+oq1X0xMVdqoWlxm bKAr6bvHnTLvg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 2D2C46884F; Tue, 16 Dec 2025 19:30:03 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:08 -0700 Message-ID: <20251217022823.392557-20-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: AZNENUIQELCR5ETEO42XVABQCSHOIRKS X-Message-ID-Hash: AZNENUIQELCR5ETEO42XVABQCSHOIRKS X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 19/24] pickman: Stop poll on errors List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Change poll to stop on errors rather than continuing, since errors like permission denied are not recoverable by retrying. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/agent.py | 3 +++ tools/pickman/control.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/pickman/agent.py b/tools/pickman/agent.py index e63248a1150..26f4ac3afd9 100644 --- a/tools/pickman/agent.py +++ b/tools/pickman/agent.py @@ -72,6 +72,8 @@ Steps to follow: - For regular commits: git cherry-pick -x - For merge commits (identified by "Merge" in subject): git cherry-pick -x -m 1 --allow-empty Cherry-pick one commit at a time to handle each appropriately. + IMPORTANT: Always include merge commits even if they result in empty commits. + The merge commit message is important for tracking history. 4. If there are conflicts: - Show the conflicting files - Try to resolve simple conflicts automatically @@ -91,6 +93,7 @@ Important: - Stop immediately if there's a conflict that cannot be auto-resolved - Do not force push or modify history - If cherry-pick fails, run 'git cherry-pick --abort' +- NEVER skip merge commits - always use --allow-empty to preserve them """ options = ClaudeAgentOptions( diff --git a/tools/pickman/control.py b/tools/pickman/control.py index 446697a0d5a..d9d326a0d21 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -907,7 +907,7 @@ def do_poll(args, dbs): try: ret = do_step(args, dbs) if ret != 0: - tout.warning(f'Step returned {ret}, continuing anyway...') + tout.warning(f'step returned {ret}') tout.info('') tout.info(f'Sleeping {interval} seconds...') time.sleep(interval) From patchwork Wed Dec 17 02:28:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 963 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938613; bh=8lGWb7UiZfCdhftVZrKnRIjs+XKC9PBMfV0QRX/4zTk=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ZQVoXJOKDjr2he4Iz4Q4OrG2J1qr1pNuxCKY/uz0JiA6t8JjJTmafmmyn/I/yigko H74ZVV8e3pUOr/lLGwyimw6ZWvNMFhhuzx82RaxJ7IGLpBqCU8aGwfNl808lq4K8GQ F9DiTMVPgW4T11z96xGtrVjz6jU1mjgu1BR3QjQmcda5ufVgZI+jHYYZ6+a1Zw4+tS FxosJppMBkw3ysTvLYH/5UH8gmNl22ZeqURA2/4nSjgMILuxMe6HNaAgUEt4+OdTuo xSHTaHjy1vW60OBb2HMiwEuxn4ZUafSbS0iaeNGyRwIbgOknou+aeFesaGu2TuCayf Cavn/Yc7uEhgg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5C04D68BC7 for ; Tue, 16 Dec 2025 19:30:13 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id dkv0AFs-mg0L for ; Tue, 16 Dec 2025 19:30:13 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938613; bh=8lGWb7UiZfCdhftVZrKnRIjs+XKC9PBMfV0QRX/4zTk=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=ZQVoXJOKDjr2he4Iz4Q4OrG2J1qr1pNuxCKY/uz0JiA6t8JjJTmafmmyn/I/yigko H74ZVV8e3pUOr/lLGwyimw6ZWvNMFhhuzx82RaxJ7IGLpBqCU8aGwfNl808lq4K8GQ F9DiTMVPgW4T11z96xGtrVjz6jU1mjgu1BR3QjQmcda5ufVgZI+jHYYZ6+a1Zw4+tS FxosJppMBkw3ysTvLYH/5UH8gmNl22ZeqURA2/4nSjgMILuxMe6HNaAgUEt4+OdTuo xSHTaHjy1vW60OBb2HMiwEuxn4ZUafSbS0iaeNGyRwIbgOknou+aeFesaGu2TuCayf Cavn/Yc7uEhgg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4BDE668BA8 for ; Tue, 16 Dec 2025 19:30:13 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938612; bh=qJ7qX4a1oqfDsnv7UsQlWFW0dUQO5HRt9Xe5D74JJvU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RUj8umUfV4Erdx/8oeT1HJ4wpONBB+dM+GoNNHxeNWMxcJDEpibhVxWd+EKK/w0b0 2ebBTetJUTjANUwwSxtuKCtTlkRs/WWnt8j8NRjh9x7aJi7ZTIUiVGvO64vNj4UydO SeXTc2BFJg0ZGbed9mktjIgvjf+qLk3phdglD2eXc8UPmq8bE/ly8kVqYMaXuPB8pr fHH7yDIYwnf/eRtNTG8S9rbGdDrwhfehzw8+8BUZfloD1HIsdLwdwFHpoxkieGSru+ 1Ko6RlJSb/o9JJN/ZZSF4j4sJxzmAsGM68TzspV/XrCwv81LAcxahvP/QbHe0OB0Qw S8xXYsTImNWFA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 53F4E68AFD; Tue, 16 Dec 2025 19:30:12 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id NbMTmvoJHNRk; Tue, 16 Dec 2025 19:30:12 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938608; bh=342ZcnMtFPOmK4nGcWrWtsRbeHFekg42kR9ce+y0WkE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GII6I8nCXSWFfhNQge4pMrrKhaKHGwxKlS5aMxl71XW06b0ItXbO7n0GmI5brFwvD NA3j7t6PtGBdP+G384qd7+mUvaB1ieLslL5AQuwvjcZ5YKRA3i3ta8z+3P/WknmYbZ nssCdpp3ogxKhRoAoxdq+3yChPJtw12eublcIR3ZICVZ/gKyZti2T5i5IlTqkit5Q2 TZYWOJC3GBhYtarQqHqVHLpc0wIOWpHsUnfZv2bUxl93OvWEBxNmVemgjFFaqqdQl9 4wbVcFfnEBI8MTFY6c04byC4p8tPfTQvyNWtlC7YFN4x/HmW4lqXTLTzJkB1fwERS6 tfXr+y430kHCg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id DE0626884F; Tue, 16 Dec 2025 19:30:07 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:09 -0700 Message-ID: <20251217022823.392557-21-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: QYLEYEEYJ3FSIQHTY3K5KHPA4P3ZL33Z X-Message-ID-Hash: QYLEYEEYJ3FSIQHTY3K5KHPA4P3ZL33Z X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 20/24] pickman: Add config-file support for GitLab token List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add support for storing a GitLab API token in ~/.config/pickman.conf instead of an environment variable. This allows using a dedicated bot account for pickman without affecting the user's personal GitLab credentials. Config file format: [gitlab] token = glpat-xxxxxxxxxxxxxxxxxxxx This falls back to GITLAB_TOKEN environment variable if config file is not present. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 17 ++++++++++--- tools/pickman/ftest.py | 49 +++++++++++++++++++++++++++++++++++++ tools/pickman/gitlab_api.py | 39 ++++++++++++++++++++++++++++- 3 files changed, 101 insertions(+), 4 deletions(-) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index 4b99de552b2..ce520f09a45 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -192,9 +192,20 @@ To use the ``-p`` (push) option for GitLab integration, install python-gitlab:: pip install python-gitlab -You will also need a GitLab API token set in the ``GITLAB_TOKEN`` environment -variable. See `GitLab Personal Access Tokens`_ for instructions on creating one. -The token needs ``api`` scope. +You will also need a GitLab API token. The token can be configured in a config +file or environment variable. Pickman checks in this order: + +1. Config file ``~/.config/pickman.conf``:: + + [gitlab] + token = glpat-xxxxxxxxxxxxxxxxxxxx + +2. ``GITLAB_TOKEN`` environment variable +3. ``GITLAB_API_TOKEN`` environment variable + +See `GitLab Personal Access Tokens`_ for instructions on creating a token. +The token needs ``api`` scope. Using a dedicated bot account for pickman is +recommended. .. _GitLab Personal Access Tokens: https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 3e898eff188..66e087e6625 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1431,6 +1431,55 @@ class TestCheckAvailable(unittest.TestCase): self.assertTrue(result) +class TestConfigFile(unittest.TestCase): + """Tests for config file support.""" + + def setUp(self): + """Set up test fixtures.""" + self.config_dir = tempfile.mkdtemp() + self.config_file = os.path.join(self.config_dir, 'pickman.conf') + + def tearDown(self): + """Clean up test fixtures.""" + shutil.rmtree(self.config_dir) + + def test_get_token_from_config(self): + """Test getting token from config file.""" + with open(self.config_file, 'w', encoding='utf-8') as fhandle: + fhandle.write('[gitlab]\ntoken = test-config-token\n') + + with mock.patch.object(gitlab_api, 'CONFIG_FILE', self.config_file): + token = gitlab_api.get_token() + self.assertEqual(token, 'test-config-token') + + def test_get_token_fallback_to_env(self): + """Test falling back to environment variable.""" + # Config file doesn't exist + with mock.patch.object(gitlab_api, 'CONFIG_FILE', '/nonexistent/path'): + with mock.patch.dict(os.environ, {'GITLAB_TOKEN': 'env-token'}): + token = gitlab_api.get_token() + self.assertEqual(token, 'env-token') + + def test_get_token_config_missing_section(self): + """Test config file without gitlab section.""" + with open(self.config_file, 'w', encoding='utf-8') as fhandle: + fhandle.write('[other]\nkey = value\n') + + with mock.patch.object(gitlab_api, 'CONFIG_FILE', self.config_file): + with mock.patch.dict(os.environ, {'GITLAB_TOKEN': 'env-token'}): + token = gitlab_api.get_token() + self.assertEqual(token, 'env-token') + + def test_get_config_value(self): + """Test get_config_value function.""" + with open(self.config_file, 'w', encoding='utf-8') as fhandle: + fhandle.write('[section1]\nkey1 = value1\n') + + with mock.patch.object(gitlab_api, 'CONFIG_FILE', self.config_file): + value = gitlab_api.get_config_value('section1', 'key1') + self.assertEqual(value, 'value1') + + class TestUpdateMrDescription(unittest.TestCase): """Tests for update_mr_description function.""" diff --git a/tools/pickman/gitlab_api.py b/tools/pickman/gitlab_api.py index 508168aa75c..d2297f40c93 100644 --- a/tools/pickman/gitlab_api.py +++ b/tools/pickman/gitlab_api.py @@ -6,6 +6,7 @@ """GitLab integration for pickman - push branches and create merge requests.""" from collections import namedtuple +import configparser import os import re import sys @@ -50,12 +51,48 @@ def check_available(): return True +CONFIG_FILE = os.path.expanduser('~/.config/pickman.conf') + + +def get_config_value(section, key): + """Get a value from the pickman config file + + Args: + section (str): Config section name + key (str): Config key name + + Returns: + str: Value or None if not found + """ + if not os.path.exists(CONFIG_FILE): + return None + + config = configparser.ConfigParser() + config.read(CONFIG_FILE) + + try: + return config.get(section, key) + except (configparser.NoSectionError, configparser.NoOptionError): + return None + + def get_token(): - """Get GitLab API token from environment + """Get GitLab API token from config file or environment + + Checks in order: + 1. Config file (~/.config/pickman.conf) [gitlab] token + 2. GITLAB_TOKEN environment variable + 3. GITLAB_API_TOKEN environment variable Returns: str: Token or None if not set """ + # Try config file first + token = get_config_value('gitlab', 'token') + if token: + return token + + # Fall back to environment variables return os.environ.get('GITLAB_TOKEN') or os.environ.get('GITLAB_API_TOKEN') From patchwork Wed Dec 17 02:28:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 964 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938618; bh=/y4UPesyDSLV76G+PUcMcw8Pn0TSBYZz6fobrtbp7eM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=AK6Lt5oMs9+VbLplSNray2eLu4TbvM/kwtM74Fb+PhvYtnPWB3HpaHj3BF0BV9RDy hurW8cdSCf3AwFzp90jKI0l6cIReeNyZuhSLVQP85XmvbHyNp/UlcZp25rgmfFM920 jrkVJ4+wRIpkNe8xrXAnC3ZjKTsFksoWu2FvzdbMlYER9qhSPqmFJEf+cFM1FVzsr4 SLGPMPHcDWQS3CBqX4rq1+GCKko00S2Chu8uiVkOAgsWSEN6Hi30W3uOJnBtwmXtqU M3XStY+LlmXGWpLm7StLrC5K9kqXsFM9gn6fDaIqyINnbUFRtViy+omk0/aVXI/cF7 QyN6Qj16ZAsxA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 461A668BD0 for ; Tue, 16 Dec 2025 19:30:18 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id kyCgze2tb_rz for ; Tue, 16 Dec 2025 19:30:18 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938618; bh=/y4UPesyDSLV76G+PUcMcw8Pn0TSBYZz6fobrtbp7eM=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=AK6Lt5oMs9+VbLplSNray2eLu4TbvM/kwtM74Fb+PhvYtnPWB3HpaHj3BF0BV9RDy hurW8cdSCf3AwFzp90jKI0l6cIReeNyZuhSLVQP85XmvbHyNp/UlcZp25rgmfFM920 jrkVJ4+wRIpkNe8xrXAnC3ZjKTsFksoWu2FvzdbMlYER9qhSPqmFJEf+cFM1FVzsr4 SLGPMPHcDWQS3CBqX4rq1+GCKko00S2Chu8uiVkOAgsWSEN6Hi30W3uOJnBtwmXtqU M3XStY+LlmXGWpLm7StLrC5K9kqXsFM9gn6fDaIqyINnbUFRtViy+omk0/aVXI/cF7 QyN6Qj16ZAsxA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 30ED168BA8 for ; Tue, 16 Dec 2025 19:30:18 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938617; bh=6YsO/qRJ+3F/MOfZIQWKAz/mhlUGprylD2JVLUoevj0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SY/ZHqxVpLBYQdAT5KFooW/kRKJKbHqiL6GTNfHP1IDbNOSB0Qnq8ArsfbmiTVn+4 +mAUqJgmyYVmd2QqC9YJqL48CMxeGON5Fx6RlNylfxktYDTxbwetWvpsj8+t2yddS1 bIULxiihGLWSjZeLPgJZjdCJvQtmY1ZHjOehPb9JfUH9ke5sVJKJ9QTLyzID4i0JXK NLZH4aaR6kpDl9qpES5y6YtPtTLq8B2lTmlqiKS939dSYhNXPjT8VND67vw3W76yRT sOvGmAN2vXfQhNrPcBRIR/i2uCqKg90ZGVi3eu9QR+YtUeMoUmNYfy21vP+cVT+2hZ DifP455snBgQw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 1722468BA8; Tue, 16 Dec 2025 19:30:17 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id iImrkbzZk0Ew; Tue, 16 Dec 2025 19:30:17 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938612; bh=aqe1IQLyrCMpjAIb5eHhTIqa+1RhqjmjYg9h199kenQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lYQV5rEv5FYc9EdzAdJcdb23aseyBdFAlrzjxzblkPIGkLL3sq9wgTlmxmyviLUKf VGvEVzwEbbSOA1VHbkBCWdrAAXhJCV+HnabrBFyuWyXW5m56b+bhIjdWnVaAe7vgv/ ZiEqyTfvKIUATRkZwaNfnpG5Z9OCKi692Gd8F3HpkLmz+P5+M/kL1ANd/FUjLx1IyD KrDewzLRLx/InfcOY1zPQ7dHCPsWJyBfQyGqzKzY9yRUIYRUV4Q4LyJTuYunNRB8us 4+mh5i2U0iCkvjHg4mLMWxcIVcpPRY1RAoZAerH0Z8MxsYoqJ53tAFdUUOuG4P6YEU 4Se4nrx/vMQBQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 970286884F; Tue, 16 Dec 2025 19:30:12 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:10 -0700 Message-ID: <20251217022823.392557-22-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: N5ZGXFJYYR7LHU6XE3A4IY3TTC5HVMDQ X-Message-ID-Hash: N5ZGXFJYYR7LHU6XE3A4IY3TTC5HVMDQ X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 21/24] pickman: Add check-gitlab command List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add check-gitlab command to verify GitLab permissions for the configured token. This helps diagnose issues like 403 Forbidden errors when trying to create merge requests. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/README.rst | 7 +++ tools/pickman/__main__.py | 5 +++ tools/pickman/control.py | 40 +++++++++++++++++ tools/pickman/ftest.py | 53 ++++++++++++++++++++++ tools/pickman/gitlab_api.py | 88 +++++++++++++++++++++++++++++++++++++ 5 files changed, 193 insertions(+) diff --git a/tools/pickman/README.rst b/tools/pickman/README.rst index ce520f09a45..ff3ca3d58c3 100644 --- a/tools/pickman/README.rst +++ b/tools/pickman/README.rst @@ -60,6 +60,13 @@ This shows: master branch (ci/master) - The last common commit between the two branches +To check GitLab permissions for the configured token:: + + ./tools/pickman/pickman check-gitlab + +This verifies that the GitLab token has the required permissions to push +branches and create merge requests. Use ``-r`` to specify a different remote. + To show the next set of commits to cherry-pick from a source branch:: ./tools/pickman/pickman next-set us/next diff --git a/tools/pickman/__main__.py b/tools/pickman/__main__.py index 1258a0835c2..8a56976b872 100755 --- a/tools/pickman/__main__.py +++ b/tools/pickman/__main__.py @@ -48,6 +48,11 @@ def parse_args(argv): apply_cmd.add_argument('-t', '--target', default='master', help='Target branch for MR (default: master)') + check_gl = subparsers.add_parser('check-gitlab', + help='Check GitLab permissions') + check_gl.add_argument('-r', '--remote', default='ci', + help='Git remote (default: ci)') + commit_src = subparsers.add_parser('commit-source', help='Update database with last commit') commit_src.add_argument('source', help='Source branch name') diff --git a/tools/pickman/control.py b/tools/pickman/control.py index d9d326a0d21..b07ef703ba2 100644 --- a/tools/pickman/control.py +++ b/tools/pickman/control.py @@ -3,6 +3,7 @@ # Copyright 2025 Canonical Ltd. # Written by Simon Glass # +# pylint: disable=too-many-lines """Control module for pickman - handles the main logic.""" from collections import namedtuple @@ -147,6 +148,44 @@ def do_compare(args, dbs): # pylint: disable=unused-argument return 0 +def do_check_gitlab(args, dbs): # pylint: disable=unused-argument + """Check GitLab permissions for the configured token + + Args: + args (Namespace): Parsed arguments with 'remote' attribute + dbs (Database): Database instance (unused) + + Returns: + int: 0 on success with sufficient permissions, 1 otherwise + """ + remote = args.remote + + perms = gitlab_api.check_permissions(remote) + if not perms: + return 1 + + tout.info(f"GitLab permission check for remote '{remote}':") + tout.info(f" Host: {perms.host}") + tout.info(f" Project: {perms.project}") + tout.info(f" User: {perms.user}") + tout.info(f" Access level: {perms.access_name}") + tout.info('') + tout.info('Permissions:') + tout.info(f" Push branches: {'Yes' if perms.can_push else 'No'}") + tout.info(f" Create MRs: {'Yes' if perms.can_create_mr else 'No'}") + tout.info(f" Merge MRs: {'Yes' if perms.can_merge else 'No'}") + + if not perms.can_create_mr: + tout.warning('') + tout.warning('Insufficient permissions to create merge requests!') + tout.warning('The user needs at least Developer access level.') + return 1 + + tout.info('') + tout.info('All required permissions are available.') + return 0 + + def get_next_commits(dbs, source): """Get the next set of commits to cherry-pick from a source @@ -940,6 +979,7 @@ def do_test(args, dbs): # pylint: disable=unused-argument COMMANDS = { 'add-source': do_add_source, 'apply': do_apply, + 'check-gitlab': do_check_gitlab, 'commit-source': do_commit_source, 'compare': do_compare, 'count-merges': do_count_merges, diff --git a/tools/pickman/ftest.py b/tools/pickman/ftest.py index 66e087e6625..769813122fb 100644 --- a/tools/pickman/ftest.py +++ b/tools/pickman/ftest.py @@ -1480,6 +1480,59 @@ class TestConfigFile(unittest.TestCase): self.assertEqual(value, 'value1') +class TestCheckPermissions(unittest.TestCase): + """Tests for check_permissions function.""" + + @mock.patch.object(gitlab_api, 'get_remote_url') + @mock.patch.object(gitlab_api, 'get_token') + @mock.patch.object(gitlab_api, 'AVAILABLE', True) + def test_check_permissions_developer(self, mock_token, mock_url): + """Test checking permissions for a developer.""" + mock_token.return_value = 'test-token' + mock_url.return_value = 'git@gitlab.com:group/project.git' + + mock_user = mock.MagicMock() + mock_user.username = 'testuser' + mock_user.id = 123 + + mock_member = mock.MagicMock() + mock_member.access_level = 30 # Developer + + mock_project = mock.MagicMock() + mock_project.members.get.return_value = mock_member + + mock_glab = mock.MagicMock() + mock_glab.user = mock_user + mock_glab.projects.get.return_value = mock_project + + with mock.patch('gitlab.Gitlab', return_value=mock_glab): + perms = gitlab_api.check_permissions('origin') + + self.assertIsNotNone(perms) + self.assertEqual(perms.user, 'testuser') + self.assertEqual(perms.access_level, 30) + self.assertEqual(perms.access_name, 'Developer') + self.assertTrue(perms.can_push) + self.assertTrue(perms.can_create_mr) + self.assertFalse(perms.can_merge) + + @mock.patch.object(gitlab_api, 'AVAILABLE', False) + def test_check_permissions_not_available(self): + """Test check_permissions when gitlab not available.""" + with terminal.capture(): + perms = gitlab_api.check_permissions('origin') + self.assertIsNone(perms) + + @mock.patch.object(gitlab_api, 'get_token') + @mock.patch.object(gitlab_api, 'AVAILABLE', True) + def test_check_permissions_no_token(self, mock_token): + """Test check_permissions when no token set.""" + mock_token.return_value = None + with terminal.capture(): + perms = gitlab_api.check_permissions('origin') + self.assertIsNone(perms) + + class TestUpdateMrDescription(unittest.TestCase): """Tests for update_mr_description function.""" diff --git a/tools/pickman/gitlab_api.py b/tools/pickman/gitlab_api.py index d2297f40c93..0db251bd9b8 100644 --- a/tools/pickman/gitlab_api.py +++ b/tools/pickman/gitlab_api.py @@ -427,3 +427,91 @@ def push_and_create_mr(remote, branch, target, title, desc=''): tout.info(f'Merge request created: {mr_url}') return mr_url + + +# Access level constants from GitLab +ACCESS_LEVELS = { + 0: 'No access', + 5: 'Minimal access', + 10: 'Guest', + 20: 'Reporter', + 30: 'Developer', + 40: 'Maintainer', + 50: 'Owner', +} + +# Permission info returned by check_permissions() +PermissionInfo = namedtuple('PermissionInfo', [ + 'user', 'user_id', 'access_level', 'access_name', + 'can_push', 'can_create_mr', 'can_merge', 'project', 'host' +]) + + +def check_permissions(remote): # pylint: disable=too-many-return-statements + """Check GitLab permissions for the current token + + Args: + remote (str): Remote name + + Returns: + PermissionInfo: Permission info, or None on failure + """ + if not check_available(): + return None + + token = get_token() + if not token: + tout.error('No GitLab token configured') + tout.error('Set token in ~/.config/pickman.conf or GITLAB_TOKEN env var') + return None + + remote_url = get_remote_url(remote) + host, proj_path = parse_url(remote_url) + + if not host or not proj_path: + tout.error(f"Could not parse GitLab URL from remote '{remote}'") + return None + + try: + glab = gitlab.Gitlab(f'https://{host}', private_token=token) + glab.auth() + user = glab.user + + project = glab.projects.get(proj_path) + + # Get user's access level in this project + access_level = 0 + try: + # Try to get the member directly + member = project.members.get(user.id) + access_level = member.access_level + except gitlab.exceptions.GitlabGetError: + # User might have inherited access from a group + try: + member = project.members_all.get(user.id) + access_level = member.access_level + except gitlab.exceptions.GitlabGetError: + pass + + access_name = ACCESS_LEVELS.get(access_level, f'Unknown ({access_level})') + + return PermissionInfo( + user=user.username, + user_id=user.id, + access_level=access_level, + access_name=access_name, + can_push=access_level >= 30, # Developer or higher + can_create_mr=access_level >= 30, # Developer or higher + can_merge=access_level >= 40, # Maintainer or higher + project=proj_path, + host=host, + ) + except gitlab.exceptions.GitlabAuthenticationError as exc: + tout.error(f'Authentication failed: {exc}') + return None + except gitlab.exceptions.GitlabGetError as exc: + tout.error(f'Could not access project: {exc}') + return None + except gitlab.exceptions.GitlabError as exc: + tout.error(f'GitLab API error: {exc}') + return None From patchwork Wed Dec 17 02:28:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 965 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938620; bh=VXtaspFqp8TrdFiVndjf8U0277pZqvuyHbUtqhEfV7Q=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=NCD7yfQ2tRghLvkUxHp0dEDYawTsCt94G6D36E0+xrhUSQ+frkZjsqbJ4GiftIETT A51KKgl6pQZQfmEpthOHUz22fp6hNx64RJPx168BNluqPdntQGx4eyznBfD3NVDUlN p6jkq2+E2+FeT/eMmM+WCltWOcMF+9Qj0xBAj49d/Fj6NCR/bu/bvTnIJGjUie6bDq x6z6BZIhObI2HOTTvm4V1YfH9x5f9eO/TM3nXg639Y9aqIDxx9bwP17VgIYD6YeAtl /pHfnyBfYdF17gU5TimugiGxImLr9+QtbrZPdkaqCfrYw54rlP6qsf30YAGvog7ROS 7OkZvC9HpUYAQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id AF8A768BD9 for ; Tue, 16 Dec 2025 19:30:20 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 7bSwjMaoKIol for ; Tue, 16 Dec 2025 19:30:20 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938619; bh=VXtaspFqp8TrdFiVndjf8U0277pZqvuyHbUtqhEfV7Q=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=PChgbLSg0OfpcvhPqidsrlFnpWCf2YQWE2+RbH4vEsREOBrp115CaVDbzrzHZrzQ4 Fdt673co7Pq5wUbF1iA+/T8/k88e/x2zrRUBqIHPkq6SJkELpt74SbckDr+9SPNs+x Ut0uHo3V6+uCua+XSi5IsL7kNtKW4CwHEF0uznYMTa94vygU0Mg26TidhGVSegDb9r h0nYk6Jc1xKE5kDwpQo9HkIq8ovDz/ZX2nuHU8Yt2/irDNouL6nvHLt368u2I58l1+ CxV8HcwhKHmsCQSPtvnyh9FgetCG2uTWm+WfksJWmJGX5pjYL7RZiwWmGW0s7m7iGl YF+ugHTHKGdiQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A7D3168AFD for ; Tue, 16 Dec 2025 19:30:19 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938617; bh=cy8CByNnTorqPEVq99Vo5/WFXHGZ+62PcYUZu9awXGY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bkfi49bO0VTjJZlmXyt/F5GwuHmjIBV0W1jZqtoJzwDzAHnJ1mP4wrnDSA/xOaBun YRUb6UykVsiD2RKU0V2U6Y3tyl3R3ReIk9QvX4DlMefMSGpLqsuT1rtI4tKVdTW30m SwwbLnuPCythXiQ78nEVO59KgEtO7W3FhPjzkx56F//JMGmQOaSJYlToLSQqyfJyPs wBFTCaq4l/6tZ07/+mU2OMUFQPAWy8cITv5RZIfjwX9FpglQmwFydquhtMc0aPYc0f F8QDxdI0G+7FGOy1Z4AoETG9NYnAqaJewODFfI2HauRFXAj/+S3fOHcaCQK9EtuJ7v qnoUhs+BwHtzw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id EF36068AFD; Tue, 16 Dec 2025 19:30:17 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 7HV0F2uPjTuP; Tue, 16 Dec 2025 19:30:17 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938617; bh=jcGlRhSMPaMvbc2b2FrGhyoXthtg/xptW/XaXbTcUb4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WWLZQQp8DjG31sq5qVEoo08rorAPAj7RyAmjN0wK1xW0/+a6O1eUriWjveZs1aX+4 +rrk5vduw8zkgJqtXP02xg23J7aTFdlNseY1YGD9iUvS5y9IkXfSaWOtwjPUZIIaFp Y7eSP0H0/LO0v7VSreN1psGc2Qo1i1laOmYOvSH5LAJnf8BW+yOSGy9J/X6/sL3cuT e46+6adFTzalZ9VLQ5raimKzB5pSuxLVqhf6l0WBTnvpLNKWJoTuTzspw0oefCcFb/ /w3ZzvyN9Wn2Erg6opTkmpFF3+WM6Od8Kq+3N0Y8n3qnNKD867hTQzOHYsws/g8iNu ZRzmEBR3+TNzw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 5966C6884F; Tue, 16 Dec 2025 19:30:17 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:11 -0700 Message-ID: <20251217022823.392557-23-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: SNQY7XNCP42XMVO5OWGGQ7FSBQOFKKHX X-Message-ID-Hash: SNQY7XNCP42XMVO5OWGGQ7FSBQOFKKHX X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 22/24] pickman: Skip push pipeline for MR branches List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Remove SJG_LAB CI variable from push since lab tests are now triggered automatically for MR pipelines via .gitlab-ci.yml rules. Keep ci.skip to avoid running a pipeline on push; the MR pipeline will run when the merge request is created. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/pickman/gitlab_api.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/tools/pickman/gitlab_api.py b/tools/pickman/gitlab_api.py index 0db251bd9b8..50c3ec67909 100644 --- a/tools/pickman/gitlab_api.py +++ b/tools/pickman/gitlab_api.py @@ -146,10 +146,8 @@ def push_branch(remote, branch, force=False): bool: True on success """ try: - # Use ci.skip to avoid duplicate pipeline (MR pipeline will still run) - # Set SJG_LAB=1 CI variable for the MR pipeline - args = ['git', 'push', '-u', '-o', 'ci.skip', - '-o', 'ci.variable=SJG_LAB=1'] + # Skip push pipeline; MR pipeline will run when MR is created + args = ['git', 'push', '-u', '-o', 'ci.skip'] if force: args.append('--force-with-lease') args.extend([remote, branch]) From patchwork Wed Dec 17 02:28:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 966 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938625; bh=qsYalIYSSlR3902VCkiAHgQ4QILW56d9/hpUBrTylW8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=LdAi6rKruhw6GiiZViixx063hmdQnlBHrwADMwEf+DJvILBJWkNBAyliFr//sRrqW CPOLmE1YFor4zzNqKoY5f7K0pqszNKBaNqlkOztbS2NIMOk2ryhfWUEqjJdfuA54mp Fp2H4FPoYYzFsXlgsCRyhAxRBS6leeFCFwI7P94u3WJtjF++/JtkZCY20pB3TmC86K ixyYnHuPwI1o4qA9Qqw9yqUql8H3lZYllVha00txiU2P4Xd7IWMGjlX2fbGhSjzjoU MmzssxGlIbXErHP25DKL7w9bHY11hCwwDfQExx2lkrS3ZhB6+WkINT+eiQFMli3QTJ 7YwFxF4lpNjnw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3A09A68BC7 for ; Tue, 16 Dec 2025 19:30:25 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id FWsfYjd_rGL6 for ; Tue, 16 Dec 2025 19:30:25 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938625; bh=qsYalIYSSlR3902VCkiAHgQ4QILW56d9/hpUBrTylW8=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=LdAi6rKruhw6GiiZViixx063hmdQnlBHrwADMwEf+DJvILBJWkNBAyliFr//sRrqW CPOLmE1YFor4zzNqKoY5f7K0pqszNKBaNqlkOztbS2NIMOk2ryhfWUEqjJdfuA54mp Fp2H4FPoYYzFsXlgsCRyhAxRBS6leeFCFwI7P94u3WJtjF++/JtkZCY20pB3TmC86K ixyYnHuPwI1o4qA9Qqw9yqUql8H3lZYllVha00txiU2P4Xd7IWMGjlX2fbGhSjzjoU MmzssxGlIbXErHP25DKL7w9bHY11hCwwDfQExx2lkrS3ZhB6+WkINT+eiQFMli3QTJ 7YwFxF4lpNjnw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 28B0668AFD for ; Tue, 16 Dec 2025 19:30:25 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938623; bh=Yu1JXjZd+LOisxVk/eT7bxr2FFzZ3w2wk9kUcXfm4So=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CUzoZcmdTtz2/FmDaqzwWG1x5LPX2vBF7wG1cusIGzfkDCGmxZevOxKog+qKzOsS3 lfOgUt5PLXRbo5k5i79mHxixJAcKXN1p4m97BRB/kz0E/lROK0hHCnK1v/hxM5dGd2 KG/rlr2s4Kq5GRunyVk1wfkOMy2WdQj8g7QL917L88ItgaHUV0oW1GTCa4e9JeE4uQ QNXRYoGDdxNUrKEdhQxuFcpafDEdgSwFQhn7o5IrrGQI1mg3MzOTJ5qr3QEhyzSjM7 JwzC5JHoM6BgNAfV6/57OqerW1sSbpX7eTnRkqSntYFw2N6A4YoyJiSfljM8dh7ImK jcvc0iQUrvqww== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 06D2B68AFD; Tue, 16 Dec 2025 19:30:23 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id uenKteuRoomY; Tue, 16 Dec 2025 19:30:22 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938618; bh=o7c0oZo/mv02Nr3VbqR9rySkMAFrN2oOnWtQB/FW52w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f6htiWEff85Ubt/+onHQPoBYKNovjIg02sLn1b57brCEo6xmYXq+vs/oOWr/BeHmN FVzVKsYf1YSEWHXIAbDeVeNGWXPxRk3ECofL2PPnxd60nP0PvqdEChwhW5I5WRi7MZ v5wrm/IVQmIHvS/ZTy3tAboyozQQacr7PJNRLQy6zzjxcqjcNEpS9urNihzU9W+wad WdBzMz8NCQAKl75S3llDA0CKMN50PtJzOmjC0sJsYQjuLhCB2dTRkXqE2OTU4fXmuc pRHF9gaaSCv7ISNEpaBgXNPYratM0VOUvk1jMfWDEaSAvTM9w3Qm7jKL6s87vsKuIA dRm6iPfH9mzrA== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 3D46768BC7; Tue, 16 Dec 2025 19:30:18 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:12 -0700 Message-ID: <20251217022823.392557-24-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: CHFMS5GYV5YRSCE3I4O45C6YCQIRPTJH X-Message-ID-Hash: CHFMS5GYV5YRSCE3I4O45C6YCQIRPTJH X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 23/24] gitlab-ci: Run lab tests automatically for MR pipelines List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add rule to run lab tests automatically when the pipeline is triggered by a merge request event. This allows pickman-created MRs to have lab tests run without needing to set SJG_LAB=1 manually. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- .gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0d63bd4c358..ed517d123b0 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -722,6 +722,8 @@ coreboot test.py: rules: - if: $SJG_LAB == $ROLE || $SJG_LAB == "1" when: always + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + when: always - if: $SJG_LAB != "" && $SJG_LAB != "1" && $SJG_LAB != $ROLE when: never - if: $SJG_LAB == "" From patchwork Wed Dec 17 02:28:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 967 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938629; bh=jbAl9CT2dIEG+j197aiMSIdZWHs2Kgpm9zdJmixC0UA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=JreQTcMqzsteBe010xxFoFLDwcKmErmNElstKJec+FfYiczA+l+36oCITlHzJTuyX bTDciskHORMUnNXTvGhuHrMupgciFjt+01xki6QuZ4czg9KSDkQK2dSnrltbyHuNOY ocvlVPb7alZsz/7TI53kiKOScg/erL3h8ZGZyupSBKOovkobBhb7XH2lPYV1EdZBFV BmBKAwK2OY/3bCI5vlDNq1BtjXkxkjRJRJkqstShub+y26xtfPahiV4D3eCbfvmO4P lfOBMq+2YN7g8gNr8mna8kMEWGAlT7enmXce+XpPELGpNYV7R6uUPvcAVbF7s2bgeZ PogwP+TP3ZSDg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id AA30E68BB6 for ; Tue, 16 Dec 2025 19:30:29 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id cNy3Lr-DwGgs for ; Tue, 16 Dec 2025 19:30:29 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938629; bh=jbAl9CT2dIEG+j197aiMSIdZWHs2Kgpm9zdJmixC0UA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=JreQTcMqzsteBe010xxFoFLDwcKmErmNElstKJec+FfYiczA+l+36oCITlHzJTuyX bTDciskHORMUnNXTvGhuHrMupgciFjt+01xki6QuZ4czg9KSDkQK2dSnrltbyHuNOY ocvlVPb7alZsz/7TI53kiKOScg/erL3h8ZGZyupSBKOovkobBhb7XH2lPYV1EdZBFV BmBKAwK2OY/3bCI5vlDNq1BtjXkxkjRJRJkqstShub+y26xtfPahiV4D3eCbfvmO4P lfOBMq+2YN7g8gNr8mna8kMEWGAlT7enmXce+XpPELGpNYV7R6uUPvcAVbF7s2bgeZ PogwP+TP3ZSDg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 999F668AEE for ; Tue, 16 Dec 2025 19:30:29 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938627; bh=hKTPqA/EKWxwTGt3OYvA5P403Hag0ib2wAFWvirAtZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iJWKiaQJDm9PgDS6wEYMfcFSxu6eWI3WtBmJ5rbHz9IJZ+EOUj1qwsMYpoAf3d4D0 l8BczCWP8TB/21J8LJJ1DkrphRaXxNKzl5zZ2gGPtaYgLYCo0EIhdyhQnibyKeGsKD agL7FrCnm1z6GpC6uXhURKMC579GnTrUA7o7WNOiribj43ALPSiOBO/TxgqCgGdKBc 0of1wZjY1DcaXPWG5kBWx3vcjwF2s9eyg5l8Vx3cEej6l6zSnnhvkMjXj6++/4V5vo kFktf474dFK7ao3D7bf9AEyaBFywhN4a4xfe94UH0HmEnbZ+4WJ3hl9brONs00budP ZwB1whe/UG3Uw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 9DF0868AEE; Tue, 16 Dec 2025 19:30:27 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id XwufoSANYqeD; Tue, 16 Dec 2025 19:30:27 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765938623; bh=Fvo56WWLxx5VyG3Xuen1pzfsyyGv/VAyQiijXtYJL2s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uWRYufzKx96W1YTRp/3BKbe34ywfl7LgiEVkx7HRxOltD5VFOIkf+Y/xeBTTMvXjr 5a77+iqYlQq0SxDP5LCNV5Q+zpb0fsZhJMI7eQb6UFV/WKIeomBEhXetZsw1gHSQAr nxVAY1pkdPpCmm/E2VW6Xy3ySL2KYJRc8Xlt8XrWOd20bEO1aCAj5eNReOpNsC1UHG CDTejrXuaaZklcbsNsKDhq7BJSAw/YjpLkSqlcVz2NFMAIZ3VG7R2btNCZf08DB9Gn 87dcW/2G+LalAkJDLHSu7nSKHuYfFNLEhvppSMRZxWpPW/5C4zzFfo5TciPGX9ufJ0 c+XkWSSFSpdRg== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id C5D0E6884F; Tue, 16 Dec 2025 19:30:22 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Tue, 16 Dec 2025 19:28:13 -0700 Message-ID: <20251217022823.392557-25-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251217022823.392557-1-sjg@u-boot.org> References: <20251217022823.392557-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 5LRQXL2ZHNLXAQMO5Y3K6RKMV3DK3LDT X-Message-ID-Hash: 5LRQXL2ZHNLXAQMO5Y3K6RKMV3DK3LDT X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 24/24] gitlab-ci: Add pickman tests to CI List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add pickman test suite to the existing tool test job alongside binman, buildman, dtoc and patman tests. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- .gitlab-ci.yml | 6 ++++-- tools/pickman/requirements.txt | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) create mode 100644 tools/pickman/requirements.txt diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ed517d123b0..4d97291b484 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -273,7 +273,7 @@ Build tools-only and envtools: make mrproper; make tools-only_config envtools -j$(nproc) -Run binman, buildman, dtoc, hwids_to_dtsi, Kconfig and patman test suites: +Run binman, buildman, dtoc, hwids_to_dtsi, Kconfig, patman and pickman suites: extends: .test_suites script: - git config --global user.name "GitLab CI Runner"; @@ -284,7 +284,7 @@ Run binman, buildman, dtoc, hwids_to_dtsi, Kconfig and patman test suites: . /tmp/venv/bin/activate; pip install -r test/py/requirements.txt -r tools/binman/requirements.txt -r tools/buildman/requirements.txt -r tools/patman/requirements.txt - -r tools/u_boot_pylib/requirements.txt; + -r tools/pickman/requirements.txt -r tools/u_boot_pylib/requirements.txt; export UBOOT_TRAVIS_BUILD_DIR=/tmp/tools-only; export PYTHONPATH="${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc/pylibfdt"; export PATH="${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc:${PATH}"; @@ -299,6 +299,8 @@ Run binman, buildman, dtoc, hwids_to_dtsi, Kconfig and patman test suites: ./tools/buildman/buildman -t; ./tools/dtoc/dtoc -t; ./tools/patman/patman test; + ./tools/pickman/pickman test; + ./tools/pickman/pickman test -T; python3 ./test/scripts/test_hwids_to_dtsi.py; python3 -m pytest ./test/scripts/test_release_version.py; make testconfig diff --git a/tools/pickman/requirements.txt b/tools/pickman/requirements.txt new file mode 100644 index 00000000000..fb4a8c12692 --- /dev/null +++ b/tools/pickman/requirements.txt @@ -0,0 +1,4 @@ +# Requirements for pickman +# Install with: pip install -r tools/pickman/requirements.txt + +python-gitlab