From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 66A3D3858CDB for ; Sat, 25 Nov 2023 22:44:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 66A3D3858CDB Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 66A3D3858CDB Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=45.83.234.184 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700952246; cv=none; b=bWgV2crBTLPhlFkEKbBitRyQke54dOFDFhSTTtQ6iedmUkRpNO0jhskHVQurm3dpIrgRyH2RcvcphehAKsDH1G2KeSR3pogS1HidfoDNCOSqoNr8XDWpBk1MvtNnBVJrSOnssCa+IbVv4hRq5mGc2m0gnUgo0+hWcXa2VdCh51M= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700952246; c=relaxed/simple; bh=+C5NRSbaGxgV6yUBFNMo2AEush6/O6GWl8nzAxceQ5k=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=W2nYVKOyBOLdxOvdft46ubjGQ6fQQSNYz2g1CPbAIVqZkDA0fvMrBtaU0X88hOaqsnG+HCv3lv0u9bwJ4YhMyMZRRRw3mrd4K7eZqWuFWGSou5D9dUriEr+7i2KtjMBBVskhrAJwYJXfRa8eaucKeG+ITTJCzjm3c4yuXpSd/pc= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from csb.redhat.com (deer0x03.wildebeest.org [172.31.17.133]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id DD23E30291AB; Sat, 25 Nov 2023 23:44:02 +0100 (CET) Received: by csb.redhat.com (Postfix, from userid 10916) id 3B434DD484; Sat, 25 Nov 2023 23:44:01 +0100 (CET) From: Mark Wielaard To: buildbot@sourceware.org Cc: Mark Wielaard Subject: [PATCH] Upgrade to buildbot 3.9.2 on python 3.9.18 Date: Sat, 25 Nov 2023 23:43:31 +0100 Message-Id: <20231125224331.2906047-1-mark@klomp.org> X-Mailer: git-send-email 2.39.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00,GIT_PATCH_0,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,KAM_NUMSUBJECT,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Document the process and add the one buildbot patch we need (to make the Mailnotifier to work) in SETUP. --- SETUP | 58 ++++++++++++++++++++++++ buildbot.requirements | 100 ++++++++++++++++++++---------------------- 2 files changed, 106 insertions(+), 52 deletions(-) diff --git a/SETUP b/SETUP index 6a0e560..cd7149e 100644 --- a/SETUP +++ b/SETUP @@ -149,3 +149,61 @@ that the hosts are in .known_hosts (by logging in by hand once). The snapshots builder is just like a DockerLatentWorker. See also 'Adding a snapshots worker' in README_workers. + += Upgrades = + +When updating python (say from 3.6 to 3.9) it is best to just +recreate the whole python environment: + +$ mv buildbot-env buildbot-env-old +$ python3.9 -m venv buildbot-env + +To upgrade just the buildbot package: + +$ cp buildbot/builder/state.sqlite{,.old} +$ source buildbot-env/bin/activate +$ pip install --upgrade pip +$ python3 -m pip install buildbot.packages +$ python3 -m pip install $(cat buildbot.packages) + +$ python3 -m pip freeze > buildbot.requirements + +Currently you have to apply the following patch after installing +a new buildbot version, otherwise the the MailNotifier will not +sent any emails and produce the following error in twistd.log: + + [-] sending mail (1392 bytes) to ['builder@sourceware.org'] + [-] Starting factory + [ESMTPSender,client] SMTP Client retrying server. Retry: 5 + [ESMTPSender,client] SMTP Client retrying server. Retry: 4 + [ESMTPSender,client] SMTP Client retrying server. Retry: 3 + [ESMTPSender,client] SMTP Client retrying server. Retry: 2 + [-] Stopping container f5f425... + [ESMTPSender,client] SMTP Client retrying server. Retry: 1 + [ESMTPSender,client] Got exception when handling reporter events + Traceback (most recent call last): + File "/sourceware1/builder/buildbot-env/lib64/python3.9/site-packages/twisted/internet/defer.py", line 1792, in gotResult + _inlineCallbacks(r, gen, status, context) + File "/sourceware1/builder/buildbot-env/lib64/python3.9/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks + result = context.run( + File "/sourceware1/builder/buildbot-env/lib64/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator + return g.throw(self.type, self.value, self.tb) + File "/sourceware1/builder/buildbot-env/lib64/python3.9/site-packages/buildbot/reporters/base.py", line 119, in _got_event + log.err(e, 'Got exception when handling reporter events') + --- --- + File "/sourceware1/builder/buildbot-env/lib64/python3.9/site-packages/buildbot/reporters/base.py", line 117, in _got_event + yield self.sendMessage(reports) + twisted.mail._except.SMTPConnectError: Unable to connect to server. + +--- buildbot-env/lib/python3.9/site-packages/buildbot/reporters/mail.py 2023-11-25 16:25:51.113979068 +0000 ++++ buildbot-env/lib/python3.9/site-packages/buildbot/reporters/mail.py 2022-10-08 23:06:39.679022295 +0000 +@@ -336,7 +336,7 @@ + unicode2bytes(self.smtpUser), unicode2bytes(self.smtpPassword), + parseaddr(self.fromaddr)[1], recipients, BytesIO(s), + result, requireTransportSecurity=self.useTls, +- requireAuthentication=useAuth, hostname=self.relayhost) ++ requireAuthentication=useAuth) + + if self.useSmtps: + reactor.connectSSL(self.relayhost, self.smtpPort, + diff --git a/buildbot.requirements b/buildbot.requirements index 2e294e6..8af656d 100644 --- a/buildbot.requirements +++ b/buildbot.requirements @@ -1,59 +1,55 @@ -alembic==1.7.7 -attrs==21.4.0 -autobahn==21.2.1 -Automat==20.2.0 -bcrypt==3.2.0 -buildbot==3.6.1 -buildbot-badges==3.6.1 -buildbot-console-view==3.6.1 -buildbot-worker==3.6.1 -buildbot-www==3.6.1 -cairocffi==1.2.0 -CairoSVG==2.5.2 -certifi==2021.10.8 -cffi==1.15.0 -characteristic==14.3.0 -charset-normalizer==2.0.12 -constantly==15.1.0 -cryptography==36.0.2 -cssselect2==0.4.1 -dataclasses==0.8 +alembic==1.12.1 +attrs==23.1.0 +autobahn==23.6.2 +Automat==22.10.0 +bcrypt==4.0.1 +buildbot==3.9.2 +buildbot-badges==3.9.2 +buildbot-console-view==3.9.2 +buildbot-worker==3.9.2 +buildbot-www==3.9.2 +cairocffi==1.6.1 +CairoSVG==2.7.1 +certifi==2023.11.17 +cffi==1.16.0 +charset-normalizer==3.3.2 +constantly==23.10.4 +cryptography==41.0.5 +cssselect2==0.7.0 defusedxml==0.7.1 -docker==5.0.3 -future==0.18.2 -greenlet==1.1.2 +docker==6.1.3 +future==0.18.3 +greenlet==3.0.1 hyperlink==21.0.0 -idna==3.3 -importlib-metadata==4.8.3 -importlib-resources==5.4.0 -incremental==21.3.0 -Jinja2==3.0.3 -klein==21.8.0 -Mako==1.1.6 -MarkupSafe==2.0.1 -msgpack==1.0.3 -paramiko==2.11.0 -Pillow==8.4.0 -pyasn1==0.4.8 -pyasn1-modules==0.2.8 +idna==3.6 +incremental==22.10.0 +Jinja2==3.1.2 +klein==23.5.0 +Mako==1.3.0 +MarkupSafe==2.1.3 +msgpack==1.0.7 +packaging==23.2 +paramiko==3.3.1 +Pillow==10.1.0 +pyasn1==0.5.1 +pyasn1-modules==0.3.0 pycparser==2.21 -PyJWT==2.3.0 +PyJWT==2.8.0 PyNaCl==1.5.0 -pyOpenSSL==22.0.0 +pyOpenSSL==23.3.0 python-dateutil==2.8.2 -PyYAML==6.0 -requests==2.27.1 -service-identity==21.1.0 +PyYAML==6.0.1 +requests==2.31.0 +service-identity==23.1.0 six==1.16.0 -SQLAlchemy==1.4.35 -tinycss2==1.1.1 -Tubes==0.2.0 -Twisted==22.2.0 -txaio==22.2.1 -typing_extensions==4.1.1 -urllib3==1.26.9 +SQLAlchemy==1.4.50 +tinycss2==1.2.1 +Tubes==0.2.1 +Twisted==22.10.0 +txaio==23.1.1 +typing_extensions==4.8.0 +urllib3==2.1.0 webencodings==0.5.1 -websocket-client==1.3.1 -Werkzeug==2.0.3 -zipp==3.6.0 -zope.interface==5.4.0 +websocket-client==1.6.4 +Werkzeug==3.0.1 +zope.interface==6.1 -- 2.39.3