public inbox for cygwin-apps-cvs@sourceware.org
help / color / mirror / Atom feed
* [calm - Cygwin server-side packaging maintenance script] branch master, updated. 20230209-5-g740d446
@ 2023-02-12 16:09 Jon Turney
  0 siblings, 0 replies; only message in thread
From: Jon Turney @ 2023-02-12 16:09 UTC (permalink / raw)
  To: cygwin-apps-cvs




https://sourceware.org/git/gitweb.cgi?p=cygwin-apps/calm.git;h=740d4468b4a3c4e40fcfb3388f62e092255c14f7

commit 740d4468b4a3c4e40fcfb3388f62e092255c14f7
Author: Jon Turney <jon.turney@dronecode.org.uk>
Date:   Sun Feb 12 12:36:57 2023 +0000

    Improve hint parser error for embedded double quote

https://sourceware.org/git/gitweb.cgi?p=cygwin-apps/calm.git;h=daf44bc4481feaea355b31c67028946426f71938

commit daf44bc4481feaea355b31c67028946426f71938
Author: Jon Turney <jon.turney@dronecode.org.uk>
Date:   Tue Feb 7 15:14:46 2023 +0000

    Give deploy status feedback into scallywag
    
    Also: isolate each individual deploy, by putting them in a subdirectory
    of stagingdir named after the job id.

https://sourceware.org/git/gitweb.cgi?p=cygwin-apps/calm.git;h=8ff2532aa62358379d33e0a28109ce808f32a35e

commit 8ff2532aa62358379d33e0a28109ce808f32a35e
Author: Jon Turney <jon.turney@dronecode.org.uk>
Date:   Mon Feb 6 15:49:48 2023 +0000

    Always remove files from stagingdir
    
    We want different treatment for files which are manually uploaded (into
    homedir) and files which arrive from the build service deployer (into
    stagingdir)
    
    Broken manual uploads were left in place so they can be inspected and
    manually fixed by someone.
    
    Currently, broken build deploys will also hang around forever, but they
    are expected to be "correct", and the initator of the build has no
    access to the stagingdir to fix things. Instead make them get
    automatically removed.

https://sourceware.org/git/gitweb.cgi?p=cygwin-apps/calm.git;h=864e73987907874171bbb8dda6702b5d4f9a4ba3

commit 864e73987907874171bbb8dda6702b5d4f9a4ba3
Author: Jon Turney <jon.turney@dronecode.org.uk>
Date:   Tue Jan 17 14:39:07 2023 +0000

    Revise maintainers module
    
    The data ordered by package name is the one we use the most often, so
    make that the primary form.
    
    Also use mtime_cache for the result of parsing cygwin-pkg-maint.

https://sourceware.org/git/gitweb.cgi?p=cygwin-apps/calm.git;h=6be5440ff6d88db80bf862d04504c74610f7ed7a

commit 6be5440ff6d88db80bf862d04504c74610f7ed7a
Author: Jon Turney <jon.turney@dronecode.org.uk>
Date:   Wed Feb 8 14:38:15 2023 +0000

    Fix invalid packageset after process_relarea
    
    If post-stale removal package set validation fails, don't change the
    packageset during process_relarea(). This makes it similar to
    process_uploads, in that the packageset can't be changed to something
    invalid.
    
    (Scenario: vaulting is requested via 'vault' command, which triggers
    process_relarea(), which applies any requested vaultings during it's
    stale evaluation.  If that fails to validate, the packageset is broken
    and any subsequent uploads will fail until after another relarea scan
    takes place)
    
    Fixes: 0939d5bd86f4 ("Add 'calm-tool vault'")


Diff:
---
 calm/calm.py                                       | 84 ++++++++++++-------
 calm/hint.py                                       |  2 +-
 calm/maintainers.py                                | 94 ++++++++++++++--------
 calm/mkgitoliteconf.py                             | 21 ++---
 calm/package.py                                    | 17 ++--
 calm/pkg2html.py                                   | 10 ++-
 calm/reports.py                                    |  5 +-
 calm/scallywag_db.py                               | 54 +++++++++++++
 calm/tool_util.py                                  | 11 +--
 test/test_calm.py                                  |  6 +-
 .../libtextcat/libtextcat-2.2-2-src.expected       |  2 +-
 .../release/libtextcat/libtextcat-2.2-2.expected   |  2 +-
 .../libtextcat-devel-2.2-2.expected                |  2 +-
 .../libtextcat0/libtextcat0-2.2-2.expected         |  2 +-
 .../libproj-devel/libproj-devel-4.8.0-1.expected   |  3 +-
 .../proj/libproj1/libproj1-4.8.0-1.expected        |  3 +-
 .../x86_64/release/proj/proj-4.8.0-1.expected      |  3 +-
 17 files changed, 207 insertions(+), 114 deletions(-)

diff --git a/calm/calm.py b/calm/calm.py
index b6e7e6e..b97607d 100755
--- a/calm/calm.py
+++ b/calm/calm.py
@@ -72,6 +72,7 @@ from . import package
 from . import pkg2html
 from . import repology
 from . import reports
+from . import scallywag_db
 from . import setup_exe
 from . import uploads
 from . import utils
@@ -121,7 +122,11 @@ def process_relarea(args, state):
     # packages can be stale due to changes made directly in the release
     # area, so first check here if there are any stale packages to vault
     if args.stale:
-        stale_to_vault = remove_stale_packages(args, packages, state)
+        fresh_packages = {}
+        for arch in common_constants.ARCHES:
+            fresh_packages[arch] = package.merge(packages[arch])
+
+        stale_to_vault = remove_stale_packages(args, fresh_packages, state)
         if stale_to_vault:
             for arch in common_constants.ARCHES + ['noarch', 'src']:
                 logging.info("vaulting %d old package(s) for arch %s" % (len(stale_to_vault[arch]), arch))
@@ -130,6 +135,8 @@ def process_relarea(args, state):
             logging.error("error while evaluating stale packages")
             return None
 
+        packages = fresh_packages
+
     # clean up any empty directories
     if not args.dryrun:
         utils.rmemptysubdirs(args.rel_area)
@@ -144,10 +151,10 @@ def process_relarea(args, state):
 
 def process_uploads(args, state):
     # read maintainer list
-    mlist = maintainers.read(args)
+    mlist = maintainers.maintainer_list(args)
 
     # make the list of all packages
-    all_packages = maintainers.all_packages(mlist)
+    all_packages = maintainers.all_packages(args.pkglist)
 
     # for each maintainer
     for name in sorted(mlist.keys()):
@@ -155,7 +162,14 @@ def process_uploads(args, state):
 
         with logfilters.AttrFilter(maint=m.name):
             process_maintainer_uploads(args, state, all_packages, m, args.homedir, 'upload')
-            process_maintainer_uploads(args, state, all_packages, m, args.stagingdir, 'staging')
+
+    # for each deploy job
+    def deploy_upload(r):
+        m = mlist[r.user]
+        with logfilters.AttrFilter(maint=m.name):
+            return process_maintainer_uploads(args, state, all_packages, m, os.path.join(args.stagingdir, r.id), 'staging', scrub=True)
+
+    scallywag_db.do_deploys(deploy_upload)
 
     # record updated reminder times for maintainers
     maintainers.update_reminder_times(mlist)
@@ -163,14 +177,12 @@ def process_uploads(args, state):
     return state.packages
 
 
-def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
-    name = m.name
-
+def process_maintainer_uploads(args, state, all_packages, m, basedir, desc, scrub=False):
     # for each arch and noarch
     scan_result = {}
-    skip_maintainer = False
+    success = True
     for arch in common_constants.ARCHES + ['noarch', 'src'] + common_constants.ARCHIVED_ARCHES:
-        logging.debug("reading uploaded arch %s packages from maintainer %s" % (arch, name))
+        logging.debug("reading uploaded arch %s packages from maintainer %s" % (arch, m.name))
 
         # read uploads
         scan_result[arch] = uploads.scan(basedir, m, all_packages, arch, args)
@@ -179,18 +191,33 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
         uploads.remove(args, scan_result[arch].remove_always)
 
         if scan_result[arch].error:
-            logging.error("error while reading uploaded arch %s packages from maintainer %s" % (arch, name))
-            skip_maintainer = True
+            logging.error("error while reading uploaded arch %s packages from maintainer %s" % (arch, m.name))
+            success = False
             continue
 
+    if success:
+        success = _process_maintainer_uploads(scan_result, args, state, all_packages, m, basedir, desc)
+
+    # remove upload files on success in homedir, always in stagingdir
+    for arch in common_constants.ARCHES + ['noarch', 'src']:
+        if scrub or success:
+            uploads.remove(args, scan_result[arch].remove_success)
+
+    # clean up any empty directories
+    if not args.dryrun:
+        utils.rmemptysubdirs(os.path.join(basedir, m.name))
+
+    return success
+
+
+def _process_maintainer_uploads(scan_result, args, state, all_packages, m, basedir, desc):
+    name = m.name
+
     # if there are no added or removed files for this maintainer, we
     # don't have anything to do
     if not any([scan_result[a].to_relarea or scan_result[a].to_vault for a in scan_result]):
         logging.debug("nothing to do for maintainer %s" % (name))
-        skip_maintainer = True
-
-    if skip_maintainer:
-        return
+        return True
 
     # for each arch
     merged_packages = {}
@@ -219,7 +246,7 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
     # if an error occurred ...
     if not valid:
         # ... discard move list and merged_packages
-        return
+        return False
 
     # check for packages which are stale as a result of this upload,
     # which we will want in the same report
@@ -230,7 +257,7 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
         if not stale_to_vault:
             # ... discard move list and merged_packages
             logging.error("error while evaluating stale packages for %s" % (name))
-            return
+            return False
 
     # check for conflicting movelists
     conflicts = False
@@ -243,7 +270,7 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
     if conflicts:
         # ... discard move list and merged_packages
         logging.error("error while validating movelists for %s" % (name))
-        return
+        return False
 
     # for each arch and noarch
     for arch in common_constants.ARCHES + ['noarch', 'src']:
@@ -253,10 +280,11 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
         if scan_result[arch].to_vault:
             logging.info("vaulting %d package(s) for arch %s, by request" % (len(scan_result[arch].to_vault), arch))
         scan_result[arch].to_vault.move_to_vault(args)
-        uploads.remove(args, scan_result[arch].remove_success)
+
         if scan_result[arch].to_relarea:
             logging.info("adding %d package(s) for arch %s" % (len(scan_result[arch].to_relarea), arch))
         scan_result[arch].to_relarea.move_to_relarea(m, args, desc)
+
         # XXX: Note that there seems to be a separate process, not run
         # from cygwin-admin's crontab, which changes the ownership of
         # files in the release area to cyguser:cygwin
@@ -273,10 +301,6 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
         # use merged package list
         state.packages[arch] = merged_packages[arch]
 
-    # clean up any empty directories
-    if not args.dryrun:
-        utils.rmemptysubdirs(os.path.join(basedir, m.name))
-
     # report what we've done
     added = []
     for arch in common_constants.ARCHES + ['noarch', 'src']:
@@ -285,6 +309,8 @@ def process_maintainer_uploads(args, state, all_packages, m, basedir, desc):
     logging.debug(msg)
     irk.irk("calm %s" % msg)
 
+    return True
+
 
 #
 #
@@ -656,13 +682,13 @@ def mail_cb(state, loghandler):
 
     # send each maintainer mail containing log entries caused by their actions,
     # or pertaining to their packages
-    #
-    # XXX: prev_maint=False here is a kind of wrong: it prevents the previous
-    # maintainer of an orphaned package from getting mails about it being
-    # altered by a trusted maintainer, but also stops them getting mails if the
-    # do something themselves...
-    mlist = maintainers.read(state.args, prev_maint=False)
+    mlist = maintainers.maintainer_list(state.args)
     for m in mlist.values():
+        # may happen for previous maintainers who orphaned all their packages
+        # before an email became mandatory
+        if not m.email:
+            continue
+
         email = m.email
         if m.name == 'ORPHANED':
             email = common_constants.EMAILS.split(',')
diff --git a/calm/hint.py b/calm/hint.py
index 28a81b0..71fa7b9 100755
--- a/calm/hint.py
+++ b/calm/hint.py
@@ -232,7 +232,7 @@ def hint_file_parse(fn, kind, strict=False):
                     errors.append('%s at line %d' % (error, i))
 
                 if (item.count('"') != 0) and (item.count('"') != 2):
-                    errors.append('embedded quote at line %d' % (i))
+                    errors.append('double-quote within double-quotes at line %d (hint files have no escape character)' % (i))
 
                 # key:value
                 match = re.match(r'^([^:\s]+):\s*(.*)$', item, re.DOTALL)
diff --git a/calm/maintainers.py b/calm/maintainers.py
index 7c1fc7d..f9ef734 100644
--- a/calm/maintainers.py
+++ b/calm/maintainers.py
@@ -35,19 +35,33 @@
 # - the timestamp when 'ignoring' warnings were last emitted
 #
 
-# XXX: Rather than this implementing an object which reads cygwin-pkg-maint when
-# constructed at specific places in the code, perhaps this needs to contain the
-# list (and it's inversion) and accessors, and invalidate that stored list when
-# cygwin-pkg-maint changes...
-
-import itertools
 import logging
 import os
 import re
-from collections import defaultdict
+from collections import UserString
 
 from . import utils
 
+
+#
+# MaintainerPackage object behaves like a string of the package name, but also
+# supports is_orphaned() and maintainers() methods
+#
+class MaintainerPackage(UserString):
+    def __init__(self, name, maintainers, orphaned):
+        super().__init__(name)
+        self._maintainers = maintainers
+        self._orphaned = orphaned
+
+    # XXX: for historical reasons, 'ORPHANED' still appears in the maintainer
+    # list, when this is True.  Probably should fix that...
+    def is_orphaned(self):
+        return self._orphaned
+
+    def maintainers(self):
+        return self._maintainers
+
+
 #
 #
 #
@@ -128,17 +142,19 @@ def add_directories(mlist, homedirs):
                             m.quiet = True
                         elif l:
                             m.email.append(l)
-        if not m.email:
-            logging.error("no email address known for maintainer '%s'" % (m.name))
 
     return mlist
 
 
 # add maintainers from the package maintainers list, with the packages they
 # maintain
-def add_packages(mlist, pkglist, prev_maint=True):
+@utils.mtime_cache
+def _read_pkglist(pkglist):
+    mpkgs = {}
+
     with open(pkglist) as f:
         for (i, l) in enumerate(f):
+            orphaned = False
             l = l.rstrip()
 
             # match lines of the form '<package> <maintainer(s)|status>'
@@ -159,12 +175,12 @@ def add_packages(mlist, pkglist, prev_maint=True):
                     # orphaned packages are assigned to 'ORPHANED'
                     elif status == 'ORPHANED':
                         m = status
+                        orphaned = True
 
-                        if prev_maint:
-                            # also add any previous maintainer(s) listed
-                            prevm = re.match(r'^ORPHANED\s\((.*)\)', rest)
-                            if prevm:
-                                m = m + '/' + prevm.group(1)
+                        # also add any previous maintainer(s) listed
+                        prevm = re.match(r'^ORPHANED\s\((.*)\)', rest)
+                        if prevm:
+                            m = m + '/' + prevm.group(1)
                     else:
                         logging.error("unknown package status '%s' in line %s:%d: '%s'" % (status, pkglist, i, l))
                         continue
@@ -172,6 +188,7 @@ def add_packages(mlist, pkglist, prev_maint=True):
                     m = rest
 
                 # joint maintainers are separated by '/'
+                maintainers = set()
                 for name in m.split('/'):
                     name = name.strip()
 
@@ -185,35 +202,46 @@ def add_packages(mlist, pkglist, prev_maint=True):
                         logging.error("non-ascii maintainer name '%s' in line %s:%d, skipped" % (rest, pkglist, i))
                         continue
 
-                    m = Maintainer._find(mlist, name)
-                    m.pkgs.append(pkg)
+                    maintainers.add(name)
+
+                mpkgs[pkg] = MaintainerPackage(pkg, maintainers, orphaned)
 
             else:
                 logging.error("unrecognized line in %s:%d: '%s'" % (pkglist, i, l))
 
-    return mlist
+    return mpkgs
+
+
+#
+def pkg_list(pkglist):
+    return _read_pkglist(pkglist)
 
 
 # create maintainer list
-def read(args, prev_maint=True):
+def maintainer_list(args):
     mlist = {}
-    mlist = add_directories(mlist, args.homedir)
-    mlist = add_packages(mlist, args.pkglist, prev_maint)
 
-    return mlist
+    # add all maintainers for all packages
+    for p in pkg_list(args.pkglist).values():
+        for m in p.maintainers():
+            Maintainer._find(mlist, m).pkgs.append(p)
 
+    # read information from homedirs
+    mlist = add_directories(mlist, args.homedir)
 
-# invert to a per-package list of maintainers
-def invert(mlist):
-    _pkgs = defaultdict(list)
-    # for each maintainer
+    # check all maintainers have an email
     for m in mlist.values():
-        # for each package
-        for p in m.pkgs:
-            # add the maintainer name
-            _pkgs[p].append(m.name)
+        if m.name == 'ORPHANED':
+            continue
 
-    return _pkgs
+        # not required if only a previous maintainer for some orphaned packages
+        if all(p.is_orphaned() for p in m.pkgs):
+            continue
+
+        if not m.email:
+            logging.error("no email address known for maintainer '%s'" % (m.name))
+
+    return mlist
 
 
 def update_reminder_times(mlist):
@@ -222,5 +250,5 @@ def update_reminder_times(mlist):
 
 
 # a list of all packages
-def all_packages(mlist):
-    return list(itertools.chain.from_iterable(mlist[m].pkgs for m in mlist))
+def all_packages(pkglist):
+    return pkg_list(pkglist).keys()
diff --git a/calm/mkgitoliteconf.py b/calm/mkgitoliteconf.py
index c67277e..b75b4bb 100755
--- a/calm/mkgitoliteconf.py
+++ b/calm/mkgitoliteconf.py
@@ -27,7 +27,6 @@
 
 import argparse
 import sys
-from collections import defaultdict
 
 from . import common_constants
 from . import maintainers
@@ -49,21 +48,8 @@ def transform_username(name):
 #
 
 def do_main(args):
-    # read maintainer list
-    mlist = {}
-    mlist = maintainers.add_packages(mlist, args.pkglist)
-
-    # make the list of all packages
-    maintainers.all_packages(mlist)
-
-    # invert to a per-package list of maintainers
-    pkgs = defaultdict(list)
-    # for each maintainer
-    for m in mlist.values():
-        # for each package
-        for p in m.pkgs:
-            # add the maintainer name
-            pkgs[p].append(m.name)
+    # Get per-package list of maintainers
+    pkgs = maintainers.pkg_list(args.pkglist)
 
     # header
     print("# automatically generated by mkgitoliteconf")
@@ -92,6 +78,9 @@ def do_main(args):
         users = ' '.join(map(transform_username, pkgs[p]))
         owner = pkgs[p][0]  # first named maintainer
 
+        if p.is_orphaned():
+            owner = 'ORPHANED'
+
         print("repo git/cygwin-packages/%s" % (p))
         print("C  = %s @leads" % (users))
         print("RW master$ = %s" % (users))
diff --git a/calm/package.py b/calm/package.py
index 81d07fe..21639cb 100755
--- a/calm/package.py
+++ b/calm/package.py
@@ -1086,13 +1086,11 @@ def validate_package_maintainers(args, packages):
     if not args.pkglist:
         return error
 
-    # read maintainer list
-    mlist = {}
-    mlist = maintainers.add_packages(mlist, args.pkglist)
-    pkg_maintainers = maintainers.invert(mlist)
+    # read package maintainer list
+    pkg_maintainers = maintainers.pkg_list(args.pkglist)
 
     # make the list of all packages
-    all_packages = maintainers.all_packages(mlist)
+    all_packages = pkg_maintainers.keys()
 
     # validate that all packages are in the package list
     for p in sorted(packages):
@@ -1120,7 +1118,7 @@ def validate_package_maintainers(args, packages):
                         logging.error("package '%s' is not obsolete, but has no maintainer" % (p))
                         error = True
 
-                if 'ORPHANED' in pkg_maintainers[es_pn]:
+                if (es_pn in pkg_maintainers) and (pkg_maintainers[es_pn].is_orphaned()):
                     # note orphaned packages
                     packages[p].orphaned = True
 
@@ -1353,8 +1351,7 @@ def write_repo_json(args, packages, f):
     for arch in packages:
         package_list.update(packages[arch])
 
-    mlist = maintainers.read(args, None)
-    pkg_maintainers = maintainers.invert(mlist)
+    pkg_maintainers = maintainers.pkg_list(args.pkglist)
 
     pl = []
     for pn in sorted(package_list):
@@ -1397,8 +1394,8 @@ def write_repo_json(args, packages, f):
         if 'license' in po.version_hints[bv]:
             d['license'] = po.version_hints[bv]['license']
 
-        if pkg_maintainers[po.orig_name] and ('ORPHANED' not in pkg_maintainers[po.orig_name]):
-            d['maintainers'] = sorted(pkg_maintainers[po.orig_name])
+        if (po.orig_name in pkg_maintainers) and (not pkg_maintainers[po.orig_name].is_orphaned()):
+            d['maintainers'] = sorted(pkg_maintainers[po.orig_name].maintainers())
 
         pl.append(d)
 
diff --git a/calm/pkg2html.py b/calm/pkg2html.py
index 2a0d25d..0e7778b 100755
--- a/calm/pkg2html.py
+++ b/calm/pkg2html.py
@@ -142,8 +142,7 @@ def update_package_listings(args, packages):
     summaries = os.path.join(args.htdocs, 'summary')
     ensure_dir_exists(args, summaries)
 
-    mlist = maintainers.read(args, None)
-    pkg_maintainers = maintainers.invert(mlist)
+    pkg_maintainers = maintainers.pkg_list(args.pkglist)
 
     toremove = glob.glob(os.path.join(summaries, '*'))
 
@@ -262,11 +261,14 @@ def update_package_listings(args, packages):
                     es_po = arch_package(packages, es)
                     if not es_po:
                         es_po = po
+
                     m_pn = es_po.orig_name
-                    if 'ORPHANED' in pkg_maintainers[m_pn]:
+                    if m_pn not in pkg_maintainers:
+                        m = None
+                    elif pkg_maintainers[m_pn].is_orphaned():
                         m = 'ORPHANED'
                     else:
-                        m = ', '.join(sorted(pkg_maintainers[m_pn]))
+                        m = ', '.join(sorted(pkg_maintainers[m_pn].maintainers()))
 
                     if m:
                         print('<span class="detail">maintainer(s)</span>: %s ' % m, file=f)
diff --git a/calm/reports.py b/calm/reports.py
index 3932d78..c2fac58 100644
--- a/calm/reports.py
+++ b/calm/reports.py
@@ -65,8 +65,7 @@ def linkify(pn, po):
 # produce a report of unmaintained packages
 #
 def unmaintained(args, packages, reportsdir):
-    mlist = maintainers.read(args, None)
-    pkg_maintainers = maintainers.invert(mlist)
+    pkg_maintainers = maintainers.pkg_list(args.pkglist)
 
     um_list = []
 
@@ -78,7 +77,7 @@ def unmaintained(args, packages, reportsdir):
         if po.kind != package.Kind.source:
             continue
 
-        if 'ORPHANED' not in pkg_maintainers[po.orig_name]:
+        if not pkg_maintainers[po.orig_name].is_orphaned():
             continue
 
         # the highest version we have
diff --git a/calm/scallywag_db.py b/calm/scallywag_db.py
new file mode 100644
index 0000000..aa37b58
--- /dev/null
+++ b/calm/scallywag_db.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+#
+# Copyright (c) 2023 Jon Turney
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+#
+
+import contextlib
+import os
+import sqlite3
+from collections import namedtuple
+
+basedir = os.path.dirname(os.path.realpath(__file__))
+dbfile = os.path.join(basedir, '..', '..', 'scallywag', 'carpetbag.db')
+
+
+def namedtuple_factory(cursor, row):
+    fields = [column[0] for column in cursor.description]
+    row_cls = namedtuple("row_cls", fields)
+    return row_cls(*row)
+
+
+def do_deploys(cb):
+    if not os.path.exists(dbfile):
+        return
+
+    with contextlib.closing(sqlite3.connect(dbfile)) as conn:
+        conn.row_factory = namedtuple_factory
+
+        cur = conn.execute("SELECT * FROM jobs WHERE status = 'deploying'")
+        rows = cur.fetchall()
+
+        for r in rows:
+            status = 'deploy successful'
+            if not cb(r):
+                status = 'deploy failed'
+
+            conn.execute("UPDATE jobs SET status = '?' WHERE id = ?", (status, r.id))
diff --git a/calm/tool_util.py b/calm/tool_util.py
index 712b0e2..f677f86 100644
--- a/calm/tool_util.py
+++ b/calm/tool_util.py
@@ -43,26 +43,19 @@ def split(pvr):
 
 
 def permitted(p):
-    # check CYGNAME is a maintainer for package
     cygname = os.environ.get('CYGNAME', None)
     if not cygname:
         logging.error("who are you?")
         return False
 
-    mlist = {}
-    mlist = maintainers.add_packages(mlist, common_constants.PKGMAINT)
-
     # CYGNAME is a maintainer for package
-    if p in mlist[cygname].pkgs:
+    pkg_list = maintainers.pkg_list(common_constants.PKGMAINT)
+    if cygname in pkg_list[p].maintainers():
         return True
 
     # CYGNAME is a trusted maintainer
     if cygname in common_constants.TRUSTEDMAINT.split('/'):
         return True
 
-    if cygname not in mlist:
-        logging.error("'%s' is not a package maintainer" % (cygname))
-        return False
-
     logging.error("package '%s' is not in the package list for maintainer '%s'" % (p, cygname))
     return False
diff --git a/test/test_calm.py b/test/test_calm.py
index bd8390d..bc39fdc 100755
--- a/test/test_calm.py
+++ b/test/test_calm.py
@@ -317,9 +317,11 @@ class CalmTest(unittest.TestCase):
     def test_maint_pkglist(self):
         self.maxDiff = None
 
+        args = types.SimpleNamespace()
+        args.homedir = 'testdata/homes'
+        args.pkglist = 'testdata/pkglist/cygwin-pkg-maint'
         mlist = {}
-        mlist = maintainers.add_directories(mlist, 'testdata/homes')
-        mlist = maintainers.add_packages(mlist, 'testdata/pkglist/cygwin-pkg-maint')
+        mlist = maintainers.maintainer_list(args)
 
         compare_with_expected_file(self, 'testdata/pkglist', mlist)
 
diff --git a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2-src.expected b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2-src.expected
index 4a76a30..d8d5310 100644
--- a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2-src.expected
+++ b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2-src.expected
@@ -7,4 +7,4 @@
           'BSD License.\n'
           'http://software.wise-guys.nl/libtextcat/"',
  'category': 'Text',
- 'parse-errors': ['embedded quote at line 7']}
+ 'parse-errors': ['double-quote within double-quotes at line 7 (hint files have no escape character)']}
diff --git a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2.expected b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2.expected
index 498e7e5..7fb8dd8 100644
--- a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2.expected
+++ b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-2.2-2.expected
@@ -8,4 +8,4 @@
           'http://software.wise-guys.nl/libtextcat/"',
  'category': 'Text',
  'requires': 'libtextcat0',
- 'parse-errors': ['embedded quote at line 7']}
+ 'parse-errors': ['double-quote within double-quotes at line 7 (hint files have no escape character)']}
diff --git a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-devel/libtextcat-devel-2.2-2.expected b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-devel/libtextcat-devel-2.2-2.expected
index f60d34d..a038cd8 100644
--- a/test/testdata/hints/x86_64/release/libtextcat/libtextcat-devel/libtextcat-devel-2.2-2.expected
+++ b/test/testdata/hints/x86_64/release/libtextcat/libtextcat-devel/libtextcat-devel-2.2-2.expected
@@ -9,4 +9,4 @@
  'category': 'Devel Text',
  'requires': 'libtextcat0 libtextcat',
  'external-source': 'libtextcat',
- 'parse-errors': ['embedded quote at line 7']}
+ 'parse-errors': ['double-quote within double-quotes at line 7 (hint files have no escape character)']}
diff --git a/test/testdata/hints/x86_64/release/libtextcat/libtextcat0/libtextcat0-2.2-2.expected b/test/testdata/hints/x86_64/release/libtextcat/libtextcat0/libtextcat0-2.2-2.expected
index 7f186d6..12d0872 100644
--- a/test/testdata/hints/x86_64/release/libtextcat/libtextcat0/libtextcat0-2.2-2.expected
+++ b/test/testdata/hints/x86_64/release/libtextcat/libtextcat0/libtextcat0-2.2-2.expected
@@ -9,4 +9,4 @@
  'category': 'Text',
  'requires': 'cygwin',
  'external-source': 'libtextcat',
- 'parse-errors': ['embedded quote at line 7']}
+ 'parse-errors': ['double-quote within double-quotes at line 7 (hint files have no escape character)']}
diff --git a/test/testdata/hints/x86_64/release/proj/libproj-devel/libproj-devel-4.8.0-1.expected b/test/testdata/hints/x86_64/release/proj/libproj-devel/libproj-devel-4.8.0-1.expected
index 620f019..5c3d624 100644
--- a/test/testdata/hints/x86_64/release/proj/libproj-devel/libproj-devel-4.8.0-1.expected
+++ b/test/testdata/hints/x86_64/release/proj/libproj-devel/libproj-devel-4.8.0-1.expected
@@ -3,4 +3,5 @@
  'sdesc': '""The PROJ Cartographic Projections Software (devel)"\n'
           'ldesc: "Cartographic projection library and utilities"',
  'external-source': 'proj',
- 'parse-errors': ['embedded quote at line 3', 'key sdesc has multi-line value']}
+ 'parse-errors': ['double-quote within double-quotes at line 3 (hint files have no escape character)',
+                  'key sdesc has multi-line value']}
diff --git a/test/testdata/hints/x86_64/release/proj/libproj1/libproj1-4.8.0-1.expected b/test/testdata/hints/x86_64/release/proj/libproj1/libproj1-4.8.0-1.expected
index cc05b73..b7487fe 100644
--- a/test/testdata/hints/x86_64/release/proj/libproj1/libproj1-4.8.0-1.expected
+++ b/test/testdata/hints/x86_64/release/proj/libproj1/libproj1-4.8.0-1.expected
@@ -3,4 +3,5 @@
  'sdesc': '""The PROJ Cartographic Projections Software (runtime)"\n'
           'ldesc: "Cartographic projection library and utilities"',
  'external-source': 'proj',
- 'parse-errors': ['embedded quote at line 3', 'key sdesc has multi-line value']}
+ 'parse-errors': ['double-quote within double-quotes at line 3 (hint files have no escape character)',
+                  'key sdesc has multi-line value']}
diff --git a/test/testdata/hints/x86_64/release/proj/proj-4.8.0-1.expected b/test/testdata/hints/x86_64/release/proj/proj-4.8.0-1.expected
index 40db455..f43b179 100644
--- a/test/testdata/hints/x86_64/release/proj/proj-4.8.0-1.expected
+++ b/test/testdata/hints/x86_64/release/proj/proj-4.8.0-1.expected
@@ -2,4 +2,5 @@
  'requires': 'libproj1',
  'sdesc': '""The PROJ Cartographic Projections Software (utilities)"\n'
           'ldesc: "Cartographic projection library and utilities"',
- 'parse-errors': ['embedded quote at line 3', 'key sdesc has multi-line value']}
+ 'parse-errors': ['double-quote within double-quotes at line 3 (hint files have no escape character)',
+                  'key sdesc has multi-line value']}


^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2023-02-12 16:09 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-12 16:09 [calm - Cygwin server-side packaging maintenance script] branch master, updated. 20230209-5-g740d446 Jon Turney

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).