Re: inbox.sourceware.org experiment

[Mark Wielaard <mark@klomp.org>]

Hi,

On Sun, Aug 21, 2022 at 07:41:43PM +0200, Mark Wielaard via Overseers wrote:
> I did reimport some of the lists as V2 archives with full indexing. So
> you can now easily search through the following cygwin lists:
> 
> cygwin, cygwin-announce, cygwin-apps, cygwin-developers,
> cygwin-licensing, cygwin-patches and cygwin-talk
> 
> The following gcc lists:
> 
> fortran, gcc, gcc-announce, gcc-help, gcc-patches, gcc-rust,
> gnutools-advocacy, java, java-announce, java-patches, jit and
> libstdc++
> 
> But https://inbox.sourceware.org/libstdc++ doesn't work, I suspect the
> ++ should be URL escaped somehow.

This has been fixed now with a minimal inbox_re_workaround.psgi as
Eric suggested.

> And the following sourceware lists:
> 
> archer, bfd, binutils, buildbot, bunsen, bzip2-devel, c++-embedded,
> cgen, crossgcc, debugedit, docbook-tools-announce,
> docbook-tools-discuss, dominion-hackers, dwz, eclipse, ecos-announce,
> ecos-devel, ecos-discuss, ecos-maintainers, ecos-patches,
> elfutils-devel, elix, elix-announce, frysk, gas2, gdb, gdb-announce,
> gdb-patches, gnats-announce, gnats-devel, gnu-gabi, gsl-announce,
> gsl-discuss, guile-emacs, guile-gtk, infinity, insight,
> insight-announce, installshell, kawa, libabigail, libc-alpha,
> libc-announce, libc-hacker, libc-help, libc-locales, libc-ports,
> libc-stable, libffi-announce, libffi-discuss, mauve-discuss,
> mauve-patches, mingw-dvlpr, netresolve, newlib, patchutils-list,
> prelink, pthreads-win32, rda, rhdb, rhdb-admin, rhdb-cc, rhdb-explain,
> rhug-rhats, sharutils-alpha, sid, sid-announce, sourcenav,
> sourcenav-announce, sourceware-announce, springfield, systemtap,
> xconq-announce and xconq7
> 
> See also the
> mailman.lists/{cygwin.com,gcc.gnu.org,sourceware.org}.lists.full lists
> and import_{cygwin,gcc,sourceware}_from_mbox scripts in the inbox
> homedir.

It should be said that this increases the disk usage by ~3.5x. These
lists used to take ~9GB of storage, now they take ~32GB. The total
public-inbox lists storage is now 52GB. There is still 250G free
space. And when we are happy we can reclaim the lists.old 9GB of
storage.

> I did remove the "test" list, and the cronjob that kept it
> populated. But all other lists have been kept as V1 and basic
> indexing.
> 
> There are some lists which never seen any messages. I think we should
> remove them because they probably won't see any messages ever. And it
> makes looking for real lists more difficult (it is ~25% of the lists,
> 97 out of the total 260 lists are just empty).
> 
> anonymous, autobook-cvs, autobook-webpages-cvs, autoconf-cvs,
> autoconf-webpages-cvs, binutils-webpages-cvs, bzip2-cvs,
> bzip2-webpages-cvs, catapult-cvs, catapult-webpages-cvs,
> c++-embedded-cvs, c++-embedded-webpages-cvs, cgen-prs,
> cgen-webpages-cvs, cluster-webpages-cvs, cygwin-webpages-cvs, dm-cvs,
> dm-webpages-cvs, docbook-tools-hackers, docbook-tools-webpages-cvs,
> dominion-announce, dominion-cvs, dominion-discuss,
> dominion-webpages-cvs, ecos-webpages-cvs, elix-cvs, elix-webpages-cvs,
> gcc-cvs-testrun, gcc-maintainers, gcc-ppc, gcc-sc, gcc-testlist,
> gdb-webpages-cvs, gettext-alpha, gettext-announce,
> gettext-webpages-cvs, glibc-webpages-cvs, global, gnats-admin,
> gnats-webpages-cvs, gsl-webpages-cvs, guile-emacs-cvs,
> guile-webpages-cvs, insight-cvs, insight-webpages-cvs, inti-cvs, inti,
> inti-webpages-cvs, ip-over-scsi-cvs, ip-over-scsi-webpages-cvs,
> java-cvs, jffs2-webpages-cvs, kawa-cvs, kawa-webpages-cvs, libaio,
> libaio-webpages-cvs, libc-alpha1, libffi-cvs, libffi-webpages-cvs,
> libstdc++-webpages-cvs, mailer-daemon, mauve-announce, mauve-cvs,
> mauve-webpages-cvs, newlib-webpages-cvs, piranha-webpages-cvs,
> postmaster, prelink-svn, psim-cvs, psim-webpages-cvs,
> pthreads-win32-cvs, pthreads-win32-webpages-cvs, rhdb-installer,
> rhdb-jdbc, rhdb-utils, root, rpm2html, rpm2html-prs,
> sharutils-announce, sharutils-cvs, sharutils-webpages-cvs,
> sid-webpages-cvs, sourcemaster, sourcenav-prs, sourceware-cvs,
> sourceware-cvs-sourceware, sourceware-cvs-sourceware-webpages,
> sourceware-infra-cvs, sourceware-webpages-cvs, systemtap-webpages-cvs,
> testcvs-cvs, testcvs-webpages-cvs, webmaster, win32-x11-cvs,
> win32-x11-webpages-cvs, xconq-prs and xconq-webpages-cvs

I removed them all. If people want them back it will be easy since
they didn't contain any messages to begin with.

> There are also the following 9 lists, which are either private (in
> which case they show up as empty above) or not publicly advertised
> lists:
> 
> cygwin-xfree, cygwin-xfree-announce, gcc-sc, mailman, overseers,
> postmaster, root, sourcemaster and test-list.
> 
> The cygwin lists might still be interesting. Likewise for this list
> overseers. But the others probably should be removed.

I kept cygwin-xfree, cygwin-xfree-announce, overseers and test-list
(which I used to test :)

This leaves us with 162 public-inbox lists.

Cheers,

Mark