Re: Testing Setup - More Tests and Automation?

[Mark Wielaard <mark@klomp.org>]

Hi Vanessa,

On Sat, Apr 30, 2022 at 02:36:47PM -0600, v wrote:
> The links are private because this is Ben's personal repository for his
> account - given that libabigail was developed under a proper organization
> it would be public. You can look at example OSS projects (e.g., tensorflow
> https://github.com/tensorflow/tensorflow/runs/6241307726?check_suite_focus=true)
> to get a sense of what the UI looks like.

That shows the same thing "Sign in to view logs".

> I understand the desire to not use proprietary software, but if you needed
> to move the setup elsewhere that's not hard to do! Modern software
> developers use GitHub, and all major companies (including RedHat) grow
> strong communities of developers on there. Podman is a good example:
> https://github.com/containers/podman. Git is git, so you could develop on
> GitHub and keep a backup on sourceware, and have an automated workflow to
> do that. In other words, there are ways to have your cake and eat it too.

I have contributed to some projects that use github (see e.g. rpm or
gccrust) but found it a major pita to be honest. Their website
basically doesn't work without javascript. And you have to depend on
people having a github account to do pull requests or accept patches
for you. Maybe it works better if you are able to create an account
and don't mind running all this proprietary javascript. But the legal
language involved in creating one is so large that I cannot even tell
if it is acceptable or not.

I happen to work for Red Hat myself as day job. But not directly
on libabigail, nor on any project that uses github. I know it is
possible to extract some stuff from github, I do keep some mirrors in
my own git setup https://code.wildebeest.org/ so I can sent patches
or pull requests. But I don't find the experience of interacting with
github using projects great.

> You have *eighteen* contributors with over 5 commits in a project that is 8
> years old, and that includes repeated individuals using different
> alias/email. Out of those 18, it looks like there are maybe 3 that have
> contributed meaningfully (and recently), and talking with Ben (and seeing
> number of commits), I get a sense that the main developer is Dodji and he
> is overwhelmed. So (to me) for a library that is as important as
> libabigail, this is problematic. The bus factor is really high.

I agree it would be good to have more active developers. I doubt it
matters whether or not the project has a github webpage. For me
personally I stopped most contributions since the licence
change. Which might or might not be a factor to others. Personally I
like contributing to projects that use strong copyleft.

> Compare
> that to Podman https://github.com/containers/podman/graphs/contributors,
> which is also a niche technology / area of work, and the project has only
> been around since late 2017 (and I don't know when it went up on GitHub
> from perhaps being private at RedHat) and it has almost 400 contributors
> (just over 100 with >= 5 commits) and many others that haven't contributed
> directly to the code (myself included) but have contributed via issues and
> other interaction.

I am not sure that libabigail and podman are comparable projects. But
I get that there are ways to interact with the project on the github
website that you prefer to interacting with the libabigail project?

> So (to me) this is a problem that needs to be solved, and I don't think
> keeping things the way that they are is that solution, regardless of how
> you innovate a bit with adding automation. The cost of not changing your
> current setup is very high, and I worry about the future of libabigail.

So which changes would you like to see? Note that I don't mind a forge
like setup, I just think that github is a pretty bad one. But
something like sourcehut, self-hosted gitlab or pagure would be pretty
cool. I do think adding automated testing is a pretty valuable thing,
especially if contributors can use it as pre-commit gating, which is
why I am working on adding buildbot and container based automation to
sourceware for all project hosted there. So I am a bit surprised you
believe that isn't part of the solution.

> As
> a developer I want to contribute and champion the work, but in its current
> state I find the project feels very closed and not trying bring me in. The
> students I spoke to at CU Boulder were inspired, but they will stop their
> exploration when they realize it's not easy to ask questions or for help.

Is that because they don't use email, bugzilla or irc? Do you think we
should have some kind of web-forum for the project?

Would their opinion change if they realize libabigail is mirrored on
sourcehut already?

> In other words, you are 100% losing out in terms of new contributors and
> developers by not using these modern practices and services.

Which modern practices and services are those exactly?

> That is of
> course your choice (and I respect the choices you make generally) but I
> strongly suggest you have a conversation with your team about this issue,
> and action you can take to address it. I really have enjoyed using
> libabigail and I think it can be better in many ways, not just the library
> itself but supported tools, automation, and branding, and this burden
> cannot just be on one or two developers.

I hope you can help and I hope we can provide you with the tools to do so.

Thanks,

Mark