Web part of bugzilla is ready (fwd)

Web part of bugzilla is ready (fwd)

[Daniel Berlin]

The fast i can get answers to the questions below, the faster i can move
gcc to bugzilla.

Chris said he didn't know, and to mail overseers.
I seriously hope *someone* knows the answers to the questions below.
I don't have perms to the necessary directories on sources to try to
figure out the answers myself.


 On Fri, Jan 31, 2003 at 06:11:49PM -0500, Daniel Berlin wrote:
> The web portion of bugzilla is ready.
>
>
> 1. I need to know how gcc-gnats actually works (IE the mail flow).
> To auto-parse incoming gnats reports, i need to feed the entire
> message
> into a perl script that has access to bugzilla.
>
> How does mail to gcc-gnats end up on gcc-bugs? (IE is it
> auto-mirroring in qmail, or is it done by a gnats handling script).

>
> 2. How does gcc-bugs email get appended to a bug?  For bugzilla, it's,
> as one would imagine, a perl script that does it.

Re: Web part of bugzilla is ready (fwd)

[Christopher Faylor]

Ok, I did a little research.

On Mon, Feb 03, 2003 at 11:17:17PM -0500, Daniel Berlin wrote:
>The fast i can get answers to the questions below, the faster i can move
>gcc to bugzilla.
>
>Chris said he didn't know, and to mail overseers.
>I seriously hope *someone* knows the answers to the questions below.
>I don't have perms to the necessary directories on sources to try to
>figure out the answers myself.
>
>
> On Fri, Jan 31, 2003 at 06:11:49PM -0500, Daniel Berlin wrote:
>> The web portion of bugzilla is ready.
>>
>>
>> 1. I need to know how gcc-gnats actually works (IE the mail flow).
>> To auto-parse incoming gnats reports, i need to feed the entire
>> message
>> into a perl script that has access to bugzilla.
>>
>> How does mail to gcc-gnats end up on gcc-bugs? (IE is it
>> auto-mirroring in qmail, or is it done by a gnats handling script).

There is a qmail alias set up which redirects mail delivered to
gcc-gnats to a program (/usr/local/libexec/gnats/queue-pr) which
eventually sends email to gcc-bugs and gcc-prs.

If you want me to set up a similar situation for bugzilla, let me know
what and where the program is.  I assume that we should set up a
gcc-bugzilla alias and everntually gcc-bugzilla and gcc-gnats will
go to the same place.

>> 2. How does gcc-bugs email get appended to a bug?  For bugzilla, it's,
>> as one would imagine, a perl script that does it.

I think that bug data is only appended when gcc-gnats is cc'ed.

cgf

Re: Web part of bugzilla is ready (fwd)

[Jason Molenda]

On Mon, Feb 03, 2003 at 11:36:13PM -0500, Christopher Faylor wrote:
> Ok, I did a little research.

Sorry for not following up to this - I should have answered.


> >> 1. I need to know how gcc-gnats actually works (IE the mail flow).
> >> To auto-parse incoming gnats reports, i need to feed the entire
> >> message
> >> into a perl script that has access to bugzilla.
> >>
> >> How does mail to gcc-gnats end up on gcc-bugs? (IE is it
> >> auto-mirroring in qmail, or is it done by a gnats handling script).
> 
> There is a qmail alias set up which redirects mail delivered to
> gcc-gnats to a program (/usr/local/libexec/gnats/queue-pr) which
> eventually sends email to gcc-bugs and gcc-prs.

Yeah, queue-pr (and gnatsd (gnatsweb)) save the incoming PRs as
plain text in a holding directory until a crontab runs (once every
ten minutes) to read in the newly-arrived PRs and put them in their
proper place under the proper uid.

Specifically, /qmail/alias/.qmail-gcc-gnats contains

|/sourceware/infra/bin/mlcheck --preferred-domain=gcc.gnu.org
|/qmail/bin/preline /usr/local/libexec/gnats/queue-pr --directory=/sourceware/gnats/gcc-db -q

More than one program can be called out of .qmail-gcc-gnats, so if
you want to also get a copy of newly arriving gnats PRs, another
script can be handed them as they come in.  They don't yet have
their gnats #'s assigned, of course (you'd have to add a script to
the gcc-prs mailing list to see that).  And you're running under
the one of the mail processing UIDs so you want to exercise some
security caution with what you plug in here.

> >> 2. How does gcc-bugs email get appended to a bug?  For bugzilla, it's,
> >> as one would imagine, a perl script that does it.
> 
> I think that bug data is only appended when gcc-gnats is cc'ed.

Yeah, e-mail messages being appended to a bug will come in through
gcc-gnats@gcc and they'll be added to the existing PR if their PR
category/# matches correctly.

When a user changes the state of a PR or whatever through gnatsweb, a
notification is sent out to the gcc-prs list.  Note that nothing is
sent to gcc-gnats@, so if you just monitor gcc-gnats you'll only see
new PRs and PRs with e-mail additions, not state changes.  If you
monitor gcc-prs, you'll see new PRs, additions to PRs, and state
changes to PRs.  At least you _should_ see all those things. :-)


J

Re: Web part of bugzilla is ready (fwd)

[Daniel Berlin]

On Wednesday, February 5, 2003, at 03:04  AM, Jason Molenda wrote:

> On Mon, Feb 03, 2003 at 11:36:13PM -0500, Christopher Faylor wrote:
>> Ok, I did a little research.
>
> Sorry for not following up to this - I should have answered.
>
>
>>>> 1. I need to know how gcc-gnats actually works (IE the mail flow).
>>>> To auto-parse incoming gnats reports, i need to feed the entire
>>>> message
>>>> into a perl script that has access to bugzilla.
>>>>
>>>> How does mail to gcc-gnats end up on gcc-bugs? (IE is it
>>>> auto-mirroring in qmail, or is it done by a gnats handling script).
>>
>> There is a qmail alias set up which redirects mail delivered to
>> gcc-gnats to a program (/usr/local/libexec/gnats/queue-pr) which
>> eventually sends email to gcc-bugs and gcc-prs.
>
> Yeah, queue-pr (and gnatsd (gnatsweb)) save the incoming PRs as
> plain text in a holding directory until a crontab runs (once every
> ten minutes) to read in the newly-arrived PRs and put them in their
> proper place under the proper uid.
>
> Specifically, /qmail/alias/.qmail-gcc-gnats contains
>
> |/sourceware/infra/bin/mlcheck --preferred-domain=gcc.gnu.org
> |/qmail/bin/preline /usr/local/libexec/gnats/queue-pr 
> --directory=/sourceware/gnats/gcc-db -q
>
> More than one program can be called out of .qmail-gcc-gnats, so if
> you want to also get a copy of newly arriving gnats PRs, another
> script can be handed them as they come in.  They don't yet have
> their gnats #'s assigned, of course (you'd have to add a script to
> the gcc-prs mailing list to see that).
They don't need PR numbers, since they will be converted to bugzilla 
bugs.
IE when gcc moves over, we are not going to have it do insertion into 
both databases, since gnats won't be used for gcc anymore. It will take 
an incoming GNATS bug report, and insert it into bugzilla, not GNATS.

> And you're running under
> the one of the mail processing UIDs so you want to exercise some
> security caution with what you plug in here.
>

Well, it *has* to have read access to bugzilla/ and bugzilla/data, and 
write access to some temp dir (the mime decoder module wants this). But 
all the *real* data goes into the mysql database.  It's perl, so buffer 
overflows presumably won't be a concern.

>>>> 2. How does gcc-bugs email get appended to a bug?  For bugzilla, 
>>>> it's,
>>>> as one would imagine, a perl script that does it.
>>
>> I think that bug data is only appended when gcc-gnats is cc'ed.
>
> Yeah, e-mail messages being appended to a bug will come in through
> gcc-gnats@gcc and they'll be added to the existing PR if their PR
> category/# matches correctly.

Through the same type of script process as new bug reports do?

>
> When a user changes the state of a PR or whatever through gnatsweb, a
> notification is sent out to the gcc-prs list.  Note that nothing is
> sent to gcc-gnats@, so if you just monitor gcc-gnats you'll only see
> new PRs and PRs with e-mail additions, not state changes.  If you
> monitor gcc-prs, you'll see new PRs, additions to PRs, and state
> changes to PRs.  At least you _should_ see all those things. :-)
>
>
> J

Re: Web part of bugzilla is ready (fwd)

[Christopher Faylor]

Daniel,
I am not exactly sure need here but it sounds like what we need is:

1) Something to accept incoming, specially formatted email.

2) Something to generate email when a bug is entered in bugzilla.

Neither is hard to do.  I can set up 1 immediately, if you want.  We can
set up a gcc-bugzilla mailing list and have it go directly into your
database.  Just tell me what program needs to be run.

On the other side, I think that anything that shows up in bugzilla should
generate gcc-bugs email.  There's no reason to generate gcc-prs mail,
AFAICT.  You can do that now with no help from me, I think.

cgf

On Wed, Feb 05, 2003 at 07:31:22PM -0500, Daniel Berlin wrote:
>
>On Wednesday, February 5, 2003, at 03:04  AM, Jason Molenda wrote:
>
>>On Mon, Feb 03, 2003 at 11:36:13PM -0500, Christopher Faylor wrote:
>>>Ok, I did a little research.
>>
>>Sorry for not following up to this - I should have answered.
>>
>>
>>>>>1. I need to know how gcc-gnats actually works (IE the mail flow).
>>>>>To auto-parse incoming gnats reports, i need to feed the entire
>>>>>message
>>>>>into a perl script that has access to bugzilla.
>>>>>
>>>>>How does mail to gcc-gnats end up on gcc-bugs? (IE is it
>>>>>auto-mirroring in qmail, or is it done by a gnats handling script).
>>>
>>>There is a qmail alias set up which redirects mail delivered to
>>>gcc-gnats to a program (/usr/local/libexec/gnats/queue-pr) which
>>>eventually sends email to gcc-bugs and gcc-prs.
>>
>>Yeah, queue-pr (and gnatsd (gnatsweb)) save the incoming PRs as
>>plain text in a holding directory until a crontab runs (once every
>>ten minutes) to read in the newly-arrived PRs and put them in their
>>proper place under the proper uid.
>>
>>Specifically, /qmail/alias/.qmail-gcc-gnats contains
>>
>>|/sourceware/infra/bin/mlcheck --preferred-domain=gcc.gnu.org
>>|/qmail/bin/preline /usr/local/libexec/gnats/queue-pr 
>>--directory=/sourceware/gnats/gcc-db -q
>>
>>More than one program can be called out of .qmail-gcc-gnats, so if
>>you want to also get a copy of newly arriving gnats PRs, another
>>script can be handed them as they come in.  They don't yet have
>>their gnats #'s assigned, of course (you'd have to add a script to
>>the gcc-prs mailing list to see that).
>They don't need PR numbers, since they will be converted to bugzilla 
>bugs.
>IE when gcc moves over, we are not going to have it do insertion into 
>both databases, since gnats won't be used for gcc anymore. It will take 
>an incoming GNATS bug report, and insert it into bugzilla, not GNATS.
>
>>And you're running under
>>the one of the mail processing UIDs so you want to exercise some
>>security caution with what you plug in here.
>>
>
>Well, it *has* to have read access to bugzilla/ and bugzilla/data, and 
>write access to some temp dir (the mime decoder module wants this). But 
>all the *real* data goes into the mysql database.  It's perl, so buffer 
>overflows presumably won't be a concern.
>
>>>>>2. How does gcc-bugs email get appended to a bug?  For bugzilla, 
>>>>>it's,
>>>>>as one would imagine, a perl script that does it.
>>>
>>>I think that bug data is only appended when gcc-gnats is cc'ed.
>>
>>Yeah, e-mail messages being appended to a bug will come in through
>>gcc-gnats@gcc and they'll be added to the existing PR if their PR
>>category/# matches correctly.
>
>Through the same type of script process as new bug reports do?
>
>>
>>When a user changes the state of a PR or whatever through gnatsweb, a
>>notification is sent out to the gcc-prs list.  Note that nothing is
>>sent to gcc-gnats@, so if you just monitor gcc-gnats you'll only see
>>new PRs and PRs with e-mail additions, not state changes.  If you
>>monitor gcc-prs, you'll see new PRs, additions to PRs, and state
>>changes to PRs.  At least you _should_ see all those things. :-)
>>
>>
>>J

Re: Web part of bugzilla is ready (fwd)

[Daniel Berlin]

On Fri, 7 Feb 2003, Christopher Faylor wrote:

> Daniel,
> I am not exactly sure need here but it sounds like what we need is:
>
> 1) Something to accept incoming, specially formatted email.

Yup.

There are two email interfaces for gcc bugzilla.
The first takes gnats email input, for reporting bugs by email.  Thus,
this script will need to be put in place of the queue-pr command called
now, since we won't have GNATS anymore for gcc.
This script is still being worked on a bit, i haven't updated it in a
while, and i'm still running tests on it.  Shouldn't be more than a day or
two, hopefully.

The second is a more general interface, that can do more (IE do bug
searches, etc, by email). It just needs a new email address.
This script is done and ready to go, but secondary, so if you are busy at
the moment, don't worry about it.


>
> 2) Something to generate email when a bug is entered in bugzilla.
>
This is already done by bugzilla on its own.


> Neither is hard to do.  I can set up 1 immediately, if you want.  We can
> set up a gcc-bugzilla mailing list and have it go directly into your
> database.  Just tell me what program needs to be run.
>

I shall, the second i have it done.

> On the other side, I think that anything that shows up in bugzilla should
> generate gcc-bugs email.  There's no reason to generate gcc-prs mail,
> AFAICT.  You can do that now with no help from me, I think.
Yup.
Bugzilla calls a processmail script on its own when it needs to send out
mail.

>
> cgf
>
> On Wed, Feb 05, 2003 at 07:31:22PM -0500, Daniel Berlin wrote:
> >
> >On Wednesday, February 5, 2003, at 03:04  AM, Jason Molenda wrote:
> >
> >>On Mon, Feb 03, 2003 at 11:36:13PM -0500, Christopher Faylor wrote:
> >>>Ok, I did a little research.
> >>
> >>Sorry for not following up to this - I should have answered.
> >>
> >>
> >>>>>1. I need to know how gcc-gnats actually works (IE the mail flow).
> >>>>>To auto-parse incoming gnats reports, i need to feed the entire
> >>>>>message
> >>>>>into a perl script that has access to bugzilla.
> >>>>>
> >>>>>How does mail to gcc-gnats end up on gcc-bugs? (IE is it
> >>>>>auto-mirroring in qmail, or is it done by a gnats handling script).
> >>>
> >>>There is a qmail alias set up which redirects mail delivered to
> >>>gcc-gnats to a program (/usr/local/libexec/gnats/queue-pr) which
> >>>eventually sends email to gcc-bugs and gcc-prs.
> >>
> >>Yeah, queue-pr (and gnatsd (gnatsweb)) save the incoming PRs as
> >>plain text in a holding directory until a crontab runs (once every
> >>ten minutes) to read in the newly-arrived PRs and put them in their
> >>proper place under the proper uid.
> >>
> >>Specifically, /qmail/alias/.qmail-gcc-gnats contains
> >>
> >>|/sourceware/infra/bin/mlcheck --preferred-domain=gcc.gnu.org
> >>|/qmail/bin/preline /usr/local/libexec/gnats/queue-pr
> >>--directory=/sourceware/gnats/gcc-db -q
> >>
> >>More than one program can be called out of .qmail-gcc-gnats, so if
> >>you want to also get a copy of newly arriving gnats PRs, another
> >>script can be handed them as they come in.  They don't yet have
> >>their gnats #'s assigned, of course (you'd have to add a script to
> >>the gcc-prs mailing list to see that).
> >They don't need PR numbers, since they will be converted to bugzilla
> >bugs.
> >IE when gcc moves over, we are not going to have it do insertion into
> >both databases, since gnats won't be used for gcc anymore. It will take
> >an incoming GNATS bug report, and insert it into bugzilla, not GNATS.
> >
> >>And you're running under
> >>the one of the mail processing UIDs so you want to exercise some
> >>security caution with what you plug in here.
> >>
> >
> >Well, it *has* to have read access to bugzilla/ and bugzilla/data, and
> >write access to some temp dir (the mime decoder module wants this). But
> >all the *real* data goes into the mysql database.  It's perl, so buffer
> >overflows presumably won't be a concern.
> >
> >>>>>2. How does gcc-bugs email get appended to a bug?  For bugzilla,
> >>>>>it's,
> >>>>>as one would imagine, a perl script that does it.
> >>>
> >>>I think that bug data is only appended when gcc-gnats is cc'ed.
> >>
> >>Yeah, e-mail messages being appended to a bug will come in through
> >>gcc-gnats@gcc and they'll be added to the existing PR if their PR
> >>category/# matches correctly.
> >
> >Through the same type of script process as new bug reports do?
> >
> >>
> >>When a user changes the state of a PR or whatever through gnatsweb, a
> >>notification is sent out to the gcc-prs list.  Note that nothing is
> >>sent to gcc-gnats@, so if you just monitor gcc-gnats you'll only see
> >>new PRs and PRs with e-mail additions, not state changes.  If you
> >>monitor gcc-prs, you'll see new PRs, additions to PRs, and state
> >>changes to PRs.  At least you _should_ see all those things. :-)
> >>
> >>
> >>J
>

Re: Web part of bugzilla is ready (fwd)

[Joseph S. Myers]

On Fri, 7 Feb 2003, Christopher Faylor wrote:

> On the other side, I think that anything that shows up in bugzilla should
> generate gcc-bugs email.  There's no reason to generate gcc-prs mail,
> AFAICT.  You can do that now with no help from me, I think.

I believe that the gcc-bugs list should be told that gcc-bugzilla (or 
whatever - the Bugzilla address that mail followups to Bugzilla bugs get 
sent to) is another name for itself, so it accepts messages without 
gcc-bugs explicitly in their headers.  (A followup mail sent to 
gcc-bugzilla should appear on the list and in the database.  If people CC 
mails to both this means lots of messages get duplicated on the list, 
which is undesirable, so it's good if the messages on gcc-bugs don't 
actually mention gcc-bugs in the To/CC headers if they mention 
gcc-bugzilla.  This can always be dealt with after the transition if 
complicated; duplicate messages aren't too serious a problem.  The gcc-prs 
list can then go away - the only messages it gets that gcc-bugs doesn't 
are mail followups to PRs sent to gcc-gnats only, and they should appear 
on gcc-bugs in future.)

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Web part of bugzilla is ready (fwd)

[Christopher Faylor]

On Fri, Feb 07, 2003 at 11:53:41PM -0500, Daniel Berlin wrote:
>The second is a more general interface, that can do more (IE do bug
>searches, etc, by email). It just needs a new email address.
>This script is done and ready to go, but secondary, so if you are busy at
>the moment, don't worry about it.

This is easy to set up.  I just have to create a gcc-bugzilla alias that
runs this program, similar to the gcc-gnats alias.  I'll set up the
alias if you provide the path to the program.

>> 2) Something to generate email when a bug is entered in bugzilla.
>>
>This is already done by bugzilla on its own.

Cool.

>> Neither is hard to do.  I can set up 1 immediately, if you want.  We can
>> set up a gcc-bugzilla mailing list and have it go directly into your
>> database.  Just tell me what program needs to be run.
>
>I shall, the second i have it done.

Ok.  I just didn't want to leave the impression that it was stalled on
the sysadmin side of things.

cgf

Re: Web part of bugzilla is ready (fwd)

[Christopher Faylor]

On Sat, Feb 08, 2003 at 11:25:40AM -0500, Christopher Faylor wrote:
>Ok.  I just didn't want to leave the impression that it was stalled on
>the sysadmin side of things.

Actually, on reading this, it sounds like I was saying things were stalled
on Daniel's end, which wasn't my intent either.  It sounds like we're close
to getting bugzilla operational, in fact.

cgf

Re: Web part of bugzilla is ready (fwd)

[Daniel Berlin]

On Saturday, February 8, 2003, at 11:37  AM, Christopher Faylor wrote:

> On Sat, Feb 08, 2003 at 11:25:40AM -0500, Christopher Faylor wrote:
>> Ok.  I just didn't want to leave the impression that it was stalled on
>> the sysadmin side of things.
>
> Actually, on reading this, it sounds like I was saying things were 
> stalled
> on Daniel's end, which wasn't my intent either.  It sounds like we're 
> close
> to getting bugzilla operational, in fact.

Yup.
I just like to make sure it works perfectly on my system before moving 
it to sources.

I hadn't touched that gnats *email* parsing script in a while, and I 
had to merge it with the latest version of the gnats db conversion 
script (The email parsing script is a bastardized version of the gnats 
db->bugzilla db converter  script i wrote).

Thus, i'm running it through random emails selected from the gcc-gnats 
list by hand to make sure it works.

>
> cgf

Re: Web part of bugzilla is ready (fwd)

[Daniel Berlin]

On Saturday, February 8, 2003, at 11:25  AM, Christopher Faylor wrote:

> On Fri, Feb 07, 2003 at 11:53:41PM -0500, Daniel Berlin wrote:
>> The second is a more general interface, that can do more (IE do bug
>> searches, etc, by email). It just needs a new email address.
>> This script is done and ready to go, but secondary, so if you are 
>> busy at
>> the moment, don't worry about it.
>
> This is easy to set up.  I just have to create a gcc-bugzilla alias 
> that
> runs this program, similar to the gcc-gnats alias.  I'll set up the
> alias if you provide the path to the program.


Okay, it's ready.

The program is /pool/bugzilla/bugzilla-2.17.3/contrib/bugzilla_email.pl


It expects to be run with a current working directory of 
/pool/bugzilla/bugzilla-2.17.3/contrib/, but only because i didn't use 
absolute paths to tell it where to find files, i used "../".


>
>>> 2) Something to generate email when a bug is entered in bugzilla.
>>>
>> This is already done by bugzilla on its own.
>
> Cool.
>
>>> Neither is hard to do.  I can set up 1 immediately, if you want.  We 
>>> can
>>> set up a gcc-bugzilla mailing list and have it go directly into your
>>> database.  Just tell me what program needs to be run.
>>
>> I shall, the second i have it done.
>
> Ok.  I just didn't want to leave the impression that it was stalled on
> the sysadmin side of things.
>

Okay, i'm actually happy with all the scripts now, but since they all 
rely on basically the same perl code to do most of the work, i'd rather 
start with the non-intrusive one, which is the one i was talking about 
above (for  the gcc-bugzilla email address).

> cgf

Re: Web part of bugzilla is ready (fwd)

[Christopher Faylor]

On Sun, Feb 09, 2003 at 08:23:33PM -0500, Daniel Berlin wrote:
>
>On Saturday, February 8, 2003, at 11:25  AM, Christopher Faylor wrote:
>
>>On Fri, Feb 07, 2003 at 11:53:41PM -0500, Daniel Berlin wrote:
>>>The second is a more general interface, that can do more (IE do bug
>>>searches, etc, by email). It just needs a new email address.
>>>This script is done and ready to go, but secondary, so if you are 
>>>busy at
>>>the moment, don't worry about it.
>>
>>This is easy to set up.  I just have to create a gcc-bugzilla alias 
>>that
>>runs this program, similar to the gcc-gnats alias.  I'll set up the
>>alias if you provide the path to the program.
>
>
>Okay, it's ready.
>
>The program is /pool/bugzilla/bugzilla-2.17.3/contrib/bugzilla_email.pl
>
>
>It expects to be run with a current working directory of 
>/pool/bugzilla/bugzilla-2.17.3/contrib/, but only because i didn't use 
>absolute paths to tell it where to find files, i used "../".

I just noticed this.  You are not using /pool/bugzilla everywhere are
you?

/pool is subject to change /sourceware/bugzilla isn't.

Anyway, I've set up a gcc-bugzilla alias, I hope.

cgf