XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

Dear GCC developers,

I was really suprized and happy to see the patches applied to the 
ast-optimizer-branch from Sebastian Pop. 
His XML dumping of call graphs is very useful. 
Can I take this as general acceptance of XML dumping in the gcc? 

At least it has not been attacked or criticized on the mailling list.

Also the dump-tree-dot module dumps into a format for the non-free
software dot, a part of graphviz. I take this is an accepted practice
of dumping tree structures for explict usage by non-free software? 

I hope that this is an indication that there is no longer any policy
problem of using the ast-dumps from the gcc by other free software
modules? I mean if the output is meant for non-free software, then you
must be happy to have this output used by free software.

Or is the contents of the CVS not an indication of policy at all, 
with the policy makers not enforcing them, and the cvs commiters not
asking for permission?

There have been many people who are interested in using the
introspector for creating free software. I have been trying to convince
people that it is better to not output asts to non-free software or
xml. But if the gcc is accepting patches to do the same, then I would
be silly not to follow suite. 

In fact, I cannot see any real reason anymore to get stressed out about
this issue at all. 

We am going to be re-starting up the binary distribution of the patched
gcc for full dumping of function bodies into xml and post processing in
perl. I had taken them down out of respect to some or Also we we will
be creating a local branch of the gcc ast-optimizer branch in the
introspector.sf.net repository.

My goal is to create a statically linked visualization and manipulation
tool under the GPL by gluing togeather the components of the VCG, GNOME
DIA, GCC and Perl. 

The VCG is a GPLed software that provides similar functions to 
GraphVis, In fact I have now gotten an "Un-Uglified" copy of the
sources from the author, and will be pushing to get the Gcc users to
switch over to use free software.

Also I have planned out an exchange of the ast information via
in-memory trees stored in the libxml2 dom structures. That would allow
for the visualization tools to get a "Live feed" from the compiler as a
push data feed. I hope that it will eliminate the need for external
representation, in fact the entire system could be statically linked
into one huge monolithic application. That would also include an
embedded perl for a script interpreter.

Anyway, we have many discussions about this issue on the list in the
past, and I think this whole issue of external representation is really
turning out to be a free-for-all where is it easier to ask for
forgiveness then for permisison, not following any real guidelines. It
is now time for me to get back in on this free-for-all, otherwise I
will just be left out.

Mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Joe Buck]

James Michael DuPont writes:

> I was really suprized and happy to see the patches applied to the 
> ast-optimizer-branch from Sebastian Pop. 
> His XML dumping of call graphs is very useful. 
> Can I take this as general acceptance of XML dumping in the gcc? 

Not necessarily.  RMS still has to approve anything like this before we
move it to the release branch.  We can justify it only if the XML is in
some sense incomplete: insufficient for use in proprietary back ends.

> At least it has not been attacked or criticized on the mailling list.

That's because it's incomplete (not a complete dump of the program
structure).

> Also the dump-tree-dot module dumps into a format for the non-free
> software dot, a part of graphviz. I take this is an accepted practice
> of dumping tree structures for explict usage by non-free software? 

Again, no.  RMS has explicitly objected to the graphviz support, so it
probably won't be released.  However, the situation is more complicated
than you suggest: the (free software) Boost graph library supports
graphviz reading and writing, as do other free programs.  But let's not
fool ourselves: the main reason people want graphviz output is to see the
graph visually, and the way they do that is with the non-free program.  So
we might not be able to convince RMS to let us release that.

> Or is the contents of the CVS not an indication of policy at all, 
> with the policy makers not enforcing them, and the cvs commiters not
> asking for permission?

Especially on side branches, you should assume that anything there is
experimental and not necessarily accepted for release.  Non-acceptance
might be based on technical reasons or policy reasons.

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

> Again, no.  RMS has explicitly objected to the graphviz support, so
> it
> probably won't be released.  However, the situation is more
> complicated
> than you suggest: the (free software) Boost graph library supports
> graphviz reading and writing, as do other free programs.  But let's
> not
> fool ourselves: the main reason people want graphviz output is to see
> the
> graph visually, and the way they do that is with the non-free
> program.  So
> we might not be able to convince RMS to let us release that.

I know all about the GraphViz.

Well in my many mailings to RMS about my project, and when I met with
him on July 3rd in Berlin, he has explicitly stated his support for
tools that directly interact with the user and are not meant to extract
to the files for that purpose. I said that I will in turn reduce any
external representation and try and statically linke everything in.

That is why I have decided to use libxml as a tranport layer and link
in the modules all statically to the gcc. Even if someone were to dump
that xml to a file to view it, the xml is only used as an intermediate
layer. The LGPL supports the reversal of the code back to GPL, to I can
then provide the XML into other tools statically via libxml, and it
would be a GPLed interface... Go figure.!

Also RMS has supported my fight with the University of Saarland that
tried to block the release of the uglified parts of the vcg tool. 

I now have that non-uglified code from the author and will be
re-releasing VCG as a GNU project later on when I find some time.

That will allow for a complete GPL code information extraction and
graph layout tool. I have been hacking the Gnome DIA tool to later
provide a final rendering in UML.

> 
> > Or is the contents of the CVS not an indication of policy at all, 
> > with the policy makers not enforcing them, and the cvs commiters
> not
> > asking for permission?
> 
> Especially on side branches, you should assume that anything there is
> experimental and not necessarily accepted for release. 
> Non-acceptance
> might be based on technical reasons or policy reasons.

Fine, then you wont have any problem with me branching the
ast-optimizer branch and maintaining that Branch for visualization.

I wont ask for offical support, but will be trying to maintain my
patches to your standards. Hopefully they will be usable by you some
day if you need them.

All I want to do is be able to use the GCC to help understand code, 
I dont need XML, I need something that I can look at.

mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Joe Buck]

James Michael DuPont writes:

> Also RMS has supported my fight with the University of Saarland that
> tried to block the release of the uglified parts of the vcg tool. 
> 
> I now have that non-uglified code from the author and will be
> re-releasing VCG as a GNU project later on when I find some time.

This is great news and would be a huge contribution to the free software
world, especially if it reads graphviz format.

> > Especially on side branches, you should assume that anything there is
> > experimental and not necessarily accepted for release. 
> > Non-acceptance
> > might be based on technical reasons or policy reasons.
> 
> Fine, then you wont have any problem with me branching the
> ast-optimizer branch and maintaining that Branch for visualization.

Ah, the argument from precedent again!  *I* have no problem, and if you're
in regular communication with RMS then you'll know whether or not *he* has
a problem, but of course there's a strong difference between "I don't like
it" and "I'm going to try to stop you" and I suspect that his feelings
will be more in the former category.

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

--- Joe Buck <jbuck@synopsys.com> wrote:
> James Michael DuPont writes:
> 
> > Also RMS has supported my fight with the University of Saarland
> that
> > tried to block the release of the uglified parts of the vcg tool. 
> > 
> > I now have that non-uglified code from the author and will be
> > re-releasing VCG as a GNU project later on when I find some time.
> 
> This is great news and would be a huge contribution to the free
> software
> world, especially if it reads graphviz format.

Thanks Joe.
It means alot to me.

I see the vcg as an important tool to help understand programs.

Now, if we look at the problem of communication, how can the gcc safely

communicate the asts to the vcg for layout. 

You will see that any form of communcation is subject to a
man-in-the-middle attack that can dump the data out to a pipe.

Now you also need the ability to focus the results on something
visible,
to search and filter the ast nodes. Others will want to transform the
trees selecting certain attributes and turning those into graph nodes.

If the asts are put into an libxml dom structure, those transformations
can be done by a simple xslt style sheet, or perl script linked in, and
the passed on to the visualizer, all without leaving main memory.

So you see that to do anything usefull with this information, it must
run the risk to be intercepted.

Now, I am not a laywer, but 
I think that if we can show that if the data is put into such a form
that it is indended for transport between modules, and a third party
takes that data a runs off with it, we can then say that they got
between two statically linked modules . Maybe that will offer some
protection against non-free modules getting in between?

But on the other hand, what if you take these graphs and run the
through some 3d transformations on your video card, to use the
accelerated chips for doing things like that. It could very well be
non-free software, so what about that?

> > > Especially on side branches, you should assume that anything
> there is
> > > experimental and not necessarily accepted for release. 
> > > Non-acceptance
> > > might be based on technical reasons or policy reasons.
> > 
> > Fine, then you wont have any problem with me branching the
> > ast-optimizer branch and maintaining that Branch for visualization.
> 
> Ah, the argument from precedent again!  *I* have no problem, and if
> you're
> in regular communication with RMS then you'll know whether or not
> *he* has
> a problem, but of course there's a strong difference between "I don't
> like
> it" and "I'm going to try to stop you" and I suspect that his
> feelings
> will be more in the former category.
Fine, and that was got me into problems before. 
It is easier to ask for forgiveness than for permission.

I will be branching the code, like it or not. I will be updating it
regulary from the gcc base. 

mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Joe Buck]

> Now, if we look at the problem of communication, how can the gcc safely
> communicate the asts to the vcg for layout. 

You'll have to assume, I think, that anything gcc can write anyone can
use.

> Now, I am not a laywer, but 
> I think that if we can show that if the data is put into such a form
> that it is indended for transport between modules, and a third party
> takes that data a runs off with it, we can then say that they got
> between two statically linked modules . Maybe that will offer some
> protection against non-free modules getting in between?

I think it would be hard to make such an argument if what is communicated
is basically just information about the structure of the user's own code.
But this list is not the best place for such discussions.

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Joseph S. Myers]

On Tue, 15 Oct 2002, James Michael DuPont wrote:

> I now have that non-uglified code from the author and will be
> re-releasing VCG as a GNU project later on when I find some time.

Ulrich Drepper had some fixes to VCG (that were included in a version in
the infrastructure directory on the FTP site, which had to be removed
because uglified source isn't source within the meaning of the GPL, so we
couldn't legally distribute the uglified version at all); these fixes
ought to be merged to the the distributabel non-uglified version.  Then
the link in <http://gcc.gnu.org/news/egcs-vcg.html> can be fixed/updated.

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

--- "Joseph S. Myers" <jsm28@cam.ac.uk> wrote:
> On Tue, 15 Oct 2002, James Michael DuPont wrote:
> 
> > I now have that non-uglified code from the author and will be
> > re-releasing VCG as a GNU project later on when I find some time.
> 
> Ulrich Drepper had some fixes to VCG (that were included in a version
> in
> the infrastructure directory on the FTP site, which had to be removed
> because uglified source isn't source within the meaning of the GPL,
> so we
> couldn't legally distribute the uglified version at all); these fixes
> ought to be merged to the the distributabel non-uglified version. 
> Then
> the link in <http://gcc.gnu.org/news/egcs-vcg.html> can be
> fixed/updated.

Thanks Joseph, I will try and setup a savannah project this week and
get all the sources under cvs. 
I hope that we can make vcg become what it should have been, a usefull
tool and not a licencing problem.

Mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Zack Weinberg]

On Tue, Oct 15, 2002 at 11:04:52AM -0700, James Michael DuPont wrote:
> Now, if we look at the problem of communication, how can the gcc safely
> communicate the asts to the vcg for layout. 
> 
> You will see that any form of communcation is subject to a
> man-in-the-middle attack that can dump the data out to a pipe.
> 
> Now you also need the ability to focus the results on something
> visible, to search and filter the ast nodes. Others will want to
> transform the trees selecting certain attributes and turning those
> into graph nodes.
> 
> If the asts are put into an libxml dom structure, those transformations
> can be done by a simple xslt style sheet, or perl script linked in, and
> the passed on to the visualizer, all without leaving main memory.
> 
> So you see that to do anything usefull with this information, it must
> run the risk to be intercepted.

This is silly.

The relevant policy goal of the FSF in this context is to prevent
people from writing front ends, optimization modules, or back ends for
GCC which are not free software.  Rendering the AST, or the CFG, or
whatever, in a form suitable for graphing will not facilitate any of
these goals.  In fact, we already *have* code to generate VCG graphs
from the debugging dumps.

So just write your graph to a file in whatever format seems sensible
to you.  (We are unlikely to take the patch if it is a format readable
only by proprietary software, but then until just now that was true of
the existing graph dumper.)

zw

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

--- Robert Dewar <dewar@gnat.com> wrote:
> <<I am not planning in any way to create non-free software, just
> pointing
> out that the .dot files are a file format that is read by the
> non-free
> software GraphViz. It strikes me funny that the gcc would explicitly
> support a non-free software, and serve it files.
> >>
> 
> This is confused, it is not at all correct to say that gcc is
> "explicitly
> supporting a non-free software and serving it files". If gcc is used
> int
> this way, it might well be a copyright violation. And the only way to
> determine this would be through litigation. Certainly there is no
> policy
> statement that says that this is permissible.

Wait a second... 
What other way to use these .dot files, but to feed them to the dot
program? 

"If the gcc is used in this way"? 
it modified by one of the gcc developers to be used that way. 

This issue is with the dump-tree-dot function.
It used to be named dump-tree-graphviz !

Diego has tried to remove references to the origins 
of this tool, but all I can say is that support GraphViz is much less
in line with FSF policy then by trying to replace it.

See the change logs:
2002-05-02
* tree-dump.c (dump_files): Rename -fdump-tree-graphviz to
	-fdump-tree-dot.
2001-08-20
   the code was first added.

So, I would say that proverbial right hand does not know what the left
hand is doing. 

mike


=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Robert Dewar]

<<I am not planning in any way to create non-free software, just pointing
out that the .dot files are a file format that is read by the non-free
software GraphViz. It strikes me funny that the gcc would explicitly
support a non-free software, and serve it files.
>>

This is confused, it is not at all correct to say that gcc is "explicitly
supporting a non-free software and serving it files". If gcc is used int
this way, it might well be a copyright violation. And the only way to
determine this would be through litigation. Certainly there is no policy
statement that says that this is permissible.

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[James Michael DuPont]

--- Robert Dewar <dewar@gnat.com> wrote:
> <<Also the dump-tree-dot module dumps into a format for the non-free
> software dot, a part of graphviz. I take this is an accepted practice
> of dumping tree structures for explict usage by non-free software?
> >>
> 
> You should make no such assumption. The mere use of an intermediate
> file
> does not get around the limitations in the GPL license automatically.
> Indeed there are obvious cases where such an approach would be a
> clear
> violation of the license.
> 
> As always, you need to consult an attorney for this kind of approach.

I am not planning in any way to create non-free software, just pointing
out that the .dot files are a file format that is read by the non-free
software GraphViz. It strikes me funny that the gcc would explicitly
support a non-free software, and serve it files.

In fact with the policy of the FSF being to not use non-free software,
and to do without or replace it by free software.

But on the other hand the gcc developers seem to discourage people from
producing free software that would use the same AST information in an
external form. 

You must admit that there seems to be a double standard, or at least no
enforced standard at all. 

> <<Or is the contents of the CVS not an indication of policy at all,
> with the policy makers not enforcing them, and the cvs commiters not
> asking for permission?
> >>
> 
> Certainly the contents of the CVS is not necessarily an indication
> of policy. And most certainly no legal conclusions can be drawn
> from the contents of the CVS!

Fine.

My point is that I want to use this information for free software only.


I have been holding back out of respect for many wishes to avoid
"dangerous" software. 

All indications are that I am just wasting time here, I am the only one
who is holding back. 

The offical gcc developers are adding in more features that do exactly
what I was planning, so I have some catching up to do. I will also try
and avoid using intermediate files where I can and link the modules
statically to the gcc, as a symbol of my understanding of the issues.

I hope that my patches will be some day accepted into the gcc source
tree.

mike

=====
James Michael DuPont
http://introspector.sourceforge.net/

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: XML dumping and GraphViz/VCG in the GCC ast-optimizer-branch

[Robert Dewar]

<<Also the dump-tree-dot module dumps into a format for the non-free
software dot, a part of graphviz. I take this is an accepted practice
of dumping tree structures for explict usage by non-free software?
>>

You should make no such assumption. The mere use of an intermediate file
does not get around the limitations in the GPL license automatically.
Indeed there are obvious cases where such an approach would be a clear
violation of the license.

As always, you need to consult an attorney for this kind of approach.

<<Or is the contents of the CVS not an indication of policy at all,
with the policy makers not enforcing them, and the cvs commiters not
asking for permission?
>>

Certainly the contents of the CVS is not necessarily an indication
of policy. And most certainly no legal conclusions can be drawn
from the contents of the CVS!