Re: GSL, VSIPL, and the ultimate numerical library

Re: GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Brian Gough wrote:

> That is an interesting message.  Here are my thoughts.
> A single grand library may not be a desirable goal --
> it would constrain the user to operate within its parameters
> and implies a centralized development model.
> I believe the most flexible approach
> involves having many libraries:
> 
>  specialised libraries for different specialist areas 
>  a general library covering the basics 
> 
> LAPACK, FFTW, etc would be examples of specialised libraries.
> GSL would fall into the general category.
> The role of the general library is
> to provide basic functionality and interoperability.
> In the C/FORTRAN and C-based VHLL world,
> this is fairly straightforward.
> 
> For everyday purposes,
> the general library should be sufficient
> so that one can carry out simple tasks easily.
> This is the position of GSL: with it one can
> compute a random number,
> find a root or a minimum,
> obtain the value of a special function,
> compute simple statistics,
> interpolate a table of values,
> etc.

In general, different library developers
will use different data representations
for vector, matrix and tensor objects.
You will need to standardize the data representation
if you are going to insist upon interoperability
and that will preclude optimal performance
for some target platforms.

It would be better if library developers provided
interfaces which accept references (pointers)
to a GSL vector or matrix objects.
Any changes to GSL vector and matrix
data representations would need to be coordinated
with the different library developers
but not with application programmers.

Re: GSL, VSIPL, and the ultimate numerical library

[Randall Judd]

The TASP code is free for any use. I am not allowed to copyright since I am 
a Government employee and much of my implementation is done on my 
Government time. The HRL, LLC and MSU code in the TASP implementation have 
copyright statements that allow people to make use of it for any purpose. 
You only need to leave their copyright statement in if you derive code from 
their code. So if somebody took the TASP VSIPL implementation and changed a 
bunch of stuff for a particular piece of hardware, and added some sort of 
GPL copyright on with the other copyrights, I don't see why the could not 
do that. I am not a lawyer so I don't understand all the copyright issues, 
but I know the intent of the TASP VSIPL code is to jump start anybody who 
wants to do a VSIPL implementation, either free or comercial, or who just 
wants to learn about VSIPL..

                Randy Judd



At 12:48 AM 11/07/2000 -0500, Robert W. Brewer wrote:
> > > I may try to optimize parts of the TASP implementation
> > > for my target du jour.
>
> > Will you share?  For free?  For profit?
>
>I'd probably like to share via the LGPL if possible.
>But I'm no licensing expert or lawyer, so I don't know
>if that would be compatible with the licenses currently
>on the TASP code.  Of coures this is all just talk until
>I actually get off my duff and write something.
>
>Does anyone have any thoughts on the licensing issues?
>
>-Rob
>--
>Robert W. Brewer
>Jesus rules!

Re: GSL, VSIPL, and the ultimate numerical library

[Robert W. Brewer]

> > I may try to optimize parts of the TASP implementation
> > for my target du jour.

> Will you share?  For free?  For profit?

I'd probably like to share via the LGPL if possible.  
But I'm no licensing expert or lawyer, so I don't know
if that would be compatible with the licenses currently
on the TASP code.  Of coures this is all just talk until
I actually get off my duff and write something.

Does anyone have any thoughts on the licensing issues?

-Rob
-- 
Robert W. Brewer
Jesus rules!

Re: GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Gerard Jungman wrote:

> One last point of emphasis,
> at the risk of repeating myself...
> 
> Because of the VSIPL goals,
> all of the discussion in the VSIPL/GSL thread
> has concentrated on linear algebra operations,
> and other large linear operations like FFTs.
> 
> Scanning the web for scientific software
> (a sad hobby of mine),
> you will find many projects
> (and many of these failed)
> which declare one of their main goals to be
> "to provide efficient and portable
> vector/matrix objects...".

You probably meant portable vector/matrix classes.
Objects don't exist until you actually run the program
so they can never be portable.

> Perhaps they also got as far as providing
> some kind of linear algebra,
> either native or through interfaces to lapack.
> 
> We can judge the success of these attempts
> by the fact that we are still arguing
> about vector/matrix implementations.
> 
> So I would just like to say that
> it is very important to draw a distinction
> between these "linear" parts of a generic library
> and the other functionality.  I'm glad that
> the GSL did not become one of these zombie projects,
> wandering in circles in the "vector/matrix" swamp.

We don't know that yet.  The GSL is not complete.
There is no reason to believe that the GSL
will survive any longer than the GNUSSL.

> On the other hand, it means that GSL could not afford
> to do anything substantial about these issues;
> we can't drain the swamp,
> and I doubt if it is even possible,
> given the limitations
> imposed by the implementation language.

Well, you can implement anything in ANSI C.
It's just harder to design a practical user interface
for ANSI C application programmers to use.

> And although VSIPL is a very reasonable spec in C,
> I think implementations will suffer
> from the standard abstraction penalties
> in generic situations.
> I don't think this is a controversial statement,
> but people are free to disagree.

Perhaps.  But it isn't ever necessary.
You can always implement specializations
to avoid any abstraction penalty.

> Also, the concentration on dense arrays
> is very limiting from the standpoint
> of general scientific computing.
> Maybe this will be addressed in later specs,

Maybe.
But meanwhile functions like gather and scatter
are meant to bridge the gap.

> but it looks like the standard combinatorial explosion
> of interfaces will be unavoidable.

Why should that bother you?
You're not an ANSI C compiler.

> These are the issues
> which I am most concerned with.
> 
> I optimistically hope that
> the brave new template world will come to the rescue.
> Right now, things are looking pretty good for that.
> Maybe we will finally be able to move on in life.
> I personally find linear algebra
> to be extremely tedious.

The problem with genericity in numerical computing is that
vector and matrix classes are NOT container classes.
Templates can help library developers implement
vector and matrix classes efficiently
but they require so many specializations that
they don't really help application programmers very much
in extending vector and matrix classes to new types.

Re: GSL, VSIPL, and the ultimate numerical library

[Gerard Jungman]

One last point of emphasis, at the risk of repeating
myself...

Because of the VSIPL goals, all of the discussion in
the VSIPL/GSL thread has concentrated on linear algebra
operations, and other large linear operations like FFTs.

Scanning the web for scientific software (a sad hobby of mine),
you will find many projects (and many of these failed)
which declare one of their main goals to be "to provide
efficient and portable vector/matrix objects...".
Perhaps they also got as far as providing some
kind of linear algebra, either native or through
interfaces to lapack.

We can judge the success of these attempts by the fact
that we are still arguing about vector/matrix implementations.

So I would just like to say that it is very important
to draw a distinction between these "linear" parts of
a generic library and the other functionality. I'm glad
that GSL did not become one of these zombie projects,
wandering in circles in the "vector/matrix" swamp.

On the other hand, it means that GSL could not afford
to do anything substantial about these issues; we
can't drain the swamp, and I doubt if it is even
possible, given the limitations imposed by the
implementation language.

And although VSIPL is a very reasonable spec
in C, I think implementations will suffer
from the standard abstraction penalties
in generic situations. I don't think this
is a controversial statement, but people are
free to disagree. Also, the concentration on
dense arrays is very limiting from the standpoint
of general scientific computing. Maybe this
will be addressed in later specs, but
it looks like the standard combinatorial
explosion of interfaces will be unavoidable.
These are the issues which I am most
concerned with.

I optimistically hope that the brave new
template world will come to the rescue. Right
now things are looking pretty good for that.
Maybe we will finally be able to move on
in life. I personally find linear algebra
to be extremely tedious.


-- 
G. Jungman

Re: GSL, VSIPL, and the ultimate numerical library

[Gerard Jungman]

"Robert W. Brewer" wrote:
> 
> It sounds like that is what Gerard was getting at with
> the GSE: an interactive way to simulate, test, and debug an
> algorithm at a very high level, together with additional analysis
> and graphing tools.  And then a migration path to deployment
> on a target, which would be further and further optimized in
> various ways for the specific application.  That would
> be very nice, but it is also very lofty.

As long as we're shooting for the moon, I would like to add
the need for component deployment as well as development.
Not only should people be able to develop "in-house"
applications with a degree of flexibility, but they should
be able to create components targetted to their problem
domain, not only for their own re-use, but for others
as well. Scientific computing tends to be "I have an
idea for a CFD solver (or whatever), let me go ahead
and code a giant black-box monolithic application,
reinventing several wheels along the way, creating
an inflexible tool. Maybe I will be able to solve
a few problems with it before it has to be thrown
away." Rather, it could be "Let me develop my solver
as a component to be dropped into a reusable framework,
developed over time by other experts in this and related
fields, and when I shelve the project I won't have to
throw away all the developed infrastructure."

This is not appropriate for every project, but I see
a number of "research/production" codes which are
doomed from the start by their monolithic nature.
The wasted effort can easily dominate the project.
Maybe people would change if they were
given significantly better tools. Maybe.

> Currently the way
> I would do that, and the way I've mainly seen that done is
> like this:
> ... [outline deleted]
>

Even that outline is often a dream. In high performance
computing I have seen an outline more like:

  - spend years writing giant F90 code, reinventing all aspects,
    including data structures, communication methodology,
    file formats, etc., often at the lowest conceivable
    level of abstraction
  - have a single developer hack at the code until it is
    comprehensible only to him
  - solve a few problems, then hack some more, introducing
    "options" which were found necessary to get good results
    on certain types of problems; make the changes as non-dynamic
    as possible by introducing them into the build process, so
    the code is no longer a single platform for problem solving,
    but a confusing multi-headed gorgon
  - insist that refactoring or re-engineering is not a
    viable option because further investment in "development"
    cannot be justified (it would probably be impossible anyway)
  - repeat until dead

At some level, the problem comes from a lack of any
viable tools or models for prototyping and component
deployment, especially on massively parallel platforms
where nothing ever works right and the system changes
on a monthly or weekly basis.

I hope we're not just doomed.


> Now certainly a library or a GSE is no substitute for
> discipline and keeping the analysis and design documentation
> and models updated.  But I think it can help, especially
> when the library has high-level enough functionality that
> the final code is forced to be somewhat self-documenting.

Yes. Systems should be natural, and their naturalness
should flow directly from the design process.

Maybe we are doomed after all.
Oh dear.


-- 
G. Jungman

Re: GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Robert W. Brewer wrote:

> I've really enjoyed the responses.
> Based on Gerard's comments about a GNU Scientific Environment,
> I can see that the ideal for me would be a way to write
> high-level, Octave or Matlab style code
> but still have it be fast when needed.
> I have seen Matlab compilers
> which supposedly help with this sort of thing.
> 
> It sounds like that is what Gerard was getting at with the GSE:
> an interactive way to simulate, test, and debug
> an algorithm at a very high level,
> together with additional analysis and graphing tools.
> And then a migration path to deployment on a target,
> which would be further and further optimized in various ways
> for the specific application.
> That would  be very nice, but it is also very lofty.
> Currently the way I would do that
> and the way I've mainly seen that done is like this:
>   Read books, papers, and articles (and think)
>      to find candidate algorithms and approaches,
>   Use Matlab, Octave, or one of many simulation tools
>      to simulate and analyze some candidate solutions.
>      This often involves writing a lot of code
>      in their respective languages.

This is usually caller the "reference version"
or, sometimes, "the golden copy."

>   Choose best algorithm.
>   Reimplement algorithm in C or C++ using whatever helper libraries
>      are available for the target platform.

Unfortunately, developers often stop maintaining the reference version
shortly after the performance version is implemented.

>   If still not fast enough,
>      reimplement inner loops with specialized assembly code.
>   Go back to any previous step if it's still not working as expected.
> 
>   If it's working correctly,
>      gradually misplace books, papers, articles, and thinking.
>      Gradually misplace Matlab simulation code.
>      Gradually misplace everything
>      except working C, C++, and assembly code.
> 
>   Decide to switch target platforms or upgrade algorithm.  
>      Often must go back to step one
>      since results of intermediate stages are gone
>      and working C, C++, and assembly code is too hard to figure out.
> 
> Now certainly a library or a GSE is no substitute for discipline
> and keeping the analysis and design documentation and models updated.
> But I think it can help,
> especially when the library has high-level enough functionality
> that the final code is forced to be somewhat self-documenting.
> 
> OK, back to the topic.  I spent some time considering
> how to create a genericized VSIPL implementation,
> and finally concluded that to ignore all the good work
> that has gone into the TASP VSIPL code would be a big mistake.
> Gerard's comments about the sin of reinventing BLAS helped win me over.

People reinvent the BLAS library all the time.
The BLAS library is really just a standard API
that everybody implements the way that they see fit.
You can get the BLAS [portable] reference implementation from

	http://www.netlib.org/blas/index.html

Just click on blas.tgz

Also, take a look at
The ANSI C Numerical Class Library

	http://www.netwood.net/~edwin/svmt/

The .../cncl/bin/genclass script converts prototype files like
.../cncl/src/vector/vector.hP and .../cncl/src/vector/vector.cP
into ANSI C header and source files.

> Now I think I understand the difference
> between the original Hughes reference code and the TASP code, 
> and it sounds like the reference code may still be lurking
> inside Hughes somewhere.
> Is there any chance of publicly releasing it?

I'm not sure.
I believe that HRL Laboratories LLC was supposed to
bring the Portable Reference Library up to date,
transfer the copyright to the VSIPL Forum and
release it to the public.
I don't believe that is going to happen now.
It might be counter productive
to release the Portable Reference Library as is
because it might be confused with VSIPL API specification.

Anyway, send email to David Schwartz daSchwartz@HRL.com
and ask him for a copy of 

	VSIP Library Reference Implementation (9/10/97)

> I don't have my copy anymore,
> but even if I did I had agreed to keep it private when I received it.
> That would be another codebase for me to consider mucking with.
> But for now I think I'm going to concentrate
> on getting some algorithms to work that would use VSIPL
> and then, if they are too slow,
> I may try to optimize parts of the TASP implementation
> for my target du jour.

Will you share?  For free?  For profit?

> I don't want to fill up too much more of the GSL list
> with purely VSIPL stuff,
> so does anyone know of an active VSIPL list?
> I think I subscribed to some on the VSIPL website years ago
> and have only gotten a few messages a year
> about meeting announcements.

Take a look at the VSIPL Forum home page

	http://www.vsipl.org/forum.html

and click on the GETTING INVOLVED radio button
on the left hand side of the page.

	Join the VSIPL Mail List 
	VSIPL Forum Meeting Schedule 
	Post a comment to the VSIPL Forum 
	Read VSIPL Comments 

> Now that I've spent several hours considering
> how to implement one of these numeric library beasts,
> I have a better appreciation
> for what all the GSL and VSIPL folks have done.
> I even learned a lot of interesting stuff
> just from reading parts of the GSL and VSIPL documentation.

Re: GSL, VSIPL, and the ultimate numerical library

[Robert W. Brewer]

I've really enjoyed the responses.  Based on Gerard's
comments about a GNU Scientific Environment, I can
see that the ideal for me would be a way to write high-level,
Octave or Matlab style code, but still have it be fast
when needed.  I have seen Matlab compilers, which supposedly
help with this sort of thing.

It sounds like that is what Gerard was getting at with
the GSE: an interactive way to simulate, test, and debug an 
algorithm at a very high level, together with additional analysis
and graphing tools.  And then a migration path to deployment
on a target, which would be further and further optimized in
various ways for the specific application.  That would 
be very nice, but it is also very lofty.  Currently the way
I would do that, and the way I've mainly seen that done is 
like this:
  Read books, papers, and articles (and think) to find candidate
     algorithms and approaches
  Use Matlab, Octave, or one of many simulation tools to 
     simulate and analyze some candidate solutions.  This
     often involves writing a lot of code in their respective
     languages.
  Choose best algorithm.
  Reimplement algorithm in C or C++ using whatever helper libraries
     are available for the target platform.
  If still not fast enough, reimplement inner loops with specialized
     assembly code.
  Go back to any previous step if it's still not working as expected.

  If it's working correctly, gradually misplace books, papers,
     articles, and thinking.  Gradually misplace Matlab
     simulation code.  Gradually misplace everything except
     working C, C++, and assembly code.

  Decide to switch target platforms or upgrade algorithm.  
     Often must go back to step one since results of 
     intermediate stages are gone, and 
     working C, C++, and assembly code is too hard to 
     figure out.

Now certainly a library or a GSE is no substitute for 
discipline and keeping the analysis and design documentation
and models updated.  But I think it can help, especially
when the library has high-level enough functionality that
the final code is forced to be somewhat self-documenting.

OK, back to the topic.  I spent some time considering
how to create a genericized VSIPL implementation,
and finally concluded that to ignore all the good work
that has gone into the TASP VSIPL code would be 
a big mistake.  Gerard's comments about the sin
of reinventing BLAS helped win me over.

Now I think I understand the difference between
the original Hughes reference code and the TASP code, 
and it sounds like the reference code may still be 
lurking inside Hughes somewhere.  Is there any
chance of publicly releasing it?  I don't have
my copy anymore, but even if I did I had agreed
to keep it private when I received it.  That
would be another codebase for me to consider
mucking with.  But for now I think I'm going
to concentrate on getting some algorithms to 
work that would use VSIPL, and then if they
are too slow I may try to optimize parts 
of the TASP implementation for my target du jour.

I don't want to fill up too much more of the GSL
list with purely VSIPL stuff, so does anyone
know of an active VSIPL list?  I think I subscribed
to some on the VSIPL website years ago and have
only gotten a few messages a year about meeting
announcements.

Now that I've spent several hours considering
how to implement one of these numeric library
beasts, I have a better appreciation for what
all the GSL and VSIPL folks have done.  I even
learned a lot of interesting stuff just from 
reading parts of the GSL and VSIPL documentation.

-Rob
-- 
Robert W. Brewer  PGP RSA 2048-bit Key ID: 03E0E635
Jesus rules!      Fingerprint: 6327 8034 7BDA D144 B40C C5E2 F760 13BB

Re: GSL, VSIPL, and the ultimate numerical library

[Gerard Jungman]

Les Schaffer wrote:
> 
> > Somebody was following the releases fairly closely and creating
> > python bindings. We haven't heard from him in a while.
> 
> i'm interested in this myself. is there one grand .h file from gsl
> that can be wrapped, or which set of .h files implements the external
> interface? i;d like to give swig a shot.

Anything of the form 'gsl_XXX.h' is an interface header.
There's no "grand .h file", but that's not really necessary.

-- 
G. Jungman

RE: GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Jiri Hajek wrote:

> E. Robert Tisdale wrote:
> 
> > You could implement
> > The Scalar, Vector, Matrix and Tensor class library
> >
> >       http://www.netwood.net/~edwin/svmt/
> >
> > on top of the GNU Scientific Library (GSL).
> 
> I would like to do it!
> The problem is that I don't have enough time
> to maintain completely new project.
> So my question is:
> isn't there anybody else willing to do it?
> I would definitely like to contribute to such project.

You could pay to have someone else do it.

The problem is that you wouldn't be able to protect your investment
because the GSL is protected by the GPL instead of the LGPL.
You are obliged to publish your application source code
if you distribute any application program which uses the GSL.
This virtually precludes use of the GSL for commercial applications
and, consequently, any hope of generating revenue
from sales of the GSL so there is no money to support
continuing development and maintenance of the GSL.
This means that the GSL depends entirely upon the contributions
of volunteers who we expect, if they are any good,
will eventually find paying jobs and abandon the GSL.

You would probably be better off to pay someone to implement
The C++ Scalar, Vector, Matrix and Tensor class library
from scratch on top of commercial, high performance
numerical libraries -- the BLAS library, LAPACK, etc.
That way, you could copyright your SVMT source code
and leave most of the maintenance up to
reliable commercial library developers.

RE: GSL, VSIPL, and the ultimate numerical library

[Jiri Hajek]

E. Robert Tisdale wrote:

> You could implement
> The Scalar, Vector, Matrix and Tensor class library
>
> 	http://www.netwood.net/~edwin/svmt/
>
> on top of the GNU Scientific Library (GSL).

I would like to do it! The problem is that I don't have enough time to
maintain completely new project. So my question is: isn't there anybody else
willing to do it? I would definitely like to contribute to such project.

Jiri Hajek

GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Robert W. Brewer wrote:

> I just read a good chunk of the year 2000 list archives.
> I've been following the VSIPL forum work on and off
> for the past couple of years, and I was pretty excited
> when I saw the announcement of GSL 0.7 on freshmeat.net,
> because it sounded like the GSL might have already
> beaten the VSIPL to the punch so to speak.
> It appears that a lot of good work has gone into GSL so far.
> 
> I've been thinking about what I would want
> in a grand numerical library.
> The things that I myself would probably use
> a good numerical library for are
> some real-time speech processing under Linux
> and maybe as a basis for a neural networks library.
> Performance is important for both of those, otherwise,
> for offline use, I would just use Octave or Matlab.  
> If the Intel Signal Processing library ran under Linux,
> that would be great but it doesn't
> and it is also a closed-source library
> which I'm generally against. 
> 
> The TASP VSIPL code is a pretty good choice
> but it looks hard to maintain
> since there is so much code replication
> to support the different data types.
> I recall an early version the Hughes VSIPL code
> which used a preprocessor template scheme
> that seemed pretty slick.
> It looks like GSL has a similar problem.
> Many functions are written 3 and 4 times
> just to support the different float and int types
> the user might want to instantiate.
> And after all that, it still can't support neat things
> like arbitrary precision numbers,
> while a templating scheme might be able to.
> 
> To boil it down, the library that I would like to use
> would be (most important first):
>    high performance
>    generic to allow optimizing under the hood 
>       and advanced algorithms
>    templated for data type choices
>       and easier maintenance open source
>    easily tested for correctness with an automated suite
> 
> I think the VSIPL API has a lot of good ideas
> for the  functions that it supports
> but the reference implementation would be easier
> to deal with if it were templatized.
> Maybe it's my personality,
> but making the same change in more than one place
> just wears me out.
> 
> The GSL team has implemented a large breadth of functions,
> many of which have no equivalent in the VSIPL specification
> but which could probably benefit from some of its principles.
> Since both the TASP reference implementation and GSL
> are open source,
> it might be possible to gradually combine them,
> as Randall Judd seems interested in doing.
> Many GSL functions not present in VSIPL might be wrapped
> to present an interface similar to the VSIPL API.
> Then gradually the underlying GSL code could be moved
> into the wrapper to increase performance,
> better conform to VSIPL, etc.
> 
> It might also be nice to go back to the technique
> in the Hughes code of implementing most of the functions
> in terms of other VSIPL/GSL functions
> or lower-level helper functions whenever possible.
> This could have the advantage of allowing
> a mostly optimized version for a specific hardware platform
> by just optimizing the lowest-level vector routines
> and helpers for that platform and allowing
> the more complicated routines to ride on top and still benefit.
> The hope would be that, say, 80% of the Altivec
> (or Intel's SIMD) performance gain
> could come from  writing Altivec assembly
> in only, say, 20% of the functions.  
> I mention this because the neat thing, to me,
> about having a nice numerical library is being able to do some
> impressive real-time signal processing tricks on a desktop PC.
> Intel's MMX and Streaming SIMD Extensions,
> AMD's 3D Now instructions and PowerPC's Altivec technology
> all allow for amazingly fast computation
> but since standard compilers can't take advantage of them,
> writing assembly code with a high-level API is the way to go.
> But the idea of writing Rob's Gee Whiz Altivec Library
> is not very appealing to me, first because VSIPL and GSL
> have put a lot of thought into how to do it right
> and second because I'd rather not start from scratch
> if I switch to a non-Altivec machine.
> 
> I guess in the end I can understand
> why Tisdale is so interested in the vision and goals of GSL,
> so it is easier to decide if it is the same direction 
> that he would like to go.  I am in a similar quandary,
> there are several interesting-looking projects
> with various tradeoffs, and looking into the future a little
> would help me bind to one or decide to start my own.

Hi Rob,

The Vector, Signal and Image Processing Library (VSIPL)
has a limited scope.  But Randy Judd and I are both Physicists
so we are interested in general purpose numerical computing
as well as digital signal and image processing.
My hope was that the GNU Scientific Library (GSL)
would not be restricted to linear algebra
like the Basic Linear Algebra Subroutine (BLAS) library
and the Linear Algebra PACKage (LAPACK)
and that it would subsume the VSIPL API standard.

I implemented the VSIPL Portable Reference Library
that you got from Hughes Research Laboratories, Inc.
(now HRL Laboratories, LLC).
The Portable Reference Library was designed to be
easy to read, understand and maintain
because there wasn't any money to pay a programmer
to maintain the Portable Reference Library.
We actually began work on the Portable Reference Library
at the same time that we began work on the VSIPL
Application Programmer's Interface (API) specification
so that VSIPL Forum members could test their proposals
using the Portable Reference Library
before they presented them to the VSIPL Forum.
I suspect that the Portable Reference Library
has already served it's purpose and I suppose
that is the reason why nobody cares
whether it is ever released to the public or not.

I used a modified GNU genclass shell script
which also called the m4 preprocessor
to convert prototype files (containing what you call templates)
into ANSI C header and source files.
It was never my intention to release the prototype files
to the public but I did distribute them
to the members of the VSIPL Forum
along with the header and source files produced from them.

Vector, matrix and tensor classes are not container classes.
They are arrays of NUMBERS and NOT arbitrary types.
The decision to use templates, prototypes
or any other GENERIC method is an IMPLEMENTATION DETAIL
best left up to the LIBRARY DEVELOPER.
If you need a generic implementation,
you will need to negotiate that with the library developer
which, I guess, is what you are attempting to do here.

Randy Judd has implemented a high performance version
of the VSIPL API standard targeted
for a restricted class of computational platforms --
workstations (including personal computers).
He elected not to use any generic method
to implement his library and his implementation
of the VSIPL API standard happens to be similar
to the current implementation of the GSL.
If they were just a little more similar,
they could be INTEROPERABLE.  That is to say that
you could call GSL functions from Randy's VSIPL implementation
and you could call VSIPL functions from the current GSL.

If I could find someone to pay me to implement
a high performance version of the VSIPL API today,
I would probably write a perl script to replace
the old genclass shell script and accept templates
from a prototype file to emit ANSI C header and
source files for each supported type.
I would certainly try to leverage off of any
existing high performance numerical libraries
appropriate for the target platform.

What would I want in a grand numerical library?
Well just open up your copy
of Numerical Recipes in C
and read the table of contents.
I think that would be a very good place to start.
The problem with Numerical Recipes in C
is not that the implementation is so bad
but that the API makes it impossible
to re-implement the library.
And that's exactly what's wrong with the GSL as well.
No matter how clever and gifted the programmer,
it is practically impossible to implement
a high performance numerical library
that will port everywhere.
If you aren't very careful about how you design the API,
you will paint yourself into a corner
from which you will not be able to escape
without changing your API and breaking
existing application programs that depend upon it.

What we really need from the GSL designers
is some documentation of their rationale.
Something that we could use to determine
whether or not the GSL is appropriate for our purposes.
We need answers to very simple, basic questions.

1. What is the target platform?
2. Who are the target users?
3. What is the application domain?
4. What are the priorities?
5. What is the expected life of the library?
6. Who will provide support for the library?
7. What are the plans for the future?
   etc.

Otherwise, we will be obliged to wait and see
what sort of creature the GSL evolves into.

Re: GSL, VSIPL, and the ultimate numerical library

[Brian Gough]

That is an interesting message. Here are my thoughts.  A single grand
library may not be a desirable goal -- it would constrain the user to
operate within its parameters and implies a centralized development
model. I believe the most flexible approach involves having many
libraries:

 specialised libraries for different specialist areas 
 a general library covering the basics 

LAPACK, FFTW, etc would be examples of specialised libraries.  GSL
would fall into the general category.  The role of the general library
is to provide basic functionality and interoperability.  In the
C/FORTRAN and C-based VHLL world this is fairly straightforward.

For everyday purposes the general library should be sufficient, so
that one can carry out simple tasks easily. This is the position of
GSL: with it one can compute a random number, find a root or a
minimum, obtain the value of a special function, compute simple
statistics, interpolate a table of values, etc. 



Robert W. Brewer writes:
 > I've been thinking about what I would want in a grand
 > numerical library.  The things I myself would probably use a 
 > good numerical library for are some real-time speech processing
 > under Linux, and maybe as a basis for a neural networks
 > library.  Performance is important for both of those, otherwise
 > for offline use I would just use Octave or Matlab.  
 > If the Intel Signal Processing library ran under Linux that 
 > would be great, but it doesn't, and it is also a closed-source
 > library, which I'm generally against. 

Re: GSL, VSIPL, and the ultimate numerical library

[Randall Judd]

At 01:06 AM 11/02/2000 -0500, Robert W. Brewer wrote:


>The TASP VSIPL code is a pretty good choice, but it looks
>hard to maintain since there is so much code replication to
>support the different data types.  I recall an early version
>the Hughes VSIPL code which used a preprocessor template
>scheme that seemed pretty slick.  It looks like GSL has a
>similar problem, many functions are written 3 and 4 times
>just to support the different float and int types the user might want
>to instantiate.  And after all that, it still can't support
>neat things like arbitrary precision numbers, while a templating
>scheme might be able to.

It's not that hard to maintain, although there are some problems. I have 
found that templates tend to be more trouble than they are worth. Of course 
I am not a template or m4 expert.  M4 is what Bob was using. The time it 
takes to convert a float routine to a double routine is trivial. If I make 
major changes to a routine in float I usually just convert it wholesale 
instead of trying to change the double to match.

GSL and the TASP implementation of VSIPL are both trying to stay with ANSI 
C. I think writing routines which will support arbitrary precision would be 
difficult if not impossible, and of little use. VSIPL has the name space 
reserved for this precision but this was mostly to support any kind of chip 
precision at the native level. It was never fully developed and documented 
but the the idea of a vendor supplied tool (shell script) which would 
produce a header file to support particular precision of application code 
is supported in VSIPL.  If  the implementation did not support the users 
requested precision then the tool would fail (buy a library which supports 
your application precision requirements).


>To boil it down, the library I would like to use would be
>(most important first):
>    high performance
>    generic to allow optimizing under the hood
>       and advanced algorithms
>    templated for data type choices and easier maintenance
>    open source
>    easily tested for correctness with an automated suite
>
>I think the VSIPL API has a lot of good ideas for the
>functions that it supports, but the reference implementation
>would be easier to deal with if it were templatized.
>Maybe it's my personality, but making the same change
>in more than one place just wears me out.
The most important thing for VSIPL was portability. We want applications to 
run the same everywhere.

When you say "reference" I assume your talking about my implementation.

It would only easier to deal with if the person getting it understands and 
uses the same template scheme. What I go for is simplicity. This is mostly 
because I am kind of simple minded and frequently prefer a brut force 
method. But I see a lot of really nice code out there that I would never be 
able to figure out how to edit in this lifetime. There are things I would 
do differently (I am not as simple as I was when I started this project) if 
I was to start over. I am trying to work some of that stuff into my next 
release which should be out around January. One of them is to put all the 
source in a single directory so it is easier to build with tools like VC++.

The point I am trying to make here (not very well) is you can waste a lot 
of time trying to write something that lets a computer write your code for 
you. I find it is easier and surer to just do it the strait forward way.



>The GSL team has implemented a large breadth
>of functions, many of which have no equivalent in the VSIPL
>spec, but which could probably benefit from some of its principles.
>Since both the TASP reference implementation and
>GSL are open source, it might be possible to gradually
>combine them, as Randall Judd seems interested
>in doing.  Many GSL functions not present in VSIPL might
>be wrapped to present an interface similar to the VSIPL API.
>Then gradually the underlying GSL code could be moved into
>the wrapper to increase performance, better conform to
>VSIPL, etc.
I don't want to combine them. Just make sure they can work together.

>It might also be nice to go back to the
>technique in the Hughes code of implementing most of the
>functions in terms of other VSIPL/GSL functions or lower-level
>helper functions whenever possible.
I thought some of the stuff in Bobs code was neat, but almost none of the 
vendors liked it. It was also very (very, very,...) slow.  But VSIPL is a 
library specification, not an implementation. Anybody who is interested in 
doing some research in this area is more than welcome.


                                      Randy Judd

RE: GSL, VSIPL, and the ultimate numerical library

[E. Robert Tisdale]

Jiri Hajek wrote:

> In my opinion GSL is quite complex and useful library,
> though I have seen just documentation so far,
> because I haven't compiled it in Windows yet.
> However, there is one thing I miss a lot:
> C++ version of this library, at least for matrix algebra,
> because there is big diffence
> between writing several matrix operations
> using C++ operators and plain C functions.

C application programs are about three times larger
than C++ application programs
that do vector and matrix arithmetic.

> There is written "while allowing wrappers to be written
> for very high level languages" on the GSL web page.
> Did anybody try this or anybody plans to do it?

You could implement
The Scalar, Vector, Matrix and Tensor class library

	http://www.netwood.net/~edwin/svmt/

on top of the GNU Scientific Library (GSL).
But I am disinclined to do so myself right now.
Any application that depends upon the GSL will break
if the GSL library developers make any changes
in the data representation or algorithm implementation.
I also fear that the GSL developers may abandon the GSL
before it is completed like Robert D. Pierce
abandoned the GNU Scientific Software Library (GNUSSL).

	ftp://ftp.gnu.org/gnu/gnussl/

I can't even get the GNUSSL to compile anymore.

Re: GSL, VSIPL, and the ultimate numerical library

[Gerard Jungman]

Jiri Hajek wrote:
> because I haven't compiled it in Windows yet.

Has anybody compiled under Windows? In any form? Cygwin?
We don't see alot of reports.


> However, there is one thing I miss a lot: C++ version of this library, at

Well, there will never be a "C++ version" of GSL. GSL is
a C library from head to toe. A C++ project might reuse
some of the code (and more of the thinking), but would be
a completely distinct project.


> least for matrix algebra, because there is big diffence between writing
> several matrix operations using C++ operators and plain C functions.

Well, maybe operator overloading (together with the expression template
goop needed to make it satisfactory) is not a bad idea. But in itself
it is not a major issue. I say this only to try and enlighten anybody
who thinks that gimmicks will save the day. Gimmicks are gimmicks;
meanwhile there are real structural problems to be addressed.
Go to oonumerics.org to see the broad picture of what people
are concerned with.


> There is written "while allowing wrappers to be written for very high level
> languages" on the GSL web page. Did anybody try this or anybody plans to do
> it?

Somebody was following the releases fairly closely and creating
python bindings. We haven't heard from him in a while.


-- 
G. Jungman

Re: GSL, VSIPL, and the ultimate numerical library

[Gerard Jungman]

"Robert W. Brewer" wrote:
> Performance is important for both of those, otherwise
> for offline use I would just use Octave or Matlab.

As a general comment, I would like to say that this statement
already introduces something important to the discussion.
Some people like to say that "GSL is some kind of GPL'ed
numerical recipes, but better". For several reasons, some
of them obvious, I felt that was a pathetic design
goal.

If we want progress on our own "numerical science" desktops,
we need to understand what it is that people like us really do.
This means addressing issues like how to obtain coherence
across different systems, at different abstraction levels,
like Octave vs. heavy-duty compiled stuff. I hope that
people start thinking in terms of a "GNU Scientific Environment".
Some already are; the ones that I know about are listening
on this list. Jim Amundson coined "GNU Scientific Environment",
and has put real thought into what it might mean. Maybe there
are more people hiding out there with good ideas. Numerical
computation is itself only a part of the vision; there are also
questions like "what can we do with the GPL'ed macsyma?".

I don't use Octave myself, but as an example, I do like
to wrap library functionality in python, so that is a model
I understand for coherence at different levels of abstraction.
GSL is supposed to be "easy to wrap into high level langauges",
at least that is one of the design goals. Well, of course it
is easy to wrap (roughly speaking, if you can type 'swig' and
hit the return key, then you are in business). But that does
nothing to address coherence, because then you just get the
same basically inflexible GSL semantics, bound to another
language. It doesn't help much.

As we've learned in other areas, the key seems to be
a high level of genericity, separating data from
algorithms. Once you get rid of the baggage which is
not relevant to the problem domain, you have a hope
of being able to mix and match solutions.

This is a hard but fundamentally important problem.
I have hope basically only because of the actual
progress that has been made on genericity in C++
over the last 5 years or so. Not only are all these
template shenanigans nifty within the context of C++,
but they provide a model for what is needed outside
of any specific language; the language simply provides
the tool which makes it possible to explore all these
things that people have been trying to get at for
so long.

 
> It looks like GSL has a
> similar problem, many functions are written 3 and 4 times
> just to support the different float and int types the user might want
> to instantiate. 

Yup. That really stinks.

> And after all that, it still can't support
> neat things like arbitrary precision numbers, while a templating
> scheme might be able to.

Exactly. I am interested in that specific issue.
I think fundamental questions like "ok, but what
happens when this is 'long double' or 'float'"
need to be addressed on day one. Not just from the
standpoint of genericity in containers, but also
in algorithms. In principle, algorithms change too,
and you need to encapsulate the parts which vary.

I would be happy to start with some sort of
traits-class or similar solution, which is
basically a table-driven approach, lets say
assuming the existence of a complete and valid
std::numeric_traits implementation.

Support for actual (but static, compile-time) arbitrary
precision would be much harder. But we can almost see
how something like that could work.


> To boil it down, the library I would like to use would be
> (most important first):
>    high performance

Performance is a multidimensional space. Memory access
should be tightly controlled and appropriately parametrized
for different platforms. But I'm willing to loose
cycles; cycles are dirt cheap.

>    generic to allow optimizing under the hood
>       and advanced algorithms

Absolutely. Genericity is the key.

A certain group of language gurus (the aspect-oriented
programming community) likes to talk about what they
call "cross-cutting". To the extent that I understand
what this is, it seems to be one of the pervasive problems
in numerical computing. We have problem domains, which give
certain kinds of abstractions, and we have algorithms and
data, each of which provide other kinds of abstractions.
But these abstractions tend to cut across each other,
sometimes interfering destructively. Somehow, we have
to get more generic in order to defeat this problem.

>    templated for data type choices and easier maintenance

Yup. I think I can tell from your examples (emphasis on data flow,
linear operations, FFTs,...) that your emphasis is genericity
of containers. But as mentioned above, in the larger view
we must also understand how this works together with
parametrization of algorithms.

>    open source

Absolutely.

>    easily tested for correctness with an automated suite

Ahh, testing. What are we going to do about testing?
I though it was very interesting when the software-carpentry
contest had to give up on the testing module the first
time around. That is a very hard problem.

As somebody said in that context (I'm sure it's been said
many times), if your project was not designed from the
start with testing in mind, then you're cooked. The
idea of some kind of magic universal test harness is
pure fantasy.

I don't know what to do about it. I see very little here
or on the horizon which would help. I think we would have
to cobble together bits and pieces of existing tools and
hope for the best.


> I think the VSIPL API has a lot of good ideas for the
> functions that it supports, but the reference implementation
> would be easier to deal with if it were templatized.

I agree. Oh well.


> Maybe it's my personality, but making the same change
> in more than one place just wears me out.

Yup. We only live so long. And besides, when something
wears you out like that, you get angry with it and start
losing faith in the implementation. Just more tedious
foo to keep track of. I don't see people talk about
this much, but I think the project design has to
support positive psychology. This isn't Microsoft;
we should be able to afford doing things the right way.


> The GSL team has implemented a large breadth
> of functions, many of which have no equivalent in the VSIPL
> spec, but which could probably benefit from some of its principles.
> Since both the TASP reference implementation and
> GSL are open source, it might be possible to gradually
> combine them, as Randall Judd seems interested
> in doing.  Many GSL functions not present in VSIPL might
> be wrapped to present an interface similar to the VSIPL API.
> Then gradually the underlying GSL code could be moved into
> the wrapper to increase performance, better conform to
> VSIPL, etc.

I guess I have no real comment on this. There's no reason they
can't get along. But I'm not sure if either one is what I am
really interested in. The level of combining that was discussed
was getting them a little more semantically aligned so that
they could coexist (specifically memory management issues).
I don't think anything else was intended.

I like the opaqueness of the VSIPL data objects, and I don't
very much like the defacto translucency of the GSL data
objects. But in any case, it's hard to see how we can stop
short of a full-blown template-based implementation (something like MTL)
and still get what we want. But maybe this is still the wrong emphasis,
from the standpoint of coherence. I don't know.

Personally, I would rather not have to think about
things like vector/matrix implementations. I am ready, at
least as a first approximation, to declare this problem
as solved (say by MTL, or one of the few other related
projects) and move on. After all, if we have achieved the
correct level of genericity, then it shouldn't matter
which data representation I choose to use today.


> It might also be nice to go back to the
> technique in the Hughes code of implementing most of the
> functions in terms of other VSIPL/GSL functions or lower-level
> helper functions whenever possible.

Sure, isolating a kernel of operations is a time-honored approach.
For instance, in GSL we have a BLAS kernel. I made it so that you
could use the native GSL implementation (nothing great), or link
in your own high performance BLAS kernel.

Now, why the GSL BLAS impl had to be completely native is a mystery
even to me. That was a dumb idea. I blame myself, although only
in part since it was based on general brain-damaged GSL philosophy.
It's still not even finished. We should have just used the
CBLAS model for wrapping existing fortran BLAS and provided
a copy of the fortran reference, with the same "link your
own if you like" methodology.

BLAS is BLAS. It works. It's the same everywhere. It will
never change. GSL commited a serious sin of wheel-reinventing
there, and it stains the whole project.

I would have also liked to see a LAPACK kernel, with all the
same thoughts. Again, CLAPACK solved all the problems (not
nearly as easy as BLAS), and we should have leveraged that
as far as it could go.

Will these sins be corrected? Before version 1.0?


> But the idea of writing Rob's Gee Whiz Altivec Library
> is not very appealing to me, first because VSIPL and GSL have
> put a lot of thought into how to do it right,

I wouldn't oversell the level of thought in GSL.
I know; I was there.


> I guess in the end I can understand why Tisdale
> is so interested in the vision and goals of GSL, so
> it is easier to decide if it is the same direction
> that he would like to go.  I am in a similar quandary,
> there are several interesting-looking projects with
> various tradeoffs, and looking into the future a little would
> help me bind to one or decide to start my own.

It's a real problem. GUI's were in the same situation
for a decade. Which solution do you use? Motif? (GACKKK!)
Xt? (mega GACKKK!), one of the horde of half-assed libraries?
(GACKKK, GACKKK, GACKKK,...)

Now we have gnome and kde. Both are good. Both are defacto
standards. Everybody benefits.

Standards help everybody. It doesn't
matter so much how they come about.
But if we can all more or less agree that
something is good, then we can feel good
about using it. This has not yet happened
in the world of numerical tools.


Thanks for your thoughts (and another
chance for me to stand on the soapbox).

-- 
G. Jungman

RE: GSL, VSIPL, and the ultimate numerical library

[Jiri Hajek]

> Comments and replies welcome!

In my opinion GSL is quite complex and useful library, though I have seen
just documentation so far, because I haven't compiled it in Windows yet.
However, there is one thing I miss a lot: C++ version of this library, at
least for matrix algebra, because there is big diffence between writing
several matrix operations using C++ operators and plain C functions.

There is written "while allowing wrappers to be written for very high level
languages" on the GSL web page. Did anybody try this or anybody plans to do
it?

Continue in developement, please!

Jiri

GSL, VSIPL, and the ultimate numerical library

[Robert W. Brewer]

I just read a good chunk of the year 2000
list archives.  I've been following the VSIPL
forum work on and off for the past couple of 
years, and I was pretty excited when I saw the announcement
of GSL 0.7 on freshmeat.net, because it sounded 
like it might have already beaten VSIPL to the punch
so to speak.  It appears that a lot of good work
has gone into GSL so far.

I've been thinking about what I would want in a grand
numerical library.  The things I myself would probably use a 
good numerical library for are some real-time speech processing
under Linux, and maybe as a basis for a neural networks
library.  Performance is important for both of those, otherwise
for offline use I would just use Octave or Matlab.  
If the Intel Signal Processing library ran under Linux that 
would be great, but it doesn't, and it is also a closed-source
library, which I'm generally against. 

The TASP VSIPL code is a pretty good choice, but it looks
hard to maintain since there is so much code replication to
support the different data types.  I recall an early version
the Hughes VSIPL code which used a preprocessor template
scheme that seemed pretty slick.  It looks like GSL has a
similar problem, many functions are written 3 and 4 times
just to support the different float and int types the user might want
to instantiate.  And after all that, it still can't support
neat things like arbitrary precision numbers, while a templating
scheme might be able to.

To boil it down, the library I would like to use would be
(most important first):
   high performance
   generic to allow optimizing under the hood 
      and advanced algorithms
   templated for data type choices and easier maintenance
   open source
   easily tested for correctness with an automated suite

I think the VSIPL API has a lot of good ideas for the 
functions that it supports, but the reference implementation 
would be easier to deal with if it were templatized.
Maybe it's my personality, but making the same change 
in more than one place just wears me out.

The GSL team has implemented a large breadth
of functions, many of which have no equivalent in the VSIPL
spec, but which could probably benefit from some of its principles.
Since both the TASP reference implementation and
GSL are open source, it might be possible to gradually
combine them, as Randall Judd seems interested
in doing.  Many GSL functions not present in VSIPL might
be wrapped to present an interface similar to the VSIPL API.
Then gradually the underlying GSL code could be moved into
the wrapper to increase performance, better conform to
VSIPL, etc.

It might also be nice to go back to the
technique in the Hughes code of implementing most of the
functions in terms of other VSIPL/GSL functions or lower-level
helper functions whenever possible.
This could have the advantage of allowing a mostly optimized version
for a specific hardware platform by just optimizing the lowest-level
vector routines and helpers for that platform, and allowing the 
more complicated routines to ride on top and still benefit.
The hope would be that, say, 80% of the Altivec
(or Intel's SIMD) performance gain could come from 
writing Altivec assembly in only, say, 20% of the functions.  
I mention this because the neat thing to me about having 
a nice numerical library is being able to do some impressive
real-time signal processing tricks on a desktop PC.
Intel's MMX and Streaming SIMD Extensions, AMD's 3D Now instructions,
and PowerPC's Altivec technology all allow for amazingly fast
computation, but since standard compilers can't take advantage
of them, writing assembly code with a high-level API is the
way to go.  But the idea of writing Rob's Gee Whiz Altivec Library
is not very appealing to me, first because VSIPL and GSL have
put a lot of thought into how to do it right, and second because
I'd rather not start from scratch if I switch to a non-Altivec machine.

I guess in the end I can understand why Tisdale
is so interested in the vision and goals of GSL, so
it is easier to decide if it is the same direction 
that he would like to go.  I am in a similar quandary,
there are several interesting-looking projects with
various tradeoffs, and looking into the future a little would
help me bind to one or decide to start my own.

Comments and replies welcome!

-Rob
-- 
Robert W. Brewer  PGP RSA 2048-bit Key ID: 03E0E635
Jesus rules!      Fingerprint: 6327 8034 7BDA D144 B40C C5E2 F760 13BB