Parallel computing abilitites in GSL?

Parallel computing abilitites in GSL?

[Wartan Hachaturow]

Hello.

First of all I'd really like to thank you guys for this _brilliant_ library.
(It does even has those braindamaged ntuples -- and I don't have to
use Fortran again to parse them! :)

The question is: what do you think -- is it overall possible to create
some (perhaps, built-in in the library (best) or it could be separate
compilable wrapper, or something like that) support for out-of-the-box
paralleling of gsl-using applications?
Say, for example, one might expect GSL (being told about the resources
available) to automatically run parts of the algorithms (that does allow
that) on multiple machines, using PVM/MPI/whatever. That would allow
people to easily create paralleled applications, using their favourite
scientific library..

-- 
Regards, Wartan.
"Computers are not intelligent. They only think they are."

Parallel computing abilitites in GSL?

[Wartan Hachaturow]

Hello.

First of all I'd really like to thank you guys for this _brilliant_ library.
(It does even has those braindamaged ntuples -- and I don't have to
use Fortran again to parse them! :)

The question is: what do you think -- is it overall possible to create
some (perhaps, built-in in the library (best) or it could be separate
compilable wrapper, or something like that) support for out-of-the-box
paralleling of gsl-using applications?
Say, for example, one might expect GSL (being told about the resources
available) to automatically run parts of the algorithms (that does allow
that) on multiple machines, using PVM/MPI/whatever. That would allow
people to easily create paralleled applications, using their favourite
scientific library..

-- 
Regards, Wartan.
"Computers are not intelligent. They only think they are."

Re: Parallel computing abilitites in GSL?

[Brian Gough]

Wartan Hachaturow writes:
 > The question is: what do you think -- is it overall possible to create
 > some (perhaps, built-in in the library (best) or it could be separate
 > compilable wrapper, or something like that) support for out-of-the-box
 > paralleling of gsl-using applications?

Hi, 

Basically parallel algorithms are outside the scope of GSL, because
they would require a different design.

regards
Brian Gough

Re: Parallel computing abilitites in GSL?

[Brian Gough]

Wartan Hachaturow writes:
 > The question is: what do you think -- is it overall possible to create
 > some (perhaps, built-in in the library (best) or it could be separate
 > compilable wrapper, or something like that) support for out-of-the-box
 > paralleling of gsl-using applications?

Hi, 

Basically parallel algorithms are outside the scope of GSL, because
they would require a different design.

regards
Brian Gough

Re: Parallel computing abilitites in GSL?

[Hossein S. Zadeh]

On Sun, 7 Jul 2002, Brian Gough wrote:

> Wartan Hachaturow writes:
>  > The question is: what do you think -- is it overall possible to create
>  > some (perhaps, built-in in the library (best) or it could be separate
>  > compilable wrapper, or something like that) support for out-of-the-box
>  > paralleling of gsl-using applications?
> 
> Basically parallel algorithms are outside the scope of GSL, because
> they would require a different design.

Hi there,
I agree and disagree!! I agree that a generalised kind of parallelism is
well outside the scope of GSL. But algorithms like Simulated Annealing can
have a parameter to specify how many child threads to spawn. On a
multi-cpu system this could help a lot. And even though this doesn't help
with load distribution over a cluster, it makes it easier to incorporate
it in things like PVM. (I haven't actually used Simulated Annealing in
GSL, so this might already be provided??)

cheers,
Hossein

Re: Parallel computing abilitites in GSL?

[Brian Gough]

Hossein S. Zadeh writes:
 > I agree and disagree!! I agree that a generalised kind of parallelism is
 > well outside the scope of GSL. But algorithms like Simulated Annealing can
 > have a parameter to specify how many child threads to spawn. On a
 > multi-cpu system this could help a lot. And even though this doesn't help
 > with load distribution over a cluster, it makes it easier to incorporate
 > it in things like PVM. (I haven't actually used Simulated Annealing in
 > GSL, so this might already be provided??)

For algorithms like simulated annealing or monte carlo integration I
think the user could start up each thread from their application.
However, what GSL certainly needs for parallel codes is improvement of
the RNGs to support a larger seed space.  The current 32-bit seed is
inadequate for parallel programs.   

What's probably needed is an extra method for each generator which
accepts a arbitrary-sized array of seed bits, and hashes it up or down
in size to fit each generator. This could be used to provide a two
argument seed function gsl_rng_seed2 (thread_id,seed).

Brian

Re: Parallel computing abilitites in GSL?

[Wartan Hachaturow]

On Sun, Jul 07, 2002 at 11:21:33PM +0100, Brian Gough wrote:

> Basically parallel algorithms are outside the scope of GSL, because
> they would require a different design.

Well, actually the idea wasn't to create an underlying parallel level,
that would lead to a redisign of GSL (though that might be cool), but
to create ready-to-be-paralleled versions of some algoritms, and include
them into the distribution, similar to the way GSL makes error-reporting
versions of functions.
Say, it could look like GSL would provide 3 functions
gsl_some_function()
gsl_e_some_function()
gsl_p_some_function(additional arguments).

The idea behind this is that most of the ready-paralleled numeric
libraries (such as those provided by Numeric Solutions) are not free,
and it would be very good to provide a free software to do this.
GSL has a nice ready base for this, IMO.
But this is no more than just an idea now :)

-- 
Regards, Wartan.
"Computers are not intelligent. They only think they are."

Re: Parallel computing abilitites in GSL?

[Brian Gough]

Wartan Hachaturow writes:
 > Well, actually the idea wasn't to create an underlying parallel level,
 > that would lead to a redisign of GSL (though that might be cool), but
 > to create ready-to-be-paralleled versions of some algoritms, and include
 > them into the distribution, similar to the way GSL makes error-reporting
 > versions of functions.

If anyone wants to write some parallel code they could use the same
API as the corresponding GSL routine. It could then be dropped in to
an application without changes by the user. This would be the best way
to do it, as with BLAS.  I'd certainly be happy to make it available
in the gsl ftp area if anyone did that.

Brian

Re: Parallel computing abilitites in GSL?

[tlb]

> paralleling of gsl-using applications?

In fact, the freely available ATLAS library also uses multiple
processors on suitable systems. Try it first. You should be using this
anyway, as it's about twice as fast as the BLAS library included with
GSL on a single Pentium machine.

http://math-atlas.sourceforge.net

--
Trevor Blackwell         tlb@tlb.org          (650) 776-7870

Re: Parallel computing abilitites in GSL?

[tlb]

> paralleling of gsl-using applications?

In fact, the freely available ATLAS library also uses multiple
processors on suitable systems. Try it first. You should be using this
anyway, as it's about twice as fast as the BLAS library included with
GSL on a single Pentium machine.

http://math-atlas.sourceforge.net

--
Trevor Blackwell         tlb@tlb.org          (650) 776-7870

Re: Parallel computing abilitites in GSL?

[tlb]

> paralleling of gsl-using applications?

If most of the work is large matrix operations, there are some
parallel BLAS libraries that can harness multiple processors for doing
the low-level matrix ops.

Intel sells a BLAS that uses multiple processors on Linux and Windows:

  http://www.intel.com/software/products/mkl/mkl52/features.htm

I haven't tried it, but they offer a free trial. I would guess that
parallelism only helps with matrices larger than about 30x30.

SGI also provides parallel BLAS libraries for some of their hardware.

Automatically parallelizing entire applications is an unsolved
research problem. You have to do it manually.

--
Trevor Blackwell         tlb@tlb.org          (650) 776-7870

Re: Parallel computing abilitites in GSL?

[tlb]

> paralleling of gsl-using applications?

If most of the work is large matrix operations, there are some
parallel BLAS libraries that can harness multiple processors for doing
the low-level matrix ops.

Intel sells a BLAS that uses multiple processors on Linux and Windows:

  http://www.intel.com/software/products/mkl/mkl52/features.htm

I haven't tried it, but they offer a free trial. I would guess that
parallelism only helps with matrices larger than about 30x30.

SGI also provides parallel BLAS libraries for some of their hardware.

Automatically parallelizing entire applications is an unsolved
research problem. You have to do it manually.

--
Trevor Blackwell         tlb@tlb.org          (650) 776-7870

Re: Parallel computing abilitites in GSL?

[E. Robert Tisdale]

Wartan Hachaturow wrote:

> Is it overall possible to create some support
> (perhaps, built-in in the library (best)
>  or it could be separate compilable wrapper
>  or something like that)
> for out-of-the-box paralleling of gsl-using applications?
> Say, for example, one might expect GSL
> (being told about the resources available)
> to automatically run parts of the algorithms
> (that does allow that) on multiple machines,
> using PVM/MPI/whatever.
> That would allow people to easily create
> parallel applications using their favourite scientific library.

Take a look at
the Vector, Signal and Image Processing Library (VSIPL) home page:

	http://www.vsipl.org/

    HPEC-SI CONTRIBUTES TO EXTENSION OF VSIPL 
    The High Performance Embedded Computing Software Initiative,
    sponsored by the U.S. Dept. of Defense
    and its High Performance Computing Modernization Office,
    is further advancing the development
    of embedded computing software standards
    with a goal of software portability:
    to enable "write-once/run-anywhere"
    for applications of high performance embedded computing.
    The VSIPL Forum is cooperating with HPEC-SI
    in the development of proposals for "VSIPL++",
    a C++ binding for VSIPL, and a "parallel VSIPL++"
    for multiprocessor systems.

The GSL and the VSIPL are very similar.
The VSIPL has a much narrower scope than the GSL
but parallel VSIPL++ will use
the Message Passing Interface (MPI) standard.