New routines for moving window statistics and filters

New routines for moving window statistics and filters

[Patrick Alken]

Dear GSL users/developers,

I have added a new module called gsl_movstat to GSL, which provides 
routines for moving window statistics (also called sliding window 
statistics, rolling statistics, running statistics, etc). Currently 
there is support for the following:

moving mean, sum, min/max, variance/stddev, median, MAD, q-quantile 
range, Q_n, S_n

I have also added some robust statistics routines to the gsl_stats area, 
including MAD, S_n, Q_n, Gastwirth and trimmed mean routines.

Finally, I added a new module called gsl_filter, which currently 
contains a small number of common filtering routines. Currently, there 
is 1 linear filter (Gaussian smoothing) and 3 nonlinear filters (median, 
recursive median and impulse-rejection filters). I would like to 
eventually add other common filters (like Butterworth, Chebyshev) and 
possibly some routines to allow users to design their own filters with 
various criteria. This probably won't happen before the next release 
however.

I have put everything into the master branch of the git with 
documentation. Any feedback/suggestions are welcome.

Thanks,

Patrick

Re: New routines for moving window statistics and filters

[Brian Gladman]

On 02/05/2018 21:58, Patrick Alken wrote:
> Dear GSL users/developers,
> 
> I have added a new module called gsl_movstat to GSL, which provides
> routines for moving window statistics (also called sliding window
> statistics, rolling statistics, running statistics, etc). Currently
> there is support for the following:
> 
> moving mean, sum, min/max, variance/stddev, median, MAD, q-quantile
> range, Q_n, S_n
> 
> I have also added some robust statistics routines to the gsl_stats area,
> including MAD, S_n, Q_n, Gastwirth and trimmed mean routines.
> 
> Finally, I added a new module called gsl_filter, which currently
> contains a small number of common filtering routines. Currently, there
> is 1 linear filter (Gaussian smoothing) and 3 nonlinear filters (median,
> recursive median and impulse-rejection filters). I would like to
> eventually add other common filters (like Butterworth, Chebyshev) and
> possibly some routines to allow users to design their own filters with
> various criteria. This probably won't happen before the next release
> however.
> 
> I have put everything into the master branch of the git with
> documentation. Any feedback/suggestions are welcome.

Hi Patrick,

I have been trying to add your new material to the Visual Studio build
of GSL and have found a very large number of errors with the Microsoft
compiler because of the assumption that a (void*) pointer points to an
object of length one.  This is a non standard GCC extension.

One of many examples is on line 363 of median.c is:

  state->minmax_state = vstate + sizeof(rmedian_state_t);

where vstate is defined as a (void*).

Although I could change all these (void*) to (char*), I am reluctant to
do this as it seems potentially very error prone.

Since GSL is (I believe) supposed to be written in 'standard C' I also
assume that the use of this GCC extension needs to be removed anyway.

I would hence be grateful for your advice on how to errors of this type.

   best regards,

     Brian Gladman

Re: New routines for moving window statistics and filters

[Patrick Alken]

Brian,

   Thanks for testing. gcc never complained about the void* arithmetic 
so I didn't catch this error. I have replaced these lines by casting the 
void pointers to unsigned char *, which should correct the issue. Can 
you try out the latest git and let me know if it fixes the issue?

Due to the many different algorithms used in the moving statistics 
module, I am trying to keep the size of the workspace smaller by reusing 
the same workspace for the different algorithms. Therefore each moving 
window algorithm initializes a void* workspace as needed, hence the 
reason I am doing the pointer arithmetic.

It seems that by casting everything to unsigned char *, it should 
conform to ansi C. I have tested it with strict compiler flags as well 
as compiling under the C89 and C99 standard. Let me know if it works now.

Thanks,
Patrick

On 05/03/2018 04:36 AM, Brian Gladman wrote:
> On 02/05/2018 21:58, Patrick Alken wrote:
>> Dear GSL users/developers,
>>
>> I have added a new module called gsl_movstat to GSL, which provides
>> routines for moving window statistics (also called sliding window
>> statistics, rolling statistics, running statistics, etc). Currently
>> there is support for the following:
>>
>> moving mean, sum, min/max, variance/stddev, median, MAD, q-quantile
>> range, Q_n, S_n
>>
>> I have also added some robust statistics routines to the gsl_stats area,
>> including MAD, S_n, Q_n, Gastwirth and trimmed mean routines.
>>
>> Finally, I added a new module called gsl_filter, which currently
>> contains a small number of common filtering routines. Currently, there
>> is 1 linear filter (Gaussian smoothing) and 3 nonlinear filters (median,
>> recursive median and impulse-rejection filters). I would like to
>> eventually add other common filters (like Butterworth, Chebyshev) and
>> possibly some routines to allow users to design their own filters with
>> various criteria. This probably won't happen before the next release
>> however.
>>
>> I have put everything into the master branch of the git with
>> documentation. Any feedback/suggestions are welcome.
> Hi Patrick,
>
> I have been trying to add your new material to the Visual Studio build
> of GSL and have found a very large number of errors with the Microsoft
> compiler because of the assumption that a (void*) pointer points to an
> object of length one.  This is a non standard GCC extension.
>
> One of many examples is on line 363 of median.c is:
>
>    state->minmax_state = vstate + sizeof(rmedian_state_t);
>
> where vstate is defined as a (void*).
>
> Although I could change all these (void*) to (char*), I am reluctant to
> do this as it seems potentially very error prone.
>
> Since GSL is (I believe) supposed to be written in 'standard C' I also
> assume that the use of this GCC extension needs to be removed anyway.
>
> I would hence be grateful for your advice on how to errors of this type.
>
>     best regards,
>
>       Brian Gladman

Re: New routines for moving window statistics and filters

[Brian Gladman]

On 03/05/2018 17:32, Patrick Alken wrote:
> Brian,
> 
>   Thanks for testing. gcc never complained about the void* arithmetic so
> I didn't catch this error. I have replaced these lines by casting the
> void pointers to unsigned char *, which should correct the issue. Can
> you try out the latest git and let me know if it fixes the issue?

Hi Patrick

That solved it and all the new material now works with Visual Studio 2017.

I had to make a small change in filter/test_median.c because the Unix
function gettimeofday() is not available on Windows so I had to add a
local one.  I have added this change to the main repository.

     Brian