[ECOS] On demand FP context switch major problem.

[ECOS] On demand FP context switch major problem.

[Sergei Organov]

Hello,

I've almost finished the "on demand" FP context switch support in the PowerPC
HAL, but at the end faced with a problem for which I don't see a reasonable
solution. Here is the description of the problem.

To switch FP context on demand we need to disable FP in the usual context
switch routine by means of clearing the "FP enable" bit in the MSR. It means
that at thread level of execution the FP bit in the MSR could be cleared any
time if scheduler isn't locked and/or interrupts aren't disabled. It in turn
means that in thread code we can safely change MSR only if above condition(s)
is(are) met. But the EE (external interrupt enable) bit is also located in the
MSR, so it's unsafe to disable interrupts if scheduler isn't locked!

The above has unfortunate consequences for the code in the
HAL_DISABLE_INTERRUPTS macro. With "on demand" FP context switch is used, this
macro should somehow lock the scheduler before reading the MSR and then unlock
it back after writing the MSR :-(

The above discussion didn't take into consideration support for the use of FPU
in the interrupt handlers (ISRs/DSRs). If this support is added, then even
locking of the scheduler doesn't help. In this case only disabling of
interrupts gives us a safe way to change MSR, but once again disabling of
interrupts needs changing of MSR :-(

Any suggestions? Hopefully I'm just missing something obvious.

If above explanation of the problem isn't clear, here is a possible scenario:

1. Assume Thread1 is running with FP enable bit set (it owns FP context) and
   invokes HAL_DISABLE_INTERRUPTS macro.
2. HAL_DISABLE_INTERRUPTS macro reads current MSR value to a GPR, saves it
   into memory and clears EE bit in the GPR.
3. Thread1 is preempted by Thread2 that executes FP instruction and thus
   receives ownership of the FP context.
4. Thread2 is put to the waiting state and execution returns to the Thread1.
5. Thread1 continues execution but now FP enable bit in the MSR is cleared,
   because FP context is owned by the Thread2.
6. HAL_DISABLE_INTERRUPTS macro writes the GPR with EE bit cleared to the
   MSR and thus effectively sets the FP bit in the MSR!
7. Thread1 continues to execute with interrupts disabled and executes floating
   point instruction. As the FP enable bit is set, this doesn't produce "FP
   unavailable" exception and thus FP context isn't switched. As a result, the
   FP context of the Thread2 is clobbered by the Thread1 :-(

BR,
Sergei.

Re: [ECOS] On demand FP context switch major problem.

[Nick Garnett]

Sergei Organov <osv@javad.ru> writes:

> Hello,
> 
> I've almost finished the "on demand" FP context switch support in the PowerPC
> HAL, but at the end faced with a problem for which I don't see a reasonable
> solution. Here is the description of the problem.
> 
> To switch FP context on demand we need to disable FP in the usual context
> switch routine by means of clearing the "FP enable" bit in the MSR. It means
> that at thread level of execution the FP bit in the MSR could be cleared any
> time if scheduler isn't locked and/or interrupts aren't disabled. It in turn
> means that in thread code we can safely change MSR only if above condition(s)
> is(are) met. But the EE (external interrupt enable) bit is also located in the
> MSR, so it's unsafe to disable interrupts if scheduler isn't locked!
> 
> The above has unfortunate consequences for the code in the
> HAL_DISABLE_INTERRUPTS macro. With "on demand" FP context switch is used, this
> macro should somehow lock the scheduler before reading the MSR and then unlock
> it back after writing the MSR :-(
> 
> The above discussion didn't take into consideration support for the use of FPU
> in the interrupt handlers (ISRs/DSRs). If this support is added, then even
> locking of the scheduler doesn't help. In this case only disabling of
> interrupts gives us a safe way to change MSR, but once again disabling of
> interrupts needs changing of MSR :-(
> 
> Any suggestions? Hopefully I'm just missing something obvious.


This is one of the annoying features of the PowerPC, and other RISC
processors, that "global" state and "per-thread" state are mixed up in
the same register.

One option would be to always clear the FP enable bit in
HAL_DISABLE_INTERRUPTS (probably HAL_ENABLE_INTERRUPTS and
HAL_RESTORE_INTERRUPTS too). This would make your described scenario
work correctly since you would get an FP exception at step 7. In the
simple case you would also get an FP trap on the next FP instruction,
which would see that the current thread is already the FPU owner and
just set the FP enable bit.

Taking the extra trap is unpleasant, but with a suitable fast path it
should not be too expensive. It is unlikely that code which plays with
the interrupt enable is also doing a lot of floating point, and the
thread may well be timesliced or preempted before moving from one to
the other anyway. So the chances are that this will not result in any
more FP traps than before. There has to be a FP trap per thread switch
anyway, and changing the interrupt enable should be a lot less common
than that.



-- 
Nick Garnett
Cygnus Solutions, a Red Hat Company
Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Sergei Organov]

Nick,

What you suggest seems to be the most reasonable (if not the only) solution to
the problem.

Thanks,
Sergei.

Nick Garnett <nickg@cygnus.co.uk> writes:
> Sergei Organov <osv@javad.ru> writes:
[...]
> This is one of the annoying features of the PowerPC, and other RISC
> processors, that "global" state and "per-thread" state are mixed up in
> the same register.
>
> One option would be to always clear the FP enable bit in
> HAL_DISABLE_INTERRUPTS (probably HAL_ENABLE_INTERRUPTS and
> HAL_RESTORE_INTERRUPTS too). This would make your described scenario
> work correctly since you would get an FP exception at step 7. In the
> simple case you would also get an FP trap on the next FP instruction,
> which would see that the current thread is already the FPU owner and
> just set the FP enable bit.
>
> Taking the extra trap is unpleasant, but with a suitable fast path it
> should not be too expensive. It is unlikely that code which plays with
> the interrupt enable is also doing a lot of floating point, and the
> thread may well be timesliced or preempted before moving from one to
> the other anyway. So the chances are that this will not result in any
> more FP traps than before. There has to be a FP trap per thread switch
> anyway, and changing the interrupt enable should be a lot less common
> than that.
>
>
>
> --
> Nick Garnett
> Cygnus Solutions, a Red Hat Company
> Cambridge, UK

[ECOS] eCos synthetic target on Sun/Sparc ?

[Arvind]

Hi

Is it possible to have a eCos synthetic target on SunOS/Ultra Sparc instead
of Linux/x86.
Motivation: i am looking for cache profiling tools on the net, most of the
ones that were useful to me run on Sparc machines, and the only Sparc
machine that we have runs SunOS.
If you guys know any good cache profiling tools on Linux/x86, i would be
interested in hearing about them also.
Thanks for any response,

regards
Arvind

Re: [ECOS] eCos synthetic target on Sun/Sparc ?

[Bart Veer]

>>>>> "Arvind" == Arvind  <arvindj@sasi.com> writes:

    Arvind> Is it possible to have a eCos synthetic target on
    Arvind> SunOS/Ultra Sparc instead of Linux/x86.

    Arvind> Motivation: i am looking for cache profiling tools on the
    Arvind> net, most of the ones that were useful to me run on Sparc
    Arvind> machines, and the only Sparc machine that we have runs
    Arvind> SunOS.

It may be possible, but it will require a fair bit of porting work.

The first requirement is that the eCos synthetic target application
must be linked without using any of the SunOS libraries, because of
name clashes etc. Your synthetic target app will want to use the eCos
version of printf(), not the SunOS one.

However, the synthetic target app must still make certain kernel
calls, for example read() and write(). This list will be extended in
future as the synthetic target is developed further. Under Linux all
of the calls are available as system traps, so read is defined
using something like:

        .globl cyg_hal_sys_read;                \
                                                \
cyg_hal_sys_read:                               \
                                                \
        push %ebx;                              \
        mov 8(%esp), %ebx;                      \
        mov 12(%esp), %ecx;                     \
        mov 16(%esp), %edx;                     \
        lea     3, %eax;                        \
        int $0x80;                              \
        pop %ebx;                               \
        ret;                                    \

The actual code uses various macros to keep things simple. So the eCos
HAL can make Linux system calls without linking with any Linux
libraries. You would have to figure out the SunOS equivalent, i.e. how
to make system calls directly without linking with any SunOS
libraries.

There may be other problems as well, you would certainly need to take
a good look at the existing synthetic target code. However the above
work would certainly need to be done.

    Arvind> If you guys know any good cache profiling tools on
    Arvind> Linux/x86, i would be interested in hearing about them
    Arvind> also. Thanks for any response,

By a strange coincidence, I spotted the text below on another mailing
list today. It may be useful.

Bart

------------------------------------------------------------------------
I was just catching up on my linux-kernel mailing list and noticed:

| From owner-linux-kernel-outgoing@vger.rutgers.edu  Sat May 13 18:55:48 2000  
| Date:   Sun, 14 May 2000 00:49:18 +0200 (MET DST)
| From: Mikael Pettersson <mikpe@csd.uu.se>
| Message-Id: <200005132249.AAA01834@harpo.it.uu.se>
| To: linux-kernel@vger.rutgers.edu
| Subject: [Announce] Version 1.1 of x86 performance counters driver
| 
| Version 1.1 of my x86 performance-monitoring counters driver is
| now available at http://www.csd.uu.se/~mikpe/linux/perfctr/ .
| 

           Linux x86 Performance-Monitoring Counters Driver
                by Mikael Pettersson <mikpe@csd.uu.se>
                       [Last updated 2000-05-13]
========================================================================

Latest Updates
--------------
Version 1.1, 2000-05-13
- Support for Linux kernels 2.2.14, 2.2.15 and 2.3.99-pre8.
- Changes to the driver and user-space library to reduce the
  number of getpid() calls. (Suggested by Ulrich Drepper.)
- Added support for the VIA Cyrix III processor.
- Performance improvements in the x86 driver interface.
- Some code cleanups.

Overview
--------
This package adds support to the Linux kernel for using the
Performance-Monitoring Counters (PMCs) found in many modern
x86-class processors. Supported processors are:
- All Intel family 5 and 6 processors, i.e. Pentium, Pentium MMX,
  Pentium Pro, Pentium II, Celeron, and Pentium III.
- AMD K7 Athlon. (Note: the K6 family has no PMCs.)
- Cyrix 6x86MX, MII, and III.
- WinChip C6, 2, 2A, and 3.

PMCs are "event counters" capable of recording any of a large
number of performance-related events during execution.
These events typically include instructions executed, cache
misses, TLB misses, stalls, and other events specific to
the microarchitecture of the processor being used.

<snip>

Status
------
The code has been tested on Intel Pentium, Pentium MMX,
Pentium II, Pentium III, and AMD K7 processors, UP and SMP
kernels, with and without modules.

This and future versions of this package can be downloaded from
http://www.csd.uu.se/~mikpe/linux/perfctr/ .

Re: [ECOS] On demand FP context switch major problem.

[Sergei Organov]

Nick,

I was very busy last two weeks, but now when I've got a chance to return to
the FP implementation, I've found another related problem.

The default interrupt handler calls 'hal_cpu_int_disable' close to the end of
the ISR processing. If I disable FP there, then *every* interrupt will result
in disabling of FP, that is obviously unacceptable. 

Yet another similar place is in the
'hal_interrupt_stack_call_pending_DSRs' that restores interrupt settings
before return.

Do you see a way to handle above cases?

BR,
Sergei.

Sergei Organov <osv@javad.ru> writes:
> Nick,
> 
> What you suggest seems to be the most reasonable (if not the only) solution
> to the problem.
> 
> Thanks,
> Sergei.
> 
> Nick Garnett <nickg@cygnus.co.uk> writes:
> > Sergei Organov <osv@javad.ru> writes:
> [...]
> > This is one of the annoying features of the PowerPC, and other RISC
> > processors, that "global" state and "per-thread" state are mixed up in
> > the same register.
> >
> > One option would be to always clear the FP enable bit in
> > HAL_DISABLE_INTERRUPTS (probably HAL_ENABLE_INTERRUPTS and
> > HAL_RESTORE_INTERRUPTS too). This would make your described scenario
> > work correctly since you would get an FP exception at step 7. In the
> > simple case you would also get an FP trap on the next FP instruction,
> > which would see that the current thread is already the FPU owner and
> > just set the FP enable bit.
> >
> > Taking the extra trap is unpleasant, but with a suitable fast path it
> > should not be too expensive. It is unlikely that code which plays with
> > the interrupt enable is also doing a lot of floating point, and the
> > thread may well be timesliced or preempted before moving from one to
> > the other anyway. So the chances are that this will not result in any
> > more FP traps than before. There has to be a FP trap per thread switch
> > anyway, and changing the interrupt enable should be a lot less common
> > than that.
> >
> >
> >
> > --
> > Nick Garnett
> > Cygnus Solutions, a Red Hat Company
> > Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Nick Garnett]

Sergei Organov <osv@javad.ru> writes:

> Nick,
> 
> I was very busy last two weeks, but now when I've got a chance to return to
> the FP implementation, I've found another related problem.
> 
> The default interrupt handler calls 'hal_cpu_int_disable' close to the end of
> the ISR processing. If I disable FP there, then *every* interrupt will result
> in disabling of FP, that is obviously unacceptable. 
> 
> Yet another similar place is in the
> 'hal_interrupt_stack_call_pending_DSRs' that restores interrupt settings
> before return.
> 
> Do you see a way to handle above cases?


I haven't thought a lot of this through, but here are some (random)
thoughts:

There is no reason why the asm level macros should behave the same way
as the user level ones do. These are all used in very specialized
places where it may not matter. Looking at the code,apart from the
merge in context.S, these are only used in the places you mention in
vectors.S.

The disable in restore_state only actually disables interrupts when
returning from an exception. When returning from an interrupt, they
are already disabled at this point. Perhaps the disable should be
taken out of here and put into a path that only the exception code
will go through.

Perhaps the macros should only clear the FP enable if they actually
change the interrupt state.

In the DSR code, we know that any interrupt that occurs will not cause
a thread switch, since the scheduler lock is non-zero. Hence the FPU
state will not have changed. So I don't think we have to clear the FP
enable at all through this code.


-- 
Nick Garnett
Cygnus Solutions, a Red Hat Company
Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Sergei Organov]

I agree with your (random) thoughts. However for me it seems that on-demand
context switch is incompatible with ability to use of the floating point in
ISRs/DSRs. I actually even didn't start to try to implement this capability
when on-demand FP context switch is configured, but it seems that if
implemented, then *any* interrupt could clear FP bit in MSR, and thus the fact 
that scheduler is locked in the DSR code doesn't help :-(

Is the capability to use FP in ISRs/DSRs significant enough to deny on-demand
context switch implementation on the PowerPC?

Nick Garnett <nickg@cygnus.co.uk> writes:
> Sergei Organov <osv@javad.ru> writes:
> 
> > Nick,
> > 
> > I was very busy last two weeks, but now when I've got a chance to return to
> > the FP implementation, I've found another related problem.
> > 
> > The default interrupt handler calls 'hal_cpu_int_disable' close to the end of
> > the ISR processing. If I disable FP there, then *every* interrupt will result
> > in disabling of FP, that is obviously unacceptable. 
> > 
> > Yet another similar place is in the
> > 'hal_interrupt_stack_call_pending_DSRs' that restores interrupt settings
> > before return.
> > 
> > Do you see a way to handle above cases?
> 
> 
> I haven't thought a lot of this through, but here are some (random)
> thoughts:
> 
> There is no reason why the asm level macros should behave the same way
> as the user level ones do. These are all used in very specialized
> places where it may not matter. Looking at the code,apart from the
> merge in context.S, these are only used in the places you mention in
> vectors.S.
> 
> The disable in restore_state only actually disables interrupts when
> returning from an exception. When returning from an interrupt, they
> are already disabled at this point. Perhaps the disable should be
> taken out of here and put into a path that only the exception code
> will go through.
> 
> Perhaps the macros should only clear the FP enable if they actually
> change the interrupt state.
> 
> In the DSR code, we know that any interrupt that occurs will not cause
> a thread switch, since the scheduler lock is non-zero. Hence the FPU
> state will not have changed. So I don't think we have to clear the FP
> enable at all through this code.
> 
> 
> -- 
> Nick Garnett
> Cygnus Solutions, a Red Hat Company
> Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Nick Garnett]

Sergei Organov <osv@javad.ru> writes:

> I agree with your (random) thoughts. However for me it seems that on-demand
> context switch is incompatible with ability to use of the floating point in
> ISRs/DSRs. I actually even didn't start to try to implement this capability
> when on-demand FP context switch is configured, but it seems that if
> implemented, then *any* interrupt could clear FP bit in MSR, and thus the fact 
> that scheduler is locked in the DSR code doesn't help :-(
>

The management of the memory for storing nested FP contexts is a
little involved but not impossible. The main problem is the management
of the FP enable bit. I think that the general approach here would be
to disable the FPU on interrupt entry, take FPU traps as normal during
interrupt processing and on interrupt return decide whether the FPU
had been used, and if not, clear the FP bit. The interrupt's FPU
context at this point is redundant, so it can just be overwritten.

> Is the capability to use FP in ISRs/DSRs significant enough to deny on-demand
> context switch implementation on the PowerPC?

I think we can expect writers of ISRs and DSRs to be reasonably
disciplined and to avoid any FP code. So banning FP use in these
contexts is not a major problem. The main area of worry is in the
alarm handler routines. At present these are called from a DSR and may
contain any user code. We can impose the same restrictions on these
functions as on DSRs, but it would be nice to lift it one day. One
approach to this is to move alarm handling to a proper thread
context, but this is still in the future.


-- 
Nick Garnett
Cygnus Solutions, a Red Hat Company
Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Sergei Organov]

Nick Garnett <nickg@cygnus.co.uk> writes:
> Sergei Organov <osv@javad.ru> writes:
>
> The management of the memory for storing nested FP contexts is a
> little involved but not impossible. The main problem is the management
> of the FP enable bit. I think that the general approach here would be
> to disable the FPU on interrupt entry, take FPU traps as normal during
> interrupt processing and on interrupt return decide whether the FPU
> had been used, and if not, clear the FP bit. The interrupt's FPU
> context at this point is redundant, so it can just be overwritten.

If FPU hadn't been used, then FP bit is still cleared on interrupt return, so
there is no reason to clear it again. Did you mean that FP bit should be
cleared on exit from ISR if FPU had been used? What "interrupt entry" and
"interrupt return" mean exactly, - before/after call to particular ISR, or
begin/end of the default VSR handler?

I must say that I still don't see how to handle FP bit if FPU usage in
ISRs/DSRs is enabled. While situation with HAL_DISABLE_INTERRUPTS is now
resolved (just clear FP bit), the following places have troubles:

1. hal_cpu_int_merge at the end of hal_thread_switch_context.
2. hal_cpu_int_merge at the end of hal_interrupt_stack_call_pending_DSRs.
3. hal_cpu_int_disable at the end of cyg_hal_default_interrupt_vsr.
4. a few places where FP bit is changed (added to support on-demand FP).

If FP usage in ISRs/DSRs is disabled, all these places don't have troubles
because scheduler is locked there, and thus interrupts will not change the FP
bit. If FP usage in ISRs/DSRs is enabled, then possibility that interrupt will
clear FP bit exists, and we can't safely modify MSR even if scheduler is
locked. We also can't clear FP bit there because this will result in a lot of
"FP unavailable" traps.  While these traps will not actually switch context
most of time, they still have overhead.

As you wrote in previous mail, in the case (3) interrupts are already disabled
when we return from interrupt, so there is no reason to disable them again. I
guess that's because they are merged in (1). But what if interrupts are
re-enabled inside the ISR? Or is it prohibited to enable interrupts in ISR
handlers?

Anyway, (1), (2), and (4) still have troubles, I think.

> > Is the capability to use FP in ISRs/DSRs significant enough to deny on-demand
> > context switch implementation on the PowerPC?
>
> I think we can expect writers of ISRs and DSRs to be reasonably
> disciplined and to avoid any FP code. So banning FP use in these
> contexts is not a major problem. The main area of worry is in the
> alarm handler routines. At present these are called from a DSR and may
> contain any user code. We can impose the same restrictions on these
> functions as on DSRs, but it would be nice to lift it one day. One
> approach to this is to move alarm handling to a proper thread
> context, but this is still in the future.

Possibility to configure eCos to use simple variant of FP context switch still
remains. If application requires FPU even in ISRs/DSRs/alarm handlers, then
it's likely that most of tasks require FPU as well, so there is no reason to
use on-demand context switch implementation that has some overhead anyway.

>
>
> --
> Nick Garnett
> Cygnus Solutions, a Red Hat Company
> Cambridge, UK

BR,
Sergei Organov.

Re: [ECOS] On demand FP context switch major problem.

[Nick Garnett]

Sergei Organov <osv@javad.ru> writes:

> Nick Garnett <nickg@cygnus.co.uk> writes:
> > Sergei Organov <osv@javad.ru> writes:
> >
> > The management of the memory for storing nested FP contexts is a
> > little involved but not impossible. The main problem is the management
> > of the FP enable bit. I think that the general approach here would be
> > to disable the FPU on interrupt entry, take FPU traps as normal during
> > interrupt processing and on interrupt return decide whether the FPU
> > had been used, and if not, clear the FP bit. The interrupt's FPU
> > context at this point is redundant, so it can just be overwritten.
> 
> If FPU hadn't been used, then FP bit is still cleared on interrupt return, so
> there is no reason to clear it again. Did you mean that FP bit should be
> cleared on exit from ISR if FPU had been used?

Actually, what I think I meant was that if the interrupt had not used
the FPU, but the thread was using it, then we should re-set (not
clear) the FP enable bit on exit. If the interrupt has not used the
FPU then is still contains the thread's FP state, so setting the bit
will avoid the subsequent FP trap. Obviously some care needs to be
taken not to set the bit spuriously if interrupt_end() had caused a
thread switch.

> What "interrupt entry" and
> "interrupt return" mean exactly, - before/after call to particular ISR, or
> begin/end of the default VSR handler?

I meant the VSR handler code.

> 
> I must say that I still don't see how to handle FP bit if FPU usage in
> ISRs/DSRs is enabled. While situation with HAL_DISABLE_INTERRUPTS is now
> resolved (just clear FP bit), the following places have troubles:
> 
> 1. hal_cpu_int_merge at the end of hal_thread_switch_context.
> 2. hal_cpu_int_merge at the end of hal_interrupt_stack_call_pending_DSRs.
> 3. hal_cpu_int_disable at the end of cyg_hal_default_interrupt_vsr.
> 4. a few places where FP bit is changed (added to support on-demand FP).
> 
> If FP usage in ISRs/DSRs is disabled, all these places don't have troubles
> because scheduler is locked there, and thus interrupts will not change the FP
> bit. If FP usage in ISRs/DSRs is enabled, then possibility that interrupt will
> clear FP bit exists, and we can't safely modify MSR even if scheduler is
> locked. We also can't clear FP bit there because this will result in a lot of
> "FP unavailable" traps.  While these traps will not actually switch context
> most of time, they still have overhead.
> 
> As you wrote in previous mail, in the case (3) interrupts are already disabled
> when we return from interrupt, so there is no reason to disable them again. I
> guess that's because they are merged in (1). But what if interrupts are
> re-enabled inside the ISR? Or is it prohibited to enable interrupts in ISR
> handlers?
>

An ISR is allowed to enable its own interrupt, either in
the hardware it controls, or in the interrupt controller, but
shouldn't enable interrupts globally.

> Anyway, (1), (2), and (4) still have troubles, I think.
> 
> > > Is the capability to use FP in ISRs/DSRs significant enough to deny on-demand
> > > context switch implementation on the PowerPC?
> >
> > I think we can expect writers of ISRs and DSRs to be reasonably
> > disciplined and to avoid any FP code. So banning FP use in these
> > contexts is not a major problem. The main area of worry is in the
> > alarm handler routines. At present these are called from a DSR and may
> > contain any user code. We can impose the same restrictions on these
> > functions as on DSRs, but it would be nice to lift it one day. One
> > approach to this is to move alarm handling to a proper thread
> > context, but this is still in the future.
> 
> Possibility to configure eCos to use simple variant of FP context switch still
> remains. If application requires FPU even in ISRs/DSRs/alarm handlers, then
> it's likely that most of tasks require FPU as well, so there is no reason to
> use on-demand context switch implementation that has some overhead anyway.

I suspect that this is the best approach to this problem. At least for
the present, or until someone works out how to do it properly. I would
much rather not provide a facility than supply a badly thought-out or
poorly implemented version.

-- 
Nick Garnett
Cygnus Solutions, a Red Hat Company
Cambridge, UK

Re: [ECOS] On demand FP context switch major problem.

[Sergei Organov]

Nick Garnett <nickg@cygnus.co.uk> writes:
> Sergei Organov <osv@javad.ru> writes:
[...]
> > 
> > Possibility to configure eCos to use simple variant of FP context switch still
> > remains. If application requires FPU even in ISRs/DSRs/alarm handlers, then
> > it's likely that most of tasks require FPU as well, so there is no reason to
> > use on-demand context switch implementation that has some overhead anyway.
> 
> I suspect that this is the best approach to this problem. At least for
> the present, or until someone works out how to do it properly. I would
> much rather not provide a facility than supply a badly thought-out or
> poorly implemented version.

Exactly. So I'll focus on implementing on-demand without capability to use FP
in ISRs/DSRs, and leave capability to use FP in ISRs/DSRs only when simple FP
context switch policy is configured.

> 
> -- 
> Nick Garnett
> Cygnus Solutions, a Red Hat Company
> Cambridge, UK

BR,
Sergei.

Re: [ECOS] On demand FP context switch major problem.

[Nick Garnett]

Sergei Organov <osv@javad.ru> writes:

> Hello,
> 
> I've almost finished the "on demand" FP context switch support in the PowerPC
> HAL, but at the end faced with a problem for which I don't see a reasonable
> solution. Here is the description of the problem.
> 
> To switch FP context on demand we need to disable FP in the usual context
> switch routine by means of clearing the "FP enable" bit in the MSR. It means
> that at thread level of execution the FP bit in the MSR could be cleared any
> time if scheduler isn't locked and/or interrupts aren't disabled. It in turn
> means that in thread code we can safely change MSR only if above condition(s)
> is(are) met. But the EE (external interrupt enable) bit is also located in the
> MSR, so it's unsafe to disable interrupts if scheduler isn't locked!
> 
> The above has unfortunate consequences for the code in the
> HAL_DISABLE_INTERRUPTS macro. With "on demand" FP context switch is used, this
> macro should somehow lock the scheduler before reading the MSR and then unlock
> it back after writing the MSR :-(
> 
> The above discussion didn't take into consideration support for the use of FPU
> in the interrupt handlers (ISRs/DSRs). If this support is added, then even
> locking of the scheduler doesn't help. In this case only disabling of
> interrupts gives us a safe way to change MSR, but once again disabling of
> interrupts needs changing of MSR :-(
> 
> Any suggestions? Hopefully I'm just missing something obvious.


This is one of the annoying features of the PowerPC, and other RISC
processors, that "global" state and "per-thread" state are mixed up in
the same register.

One option would be to always clear the FP enable bit in
HAL_DISABLE_INTERRUPTS (probably HAL_ENABLE_INTERRUPTS and
HAL_RESTORE_INTERRUPTS too). This would make your described scenario
work correctly since you would get an FP exception at step 7. In the
simple case you would also get an FP trap on the next FP instruction,
which would see that the current thread is already the FPU owner and
just set the FP enable bit.

Taking the extra trap is unpleasant, but with a suitable fast path it
should not be too expensive. It is unlikely that code which plays with
the interrupt enable is also doing a lot of floating point, and the
thread may well be timesliced or preempted before moving from one to
the other anyway. So the chances are that this will not result in any
more FP traps than before. There has to be a FP trap per thread switch
anyway, and changing the interrupt enable should be a lot less common
than that.



-- 
Nick Garnett
Cygnus Solutions, a Red Hat Company
Cambridge, UK