[ECOS] Hardware Watchdog

[ECOS] Hardware Watchdog

[Christoph Csebits]

hi

im having a MPC860T based board with
an hardware watchdog on it. It must be
triggered within a second, otherwise
the MPC will be reset.

In eCos applications i am having a
separate thread triggering the watchdog.
No problem.

How should i deal with it in RedBoot:

Currently i scattered several of 
TOGGLE_WATCHDOG macros into the
RedBoot code, but this should only
be a preliminarily solution.

What do you think about using some
of the timer interrupts for generating
the watchdog triggers.

Thanks for advice, and how to start.

best regards, christoph
-- 

Re: [ECOS] Hardware Watchdog

[fche]

Content-Type: text/plain; charset=US-ASCII
From: fche@redhat.com (Frank Ch. Eigler)
Date: 26 Jul 2001 07:56:53 -0400
In-Reply-To: < 20010726125313.A14929@frequentis.com >
Message-ID: <o5ofq7ucmy.fsf@touchme.toronto.redhat.com>
Lines: 23
User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.1 (Cuyahoga Valley)
MIME-Version: 1.0

Christoph Csebits <christoph.csebits@frequentis.com> writes:

: [...]
: im having a MPC860T based board with
: an hardware watchdog on it.
: [...]
: In eCos applications i am having a
: separate thread triggering the watchdog.
: No problem.
: [...]
: What do you think about using some
: of the timer interrupts for generating
: the watchdog triggers.

Is it typical to put watchdog reset calls into separate dedicated
threads, or low-level automatically-invoked code like interrupt
handlers?  I was under the impression that watchdog resets are most
appropriately placed within the activity loops of the application code
itself -- it is their demise that the watchdog mechanism is meant to
detect.


- FChE

Re: [ECOS] Hardware Watchdog

[Robin Farine]

Christoph Csebits <christoph.csebits@frequentis.com> writes:

> On Fri, Jul 27, 2001 at 08:48:14AM +0200, Robin Farine wrote:
> > Ok, have you already tried to configure the GPIO pin as *input*? This would
> > set the line in high-impedance mode and the MAX should disable its watchdog
> > function. Check your ROM startup code for access to the GPDR (IO direction
> > register). Then, in the application, reconfigure this bit as output to use the
> > watchdog.
> 
> basically its right, but the hardware designers
> made a good job of it, and there is a pull down
> resistor preventing the watchdog input from floating.
> (which would disable the watchdog timer)

That's exactly what I suspected :-(

> 
> i dont think the soldering iron is an acceptable solution. :-)
> 

Why not? I would call this a design flaw and report it to the manufacturer. The
fact that a place for a pull down resistor exists is a good thing (we have seen
some chips with specified internal pull down/up resistors but in the reality,
those pins were floating so we now always put some pads just in case), but in
this case the resistor should not be fitted.

Robin

Re: [ECOS] Hardware Watchdog

[Christoph Csebits]

On Fri, Jul 27, 2001 at 08:48:14AM +0200, Robin Farine wrote:
> Ok, have you already tried to configure the GPIO pin as *input*? This would
> set the line in high-impedance mode and the MAX should disable its watchdog
> function. Check your ROM startup code for access to the GPDR (IO direction
> register). Then, in the application, reconfigure this bit as output to use the
> watchdog.

basically its right, but the hardware designers
made a good job of it, and there is a pull down
resistor preventing the watchdog input from floating.
(which would disable the watchdog timer)

i dont think the soldering iron is an acceptable
solution. :-)

thanks, christoph
- 

Re: [ECOS] Hardware Watchdog

[Robin Farine]

Christoph Csebits <christoph.csebits@frequentis.com> writes:

> On Thu, Jul 26, 2001 at 08:24:59PM +0200, Robin Farine wrote:
> > Christoph Csebits <christoph.csebits@frequentis.com> writes:
> > 
> > Does your board use an external chip such as the Maxim MAX691A as watchdog? And
> > if yes, does it use a GPIO pin of the CPU to keep the watchdog quiet?
> 
> Yes you are right (MAX690T) and it is served an a GPIO pin (PortB) of
> the MPC.

Ok, have you already tried to configure the GPIO pin as *input*? This would
set the line in high-impedance mode and the MAX should disable its watchdog
function. Check your ROM startup code for access to the GPDR (IO direction
register). Then, in the application, reconfigure this bit as output to use the
watchdog.

Robin

Re: [ECOS] Hardware Watchdog

[Christoph Csebits]

On Thu, Jul 26, 2001 at 08:24:59PM +0200, Robin Farine wrote:
> Christoph Csebits <christoph.csebits@frequentis.com> writes:
> 
> Does your board use an external chip such as the Maxim MAX691A as watchdog? And
> if yes, does it use a GPIO pin of the CPU to keep the watchdog quiet?

Yes you are right (MAX690T) and it is served an a GPIO pin (PortB) of
the MPC.
-- 
___________________________________
DI Christoph Csebits, System Design
FREQUENTIS   -   www.frequentis.com
A-1120 Wien, Spittelbreitengasse 34
voice:  +43/1/81150-2149, fax: 2499

RE: [ECOS] Hardware Watchdog

[Doug Fraser]

Very possibly you are correct, but I do
not know the gdb stubs at that level.
To honk the thing, you just write a register.
The problem is, if you are late even one
clock tick, the processor resets, and there
is absolutely nothing you can do about that.

That is why I propose finding all the normal
surface loops, then just plain shutting the
watchdog down if GDB starts.

Doug

> 
> So the idea is that you need some piece of code telling
> the watchdog "I'm alive", correct?  Surely this can be
> done without all of the baggage of handling interrupts.

Yup, as long as you cover every single loop location.


> I don't see how you can handle interrupts without enabling then.
> Once they are enabled, then other interrupt conditions might need
> to be serviced, etc.

Install NULL handlers....


> 
> I think it much better/wiser to figure out where it might be "looping"
> and put in stuff to tickle the watchdog appropriately.
> 

Re: [ECOS] Hardware Watchdog

[Robin Farine]

Christoph Csebits <christoph.csebits@frequentis.com> writes:

Does your board use an external chip such as the Maxim MAX691A as watchdog? And
if yes, does it use a GPIO pin of the CPU to keep the watchdog quiet?

Robin

RE: [ECOS] Hardware Watchdog

[Gary Thomas]

On 26-Jul-2001 Doug Fraser wrote:
> Interrupts may not be enabled in redboot, but
> does that stop him from installing his own
> tick handler and servicing the watchdog from there?
> The maximum time on the 860 watchdog is around
> three seconds with a 50Mhz CPU clock.
> As Christoph mentioned, the watchdog activates
> a hard reset cycle, and there is nothing you can
> do to catch it. The other problem is that if you
> disable the watchdog, you cannot reenable it
> without performing a hard reset.
> 

So the idea is that you need some piece of code telling
the watchdog "I'm alive", correct?  Surely this can be
done without all of the baggage of handling interrupts.

> As well as servicing it with a tick interrupt
> during redboot, you could also disable it if
> the redboot polling mechanism detects a GDB
> connect. You probably want the watchdog off
> during debugging anyway, so that would allow the
> target to boot with a watchdog in a normal run
> state, yet reduce the complexity of having to
> service it during a debug session.
> 

I don't see how you can handle interrupts without enabling then.
Once they are enabled, then other interrupt conditions might need
to be serviced, etc.

I think it much better/wiser to figure out where it might be "looping"
and put in stuff to tickle the watchdog appropriately.

RE: [ECOS] Hardware Watchdog

[Doug Fraser]

Interrupts may not be enabled in redboot, but
does that stop him from installing his own
tick handler and servicing the watchdog from there?
The maximum time on the 860 watchdog is around
three seconds with a 50Mhz CPU clock.
As Christoph mentioned, the watchdog activates
a hard reset cycle, and there is nothing you can
do to catch it. The other problem is that if you
disable the watchdog, you cannot reenable it
without performing a hard reset.

As well as servicing it with a tick interrupt
during redboot, you could also disable it if
the redboot polling mechanism detects a GDB
connect. You probably want the watchdog off
during debugging anyway, so that would allow the
target to boot with a watchdog in a normal run
state, yet reduce the complexity of having to
service it during a debug session.

good luck...

Doug

> -----Original Message-----
> From: Gary Thomas [ mailto:gthomas@redhat.com ]
> Sent: Thursday, July 26, 2001 12:24 PM
> To: Christoph Csebits
> Cc: Doug Fraser; fche@redhat.com; ecos-discuss@sources.redhat.com
> Subject: Re: [ECOS] Hardware Watchdog
> 
> 
> 
> On 26-Jul-2001 Christoph Csebits wrote:
> > On Thu, Jul 26, 2001 at 09:49:29AM -0600, Gary Thomas wrote:
> >> 
> >> On 26-Jul-2001 Christoph Csebits wrote:
> >> > Nevertheless my watchdog is not
> >> > software controlled. :-|
> >> > 
> >> 
> >> What happens when your watchdog fires?   
> > 
> > the MPCs reset lines are pulled accordingly
> > to do an hard reset. no way to catch it. :-)
> > 
> >> Fixing this may be hard.  Within RedBoot, there is a 
> notion of "idle"
> >> routines which will be called while RedBoot waits.  
> However, once the
> >> GDB stubs start running, all of that stops and everything is simply
> >> polled based on the serial port.
> > 
> > I have a running version of RedBoot, but as i told, from
> > time to time i am running into a loop, and the board resets.
> > Then i start to locate the loop and fix it.
> > 
> > The best solution may be a timer interrupt routine
> > with detaching the interrupt when starting an application.
> > (Someone mentioned it earlier, thanks to him
> > - i lost the mail and do not remember the name)
> > 
> > Is it the right way to do it with
> > cyg_drv_interrupt_create( );
> > cyg_drv_interrupt_attach( );
> > cyg_drv_interrupt_acknowledge( );
> > cyg_drv_interrupt_unmask( );
> >:
> 
> Sorry, interrupts are not enabled in RedBoot, so this just 
> isn't going to
> cut it.
> 
> What is the period on the watchdog?
> Do you have any ideas what's happening (in RedBoot or 
> otherwise) then it fails?
> 

Re: [ECOS] Hardware Watchdog

[Gary Thomas]

On 26-Jul-2001 Christoph Csebits wrote:
> On Thu, Jul 26, 2001 at 09:49:29AM -0600, Gary Thomas wrote:
>> 
>> On 26-Jul-2001 Christoph Csebits wrote:
>> > Nevertheless my watchdog is not
>> > software controlled. :-|
>> > 
>> 
>> What happens when your watchdog fires?   
> 
> the MPCs reset lines are pulled accordingly
> to do an hard reset. no way to catch it. :-)
> 
>> Fixing this may be hard.  Within RedBoot, there is a notion of "idle"
>> routines which will be called while RedBoot waits.  However, once the
>> GDB stubs start running, all of that stops and everything is simply
>> polled based on the serial port.
> 
> I have a running version of RedBoot, but as i told, from
> time to time i am running into a loop, and the board resets.
> Then i start to locate the loop and fix it.
> 
> The best solution may be a timer interrupt routine
> with detaching the interrupt when starting an application.
> (Someone mentioned it earlier, thanks to him
> - i lost the mail and do not remember the name)
> 
> Is it the right way to do it with
> cyg_drv_interrupt_create( );
> cyg_drv_interrupt_attach( );
> cyg_drv_interrupt_acknowledge( );
> cyg_drv_interrupt_unmask( );
>:

Sorry, interrupts are not enabled in RedBoot, so this just isn't going to
cut it.

What is the period on the watchdog?
Do you have any ideas what's happening (in RedBoot or otherwise) then it fails?

Re: [ECOS] Hardware Watchdog

[Christoph Csebits]

On Thu, Jul 26, 2001 at 09:49:29AM -0600, Gary Thomas wrote:
> 
> On 26-Jul-2001 Christoph Csebits wrote:
> > Nevertheless my watchdog is not
> > software controlled. :-|
> > 
> 
> What happens when your watchdog fires?   

the MPCs reset lines are pulled accordingly
to do an hard reset. no way to catch it. :-)

> Fixing this may be hard.  Within RedBoot, there is a notion of "idle"
> routines which will be called while RedBoot waits.  However, once the
> GDB stubs start running, all of that stops and everything is simply
> polled based on the serial port.

I have a running version of RedBoot, but as i told, from
time to time i am running into a loop, and the board resets.
Then i start to locate the loop and fix it.

The best solution may be a timer interrupt routine
with detaching the interrupt when starting an application.
(Someone mentioned it earlier, thanks to him
- i lost the mail and do not remember the name)

Is it the right way to do it with
cyg_drv_interrupt_create( );
cyg_drv_interrupt_attach( );
cyg_drv_interrupt_acknowledge( );
cyg_drv_interrupt_unmask( );
:

Re: [ECOS] Hardware Watchdog

[Gary Thomas]

On 26-Jul-2001 Christoph Csebits wrote:
> Nevertheless my watchdog is not
> software controlled. :-|
> 

What happens when your watchdog fires?   
Is there an exception that you can catch?
What is the watchdog time [limit]?  Can you adjust it?

Fixing this may be hard.  Within RedBoot, there is a notion of "idle"
routines which will be called while RedBoot waits.  However, once the
GDB stubs start running, all of that stops and everything is simply
polled based on the serial port.

Re: [ECOS] Hardware Watchdog

[Christoph Csebits]

On Thu, Jul 26, 2001 at 10:24:08AM -0400, Doug Fraser wrote:
> [......]
> 
> Doug

Thanks for your statement, I basically
agree with you. 

But my really problem is porting
RedBoot to my platform.

RedBoot doesnt support scheduling,
its a single task, and finding
every single little loop that could
last longer as my watchdog period
is really annoying, and time consuming.

And you never know, if you missed one.

The best solution would be a switchable
watchdog. During the Bootstrapper is
running the watchdog is disabled.
Starting the real application goes
hand in hand with enabling the watchdog
so the application can handle it in
a case dependend manner.

Nevertheless my watchdog is not
software controlled. :-|

comments would be appreciated
Is nobody running RedBoot with such a device?

regards, christoph
-- 

RE: [ECOS] Hardware Watchdog

[Doug Fraser]

There are several opinions on this,
so don't be surprised by disagreements.

With a multi tasking OS, one could
take two task based views.

One: Stick it at the highest priority level
(thread, process, whatever....) and the
watchdog fires if the kernel crashes
or stops scheduling for any reason.

Two: Stick it at the lowest priority level
and the watchdog will fire in any starvation
situation.

There are good and bad points for either,
and I won't argue them here. (the list
is arbitrary and HUGE!)

In either case, a still running system can
make other decisions about its own health
and kill or block the watchdog thread,
forcing a reset. You might even consider
adding a rendezvous point to the watchdog thread.

On acknowledging the watchdog from an interrupt...

Depending on you interrupt structure, you
should not associate it with an interrupt.
The reason being that on many systems, the
interrupts will continue to fire even if
the kernel is dead or scheduling has ceased.

That would be bad if your periodic timer
interrupt was hammering away happily on your
otherwise dead system, keeping the much
needed watchdog at bay.

Doug

> -----Original Message-----
> From: fche@redhat.com [ mailto:fche@redhat.com ]
> Sent: Thursday, July 26, 2001 7:57 AM
> To: ecos-discuss@sources.redhat.com
> Subject: Re: [ECOS] Hardware Watchdog
> 
> 
> Content-Type: text/plain; charset=US-ASCII
> From: fche@redhat.com (Frank Ch. Eigler)
> Date: 26 Jul 2001 07:56:53 -0400
> In-Reply-To: < 20010726125313.A14929@frequentis.com >
> Message-ID: <o5ofq7ucmy.fsf@touchme.toronto.redhat.com>
> Lines: 23
> User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.1 (Cuyahoga Valley)
> MIME-Version: 1.0
> 
> Christoph Csebits <christoph.csebits@frequentis.com> writes:
> 
> : [...]
> : im having a MPC860T based board with
> : an hardware watchdog on it.
> : [...]
> : In eCos applications i am having a
> : separate thread triggering the watchdog.
> : No problem.
> : [...]
> : What do you think about using some
> : of the timer interrupts for generating
> : the watchdog triggers.
> 
> Is it typical to put watchdog reset calls into separate dedicated
> threads, or low-level automatically-invoked code like interrupt
> handlers?  I was under the impression that watchdog resets are most
> appropriately placed within the activity loops of the application code
> itself -- it is their demise that the watchdog mechanism is meant to
> detect.
> 
> 
> - FChE
>