[ECOS] FW: Mount JFFS2 root file system issue

[ECOS] FW: Mount JFFS2 root file system issue

[Lin, Bin (Bin)]

Hello All,

I have ARM9 systems with JFFS2 root file system. For some reasons, e.g.
system crash or power failure, when system boots up next time, it takes
much longer time than usual to mount jffs2 root file system. It can hang
at following message for 3-10 min. The system will eventually come up
and function well. This problem occurred on less than 10% of boards.

NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
Fast Floating Point Emulator V0.9 (c) Peter Teichmann.

There are some symbolic links in the root file system. These links is
linked to the files in another partition. Could that cause problem if
the file in the other partition gets corrupted? This behaviour is not
reproducible. Looks like the files in the flash are not corrupted. I
guess the corruption could be in jffs2 file system.

Anyone has seen this issue, and solution?

Appreciate very much for any input,

Ben

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] FW: Mount JFFS2 root file system issue

[Gary Thomas]

Lin, Bin (Bin) wrote:
> Hello All,
> 
> I have ARM9 systems with JFFS2 root file system. For some reasons, e.g.
> system crash or power failure, when system boots up next time, it takes
> much longer time than usual to mount jffs2 root file system. It can hang
> at following message for 3-10 min. The system will eventually come up
> and function well. This problem occurred on less than 10% of boards.

I think this can happen (after such a failure) when JFFS2
realizes that it needs to rebuild/recover the active
nodes in the file system.  This can require a scan of
the entire file system, along with rewriting some nodes, etc.

> NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> Fast Floating Point Emulator V0.9 (c) Peter Teichmann.
> 
> There are some symbolic links in the root file system. These links is
> linked to the files in another partition. Could that cause problem if
> the file in the other partition gets corrupted? This behaviour is not
> reproducible. Looks like the files in the flash are not corrupted. I
> guess the corruption could be in jffs2 file system.
> 
> Anyone has seen this issue, and solution?
> 
> Appreciate very much for any input,

This looks to be a Linux kernel issue - I don't see what it
has to do with eCos.

You should probably ask on some appropriate Linux mailing list.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] FW: Mount JFFS2 root file system issue

[Lin, Bin (Bin)]

Hi Gary,

Thanks a lot for your reply. 

It is understandable that JFFS2 needs to rebuild/recover the active
nodes in the next reboot after failure.  But once this happen, every
reboot later on will take the same amount of time which is much long
than normal. I have to re-program the entire flash to get it back to
normal mount time. 

Also this problem can not reproduced in the lab. Is there a way to
craete this problem quickly?

Thanks again,
Ben


-----Original Message-----
From: Gary Thomas [mailto:gary@mlbassoc.com] 
Sent: Friday, January 11, 2008 7:40 AM
To: Lin, Bin (Bin)
Cc: ecos-discuss@sourceware.org
Subject: Re: [ECOS] FW: Mount JFFS2 root file system issue

Lin, Bin (Bin) wrote:
> Hello All,
> 
> I have ARM9 systems with JFFS2 root file system. For some reasons,
e.g.
> system crash or power failure, when system boots up next time, it 
> takes much longer time than usual to mount jffs2 root file system. It 
> can hang at following message for 3-10 min. The system will eventually

> come up and function well. This problem occurred on less than 10% of
boards.

I think this can happen (after such a failure) when JFFS2 realizes that
it needs to rebuild/recover the active nodes in the file system.  This
can require a scan of the entire file system, along with rewriting some
nodes, etc.

> NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> Fast Floating Point Emulator V0.9 (c) Peter Teichmann.
> 
> There are some symbolic links in the root file system. These links is 
> linked to the files in another partition. Could that cause problem if 
> the file in the other partition gets corrupted? This behaviour is not 
> reproducible. Looks like the files in the flash are not corrupted. I 
> guess the corruption could be in jffs2 file system.
> 
> Anyone has seen this issue, and solution?
> 
> Appreciate very much for any input,

This looks to be a Linux kernel issue - I don't see what it has to do
with eCos.

You should probably ask on some appropriate Linux mailing list.

--
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] FW: Mount JFFS2 root file system issue

[Gary Thomas]

Lin, Bin (Bin) wrote:
> Hi Gary,
> 
> Thanks a lot for your reply. 
> 
> It is understandable that JFFS2 needs to rebuild/recover the active
> nodes in the next reboot after failure.  But once this happen, every
> reboot later on will take the same amount of time which is much long
> than normal. I have to re-program the entire flash to get it back to
> normal mount time. 
> 
> Also this problem can not reproduced in the lab. Is there a way to
> craete this problem quickly?

No clue, sorry.

As I said, since this isn't an eCos issue, you should ask the JFFS2
folks (linux-mtd@lists.infradead.org) or Linux mailing lists.

> -----Original Message-----
> From: Gary Thomas [mailto:gary@mlbassoc.com] 
> Sent: Friday, January 11, 2008 7:40 AM
> To: Lin, Bin (Bin)
> Cc: ecos-discuss@sourceware.org
> Subject: Re: [ECOS] FW: Mount JFFS2 root file system issue
> 
> Lin, Bin (Bin) wrote:
>> Hello All,
>>
>> I have ARM9 systems with JFFS2 root file system. For some reasons,
> e.g.
>> system crash or power failure, when system boots up next time, it 
>> takes much longer time than usual to mount jffs2 root file system. It 
>> can hang at following message for 3-10 min. The system will eventually
> 
>> come up and function well. This problem occurred on less than 10% of
> boards.
> 
> I think this can happen (after such a failure) when JFFS2 realizes that
> it needs to rebuild/recover the active nodes in the file system.  This
> can require a scan of the entire file system, along with rewriting some
> nodes, etc.
> 
>> NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
>> Fast Floating Point Emulator V0.9 (c) Peter Teichmann.
>>
>> There are some symbolic links in the root file system. These links is 
>> linked to the files in another partition. Could that cause problem if 
>> the file in the other partition gets corrupted? This behaviour is not 
>> reproducible. Looks like the files in the flash are not corrupted. I 
>> guess the corruption could be in jffs2 file system.
>>
>> Anyone has seen this issue, and solution?
>>
>> Appreciate very much for any input,
> 
> This looks to be a Linux kernel issue - I don't see what it has to do
> with eCos.
> 
> You should probably ask on some appropriate Linux mailing list.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] FW: Mount JFFS2 root file system issue

[Jürgen Lambrecht]

Hello Bin,

I am also struggling with JFFS2, and I did already a lot of tests.
There more files you have in jffs2, the longer it takes to mount:
- per inode, 16 B (static) or 24 B (dynamic) are written in RAM
- per file, more or less 72 B (found by testing, I should check the 
jffs2 sources..) are written in RAM
Also, during mounting, a garbage collect is done. As explained below, 
the file system (the journal) is then checked. And the bigger the flash, 
the more time it can take.

Indeed, once the boot time is long, you can only solve it by formatting 
jffs2.

I can reproduce this this way: I put 1000 files in 1 directory, then I 
delete 997 of them, and still mounting and also opening of 1 file takes 
very long. To go more in detail: mounting goes from 11s to 10s, but 
opening 1 file goes from 570ms to 820ms.

Regards,

Jürgen Lambrecht
R&D Engineer
Televic Transport Systems
http://www.televic.com
Televic NV / SA (main office)  	
Leo Bekaertlaan 1
B-8870 Izegem
Tel: +32 (0)51 303045
Fax: +32 (0)51 310670



Gary Thomas wrote:

> Lin, Bin (Bin) wrote:
>
>> Hi Gary,
>>
>> Thanks a lot for your reply.
>> It is understandable that JFFS2 needs to rebuild/recover the active
>> nodes in the next reboot after failure.  But once this happen, every
>> reboot later on will take the same amount of time which is much long
>> than normal. I have to re-program the entire flash to get it back to
>> normal mount time.
>> Also this problem can not reproduced in the lab. Is there a way to
>> craete this problem quickly?
>
>
> No clue, sorry.
>
> As I said, since this isn't an eCos issue, you should ask the JFFS2
> folks (linux-mtd@lists.infradead.org) or Linux mailing lists.
>
>> -----Original Message-----
>> From: Gary Thomas [mailto:gary@mlbassoc.com] Sent: Friday, January 
>> 11, 2008 7:40 AM
>> To: Lin, Bin (Bin)
>> Cc: ecos-discuss@sourceware.org
>> Subject: Re: [ECOS] FW: Mount JFFS2 root file system issue
>>
>> Lin, Bin (Bin) wrote:
>>
>>> Hello All,
>>>
>>> I have ARM9 systems with JFFS2 root file system. For some reasons, e.g.
>>
>>> system crash or power failure, when system boots up next time, it 
>>> takes much longer time than usual to mount jffs2 root file system. 
>>> It can hang at following message for 3-10 min. The system will 
>>> eventually
>>
>>
>>> come up and function well. This problem occurred on less than 10% of 
>>> boards.
>>
>>
>> I think this can happen (after such a failure) when JFFS2 realizes that
>> it needs to rebuild/recover the active nodes in the file system.  This
>> can require a scan of the entire file system, along with rewriting some
>> nodes, etc.
>>
>>> NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
>>> Fast Floating Point Emulator V0.9 (c) Peter Teichmann.
>>>
>>> There are some symbolic links in the root file system. These links 
>>> is linked to the files in another partition. Could that cause 
>>> problem if the file in the other partition gets corrupted? This 
>>> behaviour is not reproducible. Looks like the files in the flash are 
>>> not corrupted. I guess the corruption could be in jffs2 file system.
>>>
>>> Anyone has seen this issue, and solution?
>>>
>>> Appreciate very much for any input,
>>
>>
>> This looks to be a Linux kernel issue - I don't see what it has to do
>> with eCos.
>>
>> You should probably ask on some appropriate Linux mailing list.
>
>

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] FW: Mount JFFS2 root file system issue

[Chris Zimman]

> I am also struggling with JFFS2, and I did already a lot of tests.
> There more files you have in jffs2, the longer it takes to mount:
> - per inode, 16 B (static) or 24 B (dynamic) are written in RAM
> - per file, more or less 72 B (found by testing, I should check the
> jffs2 sources..) are written in RAM
> Also, during mounting, a garbage collect is done. As explained below,
> the file system (the journal) is then checked. And the bigger the
> flash,
> the more time it can take.
> 
> Indeed, once the boot time is long, you can only solve it by formatting
> jffs2.
> 
> I can reproduce this this way: I put 1000 files in 1 directory, then I
> delete 997 of them, and still mounting and also opening of 1 file takes
> very long. To go more in detail: mounting goes from 11s to 10s, but
> opening 1 file goes from 570ms to 820ms.

Just so you're aware, eCosCentric does provide a supported and working
version of JFFS2 with eCosPro.

--Chris


--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] FW: Mount JFFS2 root file system issue

[Jürgen Lambrecht]

Hello Chris,

We would be interested in eCosCentric's JFFS2 if - in order of importance
- it would need less RAM per file
- it would have less "strange" problems
  * if I drive JFFS2 to its limits (put it full, use very big files, put 
many small files in 1 directory) it can get corrupted
  * our simple FTP server corrupts JFFS2 if you put over 180 files with 
a FTP client on PC
  * the more you use JFFS2 (add and remove files), the more it trashes 
itself, so the more delay all actions have
- it would be faster in opening files (especially in a directory with 
over 1000 files)

Therefore, I want to repeat the warning from Thomas Köller: "Do not use 
large JFFS2 file systems if RAM is a scarce resource!" 
(http://sourceware.org/ml/ecos-discuss/2003-10/msg00247.html).
And that is just what we have done..

Kind regards,
Jürgen

Chris Zimman wrote:

>>I am also struggling with JFFS2, and I did already a lot of tests.
>>There more files you have in jffs2, the longer it takes to mount:
>>- per inode, 16 B (static) or 24 B (dynamic) are written in RAM
>>- per file, more or less 72 B (found by testing, I should check the
>>jffs2 sources..) are written in RAM
>>Also, during mounting, a garbage collect is done. As explained below,
>>the file system (the journal) is then checked. And the bigger the
>>flash,
>>the more time it can take.
>>
>>Indeed, once the boot time is long, you can only solve it by formatting
>>jffs2.
>>
>>I can reproduce this this way: I put 1000 files in 1 directory, then I
>>delete 997 of them, and still mounting and also opening of 1 file takes
>>very long. To go more in detail: mounting goes from 11s to 10s, but
>>opening 1 file goes from 570ms to 820ms.
>>    
>>
>
>Just so you're aware, eCosCentric does provide a supported and working
>version of JFFS2 with eCosPro.
>
>--Chris
>
>
>  
>

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] FW: Mount JFFS2 root file system issue

[Chris Zimman]

> We would be interested in eCosCentric's JFFS2 if - in order of
> importance
> - it would need less RAM per file
> - it would have less "strange" problems
>   * if I drive JFFS2 to its limits (put it full, use very big files,
> put
> many small files in 1 directory) it can get corrupted
>   * our simple FTP server corrupts JFFS2 if you put over 180 files with
> a FTP client on PC
>   * the more you use JFFS2 (add and remove files), the more it trashes
> itself, so the more delay all actions have
> - it would be faster in opening files (especially in a directory with
> over 1000 files)
> 
> Therefore, I want to repeat the warning from Thomas Köller: "Do not use
> large JFFS2 file systems if RAM is a scarce resource!"
> (http://sourceware.org/ml/ecos-discuss/2003-10/msg00247.html).
> And that is just what we have done..

Hi Jürgen,

I think you need to discuss these things with eCosCentric.  I don't work for
them, I just recommend them because I know they provide very good service.
As I said before, they provide a supported version of JFFS2 for eCos.  I
don't have the time right now to go and fix up the one in the public
repository.

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss