RE: [ECOS] Stress testing JFFS2

RE: [ECOS] Stress testing JFFS2

[Doug Fraser]

It becomes an interesting issue when you are using the FLASH
to store an actively updated database, which moves through
the FLASH arena at some slow but consistent rate.

Doug Fraser
dfraser@photuris.com

> -----Original Message-----
> From: Thomas Koeller [mailto:thomas.koeller@baslerweb.com]
> Sent: Wednesday, October 15, 2003 7:30 AM
> To: David Woodhouse
> Cc: ecos-discuss@sources.redhat.com; linux-mtd@lists.infradead.org
> Subject: Re: [ECOS] Stress testing JFFS2
> 
> 
> Hi David,
> 
> thanks for your quick response.
> 
> > All correct. But you miss the observation that we also keep a
> > raw_node_ref around for _obsolete_ nodes, which perhaps we 
> could avoid.
> > In fact, we do this because the raw_node_ref is in a 
> singly-linked list,
> > and it's going to be very inefficient to remove obsoleted nodes from
> > that list when they become obsolete.
> 
> I do not think this path leads anywhere I want to go. The flash size
> was chosen to meet the expected storage requirements, which means that
> at some point the flash will be filled with valid data and 
> consequently
> there will be few obsoleted nodes. I expect this to be true for most
> systems.
> 
> > Omitting the 'totlen' field should be relatively simple if 
> you're not
> > freeing obsolete refs. Observe that in 99% of cases, it's true that
> >
> > 	ref->totlen == ref_offset(ref->next_phys) - ref_offset(ref)
> >
> > Make it 100% and make me believe it, and you can remove 
> totlen from the
> > structure.
> >
> 
> I will consider that.
> 
> tk
> --------------------------------------------------
> 
> Thomas Koeller, Software Development
> 
> Basler Vision Technologies
> An der Strusbek 60-62
> 22926 Ahrensburg
> Germany
> 
> Tel +49 (4102) 463-162
> Fax +49 (4102) 463-239
> 
> mailto:thomas.koeller@baslerweb.com
> http://www.baslerweb.com
> 
> ==============================
> 
> 
> -- 
> Before posting, please read the FAQ: 
> http://sources.redhat.com/fom/ecos
> and search the list archive: http://sources.redhat.com/ml/ecos-discuss
> 

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[David Woodhouse]

On Wed, 2003-10-15 at 13:30 +0200, Thomas Koeller wrote:
> I do not think this path leads anywhere I want to go. The flash size
> was chosen to meet the expected storage requirements, which means that
> at some point the flash will be filled with valid data and consequently
> there will be few obsoleted nodes. I expect this to be true for most
> systems.

A sane observation.

> > Omitting the 'totlen' field should be relatively simple if you're not
> > freeing obsolete refs. Observe that in 99% of cases, it's true that
> >
> > 	ref->totlen == ref_offset(ref->next_phys) - ref_offset(ref)
> >
> > Make it 100% and make me believe it, and you can remove totlen from the
> > structure.
> >
> 
> I will consider that.

OK... I've made it slightly easier for you by switching all use of
'ref->totlen' to an accessor function ref_totlen(c, jeb, ref), where
'jeb' can be NULL if the caller doesn't know it -- it _can_ be
calculated in the relatively rare case that it's required, but if it's
already to hand we might as well use it.

I've written what I suspect will be the new version of that accessor
function, and included some sanity checks for it. It's in #if 0 for now
since we know it'll be broken -- look in nodelist.h. We may end up
wanting it out of line, since it's not entirely trivial.

To make it usable, we need to ensure that we have a node reference for
areas of dirty space which we've previously been ignoring -- scan.c can
no longer just skip over stuff it ignores but must allocate node
references for it, and we must allocate node references for the wasted
space at the end of eraseblocks when we've filled them _almost_ to the
end and then filed them elsewhere.

It should be relatively simple.

-- 
dwmw2


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Øyvind Harboe]

On Thu, 2003-10-16 at 09:32, Andrew Lunn wrote:
> > - GCC bug-fix. (Now comitted to eCos CVS, I think).
> 
> Err, are you sure? All i committed was a test to see if you are using
> a known broken compiler and if so, refuse to compile.

Ah, though knowing precisely where the bug is located is 95% of the work
to fix it.

Øyvind



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Andrew Lunn]

> - GCC bug-fix. (Now comitted to eCos CVS, I think).

Err, are you sure? All i committed was a test to see if you are using
a known broken compiler and if so, refuse to compile.

  Andrew

Index: fs-ecos.c
===================================================================
RCS file: /cvs/ecos/ecos/packages/fs/jffs2/current/src/fs-ecos.c,v
retrieving revision 1.10
retrieving revision 1.11
diff -u -r1.10 -r1.11
--- fs-ecos.c   23 Sep 2003 11:47:59 -0000      1.10
+++ fs-ecos.c   4 Oct 2003 17:54:26 -0000       1.11
@@ -28,6 +28,12 @@
 #include <cyg/io/config_keys.h>
 #include <cyg/io/flash.h>
 
+#if (__GNUC__ == 3) && (__GNUC_MINOR__ == 2) && (__GNUC_PATCHLEVEL__ == 1) && \
+    defined (__ARM_ARCH_4__)
+#error This compiler is known to be broken. Please see:
+#error http://ecos.sourceware.org/ml/ecos-patches/2003-08/msg00006.html
+#endif
+
 //==========================================================================
 // Forward definitions
 

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Øyvind Harboe]

W.r.t. actual usage of JFFS2, here are the "parameters" for our usage:

- We need a capability to persit options(i.e. various named parameters 
that are anything from a couple of bytes to a couple of k in size).
These parameters are updated frequently during configuration phases and
only read during normal operation. The total number of files is ~10-30.
This configuration JFFS2 image is 6x64k. Possible alternative: Redboot
parameters?
- The configuration parameters are only read during startup. Afterwards,
it would be advantagous to flush all ram that JFFS2 is using. JFFS2 can
evidently rebuild the ram nodes more than quickly enough.
- During production we have a phase where the a second JFFS2 image is
modifyable(where we store, e.g. serial number). This JFFS2 image is
afterwards write protected in hardware. This JFFS2 image is 1x64k, hence
my previous posts about tricks to get JFFS2 to deal with this. Romdisks
could have been an alternative here, but the tools are designed to run
on the developer machine.

The needs above could be covered by other solutions. However, I like
JFFS2, because it is a generic solution.

I think the complete lists of tweaks to JFFS2 that I've had to do are:

- add option to disable compression. Saves ram. Flash is "cheap".
- allow JFFS2 to write to image when there are 0 free sectors. Allows
"ROM" scheme above + it fixes bugs that showed up in stress testing of
the 6x64k JFFS2 configuration image.
- GCC bug-fix. (Now comitted to eCos CVS, I think).
- Reboot after a couple of writes to JFFS2 to free up ram.


Øyvind



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Thomas Koeller]

Dinesh Kumar wrote:
> I am newbie to JFFS2. What is the reason behind not
> freeing jffs2_raw_node_ref from RAM (even after they
> have been used)?

Their purpose is to represent physical nodes in the FLASH,
and so they must remain just as long as those.

tk
--------------------------------------------------

Thomas Koeller, Software Development

Basler Vision Technologies
An der Strusbek 60-62
22926 Ahrensburg
Germany

Tel +49 (4102) 463-162
Fax +49 (4102) 463-239

mailto:thomas.koeller@baslerweb.com
http://www.baslerweb.com

==============================


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Dinesh Kumar]

> Every operation that changes the contents of the
FLASH 
> (even deleting files!)is performed by writing new 
> nodes to the FLASH. Every such node is represented
in 
> RAM by a struct jffs2_raw_node_ref. The memory 
> occupied by these structs is never freed unless the 
> file system is unmounted or garbage collection takes

> place.
I am newbie to JFFS2. What is the reason behind not
freeing jffs2_raw_node_ref from RAM (even after they
have been used)?

Regards
Dinesh

__________________________________
Do you Yahoo!?
The New Yahoo! Shopping - with improved product search
http://shopping.yahoo.com

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Thomas Koeller]

Hi David,

thanks for your quick response.

> All correct. But you miss the observation that we also keep a
> raw_node_ref around for _obsolete_ nodes, which perhaps we could avoid.
> In fact, we do this because the raw_node_ref is in a singly-linked list,
> and it's going to be very inefficient to remove obsoleted nodes from
> that list when they become obsolete.

I do not think this path leads anywhere I want to go. The flash size
was chosen to meet the expected storage requirements, which means that
at some point the flash will be filled with valid data and consequently
there will be few obsoleted nodes. I expect this to be true for most
systems.

> Omitting the 'totlen' field should be relatively simple if you're not
> freeing obsolete refs. Observe that in 99% of cases, it's true that
>
> 	ref->totlen == ref_offset(ref->next_phys) - ref_offset(ref)
>
> Make it 100% and make me believe it, and you can remove totlen from the
> structure.
>

I will consider that.

tk
--------------------------------------------------

Thomas Koeller, Software Development

Basler Vision Technologies
An der Strusbek 60-62
22926 Ahrensburg
Germany

Tel +49 (4102) 463-162
Fax +49 (4102) 463-239

mailto:thomas.koeller@baslerweb.com
http://www.baslerweb.com

==============================


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[David Woodhouse]

On Wed, 2003-10-15 at 12:11 +0200, Thomas Koeller wrote:
> In an attempt to understand what's really going on inside JFFS2, I
> spent a day analyzing the code. Here's what I found. Please everybody
> comment on this and correct any errors on my side.
> 
> - Every operation that changes the contents of the FLASH (even deleting files!)
>   is performed by writing new nodes to the FLASH. Every such node is represented
>   in RAM by a struct jffs2_raw_node_ref. The memory occupied by these structs
>   is never freed unless the file system is unmounted or garbage collection
>   takes place. Garbage collection starts when there are only five empty
>   erase blocks left. Since every data node on the flash can hold at most one
>   page (4KB) worth of data (uncompressed), the number of in-core instances of
>   struct jffs2_raw_node_ref can grow very large. So in order to support a larger
>   JFFS2 file system, an appropriate amount of RAM is absolutely required.

All correct. But you miss the observation that we also keep a
raw_node_ref around for _obsolete_ nodes, which perhaps we could avoid.
In fact, we do this because the raw_node_ref is in a singly-linked list,
and it's going to be very inefficient to remove obsoleted nodes from
that list when they become obsolete. 

Question: What is your ratio of obsolete to valid nodes, and hence what
benefit, if any, would you gain from turning the singly-linked list into
a doubly-linked list (and increasing the size of the structure by
another pointer) in order to be able to free the obsolete ones? Nobody's
ever checked, to my knowledge.

I suppose it's also possible to take the CPU-time cost of removing the
nodes even from the singly-linked list; nobody's ever done an analysis
of what this would cost either.

Look at jffs2_mark_node_obsolete and see how much time it will take to
start at jeb->first_node, walking the ->next_phys list till you find the
newly-obsoleted raw_node_ref, and free it. You also have to find the
inode number of the inode to which the node belongs (if any), by using
jffs2_raw_ref_to_inum(), and then walk _that_ list too to remove the
raw_node_ref in question.

Note that you can't actually free obsolete node refs which refer to
obsolete directory entries, because we need to keep track of them on
NAND flash. See jffs2_garbage_collect_deletion_dirent() for the gory
details.

> - The size of a single struct jffs2_raw_node_ref is 16 bytes. In ecos, these
>   structs are allocated through calls to malloc(). If the underlying implementation
>   is dlmalloc, as is probably the case most often, the minimum allocation size is
>   24 bytes, so some memory is wasted here. A fixed-size block allocator would be
>   more appropriate.

That one should be relatively easy to do. We do it in Linux already;
each object type already has its own allocation function in malloc.c

> - In http://lists.infradead.org/pipermail/linux-mtd/2003-August/008372.html,
>   David Woodhouse comments on this, suggesting possible improvements. I do not
>   know if any work is going on to implement one of these (probably not).

Omitting the 'totlen' field should be relatively simple if you're not
freeing obsolete refs. Observe that in 99% of cases, it's true that

	ref->totlen == ref_offset(ref->next_phys) - ref_offset(ref)

Make it 100% and make me believe it, and you can remove totlen from the
structure.

You do have the choice between freeing obsolete refs and the totlen
optimisation though -- obviously they don't really work together. You
could _merge_ contiguous obsolete data-node refs and free some RAM
though, while still keeping the simple way of finding totlen.

-- 
dwmw2


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Thomas Koeller]

In an attempt to understand what's really going on inside JFFS2, I
spent a day analyzing the code. Here's what I found. Please everybody
comment on this and correct any errors on my side.

- Every operation that changes the contents of the FLASH (even deleting files!)
  is performed by writing new nodes to the FLASH. Every such node is represented
  in RAM by a struct jffs2_raw_node_ref. The memory occupied by these structs
  is never freed unless the file system is unmounted or garbage collection
  takes place. Garbage collection starts when there are only five empty
  erase blocks left. Since every data node on the flash can hold at most one
  page (4KB) worth of data (uncompressed), the number of in-core instances of
  struct jffs2_raw_node_ref can grow very large. So in order to support a larger
  JFFS2 file system, an appropriate amount of RAM is absolutely required.

- The size of a single struct jffs2_raw_node_ref is 16 bytes. In ecos, these
  structs are allocated through calls to malloc(). If the underlying implementation
  is dlmalloc, as is probably the case most often, the minimum allocation size is
  24 bytes, so some memory is wasted here. A fixed-size block allocator would be
  more appropriate.

- In http://lists.infradead.org/pipermail/linux-mtd/2003-August/008372.html,
  David Woodhouse comments on this, suggesting possible improvements. I do not
  know if any work is going on to implement one of these (probably not).

When I originally decided to design my system around JFFS2 I was totally unaware
of these topics. Although everything I found is implicit in David Woodhouse's
paper 'JFFS : The Journalling Flash File System', it is not stated explicitly
anywhere, so I'm doing it here and now: Do not use large JFFS2 file systems if
RAM is a scarce resource!

tk
--------------------------------------------------

Thomas Koeller, Software Development

Basler Vision Technologies
An der Strusbek 60-62
22926 Ahrensburg
Germany

Tel +49 (4102) 463-162
Fax +49 (4102) 463-239

mailto:thomas.koeller@baslerweb.com
http://www.baslerweb.com

==============================


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Jani Monoses]

On Wed, 08 Oct 2003 09:37:23 +0200
Øyvind Harboe <oyvind.harboe@zylin.com> wrote:

> >What I found out so far is that the memory does not seem to be lost
> >permanently, if the file system is unmounted, it's all there again.
> >I'd
> > be delighted if someone came up with a solution!
> 
> Daved Woodhouse seems to be referring to this problem in this
> thread:
> 
> http://lists.infradead.org/pipermail/linux-mtd/2003-August/008363.html
> 

So is your patch OK with David after all? It would be nice to be able to
disable compression in stock JFFS2.


--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Thomas Koeller]

Andrew Lunn wrote:
> On Tue, Oct 07, 2003 at 06:45:57PM +0200, Thomas Koeller wrote:
> > I also wrote a test program (attached) that runs on my target
> > hardware as well as the synth target. The only thing that's special
> > about it is that main() must be called from initialization context,
> > it wants to start the scheduler itself.
>
> What do you mean by this last bit?
>

I meant to say that the program assumes that eCos has been configured
with CYGSEM_LIBC_STARTUP_MAIN_INITCONTEXT. This could easily be changed,
but at the time I wrote it I used this start mode on my target. I also
should have mentioned that I am using a rather large flash device of
16 MB size, organized as 256 blocks of size 64KB each.

tk
--------------------------------------------------

Thomas Koeller, Software Development

Basler Vision Technologies
An der Strusbek 60-62
22926 Ahrensburg
Germany

Tel +49 (4102) 463-162
Fax +49 (4102) 463-239

mailto:thomas.koeller@baslerweb.com
http://www.baslerweb.com

==============================


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Øyvind Harboe]

>What I found out so far is that the memory does not seem to be lost
>permanently, if the file system is unmounted, it's all there again. I'd
> be delighted if someone came up with a solution!

Daved Woodhouse seems to be referring to this problem in this
thread:

http://lists.infradead.org/pipermail/linux-mtd/2003-August/008363.html

Øyvind



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Andrew Lunn]

On Tue, Oct 07, 2003 at 06:45:57PM +0200, Thomas Koeller wrote:
> I also wrote a test program (attached) that runs on my target
> hardware as well as the synth target. The only thing that's special
> about it is that main() must be called from initialization context,
> it wants to start the scheduler itself.

What do you mean by this last bit? 

Also, you said this works for synth? Its segment faulting for me.....

Round #14
Opened existing file '/jffs2/file_0008'
Checking contents...OK
Overwriting existing file '/jffs2/file_0008' with 11654 data bytes
Duration = 0 seconds
Round #15
Creating new file '/jffs2/file_0006' of size 3714
Wrote 3714 bytes to file '/jffs2/file_0006'
Duration = 0 seconds
Round #16
Creating new file '/jffs2/file_0007' of size 30091
Wrote 30091 bytes to file '/jffs2/file_0007'
Duration = 0 seconds
Round #17
Creating new file '/jffs2/file_0000' of size 235667
iput called for used inode
iput called for used inode
iput called for used inode
iput called for used inode
iput called for used inode
iput called for used inode
iput called for used inode
Wrote 235667 bytes to file '/jffs2/file_0000'
Duration = 0 seconds
Round #18
Creating new file '/jffs2/file_0005' of size 60276
read_cache_page
 
Program received signal SIGSEGV, Segmentation fault.
0x0101fd12 in rb_erase (elm=0x380097, head=0x2029968)
    at /home/lunn/eCos/anoncvs/packages/compat/linux/current/src/rbtree.c:281
281                             elm = left;

Now this assignment looks strange. I don't see how this can cause a
sigfault.

Please could you run it with the latest CVS on your synth target. Im
wondering if i have a compiler problem. I've got gcc 3.3.2 installed
on my Debian-testing system.


   Thanks
      Andrew

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Øyvind Harboe]

>Also, in a previous message you mentioned setting the number of
>required free blocks to zero.  Was this done in this test?  

No.

>If so then
>(as I >understand it) JFFS2 will not work as there are no free blocks
>for GCing so old nodes cannot be deleted thus you have no free flash
>left.

Actually, when I set required number of free blocks to zero,
my test case does a 100k iterations without problems.

Occasionally I see a short pause when presumably it does garbage
collection.

For me setting required number of free blocks to zero has
been a cureall.

Øyvind



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Thomas Koeller]

[-- Attachment #1: Type: text/plain, Size: 2518 bytes --]

I also wrote a test program (attached) that runs on my target hardware as well
as the synth target. The only thing that's special about it is that main() must
be called from initialization context, it wants to start the scheduler itself.

The results I obtained from running it are the same as Øyvind did: the amount
of RAM used by JFFS2 is ever-increasing over time as the program executes.
On my real target, it eventually hits the limit, while on the synthetic target
i have not seen this so far.

Just like Øyvind I was surprised that JFFS2 needs such huge amounts of RAM
(it is targeted at embedded systems after all), but I could not spend any
more time on the investigation. What I found out so far is that the memory
does not seem to be lost permanently, if the file system is unmounted, it's
all there again. I'd be delighted if someone came up with a solution!

tk

Andrew Lunn wrote:
> On Tue, Oct 07, 2003 at 03:16:48PM +0200, ?yvind Harboe wrote:
> > > Please could you give us a complete test case. Something we can run
> > > ourselves. Create a new bugzilla bug on bugs.ecos.sourceware.org and
> > > include the source code and the ecos configuration you are using.
> > >
> > > That will make it much easier for us to see whats happening.
> >
> > OK.
> >
> > But I need a few pointers. Clearly this standalone test case
> > should not be EB40a specific.
> >
> > Ideally anyone should be able to run the testcase, but setting up a
> > JFFS2 mount consists of a fair bit of hardware specific ecos.ecc
> > manipulations, etc.
>
> Yep. Ideally would be to write a test case which runs on the synth
> target. We can all work with that. This does work, i've run the
> standard test case on synth before.
>
> > This might not be eCos specific, but I would expect the eCos community
> > having greater interest in this kind of bug being fixed than
> > non-embedded applications.
>
> Actually, it could be ecos specific. There is an wrapper round jffs2
> which implements the interface between eCos fileio and jffs2. The
> problem could be in there. Also, its easier to debug jffs2 in eCos
> than inside a linux kernel.
>
>      Andrew

-- 
--------------------------------------------------

Thomas Koeller, Software Development

Basler Vision Technologies
An der Strusbek 60-62
22926 Ahrensburg
Germany

Tel +49 (4102) 463-162
Fax +49 (4102) 463-239

mailto:thomas.koeller@baslerweb.com
http://www.baslerweb.com

==============================

[-- Attachment #2: jffs2_exerciser.c.bz2 --]
[-- Type: application/x-bzip2, Size: 2815 bytes --]

[-- Attachment #3: Type: text/plain, Size: 146 bytes --]

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Andrew Lunn]

On Tue, Oct 07, 2003 at 03:16:48PM +0200, ?yvind Harboe wrote:
> > Please could you give us a complete test case. Something we can run
> > ourselves. Create a new bugzilla bug on bugs.ecos.sourceware.org and
> > include the source code and the ecos configuration you are using. 
> > 
> > That will make it much easier for us to see whats happening.
> 
> OK.
> 
> But I need a few pointers. Clearly this standalone test case 
> should not be EB40a specific. 
> 
> Ideally anyone should be able to run the testcase, but setting up a
> JFFS2 mount consists of a fair bit of hardware specific ecos.ecc
> manipulations, etc.

Yep. Ideally would be to write a test case which runs on the synth
target. We can all work with that. This does work, i've run the
standard test case on synth before.

> This might not be eCos specific, but I would expect the eCos community
> having greater interest in this kind of bug being fixed than
> non-embedded applications.

Actually, it could be ecos specific. There is an wrapper round jffs2
which implements the interface between eCos fileio and jffs2. The
problem could be in there. Also, its easier to debug jffs2 in eCos
than inside a linux kernel.

     Andrew

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Eric Donnat]

JFFS2's design paper is at:

http://lwn.net/2001/features/OLS/pdf/pdf/jffs2.pdf

While running this particular test, the filesystem is "at least" allocating
as many struct jffs2_raw_node_ref in RAM as the number of write()
calls. Nodes are only cleaned when sectors are erased. So the ramp up
for memory is huge. Since you're writing very few bytes before truncating
them, the amount of flash storage for every write() is small compared to
the size of flash sectors. Thus there are many jffs2_raw_node_ref in ram
just waiting for being freed.

But with the IPAQ, this is not very much a problem. ;-).

On Tue, 07 Oct 2003 12:57:00 +0200
Øyvind Harboe <oyvind.harboe@zylin.com> wrote:

> Today stress testing JFFS2 came up on the agenda, and I wrote a small
> piece that demonstrates a problem with JFFS2 running out of space
> when it shouldn't, leaving a corrupt JFFS2 image and continuously
> allocating more ram.
> 
> [...]

--
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Øyvind Harboe]

> Please could you give us a complete test case. Something we can run
> ourselves. Create a new bugzilla bug on bugs.ecos.sourceware.org and
> include the source code and the ecos configuration you are using. 
> 
> That will make it much easier for us to see whats happening.

OK.

But I need a few pointers. Clearly this standalone test case 
should not be EB40a specific. 

Ideally anyone should be able to run the testcase, but setting up a
JFFS2 mount consists of a fair bit of hardware specific ecos.ecc
manipulations, etc.

Does JFFS2 have some sort of platform neutral environment to test
its logic?

Something that doesn't even have to be run against an embedded
target or flash?

This might not be eCos specific, but I would expect the eCos community
having greater interest in this kind of bug being fixed than
non-embedded applications.

Øyvind





-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[Andrew Lunn]

On Tue, Oct 07, 2003 at 12:57:00PM +0200, ?yvind Harboe wrote:
> Today stress testing JFFS2 came up on the agenda, and I wrote a small
> piece that demonstrates a problem with JFFS2 running out of space
> when it shouldn't, leaving a corrupt JFFS2 image and continuously
> allocating more ram.
> 
> The test consists of writing lots of files that overwrite existing
> files. The total number of files is never >16.
> 
> /config is a JFFS2 fs. 6*0x10000bytes.
> 
> Should I expect this sort of thing to never fail and to
> reach a plateau of memory usage?

Please could you give us a complete test case. Something we can run
ourselves. Create a new bugzilla bug on bugs.ecos.sourceware.org and
include the source code and the ecos configuration you are using. 

That will make it much easier for us to see whats happening.

> Output:
> 
> 
> Created file 0  153284 bytes ram free
> Created file 100  149132 bytes ram free
> Created file 200  146732 bytes ram free
> Created file 300  144332 bytes ram free
> Created file 400  141932 bytes ram free
> Created file 500  139532 bytes ram free
> Created file 600  137132 bytes ram free
> Created file 700  134732 bytes ram free
> Created file 800  132332 bytes ram free
> Failed

What is failing? open, write, close?

There could actually to two things going one. A plain memory leak and
some other jffs2 problem. I would first investige the call that is
failing. Find out exactly where it fails and hence why.

         Andrew


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Stress testing JFFS2

[David Vrabel]

Hi,

[Note: It's been ages and ages since I did any real work with JFFS]

> 20-30k ram seems excessive for 16 files.

In your test you wrote about 800 files so that's 30000/800 ~ 40 byte per 
written file.  Doesn't seem so outrageous to me. Remember that JFFS2 is 
a log-structured file system so each write appends another node to the 
log.  Old nodes aren't recovered until garbage collection is done. Er. I 
think.

Also, in a previous message you mentioned setting the number of required 
free blocks to zero.  Was this done in this test?  If so then (as I 
understand it) JFFS2 will not work as there are no free blocks for GCing 
so old nodes cannot be deleted thus you have no free flash left.

David Vrabel
-- 
David Vrabel, Design Engineer

Arcom                         Tel: +44 (0)1223 411200 ext. 3233
Clifton Road                  Fax: +44 (0)1223 403400
Cambridge CB1 7EA             E-mail: dvrabel@arcom.com
UK                            Web: http://www.arcom.com/


________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs Email
Security System. For more information on a proactive email security
service working around the clock, around the globe, visit
http://www.messagelabs.com
________________________________________________________________________

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

[ECOS] Stress testing JFFS2

[Øyvind Harboe]

Today stress testing JFFS2 came up on the agenda, and I wrote a small
piece that demonstrates a problem with JFFS2 running out of space
when it shouldn't, leaving a corrupt JFFS2 image and continuously
allocating more ram.

The test consists of writing lots of files that overwrite existing
files. The total number of files is never >16.

/config is a JFFS2 fs. 6*0x10000bytes.

Should I expect this sort of thing to never fail and to
reach a plateau of memory usage?

20-30k ram seems excessive for 16 files.

    char *args="dummy";    
    for (int i=0; i<10000; i++)
    {
    	char name[5];
    	for (int j=0; j<(sizeof(name)-1); j++)
    	{
	    	name[j]=rand()%2+'a';
    	};
    	name[4]=0;
    	
		char nameBuffer[100];
		sprintf(nameBuffer, "/config/%s", name);

		int fd;
	    fd = open( nameBuffer, O_WRONLY|O_CREAT|O_TRUNC );
	    if (fd<0)
	    {
			diag_printf("Failed\r\n");
			break;
	    }
    
		int len=strlen(args);    
		ssize_t actual;
		actual=write(fd, args, len);
		if (actual<0)
		{
			diag_printf("Failed\r\n");
			break;
		}
		int err;
		err=close(fd);
		if (err<0)
		{
			diag_printf("Failed\r\n");
			break;
		}
		
		struct mallinfo info;
		info = mallinfo();
		
    	if ((i%100)==0)
    	{
	    	diag_printf("Created file %d  %d bytes ram free\r\n", i,
info.maxfree);
    	}
    }
    

Output:


Created file 0  153284 bytes ram free
Created file 100  149132 bytes ram free
Created file 200  146732 bytes ram free
Created file 300  144332 bytes ram free
Created file 400  141932 bytes ram free
Created file 500  139532 bytes ram free
Created file 600  137132 bytes ram free
Created file 700  134732 bytes ram free
Created file 800  132332 bytes ram free
Failed



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss