NAND review

NAND review

[Simon Kallweit]

Hi there

I took a quick look at the NAND code released by Ross. I'll just give 
you my thoughts in no particular order:

* Ross's code uses the directory 'io/nand' for the NAND framework and 
adds the NAND flash device drivers in 'devs/nand'. This is contrary to 
what was discussed on the mailing list a few months ago:

http://ecos.sourceware.org/ml/ecos-discuss/2008-09/msg00172.html

I still think that the naming scheme by Ross (and what Rutger originally 
intended to do) is the better approach. Because when we mix the flash 
chip drivers for NOR and NAND chips in one directory (devs/flash) it's 
rather easy to get confused. For example we would get something like 
synth, synth_v2 and synth_nand all in the same directory. Or we would 
have to move that to synth/v1 synth/v2 synth/nand or something. Anyway I 
would suggest we leave the flash v1/v2 stuff alone and add a new devs 
directory 'nand' for the NAND stuff.

* Ross added very primitive support for partitions. Each partition is 
defined by a first and a last block. Reading/writing pages and erasing 
sectors is always done in the context of a partition. BUT the page and 
block addresses are still absolute addresses to the flash chip base, not 
relative to the given partition. This is slightly confusing and I wonder 
if the better approach would be to change this to relative addresses. 
Absolute addresses could still be used in the absence of a partition (NULL).

I also wonder if the current approach for initializing the partition 
table is wise. Currently the flash device driver uses a macro 
NAND_DEV_PARTITION_INIT to call a potential function defined the HAL 
platform to initialize the partition table. This is flexible but we will 
have a lot of code duplication in HAL platforms if the partition table 
is going to be defined via CDL options. Couldn't this be implemented in 
a more generic fashion?

* Ross's code is not layered as heavily as Rutgers code. Rutger has a 
clear interface between the application, the NAND flash controller and 
the NAND flash chips. In Ross's code this is all much more loosly 
coupled. Both approaches have their pros and cons but I would like to 
discuss Ross's approach here. The interface between the common NAND code 
and the flash devices is very high-level (initialization, 
reading/writing pages, erasing blocks, checking factory bad blocks). 
This makes for example the implementation of a synthetic NAND device 
very, very easy. I see some issues with the interface between the NAND 
chip driver and the NAND flash controller. In the current design the 
NAND flash controller is implemented in the HAL platform. The interface 
between the chip driver and the controller is defined in the chip driver 
itself (it just calls to a few functions which the platforms needs to 
implement). Is it intended that future drivers use the same interface or 
are drivers free to define their own? I think a common interface should 
be used for simplicity, and this should probably be defined in a more 
rigid manner. I also wonder if the NAND flash controller functionality 
belongs to the platform. In my opinion this should go to the HAL variant 
instead. What should remain in the platform is the initialization of the 
flash controller as well as the definition of the flash chips used.

* The current code does not read the flash chip timings from the chip 
signature. The timings are hardcoded into the driver. This might or 
might not be the best approach. Again, when adding a generic interface 
between the chip driver and the controller, these timings could be used 
to setup the controller (as some NAND flash controllers can take core of 
these timings). Some of the timings don't belong into the driver but 
into the controller I think, but I didn't look at it in detail.

* The current synth driver is rather limited in its configurability. It 
does not yet support small page flash chips for example. This should be 
improved but I think that's pretty easy to do.

Well these are my first thoughts on the prereleased code. I hope more 
people take a look at it and we can have a discussion and soon decide 
which NAND framework we're going to use.

Simon

Re: NAND review

[Ross Younger]

Hi Simon,

Thanks for the feedback.

> * [io/nand & devs/nand]

I took the view that the path to the driver package ought to imply what
subsystem it's for. Also, putting NAND drivers in devs/flash/nand/* just
doesn't feel right. The flash subsystem takes flash drivers, the nand
subsystem takes nand drivers, and never the twain shall meet.


> * Ross added very primitive support for partitions. Each partition is
> defined by a first and a last block. Reading/writing pages and erasing
> sectors is always done in the context of a partition. BUT the page and
> block addresses are still absolute addresses to the flash chip base, not
> relative to the given partition. This is slightly confusing and I wonder
> if the better approach would be to change this to relative addresses.

I perceive a need for some sort of mechanism to draw the line between
different regions of an array. It seems quite common to have a "/boot
partition" containing one's kernel image and so forth, and to use the rest
of the array as the root filesystem.

My choice of absolute addressing was really a decision for simplicity in the
early days; I suspect that relative addressing could be factored in without
much trouble. YAFFS works just fine with absolute addressing (startBlock and
endBlock); there's no reason why we couldn't pull the wool over its eyes and
tell it to use blocks 0 through whatever, translating them as we pass
addresses onto the NAND layer, but at the same time this is not much of an
argument either way.


> I also wonder if the current approach for initializing the partition
> table is wise. Currently the flash device driver uses a macro
> NAND_DEV_PARTITION_INIT to call a potential function defined the HAL
> platform to initialize the partition table. This is flexible but we will
> have a lot of code duplication in HAL platforms if the partition table
> is going to be defined via CDL options. Couldn't this be implemented in
> a more generic fashion?

The partition definition is necessarily platform-specific, so doesn't fit
anywhere else. When I get round to working through code changes from the
feedback I've received, I'll have a think about whether this could be done
better.


> The interface between the common NAND code
> and the flash devices is very high-level (initialization,
> reading/writing pages, erasing blocks, checking factory bad blocks).

This was a conscious design decision, in order to try and keep the
complexity down.


> In the current design the
> NAND flash controller is implemented in the HAL platform. The interface
> between the chip driver and the controller is defined in the chip driver
> itself (it just calls to a few functions which the platforms needs to
> implement). Is it intended that future drivers use the same interface or
> are drivers free to define their own? 

Unclear. I think that to prescribe too rigidly risks being unduly
constrictive. NAND chips with a 16-bit data bus seem to need 16 bit read and
write functions, and 8-bit for some operations (Read ID, write command,
write address?), and I have heard of some cases where multiple chips are
ganged together to provide a virtual 32-bit device. We could try to work all
this into one place, but I'm worried that we could end up with a wobbly
complex mess which isn't quite right for anything.


> I also wonder if the NAND flash controller functionality
> belongs to the platform. In my opinion this should go to the HAL variant
> instead.

This is a tricky call. In some cases the variant HAL could be the right
place, in others it would most certainly be better suited to the platform.
For example, the NAND chip is hooked up on the EA 2468 board via the memory
controller; all my driver does is write to appropriate MMIO locations which
in turn wiggle the chip's legs. However, would it be right to put something
into the LPC2xxx variant (and if so, what)? Another LPC2xxx-based platform
might have NAND on-board which you could only access by talking to a CPLD
which was effectively a bespoke NAND controller.


> * The current code does not read the flash chip timings from the chip
> signature. The timings are hardcoded into the driver. This might or
> might not be the best approach. 

Timings are necessarily chip-specific, and I've only written the one
real-hardware driver to date. It's entirely possible that my k9 driver could
be adaptable into something more generic, at which point a lookup table
keyed off at least the device ID makes sense.


> * The current synth driver is rather limited in its configurability. It
> does not yet support small page flash chips for example. This should be
> improved but I think that's pretty easy to do.

I was aiming for compatibility with the Linux MTD layer in using a Bad Block
Table, ECC and the layout of the on-chip out-of-band (spare) area. Devices
with 256-byte pages aren't implemented simply because I haven't (yet?)
figured out how the BBT works on them. I might have missed something, but
the location for the BBT identity pattern is at offset 8 of the (8-byte!)
OOB area of the first page of the block that holds the BBT. I'll take
another look into this in due course.


By the way, speaking of Linux compatibility, I did a bit of digging to
confirm the licensing status of the ECC algorithm in the Linux MTD layer.
The latest source in the Linux kernel tree is full-GPL, but an older version
is GPL+LibException. packages/io/nand/current/src/nand_ecc_mtd.c contains
the links I used to verify this. At the time, I discussed this with the
in-house maintainers, and it seemed likely that would be OK to incorporate
into eCos, so I went with it.


Regards,


Ross

-- 
Embedded Software Engineer, eCosCentric Limited.
Barnwell House, Barnwell Drive, Cambridge CB5 8UU, UK.
Registered in England no. 4422071.                  www.ecoscentric.com

Re: NAND review

[Andrew Lunn]

> The partition definition is necessarily platform-specific, so doesn't fit
> anywhere else. 

I simply don't get this. 

Take a case i recently had with a NOR based system. I had a linux
kernel bug i had to trace down. So that i had human readable kernel
opps information, i rebuilt the kernel to include debug symbols. The
resulting kernel was too big to fit in the space allocated to it. So i
used redboot fis to zap both the root filesystem and the space holding
the kernel. I recreated the kernel partition a bit bigger and made the
root filesystem a bit smaller. I then installed the new kernel and the
root filesystem. I then had a booting system with opps with symbols,
not hex addresses.

At no point did i need to edit the HAL, rebuild and install a new
redboot. Why should NAND be different? Why cannot this partition
information be configured by redboot? Why must it be platform
specific?

        Andrew

Re: NAND review

[Andrew Lunn]

On Tue, May 19, 2009 at 10:27:05AM +0200, Simon Kallweit wrote:
> Hi there
>
> * Ross's code uses the directory 'io/nand' for the NAND framework and  
> adds the NAND flash device drivers in 'devs/nand'. This is contrary to  
> what was discussed on the mailing list a few months ago:
>
> http://ecos.sourceware.org/ml/ecos-discuss/2008-09/msg00172.html
>
> I still think that the naming scheme by Ross (and what Rutger originally  
> intended to do) is the better approach. Because when we mix the flash  
> chip drivers for NOR and NAND chips in one directory (devs/flash) it's  
> rather easy to get confused. 

At the time we discussed this i did not have that strong an
opinion. And i think i was the only person who did express an
opinion. I would not complain about this naming scheme if that is what
people want.

       Andrew

Re: NAND review

[Jonathan Larmour]

Simon Kallweit wrote:
> Well these are my first thoughts on the prereleased code. I hope more 
> people take a look at it and we can have a discussion and soon decide 
> which NAND framework we're going to use.

Just to clarify something here, I don't think it's a case of this one or 
that one. Provided someone is prepared to put in the effort, it is 
possible to have a mix of both, with the best aspects of both. It seems 
unlikely to me that one of them will be superior to the other in every way.

Like you, I'm also concerned about some aspects of Ross's use of 
partitioning (and have emailed some details privately to him about that). 
But I'm also concerned about possibly having too much layering in Rutger's 
version for small simple implementations. I guess we'll wait for Ross to 
reply with more detail on his rationale for the differences to Rutger's.

Jifl
-- 
--["No sense being pessimistic, it wouldn't work anyway"]-- Opinions==mine

Re: NAND review

[Simon Kallweit]

Jonathan Larmour wrote:
> Simon Kallweit wrote:
>> Well these are my first thoughts on the prereleased code. I hope more 
>> people take a look at it and we can have a discussion and soon decide 
>> which NAND framework we're going to use.
> 
> Just to clarify something here, I don't think it's a case of this one or 
> that one. Provided someone is prepared to put in the effort, it is 
> possible to have a mix of both, with the best aspects of both. It seems 
> unlikely to me that one of them will be superior to the other in every way.

True, but I think we're still heading for one implementation which is 
going to be refined with code/ideas of the other.

> Like you, I'm also concerned about some aspects of Ross's use of 
> partitioning (and have emailed some details privately to him about 
> that). But I'm also concerned about possibly having too much layering in 
> Rutger's version for small simple implementations. I guess we'll wait 
> for Ross to reply with more detail on his rationale for the differences 
> to Rutger's.

Yes, I generally like the overall lean design of Ross's solution a bit 
more. Currently my only concern is that there is quite a bit of code 
sitting in the platform HALs, and a lot of this will going to be 
duplicated for different ports. But I also see Ross's point about total 
flexibility here, and that it's going to be difficult to have a more 
generic solution which is going to work for all the cases without 
getting messy.

Simon

Re: NAND review

[Rutger Hofman]

Simon Kallweit wrote:
> Jonathan Larmour wrote:
>> that). But I'm also concerned about possibly having too much layering 
>> in Rutger's version for small simple implementations.

Well, there is one obvious candidate for being thinned out in my NAND 
implementation: the ANC layer that hides the presence of multiple 
controllers and/or chips. Making this optional for the (common) case of 
one controller and one (or multiple identical) chips will be easy.

This leaves 3 layers:
- common controller code, which takes care of unavoidable nuisances like 
spare layout, ECC handling, etc;
- controller-specific driver code;
- chip code that does interrogation and bad-block management; this, as 
often as not, is just the common large-page regular chip implementation, 
so no chip-specifics. Later in life we will see ONFI chips rule (maybe!) 
-- no chip-specifics here either.

I dislike the idea of code duplication, and so much is common across 
controller functionality. Hence the split into common and 
device-specific code, which enforces an API in-between. I did my best to 
design this API in a way that is flexible and powerful, but of course I 
cannot rule out that controllers exist that fit this interface only with 
a lot of workarounds. Needless to say, I am very much open to 
suggestions for improvement.

AFAIK, Linux's MTD also has an API between common and specific 
controller code, but it allows even more pluggability for controllers 
than my design. I wouldn't be surprised if all common code can be 
run-time replaced in MTD.

Rutger

Re: NAND review

[Simon Kallweit]

Rutger Hofman wrote:
>> Jonathan Larmour wrote:
>>> that). But I'm also concerned about possibly having too much layering 
>>> in Rutger's version for small simple implementations.
> 
> Well, there is one obvious candidate for being thinned out in my NAND 
> implementation: the ANC layer that hides the presence of multiple 
> controllers and/or chips. Making this optional for the (common) case of 
> one controller and one (or multiple identical) chips will be easy.

I don't really like that idea, as it cuts flexibility a lot. I think we 
will see the need to control 2 or more NAND controllers and/or chips at 
the same time. With Ross's solution this is currently possible and this 
rare case is where his implementation shines IMHO, because you just 
simply implement it in the platform instead of trying to implement it 
generically.

> I dislike the idea of code duplication, and so much is common across 
> controller functionality. Hence the split into common and 
> device-specific code, which enforces an API in-between. I did my best to 
> design this API in a way that is flexible and powerful, but of course I 
> cannot rule out that controllers exist that fit this interface only with 
> a lot of workarounds. Needless to say, I am very much open to 
> suggestions for improvement.

I'm with you on the code duplication matter and I think there should be 
a clear API between chips and the NAND controllers. I can't really judge 
if this is even possible. I have not yet used NAND flash and I don't 
know how diverse the controllers/chips out there are. Is there anyone 
with more insight?

Simon

Re: NAND review

[Bart Veer]

>>>>> "Andrew" == Andrew Lunn <andrew@lunn.ch> writes:

    >> The partition definition is necessarily platform-specific, so
    >> doesn't fit anywhere else.

    Andrew> I simply don't get this. 

    Andrew> Take a case i recently had with a NOR based system. I had
    Andrew> a linux kernel bug i had to trace down. So that i had
    Andrew> human readable kernel opps information, i rebuilt the
    Andrew> kernel to include debug symbols. The resulting kernel was
    Andrew> too big to fit in the space allocated to it. So i used
    Andrew> redboot fis to zap both the root filesystem and the space
    Andrew> holding the kernel. I recreated the kernel partition a bit
    Andrew> bigger and made the root filesystem a bit smaller. I then
    Andrew> installed the new kernel and the root filesystem. I then
    Andrew> had a booting system with opps with symbols, not hex
    Andrew> addresses.

    Andrew> At no point did i need to edit the HAL, rebuild and
    Andrew> install a new redboot. Why should NAND be different? Why
    Andrew> cannot this partition information be configured by
    Andrew> redboot? Why must it be platform specific?

I am not a NAND expert, but I think the answer is that NAND is
fundamentally different from NOR: it is unreliable.

With NOR flash we can store partition table info (i.e. FIS) in a
well-known location. RedBoot can write to and read from that location
and it is pretty much guaranteed to work. Hence the only detail that
needs to be encoded in the RedBoot executable is that location - which
is configurable although nearly all systems will use the default.
Everything else can be determined and changed at run-time.

With NAND flash we do not have the guarantee from the hardware that a
well-known location will always work. In fact Sod's law will ensure
that that location will be one of the first to have errors.

Some systems will have reliable persistent storage in addition to the
NAND, e.g. an additional bank of NOR flash. On such systems we could
store the NAND partition table in the NOR flash, using similar code
to FIS. But we cannot assume that all systems will be like that, and I
do not like the idea of having two different partitioning schemes
depending on the hardware capabilities.

Or, we could try to implement a robust layer on top of the basic NAND
layer. If you want to store N pages reliably, reserve (N+f(N)+k)
pages in the NAND flash. When one of the pages starts failing, start
using one of the spare pages. Assume that updates will be infrequent
so that you do not need to worry about wear-levelling, just bad
blocks. This would give us a way of storing the partition info in the
NAND flash with a high degree of reliability. Unfortunately it means a
fairly complicated extra layer which will not be needed by e.g.
NAND-aware flash filesystems - those expect to handle the bad pages
etc. themselves.

So, if you cannot store the partition info in the NAND chip itself
without bloat, and you may not have any other persistent storage in
the system, you are left with two choices: don't bother with partition
info at all, or embed the partition info in the code.

Ignoring partitioning info completely in the NAND layer, and instead
leaving it to higher-level code, is somewhat tempting. But it does not
actually solve anything. Consider a chip with a built-in bootloader
which can read in a secondary bootloader from NAND flash. That
built-in bootloader will impose certain restrictions on the NAND
flash, e.g. it will reserve n pages at the start of the flash. The
partitioning code has to be aware of that. But you really do not want
higher-level partitioning code in both YAFFS and UFFS having to cope
with hardware-specific details like that. Doing it in a single NAND
layer with close ties to the platform HAL seems better.

So, what you are left with is embedding the partition info in the
code. We really do not want to completely hardwire it for a given
system, so using configury is the next best thing. It is not as
flexible as FIS for NOR flash, but it will be adequate for most users.

Now, there are ways in which we should be able to avoid duplicating
lots of CDL for most platforms. We can have a default partitioning
scheme in the generic NAND code, active_if some interface. If a
platform can use that default scheme it can just implement the
interface, possibly in conjunction with some default settings and
requires properties. If a platform really needs to do something
special it can ignore the generic support and define its own
partitioning CDL, taking into account whatever weird constraints the
hardware has. It may not be possible to figure out exactly how to do
that just yet, and it may need to wait until NAND support is working
on a few more platforms.

Bart

Re: NAND review

[Rutger Hofman]

Bart Veer wrote:
>>>>>> "Andrew" == Andrew Lunn <andrew@lunn.ch> writes:
> 
>     >> The partition definition is necessarily platform-specific, so
>     >> doesn't fit anywhere else.
> 
>     Andrew> I simply don't get this. 
> 
>     Andrew> Take a case i recently had with a NOR based system. I had
>     Andrew> a linux kernel bug i had to trace down. So that i had
>     Andrew> human readable kernel opps information, i rebuilt the
>     Andrew> kernel to include debug symbols. The resulting kernel was
>     Andrew> too big to fit in the space allocated to it. So i used
>     Andrew> redboot fis to zap both the root filesystem and the space
>     Andrew> holding the kernel. I recreated the kernel partition a bit
>     Andrew> bigger and made the root filesystem a bit smaller. I then
>     Andrew> installed the new kernel and the root filesystem. I then
>     Andrew> had a booting system with opps with symbols, not hex
>     Andrew> addresses.
> 
>     Andrew> At no point did i need to edit the HAL, rebuild and
>     Andrew> install a new redboot. Why should NAND be different? Why
>     Andrew> cannot this partition information be configured by
>     Andrew> redboot? Why must it be platform specific?
> 
> I am not a NAND expert, but I think the answer is that NAND is
> fundamentally different from NOR: it is unreliable.
> 
[snip]
> Or, we could try to implement a robust layer on top of the basic NAND
> layer. If you want to store N pages reliably, reserve (N+f(N)+k)
> pages in the NAND flash. When one of the pages starts failing, start
> using one of the spare pages. Assume that updates will be infrequent
> so that you do not need to worry about wear-levelling, just bad
> blocks. This would give us a way of storing the partition info in the
> NAND flash with a high degree of reliability. Unfortunately it means a
> fairly complicated extra layer which will not be needed by e.g.
> NAND-aware flash filesystems - those expect to handle the bad pages
> etc. themselves.

FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).

Rutger

Re: NAND review

[Rutger Hofman]

Simon Kallweit wrote:
> Rutger Hofman wrote:
>>> Jonathan Larmour wrote:
>>>> that). But I'm also concerned about possibly having too much 
>>>> layering in Rutger's version for small simple implementations.
>>
>> Well, there is one obvious candidate for being thinned out in my NAND 
>> implementation: the ANC layer that hides the presence of multiple 
>> controllers and/or chips. Making this optional for the (common) case 
>> of one controller and one (or multiple identical) chips will be easy.
> 
> I don't really like that idea, as it cuts flexibility a lot. I think we 
> will see the need to control 2 or more NAND controllers and/or chips at 
> the same time. With Ross's solution this is currently possible and this 
> rare case is where his implementation shines IMHO, because you just 
> simply implement it in the platform instead of trying to implement it 
> generically.

Uhmmm... I am not sure I understand? In my current NAND implementation, 
the platform is free to hide the fact that there are multiple devices by 
having one ANC that handles the multiplicity issues transparently, or 
the platform can configure things so that the multiplicity is made 
public by using multiple ANC structs, or everything in between.

#undef-ing the multiplicity support /within/ the ANC code would be a 
hack to get leaner compiled code when ANCs have only one (type of) 
controller/chip. This leaves in multiplicity by using multiple ANCs anyway.

Rutger

Re: NAND review

[Andrew Lunn]

> FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).

Hi Rutger

You seem to know the MTD code. How does MTD handle partition
information? Where does it get it from?

    Thanks
        Andrew

Re: NAND review

[Gary Thomas]

Andrew Lunn wrote:
>> FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).
> 
> Hi Rutger
> 
> You seem to know the MTD code. How does MTD handle partition
> information? Where does it get it from?

It can be hard coded, come from the command line, or use the RedBoot
FIS directory.  (any or all of this set)

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: NAND review

[Rutger Hofman]

Andrew Lunn wrote:
>> FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).
> 
> Hi Rutger
> 
> You seem to know the MTD code. How does MTD handle partition
> information? Where does it get it from?

Well, I didn't read that part of MTD well. My guess would be that UBI 
does it, on top of MTD.

Rutger

Re: NAND review

[Ross Younger]

Andrew Lunn wrote:
> Why cannot this partition information be configured by redboot? 
> Why must it be platform specific?

When I was designing my NAND library, I was striving for compatibility with
Linux - that is to say, the ability for an eCos app (RedBoot?) and a Linux
kernel to share the same NAND array.

There has to be some sort of partition mechanism, but there is no really
standard way of doing so. The Linux MTD layer doesn't define anything in
this regard; typically, the partition info is passed to the kernel at boot
time (having been hard-coded into the bootloader or read from EEPROM or
somesuch) and it then appears in /dev as the relevant number of devices
(mtdblock*). What this means is - if I understand the situation correctly -
if whoever put the platform together wanted the chip to appear as multiple
partitions for whatever reason, they had to invent their own mechanism to
specify the dimensions.

If compatibility with Linux is not a selling point, then we can of course do
what we like. For example, it would be relatively straightforward for me to
carve off a block or two to robustly store a partition table - as Rutger
hints, the MTD bad block table as mimicked by both of our NAND libraries
already does this - and a little more work to add helper commands to RedBoot
to allow it to be manipulated interactively. However, if we went down this
route, it would be dangerous to maintain the pretence of compatibility with
Linux; this could of course be trivially given up by changing the signature
magic numbers on our BBT.

Cheers,


Ross

-- 
Embedded Software Engineer, eCosCentric Limited.
Barnwell House, Barnwell Drive, Cambridge CB5 8UU, UK.
Registered in England no. 4422071.                  www.ecoscentric.com

Re: NAND review

[Gary Thomas]

Ross Younger wrote:
> Andrew Lunn wrote:
>> Why cannot this partition information be configured by redboot? 
>> Why must it be platform specific?
> 
> When I was designing my NAND library, I was striving for compatibility with
> Linux - that is to say, the ability for an eCos app (RedBoot?) and a Linux
> kernel to share the same NAND array.
> 
> There has to be some sort of partition mechanism, but there is no really
> standard way of doing so. The Linux MTD layer doesn't define anything in
> this regard; typically, the partition info is passed to the kernel at boot
> time (having been hard-coded into the bootloader or read from EEPROM or
> somesuch) and it then appears in /dev as the relevant number of devices
> (mtdblock*). What this means is - if I understand the situation correctly -
> if whoever put the platform together wanted the chip to appear as multiple
> partitions for whatever reason, they had to invent their own mechanism to
> specify the dimensions.

I don't know what/where you were looking to come to these conclusions :-(
Linux MTD has had the ability to use the RedBoot FIS directory for many (7+) years.

> If compatibility with Linux is not a selling point, then we can of course do
> what we like. For example, it would be relatively straightforward for me to
> carve off a block or two to robustly store a partition table - as Rutger
> hints, the MTD bad block table as mimicked by both of our NAND libraries
> already does this - and a little more work to add helper commands to RedBoot
> to allow it to be manipulated interactively. However, if we went down this
> route, it would be dangerous to maintain the pretence of compatibility with
> Linux; this could of course be trivially given up by changing the signature
> magic numbers on our BBT.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: NAND review

[Andrew Lunn]

On Wed, May 20, 2009 at 07:55:58AM -0600, Gary Thomas wrote:
> Andrew Lunn wrote:
> >> FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).
> > 
> > Hi Rutger
> > 
> > You seem to know the MTD code. How does MTD handle partition
> > information? Where does it get it from?
> 
> It can be hard coded, come from the command line, or use the RedBoot
> FIS directory.  (any or all of this set)

You could map these into eCos like concepts:

Hard code     -> Hard coded
command line  -> Redboot cfg block parameter?
FIS Directory -> FIS Directory!

I find it interesting that Linux guys consider FIS directory usable,
which is against what Bart was saying.

Putting that point aside, it does show that Linux considers it
necessary to have multiple ways of configuring the partitions, so
maybe eCos also needs multiple ways of configuring partitions.

      Andrew

Re: NAND review

[Andrew Lunn]

> You could map these into eCos like concepts:
> 
> Hard code     -> Hard coded
> command line  -> Redboot cfg block parameter?
> FIS Directory -> FIS Directory!

There is also one case we must support. If we don't chaos will result.

Imagine hard coded partitions in Redboot. This is used to boot an eCos
application. It also has hard coded partitions. However they are
different! You are heading for trouble.

We need that eCos applications can use Virtual Vectors to ask the eCos
ROM redboot what the partitions are. The RAM app does not care where
the ROM redboot gets the partitions from, be it hard coded, FIS. etc,
so long as they are consistent with what redboot is using.

   Andrew

Re: NAND review

[Ross Younger]

Gary Thomas wrote:
> I don't know what/where you were looking to come to these conclusions :-(
> Linux MTD has had the ability to use the RedBoot FIS directory for many (7+) years.

At the moment, as I understand things, FIS implies io/flash, which pretty
much implies NOR. I don't have any plans to implement FIS on top of NAND;
people who do use NAND will generally be using a filesystem, so "obviously"
the right place to put static config data is on that filesystem? (Chicken,
meet egg?)

Now, if we were to need to write a primary bootloader - say, if a
OneNAND-based SOC came into the frame and we wanted to run RedBoot on it -
then we might very well find ourselves needing something along these lines.
A very very simple filesystem, combined perhaps with the ability to store a
little config data and/or partition table layout, for which it might be
worth considering reusing some or all of the FIS code? With that in mind, I
suppose that defining a scheme for a partition table and very simple FS for
NAND would be a worthwhile exercise, which could then of course be
contribbed to Linux. Comments?


Ross

-- 
Embedded Software Engineer, eCosCentric Limited.
Barnwell House, Barnwell Drive, Cambridge CB5 8UU, UK.
Registered in England no. 4422071.                  www.ecoscentric.com

Re: NAND review

[Bart Veer]

>>>>> "Andrew" == Andrew Lunn <andrew@lunn.ch> writes:

    Andrew> On Wed, May 20, 2009 at 07:55:58AM -0600, Gary Thomas wrote:
    >> Andrew Lunn wrote:
    >> >> FWIW, this is the approach taken by MTD with the BBT (Bad Block Table).
    >> > 
    >> > Hi Rutger
    >> > 
    >> > You seem to know the MTD code. How does MTD handle partition
    >> > information? Where does it get it from?
    >> 
    >> It can be hard coded, come from the command line, or use the RedBoot
    >> FIS directory.  (any or all of this set)

    Andrew> You could map these into eCos like concepts:

    Andrew> Hard code     -> Hard coded
    Andrew> command line  -> Redboot cfg block parameter?
    Andrew> FIS Directory -> FIS Directory!

    Andrew> I find it interesting that Linux guys consider FIS
    Andrew> directory usable, which is against what Bart was saying.

The approach is usable iff you have NOR flash as well as NAND flash.
    
    Andrew> Putting that point aside, it does show that Linux
    Andrew> considers it necessary to have multiple ways of
    Andrew> configuring the partitions, so maybe eCos also needs
    Andrew> multiple ways of configuring partitions.

To cope with systems which only have NAND flash, I think we must
support hard-coding via configury. Hence that functionality must be
implemented straightaway. It does not preclude adding other ways of
specifying partitions in future, e.g. by storing them in NOR flash, if
we feel that the added flexibility is worth the implementation effort
and the code and data bloat. Obviously typically Linux developers will
be less concerned about the latter than eCos developers.

Bart

Re: NAND review

[Gary Thomas]

Ross Younger wrote:
> Gary Thomas wrote:
>> I don't know what/where you were looking to come to these conclusions :-(
>> Linux MTD has had the ability to use the RedBoot FIS directory for many (7+) years.
> 
> At the moment, as I understand things, FIS implies io/flash, which pretty
> much implies NOR. I don't have any plans to implement FIS on top of NAND;
> people who do use NAND will generally be using a filesystem, so "obviously"
> the right place to put static config data is on that filesystem? (Chicken,
> meet egg?)

I implemented FIS on NAND on the MOAB board in 2002 - it has been in the eCos tree
since then.  This also supported Linux 2.4 in the same time frame (the company
that made the MOAB is out of that business, so that port has fallen by the
wayside, but it has been done before, for a long time)

And yes, that support was built on top of io/flash - not perfect and I'm
not saying that the current discussion is out of line, but [IMO] proper
partition support using RedBoot FIS on NAND is not a new idea.

As for what to do if the block(s) containing the FIS directory are bad - this
is a day-one problem for all such systems.  That's why your EXTn file system
writes out many "super blocks" so there is a fall back, etc.  If there is a
bad blocks replacement algorithm, use that (Linux MTD has pretty strong ideas
about this for NAND which I suggest you follow), otherwise find another way
to deal with such failures.

> Now, if we were to need to write a primary bootloader - say, if a
> OneNAND-based SOC came into the frame and we wanted to run RedBoot on it -
> then we might very well find ourselves needing something along these lines.
> A very very simple filesystem, combined perhaps with the ability to store a
> little config data and/or partition table layout, for which it might be
> worth considering reusing some or all of the FIS code? With that in mind, I
> suppose that defining a scheme for a partition table and very simple FS for
> NAND would be a worthwhile exercise, which could then of course be
> contribbed to Linux. Comments?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: NAND review

[Andrew Lunn]

> Date: Tue, 02 Jun 2009 19:17:58 +0100
> From: John Dallaway <john@dallaway.org.uk>
> To: Andrew Lunn <andrew.lunn@ascom.ch>
> CC: eCos Maintainers <ecos-maintainers@ecos.sourceware.org>
> Subject: Re: NAND & YAFFS
>
> Have you been able to form a view on the relative merits of the VU
> Amsterdam and eCosCentric implementations so far? Just wanting to ensure
> that this process is not stalled for any reason...
>
> John Dallaway

Hi John

I've spent a little time looking at the documentation and followed the
discussions on the mailling lists. I've not dug into the code at all.

From this i have my first impressions:

eCosCentric/Ross entered the game late, after starting behind closed
doors. There was no RFC about the design, APIs etc. VU/Rutger on the
other hand has been very open, posted his designs and APIs, been
involved in discussion and made changes that resulted from the
discussions. This gets lots of +ve points for Rutger and many -ve
points for Ross/eCosCentric.

I would say the current documentation of the two frameworks is about
equal and generally good. No complains here.

IMHO Ross's partitioning scheme/API is broken. I've tried to trigger
discussion about this, but there has not been much interest. This is a
shame, because to fix it will require API changes, so requiring
changes to the YAFFS glue code etc. I'm also not the only one with
reservations about this partitioning, i've had a private email from a
respected member of the community expressing reservations about the
concept.

I don't see it currently being a problem that Rutgers code does not
have a partitioning concept. It has a nice clean API to
application/filesystem code. My first impression from reading the
documentation suggests that adding partitioning to this will not
require any major API changes, it might even be totally transparent.

So a few +ve points to Rutger, many -ve points to Ross.

I would say the way forward with partitioning is to discuss the
requirements, where can the partitioning information come from, where
does it need to go to, what should the APIs look like. Then do some
design work to make draft APIs, collect comments and then do the
implementation work/modifications as necessary. 

There has been comments that Rutgers code has too many layers. Rutger
aims to allow as much code reuse between drivers as possible, which i
think is good. Simon commented that he thinks Ross's design may result
in a lot of code stuck in HALs where it is hard to reuse without copy
paste. However Ross argued that he thinks that making code reusable is
going to be hard, there is too many different configurations of chips
and controllers, etc.

With only one supported platform on Ross's code and two with Rutgers,
i think it is too early to tell the truth. However, im generally in
favour of layers.

Ross has a simple, but functional synthetic NAND driver. Simon started
on a synth driver for Rutgers library, but stopped when Ross made his
surprise announcement, wanting to see which way we go. Simon also
commented that he used quite a bit of MTD code. This got me a bit
worried about GPL issues, but there was an interesting comment from
Ross that an older version of MTD has a more eCos friendly license.

So, a few +ve points for Ross, with respect to synth drivers.

Initially i was a bit worried about Rutgers need for dynamic memory
allocation. This is needed for the bad block table and per thread
error reporting support. This could be a problem for Redboot, where
you don't normally have a "heap", malloc and free etc. However, on
second thoughts, i don't think this is too big an issue. YAFFS needs
malloc/free. UFFS can either use its own malloc/free implementation,
or use the system provided functions. So in general, redboot is going
to need malloc/free for supporting file systems, so having the NAND
library using it is not that bad.

Ross's code has the bad block table statically allocated, sized using
CDL. This removes the need for malloc/free. However, in some ways, it
reduces the flexibility. I've seen quite a few systems with the same
basic hardware, just different sized FLASH chipsets. With Rutgers
system, this should be no problem, is just asks for as much heap as
needed. Ross's system may need different eCos builds for each hardware
variant, since the size of the BBT is hard coded in the HAL, plus the
partition table is also hard coded in the HAL.

The important thing here is that the code for allocating/freeing the
memory is nicely encapsulated so that it is not too difficult to add
extra options, controlled via CDL, as to how it gets the memory it
needs. I'd need to review the code to see how well this is implemented
in both systems.

There was an email from Rutger that Televic are willing to contribute
their drivers for an AT91SAM9260 based board. If that happens, it
would mean two real implementations for Rutgers, vs one for Ross.

Rutgers API allows reading/writing less than a page, eg just a few
bytes. Ross's API is page based. I don't know if this is an advantage
or a disadvantage. 

Simon commented he preferred Ross's directory layout within the
repository. I have no strong preference here, and it should be trivial
to change Rutgers layout if we wanted to.

There was comments from Ross about excessive stack usage blowing his
target away. This might indicate the immaturity of his code? Rutgers
has been using his code in RFID guardian, and i presume Televic have
also been using the code. So i get the feeling Rutgers code is more
stable and mature.

Overall, i currently favour Rutgers implementation. However, as i said
at the beginning, i've not looked at the actual source code yet.

         Andrew

Re: NAND review

[Ross Younger]

Hi Andrew,

Thanks for the summary. If I may comment on a couple of technical points...


> IMHO Ross's partitioning scheme/API is broken. I've tried to trigger
> discussion about this, but there has not been much interest. 

I've been quiet recently as I've been pushing to get YAFFS and RedBoot
integration up to the "complete first cut" stage. This has now happened; I
have successfully loaded and run an eCos application image from a YAFFS
filesystem on NAND flash using RedBoot. (YAFFS is on the BZ ticket; I will
sort out the eCos-v-eCosPro differences and push my [minimal] RedBoot work
to the ticket later today.) Now I've got time to take stock of all the
useful feedback, and figuring out what I do with the partitioning scheme is
one of the major issues!


> [...] This got me a bit
> worried about GPL issues, but there was an interesting comment from
> Ross that an older version of MTD has a more eCos friendly license.

For clarity, this related only to the ECC code in MTD, which I incorporated.
The rest of my code is all fresh.


> Initially i was a bit worried about Rutgers need for dynamic memory
> allocation. [...] However, on
> second thoughts, i don't think this is too big an issue. YAFFS needs
> malloc/free. [...] So in general, redboot is going
> to need malloc/free for supporting file systems, so having the NAND
> library using it is not that bad.

The philosophical question for us all is whether NAND on its own should be
allowed to use malloc, given that a NAND array will probably always be used
in conjunction with a log-structured filesystem which will chew up
comparatively large amounts of RAM (and, of course, RAM is forever getting
cheaper). Is this a corner or even N-dimensional vertex case; will it
necessarily always be the case that a device with NAND flash will have
enough RAM to support it? Do boards with NAND but not much RAM exist, and if
so do we care about them?


> Rutgers API allows reading/writing less than a page, eg just a few
> bytes. Ross's API is page based. I don't know if this is an advantage
> or a disadvantage. 

This is a tough one to call. I went for simplicity and a tight mapping to
the hardware. One could argue that providing a bytewise API might encourage
programmers unfamiliar with NAND flash to use it in a bytewise manner and
risk prematurely wearing out their chips. (I believe MTD has something along
these lines. "If it looks like a hammer...")


> There was comments from Ross about excessive stack usage blowing his
> target away. This might indicate the immaturity of his code? 

Stack usage by my NAND layer is not a problem on my target in a default eCos
config. It only becomes an issue when you try and use it under minimal
(possibly another vertex case?), when it turns out that the default isn't
big enough. I've fiddled the CDL in my working tree as a stopgap, and my
todo list includes figuring out ways to improve this. (It's only one
particularly stack-greedy function that has caused trouble, which might well
benefit from a bit of algorithmic tuning. Or indeed, if we decide that it's
OK for NAND to require malloc, then I can do a temporary malloc and free.)

Similarly, I found that when RedBoot has CYGPKG_MEMALLOC loaded, its default
heap size isn't big enough to mount even an empty YAFFS filesystem. There's
another stopgap in place, and another todo list entry...


Ross

-- 
Embedded Software Engineer, eCosCentric Limited.
Barnwell House, Barnwell Drive, Cambridge CB5 8UU, UK.
Registered in England no. 4422071.                  www.ecoscentric.com

Re: NAND review

[Andrew Lunn]

> The philosophical question for us all is whether NAND on its own should be
> allowed to use malloc, given that a NAND array will probably always be used
> in conjunction with a log-structured filesystem which will chew up
> comparatively large amounts of RAM (and, of course, RAM is forever getting
> cheaper). Is this a corner or even N-dimensional vertex case; will it
> necessarily always be the case that a device with NAND flash will have
> enough RAM to support it? Do boards with NAND but not much RAM exist, and if
> so do we care about them?

The answer is yes. Simon, could you describe your board. From what i
understand you don't have much RAM. 

An example application area would be a data logger. It has a few
inputs, eg ADC, temperature sensors etc, and a NAND flash where it
stores the results. Its the sort of thing an AT91SAM7SE could do, 32K
RAM and 256K NOR FLASH for the code. 

There are two different major categories:

1) Using eCos/Redboot to get Linux booted. RAM is not going to be a problem.
2) eCos is the target OS and RAM could be limited.
 
> > Rutgers API allows reading/writing less than a page, eg just a few
> > bytes. Ross's API is page based. I don't know if this is an advantage
> > or a disadvantage. 
> 
> This is a tough one to call. I went for simplicity and a tight mapping to
> the hardware. One could argue that providing a bytewise API might encourage
> programmers unfamiliar with NAND flash to use it in a bytewise manner and
> risk prematurely wearing out their chips. (I believe MTD has something along
> these lines. "If it looks like a hammer...")
 
That is what i was thinking. Bytewise read makes more sense than
writing. Also, if the underlying chipset does not support bytewise
reading/writing, you end up needing a page buffer low down in the
stack, rather up in the application where it might be reusable for
other things when memory is tight.
 
        Andrew

Re: NAND review

[Simon Kallweit]

Andrew Lunn wrote:
>> The philosophical question for us all is whether NAND on its own should be
>> allowed to use malloc, given that a NAND array will probably always be used
>> in conjunction with a log-structured filesystem which will chew up
>> comparatively large amounts of RAM (and, of course, RAM is forever getting
>> cheaper). Is this a corner or even N-dimensional vertex case; will it
>> necessarily always be the case that a device with NAND flash will have
>> enough RAM to support it? Do boards with NAND but not much RAM exist, and if
>> so do we care about them?
> 
> The answer is yes. Simon, could you describe your board. From what i
> understand you don't have much RAM. 

Well yes, our current hardware has 512k on-chip flash, 64k on-chip ram 
and 128k external ram. For additional storage we currently have a 4mb 
NOR flash, but we're planning on replacing this with a NAND flash so we 
can run a tiny filesystem like UFFS. The current software does only use 
static memory with the exception of lwIP, which uses fixed memory pools 
for dynamic allocation.

For upcoming products we were thinking about getting rid of the external 
ram, which would leave us with only 64k. But this certainly would lead 
to problems in other areas as well.

Simon

Re: NAND review

[Rutger Hofman]

Andrew Lunn wrote:
>> The philosophical question for us all is whether NAND on its own should be
>> allowed to use malloc, given that a NAND array will probably always be used
>> in conjunction with a log-structured filesystem which will chew up
>> comparatively large amounts of RAM (and, of course, RAM is forever getting
>> cheaper). Is this a corner or even N-dimensional vertex case; will it
>> necessarily always be the case that a device with NAND flash will have
>> enough RAM to support it? Do boards with NAND but not much RAM exist, and if
>> so do we care about them?
> 
> The answer is yes. Simon, could you describe your board. From what i
> understand you don't have much RAM. 

As I didn't want to tie the NAND user to malloc, I designed in a 
pluggable allocator. Often, the application can calculate beforehand how 
much memory is needed: 1) BBT, which depends on the NAND size; 2) 
per-thread space for the error handling. If there is no threaded kernel, 
it would be just the BBT which can be allocated statically by the 
application/platform.

>>> Rutgers API allows reading/writing less than a page, eg just a few
>>> bytes. Ross's API is page based. I don't know if this is an advantage
>>> or a disadvantage. 
>> This is a tough one to call. I went for simplicity and a tight mapping to
>> the hardware. One could argue that providing a bytewise API might encourage
>> programmers unfamiliar with NAND flash to use it in a bytewise manner and
>> risk prematurely wearing out their chips. (I believe MTD has something along
>> these lines. "If it looks like a hammer...")

The number of writes to a page between erases is often severely limited, 
like 2 or 4. Therefore bytewise writes are not really an option.

> That is what i was thinking. Bytewise read makes more sense than
> writing. Also, if the underlying chipset does not support bytewise
> reading/writing, you end up needing a page buffer low down in the
> stack, rather up in the application where it might be reusable for
> other things when memory is tight.

The file systems I took a look at read/write a complete page. NAND chips 
all support byte/word addressing, though. Like Andrew and Ross, I am not 
completely certain which is the best way to go. Limiting reads and 
writes to complete pages would make the code a little bit simpler in one 
or two places.

Rutger

Re: Re: NAND review

[Jürgen Lambrecht]

Andrew Lunn wrote:
>> Date: Tue, 02 Jun 2009 19:17:58 +0100
>> From: John Dallaway <john@dallaway.org.uk>
>> To: Andrew Lunn <andrew.lunn@ascom.ch>
>> CC: eCos Maintainers <ecos-maintainers@ecos.sourceware.org>
>> Subject: Re: NAND & YAFFS
>>
>> Have you been able to form a view on the relative merits of the VU
>> Amsterdam and eCosCentric implementations so far? Just wanting to ensure
>> that this process is not stalled for any reason...
>>
>> John Dallaway
>>     
>
>   
<snip>

> There was an email from Rutger that Televic are willing to contribute
> their drivers for an AT91SAM9260 based board. If that happens, it
> would mean two real implementations for Rutgers, vs one for Ross.
>
>   
Indeed, we would like to contribute, but it takes time...
Stijn will try to do the contribution, I will get the FSF copyright 
assignment (but the site is down today..).

Kind regards,
Jürgen

<snip>
>          Andrew
>   

Re: NAND review

[Nick Garnett]

Andrew Lunn <andrew@lunn.ch> writes:
 
> IMHO Ross's partitioning scheme/API is broken. I've tried to trigger
> discussion about this, but there has not been much interest. This is a
> shame, because to fix it will require API changes, so requiring
> changes to the YAFFS glue code etc. I'm also not the only one with
> reservations about this partitioning, i've had a private email from a
> respected member of the community expressing reservations about the
> concept.
> 
> I would say the way forward with partitioning is to discuss the
> requirements, where can the partitioning information come from, where
> does it need to go to, what should the APIs look like. Then do some
> design work to make draft APIs, collect comments and then do the
> implementation work/modifications as necessary. 
> 

In the interests of getting things moving, I'm going to give my take
on the partitioning debate. I discussed this with Ross before he
implemented the scheme, so I guess I must take some of the blame :-/

The first thing to say is that partitioning was never intended to be
the functional equivalent of the flash FIS directory. NAND flash is
designed to be used to store filesystems (possibly very primitive)
rather be used as a read-only memory like NOR flash. There will almost
always be another layer of software dealing with bad blocks, wear
levelling and OOB data between the application and the NAND device
itself. Different partitions will be handled by different
subsystems. NAND partitions are much more like the partitions on a
disk and the choice of the term was meant to convey this. It is a
relatively primitive way to divide the NAND up into functional areas.

It is sometimes best to think about a concrete example, and my
thoughts on NAND use were informed by thinking about how to use the
NAND on an Atmel AT91SAM9 based device. These processors have no
on-chip flash and must boot from an external device. A variety of such
devices are allowed, from dataflash on an SPI bus, EEPROM on I2C and
NAND flash. NOR flash is not an option, and few of the -EK boards are
equipped with it.

To boot, a SAM9 device runs an on-chip bootloader that searches the
external memories for something it recognizes as an executable, loads
it into on-chip SRAM and jumps into it. On-chip SRAM is small, so this
can only be a secondary bootstrap which will then fetch the real
executable into external memory and run it.

When booting from NAND the on-chip bootstrap simply reads the
executable from the start of the flash, it assumes the first block is
valid. Atmel supply a secondary bootstrap, snappily named
AT91Bootstrap, that will then load the main executable. This code has
to handle factory marked bad blocks. Starting at a fixed offset in the
NAND it reads a fixed number of blocks into external RAM, skipping
those that have a bad block mark.

So, here we already have two partitions in the NAND device, one for
the secondary bootstrap and one for the main executable. If the
executable is RedBoot, then it may want to store config data and
applications in the NAND itself. It may do this in a third partition
using YAFFS or UFFS, or a virtual V2/NOR flash driver. Finally, any
applications will need to use a filesystem, which for security and
reliability should probably be distinct from that used by RedBoot.

The size and position of the first two partitions is fixed by the
on-chip ROM and the secondary bootstrap. Bad block handling and
filesystem structure would make it very difficult to move or resize
any other partitions. So, once the partitioning of a NAND has been
decided on and initialized, it is fixed for good, much like disk
partitions.

I don't see much utility in having an on-chip partition table. NAND
chips, unlike disks, are not portable between different systems, so
there is little need for it to be self-describing. There is also no
standard for NAND partitioning, even in the Linux world where
partition tables seem to be either written into the driver, or are
supplied on the command line from the bootstrap. Linux compatibility
is desirable, if only to avoid reinventing the wheel, and we should
follow their lead, rather than try to invent something of our own.

For now, defining the partitioning of the NAND statically, via CDL, is
logically equivalent to the Linux approach of writing the table into
the driver. If a partitioning standard does emerge, then the hooks are
already present to make use of it.


> There has been comments that Rutgers code has too many layers. Rutger
> aims to allow as much code reuse between drivers as possible, which i
> think is good. Simon commented that he thinks Ross's design may result
> in a lot of code stuck in HALs where it is hard to reuse without copy
> paste. However Ross argued that he thinks that making code reusable is
> going to be hard, there is too many different configurations of chips
> and controllers, etc.
> 
> With only one supported platform on Ross's code and two with Rutgers,
> i think it is too early to tell the truth. However, im generally in
> favour of layers.

Just a few quick words about layering...

We have to be very careful in eCos not to over-do the abstractions.
eCos is still an embedded operating system and not a full-featured
general purpose OS. As such we must take care not to compromise code
size and performance. There are some areas in eCos already where we
have more layers than we strictly need, and performance has
suffered. Many of the target processors are relatively slow, without
caches or much fast memory and every extra call and indirection can
cost a lot of cycles.

The layering approach in the eCosCentric code is intended to keep the
layers to a minimum where possible. However, it doesn't preclude the
use of additional abstractions where necessary.

While the driver itself lives in the platform HAL, chip specifics are
#included, rather than going through a call/return interface. This
keeps the functional separation but reduces runtime overhead and
allows the compiler to optimize the code. The author of a chip driver
can always choose to move parts of the code to a separately compiled
source file if that makes more sense.

Equally, while the current LPC2468 driver is entirely in the platform
HAL, parts of this could be moved into common code in the variant HAL
if it can be shared. However, since it is not currently very clear
where that division of labour must go, this has not been done for this
initial driver. As further drivers are developed we will get a better
feel for where this line needs to be drawn.


I think in general an approach that keeps the system defined layering
to a minimum while allowing subsystems to abstract common code
internally gets the best of both worlds.



-- 
Nick Garnett                                       eCos Kernel Architect
eCosCentric Limited    http://www.eCosCentric.com       The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.      Tel: +44 1223 245571
Registered in England and Wales:                         Reg No: 4422071

Re: NAND review

[Rutger Hofman]

Nick Garnett wrote:
> Andrew Lunn <andrew@lunn.ch> writes:
> 
>> There has been comments that Rutgers code has too many layers. Rutger
>> aims to allow as much code reuse between drivers as possible, which i
>> think is good. Simon commented that he thinks Ross's design may result
>> in a lot of code stuck in HALs where it is hard to reuse without copy
>> paste. However Ross argued that he thinks that making code reusable is
>> going to be hard, there is too many different configurations of chips
>> and controllers, etc.
>>
>> With only one supported platform on Ross's code and two with Rutgers,
>> i think it is too early to tell the truth. However, im generally in
>> favour of layers.
> 
> Just a few quick words about layering...
> 
> We have to be very careful in eCos not to over-do the abstractions.
> eCos is still an embedded operating system and not a full-featured
> general purpose OS. As such we must take care not to compromise code
> size and performance. There are some areas in eCos already where we
> have more layers than we strictly need, and performance has
> suffered. Many of the target processors are relatively slow, without
> caches or much fast memory and every extra call and indirection can
> cost a lot of cycles.

In my NAND package, there are indirect calls to the device-specific code 
of the controller. I followed the examples in the eCos documentation, 
where serial line, Ethernet etc all use this approach. Per NAND page 
that is written (typically 2KB or 4KB) there are a few indirect calls. 
My unsubstantiated intuition is that the data transfers would dominate 
performance compared to a few indirections. There are also a few 
indirect calls in the chip module, but these are used at initialization 
time only. Still, this indirection is only needed for heterogeneous 
controllers.

Besides the layering issue, there is another thing that I would like to 
point out. My NAND package has support for 3 types of NAND chip: 
small-page (the old type), 'regular' large-page (that's what you buy 
today), ONFI (I think this still has to hit the market). Small-page has 
its own instruction set, and has interrogation protocol worth 
mentioning. 'regular' and ONFI largely share the instruction set but 
differ completely in the interrogation protocol.

For a specific build, it would be not hard at all to compile only the 
support for the type of chip that is actually used, and to /not/ compile 
command implementations and interrogation for the other types.

This same observation would hold for the ECC routines, and, as I 
mentioned in an earlier email, for the code that hides away controller 
or chip heterogeneity.

Rutger

Re: NAND review

[Andrew Lunn]

> It is sometimes best to think about a concrete example, and my
> thoughts on NAND use were informed by thinking about how to use the
> NAND on an Atmel AT91SAM9 based device. 

I agree with the sentiment. Lets take another couple of examples.

Both Rutger and Simon have NOR flash to boot from. They will put one
filesystem onto the NAND device. For them partitioning is just
bloat. I think that should be a strong argument for having an API
which does not enforce the use of partitions.

http://pokylinux.org/releases/pinky-3.1/readme.hardware

The freescale ADS section talks about using NAND devices and putting
the partition table in the FIS directory. 

I found other freescale extensions for Redboot, but most information
seems to require that you register. However it looks like Freescale
have a redboot which fully integrated NAND devices into FIS and FIS
can be used to partition the flash.

Your argument about the HAL knowing about the boot block sections
makes a lot of sense. However i think we need an architecture where
additional partition information can come from other places. FIS, or a
NandFIS would be one such other place. We also need to think about how
this information is exported, eg into file systems, to Linux, and to
RAM versions of eCos running on top of ROM or ROMRAM Redboot etc.

    Andrew

           

Re: NAND review

[Nick Garnett]

Andrew Lunn <andrew@lunn.ch> writes:

> > It is sometimes best to think about a concrete example, and my
> > thoughts on NAND use were informed by thinking about how to use the
> > NAND on an Atmel AT91SAM9 based device. 
> 
> I agree with the sentiment. Lets take another couple of examples.
> 
> Both Rutger and Simon have NOR flash to boot from. They will put one
> filesystem onto the NAND device. For them partitioning is just
> bloat. I think that should be a strong argument for having an API
> which does not enforce the use of partitions.


The disk drive interface attempts to hide the partitioning behind a
conventional driver interface and is forced to take a somewhat
tortuous approach. I think trying to do the same in the NAND layer
would be similarly tortuous.

I believe that an API that is partition-aware from the start is
preferable. I don't believe the code or data structures needed to
support the partitions are particularly onerous. However, perhaps
there is scope for making some of this more conditional if only one
partition is expected.

> 
> http://pokylinux.org/releases/pinky-3.1/readme.hardware
> 
> The freescale ADS section talks about using NAND devices and putting
> the partition table in the FIS directory.
> 
> I found other freescale extensions for Redboot, but most information
> seems to require that you register. However it looks like Freescale
> have a redboot which fully integrated NAND devices into FIS and FIS
> can be used to partition the flash.

It is not entirely clear exactly what they are doing there. Does their
flash driver handle wear levelling and bad blocks?

Our approach is that a NOR flash emulation driver would operate within
a partition of the NAND, rather than cover the whole thing. I am
unconvinced that FIS is the right thing to use for partitioning a NAND
flash.

> 
> Your argument about the HAL knowing about the boot block sections
> makes a lot of sense. However i think we need an architecture where
> additional partition information can come from other places. FIS, or a
> NandFIS would be one such other place. We also need to think about how
> this information is exported, eg into file systems, to Linux, and to
> RAM versions of eCos running on top of ROM or ROMRAM Redboot etc.

My main concern is that we avoid reinventing the wheel, or end up
inventing a square wheel. Looking at the more mature Linux NAND
support, no consensus seems to have emerged over NAND partitioning. A
very small number of platforms seem to use FIS in some form, but
others just seem to define the partitions statically. I'm wary of
jumping in here and defining something of our own, just for the sake
of it. I would prefer a more cautious approach, keeping to a static
approach until a consensus emerges. 

-- 
Nick Garnett                                       eCos Kernel Architect
eCosCentric Limited    http://www.eCosCentric.com       The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.      Tel: +44 1223 245571
Registered in England and Wales:                         Reg No: 4422071

Re: NAND review

[Andrew Lunn]

On Thu, Jun 18, 2009 at 03:10:12PM +0100, Nick Garnett wrote:
> Andrew Lunn <andrew@lunn.ch> writes:
> 
> > > It is sometimes best to think about a concrete example, and my
> > > thoughts on NAND use were informed by thinking about how to use the
> > > NAND on an Atmel AT91SAM9 based device. 
> > 
> > I agree with the sentiment. Lets take another couple of examples.
> > 
> > Both Rutger and Simon have NOR flash to boot from. They will put one
> > filesystem onto the NAND device. For them partitioning is just
> > bloat. I think that should be a strong argument for having an API
> > which does not enforce the use of partitions.
> 
> 
> The disk drive interface attempts to hide the partitioning behind a
> conventional driver interface and is forced to take a somewhat
> tortuous approach. I think trying to do the same in the NAND layer
> would be similarly tortuous.

We don't need to follow the disk driver approach to partitioning. As
Ross said, he used the name "partition" because he could not think of
a better name. However the name does not imply we need to follow the
normal ways for partitions. All we need is a suitable API which is
efficient when used with partitions and without partitions.

> I believe that an API that is partition-aware from the start is
> preferable.

and i prefer an API which works well with and without partitions. 

> I don't believe the code or data structures needed to
> support the partitions are particularly onerous. However, perhaps
> there is scope for making some of this more conditional if only one
> partition is expected.

Looking at the existing use cases we know about, the systems which
don't need partitions are the ones with the most resource
constrains. Simon does not need partitions, he plans to use a very
lean and mean FS, and only has a small amount of ROM and RAM. With
Ross's API there is the partitions data structure in the HAL, the code
to get this out of the HAL and into the flash library, the extra
function call needed to get a partition handle, and then the checks
used to validate addresses are inside the partition. I've no idea how
much this adds up to in terms of code and data, so i've no idea how
big the overhead is.

> It is not entirely clear exactly what they are doing there. Does their
> flash driver handle wear levelling and bad blocks?
> 
> Our approach is that a NOR flash emulation driver would operate within
> a partition of the NAND, rather than cover the whole thing. I am
> unconvinced that FIS is the right thing to use for partitioning a NAND
> flash.

Lets thinks of a few use cases. These are somewhat hypothetical, since
i don't actually have any NAND device in my hands, but hopefully they
are realistic.

Category 1: Getting Linux booted

I see two layouts here:

1) A dedicated partition which contains the kernel and then addition
partitions for the root FS and maybe other FSs.

2) No dedicated kernel partition, the kernel is in /boot on the root
FS.

Having the kernel separate could make it easier/faster to update
during development, since you don't need to touch the rootfs. And the
same is true the other way around, you can replace the rootfs without
touching your known good kernel.

If you have a dedicates partition, sized about right for the kernel,
level wearing brings you nothing. There are no hot blocks in the
kernel image, you replace it all every time. And how often do you
replace the kernel? During development work, maybe quite often, but
once the system is deployed, rarely. What is the typical life of a
block? 10000 erases? A developer may conceivably replace the kernel
10,000 times, but in a deployed product i doubt this will happen. So i
say forget about wear leveling for this case.

Bad block are important. But a simple linear log structure should be
sufficient. When reading/writing if the BBT says the block is good,
use it, if it is bad, skip it. I'm assuming here that a NAND block
goes bad because of writes and erases, not reads and age. So once the
image has been successfully written, it should be readable forever.

The rootfs is a more interesting issue. Linux is very unlikely to boot
into a useful system without its rootfs. I see two options here. You
could tftp boot using a initramfs loaded to RAM to boot into linux
with a minimal system to perform a rootfs on NAND installation. Or
redboot has to somehow put the rootfs into the NAND partition. This
leads to three things:

1) Full read/write support for the FS in Redboot. For NOR we normally
   have only read support.

2) mkfs functionality.

3) Redboot needs to support some simple archive format, cpio, tar,
   etc, so you can upload the rootfs as an archive and unpack it onto
   the filesystem. And given that the NAND is likely to be bigger than
   RAM, maybe this has to be done as a stream.

Unlike NOR, you cannot write a filesystem image to a NAND
partition. So you need to write individual files, using the file
system code itself. You also need to be able to create the empty file
system, so mkfs is needed.

Once you have redboot with mkfs support, you have all you need to
support dynamic partition sizes. Its then just of question of having
somewhere to store this dynamic information, and FIS or NandFIS seems
the obvious candidate.

Category 2: Getting eCos images booted

The same two layouts as for Linux make sense here. 

One difference is that an eCos image could do something useful without
a "rootfs". The mkfs code could be in the eCos image, so that on first
boot it detects the filesystem is invalid and formats it. This would
allow a smaller redboot image, when using the first layout, since no
filesystem support is needed, just the simple linear log structure.
Also, dynamic partitions is again possible, the mkfs code has to be
somewhere, either in the eCos application image or in Redboot.


For me, dynamic partitions make sense. I'm not saying that release 1.0
needs to have them, we just need to make sure that the design of
release 1.0 is sufficiently flexible that adding dynamic partitions is
just a release 1.1 feature and not a rip the guts out and start again
for release 2.0.

    Andrew

Re: NAND review

[Andrew Lunn]

> My main concern is that we avoid reinventing the wheel, or end up
> inventing a square wheel. Looking at the more mature Linux NAND
> support, no consensus seems to have emerged over NAND partitioning.

Maybe the correct way to approach this is to make a design study,
power on till Linux has its file systems mounted. Figure out all the
different stages and what code is needed where. What must the HAL
provide, the flash library, eCos filesystems, Redboot, passing command
line arguments to Linux or an MTD module which gets the partition
information from NandFIS etc. Maybe see if David Woodhouse is
interested in helping since he knows the MTD side of things and is the
official embedded linux maintainer.

Write an RFC and cross post it to ecos-devel and LKML. Collect
comments and go around the loop a few times. Then do the development
work.

This should help avoid reinventing the wheel and hopefully make it
cornerles and symmetric. So long as you follow the Linux development
process, post often and early, and don't get flamed to a crisp it
could be good advertising for eCosCentric.

      Andrew

Re: NAND review

[Daniel Morris]

On Fri, Jun 19, 2009 at 10:07:47AM +0200, Andrew Lunn wrote:
> This should help avoid reinventing the wheel and hopefully make it
> cornerles and symmetric. So long as you follow the Linux development
> process, post often and early, and don't get flamed to a crisp it
> could be good advertising for eCosCentric.
> 

Did you miss the smiley off? ;-) I know that we've been too bashful and
consistently self-deprecating to raise this issue, but is interesting
that you've thought of what advertising benefit contributions make. From
a business perspective, I can see the harm that the lack of credit and
acknowledgement of contributions to eCos has made, limiting the
incentive for donations. 

Take a look at the 3.0 release announcement and pick just one example.
Now I am acutely aware of how many man-months/years eCosCentric put into
producing a coherent toolchain across the multiple host and target
architectures to help the community, to the point of turning away
business whilst we had engineers and resources working on this - but one
struggles to find reference to this effort without trawling 534
ChangeLogs or dredging threads in disparate mail-lists or examining each
commit.

eCosCentric's view has been to shrug our shoulders, ignore the trolls,
hesitate to criticise and pretty much carry on regardless, making well
reasoned contributions and at the same time building a sustainable
business so that we can lather, rinse and repeat. There's lots of stuff
that we just get on with, in the background, that is unlikely ever to
bring in any publicity, but is pivotal to the ongoing success of eCos.
However, I know from talking with engineers, managers and cohorts of
lawyers that the perceived value (and even risk) of a contribution is
often measured according to the more accessible headlines and
soundbites, rather than unrecognised deep and broad engineering. 

Perhaps making our own improvements would have greater, more immediate
and measurable benefits to the eCos community than hoping to piggy-back
on Linux's shirt-tails through guerilla marketing? ;-)

 Daniel


%<----------------------------------------------------------------------
  Daniel Morris - Sales & Marketing Director
  eCosCentric - The eCos and RedBoot experts
  Tel: +44 1223 245 571 - info@eCosCentric.com
  DDI: +44 1269 591 171 - danielm@eCosCentric.com

Re: NAND review

[Andrew Lunn]

On Fri, Jun 19, 2009 at 12:36:57PM +0100, Daniel Morris wrote:
> On Fri, Jun 19, 2009 at 10:07:47AM +0200, Andrew Lunn wrote:
> > This should help avoid reinventing the wheel and hopefully make it
> > cornerles and symmetric. So long as you follow the Linux development
> > process, post often and early, and don't get flamed to a crisp it
> > could be good advertising for eCosCentric.
> > 
> 
> Did you miss the smiley off? ;-) I know that we've been too bashful and
> consistently self-deprecating to raise this issue, but is interesting
> that you've thought of what advertising benefit contributions make. 

Well there are other advantages this route offers:

1) There is _one_true_way_ to get Linux booted from a NAND device
   using RedBoot. This means when some custom comes to you asking for
   a Redboot port, you don't need to look at the FreeScale extensions
   to RedBoot, the Samsung extensions to RedBoot, the MobLin
   extensions to RedBoot etc, you just do the _one_true_way_, which
   you have already done N times, wash, rinse and repeat...

2) It is eCosCentric code, you know it backwards/upside down and where
   the warts are, so allowing you to do a port to a new device quicker
   and cheaper than if it was somebody else code.

3) The Linux parts are in mainstream. This is always the recommended
   way to do it, compared to being out of tree. So that means less
   work for you every time there is a new kernel release.

   Andrew

Re: NAND review

[Ross Younger]

> Nick Garnett wrote:
>> I believe that an API that is partition-aware from the start is
>> preferable.

Andrew Lunn wrote:
> and i prefer an API which works well with and without partitions. 

The partitions are already a pretty slim layer on top of the main code; I
don't think that thinning things down would save very much. Still, it would
be little work for me to fillet out the API so that I provided two forms of
each of the main entrypoints, one which takes a device* and the other a
partition*. They probably ought to be switchable in and out by CDL, and I
suspect not both present at once. (Naturally, RedBoot and YAFFS would DTRT,
whatever was present.) If this sounds like a welcome move, I'll gladly add
it to my todo list.


> Category 1: Getting Linux booted
>
> 1) A dedicated partition which contains the kernel and then addition
> partitions for the root FS and maybe other FSs.
> 2) No dedicated kernel partition, the kernel is in /boot on the root
> FS.
> 
> Having the kernel separate could make it easier/faster to update
> during development, since you don't need to touch the rootfs. And the
> same is true the other way around, you can replace the rootfs without
> touching your known good kernel.

I'd personally steer people towards a dedicated kernel partition. RedBoot
has a small heap by default, and a full rootfs would necessarily have quite
a complex journal to replay which would take a lot of it, not to mention
take longer. Having the two partitions separate also vaguely allows the
possibility of reformatting the rootfs in the field without having to also
upgrade your boot loader at the same time, which sounds to me like a risky
"don't turn off the power or you'll brick the device" sort of operation.

> What is the typical life of a
> block? 10000 erases? A developer may conceivably replace the kernel
> 10,000 times, but in a deployed product i doubt this will happen. So i
> say forget about wear leveling for this case.

I have been thinking about lifespan and how likely we are to come across
worn-bad blocks. The Samsung chip on the EA 2468 board specifies 100k
write/erase cycles per block and data retention of at least ten years, which
seems to be pretty typical for NAND parts, and comparable to most NOR parts?

> I'm assuming here that a NAND block
> goes bad because of writes and erases, not reads and age. So once the
> image has been successfully written, it should be readable forever.

This is my understanding as well.

> Bad block are important. But a simple linear log structure should be
> sufficient. When reading/writing if the BBT says the block is good,
> use it, if it is bad, skip it. 

Properly NAND-aware filesystems take both kinds of bad blocks in their
stride; there's no point in adding the complexity if such a mapping if
you're using such a filesystem.

Even with something simpler, factory-bad blocks are straightforward: as you
say, one can simply linear-map them into oblivion in the driver at the cost
of extra time complexity *on every NAND operation*. (I add the emphasis
because I expect the overhead will add up fast if the NAND is heavily used.)

Worn-bad blocks are harder; you can't ever add one to the mapping, because
you'd have changed the in-flash address of everything above it.

I've been thinking about this sort of question in the context of a simple
layer to make NAND look like NOR for RedBoot, and have had an interesting
idea which I'm going to propose on this list when I've written it up.


> The rootfs is a more interesting issue. Linux is very unlikely to boot
> into a useful system without its rootfs. I see two options here. You
> could tftp boot using a initramfs loaded to RAM to boot into linux
> with a minimal system to perform a rootfs on NAND installation. Or
> redboot has to somehow put the rootfs into the NAND partition. 

You could also embed your entire rootfs into the kernel image as a cramfs.


> This leads to three things:
> 
> 1) Full read/write support for the FS in Redboot. For NOR we normally
>    have only read support.

Do you mean jffs2?

At the moment my YAFFS layer is fully read-write, but I could add a
read-only switch if this was thought to be useful?

> 2) mkfs functionality.

I don't know about other filesystems, but YAFFS always scans the NAND on
mount, and automatically does the Right Thing when asked to mount a
fully-erased array. (My RedBoot patch already includes logic to erase a nand
partition.)

> 3) Redboot needs to support some simple archive format, cpio, tar,
>    etc, so you can upload the rootfs as an archive and unpack it onto
>    the filesystem. And given that the NAND is likely to be bigger than
>    RAM, maybe this has to be done as a stream.
> 
> Unlike NOR, you cannot write a filesystem image to a NAND
> partition. So you need to write individual files, using the file
> system code itself. You also need to be able to create the empty file
> system, so mkfs is needed.

The original YAFFS tree includes a mkyaffs2image utility for turning a
directory tree into an image file which can then be programmed to NAND with
suitable software. (Being log-based, I suppose bad blocks wouldn't matter.)
I haven't looked into this, but it smells pretty straightforward to add
support to RedBoot to program such an image into NAND - much nicer than
teaching it how to unpack (say) a tarball.


> Once you have redboot with mkfs support, you have all you need to
> support dynamic partition sizes. Its then just of question of having
> somewhere to store this dynamic information, and FIS or NandFIS seems
> the obvious candidate.

Dynamic partitions are conceptually not very far from the current model. The
partition "table" is just an array hanging off the cyg_nand_device struct,
which could in theory be modified by any code which chose to - either from
logic, or interactively e.g. from RB. The interesting part is deciding where
and how the table is to be stored, which influences how we would provide a
call to write out a fresh partition table and whether this was accessible
via a hosted Linux partition. (We'd also have to think carefully what it
meant to change the table at runtime; what if there were mounted
filesystems? open files? Would (could) we make it safe?)


Ross

-- 
Embedded Software Engineer, eCosCentric Limited.
Barnwell House, Barnwell Drive, Cambridge CB5 8UU, UK.
Registered in England no. 4422071.                  www.ecoscentric.com

Re: NAND review

[Andrew Lunn]

> Properly NAND-aware filesystems take both kinds of bad blocks in their
> stride; there's no point in adding the complexity if such a mapping if
> you're using such a filesystem.
> 
> Even with something simpler, factory-bad blocks are straightforward: as you
> say, one can simply linear-map them into oblivion in the driver at the cost
> of extra time complexity *on every NAND operation*. (I add the emphasis
> because I expect the overhead will add up fast if the NAND is heavily used.)

I'm not suggesting putting this in the driver. 

I'm suggesting RedBoot commands nandfis load and nandfis create
does this.

As you said, a full NAND filesystem will handle bad blocks etc.

However, just to get an executable image, be it the Linux kernel, or
an eCos App, into memory we don't necessarily need the overhead of a
NAND filesystem. 

> > This leads to three things:
> > 
> > 1) Full read/write support for the FS in Redboot. For NOR we normally
> >    have only read support.
> 
> Do you mean jffs2?

jffs2 is i think by default, and the ext2 code is definitely read
only.
 
> At the moment my YAFFS layer is fully read-write, but I could add a
> read-only switch if this was thought to be useful?

It might be, if it saves code/data space.

> > 2) mkfs functionality.
> 
> I don't know about other filesystems, but YAFFS always scans the NAND on
> mount, and automatically does the Right Thing when asked to mount a
> fully-erased array. (My RedBoot patch already includes logic to erase a nand
> partition.)

Given the unfriendly license of YAFFS, i think uffs is more likely to
be the defacto standard NAND filesystem for eCos based systems, once
it gets ported and committed into the anoncvs tree. I've no idea what
it needs for mkfs like operations. 

YAFFS makes more sense for Linux, but i also expect there will be
pressure to add UBI/UBIFS support, since that seems to be the defacto
standard mainline NAND filesystem now, replacing JFFS2. 
 
> The original YAFFS tree includes a mkyaffs2image utility for turning a
> directory tree into an image file which can then be programmed to NAND with
> suitable software. (Being log-based, I suppose bad blocks wouldn't matter.)
> I haven't looked into this, but it smells pretty straightforward to add
> support to RedBoot to program such an image into NAND - much nicer than
> teaching it how to unpack (say) a tarball.

The problem with mkyaffs2image is that it is specific to yaffs. What
about uffs and UBIFS and even jffs2, etc. Supporting a simple archive
format, like cpio, is a much more generic solution.

> Dynamic partitions are conceptually not very far from the current model. The
> partition "table" is just an array hanging off the cyg_nand_device struct,
> which could in theory be modified by any code which chose to - either from
> logic, or interactively e.g. from RB. The interesting part is deciding where
> and how the table is to be stored, which influences how we would provide a
> call to write out a fresh partition table and whether this was accessible
> via a hosted Linux partition. (We'd also have to think carefully what it
> meant to change the table at runtime; what if there were mounted
> filesystems? open files? Would (could) we make it safe?)

This is the sort of discussion i want to see. How would Redboot do
this? What are the APIs? How does an eCos App get this information via
VV? How many times have i asked these questions?

You say not very far from the current model, so lets get is right now,
rather than have to break the world when we fix it later.

Getting the locking right would be nice, but we currently don't do any
locking with JFFS2. You can fis delete etc, when the filesystem is
mounted. Bad things then happen. 

         Andrew

Re: Re: NAND review

[Jürgen Lambrecht]

Ross Younger wrote:
>> Nick Garnett wrote:
>>     
<snip>
>> What is the typical life of a
>> block? 10000 erases? A developer may conceivably replace the kernel
>> 10,000 times, but in a deployed product i doubt this will happen. So i
>> say forget about wear leveling for this case.
>>     
>
> I have been thinking about lifespan and how likely we are to come across
> worn-bad blocks. The Samsung chip on the EA 2468 board specifies 100k
> write/erase cycles per block and data retention of at least ten years, which
> seems to be pretty typical for NAND parts, and comparable to most NOR parts?
>   
yes, but 20 years data retention (Spansion NOR).
>   
>> I'm assuming here that a NAND block
>> goes bad because of writes and erases, not reads and age. So once the
>> image has been successfully written, it should be readable forever.
>>     
>
> This is my understanding as well.
>   
mine too.
>   
>> Bad block are important. But a simple linear log structure should be
>> sufficient. When reading/writing if the BBT says the block is good,
>> use it, if it is bad, skip it.
>>     
>
> Properly NAND-aware filesystems take both kinds of bad blocks in their
> stride; there's no point in adding the complexity if such a mapping if
> you're using such a filesystem.
>
> Even with something simpler, factory-bad blocks are straightforward: as you
> say, one can simply linear-map them into oblivion in the driver at the cost
> of extra time complexity *on every NAND operation*. (I add the emphasis
> because I expect the overhead will add up fast if the NAND is heavily used.)
>
> Worn-bad blocks are harder; you can't ever add one to the mapping, because
> you'd have changed the in-flash address of everything above it.
>
> I've been thinking about this sort of question in the context of a simple
> layer to make NAND look like NOR for RedBoot, and have had an interesting
> idea which I'm going to propose on this list when I've written it up.
>
>
>   
>> The rootfs is a more interesting issue. Linux is very unlikely to boot
>> into a useful system without its rootfs. I see two options here. You
>> could tftp boot using a initramfs loaded to RAM to boot into linux
>> with a minimal system to perform a rootfs on NAND installation. Or
>> redboot has to somehow put the rootfs into the NAND partition.
>>     
>
> You could also embed your entire rootfs into the kernel image as a cramfs.
>
>
>   
>> This leads to three things:
>>
>> 1) Full read/write support for the FS in Redboot. For NOR we normally
>>    have only read support.
>>     
>
> Do you mean jffs2?
>
> At the moment my YAFFS layer is fully read-write, but I could add a
> read-only switch if this was thought to be useful?
>
>   
>> 2) mkfs functionality.
>>     
>
> I don't know about other filesystems, but YAFFS always scans the NAND on
> mount, and automatically does the Right Thing when asked to mount a
> fully-erased array. (My RedBoot patch already includes logic to erase a nand
> partition.)
>
>   
The same with JFFS2 (after a bugfix in my case).

Kind regards,
Juergen

Re: NAND review

[Nick Garnett]

Andrew Lunn <andrew@lunn.ch> writes:

> We don't need to follow the disk driver approach to partitioning. As
> Ross said, he used the name "partition" because he could not think of
> a better name. However the name does not imply we need to follow the
> normal ways for partitions. All we need is a suitable API which is
> efficient when used with partitions and without partitions.

I wasn't suggesting we follow the disk driver approach. In fact I was
using it as an example of an approach that we should not use.

> 
> > I believe that an API that is partition-aware from the start is
> > preferable.
> 
> and i prefer an API which works well with and without partitions. 

Unless I have misunderstood, you appear to be criticizing the
eCosCentric design because it makes partitioning an explicit part of
the API. My point was that any API that attempts to conceal the
presence of partitions will quickly tie iteslf in knots when the need
for such things arises. I don't believe that a partition-aware API is
particularly high overhead, a handful of extra instructions per API
routine and a few extra words per data structure. With a little
judicious ifdeffery, it can be almost totally eliminated for the
single partition case.

To paraphrase Einstein: We should make an API as simple a possible,
but not simpler than it needs to be.

> Looking at the existing use cases we know about, the systems which
> don't need partitions are the ones with the most resource
> constrains. Simon does not need partitions, he plans to use a very
> lean and mean FS, and only has a small amount of ROM and RAM. With
> Ross's API there is the partitions data structure in the HAL, the code
> to get this out of the HAL and into the flash library, the extra
> function call needed to get a partition handle, and then the checks
> used to validate addresses are inside the partition. I've no idea how
> much this adds up to in terms of code and data, so i've no idea how
> big the overhead is.

My contention is that this overhead would be minimal if the NAND
system could be configured into a single partition mode. Then, a lot
of these checks get collapsed to nothing in the source, or get
optimized away by the compiler.

> If you have a dedicates partition, sized about right for the kernel,
> level wearing brings you nothing. There are no hot blocks in the
> kernel image, you replace it all every time. And how often do you
> replace the kernel? During development work, maybe quite often, but
> once the system is deployed, rarely. What is the typical life of a
> block? 10000 erases? A developer may conceivably replace the kernel
> 10,000 times, but in a deployed product i doubt this will happen. So i
> say forget about wear leveling for this case.

Exact figures for NAND reliability are hard to find. The 100,000 erase
cycle guarantee often only applies to the first block. My
understanding of NAND is that the design of the cells makes them less
reliable that NOR cells. Any cell may go bad, or read back invalid, at
any time, not just after some predefined number of erase cycles,
albeit with lower probability early in the device's lifetime. So
software must keep CRCs and handle bad blocks. The manufacturers
exploit this existing requirement to raise yield by allowing parts
with factory bad blocks to be sold to customers.

So, while most of the time you can get away with a simple approach,
every so often you will encounter a device that has cells that go bad
very quickly. Unless you have a way of working around this, you may
end up with an unusable board.

> 
> For me, dynamic partitions make sense. I'm not saying that release 1.0
> needs to have them, we just need to make sure that the design of
> release 1.0 is sufficiently flexible that adding dynamic partitions is
> just a release 1.1 feature and not a rip the guts out and start again
> for release 2.0.

I'm less convinced that dynamic partitions are necessary or useful. I
don't believe YAFFS, or any other NAND filesystem, can handle its
partition being resized or relocated and having the positions of bad
blocks change under it feet. The same would apply to any other piece
of software that was NAND aware.

Without an existing common mechanism for defining partitions in Linux,
we have to fall into line with what happens there. The majority of
drivers seem to define fixed static blocks. While we could come up
with a definition of a partitioning scheme and promote it in the Linux
world, experience suggests that this would be something of an uphill
struggle. I know people who have spent years trying to get stuff
accepted.

Your final point is exactly the one I have been trying to make. We
should not be jumping in and specifying partitioning mechanisms right
now. I think we need to gather more experience with NAND before we do
that.  Ross's design permits any sort of partioning mechanism to be
defined; at present only one, a CDL-based static partitioning, is
implemented. I also believe that by making partioning explicit in the
API, we keep the design flexible for the future.

-- 
Nick Garnett                                       eCos Kernel Architect
eCosCentric Limited    http://www.eCosCentric.com       The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.      Tel: +44 1223 245571
Registered in England and Wales:                         Reg No: 4422071