[ECOS] NAND flash driver considerations: RFC

[ECOS] NAND flash driver considerations: RFC

[Rutger Hofman]

I am writing a NAND flash driver for eCos. I would like to get reactions 
and advice on some issues, so I can correct now while I am still coding 
(and my guess is, this won't be the last time).


NAND internal interfaces
------------------------

NAND flash is actually two devices: the flash chip(s), and the 
controller, often a separate part, sometimes integrated into a CPU (as 
with my BlackFin BF54x).
The chips seem to have a fairly uniform interface, as canonicalized by 
ONFI (http://www.onfi.org). All chips have lines for 'chip enable', 
'select command', 'select address', 'r/w', 'data bus (x8 or x16)', 
'status', which are handled in a standard way.
Consequently, any controller has a very good chance of being able to 
speak to any chip, in hardware terms.
Moreover, the command set is mostly (a subset of) ONFI.
Deviations from the ONFI standard seem to be centered in the command 
that interrogates the chip about its properties. #LUNs, #blocks/LUN, 
#pages/block, data/page, spare size, bus width (x8 or x16), support for 
optional commands, and timing characteristics can typically be obtained 
either in ONFI speak (the 'READ PARAMETER BLOCK' command) or with a 
manufacturer-specific command.

The NAND flash controllers vary in their CPU interface. They may have 
registers that reflect the basic capabilities of the chip interface 
('command', 'address', 'data', 'status'). Or they may solve that 
differently, in some serial way or whatever.

+++ My proposal:

= define a NAND controller /interface/ that specifies functions for the 
chip capabilities ('command', 'address', 'data', 'status'); any NAND 
controller /implementation/ must define them.
(See below: on top of that NAND controller interface, generic NAND code 
provides the functions specified by ONFI to read/program (partial) pages 
incl. spare erea, erase blocks, etc.)

= as far as the chip is concerned: most of the parameters are found 
through chip interrogation, only very few parameters remain to be 
specified in a .cdl or a chip .h file: bus width?, maybe some timings; 
the src for the chip must contain any routines that deviate from ONFI, 
where the device interrogation routine may be an obvious candidate. (If 
a chip doesn't support this kind of interrogation, or if it must be 
overridden, well, it must go into the chip .cdl after all.)


eCos directory structure
------------------------

Should NAND be grouped under flash in packages/io and packages/devs? To 
be honest, the interface under which NAND flash is approached only 
vaguely resembles that for NOR flash. When we would choose for 
flash/nand, then we should also have to have flash/nor/ and move all 
things currently flashing down there.

+++ My proposal:

have packages/io/nand/ and packages/devs/nand/* *beside* packages/flash/ 
and packages/devs/flash/*. They are very different. Also, who wants 
refactoring of flash/* into flash/nor/*.
Is there a case to be made for separate nand/controller/ and nand/chip/ 
trees?


NAND eCos device type
----------------

NAND flash is neither a character device (CHAR_DEVTAB_ENTRY), nor a 
block device (BLOCK_DEVTAB_ENTRY), nor a net device (NET_DEVTAB_ENTRY), 
nor a flash device in the sense of the eCos device driver structure.
NAND:
- must be erased in 'blocks'
- must be programmed/read in (partial) pages (there usually is a good 
number of pages per block)
- has a 'spare' area for each page (to store ECCs, bad block info, and 
client, e.g. file system info); spare is to be handled explicitly
- has single-bit failures, corrected by ECCs
- has bad blocks
- and may have a copy-back function to relocate data within the flash chip.
NAND controllers may calculate ECCs in hardware on the fly.

So, should we copy/paste one of those many device driver infrastructures 
and modify them to accomodate the NAND flash devices?
eCos NOR flash has solved this in its own way: it does device lookup 
based on the addresses that the NOR flash is memory-mapped to. But NAND 
flash is *not* memory-mapped. Besides, NAND flashes can/will exceed 4GB 
in size, so they cannot be addressed with 32bits.

+++ My proposal:

= I would rather create NAND devices as just DEVTAB_ENTRYs, and mark 
them as NAND (or OTHER or ...) i.s.o. CHAR/BLOCK type. The function 
dispatch block ptr can be cast, or it can be put in the private area. 
This way one can do device lookup in the normal way with cyg_io_lookup() 
("/dev/nand0" resolves to a device handle).

= I propose to create two separate device types: NAND controllers and 
NAND chips. The two device types might be designated as "/dev/nand0" for 
controller 0, and "/dev/nand0.2" for the third chip connected to 
controller 0.

I agree that this is kind of broken. But I have trouble thinking of a 
way to do device lookup otherwise. Besides, we might consider 
generalization of the device structure in the sense that the functions 
block is no longer strictly typed. Its actual type might be derived from 
the device driver type field.


External interface of NAND devices (controller on top of chip)
--------------------------------------------------------------

+++ My proposal:
the ONFI functions exported by the generic NAND controller code are 
sufficient.

These are:
- lookup/init; a controller must initialize any chips it owns, locating 
them e.g. by their names ("/dev/nand0" owns "/dev/nand0.0" .. 
"/dev/nand0.X")
- programming of a (partial) page incl spare
- reading of a (partial) page incl spare, optionally exporting the 
controller's calculated ECC
- erase of a block
- marking block as bad
- querying block badness
- retrieving initial bad block table

To be extended later:
- reading/programming of multiple pages ?within a block
- copyback
- interleaved support

The spare area will usually have fields for ECC values, either 
calculated by the layer above, or by a controller that supports it.

So, at the NAND flash layer, there is no attempt to hide bad blocks, 
spare areas, ECC handling. The layer above (file system, 'contiguous' 
programmer) must handle that.

This exactly fits the YAFFS2 interface (sorry, no idea about JFFS2, but 
I'd hope so).

I don't know yet how 'contiguous' programming should work. Typical 
application: the boot loader code. For NAND flash booting, typically a 
first-level bootstrap is read from block 0 (guaranteed to be 'good'). 
This contains code to copy the rest of the bootloader code from NAND 
flash to RAM, while taking in consideration any bad blocks. Hmmm... 
would there be any interest in this? As for me, I am currently just 
interested in getting YAFFS2 alive.


Well, I hope that this is understandable, and sorry that it is so much.

Looking forward to your comments,

Rutger Hofman
VU Amsterdam

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] Re: NAND flash driver considerations: RFC

[Rutger Hofman]

I have received no responses for this RFC.

Wouldn't anybody care to comment?

Or do you think it is more useful if I just present my proposed NAND 
controller interfaces (public and controller-specific), my proposed NAND 
chip interfaces, and my proposed .cdl config setup? For sure that is 
some stages later in the design.

Rutger

Rutger Hofman wrote:
> I am writing a NAND flash driver for eCos. I would like to get reactions 
> and advice on some issues, so I can correct now while I am still coding 
> (and my guess is, this won't be the last time).
> 
> 
> NAND internal interfaces
> ------------------------
> 
> NAND flash is actually two devices: the flash chip(s), and the 
> controller, often a separate part, sometimes integrated into a CPU (as 
> with my BlackFin BF54x).
> The chips seem to have a fairly uniform interface, as canonicalized by 
> ONFI (http://www.onfi.org). All chips have lines for 'chip enable', 
> 'select command', 'select address', 'r/w', 'data bus (x8 or x16)', 
> 'status', which are handled in a standard way.
> Consequently, any controller has a very good chance of being able to 
> speak to any chip, in hardware terms.
> Moreover, the command set is mostly (a subset of) ONFI.
> Deviations from the ONFI standard seem to be centered in the command 
> that interrogates the chip about its properties. #LUNs, #blocks/LUN, 
> #pages/block, data/page, spare size, bus width (x8 or x16), support for 
> optional commands, and timing characteristics can typically be obtained 
> either in ONFI speak (the 'READ PARAMETER BLOCK' command) or with a 
> manufacturer-specific command.
> 
> The NAND flash controllers vary in their CPU interface. They may have 
> registers that reflect the basic capabilities of the chip interface 
> ('command', 'address', 'data', 'status'). Or they may solve that 
> differently, in some serial way or whatever.
> 
> +++ My proposal:
> 
> = define a NAND controller /interface/ that specifies functions for the 
> chip capabilities ('command', 'address', 'data', 'status'); any NAND 
> controller /implementation/ must define them.
> (See below: on top of that NAND controller interface, generic NAND code 
> provides the functions specified by ONFI to read/program (partial) pages 
> incl. spare erea, erase blocks, etc.)
> 
> = as far as the chip is concerned: most of the parameters are found 
> through chip interrogation, only very few parameters remain to be 
> specified in a .cdl or a chip .h file: bus width?, maybe some timings; 
> the src for the chip must contain any routines that deviate from ONFI, 
> where the device interrogation routine may be an obvious candidate. (If 
> a chip doesn't support this kind of interrogation, or if it must be 
> overridden, well, it must go into the chip .cdl after all.)
> 
> 
> eCos directory structure
> ------------------------
> 
> Should NAND be grouped under flash in packages/io and packages/devs? To 
> be honest, the interface under which NAND flash is approached only 
> vaguely resembles that for NOR flash. When we would choose for 
> flash/nand, then we should also have to have flash/nor/ and move all 
> things currently flashing down there.
> 
> +++ My proposal:
> 
> have packages/io/nand/ and packages/devs/nand/* *beside* packages/flash/ 
> and packages/devs/flash/*. They are very different. Also, who wants 
> refactoring of flash/* into flash/nor/*.
> Is there a case to be made for separate nand/controller/ and nand/chip/ 
> trees?
> 
> 
> NAND eCos device type
> ----------------
> 
> NAND flash is neither a character device (CHAR_DEVTAB_ENTRY), nor a 
> block device (BLOCK_DEVTAB_ENTRY), nor a net device (NET_DEVTAB_ENTRY), 
> nor a flash device in the sense of the eCos device driver structure.
> NAND:
> - must be erased in 'blocks'
> - must be programmed/read in (partial) pages (there usually is a good 
> number of pages per block)
> - has a 'spare' area for each page (to store ECCs, bad block info, and 
> client, e.g. file system info); spare is to be handled explicitly
> - has single-bit failures, corrected by ECCs
> - has bad blocks
> - and may have a copy-back function to relocate data within the flash chip.
> NAND controllers may calculate ECCs in hardware on the fly.
> 
> So, should we copy/paste one of those many device driver infrastructures 
> and modify them to accomodate the NAND flash devices?
> eCos NOR flash has solved this in its own way: it does device lookup 
> based on the addresses that the NOR flash is memory-mapped to. But NAND 
> flash is *not* memory-mapped. Besides, NAND flashes can/will exceed 4GB 
> in size, so they cannot be addressed with 32bits.
> 
> +++ My proposal:
> 
> = I would rather create NAND devices as just DEVTAB_ENTRYs, and mark 
> them as NAND (or OTHER or ...) i.s.o. CHAR/BLOCK type. The function 
> dispatch block ptr can be cast, or it can be put in the private area. 
> This way one can do device lookup in the normal way with cyg_io_lookup() 
> ("/dev/nand0" resolves to a device handle).
> 
> = I propose to create two separate device types: NAND controllers and 
> NAND chips. The two device types might be designated as "/dev/nand0" for 
> controller 0, and "/dev/nand0.2" for the third chip connected to 
> controller 0.
> 
> I agree that this is kind of broken. But I have trouble thinking of a 
> way to do device lookup otherwise. Besides, we might consider 
> generalization of the device structure in the sense that the functions 
> block is no longer strictly typed. Its actual type might be derived from 
> the device driver type field.
> 
> 
> External interface of NAND devices (controller on top of chip)
> --------------------------------------------------------------
> 
> +++ My proposal:
> the ONFI functions exported by the generic NAND controller code are 
> sufficient.
> 
> These are:
> - lookup/init; a controller must initialize any chips it owns, locating 
> them e.g. by their names ("/dev/nand0" owns "/dev/nand0.0" .. 
> "/dev/nand0.X")
> - programming of a (partial) page incl spare
> - reading of a (partial) page incl spare, optionally exporting the 
> controller's calculated ECC
> - erase of a block
> - marking block as bad
> - querying block badness
> - retrieving initial bad block table
> 
> To be extended later:
> - reading/programming of multiple pages ?within a block
> - copyback
> - interleaved support
> 
> The spare area will usually have fields for ECC values, either 
> calculated by the layer above, or by a controller that supports it.
> 
> So, at the NAND flash layer, there is no attempt to hide bad blocks, 
> spare areas, ECC handling. The layer above (file system, 'contiguous' 
> programmer) must handle that.
> 
> This exactly fits the YAFFS2 interface (sorry, no idea about JFFS2, but 
> I'd hope so).
> 
> I don't know yet how 'contiguous' programming should work. Typical 
> application: the boot loader code. For NAND flash booting, typically a 
> first-level bootstrap is read from block 0 (guaranteed to be 'good'). 
> This contains code to copy the rest of the bootloader code from NAND 
> flash to RAM, while taking in consideration any bad blocks. Hmmm... 
> would there be any interest in this? As for me, I am currently just 
> interested in getting YAFFS2 alive.
> 
> 
> Well, I hope that this is understandable, and sorry that it is so much.
> 
> Looking forward to your comments,
> 
> Rutger Hofman
> VU Amsterdam


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: NAND flash driver considerations: RFC

[Andrew Lunn]

On Thu, Sep 25, 2008 at 02:29:42AM +0200, Rutger Hofman wrote:
> I have received no responses for this RFC.
>
> Wouldn't anybody care to comment?

O.K. I'll comment. However i've never used NAND, i've no idea how they
work, how they differ between devices and what quirks they have...  So
this is probably not the most educated reply....

> Or do you think it is more useful if I just present my proposed NAND  
> controller interfaces (public and controller-specific), my proposed NAND  
> chip interfaces, and my proposed .cdl config setup? For sure that is  
> some stages later in the design.

Once the basic architecture is agreed, i would say such header files
with lots of comments are a good way to go. When i wrote the first
version of the flash_v2 package that is what i did. I wrote the header
files which defined the interface between the application and the
generic part and between the generic part of the specific flash
drivers. Lots of comments described how the interfaces worked and
allowed me to get my ideas straight. Some changes were made later to
the API but they were relatively minor.

>> eCos directory structure
>> ------------------------
>>
>> Should NAND be grouped under flash in packages/io and packages/devs? To 
>> be honest, the interface under which NAND flash is approached only  
>> vaguely resembles that for NOR flash. When we would choose for  
>> flash/nand, then we should also have to have flash/nor/ and move all  
>> things currently flashing down there.
>>
>> +++ My proposal:
>>
>> have packages/io/nand/ and packages/devs/nand/* *beside* 
>> packages/flash/ and packages/devs/flash/*. They are very different. 
>> Also, who wants refactoring of flash/* into flash/nor/*.
>> Is there a case to be made for separate nand/controller/ and nand/chip/ 
>> trees?

I don't see there being any problem mixing up NOR and NAND drivers in
packages/dev/flash/ The CDL rules will prevent somebody from trying to
use a NAND driver with the NOR generic parts and vica-verse. 

You talk about separate controllers and chips directories. Do we
actually need chip packages. You said that mostly any controller can
talk to any chip. This makes me think a chip packages are not
needed. What are needed are target hardware specific packages which
contain all quirks and configuration information needed for a specific
controller and chips pairing on a specific board. I think we probably
have enough structure in the packages/dev/flash/ to handle this.

>> NAND eCos device type
>> ----------------

...

>> +++ My proposal:
>>
>> = I would rather create NAND devices as just DEVTAB_ENTRYs, and mark  
>> them as NAND (or OTHER or ...) i.s.o. CHAR/BLOCK type. The function  
>> dispatch block ptr can be cast, or it can be put in the private area.  
>> This way one can do device lookup in the normal way with 
>> cyg_io_lookup() ("/dev/nand0" resolves to a device handle).
>>
>> = I propose to create two separate device types: NAND controllers and  
>> NAND chips. The two device types might be designated as "/dev/nand0" 
>> for controller 0, and "/dev/nand0.2" for the third chip connected to  
>> controller 0.

The disk package uses the naming scheme 

/dev/XXXdiskY/Z

XXX is the type of disc, eg mmc, ide etc. Y is the disk number, Z is
the partition. Maybe for nand /dev/namdX/Y might be better. X being
the controller number and Y being the chip number. 

>> I agree that this is kind of broken. But I have trouble thinking of a  
>> way to do device lookup otherwise. Besides, we might consider  
>> generalization of the device structure in the sense that the functions  
>> block is no longer strictly typed. Its actual type might be derived 
>> from the device driver type field.

What we probably need to do is think about how we would want to use
the devices. eg for a filesystem we don't really care about how many
chips there are and how they are arranged. We just want to put a
filesystem on it, or a subsection of it. The filesystem probably does
not want to address controller:chip:block, it wants to use a more
abstract interface, maybe even just a block number.

I would say some more thought is required here...

>> External interface of NAND devices (controller on top of chip)
>> --------------------------------------------------------------
>>
>> +++ My proposal:
>> the ONFI functions exported by the generic NAND controller code are  
>> sufficient.

So you are saying there will not be any generic code in io/nand/ 

   Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: NAND flash driver considerations: RFC

[Rutger Hofman]

Andrew Lunn wrote:
> On Thu, Sep 25, 2008 at 02:29:42AM +0200, Rutger Hofman wrote:
>> I have received no responses for this RFC.
>>
>> Wouldn't anybody care to comment?
> 
> O.K. I'll comment. However i've never used NAND, i've no idea how they
> work, how they differ between devices and what quirks they have...  So
> this is probably not the most educated reply....

Ah, thanks, I am glad.

>>> eCos directory structure
...
>>> +++ My proposal:
>>>
>>> have packages/io/nand/ and packages/devs/nand/* *beside* 
>>> packages/flash/ and packages/devs/flash/*.

> I don't see there being any problem mixing up NOR and NAND drivers in
> packages/dev/flash/ The CDL rules will prevent somebody from trying to
> use a NAND driver with the NOR generic parts and vica-verse. 

I am not sure. NOR and NAND are really very different, not only 
internally but also in the API to higher layers.

E.g., NAND flash file systems usually use the spare area on the flash to 
write meta-data; they handle the ECC explicitly; they are aware 
of/manage bad blocks etc. NOR flash is memory-mapped, NAND cannot be. In 
consequence, literally nothing of the io/flash/common code can be used 
for NAND flash.

So, if we keep NAND under flash/, we would have io/flash/common and 
io/flash/nand/common. 2nd prize in beauty contest. Then, where should 
individual controller/chip packages go? Either under devs/flash/nand/, 
so not on a level with NOR parts; or under devs/flash/, obfuscating the 
fact that they are not interested in io/flash/common but in 
io/flash/nand/common/. This all seemed not very good to me and I moved 
away from my initial, obvious idea of 'keep (NAND) flash with (NOR) flash'.

> You talk about separate controllers and chips directories. Do we
> actually need chip packages. You said that mostly any controller can
> talk to any chip. This makes me think a chip packages are not
> needed. What are needed are target hardware specific packages which
> contain all quirks and configuration information needed for a specific
> controller and chips pairing on a specific board. I think we probably
> have enough structure in the packages/dev/flash/ to handle this.

For these paragraphs, I'll ignore the fact that there are also NAND 
controllers.

At the hardware level, NAND chips appear to be conformant. They have 
wires like nCE (chip enable), nR/B (Ready/Busy), ALE (address latch 
enable), CLE (command latch enable), nWE (write enable) nRE (read 
enable) and some more, and a 8-bit or 16-bit data bus.

NAND chips are controlled by sending a sequence of (generally speaking) 
a command (enable CLE, toggle the data wires), an address (enable ALE, 
...), sending/receiving data (enable nWE/nRE, ...), checking status etc. 
These command sequences are defined in the terms of the wires above. 
Command sequences are usually just named 'command' (and we hope there is 
no confusion with the wire-level command). The NAND chip data sheets 
specify which commands in what format are supported. ONFI (a 
standardization effort underway since 2006 that builds upon de-facto 
'standards') attempts to canonicalize the command set.

If a chip is ONFI-compliant, it is also conformant at the command layer; 
e.g. an ONFI page program command is: send (wire) command 0x80, send 
address = ca. 5 bytes, then send the data, then send (wire) command 
0x10, then wait until status bit[6] has a rising flank. If a chip is 
ONFI-conformant, then we can use generic ONFI encodings and no 
chip-specific code is needed.

Well, ONFI is recent, so although many chips support most of the ONFI 
command set (e.g. page program command = 0x80), they are often not 
*completely* compliant. E.g. the chip I am working with now has a custom 
command for interrogation, which is used to get essential parameters 
like block/page size, #blocks, x8 or x16, etc. So I needed to write a 
chip-specific piece of code to handle that. We wouldn't want every 
platform to repeat the chip-specific code. That is the reason that I 
think a NAND chip device type is required.

I fully agree that the platform target package must (be able to) 
configure some stuff for the NAND chips, e.g. indicate which (GPIO?) pin 
the nCE pin of some NAND chip is attached to. In general, the target 
should configure which chips are attached to which controller, their 
device names etc etc.

>>> NAND eCos device type

> The disk package uses the naming scheme 
> 
> /dev/XXXdiskY/Z
> 
> XXX is the type of disc, eg mmc, ide etc. Y is the disk number, Z is
> the partition. Maybe for nand /dev/namdX/Y might be better. X being
> the controller number and Y being the chip number. 

I will do that.

> What we probably need to do is think about how we would want to use
> the devices. eg for a filesystem we don't really care about how many
> chips there are and how they are arranged. We just want to put a
> filesystem on it, or a subsection of it. The filesystem probably does
> not want to address controller:chip:block, it wants to use a more
> abstract interface, maybe even just a block number.
> 
> I would say some more thought is required here...

Agreed. See at the bottom.

>>> External interface of NAND devices (controller on top of chip)
>>> --------------------------------------------------------------
>>>
>>> +++ My proposal:
>>> the ONFI functions exported by the generic NAND controller code are  
>>> sufficient.
> 
> So you are saying there will not be any generic code in io/nand/ 

There is certainly generic code in io/nand. It will implement one or 
more higher-level APIs in terms of the ONFI commands.

The code for the *basic* API is something like this.

Example: program (within) one page:

cyg_nand_page_program(cyg_nand_t *nand,
                       const void *data, size_t len,
                       size_t col, size_t row,
                       const void *spare, size_t spare_len)

must implement the ONFI command sequence for page program. So, it is 
essentially implemented as follows:

    nand->cmd(nand, 0x80);
    nand->addr(nand, col, row);
    nand->program_data(nand, data, len);
    nand->goto_addr(nand, spare_offset, row);
    nand->program_data(nand, spare, spare_len);
    nand->cmd(nand, 0x10);
    nand->await_status(nand, 6);

where the indirect calls are implemented by the specific controller.

A minimal set of other calls would be:
    cyg_nand_page_read()
    cyg_nand_block_erase()
    cyg_nand_bad_block_...()
    cyg_nand_chip_select(nand, chip_number)
    cyg_nand_init()

Now, it seems YAFFS only requires the following commands:
    read_page()
    program_page()
    erase_block()
    and some calls for initialization and bad block handling.

YAFFS is aware of page/block/spare size and the rest, it uses the spare 
area, it does ECC by itself, and it handles bad blocks. So, this basic 
API would suffice to run YAFFS on one chip.

I agree with your remark above that a higher-level API is also 
desirable. For starters, if the nand chips have identical page/block 
size, spare size, bus width, etc, we need only a thin layer to hide the 
fact that there are multiple chips and possibly multiple controllers. 
This would still fully expose bad blocks, spare and ECC handling. It 
would allow YAFFS to run on one 'abstract NAND'.

If we want to offer another layer that can hide the NAND-specific 
nasties (spare, ECC, bad blocks) from the upper layer, more thought is 
needed. ECC handling is not an issue, it can just be implemented. But 
e.g. if we would want to present all the NANDs as one contiguous area, 
one must handle blocks that go bad. One might use an indirection table 
for that, and reserve blocks as backups. Or there might be different 
solutions. How would one handle multiple writes to one page? Usually, 
these are limited, a typical allowed value is 4 before an erase is 
required. So that would require buffering, flushing, and/or relocation 
of pages. What if we want to hide the fact that there are 
pages/blocks/luns/chips? Then we must address the bytes in this abstract 
NAND, but 32bit addresses will be insufficient.

So, I agree that lots of thought is required for the fancier 
higher-level APIs. But if we implement the basic API plus the extension 
of abstracting away uniform chips, I think we can serve the most 
important target: flash file systems like YAFFS.

Rutger



-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: NAND flash driver considerations: RFC

[Andrew Lunn]

>> I don't see there being any problem mixing up NOR and NAND drivers in
>> packages/dev/flash/ The CDL rules will prevent somebody from trying to
>> use a NAND driver with the NOR generic parts and vica-verse. 
>
> I am not sure. NOR and NAND are really very different, not only  
> internally but also in the API to higher layers.
>
> E.g., NAND flash file systems usually use the spare area on the flash to  
> write meta-data; they handle the ECC explicitly; they are aware  
> of/manage bad blocks etc. NOR flash is memory-mapped, NAND cannot be. In  
> consequence, literally nothing of the io/flash/common code can be used  
> for NAND flash.

Sure, but that is not an issue when we talk about what files go where
in the directory hierarchy of packages. They are all device drivers
for flash devices. 

> So, if we keep NAND under flash/, we would have io/flash/common and  
> io/flash/nand/common. 2nd prize in beauty contest. Then, where should  
> individual controller/chip packages go? Either under devs/flash/nand/,  
> so not on a level with NOR parts; or under devs/flash/, obfuscating the  
> fact that they are not interested in io/flash/common but in  
> io/flash/nand/common/. This all seemed not very good to me and I moved  
> away from my initial, obvious idea of 'keep (NAND) flash with (NOR) 
> flash'.

There are many different ways at looking at the packages. The CDL
language imposes a hierarchy as well which has nothing to do with the
directory hierarchy. We could have packages/io/flash_nand/current for
the common code. This is not unknown. Eg we have packages/net/tcpip
and packages/net/bsd_tcpip. The first one is no longer supported and
the second one is what everybody uses.

I don't really see that mixing up the packages is obfuscating them. We
already have two different classes of flash drivers in
packages/dev/flash. We have chip drivers and we have target board
drivers. When the flash_v2 branch gets merged in we will have another
two classes added. 

What is important here is the CDL. The new packages need to have

    parent        CYGPKG_IO_FLASH_NAND
    active_if     CYGPKG_IO_FLASH_NAND 

so that they are correctly placed into the CDL tree and only active
when appropriate.

[..]

> Well, ONFI is recent, so although many chips support most of the ONFI  
> command set (e.g. page program command = 0x80), they are often not  
> *completely* compliant. E.g. the chip I am working with now has a custom  
> command for interrogation, which is used to get essential parameters  
> like block/page size, #blocks, x8 or x16, etc. So I needed to write a  
> chip-specific piece of code to handle that. We wouldn't want every  
> platform to repeat the chip-specific code. That is the reason that I  
> think a NAND chip device type is required.

O.K. Now i understand.

> The code for the *basic* API is something like this.
>
> Example: program (within) one page:
>
> cyg_nand_page_program(cyg_nand_t *nand,
>                       const void *data, size_t len,
>                       size_t col, size_t row,
>                       const void *spare, size_t spare_len)
>
> must implement the ONFI command sequence for page program. So, it is  
> essentially implemented as follows:
>
>    nand->cmd(nand, 0x80);
>    nand->addr(nand, col, row);
>    nand->program_data(nand, data, len);
>    nand->goto_addr(nand, spare_offset, row);
>    nand->program_data(nand, spare, spare_len);
>    nand->cmd(nand, 0x10);
>    nand->await_status(nand, 6);
>
> where the indirect calls are implemented by the specific controller.
>
> A minimal set of other calls would be:
>    cyg_nand_page_read()
>    cyg_nand_block_erase()
>    cyg_nand_bad_block_...()
>    cyg_nand_chip_select(nand, chip_number)
>    cyg_nand_init()

Given this API, what does the device filenames /dev/nand0/1 have to do
with anything. How do i get from a file descriptor returned with
open() to a cyg_nand_t * nand structure? Do file descriptors and /dev/
files make any sense? The NOR devices don't do this?

> I agree with your remark above that a higher-level API is also  
> desirable. For starters, if the nand chips have identical page/block  
> size, spare size, bus width, etc, we need only a thin layer to hide the  
> fact that there are multiple chips and possibly multiple controllers.  
> This would still fully expose bad blocks, spare and ECC handling. It  
> would allow YAFFS to run on one 'abstract NAND'.
>
> If we want to offer another layer that can hide the NAND-specific  
> nasties (spare, ECC, bad blocks) from the upper layer, more thought is  
> needed. ECC handling is not an issue, it can just be implemented. But  
> e.g. if we would want to present all the NANDs as one contiguous area,  
> one must handle blocks that go bad. One might use an indirection table  
> for that, and reserve blocks as backups. Or there might be different  
> solutions. How would one handle multiple writes to one page? Usually,  
> these are limited, a typical allowed value is 4 before an erase is  
> required. So that would require buffering, flushing, and/or relocation  
> of pages. What if we want to hide the fact that there are  
> pages/blocks/luns/chips? Then we must address the bytes in this abstract  
> NAND, but 32bit addresses will be insufficient.

Not necessarily bytes, but by blocks. We could define a block as being
512 bytes, and use a u32 to indicate which block to access. That would
allow access to i think 2TB. For an embedded system that is quite a
lot. Is 512 sensible? I have no idea.

> So, I agree that lots of thought is required for the fancier  
> higher-level APIs. But if we implement the basic API plus the extension  
> of abstracting away uniform chips, I think we can serve the most  
> important target: flash file systems like YAFFS.

YAFFS is interesting, but from a licensing point of view, it is
bad. It is pure GPL. We can never include it into the eCos tree.  We
should also be looking at support for JFFS2, which we can and do
import into eCos. People who are using Redboot as a Linux bootloader
will want the NAND implementation to the compatible with what linux
does so that they can boot there Linux kernel out of a NAND filesystem
with JFFS2. Taking this further, UBI and UBIFS is of interest for the
Linux world, however it has the same licensing problems as YAFFS.

      Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: NAND flash driver considerations: RFC

[Rutger Hofman]

Andrew Lunn wrote:
>>> I don't see there being any problem mixing up NOR and NAND drivers in
>>> packages/dev/flash/ The CDL rules will prevent somebody from trying to
> 
> There are many different ways at looking at the packages. The CDL
> language imposes a hierarchy as well which has nothing to do with the
> directory hierarchy. We could have packages/io/flash_nand/current for
> the common code. This is not unknown. Eg we have packages/net/tcpip
> and packages/net/bsd_tcpip. The first one is no longer supported and
> the second one is what everybody uses.
> 
> I don't really see that mixing up the packages is obfuscating them. We
> already have two different classes of flash drivers in
> packages/dev/flash. We have chip drivers and we have target board
> drivers. When the flash_v2 branch gets merged in we will have another
> two classes added. 
> 
> What is important here is the CDL. The new packages need to have
> 
>     parent        CYGPKG_IO_FLASH_NAND
>     active_if     CYGPKG_IO_FLASH_NAND 
> 
> so that they are correctly placed into the CDL tree and only active
> when appropriate.

OK, I refactored my test setup to reflect what you suggest. So now we 
have packages:

io/flash_nand/
   + common/
   + controller/
   + chip/

devs/flash/bfin/
   + nfc/
   + ez_kit_bf548/nand/
  (+ ez_kit_bf548/nor/ for the NOR flash also on this board)

devs/flash/st-micro/nand0xG/

and I refactored the .cdl's, the #defines, the includes etc etc. The 
.cdl-s have the parentage you suggested.

Is this OK?

>> The code for the *basic* API is something like this.
>>
>> Example: program (within) one page:
>>
>> cyg_nand_page_program(cyg_nand_t *nand,
>>                       const void *data, size_t len,
>>                       size_t col, size_t row,
>>                       const void *spare, size_t spare_len)
>>
>> must implement the ONFI command sequence for page program. So, it is  
>> essentially implemented as follows:
>>
>>    nand->cmd(nand, 0x80);
>>    nand->addr(nand, col, row);
>>    nand->program_data(nand, data, len);
>>    nand->goto_addr(nand, spare_offset, row);
>>    nand->program_data(nand, spare, spare_len);
>>    nand->cmd(nand, 0x10);
>>    nand->await_status(nand, 6);
>>
>> where the indirect calls are implemented by the specific controller.
>>
>> A minimal set of other calls would be:
>>    cyg_nand_page_read()
>>    cyg_nand_block_erase()
>>    cyg_nand_bad_block_...()
>>    cyg_nand_chip_select(nand, chip_number)
>>    cyg_nand_init()
> 
> Given this API, what does the device filenames /dev/nand0/1 have to do
> with anything. How do i get from a file descriptor returned with
> open() to a cyg_nand_t * nand structure?

That is not the idea. There must be a NAND filesystem that is layered on 
top of flash_nand devices. That file system has an open() implementation 
(like yaffs_open()).

 > Do file descriptors and /dev/
> files make any sense? The NOR devices don't do this?

NOR flash is memory-mapped, it uses the memory mapping to locate
devices. NAND flash is not memory-mapped, so I had to come up with some
other mechanism to locate devices. The target cdl must tell the devices 
their names, and the implementation must locate the driver data 
structures from that.

My proposal was to use cyg_io_lookup() to locate the controller and chip 
devices by name. Well, as I wrote in my RFC, this is obviously broken 
because NAND controller and chip devices must not be accessed through 
the file system. But we still need a mechanism to look them up and then 
tie them together.

Then, the File System must retrieve its NAND controller (+ associated 
chip(s)) from the name it is passed at cdl or mount time. I thought 
cyg_io_lookup() can be used again to get the nand_t pointer. (Or ad lib 
replace NAND controller here with "Abstract NAND Device".)

Would you care to suggest a better scheme for lookup for chip, 
controller, FS? Or would you already be satisfied if the names are not 
like /dev/nand0 but like '#nand0' or whatever non-filesystem-capable string?

[...]
>> of pages. What if we want to hide the fact that there are  
>> pages/blocks/luns/chips? Then we must address the bytes in this abstract  
>> NAND, but 32bit addresses will be insufficient.
> 
> Not necessarily bytes, but by blocks. We could define a block as being
> 512 bytes, and use a u32 to indicate which block to access. That would
> allow access to i think 2TB. For an embedded system that is quite a
> lot. Is 512 sensible? I have no idea.

Blocks are the unit of erase. Pages are the unit of read/program (with 
ECC in the page's spare area). Blocks usually consist of a good number 
of pages (think 64). Pages can be 256, 512, 1K, 2K, 4K bytes (with 
larger to come, I'd guess).

Even at the chip level, NAND allows addressing of individual x8 or x16 
words (bytes or shorts). This saves on IO between CPU and NAND 
peripherals -- it allows random access (within a page). I wouldn't 
suggest to throw that faculty away. So, my proposal is to have two 
32-bit indices, column (= x8/x16 word within page) and row = 
page+block+lun. This resembles the ONFI chip specification, where 
addressing is done in tuples too (some integral number of bytes for the 
column, some integral number of bytes for the row). It also makes 
evident that some things are different with NAND flash; e.g. it is 
impossible to program/read more than 1 page at a time (although there is 
an interface to do pipelining and interleaving (which I have skipped) of 
pages).

Brainstorm: it might be possible to generalize my previous suggestion of 
'uniform chips', and let the Abstract Nand Device export one contiguous 
NAND 'chip'; it must translate between the abstract blocks and pages and 
the blocks and pages of individual chips. The block size (unit of erase) 
of the Abstract Chip must be the maximum of the block size of the chips. 
The page size (unit of ECC) must be handled carefully to translate ECC 
addresses within the spare area to match potentially different page sizes.

> YAFFS is interesting, but from a licensing point of view, it is
> bad. It is pure GPL. We can never include it into the eCos tree.  We
> should also be looking at support for JFFS2, which we can and do
> import into eCos. People who are using Redboot as a Linux bootloader
> will want the NAND implementation to the compatible with what linux
> does so that they can boot there Linux kernel out of a NAND filesystem
> with JFFS2. Taking this further, UBI and UBIFS is of interest for the
> Linux world, however it has the same licensing problems as YAFFS.

YAFFS is also Linuxware. It has been in the kernel for a long time. I 
understand that the licensing of incorporating YAFFS into their RedBoot 
may bite (i.e. cost) people. Just for curiosity: does anyone know what a 
commercial license of YAFFS would cost?

JFFS2 uses mtd. Should we then make a thin mtd portability layer on top 
of cyg_nand? Or should we hack JFFS2 to speak native cyg_nand (and again 
loose JFFS2 compatibility)? I seem to read that JFFS2 has been frozen, 
and JFFS3 is now UBI/UBIFS.

If we make a thin mtd portability layer on top of cyg_nand, I have to 
read up on mtd, or at least those parts of mtd that are actually used by 
JFFS2. I skimmed through mtd when I started this. mtd's interface is *huge*.

Rutger

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] SMP on Intel x86

[Steve West]

Hello,
    I am still having some problems with SMP on Intel x86. I am using Core 2 
duo CPU's with a Intel DG35EC Motherboard. Problem is that the Bios only 
reports 1 CPU entry in the MP table. I assume this is the boot CPU. I know 
this setup works ok with XP. Xp reports 2 CPU's. Is there somthing I am 
missing with the MP table?

Steve 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] SMP on Intel x86

[Nick Garnett]

"Steve West" <swest@cyglan.com> writes:

> Hello,
>     I am still having some problems with SMP on Intel x86. I am using
> Core 2 duo CPU's with a Intel DG35EC Motherboard. Problem is that the
> Bios only reports 1 CPU entry in the MP table. I assume this is the
> boot CPU. I know this setup works ok with XP. Xp reports 2 CPU's. Is
> there somthing I am missing with the MP table?

In the 7 years since I wrote this code the state of x86 SMP technology
has moved on considerably. The MP tables are now obsolete and replaced
by ACPI tables, which is what XP is using. Most BIOSes still have an
MP table for backward compatibility, but it only describes a single
CPU.

So the first step is to add code to parse ACPI table.

However, even in 2001 SMP support was experimental and incomplete. It
was just about possible to get simple stand-alone programs up and
running, but RedBoot didn't work, none of the device drivers had been
converted to SMP awareness, some of the test programs didn't work,
debugging was difficult. Little has been done with the code since and
bitrot and changes to the rest of eCos mean that even the minimal
functionality it used to have has probably been lost.

Of course it can probably be fixed, but that is several weeks work.

There is also a question of suitability. eCos is intended for small,
cheap, low power embedded devices. Anything with a Core 2 duo in it
will have plenty of RAM, peripherals, graphics, and high power
consumption. Most likely a PC motherboard. It would make more sense to
run an embedded Linux on that than put the effort in to getting eCos
running.

-- 
Nick Garnett                                      eCos Kernel Architect
eCosCentric Limited    http://www.eCosCentric.com      The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.     Tel: +44 1223 245571
Registered in England and Wales:                        Reg No: 4422071
   >>>> Visit us on stand 905 at the Embedded Systems Show 2008 <<<<
   >>>> Oct 1-2, NEC, Birmingham, UK http://www.embedded.co.uk  <<<<


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] SMP on Intel x86

[Steve West]

> Of course it can probably be fixed, but that is several weeks work.
>
> There is also a question of suitability. eCos is intended for small,
> cheap, low power embedded devices. Anything with a Core 2 duo in it
> will have plenty of RAM, peripherals, graphics, and high power
> consumption. Most likely a PC motherboard. It would make more sense to
> run an embedded Linux on that than put the effort in to getting eCos
> running.
>
We have been using eCos since 2002. There was little for embedded OS's that 
where as powerful as eCos back then. We would like to continue to use it, 
but may be forced to an embedded Linux as the hardware changes are forcing 
us to a dual CPU.
    We would like to test the SMP on eCos. For now we can just get the info 
from the ACPI table by hand and try the code. I have been doing some 
investigation and find that setting the interrupt and starting the CPU may 
be different now. Could you shed some light on this?

Steve 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] SMP on Intel x86

[Nick Garnett]

"Steve West" <swest@cyglan.com> writes:

> > Of course it can probably be fixed, but that is several weeks work.
> >
> > There is also a question of suitability. eCos is intended for small,
> > cheap, low power embedded devices. Anything with a Core 2 duo in it
> > will have plenty of RAM, peripherals, graphics, and high power
> > consumption. Most likely a PC motherboard. It would make more sense to
> > run an embedded Linux on that than put the effort in to getting eCos
> > running.
> >
> We have been using eCos since 2002. There was little for embedded OS's
> that where as powerful as eCos back then. We would like to continue to
> use it, but may be forced to an embedded Linux as the hardware changes
> are forcing us to a dual CPU.
>     We would like to test the SMP on eCos. For now we can just get the
> info from the ACPI table by hand and try the code. I have been doing
> some investigation and find that setting the interrupt and starting
> the CPU may be different now. Could you shed some light on this?

I'm afraid my knowledge of x86 SMP hardware is very rusty, and I
haven't looked at it for ages. I would hope that the mechanism for
getting secondary CPUs running is better than it was before, it was a
somewhat weird process when I did it.

I'm afraid you will have to work this out for yourself. The first step
is to get the necessary documentation, probably from Intel's
website. For example code take a look at the Linux kernel, that was my
main source of information on the details. Of course if you intend to
link proprietary code with it, or contribute it back, don't copy any
of the Linux code.

Sorry I cannot be more help.


-- 
Nick Garnett                                      eCos Kernel Architect
eCosCentric Limited    http://www.eCosCentric.com      The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.     Tel: +44 1223 245571
Registered in England and Wales:                        Reg No: 4422071
   >>>> Visit us on stand 905 at the Embedded Systems Show 2008 <<<<
   >>>> Oct 1-2, NEC, Birmingham, UK http://www.embedded.co.uk  <<<<


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss