Synth NAND Flash

Synth NAND Flash

[Simon Kallweit]

Hi there

I merged the NAND code from Rutger into my repo and tried to figure out 
how to write synthetic target support, which in my opinion would be a 
great addition so we can test future filesystems (UFFS) without a 
target, do wear-leveling analysis and stuff like that.

First, I noticed a few things I would like to clear up in front. 
Currently, the NAND subsystem sits in io/flash_nand which I think is 
fine. But the devices sit under devs/flash, which is the same location 
as for NOR flash. I think we should rename this to devs/flash_nand. I 
already did this in my merge. This would make the distinction between 
NOR and NAND flash more clearer. I also thought about renaming the whole 
framework from flash_nand to simply nand. This would also match the API 
names cyg_nand_xxx better. Are there any objections?

I started out with eCos in it's default template. I was getting some 
errors because of the missing ssize_t type. Either we should get rid of 
it or add something like this to the CDL:

require CYGBLD_ISO_SSIZE_T_HEADER

Other than that there is a little bit of cleaning up to do, but I think 
that's all minor stuff.

Next I copied the GPIO nand flash controller to make a synth version and 
also copied a st-micro NAND chip driver to make a synth version. For 
now, they are pretty empty skeletons. But I was able to build the NAND 
subsystems with those dummy drivers. Of course, the test cases don't 
work. It also occurred to me that the "shell.c" test has quite a few 
dependencies. I just removed this test for the moment.

Now for the actual design of the synth driver. I think the best way 
would be to implement a NAND simulator based on the ONFI specification. 
Something similar has been done for the MTD framework, but I guess other 
than for inspiration we're not allowed to use that code. So basically we 
would simulate the interface to the chip. I guess we don't have to 
simulate the signal lines. We just need some mechanism for chipselect 
and reset I guess. The interface will more be along the lines of writing 
commands, addresses, reading back etc. This means that the simulator 
will be implemented as a state machine. There is even one described in 
the ONFI specification for reference.

I think the basics can be implemented rather quickly. I guess we don't 
need to simulate multiple concurrent LUNs, or does the framework already 
support these?

Well that's about it. I'll try to implement a simple simulator tomorrow 
and see where I get. I post back some results as soon as I have something.

Simon

Re: Synth NAND Flash

[Simon Kallweit]

Simon Kallweit wrote:
> Now for the actual design of the synth driver. I think the best way 
> would be to implement a NAND simulator based on the ONFI specification. 
> Something similar has been done for the MTD framework, but I guess other 
> than for inspiration we're not allowed to use that code. So basically we 
> would simulate the interface to the chip. I guess we don't have to 
> simulate the signal lines. We just need some mechanism for chipselect 
> and reset I guess. The interface will more be along the lines of writing 
> commands, addresses, reading back etc. This means that the simulator 
> will be implemented as a state machine. There is even one described in 
> the ONFI specification for reference.

Well after some more thought it's probably a better idea to keep the 
interface to the NAND chip simulator very close to the hardware. That 
means, a function for setting signal lines (CE, CLE, ALE, WP) as well as 
two functions for read and write access.

I don't intend to support multiple LUNs for the moment, just to keep it 
simple.

I have already done the interface to the NAND chip simulator. This seems 
to work fine. I'll keep you updated.

Simon

Re: Synth NAND Flash

[Sergei Gavrikov]

On Tue, May 12, 2009 at 10:36:34AM +0200, Simon Kallweit wrote:
> Simon Kallweit wrote:
>> Now for the actual design of the synth driver. I think the best way
>> would be to implement a NAND simulator based on the ONFI
>> specification.  Something similar has been done for the MTD
>> framework, but I guess other than for inspiration we're not allowed
>> to use that code. So basically we would simulate the interface to the
>> chip. I guess we don't have to simulate the signal lines. We just
>> need some mechanism for chipselect and reset I guess. The interface
>> will more be along the lines of writing commands, addresses, reading
>> back etc. This means that the simulator will be implemented as a
>> state machine. There is even one described in the ONFI specification
>> for reference.
>
> Well after some more thought it's probably a better idea to keep the
> interface to the NAND chip simulator very close to the hardware. That
> means, a function for setting signal lines (CE, CLE, ALE, WP) as well
> as  two functions for read and write access.
>
> I don't intend to support multiple LUNs for the moment, just to keep
> it  simple.
>
> I have already done the interface to the NAND chip simulator. This
> seems  to work fine. I'll keep you updated.

Hello Simon

Thank you for your efforts! Can I ask you about UFFS itself? How did it
look for you, Is UFFS stable enough to use it? Thanks to your post about
UFFS I looked on its sources 2 days ago and just tried to know what its a
memory amount. I did stand up a small test sandbox here
http://bitbucket.org/tickling/uffs4ecos/

so, I knew that UFFS needs much less of .text, .bss than JFFS2 needs. As
I could understand UFFS has not GC like JFFS2 and, perhaps, it can be
more suitable for the small memory foot print systems. But, I do not
know, is UFFS stable, bug-less, etc.

I see that you started from NAND flash driver for eCos to wire it with
UFFS core then. Fortunately or unfortunately I have no NAND flash parts
to play with it, and I looked in a side of a UFFS SIMRAM class which was
implemented by UFFS's author to debug and play with UFFS (I did import
uffs-1.3.0 sources). So, that my stub sandbox FS_UFFS does not seat on
FLASH_IO layer, instead, I thought to try to implement of a set of the
file system commands like uffs_mount, uffs_umount, uffs_open, etc. to
get the UFFS stuff like the eCos RAMFS file system for the test
purposed. It seemed for me that e.g. 512b x 512 or 256K UFFS partition
will be suitable for some targets and of course for synth Linux target.
What is your opinion about this way to test UFFS? Does it look wrong on
your view? If I miss understood something, please, enlighten me and I
will stop those my evening drops on bitbucket and will be wait a success
story from you.

Thank you.

Sergei

> Simon

Re: Synth NAND Flash

[Simon Kallweit]

Sergei Gavrikov wrote:
> Thank you for your efforts! Can I ask you about UFFS itself? How did it
> look for you, Is UFFS stable enough to use it? Thanks to your post about
> UFFS I looked on its sources 2 days ago and just tried to know what its a
> memory amount. I did stand up a small test sandbox here
> http://bitbucket.org/tickling/uffs4ecos/

Hmm, I don't have any experience with UFFS yet. But as I need something 
more economical than JFFS2 and public domain also UFFS seemed like a 
good option.

> so, I knew that UFFS needs much less of .text, .bss than JFFS2 needs. As
> I could understand UFFS has not GC like JFFS2 and, perhaps, it can be
> more suitable for the small memory foot print systems. But, I do not
> know, is UFFS stable, bug-less, etc.

UFFS does not use any GC. It's memory requirements are currently linear 
to the disk size, but should be very economical for smaller FS. It also 
should be, according to the author, stable to be used in real-world 
applications.

> I see that you started from NAND flash driver for eCos to wire it with
> UFFS core then. Fortunately or unfortunately I have no NAND flash parts
> to play with it, and I looked in a side of a UFFS SIMRAM class which was
> implemented by UFFS's author to debug and play with UFFS (I did import
> uffs-1.3.0 sources). So, that my stub sandbox FS_UFFS does not seat on
> FLASH_IO layer, instead, I thought to try to implement of a set of the
> file system commands like uffs_mount, uffs_umount, uffs_open, etc. to
> get the UFFS stuff like the eCos RAMFS file system for the test
> purposed. It seemed for me that e.g. 512b x 512 or 256K UFFS partition
> will be suitable for some targets and of course for synth Linux target.
> What is your opinion about this way to test UFFS? Does it look wrong on
> your view? If I miss understood something, please, enlighten me and I
> will stop those my evening drops on bitbucket and will be wait a success
> story from you.

This seems like a reasonable plan to me. I originally intended to do the 
same, but then decided to first focus on the NAND subsystem itself. Once 
we have synthetic NAND in place, implementing the NAND glue code for 
UFFS will be relatively easy and testable.

Of course it would be really great if you already write the glue code 
for the FS itself. This can be tested independently (with the SIMRAM class).

Simon

Re: Synth NAND Flash

[Rutger Hofman]

Simon Kallweit wrote:
> Hi there
> 
> I merged the NAND code from Rutger into my repo and tried to figure out 
> how to write synthetic target support, which in my opinion would be a 
> great addition so we can test future filesystems (UFFS) without a 
> target, do wear-leveling analysis and stuff like that.
> 
> First, I noticed a few things I would like to clear up in front. 
> Currently, the NAND subsystem sits in io/flash_nand which I think is 
> fine. But the devices sit under devs/flash, which is the same location 
> as for NOR flash. I think we should rename this to devs/flash_nand. I 
> already did this in my merge. This would make the distinction between 
> NOR and NAND flash more clearer. I also thought about renaming the whole 
> framework from flash_nand to simply nand. This would also match the API 
> names cyg_nand_xxx better. Are there any objections?

In fact, this is how I started out originally. But Andrew Lunn convinced 
me to do otherwise:

http://ecos.sourceware.org/ml/ecos-discuss/2008-09/msg00172.html

> I started out with eCos in it's default template. I was getting some 
> errors because of the missing ssize_t type. Either we should get rid of 
> it or add something like this to the CDL:
> 
> require CYGBLD_ISO_SSIZE_T_HEADER

My Ubuntu man page says that ssize_t should be defined by sys/types.h; 
requiring CYGPKG_ISOINFRA ought to be good enough.

> Other than that there is a little bit of cleaning up to do, but I think 
> that's all minor stuff.

I'm open for recommendations.

> Next I copied the GPIO nand flash controller to make a synth version and 
> also copied a st-micro NAND chip driver to make a synth version. For 
> now, they are pretty empty skeletons. But I was able to build the NAND 
> subsystems with those dummy drivers. Of course, the test cases don't 
> work. It also occurred to me that the "shell.c" test has quite a few 
> dependencies. I just removed this test for the moment.

Yes, sure, "shell.c" allows a number of operations to be done 
interactively. It is not so much in the vein of automated stress 
testing. It was in fact my testing vehicle for a raw terminal program. 
I'm fine with removing it from the standard test build, but I would be 
sorry to see it go.

> Now for the actual design of the synth driver. I think the best way 
> would be to implement a NAND simulator based on the ONFI specification. 
> Something similar has been done for the MTD framework, but I guess other 
> than for inspiration we're not allowed to use that code. So basically we 
> would simulate the interface to the chip. I guess we don't have to 
> simulate the signal lines. We just need some mechanism for chipselect 
> and reset I guess. The interface will more be along the lines of writing 
> commands, addresses, reading back etc. This means that the simulator 
> will be implemented as a state machine. There is even one described in 
> the ONFI specification for reference.

I never heard of an actual ONFI chip yet. Current chips are usually of 
the regular large-page kind. It might be most practical to make a 
generic regular large-page chip emulator that can be parameterized with 
the Device IDs as enumerated in cyg_nand_chip_id[] in file 
flash_nand/src/chip/io_nand_chip.c .

> I think the basics can be implemented rather quickly. I guess we don't 
> need to simulate multiple concurrent LUNs, or does the framework already 
> support these?

Multiple LUNs is the same as multiple chips, as far as the NAND package 
(or ONFI) is concerned. It should be supported, but it is untested 
because I lacked hardware.

> Well that's about it. I'll try to implement a simple simulator tomorrow 
> and see where I get. I post back some results as soon as I have something.
> 
> Simon

Rutger

Re: Synth NAND Flash

[Rutger Hofman]

Simon Kallweit wrote:
> Simon Kallweit wrote:
>> Now for the actual design of the synth driver. I think the best way 
>> would be to implement a NAND simulator based on the ONFI 
>> specification. Something similar has been done for the MTD framework, 
>> but I guess other than for inspiration we're not allowed to use that 
>> code. So basically we would simulate the interface to the chip. I 
>> guess we don't have to simulate the signal lines. We just need some 
>> mechanism for chipselect and reset I guess. The interface will more be 
>> along the lines of writing commands, addresses, reading back etc. This 
>> means that the simulator will be implemented as a state machine. There 
>> is even one described in the ONFI specification for reference.
> 
> Well after some more thought it's probably a better idea to keep the 
> interface to the NAND chip simulator very close to the hardware. That 
> means, a function for setting signal lines (CE, CLE, ALE, WP) as well as 
> two functions for read and write access.

See my suggestion on a NAND chip simulator in my previous mail. Having 
it be accessed using the control lines is the right way to go IMHO: this 
is where the package boundary/API for NAND chips is.

> I don't intend to support multiple LUNs for the moment, just to keep it 
> simple.
> 
> I have already done the interface to the NAND chip simulator. This seems 
> to work fine. I'll keep you updated.

I'm looking forward!

Rutger

Re: Synth NAND Flash

[Rutger Hofman]

Sergei Gavrikov wrote:
[snip]
> I see that you started from NAND flash driver for eCos to wire it with
> UFFS core then. Fortunately or unfortunately I have no NAND flash parts
> to play with it, and I looked in a side of a UFFS SIMRAM class which was
> implemented by UFFS's author to debug and play with UFFS (I did import
> uffs-1.3.0 sources). So, that my stub sandbox FS_UFFS does not seat on
> FLASH_IO layer, instead, I thought to try to implement of a set of the
> file system commands like uffs_mount, uffs_umount, uffs_open, etc. to
> get the UFFS stuff like the eCos RAMFS file system for the test
> purposed.

You could also take a look at my interface layer with YAFFS; it has code 
both to integrate YAFFS into fileio, and interfacing with the NAND 
layer. Tarballs from 
http://www.cs.vu.nl/~rutger/software/ecos/nand-flash/ below the tarballs 
for the NAND flash package.

Rutger

Re: Synth NAND Flash

[Sergei Gavrikov]

Rutger Hofman writes:
> Sergei Gavrikov wrote:
> [snip]
>> I see that you started from NAND flash driver for eCos to wire it with
>> UFFS core then. Fortunately or unfortunately I have no NAND flash parts
>> to play with it, and I looked in a side of a UFFS SIMRAM class which was
[snip]
>> file system commands like uffs_mount, uffs_umount, uffs_open, etc. to
>> get the UFFS stuff like the eCos RAMFS file system for the test
>> purposed.
>
> You could also take a look at my interface layer with YAFFS; it has code  
> both to integrate YAFFS into fileio, and interfacing with the NAND  
> layer. Tarballs from  
> http://www.cs.vu.nl/~rutger/software/ecos/nand-flash/ below the tarballs  
> for the NAND flash package.
>
> Rutger

Thank you, Rutger. I will look on your implementation. It's better to
have three examples. Unfortunately, the original UFFS's `mkuffs' utility
has only a few FS opers. So, it's better to have more examples of the
different implementations.

Sergei

Re: Synth NAND Flash

[Simon Kallweit]

Rutger Hofman wrote:
>> First, I noticed a few things I would like to clear up in front. 
>> Currently, the NAND subsystem sits in io/flash_nand which I think is 
>> fine. But the devices sit under devs/flash, which is the same location 
>> as for NOR flash. I think we should rename this to devs/flash_nand. I 
>> already did this in my merge. This would make the distinction between 
>> NOR and NAND flash more clearer. I also thought about renaming the 
>> whole framework from flash_nand to simply nand. This would also match 
>> the API names cyg_nand_xxx better. Are there any objections?
> 
> In fact, this is how I started out originally. But Andrew Lunn convinced 
> me to do otherwise:
> 
> http://ecos.sourceware.org/ml/ecos-discuss/2008-09/msg00172.html

OK, then I have to rename my stuff again :)

>> I started out with eCos in it's default template. I was getting some 
>> errors because of the missing ssize_t type. Either we should get rid 
>> of it or add something like this to the CDL:
>>
>> require CYGBLD_ISO_SSIZE_T_HEADER
> 
> My Ubuntu man page says that ssize_t should be defined by sys/types.h; 
> requiring CYGPKG_ISOINFRA ought to be good enough.

Seems like there is a bug in eCos as the option is called 
CYGBLD_ISO_SSIZE_T_HEADER but used as CYGBLD_ISO_SSIZET_HEADER. I submit 
a patch for it.

>> Other than that there is a little bit of cleaning up to do, but I 
>> think that's all minor stuff.
> 
> I'm open for recommendations.

I'll post patches when ready.

>> Next I copied the GPIO nand flash controller to make a synth version 
>> and also copied a st-micro NAND chip driver to make a synth version. 
>> For now, they are pretty empty skeletons. But I was able to build the 
>> NAND subsystems with those dummy drivers. Of course, the test cases 
>> don't work. It also occurred to me that the "shell.c" test has quite a 
>> few dependencies. I just removed this test for the moment.
> 
> Yes, sure, "shell.c" allows a number of operations to be done 
> interactively. It is not so much in the vein of automated stress 
> testing. It was in fact my testing vehicle for a raw terminal program. 
> I'm fine with removing it from the standard test build, but I would be 
> sorry to see it go.

Ok, then we should make the building of this 'test' configurable and add 
a list of requirements for building. Anyone disagrees?

>> Now for the actual design of the synth driver. I think the best way 
>> would be to implement a NAND simulator based on the ONFI 
>> specification. Something similar has been done for the MTD framework, 
>> but I guess other than for inspiration we're not allowed to use that 
>> code. So basically we would simulate the interface to the chip. I 
>> guess we don't have to simulate the signal lines. We just need some 
>> mechanism for chipselect and reset I guess. The interface will more be 
>> along the lines of writing commands, addresses, reading back etc. This 
>> means that the simulator will be implemented as a state machine. There 
>> is even one described in the ONFI specification for reference.
> 
> I never heard of an actual ONFI chip yet. Current chips are usually of 
> the regular large-page kind. It might be most practical to make a 
> generic regular large-page chip emulator that can be parameterized with 
> the Device IDs as enumerated in cyg_nand_chip_id[] in file 
> flash_nand/src/chip/io_nand_chip.c.

Seems like a good idea. Although it might be nice to also simulate the 
more esoteric types. Configuration could be done by just specifying the 
ID bytes. I didn't yet figure out the content of bytes 3-4 though. Are 
they standardized? We could also provide a configuration interface for 
custom chips, so pretty much every configuration could be done, but this 
might well be overkill.

>> I think the basics can be implemented rather quickly. I guess we don't 
>> need to simulate multiple concurrent LUNs, or does the framework 
>> already support these?
> 
> Multiple LUNs is the same as multiple chips, as far as the NAND package 
> (or ONFI) is concerned. It should be supported, but it is untested 
> because I lacked hardware.

I see. I think it's a good idea to first test behaviour with real 
hardware and then see how the simulator needs to be adapted.

>> Well that's about it. I'll try to implement a simple simulator 
>> tomorrow and see where I get. I post back some results as soon as I 
>> have something.

I have the basics running. But I still ended up using a bit of code from 
the MTD NAND simulator code. I hope we can get an agreement with the 
original author to use bits of his code. Anyone thinks this is a problem?

Simon