[ECOS] Re: Simple flash filesystem?

["Grant Edwards" <grante@visi.com>]

Lewin A.R.W. Edwards writes:

> >The filesystem will basically contain web pages and Java applets.
> 
> I figured this. So you're dealing with numerous small filelets which will 
> mostly be smaller than the block size, and it is not unreasonable to 
> reserve at least one block for directory information.
> 
> How about this for a solution: Reserve one block (64K) as a directory 
> block. Keep that whole block in RAM at all times. Inside it, have a number 
> of structures like this:
> 
> 32bits pointer to file data or NULL if no file exists
> 32bits file size in bytes
> asciiz variable length filename (\0 for non-file)
> 
> Keep the directory sorted by file pointer.
> 
> When you want to create a new file, you can follow this algorithm:
> 
> * if there is a free block (64K), allocate part of it for the file, erase 
> that block and write the file to the start of it, and update the directory.
> * if there is no completely free block, search for a block that has free 
> space in its tail big enough to accommodate your file. Read that block into 
> RAM, append your new file into the tail, and erase/writeback the block.

And the erase/writeback might not be needed.  If the space is
unused, it may still be all 1's from the last time it was erased.

> The same algorithm can be used to replace an existing file - simply change 
> the existing file's pointer in the directory to indicate that it's no 
> longer in use, then follow the steps above.
> 
> Depending on how power-failure-tolerant you need this to be, 

Not very.  If the power fails while somebody is downloading (either
a new file or overwriting a file), It's fine if that entire file is lost
as long as the filesystem doesn't leak memory or loose other files.  The
latter can be avoided by designing the erase/writeback operation so that
the writeback goes to a different block before the original block is 
deallocated.  It's possible to end up with duplicate files, but that's
easy enough to repair during start-up.

> you can get a 
> big performance improvement on writes because your directory is all in RAM 
> and you only need to commit it back to flash periodically.

Write performance isn't much of a concern.  It would be nice if I could
write on the fly during a TFTP transfer, but not required.
 
> This does not perform any explicit wear leveling, obviously, and it also 
> doesn't make the most efficient use of space (note the lack of support for 
> noncontiguous allocation!). But I think this system might do well for your 
> application where you're mostly dealing with tiny files that are not going 
> to be rewritten frequently.

Right -- I'm not too worried about wear levelling.  The device will
probably
get written to a couple dozen times while the customer tests web pages,
then
it will sit for years.
 
> >I may just forget about writing individual files and force the user
> >to download an entire ROM filesystem everytime anything changes. That
> >would be way simpler, but a little slower.
> 
> I wanted to do this for one of our products, but I was shouted down 
> (legitimately, I think) on the grounds that end-users want to be able to 
> read/write randomly. So the project is sitting on the back-burner right now ;)

The secret is to start with something so horrific that your fallback
position sounds good.  I initially told them that the web pages and
java applets will be linked in with the executable at build time, so 
the customer will have to have a complete toolset and object files.
Which is they way things work in the prototype.

By comparison, running a host-end application to build a ROM filesystem
from a directory tree and then download it will seem brilliant. ;)

I'd be happy to do a nice little filesystem, but I don't think there's
room in the schedule.

-- 
Grant Edwards
grante@visi.com