[ECOS] Adding checksum to elf file

[ECOS] Adding checksum to elf file

[Christopher Cordahi]

Hello ecos

I'd like to add some sort of checksum into the .elf files.
Is there a standard way of doing this?

Googling I see some mention of a .dynamic tag DT_CHECKSUM
but not how to use it properly
and a short discussion in
http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
but it doesn't seem very standard.

Any ideas?
-- 
Chris

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Sergei Gavrikov]

On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> Hello ecos
> 
> I'd like to add some sort of checksum into the .elf files.
> Is there a standard way of doing this?
> 
> Googling I see some mention of a .dynamic tag DT_CHECKSUM
> but not how to use it properly
> and a short discussion in
> http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> but it doesn't seem very standard.
> 
> Any ideas?

It seems, it's more suitable to verify ELF using a parallel md5sum file
That is 128-bit! There are sources http://www.ietf.org/rfc/rfc1321.txt.

Sergei

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Christopher Cordahi]

On 05/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> > Hello ecos
> >
> > I'd like to add some sort of checksum into the .elf files.
> > Is there a standard way of doing this?
> >
> > Googling I see some mention of a .dynamic tag DT_CHECKSUM
> > but not how to use it properly
> > and a short discussion in
> > http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> > but it doesn't seem very standard.
> >
> > Any ideas?
>
> It seems, it's more suitable to verify ELF using a parallel md5sum file
> That is 128-bit! There are sources http://www.ietf.org/rfc/rfc1321.txt.

Thanks,
However I only want it to detect accidental file corruption, something that
can be added to the file at file creation and can be easily verified by the
program loader (redboot).  At most I was thinking about implementing a
32 bit CRC algorithm and probably only a 16 bits.  Maintaining and
distributing only 1 file is much easier than 2 so I am hoping to add the
CRC to the file in a standard way.  If there is no such standard I'll have
to resort to a method similar to that mentioned in the discussion.

-- 
Chris

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Sergei Gavrikov]

On Fri, Jun 08, 2007 at 10:08:28PM -0400, Christopher Cordahi wrote:
> On 05/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> >On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> >> Hello ecos
> >>
> >> I'd like to add some sort of checksum into the .elf files.
> >> Is there a standard way of doing this?
> >>
> >> Googling I see some mention of a .dynamic tag DT_CHECKSUM
> >> but not how to use it properly
> >> and a short discussion in
> >> http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> >> but it doesn't seem very standard.
> >>
> >> Any ideas?
> >
> >It seems, it's more suitable to verify ELF using a parallel md5sum file
> >That is 128-bit! There are sources http://www.ietf.org/rfc/rfc1321.txt.
> 
> Thanks,
> However I only want it to detect accidental file corruption, something that
> can be added to the file at file creation and can be easily verified by the
> program loader (redboot).  At most I was thinking about implementing a
> 32 bit CRC algorithm and probably only a 16 bits.  Maintaining and
> distributing only 1 file is much easier than 2 so I am hoping to add the
> CRC to the file in a standard way.  If there is no such standard I'll have
> to resort to a method similar to that mentioned in the discussion.

I have an idea to do it with no blood. You won't need tweak ELF. You can
use zipped ELF and use RedBoot's [fis load] command with -d option
(decompress).  Any corruption in your gzip file will be found by gzip
layer as well. Gzip has a nice crc checking. It quite works. There is a
demo below

Let's do it in shell

make hello
gzip hello
nano hello.gz ; imitate a corruption :-)

and this one let's do in RedBoot

load -r -b %{freememlo}
fis create hello.gz
fis load -d hello.gz
decompression error: invalid literal/lengths set

If you can see, we catch that corruption before a run! More that, I
build-in GUNZIP behavior in RedBoot every time to decrease an upload
time when I use Ymodem to load ELFs. Gzipped, even non-stripped ELF
files don't eat my FLASH.

Note: your redboot_ROM.ecm should contain such things

cdl_configuration eCos {
...
    package CYGPKG_COMPRESS_ZLIB current ;
};

...

cdl_option CYGBLD_BUILD_REDBOOT_WITH_GUNZIP {
    user_value 1
};


-- Sergei

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Sergei Gavrikov]

On Sat, Jun 09, 2007 at 11:46:18AM +0300, Sergei Gavrikov wrote:
> On Fri, Jun 08, 2007 at 10:08:28PM -0400, Christopher Cordahi wrote:
> > On 05/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> > >On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> > >> Hello ecos
> > >>
> > >> I'd like to add some sort of checksum into the .elf files.
> > >> Is there a standard way of doing this?
> > >>
> > >> Googling I see some mention of a .dynamic tag DT_CHECKSUM
> > >> but not how to use it properly
> > >> and a short discussion in
> > >> http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> > >> but it doesn't seem very standard.
> > >>
> > >> Any ideas?
> > >
> > >It seems, it's more suitable to verify ELF using a parallel md5sum file
> > >That is 128-bit! There are sources http://www.ietf.org/rfc/rfc1321.txt.
> > 
> > Thanks,
> > However I only want it to detect accidental file corruption, something that
> > can be added to the file at file creation and can be easily verified by the
> > program loader (redboot).  At most I was thinking about implementing a
> > 32 bit CRC algorithm and probably only a 16 bits.  Maintaining and
> > distributing only 1 file is much easier than 2 so I am hoping to add the
> > CRC to the file in a standard way.  If there is no such standard I'll have
> > to resort to a method similar to that mentioned in the discussion.
> 
> I have an idea to do it with no blood. You won't need tweak ELF. You can
> use zipped ELF and use RedBoot's [fis load] command with -d option
  ^^^^^^^^^^^^^^

Ups, I should correct myself. That should be compressed binary BFD.

> (decompress).  Any corruption in your gzip file will be found by gzip
> layer as well. Gzip has a nice crc checking. It quite works. There is a
> demo below
> 
> Let's do it in shell
> 
> make hello

objcopy -O binary hello ;# <-- I passed this conversion

> gzip hello
> nano hello.gz ; imitate a corruption :-)
> 
> and this one let's do in RedBoot
> 
> load -r -b %{freememlo}
> fis create hello.gz
> fis load -d hello.gz
> decompression error: invalid literal/lengths set
> 
> If you can see, we catch that corruption before a run! More that, I
> build-in GUNZIP behavior in RedBoot every time to decrease an upload
> time when I use Ymodem to load ELFs. Gzipped, even non-stripped ELF
> files don't eat my FLASH.
> 
> Note: your redboot_ROM.ecm should contain such things
> 
> cdl_configuration eCos {
> ...
>     package CYGPKG_COMPRESS_ZLIB current ;
> };
> 
> ...
> 
> cdl_option CYGBLD_BUILD_REDBOOT_WITH_GUNZIP {
>     user_value 1
> };
> 
> 
> -- Sergei

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Christopher Cordahi]

On 10/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> On Sat, Jun 09, 2007 at 11:46:18AM +0300, Sergei Gavrikov wrote:
> > On Fri, Jun 08, 2007 at 10:08:28PM -0400, Christopher Cordahi wrote:
> > > On 05/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> > > >On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> > > >> Hello ecos
> > > >>
> > > >> I'd like to add some sort of checksum into the .elf files.
> > > >> Is there a standard way of doing this?
> > > >>
> > > >> Googling I see some mention of a .dynamic tag DT_CHECKSUM
> > > >> but not how to use it properly
> > > >> and a short discussion in
> > > >> http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> > > >> but it doesn't seem very standard.
> > > >>
> > > >> Any ideas?
> > > >
> > > >It seems, it's more suitable to verify ELF using a parallel md5sum file
> > > >That is 128-bit! There are sources http://www.ietf.org/rfc/rfc1321.txt.
> > >
> > > Thanks,
> > > However I only want it to detect accidental file corruption, something that
> > > can be added to the file at file creation and can be easily verified by the
> > > program loader (redboot).  At most I was thinking about implementing a
> > > 32 bit CRC algorithm and probably only a 16 bits.  Maintaining and
> > > distributing only 1 file is much easier than 2 so I am hoping to add the
> > > CRC to the file in a standard way.  If there is no such standard I'll have
> > > to resort to a method similar to that mentioned in the discussion.
> >
> > I have an idea to do it with no blood. You won't need tweak ELF. You can
> > use zipped ELF and use RedBoot's [fis load] command with -d option
>   ^^^^^^^^^^^^^^
>
> Ups, I should correct myself. That should be compressed binary BFD.
>
> > (decompress).  Any corruption in your gzip file will be found by gzip
> > layer as well. Gzip has a nice crc checking. It quite works. There is a
> > demo below
> >
> > Let's do it in shell
> >
> > make hello
>
> objcopy -O binary hello ;# <-- I passed this conversion
>
> > gzip hello
> > nano hello.gz ; imitate a corruption :-)
> >
> > and this one let's do in RedBoot
> >
> > load -r -b %{freememlo}
> > fis create hello.gz
> > fis load -d hello.gz
> > decompression error: invalid literal/lengths set
> >
> > If you can see, we catch that corruption before a run! More that, I
> > build-in GUNZIP behavior in RedBoot every time to decrease an upload
> > time when I use Ymodem to load ELFs. Gzipped, even non-stripped ELF
> > files don't eat my FLASH.
> >
> > Note: your redboot_ROM.ecm should contain such things
> >
> > cdl_configuration eCos {
> > ...
> >     package CYGPKG_COMPRESS_ZLIB current ;
> > };
> >
> > ...
> >
> > cdl_option CYGBLD_BUILD_REDBOOT_WITH_GUNZIP {
> >     user_value 1
> > };
> >
> >
> > -- Sergei
>

Thanks,
that's a very good solution, I hadn't noticed the zip option,
I also like the idea of saving 50 % of the Flash and transfer time.
also less data = less chance of corruption.

How much longer does it take to do the decompression?
Loading the .elf file takes long enough.
However reading less data from slow Flash should speed it up.
Guess I'll just have to try it out.

Thanks again,
-- 
Chris

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Adding checksum to elf file

[Sergei Gavrikov]

On Thu, Jun 14, 2007 at 11:46:48AM -0400, Christopher Cordahi wrote:
> On 10/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> >On Sat, Jun 09, 2007 at 11:46:18AM +0300, Sergei Gavrikov wrote:
> >> On Fri, Jun 08, 2007 at 10:08:28PM -0400, Christopher Cordahi wrote:
> >> > On 05/06/07, Sergei Gavrikov <w3sg@softhome.net> wrote:
> >> > >On Tue, Jun 05, 2007 at 01:49:36PM -0400, Christopher Cordahi wrote:
> >> > >> Hello ecos
> >> > >>
> >> > >> I'd like to add some sort of checksum into the .elf files.
> >> > >> Is there a standard way of doing this?
> >> > >>
> >> > >> Googling I see some mention of a .dynamic tag DT_CHECKSUM
> >> > >> but not how to use it properly
> >> > >> and a short discussion in
> >> > >> http://ecos.sourceware.org/ml/binutils/2003-02/msg00242.html
> >> > >> but it doesn't seem very standard.
> >> > >>
> >> > >> Any ideas?
> >> > >
> >> > >It seems, it's more suitable to verify ELF using a parallel md5sum 
> >file
> >> > >That is 128-bit! There are sources 
> >http://www.ietf.org/rfc/rfc1321.txt.
> >> >
> >> > Thanks,
> >> > However I only want it to detect accidental file corruption, something 
> >that
> >> > can be added to the file at file creation and can be easily verified 
> >by the
> >> > program loader (redboot).  At most I was thinking about implementing a
> >> > 32 bit CRC algorithm and probably only a 16 bits.  Maintaining and
> >> > distributing only 1 file is much easier than 2 so I am hoping to add 
> >the
> >> > CRC to the file in a standard way.  If there is no such standard I'll 
> >have
> >> > to resort to a method similar to that mentioned in the discussion.
> >>
> >> I have an idea to do it with no blood. You won't need tweak ELF. You can
> >> use zipped ELF and use RedBoot's [fis load] command with -d option
> >  ^^^^^^^^^^^^^^
> >
> >Ups, I should correct myself. That should be compressed binary BFD.
> >
> >> (decompress).  Any corruption in your gzip file will be found by gzip
> >> layer as well. Gzip has a nice crc checking. It quite works. There is a
> >> demo below
> >>
> >> Let's do it in shell
> >>
> >> make hello
> >
> >objcopy -O binary hello ;# <-- I passed this conversion
> >
> >> gzip hello
> >> nano hello.gz ; imitate a corruption :-)
> >>
> >> and this one let's do in RedBoot
> >>
> >> load -r -b %{freememlo}
> >> fis create hello.gz
> >> fis load -d hello.gz
> >> decompression error: invalid literal/lengths set
> >>
> >> If you can see, we catch that corruption before a run! More that, I
> >> build-in GUNZIP behavior in RedBoot every time to decrease an upload
> >> time when I use Ymodem to load ELFs. Gzipped, even non-stripped ELF
> >> files don't eat my FLASH.
> >>
> >> Note: your redboot_ROM.ecm should contain such things
> >>
> >> cdl_configuration eCos {
> >> ...
> >>     package CYGPKG_COMPRESS_ZLIB current ;
> >> };
> >>
> >> ...
> >>
> >> cdl_option CYGBLD_BUILD_REDBOOT_WITH_GUNZIP {
> >>     user_value 1
> >> };
> >>
> >>
> >> -- Sergei
> >
> 
> Thanks, that's a very good solution, I hadn't noticed the zip option,
> I also like the idea of saving 50 % of the Flash and transfer time.
> also less data = less chance of corruption.
>
> How much longer does it take to do the decompression?  Loading the
> .elf file takes long enough.  However reading less data from slow
> Flash should speed it up.  Guess I'll just have to try it out.
> 
> Thanks again,
> -- 
> Chris

My experience show that ratio aprox. 3:1 for ELFs, and one my FPGA RBF
file, for example, in 78K becomes itself a 18k gzcompressed image. I
decompress it using a procedure like do_gunzip(), look at the original:
redboot/current/src/gunzip.c. It's very fast. Any decompression is quite
faster than a compression.

Yet another way to decompess ELF on a fly. Try in RedBoot (using some
X/Y/Zmodem) to load and run gzipped ELF (be sure that you have zlib
support in your RedBoot)

RedBoot> load -d -m y
RedBoot> go

I use that every time. My `install' rule in Makefile gzips an ELF and
places the result image in a minicom download directory.


--Sergei




-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss