loading of shared objects and executables

loading of shared objects and executables

[Michael Zintakis]

Though not strictly 100% GCC subject, it is still GCC-related. A couple 
of queries I am hoping someone would be able to address for me.

I am trying to find out what is the process of loading a shared object 
file (.so) and the process of loading and executing a binary file (like 
/bin/bash for example) and what part of the Linux system (or files - 
like ld-linux.so may be?) is responsible for that process?

In other words, when I try to execute a file, I presume "something" 
loads it first into memory, resolves what other shared objects need to 
be loaded and then loads them, before passing the execution to the entry 
point of that executable. I am not interested in the process of the 
actual execution or resolving the external references - just the loading 
process of shared objects and executables, nothing more than that (at 
least not at this stage anyway).

Why would I like to know that?

I am trying to "intervene" in this process, so that before every file 
(shared object or executable) is loaded, I "verify" it and then pass the 
appropriate control to the "standard" loaders for either execution or 
resolving external references, or raise an exception if I don't like the 
file or the shared object about to be loaded.

All this would be done in a Linux environment (x86 - both 32 and 64-bit, 
as well as ARM), no Windoze! I would be glad if someone could give me a 
few pointers - at least to start with. Thanks!


MZ

Re: loading of shared objects and executables

[Marc Glisse]

On Sat, 20 Oct 2012, Michael Zintakis wrote:

> I am trying to find out what is the process of loading a shared object file 
> (.so) and the process of loading and executing a binary file (like /bin/bash 
> for example) and what part of the Linux system (or files - like ld-linux.so 
> may be?) is responsible for that process?

I guess you could start from here:
http://en.wikipedia.org/wiki/Dynamic_linker
and follow the links.

-- 
Marc Glisse

RE: loading of shared objects and executables

[Bolshakov, Roman]

Hi, Mike.

I also studied that in the spring. You can find a lot of details on the page: http://www.acsu.buffalo.edu/~charngda/elf.html


Dynamic linker is loaded by kernel after the kernel reads ELF headers. Then it passes control to the entry point of the linker. If you use glibc the entry point is named _dl_start. So, in general, you need to make breakpoint on _dl_start.

Roman

Re: loading of shared objects and executables

[Michael Zintakis]

Hi Roman,


Bolshakov, Roman wrote:
> I also studied that in the spring. You can find a lot of details on the page: http://www.acsu.buffalo.edu/~charngda/elf.html
>   
Very good reference this, thank you - I'll look into it in more detail 
over the weekend! OK, in a nutshell, what I am after is, basically, two 
things:

1. Find the portion of the kernel code, which is responsible for the 
initial loading the file for execution (i.e. when I type "iptables" from 
the command line for example). This will help me to modify that code so 
that the file which is about to be loaded is "verified" first and raise 
a security exception if something is wrong with it.
2. Once I've done that, if there is a possibility to identify all .so 
(shared objects) on which this executable depends (by looking at its 
headers for example), I could do the same as step 1 above for all 
dependent .so files. If not, then I'll have to pass control to ld.so and 
intercept the point prior to where these .so are about to be loaded, so 
that I could do the verification - as in step one above, but for the 
shared objects. If everything checks out, then I'll pass control over to 
ld.so.

I am guessing the picture when statically compiled executables are run 
is more or less the same with the exception of loading тхе shared 
libraries, as they are included as part of the file which needs to be 
executed - am I correct in thinking that?

> Dynamic linker is loaded by kernel after the kernel reads ELF headers.
And it is precisely at that point I need to take control to "verify" the 
executable and, if possible, all .so files on which this file depends.

>  Then it passes control to the entry point of the linker. If you use glibc the entry point is named _dl_start. So, in general, you need to make breakpoint on _dl_start.
>   
I guess I would need to do that only if I could not find the .so files 
on which this executable depends, otherwise the verification could be 
done much earlier and I could then let ld.so do its job, right?

Re: loading of shared objects and executables

[Ángel González]

On 27/10/12 01:28, Michael Zintakis wrote:
> Hi Roman,
>
>
> Bolshakov, Roman wrote:
>> I also studied that in the spring. You can find a lot of details on
>> the page: http://www.acsu.buffalo.edu/~charngda/elf.html
>>   
> Very good reference this, thank you - I'll look into it in more detail
> over the weekend! OK, in a nutshell, what I am after is, basically,
> two things:
>
> 1. Find the portion of the kernel code, which is responsible for the
> initial loading the file for execution (i.e. when I type "iptables"
> from the command line for example). This will help me to modify that
> code so that the file which is about to be loaded is "verified" first
> and raise a security exception if something is wrong with it.
> 2. Once I've done that, if there is a possibility to identify all .so
> (shared objects) on which this executable depends (by looking at its
> headers for example), I could do the same as step 1 above for all
> dependent .so files. If not, then I'll have to pass control to ld.so
> and intercept the point prior to where these .so are about to be
> loaded, so that I could do the verification - as in step one above,
> but for the shared objects. If everything checks out, then I'll pass
> control over to ld.so.
>
> I am guessing the picture when statically compiled executables are run
> is more or less the same with the exception of loading тхе shared
> libraries, as they are included as part of the file which needs to be
> executed - am I correct in thinking that?
>
>> Dynamic linker is loaded by kernel after the kernel reads ELF headers.
> And it is precisely at that point I need to take control to "verify"
> the executable and, if possible, all .so files on which this file
> depends.
>
>>  Then it passes control to the entry point of the linker. If you use
>> glibc the entry point is named _dl_start. So, in general, you need to
>> make breakpoint on _dl_start.
>>   
> I guess I would need to do that only if I could not find the .so files
> on which this executable depends, otherwise the verification could be
> done much earlier and I could then let ld.so do its job, right?
>
Don't forget about dlopen()
The appropiate place would generally be on the dynamic linker, except
for static binaries.

I would probably try to hook the mmap() calls with PROT_EXEC and block
them if it isn't a verified file.

A verified binary could bypass it by loading another program in a
different way, but as I guess you wouldn't "verify" such binary, that's
probably ok, since normal users will use mmap(). The problem of
verifying binaries (scripts) in dynamic languages, is worse anyway.

Re: loading of shared objects and executables

[Michael Zintakis]

> Don't forget about dlopen()
> The appropiate place would generally be on the dynamic linker, except
> for static binaries.
>
> I would probably try to hook the mmap() calls with PROT_EXEC and block
> them if it isn't a verified file.
>   
Duly noted, thanks.

> A verified binary could bypass it by loading another program in a
> different way, but as I guess you wouldn't "verify" such binary, that's
> probably ok, since normal users will use mmap().
I would aim to prevent such a case - I'll seek to modify the appropriate 
kernel functions responsible for loading/execution (process.c, exec.c 
etc) and will only allow the binary formats which can be verified to be 
loaded/executed. This should be OK, at least in my case, because I plan 
to use this on a hardened system, so the case of loading anything else, 
apart from elf-type binaries is not likely. Another possible way of 
"preventing" verification would be to replace ld.so with a rogue one, 
but I'll make the kernel check that as well.

Currently, I am torn between adopting two quite different approaches 
(still have quite a bit to catch up on on the good reference Roman was 
kind enough to provide though): 1. attach all verification data at the 
end of a given executable/.so file (by "verification data" I mean, at 
the very least, a hash on the entire executable/.so file, calculated 
using a private key, and a signature ID - as text - for the public key 
to be used to verify that hash); or 2. create a separate header/section 
(called ".security" for example) and attach that verification data there.

I'm inclined to use the second approach, not least because it could 
survive any potential stripping, though it would be more involved as I 
would have to, somehow, instruct the linker to create this additional 
section and also include the verification data just after the 
executable/.so file is built (in other words, to invoke the creation of 
the verification data as part of the linking process). If I adopt this 
route, I am still to figure out how to go about this as my knowledge of 
the gcc linker doesn't go that deep (yet!).

The first approach - to attach the verification data at the end of the 
file - has the benefit of adding verification data on files already 
built, without the need to recompile them, but as I pointed out above, I 
would try the "custom" header section first and if I can't pursue it, 
then I'll adopt this one.

>  The problem of verifying binaries (scripts) in dynamic languages, is worse anyway.
>   
Indeed. Java, perl, python, bash, to just name a few - can't be 
protected against, not to mention executing a "verified" "dd 
if=/dev/zero of=/dev/sda bs=512 count=1000" for example, but no defence 
or protection built is ever 100% secure, so of course there are 
potential exploits no matter what system is used.

RE: loading of shared objects and executables

[Bolshakov, Roman]

> Currently, I am torn between adopting two quite different approaches 
> (still have quite a bit to catch up on on the good reference Roman was 
> kind enough to provide though): 1. attach all verification data at the 
> end of a given executable/.so file (by "verification data" I mean, at 
> the very least, a hash on the entire executable/.so file, calculated 
> using a private key, and a signature ID - as text - for the public key 
> to be used to verify that hash); or 2. create a separate header/section 
> (called ".security" for example) and attach that verification data there.

As Linux uses ELF, I personally like the second approach as it's much more consistent.

> I'm inclined to use the second approach, not least because it could 
> survive any potential stripping, though it would be more involved as I 
> would have to, somehow, instruct the linker to create this additional 
> section and also include the verification data just after the 
> executable/.so file is built (in other words, to invoke the creation of 
> the verification data as part of the linking process). If I adopt this 
> route, I am still to figure out how to go about this as my knowledge of 
> the gcc linker doesn't go that deep (yet!).

To make a custom section layout you can use linker scripts (http://sourceware.org/binutils/docs/ld/Scripts.html).

Re: loading of shared objects and executables

[Michael Zintakis]

Hi Roman,


>> or 2. create a separate header/section 
>> (called ".security" for example) and attach that verification data there.
>>     
>
> As Linux uses ELF, I personally like the second approach as it's much more consistent.
>   
Indeed, much easier to create/maintain and is quite flexible as well (I 
could have multiple sections for multiple purposes - .security.sig.hash, 
.security.sig.name, .security.flags for example).

> To make a custom section layout you can use linker scripts (http://sourceware.org/binutils/docs/ld/Scripts.html).
>   
Yep, that was another good one - thank you! For already-built binaries I 
could use objcopy (I tested it earlier today and it is flawless!), so 
that folds up nicely for compiling new ELF binaries.

Re: loading of shared objects and executables

[Ángel González]

On 27/10/12 16:02, Michael Zintakis wrote:
>> A verified binary could bypass it by loading another program in a
>> different way, but as I guess you wouldn't "verify" such binary, that's
>> probably ok, since normal users will use mmap().
> I would aim to prevent such a case - I'll seek to modify the
> appropriate kernel functions responsible for loading/execution
> (process.c, exec.c etc) and will only allow the binary formats which
> can be verified to be loaded/executed. This should be OK, at least in
> my case, because I plan to use this on a hardened system, so the case
> of loading anything else, apart from elf-type binaries is not likely.
> Another possible way of "preventing" verification would be to replace
> ld.so with a rogue one, but I'll make the kernel check that as well.

No, I was refering to a "userland loader". Such as instead of using
mmap() with executable protection, doing a malloc() and then mprotect()
[not actually supported by POSIX, but it works in Linux].
As you are making a hardened system, you probably also want to forbid
writable and executable memory (when possible), so it will remove many
holes of that kind.

Still, the "rogue loader" cannot be completely detected just by kernel.
The extreme example would be a program that parsed each instruction and
executed it.
As I said, it is very unlikely that such program got verified :)

RE: loading of shared objects and executables

[Bolshakov, Roman]

Hi Mike,

> 2. Once I've done that, if there is a possibility to identify all .so 
> (shared objects) on which this executable depends (by looking at its 
> headers for example), I could do the same as step 1 above for all 
> dependent .so files. If not, then I'll have to pass control to ld.so and 
> intercept the point prior to where these .so are about to be loaded, so 
> that I could do the verification - as in step one above, but for the 
> shared objects. If everything checks out, then I'll pass control over to 
> ld.so.

I'm only not sure if the kernel know all the sorts of ELF sections and segments ld.so can handle. 
Probably you'll need to modify kernel code to walk over DYNAMIC segment (which keeps the list
of dependencies) to verify the shared objects an application depends on.

> I am guessing the picture when statically compiled executables are run 
> is more or less the same with the exception of loading тхе shared 
> libraries, as they are included as part of the file which needs to be 
> executed - am I correct in thinking that?

Yes, but a statically linked executable can also call dlopen(), mmap() (As it was noted later).

Re: loading of shared objects and executables

[Michael Zintakis]

Hi Roman,


> I'm only not sure if the kernel know all the sorts of ELF sections and segments ld.so can handle. 
> Probably you'll need to modify kernel code to walk over DYNAMIC segment (which keeps the list
> of dependencies) to verify the shared objects an application depends on.
>   
It looks as though I have to reconsider my initial approach - even 
though I could read that segment (not an easy task though), there is no 
way the kernel alone could find all dependencies based on that section 
(there could be .so files, which have dependencies on other .so files 
and so on), so I have to change ld.so in order to be able to do what the 
kernel does and verify the files to be loaded. That would be in addition 
to the kernel doing the verification of the initial executable.

> Yes, but a statically linked executable can also call dlopen(), mmap() (As it was noted later).
>   
Yep, and given what I wrote in the previous paragraph above, it looks as 
though ld.so also needs to be capable of doing the verification.

On a slightly brighter node, (dynamically) adding new sections to ELF 
turned out to be an easy task, even if the executable/.so file is 
already compiled/built - all I have to do is execute "objcopy 
--add-section <section_name>=<section_file> <infile>" and then "objcopy 
--set-section-flags <section_name>=<flags> <infile>" to set the 
appropriate flags to <section_name> ("readonly" in my case). As soon as 
this is done, the section is added (with its content set from the 
content of <section_file>) and the appropriate reallocation of the ELF 
binary is "automatically" done by objcopy.

My initial intent was to create a new section in the ELF header, but I 
could not find an easy way to do that either during compile time or when 
the executable/.so file is already built, so placing my "custom" section 
in the ELF section header by using objcopy seems a good choice and I 
don't have to recompile everything.

Next, I'll look for ways in which the binary/.so loading could be 
bypassed (to prevent verification) so that when I change the kernel and 
the ld.so code later on, to prevent that from happening and enforce 
verification in all cases, when desired.