A static binary patch

A static binary patch

[H.J. Lu]

Here is a patch for glibc 2.2. It can be used to minimize the static
binary size together with the c_stubs addon. You can use

# gcc -static ..... -lc_stubs

It will stub out the gconv stuff. Not everything will work when
-lc_stubs is used. __libc_c_stubs is needed since we have to make
sure libcc_stubs.a is loaded even if nothing is referenced yet.
You can the c_stubs addon at

ftp://ftp.valinux.com/pub/support/hjl/glibc/glibc-c_stubs-2.1.tar.gz

We can add more stuff to c_stubs if necessary.


-- 
H.J. Lu (hjl@gnu.org)
--
Mon Aug 23 17:22:53 1999  H.J. Lu  <hjl@gnu.org>

	* Makeconfig (FILE): Define.

	* Rules (%.out): Run $(FILE) on the binary before running it
	and don't use the dynamic linker if it is not dynamically
	linked.

	* csu/init.c (__libc_c_stubs): New extern.
	(dummy): Reference it.

	* misc/Versions (__libc_c_stubs): Added to GLIBC_2.2.

	* misc/init-misc.c (__libc_c_stubs): New defined.

Index: Makeconfig
===================================================================
RCS file: /work/cvs/gnu/glibc/Makeconfig,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 Makeconfig
--- Makeconfig	1999/08/08 17:01:06	1.1.1.1
+++ Makeconfig	1999/08/23 23:20:28
@@ -493,6 +493,10 @@ ifndef	RANLIB
 RANLIB = ranlib
 endif
 
+ifndef FILE
+FILE := file
+endif
+
 # Extra flags to pass to GCC.
 +gccwarn := -Wall -Wwrite-strings -Winline -Wstrict-prototypes
 
Index: Rules
===================================================================
RCS file: /work/cvs/gnu/glibc/Rules,v
retrieving revision 1.1.1.2
diff -u -p -r1.1.1.2 Rules
--- Rules	1999/08/23 16:16:40	1.1.1.2
+++ Rules	1999/08/23 23:26:41
@@ -119,13 +119,33 @@ ifneq "$(strip $(tests) $(test-srcs))" "
 # These are the implicit rules for making test outputs
 # from the test programs and whatever input files are present.
 $(objpfx)%.out: %.args $(objpfx)% %.input
-	$($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` < $(word 3,$^) > $@
+	$(FILE) $(built-program-file) | grep dynamically > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` < $(word 3,$^) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) `cat $(word 1,$^)` < $(word 3,$^) > $@; \
+	fi
 $(objpfx)%.out: %.args $(objpfx)%
-	$($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` > $@
+	$(FILE) $(built-program-file) | grep dynamically > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) `cat $(word 1,$^)` > $@; \
+	fi
 $(objpfx)%.out: %.input $(objpfx)%
-	$($*-ENV) $(built-program-cmd) < $(word 1,$^) > $@
+	$(FILE) $(built-program-file) | grep dynamically > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) < $(word 1,$^) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) < $(word 1,$^) > $@; \
+	fi
 $(objpfx)%.out: /dev/null $(objpfx)%	# Make it 2nd arg for canned sequence.
-	$($*-ENV) $(built-program-cmd) > $@
+	$(FILE) $(built-program-file) | grep dynamically > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) > $@; \
+	fi
 endif	# tests
 \f
 .PHONY: distclean realclean subdir_distclean subdir_realclean \
Index: csu/init.c
===================================================================
RCS file: /work/cvs/gnu/glibc/csu/init.c,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 init.c
--- csu/init.c	1999/08/08 17:01:09	1.1.1.1
+++ csu/init.c	1999/08/24 01:18:41
@@ -34,3 +34,6 @@
 const int _IO_stdin_used = _G_IO_IO_FILE_VERSION;
 
 #endif
+
+extern int __libc_c_stubs;
+static int *dummy = &__libc_c_stubs;
Index: misc/Versions
===================================================================
RCS file: /work/cvs/gnu/glibc/misc/Versions,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 Versions
--- misc/Versions	1999/08/08 17:01:57	1.1.1.1
+++ misc/Versions	1999/08/24 01:21:55
@@ -103,6 +103,9 @@ libc {
     tdestroy; truncate64;
   }
   GLIBC_2.2 {
+    # global variables
+    __libc_c_stubs;
+
     # m*
     mkdtemp;
   }
Index: misc/init-misc.c
===================================================================
RCS file: /work/cvs/gnu/glibc/misc/init-misc.c,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 init-misc.c
--- misc/init-misc.c	1999/08/08 17:01:57	1.1.1.1
+++ misc/init-misc.c	1999/08/24 01:17:56
@@ -52,3 +52,5 @@ __init_misc (int argc, char **argv, char
 #ifdef HAVE_GNU_LD
 text_set_element (__libc_subinit, __init_misc);
 #endif
+
+int __libc_c_stubs;

Re: A static binary patch

[Mark Kettenis]

   Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
   From: hjl@varesearch.com (H.J. Lu)

   Here is a patch for glibc 2.2. It can be used to minimize the static
   binary size together with the c_stubs addon. You can use

   # gcc -static ..... -lc_stubs

   It will stub out the gconv stuff. Not everything will work when
   -lc_stubs is used. __libc_c_stubs is needed since we have to make
   sure libcc_stubs.a is loaded even if nothing is referenced yet.

Doesn't 

   # gcc -static ..... -( -lc_stubs -lc -)

work without the __libc_c_stubs hack?

Mark

Re: A static binary patch

[Mark Kettenis]

   Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
   From: hjl@varesearch.com (H.J. Lu)

   Here is a patch for glibc 2.2. It can be used to minimize the static
   binary size together with the c_stubs addon. You can use

   Mon Aug 23 17:22:53 1999  H.J. Lu  <hjl@gnu.org>

	   * Makeconfig (FILE): Define.

	   * Rules (%.out): Run $(FILE) on the binary before running it
	   and don't use the dynamic linker if it is not dynamically
	   linked.

What is the point for these changes?  And what do they have to do with
reducing the size of static binaries?  Please avoid becoming dependent
on more external tools.  Using `file' is a violation of the GNU coding
standards.

Mark

Re: A static binary patch

[H.J. Lu]

> 
>    Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
>    From: hjl@varesearch.com (H.J. Lu)
> 
>    Here is a patch for glibc 2.2. It can be used to minimize the static
>    binary size together with the c_stubs addon. You can use
> 
>    Mon Aug 23 17:22:53 1999  H.J. Lu  <hjl@gnu.org>
> 
> 	   * Makeconfig (FILE): Define.
> 
> 	   * Rules (%.out): Run $(FILE) on the binary before running it
> 	   and don't use the dynamic linker if it is not dynamically
> 	   linked.
> 
> What is the point for these changes?  And what do they have to do with
> reducing the size of static binaries?  Please avoid becoming dependent
> on more external tools.  Using `file' is a violation of the GNU coding
> standards.
> 

My c_stubs addon has tests linked statically. When you run "make check"
in c_stubs, it will try to run it with

# ........./ld-linux.so.2 the_c_stubs_test .....

Since the_c_stubs_test is linked statically, it doesn't work. If you
can find a better way to fix it, I will appreciate it.


-- 
H.J. Lu (hjl@gnu.org)

Re: A static binary patch

[H.J. Lu]

> 
>    Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
>    From: hjl@varesearch.com (H.J. Lu)
> 
>    Here is a patch for glibc 2.2. It can be used to minimize the static
>    binary size together with the c_stubs addon. You can use
> 
>    # gcc -static ..... -lc_stubs
> 
>    It will stub out the gconv stuff. Not everything will work when
>    -lc_stubs is used. __libc_c_stubs is needed since we have to make
>    sure libcc_stubs.a is loaded even if nothing is referenced yet.
> 
> Doesn't 
> 
>    # gcc -static ..... -( -lc_stubs -lc -)
> 
> work without the __libc_c_stubs hack?
> 

I removed it in my second patch.

-- 
H.J. Lu (hjl@gnu.org)

Re: A static binary patch

[Andreas Schwab]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

hjl@lucon.org (H.J. Lu) writes:

|> > 
|> >    Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
|> >    From: hjl@varesearch.com (H.J. Lu)
|> > 
|> >    Here is a patch for glibc 2.2. It can be used to minimize the static
|> >    binary size together with the c_stubs addon. You can use
|> > 
|> >    Mon Aug 23 17:22:53 1999  H.J. Lu  <hjl@gnu.org>
|> > 
|> > 	   * Makeconfig (FILE): Define.
|> > 
|> > 	   * Rules (%.out): Run $(FILE) on the binary before running it
|> > 	   and don't use the dynamic linker if it is not dynamically
|> > 	   linked.
|> > 
|> > What is the point for these changes?  And what do they have to do with
|> > reducing the size of static binaries?  Please avoid becoming dependent
|> > on more external tools.  Using `file' is a violation of the GNU coding
|> > standards.
|> > 
|> 
|> My c_stubs addon has tests linked statically. When you run "make check"
|> in c_stubs, it will try to run it with
|> 
|> # ........./ld-linux.so.2 the_c_stubs_test .....
|> 
|> Since the_c_stubs_test is linked statically, it doesn't work. If you
|> can find a better way to fix it, I will appreciate it.

How about extending the ld.so so that if it is called with a static binary
it just execve()s it?

Andreas.

-- 
Andreas Schwab                                  "And now for something
schwab@suse.de                                   completely different."
SuSE GmbH, Schanzäckerstr. 10, D-90443 Nürnberg

Re: A static binary patch

[H.J. Lu]

> How about extending the ld.so so that if it is called with a static binary
> it just execve()s it?
> 

That will work. But before someone implements it, some kludges are
still needed for my c_stubs addon.

-- 
H.J. Lu (hjl@gnu.org)

Re: A static binary patch

[Mark Kettenis]

   From: Andreas Schwab <schwab@suse.de>
   Date: 24 Aug 1999 17:30:19 +0200

   hjl@lucon.org (H.J. Lu) writes:

   |> > 
   |> >    Date: Mon, 23 Aug 1999 18:36:53 -0700 (PDT)
   |> >    From: hjl@varesearch.com (H.J. Lu)
   |> > 
   |> >    Here is a patch for glibc 2.2. It can be used to minimize the static
   |> >    binary size together with the c_stubs addon. You can use
   |> > 
   |> >    Mon Aug 23 17:22:53 1999  H.J. Lu  <hjl@gnu.org>
   |> > 
   |> > 	   * Makeconfig (FILE): Define.
   |> > 
   |> > 	   * Rules (%.out): Run $(FILE) on the binary before running it
   |> > 	   and don't use the dynamic linker if it is not dynamically
   |> > 	   linked.
   |> > 
   |> > What is the point for these changes?  And what do they have to do with
   |> > reducing the size of static binaries?  Please avoid becoming dependent
   |> > on more external tools.  Using `file' is a violation of the GNU coding
   |> > standards.
   |> > 
   |> 
   |> My c_stubs addon has tests linked statically. When you run "make check"
   |> in c_stubs, it will try to run it with
   |> 
   |> # ........./ld-linux.so.2 the_c_stubs_test .....
   |> 
   |> Since the_c_stubs_test is linked statically, it doesn't work. If you
   |> can find a better way to fix it, I will appreciate it.

   How about extending the ld.so so that if it is called with a static binary
   it just execve()s it?

Or use `ld.so --verify' to find out if it is a dynamic executable or not
instead of using `file'.

Mark

Re: A static binary patch

[Ulrich Drepper]

hjl@lucon.org (H.J. Lu) writes:

> # ........./ld-linux.so.2 the_c_stubs_test .....
> 
> Since the_c_stubs_test is linked statically, it doesn't work. If you
> can find a better way to fix it, I will appreciate it.

You simply don't need this.  If you want to run tests do it separately
while not using the standard rules.

-- 
---------------.      drepper at gnu.org  ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com   `------------------------

Re: A static binary patch

[H.J. Lu]

> 
> hjl@lucon.org (H.J. Lu) writes:
> 
> > # ........./ld-linux.so.2 the_c_stubs_test .....
> > 
> > Since the_c_stubs_test is linked statically, it doesn't work. If you
> > can find a better way to fix it, I will appreciate it.
> 
> You simply don't need this.  If you want to run tests do it separately
> while not using the standard rules.
> 

The standard rules should be able to handle it.

-- 
H.J. Lu (hjl@gnu.org)

Re: A static binary patch

[H.J. Lu]

>    |> Since the_c_stubs_test is linked statically, it doesn't work. If you
>    |> can find a better way to fix it, I will appreciate it.
> 
>    How about extending the ld.so so that if it is called with a static binary
>    it just execve()s it?
> 
> Or use `ld.so --verify' to find out if it is a dynamic executable or not
> instead of using `file'.
> 

I like it. Here is the new patch.

-- 
H.J. Lu (hjl@gnu.org)
---
Tue Aug 24 09:13:59 1999  H.J. Lu  <hjl@gnu.org>

	* Rules (%.out): Run "ld.so --verify" on the binary before
	running it and don't use the dynamic linker if it is not
	dynamically linked.

Index: Rules
===================================================================
RCS file: /work/cvs/gnu/glibc/Rules,v
retrieving revision 1.1.1.2
diff -u -p -r1.1.1.2 Rules
--- Rules	1999/08/23 16:16:40	1.1.1.2
+++ Rules	1999/08/24 16:12:37
@@ -119,13 +119,33 @@ ifneq "$(strip $(tests) $(test-srcs))" "
 # These are the implicit rules for making test outputs
 # from the test programs and whatever input files are present.
 $(objpfx)%.out: %.args $(objpfx)% %.input
-	$($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` < $(word 3,$^) > $@
+	$(elf-objpfx)$(rtld-installed-name) --verify $(built-program-file) > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` < $(word 3,$^) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) `cat $(word 1,$^)` < $(word 3,$^) > $@; \
+	fi
 $(objpfx)%.out: %.args $(objpfx)%
-	$($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` > $@
+	$(elf-objpfx)$(rtld-installed-name) --verify $(built-program-file) > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) `cat $(word 1,$^)` > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) `cat $(word 1,$^)` > $@; \
+	fi
 $(objpfx)%.out: %.input $(objpfx)%
-	$($*-ENV) $(built-program-cmd) < $(word 1,$^) > $@
+	$(elf-objpfx)$(rtld-installed-name) --verify $(built-program-file) > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) < $(word 1,$^) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) < $(word 1,$^) > $@; \
+	fi
 $(objpfx)%.out: /dev/null $(objpfx)%	# Make it 2nd arg for canned sequence.
-	$($*-ENV) $(built-program-cmd) > $@
+	$(elf-objpfx)$(rtld-installed-name) --verify $(built-program-file) > /dev/null 2>&1; \
+	if [ $$? = 0 ]; then \
+	  $($*-ENV) $(built-program-cmd) > $@; \
+	else \
+	  $($*-ENV) $(built-program-file) > $@; \
+	fi
 endif	# tests
 \f
 .PHONY: distclean realclean subdir_distclean subdir_realclean \

Re: A static binary patch

[Ulrich Drepper]

hjl@lucon.org (H.J. Lu) writes:

> The standard rules should be able to handle it.

No.  What you implement is something in extensions to the normal and
sane way to use the library.  For the few tests you have to write for
the static stuff simply use separate rules in the appropriate new
subdir.

-- 
---------------.      drepper at gnu.org  ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com   `------------------------

Re: A static binary patch

[Geoff Keating]

> Date: Tue, 24 Aug 1999 07:32:49 -0700 (PDT)
> From: hjl@lucon.org (H.J. Lu)

> My c_stubs addon has tests linked statically. When you run "make check"
> in c_stubs, it will try to run it with
> 
> # ........./ld-linux.so.2 the_c_stubs_test .....
> 
> Since the_c_stubs_test is linked statically, it doesn't work. If you
> can find a better way to fix it, I will appreciate it.

Why not just use 'make' to handle all this?  Create a rule that knows
how to run static programs, variables to list the static programs, and
so on.  I wouldn't be surprised to find that this already exists
somewhere in glibc.

You surely must already have some rule which knows how to build static
tests.

-- 
Geoffrey Keating <geoffk@cygnus.com>

Re: A static binary patch

[H.J. Lu]

> 
> > Date: Tue, 24 Aug 1999 07:32:49 -0700 (PDT)
> > From: hjl@lucon.org (H.J. Lu)
> 
> > My c_stubs addon has tests linked statically. When you run "make check"
> > in c_stubs, it will try to run it with
> > 
> > # ........./ld-linux.so.2 the_c_stubs_test .....
> > 
> > Since the_c_stubs_test is linked statically, it doesn't work. If you
> > can find a better way to fix it, I will appreciate it.
> 
> Why not just use 'make' to handle all this?  Create a rule that knows
> how to run static programs, variables to list the static programs, and

That is what my "ld.so --verify" patch is for.

> so on.  I wouldn't be surprised to find that this already exists
> somewhere in glibc.
> 

I tried. I didn't find it.

H.J.

Re: A static binary patch

[Ulrich Drepper]

hjl@lucon.org (H.J. Lu) writes:

> That is what my "ld.so --verify" patch is for.

No you haven't.  You want everything to be slower and possibly more
unreliable just because of your problem.  Create completely
independent rules then you don't have to change the others.

-- 
---------------.      drepper at gnu.org  ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com   `------------------------

Re: A static binary patch (A new patch)

[H.J. Lu]

> 
> hjl@lucon.org (H.J. Lu) writes:
> 
> > That is what my "ld.so --verify" patch is for.
> 
> No you haven't.  You want everything to be slower and possibly more
> unreliable just because of your problem.  Create completely
> independent rules then you don't have to change the others.
> 

Here is a new patch.


----
Wed Aug 25 09:59:15 1999  H.J. Lu  <hjl@gnu.org>

	* Makeconfig (built-program-cmd): When shared library is
	enabled, if $(tests-static) is not empty, try "ld.so --verify"
	on the binary before running it and don't use the dynamic
	linker if it is not dynamically linked.

	* Rules (binaries-static): Add $(tests-static).

Index: Makeconfig
===================================================================
RCS file: /work/cvs/gnu/glibc/Makeconfig,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 Makeconfig
--- Makeconfig	1999/08/08 17:01:06	1.1.1.1
+++ Makeconfig	1999/08/25 17:45:15
@@ -478,12 +478,22 @@ sysdep-library-path = \
 $(subst $(empty) ,:,$(strip $(patsubst -Wl$(comma)-rpath-link=%, %,\
 				       $(filter -Wl$(comma)-rpath-link=%,\
 						$(sysdep-LDFLAGS)))))
-define built-program-cmd
+define run-built-program-cmd
 $(elf-objpfx)$(rtld-installed-name) \
 	--library-path $(rpath-link)$(patsubst %,:%,$(sysdep-library-path)) \
 	$(built-program-file)
 endef
-endif
+ifeq ("",$(tests-static))
+built-program-cmd = $(run-built-program-cmd)
+else
+define built-program-cmd 
+	$(shell $(elf-objpfx)$(rtld-installed-name) --verify \
+			$(built-program-file) > /dev/null 2>&1; \
+		if [ $$? = 0 ]; then echo $(run-built-program-cmd); \
+		else echo $(built-program-file); fi)
+endef
+endif	# tests-static
+endif	# build-shared
 
 ifndef LD
 LD := ld -X
Index: Rules
===================================================================
RCS file: /work/cvs/gnu/glibc/Rules,v
retrieving revision 1.1.1.2
diff -u -p -r1.1.1.2 Rules
--- Rules	1999/08/23 16:16:40	1.1.1.2
+++ Rules	1999/08/25 17:12:22
@@ -93,7 +93,7 @@ endif
 
 ifeq ($(build-programs),yes)
 binaries-all = $(others) $(sysdep-others) $(tests) $(test-srcs)
-binaries-static = $(others-static)
+binaries-static = $(others-static) $(tests-static)
 else
 binaries-all = $(tests) $(test-srcs)
 binaries-static =

Re: A static binary patch

[Geoff Keating]

> Date: Tue, 24 Aug 1999 22:22:45 -0700 (PDT)
> Cc: libc-hacker@sourceware.cygnus.com (GNU C Library)
> From: hjl@lucon.org (H.J. Lu)
> 
> > 
> > > Date: Tue, 24 Aug 1999 07:32:49 -0700 (PDT)
> > > From: hjl@lucon.org (H.J. Lu)
> > 
> > > My c_stubs addon has tests linked statically. When you run "make check"
> > > in c_stubs, it will try to run it with
> > > 
> > > # ........./ld-linux.so.2 the_c_stubs_test .....
> > > 
> > > Since the_c_stubs_test is linked statically, it doesn't work. If you
> > > can find a better way to fix it, I will appreciate it.
> > 
> > Why not just use 'make' to handle all this?  Create a rule that knows
> > how to run static programs, variables to list the static programs, and
> 
> That is what my "ld.so --verify" patch is for.

That's not what I was thinking of.

I mean, you create a rule that knows how to run static programs,
_without_ using ld.so or file or anything.

Eg. (you will want to fix this a bit to get directories right):

$(staticprogs) : %.out : %
	./$< > $@

Then put your static programs in $(staticprogs).  You must already
have a list of them because otherwise how do they get linked statically?

I imagine glibc can do this already because you can compile (or could
compile) with --disable-shared.

-- 
Geoffrey Keating <geoffk@cygnus.com>