Re: interface/implementation: export keyword?

Re: interface/implementation: export keyword?

[Nathan Myers]

Jan Reimers wrote:

> > > > I'm trying to sensibly separate template definitions/declarations
> > > > for a project I'm writing in C++, as I believe (?) it's good 
> > > > practice to separate the interface from the implementation.
> > >
> > > The short answer is to put your template declarations in an .h file,
> > > and your template definitions in a .cxx (or .cc) file, and until
> > > egcs supports export, #include the .cxx file as well as the .h file
> > > when you use the templates.
> >
> > Thanks for the advice, but there's two reasons I didn't do this in the
> > first place: ...

It's better to conditionally include the template definitions in the 
headers directly, as:

  // foo.h
  #ifndef INCLUDED_FOO
  # define INCLUDED_FOO
  template <class T> void mangle(T*);
  # ifdef EXPORT_NOT_IMPLEMENTED
  #  define export
  #  include "bar.h" /* needed for mangle() definition */
  #  include "foo.tcc"
  # endif
  #endif

  // foo.tcc
  export template <class T>
    void mangle(T*) {}

This is the solution used in the libstdc++-v3 project.
When export is supported then you read the ".tcc" files
into your template repository.  In the meantime compilation
is slower because each compilation unit sees the whole 
definition.

> > Surely there must be an elegant way of doing this already, or does
> > nobody else use templates and require them to be space-efficient?
>
> I would guess that very few people don't use templates these days.  The
> standard is full of them, even istream and ostream are implemented with
> templates.

It's not a space-efficiency issue; it's a matter of convenience,
size of intermediate files, and compilation time.  The solution
to these is to implement export.

Nathan Myers
ncm@cantrip.org

RE: interface/implementation: export keyword?

[Jan Reimers]

> ----------
> From: 	Terence J. Wells[SMTP:tjw30@cam.ac.uk]
> Reply To: 	tjw30@cam.ac.uk
> Sent: 	Wednesday, October 28, 1998 3:53 AM
> To: 	Joe Buck
> Cc: 	egcs@cygnus.com
> Subject: 	Re: interface/implementation: export keyword?
> 
> 
> > > I'm trying to sensibly separate template definitions/declarations
> for
> > > a project I'm writing in C++, as I believe (?) it's good practice
> to
> > > separate the interface from the implementation.
> > 
> > The short answer is to put your template declarations in an .h file,
> > and your template definitions in a .cxx (or .cc) file, and until
> > egcs supports export, #include the .cxx file as well as the .h file
> > when you use the templates.
> 
> Thanks for the advice, but there's two reasons I didn't do this in the
> first place:
> 
> The first reason is that I couldn't get make to do proper dependency
> checking without rewriting all the rules :-(
> 
SOURCES := $(shell find . -name '*.C')
OBJECTS := $(SOURCES:.C=.o) $(wildcard repo/*.o)
DEPS    := $(SOURCES:.C=.d)

%.d: %.C
	$(SHELL) -ec '$(CXX) -MM $(INCLUDE) $< | sed
'\''s%$(*F).o%$(*D)/& $@%g'\'' > $@'



include $(DEPS)



> Also, I have a template Matrix class (say) which is used by two other
> classes: they both use a Matrix<double>.  If I put the classes in
> separate .cc files I get a bigger executable than if I put them in one
> big .cc file, which (I imagine) means that my final executable has two
> copies of the Matrix<double> code instead of one -- I'm not familiar
> with the structure of the object code so I can't be sure.  Is there
> any way of getting egcs (or the linker) to eliminate redundant
> instantiations? 
You will get bigger object files, but the final executable will have
only one copy of each template function that is used.  To do this egcs
uses 'weak' symbols for which the linker discard duplicates.

> I believe that this can be solved by explicitly instantiating the
> needed classes in the template .cc file and linking against the
> resulting .o file, but this requires some clever dependency checking
> and preprocessor magic beyond my current skills.
Is there really anything to solve?  Just include all template
definitions and let g++/linker deal with the rest.

> How does the STL do it?  Does the compiler have to emit code for all
> the template instantiations itself and put them in executables, or is
> there some magic way libraries use to handle arbitrary types for
> template instantiations?
When optimization is on the compiler only emits non-inline template
functions that are actually used in each translation unit.  

> Surely there must be an elegant way of doing this already, or does
> nobody else use templates and require them to be space-efficient?
I would guess that very few people don't use templates these days.  The
standard is full of them, even istream and ostream are implemented with
templates.

Just try what Joe said, if that doesn't work then you either forgot to
include a .cc file, your you've found a bug.  Also make sure the .cc
files you include from headers don't have any non-template definitions
or you will get multiple definition errors.

> Thanks,
> 
> TJW
> 
> --------------------------------------------------------------------
> Terence J. Wells                E-mail  : tjw30@cam.ac.uk
> St John's College               Phone   : +44.468.916617 (mobile)
> Cambridge CB2 1TP               Room    : A3 New Court, SJC
> United Kingdom                  Talk    : tjw30@tjw30.joh.cam.ac.uk
> 

interface/implementation: export keyword?

[Terence J. Wells]

I'm trying to sensibly separate template definitions/declarations for
a project I'm writing in C++, as I believe (?) it's good practice to
separate the interface from the implementation.

The use of the "export" keyword for template definitions gets flagged
by egcs-1.1b as a syntax error, which I assume means it isn't
supported.  I also tried using the #pragma interface/implementation
construction but it doesn't appear to do anything (or certainly not
what I expect it to).  As a last resort I tried to do it using
preprocessor directives, which works after a fashion but is utterly
unmaintainable.

After reading the egcs info pages and scanning the egcs source (!)
I'm none the wiser, and I couldn't find this as a FAQ anywhere
either.  I even tried looking at the source of some other C++
template libraries (STL, blitz++), but they don't appear to be
enforcing the distinction between interface and implementation at what
I'd thought was an obvious boundary.
 
I'd therefore appreciate some advice from someone who's been there and
done that, preferably with (a pointer to) a simple program
demonstrating the technique ;-)

Thanks,

TJW

--------------------------------------------------------------------
Terence J. Wells                E-mail  : tjw30@cam.ac.uk
St John's College               Phone   : +44.468.916617 (mobile)
Cambridge CB2 1TP               Room    : A3 New Court, SJC
United Kingdom                  Talk    : tjw30@tjw30.joh.cam.ac.uk

Re: interface/implementation: export keyword?

[Joe Buck]

> I'm trying to sensibly separate template definitions/declarations for
> a project I'm writing in C++, as I believe (?) it's good practice to
> separate the interface from the implementation.

The short answer is to put your template declarations in an .h file,
and your template definitions in a .cxx (or .cc) file, and until
egcs supports export, #include the .cxx file as well as the .h file
when you use the templates.

> After reading the egcs info pages and scanning the egcs source (!)
> I'm none the wiser, and I couldn't find this as a FAQ anywhere
> either.  I even tried looking at the source of some other C++
> template libraries (STL, blitz++), but they don't appear to be
> enforcing the distinction between interface and implementation at what
> I'd thought was an obvious boundary.

That's because they work with existing compilers.

Re: interface/implementation: export keyword?

[Terence J. Wells]

> > I'm trying to sensibly separate template definitions/declarations for
> > a project I'm writing in C++, as I believe (?) it's good practice to
> > separate the interface from the implementation.
> 
> The short answer is to put your template declarations in an .h file,
> and your template definitions in a .cxx (or .cc) file, and until
> egcs supports export, #include the .cxx file as well as the .h file
> when you use the templates.

Thanks for the advice, but there's two reasons I didn't do this in the
first place:

The first reason is that I couldn't get make to do proper dependency
checking without rewriting all the rules :-(

Also, I have a template Matrix class (say) which is used by two other
classes: they both use a Matrix<double>.  If I put the classes in
separate .cc files I get a bigger executable than if I put them in one
big .cc file, which (I imagine) means that my final executable has two
copies of the Matrix<double> code instead of one -- I'm not familiar
with the structure of the object code so I can't be sure.  Is there
any way of getting egcs (or the linker) to eliminate redundant
instantiations? 

I believe that this can be solved by explicitly instantiating the
needed classes in the template .cc file and linking against the
resulting .o file, but this requires some clever dependency checking
and preprocessor magic beyond my current skills.

How does the STL do it?  Does the compiler have to emit code for all
the template instantiations itself and put them in executables, or is
there some magic way libraries use to handle arbitrary types for
template instantiations?

Surely there must be an elegant way of doing this already, or does
nobody else use templates and require them to be space-efficient?

Thanks,

TJW

--------------------------------------------------------------------
Terence J. Wells                E-mail  : tjw30@cam.ac.uk
St John's College               Phone   : +44.468.916617 (mobile)
Cambridge CB2 1TP               Room    : A3 New Court, SJC
United Kingdom                  Talk    : tjw30@tjw30.joh.cam.ac.uk

Re: interface/implementation: export keyword?

[Joe Buck]

> > The short answer is to put your template declarations in an .h file,
> > and your template definitions in a .cxx (or .cc) file, and until
> > egcs supports export, #include the .cxx file as well as the .h file
> > when you use the templates.
> 
> Thanks for the advice, but there's two reasons I didn't do this in the
> first place:
> 
> The first reason is that I couldn't get make to do proper dependency
> checking without rewriting all the rules :-(

I don't understand this.  Standard approaches for "make depend" will
automatically generate the dependencies, based only on who #includes
what.

> Also, I have a template Matrix class (say) which is used by two other
> classes: they both use a Matrix<double>.  If I put the classes in
> separate .cc files I get a bigger executable than if I put them in one
> big .cc file, which (I imagine) means that my final executable has two
> copies of the Matrix<double> code instead of one --

No, if you use the GNU linker, the duplicate copies will be eliminated.

> How does the STL do it?  Does the compiler have to emit code for all
> the template instantiations itself and put them in executables,

No, the compiler puts the instantiations in object files where they
occur, but with a special symbol type called a weak symbol.  If there
are multiple definitions, the linker chooses one and ignores the others.
If you are using a non-GNU linker you will have duplicate copies in
the executable.

> Surely there must be an elegant way of doing this already, or does
> nobody else use templates and require them to be space-efficient?

If you use the GNU linker and are on an ELF system (like Linux, Solaris,
or an SVR4-based Unix), your templates are space-efficient, though your
.o files may be costing you extra space.

For other platforms, there is an alternate method: the -frepo flag.
When it is used, the compiler still must see the template definitions,
but it does not immediately expand them if they are not inline functions.
A link-time pass will cause the correct templates to be expanded.

Re: interface/implementation: export keyword?

[Martin von Loewis]

> Surely there must be an elegant way of doing this already, or does
> nobody else use templates and require them to be space-efficient?

egcs currently uses two techniques by default to safe space in
template instantiation:

a) In implicit instantiations, it emits code only if it is used in the
   same translation unit. For example, if you instantiate a list<Foo>&
   in a parameter, and then use only empty() on that parameter, it only
   emits code for .empty() (this is the theory, some other things get
   instantiated which would not need to)

b) If the same function/method is emitted multiple times in different
   object files, the linker will remove the duplicates (if it's a GNU
   linker)

This together more or less makes all other techniques
(-fno-implicit-templates, #pragma, -frepo) obsolete. 'export' will
survive (once it is born :-), as it is required by the standard.

Regards,
Martin