Large map<srting,string> Initialization

Large map<srting,string> Initialization

[Tom Browder]

I need a map with many string pairs and I instantiate it like this:

  map<const string,const string> mymap;
  mymap.insert(make_pair("long_string1", "long_string_val1"));
  ...
  mymap.insert(make_pair("long_string10000", "long_string_val10000"));

The strings generally are about 30-40 characters in length.

The source file is very long and g++ (4.0) takes a very long time (and a
large amount of memory) to compile it.

I have broken the file into smaller pieces (which helps with memory usage)
but the total compile time is about the same.

Note that the map is never modified after the initial load, it is merely
used for looking up the values.

Several questions:

1.  Is there a better way to initialize such a map?

2.  Are there g++ options that would help?

3.  Are there linker/loader tricks that would help?

Thanks.

Tom Browder

Re: Large map<srting,string> Initialization

[Stefan Strasser]

Tom Browder schrieb:
> I need a map with many string pairs and I instantiate it like this:
> 
>   map<const string,const string> mymap;

first const is not required, key type of a map is always const.

>   mymap.insert(make_pair("long_string1", "long_string_val1"));
>   ...
>   mymap.insert(make_pair("long_string10000", "long_string_val10000"));
> 
> The strings generally are about 30-40 characters in length.
> 
> The source file is very long and g++ (4.0) takes a very long time (and a
> large amount of memory) to compile it.

GCC does not have a very good algorithm complexity in this case it seems.
(I tried, doubling the "insert" statements quintupled compilation time).

this is compiled much faster(and will run faster):

std::pair<std::string,std::string> a[]={
	std::make_pair("asdklfj","kljsfd"),
	// ...
};

std::map<std::string,std::string> b(a,a + (sizeof(a) / sizeof(*a)));

> 
> I have broken the file into smaller pieces (which helps with memory usage)
> but the total compile time is about the same.
> 
> Note that the map is never modified after the initial load, it is merely
> used for looking up the values.
> 
> Several questions:
> 
> 1.  Is there a better way to initialize such a map?
> 
> 2.  Are there g++ options that would help?
> 
> 3.  Are there linker/loader tricks that would help?
> 
> Thanks.
> 
> Tom Browder
> 
> 
> 



-- 
Stefan Strasser

RE: Large map<srting,string> Initialization

[Tom Browder]

Thanks, Stefan, I'll try your suggestions.  They should help immensely.

-Tom

Re: Large map<srting,string> Initialization

[Florian Weimer]

* Tom Browder:

> 1.  Is there a better way to initialize such a map?

You could use a (preferably sorted) array of {"long_string1",
"long_string_val1"} structure values to initialize the map.  Depending
on your needs, you might be able to skip the map altogether and just
use a binary search to locate the correct strings.