corrupted profile info. Unable to use gcda profile data

corrupted profile info. Unable to use gcda profile data

[Sandeep Rikhi]

Hi
This is my first exposure to profile guided optimization. I am trying to use PGO for a fairly complex project.  I am unable to consume the gcda files. I can not seem to find a proper explanation from a day long google searches. Is there a best practices document that someone can point me to in order to avoid the errors of this nature ?

   "error corrupted profile info number of executions"     
 
The summary of the project is 

1. lots and lots of c / c++ files
2. Custom generated Shared Libraries (.so files)
3. Over  dozen applications run simultaneously by using share lib routines
4. Most critical application of interest is highly multithreaded (over 100 threads) application.
5. can not statically link the libraries. 
6. gcc / g++ version is 4.1.2-46
7. build environment is a CentOS VM (kernel version 2.6.18).

The way I am generating the instrumented binaries is by adding the following options to my gcc/g++ and did a full clean build.
CXXFLAGS = -g -pg -fprofile-arcs -ftest-coverage -fprofile-generate -O2 -march=core2 -mtune=core2 -frandom-seed=0 -lgcov
CFLAGS = -g -pg -fprofile-arcs -ftest-coverage -fprofile-generate -O2 -march=core2 -mtune=core2 -frandom-seed=0 -lgcov
 
After the critical binary is put through benchmark tests, I am able to see a bunch of gcda files... They start to appear for the shared libraries even before the main application exists. The gcda file for main application appears after a graceful exit. the problem is that when I try to do the profile-use, I am facing a large number of errors that look like
"error corrupted profile info number of executions for edge XXXX ...."
So far, all the errors seem to be from the .gcda files tied to shared library code... 
 
I am doing a clean build ("make clean") before consuming the .gcda files
 
Can someone please share some words of wisdom as to how can I avoid such corruption ?  Am I doing something wrong ?