From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 84574 invoked by alias); 27 Mar 2015 15:46:20 -0000 Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-bugs-owner@gcc.gnu.org Received: (qmail 84094 invoked by uid 55); 27 Mar 2015 15:46:12 -0000 From: "hubicka at ucw dot cz" To: gcc-bugs@gcc.gnu.org Subject: [Bug lto/65536] LTO line number information garbled Date: Fri, 27 Mar 2015 15:53:00 -0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: gcc X-Bugzilla-Component: lto X-Bugzilla-Version: 5.0 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: hubicka at ucw dot cz X-Bugzilla-Status: NEW X-Bugzilla-Priority: P3 X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-SW-Source: 2015-03/txt/msg03188.txt.bz2 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536 --- Comment #50 from Jan Hubicka --- > > +/* Do not track column numbers higher than this one. As a result, the > + range of column_bits is [7, 18] (or 0 if column numbers are > + disabled). */ > +#define LINE_MAP_MAX_COLUMN_NUMBER (1U << 17) > + > +/* Do not track column numbers if locations get higher than this. */ > +#define LINE_MAP_MAX_LOCATION_WITH_COLS 0x70000000 > + > +/* Highest possible source location encoded within an ordinary or > + macro map. */ > +#define LINE_MAP_MAX_SOURCE_LOCATION 0x7FFFFFF0 I understood the limit of 0x70000000 to give some buffer so we can still record new files but also drop the line number information? > -I/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include > -I/home/manuel/test1/pristine/libstdc++-v3/libsupc++ > -L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs > -L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs > -g -O2 -flto=jobserver -frandom-seed=1 -DIN_GCC -fno-exceptions -fno-rtti > -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings > -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic > -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror -fno-common > -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -o cc1 c/c-lang.o > c-family/stub-objc.o attribs.o c/c-errors.o c/c-decl.o c/c-typeck.o > c/c-convert.o c/c-aux-info.o c/c-objc-common.o c/c-parser.o > c/c-array-notation.o c-family/c-common.o c-family/c-cppbuiltin.o > c-family/c-dump.o c-family/c-format.o c-family/c-gimplify.o c-family/c-lex.o > c-family/c-omp.o c-family/c-opts.o c-family/c-pch.o c-family/c-ppoutput.o > c-family/c-pragma.o c-family/c-pretty-print.o c-family/c-semantics.o > c-family/c-ada-spec.o c-family/c-cilkplus.o c-family/array-notation-common.o > c-family/cilk.o c-family/c-ubsan.o i386-c.o glibc-c.o \ > cc1-checksum.o libbackend.a main.o tree-browser.o libcommon-target.a > libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a > ../libcpp/libcpp.a ../libbacktrace/.libs/libbacktrace.a > ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a > -L/opt/cfarm/isl-0.12.2/lib -lisl -L/opt/cfarm/gmp-4.3.2/lib > -L/opt/cfarm/mpfr-2.4.2/lib -L/opt/cfarm/mpc-0.8.1/lib -lmpc -lmpfr -lgmp > -rdynamic -ldl -L../zlib -lz > /home/manuel/test1/pristine/gcc/c/c-lang.h:26:16: error: type ???struct > lang_type??? violates one definition rule [-Werror=odr] > struct GTY(()) lang_type { > ^ > /home/manuel/test1/pristine/gcc/cp/cp-tree.h:1566:16: note: a different type is > defined in another translation unit > struct GTY(()) lang_type { > ^ > /home/manuel/test1/pristine/gcc/c/c-lang.h:28:72: note: the first difference of > corresponding definitions is field ???s??? > struct sorted_fields_type * GTY ((reorder ("resort_sorted_fields"))) s; > ^ > /home/manuel/test1/pristine/gcc/cp/cp-tree.h:1572:45: note: a field with > different name is defined in another translation unit > } GTY((desc ("%h.h.is_lang_type_class"))) u; > ^ Well, this is a bug in C/C++ FEs clearly, lang_type should not be declared twice with different meanings. One needs to be renamed. You can however safely ignore it and get build to complete with --disable-werror > > > 2) there is a bug that we sometimes output wrong line number. I think it is > > caused by the logic reallocating ORDINARY_MAP_NUMBER_OF_COLUMN_BITS as > > described in original email. > > Maybe but I do not see it yet. My guess is that it is caused either by > overflowing start_location in linemap_add, or when computing 'r' > linemap_position_for_column after linemap_line_start starts returning 0 or > after column_hint has been truncated (your patch fixes this), or when the > computation of 'r' overflows in linemap_line_start. Any of those cases will > trigger a silent overflow and return a location for something else. I will try to add assets and reproduce it (at least with cache disabled on gcc bootstrap), lets see if I get any luck. I only saw it on Chromium that is order of mangnitude bigger than GCC. Just to give you idea, GCC linktime memory consumption is about 700MB, while Firefox takes 3GB and Chromium 9GB. It has 11k files linked to one unit (which makes is so much bigger than firefox that does include many .cpp units into one to get around the bloat). Before my changes, the linemap occupied about 1GB. > > I do not think typical usage is pesimized here. Yes, code is trying to avoid > > case where we skip too many location indexes because we entered lines 1...30 > > and now we want to insert line 1000. We do not want to fast forward by > > 970*80. > > Yes, we want to. Because then set->highest_location goes up just by one and the > next map starts from that, thus we do not need to consume source_locations for > those 970*80 entries which will never be addressed (we jump directly from 30:80 > to 1000:1). Yep, i think we have oposite meaning of fast forward. I mean fast forward by not adding map entry, while you mean map entry. So I think we agree ;) > > > > I do not see why. If we insert first line 1 (80 columns), then we create a > > map 1. Now we insert line 3, we do not create new map, but we offset > > highest_line by 80*2. Now if we start line 2, all we need is to offset > > highest_line by -80 and linemap_position_for_column will still work well. > > But then you do not know which is the source_location of the highest line > encoded in the line_table, which is used in several places. I'm not saying it > is not worth it or possible, but it is not as trivial as just offsetting > highest_line (and then, it should not be called highest_line but perhaps > last_line_location). The code tracking hihest recorded location is intact (it is simple max operation). This needs to work, because no one guarantees the columns are inserted in increasing number, so linemap_position_for_column is not always returning highest location at each invocation. > > Note that I'm not discussing all this to be negative. I'm very much in favour > of supporting some kind of out-of-order insertion (in particular for PR52952), > but we should be careful to not pessimize the in-order case and we should get > all the details right. The line-maps are tricky and getting the details wrong > could be disaster. Sure, I am trying to find solution that works well (better) for both. Honza