From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2994 invoked by alias); 26 May 2010 13:43:33 -0000 Received: (qmail 2971 invoked by uid 22791); 26 May 2010 13:43:32 -0000 X-SWARE-Spam-Status: No, hits=-0.2 required=5.0 tests=AWL,BAYES_50,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SARE_LWSHORTT,TW_FN X-Spam-Check-By: sourceware.org Received: from mail-px0-f175.google.com (HELO mail-px0-f175.google.com) (209.85.212.175) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 26 May 2010 13:43:25 +0000 Received: by pxi14 with SMTP id 14so2528574pxi.20 for ; Wed, 26 May 2010 06:43:24 -0700 (PDT) Received: by 10.141.188.34 with SMTP id q34mr6647751rvp.203.1274881403216; Wed, 26 May 2010 06:43:23 -0700 (PDT) Received: from Paullaptop (124-168-184-56.dyn.iinet.net.au [124.168.184.56]) by mx.google.com with ESMTPS id b10sm82595rvn.3.2010.05.26.06.43.19 (version=SSLv3 cipher=RC4-MD5); Wed, 26 May 2010 06:43:21 -0700 (PDT) Message-ID: From: "Paul Edwards" To: Cc: "Richard Guenther" References: <200911241405.nAOE5Jsd022678@d12av02.megacenter.de.ibm.com> <7B52F224E6EE465EB95237D5A0D9A15F@Paullaptop> <84fc9c000911280802j3a0be6b1p1241d81d91f0672e@mail.gmail.com> In-Reply-To: <84fc9c000911280802j3a0be6b1p1241d81d91f0672e@mail.gmail.com> Subject: Re: i370 port - status Date: Wed, 26 May 2010 14:40:00 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="Windows-1252"; reply-type=original Content-Transfer-Encoding: 7bit Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org X-SW-Source: 2010-05/txt/msg00555.txt.bz2 > So the fix below is not a fix but papering over an > issue elswhere. Ok, this problem nearly killed me. It took months of effort to try to isolate the fault, delving into unfamiliar code, trying to get something that could be reliably reproduced. I did finally succeed in getting an executable that reliably crashed on the 3rd identical run. However, when I attempted to debug at a level above, ie in the emulator, the bug would disappear also. My working hypothesis now is that it is a bug in the Hercules emulator to do with paging. Which means I am at the mercy of the operating system as to when it pages. That operating system is MUSIC/SP, and it doesn't come with source code, and I'm not really familiar with it anyway, and the author is dead (a couple of years ago). I did succeed in finding a configuration option to stop the huge amount of paging in MUSIC/SP. That then allowed my compiles to go through cleanly, so I am now able to get GCC to reproduce itself under MUSIC/SP. The underlying presumed paging bug in either Hercules or MUSIC/SP I decided to just package up and leave for another day, as neither thing was my main or even important focus. Closer to my main focus - after getting GCC working on MUSIC/SP, the final real ("real" = "commercially used operating system, which people use to develop code, and which doesn't have a free C compiler already") target that I knew of that remained was DOS/VS (z/VSE). In April 2010 I finally got this last environment to have a hosted C compiler that was good enough to compile a "hello world". The C runtime library is only bare bones, so it can't handle #include yet, only reading from stdin, but that's enough to get the right code for a "hello world" to be generated. Getting GCC on that environment found a bug in the DOS/VS operating system, and other limitations, but these problems were worked around. As we speak, Michel et al over here: http://tech.groups.yahoo.com/group/H390-DOSVS/ is battling to try to get the DOS/VS environment to process a PARM string (in a way compatible with z/VSE). Previously the parameter was kludged to read it from stdin. The old DOS/VS doesn't have parameters basically. Then the rest of the library can be dealt with (reading from private source statement libraries is the main thing). Meanwhile, I have released GCC 3.2.3 MVS 8.0, which can be found here: http://gccmvs.sourceforge.net which includes a much nicer MVS environment (with default DCB information) for GCC et al. It also includes the afore mentioned MUSIC/SP and DOS/VS ports, even if the latter is bare-bones. Perhaps a link to that site could be put here: http://gcc.gnu.org/extensions.html So in the short term, the sort of things I am likely to be involved in are: 1. MVS source code management (10 million lines of code). 2. Improve VSE/380 port. 3. GCC 3.4.6 upgrade (from 3.2.3). Although I managed to get 3.4.6 working on MVS many months ago, I haven't formally released that port because I was trying to get the potentially last 3.2.3 out the door. But that happened a couple of days ago. So now, as long as I can bring all 4 environments along, I should be able to upgrade. GCC 3.4.6 has the advantage that the Unix build tools are able to generate all the required files for MVS etc. Before I used to handcraft them. I could probably backport that to 3.2.3 but haven't tried, as my plan was to go forward. One problem with going foward though is that I will lose the ability to generate the generated files from i370.md. The reason for that is that one of the generator programs opens lots of files by name, and those names are not appropriate for MVS usage. I can probably work around that, but rather than spending the effort doing that, I was planning on revisiting that in GCC 4, and in the meantime, just do that part of the build on Unix instead. If people truly need a self-contained MVS (EBCDIC) C compiler, then 3.2.3 will always be available. Hopefully I'll have more positive news on GCC on the EBCDIC platfoms in the months ahead. :-) Thanks to everyone who helped in getting it so far already. BFN. Paul. ----- Original Message ----- From: "Richard Guenther" To: "Paul Edwards" Cc: Sent: Sunday, November 29, 2009 2:02 AM Subject: Re: i370 port - music/sp - possible generic gcc problem On Sat, Nov 28, 2009 at 4:13 PM, Paul Edwards wrote: > I think I have found a bug in gcc, that still exists in gcc 4.4 > > I found the problem on 3.2.3 though. > > While MVS and VM have basically been working fine, when I did > the port to MUSIC/SP I started getting strange compilation failures. > > Initializing the stack to NULs made the problem go away, but I > persisted, and instead initialized the stack to x'01' to try to get > consistent failures. > > Even with that, and the malloced memory initialized to consistent > garbage, I still didn't get the desired consistency in failures. > But it was consistent enough that I could just run it 6 times and > see if there were any failures. > > > Anyway, I tracked down the particular malloc() which gave changed > behaviour depending on whether the malloc() did a memory initialization > to NULs or not. Well, GC hands out non-zeroed memory - the callers are responsible for initializing it. So the fix below is not a fix but papering over an issue elswhere. Richard. > It's this one (showing 3.2.3): > > C:\devel\gcc\gcc>cvs diff -l -c15 > cvs diff: Diffing . > Index: ggc-page.c > =================================================================== > RCS file: c:\cvsroot/gcc/gcc/ggc-page.c,v > retrieving revision 1.1.1.1 > diff -c -1 -5 -r1.1.1.1 ggc-page.c > *** ggc-page.c 15 Feb 2006 10:22:25 -0000 1.1.1.1 > --- ggc-page.c 28 Nov 2009 14:13:41 -0000 > *************** > *** 640,670 **** > #ifdef USING_MALLOC_PAGE_GROUPS > else > { > /* Allocate a large block of memory and serve out the aligned > pages therein. This results in much less memory wastage > than the traditional implementation of valloc. */ > > char *allocation, *a, *enda; > size_t alloc_size, head_slop, tail_slop; > int multiple_pages = (entry_size == G.pagesize); > > if (multiple_pages) > alloc_size = GGC_QUIRE_SIZE * G.pagesize; > else > alloc_size = entry_size + G.pagesize - 1; > ! allocation = xmalloc (alloc_size); > > page = (char *) (((size_t) allocation + G.pagesize - 1) & > -G.pagesize); > head_slop = page - allocation; > if (multiple_pages) > tail_slop = ((size_t) allocation + alloc_size) & (G.pagesize - 1); > else > tail_slop = alloc_size - entry_size - head_slop; > enda = allocation + alloc_size - tail_slop; > > /* We allocated N pages, which are likely not aligned, leaving > us with N-1 usable pages. We plan to place the page_group > structure somewhere in the slop. */ > if (head_slop >= sizeof (page_group)) > group = (page_group *)page - 1; > else > --- 640,670 ---- > #ifdef USING_MALLOC_PAGE_GROUPS > else > { > /* Allocate a large block of memory and serve out the aligned > pages therein. This results in much less memory wastage > than the traditional implementation of valloc. */ > > char *allocation, *a, *enda; > size_t alloc_size, head_slop, tail_slop; > int multiple_pages = (entry_size == G.pagesize); > > if (multiple_pages) > alloc_size = GGC_QUIRE_SIZE * G.pagesize; > else > alloc_size = entry_size + G.pagesize - 1; > ! allocation = xcalloc (1, alloc_size); > > page = (char *) (((size_t) allocation + G.pagesize - 1) & > -G.pagesize); > head_slop = page - allocation; > if (multiple_pages) > tail_slop = ((size_t) allocation + alloc_size) & (G.pagesize - 1); > else > tail_slop = alloc_size - entry_size - head_slop; > enda = allocation + alloc_size - tail_slop; > > /* We allocated N pages, which are likely not aligned, leaving > us with N-1 usable pages. We plan to place the page_group > structure somewhere in the slop. */ > if (head_slop >= sizeof (page_group)) > group = (page_group *)page - 1; > else > > > I suspect that it has stayed hidden for so long because most people > probably have this mmap_anon: > > /* Define if mmap can get us zeroed pages using MAP_ANON(YMOUS). */ > #undef HAVE_MMAP_ANON > > > #ifdef HAVE_MMAP_ANON > # undef HAVE_MMAP_DEV_ZERO > > # include > # ifndef MAP_FAILED > # define MAP_FAILED -1 > # endif > # if !defined (MAP_ANONYMOUS) && defined (MAP_ANON) > # define MAP_ANONYMOUS MAP_ANON > # endif > # define USING_MMAP > > #endif > > #ifndef USING_MMAP > #define USING_MALLOC_PAGE_GROUPS > #endif > > > Seems pretty clear that zeroed pages are required. > > Looking at the code above and below this block, it uses xcalloc > instead. > > It will take a couple of days to confirm that this is the last > presumed bug affecting the MUSIC/SP port. > > In the meantime, can someone confirm that this code is wrong, > and that xcalloc is definitely required? > > I had a look at the gcc4 code, and it is calling XNEWVEC which > is also using xmalloc instead of xcalloc, so I presume it is still > a problem today, under the right circumstances. > > It took about a week to track this one down. :-) > > One problem I have had for years, even on the MVS port, is that > I need to use -Os to get the exact same register selection on the > PC as the mainframe. -O0 and -O2 get slightly different register > allocations, although all versions of the code are correct. If I'm > lucky, this fix may make that problem go away, but as I said, > it'll take a couple of days before the results are in, as each build > takes about 2 hours (partly because i need to use -Os for > consistency). > > Thanks. Paul. > >