From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) by sourceware.org (Postfix) with ESMTPS id 7FAF13893664 for ; Thu, 18 Feb 2021 20:28:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 7FAF13893664 Received: by mail-yb1-xb30.google.com with SMTP id p186so3450938ybg.2 for ; Thu, 18 Feb 2021 12:28:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=rpqqwDWG0IjMqA13LexLIruKrdfm1gREqGlnGsIyLz8=; b=Er8DuUTxqfsZgp18OPiHElii2qGzoQPCuuvzhnmVYv0+/iDrV3xiriVT94qb74S6Z3 www809IL//NJeaHKsYekyk0bFeITivL4PW9nmsnLiTLP+74bxXaY3lbMygF1L8nyvjTx uOcRle5LbAfEV1GpkrDmDZf9cP33t0BhRpp5hi3y9y9iKrjgQ2bV+yzBDzcM3bao9d3h Sux/qZ1smsV63QLmBjltWKaLIRIa0Acp1UVTsLHdmzgf+pq/V36A7UGATUYKX6iI9IU/ RgTQxji4vsNDaf0AHdEYLDyumuzqBxyl802EnOTpbUYmpZqQfXLbPzTGpxysfktWRFEf +xkw== X-Gm-Message-State: AOAM533u4wl4VeJMCKwPuyLsJLhZX0IAy0xtd8L4UnJ87n2ZRwJ83LNm M6bWkYqy6Y4r6QzEJKFGt71rJGHaRAdX3JFvcFk= X-Google-Smtp-Source: ABdhPJxrdjj8i6MlqrnrX9bL0Rot8HcnBAkEEWRr/5TUZghio6mjV+Xouq+hb2nQvN1kLlHmy+XqT6PnkoQBeBd8j2U= X-Received: by 2002:a25:c793:: with SMTP id w141mr9218323ybe.29.1613680137865; Thu, 18 Feb 2021 12:28:57 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Andrew Pinski Date: Thu, 18 Feb 2021 12:28:46 -0800 Message-ID: Subject: Re: GCC generates non-compliant MIPS relocation data? Obscure GNU extension? To: Project Revolution Cc: "gcc@gcc.gnu.org" , "kenixwhisperwind@gmail.com" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2021 20:29:00 -0000 On Thu, Feb 18, 2021 at 12:15 PM Project Revolution via Gcc wrote: > > Hi GCC folks, > > We were working on a decompilation project for a Nintendo 64 title and at= tempting to enable support for using GCC instead of the emulated IRIX compi= ler and we ran into a big problem with how GCC generates relocations for th= e MIPS target which appears to show that GCC is generating non-compliant re= location data for MIPS target. Try compiling with -fno-section-anchors . https://gcc.gnu.org/legacy-ml/gcc-help/2009-07/msg00455.html Thanks, Andrew > > In summary: the Nintendo 64 title has a limited amount of RAM (4MB, 8MB i= f you add Expansion Pak, which our ROM target uses for debug reasons); in o= rder to accomplish this, the codebase packs actors/objects into overlays wh= ich the game determines need to be loaded per room/system transition. Once = loaded into RAM, the game applies the overlay's relocations generated at co= mpile time to the code to move the data and code correctly and make sure th= e jumps and loads get recalculated correctly. > > Unfortunately.. there's a problem. Here's the function that applies the r= elocs to MIPS: https://github.com/zeldaret/oot/blob/master/src/code/relocat= ion.c > > While enabling overlays to be recompiled with GCC instead of the IDO comp= iler, we have found the relocs generated did not guarantee 0x45/0x46 (Hi/lo= pairs) pairs to be 1:1, and GCC would share any possible hi/lo in O2 mode.= While O0 and O1 gcc overlays will work, or even Og, this is not a solution= for an N64 ROM due to limited RAM and space will quickly run out since mem= ory is so tight. While investigating why gcc will optimize relocs, we have = found the following: > > The MIPS ABI specified at https://refspecs.linuxfoundation.org/elf/mipsab= i.pdf on pages 79-80 (page 82 regarding the GP caveat) demands that hi/los = be in pairs). Thus, we have found that the reloc data generated erroneously= applies the relocation twice. Several LOs following a HI seems to be in a = spec, but several HIs following a LO does not. This is causing issues for o= ur relocation due to the relocs being applied incorrectly as a result of no= n-compliant relocation data. It turned out this reloc optimization is cause= d by an unmentioned, undocumented GNU extension. > > We have found the GNU extension was ONLY ever mentioned here: https://peo= ple.freebsd.org/~adrian/mips/20160819-mips-elf-reloc-gcc-5.3-3.diff > > Here is the file we compiled: https://github.com/zeldaret/oot/blob/master= /src/overlays/actors/ovl_En_Heishi4/z_en_heishi4.c > > This is the line we used to compile it: > > mips-linux-gnu-gcc -c -O2 -c -G 0 -nostdinc -Iinclude -Isrc -Iassets -Ibu= ild -I. -DNON_MATCHING=3D1 -DNON_EQUIVALENT=3D1 -DAVOID_UB=3D1 -mno-shared = -march=3Dvr4300 -mfix4300 -mabi=3D32 -mhard-float -mdivide-breaks -fno-stac= k-protector -fno-common -fno-zero-initialized-in-bss -mno-abicalls -fno-str= ict-aliasing -fno-inline-functions -fno-inline-small-functions -fno-topleve= l-reorder -ffreestanding -fwrapv -Wall -Wextra -g -fno-gcse -fno-cse-follow= -jumps -mno-load-store-pairs -mno-explicit-relocs -fno-peephole2 -mips3 -o = build/src/overlays/actors/ovl_En_Heishi4/z_en_heishi4.o src/overlays/actors= /ovl_En_Heishi4/z_en_heishi4.c > > To note, we have tried with and without explicit relocs and with and with= out peephole2 and with and without mips2/3 and it didnt make a difference: = the relocs were still noncompliant per the PDF posted earlier. We need a wa= y to turn this undocumented GNU extension off because it is causing relocs = when optimized to not be processed correctly. To note, our use case is atte= mpting to compile this repo with GCC (this file is a test case) but if you = were to compile the ROM with the Heishi4 file being compiled as GCC using t= he above call (make any Makefile alterations to force the object to be GCC)= , spawn on the SPOT00 map at the start of the game and go inside the castle= town area and observe the crash which takes like 60 seconds. This is ultim= ately what we're trying to fix which following this rabbit hole leads us to= this GNU extension in a haystack hunt. Can you guys help us resolve this? > > v/r, > Revo >