From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by sourceware.org (Postfix) with ESMTPS id 93B6F38438DE for ; Wed, 11 May 2022 19:33:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 93B6F38438DE Received: by mail-pl1-x62e.google.com with SMTP id j14so2854484plx.3 for ; Wed, 11 May 2022 12:33:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=aWlcSbQO41Isl0hHFTB51vCmWyahu7YieUX2zn/5T88=; b=m+7pG1qPJc1CuSY1N5NU6d88FmyMDDv3ZPzRra1OvSAJz9A9wM9JdMuVw7deoThTbj bUBRDZsIl09lFS79iG7xWHuf2y7Vu813oORANjGIGMizR5gaQJhxulS7XItQvP4aqaVt pque+GW6RYyrwkyj6dv+tEXn5KRRPOOeqZRH6NU8r9nyU/xbgMqTRWNglbLrlTh700Q/ EfeeaT5HZ1rLbKlRTOD+MydQqCya/JPLE/rcO/1k4uo7LDMIva8f/sFUSVZwJ6ws7bsb mVR8s5382FiGaEBysVz0tp9OJ//hRorPrq3HVBcwuZ3h0kL7V2sVi5+WatzGiopstcru lmXQ== X-Gm-Message-State: AOAM5309ho6FYhV5n192k8dr+Pu70hqVU/N1BeWAfEWTGZBe+I4HN6VU /7ltxar9qqBB59YaB96dGPiF0g== X-Google-Smtp-Source: ABdhPJzSNiIn18Mj/c+ykLLUJJ4J6x6CubphhgSRByhXFKjGHVt7P8vFDi8RSedHzJbXf9O4XGteEg== X-Received: by 2002:a17:902:ea57:b0:15a:6173:87dd with SMTP id r23-20020a170902ea5700b0015a617387ddmr27175616plg.147.1652297633408; Wed, 11 May 2022 12:33:53 -0700 (PDT) Received: from google.com ([2620:15c:2ce:200:3228:6d2d:ebc8:7bc1]) by smtp.gmail.com with ESMTPSA id w20-20020a17090aaf9400b001cd4989ff65sm280034pjq.44.2022.05.11.12.33.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 May 2022 12:33:53 -0700 (PDT) Date: Wed, 11 May 2022 12:33:49 -0700 From: Fangrui Song To: "H.J. Lu" Cc: Florian Weimer , GNU C Library , Binutils Subject: Re: PT_GNU_RELRO is somewhat broken Message-ID: <20220511193349.zxexikfzpbqqg3x4@google.com> References: <871qx0dmz5.fsf@oldenburg.str.redhat.com> <20220511181704.y4pldvlqnbix3p53@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-17.9 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, KAM_INFOUSMEBIZ, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2022 19:33:56 -0000 On 2022-05-11, H.J. Lu wrote: >On Wed, May 11, 2022 at 11:17 AM Fangrui Song wrote: >> >> On 2022-05-11, H.J. Lu via Libc-alpha wrote: >> >On Wed, May 11, 2022 at 9:59 AM Florian Weimer via Libc-alpha >> > wrote: >> >> >> >> PT_GNU_RELRO is supposed to identify a region in the process image which >> >> has to be flipped to PROT_READ (only) permission after relocation >> >> (“Read-Only after RELocation”). >> >> >> >> glibc has this code in the dynamic loader in elf/dl-reloc.c: >> >> >> >> | void >> >> | _dl_protect_relro (struct link_map *l) >> >> | { >> >> | ElfW(Addr) start = ALIGN_DOWN((l->l_addr >> >> | + l->l_relro_addr), >> >> | GLRO(dl_pagesize)); >> >> | ElfW(Addr) end = ALIGN_DOWN((l->l_addr >> >> | + l->l_relro_addr >> >> | + l->l_relro_size), >> >> | GLRO(dl_pagesize)); >> >> | if (start != end >> >> | && __mprotect ((void *) start, end - start, PROT_READ) < 0) >> >> | { >> >> | static const char errstring[] = N_("\ >> >> | cannot apply additional memory protection after relocation"); >> >> | _dl_signal_error (errno, l->l_name, NULL, errstring); >> >> | } >> >> | } >> >> >> >> I assume the intent is to conservatively apply the largest possible >> >> RELRO region given GLRO(dl_pagesize), the run-time page size reported by >> >> the kernel. If the binary is built to a smaller page size (to save disk >> >> space), glibc can still load it, but apply only some RELRO protection. >> >> But _dl_relocate_object has a bug: to be conservative, it would have to >> >> use ALGIN_UP for the start (lower) address of the range. >> >> >> >> But it turns out we can't make this change without incurring a loss of >> >> hardening: BFD ld does not align the start address to a page boundary. >> >> For example, /bin/true in Fedora 35 x86-64 has this: >> >> >> >> | $ readelf -l /bin/true >> >> | >> >> | Elf file type is DYN (Position-Independent Executable file) >> >> | Entry point 0x1960 >> >> | There are 13 program headers, starting at offset 64 >> >> | >> >> | Program Headers: >> >> | Type Offset VirtAddr PhysAddr >> >> | FileSiz MemSiz Flags Align >> >> | PHDR 0x0000000000000040 0x0000000000000040 0x0000000000000040 >> >> | 0x00000000000002d8 0x00000000000002d8 R 0x8 >> >> | INTERP 0x0000000000000318 0x0000000000000318 0x0000000000000318 >> >> | 0x000000000000001c 0x000000000000001c R 0x1 >> >> | [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] >> >> | LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> >> | 0x0000000000000ff8 0x0000000000000ff8 R 0x1000 >> >> | LOAD 0x0000000000001000 0x0000000000001000 0x0000000000001000 >> >> | 0x00000000000029a1 0x00000000000029a1 R E 0x1000 >> >> | LOAD 0x0000000000004000 0x0000000000004000 0x0000000000004000 >> >> | 0x0000000000000d38 0x0000000000000d38 R 0x1000 >> >> | LOAD 0x0000000000005c78 0x0000000000006c78 0x0000000000006c78 >> >> | 0x0000000000000390 0x00000000000003a0 RW 0x1000 >> >> | DYNAMIC 0x0000000000005c90 0x0000000000006c90 0x0000000000006c90 >> >> | 0x00000000000001f0 0x00000000000001f0 RW 0x8 >> >> | NOTE 0x0000000000000338 0x0000000000000338 0x0000000000000338 >> >> | 0x0000000000000050 0x0000000000000050 R 0x8 >> >> | NOTE 0x0000000000000388 0x0000000000000388 0x0000000000000388 >> >> | 0x0000000000000044 0x0000000000000044 R 0x4 >> >> | GNU_PROPERTY 0x0000000000000338 0x0000000000000338 0x0000000000000338 >> >> | 0x0000000000000050 0x0000000000000050 R 0x8 >> >> | GNU_EH_FRAME 0x00000000000049c4 0x00000000000049c4 0x00000000000049c4 >> >> | 0x000000000000007c 0x000000000000007c R 0x4 >> >> | GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> >> | 0x0000000000000000 0x0000000000000000 RW 0x10 >> >> | GNU_RELRO 0x0000000000005c78 0x0000000000006c78 0x0000000000006c78 >> >> | 0x0000000000000388 0x0000000000000388 R 0x1 >> >> | […] >> >> >> >> The virtual address for PT_GNU_RELRO is 0x388, which is definitely not >> >> aligned to a 4K page. (0x388 + 0x6c78 == 0x7000, so at least the end >> >> address is aligned.) In practice, this seems to work because the RELRO >> >> area seems to be at the start of the RW LOAD segment, so we can safely >> >> flip the slack space at the start of the page to RO. It still looks >> >> like a major wart to me, though. >> > >> >After relocation, we change the end of the RO segment (aligned down from >> >the beginning of the RELRO area) to the end of the RELRO segment to RO. >> >Since the end of the RELRO segment must be aligned to the page size, >> >ALIGN_DOWN on the end of the RELRO segment doesn't lose any protection. >> > >> >> Any suggestions what should we do to fix this properly, mainly for >> >> targets that have varying page size in practice? >> > >> >The end of the RELRO segment should be aligned to the maximum page >> >size. >> > >> >> PT_GNU_RELRO is designed/implemented this way: >> >> * there can be at most one PT_GNU_RELRO >> * p_vaddr(PT_GNU_RELRO) = p_vaddr(first RW PT_LOAD); https://sourceware.org/binutils/docs/ld/Builtin-Functions.html DATA_SEGMENT_RELRO_END is designed this way >> * p_vaddr(PT_GNU_RELRO) + p_memsz(PT_GNU_RELRO) is aligned by common-page-size. comon page size is chosen probably because of less waste > >ld aligns DATA_SEGMENT_RELRO_END to the maximum page size. > >> If the proposal is to align p_vaddr(PT_GNU_RELRO) + >> p_memsz(PT_GNU_RELRO) to max page size, that will penalize the size of >> many max-page-size>4096 ports with the current GNU ld section/segment >> layout. See https://sourceware.org/bugzilla/show_bug.cgi?id=24490 and >> https://sourceware.org/bugzilla/show_bug.cgi?id=23704 for GNU ld's -z >> separate-code complaints. > >Separate RW PT_LOAD for PT_GNU_RELRO can reduce file size >for -z no-separate-code. ld implements -z separate-code in such a >way that not only executable sections are in separate RE pages in >memory, but also mapping the RE segment won't map in other contents >on disk. > >> Note: ld.lld used (before 9.0.0) to place PT_GNU_RELRO in the middle of >> the RW PT_LOAD. I changed it to the start in >> https://reviews.llvm.org/D58892 With the new scheme, it doesn't really >> matter whether p_vaddr(PT_GNU_RELRO) + p_memsz(PT_GNU_RELRO) is aligned >> to max-page-size or common-page-size: the file size does not change. > >This layout is generated by lld: > > LOAD 0x000000 0x0000000000200000 0x0000000000200000 >0x00055c 0x00055c R 0x1000 > LOAD 0x000560 0x0000000000201560 0x0000000000201560 >0x000160 0x000160 R E 0x1000 > LOAD 0x0006c0 0x00000000002026c0 0x00000000002026c0 >0x0001a0 0x0001a0 RW 0x1000 > LOAD 0x000860 0x0000000000203860 0x0000000000203860 >0x000028 0x000029 RW 0x1000 > DYNAMIC 0x0006d0 0x00000000002026d0 0x00000000002026d0 >0x000180 0x000180 RW 0x8 > GNU_RELRO 0x0006c0 0x00000000002026c0 0x00000000002026c0 >0x0001a0 0x000940 R 0x1 > >The beginning of the first RE page in memory also includes the end >of the previous R segment and the end of the last RE page also >includes the beginning of the next RW segment. --rosegment still >leaves non-instructions bytes in the RE pages. lld chooses to default to -z noseparate-code. One can add -z separate-code to make R and RE separate. I have some notes about this on https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities