From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 3BA7C3858402 for ; Sat, 29 Jan 2022 17:39:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 3BA7C3858402 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-575-hPs_qNhsNTWURvQwU1rkEw-1; Sat, 29 Jan 2022 12:39:37 -0500 X-MC-Unique: hPs_qNhsNTWURvQwU1rkEw-1 Received: by mail-qt1-f199.google.com with SMTP id h22-20020ac85696000000b002d258f68e98so6892794qta.22 for ; Sat, 29 Jan 2022 09:39:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=HoaPLmXjaBng1L/lEtzrghIOfglqIHkn7CqOKQ+//uI=; b=ARL1/3KUcLE6QHyVAcKtM3tL6B+3daXkBx0aW3NDe1nX6AeA8aLWGUqcFfb5x87GdY ocV9nEEYntIehTUoAfnvgpXN2dxQFa4iuBvg220iBWeO+yfiejlH8aaIMIYBNCKKvx+T NEuHw4T0g7VdPdsBnJsBJZnj3KKNzHWaXG2lFhnJZnusU26qVCPh663wBYz3+Y+AdhVi +TMU0cQa8foqTkzO2ryA5hWPwGOZr9LuKc9zFc7XxozgXVCy6Ccw178UKUReDsdu00IQ J5h45MIDdaAAk0dXkocvLMp57R+5f3HSHWvfp9ak+4ulrmYyyI9tG3Bv/CqgmQs1uOsT B+8A== X-Gm-Message-State: AOAM531i3TGy8hk8wGkj1Q1y98eOUTEP2NQv3HNOL7P2tYUH7w2Qv8dB n7fnu6lCnWmhih9/+sWu1siO5iQ4tPc++Ztbg67e4ISCljCK/TgbOJImAQdjAd6kFTgl8Ws9/BZ 7c6zStAc= X-Received: by 2002:a37:9203:: with SMTP id u3mr9026503qkd.775.1643477977324; Sat, 29 Jan 2022 09:39:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJy76+tY4RqowDLhRMVCg1IQq1Wdoj4cMGoReeOoSz2ZMHucGt/nqV7GgMTNQAH0qLIdNJXEWA== X-Received: by 2002:a37:9203:: with SMTP id u3mr9026491qkd.775.1643477976985; Sat, 29 Jan 2022 09:39:36 -0800 (PST) Received: from t14s.localdomain (c-73-69-212-193.hsd1.ma.comcast.net. [73.69.212.193]) by smtp.gmail.com with ESMTPSA id m22sm4947535qkn.35.2022.01.29.09.39.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 Jan 2022 09:39:36 -0800 (PST) Message-ID: <2cc1022a7d7bc2b00ceac2175ada29554d961bff.camel@redhat.com> Subject: Re: GSoC: Working on the static analyzer From: David Malcolm To: Mir Immad , gcc@gcc.gnu.org Date: Sat, 29 Jan 2022 12:39:35 -0500 In-Reply-To: References: <4eec5fa69b9daedcec5361c2cc18df7f1ef397af.camel@redhat.com> User-Agent: Evolution 3.38.4 (3.38.4-1.fc33) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-6.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2022 17:39:45 -0000 On Sat, 2022-01-29 at 20:22 +0530, Mir Immad wrote: > Thank you for the detailed information. > > I've been looking into the integer posix  file descriptor APIs and I > decided to write proof-of-concept  checker for them. (not caring > about > errno). The checker tracks the fd returned by open(), warns if dup() > is > called with closed fd otherwise tracks the fd returned by dup(), it > also > warns if read() and write() functions were called on closed fd. I'm > attaching a text file that lists some c sources and warnings by the > static > analyzer. I've used the diagnostic meta-data from sm-file. Is this > something that could also be added to the analyzer? This looks great, and very promising as both new functionality for GCC 13, and as a GSoC 2022 project. BTW, it looks like you're working with GCC 11, but the analyzer has changed quite a bit on trunk for GCC 12, so it's worth trying to track trunk. I wonder if it's worth checking for attempts to write to a fd that was opened with O_RDONLY, or the converse? (I'm not sure, just thinking aloud - how much state does it make sense to track for a fd?). Also, at some point, we're going to have to handle "errno" - but given that might be somewhat fiddly it's OK to defer that until you're more familiar with the code. > > About the fd leak, that's the next thing I'll try to get working. > Since > you've mentioned that it could be a GSoC project, this is what I'm > going to > focus on. Excellent. Let me know (via this mailing list) if you have any questions. Thanks Dave > > Regards. > > > > On Wed, Jan 26, 2022 at 7:56 PM David Malcolm > wrote: > > > On Mon, 2022-01-24 at 01:41 +0530, Mir Immad wrote: > > > Hi, sir. > > > > > > I've been trying to understand the static analyzer's code. I > > > spent most > > > of > > > my time learning the state machine's API. I learned how state > > > machine's > > > on_stmt is supposed to "recognize" specific functions and how > > > on_transition > > > takes a specific tree from one state to another, and how the > > > captured > > > states are used by pending_diagnostics to report the errors. > > > Furthermore, I > > > was able to create a dummy checker that mimicked the behaviour of > > > sm- > > > file's > > > double_fclose and compile GCC with these changes. Is this the > > > right way > > > of > > > learning? > > > > This sounds great. > > > > > > > > As you've mentioned on the projects page that you would like to > > > add > > > more > > > support for some POSIX APIs. Can you please write (or refer me to > > > a) a > > > simple C program that uses such an API (and also what the > > > analyzer > > > should > > > have done) so that I can attempt to add such a checker to the > > > analyzer. > > > > A couple of project ideas: > > > > (i) treat data coming from a network connection as tainted, by > > somehow > > teaching the analyzer about networking APIs.  Ideally: look at some > > subset of historical CVEs involving network-facing attacks on user- > > space daemons, and find a way to detect them in the analyzer (need > > to > > find a way to mark the incoming data as tainted, so that the > > analyer > > "know" about the trust boundary - that the incoming data needs to > > be > > sanitized and treated with extra caution; see > > https://gcc.gnu.org/pipermail/gcc-patches/2021-November/584372.html > > for my attempts to do this for the Linux kernel). > > > > Obviously this is potentially a huge project, so maybe just picking > > a > > tiny subset and getting that working as a proof-of-concept would be > > a > > good GSoC project.  Maybe find an old CVE that someone has written > > a > > good write-up for, and think about "how could GCC/-fanalyzer have > > spotted it?" > > > > (ii) add leak-detection for POSIX file descriptors: i.e. the > > integer > > values returned by "open", "dup", etc.  It would be good to have a > > check that the user's code doesn't leak these values e.g. on error- > > handling paths, by failing to close a file-descriptor (and not > > storing > > it anywhere).  I think that much of this could be done by analogy > > with > > the sm-file.cc code. > > > > > > > > > > Also, I didn't realize the complexity of adding SARIF when I > > > mentioned > > > it. > > > I'd rather work on adding more checkers. > > > > Fair enough. > > > > Hope this above is constructive. > > > > Dave > > > > > > > > Regards. > > > > > > Mir Immad > > > > > > On Sun, Jan 23, 2022, 11:04 PM Mir Immad > > > wrote: > > > > > > > Hi Sir, > > > > > > > > I've been trying to understand the static analyzer's code. I > > > > spent > > > > most of > > > > my time learning the state machine's API. I learned how state > > > > machine's > > > > on_stmt is supposed to "recognize" specific functions and takes > > > > a > > > > specific > > > > tree from one state to another, and how the captured states are > > > > used > > > > by > > > > pending_diagnostics to report the errors. Furthermore, I was > > > > able to > > > > create > > > > a dummy checker that mimicked the behaviour of sm-file's > > > > double_fclose and > > > > compile GCC with these changes. Is this the right way of > > > > learning? > > > > > > > > As you've mentioned on the projects page that you would like to > > > > add > > > > more > > > > support for some POSIX APIs. Can you please write (or refer me > > > > to a) > > > > a > > > > simple C program that uses such an API (and also what the > > > > analyzer > > > > should > > > > have done) so that I can attempt to add such a checker to the > > > > analyzer. > > > > > > > > Also, I didn't realize the complexity of adding SARIF when I > > > > mentioned it. > > > > I'd rather work on adding more checkers. > > > > > > > > Regards. > > > > Mir Immad > > > > > > > > On Mon, Jan 17, 2022 at 5:41 AM David Malcolm < > > > > dmalcolm@redhat.com> > > > > wrote: > > > > > > > > > On Fri, 2022-01-14 at 22:15 +0530, Mir Immad wrote: > > > > > > HI David, > > > > > > I've been tinkering with the static analyzer for the last > > > > > > few > > > > > > days. I > > > > > > find > > > > > > the project of adding SARIF output to the analyzer > > > > > > intresting. > > > > > > I'm > > > > > > writing > > > > > > this to let you know that I'm trying to learn the codebase. > > > > > > Thank you. > > > > > > > > > > Excellent. > > > > > > > > > > BTW, I think adding SARIF output would involve working more > > > > > with > > > > > GCC's > > > > > diagnostics subsystem than with the static analyzer, since > > > > > (in > > > > > theory) > > > > > all of the static analyzer's output is passing through the > > > > > diagnostics > > > > > subsystem - though the static analyzer is probably the only > > > > > GCC > > > > > component generating diagnostic paths. > > > > > > > > > > I'm happy to mentor such a project as I maintain both > > > > > subsystems > > > > > and > > > > > SARIF output would benefit both - but it would be rather > > > > > tangential > > > > > to > > > > > the analyzer - so if you had specifically wanted to be > > > > > working on > > > > > the > > > > > guts of the analyzer itself, you may want to pick a different > > > > > subproject. > > > > > > > > > > The SARIF standard is rather long and complicated, and we > > > > > would > > > > > want to > > > > > be compatible with clang's implementation. > > > > > > > > > > It would be very cool if gcc could also accept SARIF files as > > > > > an > > > > > *input* format, and emit them as diagnostics; that might help > > > > > with > > > > > debugging SARIF output.   (I have a old patch for adding JSON > > > > > parsing > > > > > support to GCC that could be used as a starting point for > > > > > this). > > > > > > > > > > Hope the above makes sense > > > > > Dave > > > > > > > > > > > > > > > > > On Tue, Jan 11, 2022, 7:09 PM David Malcolm < > > > > > > dmalcolm@redhat.com> > > > > > > wrote: > > > > > > > > > > > > > On Tue, 2022-01-11 at 11:03 +0530, Mir Immad via Gcc > > > > > > > wrote: > > > > > > > > Hi everyone, > > > > > > > > > > > > > > Hi, and welcome. > > > > > > > > > > > > > > > I intend to work on the static analyzer. Are these > > > > > > > > documents > > > > > > > > enough to > > > > > > > > get > > > > > > > > started: https://gcc.gnu.org/onlinedocs/gccint and > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://gcc.gnu.org/onlinedocs/gccint/Analyzer-Internals.html#Analyzer-Internals > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > There are also some high-level notes here: > > > > > > >   https://gcc.gnu.org/wiki/DavidMalcolm/StaticAnalyzer > > > > > > > > > > > > > > Also, given that the analyzer is part of GCC, the more > > > > > > > general > > > > > > > introductions to hacking on GCC will be useful. > > > > > > > > > > > > > > I recommend creating a trivial C source file with a bug > > > > > > > in it > > > > > > > (e.g. > > > > > > > a > > > > > > > 3-line function with a use-after-free), and stepping > > > > > > > through > > > > > > > the > > > > > > > analyzer to get a sense of how it works. > > > > > > > > > > > > > > Hope this is helpful; don't hesitate to ask questions. > > > > > > > Dave > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >