From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by sourceware.org (Postfix) with ESMTP id A1B9438708F9 for ; Sat, 20 Jun 2020 15:46:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A1B9438708F9 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-GHGtauBGN9uRDTFsfm0asA-1; Sat, 20 Jun 2020 11:46:18 -0400 X-MC-Unique: GHGtauBGN9uRDTFsfm0asA-1 Received: by mail-wm1-f71.google.com with SMTP id u15so3888431wmm.5 for ; Sat, 20 Jun 2020 08:46:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=SLI16mnk/wg+TYupQCL7MFSjy32k9caHllyQvEJnOrI=; b=qiYnnkolGFSswEKSdGkCUkR9nwMS9hsSkwW7m38CanBykLrz5V/YX8UY/Lch/F2RE+ msyaleMNY5MZg/u7E93ctQOqcTyCFFrqy2afJeF6SNyzDbISRkUUdKAllEMeyRwf0DgR hpyNg5Q2clDcZBtMu1tWVOBDA2NsqkKbxCnrgXL2r3E0LDylpbjMKtAjJ95B/2dDyaoR Tk9vdc1oOARQ8WmnKpdXY6NJ9TjGR5Md/kC5O4FrxHrPAiP2l4wjPdVRsrwGo7R4z8et 5ZCjqzXjrV1sghdvmzxxKG5QHhdbwo0aiyRR3nhHiCuUKmEIqoX8Aql6urBVHSMkF+VP FYhQ== X-Gm-Message-State: AOAM532CYG4EXEd0NHFcKWDm2yLUxGsNJa2ocYHzj6ka+JHhr6Z+YxXN vA2nz2jdUx5sBJa7xAQcRIzkBFTw8s1kul/HH/M4WMD5DHu+mkwD0GvUbBLKtOEFf1BO7NAPTd2 50NB1IexskcLpRIlFdwvu0A== X-Received: by 2002:a7b:c5c1:: with SMTP id n1mr8039380wmk.21.1592667976861; Sat, 20 Jun 2020 08:46:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzDNDEVBmrtZQclt6aJc8Ho1R9Mt7fkgZOwzKKNWXRwg2d43GpNpkubFfKpLJLdtFz7UPbSAQ== X-Received: by 2002:a7b:c5c1:: with SMTP id n1mr8039370wmk.21.1592667976667; Sat, 20 Jun 2020 08:46:16 -0700 (PDT) Received: from ?IPv6:2001:8a0:f922:c400:56ee:75ff:fe8d:232b? ([2001:8a0:f922:c400:56ee:75ff:fe8d:232b]) by smtp.gmail.com with ESMTPSA id a12sm828422wrv.41.2020.06.20.08.46.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 20 Jun 2020 08:46:16 -0700 (PDT) Subject: Re: [RFC][gdb/testsuite] Add gdbfail/envfail aliases for kfail/xfail To: Tom de Vries , gdb-patches@sourceware.org References: <20200430064319.GA24688@delia> Cc: Tom Tromey From: Pedro Alves Message-ID: <6ac2b328-76ce-1c0e-8b1d-bd818a0caeae@redhat.com> Date: Sat, 20 Jun 2020 16:46:14 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20200430064319.GA24688@delia> Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2020 15:46:23 -0000 On 4/30/20 7:43 AM, Tom de Vries wrote: > Hi, > > The difference between an xfail and a kfail (in the context of gdb testing) is > that: > - a kfail marks a problem in gdb > - an xfail marks a problem in the environment (compiler, libc, kernel) > > However, this difference is not immediately obvious from the contrast between > "known" and "expected". I memorized the the X in xfail as "external", not "expected". So I always think of them as "known" vs "external", which makes the difference much more obvious, IMO. I propose that the documentation references the "external" mnemonic to help. KFAIL and XFAIL are documented in testsuite/README: XFAIL Short for "expected failure", this indicates a known problem with the environment. This could include limitations of the operating system, compiler version, and other components. Maybe some new entry on KFAIL/XFAIL at: https://sourceware.org/gdb/wiki/GDBTestcaseCookbook would help too. > > Add aliases gdbfail for kfail, and envfail for xfail, that are more intuitive > to use. > > Tested on x86_64-linux. > > Any comments? I think this isn't a good idea: - We still end up with XFAIL / KFAIL in result logs anyway. People still need to know the difference. - It adds different ways to do the same thing. I.e., more things to learn, when you still need to know about the old way anyway. Thanks, Pedro Alves