From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by sourceware.org (Postfix) with ESMTPS id D036D3858C31; Mon, 6 Nov 2023 17:48:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D036D3858C31 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=quicinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D036D3858C31 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699292897; cv=none; b=fqBE+9iPkKlLfVHCGVWDzPA9c8CGEwPlkgOI0mQfApPhgp/jbtZTACCELt1CnIL0z7pQIGLHlAhtkU5cciEUXJ3qAgBMli2WZm8EHsee44TX3M6/It8MV1WZEtJOCNMwhVAf4klR8TQzLbUnGorncLQdYDTRK2VtDz65i70hyXg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699292897; c=relaxed/simple; bh=2ED6yjCEtSFOcWg9dFkEoa6R4l9AWx9XHYY+O/cEOtQ=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=vzc2mBJyEsmZXUKNOGszHVOKWWt84+ibwGVKKPR3qXZLrtjx6RDV2fTDR2JxJLUY/cQMeYkzuJ/hBWcuNCXIlbharQcxpxXU0VCL+MdQNWH5srikAClBNjIsHPAHPZf6DlGkOlD7m11Yuu7u0auhdxj0gr2HqsxZ4XeVWecxXUU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3A6CWQfi026712; Mon, 6 Nov 2023 17:48:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=LNY8Sk41IF7oWcTSBV06f1Qs0FvlHw5fWGC0OMzYoU8=; b=DiUcwvANYFGRGsOUez2r7BevhPuoD/KADPyYuRGDcnpj/2qSP/SyQ4MOxt0Y0qbCuX5G 7PC0rZ6V9ZEgL/MM1v+pZ1obY+L0lv3PyWKU3BB87IpXpbAm/UmJI+/dhTjvYa1YHQUl b7JFEpqGWU4PXX9mj/ogj+6uehsFItPyfL338xGiQoT22xzhSFrSuMW+iM/0SL//QB+4 6dDTMxTFj041NZn8ttzvS+fC8r7KTuQEmT5FFav0XBI92P8+sBUtRPvXLZlwcRwe3cTx PsSm9jisHbPKUELVE+FLdGZ474BolsVEhbX3AhzhW4vpxp9bFZdi8+1s3qJOaaYeS7EH Ww== Received: from nalasppmta03.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3u6xdu9668-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Nov 2023 17:48:10 +0000 Received: from nalasex01b.na.qualcomm.com (nalasex01b.na.qualcomm.com [10.47.209.197]) by NALASPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 3A6Hm94k022587 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 6 Nov 2023 17:48:09 GMT Received: from [10.110.120.154] (10.80.80.8) by nalasex01b.na.qualcomm.com (10.47.209.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Mon, 6 Nov 2023 09:48:09 -0800 Message-ID: <16f3e24f-a635-46cd-9e16-c45f7700936c@quicinc.com> Date: Mon, 6 Nov 2023 09:47:17 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [Bug default/31017] Flex array conversion suppression Content-Language: en-US To: Dodji Seketeli , quic_johmoo at quicinc dot com CC: References: <87bkc8cz84.fsf@seketeli.org> From: John Moon In-Reply-To: <87bkc8cz84.fsf@seketeli.org> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01b.na.qualcomm.com (10.47.209.197) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: -f3aj5M42YXtizH1YtfiUogtBrLdufcK X-Proofpoint-ORIG-GUID: -f3aj5M42YXtizH1YtfiUogtBrLdufcK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-11-06_13,2023-11-02_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 spamscore=0 impostorscore=0 mlxscore=0 malwarescore=0 clxscore=1011 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2310240000 definitions=main-2311060145 X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 11/5/2023 1:43 AM, Dodji Seketeli wrote: > "quic_johmoo at quicinc dot com" a > écrit: > >> https://sourceware.org/bugzilla/show_bug.cgi?id=31017 >> >> --- Comment #2 from John Moon --- >> >> Thanks for the implementation! I think this looks great. I tested it and it >> seems to be working properly for our use case in the kernel! >> >> One question about this block: >> >> + // Support for the >> + // "has_strict_flexible_array_data_member_conversion = true" >> + // clause. >> + if (has_strict_fam_conversion()) >> + { >> + // Let's detect if the first class of the diff has a fake >> + // flexible array data member that got turned into a real >> + // flexible array data member. >> + if (!( >> + (has_fake_flexible_array_data_member(first_class) >> + && has_flexible_array_data_member(second_class)) >> + // A fake flexible array member has been changed into >> + // a real flexible array ... >> + && >> + ((first_class->get_size_in_bits() >> + == second_class->get_size_in_bits()) >> + || get_has_size_change()) >> + // There was no size change or the suppression has a >> + // "has_size_change = true" clause. >> + )) >> + return false; >> + } >> >> Is it possible for a structure to meet the first condition (fake flex -> flex) >> *without* a size change? > > As a general rule, suppression specifications (aka supprspecs) don't apply to a type > which size has changed, unless the user /really/ wants the supprspec to > apply. If she really wants it, then she has to explicitly say > "has_size_change = yes". This is prevents "too eager" supprspecs to be > applied without the user noticing the supprspecs is too eager. > > Basically, if a type's size changed, more often than not, we don't want > to suppress its change report, for obvious reasons. > > That is why, throughout type_suppression::suppresses_diff you see the > careful attention to the size change condition, when evaluating > supprspecs. Right, I wasn't suggesting to apply the suppression even without the "has_size_change = true" clause. I just thought we could avoid the check as it was always true. > > In this particular case, I think that we can have "fake flex -> flex" > changes without a size change because there can other /additional/ > changes that counter the size change we would have expected. That > change could have been introduced, on purpose, to keep the ABI stable. > For instance: > > $ diff -u test-PR31017-2-v0.c test-PR31017-2-v1.c > --- test-PR31017-2-v0.c 2023-11-05 10:04:36.433000539 +0100 > +++ test-PR31017-2-v1.c 2023-11-05 10:07:00.579810832 +0100 > @@ -2,7 +2,8 @@ > { > int x; > int y; > - int end[1]; > + int padding; > + int end[]; > }; > > $ /home/dodji/git/libabigail/PR31017/build/tools/abidiff test-PR31017-2-v0.o test-PR31017-2-v1.o > Functions changes summary: 0 Removed, 1 Changed, 0 Added function > Variables changes summary: 0 Removed, 0 Changed, 0 Added variable > > 1 function with some indirect sub-type change: > > [C] 'function void fun(foo*)' at test-PR31017-2-v0.c:9:1 has some indirect sub-type changes: > parameter 1 of type 'foo*' has sub-type changes: > in pointed to type 'struct foo' at test-PR31017-2-v1.c:1:1: > type size hasn't changed > 1 data member insertion: > 'int padding', at offset 64 (in bits) at test-PR31017-2-v1.c:5:1 > 1 data member change: > type of 'int end[1]' changed: > type name changed from 'int[1]' to 'int[]' > array type size changed from 32 to 'unknown' > array type subrange 1 changed length from 1 to 'unknown' > and offset changed from 64 to 96 (in bits) (by +32 bits) > > $ > And this proves I was wrong! :) I thought libabigail would consider the whole structure size as "unknown" if there was a flex array at the end, but this is clearly not the case. It just takes the size of the known structs (as the compiler does). > One reason why I think it's important to keep this "rigour" with the > type size change thing is that abidiff actually returns a code that is a > bit field that tells callers about the categories of the changes it > encountered. From > https://sourceware.org/libabigail/manual/abidiff.html#return-values, we > see that if the ABIDIFF_ABI_CHANGE bit is set, it means there were some > ABI changes. But then if the ABIDIFF_ABI_INCOMPATIBLE_CHANGE bit is > set, it means abidiff is 100% sure that at least of the changes causes > an ABI incompatibility. In a continuous integration context, for > instance, if the ABIDIFF_ABI_INCOMPATIBLE_CHANGE bit is set, it means we > are sure the change is incompatible, whereas if only the > ABIDIFF_ABI_CHANGE bit is set, it means the change might or might not > incompatible and thus needs a human review to decide. > > To wraps this all up, I'd say that only changes that would NOT set the > ABIDIFF_ABI_INCOMPATIBLE_CHANGE bit should be able to be suppressed, > unless the user really knows what she is doing. > > Thinking about this, maybe check-uapi.sh could use the return code of > abidiff rather than grepping its output. check-uapi.sh would then only > reject changes categorically only if the ABIDIFF_ABI_INCOMPATIBLE_CHANGE > bit is set. If only ABIDIFF_ABI_CHANGE bit is set, check-uapi.sh would > just propose the user to review the changes detected and possibly waive > them. One result of the waiving process would thus be a new supprspec > written and added to the stock of supprspecs to make sure the same kind > of reviews is not requested in the future. Agreed, and in v6 of the script, we do this! If you pass the flag "-i" to the script, it will ignore abidiff results when return code is 4 (ABIDIFF_ABI_CHANGE, but not ABIDIFF_ABI_INCOMPATIBLE_CHANGE). > > With time, if a supprspec is recognized to be needed for this particular > project, libabigail can even integrate it and install it by default for > that project to use. We do this for various projects and their default > supprspecs are included in the default.abignore file that is installed > libabigail. You can browse it at > https://sourceware.org/git/?p=libabigail.git;a=blob;f=default.abignore > and learn about what project requested default supprspecs. > >> I'd think not, but may be missing something. > > I hope my explanation above helps shed some light in this apparently > weird way of doing things. It certainly did, thank you! > >> Basically, I think you can get rid of the first_class->get_size_in_bits() == >> second_class->get_size_in_bits() check. > > I would rather keep it, at least for the sake of consistency in the > behaviour supprspecs evaluation in general, especially with the > unwritten rule: > > "only changes that would NOT set the ABIDIFF_ABI_INCOMPATIBLE_CHANGE > bit should be able to be suppressed, unless the user really knows > what she is doing." > > I guess we need (better) documentation about all this :-( I think the documentation is clear, I just made an incorrect assumption. > >> Also, if we know a size change is a tautology, you could move the >> get_has_size_change() check to be first and save a few CPU cycles. >> >> Other than that, LGTM! >> >> I went ahead and made that change and added (at least a start) on the >> documentation/tests. I don't have access to make a branch, so I just sent a >> patch separately. We can continue discussion there as needed. > > Thanks a lot for moving forward on this! > > I'll wait for your feedback on these comments and we can proceed with > merging your patch accordingly. In the mean time, if I have comments, > I'll follow-up on the patch thread, indeed. > Sounds good, thank you! - John