From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com [IPv6:2a00:1450:4864:20::231]) by sourceware.org (Postfix) with ESMTPS id BC7BF3861037 for ; Wed, 5 Jun 2024 07:26:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BC7BF3861037 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BC7BF3861037 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::231 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717572383; cv=none; b=qgbEmww2zH5EEF3SfdYUbjBKit+Qp4N+31Cmp/TMp2rq9y+xtYT3ioBFVCx8uezDPJhD9BjDpEDboip3ZEo3EVhcYQQcIN+8dJqKBaW8hBa8ZZsdlBrmr4QtCyY6UMbd+z58i+AvEhW7Yq1g2Efw0OCi6VBmwh5feALMn7kv5M0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717572383; c=relaxed/simple; bh=piCr+Ks1SS48jj0bz82/iXDIqRPpuIvPM6E4cwYuCzs=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=HYPZiFJFj3BPc4MiFCHAONjs3ayongjzkNhJW5yJHT75iaTH/CAz/CWL/1qcvUlYCBuizFYJtYfiOVHyBQagsGJ7/gfDO2KZvcLqsN+jM2YphjtfPJcxYbEJpvHana152ngugreQLDjoBWhZ9XN8SJ90+rDjudOqDjSW4AdYd3Y= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lj1-x231.google.com with SMTP id 38308e7fff4ca-2eaac465915so19197731fa.1 for ; Wed, 05 Jun 2024 00:26:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717572378; x=1718177178; darn=gcc.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=SSFWCFZZU028O643be97hnM6vVgf13y98h8b0oA3mAg=; b=QM9mRk0kB1omvvIFOHrDn7XABR4LaRjlZ5APyCHa2E8lPcx3r3oU5NkP41LgMWgApL ed0bo+5Bf4lY0Sr0DL5GHRbbtZSNGxRmlQC3Vr4VU8aYWA/8urDjuEJXifyc011NQOwO mhatr6AR0264ARJL9Pjze+3WrLatrP0+aJPIx1bpA17fEzZghHW6Q1cHGmerpHQmWC+U 2B5RE7gNXlHV9V7OlAef5rGpg+l8+LKsnwLbtUcTp9TpQl8Ioc1h0MdqtxBJIx7ZqHSf TEjCfXxkcd4NYOCavZk2crH6oRR20jebfdhktJRFGUsMEq7LMv+5+QCVD8jgJxTcMIjA Izjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717572378; x=1718177178; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SSFWCFZZU028O643be97hnM6vVgf13y98h8b0oA3mAg=; b=kmRbloCbPMP2BChPw6QnVhYVaI9pb0WPOkSxb/TH59SI4Q1lQKHJzLhUxieG33BFNj acmx2Jt9M4aF1CumgQGmQMK16zoKNEMSIASluxbwuUWQfNOfRwmVvrC30InE4Yelcs/E BQC0qtKb33SNh9557SQRiJV59FipmhtwE1ahF0zmMo3RClw8Cia+e2yMos/7vu0WVtwS OmInWs9TD1t1XWNWCfRTp9x91qcwZNbGwB+lg1xyF6Rl6lm6DQPMaheT/du0xPsgRigU RQkzqzmIJBYqmNwD3g4de9BO1ssoo2LHVgwXDhj+Jr5FG45+N8M3qsZtXDXrvaCZ70kV IPSw== X-Forwarded-Encrypted: i=1; AJvYcCXhNx7SBS9AGic39V0oAWWpjxv3wYgL0yBBGZNY14sZl4jkIhQ/grNr25B3OAmb3U9CNTgG86eVNLHWmxWj+l/XUWBTk7uv2w== X-Gm-Message-State: AOJu0Yy4+lmFJVQ71Wo04Ov6yLSsfNspbIY/DdlZwIOX2f8aS9UsuS0A /aXpgdDq6Mih9DJWgD8XLWw37Pq9PNDZZ7UXhzGEFAzFMuq4AibqSbOXxrm0/xAkXXatwj6ZgdS koc4QZ07Dq7WbbFIDgDWhJRAGzw4= X-Google-Smtp-Source: AGHT+IGIx4tURhNWiha+2angbWd+1+F24CMxHpKpw+Heq/Sb6nOAip0Nte3pLeN76gaJLHKt+2V8HCEupitXpj4syyw= X-Received: by 2002:a05:6512:39c8:b0:523:72b8:8002 with SMTP id 2adb3069b0e04-52baca5abb4mr1050423e87.30.1717572377930; Wed, 05 Jun 2024 00:26:17 -0700 (PDT) MIME-Version: 1.0 References: <20240513194830.1676938-1-qing.zhao@oracle.com> <35799652-55op-79os-ro61-56094939s0o2@fhfr.qr> <8c3c2c188421e4817249a5bc6d51dcf43eb079db.camel@redhat.com> <96A847FA-C41E-4BE3-BECB-83935E0D8415@oracle.com> <552015E4-10FF-4E20-BC2F-8D88A6C55C0B@oracle.com> <6D93ECB0-17CF-4BF8-8644-83EC1955DCEC@oracle.com> <978fa12b94451d07875b9bc524ef0ef26ac641ec.camel@redhat.com> In-Reply-To: From: Richard Biener Date: Wed, 5 Jun 2024 09:26:02 +0200 Message-ID: Subject: Re: [RFC][PATCH] PR tree-optimization/109071 - -Warray-bounds false positive warnings due to code duplication from jump threading To: Qing Zhao Cc: David Malcolm , GCC Patches , Andrew Pinski , kees Cook Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-7.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, Jun 4, 2024 at 10:31=E2=80=AFPM Qing Zhao wr= ote: > > > > > On Jun 4, 2024, at 03:43, Richard Biener w= rote: > > > > On Mon, Jun 3, 2024 at 4:48=E2=80=AFPM David Malcolm wrote: > >> > >> On Mon, 2024-06-03 at 08:29 +0200, Richard Biener wrote: > >>> On Fri, May 31, 2024 at 11:23=E2=80=AFPM Qing Zhao > >>> wrote: > >>>> > >>>> > >>>> > >>>>> On May 23, 2024, at 07:46, Richard Biener > >>>>> wrote: > >>>>> > >>>>> On Wed, May 22, 2024 at 8:53=E2=80=AFPM Qing Zhao > >>>>> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On May 22, 2024, at 03:38, Richard Biener > >>>>>>> wrote: > >>>>>>> > >>>>>>> On Tue, May 21, 2024 at 11:36=E2=80=AFPM David Malcolm > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> On Tue, 2024-05-21 at 15:13 +0000, Qing Zhao wrote: > >>>>>>>>> Thanks for the comments and suggestions. > >>>>>>>>> > >>>>>>>>>> On May 15, 2024, at 10:00, David Malcolm > >>>>>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> On Tue, 2024-05-14 at 15:08 +0200, Richard Biener > >>>>>>>>>> wrote: > >>>>>>>>>>> On Mon, 13 May 2024, Qing Zhao wrote: > >>>>>>>>>>> > >>>>>>>>>>>> -Warray-bounds is an important option to enable > >>>>>>>>>>>> linux kernal to > >>>>>>>>>>>> keep > >>>>>>>>>>>> the array out-of-bound errors out of the source > >>>>>>>>>>>> tree. > >>>>>>>>>>>> > >>>>>>>>>>>> However, due to the false positive warnings > >>>>>>>>>>>> reported in > >>>>>>>>>>>> PR109071 > >>>>>>>>>>>> (-Warray-bounds false positive warnings due to code > >>>>>>>>>>>> duplication > >>>>>>>>>>>> from > >>>>>>>>>>>> jump threading), -Warray-bounds=3D1 cannot be added > >>>>>>>>>>>> on by > >>>>>>>>>>>> default. > >>>>>>>>>>>> > >>>>>>>>>>>> Although it's impossible to elinimate all the false > >>>>>>>>>>>> positive > >>>>>>>>>>>> warnings > >>>>>>>>>>>> from -Warray-bounds=3D1 (See PR104355 Misleading - > >>>>>>>>>>>> Warray-bounds > >>>>>>>>>>>> documentation says "always out of bounds"), we > >>>>>>>>>>>> should minimize > >>>>>>>>>>>> the > >>>>>>>>>>>> false positive warnings in -Warray-bounds=3D1. > >>>>>>>>>>>> > >>>>>>>>>>>> The root reason for the false positive warnings > >>>>>>>>>>>> reported in > >>>>>>>>>>>> PR109071 is: > >>>>>>>>>>>> > >>>>>>>>>>>> When the thread jump optimization tries to reduce > >>>>>>>>>>>> the # of > >>>>>>>>>>>> branches > >>>>>>>>>>>> inside the routine, sometimes it needs to duplicate > >>>>>>>>>>>> the code > >>>>>>>>>>>> and > >>>>>>>>>>>> split into two conditional pathes. for example: > >>>>>>>>>>>> > >>>>>>>>>>>> The original code: > >>>>>>>>>>>> > >>>>>>>>>>>> void sparx5_set (int * ptr, struct nums * sg, int > >>>>>>>>>>>> index) > >>>>>>>>>>>> { > >>>>>>>>>>>> if (index >=3D 4) > >>>>>>>>>>>> warn (); > >>>>>>>>>>>> *ptr =3D 0; > >>>>>>>>>>>> *val =3D sg->vals[index]; > >>>>>>>>>>>> if (index >=3D 4) > >>>>>>>>>>>> warn (); > >>>>>>>>>>>> *ptr =3D *val; > >>>>>>>>>>>> > >>>>>>>>>>>> return; > >>>>>>>>>>>> } > >>>>>>>>>>>> > >>>>>>>>>>>> With the thread jump, the above becomes: > >>>>>>>>>>>> > >>>>>>>>>>>> void sparx5_set (int * ptr, struct nums * sg, int > >>>>>>>>>>>> index) > >>>>>>>>>>>> { > >>>>>>>>>>>> if (index >=3D 4) > >>>>>>>>>>>> { > >>>>>>>>>>>> warn (); > >>>>>>>>>>>> *ptr =3D 0; // Code duplications since > >>>>>>>>>>>> "warn" does > >>>>>>>>>>>> return; > >>>>>>>>>>>> *val =3D sg->vals[index]; // same this line. > >>>>>>>>>>>> // In this path, > >>>>>>>>>>>> since it's > >>>>>>>>>>>> under > >>>>>>>>>>>> the condition > >>>>>>>>>>>> // "index >=3D 4", the > >>>>>>>>>>>> compiler > >>>>>>>>>>>> knows > >>>>>>>>>>>> the value > >>>>>>>>>>>> // of "index" is > >>>>>>>>>>>> larger then 4, > >>>>>>>>>>>> therefore the > >>>>>>>>>>>> // out-of-bound > >>>>>>>>>>>> warning. > >>>>>>>>>>>> warn (); > >>>>>>>>>>>> } > >>>>>>>>>>>> else > >>>>>>>>>>>> { > >>>>>>>>>>>> *ptr =3D 0; > >>>>>>>>>>>> *val =3D sg->vals[index]; > >>>>>>>>>>>> } > >>>>>>>>>>>> *ptr =3D *val; > >>>>>>>>>>>> return; > >>>>>>>>>>>> } > >>>>>>>>>>>> > >>>>>>>>>>>> We can see, after the thread jump optimization, the > >>>>>>>>>>>> # of > >>>>>>>>>>>> branches > >>>>>>>>>>>> inside > >>>>>>>>>>>> the routine "sparx5_set" is reduced from 2 to 1, > >>>>>>>>>>>> however, due > >>>>>>>>>>>> to > >>>>>>>>>>>> the > >>>>>>>>>>>> code duplication (which is needed for the > >>>>>>>>>>>> correctness of the > >>>>>>>>>>>> code), > >>>>>>>>>>>> we > >>>>>>>>>>>> got a false positive out-of-bound warning. > >>>>>>>>>>>> > >>>>>>>>>>>> In order to eliminate such false positive out-of- > >>>>>>>>>>>> bound warning, > >>>>>>>>>>>> > >>>>>>>>>>>> A. Add one more flag for GIMPLE: is_splitted. > >>>>>>>>>>>> B. During the thread jump optimization, when the > >>>>>>>>>>>> basic blocks > >>>>>>>>>>>> are > >>>>>>>>>>>> duplicated, mark all the STMTs inside the original > >>>>>>>>>>>> and > >>>>>>>>>>>> duplicated > >>>>>>>>>>>> basic blocks as "is_splitted"; > >>>>>>>>>>>> C. Inside the array bound checker, add the > >>>>>>>>>>>> following new > >>>>>>>>>>>> heuristic: > >>>>>>>>>>>> > >>>>>>>>>>>> If > >>>>>>>>>>>> 1. the stmt is duplicated and splitted into two > >>>>>>>>>>>> conditional > >>>>>>>>>>>> paths; > >>>>>>>>>>>> + 2. the warning level < 2; > >>>>>>>>>>>> + 3. the current block is not dominating the exit > >>>>>>>>>>>> block > >>>>>>>>>>>> Then not report the warning. > >>>>>>>>>>>> > >>>>>>>>>>>> The false positive warnings are moved from -Warray- > >>>>>>>>>>>> bounds=3D1 to > >>>>>>>>>>>> -Warray-bounds=3D2 now. > >>>>>>>>>>>> > >>>>>>>>>>>> Bootstrapped and regression tested on both x86 and > >>>>>>>>>>>> aarch64. > >>>>>>>>>>>> adjusted > >>>>>>>>>>>> -Warray-bounds-61.c due to the false positive > >>>>>>>>>>>> warnings. > >>>>>>>>>>>> > >>>>>>>>>>>> Let me know if you have any comments and > >>>>>>>>>>>> suggestions. > >>>>>>>>>>> > >>>>>>>>>>> At the last Cauldron I talked with David Malcolm > >>>>>>>>>>> about these kind > >>>>>>>>>>> of > >>>>>>>>>>> issues and thought of instead of suppressing > >>>>>>>>>>> diagnostics to > >>>>>>>>>>> record > >>>>>>>>>>> how a block was duplicated. For jump threading my > >>>>>>>>>>> idea was to > >>>>>>>>>>> record > >>>>>>>>>>> the condition that was proved true when entering the > >>>>>>>>>>> path and do > >>>>>>>>>>> this > >>>>>>>>>>> by recording the corresponding locations > >>>>>>>>> > >>>>>>>>> Is only recording the location for the TRUE path enough? > >>>>>>>>> We might need to record the corresponding locations for > >>>>>>>>> both TRUE and > >>>>>>>>> FALSE paths since the VRP might be more accurate on both > >>>>>>>>> paths. > >>>>>>>>> Is only recording the location is enough? > >>>>>>>>> Do we need to record the pointer to the original > >>>>>>>>> condition stmt? > >>>>>>>> > >>>>>>>> Just to be clear: I don't plan to work on this myself (I > >>>>>>>> have far too > >>>>>>>> much already to work on...); I'm assuming Richard Biener is > >>>>>>>> hoping/planning to implement this himself. > >>>>>>> > >>>>>>> While I think some of this might be an improvement to those > >>>>>>> vast > >>>>>>> number of "false positive" diagnostics we have from (too) > >>>>>>> late diagnostic > >>>>>>> passes I do not have the cycles to work on this. > >>>>>> > >>>>>> I can study a little bit more on how to improve the diagnostics > >>>>>> for PR 109071 first. > >>>>>> > >>>>>> FYI, I just updated PR109071=E2=80=99s description as: Need more > >>>>>> context for -Warray-bounds warnings due to code duplication > >>>>>> from jump threading. > >>>>>> > >>>>>> As we already agreed, this is NOT a false positive. It caught a > >>>>>> real bug in linux kernel that need to be patched in the kernel > >>>>>> source. (See > >>>>>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D109071#c11) > >>>>>> > >>>>>> In order to add more context to the diagnostics to help the end > >>>>>> user locate the bug accurately in their source code, compiler > >>>>>> needs: > >>>>>> > >>>>>> 1. During jump threading phase, record the following > >>>>>> information for each duplicated STMT: > >>>>>> A. A pointer to the COND stmt; > >>>>>> B. True/false for such COND > >>>>>> 2. During array out-of-bound checking phase, whenever locate an > >>>>>> out-of-bound case, > >>>>>> A. Use a rich_location for the diagnostic; > >>>>>> B. Create an instance of diagnositic_path, add events to > >>>>>> this diagnostic_path based on the COND stmt, True/False info > >>>>>> recorded in the STMT; > >>>>>> C. Call richloc.set_path() to associate the path with > >>>>>> the rich_location; > >>>>>> D. Then issue warning with this rich_location instead of > >>>>>> the current regular location. > >>>>>> > >>>>>> Any comment and suggestion to the above? > >>>>> > >>>>> I was originally thinking of using location_adhoc_data to store > >>>>> 1.A > >>>>> and 1.B as a common bit to associate to each > >>>>> copied stmt. IIRC location_adhoc_data->data is the stmts > >>>>> associated > >>>>> lexical block so we'd need to stuff another > >>>>> 'data' field into this struct, like a "copy history" where we > >>>>> could > >>>>> then even record copied-of-copy-of-X. > >>>>> locataion_adhoc_data seems imperfectly packed right now, with a > >>>>> 32bit > >>>>> hole before 'data' and 32bit unused > >>>>> at its end, so we might get away without memory use by re- > >>>>> ordering > >>>>> discriminator before 'data' ... > >>>> > >>>> Like this? > >>>> > >>>> diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h > >>>> index e6e2b0897572..ee344f91333b 100644 > >>>> --- a/libcpp/include/line-map.h > >>>> +++ b/libcpp/include/line-map.h > >>>> @@ -761,8 +761,9 @@ struct GTY(()) maps_info_macro { > >>>> struct GTY(()) location_adhoc_data { > >>>> location_t locus; > >>>> source_range src_range; > >>>> - void * GTY((skip)) data; > >>>> unsigned discriminator; > >>>> + void * GTY((skip)) data; > >>>> + void * GTY((skip)) copy_history; > >>>> }; > >>>> struct htab; > >>> > >>> Yes. > >>> > >>>> How about the copy_history? Do we need a new data structure (like > >>>> the following, or other suggestion) for this? Where should I add > >>>> this new data structure? > >>> > >>> As it needs to be managed by libcpp it should be in this very same > >>> file. > >>> > >>>> struct copy_history { > >>>> location_t condition; > >>>> Bool is_true_path; > >>>> } > >>> > >>> I think we want a pointer to the previous copy-of state as well in > >>> case a stmt > >>> was copied twice. We'll see whether a single (condition) location > >>> plus edge flag > >>> is sufficient. I'd say we should plan for an enum to indicate the > >>> duplication > >>> reason at least (jump threading, unswitching, unrolling come to my > >>> mind). For > >>> jump threading being able to say "when is true/false" is > >>> probably > >>> good enough, though it might not always be easy to identify a single > >>> condition > >>> here given a threading path starts at an incoming edge to a CFG merge > >>> and > >>> will usually end with the outgoing edge of a condition that we can > >>> statically > >>> evaluate. The condition controlling the path entry might not be > >>> determined > >>> fully by a single condition location. > >>> > >>> Possibly building a full "diagnostic path" object at threading time > >>> might be > >>> the only way to record all the facts, but that's also going to be > >>> more > >>> expensive. > >> > >> Note that a diagnostic_path represents a path through some kind of > >> graph, whereas it sounds like you want to be storing the *nodes* in th= e > >> graph, and later generating the diagnostic_path from that graph when w= e > >> need it (which is trivial if the graph is actually just a tree: just > >> follow the parent links backwards, then reverse it). > > > > I think we are mixing two things - one is that a single transform like = jump > > threading produces a stmt copy and when we emit a diagnostic on that > > copied statement we want to tell the user the condition under which the > > copy is executed. That "condition" can be actually a sequence of > > conditionals. I wanted to point out that a diagnostic_path instance co= uld > > be used to describe such complex condition. > > > > But then the other thing I wanted to address with the link to a previou= s > > copy_history - that's when a statement gets copied twice, for example > > by two distinct jump threading optimizations. Like when dumping > > the inlining decisions for diagnostics we could dump the logical "and" > > of the conditions of the two threadings. Since we have a single > > location per GIMPLE stmt we'd have to keep a "stack" of copy events > > associated with it. That's the linked list (I think a linked list shou= ld > > work fine here). > Yes, the linked list to keep the =E2=80=9Cstack=E2=80=9D of copy events s= hould be good enough > to form the sequence of conditionals event for the diagnostic_path insta= nce. > > > > I realize things may get a bit "fat", but eventually we are not duplica= ting > > statements that much. I do hope we can share for example a big > > diagnostic_path when we duplicate a basic block during threading > > and use one instance for all stmts in such block copy (IIRC we never > > release locations or their ad-hoc data, we just make sure to never > > use locations with ad-hoc data pointing to BLOCKs that we released, > > but the linemap data will still have pointers in "dead" location entrie= s, > > right?) > Are you still suggesting to add two artificial stmts in the beginning and= the > end of the duplicated block to carry the copy history information for all= the > stmts in the block to save space? > > Compared with the approach to carry such information to each duplicated s= tmts (which I preferred), > The major concerns with the approach are: > 1. Optimizations might move stmts out of these two artificial stmts, t= herefore we need add > Some memory barrier on these two artificial stmts to prevent such= movements. > This might prevent good optimization from happening and result in = runtime performance loss; > 2. Every time when query whether the stmt is copied and get its copy = history, we have to search backward or > Forward through the stmt chain to get to the artificial stmts tha= t carry the copy history, compilation time will > be slower. > > I still think that carrying the copy history info to each duplicated stmt= s might be better. We can limit the length of > the history to control the space, what do you think? Copying to each stmt is definitely easier so I think it's better to start with that and only resort to other things when this fails. Note we could, instead of putting a pointer to data into the ad-hoc info, put in an index and have the actual data be maintained elsewhere. That data could even only keep the "last" 1000 contexts and simply discard "older" ones. Before IPA info can still be transfered between functions, but after IPA where most of the threadings happen the extra info could be discarded once we get a function to the last pass emitting diagnostics we want to amend. Doing an index lookup makes it possible to not worry about "stale" entries. Richard. > > Qing > > > > > Richard. > > > >> > >> Dave > >> > >>> I can think of having a -fdiagnostic-try-to-explain-harder option to > >>> clarify confusing > >>> diagnostics people could use on-demand? > >>> > >>> Richard. > >>> > >>>> Qing > >>>>> > >>>>> Richard. > >>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> Qing > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> Richard. > >>>>>>> > >>>>>>>> But feel free to ask me any questions about the > >>>>>>>> diagnostic_path > >>>>>>>> machinery within the diagnostics subsystem. > >>>>>>>> > >>>>>>>> [...snip...] > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> Dave > >