From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rock.gnat.com (rock.gnat.com [205.232.38.15]) by sourceware.org (Postfix) with ESMTPS id 131E83856263; Thu, 23 Jun 2022 11:39:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 131E83856263 Received: from localhost (localhost.localdomain [127.0.0.1]) by filtered-rock.gnat.com (Postfix) with ESMTP id BDB3F1161BF; Thu, 23 Jun 2022 07:39:20 -0400 (EDT) X-Virus-Scanned: Debian amavisd-new at gnat.com Received: from rock.gnat.com ([127.0.0.1]) by localhost (rock.gnat.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id KY7XLtePmwMv; Thu, 23 Jun 2022 07:39:20 -0400 (EDT) Received: from free.home (tron.gnat.com [IPv6:2620:20:4000:0:46a8:42ff:fe0e:e294]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by rock.gnat.com (Postfix) with ESMTPS id 852071161A1; Thu, 23 Jun 2022 07:39:20 -0400 (EDT) Received: from livre (livre.home [172.31.160.2]) by free.home (8.15.2/8.15.2) with ESMTPS id 25NBdAjY763475 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Jun 2022 08:39:11 -0300 From: Alexandre Oliva To: Jonathan Wakely Cc: gcc Patches , "libstdc++" Subject: Re: [PATCH] libstdc++: testsuite: avoid predictable mkstemp Organization: Free thinker, does not speak for AdaCore References: Errors-To: aoliva@lxoliva.fsfla.org Date: Thu, 23 Jun 2022 08:39:10 -0300 In-Reply-To: (Jonathan Wakely's message of "Wed, 22 Jun 2022 10:16:40 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.84 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libstdc++@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libstdc++ mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2022 11:39:22 -0000 On Jun 22, 2022, Jonathan Wakely wrote: > On Wed, 22 Jun 2022 at 07:05, Alexandre Oliva via Libstdc++ > wrote: >> It was prompted by a target system with a non-random implementation of >> mkstemp, that returns a predictable sequence of filenames and selects >> the first one that isn't already taken. > OK And here's the patch that enabled me to stop worrying about the above. Regstrapped on x86_64-linux-gnu, also tested with a cross to aarch64-rtems6. Ok to install? __gnu_test::nonexistent_path: Always include counter in filename returned From: Joel Brobecker We have noticed that, on RTEMS, a small number of testscases are failing because two calls to this method return the same filename. This happens for instance in 27_io/filesystem/operations/copy_file.cc where it does: auto from = __gnu_test::nonexistent_path(); auto to = __gnu_test::nonexistent_path(); We tracked this issue down to the fact that the implementation of mkstemp on that system appears to use a very predictable algorithm for chosing the name of the temporary file, where the same filename appears to be tried in the same order, regardless of past calls. So, as long as the file gets deleted after a call to mkstemp (something we do here in our nonexistent_path method), the next call to mkstemps ends up returning the same filename, causing the collision we se above. This commit enhances the __gnu_test::nonexistent_path method to introduce in the filename being returned a counter which gets incremented at every call of this method. libstdc++-v3/ChangeLog: * testsuite/util/testsuite_fs.h (__gnu_test::nonexistent_path): Always include a counter in the filename returned. --- libstdc++-v3/testsuite/util/testsuite_fs.h | 31 ++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/libstdc++-v3/testsuite/util/testsuite_fs.h b/libstdc++-v3/testsuite/util/testsuite_fs.h index 037d9ffc0f429..206ea67779003 100644 --- a/libstdc++-v3/testsuite/util/testsuite_fs.h +++ b/libstdc++-v3/testsuite/util/testsuite_fs.h @@ -38,9 +38,9 @@ namespace test_fs = std::experimental::filesystem; #if defined(_GNU_SOURCE) || _XOPEN_SOURCE >= 500 || _POSIX_C_SOURCE >= 200112L #include // mkstemp -#else -#include // std::random_device +#include // strcpy #endif +#include // std::random_device #if defined(__MINGW32__) || defined(__MINGW64__) \ || !defined (_GLIBCXX_HAVE_SYMLINK) @@ -125,8 +125,32 @@ namespace __gnu_test file.erase(0, pos+1); test_fs::path p; + // A counter, starting from a random value, to be included as part + // of the filename being returned, and incremented each time + // this method is used. It allows us to ensure that two calls + // to this method can never return the same filename, something + // testcases do when they need multiple non-existent filenames + // for their purposes. + static unsigned counter = std::random_device{}(); + #if defined(_GNU_SOURCE) || _XOPEN_SOURCE >= 500 || _POSIX_C_SOURCE >= 200112L - char tmp[] = "filesystem-test.XXXXXX"; + // Use mkstemp to determine the name of a file which does not exist yet. + // + // Note that we have seen on some systems (such as RTEMS, for instance) + // that mkstemp behaves very predictably, causing it to always try + // the same sequence of file names. In other words, if we call mkstemp + // with a pattern, delete the file it created (which is what we do, here), + // and call mkstemp with the same pattern again, it returns the same + // filename once more. While most implementations introduce a degree + // of randomness, it is not mandated by the standard, and this is why + // we include a counter in the template passed to mkstemp. + std::string mkstemp_template ("filesystem-test."); + mkstemp_template.append(std::to_string (counter++)); + mkstemp_template.append(".XXXXXX"); + + char tmp[mkstemp_template.length() + 1]; + std::strcpy (tmp, mkstemp_template.c_str()); + int fd = ::mkstemp(tmp); if (fd == -1) throw test_fs::filesystem_error("mkstemp failed", @@ -141,7 +165,6 @@ namespace __gnu_test if (file.length() > 64) file.resize(64); char buf[128]; - static unsigned counter = std::random_device{}(); #if _GLIBCXX_USE_C99_STDIO std::snprintf(buf, 128, #else -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer Disinformation flourishes because many people care deeply about injustice but very few check the facts. Ask me about