From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 23273 invoked by alias); 19 Oct 2014 22:45:08 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 23250 invoked by uid 89); 19 Oct 2014 22:45:06 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 X-HELO: mail-oi0-f46.google.com Received: from mail-oi0-f46.google.com (HELO mail-oi0-f46.google.com) (209.85.218.46) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-SHA encrypted) ESMTPS; Sun, 19 Oct 2014 22:45:02 +0000 Received: by mail-oi0-f46.google.com with SMTP id h136so2820396oig.19 for ; Sun, 19 Oct 2014 15:45:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=40IOEzmvh01Kt0Kuw4klctO7xi3pmXdKWqn+V4DcHQE=; b=WV2FgRIXbg7l/LqAQ9RUYswKRpmq5RHUsFmlVEOg31m+79HB17vgeOkwLA176elNYI V5XmGNqASKhVvgsUtxfpeQwBoBgLVOJeZGLY95uhfqEoUj5A9r1yBq6PJq4vEae+ZWb5 TxtLR/itFFEwbb7bFVLGaEGFDiJ21g8UV1HKma3UAh6trJ3kUwboLG3nc2cWyIiaJcja D0Tn/40KZmX77V09WrnqIPwoRBAgGKTjxLdTHIIchvSo1Ec9LSKW4bU4M9GjSLksoxtP 2KzUcjxHzOO6+3r49Pq+NECJx3AXGlsfsjPV577wAKha+4916osAU1gUP54iK+UoIBoz Jajg== X-Gm-Message-State: ALoCoQlhxMZqACNTzQfqtMqz5Q0S8O3GrdmmY35yIjRqw0g9TdMV7QiurMmpvrEShz3OxS15M5Kp MIME-Version: 1.0 X-Received: by 10.60.146.234 with SMTP id tf10mr19864750oeb.3.1413758700252; Sun, 19 Oct 2014 15:45:00 -0700 (PDT) Received: by 10.60.227.67 with HTTP; Sun, 19 Oct 2014 15:44:59 -0700 (PDT) In-Reply-To: <20141016155225.GK10376@tucnak.redhat.com> References: <20141016155225.GK10376@tucnak.redhat.com> Date: Mon, 20 Oct 2014 06:57:00 -0000 Message-ID: Subject: Re: ping x 7: [PATCH] [libgomp] make it possible to use OMP on both sides of a fork From: Nathaniel Smith To: Jakub Jelinek Cc: gcc-patches@gcc.gnu.org Content-Type: multipart/mixed; boundary=047d7b5d314e20a1000505ce5925 X-IsSubscribed: yes X-SW-Source: 2014-10/txt/msg01872.txt.bz2 --047d7b5d314e20a1000505ce5925 Content-Type: text/plain; charset=UTF-8 Content-length: 7623 Hi Jakub, Thanks for your feedback! See below. On Thu, Oct 16, 2014 at 4:52 PM, Jakub Jelinek wrote: > On Mon, Oct 13, 2014 at 10:16:19PM +0100, Nathaniel Smith wrote: >> Got total silence the last 4 times I posted this, and users have been >> bugging me about it offline, so trying again. >> >> This patch fixes a showstopper problem preventing the transparent use >> of OpenMP in scientific libraries, esp. with Python. Specifically, it >> is currently not possible to use GNU OpenMP -- even in a limited, >> temporary manner -- in any program that uses (or might use) fork() for >> parallelism, even if the fork() and the use of OpenMP occur at totally >> different times. This limitation is unique to GNU OpenMP -- every >> competing OpenMP implementation already contains something like this >> patch. While technically not fully POSIX-compliant (because POSIX >> gives much much weaker guarantees around fork() than any real Unix), >> the approach used in this patch (a) performs only POSIX-compliant >> operations when the host program is itself fully POSIX-compliant, and >> (b) actually works perfectly reliably in practice on all commonly used >> platforms I'm aware of. > > 1) gomp_we_are_forked in your patch will attempt to free the pool > of the thread that encounters it, which is racy; consider a program > after fork calling pthread_create several times, each thread > thusly created then ~ at the same time doing #pragma omp parallel > and the initial thread too. You really should clean up the pool > data structure only in the initial thread and nowhere else; > for native TLS (non-emulated, IE model) the best would be to have a flag > in the gomp_thread_pool structure, > struct gomp_thread *thr = gomp_thread (); > if (thr && thr->thread_pool) > thr->thread_pool->after_fork = true; > should in that case be safe in the atfork child handler. > For !HAVE_TLS or emulated TLS not sure if it is completely safe, > it would call pthread_getspecific. Perhaps just don't register > atfork handler on those targets at all? Good point. The updated patch below takes a slightly different approach. I moved we_are_forked to the per-thread struct, and then I moved the setting of it into the *parent* process's fork handlers -- the before-fork handler toggles it to true, then the child spawns off and inherits this setting, and then the parent after-fork handler toggles it back again. (Since it's per-thread, there's no race condition here.) This lets us remove the child after-fork handler entirely, and -- since the parent handlers aren't subject to any restrictions on what they can call -- it works on all platforms regardless of the TLS implementation. > 2) can you explain why are you removing the cleanups from > gomp_free_pool_helper ? They aren't removed, but rather moved from the helper function (which runs in the helper threads) into gomp_free_thread_pool (which runs in the main thread) -- which makes it easier to run the appropriate cleanups even in the case where the helper threads aren't running. (But see below -- we might prefer to drop this part of the patch entirely.) > 3) you can call pthread_atfork many times (once for each pthread > that creates a thread pool), that is undesirable, you want to do that > only if the initial thread creates thread pool Good point. I've moved the pthread_atfork call to initialize_team, which is an __attribute__((constructor)). I am a little uncertain whether this is the best approach, though, because of the comment in team_destructor about wanting to correctly handle dlopen/dlclose. One of pthread_atfork's many (many) limitations is that there's no way to unregister handlers, so if dlopen/dlclose is important (is it?) then we can't call pthread_atfork from initialize_team. If this is a problem, then we could delay the pthread_atfork until e.g. the first thread pool is spawned -- would this be preferred? > 4) the testcase is clearly not portable enough, should be probably limited > to *-*-linux* only, fork etc. will likely not work on many targets. I think it should work on pretty much any target that has fork(); we definitely care about having this functionality on e.g. OS X. I've added some genericish target specifications. > In any case, even with the patch, are you aware that you'll leak megabytes > of thread stacks etc.? Well, err, I wasn't, no :-). Thanks for pointing that out. To me this does clinch the argument that a better approach would be the one I suggested in https://gcc.gnu.org/ml/gcc-patches/2014-02/msg00979.html i.e., of tracking whether any threadprivate variables were present, and if not then simply shutting down the thread pools before forking. But this would be a much more invasive change to gomp (I wouldn't know where to start). In the mean time, the current patch is still worthwhile. The cost is not that bad: I wouldn't think of it as "leaking" so much as "overhead of supporting OMP->fork->OMP". No-one forks a child which forks a child which forks a child etc., so the cost is pretty much bounded in practice. The most common use case is probably using fork() to spawn a set of worker processes, which will end up COW-sharing the thread stacks (which will just end up resting peacefully in swap). And when doing computational work where working set sizes are often in the gigabytes, spending a few megabytes is small change -- esp. compared to the current cost, which involves first wasting hours of programmer time figuring out why things are just locking up, and then (in many cases) having to rewrite the code entirely because there's no fix for this, you just have to redesign you parallelization architecture to either avoid OMP to avoid fork(). However, the thread stack issue does make me wonder if it's worth spending so much effort on cleaning up a few semaphores and mutexes. So I split the patch into two parts. The first enables the basic functionality and passes the test, but it doesn't even try to clean up the thread pool -- it just forgets that it existed and moves on. The second patch goes on top of the first, and adds the best-effort cleanup of synchronization objects and easily free-able heap. So patch #1 alone will do the job, and patch #2 is optional -- applying means we leak a bit yes, but does increase the chance of portability gremlins cropping up. Changelog for patch #1: 2014-10-19 Nathaniel J. Smith * libgomp.h (struct gomp_thread): New member we_are_forked. * team.c (gomp_thread_start): Add we_are_forked to gomp_thread initialization. (gomp_before_fork_callback) (gomp_after_fork_parent_callback): New functions. (initialize_team): Register atfork handlers. (gomp_team_start): Check for fork on entry, and clear thread state if found. * testsuite/libgomp.c/fork-1.c: New test. Changelog for patch #2: 2014-10-19 Nathaniel J. Smith * team.c (gomp_free_pool_helper): Move per-thread cleanup to main thread. (gomp_free_thread): Delegate implementation to... (gomp_free_thread_pool): ...this new function. Like old gomp_free_thread, but does per-thread cleanup, and has option to skip everything that involves interacting with actual threads, which is useful when called after fork. (gomp_team_start): Call gomp_free_thread_pool to release (some) resources after fork. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org --047d7b5d314e20a1000505ce5925 Content-Type: text/plain; charset=US-ASCII; name="gomp-safe-fork-patch-v3-part1.diff" Content-Disposition: attachment; filename="gomp-safe-fork-patch-v3-part1.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i1gz873f0 Content-length: 9232 Y29tbWl0IDgyYWVkMTBjOTg1ODYxN2EzOTY0Y2JiM2QzNGViZGQzNWExODA0 YjQKQXV0aG9yOiBOYXRoYW5pZWwgSi4gU21pdGggPG5qc0Bwb2JveC5jb20+ CkRhdGU6ICAgU3VuIE9jdCAxOSAyMDoyMzo1MiAyMDE0ICswMTAwCgogICAg cGF0Y2ggMQoKZGlmZiAtLWdpdCBhL2xpYmdvbXAvbGliZ29tcC5oIGIvbGli Z29tcC9saWJnb21wLmgKaW5kZXggYTE0ODJjYy4uZWYzYTdmNCAxMDA2NDQK LS0tIGEvbGliZ29tcC9saWJnb21wLmgKKysrIGIvbGliZ29tcC9saWJnb21w LmgKQEAgLTQ0MSw2ICs0NDEsOSBAQCBzdHJ1Y3QgZ29tcF90aHJlYWQKIAog ICAvKiBVc2VyIHB0aHJlYWQgdGhyZWFkIHBvb2wgKi8KICAgc3RydWN0IGdv bXBfdGhyZWFkX3Bvb2wgKnRocmVhZF9wb29sOworCisgIC8qIFRoaXMgaXMg dG8gZW5hYmxlIGJlc3QtZWZmb3J0IGNsZWFudXAgYWZ0ZXIgZm9yay4gICov CisgIGludCB3ZV9hcmVfZm9ya2VkOwogfTsKIAogCmRpZmYgLS1naXQgYS9s aWJnb21wL3RlYW0uYyBiL2xpYmdvbXAvdGVhbS5jCmluZGV4IGU2YTZkOGYu LjE5YjNjYzggMTAwNjQ0Ci0tLSBhL2xpYmdvbXAvdGVhbS5jCisrKyBiL2xp YmdvbXAvdGVhbS5jCkBAIC04Niw2ICs4Niw3IEBAIGdvbXBfdGhyZWFkX3N0 YXJ0ICh2b2lkICp4ZGF0YSkKICAgdGhyLT50cyA9IGRhdGEtPnRzOwogICB0 aHItPnRhc2sgPSBkYXRhLT50YXNrOwogICB0aHItPnBsYWNlID0gZGF0YS0+ cGxhY2U7CisgIHRoci0+d2VfYXJlX2ZvcmtlZCA9IDA7CiAKICAgdGhyLT50 cy50ZWFtLT5vcmRlcmVkX3JlbGVhc2VbdGhyLT50cy50ZWFtX2lkXSA9ICZ0 aHItPnJlbGVhc2U7CiAKQEAgLTI2Niw2ICsyNjcsNjIgQEAgZ29tcF9mcmVl X3RocmVhZCAodm9pZCAqYXJnIF9fYXR0cmlidXRlX18oKHVudXNlZCkpKQog ICAgIH0KIH0KIAorLyogQWNjb3JkaW5nIHRvIFBPU0lYLCBpZiBhIHByb2Nl c3Mgd2hpY2ggdXNlcyB0aHJlYWRzIGNhbGxzIGZvcmsoKSwgdGhlbgorICAg dGhlcmUgYXJlIHZlcnkgZmV3IHRoaW5ncyB0aGF0IHRoZSByZXN1bHRpbmcg Y2hpbGQgcHJvY2VzcyBjYW4gZG8gc2FmZWx5IC0tCisgICBtb3N0bHkganVz dCBleGVjKCkuCisKKyAgIEhvd2V2ZXIsIGluIHByYWN0aWNlLCAoYWxtb3N0 PykgYWxsIFBPU0lYIGltcGxlbWVudGF0aW9ucyBkbyBhbGxvdworICAgYXJi aXRyYXJ5IGNvZGUgdG8gcnVuIGluc2lkZSB0aGUgY2hpbGQsICppZiogdGhl IHBhcmVudCBwcm9jZXNzJ3MgdGhyZWFkcworICAgYXJlIGluIGEgd2VsbC1k ZWZpbmVkIHN0YXRlIHdoZW4gdGhlIGZvcmsgb2NjdXJzLiBBbmQgdGhpcyBj aXJjdW1zdGFuY2UgY2FuCisgICBlYXNpbHkgYXJpc2UgaW4gT01QLXVzaW5n IHByb2dyYW1zLCBlLmcuIHdoZW4gYSBsaWJyYXJ5IGZ1bmN0aW9uIGxpa2Ug REdFTU0KKyAgIHVzZXMgT01QIGludGVybmFsbHksIGFuZCBzb21lIG90aGVy IHVucmVsYXRlZCBwYXJ0IG9mIHRoZSBwcm9ncmFtIGNhbGxzCisgICBmb3Jr KCkgYXQgc29tZSBvdGhlciB0aW1lLCB3aGVuIG5vIE9NUCBzZWN0aW9ucyBh cmUgcnVubmluZy4KKworICAgVGhlcmVmb3JlLCB3ZSBtYWtlIGEgYmVzdCBl ZmZvcnQgYXR0ZW1wdCB0byBoYW5kbGUgdGhlIGNhc2U6CisKKyAgICAgT01Q IHNlY3Rpb24gKGluIHBhcmVudCkgLT4gT01QIHF1aWVzY2UgLT4gZm9yayAt PiBPTVAgc2VjdGlvbiAoaW4gY2hpbGQpCisKKyAgICJCZXN0LWVmZm9ydCIg aGVyZSBtZWFucyB0aGF0OgorICAgLSBZb3VyIHN5c3RlbSBtYXkgb3IgbWF5 IG5vdCBiZSBhYmxlIHRvIGhhbmRsZSB0aGlzIGtpbmQgb2YgY29kZSBhdCBh bGw7CisgICAgIG91ciBnb2FsIGlzIGp1c3QgdG8gbWFrZSBzdXJlIHRoYXQg aWYgaXQgZmFpbHMgaXQncyBub3QgZ29tcCdzIGZhdWx0LgorICAgLSBBbGwg dGhyZWFkcHJpdmF0ZSB2YXJpYWJsZXMgd2lsbCBiZSByZXNldCBpbiB0aGUg Y2hpbGQuIEZvcnR1bmF0ZWx5IHRoaXMKKyAgICAgaXMgZW50aXJlbHkgY29t cGxpYW50IHdpdGggdGhlIHNwZWMsIGFjY29yZGluZyB0byB0aGUgcnVsZSBv ZiBuYXNhbAorICAgICBkZW1vbnMuCisgICAtIFdlIG11c3QgaGF2ZSBtaW5p bWFsIHNwZWVkIGltcGFjdCwgYW5kIG5vIGNvcnJlY3RuZXNzIGltcGFjdCwg b24KKyAgICAgY29tcGxpYW50IHByb2dyYW1zLgorCisgICBPdXIgYXBwcm9h Y2ggaXMgdG8gdXNlIHB0aHJlYWRfYXRmb3JrIHRvIG1ha2UgYSBub3RlIHdo ZW5ldmVyIGEgZm9yaygpIGhhcworICAgb2NjdXJyZWQuICpBbGwqIHdlIGRv IGlzIG1ha2UgYSBub3RlLiBXZSBjYW4ndCBhY3R1YWxseSBzaHV0IGRvd24g b3VyCisgICB0aHJlYWQgcG9vbCBpbiB0aGUgcGFyZW50LCBiZWNhdXNlIHRo aXMgd291bGQgdmlvbGF0ZSB0aGUgT01QIHNwZWMgKGl0CisgICB3b3VsZCBj YXVzZSB0aHJlYWRwcml2YXRlIHZhcmlhYmxlcyB0byBkaXNhcHBlYXIgd2hl bmV2ZXIgdGhlIHByb2Nlc3MgZGlkCisgICBmb3JrK2V4ZWMpLiBBbmQgd2Ug Y2FuJ3QgaW1tZWRpYXRlbHkgc2h1dCBpdCBkb3duIGluIHRoZSBjaGlsZCwg YmVjYXVzZQorICAgdGhhdCByZXF1aXJlcyBjYWxsaW5nIG5vbi1hc3luYy1z aWduYWwtc2FmZSBmdW5jdGlvbnMsIGFuZCB0aHVzIHdvdWxkCisgICB2aW9s YXRlIFBPU0lYIGluIHRoZSBjYXNlIHdoZXJlIHRoZSBob3N0IHByb2dyYW0g aXMganVzdCB0cnlpbmcgdG8KKyAgIGZvcmsrZXhlYy4gSW4gZmFjdCwgd2Ug Y2FuJ3QgZXZlbiBhY2Nlc3Mgb3VyIHdlX2FyZV9mb3JrZWQgZmxhZyBmcm9t IHRoZQorICAgY2hpbGQsIGJlY2F1c2UgaXQncyBzdG9yZWQgaW4gVExTIGFu ZCBhY2Nlc3NpbmcgVExTIG9uIHNvbWUgcGxhdGZvcm1zCisgICByZXF1aXJl cyBhIG5vbi1hc3luYy1zaWduYWwtc2FmZSBjYWxsIHRvIHB0aHJlYWRfZ2V0 c3BlY2lmaWMoKS4gU28gd2hhdCB3ZQorICAgZG8gaXMgc2V0IHRoZSBmbGFn IGluIHRoZSBwYXJlbnQganVzdCBiZWZvcmUgY2FsbGluZyBmb3JrLCBsZXQg dGhlIGNoaWxkCisgICBwcm9jZXNzIGluaGVyaXQgdGhlIGZsYWcsIGFuZCB0 aGVuIHVuc2V0IHRoZSBmbGFnIGluIHRoZSBwYXJlbnQuIFRoaXMgaXMKKyAg IHNhZmUsIGJlY2F1c2UgaW4gdGhlIHBhcmVudCB0aGUgZmxhZyBpcyBvbmx5 IHZpc2libGUgdG8gdGhlIHRocmVhZCBjYWxsaW5nCisgICBmb3JrKCksIGFu ZCBieSB0aGUgdGltZSBmb3JrKCkgaGFzIHJldHVybmVkIHRoZSBmbGFnIGlz IHNldCBiYWNrIHRvIGl0cworICAgY29ycmVjdCB2YWx1ZS4KKyovCitzdGF0 aWMgdm9pZAorZ29tcF9iZWZvcmVfZm9ya19jYWxsYmFjayAodm9pZCkKK3sK KyAgc3RydWN0IGdvbXBfdGhyZWFkICp0aHIgPSBnb21wX3RocmVhZCAoKTsK KyAgLyogVXNlIGluY3JlbWVudC9kZWNyZW1lbnQgdG8gaGFuZGxlIHRoZSBj YXNlIHdoZXJlIHRoZSBjaGlsZCBvZiBvdXIgY2hpbGQKKyAgICAgZW50ZXJz IGFuIE9NUCBzZWN0aW9uLiAqLworICB0aHItPndlX2FyZV9mb3JrZWQrKzsK K30KKworc3RhdGljIHZvaWQKK2dvbXBfYWZ0ZXJfZm9ya19wYXJlbnRfY2Fs bGJhY2sgKHZvaWQpCit7CisgIHN0cnVjdCBnb21wX3RocmVhZCAqdGhyID0g Z29tcF90aHJlYWQgKCk7CisgIHRoci0+d2VfYXJlX2ZvcmtlZC0tOworfQor CiAvKiBMYXVuY2ggYSB0ZWFtLiAgKi8KIAogdm9pZApAQCAtMjg3LDYgKzM0 NCwxNSBAQCBnb21wX3RlYW1fc3RhcnQgKHZvaWQgKCpmbikgKHZvaWQgKiks IHZvaWQgKmRhdGEsIHVuc2lnbmVkIG50aHJlYWRzLAogICBzdHJ1Y3QgZ29t cF90aHJlYWQgKiphZmZpbml0eV90aHIgPSBOVUxMOwogCiAgIHRociA9IGdv bXBfdGhyZWFkICgpOworICBpZiAoX19idWlsdGluX2V4cGVjdCAodGhyLT53 ZV9hcmVfZm9ya2VkLCAwKSkKKyAgICB7CisgICAgICAvKiBUaGVyZSB3YXMg c29tZSBwYXJlbnQgcHJvY2VzcyB3aG8gd2FzIHVzaW5nIE9NUCwgYW5kIHRo ZW4gY2FsbGVkCisJIGZvcmsoKS4gV2UgYXJlIHRoZSBtYWluIHRocmVhZCBv ZiB0aGUgcmVzdWx0aW5nIGNoaWxkIHByb2Nlc3MuIE91cgorCSB0aHJlYWQg c3RydWN0dXJlIGNvbnRhaW5zIHN0YWxlIGRhdGEgcmVmZXJyaW5nIHRvIHRo ZSBwYXJlbnQgdGhyZWFkCisJIHdobyBjYWxsZWQgZm9yaygpLiBSZXNldCBp dCB0byByZWZsZWN0IG91ciBuZXcgbWFpbi10aHJlYWQKKwkgc3RhdHVzLiAo VGhpcyBsZWFrcywgYnV0IHRoYXQncyBiZXR0ZXIgdGhhbiBkZWFkbG9ja2lu Zy4pICAqLworICAgICAgbWVtc2V0ICh0aHIsIDAsIHNpemVvZihzdHJ1Y3Qg Z29tcF90aHJlYWQpKTsKKyAgICB9CiAgIG5lc3RlZCA9IHRoci0+dHMudGVh bSAhPSBOVUxMOwogICBpZiAoX19idWlsdGluX2V4cGVjdCAodGhyLT50aHJl YWRfcG9vbCA9PSBOVUxMLCAwKSkKICAgICB7CkBAIC05MjUsNiArOTkxLDEw IEBAIGluaXRpYWxpemVfdGVhbSAodm9pZCkKIAogICBpZiAocHRocmVhZF9r ZXlfY3JlYXRlICgmZ29tcF90aHJlYWRfZGVzdHJ1Y3RvciwgZ29tcF9mcmVl X3RocmVhZCkgIT0gMCkKICAgICBnb21wX2ZhdGFsICgiY291bGQgbm90IGNy ZWF0ZSB0aHJlYWQgcG9vbCBkZXN0cnVjdG9yLiIpOworCisgIHB0aHJlYWRf YXRmb3JrIChnb21wX2JlZm9yZV9mb3JrX2NhbGxiYWNrLAorCQkgIGdvbXBf YWZ0ZXJfZm9ya19wYXJlbnRfY2FsbGJhY2ssCisJCSAgTlVMTCk7CiB9CiAK IHN0YXRpYyB2b2lkIF9fYXR0cmlidXRlX18oKGRlc3RydWN0b3IpKQpkaWZm IC0tZ2l0IGEvbGliZ29tcC90ZXN0c3VpdGUvbGliZ29tcC5jL2ZvcmstMS5j IGIvbGliZ29tcC90ZXN0c3VpdGUvbGliZ29tcC5jL2ZvcmstMS5jCm5ldyBm aWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAuLjk3YmIzOTEKLS0tIC9k ZXYvbnVsbAorKysgYi9saWJnb21wL3Rlc3RzdWl0ZS9saWJnb21wLmMvZm9y ay0xLmMKQEAgLTAsMCArMSw4MCBAQAorLyogVGhpcyB0ZXN0IHJlcXVpcmVz IGZvcmsoKS4gSXQgb3VnaHQgdG8gd29yayBldmVyeXdoZXJlIHRoYXQgZm9y aygpIGRvZXMsCisgICB0aG91Z2guIFVuZm9ydHVuYXRlbHkgdGhhdCBpcyBu b3Qgc28gZWFzeSB0byB3cml0ZSBkb3duLi4uICovCisvKiB7IGRnLWRvIHJ1 bgorICAgICB7dGFyZ2V0ICotKi1saW51eCogKi0qLWdudSogKi0qLWZyZWVi c2QqICotKi1kYXJ3aW4qICotKi1zb2xhcmlzKiB9IH0gKi8KKy8qIHsgZGct dGltZW91dCAxMCB9ICovCisKKyNpbmNsdWRlIDxvbXAuaD4KKyNpbmNsdWRl IDxzdHJpbmcuaD4KKyNpbmNsdWRlIDxzeXMvd2FpdC5oPgorI2luY2x1ZGUg PHVuaXN0ZC5oPgorI2luY2x1ZGUgPGFzc2VydC5oPgorCitzdGF0aWMgaW50 IHNhd1s0XTsKKworc3RhdGljIHZvaWQKK2NoZWNrX3BhcmFsbGVsIChpbnQg ZXhpdF9vbl9mYWlsdXJlKQoreworICBtZW1zZXQgKHNhdywgMCwgc2l6ZW9m IChzYXcpKTsKKyAgI3ByYWdtYSBvbXAgcGFyYWxsZWwgbnVtX3RocmVhZHMg KDIpCisgIHsKKyAgICBpbnQgaWFtID0gb21wX2dldF90aHJlYWRfbnVtICgp OworICAgIHNhd1tpYW1dID0gMTsKKyAgfQorCisgIC8vIEVuY29kZSBmYWls dXJlIGluIHN0YXR1cyBjb2RlIHRvIHJlcG9ydCB0byBwYXJlbnQgcHJvY2Vz cworICBpZiAoZXhpdF9vbl9mYWlsdXJlKQorICAgIHsKKyAgICAgIGlmIChz YXdbMF0gIT0gMSkKKyAgICAgICAgX2V4aXQoMSk7CisgICAgICBlbHNlIGlm IChzYXdbMV0gIT0gMSkKKyAgICAgICAgX2V4aXQoMik7CisgICAgICBlbHNl IGlmIChzYXdbMl0gIT0gMCkKKyAgICAgICAgX2V4aXQoMyk7CisgICAgICBl bHNlIGlmIChzYXdbM10gIT0gMCkKKyAgICAgICAgX2V4aXQoNCk7CisgICAg ICBlbHNlCisgICAgICAgIF9leGl0KDApOworICB9CisgIC8vIFVzZSByZWd1 bGFyIGFzc2VydGlvbnMKKyAgZWxzZQorICAgIHsKKyAgICAgIGFzc2VydCAo c2F3WzBdID09IDEpOworICAgICAgYXNzZXJ0IChzYXdbMV0gPT0gMSk7Cisg ICAgICBhc3NlcnQgKHNhd1syXSA9PSAwKTsKKyAgICAgIGFzc2VydCAoc2F3 WzNdID09IDApOworICAgIH0KK30KKworaW50CittYWluICgpCit7CisgIC8v IEluaXRpYWxpemUgdGhlIE9NUCB0aHJlYWQgcG9vbCBpbiB0aGUgcGFyZW50 IHByb2Nlc3MKKyAgY2hlY2tfcGFyYWxsZWwgKDApOworICBwaWRfdCBmb3Jr X3BpZCA9IGZvcmsoKTsKKyAgaWYgKGZvcmtfcGlkID09IC0xKQorICAgIHJl dHVybiAxOworICBlbHNlIGlmIChmb3JrX3BpZCA9PSAwKQorICAgIHsKKyAg ICAgIC8vIENhbGwgT01QIGFnYWluIGluIHRoZSBjaGlsZCBwcm9jZXNzIGFu ZCBlbmNvZGUgZmFpbHVyZXMgaW4gZXhpdAorICAgICAgLy8gY29kZS4KKyAg ICAgIGNoZWNrX3BhcmFsbGVsICgxKTsKKyAgICB9CisgIGVsc2UKKyAgICB7 CisgICAgICAvLyBDaGVjayB0aGF0IE9NUCBydW50aW1lIGlzIHN0aWxsIGZ1 bmN0aW9uYWwgaW4gcGFyZW50IHByb2Nlc3MgYWZ0ZXIKKyAgICAgIC8vIHRo ZSBmb3JrLgorICAgICAgY2hlY2tfcGFyYWxsZWwgKDApOworCisgICAgICAv LyBXYWl0IGZvciB0aGUgY2hpbGQgdG8gZmluaXNoIGFuZCBjaGVjayB0aGUg ZXhpdCBjb2RlLgorICAgICAgaW50IGNoaWxkX3N0YXR1cyA9IDA7CisgICAg ICBwaWRfdCB3YWl0X3BpZCA9IHdhaXQoJmNoaWxkX3N0YXR1cyk7CisgICAg ICBhc3NlcnQgKHdhaXRfcGlkID09IGZvcmtfcGlkKTsKKyAgICAgIGFzc2Vy dCAoV0VYSVRTVEFUVVMgKGNoaWxkX3N0YXR1cykgPT0gMCk7CisKKyAgICAg IC8vIENoZWNrIHRoYXQgdGhlIHRlcm1pbmF0aW9uIG9mIHRoZSBjaGlsZCBw cm9jZXNzIGRpZCBub3QgaW1wYWN0CisgICAgICAvLyBPTVAgaW4gcGFyZW50 IHByb2Nlc3MuCisgICAgICBjaGVja19wYXJhbGxlbCAoMCk7CisgICAgfQor ICByZXR1cm4gMDsKK30K --047d7b5d314e20a1000505ce5925 Content-Type: text/plain; charset=US-ASCII; name="gomp-safe-fork-patch-v3-part2.diff" Content-Disposition: attachment; filename="gomp-safe-fork-patch-v3-part2.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i1gz8enm1 Content-length: 5332 Y29tbWl0IDZkOTQ4MTc2YmRjZmRlZDMyOWUzZTRlMzA5Nzg0Mjg0NmJjMzQ5 MWMKQXV0aG9yOiBOYXRoYW5pZWwgSi4gU21pdGggPG5qc0Bwb2JveC5jb20+ CkRhdGU6ICAgU3VuIE9jdCAxOSAyMDoyMzo1NyAyMDE0ICswMTAwCgogICAg cGF0Y2ggMgoKZGlmZiAtLWdpdCBhL2xpYmdvbXAvdGVhbS5jIGIvbGliZ29t cC90ZWFtLmMKaW5kZXggMTliM2NjOC4uNTQyMDg4MSAxMDA2NDQKLS0tIGEv bGliZ29tcC90ZWFtLmMKKysrIGIvbGliZ29tcC90ZWFtLmMKQEAgLTIwNSw0 MiArMjA1LDQxIEBAIHN0YXRpYyBzdHJ1Y3QgZ29tcF90aHJlYWRfcG9vbCAq Z29tcF9uZXdfdGhyZWFkX3Bvb2wgKHZvaWQpCiAgIHJldHVybiBwb29sOwog fQogCisvKiBGcmVlIGEgdGhyZWFkIHBvb2wgYW5kIHJlbGVhc2UgaXRzIHRo cmVhZHMuICovCisKIHN0YXRpYyB2b2lkCiBnb21wX2ZyZWVfcG9vbF9oZWxw ZXIgKHZvaWQgKnRocmVhZF9wb29sKQogewotICBzdHJ1Y3QgZ29tcF90aHJl YWQgKnRociA9IGdvbXBfdGhyZWFkICgpOwogICBzdHJ1Y3QgZ29tcF90aHJl YWRfcG9vbCAqcG9vbAogICAgID0gKHN0cnVjdCBnb21wX3RocmVhZF9wb29s ICopIHRocmVhZF9wb29sOwogICBnb21wX2JhcnJpZXJfd2FpdF9sYXN0ICgm cG9vbC0+dGhyZWFkc19kb2NrKTsKLSAgZ29tcF9zZW1fZGVzdHJveSAoJnRo ci0+cmVsZWFzZSk7Ci0gIHRoci0+dGhyZWFkX3Bvb2wgPSBOVUxMOwotICB0 aHItPnRhc2sgPSBOVUxMOwogICBwdGhyZWFkX2V4aXQgKE5VTEwpOwogfQog Ci0vKiBGcmVlIGEgdGhyZWFkIHBvb2wgYW5kIHJlbGVhc2UgaXRzIHRocmVh ZHMuICovCi0KLXZvaWQKLWdvbXBfZnJlZV90aHJlYWQgKHZvaWQgKmFyZyBf X2F0dHJpYnV0ZV9fKCh1bnVzZWQpKSkKK3N0YXRpYyB2b2lkCitnb21wX2Zy ZWVfdGhyZWFkX3Bvb2wgKGJvb2wgdGhyZWFkc19hcmVfcnVubmluZykKIHsK ICAgc3RydWN0IGdvbXBfdGhyZWFkICp0aHIgPSBnb21wX3RocmVhZCAoKTsK ICAgc3RydWN0IGdvbXBfdGhyZWFkX3Bvb2wgKnBvb2wgPSB0aHItPnRocmVh ZF9wb29sOwogICBpZiAocG9vbCkKICAgICB7CisgICAgICBpbnQgaTsKICAg ICAgIGlmIChwb29sLT50aHJlYWRzX3VzZWQgPiAwKQogCXsKLQkgIGludCBp OwotCSAgZm9yIChpID0gMTsgaSA8IHBvb2wtPnRocmVhZHNfdXNlZDsgaSsr KQorCSAgaWYgKHRocmVhZHNfYXJlX3J1bm5pbmcpCiAJICAgIHsKLQkgICAg ICBzdHJ1Y3QgZ29tcF90aHJlYWQgKm50aHIgPSBwb29sLT50aHJlYWRzW2ld OwotCSAgICAgIG50aHItPmZuID0gZ29tcF9mcmVlX3Bvb2xfaGVscGVyOwot CSAgICAgIG50aHItPmRhdGEgPSBwb29sOworCSAgICAgIGZvciAoaSA9IDE7 IGkgPCBwb29sLT50aHJlYWRzX3VzZWQ7IGkrKykKKwkJeworCQkgIHN0cnVj dCBnb21wX3RocmVhZCAqbnRociA9IHBvb2wtPnRocmVhZHNbaV07CisJCSAg bnRoci0+Zm4gPSBnb21wX2ZyZWVfcG9vbF9oZWxwZXI7CisJCSAgbnRoci0+ ZGF0YSA9IHBvb2w7CisJCX0KKwkgICAgICAvKiBUaGlzIGJhcnJpZXIgdW5k b2NrcyB0aHJlYWRzIGRvY2tlZCBvbiBwb29sLT50aHJlYWRzX2RvY2suICAq LworCSAgICAgIGdvbXBfYmFycmllcl93YWl0ICgmcG9vbC0+dGhyZWFkc19k b2NrKTsKKwkgICAgICAvKiBBbmQgdGhpcyB3YWl0cyB0aWxsIGFsbCB0aHJl YWRzIGhhdmUgY2FsbGVkCisJCSBnb21wX2JhcnJpZXJfd2FpdF9sYXN0IGlu IGdvbXBfZnJlZV9wb29sX2hlbHBlci4gICovCisJICAgICAgZ29tcF9iYXJy aWVyX3dhaXQgKCZwb29sLT50aHJlYWRzX2RvY2spOwogCSAgICB9Ci0JICAv KiBUaGlzIGJhcnJpZXIgdW5kb2NrcyB0aHJlYWRzIGRvY2tlZCBvbiBwb29s LT50aHJlYWRzX2RvY2suICAqLwotCSAgZ29tcF9iYXJyaWVyX3dhaXQgKCZw b29sLT50aHJlYWRzX2RvY2spOwotCSAgLyogQW5kIHRoaXMgd2FpdHMgdGls bCBhbGwgdGhyZWFkcyBoYXZlIGNhbGxlZCBnb21wX2JhcnJpZXJfd2FpdF9s YXN0Ci0JICAgICBpbiBnb21wX2ZyZWVfcG9vbF9oZWxwZXIuICAqLwotCSAg Z29tcF9iYXJyaWVyX3dhaXQgKCZwb29sLT50aHJlYWRzX2RvY2spOwogCSAg LyogTm93IGl0IGlzIHNhZmUgdG8gZGVzdHJveSB0aGUgYmFycmllciBhbmQg ZnJlZSB0aGUgcG9vbC4gICovCiAJICBnb21wX2JhcnJpZXJfZGVzdHJveSAo JnBvb2wtPnRocmVhZHNfZG9jayk7CiAKQEAgLTI1Miw2ICsyNTEsMTQgQEAg Z29tcF9mcmVlX3RocmVhZCAodm9pZCAqYXJnIF9fYXR0cmlidXRlX18oKHVu dXNlZCkpKQogCSAgZ29tcF9tYW5hZ2VkX3RocmVhZHMgLT0gcG9vbC0+dGhy ZWFkc191c2VkIC0gMUw7CiAJICBnb21wX211dGV4X3VubG9jayAoJmdvbXBf bWFuYWdlZF90aHJlYWRzX2xvY2spOwogI2VuZGlmCisJICAvKiBDbGVhbiB1 cCB0aHJlYWQgb2JqZWN0cyAqLworCSAgZm9yIChpID0gMTsgaSA8IHBvb2wt PnRocmVhZHNfdXNlZDsgaSsrKQorCSAgICB7CisJICAgICAgc3RydWN0IGdv bXBfdGhyZWFkICpudGhyID0gcG9vbC0+dGhyZWFkc1tpXTsKKwkgICAgICBn b21wX3NlbV9kZXN0cm95ICgmbnRoci0+cmVsZWFzZSk7CisJICAgICAgbnRo ci0+dGhyZWFkX3Bvb2wgPSBOVUxMOworCSAgICAgIG50aHItPnRhc2sgPSBO VUxMOworCSAgICB9CiAJfQogICAgICAgZnJlZSAocG9vbC0+dGhyZWFkcyk7 CiAgICAgICBpZiAocG9vbC0+bGFzdF90ZWFtKQpAQCAtMjY3LDYgKzI3NCwx NiBAQCBnb21wX2ZyZWVfdGhyZWFkICh2b2lkICphcmcgX19hdHRyaWJ1dGVf XygodW51c2VkKSkpCiAgICAgfQogfQogCisvKiBUaGlzIGlzIGNhbGxlZCB3 aGVuZXZlciBhIHRocmVhZCBleGl0cyB3aGljaCBoYXMgYSBub24tTlVMTCB2 YWx1ZSBmb3IKKyAgIGdvbXBfdGhyZWFkX2Rlc3RydWN0b3IuIEluIHByYWN0 aWNlLCB0aGUgb25seSB0aHJlYWQgZm9yIHdoaWNoIHRoaXMgb2NjdXJzCisg ICBpcyB0aGUgb25lIHdoaWNoIGNyZWF0ZWQgdGhlIHRocmVhZCBwb29sLgor Ki8KK3ZvaWQKK2dvbXBfZnJlZV90aHJlYWQgKHZvaWQgKmFyZyBfX2F0dHJp YnV0ZV9fKCh1bnVzZWQpKSkKK3sKKyAgZ29tcF9mcmVlX3RocmVhZF9wb29s ICh0cnVlKTsKK30KKwogLyogQWNjb3JkaW5nIHRvIFBPU0lYLCBpZiBhIHBy b2Nlc3Mgd2hpY2ggdXNlcyB0aHJlYWRzIGNhbGxzIGZvcmsoKSwgdGhlbgog ICAgdGhlcmUgYXJlIHZlcnkgZmV3IHRoaW5ncyB0aGF0IHRoZSByZXN1bHRp bmcgY2hpbGQgcHJvY2VzcyBjYW4gZG8gc2FmZWx5IC0tCiAgICBtb3N0bHkg anVzdCBleGVjKCkuCkBAIC0zNDksOCArMzY2LDkgQEAgZ29tcF90ZWFtX3N0 YXJ0ICh2b2lkICgqZm4pICh2b2lkICopLCB2b2lkICpkYXRhLCB1bnNpZ25l ZCBudGhyZWFkcywKICAgICAgIC8qIFRoZXJlIHdhcyBzb21lIHBhcmVudCBw cm9jZXNzIHdobyB3YXMgdXNpbmcgT01QLCBhbmQgdGhlbiBjYWxsZWQKIAkg Zm9yaygpLiBXZSBhcmUgdGhlIG1haW4gdGhyZWFkIG9mIHRoZSByZXN1bHRp bmcgY2hpbGQgcHJvY2Vzcy4gT3VyCiAJIHRocmVhZCBzdHJ1Y3R1cmUgY29u dGFpbnMgc3RhbGUgZGF0YSByZWZlcnJpbmcgdG8gdGhlIHBhcmVudCB0aHJl YWQKLQkgd2hvIGNhbGxlZCBmb3JrKCkuIFJlc2V0IGl0IHRvIHJlZmxlY3Qg b3VyIG5ldyBtYWluLXRocmVhZAotCSBzdGF0dXMuIChUaGlzIGxlYWtzLCBi dXQgdGhhdCdzIGJldHRlciB0aGFuIGRlYWRsb2NraW5nLikgICovCisJIHdo byBjYWxsZWQgZm9yaygpLiBEbyBvdXIgYmVzdCB0byBmcmVlIHdoYXQgd2Ug Y2FuLCBhbmQgdGhlbiByZXNldAorCSB0aHIgdG8gcmVmbGVjdCBvdXIgbmV3 IG1haW4tdGhyZWFkIHN0YXR1cy4gICovCisgICAgICBnb21wX2ZyZWVfdGhy ZWFkX3Bvb2wgKDApOwogICAgICAgbWVtc2V0ICh0aHIsIDAsIHNpemVvZihz dHJ1Y3QgZ29tcF90aHJlYWQpKTsKICAgICB9CiAgIG5lc3RlZCA9IHRoci0+ dHMudGVhbSAhPSBOVUxMOwo= --047d7b5d314e20a1000505ce5925--