From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 8F7A33858D1E for ; Sat, 3 Sep 2022 01:23:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8F7A33858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1662168224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mzBQWd3z28Q4YcWcWhlKbwYqdFFFQ6Wn1C9YpBIOeGs=; b=F9QMht3rHQEadFqE/3BDK7MlughJ2u5w99nt5NjlcJQUQH8jNGgpf1gWB66hIVKlmURjsw zv0bTvKf6SvGzjhwn+npMl5VynAswI2ccOcVvCf8eLgYnsbzFq0+oCibKi4tUQ3RLZ7Gtu D3/kFlred2+JrwOd7egZuzqCBnqMgaU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-440-mVtRbsOEPDusaeNEnEMWhg-1; Fri, 02 Sep 2022 21:23:43 -0400 X-MC-Unique: mVtRbsOEPDusaeNEnEMWhg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CE9BD801231 for ; Sat, 3 Sep 2022 01:23:42 +0000 (UTC) Received: from localhost.localdomain (unknown [10.22.32.33]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B39C1121314; Sat, 3 Sep 2022 01:23:42 +0000 (UTC) From: Aaron Merey To: elfutils-devel@sourceware.org Cc: Aaron Merey Subject: [PATCH] debuginfod: Use auto-sized connection pool when -C is not given with arg Date: Fri, 2 Sep 2022 21:23:32 -0400 Message-Id: <20220903012332.1456366-1-amerey@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Since commit 4b42d9ad, libmicrohttpd's epoll event loop is used when available in which case we must disable its setting for spawning a thread per request. This contradicts the debuginfod doc's description of '-C', which indicates that if this command line option is not given then the thread pool size is unbounded. Fix this by using an auto-sized thread pool when '-C' is not given, just as we do when it's given with no argument. Update the doc's description of '-C'. Also use a fixed-size pool even if epoll is not supported. The unbounded pool config cannot be considered entirely reliable as it appears to cause random fails in the run-debuginfod-webapi-concurrency test. Signed-off-by: Aaron Merey --- debuginfod/ChangeLog | 7 +++++++ debuginfod/debuginfod.cxx | 22 ++++++++++------------ doc/ChangeLog | 4 ++++ doc/debuginfod.8 | 10 ++++------ 4 files changed, 25 insertions(+), 18 deletions(-) diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog index da58e9a8..c692a389 100644 --- a/debuginfod/ChangeLog +++ b/debuginfod/ChangeLog @@ -1,3 +1,10 @@ +2022-09-02 Aaron Merey + + * debuginfod.cxx (parse_opt): If '-C' is given with no arg, do not + update connection_pool since it will be done at a later point. + (main): Use auto-sized connection_pool if '-C' isn't given with an + arg. Do not use MHD_USE_THREAD_PER_CONNECTION. + 2022-08-17 Martin Liska * debuginfod.cxx (handle_buildid): Update HTTP statistics only if diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index 3e2dd9ef..8680c048 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -575,8 +575,6 @@ parse_opt (int key, char *arg, if (connection_pool < 2) argp_failure(state, 1, EINVAL, "-C NUM minimum 2"); } - else // arg not given - connection_pool = std::thread::hardware_concurrency() * 2 ?: 2; break; case 'I': // NB: no problem with unconditional free here - an earlier failed regcomp would exit program @@ -3937,6 +3935,11 @@ main (int argc, char *argv[]) } } + /* If '-C' wasn't given or was given with no arg, pick a reasonable default + for the number of worker threads. */ + if (connection_pool == 0) + connection_pool = std::thread::hardware_concurrency() * 2 ?: 2; + /* Note that MHD_USE_EPOLL and MHD_USE_THREAD_PER_CONNECTION don't work together. */ unsigned int use_epoll = 0; @@ -3944,12 +3947,11 @@ main (int argc, char *argv[]) use_epoll = MHD_USE_EPOLL; #endif - unsigned int mhd_flags = ((connection_pool || use_epoll - ? 0 : MHD_USE_THREAD_PER_CONNECTION) + unsigned int mhd_flags = ( #if MHD_VERSION >= 0x00095300 - | MHD_USE_INTERNAL_POLLING_THREAD + MHD_USE_INTERNAL_POLLING_THREAD #else - | MHD_USE_SELECT_INTERNALLY + MHD_USE_SELECT_INTERNALLY #endif | MHD_USE_DUAL_STACK | use_epoll @@ -3964,12 +3966,8 @@ main (int argc, char *argv[]) handler_cb, NULL, /* handler callback */ MHD_OPTION_EXTERNAL_LOGGER, error_cb, NULL, - (connection_pool - ? MHD_OPTION_THREAD_POOL_SIZE - : MHD_OPTION_END), - (connection_pool - ? (int)connection_pool - : MHD_OPTION_END), + MHD_OPTION_THREAD_POOL_SIZE, + (int)connection_pool, MHD_OPTION_END); MHD_Daemon *d4 = NULL; diff --git a/doc/ChangeLog b/doc/ChangeLog index ceec2467..c446e99e 100644 --- a/doc/ChangeLog +++ b/doc/ChangeLog @@ -1,3 +1,7 @@ +2022-09-02 Aaron Merey + + * debuginfod.8 (-C): Update description. + 2022-06-03 Michael Trapp * debuginfod.8 (--disable-source-scan): Document. diff --git a/doc/debuginfod.8 b/doc/debuginfod.8 index 50fce7f5..7c1dc3dd 100644 --- a/doc/debuginfod.8 +++ b/doc/debuginfod.8 @@ -212,15 +212,13 @@ following table summarizes the interpretaton of this option and its optional NUM parameter. .TS l l. -no option clone new thread for every request, no fixed pool -\-C use a fixed thread pool sized automatically +no option, \-C use a fixed thread pool sized automatically \-C=NUM use a fixed thread pool sized NUM, minimum 2 .TE -The first mode is useful for friendly bursty traffic. The second mode -is a simple and safe configuration based on the number of processors. -The third mode is suitable for tuned load-limiting configurations -facing unruly traffic. +The first mode is a simple and safe configuration based on the number +of processors. The second mode is suitable for tuned load-limiting +configurations facing unruly traffic. .TP .B "\-L" -- 2.37.1