From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 2E75D3856DE4 for ; Fri, 22 Apr 2022 22:56:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2E75D3856DE4 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-502-zE7wjg9FMimdkNMpNcYrzg-1; Fri, 22 Apr 2022 18:56:46 -0400 X-MC-Unique: zE7wjg9FMimdkNMpNcYrzg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 88965185A79C; Fri, 22 Apr 2022 22:56:46 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.22.34.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4424D568655; Fri, 22 Apr 2022 22:56:46 +0000 (UTC) From: Aaron Merey To: mark@klomp.org Cc: elfutils-devel@sourceware.org, Aaron Merey Subject: Re: [PATCH] debuginfod: Use the debuginfod-size response header Date: Fri, 22 Apr 2022 18:56:41 -0400 Message-Id: <20220422225641.317366-1-amerey@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Apr 2022 22:56:50 -0000 I've updated the patch below with the changes Mark recommended. A couple X-DEBUGINFOD-SIZE tests were added in another patch I recently posted [1] that also fixes a bug when computing this header's value for an archived file. Aaron [1] https://sourceware.org/pipermail/elfutils-devel/2022q2/004936.html >From b56f1568b832fe1c23ffb711aa0486fbd2c5067f Mon Sep 17 00:00:00 2001 From: Aaron Merey Date: Tue, 11 Jan 2022 22:07:55 -0500 debuginfod: Use the debuginfod-size response header In some cases the content-length header may not be available in order to pass to a progressfn. If content-length isn't available then attempt to get the size of the download from the debuginfod-size header instead. It should be mentioned that if a compressed file (ex. gzip) is being transferred, the actual transfer length will be less than debuginfod-size. In this case debuginfod-size is a best-guess upper bound on the size of the transfer. Signed-off-by: Aaron Merey --- debuginfod/debuginfod-client.c | 83 ++++++++++++++++++---------------- 1 file changed, 43 insertions(+), 40 deletions(-) diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c index 58ef6442..de7ea1af 100644 --- a/debuginfod/debuginfod-client.c +++ b/debuginfod/debuginfod-client.c @@ -1130,11 +1130,45 @@ debuginfod_query_server (debuginfod_client *c, goto out2; } + long dl_size = 0; + if (target_handle && (c->progressfn || maxsize > 0)) + { + /* Get size of file being downloaded. NB: If going through + deflate-compressing proxies, this number is likely to be + unavailable, so -1 may show. */ + CURLcode curl_res; +#ifdef CURLINFO_CONTENT_LENGTH_DOWNLOAD_T + curl_off_t cl; + curl_res = curl_easy_getinfo(target_handle, + CURLINFO_CONTENT_LENGTH_DOWNLOAD_T, + &cl); + if (curl_res == CURLE_OK && cl >= 0) + dl_size = (cl > LONG_MAX ? LONG_MAX : (long)cl); +#else + double cl; + curl_res = curl_easy_getinfo(target_handle, + CURLINFO_CONTENT_LENGTH_DOWNLOAD, + &cl); + if (curl_res == CURLE_OK) + dl_size = (cl >= (double)(LONG_MAX+1UL) ? LONG_MAX : (long)cl); +#endif + /* If Content-Length is -1, try to get the size from + X-Debuginfod-Size */ + if (dl_size == -1 && c->winning_headers != NULL) + { + long xdl; + char *hdr = strcasestr(c->winning_headers, "x-debuginfod-size"); + + if (hdr != NULL + && sscanf(hdr, "x-debuginfod-size: %ld", &xdl) == 1) + dl_size = xdl; + } + } + if (c->progressfn) /* inform/check progress callback */ { loops ++; - long pa = loops; /* default params for progress callback */ - long pb = 0; /* transfer_timeout tempting, but loops != elapsed-time */ + long pa = loops; /* default param for progress callback */ if (target_handle) /* we've committed to a server; report its download progress */ { CURLcode curl_res; @@ -1154,50 +1188,19 @@ debuginfod_query_server (debuginfod_client *c, pa = (dl >= (double)(LONG_MAX+1UL) ? LONG_MAX : (long)dl); #endif - /* NB: If going through deflate-compressing proxies, this - number is likely to be unavailable, so -1 may show. */ -#ifdef CURLINFO_CONTENT_LENGTH_DOWNLOAD_T - curl_off_t cl; - curl_res = curl_easy_getinfo(target_handle, - CURLINFO_CONTENT_LENGTH_DOWNLOAD_T, - &cl); - if (curl_res == 0 && cl >= 0) - pb = (cl > LONG_MAX ? LONG_MAX : (long)cl); -#else - double cl; - curl_res = curl_easy_getinfo(target_handle, - CURLINFO_CONTENT_LENGTH_DOWNLOAD, - &cl); - if (curl_res == 0) - pb = (cl >= (double)(LONG_MAX+1UL) ? LONG_MAX : (long)cl); -#endif } - if ((*c->progressfn) (c, pa, pb)) + if ((*c->progressfn) (c, pa, dl_size)) break; } + /* Check to see if we are downloading something which exceeds maxsize, if set.*/ - if (maxsize > 0 && target_handle) + if (target_handle && dl_size > maxsize && maxsize > 0) { - long dl_size = 0; -#ifdef CURLINFO_SIZE_DOWNLOAD_T - curl_off_t download_size_t; - if (curl_easy_getinfo(target_handle, CURLINFO_CONTENT_LENGTH_DOWNLOAD_T, - &download_size_t) == CURLE_OK) - dl_size = download_size_t >= (double)(LONG_MAX+1UL) ? LONG_MAX : (long)download_size_t; -#else - double download_size; - if (curl_easy_getinfo(target_handle, CURLINFO_CONTENT_LENGTH_DOWNLOAD, - &download_size) == CURLE_OK) - dl_size = download_size >= (double)(LONG_MAX+1UL) ? LONG_MAX : (long)download_size; -#endif - if (dl_size > maxsize) - { - if (vfd >=0) - dprintf(vfd, "Content-Length too large.\n"); - rc = -EFBIG; - goto out2; - } + if (vfd >=0) + dprintf(vfd, "Content-Length too large.\n"); + rc = -EFBIG; + goto out2; } } while (still_running); -- 2.35.1