From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 5FFF4386184F; Wed, 31 Mar 2021 16:03:26 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5FFF4386184F From: "fche at redhat dot com" To: elfutils-devel@sourceware.org Subject: [Bug debuginfod/27673] [debuginfod] Handle source requests for same buildid more efficiently Date: Wed, 31 Mar 2021 16:03:26 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: elfutils X-Bugzilla-Component: debuginfod X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: enhancement X-Bugzilla-Who: fche at redhat dot com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2021 16:03:26 -0000 https://sourceware.org/bugzilla/show_bug.cgi?id=3D27673 --- Comment #4 from Frank Ch. Eigler --- > The time it takes for the client to see the response of the server to the > request consist of: > - time for request to travel to server (latency) > - time for server to react to request > - time for answer to travel back to client (latency again) >=20 > I've looked at the documentation of the option fdcache-prefetch, and AFAIU > this improves "time for server to react to request". That's correct. > debuginfod: > - when receiving a source request and ENOENT, send as reply the list of > available files for the buildid >=20 > client: > - when receiving a list of available files for a buildid, store it and > use it to reply to source requests related to the buildid. That is, > if the file is not in the list, reply with -2. Otherwise, send a > request to debuginfod, and expect it to succeed. Interesting. A more first-class solution could be a new webapi to enumerate source files: a "/buildid/HEXCODE/sourcelist" query that returns a structured piece of data. This can be computed by debuginfod fairly rapidly. The client could cache that and use it to drive a negative-cache hit on a subsequent source query. > Proposal b: >=20 > debuginfod: > - when receiving a source request, send a package with the sources > for that buildid to the client. >=20 > client: > - when receiving a package with the sources for a buildid, store them > and use them to reply to source requests related to the buildid. So this could be a "/buildid/HEXCODE/sources" query that returns a tarball = of all sources related to a given buildid. This is challenging in principle because sources may not live in a single upstream package we can just relay verbatim. debuginfod may have to assemble a new one on the fly, kind of li= ke gitweb's 'archive' buttons ... which are disabled by default for performance reasons. Worth a consideration I guess, but risky to deploy. By the way, a client also has another option: querying in parallel. If it knows all interesting file names, it can fork N threads and make N concurre= nt requests to debuginfod. The poor server may get larger bursts of load but total elapsed time should be better. And another option: if connection establishment / teardown are a bit part of the problem - and they can be with TLS - we could teach the client code to activate as much curl level http-keepalive as possible. So as long as a si= ngle debuginfod_client object were reused, it could avoid the TCP/TLS handshakes= .=20 (It MIGHT already be doing that.) --=20 You are receiving this mail because: You are on the CC list for the bug.=