Re: PATCH: Bug debuginfod/29472 followup

[Mark Wielaard <mark@klomp.org>]

Hi Frank,

The patch as posted (users/fche/try-pr29472) doesn't apply anymore,
have you tried to rebase it to the current git master branch?

On Thu, Nov 10, 2022 at 12:12:17PM -0500, Frank Ch. Eigler via Elfutils-devel wrote:
> > Are you sure the interface is correct? 
> 
> Hard to be sure, so it's left generalized.  The APIs would be
> unchanged if future search strategies are added (or subtracted);
> they'd affect the choice of acceptable KEY strings.

OK, so the interfaces added are:

New debuginfod-client interface:

/*  Query the urls contained in $DEBUGINFOD_URLS for metadata
    with given query key/value.

   If successful, return 0, otherwise return a posix error code.
   If successful, set *metadata to a malloc'd json array
   with each entry being a json object of metadata for 1 file.
   Caller must free() it later.  metadata MUST be non-NULL.  */
int debuginfod_find_metadata (debuginfod_client *client,
                              const char *key, const char* value,
                              char** metadata);

Might need an explanation of which characters are allowed in key and
value. In particular is '=' allowed? Any other special chars? What
encoding is used (utf8)?

New server query: metadata?key=KEY&value=VALUE

Same as above, plus maybe explictly descibe the URL encoding?

New debuginfod-find command arguments:

debuginfod-find [OPTION]... metadata KEY VALUE 

Where:

KEY	VALUE	DESCRIPTION

file	path	match exact path, including in archives
glob	pattern	sqlite glob match pattern, including in archives

And as output a json array with objects that might contain the
following:

NAME	TYPE	DESCRIPTION

buildid	string	hexadecimal buildid associated with the file
type	string	one of debuginfo or executable or source
file	string	matched file name, outside or inside the archive

Having the associated buildid with every result object might be a lot,
especially when listing source files, is there a representation where
the buildid is only listed one for a group of results?

Is file here the full path?

> We know we want glob patterns over executable file names.  I've seen
> cases where an exact match query produces a different sqlite query
> plan from the glob one, but not sure how much performance difference
> that implies.  Searching for source files by glob/match is removed
> from today's version because it doesn't run fast enough (without a new
> large index).

Aha, that wasn't really clear from the above description.
Maybe the key name "glob" is a but generic then?
 
> > Is the sqlite "glob" pattern standardized? 
> 
> Yes.

Could you add a reference or description?

I believe globs include the ? (single char) and * (zero or many)
wildcards.  Does it include (negative!) [..] ranges?

> > Why does debuginfod-client.c use json-c? Can't the server sent the
> > json object as a normal char string? Why does the string from the
> > server need to be interpreted as a json object and then turned into a
> > string again?
> 
> Use of the library allows robust processing (checking & merging) of
> incoming json data from multiple upstream servers.  Luckily, json-c is
> a small & self-contained library.

aha, I see, the metadata query is different from other queries. It
"merges" the replies from all servers instead of picking the first one
that gives an answer.

Does it really merge the results? It looks like it just adds all
elements to the array whether or not they are there.

Should the result also contain the server URL from which the entry
came?

Cheers,

Mark