Re: Pytest usage in DejaGNU?

[Jeff Law <law@redhat.com>]

On 12/14/20 9:02 AM, Martin Liška wrote:
> Hello.
>
> GCOV tests suffer from tests that would cover the intermediate format.
> It's a JSON format and and I'm attaching an example of its output.
>
> I would really like to use Python to make more complex tests:
>
> $ cat test_json.py
> import pytest
> import json
>
> def test_gcov_output():
>     data = json.load(open('gcov.json'))
>     assert len(data['files']) == 1
>     f0 = data['files'][0]
>     assert f0['file'] == 'gcov-lambda.C'
>     assert len(f0['functions']) == 3
>
>     fns = {}
>     for fn in f0['functions']:
>         fns[fn['name']] = fn
>     lines = f0['lines']
>
>     for line in lines:
>         lineno = line['line_number']
>         linefn = line['function_name']
>         assert linefn in fns
>         fn = fns[linefn]
>         assert fn['start_line'] <= lineno and lineno <= fn['end_line']
>
> I see it pretty complicated to do the same in DejaGNU. Mainly due the
> missing
> JSON parser.
>
> Would it be possible to make optional Python tests in our testsuite?
> I can imagine a simple pytest wrapper that will do something like:
>
> +proc pytest-execute { dgargs } {
> +    verbose "dg-pytest-execute: ${dgargs}" 2
> +    set script [lindex $dgargs 0]
> +    verbose "  script: ${script}" 2
> +
> +    spawn -noecho pytest -rA -s --tb=no $script
> +
> +    expect {
> +      -re "FAILED .*" {
> +       fail "pytest $expect_out(0,string)"
> +      }
> +      -re "PASSED .*" {
> +       pass "pytest $expect_out(0,string)"
> +      }
> +    }
> +}
>
> as Pytest can provide a reasonable close output:
>
> ===========================================================================================================================
> short test summary info
> ============================================================================================================================
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
I thought we already approved using python elsewhere (JIT?  Analyzer?)

jeff