* Pytest usage in DejaGNU?
@ 2020-12-14 16:02 Martin Liška
2020-12-14 20:30 ` Jeff Law
2020-12-14 21:21 ` Joseph Myers
0 siblings, 2 replies; 4+ messages in thread
From: Martin Liška @ 2020-12-14 16:02 UTC (permalink / raw)
To: GCC Development
[-- Attachment #1: Type: text/plain, Size: 1868 bytes --]
Hello.
GCOV tests suffer from tests that would cover the intermediate format.
It's a JSON format and and I'm attaching an example of its output.
I would really like to use Python to make more complex tests:
$ cat test_json.py
import pytest
import json
def test_gcov_output():
data = json.load(open('gcov.json'))
assert len(data['files']) == 1
f0 = data['files'][0]
assert f0['file'] == 'gcov-lambda.C'
assert len(f0['functions']) == 3
fns = {}
for fn in f0['functions']:
fns[fn['name']] = fn
lines = f0['lines']
for line in lines:
lineno = line['line_number']
linefn = line['function_name']
assert linefn in fns
fn = fns[linefn]
assert fn['start_line'] <= lineno and lineno <= fn['end_line']
I see it pretty complicated to do the same in DejaGNU. Mainly due the missing
JSON parser.
Would it be possible to make optional Python tests in our testsuite?
I can imagine a simple pytest wrapper that will do something like:
+proc pytest-execute { dgargs } {
+ verbose "dg-pytest-execute: ${dgargs}" 2
+ set script [lindex $dgargs 0]
+ verbose " script: ${script}" 2
+
+ spawn -noecho pytest -rA -s --tb=no $script
+
+ expect {
+ -re "FAILED .*" {
+ fail "pytest $expect_out(0,string)"
+ }
+ -re "PASSED .*" {
+ pass "pytest $expect_out(0,string)"
+ }
+ }
+}
as Pytest can provide a reasonable close output:
=========================================================================================================================== short test summary info ============================================================================================================================
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output
Thoughts?
Martin
[-- Attachment #2: gcov.json --]
[-- Type: application/json, Size: 4817 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Pytest usage in DejaGNU?
2020-12-14 16:02 Pytest usage in DejaGNU? Martin Liška
@ 2020-12-14 20:30 ` Jeff Law
2020-12-14 21:21 ` Joseph Myers
1 sibling, 0 replies; 4+ messages in thread
From: Jeff Law @ 2020-12-14 20:30 UTC (permalink / raw)
To: Martin Liška, GCC Development
On 12/14/20 9:02 AM, Martin Liška wrote:
> Hello.
>
> GCOV tests suffer from tests that would cover the intermediate format.
> It's a JSON format and and I'm attaching an example of its output.
>
> I would really like to use Python to make more complex tests:
>
> $ cat test_json.py
> import pytest
> import json
>
> def test_gcov_output():
> data = json.load(open('gcov.json'))
> assert len(data['files']) == 1
> f0 = data['files'][0]
> assert f0['file'] == 'gcov-lambda.C'
> assert len(f0['functions']) == 3
>
> fns = {}
> for fn in f0['functions']:
> fns[fn['name']] = fn
> lines = f0['lines']
>
> for line in lines:
> lineno = line['line_number']
> linefn = line['function_name']
> assert linefn in fns
> fn = fns[linefn]
> assert fn['start_line'] <= lineno and lineno <= fn['end_line']
>
> I see it pretty complicated to do the same in DejaGNU. Mainly due the
> missing
> JSON parser.
>
> Would it be possible to make optional Python tests in our testsuite?
> I can imagine a simple pytest wrapper that will do something like:
>
> +proc pytest-execute { dgargs } {
> + verbose "dg-pytest-execute: ${dgargs}" 2
> + set script [lindex $dgargs 0]
> + verbose " script: ${script}" 2
> +
> + spawn -noecho pytest -rA -s --tb=no $script
> +
> + expect {
> + -re "FAILED .*" {
> + fail "pytest $expect_out(0,string)"
> + }
> + -re "PASSED .*" {
> + pass "pytest $expect_out(0,string)"
> + }
> + }
> +}
>
> as Pytest can provide a reasonable close output:
>
> ===========================================================================================================================
> short test summary info
> ============================================================================================================================
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
I thought we already approved using python elsewhere (JIT? Analyzer?)
jeff
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Pytest usage in DejaGNU?
2020-12-14 16:02 Pytest usage in DejaGNU? Martin Liška
2020-12-14 20:30 ` Jeff Law
@ 2020-12-14 21:21 ` Joseph Myers
2020-12-14 21:53 ` Matthias Klose
1 sibling, 1 reply; 4+ messages in thread
From: Joseph Myers @ 2020-12-14 21:21 UTC (permalink / raw)
To: Martin Liška; +Cc: GCC Development
On Mon, 14 Dec 2020, Martin Liška wrote:
> + spawn -noecho pytest -rA -s --tb=no $script
"pytest" might not be the right command everywhere. If I install
python3-pytest on Ubuntu 20.04 I only get /usr/bin/pytest-3 not
/usr/bin/pytest.
There is a clear advantage to any usage of Python only depending on the
standard library and not other Python packages.
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
> PASSED test_json.py::test_gcov_output
The names after "PASS: " or "FAIL: " in DejaGnu output should be unique
(and not depend on whether the test in question passes or fails, or on
variables such as the build directory); there should not be three tests
with the same name.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Pytest usage in DejaGNU?
2020-12-14 21:21 ` Joseph Myers
@ 2020-12-14 21:53 ` Matthias Klose
0 siblings, 0 replies; 4+ messages in thread
From: Matthias Klose @ 2020-12-14 21:53 UTC (permalink / raw)
To: Joseph Myers, Martin Liška; +Cc: GCC Development
On 12/14/20 10:21 PM, Joseph Myers wrote:
> On Mon, 14 Dec 2020, Martin Liška wrote:
>
>> + spawn -noecho pytest -rA -s --tb=no $script
>
> "pytest" might not be the right command everywhere. If I install
> python3-pytest on Ubuntu 20.04 I only get /usr/bin/pytest-3 not
> /usr/bin/pytest.
<python> -m pytest should work everywhere.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-12-14 21:53 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-14 16:02 Pytest usage in DejaGNU? Martin Liška
2020-12-14 20:30 ` Jeff Law
2020-12-14 21:21 ` Joseph Myers
2020-12-14 21:53 ` Matthias Klose
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).