From: David Malcolm <dmalcolm@redhat.com>
To: gcc-patches@gcc.gnu.org
Cc: David Malcolm <dmalcolm@redhat.com>
Subject: [PATCH 16/22] Add checkers/coverity.py
Date: Fri, 04 Aug 2017 21:36:00 -0000 [thread overview]
Message-ID: <1501884293-9047-17-git-send-email-dmalcolm@redhat.com> (raw)
In-Reply-To: <1501884293-9047-1-git-send-email-dmalcolm@redhat.com>
This patch is an example of supporting a proprietary 3rd-party tool:
a harness for invoking the Coverity checker.
It uses the '--json-output-v2' option to cov-format-errors, and then uses
firehose.parsers.coverity.parse_json_v2 to parse the generated Coverity
JSON format, turning it into firehose JSON.
This isn't a great example of use of either the checker infrastructure, or
of Coverity.
As far as I can tell, Coverity is designed to be run on a
number of source files at once, performing a relatively cheap data-gathering
phase per-source-file, and then performing a more expensive LTO-style
analysis that can follow dataflow between source files, thus obtaining
much more accurate results that a purely one-file-at-a-time checker can.
In contrast, the checker machinery in this patch kit is designed to run
one file at a time. The harness code in this patch attempts to "square
this circle", but it's not a good fit; it can detect problems that
are within one source file, but prevents the checker from finding
the more interesting problems that it's normally capable of.
checkers/ChangeLog:
* coverity.py: New file.
---
checkers/coverity.py | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)
create mode 100644 checkers/coverity.py
diff --git a/checkers/coverity.py b/checkers/coverity.py
new file mode 100644
index 0000000..533a6ae
--- /dev/null
+++ b/checkers/coverity.py
@@ -0,0 +1,141 @@
+#!/usr/bin/env python
+# Copyright 2017 David Malcolm <dmalcolm@redhat.com>
+# Copyright 2017 Red Hat, Inc.
+#
+# This is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see
+# <http://www.gnu.org/licenses/>.
+
+# Coverity is a trademark of Synopsys, Inc. in the U.S. and/or other
+# countries.
+
+import os
+import sys
+import tempfile
+import unittest
+
+from gccinvocation import GccInvocation
+
+from checker import Checker, Context, CheckerTests, make_file, make_stats, \
+ tool_main
+
+from firehose.model import Analysis, Generator, Metadata, Failure, \
+ Location, File, Message, Issue, Trace
+from firehose.parsers.coverity import parse_json_v2
+
+os.environ['PATH'] = '/opt/coverity/bin:' + os.environ['PATH']
+
+class InvokeCoverity(Checker):
+ """
+ Checker subclass that invokes Coverity
+ """
+ name = 'coverity'
+
+ def __init__(self, ctxt, verbose=False):
+ Checker.__init__(self, ctxt)
+ self.verbose = verbose
+
+ def raw_invoke(self, gccinv, sourcefile):
+ # tempfile.TemporaryDirectory is only available from Python 3.2 onwards,
+ # so handle tempdir cleanup "by hand"
+ try:
+ tempdir_name = tempfile.mkdtemp()
+
+ json_name = os.path.join(tempdir_name, 'output.json')
+ build_args = ['cov-build', '--dir', tempdir_name, 'gcc'] + gccinv.argv[1:]
+ if self.verbose:
+ print(build_args)
+ build_result = self.run_subprocess(sourcefile, build_args)
+ if self.verbose:
+ print(build_result)
+
+ analyze_args = ['cov-analyze', '--dir', tempdir_name,
+ '--wait-for-license']
+ analyze_result = self.run_subprocess(sourcefile, analyze_args)
+ if self.verbose:
+ print(analyze_result)
+
+ format_args = ['cov-format-errors', '--dir', tempdir_name,
+ '--json-output-v2', json_name]
+ format_result = self.run_subprocess(sourcefile, format_args)
+ if self.verbose:
+ print(format_result)
+
+ # Parse the output, returning an Analysis instance
+ analysis = parse_json_v2(json_name)
+ if self.verbose:
+ print(analysis)
+ return analysis
+
+ # FIXME: timing metadata?
+
+ finally:
+ pass # FIXME: cleanup tempdir
+
+class CoverityTests(CheckerTests):
+ def make_tool(self):
+ return self.make_tool_from_class(InvokeCoverity)
+
+ def verify_basic_metadata(self, analysis, sourcefile):
+ # Verify basic metadata:
+ self.assert_metadata(analysis, 'coverity', sourcefile)
+
+ def test_file_not_found(self):
+ analysis = self.invoke('does-not-exist.c')
+ self.assertEqual(len(analysis.results), 0)
+
+ def test_timeout(self):
+ sourcefile = 'test-sources/harmless.c'
+ tool = self.make_tool()
+ tool.timeout = 0
+ gccinv = GccInvocation(['gcc', sourcefile])
+ analysis = tool.checked_invoke(gccinv, sourcefile)
+ self.assert_metadata(analysis, 'coverity', sourcefile)
+ self.assertEqual(len(analysis.results), 1)
+ r0 = analysis.results[0]
+ self.assertIsInstance(r0, Failure)
+ self.assertEqual(r0.failureid, 'timeout')
+
+ def test_out_of_bounds(self):
+ analysis = self.invoke('test-sources/out-of-bounds.c')
+ if 0:
+ print(analysis)
+ self.assertEqual(len(analysis.results), 2)
+
+ r0 = analysis.results[0]
+ self.assertIsInstance(r0, Issue)
+ self.assertEqual(r0.testid, 'OVERRUN')
+ self.assertEqual(r0.location.point.line, 5)
+ self.assertEqual(r0.message.text,
+ 'Overrunning array "arr" of 10 4-byte elements at'
+ ' element index 15 (byte offset 60) using index "15".')
+ self.assertIsInstance(r0.trace, Trace)
+ self.assertEqual(len(r0.trace.states), 1)
+
+ r1 = analysis.results[1]
+ self.assertIsInstance(r1, Issue)
+ self.assertEqual(r1.testid, 'UNINIT')
+ self.assertEqual(r1.location.point.line, 5)
+ self.assertEqual(r1.message.text,
+ 'Using uninitialized value "arr[15]".')
+ self.assertIsInstance(r1.trace, Trace)
+ self.assertEqual(len(r1.trace.states), 2)
+ self.assertEqual(r1.trace.states[0].location.point.line, 3)
+ self.assertEqual(r1.trace.states[0].notes.text,
+ 'Declaring variable "arr" without initializer.')
+ self.assertEqual(r1.trace.states[1].location.point.line, 5)
+ self.assertEqual(r1.trace.states[1].notes.text,
+ 'Using uninitialized value "arr[15]".')
+
+if __name__ == '__main__':
+ sys.exit(tool_main(sys.argv, InvokeCoverity))
--
1.8.5.3
next prev parent reply other threads:[~2017-08-04 21:36 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-04 21:30 [PATCH 00/22] RFC: integrated 3rd-party static analysis support David Malcolm
2017-08-04 21:30 ` [PATCH 03/22] Add JSON implementation David Malcolm
2017-09-01 17:56 ` Jeff Law
2017-08-04 21:30 ` [PATCH 01/22] Expose assert_loceq outside of input.c; add ASSERT_LOCEQ David Malcolm
2017-09-01 17:49 ` Jeff Law
2017-08-04 21:30 ` [PATCH 02/22] libcpp: add linemap_position_for_file_line_and_column David Malcolm
2017-09-01 17:50 ` Jeff Law
2017-08-04 21:36 ` [PATCH 09/22] Add selftest::read_file (..., FILE *, ...) David Malcolm
2017-08-04 21:36 ` [PATCH 22/22] Add contrib/get-static-analysis.py David Malcolm
2017-08-04 21:36 ` David Malcolm [this message]
2017-08-04 21:36 ` [PATCH 19/22] Add checkers/ianal.py David Malcolm
2017-08-04 21:36 ` [PATCH 08/22] Add GNU_BUILD_ATTRIBUTE_STATIC_ANALYSIS to annobin.h David Malcolm
2017-08-04 21:36 ` [PATCH 10/22] Add checkers.h/cc David Malcolm
2017-08-04 21:36 ` [PATCH 07/22] Add minimal version of Nick Clifton's annobin code David Malcolm
2017-09-01 18:17 ` Jeff Law
2017-08-04 21:36 ` [PATCH 17/22] Add checkers/cppcheck.py David Malcolm
2017-08-04 21:37 ` [PATCH 15/22] Add checkers/clang_analyzer.py David Malcolm
2017-08-04 21:37 ` [PATCH 11/22] Add checkers/test-sources David Malcolm
2017-08-04 21:37 ` [PATCH 04/22] Add firehose.h/cc David Malcolm
2017-08-04 21:38 ` [PATCH 12/22] Add -Wrun-analyzers= to common.opt, toplev.c, and invoke.texi David Malcolm
2017-08-04 21:38 ` [PATCH 06/22] Makefile.in: hack in -lpthread David Malcolm
2017-09-01 18:13 ` Jeff Law
2017-08-04 21:38 ` [PATCH 13/22] Add checkers/checker.py David Malcolm
2017-08-04 21:38 ` [PATCH 21/22] Add checkers/Makefile David Malcolm
2017-08-04 21:38 ` [PATCH 14/22] Add checkers/always_fails.py David Malcolm
2017-08-04 21:39 ` [PATCH 20/22] Add checkers/splint.py David Malcolm
2017-08-04 21:39 ` [PATCH 05/22] diagnostic.c/h: add support for external tools David Malcolm
2017-09-01 18:18 ` Jeff Law
2017-08-04 21:39 ` [PATCH 18/22] Add checkers/flawfinder.py David Malcolm
2017-08-05 1:00 ` [PATCH 00/22] RFC: integrated 3rd-party static analysis support Eric Gallager
2017-08-08 0:23 ` David Malcolm
2017-08-06 21:21 ` Martin Sebor
2017-08-08 17:57 ` Richard Sandiford
2017-09-01 17:46 ` Jeff Law
2017-09-02 2:46 ` Trevor Saunders
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1501884293-9047-17-git-send-email-dmalcolm@redhat.com \
--to=dmalcolm@redhat.com \
--cc=gcc-patches@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).