QTapTestLogger Class Reference

#include <qtaptestlogger_p.h>

Inheritance diagram for QTapTestLogger:

Collaboration diagram for QTapTestLogger:

Public Member Functions
	QTapTestLogger (const char *filename)
	~QTapTestLogger ()
void	startLogging () override
	Called before the start of a test run.
void	stopLogging () override
	Called after the end of a test run.
void	enterTestFunction (const char *) override
	This virtual method is called before each test function is invoked.
void	leaveTestFunction () override
	This virtual method is called after a test function has completed, to match \l enterTestFunction().
void	enterTestData (QTestData *data) override
	This virtual method is called before and after each call to a test function.
void	addIncident (IncidentTypes type, const char *description, const char *file=nullptr, int line=0) override
	This virtual method is called when an event occurs that relates to the resolution of the test.
void	addMessage (MessageTypes type, const QString &message, const char *file=nullptr, int line=0) override
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
void	addBenchmarkResult (const QBenchmarkResult &) override
	This virtual method is called after a benchmark has been run enough times to produce usable data.
Public Member Functions inherited from QAbstractTestLogger
	QAbstractTestLogger (const char *filename)
	Constructs the base-class parts of the logger.
virtual	~QAbstractTestLogger ()
	Destroys the logger object.
virtual void	startLogging ()
	Called before the start of a test run.
virtual void	stopLogging ()
	Called after the end of a test run.
virtual void	enterTestFunction (const char *function)=0
	This virtual method is called before each test function is invoked.
virtual void	leaveTestFunction ()=0
	This virtual method is called after a test function has completed, to match \l enterTestFunction().
virtual void	enterTestData (QTestData *)
	This virtual method is called before and after each call to a test function.
virtual void	addIncident (IncidentTypes type, const char *description, const char *file=nullptr, int line=0)=0
	This virtual method is called when an event occurs that relates to the resolution of the test.
virtual void	addBenchmarkResult (const QBenchmarkResult &result)=0
	This virtual method is called after a benchmark has been run enough times to produce usable data.
virtual void	addBenchmarkResults (const QList< QBenchmarkResult > &result)
virtual void	addMessage (QtMsgType, const QMessageLogContext &, const QString &)
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.This virtual method is called from the custom message handler QtTest installs in place of Qt's default message handler for the duration of testing, unless QTest::ignoreMessage() was used to ignore it, or too many messages have previously been processed.
virtual void	addMessage (MessageTypes type, const QString &message, const char *file=nullptr, int line=0)=0
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
bool	isLoggingToStdout () const
	Returns true if the output stream is standard output.
void	outputString (const char *msg)
	Convenience method to write msg to the output stream.

Additional Inherited Members
Public Types inherited from QAbstractTestLogger
enum	IncidentTypes {   Skip , Pass , XFail , Fail ,   XPass , BlacklistedPass , BlacklistedFail , BlacklistedXPass ,   BlacklistedXFail }
	\value Pass The test ran to completion successfully. More...
enum	MessageTypes {   QDebug , QInfo , QWarning , QCritical ,   QFatal , Info , Warn }
	The members whose names begin with Q describe messages that originate in calls, by the test or code under test, to Qt logging functions (implemented as macros) whose names are similar, with a q in place of the leading Q. More...
Protected Member Functions inherited from QAbstractTestLogger
void	filterUnprintable (char *str) const
	Helper utility to blot out unprintable characters in str.
Protected Attributes inherited from QAbstractTestLogger
FILE *	stream

Detailed Description

\inmodule QtTest

QTapTestLogger implements the Test Anything Protocol v13.

The \l{Test Anything Protocol} (TAP) is a simple plain-text interface between testing code and systems for reporting and analyzing test results. Since QtTest doesn't build the table for a data-driven test until the test is about to be run, we don't typically know how many tests we'll run until we've run them, so we put The Plan at the end, rather than the beginning. We summarise the results in comments following The Plan.

The TAP protocol supports inclusion, after a Test Line, of a "diagnostic block" in YAML, containing supporting information. We use this to package other information from the test, in combination with comments to record information we're unable to deliver at the same time as a test line. By convention, TAP producers limit themselves to a restricted subset of YAML, known as YAMLish, for maximal compatibility with TAP consumers.

YAML (see yaml.org for details) supports three data-types: mapping (hash or dictionary), sequence (list, array or set) and scalar (string or number), much as JSON does. It uses indentation to indicate levels of nesting of mappings and sequences within one another. A line starting with a dash (at the same level of indent as its context) introduces a list entry. A line starting with a key (more indented than its context, aside from any earlier keys at its level) followed by a colon introduces a mapping entry; the key may be either a simple word or a quoted string (within which numeric escapes may be used to indicate unicode characters). The value associated with a given key, or the entry in a list, can appar after the key-colon or dasy either on the same line or on a succession of subsequent lines at higher indent. Thus

- first list item
-
  second: list item is a mapping
  with:
  - two keys
  - the second of which is a list
- back in the outer list, a third item

In YAMLish, the top-level structure should be a hash. The keys supported for this hash, with the meanings for their values, are:

\list

• message: free text supplying supporting details
• severity: how bad is it ?
• source: source describing the test, as an URL (compare file, line)
• at: identifies the function (with file and line) performing the test
• datetime: when the test was run (ISO 8601 or HTTP format)
• file: source of the test as a local file-name, when appropriate
• line: line number within the source
• name: test name
• extensions: sub-hash in which to store random other stuff
• actual: what happened (a.k.a. found; contrast expected)
• expected: what was expected (a.k.a. wanted; contrast actual)
• display: description of the result, suitable for display
• dump: a sub-hash of variable values when the result arose
• error: describes the error
• backtrace: describes the call-stack of the error \endlist

In practice, only at, expected and actual appear to be generally supported.

We can also have several messages produced within a single test, so the simple message / severity pair of top-level keys does not serve us well. We therefore use extensions with a sub-tag messages in which to package a list of messages.

See also

QAbstractTestLogger

Definition at line 22 of file qtaptestlogger_p.h.

Constructor & Destructor Documentation

QTapTestLogger()

QTapTestLogger::QTapTestLogger

(

const char *

filename

)

Definition at line 95 of file qtaptestlogger.cpp.

~QTapTestLogger()

QTapTestLogger::~QTapTestLogger ( )

default

Member Function Documentation

addBenchmarkResult()

void QTapTestLogger::addBenchmarkResult ( const QBenchmarkResult &  result )

inlineoverridevirtual

This virtual method is called after a benchmark has been run enough times to produce usable data.

It is passed the median result from all cycles of the code controlled by the test's QBENCHMARK loop.

Every logging implementation must implement this method.

See also

addIncident(), addMessage()

Implements QAbstractTestLogger.

Definition at line 41 of file qtaptestlogger_p.h.

addIncident()

void QTapTestLogger::addIncident ( IncidentTypes  type, const char *  description, const char *  file = nullptr, int  line = 0  )

overridevirtual

This virtual method is called when an event occurs that relates to the resolution of the test.

The type indicates whether this was a pass, a fail or a skip, whether a failure was expected, and whether the test being run is blacklisted. The description may be empty (for a pass) or a message describing the nature of the incident. Where the location in code of the incident is known, it is indicated by file and line; otherwise, these are nullptr and 0, respectively.

Every logging implementation must implement this method. Note that there are circumstances where more than one incident may be reported, in this way, for a single run of a test on a single dataset. It is the implementation's responsibility to recognize such cases and decide what to do about them. For purposes of counting resolutions of tests in the "Totals" report at the end of a test run, QtTest considers the first incident (excluding XFail and its blacklisted variant) decisive.

See also

addMessage(), addBenchmarkResult()

Implements QAbstractTestLogger.

Definition at line 216 of file qtaptestlogger.cpp.

References QTestPrivate::appendCharBuffer(), QAbstractTestLogger::BlacklistedFail, QAbstractTestLogger::BlacklistedPass, QAbstractTestLogger::BlacklistedXFail, QAbstractTestLogger::BlacklistedXPass, Compare(), QTestCharBuffer::constData(), QTestResult::currentTestFunction(), QTestResult::currentTestObjectName(), expected, QAbstractTestLogger::Fail, file, QString::fromUtf8(), info, QTestCharBuffer::isEmpty(), QLatin1StringView::latin1(), line, QRegularExpression::match(), match(), ok, QAbstractTestLogger::Pass, Q_FALLTHROUGH, qPrintable, QTest::qt_asprintf(), QAbstractTestLogger::Skip, QString::startsWith(), QTestLog::totalCount(), Unknown, QAbstractTestLogger::XFail, QAbstractTestLogger::XPass, and YAML_INDENT.

Here is the call graph for this function:

addMessage()

void QTapTestLogger::addMessage ( MessageTypes  type, const QString &  message, const char *  file = nullptr, int  line = 0  )

overridevirtual

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

This virtual method is called, via its QtMsgType overload, from the custom message handler QtTest installs. It is also used to warn about various situations detected by QtTest itself, such as failure to see a message anticipated by QTest::ignoreMessage() and, particularly when verbosity options have been enabled via the command-line, to log some extra information.

Every logging implementation must implement this method. The type indicates the category of message and the message is the content to be reported. When the message is associated with specific code, the name of the file and line number within it are also supplied; otherwise, these are \nullptr and 0, respectively.

See also

QTest::ignoreMessage(), addIncident()

Implements QAbstractTestLogger.

Definition at line 429 of file qtaptestlogger.cpp.

References QTestPrivate::appendCharBuffer(), QTestCharBuffer::constData(), file, QAbstractTestLogger::Info, line, QAbstractTestLogger::outputString(), Q_UNUSED, QAbstractTestLogger::QCritical, QAbstractTestLogger::QDebug, QAbstractTestLogger::QFatal, QAbstractTestLogger::QInfo, qPrintable, QTest::qt_asprintf(), QAbstractTestLogger::QWarning, QAbstractTestLogger::Warn, and YAML_INDENT.

Here is the call graph for this function:

enterTestData()

void QTapTestLogger::enterTestData ( QTestData *  )

overridevirtual

This virtual method is called before and after each call to a test function.

For a data-driven test, the call before is passed the name of the test data row. This may combine a global data row name with a local data row name. For non-data-driven tests and for the call after a test function, \nullptr is passed

A logging implementation might chose to record the data row name for reporting of results from the test for that data row. It should, in such a case, clear its record of the name when called with \nullptr.

See also

enterTestFunction(), leaveTestFunction()

Reimplemented from QAbstractTestLogger.

Definition at line 140 of file qtaptestlogger.cpp.

References QTestCharBuffer::clear(), and QTestCharBuffer::isEmpty().

Here is the call graph for this function:

enterTestFunction()

void QTapTestLogger::enterTestFunction ( const char *  function )

overridevirtual

This virtual method is called before each test function is invoked.

It is passed the name of the test function (without its class prefix) as function. It is likewise called for {initTestCase()} at the start of testing, after \l startLogging(), and for {cleanupTestCase()} at the end of testing, in each case passing the name of the function. It is also called with \nullptr as function after the last of these functions, or in the event of an early end to testing, before \l stopLogging().

For data-driven test functions, this is called only once, before the data function is called to set up the table of datasets and the test is run with its first dataset.

Every logging implementation must implement this method. It shall typically need to record the name of the function for later use in log messages.

See also

leaveTestFunction(), enterTestData()

Implements QAbstractTestLogger.

Definition at line 131 of file qtaptestlogger.cpp.

References QTestCharBuffer::clear(), function, QTestCharBuffer::isEmpty(), and Q_ASSERT.

Here is the call graph for this function:

leaveTestFunction()

void QTapTestLogger::leaveTestFunction ( )

inlineoverridevirtual

This virtual method is called after a test function has completed, to match \l enterTestFunction().

For data-driven test functions, this is called only once, after the test is run with its last dataset.

Every logging implementation must implement this method. In some cases it may be called more than once without an intervening call to \l enterTestFunction(). In such cases, the implementation should ignore these later calls, until the next call to enterTestFunction().

See also

enterTestFunction(), enterTestData()

Implements QAbstractTestLogger.

Definition at line 32 of file qtaptestlogger_p.h.

startLogging()

void QTapTestLogger::startLogging ( )

overridevirtual

Called before the start of a test run.

This virtual method is called before the first tests are run. A logging implementation might open a file, write some preamble, or prepare in other ways, such as setting up initial values of variables. It can use the usual Qt logging infrastucture, since it is also called before QtTest installs its own custom message handler.

See also

stopLogging()

Reimplemented from QAbstractTestLogger.

Definition at line 102 of file qtaptestlogger.cpp.

References QTestResult::currentTestObjectName(), QAbstractTestLogger::outputString(), preamble, QTest::qt_asprintf(), and QAbstractTestLogger::startLogging().

Here is the call graph for this function:

stopLogging()

void QTapTestLogger::stopLogging ( )

overridevirtual

Called after the end of a test run.

This virtual method is called after all tests have run. A logging implementation might collate information gathered from the run, write a summary, or close a file. It can use the usual Qt logging infrastucture, since it is also called after QtTest has restored the default message handler it replaced with its own custom message handler.

See also

startLogging()

Reimplemented from QAbstractTestLogger.

Definition at line 115 of file qtaptestlogger.cpp.

References QTestCharBuffer::data(), QTestLog::failCount(), QAbstractTestLogger::outputString(), QTestLog::passCount(), QTest::qt_asprintf(), QAbstractTestLogger::stopLogging(), and QTestLog::totalCount().

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• qtbase/src/testlib/qtaptestlogger_p.h
• qtbase/src/testlib/qtaptestlogger.cpp