Skip to content

Commit c9026cd

Browse files
pekkaklarckpekkaklarck
committed
Enhance README
1 parent 4029185 commit c9026cd

File tree

1 file changed

+132
-127
lines changed

1 file changed

+132
-127
lines changed

‎README.rst‎

Lines changed: 132 additions & 127 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,11 @@ Introduction
88
------------
99

1010
StatusChecker is a tool for validating that executed `Robot Framework`_
11-
test cases have expected statuses and log messages. It is mainly useful
12-
for Robot Framework test library developers who want to use Robot
13-
Framework to also test their libraries. StatusChecker 1.3 and newer are
14-
compatible both with Python 2 and Python 3.
11+
tests have expected statuses and log messages. It is mainly useful
12+
for Robot Framework library developers who want to test their libraries
13+
using Robot Framework.
1514

16-
StatusChecker project is hosted at GitHub_ and downloads are at
17-
PyPI_.
15+
StatusChecker project is hosted at GitHub_ and downloads are at PyPI_.
1816

1917
.. _Robot Framework: http://robotframework.org
2018
.. _GitHub: https://github.com/robotframework/statuschecker
@@ -50,28 +48,23 @@ Programmatically:
5048
process_output('infile.xml', 'outfile.xml')
5149

5250
If an output file is not given, the input file is edited in place.
51+
If you want to get a log and a report, you need to use the Rebot tool
52+
separately afterwards.
5353

5454
Defining expected test status
5555
-----------------------------
5656

57-
By default, all test cases are expected to *PASS* and have no
58-
message. Changing the expected status to *FAIL* is done by having
59-
the word ``FAIL`` (in uppercase) somewhere in the test case
60-
documentation. The expected error message must then follow
61-
the ``FAIL`` marker.
57+
By default, all tests are expected to *PASS* and have no message.
58+
Changing the expected status to *FAIL* is done by having the word
59+
``FAIL`` (case-sensitive) somewhere in the test documentation.
60+
The expected error message must then follow the ``FAIL`` marker.
6261

63-
For robotframework version 4 you can also change the expected status
64-
to *SKIP* by adding the word ``SKIP`` in the test case documentation.
65-
Like Fail, the expected skip message must follow the word ``SKIP``.
66-
If a test documentation contains the words ``FAIL`` and ``SKIP``, ``SKIP``
67-
will be ignored and the expected status will be *FAIL*.
68-
69-
If a test is expected to *PASS* with a certain message, the word
70-
``PASS`` must be added to its documentation explicitly and the
71-
expected message given after that.
72-
73-
If a message check should happen in test setup or teardown, that check
74-
must be prefixed with ``SETUP`` or ``TEARDOWN`` word.
62+
If the test is expected to be skipped, you can change the expected status
63+
to *SKIP* by adding the word ``SKIP`` in the documentation. Also in
64+
this case the expected message must follow the marker. If a test is
65+
expected to *PASS* with a certain message, the word ``PASS`` must be
66+
added to its documentation explicitly and the expected message given
67+
after that.
7568

7669
The expected message can also be specified as a regular expression by
7770
prefixing it with ``REGEXP:``. The specified regular expression
@@ -92,131 +85,143 @@ statuses and messages:
9285
.. sourcecode:: robotframework
9386

9487
*** Test Cases ***
95-
Simple failure
96-
[Documentation] FAIL Expected error message
97-
Steps
88+
Implicit PASS
89+
Log Hello!
90+
91+
Explicit PASS with message
92+
[Documentation] PASS Expected message
93+
Pass Execution Expected message
9894

99-
Check in test setup is done by SETUP marker
100-
[Documentation] LOG SETUP This first log message in test setup
101-
[Setup] Test specific setup
102-
Steps
95+
Expected FAIL
96+
[Documentation] FAIL Expected failure
97+
Fail Expected failure
10398

104-
Exclude documentation before marker
105-
[Documentation] This text is ignored FAIL Expected error message
106-
Steps
99+
Expected SKIP
100+
[Documentation] Text before marker is ignored SKIP Expected skip
101+
Skip Expected skip
107102

108-
Regexp example
103+
Message using REGEXP
109104
[Documentation] FAIL REGEXP: (IOError|OSError): .*
110-
Steps
105+
Fail IOError: Unknown error
111106

112-
Glob example
107+
Message using GLOB
113108
[Documentation] FAIL GLOB: ??Error: *
114-
Steps
109+
Fail IOError: Unknown error
115110

116-
Start example
111+
Message using STARTS
117112
[Documentation] FAIL STARTS: IOError:
118-
Steps
119-
120-
Passing without message
121-
Steps
113+
Fail IOError: Unknown error
122114

123-
Passing with message
124-
[Documentation] PASS Expected message
125-
Steps
126115

127116
Defining expected log messages
128117
------------------------------
129118

130-
The expected keyword log messages can also be defined in the test case
131-
documentation using a syntax such as::
132-
133-
LOG x.y:z LEVEL Actual message
134-
135-
The part before the colon specifies the keyword to check. For
136-
example, ``1`` means first keyword, ``1.2`` is the second child
137-
keyword of the first keyword, and so on.
138-
139-
The part after the colon species the message. For example, ``1:2``
140-
means the second message of the first keyword and ``1.2:3`` is
141-
the third message of the second child keyword of the first keyword.
142-
The message index is optional and defaults to ``1``.
143-
The message index also supports wildcard ``*``. For example ``1:*``
144-
matches any message of the first keyword.
145-
146-
Message level is specified before the actual message, and it can be
147-
any of the valid log levels in capital letters. If the level is not
148-
given it defaults to ``INFO``. Starting from 1.4 release also
149-
``ERROR`` level is supported. The message level also supports wildcard
150-
``ANY`` which will match all log levels.
151-
152-
Possible leading and trailing whitespace is ignored both in the expected
153-
and in the actual log message.
154-
155-
This syntax can be used multiple times to test multiple messages. It
156-
also works together with specifying the expected error message with
157-
``FAIL``, but it that case ``FAIL`` and the expected error must
158-
be first.
159-
160-
It is also possible to give the message as a regular expression or glob
161-
pattern or to give just the start of the message. This is accomplished
162-
by prefixing the message with ``REGEXP:``, ``GLOB:`` or ``STARTS:``,
163-
respectively, exactly like when `defining expected test status`_.
164-
165-
Finally, to check that a keyword does not have a certain message, it
166-
is possible to use ``NONE`` in the place of the message.
119+
In addition to verifying test statuses and messages, it possible to verify
120+
messages logged by keywords. Expected log messages are defined in the test
121+
documentation using this syntax::
122+
123+
LOG x.y.z LEVEL Actual message
124+
125+
The syntax consists of the following parts:
126+
127+
- ``LOG`` marker (case-sensitive).
128+
- Locator used for finding the message. Locators typically consists of 1-based
129+
indices like ``2.1.3`` matching items in test and keyword body. In addition
130+
to that, they can contain ``setup`` and ``teardown`` markers mathing test and
131+
keyword setup and teardown.
132+
- Optional, case-sensitive log level. If omitted, the level is ``INFO``.
133+
Special value ``ANY`` can be used to accept any level.
134+
- The actual log message. Possible leading and trailing whitespace is ignored.
135+
Special value ``NONE`` (case-sensitive) can be used to indicate that there
136+
should be no log message.
137+
138+
The locator can either point directly to the message to be verified or
139+
to the parent element of the message. In the latter case the actual message
140+
is expected to be the first item in parent's body. If the message index
141+
is not known, it is possible use the asterisk as a wildcard like ``2.*``
142+
to match any message. When a locator points directly to a message, it is
143+
possible to use ``:`` as the message separator instead of ``.``, but this
144+
support is deprecated and may be removed in the future.
145+
146+
If test status and message is also tested, they must be specified before
147+
the ``LOG`` marker using the syntax explained in the previous section.
148+
If there are multiple message to be tested, the ``LOG`` marker can be used
149+
multiple times. In such cases it is often a good idea to split the documentation
150+
to multiple lines.
167151

168152
.. sourcecode:: robotframework
169153

170154
*** Test cases ***
171-
Simple example
172-
[Documentation] LOG 1 Hello, world!
173-
Steps
174-
175-
Nested keywords
176-
[Documentation] LOG 2.1 1st child of 2nd kw
177-
Steps
155+
Locator points to message parent
156+
[Documentation] LOG 1 Hello! LOG 2 first LOG 3.1 Nested!
157+
Log Hello!
158+
Log Many first second third
159+
User Keyword
160+
161+
Locator points to directly to message
162+
[Documentation] Splitting can enhance readability. This text is ignored.
163+
... LOG 1.1 Hello!
164+
... LOG 2.2 second
165+
... LOG 3.1.1 Nested!
166+
Log Hello!
167+
Log Many first second third
168+
User Keyword
169+
170+
Message in setup and teardown
171+
[Documentation]
172+
... LOG setup Hello!
173+
... LOG teardown.1 Nested!
174+
[Setup] Log Hello!
175+
No Operation
176+
[Teardown] User Keyword
177+
178+
Wildcard
179+
[Documentation] LOG 1.* first
180+
Log Many first second third
178181

179-
Message index
180-
[Documentation] LOG 2:2 2nd msg of 2nd kw
181-
Steps
182-
183-
Nested and index
184-
[Documentation] LOG 3.1:2 2nd msg of 3rd kw's 1st child
185-
Steps
182+
No message
183+
[Documentation]
184+
... LOG 1.1 one
185+
... LOG 1.2 two
186+
... LOG 1.3 NONE
187+
Log Many one two
186188

187189
Log levels
188-
[Documentation] LOG 2 DEBUG Debug-level message
189-
... LOG 1.2:3 WARN Warning
190-
Steps
191-
192-
Multiple messages
193-
[Documentation] LOG 1 First tested message
194-
... LOG 1.2 Second tested message
195-
... LOG 2.2.1 DEBUG Third tested message
196-
Steps
190+
[Documentation]
191+
... LOG 1 DEBUG first
192+
... LOG 2 INFO second
193+
... LOG 3 ANY third
194+
Log first level=DEBUG
195+
Log second level=INFO
196+
Log third level=DEBUG
197+
198+
Test status and log message
199+
[Documentation] FAIL Expected failure
200+
... LOG 1 INFO Hello!
201+
... LOG 2 FAIL Expected failure
202+
Log Hello!
203+
Fail Expected failure
204+
205+
*** Keywords ***
206+
User Keyword
207+
Log Nested!
208+
209+
If the message is not known exactly, it is possible to match it as a regular
210+
expression or glob pattern or to give just the beginning of the message.
211+
This is accomplished by prefixing the message with ``REGEXP:``, ``GLOB:``
212+
or ``STARTS:``, respectively, exactly like when `defining expected test status`_.
197213

198-
Status and log
199-
[Documentation] FAIL Expected error message
200-
... LOG 1.2 Expected log message
201-
Steps
202-
203-
Regexp message
204-
[Documentation] LOG 1 REGEXP: (Hello|Hi) world!
205-
Steps
206-
207-
Glob message
208-
[Documentation] LOG 1 GLOB: * world!
209-
Steps
214+
.. sourcecode:: robotframework
210215

211-
Start of the message
212-
[Documentation] LOG 1 STARTS: Hello w
213-
Steps
216+
*** Test cases ***
217+
Log message using REGEXP
218+
[Documentation] LOG 1 REGEXP: Hello, .*!
219+
Log Hello, Robots!
214220

215-
No message
216-
[Documentation] LOG 1:1 Test that we have only 1 msg
217-
... LOG 1:2 NONE
218-
Steps
221+
Log message using GLOB
222+
[Documentation] LOG 1 GLOB: Hello, *!
223+
Log Hello, Robots!
219224
220-
Count Messages
221-
[Documentation] LOG 4 COUNT: 2 # Fourth keyword should have excatly 2 messages.
222-
Steps
225+
Log message using STARTS
226+
[Documentation] LOG 1 STARTS: Hello
227+
Log Hello, Robots!

0 commit comments

Comments
 (0)