chapter21_behavior_verification.txt

{{toc}}

! Behavior Verification

    How do we make tests self-checking when there is no state to verify?

    We capture the indirect outputs of the SUT as they occur and compare them to the expected behavior.

A Self-Checking Test (see page 26) must verify that the expected
outcome has occurred without manual intervention by whoever is running
the test. But what do we mean by “expected outcome”? The SUT may or
may not be “stateful”; if it is stateful, it may or may not be
expected to end up in a different state after it has been
exercised. The SUT may also be expected to invoke methods on other
objects or components.

* セルフチェックテスト (26ページ) では、テストの実行中人手の介入無しに発生する期待した出力を検証しなければなりませんでした。
* しかし "期待した出力" とは何を意味するのでしょうか ?
** SUT は "ステイトフル" かもしれないし、そうでないかもしれません。
** ステイトフルだとして、exercise 後の状態は期待したものになるかもしれないし、そうでないかもしれません。
** SUT は他のオブジェクトやコンポーネントのメソッドを実行することを期待してるかもしれません。

Behavior Verification involves verifying the indirect outputs of the
SUT as it is being exercised.

* Behavior Verifiction は、exercised した SUT の間接出力を検証するものです。

!! How It Works

Each test specifies not only how the client of the SUT interacts with
it during the exercise SUT phase of the test, but also how the SUT
interacts with the components on which it should depend. This ensures
that the SUT really is behaving as specified rather than just ending
up in the correct post-exercise state.

* テストが規程すること
** SUT のクライアントと、テストにおける exercise フェーズ中の SUT のやりとり
** SUT と、SUT 自身の依存するコンポーネントのやりとり
* テストが保証すること
** SUT が exercise 後の状態に至る迄の振る舞いが正しいこと
** SUT が exercise 後に正しい状態に到達する事ではない

Behavior Verification almost always involves interacting with or
replacing a depended-on component (DOC) with which the SUT interacts
at runtime. The line between Behavior Verification and State
Verification (page 462) can get a bit blurry when the SUT stores its
state in the DOC because both forms of verification involve
layer-crossing tests. We can distinguish between the two cases based
on whether we are verifying the post-test state in the DOC (State
Verification) or whether we are verifying the method calls made by the
SUT on the DOC (Behavior Verification).

* Behavior Verification に必要なこと
** SUT が実行時に依存する DOC とのやりとりできること
** DOC を置き換えできること
* Behavior Verification と State Verification (462ページ) の違い
** SUT の状態が DOC に保存される場合は曖昧
*** いずれにしてもレイヤー交差テストを含むため
* Behavior Verification と State Verification (462ページ) の区別
** State Verification ではテスト後の DOC の状態を検証する
** Behavior Verification では DOC に対する SUT からのメソッド呼び出しを検証する

""担当者ノート
""*DOC = depended-on component

!! When to Use It

Behavior Verification is primarily a technique for unit tests and
component tests. We can use Behavior Verification whenever the SUT
calls methods on other objects or components. We must use Behavior
Verification whenever the expected outputs of the SUT are transient
and cannot be determined simply by looking at the post-exercise state
of the SUT or the DOC. This forces us to monitor these indirect
outputs as they occur.

* Behavior Verification はユニットテスト、コンポーネントテストにおける基本的な手法です。
* SUT が他のオブジェクトやコンポーネントのメソッドを呼び出しているようなら、いつでも利用できます。
* 使う場面
** SUT からの期待する出力が一時的なものである場合
** exercise 後の SUT や DOC の状態を簡単に参照できないような場合
** 期待する出力ないし振る舞いが発生したことを、間接的な出力によって監視するしかない

A common application of Behavior Verification is when we are writing
our code in an “outside-in” manner. This approach, which is often
called need-driven development, involves writing the client code
before we write the DOC. It is a good way to find out exactly what the
interface provided by the DOC needs to be based on real, concrete
examples rather than on speculation. The main objection to this
approach is that we need to use a lot of Test Doubles (page 522) to
write these tests. That could result in Fragile Tests (page 239)
because each test knows so much about how the SUT is
implemented. Because the tests specify the behavior of the SUT in
terms of its interactions with the DOC, a change in the implementation
of the SUT could break a lot of tests. This kind of Overspecified
Software (see Fragile Test) could lead to High Test Maintenance Cost
(page 265).

* Behavior Verification の典型的な用途
** "outside-in" マナーに従ってコードを構築している場合
* need-driven development
** DOC を実装する前にそれを必要とするクライアントコードから実装していくことから、ニーズ駆動開発と呼ばれます。
** DOC が提供するインターフェースを、実際的な要求や事例に基づいて発見できる
*** 推測に基づく方法よりも好ましい方法です。
* main objection to need-driven development
** テストを書くために多数の Test Doubles (522ページ) が必要になること
** これらのテストは、SUT がどう実装されているか知りすぎてしまうこと
** Fragile Test (239ページ)
* beacause...
** テストが DOC との相互作用において SUT の振る舞いを規程する
** SUT の実装の変更により、多くのテストが動かなくなるから
* Overspecified Software
** High Test Maintenance Cost (265ページ) を導くことになります。

The jury is still out on whether Behavior Verification is a better
approach than State Verification. In most cases, State Verification is
clearly necessary; in some cases, Behavior Verification is clearly
necessary. What has yet to be determined is whether Behavior
Verification should be used in all cases or whether we should use
State Verification most of the time and resort to Behavior
Verification only when State Verification falls short of full test
coverage.  

* 未決課題
** Behavior Verification が State Verification よりもよい手法であるかどうかはまだ決着していない問題です。
** 場合によりけり
** Behavior Verification をいかなる場合でも使用すべきか
** State Verification ではテストカバレッジを満たせない場合だけ使用するべきなのか
** 断言することはできません。

!! Implementation Notes

Before we exercise the SUT by invoking the methods of interest, we
must ensure that we have a way of observing its behavior. Sometimes
the mechanisms that the SUT uses to interact with the components
surrounding it make such observation possible; when this is not the
case, we must install some sort of Test Double to monitor the SUT’s
indirect outputs. We can use a Test Double as long as we have a way to
replace the DOC with the Test Double. This could be via Dependency
Injection (page 678) or by Dependency Lookup (page 686).

* SUT の exercise 前に、その振る舞いを観察する方法を保証しなければなりません。
* SUT には、その周辺コンポーネントに振る舞いの観察をできるようにする仕組みが備わっていることがあります。
* しかし、そういった仕組みが無い場合、SUT の間接的な出力を観測するため、何らかの Test Double をインストールする必要があります。
* DOC を Test Double に置き換える方法がある限り、Test Double を使います。
** Dependency Injection (678 ページ)
** Dependency Lookup (686ページ)

There are two fundamentally different ways to implement Behavior
Verification, each with its own proponents. The Mock Object (page 544)
community has been very vocal about the use of “mocks” as an
Expected Behavior Specification, so it is the more commonly used
approach. Nevertheless, Mock Objects are not the only way of doing
Behavior Verification.

* Behavior Verification 実装の派閥
** それぞれに養護者のいる根本的に異なる 2 つの派閥があります。
** 期待する振る舞いを仕様化するために "mock" を使う派 (多数派)
** もう 1 つは ?

!!! Variation: Procedural Behavior Verification

In Procedural Behavior Verification, we capture the method calls made
by the SUT as it executes and later get access to them from within the
Test Method (page 348). Then we use Equality Assertions (see Assertion
Method on page 362) to compare them with the expected results.

* SUT を駆動したことによるメソッド呼び出しをキャプチャし
* テストメソッドからその結果にアクセス
* Equality Assertions (362 ページ、Assertion Method) により、期待する結果と比較

The most common way of trapping the indirect outputs of the SUT is to
install a Test Spy (page 538) in place of the DOC during the fixture
setup phase of the Four-Phase Test (page 358). During the result
verification phase of the test, we ask the Test Spy how it was used by
the SUT during the exercise SUT phase. Use of a Test Spy does not
require any advance knowledge of how the methods of the DOC will be
called.

* SUT の間接出力をひっかける
** fixture setup フェーズにおいて、DOC に Test Spy をインストールする方法が一般的
** 結果検証フェーズにおいて、exercise フェーズ中に SUT がどのように利用されたのかを、Test Spy に問い合わせます。
* Test Spy を利用するメリット
** DOC のメソッドがどのように呼ばれるか詳しく知る必要がありません。

The alternative is to ask the real DOC how it was used. Although this
scheme is not always feasible, when it is, it avoids the need to use a
Test Double and minimizes the degree to which we have Overspecified
Software.

* 本物の DOC にどのように呼ばれたかを問合せる方法
** この方法はいつでも使えるとは限りません。
** この方法が利用できるなら、Test Double を使わなくてもよい
*** Overspecified Software になってしまう度合いを最小化できます。

We can reduce the amount of code in the Test Method (and avoid Test
Code Duplication; see page 213) by defining Expected Objects (see
State Verification) for the arguments of method calls or by delegating
the verification of them to Custom Assertions (page 474).

* テストメソッドのコード量を減らす方法 (あるいは Test Code Duplication を回避する方法)
** メソッド呼び出しの引数に Expected Object を使う
** 結果の検証をカスタムアサーションに任せる

!!! Variation: Expected Behavior Specification

Expected Behavior Specification is a different way of doing Behavior
Verification. Instead of waiting until after the fact to verify the
indirect outputs of the SUT by using a sequence of assertions, we load
the Expected Behavior Specification into a Mock Object and let it
verify that the method calls are correct as they are received.

* Behavior Verification のもう 1 つの方法
** 元の方法: 連続したアサーションにより、SUT の間接出力を検証し終わるまで待つ
** Mock Object に、あるメソッドが呼ばれた時点でそれが正しいかを検証する機能を埋め込む

We can use an Expected Behavior Specification when we know exactly
what should happen ahead of time and we want to remove all Procedural
Behavior Verification from the Test Method. This pattern variation
tends to make the test shorter (assuming we are using a compact
representation of the expected behavior) and can be used to cause the
test to fail on the first deviation from the expected behavior if we
so choose.

* 次のようなケースで利用される
** 条件: 具体的に何がいつ起きるかを知っている
** 目的: テストメソッドから Procedural Behavior Verification を排除したい
* この亜種のメリット
** 期待する振る舞いをコンパクトに表現できるとすると、テストコードを短くできる
** 期待した振る舞いとは異なる振る舞いをしたその時点で、テストを失敗できるようにする

One distinct advantage of using Mock Objects is that Test Double
generation tools are available for many members of the xUnit
family. They make implementing Expected Behavior Specification very
easy because we don’t need to manually build a Test Double for each
set of tests. One drawback of using a Mock Object is that it requires
that we can predict how the methods of the DOC will be called and what
arguments will be passed to it in the method calls.

* Mock Object をお勧めする理由
** 多くの xUnit ファミリーには、利用可能な Test Double ジェネレータが存在している
** 手動で Test Double を作成しなくてよい
* Mock Object をお勧めしない理由
** DOC のメソッドがどう呼ばれるかを予測しないといけない
** DOC のメソッド呼び出しにどんな引数が渡されるかを予測しないといけない

!! Motivating Example

The following test is not a Self-Checking Test because it does not
verify that the expected outcome has actually occurred; it contains no
calls to Assertion Methods, nor does it set up any expectations on a
Mock Object. Because we are testing the logging functionality of the
SUT, the state that interests us is actually stored in the logger
rather than within the SUT itself. The writer of this test hasn’t
found a way to access the state we are trying to verify.

* 以下のテストコードは、セルフチェックテストではない
** 実際の結果を検証してない
** アサーションメソッドの呼び出しがない
** Mock Object 中に期待値を設定しているわけでもない

   public void testRemoveFlightLogging_NSC() throws Exception {
      // setup
      FlightDto expectedFlightDto = createARegisteredFlight();
      FlightManagementFacade facade =
            new FlightManagementFacadeImpl();
      // exercise
      facade.removeFlight(expectedFlightDto.getFlightNumber());
      // verify
      // have not found a way to verify the outcome yet
      //  Log contains record of Flight removal
   }

To verify the outcome, whoever is running the tests must access the
database and the log console and compare what was actually output to
what should have been output.

結果を検証するために、テストの実行者はデータベースやログコンソールにアクセスしなければならない。

One way to make the test Self-Checking is to enhance the test with
Expected State Specification (see State Verification) of the SUT as
follows:

Expected State Specification によってこのテストをセルフチェックテストにする。

   public void testRemoveFlightLogging_ESS() throws Exception {
      // fixture setup
      FlightDto expectedFlightDto = createAnUnregFlight();
      FlightManagementFacadeImplTI facade =
            new FlightManagementFacadeImplTI();
      // exercise
      facade.removeFlight(expectedFlightDto.getFlightNumber());
      // verify
      assertFalse("flight still exists after being removed",
                  facade.flightExists( expectedFlightDto.
                                              getFlightNumber()));
   }


Unfortunately, this test does not verify the logging function of the
SUT in any way. It also illustrates one reason why Behavior
Verification came about: Some functionality of the SUT is not visible
within the end state of the SUT itself, but can be seen only if we
intercept the behavior at an internal observation point between the
SUT and the DOC or if we express the behavior in terms of state
changes for the objects with which the SUT interacts.

* 残念ながら、このテストでは SUT のログ出力機能を検証できていない。
* Behavior Verification ができた理由
** 事実: SUT のいくつかの機能は、最終状態からアクセスできない
** 工夫: SUT〜DOC間の内部観測ポイントを intercept すれば確認できる
** 工夫: SUT と相互作用するオブジェクトの状態遷移によって、その振る舞いを表現できる

!! Refactoring Notes

When we made the changes in the second code sample in the “Motivating
Example,” we weren’t really refactoring; instead, we added
verification logic to make the tests behave differently. There are,
however, several refactoring cases that are worth discussing.

To refactor from State Verification to Behavior Verification, we must
do a Replace Dependency with Test Double (page 522) refactoring to
gain visibility of the indirect outputs of the SUT via a Test Spy or
Mock Object.

* State Verification から Behavior Verification へのリファクタリング
** Test Double による依存性の置換 (522 ぺーじ)
** Test Spy か Mock Object により SUT の間接出力を見える化する

To refactor from an Expected Behavior Specification to Procedural
Behavior Verification, we install a Test Spy instead of the Mock
Object. After exercising the SUT, we make assertions on values
returned by the Test Spy and compare them with the expected values
that were originally used as arguments when we initially configured
the Mock Object (the one that we just converted into a Test Spy).

* Expected Behavior Specification から Procedural Behavior Verifiction へのリファクタリング
** Mock Object の代わりに Test Spy をインストール
** Mock Object の設定時に引数としていた値を期待値として、Test Spy の返す値のアサーションを行う

To refactor from Procedural Behavior Verification to an Expected
Behavior Specification, we configure a Mock Object with the expected
values from the assertions made on values returned by the Test Spy and
install the Mock Object instead of the Test Spy.

* Procedural Behavior Verification から Expected Behavior Specification へのリファクタリング
** Test Spy の返り値のアサーションで使っている期待値を、Mock Object に設定
** Test Spy の代わりに Mock Object をインストール

!!! Example: Procedural Behavior Verification

The following test verifies the basic functionality of creating a
flight but uses Procedural Behavior Verification to verify the
indirect outputs of the SUT. That is, it uses a Test Spy to capture
the indirect outputs and then verifies those outputs are correct by
making in-line calls to the Assertion Methods.

   public void testRemoveFlightLogging_recordingTestStub()
            throws Exception {
      // fixture setup
      FlightDto expectedFlightDto = createAnUnregFlight();
      FlightManagementFacade facade =
            new FlightManagementFacadeImpl();
      //    Test Double setup
      AuditLogSpy logSpy = new AuditLogSpy();
      facade.setAuditLog(logSpy);
      // exercise
      facade.removeFlight(expectedFlightDto.getFlightNumber());
      // verify
      assertEquals("number of calls", 1,
                   logSpy.getNumberOfCalls());
      assertEquals("action code",
                   Helper.REMOVE_FLIGHT_ACTION_CODE,
                   logSpy.getActionCode());
      assertEquals("date", helper.getTodaysDateWithoutTime(),
                   logSpy.getDate());
      assertEquals("user", Helper.TEST_USER_NAME,
                   logSpy.getUser());
      assertEquals("detail",
                   expectedFlightDto.getFlightNumber(),
                   logSpy.getDetail());
   }

!!! Example: Expected Behavior Specification

In this version of the test, we use the JMock framework to define the
expected behavior of the SUT. The method expects on mockLog configures
the Mock Object with the Expected Behavior Specification
(specifically, the expected log message).

   public void testRemoveFlight_JMock() throws Exception {
      // fixture setup
      FlightDto expectedFlightDto = createAnonRegFlight();
      FlightManagementFacade facade =
            new FlightManagementFacadeImpl();
      // mock configuration
      Mock mockLog = mock(AuditLog.class);
      mockLog.expects(once()).method("logMessage")
               .with(eq(helper.getTodaysDateWithoutTime()),
                     eq(Helper.TEST_USER_NAME),
                     eq(Helper.REMOVE_FLIGHT_ACTION_CODE),
                     eq(expectedFlightDto.getFlightNumber()));
      // mock installation
      facade.setAuditLog((AuditLog) mockLog.proxy());
      // exercise
      facade.removeFlight(expectedFlightDto.getFlightNumber());
      // verify
      // verify() method called automatically by JMock
   }


Local Variables:
coding: utf-8
End: