Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -703,14 +703,14 @@ Tasty executes tests in parallel to make them finish faster.
If this parallelism is not desirable, you can declare *dependencies* between
tests, so that one test will not start until certain other tests finish.

Dependencies are declared using the `after` or `sequentialTestGroup` combinator:
Dependencies are declared using the `after` or `dependentTestGroup` combinator:

* `after AllFinish "pattern" my_tests` will execute the test tree `my_tests` only after all
tests that match the pattern finish.
* `after AllSucceed "pattern" my_tests` will execute the test tree `my_tests` only after all
tests that match the pattern finish **and** only if they all succeed. If at
least one dependency fails, then `my_tests` will be skipped.
* `sequentialTestGroup groupName dependencyType [tree1, tree2, ..]` will execute all tests
* `dependentTestGroup groupName dependencyType [tree1, tree2, ..]` will execute all tests
in `tree1` first, after which it will execute all tests in `tree2`, and so forth. Like
`after`, `dependencyType` can either be set to `AllFinish` or `AllSucceed`.

Expand All @@ -723,7 +723,7 @@ after
-> TestTree -- ^ the subtree that depends on other tests
-> TestTree -- ^ the subtree annotated with dependency information

sequentialTestGroup
dependentTestGroup
:: TestName -- ^ name of the group
-> DependencyType -- ^ whether to run the tests even if some of the dependencies fail
-> [TestTree] -- ^ trees to execute sequentially
Expand Down Expand Up @@ -792,7 +792,7 @@ Here are some caveats to keep in mind when using patterns to specify dependencie
test tree, searching for the next test to execute may also have an
overhead quadratic in the number of tests.

Use `sequentialTestGroup` to mitigate these problems.
Use `dependentTestGroup` to mitigate these problems.


## FAQ
Expand Down
2 changes: 1 addition & 1 deletion core-tests/DependentTestGroup.hs
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ toTestTree (GenUniqueLabels genUniqueLabels) tree =

Test n -> do
-- Caller might opt to not generate unique labels for each test:
-- sequentialTestGroup should still function properly in face of name collisions.
-- dependentTestGroup should still function properly in face of name collisions.
let label = if genUniqueLabels then "T" ++ show n else "T"

testCase label $ do
Expand Down
10 changes: 5 additions & 5 deletions core/Control/Concurrent/Async.hs
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ import GHC.IO hiding (onException)
--
data Async a = Async
{ asyncThreadId :: {-# UNPACK #-} !ThreadId
-- ^ Returns the 'ThreadId' of the thread running
-- the given 'Async'.
-- ^ Returns the t'ThreadId' of the thread running
-- the given t'Async'.
, _asyncWait :: STM (Either SomeException a)
}

Expand Down Expand Up @@ -150,12 +150,12 @@ waitCatchSTM :: Async a -> STM (Either SomeException a)
waitCatchSTM (Async _ w) = w

-- | Cancel an asynchronous action by throwing the @AsyncCancelled@
-- exception to it, and waiting for the `Async` thread to quit.
-- Has no effect if the 'Async' has already completed.
-- exception to it, and waiting for the t'Async' thread to quit.
-- Has no effect if the t'Async' has already completed.
--
-- > cancel a = throwTo (asyncThreadId a) AsyncCancelled <* waitCatch a
--
-- Note that 'cancel' will not terminate until the thread the 'Async'
-- Note that 'cancel' will not terminate until the thread the t'Async'
-- refers to has terminated. This means that 'cancel' will block for
-- as long said thread blocks when receiving an asynchronous exception.
--
Expand Down
55 changes: 36 additions & 19 deletions core/Test/Tasty/Core.hs
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ resultSuccessful r =
Success -> True
Failure {} -> False

-- | Shortcut for creating a 'Result' that indicates exception
-- | Shortcut for creating a t'Result' that indicates exception
exceptionResult :: SomeException -> Result
exceptionResult e = Result
{ resultOutcome = Failure $ TestThrewException e
Expand Down Expand Up @@ -204,7 +204,7 @@ class Typeable t => IsTest t where
-- | Run the test
--
-- This method should cleanly catch any exceptions in the code to test, and
-- return them as part of the 'Result', see 'FailureReason' for an
-- return them as part of the t'Result', see 'FailureReason' for an
-- explanation. It is ok for 'run' to raise an exception if there is a
-- problem with the test suite code itself (for example, if a file that
-- should contain example data or expected output is not found).
Expand All @@ -222,7 +222,7 @@ class Typeable t => IsTest t where
-- @since 0.1
type TestName = String

-- | 'ResourceSpec' describes how to acquire a resource (the first field)
-- | t'ResourceSpec' describes how to acquire a resource (the first field)
-- and how to release it (the second field).
--
-- @since 0.6
Expand Down Expand Up @@ -325,7 +325,7 @@ data TestTree
-- @since 1.2

-- | Create a named group of test cases or other groups. Tests are executed in
-- parallel. For sequential execution, see 'sequentialTestGroup'.
-- parallel. For sequential execution, see 'dependentTestGroup'.
--
-- @since 0.1
testGroup :: TestName -> [TestTree] -> TestTree
Expand All @@ -337,34 +337,51 @@ testGroup = TestGroup
-- @since 1.5
sequentialTestGroup :: TestName -> DependencyType -> [TestTree] -> TestTree
sequentialTestGroup = dependentTestGroup

-- | Create a named group of test cases or other groups. Tests are executed in
-- order and each test is considered a dependency of the next one. If a filter
-- is applied, any dependencies are run too, even if they would otherwise not
-- match the filter's criteria.
-- order and each test is considered a dependency of the next one.
-- If filtering by t'TestPattern' (@--pattern@) is in action,
-- any test matching the pattern also shields earlier tests from filtering out,
-- even if they themselves do not match the pattern.
--
-- For parallel execution, see 'testGroup'. For ordered test execution, but
-- without dependencies, see 'inOrderTestGroup'.
-- with default filtering behavior, see 'inOrderTestGroup'.
--
-- Note that this is will only work when used with the default `TestManager`.
-- If you use another manager, like `tasty-rerun` for instance, sequentiality
-- Note that this is will only work when used with the default 'Test.Tasty.Ingredients.TestManager'.
-- If you use another manager, like @tasty-rerun@ for instance, sequentiality
-- might possibly be ignored.
--
-- @since 1.5
dependentTestGroup :: TestName -> DependencyType -> [TestTree] -> TestTree
-- @since 1.5.4
dependentTestGroup
:: TestName
-- ^ name of the group
-> DependencyType
-- ^ whether to run subsequent tests even if earlier ones have failed
-> [TestTree]
-- ^ tests to execute sequentially
-> TestTree
dependentTestGroup nm depType = setDependent . TestGroup nm . map setParallel
where
setParallel = PlusTestOptions (setOption $ Independent Parallel)
setDependent = PlusTestOptions (setOption (Dependent depType))


-- | Create a named group of test cases that will be played sequentially,
-- in the exact order provided, though filters are still applied.
-- in the exact order provided. Similarly to 'testGroup'
-- and in contrast to 'dependentTestGroup',
-- filtering by t'TestPattern' is applied uniformly.
--
-- Note that this is will only work when used with the default `TestManager`.
-- If you use another manager, like `tasty-rerun` for instance, the fact that
-- Note that this is will only work when used with the default 'Test.Tasty.Ingredients.TestManager'.
-- If you use another manager, like @tasty-rerun@ for instance, the fact that
-- these tests should be run in the given order might possibly be ignored.
inOrderTestGroup :: TestName -> [TestTree] -> TestTree
--
-- @since 1.5.4
inOrderTestGroup
:: TestName
-- ^ name of the group
-> [TestTree]
-- ^ tests to execute sequentially
-> TestTree
inOrderTestGroup nm = setSequential . TestGroup nm . map setParallel
where
setParallel = PlusTestOptions (setOption $ Independent Parallel)
Expand Down Expand Up @@ -490,7 +507,7 @@ type TestMatched = Any
-- a user's filter. This is used to force dependencies of a test to run. For
-- example, if test @A@ depends on test @B@ and test @A@ is selected to run, test
-- @B@ will be forced to match. Note that this only applies to dependencies
-- specified using 'sequentialTestGroup'.
-- specified using 'dependentTestGroup'.
type ForceTestMatch = Any

-- | Fold a test tree into a single value.
Expand Down Expand Up @@ -544,7 +561,7 @@ foldTestTree0 empty (TreeFold fTest fGroup fResource fAfter) opts0 tree0 =
AnnWithResource opts res0 tree -> fResource opts res0 $ \res -> go (tree res)
AnnAfter opts deptype dep tree -> fAfter opts deptype dep (go tree)

-- | 'TestTree' with arbitrary annotations, e. g., evaluated 'OptionSet'.
-- | 'TestTree' with arbitrary annotations, e. g., evaluated t'OptionSet'.
data AnnTestTree ann
= AnnEmptyTestTree
-- ^ Just an empty test tree (e. g., when everything has been filtered out).
Expand Down
6 changes: 3 additions & 3 deletions core/Test/Tasty/Ingredients.hs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ import Control.Concurrent.Async (concurrently)
-- which options it cares about, so that those options are presented to
-- the user if the ingredient is included in the test suite.
--
-- An ingredient can choose, typically based on the 'OptionSet', whether to
-- An ingredient can choose, typically based on the t'OptionSet', whether to
-- run. That's what the 'Maybe' is for. The first ingredient that agreed to
-- run does its work, and the remaining ingredients are ignored. Thus, the
-- order in which you arrange the ingredients may matter.
Expand Down Expand Up @@ -87,7 +87,7 @@ data Ingredient

-- | Try to run an 'Ingredient'.
--
-- If the ingredient refuses to run (usually based on the 'OptionSet'),
-- If the ingredient refuses to run (usually based on the t'OptionSet'),
-- the function returns 'Nothing'.
--
-- For a 'TestReporter', this function automatically starts running the
Expand Down Expand Up @@ -117,7 +117,7 @@ tryIngredients ins opts' tree' =
--
-- Note that this isn't the same as simply pattern-matching on
-- 'Ingredient'. E.g. options for a 'TestReporter' automatically include
-- 'NumThreads'.
-- t'NumThreads'.
--
-- @since 0.4
ingredientOptions :: Ingredient -> [OptionDescription]
Expand Down
6 changes: 3 additions & 3 deletions core/Test/Tasty/Ingredients/ConsoleReporter.hs
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ terminalWidth = unsafePerformIO $ do
approxMaxResultShortDescriptionWidth :: Int
approxMaxResultShortDescriptionWidth = 20

-- | Build the 'TestOutput' for a 'TestTree' and 'OptionSet'. The @colors@
-- | Build the 'TestOutput' for a 'TestTree' and t'OptionSet'. The @colors@
-- ImplicitParam controls whether the output is colored.
--
-- @since 0.11.3
Expand Down Expand Up @@ -276,7 +276,7 @@ foldTestOutput
:: Monoid b
=> (String -> IO () -> IO Result -> (Result -> IO ()) -> b)
-- ^ Eliminator for test cases. The @IO ()@ prints the testname. The
-- @IO Result@ blocks until the test is finished, returning it's 'Result'.
-- @IO Result@ blocks until the test is finished, returning it's t'Result'.
-- The @Result -> IO ()@ function prints the formatted output.
-> (String -> IO () -> b -> b)
-- ^ Eliminator for test groups. The @IO ()@ prints the test group's name.
Expand Down Expand Up @@ -420,7 +420,7 @@ instance Monoid Statistics where
mappend = (Sem.<>)
#endif

-- | @computeStatistics@ computes a summary 'Statistics' for
-- | @computeStatistics@ computes a summary t'Statistics' for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, learned myself something new.
Yet the t qualifier is default, so not really needed here:
From: https://haskell-haddock.readthedocs.io/latest/markup.html#hyperlinking-and-re-exported-entities

Since values and types live in different namespaces in Haskell, it is possible for a reference such as 'X' to be ambiguous. In such a case, Haddock defaults to pointing to the type. The ambiguity can be overcome by explicitly specifying a namespace, by way of a v (for value) or t (for type) immediately before the link

But of course, does not hurt.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really know why but without t'...' cabal haddock keeps spamming warnings about "ambiguity", very annoyingly.

-- a given state of the 'StatusMap'.
-- Useful in combination with 'printStatistics'.
--
Expand Down
2 changes: 1 addition & 1 deletion core/Test/Tasty/Options.hs
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ lookupOption (OptionSet s) =
changeOption :: forall v . IsOption v => (v -> v) -> OptionSet -> OptionSet
changeOption f s = setOption (f $ lookupOption s) s

-- | Create a singleton 'OptionSet'.
-- | Create a singleton t'OptionSet'.
--
-- @since 0.8
singleOption :: IsOption v => v -> OptionSet
Expand Down
8 changes: 4 additions & 4 deletions core/Test/Tasty/Options/Core.hs
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ data Timeout
| NoTimeout
deriving
( Eq
-- ^ Auto-derived instance, just to allow storing in a 'Map' and such.
-- ^ Auto-derived instance, just to allow storing in a 'Data.Map.Map' and such.
--
-- @since 1.5.1
, Ord
-- ^ Auto-derived instance, just to allow storing in a 'Map' and such.
-- ^ Auto-derived instance, just to allow storing in a 'Data.Map.Map' and such.
--
-- @since 1.5.1
, Show
Expand Down Expand Up @@ -100,7 +100,7 @@ parseDuration str =
_ -> Nothing
_ -> Nothing

-- | A shortcut for creating 'Timeout' values.
-- | A shortcut for creating v'Timeout' values.
--
-- @since 0.8
mkTimeout
Expand All @@ -127,7 +127,7 @@ instance IsOption HideProgress where

-- | The list of all core options, i.e. the options not specific to any
-- provider or ingredient, but to tasty itself. Currently contains
-- 'TestPattern' and 'Timeout'.
-- t'TestPattern', t'Timeout' and t'HideProgress'.
--
-- @since 0.1
coreOptions :: [OptionDescription]
Expand Down
2 changes: 1 addition & 1 deletion core/Test/Tasty/Parallel.hs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import Control.Concurrent.Async
import Control.Concurrent.STM
import Foreign.StablePtr

-- | What to do about an 'Action'?
-- | What to do about an t'Action'?
data ActionStatus
= ActionReady
-- ^ the action is ready to be executed
Expand Down
2 changes: 1 addition & 1 deletion core/Test/Tasty/Patterns/Parser.hs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import Test.Tasty.Patterns.Expr

type Token = ReadP

-- | A separate 'Parser' data type ensures that we don't forget to skip
-- | A separate t'Parser' data type ensures that we don't forget to skip
-- spaces.
--
-- @since 1.0
Expand Down
6 changes: 3 additions & 3 deletions core/Test/Tasty/Providers.hs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import Test.Tasty.Providers.ConsoleFormat (ResultDetailsPrinter, noResultDetails
singleTest :: IsTest t => TestName -> t -> TestTree
singleTest = SingleTest

-- | 'Result' of a passed test.
-- | t'Result' of a passed test.
--
-- @since 0.8
testPassed
Expand All @@ -37,7 +37,7 @@ testPassed desc = Result
, resultDetailsPrinter = noResultDetails
}

-- | 'Result' of a failed test.
-- | t'Result' of a failed test.
--
-- @since 0.8
testFailed
Expand All @@ -51,7 +51,7 @@ testFailed desc = Result
, resultDetailsPrinter = noResultDetails
}

-- | 'Result' of a failed test with custom details printer
-- | t'Result' of a failed test with custom details printer
--
-- @since 1.3.1
testFailedDetails
Expand Down
20 changes: 10 additions & 10 deletions core/Test/Tasty/Run.hs
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ instance Show DependencySpec where
data Dependency = Dependency DependencyType DependencySpec
deriving (Eq, Show)

-- | Is given 'Dependency' a dependency that was introduced with 'After'?
-- | Is given t'Dependency' a dependency that was introduced with 'After'?
isPatternDependency :: Dependency -> Bool
isPatternDependency (Dependency _ (PatternDep {})) = True
isPatternDependency _ = False
Expand All @@ -300,7 +300,7 @@ mapAccumM f acc (x:xs) = do
-- | An action with meta information
data TestAction act = TestAction
{ testAction :: act
-- ^ Some action, typically 'UnresolvedAction', 'ResolvedAction', or 'Action'.
-- ^ Some action, typically 'UnresolvedAction', 'ResolvedAction', or t'Action'.
, testPath :: Path
-- ^ Path pointing to this action (a series of group names + a test name)
, testDeps :: Seq Dependency
Expand All @@ -321,8 +321,8 @@ type ResolvedAction = IO ()
type Size = Int

-- | Simplified version of 'TestTree' that only includes the tests to be run (as
-- a 'TestAction') and the resources needed to run them (as 'Initializer's and
-- 'Finalizer's).
-- a t'TestAction') and the resources needed to run them (as t'Initializer's and
-- t'Finalizer's).
data TestActionTree act
= TResource Initializer Finalizer (TestActionTree act)
| TGroup Size [TestActionTree act]
Expand Down Expand Up @@ -419,8 +419,8 @@ createTestActions opts0 tree = do
Independent NonParallel -> foldSequential AllFinish trees
Dependent depType -> foldSequential depType trees

foldSequential :: DependencyType -> [Tr] -> ReaderT (Path, Seq Dependency) IO [TestActionTree UnresolvedAction]
foldSequential depType =
foldSequential :: DependencyType -> [Tr] -> ReaderT (Path, Seq Dependency) IO [TestActionTree UnresolvedAction]
foldSequential depType =
fmap snd . mapAccumM (goSeqGroup depType) mempty

-- * Utility functions
Expand Down Expand Up @@ -576,7 +576,7 @@ destroyResource restore (Finalizer doRelease stateVar _) = join . atomically $ d
FailedToCreate {} -> return $ return Nothing
Destroyed -> return $ return Nothing

-- While tasty allows to configure 'OptionSet' at any level of test tree,
-- While tasty allows to configure t'OptionSet' at any level of test tree,
-- it often has any effect only on options of test providers (class IsTest).
-- But test runners and reporters typically only look into the OptionSet
-- they were given as an argument. This is not unreasonable: e. g., if an option
Expand All @@ -585,9 +585,9 @@ destroyResource restore (Finalizer doRelease stateVar _) = join . atomically $ d
-- a global option, without passing it via command line.
--
-- 'applyTopLevelPlusTestOptions' allows for a compromise: unwrap top-level
-- 'PlusTestOptions' from the 'TestTree' and apply them to the 'OptionSet'
-- 'PlusTestOptions' from the 'TestTree' and apply them to the t'OptionSet'
-- from command line. This way a user can wrap their tests in
-- 'adjustOption' / 'localOption' forcing, for instance, 'NumThreads' to 1.
-- 'adjustOption' / 'localOption' forcing, for instance, t'NumThreads' to 1.
--
-- This function is not publicly exposed.
applyTopLevelPlusTestOptions
Expand All @@ -606,7 +606,7 @@ applyTopLevelPlusTestOptions opts tree = (opts, tree)
--
-- Once the callback returns, stop running the tests.
--
-- The number of test running threads is determined by the 'NumThreads'
-- The number of test running threads is determined by the t'NumThreads'
-- option.
--
-- @since 0.10
Expand Down
Loading
Loading