Running trial tests within Twisted
==================================

Trial tests are used to test in some circumstances the integration between
Zope and Twisted. The standard Zope test suites don't work since a lot of the
methods we are testing require use to return deferred objects. These deferred
objects contain callbacks and error callbacks which we need to test.

In order to set up the test suite correctly in order to test the deferreds
we need our test suite to extend the twisted.trial.unittest.TestCase object.
Then our setup methods must call the zope.testing.trial.setUp method and our
teardown method must call the zope.testing.trial.tearDown methods.

Our first test we will rerun a test suite which has 1 failing test, 1 test
which contains an error and a successful test.

    >>> import os.path, sys
    >>> from zope.testing import testrunner
    >>> defaults = [
    ...     '--path', this_directory,
    ...     '--tests-pattern', '^trialtest$',
    ...     ]

    >>> sys.argv = 'test --tests-pattern ^trialtest$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running unit tests:
    <BLANKLINE>
    Error in test test_error (trialtest.TestTrialTests)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", Line NNN, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtest.py", Line NNN, in test_error
        raise Exception, "this test is a broken trial test :-)"
    exceptions.Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtest.TestTrialTests)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", Line NNN, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtest.py", Line NNN, in test_failure
        self.assert_(False, "I am a failed trial test")
    twisted.trial.unittest.FailTest: I am a failed trial test
    <BLANKLINE>
      Ran 3 tests with 1 failures and 1 errors in N.NNN seconds.
    True

Here is the output from the same tests above but the setup method contains
errors.

    >>> sys.argv = 'test --tests-pattern ^trialtestfs$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running unit tests:
    <BLANKLINE>
    Error in test test_assert_ok (trialtestfs.TestTrialBrokenSetup)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestfs.py", line 28, in setUp
        raise Exception, 'broken setup'
    exceptions.Exception: broken setup
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestfs.TestTrialBrokenSetup)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestfs.py", line 28, in setUp
        raise Exception, 'broken setup'
    exceptions.Exception: broken setup
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_failure (trialtestfs.TestTrialBrokenSetup)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestfs.py", line 28, in setUp
        raise Exception, 'broken setup'
    exceptions.Exception: broken setup
    <BLANKLINE>
      Ran 3 tests with 0 failures and 3 errors in N.NNN seconds.
    True

Here is the output from the same tests but the teardown method contains
errors.

    >>> sys.argv = 'test --tests-pattern ^trialtestft$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running unit tests:
    <BLANKLINE>
    Error in test test_assert_ok (trialtestft.TestTrialBrokenTeardown)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestft.py", line 30, in tearDown
        raise Exception, 'broken tearDown'
    exceptions.Exception: broken tearDown
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestft.TestTrialBrokenTeardown)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestft.py", line 33, in test_error
        raise Exception, "this test is a broken trial test :-)"
    exceptions.Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestft.TestTrialBrokenTeardown)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestft.py", line 30, in tearDown
        raise Exception, 'broken tearDown'
    exceptions.Exception: broken tearDown
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtestft.TestTrialBrokenTeardown)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestft.py", line 36, in test_failure
        self.assert_(False, "I am a failed trial test")
      File "...twisted/trial/unittest.py", line 223, in failUnless
        raise self.failureException(msg)
    twisted.trial.unittest.FailTest: I am a failed trial test
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_failure (trialtestft.TestTrialBrokenTeardown)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", line 107, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtestft.py", line 30, in tearDown
        raise Exception, 'broken tearDown'
    exceptions.Exception: broken tearDown
    <BLANKLINE>
      Ran 3 tests with 1 failures and 4 errors in 0.118 seconds.
    True

Make sure that errors are handled correctly if the postCaseCleanup method
fails for what ever reason.

    >>> import twisted.trial.util
    >>> old_doCleanReactor = twisted.trial.util._Janitor.do_cleanReactor
    >>> def new_doCleanReactor(cls):
    ...     raise Exception, "Can't clean the reactor"
    ...
    >>> twisted.trial.util._Janitor.do_cleanReactor = new_doCleanReactor

    >>> sys.argv = 'test --tests-pattern ^trialtest$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running unit tests:
    <BLANKLINE>
    Error in test test_error (trialtest.TestTrialTests)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", Line NNN, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtest.py", Line NNN, in test_error
        raise Exception, "this test is a broken trial test :-)"
    exceptions.Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtest.TestTrialTests)
    Traceback (most recent call last):
    ...
    --- <exception caught here> ---
      File "...twisted/internet/defer.py", Line NNN, in maybeDeferred
        result = f(*args, **kw)
      File "...app/twisted/ftp/tests/trialtest.py", Line NNN, in test_failure
        self.assert_(False, "I am a failed trial test")
    twisted.trial.unittest.FailTest: I am a failed trial test
    <BLANKLINE>
      Ran 3 tests with 1 failures and 1 errors in N.NNN seconds.
    True

Now un-patch the twisted trial janitor, so that it does actually clean the
reactor.

    >>> twisted.trial.util._Janitor.do_cleanReactor = old_doCleanReactor
