Add tests

[joeyespo]

There's a growing need to be sure the CLI arguments all play nicely together, that it's cross-platform, and runs on Python 2.7 and 3.4+.

Funny enough, I'm not exactly sure how to approach it here.

Ideas?

[aldanor]

Now that we switched to using pytest.main, we can mock/monkeypatch it?

[aldanor]

(Btw I would also refactor the code so that you don't have to pass a gazillion of parameters twice, first from main to watch and then to ChangeHandler, the watch itself could be a handler's method, why not?)

[aldanor]

I would use pytest/tox for testing and set up builds on travis and appveyor -- this way we'll test it on win/linux for all interpreter versions.

[joeyespo]

Now that we switched to using pytest.main, we can mock/monkeypatch it

Interesting. Maybe. If it's our own custom monkey-patching code, it seems like a big place to make mistakes. I think integration tests are going to find many more bugs than unit tests in our case. We don't have too many functions to tests (but by all means, we should still test those).

For making sure that ptw --poll -- myfile still works though, it might make more sense not to try to go around py.test (since it's error prone), but use it in the tests so we're guaranteed that we haven't broken anything. Something that runs ptw top-level with a timeout and observes STDOUT.

Then we should also add tests for multiple ptw / py.test version combinations. So the combinations would be ptw version, py.test version, Linux | Mac | Windows, Python 2.7 | 3.2? | 3.3? | 3.4 | 3.5. (Let's only include earlier versions of 3 if we already support it :-)

What do you think?

Btw I would also refactor the code so that you don't have to pass a gazillion of parameters twice, first from main to watch and then to ChangeHandler, the watch itself could be a handler's method, why not?

Yeah, agreed. It's time to refactor.

[bendtherules]

What if we just mock pytest.main as @aldanor said and check whether the parameters provided are correct? Good thing about mocking, we dont need much of our own code to prepare the mock.

With that said, our subprocess call to py.test probably wont pick up the mocked pytest.main, we need to subprocess call something like our mocked py.test-mock executable, by setting -runner

Also, some way to run pytest main loop just once instead of the infinite watching loop, using timeouts always is not desirable.

[apast]

Hello,

can I proceed with some tests over main and parameters parsing?

I want to try some tests on #65, but before I need some basic tests coverage to keep current plugin funcionality.

Do you have some WIP on this issue or can I start here from zero?

Initially, I aim to test only for Linux + python 3+.

Thanks.

[bendtherules]

Maybe take a look at #37 ?
Added this small config tests long back, so please feel free to take over.

Short note regarding the PR:

1. Please ignore test_works_directories_ini_option check - We decided to not do it - Allow setting <directories> argument from pytest.ini #41 .
2. This only tests the config - doesn't ensure that ptw actually works with pytest - as mentioned above (Add tests #14 (comment))

Completely out of context now, but @joeyespo can suggest other things we need to fix there,

[apast]

Good!

I started from the basics before any step forward.

Initial test cases are being applied to CLI arguments, aiming to cover all pytest_watch.command file.

I agree with and am following #14 comment about focus on ptw functional, before integration. It sounds the best for the moment.