Let's say I have the following in my project.clj:
...
:eval-in :leiningen
:profiles {:midje {:dependencies [[midje "1.5.1"]]
:plugins [[lein-midje "3.1.3-RC2"]]}}
...
No tests are defined so far, so midje should not encounter any failures. I call:
$ lein with-profile midje midje
Performing task 'midje' with profile(s): 'midje'
No facts were checked. Is that what you wanted?
Error encountered performing task 'midje' with profile(s): 'midje'
Suppressed exit
Exit code is 1, things like TravisCI will fail. Note that this works fine for non-:eval-in-leiningen projects, the fact that leiningen.core.main/exit is called even if no failure is encountered just breaks testing of, e.g., plugin projects in that manner.