Byteman extension is incomplete

[rachmatowicz]

Is anyone planning to finish this project? Because at the moment, its only about 40% complete.

As far as I can tell, this only implements a small subset of Byteman's configurable features, namely the annotations BMRules and BMRule. What about BMConfig, BMScript, and other means of configuring the agent? Also, there is precious little in the way of documenting how to use it: for example, in a test involving a client and two servers, how to direct some rules to install on one server or client versus another.

Byteman is used successfully in a number of projects, but always via the BMUnit test runner - which works out of the box. In the Wildfly testsuite, which is based on ARQ, nobody's using it, and i'm guessing that the reason is the state of this extension.

Is anyone planning on addressing this? It's a pretty serious limitation of our ability to test when we are dependent on ARQ as a test runner.

[bartoszmajsak]

Thanks for opening the issue. Could you please help me understand what are the important features you need so we can plan and prioritize the improvements? Are there any workarounds which we could document before the desired capabilities will land in this extension?

[rachmatowicz]

Hi Bartosz
I believe I listed many of the missing features in my original post. Whatever annotations are available for BMUnitRunnner should also be available in this extension - in other words, if annotations represent configurable features of Byteman for JUnit and or testNG tests, they should be available from this extension. You can add BMScripts to this list but there may be more. And providing the same features via a different means is not a good idea for compatability purposes.

But beyond that, because ARQ allows managed/unmanaged deployment of multiple servers in a test case, it should be able to cater to installing specific rules on specific servers (e.g. rule X to be installed on server 1 and rule Y to be installed on server 2). This may already exist via ExecType, but I haven't seen it documented.

The other problem I found and spent a lot of time trying to debug was the fact that you re-define the Byteman annotations in an arquillian-byteman-extension package. The auto-completion on my IDE pulled in the original Byteman annotations so that the test case had annotations from org.jboss.byteman.contrib.bmunit.BMRule and the extension was looking for org.jboss.arquillian.extension.byteman.api.BMRule. My rules were not working and it took me a lot of time to discover why. You should either use the original Byteman annotations or provide warnings when they are seen by the extension.

In terms of priority, getting @bmscript and @BMConfig implemented would be top on my list. People like me usually use these to define collections of rules in the test and to indicate where they can be found.

That's about all I can think of for now.