Extensions: Provide a means for extensions to add and implement new Locator types

[jeremybramwell]

I'm not very familiar with the exact restrictions imposed by the browsers and the current plans for how extensions that add/modify selenium IDE functionality are going to work. So my thoughts below make a lot of likely incorrect assumptions.

Goal: Allow "Add on" WebExtensions to add new Locators to the list of locators that selenium IDE uses to convert an element to an "address" when recording and from an address to an element during playback.

Requirements:

• Provide a means for a separate WebExtention (Add-on Extension) to register additional Locators (LocatorStrategies) with Selenium IDE.
  • This might be accomplished by accepting something like an "AddLocator" message type that the add-on can send to selenium IDE.
  • If there is a more general mechanism for an add-on to register itself with selenium IDE then a list of the locators the add-on provides might be included in that registration.
  • Registration should allow for multiple Locators to be added by an add-on.
  • Registration might also allow for a display name and description to be provided for each Locator added so that it can be displayed in the locator ordering list and in the dropdown selector when editing a recorded step.
• Allow users to change the order of added Locators along with the built-in Locators.
• Allow users to enable/disable Locators (both built-in and add-on).
• Deal with the case where an add-on Extension that adds Locators is removed or disabled.
• During recording Selenium IDE would send a message to the add-on with the id of the Locator to resolve and the element to generate an "address" for. The recorder would then wait for all locators to be resolved. A timeout, which might be configurable, would limit how long Selenium IDE would wait for add-on locators to be resolved.
• During playback Selenium IDE would send a message to the add-on with the id of the Locator to resolve and the "address" to resolve into an element in the current DOM. Selenium IDE would then wait for a reply message from the add-on which would contain the resolved element.

Use Case:
Our company has a very complex single page application and would like to use Selenium IDE to allow our non-programmer testers to generate test scripts. Our application generates unique element ids and classes per session which prevents any locators using those attributes to generate scripts that can be replayed. Other locators like xpath are prone to break when the application is enhanced or dictionaries that our customers use to customize the application are modified. So we have built custom logical identifiers into the DOM elements that allow for robust addressing of important logical elements. We implemented this capability in the prior version of Selenium IDE with a corresponding plugin for selenium (not the IDE) for consuming scripts which use this specialized locator.

[corevo]

As part of #79, this would be a great use case for it, and would gladly add that ability,
The problem is that we need to think of a fallback mechanism, in case the plugin is not available.
I can see two solutions for this

• Selenium IDE will send the plugin the best locator is has found, and if your plugin can find a better one, you can override it. This way we keep the single locator mechanism, but the issue is when we have 2 plugins, then Selenium IDE can't decide which is better (unless the user can rearrange the locator list). We can keep aside the original locator for fallback.
• Selenium IDE will be able to keep multiple locators, then we can try all of them according to the user's order, this way multiple plugins can offer better locators. The problem here, is that manual editing can become a chore, the table becomes a lot less readable.

In any case, Selenium IDE will send that locator to the respective plugin, which will then return an address to that element.

[gwtuser]

Being able to create custom locators by looking at the DOM element and interacting with the JavaScript on the target page is a big part of what our extensions do. But they also:

• Override the behavior of the click() action as it's played back (in the old Selenium IDE, via overriding Selenium.prototype.doClick()) to execute custom JavaScript. (That's just an example of course - many other APIs are also overriden.)
• Capture events, such as mouseDown, during recording of scripts, to customize what's recorded.

I explained our situation in a bit more depth in the issue that was closed above (#184), but basically, we've written a Java wrapper around the WebDriver JARs which handles our locators and we tell our customers to use that when writing Java, and we also wrote an interpreter for Selenium v1 which executes Selenese against our WebDriver wrapper layer. However, for Selenium IDE, since it never used the Selenium JARs we didn't take any action and assumed it would continue to run Selenese with our user extensions.

If support for the two actions I list above - customizing click() (for example) during playback and running special code for events like mouseDown during recording - won't be supported, we'll have to tell our customers to stop using Selenium IDE, or just use it to get locators that they'll put in the Java code. (Customers who will be writing Java to call our Java wrapper already use it like that, but legacy customers want to be able to continue writing Selenese v1 scripts which will only work with the full functionality of our user extensions.)

[corevo]

@gwtuser I have a bit of a problem with your suggestions.
Firstly it is forbidden to inject code into the extension, meaning when we submit Selenium IDE to Mozilla, they sign the very specific version of Selenium.prototype.doClick that we've submitted.
If you override that, it means that this signature is no longer valid, while you have good intentions, it can be used to leverage malicious intents, such as keyloggers and the likes.
What is possible in terms of extension policies is extending the logic, rather than overriding it, it's a bit different, it means that will the plugins support, you'd be able to execute your own logic, but within your own extension, without tampering with the IDE's code (no monkey patching).
That being said, there is a logistical issue with overriding the default IDE behavior, say you write an extension that wants to re-implement click, the user installs a second extension that re-implements it, which one will handle the click?
It can be both of them, because it's a mutating event, unlike assertions that I wouldn't mind calling twice, the best argument for that case, is that most likely only one extension will do it.
But still, if you can find a way to be more flexible, rather than monkey patching the entire IDE, that could create a paradigm that is easier to extend upon, for all plugins.

[gwtuser]

I don't think there's agreement that Mozilla's decision was right with regard to development tools in a work environment. It seems like an overreaction to throw everyone under the bus - including power users and degreed engineers - just to ensure that the typical Firefox user can't hurt himself by being careless with unsigned extensions. I understand that you have to deal with the situation Mozilla has created, but you don't have to agree with it.

For the technical point, it sounds like you're just suggesting that our user extensions be implemented as a separate plugin layer on top of Selenium IDE (or perhaps a separate FF extension?), so that Selenium IDE's internal state can't be modified (though readable I assume)?

Having to refactor our extensions to use with Selenium IDE would obviously be better than having no support at all - but I'm not sure I understand the problem you're alluding to. If the user installs our plugin to Selenium IDE (if that's how it would work), obviously, they're trusting us to decide whether to make the super method call into your code or not (sometimes we don't). In answer to question about the click - obviously if our extension plugged into Firefox directly and not some API your extension provided, there'd have to be a way to give it priority (or "first crack") at handling any event before Selenium IDE gets a chance, and a way of forwarding the event to Selenium IDE.

In short, if there were a way to extend the event handlers Selenium IDE provides, via a Selenium IDE plugin (rather than a separate FF extension let's say), then that would seem to be a good modular design, and would work for us as long as that plugin got to decide whether to call the underlying Selenium IDE handler.

Or, are you saying that plugins into signed extensions are now completely prohibited? If so, since Selenium IDE is open source, I assume it would be possible to submit "releases" of our user extensions overriding Selenium IDE's code, bundled together as "our" signed Firefox extension to the Selenium IDE (subject to GPL license, or what not)? That would be painful, but perhaps the only path?

[corevo]

First of all, Mozilla in their mind aren't throwing anyone under the bus, we have to comply with new regulations, designed to increase users' trust and to protect their privacy, which I very much support in that sense.
Secondly, I do have to agree with what they ask of me, or else the extension will simply not be there, as a side note, Chrome disallows these kind of patching as well, they just don't enforce it.

Putting that aside I do suggest you'll write your own extension, that will work side-by-side with Selenium IDE, I'm working on this extension protocol right now.

Your argument to allow this kind of behavior is that, if a user willingly installs you, he trusts you to do the right thing, that is not correct, but this is a different discussion, if it was correct phishing wouldn't work.

Anyway back on topic, with this protocol release each command is unique, for example click can only be handled by one handler and that's Selenium IDE, and the reason, is I both don't know how to prioritize if multiple plugins override the same behavior, and that code has to be ran in command line, which one will yield the right code?

Summing up my thoughts,
What is currently prohibited, is evaling anything inside a background script, inside content script is highly discouraged, unless there is no other way, and the only way that is fine is with sandboxes.
In order to properly override the IDE's behavior that would require eval at some elevated code blocks that could create problems.
And the reason is not you specifically, if someone were to write rogue extension, that wouldn't ask for any permissions, it would seem harmless, but then inject code that steals info through Selenium IDE, because Selenium IDE, has fairly wide permissions over the users' browser.
The way browser vendors go to extending extensions behavior is through cross extension messaging which is what we're implementing atm.
At first release default behavior won't be overridden, but that is not the final word on it, and once we find a solution, and time to implement it, we will do that too.
What will be released with the first release is the ability to add new commands to the IDE, then I'll work on adding new locators.

[gwtuser]

I see that plugin support has been added to Selenium IDE- that's great!

To what extent do the new plugin APIs support custom locators and the ability to add new Selenese commands or override existing commands?

[corevo]

I'm going to break this into the 3 requests:

• The new API supports adding new commands to the IDE.
• Adding new locators is in the pipeline, we have stumbled upon 2 issues which makes things harder to accomplish.
• As of this moment, there is no way to override the default IDE behavior.

For more information you can check the plugin wiki.

[gwtuser]

Unfortunately, then, it looks like we're stuck with Selenium 2.9/FF52ESR until support is added to override built in commands such as "click()".

However, there's one possibility, which is if we could somehow translate the script upon load and as the user adds new commands. The WIKI mentions the plugin can "bootstrap setup" but do you think the script itself could be remapped? This would allow us to remap "click" to "ourClick" for example. We'd remap everything to new commands, which is then within supported behavior.

Obviously, scripts could be remapped outside Selenium IDE via perl or whatnot, but that sounds like more trouble than just telling our users to stick with Selenium 2.9/FF52ESR.

[gwtuser]

Just to clarify. I know we've already discussed that the playback of a predefined command can't be altered by a plugin, but looking at the "record api" can I also assume there's no way to prevent, say, "click()" from being recorded by substituting your own command? The "IDE Events" section of the WIKI seems rather brief as I don't see any details of what actual DOM interactions are forwarded to the plugin.

[corevo]

So no events are propagated back to plugins.
I think adding your own click instead of overriding is a great solution.
The question is if I propagate the recording events, so that a plugin can modify them, which plugin "wins", as multiple plugins can listen on the same event, but only one can respond.