Ship with multiple default rulesets

[Lakitna]

During lint rule severity discussions, we often talk about different kind of users. For example: A rule should maybe be error for an intermediate user bug warn for a beginner.

What do you think about shipping with multiple predefined rulesets? For example:

• Default
• Strict
• Loose

Alternatively, maybe a better way to grab a ruleset from a pypi package. This would, for example, allow me to publish robocop-ruleset-super-duper-strict. This is a similar idea to Eslint. Example config: https://github.com/google/eslint-config-google

Note: This is only about configuring lint and format rules. This does not include new rules or new formatters.

Background: In Slack we were discussing the potential need for highly opinionated docs/tools/configs. This is a way that we can achieve highly opinionated Robocop config.

[bhirsz]

I remembered that there was similar issue, now I found it (in the temporary repository created during migration of the tools):

bhirsz/robotframework-cop-academy#74

I agree that we can introduce rule sets and it's good idea.

[Lakitna]

I did remember correctly! I just couldn't find it :(

[bhirsz]

Same, I spent some time searching because I remembered it.. I thought I was going crazy ;)

As for grabing rulesets, it would be possible to use plugin-style approach some of the PyPi packages are using - where plugin uses the same structure as Robocop, so any extra files are 'added' to our internal directory when installing packages.

So let's say we would have this structure:

src/robocop/linter/rulesets

Then Pypi packages would have:

src/robocop/linter/rulesets/some_ruleset.toml

It would be automatically loaded by Robocop. It can be already done if you want custom rules to appear as 'internal' Robocop rules (and do not use custom rules option) since Robocop loads rules and reports dynamically.

Obiously we would also add option to loads them from cli, but having the installation option is useful to ensure that everyone in the organization actually use it.

[Lakitna]

That sounds like a good solution! It would definitely work for this.

It may get messy if you're using multiple plugins 🤔. Is there a user controllable load order concept? Not sure yet if that's an issue in the real world anytime soon though.

I do think that some file-based configuration is a must here. If you can only do this from CLI it will, like you said, just not be used by a lot of people. A file-based config could also be in pypoject.toml though.

[bhirsz]

I would get messy if someone used the same directory / file names (ie. the same plugin name). Though it may not be the issue right now so I wouldn't worry about it. It would be possible to add load order, we have similar features (ie. we can force order of formatters).

If we do it in Eslint style (requiring to use extends) then we would enforce order by default. Such rule sets could be also good idea to contribute together with company custom rules (to provide default configuration for them etc). That way it would be also easy to enable/disable them (other than using custom rules) :

extends = ["your_company"]

So basically rule sets would be 'installable configuration files' under disguise :)

[Lakitna]

As long as we keep the load order caveat in mind, this should work great.

For completeness sake: How do you feel about including rulesets/configs in Robocop itself?

[bhirsz]

I think we should offer some selection of rulesets - for example see linked issue about Context7, where it was proposed to have less strict rules for AI use (so it doesn't spend time on fixing less important stuff). So there are genuine use cases to include them in Robocop itself.

However we should still offer loading them from external sources, as there will be custom / company owned rule sets etc we don't have control over.

[Lakitna]

Sounds good. So that's essentially 2 functionalities (though they could be 1 technical implementation):

1. Allow users to use externally hosted default configs (most likely: A PyPI package)
2. Allow users to use different Robocop configs

Do have good ideas on how to implement it yet? I'd have to dive deep into the config code first to come up with anything..

[bhirsz]

Yes, I have. For me it's actually one technical implementation and 1 main functionality: allow to extend configuration files (using internal / external sources). Then some of the features like using externally hosted configs is more of documentation work than technical since if we use Pypi package approach with installing in Robocop directory directly.

To summarize design:

New extends configuration option that allows to extend current configuration. User can select Robocop default configuration files (that may offer for example pre-selected or/and pre-configured list of rules/formatters) or can select external configuration files if he/she install it.
Installing configuration file is possible using Pypi package and plugin architecture: https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/ . We will need to document how to prepare such repository - possibly offer template that can be copied over and reused.

extends will also support relative/absolute physical paths (if someone wishes to use other configuration file directly).

Example usages:

[tool.robocop]
extends = ["MyCompany",  "strict", "..\config\common_config.toml"]

[Lakitna]

Looks great!

One potential addition. It may be a good idea to scope the config names of those that ship with Robocop. For example:

[tool.robocop]
extends = ["MyCompany", "robocop:strict", "../config/common_config.toml"]

[bhirsz]

I have developed extends options which will be deployed in 6.12.0. Separetely I will add an option to register robocop plugins (for externally hosted configs) but for now the extends will support physical paths to toml files.

Technically we will be able to finish this issue but I will need to think what's considered as strict / loose configuration etc (unless we could just ship such configuration files on later date).

[Lakitna]

Awesome! Very quick too 🚀🚀

I'm good with either closing this one and making new issues or keeping it open. Whatever works best for you ;)

I'm at a conference this week, so I can't make configs atm. However, I can share my current config which would be a good starting point for what I think a strict preset should look like.. I should be able to share that tonight or tomorrow.

[bhirsz]

Keeping it open is fine (since it's about different rulesets). We don't need to hurry and we can go over possible configs and adjust details. The design is modular so we will be able to keep adding those configs if needed on later date.

For loose configs, I can go over existing INFO rules (as potential candidates to be disabled). Missing documentation rules can also be disabled.