Ignore index license restrictions when running tests against in-memory instance

[lahma]

As far as I know running in memory is no way a production feature and used for testing. It would be nice to ignore license restrictions when you run integration tests against embedded server and want to run the in highly parallel manner.

Now when running integration test suite against RavenDB 6.0 with parallelism will cause cluster index limits to be reached - not a very good cluster when in-memory anyway.

Raven.Client.Exceptions.Commercial.LicenseLimitException: The maximum number of static indexes per cluster cannot exceed the limit of: 60

[ayende]

Is there a reason you are not using a developer license for that purpose?

[lahma]

OK thanks for the clarification. Then it would make sense for everyone and every pre-production installation to just use that same automatically refreshed license to keep things simple and not to require manual intervention.

[SbiCA]

I'm somehow not following on the approach here but we're struggling with the same issue, currently updated the tests and now our CI pipeline starts to fail and I'm wondering if it's in general a good idea to put a license into a CI job which runs on anyone devs local machine as well since we're "just" using the RavenTestDriver to make sure all still works with end to end. so in our case we have the following use cases

Local dev: runs tests can use personalized license but how to put that into the test driver
CI (Az devops): would run tests on behalf of ??? since it would require a license as well 🤷

[lahma]

So far seems like a good idea to stay on 5.4 as it so much easier to work with developer-experience-wise. Offloading some data/queries to another system like PostgreSQL+Marten can also be helpful when you want to minimize index usage in CI flow (as silly as it may sound when it's not a production usage problem).

[ayende]

@SbiCA For reference, you can register a license in the RavenTestDriver using:

RavenTestDriver.ConfigureServer(new TestServerOptions
{
    CommandLineArgs = new List<string>
    {
        "--License.Path=/path/to/license.json"
    }
});

In your startup

[DejanMilicic]

We just removed limitation on number of static and auto indexes #18564

[mcrio]

Hi, I'm not sure what's the best approach handling the TestDriver license.

Let's say we have a CI pipeline that runs the tests and there are multiple developers working on the project.

So we'd register one developer license and hard code it so anyone checking out the repository would be able to run the tests? But that license is tied to an email address so I guess it's better not to expose it. When renewing it programmatically, is the license code going to stay the same?... it's an open source project on GitHub

[ayende]

The easiest way is to handle that as a secret in the CI pipeline. Then you just register that with an env variable, etc.
You can keep the old license (it will auto update as needed)

[mcrio]

I hope some day you'll find a way to not require a license for using the RavenDbTestDriver.

It's not very flexible when different people might be changing code and want to run the tests locally. They all need to register a license first, follow the onboarding docs and assign an environment variable, renew the license when it expires etc, and the case might be they never touched the persistence layer but updated some unrelated code. It's ok, we'll do it that way, but it's just that it introduces more things having to take care of.

I remember when I first tried out RavenDB, being able to use the embedded server as part of the test run was (and is) a killer feature.