Refactoring the lowerQuakeCode out of the QPU

[Renaud-K]

These changes introduce 2 new files in the cudaq-mlir-runtime library. Compiler.h and Compiler.cpp.
These files define a class Compiler class into which lowerQuakeCode and all its dependencies have been moved into from BaseRemoteRESTQPU. For now Compiler reflects the same interface as BaseRemoteRESTQPU to make the move as easy as possible and avoid errors.

At this stage, the compiler is still instantiated in BaseRemoteRESTQPU and essentially

codes = lowerQuakeCode(kernelName, module, rawArgs);
completeLaunchKernel(kernelName, std::move(codes));

becomes

Compiler compiler(...);
codes = compiler.lowerQuakeCode(kernelName, module, rawArgs);
completeLaunchKernel(kernelName, std::move(codes));

which clearly shows the data exchanges between QPUs and Compiler that may still have to be addressed.

Most header file dependencies with MLIR have been removed from BaseRemoteRESTQPU.h and Compiler.h only has forward declaration of MLIR objects.

There is still some work to be done:

1. Address data exchanges between QPUs and Compiler.
2. Move compiler instance out of the QPU.
3. Remove cudaq-mlir-runtime library from the QPU backends.

But I feel, this is a good starting point for the next steps.

[github-actions]

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

[github-actions]

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

[lmondada]

This is a step in the right direction, happy to approve when tests pass. Thanks!

This is currently not used outside of BaseRemoteRESTQPU. I'm interested in your opinions for the next steps:

1. Is your intention that over time other QPUs (or more generally, other execution paths) will use this same class?
2. Do you think this is the right place to manage the jitEngines, or will we move these out at some point?
3. Related to point 2, this Compiler class stores and returns the compiled results as string (in ServerHelper's KernelExecution class), but separately instantiates the jitEngines from the in-memory MLIR modules. If we want to separate the jit engines, we will have to come up with a replacement for KernelExecution (or roundtrip through string...)

[Renaud-K]

_> This is a step in the right direction, happy to approve when tests pass. Thanks!

This is currently not used outside of BaseRemoteRESTQPU. I'm interested in your opinions for the next steps:

1. Is your intention that over time other QPUs (or more generally, other execution paths) will use this same class?_
   Yes. Except for the simulation QPUs all QPUs are already using this class. For simulation, this class can act as passthrough, until we introduce some simulator specific optimization.

> 2. Do you think this is the right place to manage the jitEngines, or will we move these out at some point?
My concern for now is to mimic the current behavior, whereby all jitEngines are cleaned up after a run. They are now cleaned up when the compiler class goes out of scope. This, I think, is a lot better than the explicit clean-ups we had before. Ultimately, I think they will be held in a cache a bit like the JITExecutionCache.

> 3. Related to point 2, this Compiler class stores and returns the compiled results as string (in ServerHelper's KernelExecution class), but separately instantiates the jitEngines from the in-memory MLIR modules. If we want to separate the jit engines, we will have to come up with a replacement for KernelExecution (or roundtrip through string...)
ServerHelper.h itself is an odd place to host the KernelExecution or any other data exchange format between compiler and QPU but the KernelExecution is the most convenient place for now. Also note that the compiler is setting up every field of the KernelExecution so using something else could mean moving code out of the compiler. Nothing is set in stone here and this is only a first rough cut.

[github-actions]

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

[sacpis]

LGTM. Thanks @Renaud-K. Left a comment.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Renaud-K]

/ok to test 29e57da

Command Bot: Processing...

[github-actions]

CUDA Quantum Docs Bot: A preview of the documentation can be found here.

[Renaud-K]

/ok to test 69aeb0d

Command Bot: Processing...

[github-actions]

CUDA Quantum Docs Bot: A preview of the documentation can be found here.