LCORE-399: Simplify image building and dependency list. Drop requirements.txt

[matysek]

Description

LCORE-399: Simplify image building and dependency list. Drop requirements.txt
The lightspeed-stack image 0.2.0 bundles now all required Python dependencies. Also bump up the lightspeed-stack image to 0.2.0.

Testing

Steps to test

1. Pull down the PR
2. Try building the image
3. Run tests

Production deployment

• This code change is ready for production on its own
• This code change requires the following considerations before going to production:

[manstis]

@romartin @TamiTakamiya @justjais @ldjebran Could you please take a look?

It looks OK to me on face value.. but I'm worried about moving to llama-stack:0.2.18 as patch releases have broken things before (we're on llama-stack:0.2.16 at the moment in lightspeed-stack:0.1.3).

I also suspect lightspeed-stack-providers would need bumping to llama-stack:0.2..18 too.. and we've also kept our RAG pipelines aligned on llama-stack version too.

@matysek Do you know if llama-stack:0.2.18 broke anything from llama-stack:0.2.16?

[matysek]

@romartin I updated the dependency to lightspeed-stack 0.2.0. It is based on llama-stack:0.2.17. Lightspeed-stack in git was updated to 0.2.18 after lightspeed-stack 0.2.0 release.

[ldjebran]

I am afraid that lightspeed provider will install llama-stack:0.2.16 on top, so we need to bump up to llama-stack:0.2.18 first and aap-rag
will try to test it locally for aap usage.

[matysek]

@ldjebran thank you for checking. I checked it and fixed adding the providers. I tested building the providers should be now bundled properly without upgrading llama-stack version.

[omaciel]

Hi @matysek I guess we need to rebase this? :/

@ldjebran @de1987 @romartin can you keep an eye on this?

[romartin]

Hey @matysek

Sorry for delayed response. Most part of this PR is already out of date, as we're actually using lightspeed-stack:0.3.1. Can you please rebase properly?

There are other small improvements that still apply, and would be great incorporating. If that's a problem, please let us know and we'll work on it.

Thanks in advance!