WIP: CAI-302 upgrade local langfuse to v3

[amanfrinati]

List of Changes

This pull request updates the local development environment for the chatbot application, with a focus on improving Docker Compose configuration, service orchestration, and setup documentation. The most important changes include a major refactor of the docker/compose.yaml file to enhance compatibility and support for Langfuse, ClickHouse, and Minio, as well as improvements to health checks and local setup instructions.

Docker Compose service orchestration and compatibility:

• Refactored docker/compose.yaml to split Langfuse into langfuse-web and langfuse-worker services, and added new dependencies (ClickHouse and Minio) with detailed environment configuration for Langfuse integration. This enables more advanced observability and analytics features for the chatbot.
• Added explicit platform: linux/amd64 specification to several services (api, create-redis-index, jupyter) to improve cross-platform compatibility, especially on Apple Silicon and other non-amd64 hosts. [1] [2] [3]

Service health checks and reliability:

• Introduced health checks for Redis and ClickHouse services to improve reliability and ensure dependent services start only when their dependencies are healthy. [1] [2]
• Updated service dependency management, replacing the old ntw network with direct dependencies and health checks, and removing unnecessary network definitions for simpler configuration. [1] [2] [3]

Local development setup improvements:

• Updated apps/chatbot/README.md to include instructions for copying the Google service account example file, clarifying the initial setup steps for new developers.
• Added a Dockerfile step to automatically download and install the AWS Lambda Runtime Interface Emulator, simplifying local Lambda testing.

Documentation fixes:

• Fixed a typo in the main README.md ("Copile" → "Compile") to improve clarity.

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

• Chore (nothing changes by a user perspective)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 1da4745

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

Jira Pull request Link

@amanfrinati it seems this Pull Request has no issues that refers to Jira!
Please check it out.
PR's title should be something like: [B2B-1] Title where B2B-1 is the activity's id on Jira.

[github-actions]

Branch is not up to date with base branch

@amanfrinati it seems this Pull Request is not updated with base branch.
Please proceed with a merge or rebase to solve this.