|
| 1 | +# Self-Host Langfuse on Amazon ECS with Fargate using TypeScript CDK |
| 2 | + |
| 3 | +Langfuse is an Open Source LLM Engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. |
| 4 | + |
| 5 | +This repository demonstrates how to deploy a self-hosted Langfuse solution using AWS Fargate for Amazon ECS. It is designed for initial experimentation and is not suitable for production use. Additionally, it does not include all capabilities available in the Langfuse Enterprise Edition. For production-ready deployments, check out the [Langfuse offerings through AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=seller-nmyz7ju7oafxu). |
| 6 | + |
| 7 | +## Architecture overview |
| 8 | + |
| 9 | +This deployment involves multiple AWS services to host Langfuse components as described in their [official documentation on self-hosting](https://langfuse.com/self-hosting#architecture). In this sample, shown also in the architecture diagram below, we use: |
| 10 | + |
| 11 | +1. [AWS Fargate for Amazon ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html) to host the required containers without the need to manage servers or clusters of Amazon EC2 instances |
| 12 | +2. [Amazon RDS](https://aws.amazon.com/rds/postgresql/) for the Postgres OLTP store |
| 13 | +3. [Amazon Elasticache](https://aws.amazon.com/elasticache/) for the Valkey cache |
| 14 | +4. [Amazon EFS](https://aws.amazon.com/efs/) for durable managed storage to back the deployed [ClickHouse](https://clickhouse.com/docs/intro) OLAP system |
| 15 | +5. [Amazon CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Introduction.html) for high-performance CDN and HTTPS connectivity to the deployed Langfuse service. |
| 16 | + |
| 17 | + |
| 18 | + |
| 19 | +### Request Flow Sequence: |
| 20 | + |
| 21 | +1. **Client Request**: LLM applications and web browsers initiate requests through the Langfuse SDK or directly via the web interface. These requests are routed to Amazon CloudFront, which serves as the Content Delivery Network (CDN) to provide HTTPS termination, caching, and global distribution. |
| 22 | +2. **Routing Through Application Load Balancer**: CloudFront forwards the requests to the Application Load Balancer (ALB) within the Virtual Private Cloud (VPC). The ALB ensures proper routing of traffic to the ECS cluster hosting Langfuse components. |
| 23 | +3. **Processing in ECS Cluster**: The ECS cluster, powered by AWS Fargate, processes requests using two primary components: |
| 24 | + |
| 25 | + * Langfuse Web Server: Handles API endpoints and user-facing interactions. |
| 26 | + * Langfuse Worker: Processes background tasks such as event batching and analytics. |
| 27 | + |
| 28 | +4. **Data Storage Operations**: Depending on the type of data being processed, requests interact with various storage systems: |
| 29 | + |
| 30 | + * Amazon RDS Aurora Postgres OLTP: Stores transactional data such as user configurations, API keys, and project metadata. |
| 31 | + * Amazon EFS-backed ClickHouse OLAP: Handles analytical workloads for high-volume trace and span data. |
| 32 | + * Amazon S3: Stores raw objects like logs or files uploaded during processing. |
| 33 | + |
| 34 | +5. **Caching with ElastiCache**: The system utilizes Amazon ElastiCache (Valkey cache) for Redis-compatible caching to optimize performance by reducing database load and enabling faster access to frequently queried data. |
| 35 | +6. **Response Delivery**: Once processing is complete, responses are sent back to clients through the same path, ensuring secure delivery via CloudFront. Metrics and logs generated during processing from ECS are also forwarded to Amazon CloudWatch for observability. |
| 36 | + |
| 37 | +This flow ensures efficient handling of requests while leveraging AWS's managed services for scalability, reliability, and security. |
| 38 | + |
| 39 | +## Deployment option 1: Quick start |
| 40 | + |
| 41 | +If you don't have CDK development tooling set up already, and would just like to deploy the Langfuse architecture with the default settings - you can use the [AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html) template in [cfn_bootstrap.yml](./cfn_bootstrap.yml): Open the [CloudFormation Console](https://console.aws.amazon.com/cloudformation/home?#/stacks/create) in your target AWS Account and Region, click **Create stack**, and upload the template file. |
| 42 | + |
| 43 | +This "bootstrap" stack sets up a CI project in [AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/welcome.html) which pulls the sample code and performs the below app CDK setup steps for you automatically in the Cloud - with no local development environment required. ⚠️ **Note** though, that the CodeBuild project is granted *broad permissions* to deploy all the sample's AWS resources on your behalf - so is not recommended for use in production environments as-is. |
| 44 | + |
| 45 | +## Deployment option 2: Developer setup (Suggested workflow) |
| 46 | + |
| 47 | +If you are comfortable with CDK deployment or want to customize the app, you'll need to set up your local development environment rather than using the quick-start template above. |
| 48 | + |
| 49 | +### Prerequisites: |
| 50 | + |
| 51 | +1. Docker or Finch |
| 52 | + |
| 53 | + This project requires a local container build environment. If your organization doesn't support [Docker Desktop](https://www.docker.com/products/docker-desktop/), you can instead install [Finch](https://runfinch.com/). If using Finch instead of Docker, remember to: |
| 54 | + |
| 55 | + - Initialise the VM with `finch vm start`, and |
| 56 | + - Tell CDK how to build containers, by running `export CDK_DOCKER=finch` (on MacOS/Linux), or `SET CDK_DOCKER=finch` (on Windows) |
| 57 | + |
| 58 | +2. NodeJS |
| 59 | + |
| 60 | + This project requires [NodeJS](https://nodejs.org/) v20+ |
| 61 | + |
| 62 | +3. AWS CLI login |
| 63 | + |
| 64 | + To actually deploy the infrastructure to a target AWS Account (and possibly, even to synthesize a concrete template), you'll need to [configure your AWS CLI credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). Note that whatever principal you log in to AWS with (User, Role, etc) will need the relevant *IAM Permissions* to deploy and manage all types of resources used by the sample - which is a broad set. |
| 65 | + |
| 66 | + You'll also want to set your target *AWS Region*. You can check your current active AWS Account by running `aws sts get-caller-identity`, and your selected region with `aws configure get region`. |
| 67 | + |
| 68 | + |
| 69 | +### Development workflow |
| 70 | + |
| 71 | +Once your development environment is set up, this sample works like a standard CDK app project. |
| 72 | + |
| 73 | +First, install the project's dependencies: |
| 74 | + |
| 75 | +```bash |
| 76 | +npm install |
| 77 | +``` |
| 78 | + |
| 79 | +Then, you can deploy it to AWS: |
| 80 | + |
| 81 | +```bash |
| 82 | +# Deploy or update all Stacks in the app: |
| 83 | +# (Optionally specify --require-approval never to suppress approval prompts) |
| 84 | +npx cdk deploy --all |
| 85 | +``` |
| 86 | + |
| 87 | +To **delete** the deployed infrastructure when you're done exploring, and avoid ongoing charges: |
| 88 | + |
| 89 | +> ⚠️ **Warning:** Running the below will irreversibly delete any data you've stored in your deployed Langfuse instance! |
| 90 | +
|
| 91 | +```bash |
| 92 | +npx cdk destroy --all |
| 93 | +``` |
| 94 | + |
| 95 | +**Other useful commands** for the project include: |
| 96 | +- `npx cdk destroy` to delete the deployed infrastructure |
| 97 | +- `npx cdk synth` to ["synthesize"](https://docs.aws.amazon.com/cdk/v2/guide/configure-synth.html) the CDK application to deployable CloudFormation template(s), without actually deploying |
| 98 | +- `npx cdk list` to list all CDK stacks and their dependencies |
| 99 | + |
| 100 | +Refer to the [CDK CLI commands guide](https://docs.aws.amazon.com/cdk/v2/guide/ref-cli-cmd.html) for a full reference |
| 101 | + |
| 102 | + |
| 103 | +## License |
| 104 | +This library is licensed under the MIT-0 License. See the LICENSE file. |
0 commit comments