Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 4 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# serving-api

Frontend and backend API proxy for SwissAI LLM serving.
Frontend and backend API proxy for SwissAI LLM serving. For examples on how to launch models, see [model-launch](https://github.com/swiss-ai/model-launch) repo.

**Live at:**
- Prod: [serving.swissai.svc.cscs.ch](https://serving.swissai.svc.cscs.ch)
- Dev: [servingdev.swissai.svc.cscs.ch](https://servingdev.swissai.svc.cscs.ch)
- Local: with `docker compose up`
- Local: with `make run`

## Architecture

Expand All @@ -26,7 +26,6 @@ Frontend and backend API proxy for SwissAI LLM serving.
┌─────────────────┐
│ OCF │ OpenTela P2P routing → model=apertus-...
│ │
└────────┬────────┘
Expand All @@ -40,28 +39,14 @@ Frontend and backend API proxy for SwissAI LLM serving.
```
backend/ # Python API proxy (FastAPI) — auth, caching, routing
frontend/ # web UI (Astro + Svelte)
meta/ # Dockerfiles, k8s manifests, build scripts
tests/ # integration tests
tools/ # metrics & monitoring utilities
meta/ # example Dockerfiles, example k8s manifests, build scripts
```

OCF (Open Compute Framework) now renamed to OpenTela upstream is maintained at [eth-easl/OpenTela](https://github.com/eth-easl/OpenTela). We maintain a fork at [swiss-ai/OpenTela](https://github.com/swiss-ai/opentela) to control deployments to dev+prod.

## Quick Start

### Docker

```bash
docker compose up
```

### Local Development
## Dev Quick Start

```bash
make install # install backend dependencies
make run # start backend on :8080

# frontend
cd frontend
npm install && npm run dev
```
3 changes: 3 additions & 0 deletions frontend/public/favicon-dark.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions frontend/public/favicon-light.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added frontend/public/images/sponsors/epfl.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added frontend/public/images/sponsors/eth-ai-center.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions frontend/public/images/sponsors/eth-zurich.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 1 addition & 3 deletions frontend/src/components/Head.astro
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,7 @@ const { title, description, image = "/nano.png" } = Astro.props;
<!-- Global Metadata -->
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="icon" type="image/svg+xml" href="/favicon-dark.svg" media="(prefers-color-scheme: dark)">
<link rel="icon" type="image/svg+xml" href="/favicon-light.svg" media="(prefers-color-scheme: light)">
<link rel="icon" type="image/x-icon" href="/favicon-light.svg">
<link rel="icon" type="image/svg+xml" href="/favicon-light.svg">
<meta name="generator" content={Astro.generator} />

<!-- Font preloads -->
Expand Down
8 changes: 4 additions & 4 deletions frontend/src/components/SponsorCard.astro
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ interface Props {
const { name, logo, url, width = 158, height = 48 } = Astro.props;
---

<div class="col-span-2 max-h-12 w-full object-contain lg:col-span-2">
<div class="w-full h-full flex items-center justify-center">
{url ? (
<a href={url} target="_blank" rel="noopener noreferrer">
<img src={logo} alt={name} width={width} height={height} class="w-full h-full object-contain" />
<a href={url} target="_blank" rel="noopener noreferrer" class="w-full h-full flex items-center justify-center">
<img src={logo} alt={name} width={width} height={height} class="max-h-16 max-w-full object-contain" />
</a>
) : (
<img src={logo} alt={name} width={width} height={height} class="w-full h-full object-contain" />
<img src={logo} alt={name} width={width} height={height} class="max-h-16 max-w-full object-contain" />
)}
</div>
6 changes: 3 additions & 3 deletions frontend/src/content/guides/01-getting-started/index.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: "Getting Started with SP and Local Serving"
description: "Run models locally with Scratchpad and Ollama"
title: "[ARCHIVED] Getting Started with SP and Local Serving"
description: "[ARCHIVED] Run models locally with Scratchpad and Ollama"
date: "Mar 18 2024"
---

# Getting Started with SP and Local Serving
# [ARCHIVED] Getting Started with SP and Local Serving

## Run your Model Locally

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Launch Models with model-launch"
title: "Launch Models with model-launch 🚀"
description: "Framework-agnostic SLURM job submission for distributed inference"
date: "December 22 2025"
---
Expand Down
17 changes: 9 additions & 8 deletions frontend/src/pages/index.astro
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ const contributors = [
url: "https://www.cscs.ch/about/staff"
},
{
name: "Robert Matthew Smith",
name: "Robert Smith",
role: "System Expert",
avatar: "/images/contributers/robert_matthew_smith.jpeg",
url: "#"
url: "https://www.linkedin.com/in/robmsmt"
},
{
name: "Xiaozhe Yao",
Expand All @@ -81,15 +81,15 @@ const sponsors = [
},
{
name: "ETH Zurich",
logo: "https://upload.wikimedia.org/wikipedia/commons/thumb/9/99/ETH_Z%C3%BCrich_Logo_black.svg/1200px-ETH_Z%C3%BCrich_Logo_black.svg.png"
logo: "/images/sponsors/eth-zurich.svg"
},
{
name: "ETHZ AI Center",
logo: "https://ai.ethz.ch/_jcr_content/orgbox/image.imageformat.logo.1864120785.png"
logo: "/images/sponsors/eth-ai-center.png"
},
{
name: "Langfuse",
logo: "https://langfuse.com/langfuse_logo.svg"
name: "EPFL",
logo: "/images/sponsors/epfl.png"
}
];
---
Expand Down Expand Up @@ -177,7 +177,7 @@ const sponsors = [
</div>
</section>

<!-- Token Usage Statistics -->
<!-- Token Usage Statistics — hidden until rebuilt
<section class="animate space-y-8">
<div class="text-center">
<h2 class="text-3xl font-bold text-slate-900 dark:text-white mb-4">
Expand All @@ -187,11 +187,12 @@ const sponsors = [
Real-time token usage and platform statistics
</p>
</div>

<div class="card p-8">
<TokenUsageChart client:load />
</div>
</section>
-->

<!-- Documentation Section -->
<section class="animate space-y-8">
Expand Down
Loading