Repository for the Everything You Need to Know About Running LLMs Locally session.
As large language models (LLMs) become more accessible, running them locally unlocks exciting opportunities for developers, engineers, and privacy-focused users. Why rely on costly cloud AI services that share your data when you could deploy your own models tailored to your needs? In this session, we’ll dive into the advantages of local LLM deployment, from selecting the right open source model to optimizing performance on consumer hardware and integrating with your unique data. Let’s explore the journey to your own local stack for AI, and cover the important technical details such as model quantization, API integrations with IDE code assistants, and advanced methods like Retrieval-Augmented Generation (RAG) to connect your LLM to private data sources. Don’t miss out on the fun live demos that prove the bright future of open source AI is already here!
- Demo 1 - Local Model Serving with Podman AI
- Demo 2 - Retrieval Augmented Generation with AnythingLLM
- Demo 3 - Private Code Assistant with Continue
- Demo 4 - Agentic App powered by Streamlit, LangGraph and Podman AI