Skip to content

Commit f7b48c7

Browse files
kaizengrowthkaizengrowth
committed
Resources section
1 parent cbd7a04 commit f7b48c7

File tree

2 files changed

+60
-227
lines changed

2 files changed

+60
-227
lines changed

‎Intro Sessions/Session 1 - Product Engineering for AI/index.html‎

Lines changed: 60 additions & 227 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,34 @@ <h3 class="fragment fade-in" data-fragment-index="2">Components</h3>
390390
</div>
391391
</section>
392392

393+
<section>
394+
<h2>Model Evaluation in Production</h2>
395+
<div class="two-columns">
396+
<div class="column">
397+
<h3 class="fragment fade-in">Metrics to Consider</h3>
398+
<ul>
399+
<li class="fragment fade-up">Accuracy/quality of outputs</li>
400+
<li class="fragment fade-up">Latency and throughput</li>
401+
<li class="fragment fade-up">Cost per inference</li>
402+
<li class="fragment fade-up">Hallucination rate</li>
403+
<li class="fragment fade-up">Ethical considerations</li>
404+
<li class="fragment fade-up">User satisfaction</li>
405+
</ul>
406+
</div>
407+
<div class="column">
408+
<h3 class="fragment fade-in">Monitoring Tools</h3>
409+
<ul>
410+
<li class="fragment fade-up">CloudWatch metrics</li>
411+
<li class="fragment fade-up">Custom dashboards</li>
412+
<li class="fragment fade-up">A/B testing</li>
413+
<li class="fragment fade-up">User feedback loops</li>
414+
<li class="fragment fade-up">Regression testing</li>
415+
<li class="fragment fade-up">Prompt evaluation frameworks</li>
416+
</ul>
417+
</div>
418+
</div>
419+
</section>
420+
393421
<!-- SLIDE 4: UNIQUE CHALLENGES -->
394422
<section>
395423
<h2>Unique Challenges</h2>
@@ -424,11 +452,13 @@ <h3 style="font-size: 0.9em; margin-bottom: 8px;">Ethical Challenges</h3>
424452
</section>
425453

426454
<!-- SLIDE 5: HANDS-ON INTRO -->
427-
<section data-background-color="#f5f5f5">
428-
<h2>Hands-on Session: Deploying a Tiny LLM</h2>
455+
<section>
456+
<h3>Hands-on Activity</h3>
457+
<h2>Deploying a Tiny LLM</h2>
429458
<div class="fragment fade-in">
430-
<p>Today we'll deploy <span class="highlight">Qwen2.5</span>, a powerful yet efficient language model</p>
431-
<p>We'll use:</p>
459+
<p>We'll deploy <span class="highlight">Qwen2.5</span>, a small but efficient LLM.</p>
460+
<br>
461+
<p>Using:</p>
432462
<ul>
433463
<li>AWS EC2 (Free Tier)</li>
434464
<li>Ollama - Open source tool for running models locally</li>
@@ -437,249 +467,52 @@ <h2>Hands-on Session: Deploying a Tiny LLM</h2>
437467
</div>
438468
</section>
439469

440-
<!-- SLIDE 6: QWEN2.5 DETAILS -->
441-
<section>
442-
<h2>Qwen2.5 on EC2 with Ollama</h2>
443-
<div class="two-columns">
444-
<div class="column">
445-
<h3 class="fragment fade-in">What You'll Learn</h3>
446-
<ul>
447-
<li class="fragment fade-up">Setting up an EC2 instance (Free Tier)</li>
448-
<li class="fragment fade-up">Installing Ollama</li>
449-
<li class="fragment fade-up">Deploying Qwen2.5 model</li>
450-
<li class="fragment fade-up">Testing your model</li>
451-
<li class="fragment fade-up">Basic prompt engineering</li>
452-
</ul>
453-
</div>
454-
<div class="column fragment fade-in">
455-
<pre><code class="bash" data-trim data-noescape>
456-
# Install Ollama
457-
curl -fsSL https://ollama.com/install.sh | sh
458-
459-
# Pull the Qwen2.5 model
460-
ollama pull qwen2:7b
461-
462-
# Run the model
463-
ollama run qwen2:7b
464-
</code></pre>
465-
</div>
466-
</div>
467-
</section>
468-
469-
<!-- SLIDE 7: EC2 SETUP -->
470-
<section>
471-
<h2>Step 1: Setting Up EC2</h2>
472-
<ol>
473-
<li class="fragment fade-up">Log in to AWS Console and navigate to EC2</li>
474-
<li class="fragment fade-up">Launch a new instance (t2.large recommended)</li>
475-
<li class="fragment fade-up">Select Ubuntu 22.04 LTS as your AMI</li>
476-
<li class="fragment fade-up">Configure security group to allow SSH access</li>
477-
<li class="fragment fade-up">Create or use an existing key pair</li>
478-
<li class="fragment fade-up">Launch instance and connect via SSH</li>
479-
</ol>
480-
<div class="fragment fade-in">
481-
<pre><code class="bash" data-trim data-noescape>
482-
ssh -i your-key.pem ubuntu@your-instance-public-ip
483-
</code></pre>
484-
</div>
485-
</section>
486-
487-
<!-- SLIDE 8: OLLAMA INSTALLATION -->
488-
<section>
489-
<h2>Step 2: Installing Ollama</h2>
490-
<div class="fragment fade-in">
491-
<pre><code class="bash" data-trim data-noescape>
492-
# Update system packages
493-
sudo apt update && sudo apt upgrade -y
494-
495-
# Install dependencies
496-
sudo apt install -y curl
497-
498-
# Install Ollama
499-
curl -fsSL https://ollama.com/install.sh | sh
500-
501-
# Verify installation
502-
ollama --version
503-
</code></pre>
504-
</div>
505-
<p class="fragment fade-in">Ollama provides a simple API and CLI for running LLMs locally</p>
506-
</section>
507-
508-
<!-- SLIDE 9: DEPLOYING QWEN -->
509-
<section>
510-
<h2>Step 3: Deploying Qwen2.5</h2>
511-
<div class="fragment fade-in">
512-
<pre><code class="bash" data-trim data-noescape>
513-
# Pull the Qwen2.5 model (7B parameter version)
514-
ollama pull qwen2:7b
515-
516-
# This will download ~4GB of data, be patient!
517-
</code></pre>
518-
</div>
519-
<div class="fragment fade-in">
520-
<h3>About Qwen2.5</h3>
521-
<ul>
522-
<li>Developed by Alibaba Cloud</li>
523-
<li>7B parameters (compact yet powerful)</li>
524-
<li>Good performance-to-size ratio</li>
525-
<li>Supports multiple languages</li>
526-
</ul>
527-
</div>
528-
</section>
529-
530-
<!-- SLIDE 10: INTERACTING WITH MODEL -->
531-
<section>
532-
<h2>Step 4: Interacting with the Model</h2>
533-
<div class="fragment fade-in">
534-
<pre><code class="bash" data-trim data-noescape>
535-
# Start a chat session with the model
536-
ollama run qwen2:7b
537-
538-
# Try these prompts:
539-
> Explain the concept of transfer learning in AI
540-
> Write a short poem about Tulsa
541-
> How would you implement a recommendation system?
542-
</code></pre>
543-
</div>
544-
<div class="fragment fade-in">
545-
<p>You can also use the Ollama API to integrate with applications:</p>
546-
<pre><code class="bash" data-trim data-noescape>
547-
# Start the Ollama server
548-
ollama serve
549-
550-
# In another terminal, use curl to query the API
551-
curl -X POST http://localhost:11434/api/generate -d '{
552-
"model": "qwen2:7b",
553-
"prompt": "What is machine learning?"
554-
}'
555-
</code></pre>
556-
</div>
557-
</section>
558-
559-
<!-- SLIDE 11: AI MODEL TERMINOLOGY -->
560-
<section>
561-
<h2>AI Model Terminology</h2>
562-
<table class="fragment fade-in">
563-
<tr>
564-
<th>Term</th>
565-
<th>Definition</th>
566-
</tr>
567-
<tr class="fragment fade-up">
568-
<td><strong>LLM</strong></td>
569-
<td>Large Language Model, a type of AI model trained on vast text data</td>
570-
</tr>
571-
<tr class="fragment fade-up">
572-
<td><strong>Inference</strong></td>
573-
<td>Process of using a trained model to make predictions</td>
574-
</tr>
575-
<tr class="fragment fade-up">
576-
<td><strong>Token</strong></td>
577-
<td>Basic unit of text processing in LLMs (word parts or characters)</td>
578-
</tr>
579-
<tr class="fragment fade-up">
580-
<td><strong>Quantization</strong></td>
581-
<td>Process of reducing model precision to improve performance</td>
582-
</tr>
583-
<tr class="fragment fade-up">
584-
<td><strong>Context Window</strong></td>
585-
<td>The amount of text a model can consider at once</td>
586-
</tr>
587-
<tr class="fragment fade-up">
588-
<td><strong>Fine-tuning</strong></td>
589-
<td>Adapting a pre-trained model for specific tasks</td>
590-
</tr>
591-
</table>
592-
</section>
593470

594-
<!-- SLIDE 12: MODEL EVALUATION -->
471+
<!-- SLIDE 15: RESOURCES -->
595472
<section>
596-
<h2>Model Evaluation in Production</h2>
473+
<h2>Next Steps</h2>
597474
<div class="two-columns">
598475
<div class="column">
599-
<h3 class="fragment fade-in">Metrics to Consider</h3>
476+
<h3 class="fragment fade-in">Resources</h3>
600477
<ul>
601-
<li class="fragment fade-up">Accuracy/quality of outputs</li>
602-
<li class="fragment fade-up">Latency and throughput</li>
603-
<li class="fragment fade-up">Cost per inference</li>
604-
<li class="fragment fade-up">Hallucination rate</li>
605-
<li class="fragment fade-up">Ethical considerations</li>
606-
<li class="fragment fade-up">User satisfaction</li>
478+
<li class="fragment fade-up" style="font-size: 0.8em; margin-bottom: 3px;"><a href="https://huyenchip.com/books/" target="_blank">Chip Huyen's "AI Engineering" and "Designing Machine Learning Systems"</a></li>
479+
<li class="fragment fade-up" style="font-size: 0.8em; margin-bottom: 3px;"><a href="https://ollama.com/docs" target="_blank">Ollama Documentation</a></li>
480+
<li class="fragment fade-up" style="font-size: 0.8em; margin-bottom: 3px;"><a href="https://huggingface.co/Qwen/Qwen2-7B" target="_blank">Qwen2 Model Card</a></li>
481+
<li class="fragment fade-up" style="font-size: 0.8em; margin-bottom: 3px;"><a href="https://huggingface.co/docs/evaluate/llm_evaluation" target="_blank">LLM Evaluation Guide</a></li>
482+
<li class="fragment fade-up" style="font-size: 0.8em; margin-bottom: 3px;"><a href="https://www.geeksforgeeks.org/aws-certified-ai-practitioner-aif-c01/" target="_blank">AWS Certified AI Practitioner (AIF-C01) Guide</a></li>
607483
</ul>
608484
</div>
609485
<div class="column">
610-
<h3 class="fragment fade-in">Monitoring Tools</h3>
611-
<ul>
612-
<li class="fragment fade-up">CloudWatch metrics</li>
613-
<li class="fragment fade-up">Custom dashboards</li>
614-
<li class="fragment fade-up">A/B testing</li>
615-
<li class="fragment fade-up">User feedback loops</li>
616-
<li class="fragment fade-up">Regression testing</li>
617-
<li class="fragment fade-up">Prompt evaluation frameworks</li>
486+
<div class="fragment fade-in" style="text-align: center; margin-bottom: 15px;">
487+
<img src="../../assets/images/AI_Engineering_ChipHuyen.jpg" alt="Chip Huyen's AI Engineering Book" style="width: 80%; max-width: 180px;">
488+
</div>
489+
<h4 class="fragment fade-in">Coming in Session 2</h4>
490+
<ul style="font-size: 0.7em; line-height: 1.1;">
491+
<li class="fragment fade-up" style="margin-bottom: 2px;">AWS Bedrock for simplified AI orchestration</li>
492+
<li class="fragment fade-up" style="margin-bottom: 2px;">Edge AI deployment on Raspberry Pi</li>
493+
<li class="fragment fade-up" style="margin-bottom: 2px;">Environmental sensor integration</li>
494+
<li class="fragment fade-up" style="margin-bottom: 2px;">RAG patterns for knowledge augmentation</li>
618495
</ul>
619496
</div>
620497
</div>
621498
</section>
622499

623-
<!-- SLIDE 13: ARCHITECTURE OVERVIEW -->
624-
<section data-transition="zoom">
625-
<h2>Architecture Overview</h2>
626-
<div class="fragment fade-in" style="text-align: center; margin-top: 30px;">
627-
<img src="https://placeholder.pics/svg/800x400/DEDEDE/555555/AI%20Application%20Architecture" alt="Architecture Diagram">
628-
<p class="caption">Simple AI Application Architecture with EC2 and Ollama</p>
629-
</div>
630-
<div class="fragment fade-in">
631-
<pre><code class="text" data-trim data-noescape>
632-
User → API Gateway → Lambda → EC2 with Ollama → S3 Storage
633-
</code></pre>
634-
</div>
635-
</section>
636-
637-
<!-- SLIDE 14: WORKSHOP TIME -->
638-
<section data-background="#6a2ea0" data-background-transition="zoom">
639-
<h1 style="color: white;">Workshop Time!</h1>
640-
<p style="color: white; font-size: 1.5em;">Let's deploy our first AI model together</p>
641-
<div class="fragment fade-in" style="margin-top: 50px; color: white;">
642-
<p>Follow along with the instructor</p>
643-
<p>Ask questions at any time</p>
644-
<p>Collaborate with your neighbors</p>
645-
</div>
646-
</section>
647-
648-
<!-- SLIDE 15: RESOURCES -->
500+
<!-- SLIDE 16: THANK YOU -->
649501
<section>
650-
<h2>Resources & Next Steps</h2>
651-
<div class="two-columns">
502+
<h1>Thank You!</h1>
503+
<div class="two-columns" style="margin-top: 20px;">
652504
<div class="column">
653-
<h3 class="fragment fade-in">Resources</h3>
654-
<ul>
655-
<li class="fragment fade-up"><a href="https://ollama.com/docs" target="_blank">Ollama Documentation</a></li>
656-
<li class="fragment fade-up"><a href="https://docs.aws.amazon.com/ec2/" target="_blank">AWS EC2 Documentation</a></li>
657-
<li class="fragment fade-up"><a href="https://huggingface.co/Qwen/Qwen2-7B" target="_blank">Qwen2 Model Card</a></li>
658-
<li class="fragment fade-up"><a href="https://huggingface.co/docs/evaluate/llm_evaluation" target="_blank">LLM Evaluation Guide</a></li>
659-
</ul>
505+
<p class="fragment fade-in">Questions? Project ideas? Let's discuss!</p>
506+
<p class="fragment fade-in">Join us on <a href="https://techlahoma.org" target="_blank">#techlahoma</a> Slack</p>
507+
<br><br>
508+
<p class="fragment fade-in" style="margin-top: 40px; color: var(--accent-color);">See you at Session 2!</p>
660509
</div>
661-
<div class="column">
662-
<h3 class="fragment fade-in">Coming in Session 2</h3>
663-
<ul>
664-
<li class="fragment fade-up">AWS Bedrock for simplified AI orchestration</li>
665-
<li class="fragment fade-up">Edge AI deployment on Raspberry Pi</li>
666-
<li class="fragment fade-up">Environmental sensor integration</li>
667-
<li class="fragment fade-up">RAG patterns for knowledge augmentation</li>
668-
</ul>
510+
<div class="column fragment fade-in" style="display: flex; justify-content: center; align-items: center;">
511+
<img src="../../assets/images/Tulsa AI Learning Lab logo.png" alt="Tulsa AI Learning Lab Logo" style="height: 400px; max-width: 100%;">
669512
</div>
670513
</div>
671514
</section>
672515

673-
<!-- SLIDE 16: THANK YOU -->
674-
<section data-background-gradient="linear-gradient(to bottom, #6a2ea0, #9d71c7)">
675-
<h1 style="color: white;">Thank You!</h1>
676-
<div style="color: white; margin-top: 40px;">
677-
<p class="fragment fade-in">Questions? Let's discuss!</p>
678-
<p class="fragment fade-in">Join us on the #oklahom-ai channel on Slack</p>
679-
<p class="fragment fade-in" style="margin-top: 60px;">See you at Session 2!</p>
680-
</div>
681-
</section>
682-
683516

684517

685518
</div>

‎assets/images/AI_Engineering_ChipHuyen.jpg‎

186 KB
Loading

0 commit comments

Comments
 (0)