09-llms

Language Models

• Text slides
• LLM slides

Transformers and Large Language Models

among the reasons I use large pre-trained language models sparingly in my computer-generated poetry practice is that being able to know whose voices I'm speaking with is... actually important, as is being understanding how the output came to have its shape - @aparrish, full thread

• 📚 Watch an A.I. Learn to Write by Reading Nothing but ____ by Aatish Bhatia
• 📚 Attention is All You Need - Original "Transformer" paper from 2017, also Neural Machine Translation by Jointly Learning to Align and Translate -- Attention paper from 2014
• 📚 What Are Transformer Models and How Do They Work?
• 🎥 How large language models work, a visual intro to transformers by 3Blue1Brown
• 🎥 Intro to Large Language Models by Andrej Karpathy and Intro to LLMs slides
• 📖 Language Models Can Only Write Ransom Notes by Allison Parrish

LLM Training

• 🦙 LLaMA: Open and Efficient Foundation Language Models
• 🦙 The Llama 3 Herd of Models
• 🦜 On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
• 🔍 The Foundation Model Transparency Index
• 📖 Generative AI’s Illusory Case for Fair Use by Jacqueline Charlesworth

Datasets for LLMs

• 🔢 Common Crawl
• 🔢 The Pile

Climate Impact

• 🌏 The Internet’s Next Great Power Suck
• ⚡️ Carbon Emissions and Large Neural Network Training

Code Examples

• Word2Vec suggesting alternative words
• Word2vec Similar Words Floating in Space
• Sentiment
• OpenAI API + p5.js
• Gemini API + p5.js drawing, code
• Class: Gemini api p5js, Gemini api drawing
• Class: https://editor.p5js.org/yining/sketches/BmMSp-DcT (text to text)
• Class 2: canvas to text: https://editor.p5js.org/yining/sketches/UVZM7MXKi
• Class 3: webcam to text: https://editor.p5js.org/yining/sketches/xfcE8kgJd

How to obtain a Google AI Studio developer test API key:

1. Go to the Google AI Studio website: https://ai.google.dev/aistudio
2. Sign in with your Google account
3. Navigate to the API Keys section (the blue "Get API Key" button on the top left)
4. Click on the "Create API Key" button
5. Follow the prompts to create a new project or select an existing one.
6. Enable the Gemini API for your project.
7. Copy the generated API key and paste it into the "API KEY" input field in the web application.

• Note: Make sure to keep your API key secure and avoid sharing it publicly.

Example gemini curl link

Assignment

• Use Gemini APIs to build any experience with p5.js, example
• Or try chat with Gemini, Google AI studio, Chatgpt, Claude and document your findings.
• Or add more features to the Word2Vec/Sentiment sketch.
• Document your experminents in a blog post and add a link to the Assignment Wiki page. Remember to include visual documentation (screenshots, GIFs, etc)