baseline.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-12-23T10:55:47.001634Z","iopub.execute_input":"2022-12-23T10:55:47.001989Z","iopub.status.idle":"2022-12-23T10:55:47.011788Z","shell.execute_reply.started":"2022-12-23T10:55:47.001959Z","shell.execute_reply":"2022-12-23T10:55:47.010774Z"},"trusted":true},"execution_count":21,"outputs":[{"name":"stdout","text":"/kaggle/input/spotify-dataset/spotify_songs_en_pop.csv\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# <p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">Lyrics Generator</p>\n\n<img src=\"https://github.com/KarnikaKapoor/Files/blob/main/Pink%20and%20White%20Geometric%20Marketing%20Presentation.gif?raw=true\">\n\n<p style=\"font-family:newtimeroman;font-size:120%;color:#444160;\">In this project, I will be building a model to generate text. My goal is to build a song lyrics generator to explore the \"creative\" side of the Recurrent Neural Networks(RNN). RNN Text generator is one of my most desired to-do projects. I am finally checking this one off my to-do list. So yeyyy! </p> \n\n<a id='top'></a>\n<div class=\"list-group\" id=\"list-tab\" role=\"tablist\">\n<p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">TABLE OF CONTENTS</p>   \n    \n* [1. IMPORTING LIBRARIES](#1)\n    \n* [2. LOADING DATA](#2)\n    \n* [3. DATA EXPLORATION](#3)  \n    \n* [4. DATA PREPREPROCESSING](#4)  \n    \n* [5. MODEL BUILDING](#5) \n      \n* [6. EVALUATING MODELS](#6)\n    \n* [7. CONCLUSION](#7)\n    \n* [8. END](#8)\n\n\n<a id=\"1\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">IMPORTING LIBRARIES</p>\n","metadata":{}},{"cell_type":"code","source":"#importing the libraries\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport string, os\nimport nltk\nimport re\nimport keras\nimport random\nimport io\nfrom keras.utils import np_utils\nfrom keras.models import Sequential\nfrom keras.layers import LSTM, Dense, Dropout\nfrom keras.optimizers import Adamax\nimport sys\nfrom PIL import Image\nfrom wordcloud import WordCloud, STOPWORDS, ImageColorGenerator\nfrom PIL import Image, ImageDraw, ImageFont\nimport gc\nimport warnings\nfrom sklearn.model_selection import train_test_split\n\nwarnings.filterwarnings(\"ignore\")\n","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.013395Z","iopub.execute_input":"2022-12-23T10:55:47.013745Z","iopub.status.idle":"2022-12-23T10:55:47.024691Z","shell.execute_reply.started":"2022-12-23T10:55:47.013708Z","shell.execute_reply":"2022-12-23T10:55:47.023794Z"},"trusted":true},"execution_count":22,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"2\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">LOADING DATA</p>\nFor this project, I have prepared a dataset of song lyrics. Let's load it and have a look.","metadata":{}},{"cell_type":"code","source":"data = pd.read_csv(\"../input/spotify-dataset/spotify_songs_en_pop.csv\", nrows=1000)['lyrics']\ndata.head()\n\ndata, test_data = train_test_split(data, test_size=50)","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.026531Z","iopub.execute_input":"2022-12-23T10:55:47.026975Z","iopub.status.idle":"2022-12-23T10:55:47.069295Z","shell.execute_reply.started":"2022-12-23T10:55:47.026938Z","shell.execute_reply":"2022-12-23T10:55:47.068367Z"},"trusted":true},"execution_count":23,"outputs":[]},{"cell_type":"code","source":"testCorp = []\n\nfor listitem in test_data:\n    testCorp.append(listitem.lower())","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.071073Z","iopub.execute_input":"2022-12-23T10:55:47.071440Z","iopub.status.idle":"2022-12-23T10:55:47.076331Z","shell.execute_reply.started":"2022-12-23T10:55:47.071403Z","shell.execute_reply":"2022-12-23T10:55:47.075305Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"code","source":"to_remove = ['*', '_', '~', '¡', '«', '´', '¿', 'à',\n             'á', 'è', 'é', 'ê', 'í', 'ï', 'ñ', 'ó', 'ú', 'ü', 'а', 'в', 'д', 'е', 'з',\n             'и', 'й', 'к', 'л', 'м', 'н', 'о', 'п', 'р', 'с', 'т', 'у', 'ф', 'ы', 'ь',\n             'ю', 'я', '\\u2005', '\\u200a', '\\u200b', '\\u3000', '、', '「', '」', 'ぁ', 'あ',\n             'い', 'う', 'え', 'か', 'が', 'き', 'ぎ', 'く', 'け', 'げ', 'こ', 'さ', 'し', 'じ',\n             'す', 'せ', 'そ', 'た', 'だ', 'ち', 'っ', 'つ', 'づ', 'て', 'で', 'と', 'ど', 'な',\n             'に', 'ぬ', 'ね', 'の', 'は', 'び', 'ふ', 'ぶ', 'へ', 'べ', 'ほ', 'ま', 'み', 'む',\n             'め', 'も', 'よ', 'ら', 'り', 'る', 'れ', 'ろ', 'わ', 'を', 'ん', 'イ', 'ウ', 'カ', \n             'キ', 'ギ', 'ク', 'シ', 'ス', 'タ', 'ダ', 'ッ', 'ツ', 'テ', 'ト', 'ネ', 'ノ', 'ビ', \n             'ポ', 'ミ', 'メ', 'モ', 'ヤ', 'ョ', 'リ', 'ル', 'レ', 'ン', 'ー', '一', '上', '世', \n             '中', '乗', '事', '人', '仆', '今', '他', '代', '体', '何', '侵', '俺', '像', '僕', \n             '償', '先', '光', '全', '共', '分', '切', '別', '前', '動', '去', '口', '叫', '名', \n             '向', '君', '呼', '咲', '嘘', '在', '壁', '壊', '声', '変', '夜', '夢', '大', '孤',\n             '宿', '対', '尽', '巡', '巻', '序', '度', '弱', '張', '強', '影', '待', '後', '心',\n             '恍', '恐', '悟', '悪', '情', '惚', '想', '愛', '感', '戻', '手', '抜', '拓', '指', \n             '振', '探', '揺', '支', '教', '日', '明', '昔', '昙', '時', '書', '最', '未', '来', \n             '極', '歓', '止', '正', '歩', '比', '気', '泣', '溺', '無', '特', '独', '珠', '現',\n             '理', '由', '界', '痛', '白', '目', '真', '着', '瞳', '知', '破', '磨', '私', '突',\n             '章', '笑', '筋', '続', '置', '肥', '胸', '自', '良', '薇', '薔', '蝕', '行', '衝', \n             '裸', '襲', '見', '覚', '见', '触', '言', '誇', '誰', '越', '踠', '身', '輝', '过',\n             '返', '迷', '進', '過', '配', '重', '間', '闇', '限', '頭', '食', '高', '鳴', '갑',\n             '개', '걱', '것', '겐', '겠', '견', '결', '곁', '곡', '곳', '과', '광', '교', '국',\n             '군', '귀', '글', '급', '긋', '깊', '깐', '깜', '깝', '꺾', '께', '껴', '꼬', '꼭',\n             '꼴', '꽃', '꽉', '꽤', '꿰', '끊', '끽', '남', '낮', '냥', '넓', '녁', '녔', '녕',\n             '놀', '높', '놓', '놔', '놨', '뉴', '능', '닐', '님', '답', '당', '닿', '덜', '덩',\n             '델', '독', '동', '된', '될', '둑', '둘', '둡', '뒤', '드', '득', '듭', '딱', '땀',\n             '땡', '떠', '떤', '떨', '떻', '떼', '뚜', '뚫', '뜨', '뜻', '띄', '락', '럭', '럼',\n             '럽', '럿', '렘', '력', '렵', '렸', '록', '론', '롭', '루', '릇', '릭', '릴', '림',\n             '릿', '막', '많', '맘', '맛', '망', '맞', '맡', '맣', '매', '맨', '맴', '먼', '멀',\n             '며', '명', '몇', '목', '몸', '문', '민', '밀', '밖', '발', '밝', '밟', '백', '벌',\n             '베', '벼', '볍', '복', '본', '볼', '봉', '붉', '뷰', '빤', '빨', '뻑', '뻥', '뿌',\n             '뿜', '쁘', '살', '삶', '석', '설', '섬', '섭', '섹', '셀', '셋', '소', '손', '솝', \n             '송', '숑', '순', '숴', '쉬', '쉽', '쉿', '슈', '스', '슨', '슴', '습', '식', '실',\n             '싫', '십', '쌔', '쌤', '쏟', '쏴', '쑤', '쓰', '씩', '씹', '앉', '알', '았', '앞',\n             '약', '양', '얗', '언', '엇', '엔', '열', '엽', '였', '영', '옆', '옇', '예', '옥',\n             '외', '요', '용', '운', '울', '웃', '웅', '원', '윈', '유', '으', '익', '작', '잘', \n             '잠', '잡', '재', '잭', '적', '절', '젊', '점', '조', '존', '죽', '준', '줌', '줘', \n             '줬', '쥐', '증', '짓', '짝', '짧', '쩌', '찍', '착', '참', '창', '찾', '채', '챘', \n             '천', '철', '청', '쳐', '쳤', '초', '추', '축', '출', '춤', '춰', '췬', '칙', '카', \n             '커', '켜', '콘', '콤', '콱', '쿵', '크', '큰', '큼', '킬', '타', '탁', '탓', '탬', \n             '텐', '텝', '토', '통', '툭', '툰', '툴', '트', '특', '틀', '팅', '파', '팔', '패',\n             '퍼', '펴', '편', '펼', '폈', '평', '포', '폼', '표', '품', '프', '플', '피', '필',\n             '학', '할', '항', '핵', '행', '향', '험', '헤', '헷', '혀', '현', '호', '혹', '혼',\n             '화', '확', '환', '황', '회', '효', '후', '훅', '훔', '흔', '흘', '히', '힘', 'ﬁ', \n             '\\ufeff', '\\xa0', 'É', 'ä', 'å', 'ç', 'ì', 'î', 'ò', 'ô', 'ö', 'ù', 'А', 'П',\n             '가', '각', '간', '갈', '감', '같', '걍', '거', '건', '걷', '걸', '겁', '게', '겨', \n             '경', '계', '고', '곧', '골', '공', '관', '구', '굴', '궁', '그', '극', '근', '금', \n             '기', '긴', '길', '까', '깨', '꺼', '꾸', '꿈', '끄', '끈', '끌', '끔', '끝', '끼', \n             '낌', '나', '난', '날', '내', '냐', '너', '넌', '널', '넘', '네', '노', '논', '누', \n             '눈', '느', '는', '늘', '니', '닌', '다', '단', '달', '대', '더', '덕', '던', '데', \n             '도', '돌', '돼', '되', '두', '뒷', '든', '듣', '들', '듬', '듯', '디', '딜', '따', \n             '때', '떡', '또', '똑', '뛰', '라', '란', '람', '랑', '래', '랠', '러', '런', '렇', \n             '레', '려', '례', '로', '르', '른', '를', '름', '릎', '리', '린', '마', '만', '말', \n             '머', '먹', '멈', '멋', '멘', '면', '모', '몰', '못', '무', '묻', '물', '뭐', '뭔', \n             '뭘', '미', '믿', '바', '반', '받', '밤', '방', '배', '뱉', '버', '번', '범', '법', \n             '벗', '벽', '변', '별', '보', '봐', '부', '분', '불', '비', '빌', '빛', '빠', '뻔',\n             '뿐', '뿔', '삐', '사', '상', '새', '생', '서', '선', '성', '세', '셔', '속', '쇠',\n             '수', '숨', '슬', '시', '신', '심', '싶', '써', '썰', '아', '악', '안', '않', '애', \n             '앨', '야', '어', '억', '얼', '엄', '없', '었', '에', '여', '엮', '오', '온', '올', \n             '와', '완', '왔', '왜', '왠', '욕', '우', '움', '워', '위', '은', '을', '음', '의',\n             '이', '인', '일', '임', '입', '있', '잊', '자', '잖', '장', '저', '전', '정', '제',\n             '젠', '져', '좀', '좋', '주', '줄', '중', '즌', '즐', '지', '직', '진', '질', '짐', \n             '짜', '째', '쩜', '찌', '찔', '차', '처', '척', '체', '춘', '충', '취', '치', '친', \n             '칠', '침', '칵', '코', '콜', '키', '탈', '태', '터', '테', '투', '티', '팬', '펄', \n             '풀', '하', '한', '함', '해', '했', '허', '홀', '활', '흐', '힐', '々', 'お', 'ぐ', \n             'ば', 'ゃ', 'ア', 'ド', 'ブ', 'ム', '仲', '信', '値', '入', '受', '合', '味', '回', \n             '図', '場', '届', '平', '広', '意', '憧', '戦', '所', '拠', '描', '方', '望', '期',\n             '消', '眺', '確', '空', '線', '証', '誓', '論', '起', '超', '逃', '違', '遠', '開',\n             '集', '頂', '願', '駆', '鼓', '권', '넣', '둬', '뜰', '렴', '료', '밍', '션', '솔',\n             '쓸', '윤', '응', '죄', '집', '텨', '픈', 'ŏ']\n\nfor i in range(len(testCorp)):\n    for symbol in to_remove:\n        testCorp[i] = testCorp[i].replace(symbol,\" \")","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.078115Z","iopub.execute_input":"2022-12-23T10:55:47.078551Z","iopub.status.idle":"2022-12-23T10:55:47.127742Z","shell.execute_reply.started":"2022-12-23T10:55:47.078516Z","shell.execute_reply":"2022-12-23T10:55:47.126797Z"},"trusted":true},"execution_count":25,"outputs":[]},{"cell_type":"code","source":"import csv\n\nwith open('/kaggle/working/baseline.csv', 'w+') as f:\n    writer = csv.writer(f)\n\n    for title in testCorp:\n        writer.writerow([title])","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.130908Z","iopub.execute_input":"2022-12-23T10:55:47.131267Z","iopub.status.idle":"2022-12-23T10:55:47.144244Z","shell.execute_reply.started":"2022-12-23T10:55:47.131229Z","shell.execute_reply":"2022-12-23T10:55:47.143400Z"},"trusted":true},"execution_count":26,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"3\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">DATA EXPLORATION</p>\n\n**In this section, I will be:**\n* Exploring the various artists in data\n* Explore the number of songs and their corresponding information\n* Explore the various words in lyrics via wordcloud ","metadata":{}},{"cell_type":"code","source":"#Printing the names of artists in the lyrics data\n# print(\"Artists in the data:\\n\",data.track_artist.value_counts()) ","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.146199Z","iopub.execute_input":"2022-12-23T10:55:47.146587Z","iopub.status.idle":"2022-12-23T10:55:47.151606Z","shell.execute_reply.started":"2022-12-23T10:55:47.146551Z","shell.execute_reply":"2022-12-23T10:55:47.149715Z"},"trusted":true},"execution_count":27,"outputs":[]},{"cell_type":"code","source":"#Printing the size of dataset\n# print(\"Size of Dataset:\",data.shape)","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.153102Z","iopub.execute_input":"2022-12-23T10:55:47.153755Z","iopub.status.idle":"2022-12-23T10:55:47.160153Z","shell.execute_reply.started":"2022-12-23T10:55:47.153719Z","shell.execute_reply":"2022-12-23T10:55:47.159227Z"},"trusted":true},"execution_count":28,"outputs":[]},{"cell_type":"markdown","source":"So I have a total of 745 songs\n\n**I will do a little feature engineering to extract more information on the songs such as:**\n* Number of characters\n* Number of words\n* Number of lines","metadata":{}},{"cell_type":"code","source":"#Adding a column of numbers of Characters,words and sentences in each msg\n# data[\"No_of_Characters\"] = data[\"lyrics\"].apply(len)\n# data[\"No_of_Words\"]=data.apply(lambda row: nltk.word_tokenize(row[\"lyrics\"]), axis=1).apply(len)\n# data[\"No_of_Lines\"] = data[\"lyrics\"].str.split('\\n').apply(len)\n# data.describe()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.161719Z","iopub.execute_input":"2022-12-23T10:55:47.162233Z","iopub.status.idle":"2022-12-23T10:55:47.168361Z","shell.execute_reply.started":"2022-12-23T10:55:47.162199Z","shell.execute_reply":"2022-12-23T10:55:47.167551Z"},"trusted":true},"execution_count":29,"outputs":[]},{"cell_type":"markdown","source":"To be noted: On average our songs have 1400-ish characters","metadata":{}},{"cell_type":"code","source":"#Plotting the comparative song lengths for various artists\n# plt.figure(figsize=(15,15))\n# ax = sns.pairplot(data, hue=\"track_artist\", palette=\"plasma\")","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.169443Z","iopub.execute_input":"2022-12-23T10:55:47.169754Z","iopub.status.idle":"2022-12-23T10:55:47.184047Z","shell.execute_reply.started":"2022-12-23T10:55:47.169728Z","shell.execute_reply":"2022-12-23T10:55:47.182447Z"},"trusted":true},"execution_count":30,"outputs":[]},{"cell_type":"code","source":"# Generate a word cloud image\n# stopwords = set(STOPWORDS)\n# wordcloud = WordCloud(stopwords=stopwords, background_color=\"#444160\",colormap=\"Purples\", max_words=800).generate(\" \".join(data[\"lyrics\"]))\n# plt.figure(figsize=(12,12))\n# plt.imshow(wordcloud, interpolation=\"bilinear\")\n# plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.185800Z","iopub.execute_input":"2022-12-23T10:55:47.186531Z","iopub.status.idle":"2022-12-23T10:55:47.196625Z","shell.execute_reply.started":"2022-12-23T10:55:47.186469Z","shell.execute_reply":"2022-12-23T10:55:47.195374Z"},"trusted":true},"execution_count":31,"outputs":[]},{"cell_type":"markdown","source":"*Unsurprisingly one of the most frequent words in songs is \"Love\".*\n\nNext let us look at the lyrics of song No 42 in the lyrics. Why 42 you may ask! Well! because it is the answer to the Ultimate Question of Life, the Universe, and Everything. 🐬 ","metadata":{}},{"cell_type":"code","source":"# #A function to disply the song in an asthetically pleasing way! lol\n# def My_song(song):\n#     img = Image.open(\"../input/image-for-notebook/Pink and White Geometric Marketing Presentation (1).png\")\n#     Text_on_image = ImageDraw.Draw(img)\n#     myFont = ImageFont.truetype(\"../input/font-style/DancingScript-VariableFont_wght.ttf\", 45)\n#     Text_on_image.text((620,90), song, font=myFont, fill =(255, 255, 255))\n#     return img    \n# #Having a look at the first 500 charachters of a random song lyrics\n# My_song(data.lyrics[42][:500])","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.198170Z","iopub.execute_input":"2022-12-23T10:55:47.198943Z","iopub.status.idle":"2022-12-23T10:55:47.209258Z","shell.execute_reply.started":"2022-12-23T10:55:47.198897Z","shell.execute_reply":"2022-12-23T10:55:47.207866Z"},"trusted":true},"execution_count":32,"outputs":[]},{"cell_type":"markdown","source":"The generation of text with RNN involves the following workflow.  \n\n<p style=\"background-color:#B3C5E3;font-family:newtimeroman;color:#444160;text-align:center;font-size:120%;\">Loading Data ➡️ Preprocessing ➡️ Building Mapping Dictionary ➡️ Building Model ➡️ Generating Text</p>\n\nAs I have loaded and explored the data,  I will proceed further by pre-processing the text.  \n\n\n<a id=\"4\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;color:#444160;font-size:150%;text-align:center;border-radius:20px 60px;\">DATA PREPROCESSING</p>\n\n**In this section, I will be performing the following:**\n\n**Creating a Corpus of Lyrics text:** For the model, we need a sequence of the text string. I am creating a Corpus out of Lyrics column. \n\n**Removing the unrequired characters that may have sneaked in my text corpus:** The data cleaning process for NLP is crucial preprocessing. To do that, I look into the Corpus to check for what this Corpus is comprised of. That is, all the unique symbols present. After examining the Corpus, I will be eliminating any foreign language or irrelevant symbols from the Corpus. \n\n**Creating a dictionary to map characters and their indices:** The computer doesn’t understand the text. For the computer, the text is just a cluster of symbols. It works with numbers. So we create a dictionary to map each unique character in our Corpus to a number and vice versa. This will be used to encode and decode the information going in and getting out of the RNN\n\n**Splitting the corpus into smaller sentences of equal length:** Encoding and splitting the corpus into smaller sequences of equal length: At this point, Corpus contain only intended characters (i.e, lower cap English alphabets, Numbers and a few punctuations). We will encode this corpus and create small sequences of equal lengths of features and the corresponding targets. Each feature and target will contain the mapped index in the dictionary of the unique characters they signify. \n\nThe labels are then resized and normalized. Whereas the targets are one-hot encoded. Ready to be sent to the RNN for the training, but before that let us built the RNN model. \n\n**Creating a Corpus**","metadata":{}},{"cell_type":"code","source":"#Lining up all the lyrics to create corpus\nCorpus =''\nfor listitem in data:\n    Corpus += listitem\n    \nCorpus = Corpus.lower() #converting all alphabets to lowecase \ndel data\ngc.collect()\n\n# print(\"Number of unique characters:\", len(set(Corpus)))","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.211409Z","iopub.execute_input":"2022-12-23T10:55:47.211976Z","iopub.status.idle":"2022-12-23T10:55:47.466278Z","shell.execute_reply.started":"2022-12-23T10:55:47.211933Z","shell.execute_reply":"2022-12-23T10:55:47.465419Z"},"trusted":true},"execution_count":33,"outputs":[{"execution_count":33,"output_type":"execute_result","data":{"text/plain":"176"},"metadata":{}}]},{"cell_type":"markdown","source":"The total number of unique characters present in the Corpus clearly shows, that some of the foreign language scripts have sneaked in. I will take a look at all the characters present. I will then remove the unrequired characters. ","metadata":{}},{"cell_type":"code","source":"#To See all the unique characters present in the Corpus\nprint(\"The unique characters:\",sorted(set(Corpus)))","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:47.469093Z","iopub.execute_input":"2022-12-23T10:55:47.469557Z","iopub.status.idle":"2022-12-23T10:55:47.513789Z","shell.execute_reply.started":"2022-12-23T10:55:47.469515Z","shell.execute_reply":"2022-12-23T10:55:47.510573Z"},"trusted":true},"execution_count":34,"outputs":[{"name":"stdout","text":"The unique characters: [' ', '!', '\"', '#', '$', '&', \"'\", '(', ')', '*', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '?', '_', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '~', '¡', '«', '´', '¿', 'à', 'á', 'è', 'é', 'ê', 'í', 'ï', 'ñ', 'ó', 'ú', 'ü', 'а', 'в', 'д', 'е', 'з', 'и', 'й', 'к', 'л', 'м', 'н', 'о', 'п', 'р', 'с', 'т', 'у', 'ф', 'ы', 'ь', 'ю', 'я', '\\u2005', '\\u200a', '\\u200b', '–', '—', '‘', '‚', '“', '”', '…', '가', '각', '간', '갈', '감', '같', '걍', '거', '건', '걷', '걸', '겁', '게', '겨', '경', '계', '고', '곧', '골', '공', '관', '구', '굴', '궁', '권', '그', '극', '근', '금', '기', '긴', '길', '까', '깨', '꺼', '꾸', '꿈', '끄', '끈', '끌', '끔', '끝', '끼', '낌', '나', '난', '날', '내', '냐', '너', '넌', '널', '넘', '네', '노', '논', '누', '눈', '느', '는', '늘', '니', '닌', '다', '단', '달', '대', '더', '덕', '던', '데', '도', '돌', '돼', '되', '두', '뒷', '든', '듣', '들', '듬', '듯', '디', '딜', '따', '때', '떡', '또', '똑', '뛰', '뜰', '라', '란', '람', '랑', '래', '랠', '러', '런', '렇', '레', '려', '렴', '례', '로', '료', '르', '른', '를', '름', '릎', '리', '린', '마', '만', '말', '머', '먹', '멈', '멋', '멘', '면', '모', '몰', '못', '무', '묻', '물', '뭐', '뭔', '뭘', '미', '믿', '바', '반', '받', '밤', '방', '배', '뱉', '버', '번', '범', '법', '벗', '벽', '변', '별', '보', '봐', '부', '분', '불', '비', '빌', '빛', '빠', '뻔', '뿐', '뿔', '삐', '사', '상', '새', '생', '서', '선', '성', '세', '셔', '속', '솔', '쇠', '수', '숨', '슬', '시', '신', '심', '싶', '써', '썰', '쓸', '아', '악', '안', '않', '애', '앨', '야', '어', '억', '얼', '엄', '없', '었', '에', '여', '엮', '오', '온', '올', '와', '완', '왔', '왜', '왠', '욕', '우', '움', '워', '위', '윤', '은', '을', '음', '의', '이', '인', '일', '임', '입', '있', '잊', '자', '잖', '장', '저', '전', '정', '제', '젠', '져', '좀', '좋', '죄', '주', '줄', '중', '즌', '즐', '지', '직', '진', '질', '짐', '집', '짜', '째', '쩜', '찌', '찔', '차', '처', '척', '체', '춘', '충', '취', '치', '친', '칠', '침', '칵', '코', '콜', '키', '탈', '태', '터', '테', '투', '티', '팬', '펄', '풀', '픈', '하', '한', '함', '해', '했', '허', '홀', '활', '흐', '힐']\n","output_type":"stream"}]},{"cell_type":"code","source":"#Keeping only a limited set of characters. \nto_remove = ['*', '_', '~', '¡', '«', '´', '¿', 'à',\n             'á', 'è', 'é', 'ê', 'í', 'ï', 'ñ', 'ó', 'ú', 'ü', 'а', 'в', 'д', 'е', 'з',\n             'и', 'й', 'к', 'л', 'м', 'н', 'о', 'п', 'р', 'с', 'т', 'у', 'ф', 'ы', 'ь',\n             'ю', 'я', '\\u2005', '\\u200a', '\\u200b', '\\u3000', '、', '「', '」', 'ぁ', 'あ',\n             'い', 'う', 'え', 'か', 'が', 'き', 'ぎ', 'く', 'け', 'げ', 'こ', 'さ', 'し', 'じ',\n             'す', 'せ', 'そ', 'た', 'だ', 'ち', 'っ', 'つ', 'づ', 'て', 'で', 'と', 'ど', 'な',\n             'に', 'ぬ', 'ね', 'の', 'は', 'び', 'ふ', 'ぶ', 'へ', 'べ', 'ほ', 'ま', 'み', 'む',\n             'め', 'も', 'よ', 'ら', 'り', 'る', 'れ', 'ろ', 'わ', 'を', 'ん', 'イ', 'ウ', 'カ', \n             'キ', 'ギ', 'ク', 'シ', 'ス', 'タ', 'ダ', 'ッ', 'ツ', 'テ', 'ト', 'ネ', 'ノ', 'ビ', \n             'ポ', 'ミ', 'メ', 'モ', 'ヤ', 'ョ', 'リ', 'ル', 'レ', 'ン', 'ー', '一', '上', '世', \n             '中', '乗', '事', '人', '仆', '今', '他', '代', '体', '何', '侵', '俺', '像', '僕', \n             '償', '先', '光', '全', '共', '分', '切', '別', '前', '動', '去', '口', '叫', '名', \n             '向', '君', '呼', '咲', '嘘', '在', '壁', '壊', '声', '変', '夜', '夢', '大', '孤',\n             '宿', '対', '尽', '巡', '巻', '序', '度', '弱', '張', '強', '影', '待', '後', '心',\n             '恍', '恐', '悟', '悪', '情', '惚', '想', '愛', '感', '戻', '手', '抜', '拓', '指', \n             '振', '探', '揺', '支', '教', '日', '明', '昔', '昙', '時', '書', '最', '未', '来', \n             '極', '歓', '止', '正', '歩', '比', '気', '泣', '溺', '無', '特', '独', '珠', '現',\n             '理', '由', '界', '痛', '白', '目', '真', '着', '瞳', '知', '破', '磨', '私', '突',\n             '章', '笑', '筋', '続', '置', '肥', '胸', '自', '良', '薇', '薔', '蝕', '行', '衝', \n             '裸', '襲', '見', '覚', '见', '触', '言', '誇', '誰', '越', '踠', '身', '輝', '过',\n             '返', '迷', '進', '過', '配', '重', '間', '闇', '限', '頭', '食', '高', '鳴', '갑',\n             '개', '걱', '것', '겐', '겠', '견', '결', '곁', '곡', '곳', '과', '광', '교', '국',\n             '군', '귀', '글', '급', '긋', '깊', '깐', '깜', '깝', '꺾', '께', '껴', '꼬', '꼭',\n             '꼴', '꽃', '꽉', '꽤', '꿰', '끊', '끽', '남', '낮', '냥', '넓', '녁', '녔', '녕',\n             '놀', '높', '놓', '놔', '놨', '뉴', '능', '닐', '님', '답', '당', '닿', '덜', '덩',\n             '델', '독', '동', '된', '될', '둑', '둘', '둡', '뒤', '드', '득', '듭', '딱', '땀',\n             '땡', '떠', '떤', '떨', '떻', '떼', '뚜', '뚫', '뜨', '뜻', '띄', '락', '럭', '럼',\n             '럽', '럿', '렘', '력', '렵', '렸', '록', '론', '롭', '루', '릇', '릭', '릴', '림',\n             '릿', '막', '많', '맘', '맛', '망', '맞', '맡', '맣', '매', '맨', '맴', '먼', '멀',\n             '며', '명', '몇', '목', '몸', '문', '민', '밀', '밖', '발', '밝', '밟', '백', '벌',\n             '베', '벼', '볍', '복', '본', '볼', '봉', '붉', '뷰', '빤', '빨', '뻑', '뻥', '뿌',\n             '뿜', '쁘', '살', '삶', '석', '설', '섬', '섭', '섹', '셀', '셋', '소', '손', '솝', \n             '송', '숑', '순', '숴', '쉬', '쉽', '쉿', '슈', '스', '슨', '슴', '습', '식', '실',\n             '싫', '십', '쌔', '쌤', '쏟', '쏴', '쑤', '쓰', '씩', '씹', '앉', '알', '았', '앞',\n             '약', '양', '얗', '언', '엇', '엔', '열', '엽', '였', '영', '옆', '옇', '예', '옥',\n             '외', '요', '용', '운', '울', '웃', '웅', '원', '윈', '유', '으', '익', '작', '잘', \n             '잠', '잡', '재', '잭', '적', '절', '젊', '점', '조', '존', '죽', '준', '줌', '줘', \n             '줬', '쥐', '증', '짓', '짝', '짧', '쩌', '찍', '착', '참', '창', '찾', '채', '챘', \n             '천', '철', '청', '쳐', '쳤', '초', '추', '축', '출', '춤', '춰', '췬', '칙', '카', \n             '커', '켜', '콘', '콤', '콱', '쿵', '크', '큰', '큼', '킬', '타', '탁', '탓', '탬', \n             '텐', '텝', '토', '통', '툭', '툰', '툴', '트', '특', '틀', '팅', '파', '팔', '패',\n             '퍼', '펴', '편', '펼', '폈', '평', '포', '폼', '표', '품', '프', '플', '피', '필',\n             '학', '할', '항', '핵', '행', '향', '험', '헤', '헷', '혀', '현', '호', '혹', '혼',\n             '화', '확', '환', '황', '회', '효', '후', '훅', '훔', '흔', '흘', '히', '힘', 'ﬁ', \n             '\\ufeff', '\\xa0', 'É', 'ä', 'å', 'ç', 'ì', 'î', 'ò', 'ô', 'ö', 'ù', 'А', 'П',\n             '가', '각', '간', '갈', '감', '같', '걍', '거', '건', '걷', '걸', '겁', '게', '겨', \n             '경', '계', '고', '곧', '골', '공', '관', '구', '굴', '궁', '그', '극', '근', '금', \n             '기', '긴', '길', '까', '깨', '꺼', '꾸', '꿈', '끄', '끈', '끌', '끔', '끝', '끼', \n             '낌', '나', '난', '날', '내', '냐', '너', '넌', '널', '넘', '네', '노', '논', '누', \n             '눈', '느', '는', '늘', '니', '닌', '다', '단', '달', '대', '더', '덕', '던', '데', \n             '도', '돌', '돼', '되', '두', '뒷', '든', '듣', '들', '듬', '듯', '디', '딜', '따', \n             '때', '떡', '또', '똑', '뛰', '라', '란', '람', '랑', '래', '랠', '러', '런', '렇', \n             '레', '려', '례', '로', '르', '른', '를', '름', '릎', '리', '린', '마', '만', '말', \n             '머', '먹', '멈', '멋', '멘', '면', '모', '몰', '못', '무', '묻', '물', '뭐', '뭔', \n             '뭘', '미', '믿', '바', '반', '받', '밤', '방', '배', '뱉', '버', '번', '범', '법', \n             '벗', '벽', '변', '별', '보', '봐', '부', '분', '불', '비', '빌', '빛', '빠', '뻔',\n             '뿐', '뿔', '삐', '사', '상', '새', '생', '서', '선', '성', '세', '셔', '속', '쇠',\n             '수', '숨', '슬', '시', '신', '심', '싶', '써', '썰', '아', '악', '안', '않', '애', \n             '앨', '야', '어', '억', '얼', '엄', '없', '었', '에', '여', '엮', '오', '온', '올', \n             '와', '완', '왔', '왜', '왠', '욕', '우', '움', '워', '위', '은', '을', '음', '의',\n             '이', '인', '일', '임', '입', '있', '잊', '자', '잖', '장', '저', '전', '정', '제',\n             '젠', '져', '좀', '좋', '주', '줄', '중', '즌', '즐', '지', '직', '진', '질', '짐', \n             '짜', '째', '쩜', '찌', '찔', '차', '처', '척', '체', '춘', '충', '취', '치', '친', \n             '칠', '침', '칵', '코', '콜', '키', '탈', '태', '터', '테', '투', '티', '팬', '펄', \n             '풀', '하', '한', '함', '해', '했', '허', '홀', '활', '흐', '힐', '々', 'お', 'ぐ', \n             'ば', 'ゃ', 'ア', 'ド', 'ブ', 'ム', '仲', '信', '値', '入', '受', '合', '味', '回', \n             '図', '場', '届', '平', '広', '意', '憧', '戦', '所', '拠', '描', '方', '望', '期',\n             '消', '眺', '確', '空', '線', '証', '誓', '論', '起', '超', '逃', '違', '遠', '開',\n             '集', '頂', '願', '駆', '鼓', '권', '넣', '둬', '뜰', '렴', '료', '밍', '션', '솔',\n             '쓸', '윤', '응', '죄', '집', '텨', '픈', 'ŏ']\nfor symbol in to_remove:\n    Corpus = Corpus.replace(symbol,\" \")\n    \ndel to_remove\ngc.collect()\n\n#Corpus = re.sub(\"[^A-Za-z0-9'\\.\\n]\",\"\",Corpus) Alterneativly could be used but I want to pick and chose (:","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-12-23T10:55:47.517286Z","iopub.execute_input":"2022-12-23T10:55:47.519963Z","iopub.status.idle":"2022-12-23T10:55:48.296996Z","shell.execute_reply.started":"2022-12-23T10:55:47.519922Z","shell.execute_reply":"2022-12-23T10:55:48.294593Z"},"trusted":true},"execution_count":35,"outputs":[{"execution_count":35,"output_type":"execute_result","data":{"text/plain":"0"},"metadata":{}}]},{"cell_type":"code","source":"#To See all the unique characters present in the Corpus\nprint(\"The unique characters:\",sorted(set(Corpus)))","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:48.301171Z","iopub.execute_input":"2022-12-23T10:55:48.303309Z","iopub.status.idle":"2022-12-23T10:55:48.342385Z","shell.execute_reply.started":"2022-12-23T10:55:48.303266Z","shell.execute_reply":"2022-12-23T10:55:48.341437Z"},"trusted":true},"execution_count":36,"outputs":[{"name":"stdout","text":"The unique characters: [' ', '!', '\"', '#', '$', '&', \"'\", '(', ')', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '?', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '–', '—', '‘', '‚', '“', '”', '…']\n","output_type":"stream"}]},{"cell_type":"markdown","source":"**Creating a list of sorted unique characters**","metadata":{}},{"cell_type":"code","source":"# Storing all the unique characters present in my corpus to bult a mapping dic. \nsymb = sorted(list(set(Corpus)))\n\nL_corpus = len(Corpus) #length of corpus\nL_symb = len(symb) #length of total unique characters\n\n#Building dictionary to access the vocabulary from indices and vice versa\nmapping = dict((c, i) for i, c in enumerate(symb))\nreverse_mapping = dict((i, c) for i, c in enumerate(symb))\n\nprint(\"Total number of characters:\", L_corpus)\nprint(\"Number of unique characters:\", L_symb)\ndel symb\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:48.343856Z","iopub.execute_input":"2022-12-23T10:55:48.344601Z","iopub.status.idle":"2022-12-23T10:55:48.540561Z","shell.execute_reply.started":"2022-12-23T10:55:48.344561Z","shell.execute_reply":"2022-12-23T10:55:48.539582Z"},"trusted":true},"execution_count":37,"outputs":[{"name":"stdout","text":"Total number of characters: 2226491\nNumber of unique characters: 59\n","output_type":"stream"},{"execution_count":37,"output_type":"execute_result","data":{"text/plain":"44"},"metadata":{}}]},{"cell_type":"code","source":"#Splitting the Corpus in equal length of strings and output target\nlength = 40\nfeatures = []\ntargets = []\nfor i in range(0, L_corpus - length, 1):\n    feature = Corpus[i:i + length]\n    target = Corpus[i + length]\n    features.append([mapping[j] for j in feature])\n    targets.append(mapping[target])\n    \n    \nL_datapoints = len(targets)\nprint(\"Total number of sequences in the Corpus:\", L_datapoints)","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:55:48.541852Z","iopub.execute_input":"2022-12-23T10:55:48.542365Z","iopub.status.idle":"2022-12-23T10:56:03.413266Z","shell.execute_reply.started":"2022-12-23T10:55:48.542309Z","shell.execute_reply":"2022-12-23T10:56:03.411751Z"},"trusted":true},"execution_count":38,"outputs":[{"name":"stdout","text":"Total number of sequences in the Corpus: 2226451\n","output_type":"stream"}]},{"cell_type":"markdown","source":"**Encoding the Labels and Targets**","metadata":{}},{"cell_type":"code","source":"# reshape X and normalize\nX = (np.reshape(features, (L_datapoints, length, 1)))/ float(L_symb)\ndel features, L_datapoints, L_corpus\ngc.collect()\n\n# one hot encode the output variable\ny = np_utils.to_categorical(targets)\ndel targets\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:03.414590Z","iopub.execute_input":"2022-12-23T10:56:03.414937Z","iopub.status.idle":"2022-12-23T10:56:11.408486Z","shell.execute_reply.started":"2022-12-23T10:56:03.414900Z","shell.execute_reply":"2022-12-23T10:56:11.407570Z"},"trusted":true},"execution_count":39,"outputs":[{"execution_count":39,"output_type":"execute_result","data":{"text/plain":"0"},"metadata":{}}]},{"cell_type":"markdown","source":"<a id=\"5\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;font-size:150%;color:#444160;text-align:center;border-radius:20px 60px;\">MODEL BUILDING</p>\n\n\nRecurrent Neural Networks are pretty popular with generating text. In this project, I will be using a LSTM Model, an improved version of a standard recurrent neural network\n\n**Following steps are involved in the model building**\n\n* Initialising the Model\n* Defining by adding layers\n* Compiling the Model\n* Training the Model\n\n**Building the Model**","metadata":{}},{"cell_type":"code","source":"#Initialising the Model\nmodel = Sequential()\n#Adding layers\nmodel.add(LSTM(256, input_shape=(X.shape[1], X.shape[2])))\nmodel.add(Dense(y.shape[1],f activation='softmax'))\n#Compiling the model for training  \nopt = Adamax(learning_rate=0.01)\nmodel.compile(loss='categorical_crossentropy', optimizer=opt)\n\n#Model's Summary               \nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.412283Z","iopub.execute_input":"2022-12-23T10:56:11.412641Z","iopub.status.idle":"2022-12-23T10:56:11.422912Z","shell.execute_reply.started":"2022-12-23T10:56:11.412601Z","shell.execute_reply":"2022-12-23T10:56:11.421424Z"},"trusted":true},"execution_count":40,"outputs":[{"traceback":["\u001b[0;36m  File \u001b[0;32m\"<ipython-input-40-a9a31df2e36b>\"\u001b[0;36m, line \u001b[0;32m5\u001b[0m\n\u001b[0;31m    model.add(Dense(y.shape[1],f activation='softmax'))\u001b[0m\n\u001b[0m                                          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"],"ename":"SyntaxError","evalue":"invalid syntax (<ipython-input-40-a9a31df2e36b>, line 5)","output_type":"error"}]},{"cell_type":"code","source":"#Training the Model\nhistory = model.fit(X, y, batch_size=128, epochs=100)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-12-23T10:56:11.423990Z","iopub.status.idle":"2022-12-23T10:56:11.425263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\n#To be used later; I am saving the model \nmodel.save(\"Lyrics_Generator.h5\")","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-12-23T10:56:11.428205Z","iopub.status.idle":"2022-12-23T10:56:11.430328Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"<a id=\"6\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;font-size:150%;color:#444160;text-align:center;border-radius:20px 60px;\">EVALUATING MODELS</p>\n\nNow that I have my model trained on the songs lyrics let us see how it performs. I hope it creates some sensible song.\n\n**To evaluate my model, I shall be having a look at:**\n* The performance of the model via Learning Curves\n* The outcome text it generates\n\n**Plotting the learning curve for the loss function**","metadata":{}},{"cell_type":"code","source":"# history_df = pd.DataFrame(history.history)\n# #Plotting the learnings \n\n# fig = plt.figure(figsize=(15,4), facecolor=\"#B291B6\")\n# fig.suptitle(\"Learning Plot of Model for Loss\")\n# pl=sns.lineplot(data=history_df[\"loss\"],color=\"#444160\")\n# pl.set(ylabel =\"Training Loss\")\n# pl.set(xlabel =\"Epochs\")","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.432810Z","iopub.status.idle":"2022-12-23T10:56:11.433542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Generating the songs**","metadata":{}},{"cell_type":"code","source":"# The function to generate text from model\ndef Lyrics_Generator(starter,Ch_count): #,temperature=1.0):\n    generated= \"\"\n    starter = starter \n    seed=[mapping[char] for char in starter]\n    print(seed)\n    generated += starter \n    # Generating new text of given length\n    for i in range(Ch_count):\n        seed=[mapping[char] for char in starter]\n        x_pred = np.reshape(seed, (1, len(seed), 1))\n        x_pred = x_pred/ float(L_symb)\n        prediction = model.predict(x_pred, verbose=0)[0]  \n        # Getting the index of the next most probable index\n        prediction = np.asarray(prediction).astype('float64')\n        prediction = np.log(prediction) / 1.0 \n        exp_preds = np.exp(prediction)\n        prediction = exp_preds / np.sum(exp_preds)\n        probas = np.random.multinomial(1, prediction, 1)\n        index = np.argmax(prediction)\n        next_char = reverse_mapping[index]  \n        # Generating new text\n        generated += next_char\n        starter = starter[1:] + next_char\n       \n    return generated","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.441239Z","iopub.status.idle":"2022-12-23T10:56:11.441762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let us finally see the outcome by putting a seed in from one of my old blogposts [blogpost](https://karnikakapoor.blogspot.com/2017/04/killers-confession.html)","metadata":{}},{"cell_type":"code","source":"# for i in test_data:\n#     print(i)\n#     print()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.443178Z","iopub.status.idle":"2022-12-23T10:56:11.443697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"testCorp = []\n\nfor listitem in test_data:\n    testCorp.append(listitem.lower())","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.445052Z","iopub.status.idle":"2022-12-23T10:56:11.445586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Keeping only a limited set of characters. \nto_remove = ['*', '_', '~', '¡', '«', '´', '¿', 'à',\n             'á', 'è', 'é', 'ê', 'í', 'ï', 'ñ', 'ó', 'ú', 'ü', 'а', 'в', 'д', 'е', 'з',\n             'и', 'й', 'к', 'л', 'м', 'н', 'о', 'п', 'р', 'с', 'т', 'у', 'ф', 'ы', 'ь',\n             'ю', 'я', '\\u2005', '\\u200a', '\\u200b', '\\u3000', '、', '「', '」', 'ぁ', 'あ',\n             'い', 'う', 'え', 'か', 'が', 'き', 'ぎ', 'く', 'け', 'げ', 'こ', 'さ', 'し', 'じ',\n             'す', 'せ', 'そ', 'た', 'だ', 'ち', 'っ', 'つ', 'づ', 'て', 'で', 'と', 'ど', 'な',\n             'に', 'ぬ', 'ね', 'の', 'は', 'び', 'ふ', 'ぶ', 'へ', 'べ', 'ほ', 'ま', 'み', 'む',\n             'め', 'も', 'よ', 'ら', 'り', 'る', 'れ', 'ろ', 'わ', 'を', 'ん', 'イ', 'ウ', 'カ', \n             'キ', 'ギ', 'ク', 'シ', 'ス', 'タ', 'ダ', 'ッ', 'ツ', 'テ', 'ト', 'ネ', 'ノ', 'ビ', \n             'ポ', 'ミ', 'メ', 'モ', 'ヤ', 'ョ', 'リ', 'ル', 'レ', 'ン', 'ー', '一', '上', '世', \n             '中', '乗', '事', '人', '仆', '今', '他', '代', '体', '何', '侵', '俺', '像', '僕', \n             '償', '先', '光', '全', '共', '分', '切', '別', '前', '動', '去', '口', '叫', '名', \n             '向', '君', '呼', '咲', '嘘', '在', '壁', '壊', '声', '変', '夜', '夢', '大', '孤',\n             '宿', '対', '尽', '巡', '巻', '序', '度', '弱', '張', '強', '影', '待', '後', '心',\n             '恍', '恐', '悟', '悪', '情', '惚', '想', '愛', '感', '戻', '手', '抜', '拓', '指', \n             '振', '探', '揺', '支', '教', '日', '明', '昔', '昙', '時', '書', '最', '未', '来', \n             '極', '歓', '止', '正', '歩', '比', '気', '泣', '溺', '無', '特', '独', '珠', '現',\n             '理', '由', '界', '痛', '白', '目', '真', '着', '瞳', '知', '破', '磨', '私', '突',\n             '章', '笑', '筋', '続', '置', '肥', '胸', '自', '良', '薇', '薔', '蝕', '行', '衝', \n             '裸', '襲', '見', '覚', '见', '触', '言', '誇', '誰', '越', '踠', '身', '輝', '过',\n             '返', '迷', '進', '過', '配', '重', '間', '闇', '限', '頭', '食', '高', '鳴', '갑',\n             '개', '걱', '것', '겐', '겠', '견', '결', '곁', '곡', '곳', '과', '광', '교', '국',\n             '군', '귀', '글', '급', '긋', '깊', '깐', '깜', '깝', '꺾', '께', '껴', '꼬', '꼭',\n             '꼴', '꽃', '꽉', '꽤', '꿰', '끊', '끽', '남', '낮', '냥', '넓', '녁', '녔', '녕',\n             '놀', '높', '놓', '놔', '놨', '뉴', '능', '닐', '님', '답', '당', '닿', '덜', '덩',\n             '델', '독', '동', '된', '될', '둑', '둘', '둡', '뒤', '드', '득', '듭', '딱', '땀',\n             '땡', '떠', '떤', '떨', '떻', '떼', '뚜', '뚫', '뜨', '뜻', '띄', '락', '럭', '럼',\n             '럽', '럿', '렘', '력', '렵', '렸', '록', '론', '롭', '루', '릇', '릭', '릴', '림',\n             '릿', '막', '많', '맘', '맛', '망', '맞', '맡', '맣', '매', '맨', '맴', '먼', '멀',\n             '며', '명', '몇', '목', '몸', '문', '민', '밀', '밖', '발', '밝', '밟', '백', '벌',\n             '베', '벼', '볍', '복', '본', '볼', '봉', '붉', '뷰', '빤', '빨', '뻑', '뻥', '뿌',\n             '뿜', '쁘', '살', '삶', '석', '설', '섬', '섭', '섹', '셀', '셋', '소', '손', '솝', \n             '송', '숑', '순', '숴', '쉬', '쉽', '쉿', '슈', '스', '슨', '슴', '습', '식', '실',\n             '싫', '십', '쌔', '쌤', '쏟', '쏴', '쑤', '쓰', '씩', '씹', '앉', '알', '았', '앞',\n             '약', '양', '얗', '언', '엇', '엔', '열', '엽', '였', '영', '옆', '옇', '예', '옥',\n             '외', '요', '용', '운', '울', '웃', '웅', '원', '윈', '유', '으', '익', '작', '잘', \n             '잠', '잡', '재', '잭', '적', '절', '젊', '점', '조', '존', '죽', '준', '줌', '줘', \n             '줬', '쥐', '증', '짓', '짝', '짧', '쩌', '찍', '착', '참', '창', '찾', '채', '챘', \n             '천', '철', '청', '쳐', '쳤', '초', '추', '축', '출', '춤', '춰', '췬', '칙', '카', \n             '커', '켜', '콘', '콤', '콱', '쿵', '크', '큰', '큼', '킬', '타', '탁', '탓', '탬', \n             '텐', '텝', '토', '통', '툭', '툰', '툴', '트', '특', '틀', '팅', '파', '팔', '패',\n             '퍼', '펴', '편', '펼', '폈', '평', '포', '폼', '표', '품', '프', '플', '피', '필',\n             '학', '할', '항', '핵', '행', '향', '험', '헤', '헷', '혀', '현', '호', '혹', '혼',\n             '화', '확', '환', '황', '회', '효', '후', '훅', '훔', '흔', '흘', '히', '힘', 'ﬁ', \n             '\\ufeff', '\\xa0', 'É', 'ä', 'å', 'ç', 'ì', 'î', 'ò', 'ô', 'ö', 'ù', 'А', 'П',\n             '가', '각', '간', '갈', '감', '같', '걍', '거', '건', '걷', '걸', '겁', '게', '겨', \n             '경', '계', '고', '곧', '골', '공', '관', '구', '굴', '궁', '그', '극', '근', '금', \n             '기', '긴', '길', '까', '깨', '꺼', '꾸', '꿈', '끄', '끈', '끌', '끔', '끝', '끼', \n             '낌', '나', '난', '날', '내', '냐', '너', '넌', '널', '넘', '네', '노', '논', '누', \n             '눈', '느', '는', '늘', '니', '닌', '다', '단', '달', '대', '더', '덕', '던', '데', \n             '도', '돌', '돼', '되', '두', '뒷', '든', '듣', '들', '듬', '듯', '디', '딜', '따', \n             '때', '떡', '또', '똑', '뛰', '라', '란', '람', '랑', '래', '랠', '러', '런', '렇', \n             '레', '려', '례', '로', '르', '른', '를', '름', '릎', '리', '린', '마', '만', '말', \n             '머', '먹', '멈', '멋', '멘', '면', '모', '몰', '못', '무', '묻', '물', '뭐', '뭔', \n             '뭘', '미', '믿', '바', '반', '받', '밤', '방', '배', '뱉', '버', '번', '범', '법', \n             '벗', '벽', '변', '별', '보', '봐', '부', '분', '불', '비', '빌', '빛', '빠', '뻔',\n             '뿐', '뿔', '삐', '사', '상', '새', '생', '서', '선', '성', '세', '셔', '속', '쇠',\n             '수', '숨', '슬', '시', '신', '심', '싶', '써', '썰', '아', '악', '안', '않', '애', \n             '앨', '야', '어', '억', '얼', '엄', '없', '었', '에', '여', '엮', '오', '온', '올', \n             '와', '완', '왔', '왜', '왠', '욕', '우', '움', '워', '위', '은', '을', '음', '의',\n             '이', '인', '일', '임', '입', '있', '잊', '자', '잖', '장', '저', '전', '정', '제',\n             '젠', '져', '좀', '좋', '주', '줄', '중', '즌', '즐', '지', '직', '진', '질', '짐', \n             '짜', '째', '쩜', '찌', '찔', '차', '처', '척', '체', '춘', '충', '취', '치', '친', \n             '칠', '침', '칵', '코', '콜', '키', '탈', '태', '터', '테', '투', '티', '팬', '펄', \n             '풀', '하', '한', '함', '해', '했', '허', '홀', '활', '흐', '힐', '々', 'お', 'ぐ', \n             'ば', 'ゃ', 'ア', 'ド', 'ブ', 'ム', '仲', '信', '値', '入', '受', '合', '味', '回', \n             '図', '場', '届', '平', '広', '意', '憧', '戦', '所', '拠', '描', '方', '望', '期',\n             '消', '眺', '確', '空', '線', '証', '誓', '論', '起', '超', '逃', '違', '遠', '開',\n             '集', '頂', '願', '駆', '鼓', '권', '넣', '둬', '뜰', '렴', '료', '밍', '션', '솔',\n             '쓸', '윤', '응', '죄', '집', '텨', '픈', 'ŏ']\n\nfor i in range(len(testCorp)):\n    for symbol in to_remove:\n        testCorp[i] = testCorp[i].replace(symbol,\" \")\n    \ndel to_remove\ngc.collect()\n\n#Corpus = re.sub(\"[^A-Za-z0-9'\\.\\n]\",\"\",Corpus) Alterneativly could be used but I want to pick and chose (:","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.451279Z","iopub.status.idle":"2022-12-23T10:56:11.451804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for title in testCorp:\n#     print(title)\n#     print()","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.453099Z","iopub.status.idle":"2022-12-23T10:56:11.453758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Generating a song from the model\nresults = {}\n\nfor title in testCorp: \n    beginning = \" \".join(title.split()[0:5])\n    beginning = beginning.lower()\n    song = Lyrics_Generator(beginning, 400)\n    results[beginning] = song","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.455124Z","iopub.status.idle":"2022-12-23T10:56:11.455682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(results))","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.457026Z","iopub.status.idle":"2022-12-23T10:56:11.457540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import csv # move up\n\nwith open('/kaggle/working/baseline_lyrics.csv', 'w+') as f:\n    writer = csv.writer(f)\n    \n    header = ['Prompt', 'Generation', 'Coherence', 'Musical Quality', 'Pop Style']\n    writer.writerow(header)\n\n    for prompt, lyrics in results.items():\n        writer.writerow([prompt, lyrics])","metadata":{"execution":{"iopub.status.busy":"2022-12-23T10:56:11.464172Z","iopub.status.idle":"2022-12-23T10:56:11.464687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Another song generated by a seed of the lyrics of a song that's stuck in my head today. (Sunflower by Shannon Purser)","metadata":{}},{"cell_type":"markdown","source":"# <a id=\"7\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;font-size:150%;color:#444160;text-align:center;border-radius:20px 60px;\">CONCLUSION</p>\n<p style=\"font-family:newtimeroman;font-size:120%;color:#444160\">On observing the output of the Lyrics Generator, it is clear that while some of the sentences might be correct, but most of the lyrics do not make sense. It does look like a song tho. The model didn't learn the meaning of the songs. However, the character-based approach is producing some legitimate words. \nTo get to a song that makes better sense I may consider a transformer-based text generator, but that's for some other time.</p>\n\n<p style=\"font-family:newtimeroman;font-size:120%;color:#444160;\"> Neural networks amaze me all the time. There is something surreal about them that makes working with it exciting. Of course, we can peel off the layers and see the maths behind them. Get to the matrices and tensor to understand how these neurons are working. Even get the values of weights and biases and assure ourselves that this is no sorcery.  Still, when I see the result play out it is astonishing.</p>  \n\n**Some useful resources on Text generation:**\n\n[The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)\n\n[RNN ](https://towardsdatascience.com/character-level-language-model-1439f5dd87fe)\n\n[GTP-2 ](https://openai.com/blog/better-language-models/) \n\n\n**<span style=\"color:#444160;\"> If you liked this Notebook, please do upvote.</span>**\n\n**<span style=\"color:#444160;\"> Best Wishes!</span>**\n\n<a id=\"8\"></a>\n# <p style=\"background-color:#B291B6;font-family:newtimeroman;font-size:150%;color:#444160;text-align:center;border-radius:20px 60px;\">END</p>","metadata":{}}]}