ASHRAE_Great_Energy_Predictor_III.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "ASHRAE - Great Energy Predictor III.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/pea-sys/Kaggle_Public/blob/master/ASHRAE_Great_Energy_Predictor_III.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I98UzJLzJcJ7",
        "colab_type": "text"
      },
      "source": [
        "# ***ASHRAE - Great Energy Predictor III Competition NoteBook***\n",
        "\n",
        "## 【概要】(※Official和訳)\n",
        "建物の効率を改善してコストと排出量を削減するために、多大な投資が行われています。では、改善は機能していますか？それがあなたの出番です。  \n",
        "現在の推定方法は断片化されており、うまくスケールしません。特定のメータータイプを想定している場合や、異なる建物タイプでは機能しない場合もあります。  \n",
        "\n",
        "省エネの開発には2つの重要な要素があります：改善なしで将来のエネルギー使用量を予測することと、特定の改善セットが実装された後のエネルギー使用量を予測します。  \n",
        "エネルギー市場のより積極的な成長を妨げる1つの問題は、エネルギー使用量を予測するための費用効果が高く、正確で、スケーラブルな手順がないことです。   \n",
        "\n",
        "このコンペティションでは、冷水、電気、天然ガス、温水、蒸気メーターなどの分野で、建物のエネルギー使用量を正確に予測します。データは、3年間で1,000を超える建物から取得されます。 これらの省エネ投資のより良い推定により、大規模な投資家や金融機関は、効率性の構築の進展を可能にするために、この分野に投資する傾向が強くなります。  \n",
        "\n",
        "主催：ASHRAE社\n",
        "\n",
        "## 【評価指標】\n",
        "二乗平均平方根誤差(RMSLE)です\n",
        "目的変数が裾の重い分布を持ち、変換しないままだと大きな値の影響が強い場合や、真の値と予測値の比率に着目したい場合に用いられる。  \n",
        "対数をとるにあたっては、真の値が0のときに値が負に発散するのを避けるため、通常は1を加えてから対数をとる。numpyのlog1p関数が使用できる。\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "### 〇データセット\n",
        "■train.csv -- shape(20125605, 16)\n",
        "* building_id - 建物のメタデータの外部キー\n",
        "* meter - メーターIDコード。 {0：電気、1：冷水、2：蒸気、3：温水}と読みます。 すべての建物にすべてのメータータイプがあるわけではありません。\n",
        "* timestamp - 測定が行われたとき\n",
        "* meter_reading - ターゲット変数。 kWh（または同等の）単位のエネルギー消費。これは、測定誤差のある実際のデータであり、モデリング誤差のベースラインレベルを課すと予想されることに注意してください。\n",
        "\n",
        "■test.csv --- shape(41697600, 16)\n",
        "\n",
        "■building_meta.csv\n",
        "* site_id - wether.csvの外部キー。場所を表している。\n",
        "* building_id - training.csvの外部キー。\n",
        "* primary_use - [EnergyStarプロパティタイプの定義に基づいた、建物の主要な活動カテゴリのインジケータ](https://www.energystar.gov/buildings/facility-owners-and-managers/existing-buildings/use-portfolio-manager/identify-your-property-type)\n",
        "* square_feet - 建物の総床面積\n",
        "* year_built - 建物がオープンした年\n",
        "* floor_count - 建物の階数  \n",
        "\n",
        "■weather_[train/test].csv\n",
        "* site_id - wether.csvの外部キー。場所を表している。\n",
        "* air_temperature - 大気温　摂氏\n",
        "* cloud_coverage -オクタの雲に覆われた空の部分。気象学では、オクタは、気象観測所などの特定の場所での雲量を表すために使用される測定単位です。\n",
        "* dew_temperature - 露温度　摂氏\n",
        "* precip_depth_1_hr - 降水量(ミリ)\n",
        "* sea_level_pressure - 海水圧\n",
        "* wind_direction - 風向き\n",
        "* wind_speed - 風速(メートル毎秒)\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "###〇スケジュール\n",
        "* 2019年12月12日-チーム合併の締め切り。 これは、参加者がチームに参加またはマージできる最後の日です。\n",
        "* 2019年12月12日-応募締切。 競争するには、この日付の前に競争規則に同意する必要があります。\n",
        "* 2019年12月19日-最終提出期限\n",
        "\n",
        "\n",
        "---\n",
        "## ★活動記録\n",
        "- 10/16 コンペティション参加\n",
        "- 10/17 実験を回すためにデータのpickle化\n",
        "- 10/19～10/24 LGBMベースラインサブミット(Score 1.28)  。特徴生成するも効かず。たいして特徴追加していないが、早い段階でメモリ不足に悩まされる(Google Colab使用)。\n",
        "- 10/25 メモリ不足に対応するため、訓練データと評価データをsite_id毎に分割し、site_id毎に予測モデルを作った。ディスカッションにsite_id毎に時刻がずれているという書き込みあり。\n",
        "- 10/28 site_id=0の初月2か月分のメーター値が0だったので、ノイズデータとしてドロップ\n",
        "- 10/29～11/2 特徴量を色々こねくり回したがスコア上がらず\n",
        "- 11/3  [公開カーネル](https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type)を参考にメーターの種別毎にモデルを用意するように方針変更  \n",
        " 　　 前処理し銀メダル圏に入る(Score 1.1)  \n",
        " 　　ドロップ対象はイレギュラー的にメーター読み取り値が0になっていそうな箇所。これらのデータが含まれるビルの予測値がかなり低く見えたため削除した。\n",
        "- 11/5 サブミッションファイルの0.1以下クリッピング\n",
        "- 11/6～11/8 サブミッションファイルと訓練データの目視比較。特徴量設計しても相変わらず効かない。  \n",
        "- 11/10 メーター読み取り値0のデータを更に削除(Score 1.08)  \n",
        "- 11/13 訓練データの外れ値をクリッピング(Score1.07)  \n",
        "- 11/15一部のビルの予測値がリークしていることが発覚  \n",
        "こちらの[カーネル](https://www.kaggle.com/yamsam/ashrae-leak-data-station)が公平性を期するために逐次リークデータを提供してくれていた。リークデータは他にもあるとのことだったが、そこに力を入れるのはコンペの本筋ではないので定期的に上記カーネルの監視する程度で自分から探すことはしなかった。公開してくれる人には感謝。\n",
        "- 11/16 各ビル毎に時刻別にメーター値平均を算出。ビルでの活動が非アクティブな場合は、メーター値が分散していないはずと考え、そこだけ切り出して線形回帰で予測。(Score 1.04)\n",
        "- 11/17-11/19 ノートブック整理。最高スコアの再現性確認。\n",
        "- 11/20 評価関数のカスタム。rmsleの+1を0.01とかに変えてみたら精度上がるかと思ったけど、そんなことはなかった。  \n",
        "- 11/21 11/3にベースとして採用したカーネルはハイパーパラメータを1つのメーター種別で最適化していたので、各メーター毎に最適化するように修正。スコア変化なし。\n",
        "- 11/22 更新されていた公開リークデータ使用(Score 0.99)\n",
        "- 11/24 Score1.0の[公開カーネル](https://www.kaggle.com/huanglinyan/ashrae-may-make-it-up-to-1-0)とアンサンブル（Manual Weight)(Score 0.97)  \n",
        "- 11/25 Score0.99の[公開カーネル](https://www.kaggle.com/yamsam/ashrae-leak-validation-and-more)とアンサンブル（Manual Weight)(Score 0.96)\n",
        "- 11/26 最新リークデータ適用(Score 0.96) ※多分プライベートセット部分が更新\n",
        "- 11/27-11/30 11/16の延長で低分散区間を週毎、月毎、曜日毎に線形回帰して、ベースモデルの予測値に上書き。初の金メダル圏6位に到達(Score 0.95) \n",
        "- 12/1-12/13 停滞期。スコアは全く伸びず。ランクはみるみる低下。測定データが部分的に欠損しているデータに対して線形補完後に、年間でメーター値推移を見た時に7月～8月を境にほとんど左右対称になっているビルのメーターに関しては、ピークを境に日付を畳み込みするとバリデーションスコアは劇的に伸びたが、提出スコアは凄い下がった。例えば、8月1日にピークがあるメーターに関しては、8月2日のデータを7月31日に置き換える。このようにすることで、過学習できないデータセットに対して、問題を簡単化できるのでは？と考えた。対応する月との相関が高い月のみ畳み込みした。スコアがあがるものと思い込んでいたががっかり。\n",
        "- 12/16 しばらく予測値と訓練データをプロットして比較していた。線形回帰で変な値になってる箇所があったので、そこだけアンサンブルで上書きした。リーダーボードのスコアが小数点第3位まで表示されるようになっていた（Score 0.955)\n",
        "- 12/17-12/18 怪しげなアカウントが大量にジャンプアップし始めて、みるみる順位が下がる。訓練データと予測データのアンサンブル。2016年と2017年の予測値で差異が見られる箇所を2016年のデータで上書き。スコア変動はなし。\n",
        "- 12/19 コンペ最終日。一部の異常なメーター値を持つビルに関して、メーターの変化量を観察するとメーター値の変化量がある時を境に正常値に戻っていることに気付いたため、正常値とみられる訓練データのみ使用するように修正(Score 0.954)  \n",
        "Public LeaderBoardは最終的に164位前後/3700チーム前後  \n",
        "Private Scoreは457位と大幅にシェイクダウン。  \n",
        "\n",
        "---\n",
        "## ★反省点\n",
        "今回はフルで参加した2回目のコンペティションということで、1回目より大分色々な手段が思いつくようになった。公開カーネルやディスカッションも前回ほどは使っていないので、自分で考えてアプローチできるようになってきたと思う。そのアプローチ方法が良いか悪いかは置いておくとして。\n",
        "今回はメモリの問題で特徴量はせいぜい10個程度しか試しておらず、前処理に注力した。前処理はモデルに学習しやすいように問題を整形する過程であることがコンペの経験を通して実感できた。  \n",
        "仕事でデータ分析をやる機会はほとんどないが、良い経験になった。\n",
        "- 予測値検証を効率的にやる仕組みを用意するべきだった\n",
        "- ノートブックをもっと細分化するべきだった。後半めちゃくちゃ重くてスクロールも一苦労していた。\n",
        "- 前処理をほとんど人間系でやっていて、ほとんど泥臭い作業になっていたので便利なツールを探すのも一考  \n",
        "- 一回のサブミッションに複数の試みを入れることがあったので、何が効いて何が効いていないか分かっていない箇所がある\n",
        "- 色々試したけど、その過程でバグがいっぱい入っていると思う。やりたいことを優先して検証にはあまり時間をさけなかった。\n",
        "\n",
        "今回のコンペティションは正直なところ運営のお粗末さにガッカリした。  \n",
        "リークデータが2割程発見されたものの、そのリークデータの取り扱いがあいまいだったこと。最終順位の結果発表が2週間以上延期されたこと。不正者を駆逐すると公言したが実施しなかったこと。  などが挙げられる。\n",
        "\n",
        "\n",
        "PublicBoard1位の人は,privateBoardで500位以上シェイクダウン、金メダル圏の人は全員金メダル圏から落ちて波乱のコンペであった。  \n",
        "\n",
        "機械学習を学び始めてから丁度1年がたった。　\n",
        "これに懲りずにKaggleに取り組んでいきたい。\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "※尚、今回は7個程度ノートブックを作成して、それぞれを組み合わせてサブミットファイルを作成しています。Githubにあげるにあたり、１つのノートブックに要点だけ集約しています。このノートブック単体では再現性もないし実行エラーが出ると思います。\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xXWCPjhZM4IX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import google.colab\n",
        "import googleapiclient.discovery\n",
        "import googleapiclient.http\n",
        "google.colab.auth.authenticate_user()\n",
        "drive_service = googleapiclient.discovery.build('drive', 'v3')\n",
        "\n",
        "def gdrive_dl(filename):\n",
        "  upload_filename = filename\n",
        "\n",
        "  file_list = drive_service.files().list(q=\"name='\" + upload_filename + \"'\").execute().get('files')\n",
        "\n",
        "  # ファイル ID を取得します。\n",
        "  file_id = None\n",
        "  for file in file_list:\n",
        "    if file.get('name') == upload_filename:\n",
        "      file_id = file.get('id')\n",
        "      break\n",
        "\n",
        "  if file_id is None:\n",
        "    # ファイル ID を取得できなかった場合はエラーメッセージを出力します。\n",
        "    print(upload_filename + ' が見つかりません.')\n",
        "  else:\n",
        "    # colab 環境へファイルをアップロードします。\n",
        "    with open(upload_filename, 'wb') as f:\n",
        "      request = drive_service.files().get_media(fileId=file_id)\n",
        "      media = googleapiclient.http.MediaIoBaseDownload(f, request)\n",
        "\n",
        "      done = False\n",
        "      while not done:\n",
        "        progress_status, done = media.next_chunk()\n",
        "        #print(100*progress_status.progress(), end=\"\")\n",
        "        #print(\"%完了\")\n",
        "\n",
        "    print('GoogleドライブからColab環境への[' + filename +']ファイル取り込みが完了しました.')\n",
        "\n",
        "def gdrive_up(filename):\n",
        "\n",
        "  file_metadata = {'name': filename,'mimeType': 'application/octet-stream'}\n",
        "  media = googleapiclient.http.MediaFileUpload(filename,mimetype='application/octet-stream',resumable=True)\n",
        "  created = drive_service.files().create(body=file_metadata,media_body=media, fields='id').execute()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hDoUlAc1PJoJ",
        "colab_type": "code",
        "outputId": "5977e15b-0367-4c2f-8729-52d30fb68cd2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "gdrive_dl(\"kaggle.json\")\n",
        "!mkdir -p ~/.kaggle\n",
        "!cp kaggle.json ~/.kaggle/\n",
        "!chmod 600 /root/.kaggle/kaggle.json"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[kaggle.json]ファイル取り込みが完了しました.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QL0mKDtJPR7J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install git+git://github.com/pfnet/optuna.git@50759344231194046fc8f7968ce906428f36bee1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9mgu7aZ2PXwe",
        "colab_type": "code",
        "outputId": "78590bcb-2ce5-4d23-e733-4b7e510f5149",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        }
      },
      "source": [
        "import gc\n",
        "import os\n",
        "from pathlib import Path\n",
        "import sys\n",
        "\n",
        "from tqdm import tqdm_notebook as tqdm\n",
        "import numpy as np # linear algebra\n",
        "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
        "import datetime\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "from IPython.core.display import display, HTML\n",
        "\n",
        "# --- plotly ---\n",
        "from plotly import tools, subplots\n",
        "import plotly.offline as py\n",
        "py.init_notebook_mode(connected=True)\n",
        "import plotly.graph_objs as go\n",
        "import plotly.express as px\n",
        "import plotly.figure_factory as ff\n",
        "\n",
        "# --- models ---\n",
        "from sklearn import preprocessing\n",
        "from sklearn.model_selection import KFold\n",
        "import lightgbm as lgb\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "        <script type=\"text/javascript\">\n",
              "        window.PlotlyConfig = {MathJaxConfig: 'local'};\n",
              "        if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}\n",
              "        if (typeof require !== 'undefined') {\n",
              "        require.undef(\"plotly\");\n",
              "        requirejs.config({\n",
              "            paths: {\n",
              "                'plotly': ['https://cdn.plot.ly/plotly-latest.min']\n",
              "            }\n",
              "        });\n",
              "        require(['plotly'], function(Plotly) {\n",
              "            window._Plotly = Plotly;\n",
              "        });\n",
              "        }\n",
              "        </script>\n",
              "        "
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sH5VPiVxPbrR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Original code from https://www.kaggle.com/gemartin/load-data-reduce-memory-usage by @gemartin\n",
        "# Modified to support timestamp type, categorical type\n",
        "# Modified to add option to use float16 or not. feather format does not support float16.\n",
        "from pandas.api.types import is_datetime64_any_dtype as is_datetime\n",
        "from pandas.api.types import is_categorical_dtype\n",
        "\n",
        "def reduce_mem_usage(df, use_float16=False):\n",
        "    \"\"\" iterate through all the columns of a dataframe and modify the data type\n",
        "        to reduce memory usage.        \n",
        "    \"\"\"\n",
        "    start_mem = df.memory_usage().sum() / 1024**2\n",
        "    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n",
        "    \n",
        "    for col in df.columns:\n",
        "        if is_datetime(df[col]) or is_categorical_dtype(df[col]):\n",
        "            # skip datetime type or categorical type\n",
        "            continue\n",
        "        col_type = df[col].dtype\n",
        "        \n",
        "        if col_type != object:\n",
        "            c_min = df[col].min()\n",
        "            c_max = df[col].max()\n",
        "            if str(col_type)[:3] == 'int':\n",
        "                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n",
        "                    df[col] = df[col].astype(np.int8)\n",
        "                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n",
        "                    df[col] = df[col].astype(np.int16)\n",
        "                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n",
        "                    df[col] = df[col].astype(np.int32)\n",
        "                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n",
        "                    df[col] = df[col].astype(np.int64)  \n",
        "            else:\n",
        "                if use_float16 and c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n",
        "                    df[col] = df[col].astype(np.float16)\n",
        "                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n",
        "                    df[col] = df[col].astype(np.float32)\n",
        "                else:\n",
        "                    df[col] = df[col].astype(np.float64)\n",
        "        else:\n",
        "            df[col] = df[col].astype('category')\n",
        "\n",
        "    end_mem = df.memory_usage().sum() / 1024**2\n",
        "    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n",
        "    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n",
        "    \n",
        "    return df\n",
        "\n",
        "def preprocess(df):\n",
        "    df[\"hour\"] = df[\"timestamp\"].dt.hour\n",
        "    df[\"month\"] = df[\"timestamp\"].dt.month\n",
        "    df[\"dayofweek\"] = df[\"timestamp\"].dt.dayofweek\n",
        "    df[\"weekend\"] = df[\"dayofweek\"] >= 5\n",
        "\n",
        "def preprocess_ex(df):\n",
        "    df[\"hour\"] = df[\"timestamp\"].dt.hour\n",
        "    df[\"day\"] = df[\"timestamp\"].dt.day\n",
        "    df[\"month\"] = df[\"timestamp\"].dt.month\n",
        "    df[\"dayofweek\"] = df[\"timestamp\"].dt.dayofweek\n",
        "    df[\"weekend\"] = df[\"dayofweek\"] >= 5\n",
        "\n",
        "def add_lag_feature(weather_df, window=3):\n",
        "    group_df = weather_df.groupby('site_id')\n",
        "    cols = ['air_temperature', 'cloud_coverage', 'dew_temperature', 'precip_depth_1_hr', 'sea_level_pressure', 'wind_direction', 'wind_speed']\n",
        "    rolled = group_df[cols].rolling(window=window, min_periods=0)\n",
        "    lag_mean = rolled.mean().reset_index().astype(np.float16)\n",
        "    lag_max = rolled.max().reset_index().astype(np.float16)\n",
        "    lag_min = rolled.min().reset_index().astype(np.float16)\n",
        "    lag_std = rolled.std().reset_index().astype(np.float16)\n",
        "    for col in cols:\n",
        "        weather_df[f'{col}_mean_lag{window}'] = lag_mean[col]\n",
        "        weather_df[f'{col}_max_lag{window}'] = lag_max[col]\n",
        "        weather_df[f'{col}_min_lag{window}'] = lag_min[col]\n",
        "        weather_df[f'{col}_std_lag{window}'] = lag_std[col]\n",
        "\n",
        "\n",
        "def timestamp_align(df):\n",
        "    df['offset'] = df.site_id.map(site_ids_offsets)\n",
        "    df['timestamp_aligned'] = (df.timestamp - pd.to_timedelta(df.offset, unit='H'))\n",
        "    df['timestamp'] = df['timestamp_aligned']\n",
        "    del df['timestamp_aligned']\n",
        "    return df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fDorHmmDPj-d",
        "colab_type": "code",
        "outputId": "6c671695-053b-417d-e0ea-825da99f5221",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        }
      },
      "source": [
        "gdrive_dl(\"sample_submission.pickle\")\n",
        "gdrive_dl(\"building_metadata.csv\")\n",
        "gdrive_dl(\"train.csv\")\n",
        "gdrive_dl(\"weather_train.csv\")\n",
        "gdrive_dl(\"test.csv\")\n",
        "gdrive_dl(\"weather_test.csv\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[sample_submission.pickle]ファイル取り込みが完了しました.\n",
            "GoogleドライブからColab環境への[building_metadata.csv]ファイル取り込みが完了しました.\n",
            "GoogleドライブからColab環境への[train.csv]ファイル取り込みが完了しました.\n",
            "GoogleドライブからColab環境への[weather_train.csv]ファイル取り込みが完了しました.\n",
            "GoogleドライブからColab環境への[test.csv]ファイル取り込みが完了しました.\n",
            "GoogleドライブからColab環境への[weather_test.csv]ファイル取り込みが完了しました.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xf209x5yQJMG",
        "colab_type": "code",
        "outputId": "4d27e397-7676-4e35-afcd-17cefadf743b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 177
        }
      },
      "source": [
        "train_df = reduce_mem_usage(pd.read_csv('train.csv', parse_dates=['timestamp']))\n",
        "train_df['timestamp'] = pd.to_datetime(train_df['timestamp'])\n",
        "preprocess(train_df)\n",
        "weather_train_df = reduce_mem_usage(pd.read_csv('weather_train.csv', parse_dates=['timestamp']))\n",
        "building_meta_df = reduce_mem_usage(pd.read_csv('building_metadata.csv'))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 616.95 MB\n",
            "Memory usage after optimization is: 289.19 MB\n",
            "Decreased by 53.1%\n",
            "Memory usage of dataframe is 9.60 MB\n",
            "Memory usage after optimization is: 4.93 MB\n",
            "Decreased by 48.6%\n",
            "Memory usage of dataframe is 0.07 MB\n",
            "Memory usage after optimization is: 0.02 MB\n",
            "Decreased by 65.4%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kxl7_2qMuXtV",
        "colab_type": "text"
      },
      "source": [
        "#前処理  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sQS3zUpVdSc2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_df = train_df.query('not (building_id <= 104 & meter == 0 & timestamp <= \"2016-05-21\")')\n",
        "train_df = train_df.query('not (building_id == 1099 & meter == 2)')#1106極大はずれ値\n",
        "train_df = train_df.query('not (building_id == 29 & meter == 0 & timestamp <= \"2016-08-10\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id == 954 & month <= 9)')  #1110 ADD\n",
        "\n",
        "train_df.loc[(train_df['building_id'] == 163) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 176) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 195) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 200) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 244) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 252) & (train_df['meter'] == 1),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 263) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 279) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 287) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 762) & (train_df['meter'] == 2),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 1003) & (train_df['meter'] == 3),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 1013) & (train_df['meter'] == 1),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 1018) & (train_df['meter'] == 1),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 1022) & (train_df['meter'] == 1),'meter_reading'] = 0\n",
        "train_df.loc[(train_df['building_id'] == 1345) & (train_df['meter'] == 0),'meter_reading'] = 0\n",
        "\n",
        "train_df = train_df.query('not (building_id == 29 & meter == 0 & timestamp <= \"2016-08-10\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id == 40 & meter == 0 & timestamp <= \"2016-06-04\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id == 43 & meter == 1 & timestamp <= \"2016-06-17\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id == 45 & meter == 0 & timestamp <= \"2016-07-01\")')#11/4 ADD\n",
        "train_df =train_df.query('not (building_id == 53 & meter == 0 & hour == 23)') #1110 ADD Office\t87742\t1971.0\t\n",
        "train_df =train_df.query('not (building_id == 53 & meter == 0 & hour <= 4)') #1110 ADD \n",
        "\n",
        "train_df = train_df.query('not (building_id == 55 & meter == 0 & meter_reading == 0 & timestamp <= \"2016-04-01\")')#11/4 ADD\n",
        "\n",
        "train_df = train_df.query('not (building_id == 60 & meter == 1 & timestamp <= \"2016-05-21\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id == 98 & meter == 1 & timestamp <= \"2016-06-16\")')#11/4 ADD\n",
        "train_df = train_df.query('not (building_id <= 104 & meter == 0 & timestamp <= \"2016-05-21\")')\n",
        "train_df = train_df.query('not (building_id == 258 & meter == 1 & \"2016-10-01\" <= timestamp <= \"2016-12-01\")')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 260 & meter == 1 & \"2016-06-01\" <= timestamp < \"2016-09-01\")')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 260 & meter == 1 & \"2016-09-01\" <= timestamp < \"2016-12-01\")')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 331 & meter == 0 & \"2016-10-01\" <= timestamp < \"2016-12-01\" & 0 <= hour <= 16)')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 419 & meter == 0 & month == 4 & hour != 19)')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 499 & meter == 0 & 3 <= month < 6)')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 499 & meter == 0 & 11 <= month)')  #1110 ADD\n",
        "\n",
        "train_df = train_df.query('not (building_id == 604)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 681 & meter == 0 & timestamp <= \"2016-05-01\")')  #1206 ADD\n",
        "train_df = train_df.query('not (building_id == 799 & meter == 0 & month < 7)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 802 & meter == 0 & month <= 9)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 803 & meter == 0 & 10 <= month )')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 886 & meter == 0 & 6 <= month < 10 )')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 904 & meter == 0 & month <= 2)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 909 & meter == 0 & month == 1)')  #1110 ADD\n",
        "\n",
        "train_df = train_df.query('not (building_id == 986 & meter == 0 & month == 1)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 997 & meter == 0 & month <= 6)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 1094 & meter == 0 & timestamp < \"2016-09-01\")')  #1110 ADD \n",
        "train_df = train_df.query('not (building_id == 1099 & meter == 2)')#1106極大はずれ値\n",
        "train_df = train_df.query('not (building_id == 1169 & meter == 0 & month <= 7)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 1195 & meter == 0 & month <= 3)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 1195 & meter == 0 & month == 12)')  #1110 ADD\n",
        "train_df = train_df.query('not (building_id == 1221 & meter == 0 & timestamp <= \"2016-11-04\")')\n",
        "train_df = train_df.query('not (building_id == 1241 & meter == 0 & 7 < month < 11)')#1110 ADD\n",
        "train_df = train_df.query('not (building_id == 1250 & meter == 2 & timestamp <= \"2016-12-21\")')\n",
        "\n",
        "train_df = train_df.query('not (building_id == 1264 & meter == 0 & \"2016-09-01\" < timestamp )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1303 & meter == 0 & 2 == month )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1359 & meter == 0 & 5 <= month <= 6 )') #1111 ADD\n",
        "## meter1\n",
        "train_df = train_df.query('not (building_id == 207 & meter == 1 &  month == 1 )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 207 & meter == 1 &  month == 7 )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 257 & meter == 1 &  11 <= month  )') #1111 ADD 4,5は保留\n",
        "train_df = train_df.query('not (building_id == 258 & meter == 1 &  9 < month < 12  )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 259 & meter == 1 &  month < 4 )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 260 & meter == 1 &  month < 5 )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 751 & meter == 2 &   timestamp == \"2016-06-02\" )')\n",
        "train_df = train_df.query('not (building_id == 769 & meter == 1 &  month < 4 )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 780 & meter == 1 &  10 < month )') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 787 & meter == 1 &  month < 3)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 793 & meter == 1 &  month < 5)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 963 & meter == 1 &  10 < month)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 978 & meter == 1 &  month == 8)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 978 & meter == 1 &  month == 11)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 998 & meter == 1 &  month <= 2)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1001 & meter == 1 &  month <= 3)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1003 & meter == 1 &  month == 1)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1015 & meter == 1 &  month <= 2)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1028 & meter == 1 &  month <= 4)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1028 & meter == 1 &  11 <= month)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1088 & meter == 1 &  month <= 3)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1130 & meter == 1 &  month <= 8)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1219 & meter == 1 &  month <= 3)') #1111 ADD\n",
        "train_df = train_df.query('not (building_id == 1290 & meter == 1 &  month == 2)') #1111 ADD\n",
        "\n",
        "#meter 2\n",
        "train_df = train_df.query('not (building_id == 751 & meter == 2 &   timestamp <\"2016-02-04\" )') #1207 ADD   3日に数時間欠損値あり。その間修理していたかも\n",
        "train_df = train_df.query('not (building_id == 766 & meter == 2 &  month == 1 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 774 & meter == 2 &  month == 1 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 783 & meter == 2 &  month <= 11 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 798 & meter == 2 &  6 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 799 & meter == 2 &  6 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 802 & meter == 2 &  6 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1072 & meter == 2 &  month <= 6 )') #1112 ADD #1073と形が似ている。データ補完したい\n",
        "train_df = train_df.query('not (building_id == 1149 & meter == 2 &  5 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1189 & meter == 2 &  month == 1 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1218 & meter == 2 &  5 <= month <= 8 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1303 & meter == 2 &  2 <= month <= 5 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1305 & meter == 2 &  5 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1307 & meter == 2 &  6 <= month <= 8 )') #1112 ADD\n",
        "\n",
        "#meter3\n",
        "train_df = train_df.query('not (building_id == 109 & meter == 3 &  7 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 112 & meter == 3 &  7 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 114 & meter == 3 &  8 <= month <= 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 163 & meter == 3 &  month == 1 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1011 & meter == 3 &  month == 6 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1011 & meter == 3 &  month == 9 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1017 & meter == 3 & 5 <= month <= 10 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1021 & meter == 3 & 5 <= month <= 6 )') #1112 ADD\n",
        "train_df = train_df.query('not (building_id == 1302 & meter == 3 & 5 <= month <= 9 )') #1112 ADD\n",
        "\n",
        "train_df = train_df.query('not (building_id == 1232 & meter == 1 & 7 <= month <= 9 )') #1215 ADD\n",
        "train_df = train_df.query('not (building_id == 1238 & meter == 2 & 2 <= month <= 5 )') #1215 ADD\n",
        "\n",
        "#時間別1115\n",
        "train_df = train_df.query('not (building_id == 162 & meter == 1 & 0 <= hour <= 4 )') \n",
        "train_df = train_df.query('not (building_id == 171 & meter == 1 & 0 <= hour <= 4 )') \n",
        "train_df = train_df.query('not (building_id == 171 & meter == 1 & 22 <= hour)') \n",
        "train_df = train_df.query('not (building_id == 176 & meter == 1 & 22 <= hour)') \n",
        "train_df = train_df.query('not (building_id == 176 & meter == 1 & 0 <= hour <= 3 )')\n",
        "train_df = train_df.query('not (building_id == 282 & meter == 1 & 0 <= hour <= 5 )')\n",
        "train_df = train_df.query('not (building_id == 282 & meter == 1 & 21 <= hour)')\n",
        "train_df = train_df.query('not (building_id == 287 & meter == 3)')\n",
        "train_df = train_df.query('not (building_id == 290 & meter == 1 & 22 <= hour)')\n",
        "#train_df = train_df.query('not (building_id == 304 & meter == 0)')\n",
        "#train_df = train_df.query('not (building_id == 345 & meter == 0)')　\n",
        "train_df = train_df.query('not (building_id == 356 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 357 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 402 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 408 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 413 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 416 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 740 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 770 & meter == 1 & 0 <= hour <= 3)')\n",
        "train_df = train_df.query('not (building_id == 804 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 805 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 808 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 810 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 811 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 812 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 813 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 815 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 817 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 820 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 824 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 833 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 845 & meter == 0 & month <= 9)')\n",
        "train_df = train_df.query('not (building_id == 846 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 849 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 857 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 859 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 860 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 861 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 870 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 870 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 873 & meter == 0)')\n",
        "train_df = train_df.query('not (building_id == 1101 & meter == 1 & 1 <= hour <= 5)')\n",
        "train_df = train_df.query('not (building_id == 1101 & meter == 1 & 23 <= hour)')\n",
        "train_df = train_df.query('not (building_id == 1207 & meter == 2 & 20 <= hour)')\n",
        "train_df = train_df.query('not (building_id == 1207 & meter == 2 &  hour <= 2)')\n",
        "train_df = train_df.query('not (building_id == 1211 & meter == 1 & 23 <= hour)')\n",
        "train_df = train_df.query('not (building_id == 1211 & meter == 1 & hour <= 6)')\n",
        "train_df = train_df.query('not (building_id == 1213 & meter == 0 & 9 <= hour <= 17)')\n",
        "train_df = train_df.query('not (building_id == 1216 & meter == 1 & hour <= 5)')\n",
        "train_df = train_df.query('not (building_id == 1217 & meter == 1 & hour <= 5)')\n",
        "train_df = train_df.query('not (building_id == 1217 & meter == 1 & 23 <= hour )')\n",
        "train_df = train_df.query('not (building_id == 1219 & meter == 1 & 21 <= hour )')\n",
        "train_df = train_df.query('not (building_id == 1219 & meter == 1 & hour <= 5 )')\n",
        "train_df = train_df.query('not (building_id == 1245 & meter == 1 & 5 <= hour <= 9)')\n",
        "train_df = train_df.query('not (building_id == 1259 & meter == 2)')\n",
        "train_df = train_df.query('not (building_id == 1262 & meter == 3 & 3 <= hour <= 8)')\n",
        "train_df = train_df.query('not (building_id == 1270 & meter == 0 & 3 <= hour <= 11)')\n",
        "train_df = train_df.query('not (building_id == 1273 & meter == 0 & 5 <= hour <= 10)')\n",
        "train_df = train_df.query('not (building_id == 1273 & meter == 1 & 5 <= hour <= 10)')\n",
        "train_df = train_df.query('not (building_id == 1274 & meter == 1 & 4 <= hour <= 10)')\n",
        "train_df = train_df.query('not (building_id == 1336 & meter == 1)')\n",
        "train_df = train_df.query('not (building_id == 1375 & meter == 1 & 3 <= hour <= 5)')\n",
        "train_df = train_df.query('not (building_id == 1391 & meter == 1 & 0 <= hour <= 4)')\n",
        "train_df = train_df.query('not (building_id == 1398 & meter == 1 & 1 <= hour <= 4)')\n",
        "train_df = train_df.query('not (building_id == 1425 & meter == 0 & 22 <= hour)')\n",
        "train_df = train_df.query('not (building_id == 1425 & meter == 0 & hour <= 4)')\n",
        "train_df = train_df.query('not (building_id == 1446 & meter == 0 & 0 <= hour <= 5)')\n",
        "train_df = train_df.query('not (building_id == 1446 & meter == 0 & 20 <= hour <= 22)')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "STg9pG8NeDAI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#外れ値除外\n",
        "for id in train_df['building_id'].unique():\n",
        "  df = train_df[train_df['building_id'] == id]\n",
        "  for m in df['meter'].unique():\n",
        "    df = df[df['meter'] == m]\n",
        "\n",
        "    if (abs(df.sort_values('meter_reading', ascending=False)['meter_reading_log1p'][0:2].diff()) > 0.3).sum() > 0:\n",
        "      train_df = train_df.drop(df['meter_reading'].idxmax())\n",
        "    if (abs(df.sort_values('meter_reading', ascending=True)['meter_reading_log1p'][0:2].diff()) > 0.3).sum() > 0:\n",
        "      train_df = train_df.drop(df['meter_reading'].idxmin())\n",
        "\n",
        "del df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mo-0gGCyWarU",
        "colab_type": "text"
      },
      "source": [
        "#線形補完ノートブック（抜粋)　始点"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UB-reGwMWjoZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "spline = pd.DataFrame([], columns=train_df.columns)\n",
        "temp = train_df.groupby(['building_id','meter','month']).count().reset_index()\n",
        "#補完可能な欠損割合を決定(年間データが6割から9.5割のみ存在するビルの測定データを追加)\n",
        "#あまりに多くの測定データを欠損しているビルは本質的なデータが捉えられないので無視する\n",
        "temp = temp[(temp['meter_reading'] <= 24 * 29 * 0.95) & (temp['meter_reading'] >= 24 * 29 * 0.6)]\n",
        "temp['building_id'].unique()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Pr0yqF3X6BA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_df = train_df[train_df['building_id'].isin(temp['building_id'].unique())]\n",
        "del temp\n",
        "train_df['building_id'].unique()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T39DuTFWYRor",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "add_df = train_df.copy()\n",
        "add_df =add_df['timestamp'].reset_index()\n",
        "add_df = add_df.drop_duplicates(subset='timestamp')\n",
        "del add_df['index']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BD5UahPvYjLo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#補完対象のプロット目視選別(meter * 10) + building_id)\n",
        "spline_building = [10050,10097,10098,30192,30202,296,298,299,300,376,483,500,522,544,10745,20745,10789,20789,10790,20790,10791,20791,10792,20792,805,\n",
        "806,807,808,809,810,811,815,816,818,819,820,821,822,823,824,825,826,827,830,831,832,833,834,835,836,837,838,839,840,841,842,843,845,846,847,848,849,850,851,852,853,\n",
        "854,855,856,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,20878,10928,998,999,1002,31003,1004,1005,1006,1007,1014,1018,1021,1022,1148,1222,\n",
        "11239,31258,11267,31269,11273,21285,21288,11291,11293,11300,21306,21315,1318,1326,11326,21326,1327,1328,11328,1329,11329,21329,21330,1331,11331,21331,31331,1332,11332,21332,1333,11333,1334,11334,1335,1336,11336,21336,1337,11337,21337,\n",
        "1338,21338,1339,11339,21339,1341,11341,21341,1342,11342,21342,1343,11343,21343,1344,11344,21344,1345,11345,21345,\n",
        "11347,21347,1348,11348,21348,11349,1350,11350,1352,11352,1353,11354,21354,1356,1357,1358,21358,1359,11359,1360,21360,11362,1363,11363,\n",
        "21363,1364,21364,1365,1366,11366,21366,1367,11367,21367,1368,11368,1369,21369,1370,1371,11372,11375,1376,11376,1377,11377,\n",
        "21377,1378,21378,11379,21379,1380,11380,1381,21381,1382,11382,21382,1386,21387,11388,1389,11389,1390,21390,1391,1393,1394,11394,1395,11395,1396,21396,1398,1399]\n",
        "#10184,#492,#545,#20762,#767,#804,#812,#813,#814,#817,#829,#983,#21077,#11225,#11226,#1325,#31325,#21327,#1340,\n",
        "#1346,#21346,1347,\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y7LOobvmiMBb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from scipy import signal, interpolate\n",
        "from matplotlib import pylab as plt\n",
        "#spl_df = pd.DataFrame(data=None, index=None, columns=None, dtype=None, copy=False)\n",
        "for item in spline_building:\n",
        "  bid = item % 10000\n",
        "  m = item // 10000\n",
        "  spl_df = train_df.loc[(train_df.building_id == bid) & (train_df.meter == m)] \n",
        "  print(str(bid) + str(m))\n",
        "  spl_df = add_df.merge(spl_df,on='timestamp',how='left')\n",
        "  prelineCnt = len(spl_df[spl_df['meter_reading'].isnull()])\n",
        "  spl_df = spl_df.interpolate(limit_direction='both')\n",
        "  aftlineCnt = len(spl_df[spl_df['meter_reading'].isnull()])\n",
        "\n",
        "  if prelineCnt != aftlineCnt:\n",
        "    print('origin train_df',train_df.shape)\n",
        "    train_df = train_df.loc[~((train_df.building_id == bid) & (train_df.meter == m))]\n",
        "    print('drop train_df',train_df.shape)\n",
        "    train_df = pd.concat([train_df, spl_df])\n",
        "    print('train_df',train_df.shape)\n",
        "    print('building_id=',str(bid),'meter',str(m))\n",
        "\n",
        "train_df = train_df[~train_df['meter_reading'].isnull()]\n",
        "train_df = train_df[['building_id', 'meter', 'meter_reading', 'timestamp']]\n",
        "#train_df.to_csv('lineComplement.csv')\n",
        "#gdrive_up('lineComplement.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVsxtlGSjK_I",
        "colab_type": "text"
      },
      "source": [
        "#畳み込み"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SHvLRR1HjIkI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gdrive_dl('lineComplement.csv')\n",
        "train_org = reduce_mem_usage(pd.read_csv('lineComplement.csv',parse_dates=['timestamp']))\n",
        "del train_org['Unnamed: 0']\n",
        "preprocess(train_org)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Kdt_A3TkWFx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#畳み込み対象調査\n",
        "df = train_org\n",
        "\n",
        "for id in train_org['building_id'].unique():\n",
        "  for m in train_org[train_org['building_id'] == id]['meter'].unique():\n",
        "    df2 = (train_org[(train_org['building_id'] == id) & (train_org['meter'] == m)])\n",
        "    if len(df2) != 0:\n",
        "      print('building_id=',str(id),'meter=',str(m))\n",
        "      df2['9-7'] = np.log1p(df2[df2['month'] == 9]['meter_reading'].mean()) -np.log1p(df2[df2['month'] == 7]['meter_reading'].mean())\n",
        "      df2['10-6'] = np.log1p(df2[df2['month'] == 10]['meter_reading'].mean()) - np.log1p(df2[df2['month'] == 6]['meter_reading'].mean())\n",
        "      df2['11-5'] = np.log1p(df2[df2['month'] == 11]['meter_reading'].mean()) - np.log1p(df2[df2['month'] == 5]['meter_reading'].mean())\n",
        "      df2['12-4'] = np.log1p(df2[df2['month'] == 12]['meter_reading'].mean()) - np.log1p(df2[df2['month'] == 4]['meter_reading'].mean())\n",
        "      df = df.append(df2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9ZB0S9P0jYzb",
        "colab_type": "text"
      },
      "source": [
        "#線形補完ノートブック（抜粋)　終点"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zoD_oMO4etpC",
        "colab_type": "code",
        "outputId": "0db95f61-b353-45a4-e328-7dee5585598d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "gdrive_dl('lineComplement.csv')\n",
        "train_df = pd.read_csv('lineComplement.csv',parse_dates=['timestamp'])\n",
        "del train_df['Unnamed: 0']"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[lineComplement.csv]ファイル取り込みが完了しました.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hruRkyUoTenu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_df['date'] = train_df['timestamp'].dt.date\n",
        "train_df['meter_reading_log1p'] = np.log1p(train_df['meter_reading'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zQiNigbwwrna",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#低分散の線形回帰対象を訓練データから削除\n",
        "drop_lst = ['20191130_OutlierAll.csv','20191130_OutlierWeek.csv','20191130_OutlierHour.csv']\n",
        "drop_df = pd.DataFrame()\n",
        "for s in drop_lst:\n",
        "  gdrive_dl(s)\n",
        "  drop_df = drop_df.append(reduce_mem_usage(pd.read_csv(s)))\n",
        "  print(drop_df.shape)\n",
        "\n",
        "drop_df = drop_df.query('not (building_id == 279)') '精度が低いため\n",
        "drop_df =drop_df.drop('Unnamed: 0',axis=1)\n",
        "drop_df['timestamp'] = pd.to_datetime(drop_df['timestamp'])\n",
        "drop_df = drop_df.drop_duplicates()\n",
        "drop_df.shape"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2bvJ60yOQyq2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_df['merge'] = (train_df['building_id'] * 10000000) + (train_df['meter'] * 1000000) + (train_df['timestamp'].dt.month * 10000) + (train_df['timestamp'].dt.day * 100) + train_df['timestamp'].dt.hour\n",
        "drop_df['merge'] =  (drop_df['building_id']  * 10000000) + (drop_df['meter']  * 1000000) + (drop_df['timestamp'].dt.month  * 10000) + (drop_df['timestamp'].dt.day  * 100) + drop_df['timestamp'].dt.hour\n",
        "train_df = train_df[~train_df['merge'].isin(drop_df['merge'].unique())]\n",
        "del train_df['merge'] ,drop_df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qm6KGAZBQ1EA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "debug = False"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8ldJIMjXQ_8u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_group = train_df.groupby(['building_id', 'meter'])['meter_reading_log1p']\n",
        "building_mean = df_group.mean().astype(np.float16)\n",
        "building_median = df_group.median().astype(np.float16)\n",
        "building_min = df_group.min().astype(np.float16)\n",
        "building_max = df_group.max().astype(np.float16)\n",
        "building_std = df_group.std().astype(np.float16)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-qiQGP5RC4G",
        "colab_type": "code",
        "outputId": "654173e3-bfff-4766-97b4-a2341df77a56",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        }
      },
      "source": [
        "building_stats_df = reduce_mem_usage(pd.concat([building_mean, building_median, building_min, building_max, building_std], axis=1,\n",
        "                              keys=['building_mean', 'building_median', 'building_min', 'building_max', 'building_std']).reset_index())\n",
        "train_df = reduce_mem_usage(pd.merge(train_df, building_stats_df, on=['building_id', 'meter'], how='left', copy=False))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 0.06 MB\n",
            "Memory usage after optimization is: 0.06 MB\n",
            "Decreased by -7.7%\n",
            "Memory usage of dataframe is 2241.22 MB\n",
            "Memory usage after optimization is: 1353.62 MB\n",
            "Decreased by 39.6%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cRjJQz1fP0tT",
        "colab_type": "code",
        "outputId": "0b787020-5b8b-4ec6-e757-b848cd065eec",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "#1106 メーター別　当該月のの平均　当該月の週毎の平均　当該月の時間毎の平均\n",
        "#未実施メモリ不足\n",
        "'''\n",
        "#月平均\n",
        "temp = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\n",
        "temp = reduce_mem_usage(temp.groupby(['building_id','meter','month'])['meter_reading'].mean().reset_index())\n",
        "temp = temp.rename(columns={'meter_reading': 'meter_monthmean'}) \n",
        "#df = pd.merge(df,temp, on=['building_id', 'meter','month'],how='left',copy=False)\n",
        "#週平均\n",
        "temp2 = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\n",
        "temp2 = reduce_mem_usage(temp2.groupby(['building_id','month','dayofweek','meter'])['meter_reading'].mean().reset_index())\n",
        "temp2 = temp2.rename(columns={'meter_reading': 'meter_weekmean'}) \n",
        "#df = pd.merge(df,temp, on=['building_id', 'meter','month','dayofweek'],how='left',copy=False)\n",
        "\n",
        "#train_df = meteraverage(train_df)\n",
        "#train_df.head()\n",
        "df = pd.merge(temp2,temp, on=['building_id','meter','month'],how='left',copy=False)\n",
        "del temp, temp2\n",
        "\n",
        "#時間平均\n",
        "temp3 = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\n",
        "temp3 = reduce_mem_usage(temp3.groupby(['building_id','month','hour','meter'])['meter_reading'].mean().reset_index())\n",
        "temp3 = temp3.rename(columns={'meter_reading': 'meter_hourmean'}) \n",
        "#df = pd.merge(df,temp, on=['building_id', 'meter','month','hour'],how='left',copy=False)\n",
        "\n",
        "df = reduce_mem_usage(pd.merge(df,temp3, on=['building_id','meter','month'],how='left',copy=False))\n",
        "del temp3\n",
        "train_df = pd.merge(train_df,df, on=['building_id','meter','month'],how='left',copy=False)\n",
        "train_df.head()\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"\\n#月平均\\ntemp = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\\ntemp = reduce_mem_usage(temp.groupby(['building_id','meter','month'])['meter_reading'].mean().reset_index())\\ntemp = temp.rename(columns={'meter_reading': 'meter_monthmean'}) \\n#df = pd.merge(df,temp, on=['building_id', 'meter','month'],how='left',copy=False)\\n#週平均\\ntemp2 = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\\ntemp2 = reduce_mem_usage(temp2.groupby(['building_id','month','dayofweek','meter'])['meter_reading'].mean().reset_index())\\ntemp2 = temp2.rename(columns={'meter_reading': 'meter_weekmean'}) \\n#df = pd.merge(df,temp, on=['building_id', 'meter','month','dayofweek'],how='left',copy=False)\\n\\n#train_df = meteraverage(train_df)\\n#train_df.head()\\ndf = pd.merge(temp2,temp, on=['building_id','meter','month'],how='left',copy=False)\\ndel temp, temp2\\n\\n#時間平均\\ntemp3 = train_df[['building_id','meter','month','meter_reading','dayofweek','hour']]\\ntemp3 = reduce_mem_usage(temp3.groupby(['building_id','month','hour','meter'])['meter_reading'].mean().reset_index())\\ntemp3 = temp3.rename(columns={'meter_reading': 'meter_hourmean'}) \\n#df = pd.merge(df,temp, on=['building_id', 'meter','month','hour'],how='left',copy=False)\\n\\ndf = reduce_mem_usage(pd.merge(df,temp3, on=['building_id','meter','month'],how='left',copy=False))\\ndel temp3\\ntrain_df = pd.merge(train_df,df, on=['building_id','meter','month'],how='left',copy=False)\\ntrain_df.head()\\n\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hiCaE-kpRLq9",
        "colab_type": "code",
        "outputId": "16dc8e79-b572-4a09-b66a-49f2bfef173b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "weather_test_df = reduce_mem_usage(pd.read_csv('weather_test.csv', parse_dates=['timestamp']))\n",
        "weather = pd.concat([weather_train_df, weather_test_df],ignore_index=True)\n",
        "del weather_test_df\n",
        "weather_key = ['site_id', 'timestamp']"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 19.04 MB\n",
            "Memory usage after optimization is: 9.78 MB\n",
            "Decreased by 48.6%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aNl0HjgqRlL2",
        "colab_type": "code",
        "outputId": "83045cdb-45f3-4b3c-f5c4-c2cc9a185384",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "#時間補正\n",
        "# https://www.kaggle.com/nz0722/aligned-timestamp-lgbm-by-meter-type\n",
        "\n",
        "temp_skeleton = weather[weather_key + ['air_temperature']].drop_duplicates(subset=weather_key).sort_values(by=weather_key).copy()\n",
        "\n",
        "# calculate ranks of hourly temperatures within date/site_id chunks\n",
        "temp_skeleton['temp_rank'] = temp_skeleton.groupby(['site_id', temp_skeleton.timestamp.dt.date])['air_temperature'].rank('average')\n",
        "\n",
        "# create a dataframe of site_ids (0-16) x mean hour rank of temperature within day (0-23)\n",
        "df_2d = temp_skeleton.groupby(['site_id', temp_skeleton.timestamp.dt.hour])['temp_rank'].mean().unstack(level=1)\n",
        "\n",
        "# Subtract the columnID of temperature peak by 14, getting the timestamp alignment gap.\n",
        "site_ids_offsets = pd.Series(df_2d.values.argmax(axis=1) - 14)\n",
        "site_ids_offsets.index.name = 'site_id'\n",
        "\n",
        "del weather\n",
        "del temp_skeleton\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "626"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UbU3n2pr9Fco",
        "colab_type": "code",
        "outputId": "3ea12e88-9ab7-413f-e5ee-fe31181140c5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "# 畳み込み\n",
        "# スコアとても下がる\n",
        "'''\n",
        "conv = [300\t,303\t,304\t,305\t,306\t,307\t,308\t,310\t,311\t,321\t,322\t,328\t,329\t,332\t,333\t,\n",
        "334\t,336\t,342\t,343\t,348\t,349\t,353\t,354\t,355\t,357\t,358\t,359\t,361\t,366\t,369\t,370\t,\n",
        "373\t,374\t,376\t,377\t,378\t,380\t,381\t,384\t,386\t,388\t,391\t,393\t,395\t,397\t,398\t,399\t,\n",
        "400\t,406\t,410\t,412\t,414\t,415\t,416\t,419\t,422\t,423\t,424\t,425\t,427\t,428\t,429\t,430\t,\n",
        "431\t,433\t,436\t,438\t,439\t,442\t,445\t,446\t,450\t,452\t,453\t,456\t,458\t,459\t,460\t,462\t,\n",
        "464\t,465\t,467\t,468\t,470\t,472\t,473\t,474\t,476\t,477\t,478\t,480\t,481\t,490\t,493\t,497\t,498\t,502\t,504\t,511\t,\n",
        "515\t,519\t,520\t,521\t,527\t,529\t,533\t,535\t,541\t,544\t,546\t,547\t,552\t,554\t,557\t,558\t,561\t,563\t,564\t,605\t,\n",
        "623\t,625\t,676\t,682\t,686\t,693\t,694\t,695\t,746\t,20747,10748,10752,20754,10755,20757,20759,\n",
        "10763,10765,20765,20769,10770,20770,771\t,10771\t,20771,20774,10777\t,10780\t,20784\t,10786\t,\n",
        "10787\t,20787\t,10789\t,20789\t,10790\t,20790\t,10791\t,20791\t,\n",
        "10792\t,20792\t,10794\t,20794\t,10795\t,20795\t,10796\t,20796\t,10797\t,20797\t,\n",
        "10798\t,20798\t,10799\t,20799\t,858\t,859\t,862\t,869\t,871\t,10874\t,\n",
        "20875\t,10878\t,10879\t,10883\t,10885\t,20885\t,10886\t,20886\t,\n",
        "10887\t,20887\t,20888\t,10889\t,10890\t,10890\t,10893\t,10894\t,\n",
        "20894\t,10895\t,10896\t,10898\t,20898\t,10899\t,20899\t,10900\t,\n",
        "10905\t,10906\t,10908\t,20908\t,10910\t,20910\t,10911\t,\n",
        "20911\t,10912\t,20912\t,10914\t,20914\t,10917\t,20917\t,10918\t,\n",
        "20918\t,10920\t,20920\t,10921\t,20921\t,10922\t,20922\t,10924\t,\n",
        "20924\t,10925\t,20925\t,10926\t,20926\t,10928\t,\n",
        "20928\t,10929\t,20929\t,10931\t,20931\t,10932\t,10935\t,\n",
        "939\t,10948\t,20948\t,10949,20949\t,10951\t,20951\t,10956\t,20956\t,10964\t,\n",
        "20964\t,10965\t,10966\t,20966\t,10968\t,20968\t,10969\t,20969\t,\n",
        "20971\t,10972\t,10973\t,20973\t,10974\t,\n",
        "20974\t,20978\t,10979\t,20979\t,10980\t,10987\t,20987\t,10991\t,\n",
        "20991\t,10992\t,20992\t,10993\t,10995\t,20995\t,20998\t,30998]\n",
        "turn_start = datetime.date(2016, 8, 1)\n",
        "turn_end = datetime.date(2016, 11, 1)\n",
        "cnv_df = train_df.query('(\"2016-08-01\" < timestamp &  timestamp < \"2016-11-01\" )')\n",
        "train_df = train_df.query('not (\"2016-08-01\" < timestamp &  timestamp < \"2016-11-01\" )')\n",
        "for item in conv:\n",
        "  bid = item % 10000\n",
        "  m = item // 10000\n",
        "  tmp = cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]\n",
        "  if len(tmp) > 0 :\n",
        "    cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m),'timestamp'] = cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'] - ((cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'].dt.date  - turn_start) * 2)\n",
        "  \n",
        "preprocess(cnv_df)\n",
        "print(train_df.shape,cnv_df.shape)\n",
        "train_df =pd.concat([train_df,cnv_df])\n",
        "print(train_df.shape)\n",
        "del cnv_df\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\nconv = [300\\t,303\\t,304\\t,305\\t,306\\t,307\\t,308\\t,310\\t,311\\t,321\\t,322\\t,328\\t,329\\t,332\\t,333\\t,\\n334\\t,336\\t,342\\t,343\\t,348\\t,349\\t,353\\t,354\\t,355\\t,357\\t,358\\t,359\\t,361\\t,366\\t,369\\t,370\\t,\\n373\\t,374\\t,376\\t,377\\t,378\\t,380\\t,381\\t,384\\t,386\\t,388\\t,391\\t,393\\t,395\\t,397\\t,398\\t,399\\t,\\n400\\t,406\\t,410\\t,412\\t,414\\t,415\\t,416\\t,419\\t,422\\t,423\\t,424\\t,425\\t,427\\t,428\\t,429\\t,430\\t,\\n431\\t,433\\t,436\\t,438\\t,439\\t,442\\t,445\\t,446\\t,450\\t,452\\t,453\\t,456\\t,458\\t,459\\t,460\\t,462\\t,\\n464\\t,465\\t,467\\t,468\\t,470\\t,472\\t,473\\t,474\\t,476\\t,477\\t,478\\t,480\\t,481\\t,490\\t,493\\t,497\\t,498\\t,502\\t,504\\t,511\\t,\\n515\\t,519\\t,520\\t,521\\t,527\\t,529\\t,533\\t,535\\t,541\\t,544\\t,546\\t,547\\t,552\\t,554\\t,557\\t,558\\t,561\\t,563\\t,564\\t,605\\t,\\n623\\t,625\\t,676\\t,682\\t,686\\t,693\\t,694\\t,695\\t,746\\t,20747,10748,10752,20754,10755,20757,20759,\\n10763,10765,20765,20769,10770,20770,771\\t,10771\\t,20771,20774,10777\\t,10780\\t,20784\\t,10786\\t,\\n10787\\t,20787\\t,10789\\t,20789\\t,10790\\t,20790\\t,10791\\t,20791\\t,\\n10792\\t,20792\\t,10794\\t,20794\\t,10795\\t,20795\\t,10796\\t,20796\\t,10797\\t,20797\\t,\\n10798\\t,20798\\t,10799\\t,20799\\t,858\\t,859\\t,862\\t,869\\t,871\\t,10874\\t,\\n20875\\t,10878\\t,10879\\t,10883\\t,10885\\t,20885\\t,10886\\t,20886\\t,\\n10887\\t,20887\\t,20888\\t,10889\\t,10890\\t,10890\\t,10893\\t,10894\\t,\\n20894\\t,10895\\t,10896\\t,10898\\t,20898\\t,10899\\t,20899\\t,10900\\t,\\n10905\\t,10906\\t,10908\\t,20908\\t,10910\\t,20910\\t,10911\\t,\\n20911\\t,10912\\t,20912\\t,10914\\t,20914\\t,10917\\t,20917\\t,10918\\t,\\n20918\\t,10920\\t,20920\\t,10921\\t,20921\\t,10922\\t,20922\\t,10924\\t,\\n20924\\t,10925\\t,20925\\t,10926\\t,20926\\t,10928\\t,\\n20928\\t,10929\\t,20929\\t,10931\\t,20931\\t,10932\\t,10935\\t,\\n939\\t,10948\\t,20948\\t,10949,20949\\t,10951\\t,20951\\t,10956\\t,20956\\t,10964\\t,\\n20964\\t,10965\\t,10966\\t,20966\\t,10968\\t,20968\\t,10969\\t,20969\\t,\\n20971\\t,10972\\t,10973\\t,20973\\t,10974\\t,\\n20974\\t,20978\\t,10979\\t,20979\\t,10980\\t,10987\\t,20987\\t,10991\\t,\\n20991\\t,10992\\t,20992\\t,10993\\t,10995\\t,20995\\t,20998\\t,30998]\\nturn_start = datetime.date(2016, 8, 1)\\nturn_end = datetime.date(2016, 11, 1)\\ncnv_df = train_df.query(\\'(\"2016-08-01\" < timestamp &  timestamp < \"2016-11-01\" )\\')\\ntrain_df = train_df.query(\\'not (\"2016-08-01\" < timestamp &  timestamp < \"2016-11-01\" )\\')\\nfor item in conv:\\n  bid = item % 10000\\n  m = item // 10000\\n  tmp = cnv_df.loc[(cnv_df[\\'building_id\\'] == bid) & (cnv_df[\\'meter\\'] == m)]\\n  if len(tmp) > 0 :\\n    cnv_df.loc[(cnv_df[\\'building_id\\'] == bid) & (cnv_df[\\'meter\\'] == m),\\'timestamp\\'] = cnv_df[(cnv_df[\\'building_id\\'] == bid) & (cnv_df[\\'meter\\'] == m)][\\'timestamp\\'] - ((cnv_df[(cnv_df[\\'building_id\\'] == bid) & (cnv_df[\\'meter\\'] == m)][\\'timestamp\\'].dt.date  - turn_start) * 2)\\n  \\npreprocess(cnv_df)\\nprint(train_df.shape,cnv_df.shape)\\ntrain_df =pd.concat([train_df,cnv_df])\\nprint(train_df.shape)\\ndel cnv_df\\n'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EkNlIzHURo_l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "weather_train_df = timestamp_align(weather_train_df)\n",
        "weather_train_df = weather_train_df.groupby('site_id').apply(lambda group: group.interpolate(limit_direction='both'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sk0wOMKCwlmh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#1109 祝日データ\n",
        "import holidays\n",
        "\n",
        "def add_holiday(df_weather):\n",
        "  en_holidays = holidays.England()\n",
        "  ir_holidays = holidays.Ireland()\n",
        "  ca_holidays = holidays.Canada()\n",
        "  us_holidays = holidays.UnitedStates()\n",
        "\n",
        "  en_idx = df_weather.query('site_id == 1 or site_id == 5').index\n",
        "  ir_idx = df_weather.query('site_id == 12').index\n",
        "  ca_idx = df_weather.query('site_id == 7 or site_id == 11').index\n",
        "  us_idx = df_weather.query('site_id == 0 or site_id == 2 or site_id == 3 or site_id == 4 or site_id == 6 or site_id == 8 or site_id == 9 or site_id == 10 or site_id == 13 or site_id == 14 or site_id == 15').index\n",
        "\n",
        "  df_weather['IsHoliday'] = 0\n",
        "  df_weather.loc[en_idx, 'IsHoliday'] = df_weather.loc[en_idx, 'timestamp'].apply(lambda x: en_holidays.get(x, default=0))\n",
        "  df_weather.loc[ir_idx, 'IsHoliday'] = df_weather.loc[ir_idx, 'timestamp'].apply(lambda x: ir_holidays.get(x, default=0))\n",
        "  df_weather.loc[ca_idx, 'IsHoliday'] = df_weather.loc[ca_idx, 'timestamp'].apply(lambda x: ca_holidays.get(x, default=0))\n",
        "  df_weather.loc[us_idx, 'IsHoliday'] = df_weather.loc[us_idx, 'timestamp'].apply(lambda x: us_holidays.get(x, default=0))\n",
        "\n",
        "  holiday_idx = df_weather['IsHoliday'] != 0\n",
        "  df_weather.loc[holiday_idx, 'IsHoliday'] = 1\n",
        "  df_weather['IsHoliday'] = df_weather['IsHoliday'].astype(np.uint8)\n",
        "\n",
        "add_holiday(weather_train_df)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BpJBqabkRwbW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "add_lag_feature(weather_train_df, window=3)\n",
        "#add_lag_feature(weather_train_df, window=24)下がる\n",
        "add_lag_feature(weather_train_df, window=72)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LNGnBX-dR1kq",
        "colab_type": "code",
        "outputId": "e0d0b18d-d702-4c04-9f5a-6b034d9b2b08",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        }
      },
      "source": [
        "# categorize primary_use column to reduce memory on merge...\n",
        "\n",
        "primary_use_list = building_meta_df['primary_use'].unique()\n",
        "primary_use_dict = {key: value for value, key in enumerate(primary_use_list)} \n",
        "print('primary_use_dict: ', primary_use_dict)\n",
        "building_meta_df['primary_use'] = building_meta_df['primary_use'].map(primary_use_dict)\n",
        "\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "primary_use_dict:  {'Education': 0, 'Lodging/residential': 1, 'Office': 2, 'Entertainment/public assembly': 3, 'Other': 4, 'Retail': 5, 'Parking': 6, 'Public services': 7, 'Warehouse/storage': 8, 'Food sales and service': 9, 'Religious worship': 10, 'Healthcare': 11, 'Utility': 12, 'Technology/science': 13, 'Manufacturing/industrial': 14, 'Services': 15}\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uUp_XI4XR2MW",
        "colab_type": "code",
        "outputId": "fdffb328-1c15-4e6e-9707-da685d3d0abd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 177
        }
      },
      "source": [
        "train_df = reduce_mem_usage(train_df, use_float16=True)\n",
        "building_meta_df = reduce_mem_usage(building_meta_df, use_float16=True)\n",
        "weather_train_df =reduce_mem_usage(weather_train_df, use_float16=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 1353.62 MB\n",
            "Memory usage after optimization is: 954.19 MB\n",
            "Decreased by 29.5%\n",
            "Memory usage of dataframe is 0.02 MB\n",
            "Memory usage after optimization is: 0.02 MB\n",
            "Decreased by 24.1%\n",
            "Memory usage of dataframe is 21.06 MB\n",
            "Memory usage after optimization is: 18.40 MB\n",
            "Decreased by 12.7%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YnF1dDYjR8vC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "category_cols = ['building_id', 'site_id', 'primary_use', 'meter','dayofweek','building_median']\n",
        "weather_cols = [\n",
        "    'air_temperature', 'cloud_coverage',\n",
        "    'dew_temperature', 'precip_depth_1_hr', 'sea_level_pressure',\n",
        "    'wind_direction', 'wind_speed', 'air_temperature_mean_lag72',\n",
        "    'air_temperature_max_lag72', 'air_temperature_min_lag72',\n",
        "    'air_temperature_std_lag72', 'cloud_coverage_mean_lag72',\n",
        "    'dew_temperature_mean_lag72', 'precip_depth_1_hr_mean_lag72',\n",
        "    'sea_level_pressure_mean_lag72', 'wind_direction_mean_lag72',\n",
        "    'wind_speed_mean_lag72', 'air_temperature_mean_lag3',\n",
        "    'air_temperature_max_lag3',\n",
        "    'air_temperature_min_lag3', 'cloud_coverage_mean_lag3',\n",
        "    'dew_temperature_mean_lag3',\n",
        "    'precip_depth_1_hr_mean_lag3', 'sea_level_pressure_mean_lag3',\n",
        "    'wind_direction_mean_lag3', 'wind_speed_mean_lag3']\n",
        "custom_feature = ['offset','IsHoliday']#'meter_month_zero','meter_month_zerosum'\n",
        "feature_cols = ['square_feet', 'year_built'] + [\n",
        "    'hour', 'weekend' # 'month' 'dayofweek''building_median'\n",
        "    ] + weather_cols + custom_feature"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "quWXr4vZS7Pc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def create_X_y(train_df, target_meter):\n",
        "    \n",
        "    target_train_df = train_df[train_df['meter'] == target_meter]\n",
        "    target_train_df = target_train_df.merge(building_meta_df, on='building_id', how='left')\n",
        "    target_train_df = target_train_df.merge(weather_train_df, on=['site_id', 'timestamp'], how='left')\n",
        "    #https://www.kaggle.com/c/ashrae-energy-prediction/discussion/119261#latest-693904\n",
        "    target_train_df.loc[(target_train_df['site_id'] == 0) & (target_train_df['meter'] == 0), 'meter_reading'] = target_train_df[(target_train_df['site_id'] == 0) & (target_train_df['meter'] == 0)]['meter_reading'] * 0.2931\n",
        "    \n",
        "    X_train = target_train_df[feature_cols + category_cols]\n",
        "    y_train = target_train_df['meter_reading_log1p'].values\n",
        "\n",
        "    del target_train_df\n",
        "    return X_train, y_train"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0kWtR8oCS-3O",
        "colab_type": "code",
        "outputId": "549763c3-ee36-446e-9410-d5642328481e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "import optuna\n",
        "from optuna import Trial\n",
        "\n",
        "optuna.__version__"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'0.17.1'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 107
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZuR5snvuTA6I",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "debug = False\n",
        "\n",
        "train_df_original = train_df\n",
        "# Only use 10000 data,,, for fast computation for debugging.\n",
        "train_df = train_df.sample(10000)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZUGodHKrTEBg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def objective(trial: Trial, fast_check=True, target_meter=0, return_info=False):\n",
        "    folds = 5\n",
        "    seed = 666\n",
        "    shuffle = False\n",
        "    kf = KFold(n_splits=folds, shuffle=shuffle, random_state=seed)\n",
        "\n",
        "    X_train, y_train = create_X_y(train_df, target_meter=target_meter)\n",
        "    y_valid_pred_total = np.zeros(X_train.shape[0])\n",
        "    gc.collect()\n",
        "    print('target_meter', target_meter, X_train.shape)\n",
        "\n",
        "    cat_features = [X_train.columns.get_loc(cat_col) for cat_col in category_cols]\n",
        "    print('cat_features', cat_features)\n",
        "\n",
        "    models = []\n",
        "    valid_score = 0\n",
        "    for train_idx, valid_idx in kf.split(X_train, y_train):\n",
        "        train_data = X_train.iloc[train_idx,:], y_train[train_idx]\n",
        "        valid_data = X_train.iloc[valid_idx,:], y_train[valid_idx]\n",
        "\n",
        "        print('train', len(train_idx), 'valid', len(valid_idx))\n",
        "    #     model, y_pred_valid, log = fit_cb(train_data, valid_data, cat_features=cat_features, devices=[0,])\n",
        "        model, y_pred_valid, log = fit_lgbm(trial, train_data, valid_data, cat_features=category_cols,\n",
        "                                            num_rounds=1000)\n",
        "        y_valid_pred_total[valid_idx] = y_pred_valid\n",
        "        models.append(model)\n",
        "        gc.collect()\n",
        "        valid_score += log[\"valid/l2\"]\n",
        "        if fast_check:\n",
        "            break\n",
        "    valid_score /= len(models)\n",
        "    if return_info:\n",
        "        return valid_score, models, y_pred_valid, y_train\n",
        "    else:\n",
        "        return valid_score"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G5Lo9CcITIEt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Referred https://github.com/pfnet/optuna/blob/master/examples/lightgbm_simple.py\n",
        "\n",
        "def fit_lgbm(trial, train, val, devices=(-1,), seed=None, cat_features=None, num_rounds=1500):\n",
        "    \"\"\"Train Light GBM model\"\"\"\n",
        "    X_train, y_train = train\n",
        "    X_valid, y_valid = val\n",
        "    metric = 'l2'\n",
        "    params = {\n",
        "        'num_leaves': trial.suggest_int('num_leaves', 2, 256),\n",
        "        'objective': 'regression',\n",
        "#               'max_depth': -1,\n",
        "        'learning_rate': 0.1,\n",
        "        \"boosting\": \"gbdt\",\n",
        "        'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),\n",
        "        'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),\n",
        "        \"bagging_freq\": 5,\n",
        "        \"bagging_fraction\": trial.suggest_uniform('bagging_fraction', 0.1, 1.0),\n",
        "        \"feature_fraction\": trial.suggest_uniform('feature_fraction', 0.4, 1.0),\n",
        "        \"metric\": metric,\n",
        "        \"verbosity\": -1,\n",
        "    }\n",
        "    device = devices[0]\n",
        "    if device == -1:\n",
        "        # use cpu\n",
        "        pass\n",
        "    else:\n",
        "        # use gpu\n",
        "        print(f'using gpu device_id {device}...')\n",
        "        params.update({'device': 'gpu', 'gpu_device_id': device})\n",
        "\n",
        "    params['seed'] = seed\n",
        "\n",
        "    early_stop = 20\n",
        "    verbose_eval = 20\n",
        "\n",
        "    d_train = lgb.Dataset(X_train, label=y_train, categorical_feature=cat_features)\n",
        "    d_valid = lgb.Dataset(X_valid, label=y_valid, categorical_feature=cat_features)\n",
        "    watchlist = [d_train, d_valid]\n",
        "\n",
        "    print('training LGB:')\n",
        "    model = lgb.train(params,\n",
        "                      train_set=d_train,\n",
        "                      num_boost_round=num_rounds,\n",
        "                      valid_sets=watchlist,\n",
        "                      verbose_eval=verbose_eval,\n",
        "                      early_stopping_rounds=early_stop)\n",
        "\n",
        "    # predictions\n",
        "    y_pred_valid = model.predict(X_valid, num_iteration=model.best_iteration)\n",
        "    \n",
        "    print('best_score', model.best_score)\n",
        "    log = {'train/l2': model.best_score['training']['l2'],\n",
        "           'valid/l2': model.best_score['valid_1']['l2']}\n",
        "    return model, y_pred_valid, log"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0F_HYbGiTORA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "\n",
        "def fit_lgbm_with_pruning(trial, train, val, devices=(-1,), seed=None, cat_features=None, num_rounds=1500):\n",
        "    \"\"\"Train Light GBM model\"\"\"\n",
        "    X_train, y_train = train\n",
        "    X_valid, y_valid = val\n",
        "    metric = 'l2'\n",
        "    params = {\n",
        "        'num_leaves': trial.suggest_int('num_leaves', 10, 256),\n",
        "        'objective': 'regression',\n",
        "#               'max_depth': -1,\n",
        "        'learning_rate': 0.1,\n",
        "        \"boosting\": \"gbdt\",\n",
        "        'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),\n",
        "        'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),\n",
        "        \"bagging_freq\": 5,\n",
        "        \"bagging_fraction\": trial.suggest_uniform('bagging_fraction', 0.1, 1.0),\n",
        "        \"feature_fraction\": trial.suggest_uniform('feature_fraction', 0.4, 1.0),\n",
        "        \"metric\": metric,\n",
        "        \"verbosity\": -1,\n",
        "    }\n",
        "    device = devices[0]\n",
        "    if device == -1:\n",
        "        # use cpu\n",
        "        pass\n",
        "    else:\n",
        "        # use gpu\n",
        "        print(f'using gpu device_id {device}...')\n",
        "        params.update({'device': 'gpu', 'gpu_device_id': device})\n",
        "\n",
        "    params['seed'] = seed\n",
        "\n",
        "    early_stop = 20\n",
        "    verbose_eval = 20\n",
        "\n",
        "    d_train = lgb.Dataset(X_train, label=y_train, categorical_feature=cat_features)\n",
        "    d_valid = lgb.Dataset(X_valid, label=y_valid, categorical_feature=cat_features)\n",
        "    watchlist = [d_train, d_valid]\n",
        "\n",
        "    # Add a callback for pruning.\n",
        "    pruning_callback = optuna.integration.LightGBMPruningCallback(trial, 'l2', valid_name='valid_1')    \n",
        "    print('training LGB:')\n",
        "    model = lgb.train(params,\n",
        "                      train_set=d_train,\n",
        "                      num_boost_round=num_rounds,\n",
        "                      valid_sets=watchlist,\n",
        "                      verbose_eval=verbose_eval,\n",
        "                      early_stopping_rounds=early_stop,\n",
        "                      callbacks=[pruning_callback])\n",
        "\n",
        "    # predictions\n",
        "    y_pred_valid = model.predict(X_valid, num_iteration=model.best_iteration)\n",
        "    \n",
        "    print('best_score', model.best_score)\n",
        "    log = {'train/l2': model.best_score['training']['l2'],\n",
        "           'valid/l2': model.best_score['valid_1']['l2']}\n",
        "    return model, y_pred_valid, log"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aGC1bGJ1VRVl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def objective_with_prune(trial: Trial, fast_check=True, target_meter=0):\n",
        "    \"\"\"This method is completely same with previous `objective` method except calling `fit_lgbm_with_pruning`\"\"\"\n",
        "    folds = 5\n",
        "    seed = 666\n",
        "    shuffle = False\n",
        "    kf = KFold(n_splits=folds, shuffle=shuffle, random_state=seed)\n",
        "\n",
        "    X_train, y_train = create_X_y(train_df, target_meter=target_meter)\n",
        "    y_valid_pred_total = np.zeros(X_train.shape[0])\n",
        "    gc.collect()\n",
        "    print('target_meter', target_meter, X_train.shape)\n",
        "\n",
        "    cat_features = [X_train.columns.get_loc(cat_col) for cat_col in category_cols]\n",
        "    print('cat_features', cat_features)\n",
        "\n",
        "    models0 = []\n",
        "    valid_score = 0\n",
        "    for train_idx, valid_idx in kf.split(X_train, y_train):\n",
        "        train_data = X_train.iloc[train_idx,:], y_train[train_idx]\n",
        "        valid_data = X_train.iloc[valid_idx,:], y_train[valid_idx]\n",
        "\n",
        "        print('train', len(train_idx), 'valid', len(valid_idx))\n",
        "        model, y_pred_valid, log = fit_lgbm_with_pruning(trial, train_data, valid_data, cat_features=category_cols,\n",
        "                                                         num_rounds=1000)\n",
        "        y_valid_pred_total[valid_idx] = y_pred_valid\n",
        "        models0.append(model)\n",
        "        gc.collect()\n",
        "        valid_score += log[\"valid/l2\"]\n",
        "        if fast_check:\n",
        "            break\n",
        "    valid_score /= len(models0)\n",
        "    return valid_score"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0QzqWCGx7Pe",
        "colab_type": "text"
      },
      "source": [
        "# パラメータサーチ開始"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HkfaJ2YtVrC2",
        "colab_type": "code",
        "outputId": "23f4f1d3-2cf5-462e-e44b-ac33be46b8ee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "if not debug:\n",
        "    # Restore to original data size for actual computation\n",
        "    train_df = train_df_original\n",
        "    timeout = 60 * 60 * 2  # For 2 hours...\n",
        "else:\n",
        "    timeout = 60 * 1       # Debug mode, only for 1 mins\n",
        "\n",
        "print('train_df.shape', train_df.shape)\n",
        "print(f'timeout {timeout /60} min')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "train_df.shape (23268185, 15)\n",
            "timeout 120.0 min\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KkrpUQbOVuZA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "study = optuna.create_study(pruner=optuna.pruners.SuccessiveHalvingPruner(min_resource=2, reduction_factor=4, min_early_stopping_rate=0))\n",
        "study.optimize(objective_with_prune, timeout=timeout)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8UCO_zETVyGJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(f'Executed {len(study.trials)} trials, best score {study.best_value} with best_params {study.best_params}')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JvLQRBa3V1-y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "trials_df = study.trials_dataframe()\n",
        "trials_df.to_csv('trials_history.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dQtTKsdjWAqn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(\"Training model with best_params {}\".format(study.best_params))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GNL_ArHXyETT",
        "colab_type": "text"
      },
      "source": [
        "#パラメータサーチ終了"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CWDEI_IeEL9u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#1105\n",
        "params =  {'num_leaves': 251, 'lambda_l1': 4.042850956704914e-08, 'lambda_l2': 9.960105113132773, 'bagging_fraction': 0.90520386079324, 'feature_fraction': 0.8842112658876237}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HAjEpwVaWCdZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# For meter 0\n",
        "#1105\n",
        "valid_score, models0, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=0, return_info=True)\n",
        "#valid_score, models0, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(params), fast_check=False, target_meter=0, return_info=True)\n",
        "sns.distplot(y_pred_valid, label='pred')\n",
        "sns.distplot(y_train, label='ground truth')\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "del y_pred_valid, y_train"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zyLAF_R4WENH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# For meter 1\n",
        "#1105\n",
        "valid_score, models1, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=1, return_info=True)\n",
        "#valid_score, models1, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(params), fast_check=False, target_meter=1, return_info=True)\n",
        "sns.distplot(y_pred_valid, label='pred')\n",
        "sns.distplot(y_train, label='ground truth')\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "del y_pred_valid, y_train"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "paJWT9OaWGDd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# For meter 2\n",
        "#1105\n",
        "valid_score, models2, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=2, return_info=True)\n",
        "#valid_score, models2, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(params), fast_check=False, target_meter=2, return_info=True)\n",
        "sns.distplot(y_pred_valid, label='pred')\n",
        "sns.distplot(y_train, label='ground truth')\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "del y_pred_valid, y_train"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hn81lxNzWINL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# For meter 3\n",
        "#1105\n",
        "valid_score, models3, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=3, return_info=True)\n",
        "#valid_score, models3, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(params), fast_check=False, target_meter=3, return_info=True)\n",
        "\n",
        "sns.distplot(y_pred_valid, label='pred')\n",
        "sns.distplot(y_train, label='ground truth')\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "del y_pred_valid, y_train"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3AhTvHQ9WJ3Q",
        "colab_type": "code",
        "outputId": "235aa989-ba28-4376-c9c5-8481ad464beb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "try:\n",
        "    del train_df\n",
        "    del train_df_original\n",
        "    del weather_train_df\n",
        "except:\n",
        "    pass\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9646"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1BmuYDBNWLXw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print('loading...')\n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "weather_test_df = reduce_mem_usage(pd.read_csv('weather_test.csv', parse_dates=['timestamp']))\n",
        "\n",
        "print('preprocessing building...')\n",
        "test_df['date'] = test_df['timestamp'].dt.date\n",
        "preprocess(test_df)\n",
        "\n",
        "print('preprocessing weather...')\n",
        "weather_test_d = fill_weather_dataset(weather_test_d)\n",
        "weather_test_df = timestamp_align(weather_test_df)\n",
        "weather_test_df = weather_test_df.groupby('site_id').apply(lambda group: group.interpolate(limit_direction='both'))\n",
        "weather_test_df.groupby('site_id').apply(lambda group: group.isna().sum())\n",
        "add_holiday(weather_test_df)\n",
        "add_lag_feature(weather_test_df, window=3)\n",
        "#add_lag_feature(weather_test_df, window=24)\n",
        "add_lag_feature(weather_test_df, window=72)\n",
        "\n",
        "print('reduce mem usage...')\n",
        "weather_test_df = weather_test_df[['site_id', 'timestamp'] + weather_cols + ['offset', 'IsHoliday']]\n",
        "reduce_mem_usage(test_df, use_float16=True)\n",
        "reduce_mem_usage(weather_test_df, use_float16=False)\n",
        "\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HGOYvgbRVX2Z",
        "colab_type": "code",
        "outputId": "5d4a0dea-e082-44e2-b6ac-c646b136af25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "'''\n",
        "#テストデータ畳み込み\n",
        "#スコア下がるため未使用\n",
        "\n",
        "turn_start = datetime.date(2017, 8, 1)\n",
        "turn_end = datetime.date(2017, 11, 1)\n",
        "cnv_df = test_df.query('(\"2017-08-01\" < timestamp &  timestamp < \"2017-11-01\" )')\n",
        "test_df = test_df.query('not (\"2017-08-01\" < timestamp &  timestamp < \"2017-11-01\" )')\n",
        "for item in conv:\n",
        "  bid = item % 10000\n",
        "  m = item // 10000\n",
        "  tmp = cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]\n",
        "  if len(tmp) > 0 :\n",
        "    cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m),'timestamp'] = cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'] - ((cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'].dt.date  - turn_start) * 2)\n",
        "  \n",
        "preprocess(cnv_df)\n",
        "print(test_df.shape,cnv_df.shape)\n",
        "test_df =pd.concat([test_df,cnv_df])\n",
        "print(test_df.shape)\n",
        "del cnv_df\n",
        "\n",
        "turn_start = datetime.date(2018, 8, 1)\n",
        "turn_end = datetime.date(2018, 11, 1)\n",
        "cnv_df = test_df.query('(\"2018-08-01\" < timestamp &  timestamp < \"2018-11-01\" )')\n",
        "test_df = test_df.query('not (\"2018-08-01\" < timestamp &  timestamp < \"2018-11-01\" )')\n",
        "for item in conv:\n",
        "  bid = item % 10000\n",
        "  m = item // 10000\n",
        "  tmp = cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]\n",
        "  if len(tmp) > 0 :\n",
        "    cnv_df.loc[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m),'timestamp'] = cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'] - ((cnv_df[(cnv_df['building_id'] == bid) & (cnv_df['meter'] == m)]['timestamp'].dt.date  - turn_start) * 2)\n",
        "  \n",
        "preprocess(cnv_df)\n",
        "print(test_df.shape,cnv_df.shape)\n",
        "test_df =pd.concat([test_df,cnv_df])\n",
        "print(test_df.shape)\n",
        "del cnv_df\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(36444940, 9) (5252660, 9)\n",
            "(41697600, 9)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6nR8j9leWt9J",
        "colab_type": "code",
        "outputId": "857443d9-195c-4a32-bd21-80fe95d9a002",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "del df_2d\n",
        "del site_ids_offsets\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "503"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8OLrB8z2WwJ-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def create_X(test_df, target_meter):\n",
        "    target_test_df = test_df[test_df['meter'] == target_meter]\n",
        "    target_test_df = target_test_df.merge(building_meta_df, on='building_id', how='left', copy=False)\n",
        "    target_test_df = pd.merge(target_test_df, building_stats_df, on=['building_id', 'meter'], how='left', copy=False)\n",
        "    target_test_df = target_test_df.merge(weather_test_df, on=['site_id', 'timestamp'], how='left', copy=False)\n",
        "    X_test = target_test_df[feature_cols + category_cols]\n",
        "    return X_test\n",
        "\n",
        "def pred(X_test, models, batch_size=1000000):\n",
        "    iterations = (X_test.shape[0] + batch_size -1) // batch_size\n",
        "    print('iterations', iterations)\n",
        "\n",
        "    y_test_pred_total = np.zeros(X_test.shape[0])\n",
        "    for i, model in enumerate(models):\n",
        "        print(f'predicting {i}-th model')\n",
        "        for k in tqdm(range(iterations)):\n",
        "            y_pred_test = model.predict(X_test[k*batch_size:(k+1)*batch_size], num_iteration=model.best_iteration)\n",
        "            y_test_pred_total[k*batch_size:(k+1)*batch_size] += y_pred_test\n",
        "\n",
        "    y_test_pred_total /= len(models)\n",
        "    return y_test_pred_total"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "azXya30EWy2o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%time\n",
        "X_test = create_X(test_df, target_meter=0)\n",
        "gc.collect()\n",
        "\n",
        "y_test0 = pred(X_test, models0)\n",
        "\n",
        "del X_test\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B74z01oSW1Ev",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%time\n",
        "X_test = create_X(test_df, target_meter=1)\n",
        "gc.collect()\n",
        "y_test1 = pred(X_test, models1)\n",
        "\n",
        "del X_test\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HjXeRiOCW3K_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%time\n",
        "X_test = create_X(test_df, target_meter=2)\n",
        "gc.collect()\n",
        "y_test2 = pred(X_test, models2)\n",
        "\n",
        "del X_test\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TCdS_qJoW5N_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%time\n",
        "X_test = create_X(test_df, target_meter=3)\n",
        "gc.collect()\n",
        "y_test3 = pred(X_test, models3)\n",
        "\n",
        "del X_test\n",
        "gc.collect()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kb9HX2gVXCMP",
        "colab_type": "code",
        "outputId": "d2fbac61-a7e5-451c-d4ef-72867940e84b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 482
        }
      },
      "source": [
        "gdrive_dl('sample_submission.csv')\n",
        "sample_submission = pd.read_csv('sample_submission.csv')\n",
        "reduce_mem_usage(sample_submission)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[sample_submission.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 636.26 MB\n",
            "Memory usage after optimization is: 198.83 MB\n",
            "Decreased by 68.7%\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>row_id</th>\n",
              "      <th>meter_reading</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697595</th>\n",
              "      <td>41697595</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697596</th>\n",
              "      <td>41697596</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697597</th>\n",
              "      <td>41697597</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697598</th>\n",
              "      <td>41697598</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697599</th>\n",
              "      <td>41697599</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>41697600 rows × 2 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            row_id  meter_reading\n",
              "0                0              0\n",
              "1                1              0\n",
              "2                2              0\n",
              "3                3              0\n",
              "4                4              0\n",
              "...            ...            ...\n",
              "41697595  41697595              0\n",
              "41697596  41697596              0\n",
              "41697597  41697597              0\n",
              "41697598  41697598              0\n",
              "41697599  41697599              0\n",
              "\n",
              "[41697600 rows x 2 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7FGnneFFoA5q",
        "colab_type": "text"
      },
      "source": [
        "#予測結果検証開始"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MJhBOVWNh-yI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "gdrive_dl('0.956BestSubmission.csv')\n",
        "sample_submission2 = pd.read_csv('0.956BestSubmission.csv') \n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test_df = test_df.loc[test_df['timestamp'] <\"2018-01-01\"]\n",
        "test_df = pd.merge(test_df,sample_submission2, on=['row_id'],how='left')\n",
        "test_df = test_df[test_df['building_id'] > 290]\n",
        "test_df = test_df[test_df['building_id'] < 1324]\n",
        "test_df = test_df[~((test_df['building_id'] < 655) & (test_df['building_id'] > 564) & (test_df['meter'] == 0))]\n",
        "preprocess_ex(test_df)\n",
        "del test_df['dayofweek'],test_df['weekend'],test_df['row_id'],test_df['timestamp']\n",
        "test_df = reduce_mem_usage(test_df)\n",
        "test_df = test_df.rename(columns={'meter_reading': 'test1'})\n",
        "\n",
        "gdrive_dl('halfhalf.csv')\n",
        "sample_submission2 = pd.read_csv('halfhalf.csv') \n",
        "test2_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test2_df = test2_df.loc[test2_df['timestamp'] <\"2018-01-01\"]\n",
        "test2_df = pd.merge(test2_df,sample_submission2, on=['row_id'],how='left')\n",
        "test2_df = test2_df[test2_df['building_id'] > 290]\n",
        "test2_df = test2_df[test2_df['building_id'] < 1324]\n",
        "test2_df = test2_df[~((test2_df['building_id'] < 655) & (test2_df['building_id'] > 564) & (test2_df['meter'] == 0))]\n",
        "preprocess_ex(test2_df)\n",
        "del test2_df['dayofweek'],test2_df['weekend'],test2_df['row_id'],test2_df['timestamp']\n",
        "test2_df = reduce_mem_usage(test2_df)\n",
        "test2_df = test2_df.rename(columns={'meter_reading': 'test2'})\n",
        "\n",
        "gdrive_dl('1.08aitude.csv')\n",
        "sample_submission3 = pd.read_csv('1.08aitude.csv') \n",
        "test3_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test3_df = test3_df.loc[test3_df['timestamp'] <\"2018-01-01\"]\n",
        "test3_df = pd.merge(test3_df,sample_submission3, on=['row_id'],how='left')\n",
        "test3_df = test3_df[test3_df['building_id'] > 290]\n",
        "test3_df = test3_df[test3_df['building_id'] < 1324]\n",
        "test3_df = test3_df[~((test3_df['building_id'] < 655) & (test3_df['building_id'] > 564) & (test3_df['meter'] == 0))]\n",
        "preprocess_ex(test3_df)\n",
        "del test3_df['dayofweek'],test3_df['weekend'],test3_df['row_id'],test3_df['timestamp']\n",
        "test3_df = reduce_mem_usage(test3_df)\n",
        "test3_df = test3_df.rename(columns={'meter_reading': 'test3'})\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UYbDe8PNSOfE",
        "colab_type": "code",
        "outputId": "f8d8c97a-7b81-45cd-f4fd-912f23962e59",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 200
        }
      },
      "source": [
        "train_df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>building_id</th>\n",
              "      <th>meter</th>\n",
              "      <th>base</th>\n",
              "      <th>hour</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>453</th>\n",
              "      <td>291</td>\n",
              "      <td>0</td>\n",
              "      <td>125.879997</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>454</th>\n",
              "      <td>292</td>\n",
              "      <td>0</td>\n",
              "      <td>136.800003</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>455</th>\n",
              "      <td>293</td>\n",
              "      <td>0</td>\n",
              "      <td>624.619995</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>456</th>\n",
              "      <td>294</td>\n",
              "      <td>0</td>\n",
              "      <td>59.290001</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>457</th>\n",
              "      <td>295</td>\n",
              "      <td>0</td>\n",
              "      <td>27.700001</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     building_id  meter        base  hour  month  day  timestamp\n",
              "453          291      0  125.879997     0      1    1 2016-01-01\n",
              "454          292      0  136.800003     0      1    1 2016-01-01\n",
              "455          293      0  624.619995     0      1    1 2016-01-01\n",
              "456          294      0   59.290001     0      1    1 2016-01-01\n",
              "457          295      0   27.700001     0      1    1 2016-01-01"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7qV1ERe9kDNu",
        "colab_type": "code",
        "outputId": "73985afa-4761-4298-97f1-cd45ca7837c9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "#前処理まで\n",
        "preprocess_ex(train_df)\n",
        "train_df = train_df[train_df['building_id'] > 290]\n",
        "train_df = train_df[train_df['building_id'] < 1324]\n",
        "train_df = train_df[~((train_df['building_id'] < 655) & (train_df['building_id'] > 564) & (train_df['meter'] == 0))]\n",
        "train_df = train_df[['building_id','meter','base','hour','month','day','timestamp']]#meter_reading\n",
        "train_df['building_id'] = train_df['building_id'].astype(np.int64)\n",
        "train_df['meter'] = train_df['meter'].astype(np.int64)\n",
        "train_df = train_df.rename(columns={'meter_reading': 'base'})\n",
        "train_df = reduce_mem_usage(train_df)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 840.74 MB\n",
            "Memory usage after optimization is: 364.32 MB\n",
            "Decreased by 56.7%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VWQ792BDyR17",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.metrics import mean_squared_error\n",
        "def rmsle(y_true: np.ndarray, y_pred: np.ndarray) -> np.float64:\n",
        "    rmsle = mean_squared_error(np.log1p(y_true), np.log1p(y_pred))\n",
        "    return np.sqrt(rmsle)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "arLpmPp1ytp4",
        "colab_type": "code",
        "outputId": "5b4d3abb-100d-4f77-a830-28926a662930",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 411
        }
      },
      "source": [
        "merge = pd.merge(train_df,test_df, on=['building_id','meter','month','day','hour'],how='left')\n",
        "del test_df\n",
        "merge = pd.merge(merge,test2_df, on=['building_id','meter','month','day','hour'],how='left')\n",
        "del test2_df\n",
        "merge = pd.merge(merge,test3_df, on=['building_id','meter','month','day','hour'],how='left')\n",
        "del test3_df\n",
        "merge"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>building_id</th>\n",
              "      <th>meter</th>\n",
              "      <th>base</th>\n",
              "      <th>hour</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>test1</th>\n",
              "      <th>test2</th>\n",
              "      <th>test3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>291</td>\n",
              "      <td>0</td>\n",
              "      <td>125.879997</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01 00:00:00</td>\n",
              "      <td>143.130905</td>\n",
              "      <td>139.748825</td>\n",
              "      <td>149.116562</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>292</td>\n",
              "      <td>0</td>\n",
              "      <td>136.800003</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01 00:00:00</td>\n",
              "      <td>86.293098</td>\n",
              "      <td>72.515732</td>\n",
              "      <td>85.429840</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>293</td>\n",
              "      <td>0</td>\n",
              "      <td>624.619995</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01 00:00:00</td>\n",
              "      <td>547.333130</td>\n",
              "      <td>523.676025</td>\n",
              "      <td>563.438538</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>294</td>\n",
              "      <td>0</td>\n",
              "      <td>59.290001</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01 00:00:00</td>\n",
              "      <td>35.359699</td>\n",
              "      <td>32.227516</td>\n",
              "      <td>35.671875</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>295</td>\n",
              "      <td>0</td>\n",
              "      <td>27.700001</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2016-01-01 00:00:00</td>\n",
              "      <td>33.631199</td>\n",
              "      <td>30.534683</td>\n",
              "      <td>35.792892</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14692912</th>\n",
              "      <td>1318</td>\n",
              "      <td>0</td>\n",
              "      <td>184.091003</td>\n",
              "      <td>19</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2016-12-31 19:00:00</td>\n",
              "      <td>119.167099</td>\n",
              "      <td>179.859482</td>\n",
              "      <td>112.608665</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14692913</th>\n",
              "      <td>1318</td>\n",
              "      <td>0</td>\n",
              "      <td>181.565994</td>\n",
              "      <td>20</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2016-12-31 20:00:00</td>\n",
              "      <td>120.220200</td>\n",
              "      <td>180.512207</td>\n",
              "      <td>112.451782</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14692914</th>\n",
              "      <td>1318</td>\n",
              "      <td>0</td>\n",
              "      <td>184.675995</td>\n",
              "      <td>21</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2016-12-31 21:00:00</td>\n",
              "      <td>118.963699</td>\n",
              "      <td>177.309570</td>\n",
              "      <td>114.328522</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14692915</th>\n",
              "      <td>1318</td>\n",
              "      <td>0</td>\n",
              "      <td>197.557007</td>\n",
              "      <td>22</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2016-12-31 22:00:00</td>\n",
              "      <td>117.115501</td>\n",
              "      <td>158.396469</td>\n",
              "      <td>111.900917</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14692916</th>\n",
              "      <td>1318</td>\n",
              "      <td>0</td>\n",
              "      <td>202.893005</td>\n",
              "      <td>23</td>\n",
              "      <td>12</td>\n",
              "      <td>31</td>\n",
              "      <td>2016-12-31 23:00:00</td>\n",
              "      <td>113.309898</td>\n",
              "      <td>150.741409</td>\n",
              "      <td>110.022758</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>14692917 rows × 10 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "          building_id  meter        base  ...       test1       test2       test3\n",
              "0                 291      0  125.879997  ...  143.130905  139.748825  149.116562\n",
              "1                 292      0  136.800003  ...   86.293098   72.515732   85.429840\n",
              "2                 293      0  624.619995  ...  547.333130  523.676025  563.438538\n",
              "3                 294      0   59.290001  ...   35.359699   32.227516   35.671875\n",
              "4                 295      0   27.700001  ...   33.631199   30.534683   35.792892\n",
              "...               ...    ...         ...  ...         ...         ...         ...\n",
              "14692912         1318      0  184.091003  ...  119.167099  179.859482  112.608665\n",
              "14692913         1318      0  181.565994  ...  120.220200  180.512207  112.451782\n",
              "14692914         1318      0  184.675995  ...  118.963699  177.309570  114.328522\n",
              "14692915         1318      0  197.557007  ...  117.115501  158.396469  111.900917\n",
              "14692916         1318      0  202.893005  ...  113.309898  150.741409  110.022758\n",
              "\n",
              "[14692917 rows x 10 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rt8EsOeg6ol4",
        "colab_type": "code",
        "outputId": "8bef0a6e-220e-438e-fadd-aad2b989c519",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "\n",
        "merge = merge[~merge.duplicated(['building_id', 'meter','month','day','hour'])]\n",
        "merge = merge.dropna(how='any')\n",
        "len(merge['base'].isnull())\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "13071948"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fmuh-QU-kW4g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "pair = merge[~merge.duplicated(['building_id', 'meter','month','day','hour'])]\n",
        "for bid in pair['building_id'].unique():\n",
        "  for mid in pair[pair['building_id'] == bid]['meter'].unique():\n",
        "    tmp = merge[(merge['building_id'] == bid) & (merge['meter'] == mid)]\n",
        "\n",
        "    result = rmsle(tmp['base'],tmp['test1'])\n",
        "    result2 = rmsle(tmp['base'],tmp['test2'])\n",
        "    result3 = rmsle(tmp['base'],tmp['test3'])\n",
        "    if (result > 2) & (result < 2.5):\n",
        "      print ('building_id=',bid,',meter=',mid,',score=',result,',score2=',result2,',score3=',result3)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LIryuQkwbFdK",
        "colab_type": "code",
        "outputId": "77804c3f-640d-4950-fa9f-3b8f643a2d69",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "source": [
        "bid = 1221\n",
        "m = 0\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "sns.set()\n",
        "sns.set_context(\"notebook\")\n",
        "plt.figure(figsize=(10, 5))\n",
        "\n",
        "sns.lineplot(x=\"timestamp\", y=\"test1\",data= merge[(merge['building_id'] == bid) & (merge['meter'] == m)])\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f8a2e37aa90>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 242
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAFCCAYAAAB8V5DfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de1TUdf7H8RcMaqVYgeAZsl13LQzL\nCI9rqWmkJthSpNbBWLtYYdq2XS2tTUDd8pZpuWrH8mge2dqO28XY0mq72oVqrRUjKy9EKQKChojG\nOHx+f/hzjiQgyjAwn+/z8Vfz/X7m832/P9PQq/l+Z74hxhgjAAAABL3Q1i4AAAAA/kGwAwAAsATB\nDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASYa1dQCDt2bNftbXO+Nm+yMhOKi+vau0yAsqJ\nPUvO7bu5nLpuTuybnuFvLb2+oaEhOvPMjif1XEcFu9pa45hgJ8lRvR7hxJ4l5/bdXE5dNyf2Tc/w\nt7a6vpyKBQAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbAD\nAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4A\nAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgELdtu3b1daWpqSkpKUlpamwsLCBsdu27ZN8fHx\nmj179jH78vLyFBcXp1WrVrVgtQAAAMEnYMEuKytL6enpWrdundLT05WZmVnvOK/Xq6ysLA0bNuyY\nfVVVVXr88cc1ePDgli4XAAAg6AQk2JWXl6ugoEApKSmSpJSUFBUUFKiiouKYsUuXLlViYqK6d+9+\nzL5Zs2bp1ltv1ZlnntnSJQMAAASdgAS74uJide3aVS6XS5LkcrkUHR2t4uLiOuM2b96s9evX6+ab\nbz5mjvfff1/79u1TcnJyIEoGAAAIOmGtXcARHo9HU6dO1cyZM30B8IjKykrNmzdPy5cvb9YxIiM7\nNev5wSYqKry1Swg4J/YsObfv5nLqujmxb3qGv7XV9Q1IsHO73SopKZHX65XL5ZLX61Vpaancbrdv\nTFlZmYqKijR+/HhJh8OcMUZVVVVKTU1VWVmZrrvuOknSnj179O6772rv3r268847m1xHeXmVamuN\nf5tro6KiwlVWtq+1ywgoJ/YsObfv5nLqujmxb3qGv7X0+oaGhpz0h1EBCXaRkZGKi4tTbm6uUlNT\nlZubq7i4OEVERPjGxMTEKC8vz/d44cKFqq6u1uTJkyVJn3zyiW/flClTdMEFF2js2LGBKB8AACAo\nBOxbsdnZ2Vq1apWSkpK0atUqTZs2TZKUkZGh/Pz8QJUBAABgrRBjjDPOTYpTsbZzYs+Sc/tuLqeu\nmxP7pmf4W1s+FcudJwAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7\nAAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwA\nAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMA\nALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAA\nwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAA\nSxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALBEwILd9u3blZaWpqSkJKWlpamw\nsLDBsdu2bVN8fLxmz57t27ZkyRJdddVVuuaaa5SamqrXX389AFUDAAAEj7BAHSgrK0vp6elKTU3V\nq6++qszMTK1cufKYcV6vV1lZWRo2bFid7WPHjtXEiRMlSSUlJRoxYoQGDhyo008/PSD1AwAAtHUB\n+cSuvLxcBQUFSklJkSSlpKSooKBAFRUVx4xdunSpEhMT1b179zrbw8PDff9cXV2tkJAQ1dbWtmjd\nAAAAwSQgn9gVFxera9eucrlckiSXy6Xo6GgVFxcrIiLCN27z5s1av369Vq5cqcWLFx8zz/PPP6/n\nnntOu3bt0mOPPaYzzzzzhOqIjOzUvEaCTFRU+PEHWcaJPUvO7bu5nLpuTuybnuFvbXV9A3Yq9ng8\nHo+mTp2qmTNn+gLgr11//fW6/vrr9e2332rSpEnq37//CYW78vIq1dYaf5XcpkVFhausbF9rlxFQ\nTuxZcm7fzeXUdXNi3/QMf2vp9Q0NDTnpD6MCEuzcbrdKSkrk9Xrlcrnk9XpVWloqt9vtG1NWVqai\noiKNHz9eklRZWSljjKqqqjRjxow68/Xs2VPR0dH67LPPlJSUFIgWAAAA2ryABLvIyEjFxcUpNzdX\nqampys3NVVxcXJ3TsDExMcrLy/M9XrhwoaqrqzV58mRJ0pYtW3TOOedIkn788Ud98803vscAAAAI\n4KnY7OxsTZkyRYsXL1bnzp19P2WSkZGhu+66S7179270+QsXLtSWLVsUFhYml8ulRx55RD169AhE\n6QAAAEEhxBjjjIvOxDV2tnNiz5Jz+24up66bE/umZ/hbW77GjjtPAAAAWIJgBwAAYAmCHQAAgCUI\ndgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDY\nAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAH\nAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0A\nAIAlmhXsPB6PbrzxRn/VAgAAgGZoVrAzxujzzz/3Vy0AAABohrDjDRg6dGiD+4wxfi0GAAAAJ++4\nwe7nn3/W5MmT1a1bt2P21dTUaMKECS1SGAAAAE7McYNdr1691KFDB/Xv3/+YfTU1NXxqBwAA0EYc\nN9j9+c9/1qmnnlrvvnbt2mnlypV+LwoAAAAn7rjB7uKLL25wX0hIiPr16+fXggAAAHBymvyt2IYC\nXH2naAEAABB4TQ52Ho+n3m21tbV+LQgAAAAn57inYtPT0xUSEqKamhr96U9/qrNv165dSkhIaLHi\nAAAA0HTHDXbXXXedjDHKz8/Xtdde69seEhKiyMhIXXLJJS1aYLDwHKrV/7bs1iFv2/gEM7zzz9pX\neaC1ywgoJ/YsObfv5nLqujmxb3qGP3Vo79LQyE6tXUaDjhvsRo4cKUmKj49Xjx49WrygYFVQWKHF\nr2xq7TIAAEAL+/3ZETr9FFdrl1Gv4wa7I7755htJUo8ePbRt2zZlZmYqJCRE2dnZBD5J8ed00ZwJ\n/XWotm38rl9EREdVVOxv7TICyok9S87tu7mcum5O7Jue4U/tw0J1ztlnqKxsX2uXUq8mB7sFCxbo\nhRdekCTNmTNHvXv31mmnnaZp06bxW3b/r8sZ9f/eX2uIiuqk9mobITNQnNiz5Ny+m8up6+bEvukZ\nTtLkYFdRUaEuXbrol19+0X//+1899dRTCgsL4xo7AACANqLJwS4iIkI//PCDvvvuO/Xu3Vvt27fX\ngQMHuKUYAABAG9HkYHfHHXdo1KhRcrlcmj9/viTp448/1nnnnddixQEAAKDpmhzsRo0apREjRkiS\n796xF110kZ544omWqQwAAAAnpMl3npCkgwcPat26dXrmmWckSYcOHZLX622RwgAAAHBimhzsPvvs\nMyUnJ+u1117T4sWLJUk//PCDsrOzm/T87du3Ky0tTUlJSUpLS1NhYWGDY7dt26b4+HjNnj3bt23a\ntGlKTk7W1VdfrTFjxig/P7+ppQMAADhCk4PdY489pgULFmjZsmUKCzt8Bjc+Pl4bN25s0vOzsrKU\nnp6udevWKT09XZmZmfWO83q9ysrK0rBhw+psHzx4sF577TWtWbNGt99+u+69996mlg4AAOAITQ52\nO3bsUP/+/SUdvp2YJLVr165Jp2LLy8tVUFCglJQUSVJKSooKCgpUUVFxzNilS5cqMTFR3bt3r7P9\n8ssvV7t27SQdvrZv165dqq1tG7fvAgAAaAua/OWJHj166MMPP9SgQYN82z7++GPFxsYe97nFxcXq\n2rWrXK7Dt99wuVyKjo5WcXGxIiIifOM2b96s9evXa+XKlb7TvfXJyclRYmKiQkNP6BJBRbbhe7u1\nhKio8NYuIeCc2LPk3L6by6nr5sS+6Rn+1lbXt8nB7qGHHtL48eOVmJiogwcPKjMzU++8806jAexE\neDweTZ06VTNnzvQFwPr8+9//1muvvaacnJwTPkZ5eZVq28gtv1paVFR4m73dSUtxYs+Sc/tuLqeu\nmxP7pmf4W0uvb2hoyEl/GNXkYPfFF19ozZo1WrNmjUaPHi23263Vq1frjTfe0IUXXtjoc91ut0pK\nSuT1euVyueT1elVaWiq32+0bU1ZWpqKiIo0fP16SVFlZKWOMqqqqNGPGDEnSW2+9pfnz52vFihXq\n0qXLyfQLAABgrSYHu0WLFunWW29VRkZGne1LlizRuHHjGn1uZGSk4uLilJubq9TUVOXm5iouLq7O\nadiYmBjl5eX5Hi9cuFDV1dWaPHmyJOndd9/VzJkztXz5cnXr1q2pZQMAADjGcYPdJ598Ikmqra3V\np59+WucWYj/99JM6duzYpANlZ2drypQpWrx4sTp37uz7KZOMjAzddddd6t27d6PPf+ihh9SuXTvd\nddddvm0rVqzQmWee2aTjAwAA2C7EHOdmr0OGDJF0+AsQR586DQkJUVRUlDIyMjR06NCWrdJPuMbO\nbk7sWXJu383l1HVzYt/0DH8L6mvs3nnnHUnSgw8+qDlz5pzUQQAAANDymvx7IYQ6AACAtu3EfggO\nAAAAbRbBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEO\nAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsA\nAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAA\nACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAA\nsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADA\nEgQ7AAAASwQs2G3fvl1paWlKSkpSWlqaCgsLGxy7bds2xcfHa/bs2b5tr776qq666ir16tVLq1at\nCkDFAAAAwSVgwS4rK0vp6elat26d0tPTlZmZWe84r9errKwsDRs2rM72uLg4zZ8/XykpKYEoFwAA\nIOgEJNiVl5eroKDAF8pSUlJUUFCgioqKY8YuXbpUiYmJ6t69e53tsbGxOueccxQaytljAACA+gQk\nJRUXF6tr165yuVySJJfLpejoaBUXF9cZt3nzZq1fv14333xzIMoCAACwSlhrF3CEx+PR1KlTNXPm\nTF8A9LfIyE4tMm9bFRUV3tolBJwTe5ac23dzOXXdnNg3PcPf2ur6BiTYud1ulZSUyOv1yuVyyev1\nqrS0VG632zemrKxMRUVFGj9+vCSpsrJSxhhVVVVpxowZfqmjvLxKtbXGL3O1dVFR4Sor29faZQSU\nE3uWnNt3czl13ZzYNz3D31p6fUNDQ076w6iABLvIyEjFxcUpNzdXqampys3NVVxcnCIiInxjYmJi\nlJeX53u8cOFCVVdXa/LkyYEoEQAAIOgF7JsI2dnZWrVqlZKSkrRq1SpNmzZNkpSRkaH8/PzjPj83\nN1eDBw/W2rVr9eSTT2rw4MHasmVLS5cNAAAQNEKMMc44NylOxdrOiT1Lzu27uZy6bk7sm57hb235\nVCy/HQIAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAA\nYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACA\nJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACW\nINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiC\nYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmC\nHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlAhbstm/frrS0NCUlJSktLU2FhYUNjt22bZvi\n4+M1e/Zs37YDBw7onnvu0RVXXKHk5GS9++67AagaAAAgeAQs2GVlZSk9PV3r1q1Tenq6MjMz6x3n\n9XqVlZWlYcOG1dm+bNkyderUSW+99ZaefvppPfLII9q/f38gSgcAAAgKAQl25eXlKigoUEpKiiQp\nJSVFBQUFqqioOGbs0qVLlZiYqO7du9fZ/sYbbygtLU2S1L17d11wwQX64IMPWrx2AACAYBEWiIMU\nFxera9eucrlckiSXy6Xo6GgVFxcrIiLCN27z5s1av369Vq5cqcWLF9eZY+fOnTrrrLN8j91ut3bt\n2nVCdURGdmpGF8EnKiq8tUsIOCf2LDm37+Zy6ro5sW96hr+11fUNSLBrCo/Ho6lTp2rmzJm+AOhv\n5eVVqq01LTJ3WxMVFa6ysn2tXUZAObFnybl9N5dT182JfdMz/K2l1zc0NOSkP4wKSLBzu90qKSmR\n1+uVy+WS1+tVaWmp3G63b0xZWZmKioo0fvx4SVJlZaWMMaqqqtKMGTMUExOjHTt2+D7hKy4u1sUX\nXxyI8gEAAIJCQIJdZGSk4uLilJubq9TUVOXm5iouLq7OadiYmBjl5eX5Hi9cuFDV1dWaPHmyJCk5\nOVn//Oc/1bt3bxUWFio/P1/z5s0LRPkAAABBIWDfis3OztaqVauUlJSkVatWadq0aZKkjIwM5efn\nH/f5t956qyorK3XFFVfo9ttv1/Tp09Wpk7OumQMAAGhMiDHGGRediWvsbOfEniXn9t1cTl03J/ZN\nz/C3tnyNHXeeAAAAsESb+VZsIISGhrR2CQHltH4lZ/YsObfv5nLqujmxb3qGv7Xk+jZnbkedigUA\nALAZp2IBAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAA\nACxBsAMAALCEo+4VG2h79uzRgw8+qKKiIrVv316//e1vNX36dEVEROirr75SZmamfvnlF5111lma\nO3euIiMjJUn333+/8vLyVFZWpg0bNqhjx46+Offu3avp06fr66+/VlhYmEaMGKE777yz3uM3Nk9j\n+36tsVpXr16t5557TqGhoXK5XPrLX/6if/zjH0Hfc2Nje/bsqdjYWIWGHv7/ojlz5ig6OjroX+vt\n27crMzNTZWVlCgsLU+/evZWVlaVTTjlFGzZs0Jw5c1RZWSlJuuyyy/Tggw8qJKR590psq++Rxtai\nPu+8847mzJkjr9er888/XzNnztSpp56qmpoaTZw4UZs2bZIk5eXlOaJvSXr66ae1Zs0ahYWFqWPH\njpo0aZKefvrpoO65oddTkn766ScNHz5c5557rm/bk08+qUcffTSoe27svf/2229r8eLFqqmpkTFG\no0eP1i233FJvLS3B3++jDRs2aNq0ab75y8vLFRUVpZdffrne45/sa9hS8/gYtJg9e/aYTz/91Pd4\n1qxZ5qGHHjJer9cMGzbMfP7558YYYxYtWmSmTJniG/fxxx+b3bt3m9jYWFNVVVVnzttvv90sX77c\n97i0tLTB4zc2T2P7jtZYrRUVFSYhIcGUlZUZY4x5++23zfDhw4O+5+ONrW+bDa/1jz/+aL7++mtj\nzOHX/e677zZ///vfjTHGfPvtt2b79u3GGGN++eUXM2bMGPPyyy83OFdTtdV1a2wtfq2qqsoMGDDA\ntz4PP/ywWbhwoTHGGI/HYz766CNTUFBg+vXr55i+CwoKTGJiotm/f78xxpjnnnvO3HTTTUHfc0Ov\n55F5fr3Nhte5sff+V199ZXbt2mWMMaaysrJOT4HQEut7tIkTJ5pnn3223n3+Ooa/5jkap2Jb0Bln\nnKGLL77Y9/iiiy7Szp07tWnTJnXo0EF9+/aVJI0ZM0Zr1671jevfv78vrR+tsLBQ3333nW666Sbf\ntqioqAaP39A8x9t3tMZqNcbIGKP9+/dLkvbt26ezzjor6Hs+0bGSHa91t27d1KtXL0lSaGioLrzw\nQu3cuVOSFBsbq+7du0uS2rdvr169evn2NUdbXbfG1uLXPvjgA11wwQW+9RkzZozeeOMNSVJYWJgG\nDBig8PBwR/UdEhIij8ejgwcPSjr8t+Hss88O+p4bej0bYsPr3Nh7Pz4+Xl27dpUkhYeHq0ePHtqx\nY0eD9fibv9f3aOXl5froo4+Umppa735/HMOf8xyNYBcgtbW1ev755zVkyBAVFxcrJibGty8iIkK1\ntbXau3dvo3Ns2bJFXbt21V//+leNHDlSGRkZ+v7771u07sZqjYiI0PTp0zVy5EglJibqiSeeUFZW\nlm9ssPbcFDfccINSU1M1b9481dTU1NlnQ98HDx7Uv/71Lw0ZMuSYfeXl5Vq3bp0SExP9esy2um6N\nrYV07HskJiZGxcXFTZ7fxr7PO+88jRs3TkOGDNGgQYP0+uuv67777vONDdaej2f//v0aNWqURo0a\npWeffVbGGN8+G3pu7L2/detWffXVV7rkkkuaVc/J8sf6Hu2VV17RwIED1aVLl3r3++MY/pznaAS7\nAJkxY4ZOO+00jR079qTnqK2t1f/+9z+NGjVKL7/8sq677jpNnDjRj1WemKqqKuXk5Gj16tV67733\nNGXKFN15552+P2Y29ixJ7733nl566SXl5ORoy5YtWrRoUZ39wd73oUOHdO+99+qSSy7R0KFD6+yr\nqqrSxIkTdcstt/j+j99f2uK6NbYW/mJj3zt27NB//vMfvfnmm/rwww81cuRITZkyxbffxp6jo6P1\n/vvv66WXXtIzzzyjN998U6tXr/btD/aeG3vvl5aW6o477lBWVpbvE7xA88f6Hu2ll17S6NGj/TJX\noBHsAmD27Nn64YcftGDBAoWGhsrtdtf52LuiokKhoaE644wzGp3H7XbL7Xb7PrIdPny4ysrKVFFR\noSVLlig1NVWpqan69NNPT7rWX8/TWK3r169XeHi4fv/730uSrrzyShUVFWnPnj1B3fPxuN1uSVKn\nTp103XXXacOGDb59wd631+vVpEmTdPrpp+uRRx6pM/7AgQOaMGGCBg4c6PcLpNviujW0FtOmTfPN\ns23btmNq3blzp+/fEaf2vXbtWsXGxio6OlqSdM011/i+aBDMPTemffv2vlNmkZGRuuqqq3x/G4K9\n58be++Xl5Ro3bpxuu+02jRgx4rjHbQn+Wt8jvvrqK/3888+67LLLfNtO5L+NjfHXPI1q0pV4OGnz\n5s0zY8eONdXV1b5tXq/XDB06tMGLJY/49cWStbW1JiUlxXz33XfGGGM+++wzM2jQIFNbW9toDY1d\ndNmUCzsbqjU/P98MGDDA7N692xhjzCeffGIGDBhgHn/88aDuubGxe/fuNQcOHDDGHL6Q+sEHHzSP\nPvqoMcaO13rSpEnmvvvuM4cOHaqz7+DBg+bGG280c+bMafT4J6Mtrltja/Fr+/btM/3796/3SwRH\n1Hdhvc19r1271qSkpPi+PLF69Wpz7bXXBn3PR9T3eu7evdvU1NQYY4yprq42N998s1mxYkXQ99zY\ne7+iosJcddVVJicnp9E5WpI/1/eIRx55xMyePbvR4zb3GP6e52ghxhx1EQD86vvvv1dKSoq6d+/u\n+xp5t27dtGjRIm3YsEFZWVl1vt585Fz+nXfeqY0bN6qkpETR0dGKjY3VsmXLJEn5+fmaNm2aampq\ndOqpp+qvf/2rLrzwwnqP39g8je37tcZqXb58uV588UW1a9dO7du319ixYzV58uSg77mhsV9++aUy\nMzMVEhKiQ4cOKSEhQQ8//LB27twZ9K/1e++9p9tvv73OT7n06dNHWVlZysnJ0d/+9jfFxsb6xicn\nJzf79HBbfY80thb1efvttzV37lzV1tYqLi5Os2bN0mmnnSZJGj16tEpKSnw/nTBo0CDdfPPNVvdt\njNHcuXP1zjvvqH379urcubPGjRunO+64I+h7ru/1fPTRR/Xmm2/qqaeeUmhoqA4dOqTExERdffXV\nSk1NDeqeG3vvz549Wzk5Ofrd737n23fjjTcG7BRmS/z9OHjwoAYOHKgXX3xRPXr0aPT4J3uMlprn\nCIIdAACAJbjGDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAVtm5c6cSEhLk9XpbuxQA\nCDiCHYCgN2TIEH388ceSDt+z9Msvv5TL5QrY8fPy8jR48OCAHQ8AGkKwAwAAsATBDkBQe+CBB7Rz\n505NmDBBCQkJeuaZZ9SzZ08dOnRIknTDDTdo/vz5GjNmjBISEjRhwgTt2bNH999/v/r06aPRo0fr\np59+8s23detWjRs3Tv369VNSUpJef/113773339fV155pRISEjRo0CAtW7ZM1dXVysjIUGlpqRIS\nEpSQkKCSkhJt3LhRaWlp6tu3ry699FJNnz5dNTU1vrl69uypnJwcDR8+XAkJCVqwYIGKioo0ZswY\n9enTR3fffbdv/JFPBJ9++mldfPHFGjJkiNasWROgFQYQVJp04zEAaMMuv/xy89FHHxljDt/HMzY2\n1ng8HmOMMWPHjjXDhg0zP/zwg6msrDQjRowww4cPNx999JHxeDzmgQce8N2bcf/+/Wbw4MFm9erV\nxuPxmK+//tr069fPfP/99/Nnly0AAANKSURBVMYYYwYOHOi7p+PevXvNpk2bjDHGfPrpp2bQoEF1\nasrPzzdffvml8Xg85scffzTJyclm+fLlvv2xsbFmwoQJZt++fea7774z559/vrnxxhtNUVGRr86X\nXnrJN39cXJx57LHHzC+//GLy8vJMfHy82bp1a8stKoCgxCd2AKw3atQo/eY3v1F4eLgGDx6ss88+\nWwMGDFBYWJiSk5NVUFAg6fD9cs866yyNHj1aYWFh6tWrl5KSkrR27VpJUlhYmLZs2aKqqiqdfvrp\nOv/88xs85gUXXKCLLrpIYWFh6tatm9LS0vT555/XGXPbbbepU6dOOvfccxUbG6uBAwfq7LPP9tV5\npK4j7r77brVv3179+vXTZZddpjfeeMPPKwUg2IW1dgEA0NKO3FBbkjp06FDn8SmnnKLq6mpJ0o4d\nO7Rx40b17dvXt9/r9erqq6+WJD311FNasmSJ5s2bp549e+r+++9XQkJCvcfcvn27Zs2apU2bNunA\ngQPyer3HBMHG6urQoYN2797te9y5c2eddtppvscxMTEqLS09oXUAYD+CHQD8P7fbrT/84Q9avnx5\nvfsvvPBCLVmyRB6PRzk5Obrnnnv0/vvvKyQk5Jix2dnZ6tWrl+bNm6dOnTppxYoVWrdu3UnXVllZ\nqerqal+4Ky4u1rnnnnvS8wGwE6diAQS9Ll266Mcff2z2PImJiSosLNQrr7wij8cjj8ejjRs3auvW\nraqpqdGaNWu0b98+tWvXTh07dlRo6OE/oZGRkdq7d6/27dvnm2v//v3q2LGjOnbsqK1bt+r5559v\ndn0LFy5UTU2NvvjiC7333ntKTk5u9pwA7EKwAxD0xo8fryVLlqhv377N+lSsU6dOWrZsmV5//XUN\nGjRIl156qR5//HHft1NfffVVDRkyRH369NELL7yguXPnSpJ69OihP/7xjxo2bJj69u2rkpISTZ48\nWbm5uerTp4+mTp2qK6+8slk9dunSRZ07d9agQYM0adIkZWdnq0ePHs2aE4B9QowxprWLAAA0LC8v\nTw888IA++OCD1i4FQBvHJ3YAAACWINgBAABYglOxAAAAluATOwAAAEsQ7AAAACxBsAMAALAEwQ4A\nAMASBDsAAABL/B/6gqXq0tJ7HgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qAF1qb0lcGOR",
        "colab_type": "code",
        "outputId": "04ef8665-005d-4b9d-8c54-714118f80149",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "source": [
        "plt.figure(figsize=(10, 5))\n",
        "sns.lineplot(x=\"timestamp\", y=\"base\",data=merge[(merge['building_id'] == bid) & (merge['meter'] == m)])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f8a2e2fbe48>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 243
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmsAAAFCCAYAAACuB5qHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOy9a5gtVXnv+6/bnH1f1wYWC5RERTES\nCRA121zceM8xmuPxnMN263lOiPrkUbONOZrjCSdA9CEGJGqyo8E8O25zMdEkbjhCDMS4ElERBQRk\niYBcXIjr3uvWl9lzzrqcD6NG1Rij3lE9u7tqzurV7+9Ld89ZXXNUzapR7/i/NydJkgQMwzAMwzBM\nI3FHPQCGYRiGYRjGDhtrDMMwDMMwDYaNNYZhGIZhmAbDxhrDMAzDMEyDYWONYRiGYRimwbCxxjAM\nwzAM02DYWGMYhmEYhmkw/qgHUAXHjy8ijjdHubgdO6YwN7cw6mEMFT5mZjVsxnO3GY8Z2JzHvRmP\neZjUfX5d18G2bZOr/r/TwliL42TTGGsANtWxSviYmdWwGc/dZjxmYHMe92Y85mHSxPPLblCGYRiG\nYZgGw8YawzAMwzBMg2FjjWEYhmEYpsGwscYwDMMwDNNg2FhjGIZhGIZpMGysMQzDMAzDNJihle64\n7LLL0Gq10G63AQDve9/78Au/8Au4//77cdVVV6Hb7WL37t34yEc+gh07dgxrWAzDMAzDMI1mqHXW\n/uRP/gTnn39+9nccx3j/+9+PD3/4w7j00kvxyU9+EjfccAM+/OEPD3NYDMMwDMMwjWWkbtC9e/ei\n3W7j0ksvBQBcfvnluO2220Y5pIH4yr1P4/q//c6oh8EwDMMwzCZgqMra+973PiRJgksuuQS//du/\njQMHDuDss8/O3t++fTviOMaJEyewdevWgfe7Y8dUHcO18tkvPwoAmJ2dHurnSkb1uaOEj5lZDZvx\n3G3GYwY253FvxmMeJk08v0Mz1j772c9i165d6PV6uPbaa/HBD34Qr3zlKyvZ99zcwkjaQxw5Mj/0\nz5ydnR7J544SPmZmNWzGc7cZjxnYnMe9GY95mNR9fl3XWZPANDQ36K5duwAArVYLb37zm/Gd73wH\nu3btwv79+7Ntjh07Btd1V6WqMQzDMAzDnM4MxVhbWlrC/LywVJMkwZe+9CVccMEFeMELXoDl5WXc\nc889AIDPfe5zeM1rXjOMITEMwzAMw2wIhuIGnZubw2/+5m8iiiLEcYxnPetZuPrqq+G6Lq6//npc\nffXVWukOhmEYhmEYRjAUY+3cc8/FzTffTL538cUX45ZbbhnGMBiGYRiGYTYc3MGAYRiGYRimwbCx\nxjAMwzAM02DYWGMYhmEYhmkwbKwxDMMwDMM0GDbWGIZhGIZhGgwbawzDMAzDMA2GjTWGYRiGYZgG\nw8YawzAMwzBMg2FjjWEYhmEYpsGwscYwDMMwDNNg2FhjGIZhGIZpMGysrYMkSUY9BIZhGIZhTnPY\nWFsHbKoxDMMwDFM3bKytB7bWGIZhGIapGTbW1kHC1hrDMAzDMDXDxto64JA1hmEYhmHqho21dcDG\nGsMwDMMwdcPG2rpga41hGIZhmHphY20dsLLGMAzDMEzdsLG2DthWYxiGYRimbthYWwdcFJdhGIZh\nmLphY20dsK3GMAzDMEzdsLHGMAzDMAzTYNhYWwesrDEMwzAMUzdsrK0LttYYhmEYhqkXNtbWQcy2\nGsMwDMMwNcPGGsMwDMMwTINhY20dcOkOhmEYhmHqho21dcCmGsMwDMMwdcPG2npga41hGIZhmJph\nY20dsK3GMAzDMEzdsLG2DjhmjWEYhmGYumFjbR2wrcYwDMMwTN2wscYwDMMwDNNg2FhbB+wGZRiG\nYRimbthYYxiGYRiGaTBsrK2DmJU1hmEYhmFqho219cC2GsMwDMMwNcPG2jpgW41hGIZhmLphY20d\nsLHGMAzDMEzdDN1Y+9M//VM897nPxaOPPgoAuP/++/H6178er371q3HFFVdgbm5u2ENaOxyzxjAM\nwzBMzQzVWPve976H+++/H7t37wYAxHGM97///bjqqqtw++2349JLL8UNN9wwzCGtC7bVGIZhGIap\nm6EZa71eDx/84AdxzTXXZK/t3bsX7XYbl156KQDg8ssvx2233TasIa0bttUYhmEYhqkbf1gf9Md/\n/Md4/etfj3POOSd77cCBAzj77LOzv7dv3444jnHixAls3bp14H3v2DFV6VgHZdu2CczOTg/9c0fx\nmaOGj5lZDZvx3G3GYwY253FvxmMeJk08v0Mx1u677z7s3bsX73vf+2rZ/9zcAuJ4+DrXsWOLaDvD\n/czZ2WkcOTI/3A8dMXzMzGrYjOduMx4zsDmPezMe8zCp+/y6rrMmgWkoxtrdd9+Nxx9/HC9/+csB\nAAcPHsSv//qv461vfSv279+fbXfs2DG4rrsqVW2UcMwawzAMwzB1M5SYtXe84x34+te/jj179mDP\nnj0466yz8Bd/8Rd429vehuXlZdxzzz0AgM997nN4zWteM4whVQLbagzDMAzD1M3QYtYoXNfF9ddf\nj6uvvhrdbhe7d+/GRz7ykVEOaVX86PA8du+cHPUwKuFbDx3C+eduxbbp9qiHwjAMwzCMwkiMtT17\n9mS/X3zxxbjllltGMYx18+dffAgvef5Zox7Guun2I3zqi9/Drh0TuPbtLxn1cBiGYRiGUeAOBgyS\nNPju6MnlEY+EYRiGYRgTNtYYxLH4mXDGBMMwDMM0DjbWGMSpkca2GsMwDMM0DzbWGERpjbpR1Kpj\nGIZhGKYcNtaYzEhjU41hGIZhmgcbawwragzDMAzTYNhYYxBxsBrDMAzDNBY21hhW1hiGYRimwbCx\n1mC+v+847nroYO2fE7GxxjAMwzCNZaTtpphyPvJ39wFA7V0SWFljGIZhmObCyhrDxhrDMAzDNBg2\n1pisKC7DMAzDMM2DjTWGY9YYhmEYpsGwsbYBqLtnJ7tBGYZhGKa5sLG2AQijuNb9s7LGMAzDMM2F\njbUNQLdfr7HGMWsMwzAM01zYWNsA9MOajTVW1hiGYRimsbCxtgHo9aNa989uUIZhGIZpLmysbQC+\n+8RcrftXlTV2iTIMwzBMs2BjbQNw+Hin1v2rxlq/5vg4hmEYhmFWBxtrDabd8gDU76ZU1bReWK/L\nlWEYhmGY1cHGWoOJ0pId0RBLd/RYWWMYhmGYRsHGWkNJkgRhJIyo2pW1mJU1hmEYhmkqbKw1FNVA\nKyutcd8PjuAvbn2oss+qu0wIwzAMwzCrg421NXDW9gkAgO85tX2GajSFJcbaf/3Cg/jG3oPr+iwt\nZo3doAzDMAzTKNhYWwPSSGsHXm2fodZWGyRmLYrXbmSxG5RhGIZhmgsba+ugzppkPUVZGyRmbT2K\nGCcYMAzDMExzYWNtDUjTZh1i1oqoytog7aB664g1Y2WNYRiGYZoLG2trIbVtkgYpa/11tKSKWVlj\nGIZhmMbCxtoayJS1Oo21VcasddejrCVqNigrawzDMAzTJNhYWwd1ukFlNmi75Q2mrK3DyFL332Vl\njWEYhmEaBRtra0C6P4eRYDA+oLG2Hvel1huUlTWGYRiGaRT+qAew0UmSBI5Tfb016QYdb/uDGWvr\nVNZcx4HjrC9RgWEYhmGY6mFlbQ2oglpd6po0msZadmPt0LGl7Pf+epS1JIHrOgh8lxMMGIapjJML\nXfxfn/gGbrnzh6XbPbzvON5+/b9hodMfzsAYZoPBxto6GaSsxlpYWg4BANMTgTXB4KBirJV1OViJ\nOE7guQ4811lXcV2GYRiVIyeXcXy+i//va0+WbnfLnT9EFCfYd3B+SCNjmI0FG2trQDWLZLP1qlno\n9OG5DibH7Mqa+tmDZIzaiGKhrLmugxrD8BiG2WTIeWklD4Tcrs4WfgyzkWFjbS0oE88g8WRrYaHT\nx+R4AM91S4y11dVisxHHCVwHcByn1qQJhmE2F4POS9Iz4Hn8SGIYCr4z1oA6/dTlBl3s9DE1HsDz\nnKEYa57rwHXqOx6GYTYfAxtrrKwxTClsrK2TOpW1qTFfxJFZXJzqZ9vGcXKhiyv+cA+++/ic9bNk\ngoHrsrLGMHXxT9/8IX79uj2jHsZQ0ecoeh5b7oV46tACAHAYBsNYYGNtLSgTynpixcpYXBZuUNdx\nYLMHNWXNMo59h0TA7r/e+yPrZ0WpsuaAY9YYpi6+8NUnkCSbS72OlLjaxTRpyuTEQi/7nReLDEMz\ntDpr73znO/H000/DdV1MTEzg937v93DBBRfgySefxAc+8AGcOHECW7duxXXXXYfzzjtvWMNaE4li\nrUU1TS4LnT7O2zVTqnaFA/QP9Vxhj0cliRBxmmAA8GTJMHXTCyOMtTZHiUtVTVvs9DEz0Spso/ZY\nTjgZnWFIhqasXXfddfjiF7+Im2++GVdccQV+93d/FwBw9dVX481vfjNuv/12vPnNb8ZVV101rCGt\nmSQBvNS4KTOC1r7/BAudEFPjgTDWbDFrA7hBMyOsZDUvi+K6jv2zGIaphs1UeFqdlxY7tLLWV84H\nLxYZhmZoxtr09HT2+8LCAhzHwdzcHB566CG87nWvAwC87nWvw0MPPYRjx44Na1hrxksDYeuIWev1\nY4RRLIy1EgNqkASDzKgsGadU1hyH3aAMUzeyO8lmQJ27bAVv1ULcdcUAM8xGZ6gxa1deeSVe9rKX\n4WMf+xiuu+46HDhwAGeeeSY8zwMAeJ6HM844AwcOHBjmsNZE5l6soYjs4rKY1CbHfLguvdpMkgQ3\nK4UmV4qdKxtnnCqFp1OCwT/ftQ9X/OEezaBlmCbwO3/2TXzl3qdHPYyhEA1irCmt8k6X+Ydhqmao\ngRPXXnstAODmm2/G9ddfj/e85z2V7HfHjqlK9jMoruugFbjodIGZmXHMzk6v/E+roAehhu3cPolu\nJNyuO3dOZT1IZ2entYbrruugPRaQ49h/YhkA4DiudZy+76HV8hHHCYLAq/x4qmC1Y7rt208BAMYm\n29g2PVbHkGqnid/DRqHp527PfT/G5a+5oNJ9NvGYx5UYtVbbJ8c4fngx+31mevXzaROPu2424zEP\nkyae35FEuf7qr/4qrrrqKpx11lk4dOgQoiiC53mIogiHDx/Grl27VrW/ubmFocZaRVEMNzWcjs4t\nYvtEUOn+Dx8RaeydpR6WOyJT6vDhebiug9nZaRw5Mo9ON4//8FwH8wtdHDlSbNVy7JiYCLv9kHwf\nADrLfcRRjDhOsLxs325UyGNeDX5aXPOpp08g3DlZx7BqZS3HzAg2wrnzHKfSMTb1mE+c7GS/HzvR\nIcd4ZG4h+/34icVVHUdTj7tONuMxD5O6z6/rOmsSmIbiBl1cXNRcm3v27MGWLVuwY8cOXHDBBbj1\n1lsBALfeeisuuOACbN++fRjDWheDxIKtFem68zwHjiVLU/1cryQJQW5XZszGcSxKd5xGHQxavri0\nuTE000RaweaomqTOO7ZYPTVmjVsTMwzNUJS1TqeD97znPeh0OnBdF1u2bMGNN94Ix3FwzTXX4AMf\n+AA++clPYmZmBtddd90whrQuEgD+AFmWa0X2/PQ9F+nHiM/x8m3UDCpROLfcWCszWuJEWPtukpw+\nxlogTtai5biTJEEUJ5kCxzSPKI7R6UYYb3tZjOhaCaN8QdIE5GLCRhTHpcecpPfqes9L3aiLSlsW\nbJ9j1hhmRYZirO3cuRN///d/T773rGc9C//wD/8wjGFURpLkbrY6EgzypsauksiQkNsAqbFmGYf8\nv/mlPrq9CO2WV9hGPsji+PTJBg2ksrZMG2v/9M19+B93PIFPvPcXMd7eHDWvNho33vw93PvoEQDA\npz9w2Zr3M7/Uw3v+5Ov4T694Dl556blVDW9dlNVZu/+xo/iTf/wufv+KF+HcM2h3yV/e9jDueODA\nus7LMJDzVCtwtUQCla6mrJ0mExDDVEyzl2UNJivdUUOdtb7SJ08qa4lhRckaa6/7D8+E57lazTUV\n1ahb7tF1jpa6IcbbvnCDnmaTZWhZzd/xwH4AwKmlHvk+M3qkobZe5k6JJJuvPTDaLPMtky3MTIqA\n+zI36H3pcT++/6R1mzvSY2l6trNcLI63fM3dqbKoLKhYWWMYGjbW1ohN8aoC1Q2ax6yZ24iJ7xln\nTA8Us1Y21oW0abzrFI3CjYo8H32LMd1O3aTL3c1T82oj019HIVmbkTBsojjBJc+dxTPOmKpsTE2P\nyZQFt1uBq7k7VdRQhdNtscgwVcHG2hpIkqTWoriqG1RmnZqTmJqE4LqOdRwrGWtJkmBRGmuuvQ/p\nRkOu0G3156Sy0fSHHSNYz/fUlO84ShN5ghKXIAAMElYn54WmHJuNKBZzZcv3rAbqQqefJWyxssYw\nNGysrQE1waCOmDXNDWqZxKT6FnhummBQHrMG0KvWbj9CGCWYGg82TDbo/FJPc51QyK/F5iYKfKGs\nSRfZqFlaDldUFY6c6NRyvY2COElw+PiSVck1r0NbosggHD0paw2ueReVEMUJPDc1XEqUwqUV1N5O\nN4Tvi4OxnZejJzo4eMx+fodFFKXHHLjWY15cDjP3cNnlPb/Uw9OHFxrv+mWYOmBjbS0kgOet3CB9\nrch9emY2qLZNnG3jua5VWYtXUNaWlkUc26R0g24Aae23//Qb+H8+dVfpNtKosblBx9NEiy/dta/a\nwa2B5V6Id3/8Dnzhq49bt/nB0yfwf9/4TXz+K48NcWT18ZV7nsYHPnUXvvXQIfJ9tY4ggBWN8zJu\n+5b4jt0RW2vCcHHR8l30LSpTP4xwz8OHrfuIkwTv+tgdmUq1QPTb/NHhBfzOjd/E7/75XXjg8blq\nBr9GpJrY8j10LaU7FpeFsg+UK2tXf/rbuOrT38bn95we9wDDrAY21tZIVmethpWrXDkGqhvU+Bwz\nCcE2DFVxo5QbudoNfBEf13RlrdePEMUJFjr9UtVAnkOb4hikMWtowOGeWhKGyLe/b39IH007UTx1\n6PQohnn4uCiWalM2uz3xYP+JXTMAciV5LcjMbalSjwJZKsZzHfi+a1WHVOOLWgiasXv9qGgAnVjo\nZr/PnRytchxGojzOxJiPJYvB3Q9jjKWLpzJ1+eSiSAY6qRwfw2wW2FhbAwmU0h01KGthpqwpbtBC\nzJqehGAzslaKWQuN+LimC2tqjI5tpQ7k58f2kJcFOtej2FRFJ1U3yzIE5XFnRuYGp5vGbC0SyhCQ\nGyU/mRpr64kNlQubUbqQ5e3peQ58r8xYy69HKqnCfI2af9TYsFHHtEVRDN9zMDUeWMcSRnFWd842\nj8Vxkp3DMhcyw5yusLG2FpIEfo0JBpkB5bpKzJq+jZ6EYJ/kwhWMNTnZ+64D13Ea7wZVJ/yyB5E8\nP7aHonzoDRIrVjeyFlzLtxtimbF2mhTxlbFWtu9QGuJj7ZUVl5WQxksd9+qgSEPRcx34nmNdRKjn\ng0pCMLsAUNe3+n+jNtbCtPD05HiAhU5IquFhlGRFrG3fs3qctk4IDHM6c3rM/EMmQe4GraeDgXRN\nOiXZoFJZS40syzDiFRIMMsPQd+GUGH3DIIpjPP7jk3js6ZNWI+uo4tY5dsruDulnyhq9HznhJxi9\nuraYqWb07RjHCR764TEAuUq00VkgjLVON8xcZdKYlu6xMkNrJVefLBlRhwo+KJla7rqlytqh40vZ\n71T2pKkqUUafPHee62DfiN3mYRTD91xMjQcIo5g8pjCKsyLWMiSguE1+nOsp4zJMFpf7hdhLhlkr\nqzbWDhw4gPvvv7+OsWwo8jprdXQwyCd2ezZoPiE7jmON31IfCtRYcxVPuFxHGbL2tQcO4Nq/vhd/\n8Df34vZvP0Vu85V7n85+/1fld5NcWbO4QZUJXyZZjIrFFVSzex89gsf3nwJw+qgKmbGmGMrv+tgd\nePfHvwYg/35kpX/bIuKRp47j/X92J+763kHy/ThOsmtglAqqNDY914Hv2o21Pco1vVZlTaqS7cDD\nY0+fHKnBEIYxvNQNCtBKXxglmap8650/pPejHGe3IXXzVuI3P/41/M6f3TnqYTCnCQMba/v378fl\nl1+O1772tfi1X/s1AMBtt92GK6+8srbBNRXRbqq+DgZRnMBB2q/Tlg0aKzFrjv1BtKgYIrSylmee\njrp0h+wmMNbyrKqZ5zmi2wKAsnjxcCVlLYyzuMOy2LdhIB8+tuORD7iZieC0ideR16Wt9IQ0SsZT\nN6gtUeRQmqjw0L7j9H4Ug2eUMWvy3nNdB57n2DuOxAl2bhmD45Qray+/5BwA9PUtladfuuhsAMBy\nb3TXd+YGHbMba1EUr9jYXj1OW3HdJrI44oUgc/owsLF21VVX4WUvexm+853vwPfFavelL30p7rxz\nc64cHNeBg3riYOIkyRQ16QY1bahIUdbK3KDq5FiWYBD4aezbCNWHXl+k+W+ZbFljbaIowe7ZSeye\nnbSqZnGcN6Qvc4NOjYvreNQGUFmBVCA/ht2zU6eFsiYLMQP2mCppcIynyprtPpOdKLoWg0T9bm0G\n0jDIlDXPQeC71jZoC50+fuontmPHzBhplMjv/+LzZwHQyrHc5sztE+KzR+g6jzI3qPgeqV69/SjO\nYtZshIoyOer7lWFGwcDG2oMPPoh3vOMdcF2hwADA9PQ05udPj1ICqyFJcuWrlg4GaYo/AKXdlOEG\njVVFzN4marHTL02GUN2po3aD9sIIrcAtzRyLohi+64h+qJaHkO76pQ+oH8aZa6Y/YgNIGia2scrj\nmWj7GyZep4zlnii/4nsuFi1B53mCQeoGtZwbqcjY1CNpuPieO9KYNXVx5aXzhnncwogNMTUeoB3Q\nFf97Siyf6ziWBAMRAybd6qNMrBClO3I3qKmkyixPmQ1qQ56/8ba/IRYsG0n9YzYGAxtrO3bswL59\negHRxx57DLt27ap8UBsBB8KdUYdrJY6TzEizJTLkcW1O6r6k97XQ6SvVwe1uUN9z4WC0btBeP0bL\n9zA5HljdY6J9jZtm1NmMtZWDkYWyJh4g3REbQD88KBY8NsVAHs/42MZ4UK2ENMTP2j6OOEnQ6YaF\n77KQYGAt6SB+dnvlJUDG214jYtb8NMFAfU3S6YaIkwSTYwECn674L48n8MU9QNZi64tSGLIl3igV\nxX6qrE1ajDVV2S/dT5gvWDaCskYVK2aY9TCwsXbFFVfgN37jN/CFL3wBYRji1ltvxXvf+168/e1v\nr3N8jcZz61mtx3ECz9HdoObELo1E2ZLKpqwtdPqYmWiR+wDUOmsOXHe02aD9MELgi/gWW6xHmLav\nCTzX6gbVlDXCoEuSBL0wzh4go1Srfnx0Ed97UmR62gyxKIrhQBguG+FBtRIyoWN26zgAEddjPsRN\nY81maMn7wKasdbPYN3/EpTvymDVprJnXXdZNZMxHK/DI60G+1go8a1ZpN4zQCrw8CaoBbtDJMaGQ\nmve1WgC8dD9xvmDph/HI22itxKgzzJnTD3/QDd/0pjdh69at+PznP49du3bhpptuwnve8x684hWv\nqHN8jSRJADhC1aqjg0GkxazJz7Qpa7LkBr2vxU4f554xBYA2xNREhbLYt2HQ68doBx7aLc8axyXb\n13iei74ly00z1si6TuJ9qayNUq2aT6uyA3ZlrR/FQk103ZHXhKuCMNYzPUVJh4jcRmYJ2hZF0uCx\nGWuy6O7MZAvHRtgHVssGtYQlyLIsQSBaUlELFnmNtHy7utxP3aBejbUgB0W6QaXhaC1B5Lv4+Z/e\nhW/upbN65XG2leK53qibvZZga1rPMGtlYGMNAF7xildsSuPMJLXV0gbqNSlr0lizdTDQMkZp92Wv\nH6EXxrmyVlKTyZfZoKNMMEgfMi2LCwjI29ckCTBvc4NqteWK78vsS5mhNkq1ylcUBbuylmQKatPb\ngQ1Ckp7udhpvFkZJoRWRvFZl7TmbwSFfX7a4QaXLdctkqxlFcb1cWTMNrcxwcV20Ag/HibZKmbLm\ne2ncJn3ft3wvMwpH2fg8jOJsQQnYSxD5nostabgGuR9ppGbFc4Em14fmmDWmaga+3G+99VY8/rho\nNP3kk0/iLW95C9761rdmr206HNQWsxbFubLmZL1BzW3ibOXsWuqsZQ+qKbsbNNLcoKOOWRPum1bg\notePyGOSx+1Z4nUAaJl2lPEpDdQmKGvqpG5zx8rConUltAwbec/IB28YxYUYH/m9rdSGSD7sly3f\noWqsJcno3Py5sqbEkhUK3OaGi63Ze6asBSKBICTmn14oSmHkbtBRKmui3ZTjiEWlef1qCU4li0W5\nAFup00FT2Ci14JiNw8DG2sc//nFs2bIFAHDdddfhwgsvxIte9CL8/u//fm2Day65S6OOSSOJkyxW\nzVYUN4qSbDJ2LI3c8/pcLXIfQD4JyrZVo80GFYHRge8hSSzGZZxkQdo2xeDxH5/Uti98Tvpgn1yh\ndEcYxbjv0SO1tuxRP9s2wcsHnpdm6250dS0zxILcmDBjfKRybAvGl8jG9ja30yNPifprZeqy3O7B\nJ+ZWcRSrQ00IkvFZZuB/1vrNc8SCxVK6w3WcNBTAIUuACGXNLc0C73RD3P3wYTz6oxPrO7AVCKME\nfmpwUzGxUhkMfFdc36Cv78NpPb12idKaJAkeeOxo7ce0EnGc4O6HD1Wyr6cOzVtV443I4eNLOEko\nxszKDGysHTt2DDt37kS328W9996L9773vXjXu96Fhx9+uM7xNZI4EWqW67r1lO5Icjeo/FlMMFDK\ne1jcoLJy+fREQO4DyFf3MmZtlMqNTDCQagqleIVRkrqS7P0V//pfHgUAbJ1qkedFBp1PtAM4jt1l\n8d3H5/Bf/8eDuOmOJ9Z0PIOg1QGLYtqgTl2/tgLJGw0ZRyjj0fpRnBnEuXEmFFS5WLEFyd/xwIF8\nv4TK9GCavJG7U4vbdLohrvvb+/Cxv39Aa/dUJWrMmmd1g+b3YuB51kbuQSBCFnybGzSMtQQDalHz\npbv24c9u3os//Ox3cOhYPccM5DGmQOoBMIaidWIpaeF3z8OHAQBbptpiG+I+eerQAv74H7+LP/zs\nd0aqlv/g6RP4xoN07N1qCKMY1/z3u/GJm/ZWMKpm8IFP3YX3/uk3Rj2MDcnAxtr27duxb98+3HHH\nHbjwwgvRarXQ7XYbn5VTBy36PmIAACAASURBVFEkHiR+jTFrZlHcYumOOFs52xQxGbDcLsmo64Vx\nVmNN1KIanXzfjxJhrKWuDkrxiqI4DdIur7P2cz91Js4/dytpfErDYGrcz2peUcgV4Hydylr6UPn5\nC0UJHMr1FaYJBvKa2Oj3nLSX5PccRXGWDap2BvFcsYCw9aw1H8jUvZgkCV564VnWAHdzP3W1HlNj\n1mxGt2qsiRALeqwyyN63uUFTZc220AOAU0piS53KcRznc5hDhFnIv920/pz4Hzop6JlnTmPnljHr\nNicXc8VmlA3slypq7yXV4keeGq1SyDSDgRMM3vnOd+KNb3wjPM/Dxz72MQDAnXfeiec973m1Da6J\nJInoNRh4bumDfj2oqpktoytM640BsCYGyFW3+lA0kXFiAOD7drVqGERpMHKmrBHGmmxf4yAhjTX5\nUDxz+wQOHltCQpwXmWU3OR6IOD3Ld7iQbtdeoRXOepDqiYwr7IVRZlxLzIy6KE4Q1Dai+jHdoKqy\nJh/e6oLFdp+ZD2TyXkyA6fFWdj9RNcfUa74uRUZtN2Xv95uqb549fksk4aT3q8UN2k+3aUKCgTqX\nUcckbU03jWmT/2Oy0OnjmWdN50rrCtfDQqeP7TNjlRzDajFd8kmSZLHHq9rPaZaksNEXmaNmYGPt\njW98I1772tcCAMbHRX2kiy66CB/96EfrGVlDydvG1BfwHcf5ze1Z3EBRpLpB6RtBTuSyJQ9VxkLG\niQEiCy1OEu1BOUzCKEbgO7myRjw45XE7Dv3glRNly/fgWdy6ubIWpHGH9Hik2iMai9VDFj8nM1Mt\nypqvKGsb3g2ajj+7LqMkN9bS97QFi6VkySDGmiii7Cj3ERW3mZ/zumruRUqmZ66WG+NQao7Zkn3E\n4ipX1vqWBVg7cLPF3EpzVJ3P0MQoQ2TOQaSyZkmWmhwPsnId9PWQK1q2otrDwDSy1lpm5HSoqajS\nqUhx3KysSjIYHx/H+Pg4kiRBHMfYtm0bduzYUdfYGokaBFxnB4NcWaMnXC0WxNImSj6E2iUZVLLF\nk/gseiW+/+gi/v3+H2eN1utCxKO5WTVz6sEZyZpjFpetmi1ne+CpxppNweh0Q9zxwP50XPVNmj0z\nM5VYTUuXt7SfN3pGaJzFrOXK2ncfF8H98nbSsp0tiyLzgWxeD0mSiMxqRzHWiPs1HCDJY72oMWt2\nZU1xlVqOWSyupLJmvwcC34NfYqCqRl6dCo658DSVbkpxNI87jhMsLYeYHAvybYj7Wk0QWRhhA3Vz\nwTW/VDQcTy318I0HD2Txs/R+Ti9lTV1clalsYRTj6989gH0HN18ryzIGVtYOHTqED37wg7jnnntw\n6tQp7b3vf//7lQ+sqfTVuJKa6pKpRXE9i2tAtl0CYG0TFWVuUPsKu9/PJ/8sSy1K0FL8bJ/f8xge\nfGIOJ+a7+NVf+Mn1HFopYRTDd+3GWhwnSAD4rmivFUZJwcUg+3wGvl35XOz00W6JCvC2wsZ37j2Y\nFVqt01jrptl94+002N7i1vJdtzSmZyNhlu7Yd3A++57iRHynUZRkxobNDbrUNWLWzId8krsVpeFX\nFi4A1FcfK9Ri1iy1E5XWb2rmr6te32mCgdiODlvIi+LKrFPimlIMiroKuCaJuF/lIoOMWZPGmuNY\nlePlXoQEotVUFrdJfI8P7zue/T7KDgLmPfzthw7hVS96hvbarXf+EP96z9NwHQc/94KzBtrPRkeN\n5VvuRRhv0+bHTXc8gX/+1lMAgE9/4LKhjG0jMLCydvXVVyMIAnzmM5/BxMQEbrrpJlx22WWbrnRH\npBhrTk0V/3VlzWKsRWosCL1SkYalNMaoB1VXU9boyV2u/k4RK8QqEWn++cOs0LUh1pUHsY2+j7zC\nu2eNR1vo9DGVuh0d10FMPMxkAPauHRO1xvEtdvqYHPe1eDQTmd2XKzK1DWcoyIxAqazNpZ0Fnn/e\nNgDCQNG6eFi+R2lE//JLnqn9LVHLZZTFOqnXe12uJ01BsnQlkeMoc3mHUZwtqjxLko2cPzzPrqx1\nFaO0LmVNdXECMmbNso1j74Os9g+1JU2EUYwoTnDZxbsB0Ik6w0Kezz9//8sAAAuE4SjVtpOLdm9F\nrqxt8Bs+RV0UlLmp9x9dzH7nOLecgY21++67D3/wB3+ACy64AI7j4HnPex6uvfZafPrTn65zfI0j\nrzLuWGPF1osZXC1fU9FKd7h0I3fNZWtRJ/p9PWAZKBbrlAZqnRlWInEjTmN6xGs25cFT436M8y8n\nylZgVz4XOv3M7WiLWVtY7mNyzMdYy69VWZNjKYtH66dxhbnBsbFX3FnMWppIcXJBPLBkBfs4TrQ6\ngrZrN2tBJJu9E/eI+H/RqguwxKxpbtB6DBfKcCxmeOf3q9WgS0vXyO3MazOWapbrKG7Q0ShravKA\n/GlT1rR7usRYs20j56Yztk0AGG1wfq8vwhZ8z8XUeEA2dZfHVKYAnm4xa6pblzJgJepXa2sjtxkZ\n2FhzXRe+L2TLmZkZHDt2DBMTEzh0qJrifxuFLL3eT5W1Gj4jjpNssrY1Y5ZB54BYldL1ufSVOq3a\n6AHLQDFwX/5dZ9Bu3qPUybs2WLZRlTVz0lYTDFyLi3Ox08dUWhDXVltuMTWibP0XqyILnC4pYNpN\ng8ptD6qNRh6zJoysE2mJlJnMWMsTAwB78enMWAvo/qFabbNSZU11g9Yds+Zau5Kord9sSqtc0Mjt\nTNVXVfBWUmvzz61XWctja4vzlPzbcZVkKlssX4lCKo21rVMtOBhtb85eGGULYGGsFedNOZeWLYBP\nt/6i6jVXdtzqNTLKRJGmMbCx9sIXvhBf/epXAQA///M/j9/6rd/Cu9/9brzgBS+obXBNRDWAgHoy\nqVZbFNfWgD0fq2N94JkBywCdeQoA3993vDZVJ1MVlNWzGZeiGZ8DKGtlbtBJVVmzJCEIY80ls06r\nYqETYmqsPMtNlmIoMzg2EnL8cpFwIlXWsi4DcVLoj1sWaza2krKmxKxR5+4JpeNFXUHd2lis93R+\nv5a5QX21zpqprKkGaknpDlV5qiupQo5FGqcukdWrle6wHnMey2dLzpAP9cnxAEHgWo3uJw+c0txs\nddDrx9m1PTUekAaHVNseeeoE/v2+H5Nz0A+eFvXVwijBQz88VuOIh4N6b91x/37rdur3X6bAbTYG\nNtauv/56vOhFLwIAXHnllXjJS16C888/H3/0R39U2+CaiOoGdR0gqUFbE25QPUOTzAb1ZOkOujdo\nXrfJXhNOLwUg9mdOdKqB9tjTJ1EHWbyO68JJr0pzuJorKXOV6tvIscv4FjLTsxdhIg1utSmOna4I\ngK27UHCnG2JiLA+cpsurRJnxCWz8mLW872feG7QdeFliSZwkCJVsZ891SINZfi/y+jVjLVXDpcyF\nfNPXnsx+r19Zs2eDdvsxHOT3q/p/2X7SmnuAmINMZU1ur2fAEmptTzXWhhWzRodzAOWqmVYs2GLQ\nSeNnaixAy/e0mDyVD/3lPfh//9u31nxMg9AP87jC8bZPFsmVLaQOHlvCX93+CH50aKGwzeP787n2\nhs/dX9Noh4eqrN3zyJGsVZyJ+t12RpjV2zQGNtbGxsbwmc98Bq961avw4he/GDfffDM8z8v6hW4W\nMqPCd4GaEgz0GlNlCQZ5b1Cbm0g2SC4vBaC7QanP2j07CYBOQ6+CvO3VAAkGrtKahuiZKrZxrWU5\nZPkPIC17YtnG90R/xX5Yn3UkasspD2eqfVCasXu6ZINmRpTnZAaaqiZFRsyaVVmTsW82N2j6kNdc\ngiski9SlWqrXrk05Xuz0heGuqkzGcMJYcYP6xYWE6np0HHus6kKnj1dccg7G215tsah5pqf42yVi\na5MswcCeJauGSNgMOhn7NTUeoBW4I00wEAvpPN6SnF+M1+aJskhRlGSxtacDhY4jlntNndM3uheh\nSgY21q655hrcdddduPLKK/GP//iPuPLKK3H33XfjmmuuqXF4zSMzKrIEg+o/I47z1aiTTmKmIjCo\nG1StVUVlPeoJBpaehXGcBX/XJUurrg5ZqcBW4V1zg9rKNaRxSgm1H9UYtsSsyWxE33NrDejPkiqs\niSQiy61VEly90VDVn5afG2TqOYiM74g01syiz6b7Xj7klbInlHKpUte5VRcRtutbTXyxJtmEqhvU\nKRTFzc6tkk1O9SBd7kWYGg8wORbUVuZCDl3PBrXFrNmL4maxfAMkGEyNC2WNSjAYVmah3reZjie2\njV+lH8bYsWU0XRjqwFSt5TkyUc9XnSEoG42B66x95StfwZe//GXMzMwAAJ797GfjhS98IV71qlfV\nNrgmIi8ez3PhgHY/rhdRWyn/23WLPUjVAGyR6EC7Qf0VMur0BANbNmiSGWt1BXxqZQsyZU3fhgoY\ntytr+sPf9fITWoiHstSoEz1I600wCOOkNI4pS5hQSnds9NWmalC3Ag+Ly6H4TpUHcRwnWrFmW8kN\nx8ndqVRLNvn/ZcqlykrG3FpR1USbaqYZaxa3bajUn/NdV9RiU3aUmMaa6xaOWd7DUxOBNQC+CtQa\navJn4X4dwA2alUty8+Qj83ta6PThey5agWhXRwXnDyuzsDi/FLexJUiodPsRztg2nhWHXWvbqqZg\nutttJZHUS36jL0yrZGBlbefOneh0Otpr3W4Xs7OzlQ+qyciJI/DFCrmOSymK9XZP1MNKzQYVblB6\nrHIVTsVvxbHoc2omGJirmShOMN720fLd2iZ2aSB6ajZoWYKBLRhZdTeVuZBXqI4vYwI9IuOuSsJQ\nd4PaVIWybTYaasZiK7s+ldIkSYIojo06a0TnAdmjVy4yiOsbSOO3PNr4sY2tasI4gYNyd9+iUVKG\n2iaM8/te3tvqYkI1foC0vEeJCmULgK+CzMVZlg2quErtx6wo6i7tQl5IM7wdRywAKGXtiQOntO3r\nYqV+qMDgytp0mnQD0F6c/UcX8b0nN0byQT/Mr12ATnyZX+rhSeV7oubmJw+cwhe++jjufvhwPQNt\nKKXK2je/+c3s9ze84Q1429vehre+9a0488wzcfDgQXz2s5/FG97whtoH2SRk5lTgu3BQV521WJOI\nfcKg0IriurTC1+1HipvJLao2SuYkYG83FaYN1sfbPjrdmiq8E25Q84jUjDqbK0nr3Uo8FJMkEb36\nVoiHkhMulXFXFbIdUlkAvPyOAv/06Q2qGmvSBa+rX3HawSC9Li2u6jCM9WD8gvqcLwAGiVmbGg9W\nVN7Witk+Cyheu51eiLPbk9o2ZIKBLw2xvF2XxFSzPNcpuIelwjTW8jHe9nHk5PI6j46moKxRC0Y1\nZs3yPYZKSRO5vXlelrthVhE/8F2yRtl3H8vbUf35Ld/Db/9vF63twFYgMsNPKOXeeI1S/VSvB4B0\nAeNp28hkiY1Q6V921pgc93FyoUfOq5+8aa/2N5XcdeudP8R9PziK8baPn33eGbWNt2mUGmtXXnll\n4bUbb7xR+/vzn/883vGOd1Q7qgYjV0CTY4FQgGqOWQNoF6baG1TERVBjDbMG4dQ+1Gr/ABTjRt+P\ndLnaJp4qoN2gtDtEDdI2j1uvM1V8KBZiemxJCLHqBq3rAZ4bqFaDI/07UM7LRneDRorKlC0UUsMY\nEMesqRMW9TOMk6wsjdivrYNBuSo5PRHgkueegb1PzNWXYGAkTACWQtdZbTmZGZu/H6fGvTRi5UJM\ndflFmZol/hYxl3bXY53ZzuZYyJg15X7MwjBM16+SYCAFM7pIeH5eqDg81aidq8lAzcaSuX7pxZWt\nR6qKTCz6X//js/AP//Y4ojjBRk43kPf0u//nC3HtX99Lzquym4n6PyYyUaHTDTUP0+lOqbG2Z8+e\nYY1jw6C6EGxG0nox+wFSbiC1N6hrKd2xuKwXfy0WkE2VNT/fD0BVTc8biVOZTVWgZoNmqtkA7hBb\nFXibGzQPOs8NATNIW+7Hc91alTW1CGpZXS0gDa4+XZQ1pZVUyy8aa2GaVKF28aBKasiJWt4HpXXW\nSmLW4vThaqu5VwVqVxJb8oBm0BHbZIuV1KjJe+jmqoxq/ABp2ROiwb38jDpjMumYNWOb9G9X/f5N\nZS1SlTXxO90rOb2mAo/usTukxuhRnOQFy23KmkVhzN9PE4uCvEDyRr/vpbpsq+cJ2OOUVVTVdHE5\nzOKpT3c2h0laIYudvlYTqg5pTVUVADpIWHWDOkTgLlAs/mpe+Fk8VKpuUOUwRGPtvPJ6fcoaEZdi\n3rhanTW7OiG3oWJg1H0Ag7hB63uYqSUJMjedLQO2JKlio6EaLrKZuzCMpRs+0dyGHqEOAbK8St5W\nqdAbVF4LSs0xqqm5TPKwKXhVYMYxAbQLX70uAf3aDRWVFcizYFVlTW3fBNDnTlOz/PpiMlVDTP4s\na+Rue4iTsaqE6i6vA5FgUDTMuspDvs5baNCYNXWONzdRO7EMUgx7I/TQFKqwqqCuPGZaWRusE8Lp\nBhtrq0QGsgKpolXDZ6gPM4BOMNAnBDH5mDesFrDsUcpaWvogc4Omn6+6DQfoL1oFkTIhO7aH2SCT\n9gpV4OX7qvJAHZP8DnyvmHFXFYMU+8xa7XguqRSqHDy2hDv3Hqh8nFWjqmZq6Q6pkIVhPNADrx8l\nKyhrSsyaZ1cn5MLHVkS5CuiYNXOb8vqKfeVaAHJlTVUa1LIoQBrvarakSv90HAe+W2NMZkFZs7v/\nVDeoqXSHxhyk/p9Eu6YCj4xZU5W1uoofA+J60g1U/X3Zv9X388ev6TnJQlSCwRT1Oo+nKiJlTgWK\nVQfI/7F035DxiQ8+Pld4HxDn6p+/tS8rPnw6wMbaKun2Ioy1Uu+xU1O7KcNYa/luIe3Z7GAAFDU+\nWYUfoON+1ExDQHGDKvdH3pfPtaahV4EsPOt7TnZRlqb5r6CsuZZt1AQEIDW4SWVNl+wpV+l6UTNg\nV6reHigFZG0tka7+9Lfx3279fuNX2dLtCOTXnuc5mWIUxvFgjdzT7DKbi5NqU1SW+esSwfhVQSpr\nhHFZloSQqcKKuw/QrwezH6fnOQU1UY9Zqy8mM6uhlh6vQylrSoJBZnTbEgyUUACqc4M8ZluCgfqa\nGrhfNWo8cVmc3rN3b8G26Xb6mr4P6drWru+SyXcjNH2X93RZGzQTWyzfT+6aBqBn+Krc++gR/MO/\nPY6b7nhyHSNuFmysrZIEAFI7ytbmab3EiS6RT47p6fVJkmg11OSm6ljyrMc8kL6sjQtAu0FNA6mu\nmLVcBbG7QcNYNW5AbiMneq3djurWjY2HmUs3e89i1jJDoAZjLc7dWisZHJ7nZiqprdSCNL6bnoAg\nCw4DuVHiu2ovy8ESDGS2nK0lmxqTaTu/0s0vr5e66qzJzwDs2aCxEiRP9YpVq/0DSoIBpayl94fn\nFUMo1DZQMsGgrnlMGwsxB6nbZMZ6ZKpM8nv0sm3IIuHpe63AQ68fFY6pF0Z4/nnb8NILz6q15tpK\n8cTyO73gmdvwR+96KXbMjFndw+rCtNRYG1I83nqQcYX5osx+PFum8j7BJv0wwuy2CTzjzClrHKJU\n1Ooq+DwKhmKsHT9+HG9/+9vx6le/Gr/yK7+Cd7/73Th2TNSGuf/++/H6178er371q3HFFVdgbo6W\nNZtCkiTSVktLd1T/GXGsJxhMjQdYUHqkZatnQ1lT5y+z3hIVm2Uaa2VBzX5a/6y2mDVFQbK7QXPD\nZaW4H8fSGDoc4LxIN4WXxvSI8dXrBrU9wNVtpLG2UpwGVRC0SSSau4/KBtXdoJQLH5Ct0tSYHv24\n85hMe9xPXqxWBHLX5QaNkyRLnMkWV6ayFhWNWHW8uVIl/qYSDKQqrhrB5kJDdU/6ngjlqMPAp+cg\nfRu68bxpdIueqb5n30aN4W35InSh4ElIsyvrrC0H5LFZAOAQteVMVzVVf05LjvFODzdolshjMcpV\nsjhU4pi7aYtE0anCso9mr1fXxFCMNcdx8La3vQ233347brnlFpx77rm44YYbEMcx3v/+9+Oqq67C\n7bffjksvvRQ33HDDMIa0TqSRVM81YcasTRpVxs0g+awuGaWIeXZXUmi4VSgDSKtb5tZYNJSqs2aq\nZjIg37W7DeM4KbiSIsL4LFNt1PNra8FVBXk8mr2DgRqv43su2i1R8b+MvqWJdVPQMj2z78rVYpa0\nBANLnbVeWkfQVkNNqg1tVVmzlPeQ11RtxpqyAMsXI/r7CfQsZfl6tk36q9wPlWCQKdRqnTWLmuU4\nSj/gGhYjmeGoZIPaVCYZPwfQyloQuKXbqMY95R4G0od8IBY9vTCuTY3SjG6HMFANVzXlKs2vy8Hc\noGaYTBMJs9ASuxtUVX0B2qPR74vvsRW4ZPFjAOgMqVvFMBmKsbZ161a8+MUvzv6+6KKLsH//fuzd\nuxftdhuXXnopAODyyy/HbbfdNowhrZkkyY2jutygZjaoXAnKz8pXrHrdJnUopkFHBU/napZR/0nZ\nTFWiqNT7qtAC6UtKiJjbUBXey2KDqJi1w8c76HRDZZuiEVWPsZYbqCuW7kjHOzVWbA80v9TD9X/7\nnezvbsNX2arhol6fWp01s5E7cZ/1w1ioZpbJv5fFZHpWV6naYL3OBJo40ZNa1M/WxlESs2a2kpLK\n2j2P5JXcC6U7iA4cmdHn6uVSqkbt+yl/FmPW1LHYv0dZCzLw6W3UODHpHn4ybdMk6YdRWpR1MIV6\nrahjoWtk6t8RFQu8UuwtoI//+/uOV3cANaGWQwJob4U0oB3I501xPoyTBIHvoeV76Fu8CF97YD+A\njV/uRGXg3qBVEccx/u7v/g6XXXYZDhw4gLPPPjt7b/v27YjjGCdOnMDWrVsH3ueOHVN1DJWk1fIR\n+B5mZ6cxPhbAc13Mzk5Xtn95cU1Pj2X73bl9AlGcYOs2Ud1c/tyyRWyzbcuE+J8t49gyJQJWTy32\nAAAzM2Kbqak2jp7qamOd+NFJAMDs7BRmZ6cRjIk4gcmJVrbdcnovbNs6gXbbh+dXe7ySsXHx2Wed\nOZOtpiYm2tlnzc5OI0gTO84+awaL6U06s2VcG0+r5cP3xPez/fAiAGDLlolsm/lenB3P7Ow0ztwp\nzuXxpRDPOGcbAGBhSZy7LTPj2D49ln7OROXHfehUFwCwY9skzjxjBq4DjI0F2jFPTIpJ+IzZaczu\nnMT0ZAtx+p7kr/72Xjz81Ins7ynl2mkiQctHKxDf0dSkuF4nJgKceYYY8/hEC3GSYHpKfP+Tk20k\nCQrHFMYJZqba4ty5DtrGuZPXy+6zt6SLDaDVDrT9nFwQ38HMzDjG2j6WlsNazl2r5SFIj1m6rMaV\n+0wuFmbS7+5U2ilkYjK/B7rpc2fLjLjmt20X1+58qrTOzk7jR8dES8Dt2yfFuZto4cRiTzumqafF\nfb9zxxS2nhBFSLdsmcC2mWqbhh88Kc7t9m3i3pmaaCGB/j2OjYl6lWeeMZMZKO2xlraN67kYa/uY\nnZ0mt5mdnQYcBxPp+fyZ55+Fv7r9ETw9t4SX/ewzs/1EcYKZqTGctVP835hybqvFwVS676mpNpIk\n0T7HSwu/bknn5laQXxuS4x3xnW7fNpEd87Qy183OTuMzf31Ptv2hE8uNvueB9Hv0XZx15oxwafte\nYcxbptpYXA7xW2++GNf8+TfRNu5X+VzbuW0CR091ceRkhzzuY/PiHPfjZE3npYnncujG2oc+9CFM\nTEzgLW95C7785S9Xss+5uYWhWdDdbh9hFOHIkXl0u330Q/F7VcgV43Knl+13OV1BHTx0Cs84ZxsO\nHxYZMMtL6TaRmNj3PX0cu3aICVw+hOQ2gevg5EJXG+ux40sAgFMnOzji5Cu1k/PL2XZHjy4AAJYW\nu4iiGN1uWOnxSk6eFA+ZE8cXs3Mwn45jdnYaR47M4+DRBXiug4VTHcyfEtsfP7aIIzPtbD8Liz24\nDnDkyDzm0xv26LEFbBkTK/Ojc+J4FhbEvi88bxtu/ipweG4BR46Ih9Wp1FjrLPWwmK5+Dx+Zx1jF\nOnR2/k91cOTIPFzXwakF/ZiPnxDHeerkEvwkBpIES52+9h0spBOY5NDheUwojeubxlKnhyRJcOTI\nPHqpkRL2I5w8Ic7H3PElJAnQXe6n29D32XI3RBzFOHJkHp7rYD69vrNzd7ID13Fw/Jgw2gPfw4mT\nHW0/J9L7pNPpIYpiLPfqub47nX42VjlXLczn96MMhO6k3+3iQnrtzi3m9+KcOA557QLATz9rB04u\niO//yJF5HD8utpHXVBLH6Czr18sJea+dWEInvdYPHZ5H2K1WaTomx5Ke8ziK0DHmj4XFLlzHyV5z\nHQen5vXvaH6hC9/VtzmZHp/8rnu9CGFfXCNb2qIO5slTy9p++v0YvV6I5U5+zHXcJ/0wQi89zk6n\njzhO9Hk3NdaW0rk5jpPse5eo85T0kszNLWJL28uOeX6xm20/d6JTy3VbJd1uiMB3cfToAibGAhxW\nrm3JQqePX3zh2Thrpg1HuaclB4+JOQJRTF5PErkgOm5cA4Mgz29duK6zJoFpqMbaddddh3379uHG\nG2+E67rYtWsX9u/fn71/7NgxuK67KlVt2CQQEi2Qx55UiRl8ChRdIqYrjwo8N7eZHPOxuNzXuiOo\nLZ7EZ4r/VQOftQxMS72rKugriQxRRLtBF9Miv1rygDEcNd6vrCiub7iS1PgVPWaNDmiuAtNlRcVM\nqV0O5E/TBdT2dSuy6ZlhakFQyg3aTeNN1NZLlBtUtONRYjKJwHS1RIOIcaFj1uqus6Zmg1J9bc2w\nBeq6NOuWye21ItZx/jogMijNhBO1eXowBDeou8JY1HnU9xyEIfE9Kte4mCOKcXhmuSPzPpDhJbII\neF33iSyyDKS15ZAmphnuTL2zgy1u1gUQa/8nkXM7ACx09AVbE4niGGOeWDRTSR5Jkmi1Qame2Fm7\nx/Egzfol4t7iJJuvT6eiuUMr3fHRj34Ue/fuxSc+8Qm0WsLl9YIXvADLy8u45x4h537uc5/Da17z\nmmENad2IbNBqJ3fz2m3j2wAAIABJREFUAQ4oMS7pBRgaQfJUDIaZ9Tg1HiBJoMdmKcHrAB34rE4a\nrlNf9fwwyvtF5g8zfZvFTpjdyPZsUDVmLd2PFrOmnzsZjNwPifihmhMMzIeZ5xJ9HBUjVv40Dccg\n0Js7N73mUqQ8oL1M2cgNYxksXVZnLUkSrdG12MbMBo20h3zLdwup/up3XWuCQZLksVvpNU4n8hhB\n8sp3aV4vQDE43VzstYiaY1r/XLkYqeGakV+HHEsQuIXkFzM+l1qMyASDsm3UDgYAXRg3TBusyyLg\ndWVQqpmpZL08M8GAygZVtpHJIub1rR7vRjBKIqXc1NSYXxjzci9CFCd5IfcSY21qPEiv7aLBrb5W\nZ9bvsBmKsvaDH/wAn/rUp3Deeefh8ssvBwCcc845+MQnPoHrr78eV199NbrdLnbv3o2PfOQjwxjS\n+lATDCreNTUhm8HR5sROGWum0afW6JLN3U3VJk9UKE7+siNAHcVhgXSy9V1NNTMN4YVOH1NjeZFf\ndXzqeD0zYYJ6mBnByNpDUdlGGhB11FkzV9hkEkh2/nNlbamrZ4O2Cspas401SlkDcqVAjr9McQwj\nUR9NBp5TbZW6/TgzegDxADeTLx778clsHPUmGCRQDrWQJWhmcebXpaKspUNTBX0zaN/MNBR9MvUH\nmlqItizYe72Yc1Db9xBGiaZ+C0Us/x/XdfCv9z6N17z4GdiextDJhuYSqgWc2sgdEMqkaowlSZKW\n1HDJzg9VoveBlXMQIIWwLANWaXBPlXEBxPeYJHSCgfqMOHJiGXc/fBg/+7wzqj2YComUup+T4wGO\npCEeklw1Uwq5G9/zYmas+ZlSq6qWQP69SvXOrK6wURmKsfac5zwHjzzyCPnexRdfjFtuuWUYw6iE\nJMlstVo6GJi1iQAUVlZqWjcgMgQBoTxl+zHUN9nJoNNV3H2xrtpkSpRyUKqKR6WhV0UUE42uE/Ph\nG2F6opVukxp0xCTnZ4ZAsd+m2TtRPgRUl4gsmNkOvLwmUA0PcbNoKGWUyO8xq5tV0lj+1375efjv\nX3q48S1W1KK4auNy13HQ8t0sfis3OERGo2zcDlDuYeoBHmuuooBQ1n58RMRV/cSuGXz3ibmCelEV\nSazXThQt5IrFbDNlTao/at9Poyiu+J1WjtXFSNFAQrZN3qexRuVYUdYAYYDKLjBJrB/Ps3dvwf2P\nHcUPnj6JFz9fGGv9KMbEWP6oojJcVdcjIF2uRGcHz8nU2DrcoGYxclJZM+Ygh8wGzcNPkoRup+YZ\nBsjXv3ug2caaUtJkou1rXh4gV9TltUF5GuQ27ZafGd1hlGRZwkD+vW6fbmOh08ficj97bmxkuIPB\nKkmQF7es1Q1qTOwAoaxlhpgo+qlWazYnhFZQXKmr1fEBFGIq1P3I8hL19U7MWxBR7lg5LrO2HFVM\n0iwsWqqsBcVV9qIitdsq31eB6SbyXKdgFJrftUfE6/T6EWYmAlxyvpiom+4S0Yvi6u7QyfEgy/iS\nr1GdG6RxEagxa8S5W8k1FsUJ2i0PZ26fqDVmLU7yEhYA0p6c9kWENKS6yv1KhUiYZU3Mori5O5XY\nj6OXS6kac7zZwkgNOTBizf73y54t/teI51MNk8BzC8alqZ6Y8YmqUkWNoypMzwhVdqOgqFOdHZTr\nwVZ2Jpu3ATxr90xtC42qUGsnBsS9WOguQxTD1lul0SEqciEnW3k1fT4cFDbWVksCQEkwqFpZo2PW\n9AlVDfqX45g0YgBsK3X1BgmjWGtnQtZZUwq3UoGwVaEGCNvqrGnbEGMF9BgYqphkoXdievzqKlt2\ni9CMtRomQrN9EGUsyIQQacAGnltwRffCGIHvYbztwXWcxk9OUZy7BE2FbWo8wElprBkufKowtFSG\nPJdoq2Q+wIkYF9Wgq9MNmiSJNtn6vqu5eMwEA0Bkr6rKGuUGLbpT0/0YvVd7lELn1tv7NlfWxN8t\nImnC7NZCFwPWjTXPcwoxdqZBZyYYqIvXOpU1M8GDUtYKRXFdIsFAeQ7Y6qx5xjOiDoO7SrTCxUQs\npbmQphavZvcN9f8k8lqXpWhUj9NGho21VZJALYpbfQcDc3Wh/i7f0zOFBIUuB8Z+qAlKBtxKyCKy\naoIBMalUhfpgzVQzYlWVPeQtE5je4L64n7xiOtKfThr4XFTWJjVjrT43qKNO2tQxK3epUNYMY60v\nAu0dx8HkuN/4oFr1uzYnaMpYkzGZaucGs1iwcCtS14vpGis+IPIFQI3tpgzD0ff0+E8q/MGs0E67\nQemYtaJyTGWVKqr9EBJoqGQeM4uT6oMZRbFx7nRV0nQ9ys+ieqZ6Xq6s1ZFgUMz01F9Xx6KGfdhi\n1nzX3t1Evp6kv9fV17Yq9C4TbqF/KxXDa16XUXZNwaqsyWt9Oytrmxy1N6jjVB60FhkTHFBUiMwM\nQaCYCm3GrAXEBBVGSZa6LzEn/9vvfir7LDM+pkpixRBzHIfsuxonKCprhPtAzSKU/6fuQ/1/oLgK\nVzOOpKthGL0TyazHGKUPKkCv8D41Xuxw0DTUQGNzcaK5QY2YTC3bOdLVZWpiJ11jZjZopFSbt7S1\nqoI40Uv9+IYSYqrlgLwuicQXTYmi46EKrkdNWRM/HUdpqj2EdlMtSuWzKWvGMakLU99wg5pligCh\nKOrleNI5081Ld9TRoilv5WX3AOTeE/E31dkhNLKUgfKYNap0TdNQu5K0fK/Qv7U4HxZj1tR7wJbJ\n3GM3KANoXlA4KLrh1guprBlBwGY8GgC0W542+ZgXfpuYKKNID8AGipP/4z8+le2HaotSFebKWGTa\nmjdqvsK2taYJ42JwL9nWp8QA6vYjUY9J7SlZZ0yP4hKklDVtrG4xwSCMYvhpgO1Yy8Ny4+usCRUL\n0AOpAeCs7RPZdvK4x9rC4Oj2ivGWfmaYExO7cU2NtfxCz0DVEKD6xFZFYihIvq9/j6ZhAwDjLT0I\n22zkLsbsksqx/Kzx9Nx1ekQ7NXewptprxVTWxrIkJ/2YNOWYWISJDG9dlVTvR0qVDDzDGFbic12n\nvszfQvP6FbwVchtbg3tXUdaoEAkAuPL/uIQsXdM0TDcoYNQRNEsZETFreqY+nfwl97llSiQVND3h\nalDYWFslIhtUd7NVCVUUtxizlkv6EmvsSnpBB0SgcT+KNXVO7ich7nk/neRMub4qzOKYjgNiAlNX\n6XSQsBq7Qk1y1EPRLCwapmVE1H3UMRGabi1boLE6Vt8v1llT1UTfa37sipooYsZq/cxzdmbbZcHI\nhEGRuUF9ezaoqayZPXazsXj2h0NVxEmizRe+q3+PpkscSEMblKQhOWzdbUi7+QsuZLWsj7Kfsqba\n68Wcy6jYQ5uyZiou5uJKdSHbQkf0feiLtLqUqEKCgXRVqvOLTI5R+hPbYtY8z97IPYoSjLd9POvs\nLWTpmqahhqhQdQQLhi5hUKvXLjUvALn3SF5vddXTGzZsrK2FTFmrvpG7LeNLfY8KRi5OTvo2lAsi\nipJsdSKhJHm5nzqVhzjRj8d1i+dWjW+hYnEAGEVxi5McWVjUULTCUA86B+px/5oPXzLuylRkqIKg\nysPMfJA1EVEUV/yeF2+W7pH8esyzwkqMNdUNuoIqOTUeIIqTrDSL/PwyN3RVqAsNQJaf0OuAAfqE\nLFzaRWWtGLOW/88gBpIapF2nG9S817KxLBvGGhHyoStRccFYi4h4v7IsWTPhyvOcWsqVJMZYqLjZ\nXOXLr7tiyZ58/qZcw4AeTlBnpn5VqN811aHDdPP7VMxanBdP9ywLDelhkvVE63B3jwI21laJakDU\nELJGF8U1Y9YIV57r6MaNadDRpTvigrFmy/iU7oO6EgwiY9I2K7wD+iqcqkMltxmoKK6hrKkP6TCO\nlcD1GuusxdKtpU645eqQHGtsqEOawddwY011T5qqiNqNIVcL5aRcfOBlblBCWVDjIAHRcg0ws0p1\n476+dlO6ghQYSmCe6akra4tEoWvHsRslZqbhZBbvR7tT63SDqokMYizi/JsqH5VgUHCDajFrhio5\niLI2gNu8CiLjmD3C0AqNGoHUAlldbJcpa9kibaPFrJUoa2o2qJk0oYYT2Ao6SyWtHXiF4sgbGTbW\n1oCj/GLGVa0X0pjIUpSNmDXF0CquJE3ZX8jpZoKBWodKfG7uKjQNU6otSlWYRUOpsihmsUnfc8j0\nb/VmB4AvfPUJbR/y/yXmQzpU3MO1xqyZMRoWN6ipKojxKN+jsk1AJCA0DSrzV660KWVNPmC1oHIz\nG5R4WAklNt+fzQ2nJqQkqEdFNd2gZmHXRDGg8vHm/XzFNuJ1reK/EZpgPvAmxnw4jnHM6Vgcp+be\nt9IAzRaMHlq+O5Ab1FyMqCEfpippU9Yo9a1uJcpWZ81cDAJ6MXIqq12ON3MNk0WfVVW4uUZJHIsn\npXzekMkmhZIm5eV4fMt5kfsMfLeQpLORGWoj99MFOSW4NdRZk4YSVbojzwYtriRNF6VazFZiBjWH\nZIJBbvRl6oXnYKLt1xqzZiprlIpnTuyBUaUcgNbBQO6vrA2X/F2bTBX3cJ0xa4M8QMzyE9Ko6YcJ\ngvTuVc+L+SBrIqoB+ssveSZ6/Rj/8Wd2A4DWViiblNPkidBYaKjvUQ8r0w3ablF9YJWm28p9VnV7\nGlNB8j1XD7RPf6qqWTtIM+aiGK7vkW5Qx1hAmW4413HShtequ0lxNRHGf1VQ2avtlld4QJd5EeTv\n+oLFMYy11LApURzNbNvaYtaMY5bzK5VgoLb5s8esOfCiotoo/84/p1iTrEmYbui8+wCR7VwSlqDO\nh75PK2vSe9QK3LSEC7tBNyVJAm35W5cbVFuFW4riqoaYuTqTDzZtReqYRkmMwEgwcBQjSV7kb3rZ\ns0U5DbfebFCzjY6Z6CCaYed/m1XKAT1mzSMeuJTLpDCxK+5hW4mQKjDdRNTkZD7MbBXp5fdMBdo3\nDdUYGm/7+E+veE52XC2lYXdmRKWlXHQ3qCzFsIIbdABDQHUhy/+rmkKiiOHKMwski/HqZWNINyiR\nWOSY+zGUY/WakupOvUVx7Qsj87yYMV6yp2dZ9nZsGAJAWhqFMJBUFbWepCFk+xfjKCqXfUMVprJB\n5XhF5mp6HRAGXR7yUV+NwCow26BRxZgLblAihle/dumCzv0wTt3Hog9sXT1ghw0ba6skgaGsVewG\nzd0hq1fW9JVkUVkrBNJHSVFZU4w+uQKWknWdMT2mUeI4DmIQk5NquFgq0ucuNruxptpxZoBvFCXa\nClzut2oyN5GyCl8pSJ5qPK8lGLgbwA1qGOYqZpkSQJwfz6itZdZZszX3Vr9nspRLFOfJJITLqioS\nMxu0kGAgfjrkeFNjLbFcu0mSzRvmfST3UzCQFOXNNISrwnRryd/V82+W7nAcUbHfPOaisTZAgoF2\nTxsJKTVlT5pj8QkFKZu/lcVIwYuQhnw4jmNdMOp9kJsds2aWm6KKMZtzM7l4Ve7pPESlmGCQLf4I\n78tGhY211aJOuk71yloWl6JOyMaK35SUxfbFuCtAN9YKgfRUgoFbVNak2kEFfFaFeIAo4yDOrZlR\n1wq8QoKButoklTXDQAKKD7N+FGdZck6NNZmo+BZzQk5ii7JmtuxRHg5NTzAwDVAVapECAIGvP4yk\ncRGoMWsrKGtkwokRBwnUE5dpGlGmcUmpZuZCgS7dIceMbFvTWDPPjapmOY5T2zVjFn+V4zVLjXiG\n4a7OQZnCpBnxtgQDfWFKZ8fbr5cqUDNt5VgBOpM5UN2ghCFmegjKFnJ1ZjJXgSkwUFnIxaK4RKHr\nJL8WAosbtB/G2aKW8r5sVNhYWyXqZVFHNiiprKW/50Vxi9mglDoE6F0O6NgsY6JUJPl+pqx52Xt1\n1llTJ22HMFwKFel9V2t0DeiTHBV3RK3UzQeIWSx4aPEtroOnDi/gN274d7z3419Frx+JCZmMWTOU\nNeXh0G/wChsQx+0Q342J7sLUS5LkypqS3UckGJTFfgJSRTXiE2sxXPSFhk1Z0wwbw7ikY9aMbQhj\nzTQGhLKZv2+2vqoKmwFqU/nUbQoLUzUb1OyrGhfPi+eKeOIs/tZY4NZ1TxddeUWDwiw7Q8WshUTb\nPHO8pgu/yQkGxdIpRMyamXBFdjCIiXNbTDBQE5a+9+QxraD2RoWNtVWSIJ98aqmzlv6k3CHqhAzo\nBocZT2bGRcjti1mPxdIdSTbB6a6mdkv026sl1sOYtNstr1B52nSfjbd9dJb1bcyaYyaUy8RzzZg1\n3YilYieqIJ+coI2pF8Z47Ecn8OOji2mcnnJeiPpEkdbZofnKmqpcllGsWp8flzRW5aRMxrcUYtaI\nYG9FZaIKslZFgmKiiGpwZ4s06NelOt7c+Mn3Kw9PXkuUUkUq6krWbeC5hZY9VSA/UvsOPKPjQlIM\nV1AXlZQXwdpX1bOfO1PZqV0td0z1p5gco5amoWsn6up+waBT5u/GK2tGEghVjNlUSOl7Oj+3Y2nC\nkJqoI/cpz/vOreMAgGPzy9UdzIhgY221KNeOJexmfbsnlLW8Crb4m3LlmUHEZm9Q+butRIVErflj\n1n6ayqqhV9++w5y0zV6nlCJG9cE0XWwvv+QcTLTzpGczW07+rp6XvmHEihia+rPlTNdgGMWF4wmI\n+kRqbTmqw0GTSJIE/X6sJRLYKAsqz1z0vvqwKs8GpYwxrYMBYcxVhejQkf/d8j2jobn4qScWWdyg\nVKkLaZQQMWvmg74XxmgrWbeB79XiKqKyQdV4NIB2idPKmuoGdYVqVrJ4Xak2ZV1KVFEdkkaJPu+6\nSiyaiKsarL6iSqSVGHI1JbFpFLJBCcXRzAal3MOqWk7VTZSfJbf56Z/cIV5r8Jw4KGysrZIE+aRb\nhxtUzh/qpC2NmNyIIiZBI55MqkOm0RcaE2VRWcsfHOZkOzlO3xxVYE7ahertUlVQtpkkjbW4GMtH\nVDIvK4obGcZaXcHIar0rOQ6VxU5Iun4BU1lT3aAu4iRpbGaYLOirluiwUWjerazC1VpKQLExOlDM\nNKTcoGYHA6CYdVcFaqwNkMfSJCvc0+p4bTUCAd1wMa8js+ZYrx9nzcyzsdTU1NxMJCm4QamECEdV\nxIqLTjNo35ZgAFBxvqlxX1OylBmn5xOuuihKspIzgDj/UZwYiS/FhYY53jAmSgw11CgpZOMSJWOo\ngsKmQa2VK3FdTLT9gnigxfBmRc2b7W0YBDbWVot2L1TvBiUnbenqMFbYpqtUjSejaqgN5AZV9mO6\n6aiiolVhTtqTYwFZH00z6MYCLC2HWkxPkpiBxvpq04xjAqjkDMMNWlvMGv1wlix0+gW3FlX5Wz13\nthYsTcHMMC6jWFtLUT9Doc7lmbREz9Sk/AEOpPGJSiN38/2qKHQwMGIP5SeWKWt5WR/CWEtyo2Sl\nBINeGGnGcqsuZc3I9ASKRodZDDvbRjkewDDcDbUqmxtKDN0wMpU1t9auJNniiXSDxlmmM6D0OVZb\nAcZ6iy1KWQtDtdOKk/1fEzHn78DTv0NqG3IBZjwnpoz+uUAa72d2PwmbacSuBjbW1oC8VFwHFRfu\nKE/hz9ygMT1pF+JSVgg0FgYdYbiYFdMLbtDqjTVz0jZvQkoRmxwPkABYSmMWHn3qRLpNvl+zhpFw\nRxHKg1FnTTP4HAdff/BA9fGJZkFQw3D+m395BI/86AQOzC1mr7VsCQbZJEhnSDWFPMN4AGWtpGp9\n1zA4ZH3Abz10KHvNNFyo0hxmRh1QjzpRcIMaRjcZ/mCoZmZdPqBoYFqNH9XN3481Y7k2ZW0Aw9Hm\ntjVry3nEfVKmrBXdoOJnWceLKjBdeb5bvObM8JPAVo7HKLtUMNbiRHODmp/TJAqZnmSCgfhZVmfN\nvKdJ74pSeolSNjcqbKytkgQoBKtV+RCnlDUZdJwZUaAmZF1BCqNECyIGitk1VLyIqjKZRuHUWH3K\nmjlpT4376Pai7CbLz0v+P23Z7zR90Bw8tgQAuOCZ2/PjcXX1k1TWjIeZeV7O2jEBAFoD8Cow3XQv\nu+hsvOKSc3DRs3cCyCfvU0v5+ZYPebU5cRSpylqzJ6eekRhQRlnVejPuTQYZ//Xtj2SvmS5Bquht\npxtirJ1mOxMP1qqIE6PsjOHOJothG+PN4tpK4vAi4vr2XFdbjPTCSDOWWzX1TzSVYzGWYukOY7ja\n/UgZYuaCZe0JBvUnVWSFWw1DzHSJA0DfDG0ouEH18aqekUypamiZCjNZjUowMOMKx1oeojgplilS\nrpexllfI9IzivP4cu0E3MUmiF8UFqlXXsglZk/Tle7niZSY3yNZX0jCJDKkdkBOlXi6ANvr0z5M3\nx6R0gy7X4AaNi/I2kKt41KSd34i6m2h223i2jVkCJI718ghA8QGSGO6bF19wJoDqjVTTTfSMM6fx\n5leej//ypp/Gr/7Ss8j/oUp3qEYJNQk2CTnxDqSsKSfH7HnaCyMEirImjVfVCIwMY5hyjS33ouxa\nyx7wQ6izJo8/c4MaKrY6nmJRXHuIxEqlMABhMOvKWo1u0BVUbNNVLbbJF5XZIk3ZRsbbSZWWUt/M\nAsimQUcpVVVgegCodl5JYhqfRFNzI2bNp5Q1xTOSLeIaaqwVMj1dF45Du0Hls2/S0stXqxpAtJNS\nt/HYDbqZUay19Gcdypqenm+ssC2uPCCf0PtEpidVybywH8oNqqx0PNepJxvUWDGZN6q5YgUUY63w\nwMv3I13VammDwgPEMYOei5mp6liqIiGUB8nMZIt8PSAUmQRKnIdhwDYNaZysNmbNM0qS9PpxVsYE\nAJZTZU2NwaQy6oD8gSoXAqaxVlcHA0pZk0YmWZbDMDhIN6gxN1Axa67h8uv1I02VDPwaEwwoF6f2\ngI6L43UINVF5v23EeNmShsT+5XkpxqzVkmBgGJfZ4kmbXxKtREvLMD7FeAdIMIiSLOwhW8Q1tFo/\nWdx4hU4UlCfHVKgDolF7pMSsBQ33NKwGNtZWiVDWxMUiH+hVLsTlrspKd4iVmf5/lOxvJg/QmVj6\nflQlygxodhyHjBGoAipwFFCMNVJZ01UkKgC7UKuKyj4zkxCI+Dmg+lg9Kl5HMj1BG2uu42j97gox\nMoYB2zQyZW0AY82sWq/W1uqHkZbR2EldIdL1nyRJweVtGjYLhrFWZ4JBHIOMWStV1sxFGqm664s0\najFidh7phbGmStalrCUruDjFeC2uUqnuE/e9qaxRRcIL7mGztllNMWuFDgbE/WjOu1SCQWwkGLgu\n0YpOSSKjEo+aBFmCxUgKilOPUVmMtHleWoGLfklh9ExZYzfo5iNBPunKS2YQY+1Df3kP/uzmvSvv\nn4jNkp+XTcgJrYgByILpyeSBgruPcIM6+RioOknjLQ93PLC/8pWKmfWYG2uhdSyZi2GAAOzMPWx5\nmBXcoFoiQz0lS6gHq2TLFG2sAWlP1L58UOmTYOPdoDJmrcQNKr97FVG13nTl5fuYnhD/I2svmaow\nkJ8b+XBfTAsqT6YreHk9fe2B/as4osFITDeooZCSnUvM65t0gxZV97K6ZUAxwaBdU/9Es6CzHEth\nwViSEEG5fk23IWXQmQH3hQr6roPDJzr4/J4frOcQC5jqkPz5T3fty7Yx7/uWEXsr91PIBjXbKine\nE6qkT5OgjDUA+PI9P8p+N69deU//87efyrdJ9JhNz3Uxd6qLr97/4/yzEjUsRBrLzfQ0rAY21laL\n8p3n99vKF8KTB07h7ocPr7hdaXq+qg6t4AalA42LDZIpd6pZIkRdBZ57xhSAGoLtjZswk/UjPQDb\nLJQKqMqaeJ0+d/JzKGWNqKquHHO7plUrNRbJz5x/Bv6XX/pJvPySc/ChX3+R9p6qhBTS3Y0HfNMw\nA70prv4/fxb/5U0/rb1GuUzUfbzllc8FAOzeOZm9b36OqZxJpU5eaz+xawZAnl1cJeYCq5ANmr5O\nuUELhotyTPI6VRMMSDeocj2ITENlATbmYakbVh6rZ1fNDBW7ZJ6i5qA8IN+MR3O1fcj9A0oh2nQ8\nY2mh7Nu/nRsLVUB9R7NbxzTj2IxZG0/Hol53Yaxng463vMJ1qXpPpAFbR6JIFVAG9VnbJwvbqO+f\nnd7LZkKKuqiXYQQ3fe3J7DUyG5SVtc1HgiQPWctqHFW4f0tRXAeGG9R0LxhlCcwG4IC+ws6VKn0/\nogSC+D0z6JT4igvOE5mWVbc0KmTumQ8qUllLVaTsgVfcxiwoTLqJCjFrRjFVIki4Cig3kWSs7eN/\n+rnz8J9feT52z05p71HKWsEN2lBljVJKTHZsGcsyYiVU8271+p4Y8zG7dSw33InrxbxHzG0C38Vz\nz91aubs7Sev/qd+1mShS5sIvqEOUa1e5vldS1szre2osQJIU2/asFzLbnAzFKHoJZCIUdV5yZc1I\nMChJJjENVNmqqGqoefVnnjNruPsSMiZWve7MBIPJ8QCLy3pHF7WouTRgu/2Nc99f9GzRXUAtwWLO\n3c97xlbNhWwuRuT/jivfp1qjLlPTG1rKaDWwsbZaKGWtwuvApnjJOlJym5USDCh3n6utWIsrHfl3\nwQ1KGFFV1/Eyx5u5MaL8mAtjMbNBS4K01XNHlu5QVvtmrarctVix8kB8j4MQKK2KrG7Qhq+wy5Q1\nCrPOGmV0qy2pqOxhJ1VWCsaaESu5uFyt0ULFo5kJBnRyDG1cauU9iEUNFSdmJhaZtaqAGhJoCA+A\ncOeZ32NxvIWYNbLsSXoPUHMDEcPradfCmg+rFMoDILpVRNq8qh4zlcBEZcfrxpw4dnm/58Z/M92g\n1H2fqcv9fIFlzgu+UbzYFCHMHsGAnmRDlU7ZqLCxtkoS6I3cAVTqPsgmbeN1NQieVNYKLhPaBREp\n7wOEG9QwCsVr+j6A6itlmxN7XpHb/vA1M32oOCX5q5ZJWzDWYKz2dTdoXVJ6nBSN5UFoK0VMTYPD\nNGCbRrYYWeVxB6axRmYauoVkE2oxUnQr5u/XkUBDHXMxwcCurGmZno5DuvlVg26lQrS2BJo6jrt4\nrxluUFvGaCEOvhb1AAAgAElEQVROL38/dyGn6jLhWi8qa7FhCNRjrVGZqS3fQ5Lkiz0zVtj3XLRb\nntZezxzv1HiAk4u97G+5Lzk3yVCNOlz4VUDeA4aBSV0LVPiDeu7kPKj+n1B03ez/AWC518zzshrY\nWGsY1KQNiJtbukgTMsFA/CzPeqTcoMX9xNnnoDCWumKizJID5mRLuRc8Q/Eiy55kyhqy/ZArecUO\nMw3HutQqSv0cBDUbVBrN8jilAfuxv3+golFWi839vhKe5xjlD4r7CPzyyvdyP/k2SMeiqkx+LQqT\n+Jz8tWKCgXjdIa7vMkPMMQ06S6yqdCsmaamXspqGVUHda2ZhbiquTTeoxWvqHBQYyhrpATCMWLMj\nwPbp9pqPqwxqwUgZJeb8PWW01zPHOzHmI4wS3P+DowCK7bPaqRvwb/7lUZxSjLqmQN33Zm04slVa\nIWNUv6Z2bBkDkCcJAXrMmvz5xW/8EIeOL1V0NKOBjbVVkiT5hCovmnpKd+ivO8qKVNRH0983lTUq\nHkqNzaIeIHI/BWVtCG5QM53dbNNCjWWgBANCcVwxpidOCuqFg+qPOSFk/0FQG8vLccsCyGYGcNOw\nKV4r4XuuXv6AnNjdgnu4sI2jJtAUFyxjgaiaXmXMH2UUmgkGZkFQdezqAozqOCLfk9u6Rj0eOrsy\nf7+ddcWoXzkm4+dK4trK7vtCwVstRKLcDfqLF50NAPipn8i7nVQB2XHBOL9UyaSxlleaDfqzzzsD\nAHD0ZEfsQx5zei4mxwK8+PmiePfx+W5lx1MV1Pdo1oazhzbY7/v//MrzAQC70i4zgH7uXMfBq190\nLgBg7uRyZcczCthYWzXKAzu9sJIKg9as7hvHcIOuNGmv4ILIDBtiP+rELz9bYk6UVSE6C6hGof45\n1APPLFMxaGkDqiiuaAKf5MqDsonjOGnj5+qzQdcSs6Y+8GwJBk2F+h4HwU8VsUS5vgsTuxIPZVWO\n3bzm2CDuySqwJQ84TrE3qKYcE+oQVSMQMF2l+ufrvTbl/6nHXE/ZB/JeIxZGZS3vzLplgKI4GvGs\nq3GDuo6DZ5+zpfKaetR1Z8aTUf1bRVybvYPB1imhBBZiMpXdvPTCswCgUNG/CZALFqMEC5WQ4ntG\njKPx7Btr+ZiZbBnZzvp3/aK0A01Ta9ANSrNn9gaSQC0Sm75WpbJGqEOA7gY1K2ADaqucdJtYz44C\n9Crwtqw8xy2W7nCIibJyZc2YwLLJVk7IxOo5b+WiGKimUkgYsdRKXr5HuTEA4WKrulaPWCWu/v/U\nh5ktwUC817zJaZBsUArT/W6qn4CoxWa2HiPL1xjXVJl7sgqoZB7HcUQJlqwTRf66xOxgQLrwqcVI\n4fouKrGUyld1ELYtHi00jDXze1TLe+SLyvx9qXSXxbMWS3cQRcKNLPAqoBbbbTOQPikukgPfVNZi\neq6L7fM31baqKZDjNRYJtpi1vpEFXnZP59so7dQaXoNuUNhYWy1JodtU7e2mxN8OYuTuG/MhX4jx\noiZK1Q1qiR0SCh6yfYjXipNglWUsqGrzhXgdQgXJMn2UBAOq16n6/2RchDKx27JxPbcOZW31RosY\ni6IOGQ8qdZKqOquxCtbjBgX0UhfmPeC7uauUimOSf9tUSaCeSvC5Iaa/rjZQp8rOUJmeK7pBKTe/\nU3SD6qUwRqOsyVZpdAyp6bY1F575XCbnIj2xSDd0zdId+T6qL0FkjjeLsVNUVPPBW1DWjJg1Ofb8\n+hevD0MhrQLqvjdrw5HZoK5LZA8TxlpsVyXrWowMGzbWVkkCZFZa1m6qwv2XBf5LA0vEzdGTdqJM\n7OREOYAbNDEnSipOrMIVKVXaQGS9FRMMPNJwVJU1y8OsVHmQ29BqC1CU46ugrChuGU6JwaE+kKoO\nGK8CmxG1ElSwPeUyCY3zMkhhaI96sFb4wLPd02q9PDrBwFATS1ThSHGPUUHahWOu2UDNx6u/RtV6\nLPuOKDeo2MbVVCbHsSwqFYOu+Dlu5fW3yPNr9vMlvqOWrzckN4u/uo6z4nkJGlwYl7rvg6x8jaIc\nm/e0TyQYlN3Tsd4rGWBlbdOSJGpRXPlalfvX9y3REgwoo4RQ1spavVDxXeJziUbujr4PoNoig7Zy\nDuqETNVSMtsHicbo+r7NbFCy9pMSG2RTfnzPRb8ON+halTXjgScTMmYmWnjdfzgPALDYaaCyZjGG\nVyJLOFGSSYpqi7tizJrqYiMz92owXGw1DX3fy4xLKgucatJO1QgEFGWtUKLCUKosLkMRP1e/svb/\nt3fmcVJU597/VfU607MxK8OqIIOgIoOIyi56cblEosQL1xdN9NUbTUxMXMmVAJKYKxqi0SDeqwZz\n7yXmNUSNQQU0xiUgoAGEAZV9H2Zfe2Z6Pe8f1VVdy6nq7pnunp7h+X4+fmSqqqvOOXXOqec825H7\nruwfqi8LoDXzm2klRZXpS52qQXlORNDVtp1xr+SU+aypHmUUuo2LbadDVHZkAOQdDIxzkGEfZPVO\nK7rcfZkEb9zLwppa+xkrwIC7XaAq4Eq/rRiQ+fumxgsJa91B8VlLvrSmTNrgTWDSv6UPlfZ3xknb\nehNlXoJN6RrtPaRrjJq1ZJoPLLUgcloOzsdX0H3MuEEVOo1jrA+eIizDOGmkwmTSHc0az2dNfZ/K\nUVL2/2SnoEgGPPNNPBg+eGHrxYhZNKgm0pDTv1OxCjcTOOw2IWq25WnWZAFVEwUeh+bYREBSP0fd\nLFH/udSb+eX3wRjf6Vy+JipQmwvd6nYxG9NmOxgo90iZsKYShvWLSmacm512USMs6015gHbjc665\nNYPNfbxxbzeYds21ZlY+x+rvhNzGasE8Fdry3oCEtQRRy2Vyl0nmeDddSQrRnQV42iHu/qEWnTqs\n+532OXJZ+KtwIMmaNQstiNXHVz8hM44Qa9huijNRqu9jrlkTUpOuJHFZTaMd4gm6qcpInwzM3nUs\nlJQxYfMxwDNx8seAeeLcVOyxaFZnuyqJLzeSmWPiNAol2meEGV9TJWuzFAHVYIYT0xZgAEgf1bgE\najOtu02wNv3qtZKhsKKdVd8j2cmjeUlx9SZZ3qLSadcKy2apafRpiqx2dsgkeGOAFzSk79+K9k0l\npFqZQXnzod0mQhQE0qydbTDAYAZN7v3NzJNajRdPIwaor+FEg4oCWOQaM18QtfaNawaVo0GTqGUy\nM4fYbNbZ5mW/NnnisoqStdLIqPNZmZnp9NsdJYPu+qypd1zgreRz3JkrrJlpSmKhzznGNYdwtGZx\nmQRVl6Rij0UzJ3m1jx3XZ01XZzNBTP0MnkAn/y1ps/hlceqiEZMBL9cjd2HEM1VbvCP5Gk3wQAzN\nWtCkXZLvhyp9H7RpWvQCh7HODlWAAS/gCpAXjFozv/qaaCBD5mmQeONenyeTH/yly6XJ6VPqpM9y\nGxt8Ah1iRgqxiUDCWjeQ+5sSYJBUM6j2GTKSaSB6jZnflaVJUC2UcHweAHDzrGlTd2gnnmTAyxkk\nl1edtkAunxr1xM64DtjQ/J4rxKrazjR1RyqENc6kHQ+iKFqaQbNcNoiCoNn4OVPgCd3xoI9C5juv\nR33WTJPiiqIqzxoM16Rij0WzCG91ol/eWBNFbYoKy2hQ2UWCq5GJLSDpoxGTgfSO9H5iUV8yqy3B\n1Npy3jV6wdxcgydr1rTRlfI9k7lVYLxl4WmFXXYbgqEwwmGm8rvSltduE6HPLaePjreJQoaaQY3j\nXr8zjD6oQnONZtzHoVnTmbwlzXHmCbGJQMJaoqgGt35fvuTc3jhpy89SCyWmG7mrzaAcwUYur3nU\nqXWGd/3OAslAvpUxbFtltjWZtPWaQP0HUR8hxU8sGqkTM0/dkTIzaDc0a5pUDJH/21WzoCAIkejV\n5JY3GfCEkniIJkk2105ot5JKRGvDWYykoH9zTeuyHxOM2nL5N9rdCXQa9zg0jrxFGi9KMyVbyHG0\nIPI50/QqgnWqEek+2gULb1sxQN0u6YsGNReWtdGrahyOaGoauUx6s62Ucyz24jUTx32IM+71+xiH\nOG1nt8uaNatxL6qiwLXbcKnvE0jyYjvdpEVYW7FiBWbNmoXRo0dj//79yvEjR45g/vz5uOaaazB/\n/nwcPXo0HcXpEQzRDqdPJ5AM5FsZokGFqPAkmUO05/VRj9YpKlQaJL25VdSaFaXfRc9HNWvJ9+nh\nR4PGMGsJWv8WQ5QVL8dRN82gSTeZcD6s8aD+gJtqkFSCQCZhVt5Y6Mea2XsMqSZ+IIYZlBdhnIrF\niNmH1Saabu6tXKPTGpguwCwEOk20s5XGMRWa4+6MNdGo3TeOR21yYzONozbAQB8xmpoAA6v2B/jm\nbNlX0hcMmY4R7eLVROjO0HHPTd2h3y6Q03Z2Ufu94ZmHtQEG/HFvT4Fgnm7SIqxdddVVWLt2LQYP\nHqw5vnTpUtxyyy3YuHEjbrnlFixZsiQdxekZLOqzlqoEsYCJLxmLXmMWFaYxDXDMivI1ZitWjRlU\nERxVqyFRVkun42OWmKaEG2WlZA9XZcm20jia+ReJQtJXZrwVdjyofdb0G7nL6DfMzhRkn57uBhho\n+7dFXzAVSqJ+YlzNWirHNEdTojbv8LSNou4DbRRKpP+Hw+Yfs+i4D3Md04HUCC5mQSBSWczHGleg\n5rguxGMGVRY1IWYMMEiBNpHbLzmO9LycewAQCIQttUN6nzV+/rm+Me5FUfI5VteJ56cHQKNRtLKu\nyGNFn6ZF3XZ9lbQIaxMnTkR5ebnmWENDA/bt24c5c+YAAObMmYN9+/ahsbExHUXqNkwlrelNM8nA\nNKWGzjTA82mTz0n/5wsccnnNVmaiGE2Ky92XLwWaBystSKzoPo2ZiHHyF9mj5gW53GarcKbSOPK2\nMkq6yYRTlnjgOclbaZAyCZ5GLB7UQpSS+d7k4+sPhExX2HzNmvY8kCJtue642rTOi2QGjGZb0wCD\neASXsHVgUdIFF4tUOtIcFH22vrx6nzWDGVSlQeIFD4iq/iI9j2cGTcEOBiaaH6ks0UWlmRnUr9Ks\nGXzWRMEyGlR+ViZqkMzGvV2lXeYJsYYtBU0WaXrfZp5LTbLdWNJNr/msVVdXo6ysDDabpPmw2Wwo\nLS1FdXV1bxUpLhhPs5bESU6+P3dvUKae2GNM2jHMfWaToFazZhQc9RE8ycDsA6JObcALiQd0zsgc\nbaKsWfMFo9nDeXsRys8wE2L1yRmTAc8cEg/SB1z6t7kjfWZO2rz2jwe+vyX/msUvbIkrHQxfs5a6\nBRhPsyYLEzxtuVJei0jPqHAp3YPvvxWtk6WJLSVaJu0xtS9ZIj5rPM1xPAKqJhqUk2Q2FQswM4uG\n1dzsskf3D41qh4zvMWgwg/aNRZrZuFfPq1yfNcWvzXyRphb4THN2pmD+Tjf23i5AMigqyknbs0Sb\nCLfbgZKSXBQ2dAAA8vKyUFKSa/obdbSo1XUAkJXthCAKhuscThvsEcHDbrdBtImaa7yR7Po5uW6U\nlOSCMSDH49JcU5CfJf2/IBvhyCAYUJCtucbjcSLMpHJmZTkhCEBpaZ5yXp4I5DZIBl2RMVRQoG3H\n7CyHsjL3eFwAgJKSHBRF6gFIAQROpx0lJblwuuyw222ae7gjv3O5IuUVBHiynZprBtR5AQB5+VnI\ndktDIj9f1y7ZToQR+/0lgigIyHI7Te9pdjw3x40wYyguzlHapbQ4FyUDou3idNhgd9qSWt5k4HI7\nYLMZ+3csGrxSZGtOrlsZ77mRvi6TlyfV/8ujjZg7YyQAoKjIo7kmy+1AW2fAtH93+aVdH9xZ5u8l\nUVq6pIWCfqzleFwIM4aSkly43U6InHFvV/Vvm02E06Ed96JT6q9hxlBULB3PydG2S36kXQoGeCA6\nfdyyuN0OdPmCSe0vgiggW9eOA/JbpTIVZCvHCvK14z43xwUGaV7Pzo6M++Ic5Oe4lGtcLruyiLTb\nbXBF2kiNKArKewyHjfNhfl4WQmGGwkKPQYvVXZwuh2EO8kSisuWyCIKxXYobOwEA2TkuFOS5ARjb\nJcfjRENLFwCp7wNAUaG2f/e1ce90iHA47Eq7ZGVpvytFDVK75ORkqca9y9BfwjVtKCnJRWOH1NaF\nunbJznJwx5cZmdZ+QC8Ka+Xl5aipqUEoFILNZkMoFEJtba3BXBoPDQ3taVtNhIJh+HxB1NW1ob1N\nGjgNjV7UeRymv1GHh9fVtVne3+v1QeBcFw6F0eWTPiQ+XwCiKGiuaW7uUP5fV9eGcDiMrq6A5hqv\nV5qo6+rb0OL1AwDa2jo113R1BhAOM6l+Xh9EQTCURRQEtLZ3xaxLvNQ3tAMA2tt9mnsKYGjvkMrZ\n0ioN2qamDoT9qm2UGIO3w4+6ujZ0dgXAImWXkcO1GyPtEgyG4Pdr26VN9R47XdKQ8OrqFwqG4PeH\nklZnuWx+f5B7z5KSXNNndXVKbVJT24rmSLs0N3uBoHZ7qY5Iu2QSXq+f279j0RqpZ2NTB2prpd92\n6urX0eFT/i2Ph5aWTtQ5oh/ioOo98vq3vPpubUte/25o9Cp10DwrEIQ/EEZdXRu8kbLrnymAwdsh\njYsuXxA2wa65Rh7H4TBDba0kCHV16tpFNe7l3Hv6cS/PL0nt34EwArr+3d4ujbX6+nbFROHVjfvO\niHBTU9uqjM2mJi/8kX4vl1d2bZDGfZg7T7VFxnEwZBxrwcg8cqq6BVmu5HwKpffIn4NaI20eDIXh\n17V1Z+Qd1da3I+iT6t/h1bYLwgwdkffX1KTq37p696VxLwoC2iL1DARCCAZC/P7S0I48lzSO9eM+\nHAqjM9Ke9Q3SWGvXjd9wKAyf39hHeFjNvclAFIVuKZh6zQxaVFSEMWPGYP369QCA9evXY8yYMSgs\nLOytIsVJVPCK1xk5EUHS1NFYbRoAz3ypfZYUiaW7h8o0YBYNKl8jZzvnmmaSnMbCzKzltNuU/fLM\nrtGnNuDlshIQzerN83ng5p+zMMkmC16usHhQl9csCWQqnKeTAS/AIx7Ufj9m70g9zsx8V9QBBiEr\nk3gyo53jCDAw81nT+20aTYaRZ6hMnOb+WwzMbIsnIT0+azxzttk70IxHfbJrm9pPKcx1J1D7+0kB\nBtrB5tD5syYDXmoJvR8kb56SkzFr/C112j51rjBTd40Mdn/gjXspxZD53qAOnRkU4EXSRnMEmvqs\npSCaP92kRbP285//HJs2bUJ9fT1uv/12FBQU4O2338ayZcuwaNEiPP/888jLy8OKFSvSUZwewaD2\nWYvPvyWRvItmk7YgAKfrvZFrzPMXSftbMu5H0a4qr9lEKd9396EGHDjZYur0nIqN3LlZp2MMQm00\nqPEegiBEsoPLkxxHsFH5+1mlCki2gypvcooHufy7DzXg4KkWqXx9xWethwEGTW1dykbV+vuo63sq\nMlb4Pj1h1DR1IBgMGz52giAkXXCxinb2BUI4fLoVzW0+7sJIEAQcPNmC5nYfP0VF5O/Pv6pBll3Q\nHFOewxN+DD49qdnU3ExwOd3gVcplWFRGfrPvaBMOnW7lXmMTBJyq92LX/locr21HeWE29IiigNqm\nTlQdboA/GDYmSlU59ScLbkoZOahClTPSLHXH6XpvdAGmD5Zy2FTzIbjXpCJQJBmYjXu7TcTJOi8O\nnmxBQ2sXzuf0SwBobPXhlFMa0/rEuU6HiEAgrPhsAryFkYBAMPPaJRHSIqwtXrwYixcvNhwfOXIk\n/vjHP6ajCEmDqaS16NZLsYS1+DuJmaOxPxhGi9ePxtYuhMPGyVYdZRVLa6ZJjql7lLyv5K/X7TYt\no90mJjWXj9nE47BrhSyAI1yK+ihZY9s57TZl+yCrpKEhZt52qQgwYN2MBpVZ9UaV8m9+pFvmTU48\nZ/B4kLUMv3//ALZ/WQvAGF2pru/bnx4DAGQ5bdr7iAKa2nz4yX9ulc67tOelZyW37cx2MHDYRDAG\n/Py/PwcA5Hmcht/WNknm3/t/sxnlRdmmkcw7vqrFjq+kdnFx6gxoF2m8reiSvU+mVYLYF/+yz1A+\nGU+W9Fl69k/ROcgwHu3S9kE//c9PAQDlHINMTpYdOw/UY+eBeuk3Bk1V1Kk/WfDmF0EQjDsucAQO\nAPjTR4eVY3YbZz4MmO8lC2TwuDcR1rxdQbR6/fjF//4DALhBIADwyrtfKcf03z6H3YZwRFBT0p7w\ngkkyMP9cItAOBt1AVsnrIzDNSGRLE14WfgC4+pIhAIDG1i7+Ru6K+dI6UgjQCXS6a2aMH4Sffnsi\n7rh+jGkZk65ZM9GaOR3RzY3lOhmEEkEb5s9rO0+WA96Ir4fV5tKWZlBbdDPsZBFmxoknHrp8Rk2A\ny2H8QIczcHLiRcvFg/qdydrELr/WR0//kRo+MBe52VoByCZqNaQ8zWayI+rM0ix43A7d39ZrZ29X\nENm63/A+gNPGlXOvCTOmRJbyo4dTkfRZe0wvJALGdplyYTluu2a05pgxJY9RyNbzk4WX4CcLJyh/\nz5qgzfPpdKTADMpJ3QFozba81B36vgBEF84ykmYtpFhOAJMN7vvQuNePs2zdGNALrIDxG+BSbWAv\nzwF6k7d6X9W+Sr+IBk07Os1aMs2gZtqhkgIpoqvN649sWK5F5KyeuyOU2EQR55bnWX447EmeEMwm\nHqdKs2am3hZUPms8IRaQVtiyYzXjaHZ4yYJ5qTvka3gTSHfo7g4Gnb6g4RjP76cvrbBjoU9oCgAd\nPr2wpu2Tw0qNTrz6vSq56QTSZObP0X2M9R9nPW0dfsNv+Lse6PfjVC3STFOaJD+ZapiZa/fV8PYw\nPrc8T3dM9xv9bTjtUJDjQoEqglTfdsquAUncwN7Uz1fje2hsF7fTKHwayytpYuU9RAFe+prMzNRv\nNu71gpdbt+jUa0MBjgVGTnweDJn68KrTe/RVSLOWIEwlKMXrjJyQGTTM91mTJ/K2Dj/XN4tv4rQQ\nSkwmbRn9RKEm2ROCmWlGDjBgjFlmpLdy3AWAHHdUsxYzKa6JSVaeNJK9Cu+OsKYXUnio98vLJLq7\neT1PW9HRZa1Z4/VhfR8z+4AkczFiZrLSC2eOGOkjGDPWKZ4AFXWC2OjCSHtNSnzWuFpsY4F52jbZ\nFCrTnT6jx6CdS4VmzcS1QbN1HqddeAKeQViTc0YGwuaWBjGaly+TiHvcm8y7angBBoDkKmS63RTl\nWTv7YIj2J9mP5sCpFlx+wUA0tfnAGENhJE+OjHoOPFXXjsI8tyFU/Eh1KwpyXKhr6bQcuLv218Hb\nFVA0bTLyQPjiUD0GF3six7T3kCfFXQfr4fPLTtr8elqFsgtC1BR1pLoVw8tyUd3YgcJcF7JcdoQZ\nw9HqNowYlGd6DzVmphmnQwQDsLXqDKobOzT1lBFFAafrvag60oAj1a2aXEwyOVkOHK5uxfYvayyT\n4m77sgYjBuVLdeQ4qALAlqoz8Ljtyupffg9fH29CU7svahpXfaAHF3swuCQHR6pbUdfcqZwPhowO\n7vHA06zpEUUBB4+1oKGlC6IoYECutl2qG7zI9zjRFekHbqcNLV4/yos8yjX+QAhnGjsgigI8bgc6\nfEGlb8k0t/sQDIXhcTvQ2NqFnCwHAsEwiiPtcqJWSquT53HC2xkw1X7GgmcuNmrW4hDWdFpRrmmG\nAYdPt2LP4QbkZTthswnIzXLAHwyjpCALR8+0QoCAgYXZBv8wHrLcp6+21YLIjO4IMXL//nRvjbLt\nGs/XqcXrx84DdSjMdcNhFxEIhjGo2KNETR4904qmVp8yvwkCMKwsFx1dQew/0Yz8HCfyPU4cPNWC\nkoIs+PwhU581TR04L0HfNt1JpBwLWbO292gj2jr9ECCN1TBjGFiYjbqWLpysbYcoSo4vsnl8xKA8\npQ2+Pt6E5na/0l5fHGrAueXGHF02UUDV4QZs/7IGoVA4rveWrZuDZaFkW1U1vjjYAID/Hr881oJj\nZ6TUEw67iIIcl2JerG7wGnz0nA5RM+4Pn26FKAL5HhdavX7Y7aIy7jt9QXx1vAm+QAiiIMDttKPL\nH0SYMQwvy1Xuc/BUC+qbO5Vo/bqWzqQt0niBFwCwbV8NDskBVzZjv2vrCOCzr2qVpPOjhxUo/exk\nXTtO1rWjpMA6Z2pvQsJaojBAtoPKws/fdpzC+POK8fRrXwAAfrtoluYnap+1n768HRVDC7Do/0R9\nKY7XtOFnv/tc+Tuf42jscduR5bLhve3HAQAXjeCbQ6oON6LqsLRll2G7kshg37DtuOF3euTJ8eKR\nRYZzNRGn5w92nMT/btqPb047F29+cgQjBuVh8W0TsWn7Cbz2t4N45JZKjB42gHt/NbJmkuezBgC/\neGU79zwgvYOmNh9+9f+ktm+NJEVUU1KQhbaOAF74814AQG423/y0/ctaxXk9V/exkIXAte/tV46N\nGT4AD/1rJVrafVjx+52WdXzpkSuxYu0OJZpLRu9TFQ8XjSjCzgP1GDkoD4dOt2IgJxJu7xGpDzy0\negsAY5989MVtGFKSg5N1Uo67QcUenK73aq576e0v8XnEaV3m5Ueu1Hw47//NZgDA+cMK8NXxZuX4\nbxfNQpc/iKW/3a75/cUji7gf7FjIHyo1F56r9SofO7wQf9txSvm7WLegAYzvPo/T/u2dAbR3BpTx\nrObxuy7D8leksTr+vGL88FvjYpbdzAxalKcVoM/hfOSzXXaNUFqqq5NeiNGfB6J97OMvTkeP6fp3\nY6uU5+u5P+3RHL9ywmDcOns0mtp8Sr3VPH3vFGzYfhwbt58wnAOgJHGV4QmovGN6H0w9F59XjC1V\nZ5S/Lz2/1PTa/Bwnd4GT55GSIqvnQ5lJY0rx2Ve1pi4sv100Cy1eP3fcywsgNXI+PHkOysk21nnU\nkHwcONmi/G3Y5SDSJs/8YafpNVWRcf/YK58px84ZmIsl37kUgDTuzeoDAF8da8KTrxrrJI/7p//4\nBQ6qyqgmy2XHqh9Phy8Qwi/+5x+G8yM5i/ep48qVYCAAqBiSrznP6wf6MeyKaEjf+DganKH3AZQF\n89VvRoJZU4UAAB+qSURBVIOy8nOcePreqQCAJS9H56m/rBxieGYmQMJagmg1a9GBIkdtcX+jW/Hv\nP9Gs+bu53af5W+9kCUhq3Ce+ewUcbicaG70oHaCdlHmCjN7vjOeHZrW6X/3ADEv/rMORsPqvIx9p\n+e/jNdKqrqG1i/9DHd5OaSLVm4Vy9A7YnLLG42czZ/I5uHRMqWI+LtO13YBcF/7lyvPw2t8OKn8P\nK9N+OCeOLkHF0ALNu2uNTMDyRAwAN0w5Bxs/O6FoLmWa23zwB8O47vJh2FJ1Bi3tfvzLledh9qSh\nMcuvZ8b4QbhsbBmcDhGdvhDX38UKOVeTLKgB0bQwwVBYMT18dazJ8FtfIAS309iP1IKajOwnqKa2\nuZPrTB0Lt9OO5++fjtaOAILBMPI8TkN/vmR0CVb9eDqKinJw/FQTinQabgC47vLhmFBRApfDhi5/\nyKBxjEWr6l3vOxbfPsZmZtD8HBd+de8U2EQBPn/IoJEHgKd/MBW+QAidviAE8AXQ1ffPQFlZHk6e\nbuZ+3AYVe3Dd5cPw7taoUKK/j7fL+K4A4HhEQ2N23tsVRFObj3sOMGo/SwqysPL7U1QRskatr3x8\n9f0z4A+GuMEEl55figt/PB3FxTmor2+3HANP3j0Z6vyYMgNyXfjl96bggVWbDee+Pt4c09e4pZ1f\n7+svH276m9uvOx/nDclHGWeB9cD88WjvDCDMGHKzOAv2LOO4i0fjeDTyDgMWKUrkcV/Xwv+OyePe\nTFADohr/ds6CeXCxB/fPH284fuP0Ebhm0jAA0ndVPze4nDZl3NsjEcv6BYn+NyUFbqM/KOfb19Lu\nNxyL1xrUG5CwliiqEawWkKzGjF6e0GsJ4jHfANIKuaQkF26O6YxnTovHgdnKqTnW6lb2cWnTD85I\nW8TrOiF/1GOVl6dd6fTHZxJUq/l5DC6JntcLc4A0KZYUuLFfpUCQy60WSgYVe+CyiwZhTdZGDiry\nKO06qDi7eyZBQVDM1DlZidtR2zvN28zbGeCakqO/DXCFNf69jM+paezEuJHGD1U8uJ32mM/Octnh\nyXKgON/4DgFJYIrVF6zojn+TmRkUgOIAn2vSJA67CIddtFxUuZy2mNcMLYkGW/A+XGb+PLIJVO8f\nKOPtClgumAIc5/14BWSX02ZpZs5y2ZHtdsTcfcDBmTdilSUe/ybeYsTqnoBkNjbrf06HDYUWcy5v\nrk4kWCcd454xxm0Xt9PGfU+iIMR0B4g17vW/z+EKuubPUAuxPEVJpkABBgmiSrOmEdaCqklcn6pD\nH2Cgnzz0EyHPqTIWvEGr76C8STqWQGZFfcTEYbbq9scZZdXeGYAAo48GbyWpJx7/rXhQD/h426Q9\n4oOlnpzMtEY1EZ879TuJFf2XKsw+MgDQbvJRluEJYOb3Mj4nzFhc7zVTsWo7M8w0a+kk1keok5MO\nBoi6InhN6t3eGTA9B8QXDJOJmAmnMowxeE2usRI+etL39ZYGgO9zaYbVe5L7tVnOuXjHfZc/xB33\nqSSeedRKYLcSYjOJvjtr9hJMZQdVRzZVHY2aROqbO1E6QFoqN7X5cLy2XXMPb1cQW6qqMWpIAYrz\n3di2r6bH5eJ9CPTCDy8SqyfIpjO1GaTqcANqIpsSVx1pVBzDs112jD2nEJ9/VavkK1N86LYfR3bE\naV9NPA7YsSbVeFG3VbzOzKEww9ufHkN1ZD86wHylu+NAHQBtnfTvJ5X8becpCAJQNiAb+46am+/+\n8XUt7LYyfHmsiSuYfLr3DNo6/MjzOC1N/18da8LuiBO0nu441mcKG7ZHTYn+QBgf7pJ85ERBwMTR\nJch2O7DrYD3qmjrhiPjSHK2WzFDdyamXLGIJimYa6gMnm/Hp3jOmfebLY01cE7hMXxXWYhkF1n96\nDFWH+f3byszfk77PE0rinav2HG7AgZPm7+ntT4/h0jGl+GjXKe75T/eewZHq1pjP+fPfj8S9SE8W\nPZlH/7bjpGImznRIWOsG8vBw2EVlXzLZqR8A/r7nDG6aPgIA8PDqLVwzwUvrvwQA3PWNsYpTqEzF\n0ILEy8QZtLyM6MlgaGkOTtS2cyfiX6mcstXZwwHJIVxfVxmeNrHA44IgRM2pF59XbLjmkooSbFY5\nGpcUGP1+4kHdViMH8/0WJo4uxeY9ZzTHXlc5tQLAwMJsXD1xqOF41eFG2G0CivPdmDVhCP7w1wPI\n9yTmL5UIl1SU4B/765S//2fj13H97s1PjuDNT46Ynt/02Qls+ozvTK6G56QswwuIyCQ8brup1uR4\njXbh9d8bou26eU81vnfjRXiWs/uHPRJR2lsMUkXx/tOlRj/JK8cPxpt/N773Tl9Is9uAnvc/P2n5\n3OnjBiVQyt7hnIG5qGvuNH3nPN7QjW81vOCBf5o4FO99fqJHloxstx0uh02TF07vGnLJ6BL84+s6\n/U+5wTJqtu6rwVYLpUE8Y159ndMh7bYwbmQRdh9qwLSLU9cP9AvkaReXG67R5+2T+Z9N+zV/T75g\nYPIKlmQElsyU7L1EQ0N70nMEmfHDX3+CS8eU4tbZUoZtnz+EMGPo8odgswl4cNVmTL94EBZGzt/x\nxAfKb9VO7DI3Th+hDPxnfjAVNpuAbJfddMVUUpKLujr+SoBFtkvq9AeR5TRqqoCIiVZuKqF7phnG\nGPyBMDp8QbgcNtz7zMea85eMLsEd149RoqKOnWnDs3/ajQG5Lo0W7tHbLsGr7x/A4dOtWHDVKMzm\nfES6/EEMLMvD6eoWZHHaJcwYunxBOOxSHiOXw9btMP9wmKHTH7Rs/3CYAYIksP/fFX8DACy+bSIG\nR1IciKIgvQd1HXyhiIOu5LchvycrfxOr9xwv/kAI/qCUk+ntLcfw3ufRCffc8lz8YN442EQBDEBd\ncyce/29tBNfsS4di7tRzwZjk1+Gw2/DKhq80EaLfv/EijB5WgEAwDJfDhr9sOWKIDHzmh1PBwgwO\nuw2BUJgb7ZxMetp2LDKe7TZ542wBb396FO9GogYvv6AMC/9ptPLR/M3re3CkuhWCACy7fZImAnbl\n96cAMPfZSRbx1Fndd3n9W55DO/1SMMO9z3yinFv0fyZgaGlOJDAjCEDAmne/VASDn357IsoGZCMQ\nDCHX44TPH4LDLnbLpSMRkjFO1OO1yxfCi3/Ziy8ONeD6y4fjpukjNAvGnQfqseoNbcTsSw9fKf0j\n0qS8OVV+Rk9N4b5ACKUluWhsbFf2sdUTDkv9VxCAM40dmkwDz943DaKg+QRo3jMALP3OpSgdkAV/\nMAyHTQAgYNUbe/BlJOBo+R2TJOFfQNQviAG/f38/PthxClPHleNfrxoFl9MmnYox1yUD9XdNEKz7\nNyCZfX/03N+Vv+dMHo5vTh0BURSS0qesEEUBRUXGhN2xIM1agqiT4gLRffjkibgwz23q15KfY/2R\n6qkmTBAEbkSNGlEQjJsqduM5Vs6/A3KkfGtym8jbAumjxgYVeZREoAUmbeN22uGw2wzb7MiIgqCc\n66neQs4nFusaPeVF2nxbgiBomjjbbdf4DMnvKdU4HTbF5yjPo61XYZ5bk92dF7wxIJI3T0L6/wCd\nE7I+8ooXgclLj5HJqAM4ZF+XovxovbJdds07VfvD6P2CEo02TSWxPpjyed4YKC/KVtpEHm8lqiCO\nfI8z0h7SNakUTJONerxmu+1wR8peOiBLaRN5vOrTRgDxCSL6OaG7uCJj2sqlRRQFpW/q83HyzLBu\np02TbqS4wB2Zv6PXqOeH3GxHtM5ypQQoQT1ZTrvm/adjrovnu6Z+T/p2KC3ITrlA2VMowKAbWA27\nHNU+lIbf8X7W9xWbBozROXwByO20KW2SyVE4sUg0dUZvoF9p6t8JzzzD6645Ogdpsyzr/Q11+gjT\n8c+6F4DQF+DvXRntC33ZD1GP/HZ5aYv6Wj3j8efSf5diCdq9FRiVTHgJ2DOdvvuF7CWYOhyUQ06W\nA7sPNeDYmTbUNmsdsHkq67c2H01uATMAYxSqQ9GYq1ELEL0ZKddTUpFdPdXoc1fFW4fupFfpD8Qz\nmTMA23VJhPsLsXYZ6K9Cup6+Jqx1R1sUay5OtWm7N+gLc3j/a/UUE0NWw9DI5tGPvfKZJluyTRRQ\nlOfG1HFa50c5+OCKC8qSXdS0ceEIbSZ5/ZZEoihgcInWRi+bh2ZNGML9TV9g4uiSjDJzWXHRCO1O\nFMMHGn0mzh+mDWy5QLdDAADNeyzMcxk0cnIUtMzlfbhfq1Gbky4do82Wf/Ul0Yznn39Vq7gzfHPa\nuekpXIoYHQl0Mgu4GVws9YXhZZm5PU93mXKRNEefN8QY6KVPvVE5yhj0lGnIKabMvjFzp0T76Sjd\nDgIyMyoHx3yO/B2w2k0ik1B/c3hbhGUaFGCQIN9/+iNMuagct1xdYXrNohc+1WjVfv3DqRHfq6hs\n/Pv39uP9f0iRVP9y5Xm49rJhcT0/1c6PPUWdAV9NOMwQCIbhdIgJr2Iyvc6pIFV1lpN9ct8RYwiH\npf+sNCXBUDjiNMxPB6NOKGoThbSvWlPZdlZaBTkRst0uJD1NTixSVedQOGzqyA5IgSc2Uew1fx+a\nG+IjGArHHIux3vXZAgUY9BPiEW0Lcl0aYS0ny2EYJA6VWaWvqdatMPuYiaIQ16bXRGqxEjZEQYBo\nE4AYrymWGaQ/mkmA2PXqj/07ltDp4GwFRWQe8YzJdC8wiMSgt5MgkhnUeuWh3ymAt5pxqSa5vuDc\nSBAEQRBE70BSQqKw2KHI8WjKbBYbpBMEQRAEQciQsJYgLOZGJMCIQVGH3CkX8jMiy4EIADCK48hK\nEARBEAQBkM9at4ilWZsxfjBmjLeOnhk3shi/XTQriaUiCIIgCKI/Qpq1RGGxfdYIgiAIgiCSBQlr\nCcKAHm/XRBAEQRAEES8krCUIYySrEQRBEASRPkhYSxiS1giCIAiCSB8krCUII581giAIgiDSCAlr\nBEEQBEEQGQwJawnC4kiKSxAEQRAEkSxIWEuQeJLiEgRBEARBJAsS1hKFNGsEQRAEQaQREta6BUlr\nBEEQBEGkBxLWEoSBRDWCIAiCINIHCWvdgMygBEEQBEGkCxLWEoAxCi4gCIIgCCK9kLCWALKoJpBq\njSAIgiCINEHCWiKQYo0gCIIgiDRDwloCyDnWSK9GEARBEES6IGEtARSXNZLWCIIgCIJIEySsdQOS\n1QiCIAiCSBckrCVAVLNG4hpBEARBEOmBhLWEIJ81giAIgiDSS0YIa0eOHMH8+fNxzTXXYP78+Th6\n9GhvF8kSUqwRBEEQBJEuMkJYW7p0KW655RZs3LgRt9xyC5YsWdLbReJCOXEJgiAIgkg39t4uQEND\nA/bt24c1a9YAAObMmYOf/exnaGxsRGFhYS+XDti69ww2bD8OICqsUVJcgiAIgiDSRa8La9XV1Sgr\nK4PNZgMA2Gw2lJaWorq6Om5hragoJ2XlKy/rwEDV/YeU5WLGxGEoKclN2TNj0ZvP7i2ozkQinI1t\ndzbWGTg763021jmdZGL79rqwlgwaGtoRDqfGRjm8OBt33zDWcLyuri0lz4tFSUlurz27t6A6E4lw\nNrbd2Vhn4Oys99lY53SS6vYVRaFbCqZe91krLy9HTU0NQqEQACAUCqG2thbl5eW9XDKCIAiCIIje\np9eFtaKiIowZMwbr168HAKxfvx5jxozJCH81giAIgiCI3iYjzKDLli3DokWL8PzzzyMvLw8rVqzo\n7SIRBEEQBEFkBBkhrI0cORJ//OMfe7sYBEEQBEEQGUevm0EJgiAIgiAIc0hYIwiCIAiCyGBIWCMI\ngiAIgshgSFgjCIIgCILIYEhYIwiCIAiCyGAyIhq0p4ji2bVX59lWX4DqTCTG2dh2Z2OdgbOz3mdj\nndNJKtu3u/cWGGOp2aeJIAiCIAiC6DFkBiUIgiAIgshgSFgjCIIgCILIYEhYIwiCIAiCyGBIWCMI\ngiAIgshgSFgjCIIgCILIYEhYIwiCIAiCyGBIWCMIgiAIgshgSFgjCIIgCILIYEhYIwiCIAiCyGD6\nxXZT6aapqQkPP/wwjh8/DqfTieHDh2P58uUoLCzErl27sGTJEvh8PgwePBhPPfUUioqKAAAPPPAA\ntm3bhrq6OuzYsQMej0e5Z3NzM5YvX469e/fCbrfjuuuuw7333st9vtV9rM7psSrrunXr8Lvf/Q6i\nKMJms+EHP/gBfv/73/f5OltdO3r0aFRUVEAUpTXM4sWL8V//9V99us5HjhzBkiVLUFdXB7vdjosu\nughLly6F2+3Gjh078OSTT6K1tRUAMGPGDDz88MMQhJ5vtZKpY8SqPXh88MEHePLJJxEKhXDBBRfg\nP/7jP5CVlQW/34977rkHVVVVAIBt27b1+zoDwAsvvIC33noLdrsdHo8Hy5cvR3FxcZ+vN+99ypw8\neRKzZ8/GqFGjlGO//vWv8fjjj/fpOluN//fffx/PP/88/H4/GGOYN28e7rjjDm5ZUkGyx9KOHTvw\n2GOPKfdvaGhASUkJ3njjDe7zu/sOU3UfAAAjEqapqYlt3bpV+fuJJ55gP/nJT1goFGJXX301++yz\nzxhjjK1atYotWrRIuW7Lli2svr6eVVRUsPb2ds09v/vd77I1a9Yof9fW1po+3+o+VufUWJW1sbGR\nVVZWsrq6OsYYY++//z6bPXt2n69zrGv1x/rDez5x4gTbu3cvY0x65/fddx/7zW9+wxhj7Ouvv2ZH\njhxhjDHm8/nYggUL2BtvvGF6r0TI1Lazag897e3tbPLkyUob/fu//zt77rnnGGOMBQIBtnnzZrZv\n3z42adKks6LO+/btYzNnzmRer5cxxtjvfvc7duedd/aLevPep/o++mP9oc5W43/Xrl3szJkzjDHG\nWltbNXVKB6loXzX33HMPe+mll7jnkvWMZN1Hhsyg3aCgoACXXXaZ8vf48eNx+vRpVFVVweVyYeLE\niQCABQsWYMOGDcp1V1xxhSJVqzl69Cj279+Pb3/728qxkpIS0+eb3SfWOTVWZWWMgTEGr9cLAGhr\na8PgwYP7fJ0TvbY/vOchQ4Zg7NixAABRFDFu3DicPn0aAFBRUYFzzjkHAOB0OjF27FjlXE/J1Laz\nag89H3/8MS688EKljRYsWIB3330XAGC32zF58mTk5uaeNXUWBAGBQABdXV0ApHlh4MCB/aLevPdp\nRX+os9X4v/jii1FWVgYAyM3NxciRI3Hq1CnT8iSbZLevmoaGBmzevBlz587lnk/GM5J5HxkS1npI\nOBzGq6++ilmzZqG6uhqDBg1SzhUWFiIcDqO5udnyHgcPHkRZWRkeffRR3Hjjjbjrrrtw4MCBlJbb\nqqyFhYVYvnw5brzxRsycORO/+tWvsHTpUuXavlrneLj11lsxd+5crFy5En6/XzneH+rc1dWFP/3p\nT5g1a5bhXENDAzZu3IiZM2cm/bmZ2nZW7QEYx8igQYNQXV0d1737Y53PP/983H777Zg1axamTZuG\nd955B/fff7/m93213rHwer246aabcNNNN+Gll14CY0w51x/qbDX+Dx06hF27duHyyy/vUXm6SzLa\nV82bb76JKVOmoLi4mHs+Gc9I5n1kSFjrIT/72c+QnZ2NhQsXdvse4XAYX3zxBW666Sa88cYbuPnm\nm3HPPfcksZSJ0d7ejrVr12LdunX48MMPsWjRItx7773KBNUf6wwAH374IV5//XWsXbsWBw8exKpV\nq5Rzfb3OwWAQP/7xj3H55Zfjqquu0pxrb2/HPffcgzvuuENZlSeTTGw7q/ZIBv2xzqdOncJf//pX\nbNq0CZ988gluvPFGLFq0SHNNf6x3aWkpPvroI7z++ut48cUXsWnTJqxbt04539frbDX+a2tr8b3v\nfQ9Lly5VNG3pJhntq+b111/HvHnzknKvdELCWg9YsWIFjh07hmeeeQaiKKK8vFyjcm5sbIQoiigo\nKLC8T3l5OcrLyxV16ezZs1FXV4fGxkasXr0ac+fOxdy5c7F169Zul1V/H6uy/v3vf0dubi5GjBgB\nALj++utx/PhxNDU19ek6x6K8vBwAkJOTg5tvvhk7duwA0LffMwCEQiE8+OCDyM/Px+LFizXXd3Z2\n4u6778aUKVNS4kCciW1n1h6PPfaYcp/Dhw8bynr69Gmlj5yNdd6wYQMqKipQWloKAPjmN7+pccTv\ny/W2wul0KuaqoqIifOMb38jouSGROluN/4aGBtx+++248847cd1118V8bipIVvvK7Nq1Cy0tLZgx\nY4ZyLJFvoxXJuo8pMb3aCC4rV65kCxcuZB0dHcqxUCjErrrqKlOHQhm9Q2E4HGZz5sxh+/fvZ4wx\ntn37djZt2jQWDocty2DlmBiP86NZWffs2cMmT57M6uvrGWOMffrpp2zy5Mnsl7/8ZZ+us9W1zc3N\nrLOzkzEmORs//PDD7PHHH+8X7/nBBx9k999/PwsGg5pzXV1d7LbbbmNPPvmk5fO7Sya2nVV76Glr\na2NXXHEF19leRu983p/rvGHDBjZnzhwlwGDdunXsW9/6Vr+otwwvmKC+vp75/X7GGGMdHR3sO9/5\nDnvllVf6fJ2txn9jYyP7xje+wdauXWt5j1SSzPaVWbx4MVuxYoXlc3v6jGTfR0ZgTGV8J+LiwIED\nmDNnDs455xwlJHrIkCFYtWoVduzYgaVLl2pCdWXb+L333ovdu3ejpqYGpaWlqKiowMsvvwwA2LNn\nDx577DH4/X5kZWXh0Ucfxbhx47jPt7qP1Tk9VmVds2YNXnvtNTgcDjidTixcuBCPPPJIn6+z2bU7\nd+7EkiVLIAgCgsEgKisrMX/+fNx88819us4ffvghvvvd72pSkkyYMAFLly7F2rVr8fOf/xwVFRXK\n9ddee21STLOZOkas2oPH+++/j6eeegrhcBhjxozBE088gezsbADAvHnzUFNTo6QBGDduHN57771+\nW2fGGJ566il88MEHcDqdyMvLw7Jly8AY6xfvWv8+p02bhscffxybNm3Cs88+C1EUEQwGMXPmTNxw\nww2YO3dun66z1fhfsWIF1q5di3PPPVc5d9ttt6XNfJiK+aOrqwtTpkzBa6+9hpEjR1o+v7vPSNV9\nAICENYIgCIIgiAyGfNYIgiAIgiAyGBLWCIIgCIIgMhgS1giCIAiCIDIYEtYIgiAIgiAyGBLWCIIg\nCIIgMhgS1giCIAiCIDIYEtYIgsh4Tp8+jcrKSoRCod4uCkEQRNohYY0giIxk1qxZ2LJlCwBpQ/Gd\nO3fCZrOl7fnbtm3D9OnT0/Y8giAIM0hYIwiCIAiCyGBIWCMIIuN46KGHcPr0adx9992orKzEiy++\niNGjRyMYDAIAbr31Vjz99NNYsGABKisrcffdd6OpqQkPPPAAJkyYgHnz5uHkyZPK/Q4dOoTbb78d\nkyZNwjXXXIN33nlHOffRRx/h+uuvR2VlJaZNm4aXX34ZHR0duOuuu1BbW4vKykpUVlaipqYGu3fv\nxvz58zFx4kRMnToVy5cvh9/vV+41evRorF27FrNnz0ZlZSWeeeYZHD9+HAsWLMCECRNw3333KdfL\nmrsXXngBl112GWbNmoW33norTS1MEESfIubuoQRBEL3AlVdeyTZv3swYkzbYrqioYIFAgDHG2MKF\nC9nVV1/Njh07xlpbW9l1113HZs+ezTZv3swCgQB76KGHlE2TvV4vmz59Olu3bh0LBAJs7969bNKk\nSezAgQOMMcamTJmibLbc3NzMqqqqGGOMbd26lU2bNk1Tpj179rCdO3eyQCDATpw4wa699lq2Zs0a\n5XxFRQW7++67WVtbG9u/fz+74IIL2G233caOHz+ulPP1119X7j9mzBj2i1/8gvl8PrZt2zZ28cUX\ns0OHDqWuUQmC6JOQZo0giD7JTTfdhGHDhiE3NxfTp0/H0KFDMXnyZNjtdlx77bXYt28fAGkz+8GD\nB2PevHmw2+0YO3YsrrnmGmzYsAEAYLfbcfDgQbS3tyM/Px8XXHCB6TMvvPBCjB8/Hna7HUOGDMH8\n+fPx2Wefaa658847kZOTg1GjRqGiogJTpkzB0KFDlXLK5ZK577774HQ6MWnSJMyYMQPvvvtukluK\nIIi+jr23C0AQBNEdiouLlX+7XC7N3263Gx0dHQCAU6dOYffu3Zg4caJyPhQK4YYbbgAAPPvss1i9\nejVWrlyJ0aNH44EHHkBlZSX3mUeOHMETTzyBqqoqdHZ2IhQKGYQ7q3K5XC7U19crf+fl5SE7O1v5\ne9CgQaitrU2oHQiC6P+QsEYQRL+mvLwcl156KdasWcM9P27cOKxevRqBQABr167Fj370I3z00UcQ\nBMFw7bJlyzB27FisXLkSOTk5eOWVV7Bx48Zul621tRUdHR2KwFZdXY1Ro0Z1+34EQfRPyAxKEERG\nUlxcjBMnTvT4PjNnzsTRo0fx5ptvIhAIIBAIYPfu3Th06BD8fj/eeusttLW1weFwwOPxQBSlabGo\nqAjNzc1oa2tT7uX1euHxeODxeHDo0CG8+uqrPS7fc889B7/fj88//xwffvghrr322h7fkyCI/gUJ\nawRBZCT/9m//htWrV2PixIk90l7l5OTg5ZdfxjvvvINp06Zh6tSp+OUvf6lEZf75z3/GrFmzMGHC\nBPzhD3/AU089BQAYOXIk/vmf/xlXX301Jk6ciJqaGjzyyCNYv349JkyYgJ/+9Ke4/vrre1TH4uJi\n5OXlYdq0aXjwwQexbNkyjBw5skf3JAii/yEwxlhvF4IgCOJsY9u2bXjooYfw8ccf93ZRCILIcEiz\nRhAEQRAEkcGQsEYQBEEQBJHBkBmUIAiCIAgigyHNGkEQBEEQRAZDwhpBEARBEEQGQ8IaQRAEQRBE\nBkPCGkEQBEEQRAZDwhpBEARBEEQG8/8B7ttPrBxbzO4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w7OUbaVteCZT",
        "colab_type": "text"
      },
      "source": [
        "#結果検証終了"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bCibsNemXF9_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sample_submission.loc[test_df['meter'] == 0, 'meter_reading'] = np.expm1(y_test0)\n",
        "sample_submission.loc[test_df['meter'] == 1, 'meter_reading'] = np.expm1(y_test1)\n",
        "sample_submission.loc[test_df['meter'] == 2, 'meter_reading'] = np.expm1(y_test2)\n",
        "sample_submission.loc[test_df['meter'] == 3, 'meter_reading'] = np.expm1(y_test3)\n",
        "sample_submission['meter_reading'] = sample_submission['meter_reading'].clip(lower=0)#1105ADD"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "23qOeYosQgA4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sample_submission.to_csv('1215_sample_submission_org.csv')\n",
        "gdrive_up('1215_sample_submission_org.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "fJqtb6bWeQGt",
        "outputId": "26fa92a3-ac40-448b-952f-63502b1ccc71",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 307
        }
      },
      "source": [
        "'''#トレーニングデータアンサンブル\n",
        "import pandas.tseries.offsets as offsets\n",
        "train_df = reduce_mem_usage(pd.read_csv('train.csv', parse_dates=['timestamp']))\n",
        "off = reduce_mem_usage(train_df)\n",
        "off2 = off\n",
        "off['timestamp'] = off['timestamp'] + offsets.MonthOffset(12)\n",
        "off2['timestamp'] = off2['timestamp'] + offsets.MonthOffset(24)\n",
        "off = pd.concat([off,off2])\n",
        "del off2\n",
        "off.timestamp = pd.to_datetime(off.timestamp)\n",
        "off.head()\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 616.95 MB\n",
            "Memory usage after optimization is: 289.19 MB\n",
            "Decreased by 53.1%\n",
            "Memory usage of dataframe is 289.19 MB\n",
            "Memory usage after optimization is: 289.19 MB\n",
            "Decreased by 0.0%\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>building_id</th>\n",
              "      <th>meter</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>meter_reading</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-01-01</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-01-01</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-01-01</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-01-01</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-01-01</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   building_id  meter  timestamp  meter_reading\n",
              "0            0      0 2019-01-01            0.0\n",
              "1            1      0 2019-01-01            0.0\n",
              "2            2      0 2019-01-01            0.0\n",
              "3            3      0 2019-01-01            0.0\n",
              "4            4      0 2019-01-01            0.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h_yB35spE2ce",
        "colab_type": "text"
      },
      "source": [
        "#Min-Max Round"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CjioNhijNoBL",
        "colab_type": "code",
        "outputId": "e1e8d5d2-7c5b-4018-bdfa-f6d9d851b1c6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        }
      },
      "source": [
        "'''\n",
        "#Month Round　効果なし\n",
        "gdrive_dl('1210_Bestsubmission.csv')\n",
        "sample_submission = pd.read_csv('1210_Bestsubmission.csv') \n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "preprocess(test_df)\n",
        "test_df = pd.merge(test_df,sample_submission, on=['row_id'],how='left')\n",
        "test_df = test_df[['row_id','building_id','month','meter','meter_reading']]\n",
        "max = train_df.groupby(['building_id','meter','month']).max().reset_index()[['building_id','meter','month', 'meter_reading']]\n",
        "max = max.rename(columns={'meter_reading': 'meter_reading_max'})\n",
        "min = train_df.groupby(['building_id','meter','month']).min().reset_index()[['building_id','meter','month', 'meter_reading']]\n",
        "min = min.rename(columns={'meter_reading': 'meter_reading_min'})\n",
        "maxmin = pd.merge(max,min,how='inner',on=['building_id','meter','month'])\n",
        "del max ,min\n",
        "print(test_df.shape)\n",
        "test_df = pd.merge(test_df,maxmin,how='left',on=['building_id','meter','month'])\n",
        "print(test_df.shape)\n",
        "test_df.loc[test_df['meter_reading_max'].isnull(),'meter_reading_max'] = test_df['meter_reading']\n",
        "test_df.loc[test_df['meter_reading_min'].isnull(),'meter_reading_min'] = test_df['meter_reading']\n",
        "print(len(test_df[test_df['meter_reading_max'] < test_df['meter_reading']]))\n",
        "print(len(test_df[test_df['meter_reading'] < test_df['meter_reading_min']]))\n",
        "test_df.loc[test_df['meter_reading_max'] < test_df['meter_reading'], 'meter_reading'] = test_df['meter_reading_max']\n",
        "test_df.loc[test_df['meter_reading'] < test_df['meter_reading_min'], 'meter_reading'] = test_df['meter_reading_min']\n",
        "test_df = test_df[['row_id','meter_reading']]\n",
        "sample_submission = sample_submission.merge(test_df,how='inner',on='row_id')\n",
        "sample_submission = sample_submission.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "sample_submission = sample_submission[['row_id','meter_reading']]\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[1210_Bestsubmission.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n",
            "(41697600, 5)\n",
            "(41697600, 7)\n",
            "1449368\n",
            "1681806\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "720af5UPAc-o",
        "colab_type": "code",
        "outputId": "fae7c120-d330-4330-eac4-493cc518b947",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 141
        }
      },
      "source": [
        "'''\n",
        "#Dayofweek Round 1212　効果なし\n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "preprocess(test_df)\n",
        "test_df = pd.merge(test_df,sample_submission, on=['row_id'],how='left')\n",
        "test_df = test_df[['row_id','building_id','month','dayofweek','meter','meter_reading']]\n",
        "max = train_df.groupby(['building_id','meter','month','dayofweek']).max().reset_index()[['building_id','meter','month','dayofweek', 'meter_reading']]\n",
        "max = max.rename(columns={'meter_reading': 'meter_reading_max'})\n",
        "min = train_df.groupby(['building_id','meter','month','dayofweek']).min().reset_index()[['building_id','meter','month','dayofweek', 'meter_reading']]\n",
        "min = min.rename(columns={'meter_reading': 'meter_reading_min'})\n",
        "maxmin = pd.merge(max,min,how='inner',on=['building_id','meter','month','dayofweek'])\n",
        "del max ,min\n",
        "print(test_df.shape)\n",
        "test_df = pd.merge(test_df,maxmin,how='left',on=['building_id','meter','month','dayofweek'])\n",
        "print(test_df.shape)\n",
        "test_df.loc[test_df['meter_reading_max'].isnull(),'meter_reading_max'] = test_df['meter_reading']\n",
        "test_df.loc[test_df['meter_reading_min'].isnull(),'meter_reading_min'] = test_df['meter_reading']\n",
        "print(len(test_df[test_df['meter_reading_max'] < test_df['meter_reading']]))\n",
        "print(len(test_df[test_df['meter_reading'] < test_df['meter_reading_min']]))\n",
        "test_df.loc[test_df['meter_reading_max'] < test_df['meter_reading'], 'meter_reading'] = test_df['meter_reading_max']\n",
        "test_df.loc[test_df['meter_reading'] < test_df['meter_reading_min'], 'meter_reading'] = test_df['meter_reading_min']\n",
        "test_df = test_df[['row_id','meter_reading']]\n",
        "sample_submission = sample_submission.merge(test_df,how='inner',on='row_id')\n",
        "sample_submission = sample_submission.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "sample_submission = sample_submission[['row_id','meter_reading']]\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n",
            "(41697600, 6)\n",
            "(41697600, 8)\n",
            "2662576\n",
            "4211225\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2o7wyQWD2sYq",
        "colab_type": "text"
      },
      "source": [
        "#Training Validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cQH7efKEBrmL",
        "colab_type": "code",
        "outputId": "2cfdb0aa-1bb3-410d-d228-da3986529035",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 342
        }
      },
      "source": [
        "'''660 662 666 667 669　677 トレーニングアンサンブルと一部の予測値を別のサブミッションファイルで上書き\n",
        "gdrive_dl('1210_Bestsubmission.csv')\n",
        "sample_submission = pd.read_csv('1210_Bestsubmission.csv') \n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test_df['date'] = test_df['timestamp'].dt.date\n",
        "preprocess_ex(test_df)\n",
        "\n",
        "test_df = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "test_df = test_df[['row_id','building_id','meter','meter_reading','hour','month','day']]\n",
        "\n",
        "gdrive_dl('halfhalf.csv')\n",
        "sample_submission = pd.read_csv('halfhalf.csv') \n",
        "test_df = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "test_df = reduce_mem_usage(test_df)\n",
        "test_df.head()\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[1210_Bestsubmission.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n",
            "GoogleドライブからColab環境への[halfhalf.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 2346.19 MB\n",
            "Memory usage after optimization is: 1033.91 MB\n",
            "Decreased by 55.9%\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>row_id</th>\n",
              "      <th>meter_reading_x</th>\n",
              "      <th>building_id</th>\n",
              "      <th>meter</th>\n",
              "      <th>meter_reading_y</th>\n",
              "      <th>hour</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>92.729485</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>173.370300</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>41.220722</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>53.512699</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>2.686540</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>6.143000</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>139.413605</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>101.701500</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>620.063782</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>1141.240723</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   row_id  meter_reading_x  building_id  ...  hour  month  day\n",
              "0       0        92.729485            0  ...     0      1    1\n",
              "1       1        41.220722            1  ...     0      1    1\n",
              "2       2         2.686540            2  ...     0      1    1\n",
              "3       3       139.413605            3  ...     0      1    1\n",
              "4       4       620.063782            4  ...     0      1    1\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1A5utiXP6aJ0",
        "colab_type": "code",
        "outputId": "afa0c334-df38-4287-abc1-f42d24ab8740",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 253
        }
      },
      "source": [
        "'''\n",
        "#外れ値の除外終了まで\n",
        "preprocess_ex(train_df)\n",
        "train_df = train_df[['building_id','meter','meter_reading','hour','month','day']]\n",
        "train_df['building_id'] = train_df['building_id'].astype(np.int64)\n",
        "train_df['meter'] = train_df['meter'].astype(np.int64)\n",
        "train_df = train_df.rename(columns={'meter_reading': 'base'})\n",
        "train_df = reduce_mem_usage(train_df)\n",
        "test_df = pd.merge(test_df,train_df,how ='left',on=['building_id','meter','hour','day','month'])\n",
        "test_df.loc[test_df['base'] < 0,'base'] = test_df['meter_reading_x']\n",
        "#print(len(test_df.loc[abs(test_df['base'] - test_df['meter_reading_x']) > abs(test_df['base'] - test_df['meter_reading_y'])))\n",
        "test_df.loc[abs(test_df['base'] - test_df['meter_reading_x']) > abs(test_df['base'] - test_df['meter_reading_y']),'meter_reading_x'] =  test_df['meter_reading_y']\n",
        "test_df = test_df[['row_id','meter_reading_x']]\n",
        "test_df = test_df.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "test_df.tail()\n",
        "test_df = test_df.drop_duplicates(subset='row_id',keep='last')\n",
        "sample_submission = test_df\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 1246.35 MB\n",
            "Memory usage after optimization is: 400.61 MB\n",
            "Decreased by 67.9%\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>row_id</th>\n",
              "      <th>meter_reading_x</th>\n",
              "      <th>building_id</th>\n",
              "      <th>meter</th>\n",
              "      <th>meter_reading_y</th>\n",
              "      <th>hour</th>\n",
              "      <th>month</th>\n",
              "      <th>day</th>\n",
              "      <th>base</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>92.729485</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>173.370300</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>41.220722</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>53.512699</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>2.686540</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>6.143000</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>139.413605</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>101.701500</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>620.063782</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>1141.240723</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   row_id  meter_reading_x  building_id  meter  ...  hour  month  day  base\n",
              "0       0        92.729485            0      0  ...     0      1    1   NaN\n",
              "1       1        41.220722            1      0  ...     0      1    1   NaN\n",
              "2       2         2.686540            2      0  ...     0      1    1   NaN\n",
              "3       3       139.413605            3      0  ...     0      1    1   NaN\n",
              "4       4       620.063782            4      0  ...     0      1    1   NaN\n",
              "\n",
              "[5 rows x 9 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D2-gPSGaDvFt",
        "colab_type": "text"
      },
      "source": [
        "#予測が大外れの値を別のサブミッションファイルで上書き\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PjZzb3bJFhMG",
        "colab_type": "code",
        "outputId": "d770492a-a9f1-4f2f-f177-c396b9ccdd87",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        }
      },
      "source": [
        "#線形回帰が大外れの箇所の置換 1072\n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test_df['date'] = test_df['timestamp'].dt.date\n",
        "preprocess_ex(test_df)\n",
        "gdrive_dl('halfhalf.csv')\n",
        "sample_submission = pd.read_csv('halfhalf.csv') \n",
        "test_df = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "test_df = reduce_mem_usage(test_df)\n",
        "test_df = test_df.loc[((test_df['building_id'].isin([1072,1079,1098,1305]) & (test_df['meter'] == 2)) | (test_df['building_id'].isin([1219]) & (test_df['meter'] == 1)) | ((test_df['building_id'].isin([1099]) & (test_df['meter'] == 0))))]\n",
        "test_df = test_df[['row_id','meter_reading']]\n",
        "gdrive_dl('0.956BestSubmission.csv')\n",
        "sample_submission = pd.read_csv('0.956BestSubmission.csv') \n",
        "sample_submission = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "sample_submission.loc[~sample_submission['meter_reading_y'].isnull(),'meter_reading_x'] = sample_submission[~sample_submission['meter_reading_y'].isnull()]['meter_reading_y']\n",
        "sample_submission = sample_submission.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "del sample_submission['meter_reading_y']"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n",
            "GoogleドライブからColab環境への[halfhalf.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 3022.21 MB\n",
            "Memory usage after optimization is: 1471.36 MB\n",
            "Decreased by 51.3%\n",
            "GoogleドライブからColab環境への[0.957BestSubmission.csv]ファイル取り込みが完了しました.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_mZplFhlkTEU",
        "colab_type": "code",
        "outputId": "54fc6c22-5e15-410f-94b8-a7a0296701fb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        }
      },
      "source": [
        "#Half Half\n",
        "'''\n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "test_df['date'] = test_df['timestamp'].dt.date\n",
        "preprocess_ex(test_df)\n",
        "gdrive_dl('halfhalf.csv')\n",
        "sample_submission = pd.read_csv('halfhalf.csv') \n",
        "test_df = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "test_df = reduce_mem_usage(test_df)\n",
        "test_df = test_df.loc[(test_df['building_id'].isin([660,662,660,662,666,667,669,677,683,686,688,690,696,697,698,700,703,704,705,706,709,710,711,712,713,715,716,717,718,719,720,722,725,728,729,731,734,737,747,748])) & (test_df['meter'] == 0)]\n",
        "test_df = test_df[['row_id','meter_reading']]\n",
        "gdrive_dl('1210_Bestsubmission.csv')\n",
        "sample_submission = pd.read_csv('1210_Bestsubmission.csv') \n",
        "sample_submission = pd.merge(sample_submission,test_df,how ='left',on='row_id')\n",
        "sample_submission.loc[~sample_submission['meter_reading_y'].isnull(),'meter_reading_x'] = sample_submission[~sample_submission['meter_reading_y'].isnull()]['meter_reading_y']\n",
        "sample_submission = sample_submission.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "del sample_submission['meter_reading_y']\n",
        "'''"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n",
            "GoogleドライブからColab環境への[halfhalf.csv]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 3022.21 MB\n",
            "Memory usage after optimization is: 1471.36 MB\n",
            "Decreased by 51.3%\n",
            "GoogleドライブからColab環境への[1210_Bestsubmission.csv]ファイル取り込みが完了しました.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OBX7-OEhjWsU",
        "colab_type": "code",
        "outputId": "05d7c215-5801-49c9-9c2d-24179cef02d2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 411
        }
      },
      "source": [
        "sample_submission"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>row_id</th>\n",
              "      <th>meter_reading</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>173.3703</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>53.5127</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>6.1430</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>101.7015</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>1141.2407</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697595</th>\n",
              "      <td>41697595</td>\n",
              "      <td>6.5129</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697596</th>\n",
              "      <td>41697596</td>\n",
              "      <td>5.1714</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697597</th>\n",
              "      <td>41697597</td>\n",
              "      <td>0.3910</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697598</th>\n",
              "      <td>41697598</td>\n",
              "      <td>174.7597</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41697599</th>\n",
              "      <td>41697599</td>\n",
              "      <td>3.7278</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>41697600 rows × 2 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            row_id  meter_reading\n",
              "0                0       173.3703\n",
              "1                1        53.5127\n",
              "2                2         6.1430\n",
              "3                3       101.7015\n",
              "4                4      1141.2407\n",
              "...            ...            ...\n",
              "41697595  41697595         6.5129\n",
              "41697596  41697596         5.1714\n",
              "41697597  41697597         0.3910\n",
              "41697598  41697598       174.7597\n",
              "41697599  41697599         3.7278\n",
              "\n",
              "[41697600 rows x 2 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 246
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tmjBQap9LVSE",
        "colab_type": "text"
      },
      "source": [
        "#公開リークデータ上書き　\n",
        "site0 1 2 4 15"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YlITGKR0DxQA",
        "colab_type": "code",
        "outputId": "36b985a1-0b33-4fa7-f9bf-dceed8b7616b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        }
      },
      "source": [
        "import feather\n",
        "gdrive_dl('leak.feather')\n",
        "test_df = reduce_mem_usage(pd.read_csv('test.csv', parse_dates=['timestamp']))\n",
        "#データフレームの読み込み\n",
        "leak_df = feather.read_dataframe('leak.feather')\n",
        "leak_df.head()\n",
        "\n",
        "leak_df = pd.merge(leak_df,test_df, on = ['building_id', 'meter', 'timestamp'], how = \"inner\")\n",
        "leak_df = leak_df[['row_id','meter_reading']]\n",
        "\n",
        "sample_submission = pd.merge(sample_submission,leak_df, on ='row_id',how='left')\n",
        "sample_submission = sample_submission.rename(columns={'meter_reading_x': 'meter_reading'})\n",
        "sample_submission.loc[sample_submission['meter_reading_y'] >= -1,'meter_reading'] =sample_submission['meter_reading_y']\n",
        "sample_submission = sample_submission[['row_id','meter_reading']]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GoogleドライブからColab環境への[leak.feather]ファイル取り込みが完了しました.\n",
            "Memory usage of dataframe is 1272.51 MB\n",
            "Memory usage after optimization is: 596.49 MB\n",
            "Decreased by 53.1%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b-VnM303XI5N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sample_submission.to_csv('1214_2submission.csv', index=False, float_format='%.4f')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B8GwiXCF0tBB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gdrive_up('1214_2submission.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gzAU6k5I914k",
        "colab_type": "code",
        "outputId": "f92ed769-3722-49aa-aabd-7fd04dafda17",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "!kaggle competitions submit -c ashrae-energy-prediction -f 1214_2submission.csv -m \"liner re bug fix\""
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)\n",
            "100% 679M/679M [00:33<00:00, 21.5MB/s]\n",
            "Successfully submitted to ASHRAE - Great Energy Predictor III"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2abxkO-zUgjL",
        "colab_type": "code",
        "outputId": "9d9c311b-4f90-48fb-fe2c-30dad1b0a737",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 426
        }
      },
      "source": [
        "!kaggle competitions submissions -c ashrae-energy-prediction"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)\n",
            "fileName                     date                 description                  status    publicScore  privateScore  \n",
            "---------------------------  -------------------  ---------------------------  --------  -----------  ------------  \n",
            "1214_2submission.csv         2019-12-14 11:37:15  liner re bug fix             complete  0.956        None          \n",
            "1214_submission.csv          2019-12-14 05:17:23  training val                 complete  0.999        None          \n",
            "1213_2submission.csv         2019-12-13 14:44:34  half                         complete  0.961        None          \n",
            "1213_submission.csv          2019-12-13 14:06:18  shuffle                      complete  0.961        None          \n",
            "1212_submission.csv          2019-12-12 12:00:19  dayofweek round              complete  0.958        None          \n",
            "1211_2submission.csv         2019-12-11 14:26:05  training unsamble            complete  0.977        None          \n",
            "1211_submission.csv          2019-12-11 13:28:25  leak_feather replace         complete  0.957        None          \n",
            "1210_2submission.csv         2019-12-10 14:07:14  leak_feather replace         complete  0.957        None          \n",
            "1210submission.csv           2019-12-10 13:25:27  min max round                complete  0.957        None          \n",
            "12092sample_submission.csv   2019-12-09 22:36:52  site4 leak                   complete  0.967        None          \n",
            "1209sample_submission.csv    2019-12-09 12:37:12  site4 leak                   complete  0.957        None          \n",
            "1208_2sample_submission.csv  2019-12-08 14:01:00  unsamble                     complete  0.957        None          \n",
            "1207_2sample_submission.csv  2019-12-08 13:50:17  unsamble                     complete  2.147        None          \n",
            "1207_2sample_submission.csv  2019-12-07 12:43:36  unsamble                     complete  0.959        None          \n",
            "1207sample_submission.csv    2019-12-07 12:19:44  auto outer drop              complete  0.986        None          \n",
            "1203_2sample_submission.csv  2019-12-06 12:33:47  0.97 unsamble                complete  0.961        None          \n",
            "1206_sample_submission.csv   2019-12-06 11:35:27  Another 1.08 LB no leakBase  complete  0.985        None          \n",
            "1206_sample_submission.csv   2019-12-06 11:12:01  Another 1.08 LB no leakBase  error     None         None          \n",
            "1206_sample_submission.csv   2019-12-06 10:44:36  Another 1.08 LB no leakBase  error     None         None          \n",
            "1203_2sample_submission.csv  2019-12-03 13:22:27  0.97 unsamble                complete  0.960        None          \n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}