include notebook

Author: mpharrigan

dev_tools/costing-report-card.ipynb

@@ -0,0 +1,230 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "1c2b1422-5a3b-401c-88fb-0c0b723c6633",
+   "metadata": {},
+   "source": [
+    "# Timing Cost Computation\n",
+    "\n",
+    "This notebook goes through each bloq example and calls `report_on_cost_timings`, which currently times how long it takes to do the `QubitCount` cost key. This uses the `ExecuteWithTimeout` fixture."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9ae691c8-a5f8-4b66-a99c-7d82ad8ffc2e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from qualtran_dev_tools.execute_with_timeout import ExecuteWithTimeout\n",
+    "from qualtran_dev_tools.bloq_report_card import report_on_cost_timings\n",
+    "from qualtran_dev_tools.bloq_finder import get_bloq_examples"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3fcfb561-6cb9-4891-8157-547dbfa5502b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "bes = get_bloq_examples()\n",
+    "\n",
+    "# Imports to exclude certain bloqs, see following comment\n",
+    "from qualtran.bloqs.multiplexers.apply_gate_to_lth_target import ApplyGateToLthQubit"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cea586d9-55b3-4ee8-b255-ea88bdff75f4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "exec = ExecuteWithTimeout(timeout=20., max_workers=4)\n",
+    "for i, be in enumerate(bes):\n",
+    "\n",
+    "    if be.bloq_cls == ApplyGateToLthQubit:\n",
+    "        # This bloq uses a lambda function as one of its attributes, which\n",
+    "        # can't be pickled and used with multiprocessing.\n",
+    "        continue\n",
+    "    \n",
+    "    exec.submit(report_on_cost_timings, kwargs=dict(name=be.name, cls_name=be.bloq_cls.__name__, bloq=be.make()))\n",
+    "\n",
+    "records = []\n",
+    "while exec.work_to_be_done:\n",
+    "    kwargs, record = exec.next_result()\n",
+    "    print('\\r', f'{exec.work_to_be_done:5d} remaining', end='', flush=True)\n",
+    "    \n",
+    "    if record is None:\n",
+    "        records.append({\n",
+    "            'name': kwargs['name'],\n",
+    "            'cls': kwargs['cls_name'],\n",
+    "            'err': 'Timeout',\n",
+    "        })\n",
+    "    else:\n",
+    "        records.append(record)\n",
+    "\n",
+    "import pandas as pd\n",
+    "df = pd.DataFrame(records)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "abf0d3f1-bc5f-48a6-9d9b-b9fa1bbeb8d1",
+   "metadata": {},
+   "source": [
+    "## Slowest\n",
+    "\n",
+    "This prints the total number of bloq examples considered and then summarizes the 5 slowest-to-compute bloq examples."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "86f259b0-ec70-4860-b76c-0df993ca7934",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(len(df))\n",
+    "df.sort_values(by='qubitcount_dur', ascending=False).head()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a30190dd-550d-45d4-a572-94fdaa4a2f10",
+   "metadata": {},
+   "source": [
+    "## Errors and timeouts\n",
+    "\n",
+    "These bloq examples either time-out or encounter errors in the qubit computation."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "513a3421-8f80-43ae-9449-89c8063da242",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df[df['qubitcount_dur'].isna()]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "605e8869-e77e-40b3-8455-07204b7b872c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for i, row in df[df['qubitcount_dur'].isna()].iterrows():\n",
+    "    print(\"### `{}`\".format(row['name']))\n",
+    "    print(\"{}\\n\".format(row[\"err\"]))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bceb65d1-d107-4c0d-8662-8ea80c00de16",
+   "metadata": {},
+   "source": [
+    "## Timeouts\n",
+    "\n",
+    "These examples specifically time out. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "043c3d7a-f4d4-486c-aa99-eb871e23df3b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df[df['err'] == 'Timeout']"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c7b87a8d-c4eb-4b7b-abce-66dfa31a0699",
+   "metadata": {},
+   "source": [
+    "## Investigation\n",
+    "\n",
+    "Individual bloq examples can be investigated. Strangely, hubbard_time_evolution_by_gqsp times out when run through the fixture but appears reasonably quick when run directly."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d0f486e8-8cbf-4907-a84d-9ff43cfcd43e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_bloq_example(name):\n",
+    "    results = [be for be in bes if be.name == name]\n",
+    "    \n",
+    "    if len(results) == 1:\n",
+    "        return results[0]\n",
+    "    if len(results) > 1:\n",
+    "        raise ValueError(\"Found more than one result for the query\")\n",
+    "    if len(results) == 0:\n",
+    "        raise KeyError(f\"The bloq example {name} was not found\")    "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0d01d939-fa0d-4934-8a56-24f1087cd232",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from qualtran.drawing import show_call_graph\n",
+    "be = get_bloq_example('hubbard_time_evolution_by_gqsp')\n",
+    "bloq = be.make()\n",
+    "show_call_graph(bloq, max_depth=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e402cd50-8015-4f14-9c6f-1a7755bf4936",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%timeit\n",
+    "from qualtran.resource_counting import get_cost_value, QubitCount\n",
+    "\n",
+    "get_cost_value(bloq, QubitCount())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "56391b2c-9381-480e-b204-e881fe0828ae",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}