{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9b87bbc-cde7-4671-baac-fc0b107bf120",
   "metadata": {},
   "source": [
    "# Data handling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d81a2182-64ab-4e4d-83c9-7a3310a79371",
   "metadata": {},
   "source": [
    "(howto-data-handling-to-gridded)=\n",
    "## Converting dataset coordinates from flat list to grid\n",
    "\n",
    "By default measurement control returnds data in a sparse format.\n",
    "The resulting dataset has a single dimension, and all settables and gettables share it, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f6cb2db-183f-4c3b-a764-201b9be92bf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import numpy as np\n",
    "from qcodes import ManualParameter, Parameter, validators\n",
    "\n",
    "from quantify_core.data.handling import set_datadir, to_gridded_dataset\n",
    "from quantify_core.measurement import MeasurementControl\n",
    "\n",
    "set_datadir(Path.home() / \"quantify-data\")\n",
    "\n",
    "time_a = ManualParameter(\n",
    "    name=\"time_a\",\n",
    "    label=\"Time A\",\n",
    "    unit=\"s\",\n",
    "    vals=validators.Numbers(),\n",
    "    initial_value=1,\n",
    ")\n",
    "time_b = ManualParameter(\n",
    "    name=\"time_b\",\n",
    "    label=\"Time B\",\n",
    "    unit=\"s\",\n",
    "    vals=validators.Numbers(),\n",
    "    initial_value=1,\n",
    ")\n",
    "signal = Parameter(\n",
    "    name=\"sig_a\",\n",
    "    label=\"Signal A\",\n",
    "    unit=\"V\",\n",
    "    get_cmd=lambda: np.exp(time_a()) + 0.5 * np.exp(time_b()),\n",
    ")\n",
    "\n",
    "meas_ctrl = MeasurementControl(\"meas_ctrl\")\n",
    "meas_ctrl.settables([time_a, time_b])\n",
    "meas_ctrl.gettables(signal)\n",
    "meas_ctrl.setpoints_grid([np.linspace(0, 5, 10), np.linspace(5, 0, 12)])\n",
    "dset = meas_ctrl.run(\"2D-single-float-valued-settable-gettable\")\n",
    "dset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cca91a1-680a-455e-8706-92a3a9dcb040",
   "metadata": {},
   "source": [
    "This format is very close to {class}`COO sparse matrix <scipy.sparse.coo_matrix>`, except that coordinates are not integer.\n",
    "\n",
    "If the initial data is gridded, it is more convenient to resotre this structure in the dataset for processing.\n",
    "There is a utility function {func}`quantify_core.data.handling.to_gridded_dataset` for that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f472609-6836-4f47-b595-f2392b8855d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "dset_grid = to_gridded_dataset(dset)\n",
    "dset_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fab9ac66-339a-41d0-b5df-0eac9c991050",
   "metadata": {},
   "source": [
    "We see that now dataset has two dimensions (`x0` and `x1`), that represent the initial grid for settables `time_a` and `time_b`.\n",
    "Frecuently this simplifies data processing and analysis a lot, for example, we can display the data straight away:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d87a5860-6065-45e0-9eb7-dc1642808a2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "dset_grid.y0.plot(cmap=\"viridis\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}