examples/notebooks/Pivoting Data from Wide to Long.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pivot_Longer : One function to cover transformations from wide to long form."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unpivoting(reshaping data from wide to long form) in Pandas is executed either through [pd.melt](https://pandas.pydata.org/docs/reference/api/pandas.melt.html), [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html), or [pd.DataFrame.stack](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.stack.html). However, there are scenarios where a few more steps are required to massage the data into the long form that we desire. Take the dataframe below, copied from [Stack Overflow](https://stackoverflow.com/questions/64061588/pandas-melt-multiple-columns-to-tabulate-a-dataset#64062002): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>M_start_date_1</th>\n",
       "      <th>M_end_date_1</th>\n",
       "      <th>M_start_date_2</th>\n",
       "      <th>M_end_date_2</th>\n",
       "      <th>F_start_date_1</th>\n",
       "      <th>F_end_date_1</th>\n",
       "      <th>F_start_date_2</th>\n",
       "      <th>F_end_date_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  M_start_date_1  M_end_date_1  M_start_date_2  M_end_date_2  \\\n",
       "0   1          201709        201905          202004        202005   \n",
       "1   2          201709        201905          202004        202005   \n",
       "2   3          201709        201905          202004        202005   \n",
       "\n",
       "   F_start_date_1  F_end_date_1  F_start_date_2  F_end_date_2  \n",
       "0          201803        201904          201912        202007  \n",
       "1          201803        201904          201912        202007  \n",
       "2          201803        201904          201912        202007  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"id\": [1, 2, 3],\n",
    "        \"M_start_date_1\": [201709, 201709, 201709],\n",
    "        \"M_end_date_1\": [201905, 201905, 201905],\n",
    "        \"M_start_date_2\": [202004, 202004, 202004],\n",
    "        \"M_end_date_2\": [202005, 202005, 202005],\n",
    "        \"F_start_date_1\": [201803, 201803, 201803],\n",
    "        \"F_end_date_1\": [201904, 201904, 201904],\n",
    "        \"F_start_date_2\": [201912, 201912, 201912],\n",
    "        \"F_end_date_2\": [202007, 202007, 202007],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a [beautiful solution](https://stackoverflow.com/a/64062027/7175713), from Stack Overflow : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>cod</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>3</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id cod   start     end\n",
       "0    1   M  201709  201905\n",
       "1    1   M  202004  202005\n",
       "2    1   F  201803  201904\n",
       "3    1   F  201912  202007\n",
       "4    2   M  201709  201905\n",
       "5    2   M  202004  202005\n",
       "6    2   F  201803  201904\n",
       "7    2   F  201912  202007\n",
       "8    3   M  201709  201905\n",
       "9    3   M  202004  202005\n",
       "10   3   F  201803  201904\n",
       "11   3   F  201912  202007"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.set_index(\"id\")\n",
    "df1.columns = df1.columns.str.split(\"_\", expand=True)\n",
    "df1 = (\n",
    "    df1.stack(level=[0, 2, 3], future_stack=True)\n",
    "    .sort_index(level=[0, 1], ascending=[True, False])\n",
    "    .reset_index(level=[2, 3], drop=True)\n",
    "    .sort_index(axis=1, ascending=False)\n",
    "    .rename_axis([\"id\", \"cod\"])\n",
    "    .reset_index()\n",
    ")\n",
    "\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We propose an alternative, based on [pandas melt](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html) and [concat](https://pandas.pydata.org/docs/reference/api/pandas.concat.html), that abstracts the reshaping mechanism, allows the user to focus on the task, can be applied to other scenarios,  and is chainable : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>M_start_date_1</th>\n",
       "      <th>M_end_date_1</th>\n",
       "      <th>M_start_date_2</th>\n",
       "      <th>M_end_date_2</th>\n",
       "      <th>F_start_date_1</th>\n",
       "      <th>F_end_date_1</th>\n",
       "      <th>F_start_date_2</th>\n",
       "      <th>F_end_date_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  M_start_date_1  M_end_date_1  M_start_date_2  M_end_date_2  \\\n",
       "0   1          201709        201905          202004        202005   \n",
       "1   2          201709        201905          202004        202005   \n",
       "2   3          201709        201905          202004        202005   \n",
       "\n",
       "   F_start_date_1  F_end_date_1  F_start_date_2  F_end_date_2  \n",
       "0          201803        201904          201912        202007  \n",
       "1          201803        201904          201912        202007  \n",
       "2          201803        201904          201912        202007  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>cod</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>201709</td>\n",
       "      <td>201905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>3</td>\n",
       "      <td>M</td>\n",
       "      <td>202004</td>\n",
       "      <td>202005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>F</td>\n",
       "      <td>201803</td>\n",
       "      <td>201904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>3</td>\n",
       "      <td>F</td>\n",
       "      <td>201912</td>\n",
       "      <td>202007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id cod   start     end\n",
       "0    1   M  201709  201905\n",
       "1    1   M  202004  202005\n",
       "2    1   F  201803  201904\n",
       "3    1   F  201912  202007\n",
       "4    2   M  201709  201905\n",
       "5    2   M  202004  202005\n",
       "6    2   F  201803  201904\n",
       "7    2   F  201912  202007\n",
       "8    3   M  201709  201905\n",
       "9    3   M  202004  202005\n",
       "10   3   F  201803  201904\n",
       "11   3   F  201912  202007"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = df.pivot_longer(\n",
    "    index=\"id\",\n",
    "    names_to=(\"cod\", \".value\", \"date\", \"num\"),\n",
    "    names_sep=\"_\",\n",
    "    sort_by_appearance=True,\n",
    ").drop(columns=[\"date\", \"num\"])\n",
    "\n",
    "\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns = [\"id\", \"cod\", \"start\", \"end\"]\n",
    "df1 = df1.sort_values(columns, ignore_index=True)\n",
    "result = result.sort_values(columns, ignore_index=True)\n",
    "df1.equals(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html#janitor.pivot_longer) is a combination of ideas from R's [tidyr](https://tidyr.tidyverse.org/reference/pivot_longer.html) and [data.table](https://rdatatable.gitlab.io/data.table/) and is built on the powerful pandas' [melt](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html) and [concat](https://pandas.pydata.org/docs/reference/api/pandas.concat.html) functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html#janitor.pivot_longer) can melt dataframes easily; It is just a wrapper around pandas' [melt](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html).\n",
    "\n",
    "[Source Data](https://pandas.pydata.org/pandas-docs/stable/user_guide/reshaping.html#reshaping-by-melt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>last</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">person</th>\n",
       "      <th>A</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>5.5</td>\n",
       "      <td>130</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>6.0</td>\n",
       "      <td>150</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         first last  height  weight\n",
       "person A  John  Doe     5.5     130\n",
       "       B  Mary   Bo     6.0     150"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index = pd.MultiIndex.from_tuples([(\"person\", \"A\"), (\"person\", \"B\")])\n",
    "\n",
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"first\": [\"John\", \"Mary\"],\n",
    "        \"last\": [\"Doe\", \"Bo\"],\n",
    "        \"height\": [5.5, 6.0],\n",
    "        \"weight\": [130, 150],\n",
    "    },\n",
    "    index=index,\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>last</th>\n",
       "      <th>variable</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>height</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>height</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>weight</td>\n",
       "      <td>130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>weight</td>\n",
       "      <td>150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  first last variable  value\n",
       "0  John  Doe   height    5.5\n",
       "1  Mary   Bo   height    6.0\n",
       "2  John  Doe   weight  130.0\n",
       "3  Mary   Bo   weight  150.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=[\"first\", \"last\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want the data unpivoted in order of appearance, you can set `sort_by_appearance` to `True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>last</th>\n",
       "      <th>variable</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>height</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>weight</td>\n",
       "      <td>130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>height</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>weight</td>\n",
       "      <td>150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  first last variable  value\n",
       "0  John  Doe   height    5.5\n",
       "1  John  Doe   weight  130.0\n",
       "2  Mary   Bo   height    6.0\n",
       "3  Mary   Bo   weight  150.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=[\"first\", \"last\"], sort_by_appearance=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you wish to reuse the original index, you can set `ignore_index` to `False`; note that the index labels will be repeated as necessary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>last</th>\n",
       "      <th>variable</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"4\" valign=\"top\">person</th>\n",
       "      <th>A</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>height</td>\n",
       "      <td>5.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>height</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <td>John</td>\n",
       "      <td>Doe</td>\n",
       "      <td>weight</td>\n",
       "      <td>130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Bo</td>\n",
       "      <td>weight</td>\n",
       "      <td>150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         first last variable  value\n",
       "person A  John  Doe   height    5.5\n",
       "       B  Mary   Bo   height    6.0\n",
       "       A  John  Doe   weight  130.0\n",
       "       B  Mary   Bo   weight  150.0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=[\"first\", \"last\"], ignore_index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also unpivot MultiIndex columns, the same way you would with pandas' [melt](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html#pandas.melt):\n",
    "\n",
    "[Source Data](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html#pandas.melt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>D</th>\n",
       "      <th>E</th>\n",
       "      <th>F</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A  B  C\n",
       "   D  E  F\n",
       "0  a  1  2\n",
       "1  b  3  4\n",
       "2  c  5  6"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"A\": {0: \"a\", 1: \"b\", 2: \"c\"},\n",
    "        \"B\": {0: 1, 1: 3, 2: 5},\n",
    "        \"C\": {0: 2, 1: 4, 2: 6},\n",
    "    }\n",
    ")\n",
    "df.columns = [list(\"ABC\"), list(\"DEF\")]\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>(A, D)</th>\n",
       "      <th>variable_0</th>\n",
       "      <th>variable_1</th>\n",
       "      <th>num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>a</td>\n",
       "      <td>C</td>\n",
       "      <td>F</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>b</td>\n",
       "      <td>C</td>\n",
       "      <td>F</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>c</td>\n",
       "      <td>C</td>\n",
       "      <td>F</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  (A, D) variable_0 variable_1  num\n",
       "0      a          B          E    1\n",
       "1      b          B          E    3\n",
       "2      c          B          E    5\n",
       "3      a          C          F    2\n",
       "4      b          C          F    4\n",
       "5      c          C          F    6"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=[(\"A\", \"D\")], values_to=\"num\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>(A, D)</th>\n",
       "      <th>variable_0</th>\n",
       "      <th>variable_1</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>B</td>\n",
       "      <td>E</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  (A, D) variable_0 variable_1  value\n",
       "0      a          B          E      1\n",
       "1      b          B          E      3\n",
       "2      c          B          E      5"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=[(\"A\", \"D\")], column_names=[(\"B\", \"E\")])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And just like [melt](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.melt.html#pandas.melt), you can unpivot on a specific level, with `column_level`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>variable</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "      <td>B</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>B</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>B</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A variable  value\n",
       "0  a        B      1\n",
       "1  b        B      3\n",
       "2  c        B      5"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=\"A\", column_names=\"B\", column_level=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that when unpivoting MultiIndex columns, you need to pass a list of tuples to the `index` or `column_names` parameters.\n",
    "\n",
    "\n",
    "Also, `names_sep` or `names_pattern` parameters (which we shall look at in subsequent examples) only work for single indexed columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can dynamically select columns, using regular expressions or some other form supported by [pyjanitor's](https://pyjanitor-devs.github.io/pyjanitor/) [select_columns](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.select_columns.html#janitor.select_columns), especially if it is a lot of column names, and you are *lazy* like me  😄"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>artist</th>\n",
       "      <th>track</th>\n",
       "      <th>time</th>\n",
       "      <th>date.entered</th>\n",
       "      <th>wk1</th>\n",
       "      <th>wk2</th>\n",
       "      <th>wk3</th>\n",
       "      <th>wk4</th>\n",
       "      <th>wk5</th>\n",
       "      <th>...</th>\n",
       "      <th>wk67</th>\n",
       "      <th>wk68</th>\n",
       "      <th>wk69</th>\n",
       "      <th>wk70</th>\n",
       "      <th>wk71</th>\n",
       "      <th>wk72</th>\n",
       "      <th>wk73</th>\n",
       "      <th>wk74</th>\n",
       "      <th>wk75</th>\n",
       "      <th>wk76</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>2 Pac</td>\n",
       "      <td>Baby Don't Cry (Keep...</td>\n",
       "      <td>4:22</td>\n",
       "      <td>2000-02-26</td>\n",
       "      <td>87</td>\n",
       "      <td>82.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>77.0</td>\n",
       "      <td>87.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000</td>\n",
       "      <td>2Ge+her</td>\n",
       "      <td>The Hardest Part Of ...</td>\n",
       "      <td>3:15</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>91</td>\n",
       "      <td>87.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Kryptonite</td>\n",
       "      <td>3:53</td>\n",
       "      <td>2000-04-08</td>\n",
       "      <td>81</td>\n",
       "      <td>70.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Loser</td>\n",
       "      <td>4:24</td>\n",
       "      <td>2000-10-21</td>\n",
       "      <td>76</td>\n",
       "      <td>76.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000</td>\n",
       "      <td>504 Boyz</td>\n",
       "      <td>Wobble Wobble</td>\n",
       "      <td>3:35</td>\n",
       "      <td>2000-04-15</td>\n",
       "      <td>57</td>\n",
       "      <td>34.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>312</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yankee Grey</td>\n",
       "      <td>Another Nine Minutes</td>\n",
       "      <td>3:10</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>86</td>\n",
       "      <td>83.0</td>\n",
       "      <td>77.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>83.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>313</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yearwood, Trisha</td>\n",
       "      <td>Real Live Woman</td>\n",
       "      <td>3:55</td>\n",
       "      <td>2000-04-01</td>\n",
       "      <td>85</td>\n",
       "      <td>83.0</td>\n",
       "      <td>83.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>314</th>\n",
       "      <td>2000</td>\n",
       "      <td>Ying Yang Twins</td>\n",
       "      <td>Whistle While You Tw...</td>\n",
       "      <td>4:19</td>\n",
       "      <td>2000-03-18</td>\n",
       "      <td>95</td>\n",
       "      <td>94.0</td>\n",
       "      <td>91.0</td>\n",
       "      <td>85.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>315</th>\n",
       "      <td>2000</td>\n",
       "      <td>Zombie Nation</td>\n",
       "      <td>Kernkraft 400</td>\n",
       "      <td>3:30</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>99</td>\n",
       "      <td>99.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>316</th>\n",
       "      <td>2000</td>\n",
       "      <td>matchbox twenty</td>\n",
       "      <td>Bent</td>\n",
       "      <td>4:12</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>60</td>\n",
       "      <td>37.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>317 rows × 81 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     year            artist                    track  time date.entered  wk1  \\\n",
       "0    2000             2 Pac  Baby Don't Cry (Keep...  4:22   2000-02-26   87   \n",
       "1    2000           2Ge+her  The Hardest Part Of ...  3:15   2000-09-02   91   \n",
       "2    2000      3 Doors Down               Kryptonite  3:53   2000-04-08   81   \n",
       "3    2000      3 Doors Down                    Loser  4:24   2000-10-21   76   \n",
       "4    2000          504 Boyz            Wobble Wobble  3:35   2000-04-15   57   \n",
       "..    ...               ...                      ...   ...          ...  ...   \n",
       "312  2000       Yankee Grey     Another Nine Minutes  3:10   2000-04-29   86   \n",
       "313  2000  Yearwood, Trisha          Real Live Woman  3:55   2000-04-01   85   \n",
       "314  2000   Ying Yang Twins  Whistle While You Tw...  4:19   2000-03-18   95   \n",
       "315  2000     Zombie Nation            Kernkraft 400  3:30   2000-09-02   99   \n",
       "316  2000   matchbox twenty                     Bent  4:12   2000-04-29   60   \n",
       "\n",
       "      wk2   wk3   wk4   wk5  ...  wk67  wk68  wk69  wk70  wk71  wk72  wk73  \\\n",
       "0    82.0  72.0  77.0  87.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "1    87.0  92.0   NaN   NaN  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "2    70.0  68.0  67.0  66.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "3    76.0  72.0  69.0  67.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "4    34.0  25.0  17.0  17.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "..    ...   ...   ...   ...  ...   ...   ...   ...   ...   ...   ...   ...   \n",
       "312  83.0  77.0  74.0  83.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "313  83.0  83.0  82.0  81.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "314  94.0  91.0  85.0  84.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "315  99.0   NaN   NaN   NaN  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "316  37.0  29.0  24.0  22.0  ...   NaN   NaN   NaN   NaN   NaN   NaN   NaN   \n",
       "\n",
       "     wk74  wk75  wk76  \n",
       "0     NaN   NaN   NaN  \n",
       "1     NaN   NaN   NaN  \n",
       "2     NaN   NaN   NaN  \n",
       "3     NaN   NaN   NaN  \n",
       "4     NaN   NaN   NaN  \n",
       "..    ...   ...   ...  \n",
       "312   NaN   NaN   NaN  \n",
       "313   NaN   NaN   NaN  \n",
       "314   NaN   NaN   NaN  \n",
       "315   NaN   NaN   NaN  \n",
       "316   NaN   NaN   NaN  \n",
       "\n",
       "[317 rows x 81 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = \"https://raw.githubusercontent.com/tidyverse/tidyr/main/data-raw/billboard.csv\"\n",
    "df = pd.read_csv(url)\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/samuel.oranyeli/pyjanitor/janitor/functions/select.py:508: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract.\n",
      "  bools = index.str.contains(arg, na=False, regex=True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>artist</th>\n",
       "      <th>track</th>\n",
       "      <th>time</th>\n",
       "      <th>date.entered</th>\n",
       "      <th>week</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>2 Pac</td>\n",
       "      <td>Baby Don't Cry (Keep...</td>\n",
       "      <td>4:22</td>\n",
       "      <td>2000-02-26</td>\n",
       "      <td>wk1</td>\n",
       "      <td>87.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000</td>\n",
       "      <td>2Ge+her</td>\n",
       "      <td>The Hardest Part Of ...</td>\n",
       "      <td>3:15</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>wk1</td>\n",
       "      <td>91.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Kryptonite</td>\n",
       "      <td>3:53</td>\n",
       "      <td>2000-04-08</td>\n",
       "      <td>wk1</td>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Loser</td>\n",
       "      <td>4:24</td>\n",
       "      <td>2000-10-21</td>\n",
       "      <td>wk1</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000</td>\n",
       "      <td>504 Boyz</td>\n",
       "      <td>Wobble Wobble</td>\n",
       "      <td>3:35</td>\n",
       "      <td>2000-04-15</td>\n",
       "      <td>wk1</td>\n",
       "      <td>57.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24087</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yankee Grey</td>\n",
       "      <td>Another Nine Minutes</td>\n",
       "      <td>3:10</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24088</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yearwood, Trisha</td>\n",
       "      <td>Real Live Woman</td>\n",
       "      <td>3:55</td>\n",
       "      <td>2000-04-01</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24089</th>\n",
       "      <td>2000</td>\n",
       "      <td>Ying Yang Twins</td>\n",
       "      <td>Whistle While You Tw...</td>\n",
       "      <td>4:19</td>\n",
       "      <td>2000-03-18</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24090</th>\n",
       "      <td>2000</td>\n",
       "      <td>Zombie Nation</td>\n",
       "      <td>Kernkraft 400</td>\n",
       "      <td>3:30</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24091</th>\n",
       "      <td>2000</td>\n",
       "      <td>matchbox twenty</td>\n",
       "      <td>Bent</td>\n",
       "      <td>4:12</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>24092 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       year            artist                    track  time date.entered  \\\n",
       "0      2000             2 Pac  Baby Don't Cry (Keep...  4:22   2000-02-26   \n",
       "1      2000           2Ge+her  The Hardest Part Of ...  3:15   2000-09-02   \n",
       "2      2000      3 Doors Down               Kryptonite  3:53   2000-04-08   \n",
       "3      2000      3 Doors Down                    Loser  4:24   2000-10-21   \n",
       "4      2000          504 Boyz            Wobble Wobble  3:35   2000-04-15   \n",
       "...     ...               ...                      ...   ...          ...   \n",
       "24087  2000       Yankee Grey     Another Nine Minutes  3:10   2000-04-29   \n",
       "24088  2000  Yearwood, Trisha          Real Live Woman  3:55   2000-04-01   \n",
       "24089  2000   Ying Yang Twins  Whistle While You Tw...  4:19   2000-03-18   \n",
       "24090  2000     Zombie Nation            Kernkraft 400  3:30   2000-09-02   \n",
       "24091  2000   matchbox twenty                     Bent  4:12   2000-04-29   \n",
       "\n",
       "       week  value  \n",
       "0       wk1   87.0  \n",
       "1       wk1   91.0  \n",
       "2       wk1   81.0  \n",
       "3       wk1   76.0  \n",
       "4       wk1   57.0  \n",
       "...     ...    ...  \n",
       "24087  wk76    NaN  \n",
       "24088  wk76    NaN  \n",
       "24089  wk76    NaN  \n",
       "24090  wk76    NaN  \n",
       "24091  wk76    NaN  \n",
       "\n",
       "[24092 rows x 7 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# unpivot all columns that start with 'wk'\n",
    "df.pivot_longer(column_names=re.compile(\"^(wk)\"), names_to=\"week\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>artist</th>\n",
       "      <th>track</th>\n",
       "      <th>time</th>\n",
       "      <th>date.entered</th>\n",
       "      <th>week</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>2 Pac</td>\n",
       "      <td>Baby Don't Cry (Keep...</td>\n",
       "      <td>4:22</td>\n",
       "      <td>2000-02-26</td>\n",
       "      <td>wk1</td>\n",
       "      <td>87.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000</td>\n",
       "      <td>2Ge+her</td>\n",
       "      <td>The Hardest Part Of ...</td>\n",
       "      <td>3:15</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>wk1</td>\n",
       "      <td>91.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Kryptonite</td>\n",
       "      <td>3:53</td>\n",
       "      <td>2000-04-08</td>\n",
       "      <td>wk1</td>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000</td>\n",
       "      <td>3 Doors Down</td>\n",
       "      <td>Loser</td>\n",
       "      <td>4:24</td>\n",
       "      <td>2000-10-21</td>\n",
       "      <td>wk1</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000</td>\n",
       "      <td>504 Boyz</td>\n",
       "      <td>Wobble Wobble</td>\n",
       "      <td>3:35</td>\n",
       "      <td>2000-04-15</td>\n",
       "      <td>wk1</td>\n",
       "      <td>57.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24087</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yankee Grey</td>\n",
       "      <td>Another Nine Minutes</td>\n",
       "      <td>3:10</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24088</th>\n",
       "      <td>2000</td>\n",
       "      <td>Yearwood, Trisha</td>\n",
       "      <td>Real Live Woman</td>\n",
       "      <td>3:55</td>\n",
       "      <td>2000-04-01</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24089</th>\n",
       "      <td>2000</td>\n",
       "      <td>Ying Yang Twins</td>\n",
       "      <td>Whistle While You Tw...</td>\n",
       "      <td>4:19</td>\n",
       "      <td>2000-03-18</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24090</th>\n",
       "      <td>2000</td>\n",
       "      <td>Zombie Nation</td>\n",
       "      <td>Kernkraft 400</td>\n",
       "      <td>3:30</td>\n",
       "      <td>2000-09-02</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24091</th>\n",
       "      <td>2000</td>\n",
       "      <td>matchbox twenty</td>\n",
       "      <td>Bent</td>\n",
       "      <td>4:12</td>\n",
       "      <td>2000-04-29</td>\n",
       "      <td>wk76</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>24092 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       year            artist                    track  time date.entered  \\\n",
       "0      2000             2 Pac  Baby Don't Cry (Keep...  4:22   2000-02-26   \n",
       "1      2000           2Ge+her  The Hardest Part Of ...  3:15   2000-09-02   \n",
       "2      2000      3 Doors Down               Kryptonite  3:53   2000-04-08   \n",
       "3      2000      3 Doors Down                    Loser  4:24   2000-10-21   \n",
       "4      2000          504 Boyz            Wobble Wobble  3:35   2000-04-15   \n",
       "...     ...               ...                      ...   ...          ...   \n",
       "24087  2000       Yankee Grey     Another Nine Minutes  3:10   2000-04-29   \n",
       "24088  2000  Yearwood, Trisha          Real Live Woman  3:55   2000-04-01   \n",
       "24089  2000   Ying Yang Twins  Whistle While You Tw...  4:19   2000-03-18   \n",
       "24090  2000     Zombie Nation            Kernkraft 400  3:30   2000-09-02   \n",
       "24091  2000   matchbox twenty                     Bent  4:12   2000-04-29   \n",
       "\n",
       "       week  value  \n",
       "0       wk1   87.0  \n",
       "1       wk1   91.0  \n",
       "2       wk1   81.0  \n",
       "3       wk1   76.0  \n",
       "4       wk1   57.0  \n",
       "...     ...    ...  \n",
       "24087  wk76    NaN  \n",
       "24088  wk76    NaN  \n",
       "24089  wk76    NaN  \n",
       "24090  wk76    NaN  \n",
       "24091  wk76    NaN  \n",
       "\n",
       "[24092 rows x 7 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(column_names=\"wk*\", names_to=\"week\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html#janitor.pivot_longer) can also unpivot paired columns.  In this regard, it is like pandas' [wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html), but with more flexibility and power. Let's look at an example from pandas' [wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) docs : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>famid</th>\n",
       "      <th>birth</th>\n",
       "      <th>ht1</th>\n",
       "      <th>ht2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.8</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2.9</td>\n",
       "      <td>3.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2.2</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.2</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   famid  birth  ht1  ht2\n",
       "0      1      1  2.8  3.4\n",
       "1      1      2  2.9  3.8\n",
       "2      1      3  2.2  2.9\n",
       "3      2      1  2.0  3.2\n",
       "4      2      2  1.8  2.8\n",
       "5      2      3  1.9  2.4\n",
       "6      3      1  2.2  3.3\n",
       "7      3      2  2.3  3.4\n",
       "8      3      3  2.1  2.9"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"famid\": [1, 1, 1, 2, 2, 2, 3, 3, 3],\n",
    "        \"birth\": [1, 2, 3, 1, 2, 3, 1, 2, 3],\n",
    "        \"ht1\": [2.8, 2.9, 2.2, 2, 1.8, 1.9, 2.2, 2.3, 2.1],\n",
    "        \"ht2\": [3.4, 3.8, 2.9, 3.2, 2.8, 2.4, 3.3, 3.4, 2.9],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the data above, the `height`(ht) is paired with `age`(numbers). [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) can handle this easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>ht</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>famid</th>\n",
       "      <th>birth</th>\n",
       "      <th>age</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">1</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>1</th>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>1</th>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">2</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>1</th>\n",
       "      <td>1.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>1</th>\n",
       "      <td>1.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">3</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>1</th>\n",
       "      <td>2.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>1</th>\n",
       "      <td>2.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  ht\n",
       "famid birth age     \n",
       "1     1     1    2.8\n",
       "            2    3.4\n",
       "      2     1    2.9\n",
       "            2    3.8\n",
       "      3     1    2.2\n",
       "            2    2.9\n",
       "2     1     1    2.0\n",
       "            2    3.2\n",
       "      2     1    1.8\n",
       "            2    2.8\n",
       "      3     1    1.9\n",
       "            2    2.4\n",
       "3     1     1    2.2\n",
       "            2    3.3\n",
       "      2     1    2.3\n",
       "            2    3.4\n",
       "      3     1    2.1\n",
       "            2    2.9"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.wide_to_long(df, stubnames=\"ht\", i=[\"famid\", \"birth\"], j=\"age\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's see how [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) handles this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>famid</th>\n",
       "      <th>birth</th>\n",
       "      <th>age</th>\n",
       "      <th>ht</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    famid  birth age   ht\n",
       "0       1      1   1  2.8\n",
       "1       1      2   1  2.9\n",
       "2       1      3   1  2.2\n",
       "3       2      1   1  2.0\n",
       "4       2      2   1  1.8\n",
       "5       2      3   1  1.9\n",
       "6       3      1   1  2.2\n",
       "7       3      2   1  2.3\n",
       "8       3      3   1  2.1\n",
       "9       1      1   2  3.4\n",
       "10      1      2   2  3.8\n",
       "11      1      3   2  2.9\n",
       "12      2      1   2  3.2\n",
       "13      2      2   2  2.8\n",
       "14      2      3   2  2.4\n",
       "15      3      1   2  3.3\n",
       "16      3      2   2  3.4\n",
       "17      3      3   2  2.9"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"famid\", \"birth\"],\n",
    "    names_to=(\".value\", \"age\"),\n",
    "    names_pattern=r\"(.+)(.)\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first observable difference is that [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) is method chainable, while [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) is not. Now, let's learn more about the `.value` variable.\n",
    "\n",
    "\n",
    "When `.value` is used in `names_to`, a pairing is created between `names_to` and `names_pattern`. For the example above, we get this pairing :\n",
    "<br>\n",
    "<div align='center'> \".value\" --> (.+), \"age\" --> (.) </div>\n",
    "\n",
    "This tells the [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) function to keep values associated with `.value`(`.+`) as the column name, while values not associated with `.value`, in this case, the numbers, will be collated under a new column `age`. Internally, pandas `str.extract` is used to get the capturing groups before reshaping. This level of abstraction, we believe, allows the user to focus on the task, and get things done faster.\n",
    "\n",
    "Note that if you want the data returned in order of appearance you can set `sort_by_appearance` to `True`:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>famid</th>\n",
       "      <th>birth</th>\n",
       "      <th>age</th>\n",
       "      <th>ht</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    famid  birth age   ht\n",
       "0       1      1   1  2.8\n",
       "1       1      1   2  3.4\n",
       "2       1      2   1  2.9\n",
       "3       1      2   2  3.8\n",
       "4       1      3   1  2.2\n",
       "5       1      3   2  2.9\n",
       "6       2      1   1  2.0\n",
       "7       2      1   2  3.2\n",
       "8       2      2   1  1.8\n",
       "9       2      2   2  2.8\n",
       "10      2      3   1  1.9\n",
       "11      2      3   2  2.4\n",
       "12      3      1   1  2.2\n",
       "13      3      1   2  3.3\n",
       "14      3      2   1  2.3\n",
       "15      3      2   2  3.4\n",
       "16      3      3   1  2.1\n",
       "17      3      3   2  2.9"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"famid\", \"birth\"],\n",
    "    names_to=(\".value\", \"age\"),\n",
    "    names_pattern=r\"(.+)(.)\",\n",
    "    sort_by_appearance=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note also that the values in the `age` column are of `object` dtype. You can change the dtype, using pandas' [astype](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.astype.html) method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've seen already that [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) handles this already and very well, so why bother? Let's look at another scenario where [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) would need a few more steps. [Source Data](https://community.rstudio.com/t/pivot-longer-on-multiple-column-sets-pairs/43958):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>off_loc</th>\n",
       "      <th>pt_loc</th>\n",
       "      <th>pt_lat</th>\n",
       "      <th>off_lat</th>\n",
       "      <th>pt_long</th>\n",
       "      <th>off_long</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>G</td>\n",
       "      <td>100.075482</td>\n",
       "      <td>121.271083</td>\n",
       "      <td>4.472090</td>\n",
       "      <td>-7.188632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>H</td>\n",
       "      <td>75.191326</td>\n",
       "      <td>75.938453</td>\n",
       "      <td>-144.387785</td>\n",
       "      <td>-143.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C</td>\n",
       "      <td>I</td>\n",
       "      <td>122.651345</td>\n",
       "      <td>135.043791</td>\n",
       "      <td>-40.456110</td>\n",
       "      <td>21.242563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>J</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>134.511284</td>\n",
       "      <td>-46.071562</td>\n",
       "      <td>40.937417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>E</td>\n",
       "      <td>K</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>134.484374</td>\n",
       "      <td>-46.071562</td>\n",
       "      <td>40.784720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>F</td>\n",
       "      <td>L</td>\n",
       "      <td>124.010289</td>\n",
       "      <td>137.962195</td>\n",
       "      <td>-46.015943</td>\n",
       "      <td>22.905889</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  off_loc pt_loc      pt_lat     off_lat     pt_long    off_long\n",
       "0       A      G  100.075482  121.271083    4.472090   -7.188632\n",
       "1       B      H   75.191326   75.938453 -144.387785 -143.228857\n",
       "2       C      I  122.651345  135.043791  -40.456110   21.242563\n",
       "3       D      J  124.135533  134.511284  -46.071562   40.937417\n",
       "4       E      K  124.135533  134.484374  -46.071562   40.784720\n",
       "5       F      L  124.010289  137.962195  -46.015943   22.905889"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"off_loc\": [\"A\", \"B\", \"C\", \"D\", \"E\", \"F\"],\n",
    "        \"pt_loc\": [\"G\", \"H\", \"I\", \"J\", \"K\", \"L\"],\n",
    "        \"pt_lat\": [\n",
    "            100.07548220000001,\n",
    "            75.191326,\n",
    "            122.65134479999999,\n",
    "            124.13553329999999,\n",
    "            124.13553329999999,\n",
    "            124.01028909999998,\n",
    "        ],\n",
    "        \"off_lat\": [\n",
    "            121.271083,\n",
    "            75.93845266,\n",
    "            135.043791,\n",
    "            134.51128400000002,\n",
    "            134.484374,\n",
    "            137.962195,\n",
    "        ],\n",
    "        \"pt_long\": [\n",
    "            4.472089953,\n",
    "            -144.387785,\n",
    "            -40.45611048,\n",
    "            -46.07156181,\n",
    "            -46.07156181,\n",
    "            -46.01594293,\n",
    "        ],\n",
    "        \"off_long\": [\n",
    "            -7.188632000000001,\n",
    "            -143.2288569,\n",
    "            21.242563,\n",
    "            40.937416999999996,\n",
    "            40.78472,\n",
    "            22.905889000000002,\n",
    "        ],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can unpivot with [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) by first reorganising the columns : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loc_off</th>\n",
       "      <th>loc_pt</th>\n",
       "      <th>lat_pt</th>\n",
       "      <th>lat_off</th>\n",
       "      <th>long_pt</th>\n",
       "      <th>long_off</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>G</td>\n",
       "      <td>100.075482</td>\n",
       "      <td>121.271083</td>\n",
       "      <td>4.472090</td>\n",
       "      <td>-7.188632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>B</td>\n",
       "      <td>H</td>\n",
       "      <td>75.191326</td>\n",
       "      <td>75.938453</td>\n",
       "      <td>-144.387785</td>\n",
       "      <td>-143.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C</td>\n",
       "      <td>I</td>\n",
       "      <td>122.651345</td>\n",
       "      <td>135.043791</td>\n",
       "      <td>-40.456110</td>\n",
       "      <td>21.242563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>D</td>\n",
       "      <td>J</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>134.511284</td>\n",
       "      <td>-46.071562</td>\n",
       "      <td>40.937417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>E</td>\n",
       "      <td>K</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>134.484374</td>\n",
       "      <td>-46.071562</td>\n",
       "      <td>40.784720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>F</td>\n",
       "      <td>L</td>\n",
       "      <td>124.010289</td>\n",
       "      <td>137.962195</td>\n",
       "      <td>-46.015943</td>\n",
       "      <td>22.905889</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  loc_off loc_pt      lat_pt     lat_off     long_pt    long_off\n",
       "0       A      G  100.075482  121.271083    4.472090   -7.188632\n",
       "1       B      H   75.191326   75.938453 -144.387785 -143.228857\n",
       "2       C      I  122.651345  135.043791  -40.456110   21.242563\n",
       "3       D      J  124.135533  134.511284  -46.071562   40.937417\n",
       "4       E      K  124.135533  134.484374  -46.071562   40.784720\n",
       "5       F      L  124.010289  137.962195  -46.015943   22.905889"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.copy()\n",
    "df1.columns = df1.columns.str.split(\"_\").str[::-1].str.join(\"_\")\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can unpivot : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>loc</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th>set</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>off</th>\n",
       "      <td>A</td>\n",
       "      <td>121.271083</td>\n",
       "      <td>-7.188632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <th>off</th>\n",
       "      <td>B</td>\n",
       "      <td>75.938453</td>\n",
       "      <td>-143.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <th>off</th>\n",
       "      <td>C</td>\n",
       "      <td>135.043791</td>\n",
       "      <td>21.242563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <th>off</th>\n",
       "      <td>D</td>\n",
       "      <td>134.511284</td>\n",
       "      <td>40.937417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <th>off</th>\n",
       "      <td>E</td>\n",
       "      <td>134.484374</td>\n",
       "      <td>40.784720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <th>off</th>\n",
       "      <td>F</td>\n",
       "      <td>137.962195</td>\n",
       "      <td>22.905889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <th>pt</th>\n",
       "      <td>G</td>\n",
       "      <td>100.075482</td>\n",
       "      <td>4.472090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <th>pt</th>\n",
       "      <td>H</td>\n",
       "      <td>75.191326</td>\n",
       "      <td>-144.387785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <th>pt</th>\n",
       "      <td>I</td>\n",
       "      <td>122.651345</td>\n",
       "      <td>-40.456110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <th>pt</th>\n",
       "      <td>J</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <th>pt</th>\n",
       "      <td>K</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <th>pt</th>\n",
       "      <td>L</td>\n",
       "      <td>124.010289</td>\n",
       "      <td>-46.015943</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          loc         lat        long\n",
       "index set                            \n",
       "0     off   A  121.271083   -7.188632\n",
       "1     off   B   75.938453 -143.228857\n",
       "2     off   C  135.043791   21.242563\n",
       "3     off   D  134.511284   40.937417\n",
       "4     off   E  134.484374   40.784720\n",
       "5     off   F  137.962195   22.905889\n",
       "0     pt    G  100.075482    4.472090\n",
       "1     pt    H   75.191326 -144.387785\n",
       "2     pt    I  122.651345  -40.456110\n",
       "3     pt    J  124.135533  -46.071562\n",
       "4     pt    K  124.135533  -46.071562\n",
       "5     pt    L  124.010289  -46.015943"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.wide_to_long(\n",
    "    df1.reset_index(),\n",
    "    stubnames=[\"loc\", \"lat\", \"long\"],\n",
    "    sep=\"_\",\n",
    "    i=\"index\",\n",
    "    j=\"set\",\n",
    "    suffix=\".+\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get the same transformed dataframe, with less lines, using [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>set</th>\n",
       "      <th>loc</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>off</td>\n",
       "      <td>A</td>\n",
       "      <td>121.271083</td>\n",
       "      <td>-7.188632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>off</td>\n",
       "      <td>B</td>\n",
       "      <td>75.938453</td>\n",
       "      <td>-143.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>off</td>\n",
       "      <td>C</td>\n",
       "      <td>135.043791</td>\n",
       "      <td>21.242563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>off</td>\n",
       "      <td>D</td>\n",
       "      <td>134.511284</td>\n",
       "      <td>40.937417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>off</td>\n",
       "      <td>E</td>\n",
       "      <td>134.484374</td>\n",
       "      <td>40.784720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>off</td>\n",
       "      <td>F</td>\n",
       "      <td>137.962195</td>\n",
       "      <td>22.905889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>pt</td>\n",
       "      <td>G</td>\n",
       "      <td>100.075482</td>\n",
       "      <td>4.472090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>pt</td>\n",
       "      <td>H</td>\n",
       "      <td>75.191326</td>\n",
       "      <td>-144.387785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>pt</td>\n",
       "      <td>I</td>\n",
       "      <td>122.651345</td>\n",
       "      <td>-40.456110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>pt</td>\n",
       "      <td>J</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>pt</td>\n",
       "      <td>K</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>pt</td>\n",
       "      <td>L</td>\n",
       "      <td>124.010289</td>\n",
       "      <td>-46.015943</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    set loc         lat        long\n",
       "0   off   A  121.271083   -7.188632\n",
       "1   off   B   75.938453 -143.228857\n",
       "2   off   C  135.043791   21.242563\n",
       "3   off   D  134.511284   40.937417\n",
       "4   off   E  134.484374   40.784720\n",
       "5   off   F  137.962195   22.905889\n",
       "6    pt   G  100.075482    4.472090\n",
       "7    pt   H   75.191326 -144.387785\n",
       "8    pt   I  122.651345  -40.456110\n",
       "9    pt   J  124.135533  -46.071562\n",
       "10   pt   K  124.135533  -46.071562\n",
       "11   pt   L  124.010289  -46.015943"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(names_to=[\"set\", \".value\"], names_pattern=\"(.+)_(.+)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Another way to see the pairings,\n",
    "# to see what is linked to `.value`,\n",
    "\n",
    "# names_to =     [\"set\", \".value\"]\n",
    "# names_pattern = \"(.+)_(.+)\"\n",
    "# column _names =   off_loc\n",
    "#                   off_lat\n",
    "#                   off_long"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, the key here is the `.value` symbol. Pairing `names_to` with `names_pattern` and its results from [pd.str.extract](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.extract.html), we get : \n",
    "<br>\n",
    "<div align=\"center\"> set--> (.+) --> [off, pt] and  .value--> (.+) --> [loc, lat, long] </div>\n",
    "<br>\n",
    "                                     \n",
    "All values associated with `.value`(`loc, lat, long`) remain as column names, while values not associated with `.value`(`off, pt`) are lumped into a new column `set`. \n",
    "\n",
    "Notice that we did not have to reset the index - [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) takes care of that internally;  [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) allows you to focus on what you want, so you can get it and move on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the unpivoting could also have been executed with `names_sep`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>set</th>\n",
       "      <th>loc</th>\n",
       "      <th>lat</th>\n",
       "      <th>long</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>off</td>\n",
       "      <td>A</td>\n",
       "      <td>121.271083</td>\n",
       "      <td>-7.188632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pt</td>\n",
       "      <td>G</td>\n",
       "      <td>100.075482</td>\n",
       "      <td>4.472090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>off</td>\n",
       "      <td>B</td>\n",
       "      <td>75.938453</td>\n",
       "      <td>-143.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pt</td>\n",
       "      <td>H</td>\n",
       "      <td>75.191326</td>\n",
       "      <td>-144.387785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>off</td>\n",
       "      <td>C</td>\n",
       "      <td>135.043791</td>\n",
       "      <td>21.242563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pt</td>\n",
       "      <td>I</td>\n",
       "      <td>122.651345</td>\n",
       "      <td>-40.456110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>off</td>\n",
       "      <td>D</td>\n",
       "      <td>134.511284</td>\n",
       "      <td>40.937417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pt</td>\n",
       "      <td>J</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>off</td>\n",
       "      <td>E</td>\n",
       "      <td>134.484374</td>\n",
       "      <td>40.784720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pt</td>\n",
       "      <td>K</td>\n",
       "      <td>124.135533</td>\n",
       "      <td>-46.071562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>off</td>\n",
       "      <td>F</td>\n",
       "      <td>137.962195</td>\n",
       "      <td>22.905889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>pt</td>\n",
       "      <td>L</td>\n",
       "      <td>124.010289</td>\n",
       "      <td>-46.015943</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   set loc         lat        long\n",
       "0  off   A  121.271083   -7.188632\n",
       "0   pt   G  100.075482    4.472090\n",
       "1  off   B   75.938453 -143.228857\n",
       "1   pt   H   75.191326 -144.387785\n",
       "2  off   C  135.043791   21.242563\n",
       "2   pt   I  122.651345  -40.456110\n",
       "3  off   D  134.511284   40.937417\n",
       "3   pt   J  124.135533  -46.071562\n",
       "4  off   E  134.484374   40.784720\n",
       "4   pt   K  124.135533  -46.071562\n",
       "5  off   F  137.962195   22.905889\n",
       "5   pt   L  124.010289  -46.015943"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    names_to=[\"set\", \".value\"],\n",
    "    names_sep=\"_\",\n",
    "    ignore_index=False,\n",
    "    sort_by_appearance=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at another example, from [Stack Overflow](https://stackoverflow.com/questions/45123924/convert-pandas-dataframe-from-wide-to-long/45124130) : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a_1</th>\n",
       "      <th>ab_1</th>\n",
       "      <th>ac_1</th>\n",
       "      <th>a_2</th>\n",
       "      <th>ab_2</th>\n",
       "      <th>ac_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   a_1  ab_1  ac_1  a_2  ab_2  ac_2\n",
       "0    2     3     4    5     6     7"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    [{\"a_1\": 2, \"ab_1\": 3, \"ac_1\": 4, \"a_2\": 5, \"ab_2\": 6, \"ac_2\": 7}]\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data above requires extracting `a`, `ab` and `ac` from `1` and `2`. This is another example of a paired column. We could solve this using [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html); in fact there is a very good solution from [Stack Overflow](https://stackoverflow.com/a/45124775/7175713)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>ab</th>\n",
       "      <th>ac</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th>num</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        a  ab  ac\n",
       "id num           \n",
       "0  1    2   3   4\n",
       "   2    5   6   7"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.copy()\n",
    "df1[\"id\"] = df1.index\n",
    "pd.wide_to_long(df1, [\"a\", \"ab\", \"ac\"], i=\"id\", j=\"num\", sep=\"_\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you could simply pass the buck to [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>num</th>\n",
       "      <th>a</th>\n",
       "      <th>ab</th>\n",
       "      <th>ac</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  num  a  ab  ac\n",
       "0   1  2   3   4\n",
       "1   2  5   6   7"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(names_to=(\".value\", \"num\"), names_sep=\"_\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the solution above, we used the `names_sep` argument, as it is more convenient. A few more examples to get you familiar with the `.value` symbol.\n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/55403008/pandas-partial-melt-or-group-melt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>ax</th>\n",
       "      <th>ay</th>\n",
       "      <th>az</th>\n",
       "      <th>bx</th>\n",
       "      <th>by</th>\n",
       "      <th>bz</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  ax  ay  az  bx  by  bz\n",
       "0   1   1   2   3   4   5   6\n",
       "1   2   7   8   9  10  11  12"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    [[1, 1, 2, 3, 4, 5, 6], [2, 7, 8, 9, 10, 11, 12]],\n",
    "    columns=[\"id\", \"ax\", \"ay\", \"az\", \"bx\", \"by\", \"bz\"],\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>z</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>b</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>b</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id name   x   y   z\n",
       "0   1    a   1   2   3\n",
       "1   2    a   7   8   9\n",
       "2   1    b   4   5   6\n",
       "3   2    b  10  11  12"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"id\", names_to=(\"name\", \".value\"), names_pattern=\"(.)(.)\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the code above `.value` is paired with `x`, `y`, `z`(which become the new column names), while `a`, `b` are unpivoted into the `name` column. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the dataframe below, we need to unpivot the data, keeping only the suffix `hi`, and pulling out the number between `A` and `g`. \n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/35929985/melt-a-data-table-with-a-column-pattern)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>A1g_hi</th>\n",
       "      <th>A2g_hi</th>\n",
       "      <th>A3g_hi</th>\n",
       "      <th>A4g_hi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  A1g_hi  A2g_hi  A3g_hi  A4g_hi\n",
       "0   1       2       3       4       5"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    [{\"id\": 1, \"A1g_hi\": 2, \"A2g_hi\": 3, \"A3g_hi\": 4, \"A4g_hi\": 5}]\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>time</th>\n",
       "      <th>hi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id time  hi\n",
       "0   1    1   2\n",
       "1   1    2   3\n",
       "2   1    3   4\n",
       "3   1    4   5"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"id\", names_to=[\"time\", \".value\"], names_pattern=\"A(.)g_(.+)\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see an example where we have multiple values in a paired column, and we wish to split them into separate columns. [Source Data](https://stackoverflow.com/questions/64107566/how-to-pivot-longer-and-populate-with-fields-from-column-names-at-the-same-tim?noredirect=1#comment113369419_64107566) : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sony | TV | Model | value</th>\n",
       "      <th>Sony | TV | Quantity | value</th>\n",
       "      <th>Sony | TV | Max-quant | value</th>\n",
       "      <th>Panasonic | TV | Model | value</th>\n",
       "      <th>Panasonic | TV | Quantity | value</th>\n",
       "      <th>Panasonic | TV | Max-quant | value</th>\n",
       "      <th>Sanyo | Radio | Model | value</th>\n",
       "      <th>Sanyo | Radio | Quantity | value</th>\n",
       "      <th>Sanyo | Radio | Max-quant | value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A222</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>T232</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>S111</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A234</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>S3424</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>S1s1</td>\n",
       "      <td>2</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A4345</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "      <td>X3421</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>S1s2</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Sony | TV | Model | value  Sony | TV | Quantity | value  \\\n",
       "0                      A222                             5   \n",
       "1                      A234                             5   \n",
       "2                     A4345                             4   \n",
       "\n",
       "   Sony | TV | Max-quant | value Panasonic | TV | Model | value  \\\n",
       "0                             10                           T232   \n",
       "1                              9                          S3424   \n",
       "2                              9                          X3421   \n",
       "\n",
       "   Panasonic | TV | Quantity | value  Panasonic | TV | Max-quant | value  \\\n",
       "0                                  1                                  10   \n",
       "1                                  5                                  12   \n",
       "2                                  1                                  11   \n",
       "\n",
       "  Sanyo | Radio | Model | value  Sanyo | Radio | Quantity | value  \\\n",
       "0                          S111                                 4   \n",
       "1                          S1s1                                 2   \n",
       "2                          S1s2                                 4   \n",
       "\n",
       "   Sanyo | Radio | Max-quant | value  \n",
       "0                                  9  \n",
       "1                                  9  \n",
       "2                                 10  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"Sony | TV | Model | value\": {0: \"A222\", 1: \"A234\", 2: \"A4345\"},\n",
    "        \"Sony | TV | Quantity | value\": {0: 5, 1: 5, 2: 4},\n",
    "        \"Sony | TV | Max-quant | value\": {0: 10, 1: 9, 2: 9},\n",
    "        \"Panasonic | TV | Model | value\": {0: \"T232\", 1: \"S3424\", 2: \"X3421\"},\n",
    "        \"Panasonic | TV | Quantity | value\": {0: 1, 1: 5, 2: 1},\n",
    "        \"Panasonic | TV | Max-quant | value\": {0: 10, 1: 12, 2: 11},\n",
    "        \"Sanyo | Radio | Model | value\": {0: \"S111\", 1: \"S1s1\", 2: \"S1s2\"},\n",
    "        \"Sanyo | Radio | Quantity | value\": {0: 4, 1: 2, 2: 4},\n",
    "        \"Sanyo | Radio | Max-quant | value\": {0: 9, 1: 9, 2: 10},\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal is to reshape the data into long format, with separate columns for `Manufacturer`(Sony,...), `Device`(TV, Radio), `Model`(S3424, ...), `maximum quantity` and `quantity`. \n",
    "\n",
    "Below is the [accepted solution](https://stackoverflow.com/a/64107688/7175713) on Stack Overflow :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Manufacturer</th>\n",
       "      <th>Device</th>\n",
       "      <th>Model</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Max-quant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>S3424</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>T232</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>X3421</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S111</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S1s1</td>\n",
       "      <td>2</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S1s2</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A222</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A234</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A4345</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Manufacturer Device  Model  Quantity  Max-quant\n",
       "0    Panasonic     TV  S3424         5         12\n",
       "1    Panasonic     TV   T232         1         10\n",
       "2    Panasonic     TV  X3421         1         11\n",
       "3        Sanyo  Radio   S111         4          9\n",
       "4        Sanyo  Radio   S1s1         2          9\n",
       "5        Sanyo  Radio   S1s2         4         10\n",
       "6         Sony     TV   A222         5         10\n",
       "7         Sony     TV   A234         5          9\n",
       "8         Sony     TV  A4345         4          9"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.copy()\n",
    "# Create a multiIndex column header\n",
    "df1.columns = pd.MultiIndex.from_arrays(\n",
    "    zip(*df1.columns.str.split(r\"\\s?\\|\\s?\"))\n",
    ")\n",
    "\n",
    "# Reshape the dataframe using\n",
    "# `set_index`, `droplevel`, and `stack`\n",
    "(\n",
    "    df1.stack([0, 1], future_stack=True)\n",
    "    .droplevel(1, axis=1)\n",
    "    .set_index(\"Model\", append=True)\n",
    "    .rename_axis([None, \"Manufacturer\", \"Device\", \"Model\"])\n",
    "    .sort_index(level=[1, 2, 3])\n",
    "    .reset_index()\n",
    "    .drop(\"level_0\", axis=1)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we could use [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html), along with `.value` in `names_to` and a regular expression in `names_pattern` : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Manufacturer</th>\n",
       "      <th>Device</th>\n",
       "      <th>Model</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Max-quant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A222</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A234</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Sony</td>\n",
       "      <td>TV</td>\n",
       "      <td>A4345</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>T232</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>S3424</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Panasonic</td>\n",
       "      <td>TV</td>\n",
       "      <td>X3421</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S111</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S1s1</td>\n",
       "      <td>2</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Sanyo</td>\n",
       "      <td>Radio</td>\n",
       "      <td>S1s2</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Manufacturer   Device  Model    Quantity    Max-quant \n",
       "0        Sony       TV     A222           5           10\n",
       "1        Sony       TV     A234           5            9\n",
       "2        Sony       TV    A4345           4            9\n",
       "3   Panasonic       TV     T232           1           10\n",
       "4   Panasonic       TV    S3424           5           12\n",
       "5   Panasonic       TV    X3421           1           11\n",
       "6       Sanyo    Radio     S111           4            9\n",
       "7       Sanyo    Radio     S1s1           2            9\n",
       "8       Sanyo    Radio     S1s2           4           10"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    names_to=(\"Manufacturer\", \"Device\", \".value\"),\n",
    "    names_pattern=r\"(.+)\\|(.+)\\|(.+)\\|.*\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we are interested in unpivoting only a part of the entire dataframe? \n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/63044119/converting-wide-format-data-into-long-format-with-multiple-indices-and-grouped-d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>factor</th>\n",
       "      <th>variable1</th>\n",
       "      <th>variable2</th>\n",
       "      <th>variable3</th>\n",
       "      <th>variable4</th>\n",
       "      <th>variable5</th>\n",
       "      <th>variable6</th>\n",
       "      <th>O1V1</th>\n",
       "      <th>O1V2</th>\n",
       "      <th>...</th>\n",
       "      <th>O2V7</th>\n",
       "      <th>O2V8</th>\n",
       "      <th>O3V1</th>\n",
       "      <th>O3V2</th>\n",
       "      <th>O3V3</th>\n",
       "      <th>O3V4</th>\n",
       "      <th>O3V5</th>\n",
       "      <th>O3V6</th>\n",
       "      <th>O3V7</th>\n",
       "      <th>O3V8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.4</td>\n",
       "      <td>...</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.9</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.6</td>\n",
       "      <td>0.4</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>-0.6</td>\n",
       "      <td>0.4</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 32 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   time factor  variable1  variable2  variable3  variable4  variable5  \\\n",
       "0     1      a          0          0          0          2          1   \n",
       "1     2      a          0          0          2          0          0   \n",
       "2     3      b          0          1          0          1          1   \n",
       "\n",
       "   variable6  O1V1  O1V2  ...  O2V7  O2V8  O3V1  O3V2  O3V3  O3V4  O3V5  O3V6  \\\n",
       "0          0   0.0   0.0  ...   0.6   0.5   0.0   0.9   0.5   0.5   0.0   0.9   \n",
       "1          1   0.2   0.4  ...   0.1   0.2   0.7   0.2   0.2   0.2   0.7   0.2   \n",
       "2          1  -0.3  -0.9  ...  -0.3  -0.6   0.4  -0.3  -0.7  -0.6   0.4  -0.3   \n",
       "\n",
       "   O3V7  O3V8  \n",
       "0   0.5   0.5  \n",
       "1   0.2   0.2  \n",
       "2  -0.7  -0.6  \n",
       "\n",
       "[3 rows x 32 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"time\": [1, 2, 3],\n",
    "        \"factor\": [\"a\", \"a\", \"b\"],\n",
    "        \"variable1\": [0, 0, 0],\n",
    "        \"variable2\": [0, 0, 1],\n",
    "        \"variable3\": [0, 2, 0],\n",
    "        \"variable4\": [2, 0, 1],\n",
    "        \"variable5\": [1, 0, 1],\n",
    "        \"variable6\": [0, 1, 1],\n",
    "        \"O1V1\": [0, 0.2, -0.3],\n",
    "        \"O1V2\": [0, 0.4, -0.9],\n",
    "        \"O1V3\": [0.5, 0.2, -0.6],\n",
    "        \"O1V4\": [0.5, 0.2, -0.6],\n",
    "        \"O1V5\": [0, 0.2, -0.3],\n",
    "        \"O1V6\": [0, 0.4, -0.9],\n",
    "        \"O1V7\": [0.5, 0.2, -0.6],\n",
    "        \"O1V8\": [0.5, 0.2, -0.6],\n",
    "        \"O2V1\": [0, 0.5, 0.3],\n",
    "        \"O2V2\": [0, 0.2, 0.9],\n",
    "        \"O2V3\": [0.6, 0.1, -0.3],\n",
    "        \"O2V4\": [0.5, 0.2, -0.6],\n",
    "        \"O2V5\": [0, 0.5, 0.3],\n",
    "        \"O2V6\": [0, 0.2, 0.9],\n",
    "        \"O2V7\": [0.6, 0.1, -0.3],\n",
    "        \"O2V8\": [0.5, 0.2, -0.6],\n",
    "        \"O3V1\": [0, 0.7, 0.4],\n",
    "        \"O3V2\": [0.9, 0.2, -0.3],\n",
    "        \"O3V3\": [0.5, 0.2, -0.7],\n",
    "        \"O3V4\": [0.5, 0.2, -0.6],\n",
    "        \"O3V5\": [0, 0.7, 0.4],\n",
    "        \"O3V6\": [0.9, 0.2, -0.3],\n",
    "        \"O3V7\": [0.5, 0.2, -0.7],\n",
    "        \"O3V8\": [0.5, 0.2, -0.6],\n",
    "    }\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the task? This is copied verbatim from the source:\n",
    "\n",
    "<blockquote>Each row of the data frame represents a time period. There are multiple 'subjects' being monitored, namely O1, O2, and O3. Each subject has 8 variables being measured. I need to convert this data into long format where each row contains the information for one subject at a given time period, but with only the first 4 subject variables, as well as the extra information about this time period in columns 2-4, but not columns 5-8.</blockquote>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is the accepted solution, using [wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>factor</th>\n",
       "      <th>variable1</th>\n",
       "      <th>variable2</th>\n",
       "      <th>id</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.9</td>\n",
       "      <td>-0.6</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.9</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.4</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   time factor  variable1  variable2  id   V1   V2   V3   V4\n",
       "0     1      a          0          0   1  0.0  0.0  0.5  0.5\n",
       "1     1      a          0          0   2  0.0  0.0  0.6  0.5\n",
       "2     1      a          0          0   3  0.0  0.9  0.5  0.5\n",
       "3     2      a          0          0   1  0.2  0.4  0.2  0.2\n",
       "4     2      a          0          0   2  0.5  0.2  0.1  0.2\n",
       "5     2      a          0          0   3  0.7  0.2  0.2  0.2\n",
       "6     3      b          0          1   1 -0.3 -0.9 -0.6 -0.6\n",
       "7     3      b          0          1   2  0.3  0.9 -0.3 -0.6\n",
       "8     3      b          0          1   3  0.4 -0.3 -0.7 -0.6"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.rename(\n",
    "    columns={\n",
    "        x: x[2:] + x[1:2] for x in df.columns[df.columns.str.startswith(\"O\")]\n",
    "    }\n",
    ")\n",
    "\n",
    "df1 = pd.wide_to_long(\n",
    "    df1,\n",
    "    i=[\"time\", \"factor\"] + [f\"variable{i}\" for i in range(1, 7)],\n",
    "    j=\"id\",\n",
    "    stubnames=[f\"V{i}\" for i in range(1, 9)],\n",
    "    suffix=\".*\",\n",
    ")\n",
    "\n",
    "df1 = df1.reset_index().drop(\n",
    "    columns=[f\"V{i}\" for i in range(5, 9)]\n",
    "    + [f\"variable{i}\" for i in range(3, 7)]\n",
    ")\n",
    "\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can abstract the details and focus on the task with [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>factor</th>\n",
       "      <th>variable1</th>\n",
       "      <th>variable2</th>\n",
       "      <th>id</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>a</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.9</td>\n",
       "      <td>-0.6</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.9</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>b</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.4</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>-0.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   time factor  variable1  variable2 id   V1   V2   V3   V4\n",
       "0     1      a          0          0  1  0.0  0.0  0.5  0.5\n",
       "1     1      a          0          0  2  0.0  0.0  0.6  0.5\n",
       "2     1      a          0          0  3  0.0  0.9  0.5  0.5\n",
       "3     2      a          0          0  1  0.2  0.4  0.2  0.2\n",
       "4     2      a          0          0  2  0.5  0.2  0.1  0.2\n",
       "5     2      a          0          0  3  0.7  0.2  0.2  0.2\n",
       "6     3      b          0          1  1 -0.3 -0.9 -0.6 -0.6\n",
       "7     3      b          0          1  2  0.3  0.9 -0.3 -0.6\n",
       "8     3      b          0          1  3  0.4 -0.3 -0.7 -0.6"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=slice(\"time\", \"variable2\"),\n",
    "    column_names=re.compile(\".+V[1-4]$\"),\n",
    "    names_to=(\"id\", \".value\"),\n",
    "    names_pattern=\".(.)(.+)$\",\n",
    "    sort_by_appearance=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more example on the `.value` symbol for paired columns \n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/59477686/python-pandas-melt-single-column-into-two-seperate) : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>A_value</th>\n",
       "      <th>D_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>33</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  A_value  D_value\n",
       "0   1       50       60\n",
       "1   2       33       45"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({\"id\": [1, 2], \"A_value\": [50, 33], \"D_value\": [60, 45]})\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>value_type</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>A</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>D</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>D</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id value_type  value\n",
       "0   1          A     50\n",
       "1   2          A     33\n",
       "2   1          D     60\n",
       "3   2          D     45"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(index=\"id\", names_to=(\"value_type\", \".value\"), names_sep=\"_\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are scenarios where we need to unpivot the data, and group values within the column names under new columns. The values in the columns will not become new column names, so we do not need the `.value` symbol. Let's see an example below: \n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/59550804/melt-column-by-substring-of-the-columns-name-in-pandas-python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject</th>\n",
       "      <th>A_target_word_gd</th>\n",
       "      <th>A_target_word_fd</th>\n",
       "      <th>B_target_word_gd</th>\n",
       "      <th>B_target_word_fd</th>\n",
       "      <th>subject_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>mild</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "      <td>moderate</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject  A_target_word_gd  A_target_word_fd  B_target_word_gd  \\\n",
       "0        1                 1                 2                 3   \n",
       "1        2                11                12                13   \n",
       "\n",
       "   B_target_word_fd subject_type  \n",
       "0                 4         mild  \n",
       "1                14     moderate  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"subject\": [1, 2],\n",
    "        \"A_target_word_gd\": [1, 11],\n",
    "        \"A_target_word_fd\": [2, 12],\n",
    "        \"B_target_word_gd\": [3, 13],\n",
    "        \"B_target_word_fd\": [4, 14],\n",
    "        \"subject_type\": [\"mild\", \"moderate\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the dataframe above, `A` and `B` represent conditions, while the suffixes `gd` and `fd` represent value types. We are not interested in the words in the middle (`_target_word`). We could solve it this way (this is the chosen solution, copied from [Stack Overflow](https://stackoverflow.com/a/59550967/7175713)) : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject_type</th>\n",
       "      <th>subject</th>\n",
       "      <th>value</th>\n",
       "      <th>cond</th>\n",
       "      <th>value_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mild</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>mild</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mild</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mild</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>moderate</td>\n",
       "      <td>2</td>\n",
       "      <td>11</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>moderate</td>\n",
       "      <td>2</td>\n",
       "      <td>12</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>moderate</td>\n",
       "      <td>2</td>\n",
       "      <td>13</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>moderate</td>\n",
       "      <td>2</td>\n",
       "      <td>14</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  subject_type  subject  value cond value_type\n",
       "0         mild        1      1    A         gd\n",
       "2         mild        1      2    A         fd\n",
       "4         mild        1      3    B         gd\n",
       "6         mild        1      4    B         fd\n",
       "1     moderate        2     11    A         gd\n",
       "3     moderate        2     12    A         fd\n",
       "5     moderate        2     13    B         gd\n",
       "7     moderate        2     14    B         fd"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_df = pd.melt(\n",
    "    df, id_vars=[\"subject_type\", \"subject\"], var_name=\"abc\"\n",
    ").sort_values(by=[\"subject\", \"subject_type\"])\n",
    "new_df[\"cond\"] = new_df[\"abc\"].apply(lambda x: (x.split(\"_\"))[0])\n",
    "new_df[\"value_type\"] = new_df.pop(\"abc\").apply(lambda x: (x.split(\"_\"))[-1])\n",
    "new_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we could just pass the buck to [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject</th>\n",
       "      <th>subject_type</th>\n",
       "      <th>cond</th>\n",
       "      <th>value_type</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject subject_type cond value_type  value\n",
       "0        1         mild    A         gd      1\n",
       "1        2     moderate    A         gd     11\n",
       "2        1         mild    A         fd      2\n",
       "3        2     moderate    A         fd     12\n",
       "4        1         mild    B         gd      3\n",
       "5        2     moderate    B         gd     13\n",
       "6        1         mild    B         fd      4\n",
       "7        2     moderate    B         fd     14"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"subject\", \"subject_type\"],\n",
    "    names_to=(\"cond\", \"value_type\"),\n",
    "    names_pattern=\"([A-Z]).*(gd|fd)\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the code above, we pass in the new names of the columns to `names_to`('cond', 'value_type'), and pass the groups to be extracted as a regular expression to `names_pattern`. \n",
    "\n",
    "We could also reshape with `names_sep`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>subject</th>\n",
       "      <th>subject_type</th>\n",
       "      <th>cond</th>\n",
       "      <th>value_type</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>A</td>\n",
       "      <td>gd</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>A</td>\n",
       "      <td>fd</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>B</td>\n",
       "      <td>gd</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>mild</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>moderate</td>\n",
       "      <td>B</td>\n",
       "      <td>fd</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   subject subject_type cond value_type  value\n",
       "0        1         mild    A         gd      1\n",
       "1        2     moderate    A         gd     11\n",
       "2        1         mild    A         fd      2\n",
       "3        2     moderate    A         fd     12\n",
       "4        1         mild    B         gd      3\n",
       "5        2     moderate    B         gd     13\n",
       "6        1         mild    B         fd      4\n",
       "7        2     moderate    B         fd     14"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"subject\", \"subject_type\"],\n",
    "    names_to=(\"cond\", \"value_type\"),\n",
    "    names_sep=\"_target_word_\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's another example where [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) abstracts the process and makes reshaping easy.\n",
    "\n",
    "\n",
    "In the dataframe below, we would like to unpivot the data and separate the column names into individual columns(`vault` should be in an `event` column, `2012` should be in a `year` column and `f` should be in a `gender` column). \n",
    "\n",
    "[Source Data](https://dcl-wrangle.stanford.edu/pivot-advanced.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>vault_2012_f</th>\n",
       "      <th>vault_2012_m</th>\n",
       "      <th>vault_2016_f</th>\n",
       "      <th>vault_2016_m</th>\n",
       "      <th>floor_2012_f</th>\n",
       "      <th>floor_2012_m</th>\n",
       "      <th>floor_2016_f</th>\n",
       "      <th>floor_2016_m</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>48.132</td>\n",
       "      <td>46.632</td>\n",
       "      <td>46.866</td>\n",
       "      <td>45.865</td>\n",
       "      <td>45.366</td>\n",
       "      <td>45.266</td>\n",
       "      <td>45.999</td>\n",
       "      <td>43.757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Russia</td>\n",
       "      <td>46.366</td>\n",
       "      <td>46.866</td>\n",
       "      <td>45.733</td>\n",
       "      <td>46.033</td>\n",
       "      <td>41.599</td>\n",
       "      <td>45.308</td>\n",
       "      <td>42.032</td>\n",
       "      <td>44.766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>China</td>\n",
       "      <td>44.266</td>\n",
       "      <td>48.316</td>\n",
       "      <td>44.332</td>\n",
       "      <td>45.000</td>\n",
       "      <td>40.833</td>\n",
       "      <td>45.133</td>\n",
       "      <td>42.066</td>\n",
       "      <td>43.799</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         country  vault_2012_f  vault_2012_m  vault_2016_f  vault_2016_m  \\\n",
       "0  United States        48.132        46.632        46.866        45.865   \n",
       "1         Russia        46.366        46.866        45.733        46.033   \n",
       "2          China        44.266        48.316        44.332        45.000   \n",
       "\n",
       "   floor_2012_f  floor_2012_m  floor_2016_f  floor_2016_m  \n",
       "0        45.366        45.266        45.999        43.757  \n",
       "1        41.599        45.308        42.032        44.766  \n",
       "2        40.833        45.133        42.066        43.799  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"country\": [\"United States\", \"Russia\", \"China\"],\n",
    "        \"vault_2012_f\": [\n",
    "            48.132,\n",
    "            46.366,\n",
    "            44.266,\n",
    "        ],\n",
    "        \"vault_2012_m\": [46.632, 46.866, 48.316],\n",
    "        \"vault_2016_f\": [\n",
    "            46.866,\n",
    "            45.733,\n",
    "            44.332,\n",
    "        ],\n",
    "        \"vault_2016_m\": [45.865, 46.033, 45.0],\n",
    "        \"floor_2012_f\": [45.366, 41.599, 40.833],\n",
    "        \"floor_2012_m\": [45.266, 45.308, 45.133],\n",
    "        \"floor_2016_f\": [45.999, 42.032, 42.066],\n",
    "        \"floor_2016_m\": [43.757, 44.766, 43.799],\n",
    "    }\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is one way to reshape the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>event</th>\n",
       "      <th>year</th>\n",
       "      <th>gender</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>48.132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>45.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>46.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>46.033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>41.599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>44.766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>44.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>48.316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>44.332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>40.833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.799</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          country  event  year gender   score\n",
       "0   United States  vault  2012      f  48.132\n",
       "1   United States  vault  2012      m  46.632\n",
       "2   United States  vault  2016      f  46.866\n",
       "3   United States  vault  2016      m  45.865\n",
       "4   United States  floor  2012      f  45.366\n",
       "5   United States  floor  2012      m  45.266\n",
       "6   United States  floor  2016      f  45.999\n",
       "7   United States  floor  2016      m  43.757\n",
       "8          Russia  vault  2012      f  46.366\n",
       "9          Russia  vault  2012      m  46.866\n",
       "10         Russia  vault  2016      f  45.733\n",
       "11         Russia  vault  2016      m  46.033\n",
       "12         Russia  floor  2012      f  41.599\n",
       "13         Russia  floor  2012      m  45.308\n",
       "14         Russia  floor  2016      f  42.032\n",
       "15         Russia  floor  2016      m  44.766\n",
       "16          China  vault  2012      f  44.266\n",
       "17          China  vault  2012      m  48.316\n",
       "18          China  vault  2016      f  44.332\n",
       "19          China  vault  2016      m  45.000\n",
       "20          China  floor  2012      f  40.833\n",
       "21          China  floor  2012      m  45.133\n",
       "22          China  floor  2016      f  42.066\n",
       "23          China  floor  2016      m  43.799"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reshape = df.set_index(\"country\")\n",
    "reshape.columns = reshape.columns.str.split(\"_\", expand=True)\n",
    "columns = [\"event\", \"year\", \"gender\"]\n",
    "reshape.columns.names = columns\n",
    "(\n",
    "    reshape.stack(level=columns, future_stack=True)\n",
    "    .rename(\"score\")\n",
    "    .reset_index(level=[\"country\"] + columns)\n",
    "    .reset_index(drop=True)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is simplified with [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>event</th>\n",
       "      <th>year</th>\n",
       "      <th>gender</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>48.132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>46.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>44.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>48.316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>44.332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>46.033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>45.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>41.599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>40.833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>44.766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.799</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          country  event  year gender   score\n",
       "0   United States  vault  2012      f  48.132\n",
       "1          Russia  vault  2012      f  46.366\n",
       "2           China  vault  2012      f  44.266\n",
       "3   United States  vault  2012      m  46.632\n",
       "4          Russia  vault  2012      m  46.866\n",
       "5           China  vault  2012      m  48.316\n",
       "6   United States  vault  2016      f  46.866\n",
       "7          Russia  vault  2016      f  45.733\n",
       "8           China  vault  2016      f  44.332\n",
       "9   United States  vault  2016      m  45.865\n",
       "10         Russia  vault  2016      m  46.033\n",
       "11          China  vault  2016      m  45.000\n",
       "12  United States  floor  2012      f  45.366\n",
       "13         Russia  floor  2012      f  41.599\n",
       "14          China  floor  2012      f  40.833\n",
       "15  United States  floor  2012      m  45.266\n",
       "16         Russia  floor  2012      m  45.308\n",
       "17          China  floor  2012      m  45.133\n",
       "18  United States  floor  2016      f  45.999\n",
       "19         Russia  floor  2016      f  42.032\n",
       "20          China  floor  2016      f  42.066\n",
       "21  United States  floor  2016      m  43.757\n",
       "22         Russia  floor  2016      m  44.766\n",
       "23          China  floor  2016      m  43.799"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"country\",\n",
    "    names_to=[\"event\", \"year\", \"gender\"],\n",
    "    names_sep=\"_\",\n",
    "    values_to=\"score\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, if you want the data returned in order of appearance, you can turn on the `sort_by_appearance` parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>country</th>\n",
       "      <th>event</th>\n",
       "      <th>year</th>\n",
       "      <th>gender</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>48.132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>United States</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>45.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>United States</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>46.366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>46.866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>45.733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Russia</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>46.033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>41.599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Russia</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>44.766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>44.266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>48.316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>44.332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>China</td>\n",
       "      <td>vault</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>45.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>f</td>\n",
       "      <td>40.833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2012</td>\n",
       "      <td>m</td>\n",
       "      <td>45.133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>f</td>\n",
       "      <td>42.066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>China</td>\n",
       "      <td>floor</td>\n",
       "      <td>2016</td>\n",
       "      <td>m</td>\n",
       "      <td>43.799</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          country  event  year gender   score\n",
       "0   United States  vault  2012      f  48.132\n",
       "1   United States  vault  2012      m  46.632\n",
       "2   United States  vault  2016      f  46.866\n",
       "3   United States  vault  2016      m  45.865\n",
       "4   United States  floor  2012      f  45.366\n",
       "5   United States  floor  2012      m  45.266\n",
       "6   United States  floor  2016      f  45.999\n",
       "7   United States  floor  2016      m  43.757\n",
       "8          Russia  vault  2012      f  46.366\n",
       "9          Russia  vault  2012      m  46.866\n",
       "10         Russia  vault  2016      f  45.733\n",
       "11         Russia  vault  2016      m  46.033\n",
       "12         Russia  floor  2012      f  41.599\n",
       "13         Russia  floor  2012      m  45.308\n",
       "14         Russia  floor  2016      f  42.032\n",
       "15         Russia  floor  2016      m  44.766\n",
       "16          China  vault  2012      f  44.266\n",
       "17          China  vault  2012      m  48.316\n",
       "18          China  vault  2016      f  44.332\n",
       "19          China  vault  2016      m  45.000\n",
       "20          China  floor  2012      f  40.833\n",
       "21          China  floor  2012      m  45.133\n",
       "22          China  floor  2016      f  42.066\n",
       "23          China  floor  2016      m  43.799"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"country\",\n",
    "    names_to=[\"event\", \"year\", \"gender\"],\n",
    "    names_sep=\"_\",\n",
    "    values_to=\"score\",\n",
    "    sort_by_appearance=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more feature that [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) offers is to pass a list of regular expressions to `names_pattern`. This comes in handy when one single regex cannot encapsulate similar columns for reshaping to long form. This idea is inspired by the [melt](https://rdatatable.gitlab.io/data.table/reference/melt.data.table.html) function in R's [data.table](https://rdatatable.gitlab.io/data.table/). A couple of examples should make this clear.\n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/61138600/tidy-dataset-with-pivot-longer-multiple-columns-into-two-columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>actor_1</th>\n",
       "      <th>actor_2</th>\n",
       "      <th>actor_3</th>\n",
       "      <th>actor_1_FB_likes</th>\n",
       "      <th>actor_2_FB_likes</th>\n",
       "      <th>actor_3_FB_likes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>CCH_Pound…</td>\n",
       "      <td>Joel_Davi…</td>\n",
       "      <td>Wes_Studi</td>\n",
       "      <td>1000</td>\n",
       "      <td>936</td>\n",
       "      <td>855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Johnny_De…</td>\n",
       "      <td>Orlando_B…</td>\n",
       "      <td>Jack_Daven…</td>\n",
       "      <td>40000</td>\n",
       "      <td>5000</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Tom_Hardy</td>\n",
       "      <td>Christian…</td>\n",
       "      <td>Joseph_Gor…</td>\n",
       "      <td>27000</td>\n",
       "      <td>23000</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Daryl_Sab…</td>\n",
       "      <td>Samantha_…</td>\n",
       "      <td>Polly_Walk…</td>\n",
       "      <td>640</td>\n",
       "      <td>632</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>J.K._Simm…</td>\n",
       "      <td>James_Fra…</td>\n",
       "      <td>Kirsten_Du…</td>\n",
       "      <td>24000</td>\n",
       "      <td>11000</td>\n",
       "      <td>4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>Brad_Garr…</td>\n",
       "      <td>Donna_Mur…</td>\n",
       "      <td>M.C._Gainey</td>\n",
       "      <td>799</td>\n",
       "      <td>553</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 title     actor_1     actor_2      actor_3  actor_1_FB_likes  \\\n",
       "0               Avatar  CCH_Pound…  Joel_Davi…    Wes_Studi              1000   \n",
       "1  Pirates_of_the_Car…  Johnny_De…  Orlando_B…  Jack_Daven…             40000   \n",
       "2  The_Dark_Knight_Ri…   Tom_Hardy  Christian…  Joseph_Gor…             27000   \n",
       "3          John_Carter  Daryl_Sab…  Samantha_…  Polly_Walk…               640   \n",
       "4         Spider-Man_3  J.K._Simm…  James_Fra…  Kirsten_Du…             24000   \n",
       "5              Tangled  Brad_Garr…  Donna_Mur…  M.C._Gainey               799   \n",
       "\n",
       "   actor_2_FB_likes  actor_3_FB_likes  \n",
       "0               936               855  \n",
       "1              5000              1000  \n",
       "2             23000             23000  \n",
       "3               632               530  \n",
       "4             11000              4000  \n",
       "5               553               284  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    [\n",
    "        {\n",
    "            \"title\": \"Avatar\",\n",
    "            \"actor_1\": \"CCH_Pound…\",\n",
    "            \"actor_2\": \"Joel_Davi…\",\n",
    "            \"actor_3\": \"Wes_Studi\",\n",
    "            \"actor_1_FB_likes\": 1000,\n",
    "            \"actor_2_FB_likes\": 936,\n",
    "            \"actor_3_FB_likes\": 855,\n",
    "        },\n",
    "        {\n",
    "            \"title\": \"Pirates_of_the_Car…\",\n",
    "            \"actor_1\": \"Johnny_De…\",\n",
    "            \"actor_2\": \"Orlando_B…\",\n",
    "            \"actor_3\": \"Jack_Daven…\",\n",
    "            \"actor_1_FB_likes\": 40000,\n",
    "            \"actor_2_FB_likes\": 5000,\n",
    "            \"actor_3_FB_likes\": 1000,\n",
    "        },\n",
    "        {\n",
    "            \"title\": \"The_Dark_Knight_Ri…\",\n",
    "            \"actor_1\": \"Tom_Hardy\",\n",
    "            \"actor_2\": \"Christian…\",\n",
    "            \"actor_3\": \"Joseph_Gor…\",\n",
    "            \"actor_1_FB_likes\": 27000,\n",
    "            \"actor_2_FB_likes\": 23000,\n",
    "            \"actor_3_FB_likes\": 23000,\n",
    "        },\n",
    "        {\n",
    "            \"title\": \"John_Carter\",\n",
    "            \"actor_1\": \"Daryl_Sab…\",\n",
    "            \"actor_2\": \"Samantha_…\",\n",
    "            \"actor_3\": \"Polly_Walk…\",\n",
    "            \"actor_1_FB_likes\": 640,\n",
    "            \"actor_2_FB_likes\": 632,\n",
    "            \"actor_3_FB_likes\": 530,\n",
    "        },\n",
    "        {\n",
    "            \"title\": \"Spider-Man_3\",\n",
    "            \"actor_1\": \"J.K._Simm…\",\n",
    "            \"actor_2\": \"James_Fra…\",\n",
    "            \"actor_3\": \"Kirsten_Du…\",\n",
    "            \"actor_1_FB_likes\": 24000,\n",
    "            \"actor_2_FB_likes\": 11000,\n",
    "            \"actor_3_FB_likes\": 4000,\n",
    "        },\n",
    "        {\n",
    "            \"title\": \"Tangled\",\n",
    "            \"actor_1\": \"Brad_Garr…\",\n",
    "            \"actor_2\": \"Donna_Mur…\",\n",
    "            \"actor_3\": \"M.C._Gainey\",\n",
    "            \"actor_1_FB_likes\": 799,\n",
    "            \"actor_2_FB_likes\": 553,\n",
    "            \"actor_3_FB_likes\": 284,\n",
    "        },\n",
    "    ]\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we have a dataframe of movie titles, actors, and their facebook likes. It would be great if we could transform this into a long form, with just the title, the actor names, and the number of likes. Let's look at a possible solution : \n",
    "\n",
    "First, we reshape the columns, so that the numbers appear at the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actor_1</th>\n",
       "      <th>actor_2</th>\n",
       "      <th>actor_3</th>\n",
       "      <th>actor_FB_likes_1</th>\n",
       "      <th>actor_FB_likes_2</th>\n",
       "      <th>actor_FB_likes_3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Avatar</th>\n",
       "      <td>CCH_Pound…</td>\n",
       "      <td>Joel_Davi…</td>\n",
       "      <td>Wes_Studi</td>\n",
       "      <td>1000</td>\n",
       "      <td>936</td>\n",
       "      <td>855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pirates_of_the_Car…</th>\n",
       "      <td>Johnny_De…</td>\n",
       "      <td>Orlando_B…</td>\n",
       "      <td>Jack_Daven…</td>\n",
       "      <td>40000</td>\n",
       "      <td>5000</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The_Dark_Knight_Ri…</th>\n",
       "      <td>Tom_Hardy</td>\n",
       "      <td>Christian…</td>\n",
       "      <td>Joseph_Gor…</td>\n",
       "      <td>27000</td>\n",
       "      <td>23000</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John_Carter</th>\n",
       "      <td>Daryl_Sab…</td>\n",
       "      <td>Samantha_…</td>\n",
       "      <td>Polly_Walk…</td>\n",
       "      <td>640</td>\n",
       "      <td>632</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spider-Man_3</th>\n",
       "      <td>J.K._Simm…</td>\n",
       "      <td>James_Fra…</td>\n",
       "      <td>Kirsten_Du…</td>\n",
       "      <td>24000</td>\n",
       "      <td>11000</td>\n",
       "      <td>4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tangled</th>\n",
       "      <td>Brad_Garr…</td>\n",
       "      <td>Donna_Mur…</td>\n",
       "      <td>M.C._Gainey</td>\n",
       "      <td>799</td>\n",
       "      <td>553</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        actor_1     actor_2      actor_3  actor_FB_likes_1  \\\n",
       "title                                                                        \n",
       "Avatar               CCH_Pound…  Joel_Davi…    Wes_Studi              1000   \n",
       "Pirates_of_the_Car…  Johnny_De…  Orlando_B…  Jack_Daven…             40000   \n",
       "The_Dark_Knight_Ri…   Tom_Hardy  Christian…  Joseph_Gor…             27000   \n",
       "John_Carter          Daryl_Sab…  Samantha_…  Polly_Walk…               640   \n",
       "Spider-Man_3         J.K._Simm…  James_Fra…  Kirsten_Du…             24000   \n",
       "Tangled              Brad_Garr…  Donna_Mur…  M.C._Gainey               799   \n",
       "\n",
       "                     actor_FB_likes_2  actor_FB_likes_3  \n",
       "title                                                    \n",
       "Avatar                            936               855  \n",
       "Pirates_of_the_Car…              5000              1000  \n",
       "The_Dark_Knight_Ri…             23000             23000  \n",
       "John_Carter                       632               530  \n",
       "Spider-Man_3                    11000              4000  \n",
       "Tangled                           553               284  "
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.set_index(\"title\")\n",
    "header = [re.split(r\"(_?\\d)\", column) for column in df1]\n",
    "df1.columns = [f\"{first}{last}{middle}\" for first, middle, last in header]\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can reshape, using [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>actor</th>\n",
       "      <th>num_likes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th>group</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Avatar</th>\n",
       "      <th>1</th>\n",
       "      <td>CCH_Pound…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pirates_of_the_Car…</th>\n",
       "      <th>1</th>\n",
       "      <td>Johnny_De…</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The_Dark_Knight_Ri…</th>\n",
       "      <th>1</th>\n",
       "      <td>Tom_Hardy</td>\n",
       "      <td>27000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John_Carter</th>\n",
       "      <th>1</th>\n",
       "      <td>Daryl_Sab…</td>\n",
       "      <td>640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spider-Man_3</th>\n",
       "      <th>1</th>\n",
       "      <td>J.K._Simm…</td>\n",
       "      <td>24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tangled</th>\n",
       "      <th>1</th>\n",
       "      <td>Brad_Garr…</td>\n",
       "      <td>799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Avatar</th>\n",
       "      <th>2</th>\n",
       "      <td>Joel_Davi…</td>\n",
       "      <td>936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pirates_of_the_Car…</th>\n",
       "      <th>2</th>\n",
       "      <td>Orlando_B…</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The_Dark_Knight_Ri…</th>\n",
       "      <th>2</th>\n",
       "      <td>Christian…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John_Carter</th>\n",
       "      <th>2</th>\n",
       "      <td>Samantha_…</td>\n",
       "      <td>632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spider-Man_3</th>\n",
       "      <th>2</th>\n",
       "      <td>James_Fra…</td>\n",
       "      <td>11000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tangled</th>\n",
       "      <th>2</th>\n",
       "      <td>Donna_Mur…</td>\n",
       "      <td>553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Avatar</th>\n",
       "      <th>3</th>\n",
       "      <td>Wes_Studi</td>\n",
       "      <td>855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pirates_of_the_Car…</th>\n",
       "      <th>3</th>\n",
       "      <td>Jack_Daven…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The_Dark_Knight_Ri…</th>\n",
       "      <th>3</th>\n",
       "      <td>Joseph_Gor…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>John_Carter</th>\n",
       "      <th>3</th>\n",
       "      <td>Polly_Walk…</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spider-Man_3</th>\n",
       "      <th>3</th>\n",
       "      <td>Kirsten_Du…</td>\n",
       "      <td>4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tangled</th>\n",
       "      <th>3</th>\n",
       "      <td>M.C._Gainey</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 actor  num_likes\n",
       "title               group                        \n",
       "Avatar              1       CCH_Pound…       1000\n",
       "Pirates_of_the_Car… 1       Johnny_De…      40000\n",
       "The_Dark_Knight_Ri… 1        Tom_Hardy      27000\n",
       "John_Carter         1       Daryl_Sab…        640\n",
       "Spider-Man_3        1       J.K._Simm…      24000\n",
       "Tangled             1       Brad_Garr…        799\n",
       "Avatar              2       Joel_Davi…        936\n",
       "Pirates_of_the_Car… 2       Orlando_B…       5000\n",
       "The_Dark_Knight_Ri… 2       Christian…      23000\n",
       "John_Carter         2       Samantha_…        632\n",
       "Spider-Man_3        2       James_Fra…      11000\n",
       "Tangled             2       Donna_Mur…        553\n",
       "Avatar              3        Wes_Studi        855\n",
       "Pirates_of_the_Car… 3      Jack_Daven…       1000\n",
       "The_Dark_Knight_Ri… 3      Joseph_Gor…      23000\n",
       "John_Carter         3      Polly_Walk…        530\n",
       "Spider-Man_3        3      Kirsten_Du…       4000\n",
       "Tangled             3      M.C._Gainey        284"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(\n",
    "    pd.wide_to_long(\n",
    "        df1.reset_index(),\n",
    "        stubnames=[\"actor\", \"actor_FB_likes\"],\n",
    "        i=\"title\",\n",
    "        j=\"group\",\n",
    "        sep=\"_\",\n",
    "    ).rename(columns={\"actor_FB_likes\": \"num_likes\"})\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can achieve this by using `.value` multiple times, and then renaming the columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>actor</th>\n",
       "      <th>num_likes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>CCH_Pound…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Johnny_De…</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Tom_Hardy</td>\n",
       "      <td>27000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Daryl_Sab…</td>\n",
       "      <td>640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>J.K._Simm…</td>\n",
       "      <td>24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>Brad_Garr…</td>\n",
       "      <td>799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Joel_Davi…</td>\n",
       "      <td>936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Orlando_B…</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Christian…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Samantha_…</td>\n",
       "      <td>632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>James_Fra…</td>\n",
       "      <td>11000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>Donna_Mur…</td>\n",
       "      <td>553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Wes_Studi</td>\n",
       "      <td>855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Jack_Daven…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Joseph_Gor…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Polly_Walk…</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>Kirsten_Du…</td>\n",
       "      <td>4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>M.C._Gainey</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  title        actor  num_likes\n",
       "0                Avatar   CCH_Pound…       1000\n",
       "1   Pirates_of_the_Car…   Johnny_De…      40000\n",
       "2   The_Dark_Knight_Ri…    Tom_Hardy      27000\n",
       "3           John_Carter   Daryl_Sab…        640\n",
       "4          Spider-Man_3   J.K._Simm…      24000\n",
       "5               Tangled   Brad_Garr…        799\n",
       "6                Avatar   Joel_Davi…        936\n",
       "7   Pirates_of_the_Car…   Orlando_B…       5000\n",
       "8   The_Dark_Knight_Ri…   Christian…      23000\n",
       "9           John_Carter   Samantha_…        632\n",
       "10         Spider-Man_3   James_Fra…      11000\n",
       "11              Tangled   Donna_Mur…        553\n",
       "12               Avatar    Wes_Studi        855\n",
       "13  Pirates_of_the_Car…  Jack_Daven…       1000\n",
       "14  The_Dark_Knight_Ri…  Joseph_Gor…      23000\n",
       "15          John_Carter  Polly_Walk…        530\n",
       "16         Spider-Man_3  Kirsten_Du…       4000\n",
       "17              Tangled  M.C._Gainey        284"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(\n",
    "    df.pivot_longer(\n",
    "        index=\"title\",\n",
    "        names_to=(\".value\", \".value\"),\n",
    "        names_pattern=r\"(.+)_\\d(.*)\",\n",
    "    ).rename(columns={\"actor_FB_likes\": \"num_likes\"})\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we could just get our data in long form without the massaging? We know our data has a pattern to it --> it either ends in a number or *likes*.  Can't we take advantage of that? Yes, we can (I know, I know; it sounds like a campaign slogan 🤪)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>actor</th>\n",
       "      <th>num_likes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>CCH_Pound…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Johnny_De…</td>\n",
       "      <td>40000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Tom_Hardy</td>\n",
       "      <td>27000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Daryl_Sab…</td>\n",
       "      <td>640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>J.K._Simm…</td>\n",
       "      <td>24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>Brad_Garr…</td>\n",
       "      <td>799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Joel_Davi…</td>\n",
       "      <td>936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Orlando_B…</td>\n",
       "      <td>5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Christian…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Samantha_…</td>\n",
       "      <td>632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>James_Fra…</td>\n",
       "      <td>11000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>Donna_Mur…</td>\n",
       "      <td>553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Wes_Studi</td>\n",
       "      <td>855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Pirates_of_the_Car…</td>\n",
       "      <td>Jack_Daven…</td>\n",
       "      <td>1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>The_Dark_Knight_Ri…</td>\n",
       "      <td>Joseph_Gor…</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>John_Carter</td>\n",
       "      <td>Polly_Walk…</td>\n",
       "      <td>530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Spider-Man_3</td>\n",
       "      <td>Kirsten_Du…</td>\n",
       "      <td>4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Tangled</td>\n",
       "      <td>M.C._Gainey</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  title        actor  num_likes\n",
       "0                Avatar   CCH_Pound…       1000\n",
       "1   Pirates_of_the_Car…   Johnny_De…      40000\n",
       "2   The_Dark_Knight_Ri…    Tom_Hardy      27000\n",
       "3           John_Carter   Daryl_Sab…        640\n",
       "4          Spider-Man_3   J.K._Simm…      24000\n",
       "5               Tangled   Brad_Garr…        799\n",
       "6                Avatar   Joel_Davi…        936\n",
       "7   Pirates_of_the_Car…   Orlando_B…       5000\n",
       "8   The_Dark_Knight_Ri…   Christian…      23000\n",
       "9           John_Carter   Samantha_…        632\n",
       "10         Spider-Man_3   James_Fra…      11000\n",
       "11              Tangled   Donna_Mur…        553\n",
       "12               Avatar    Wes_Studi        855\n",
       "13  Pirates_of_the_Car…  Jack_Daven…       1000\n",
       "14  The_Dark_Knight_Ri…  Joseph_Gor…      23000\n",
       "15          John_Carter  Polly_Walk…        530\n",
       "16         Spider-Man_3  Kirsten_Du…       4000\n",
       "17              Tangled  M.C._Gainey        284"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"title\",\n",
    "    names_to=(\"actor\", \"num_likes\"),\n",
    "    names_pattern=(r\"\\d$\", r\"likes$\"),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A pairing of `names_to` and `names_pattern` results in:\n",
    "\n",
    "    {\"actor\": '\\d$', \"num_likes\": 'likes$'}\n",
    "                                   \n",
    "The first regex looks for columns that end with a number, while the other looks for columns that end with *likes*. [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) will then look for columns that end with a number and lump all the values in those columns under the `actor` column, and also look for columns that end with *like* and combine all the values in those columns into a new column -> `num_likes`. Underneath the hood, [numpy select](https://numpy.org/doc/stable/reference/generated/numpy.select.html) and [pd.Series.str.contains](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.contains.html) are used to pull apart the columns into the new columns. \n",
    "\n",
    "Again, it is about the goal; we are not interested in the numbers (1,2,3), we only need the names of the actors, and their facebook likes. [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) aims to give as much flexibility as possible, in addition to ease of use, to allow the end user focus on the task. \n",
    "\n",
    "Let's take a look at another example. [Source Data](https://stackoverflow.com/questions/60439749/pair-wise-melt-in-pandas-dataframe) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>Name</th>\n",
       "      <th>code</th>\n",
       "      <th>code1</th>\n",
       "      <th>code2</th>\n",
       "      <th>type</th>\n",
       "      <th>type1</th>\n",
       "      <th>type2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>ABC</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>8</td>\n",
       "      <td>S</td>\n",
       "      <td>E</td>\n",
       "      <td>T</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>XYZ</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>R</td>\n",
       "      <td>NaN</td>\n",
       "      <td>U</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id Name  code  code1 code2 type type1 type2\n",
       "0   0  ABC     1    4.0     8    S     E     T\n",
       "1   1  XYZ     2    NaN     5    R   NaN     U"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"id\": [0, 1],\n",
    "        \"Name\": [\"ABC\", \"XYZ\"],\n",
    "        \"code\": [1, 2],\n",
    "        \"code1\": [4, np.nan],\n",
    "        \"code2\": [\"8\", 5],\n",
    "        \"type\": [\"S\", \"R\"],\n",
    "        \"type1\": [\"E\", np.nan],\n",
    "        \"type2\": [\"T\", \"U\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We cannot directly use [pd.wide_to_long](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.wide_to_long.html) here without some massaging, as there is no definite suffix(the first `code` does not have a suffix), neither can we use `.value` here, again because there is no suffix. However, we can see a pattern where some columns start with `code`, and others start with `type`. Let's see how [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) solves this, using a sequence of regular expressions in the `names_pattern` argument : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>Name</th>\n",
       "      <th>code_all</th>\n",
       "      <th>type_all</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>ABC</td>\n",
       "      <td>1</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>XYZ</td>\n",
       "      <td>2</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>ABC</td>\n",
       "      <td>4.0</td>\n",
       "      <td>E</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>XYZ</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>ABC</td>\n",
       "      <td>8</td>\n",
       "      <td>T</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>XYZ</td>\n",
       "      <td>5</td>\n",
       "      <td>U</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id Name code_all type_all\n",
       "0   0  ABC        1        S\n",
       "1   1  XYZ        2        R\n",
       "2   0  ABC      4.0        E\n",
       "3   1  XYZ      NaN      NaN\n",
       "4   0  ABC        8        T\n",
       "5   1  XYZ        5        U"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"id\", \"Name\"],\n",
    "    names_to=(\"code_all\", \"type_all\"),\n",
    "    names_pattern=(\"^code\", \"^type\"),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The key here is passing the right regular expression, and ensuring the names in `names_to` is paired with the right regex in `names_pattern`; as such, every column that starts with `code` will be included in the new `code_all` column; the same happens to the `type_all` column. Easy and flexible, right? \n",
    "\n",
    "Let's explore another example, from [Stack Overflow](https://stackoverflow.com/questions/12466493/reshaping-multiple-sets-of-measurement-columns-wide-format-into-single-columns) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>DateRange1Start</th>\n",
       "      <th>DateRange1End</th>\n",
       "      <th>Value1</th>\n",
       "      <th>DateRange2Start</th>\n",
       "      <th>DateRange2End</th>\n",
       "      <th>Value2</th>\n",
       "      <th>DateRange3Start</th>\n",
       "      <th>DateRange3End</th>\n",
       "      <th>Value3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1/1/90</td>\n",
       "      <td>3/1/90</td>\n",
       "      <td>4.4</td>\n",
       "      <td>4/5/91</td>\n",
       "      <td>6/7/91</td>\n",
       "      <td>6.2</td>\n",
       "      <td>5/5/95</td>\n",
       "      <td>6/6/96</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID DateRange1Start DateRange1End  Value1 DateRange2Start DateRange2End  \\\n",
       "0   1          1/1/90        3/1/90     4.4          4/5/91        6/7/91   \n",
       "\n",
       "   Value2 DateRange3Start DateRange3End  Value3  \n",
       "0     6.2          5/5/95        6/6/96     3.3  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    [\n",
    "        {\n",
    "            \"ID\": 1,\n",
    "            \"DateRange1Start\": \"1/1/90\",\n",
    "            \"DateRange1End\": \"3/1/90\",\n",
    "            \"Value1\": 4.4,\n",
    "            \"DateRange2Start\": \"4/5/91\",\n",
    "            \"DateRange2End\": \"6/7/91\",\n",
    "            \"Value2\": 6.2,\n",
    "            \"DateRange3Start\": \"5/5/95\",\n",
    "            \"DateRange3End\": \"6/6/96\",\n",
    "            \"Value3\": 3.3,\n",
    "        }\n",
    "    ]\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the dataframe above, we need to reshape the data to have a start date, end date and value. For the `DateRange` columns, the numbers are embedded within the string, while for `value` it is appended at the end. One possible solution is to reshape the columns so that the numbers are at the end :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DateRangeStart1</th>\n",
       "      <th>DateRangeEnd1</th>\n",
       "      <th>Value1</th>\n",
       "      <th>DateRangeStart2</th>\n",
       "      <th>DateRangeEnd2</th>\n",
       "      <th>Value2</th>\n",
       "      <th>DateRangeStart3</th>\n",
       "      <th>DateRangeEnd3</th>\n",
       "      <th>Value3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1/1/90</td>\n",
       "      <td>3/1/90</td>\n",
       "      <td>4.4</td>\n",
       "      <td>4/5/91</td>\n",
       "      <td>6/7/91</td>\n",
       "      <td>6.2</td>\n",
       "      <td>5/5/95</td>\n",
       "      <td>6/6/96</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   DateRangeStart1 DateRangeEnd1  Value1 DateRangeStart2 DateRangeEnd2  \\\n",
       "ID                                                                       \n",
       "1           1/1/90        3/1/90     4.4          4/5/91        6/7/91   \n",
       "\n",
       "    Value2 DateRangeStart3 DateRangeEnd3  Value3  \n",
       "ID                                                \n",
       "1      6.2          5/5/95        6/6/96     3.3  "
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df.set_index(\"ID\")\n",
    "header = [re.split(r\"(\\d)\", column) for column in df1]\n",
    "df1.columns = [f\"{first}{last}{middle}\" for first, middle, last in header]\n",
    "df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can unpivot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>DateRangeStart</th>\n",
       "      <th>DateRangeEnd</th>\n",
       "      <th>Value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ID</th>\n",
       "      <th>num</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>1</th>\n",
       "      <td>1/1/90</td>\n",
       "      <td>3/1/90</td>\n",
       "      <td>4.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4/5/91</td>\n",
       "      <td>6/7/91</td>\n",
       "      <td>6.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/5/95</td>\n",
       "      <td>6/6/96</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       DateRangeStart DateRangeEnd  Value\n",
       "ID num                                   \n",
       "1  1           1/1/90       3/1/90    4.4\n",
       "   2           4/5/91       6/7/91    6.2\n",
       "   3           5/5/95       6/6/96    3.3"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.wide_to_long(\n",
    "    df1.reset_index(),\n",
    "    stubnames=[\"DateRangeStart\", \"DateRangeEnd\", \"Value\"],\n",
    "    i=\"ID\",\n",
    "    j=\"num\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we could allow pivot_longer worry about the massaging; simply pass to `names_pattern` a list of regular expressions that match what we are after : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>DateRangeStart</th>\n",
       "      <th>DateRangeEnd</th>\n",
       "      <th>Value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1/1/90</td>\n",
       "      <td>3/1/90</td>\n",
       "      <td>4.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4/5/91</td>\n",
       "      <td>6/7/91</td>\n",
       "      <td>6.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5/5/95</td>\n",
       "      <td>6/6/96</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID DateRangeStart DateRangeEnd  Value\n",
       "0   1         1/1/90       3/1/90    4.4\n",
       "1   1         4/5/91       6/7/91    6.2\n",
       "2   1         5/5/95       6/6/96    3.3"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"ID\",\n",
    "    names_to=(\"DateRangeStart\", \"DateRangeEnd\", \"Value\"),\n",
    "    names_pattern=(\"Start$\", \"End$\", \"^Value\"),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code above looks for columns that end with *Start*(`Start$`), aggregates all the values in those columns into `DateRangeStart` column, looks for columns that end with *End*(`End$`), aggregates all the values within those columns into `DateRangeEnd` column, and finally looks for columns that start with *Value*(`^Value`), and aggregates the values in those columns into the `Value` column. Just know the patterns, and pair them accordingly. Again, the goal is a focus on the task, to make it simple for the end user."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that for the example above, you can use multiple `.value` to get the data out: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>DateRangeStart</th>\n",
       "      <th>DateRangeEnd</th>\n",
       "      <th>Value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1/1/90</td>\n",
       "      <td>3/1/90</td>\n",
       "      <td>4.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4/5/91</td>\n",
       "      <td>6/7/91</td>\n",
       "      <td>6.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>5/5/95</td>\n",
       "      <td>6/6/96</td>\n",
       "      <td>3.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID DateRangeStart DateRangeEnd  Value\n",
       "0   1         1/1/90       3/1/90    4.4\n",
       "1   1         4/5/91       6/7/91    6.2\n",
       "2   1         5/5/95       6/6/96    3.3"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    \"ID\", names_to=(\".value\", \".value\"), names_pattern=r\"(.+)\\d(.*)\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at another example [Source Data](https://stackoverflow.com/questions/64316129/how-to-efficiently-melt-multiple-columns-using-the-module-melt-in-pandas/64316306#64316306) :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Activity</th>\n",
       "      <th>General</th>\n",
       "      <th>m1</th>\n",
       "      <th>t1</th>\n",
       "      <th>m2</th>\n",
       "      <th>t2</th>\n",
       "      <th>m3</th>\n",
       "      <th>t3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>A1</td>\n",
       "      <td>TA1</td>\n",
       "      <td>A2</td>\n",
       "      <td>TA2</td>\n",
       "      <td>A3</td>\n",
       "      <td>TA3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>B1</td>\n",
       "      <td>TB1</td>\n",
       "      <td>B2</td>\n",
       "      <td>TB2</td>\n",
       "      <td>B3</td>\n",
       "      <td>TB3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Activity General  m1   t1  m2   t2  m3   t3\n",
       "0       P1      AA  A1  TA1  A2  TA2  A3  TA3\n",
       "1       P2      BB  B1  TB1  B2  TB2  B3  TB3"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"Activity\": [\"P1\", \"P2\"],\n",
    "        \"General\": [\"AA\", \"BB\"],\n",
    "        \"m1\": [\"A1\", \"B1\"],\n",
    "        \"t1\": [\"TA1\", \"TB1\"],\n",
    "        \"m2\": [\"A2\", \"B2\"],\n",
    "        \"t2\": [\"TA2\", \"TB2\"],\n",
    "        \"m3\": [\"A3\", \"B3\"],\n",
    "        \"t3\": [\"TA3\", \"TB3\"],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a [solution](https://stackoverflow.com/a/64316306/7175713) provided by yours truly : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Activity</th>\n",
       "      <th>General</th>\n",
       "      <th>Task</th>\n",
       "      <th>M</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>TA1</td>\n",
       "      <td>A1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>TA2</td>\n",
       "      <td>A2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>TA3</td>\n",
       "      <td>A3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>TB1</td>\n",
       "      <td>B1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>TB2</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>TB3</td>\n",
       "      <td>B3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Activity General Task   M\n",
       "0       P1      AA  TA1  A1\n",
       "1       P1      AA  TA2  A2\n",
       "2       P1      AA  TA3  A3\n",
       "3       P2      BB  TB1  B1\n",
       "4       P2      BB  TB2  B2\n",
       "5       P2      BB  TB3  B3"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(\n",
    "    pd.wide_to_long(\n",
    "        df, i=[\"Activity\", \"General\"], stubnames=[\"t\", \"m\"], j=\"number\"\n",
    "    )\n",
    "    .set_axis([\"Task\", \"M\"], axis=\"columns\")\n",
    "    .droplevel(-1)\n",
    "    .reset_index()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we could use [pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html), abstract the details, and focus on the task : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Activity</th>\n",
       "      <th>General</th>\n",
       "      <th>M</th>\n",
       "      <th>Task</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>A1</td>\n",
       "      <td>TA1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>B1</td>\n",
       "      <td>TB1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>A2</td>\n",
       "      <td>TA2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>B2</td>\n",
       "      <td>TB2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>P1</td>\n",
       "      <td>AA</td>\n",
       "      <td>A3</td>\n",
       "      <td>TA3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>P2</td>\n",
       "      <td>BB</td>\n",
       "      <td>B3</td>\n",
       "      <td>TB3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Activity General   M Task\n",
       "0       P1      AA  A1  TA1\n",
       "1       P2      BB  B1  TB1\n",
       "2       P1      AA  A2  TA2\n",
       "3       P2      BB  B2  TB2\n",
       "4       P1      AA  A3  TA3\n",
       "5       P2      BB  B3  TB3"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"Activity\", \"General\"],\n",
    "    names_pattern=[\"^m\", \"^t\"],\n",
    "    names_to=[\"M\", \"Task\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright, one last example : \n",
    "\n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/64159054/how-do-you-pivot-longer-columns-in-groups)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>activity1</th>\n",
       "      <th>number_activity_1</th>\n",
       "      <th>attendees1</th>\n",
       "      <th>activity2</th>\n",
       "      <th>number_activity_2</th>\n",
       "      <th>attendees2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Birthday</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>Sleep Over</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Chris</td>\n",
       "      <td>Sleep Over</td>\n",
       "      <td>2</td>\n",
       "      <td>18</td>\n",
       "      <td>Painting</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alex</td>\n",
       "      <td>Track Race</td>\n",
       "      <td>4</td>\n",
       "      <td>100</td>\n",
       "      <td>Birthday</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Name   activity1  number_activity_1  attendees1   activity2  \\\n",
       "0   John    Birthday                  1          14  Sleep Over   \n",
       "1  Chris  Sleep Over                  2          18    Painting   \n",
       "2   Alex  Track Race                  4         100    Birthday   \n",
       "\n",
       "   number_activity_2  attendees2  \n",
       "0                  4          10  \n",
       "1                  5           8  \n",
       "2                  1           5  "
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"Name\": [\"John\", \"Chris\", \"Alex\"],\n",
    "        \"activity1\": [\"Birthday\", \"Sleep Over\", \"Track Race\"],\n",
    "        \"number_activity_1\": [1, 2, 4],\n",
    "        \"attendees1\": [14, 18, 100],\n",
    "        \"activity2\": [\"Sleep Over\", \"Painting\", \"Birthday\"],\n",
    "        \"number_activity_2\": [4, 5, 1],\n",
    "        \"attendees2\": [10, 8, 5],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The task here is to unpivot the data, and group the data under three new columns (\"activity\", \"number_activity\", and \"attendees\"). \n",
    "\n",
    "We can see that there is a pattern to the data; let's create a list of regular expressions that match the patterns and pass to `names_pattern``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>activity</th>\n",
       "      <th>number_activity</th>\n",
       "      <th>attendees</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>John</td>\n",
       "      <td>Birthday</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Chris</td>\n",
       "      <td>Sleep Over</td>\n",
       "      <td>2</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Alex</td>\n",
       "      <td>Track Race</td>\n",
       "      <td>4</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>John</td>\n",
       "      <td>Sleep Over</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Chris</td>\n",
       "      <td>Painting</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Alex</td>\n",
       "      <td>Birthday</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Name    activity  number_activity  attendees\n",
       "0   John    Birthday                1         14\n",
       "1  Chris  Sleep Over                2         18\n",
       "2   Alex  Track Race                4        100\n",
       "3   John  Sleep Over                4         10\n",
       "4  Chris    Painting                5          8\n",
       "5   Alex    Birthday                1          5"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=\"Name\",\n",
    "    names_to=(\"activity\", \"number_activity\", \"attendees\"),\n",
    "    names_pattern=(\"^activity\", \"^number_activity\", \"^attendees\"),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright, let's look at one final example:\n",
    "\n",
    "\n",
    "[Source Data](https://stackoverflow.com/questions/60387077/reshaping-and-melting-dataframe-whilst-picking-up-certain-regex)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Location</th>\n",
       "      <th>Account</th>\n",
       "      <th>Y2019:MTD:January:Expense</th>\n",
       "      <th>Y2019:MTD:January:Income</th>\n",
       "      <th>Y2019:MTD:February:Expense</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>ABC</td>\n",
       "      <td>4354</td>\n",
       "      <td>56456</td>\n",
       "      <td>235423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>XYX</td>\n",
       "      <td>769867</td>\n",
       "      <td>32556456</td>\n",
       "      <td>6785423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Rome</td>\n",
       "      <td>ABC</td>\n",
       "      <td>434654</td>\n",
       "      <td>5214</td>\n",
       "      <td>235423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Rome</td>\n",
       "      <td>XYX</td>\n",
       "      <td>632556456</td>\n",
       "      <td>46724423</td>\n",
       "      <td>46588</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Location Account  Y2019:MTD:January:Expense  Y2019:MTD:January:Income  \\\n",
       "0   Madrid     ABC                       4354                     56456   \n",
       "1   Madrid     XYX                     769867                  32556456   \n",
       "2     Rome     ABC                     434654                      5214   \n",
       "3     Rome     XYX                  632556456                  46724423   \n",
       "\n",
       "   Y2019:MTD:February:Expense  \n",
       "0                      235423  \n",
       "1                     6785423  \n",
       "2                      235423  \n",
       "3                       46588  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"Location\": [\"Madrid\", \"Madrid\", \"Rome\", \"Rome\"],\n",
    "        \"Account\": [\"ABC\", \"XYX\", \"ABC\", \"XYX\"],\n",
    "        \"Y2019:MTD:January:Expense\": [4354, 769867, 434654, 632556456],\n",
    "        \"Y2019:MTD:January:Income\": [56456, 32556456, 5214, 46724423],\n",
    "        \"Y2019:MTD:February:Expense\": [235423, 6785423, 235423, 46588],\n",
    "    }\n",
    ")\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Location</th>\n",
       "      <th>Account</th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>Expense</th>\n",
       "      <th>Income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>ABC</td>\n",
       "      <td>2019</td>\n",
       "      <td>Jan</td>\n",
       "      <td>4354</td>\n",
       "      <td>56456.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>ABC</td>\n",
       "      <td>2019</td>\n",
       "      <td>Feb</td>\n",
       "      <td>235423</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>XYX</td>\n",
       "      <td>2019</td>\n",
       "      <td>Jan</td>\n",
       "      <td>769867</td>\n",
       "      <td>32556456.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>XYX</td>\n",
       "      <td>2019</td>\n",
       "      <td>Feb</td>\n",
       "      <td>6785423</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Rome</td>\n",
       "      <td>ABC</td>\n",
       "      <td>2019</td>\n",
       "      <td>Jan</td>\n",
       "      <td>434654</td>\n",
       "      <td>5214.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Rome</td>\n",
       "      <td>ABC</td>\n",
       "      <td>2019</td>\n",
       "      <td>Feb</td>\n",
       "      <td>235423</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Rome</td>\n",
       "      <td>XYX</td>\n",
       "      <td>2019</td>\n",
       "      <td>Jan</td>\n",
       "      <td>632556456</td>\n",
       "      <td>46724423.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Rome</td>\n",
       "      <td>XYX</td>\n",
       "      <td>2019</td>\n",
       "      <td>Feb</td>\n",
       "      <td>46588</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Location Account  year month    Expense      Income\n",
       "0   Madrid     ABC  2019   Jan       4354     56456.0\n",
       "1   Madrid     ABC  2019   Feb     235423         NaN\n",
       "2   Madrid     XYX  2019   Jan     769867  32556456.0\n",
       "3   Madrid     XYX  2019   Feb    6785423         NaN\n",
       "4     Rome     ABC  2019   Jan     434654      5214.0\n",
       "5     Rome     ABC  2019   Feb     235423         NaN\n",
       "6     Rome     XYX  2019   Jan  632556456  46724423.0\n",
       "7     Rome     XYX  2019   Feb      46588         NaN"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_longer(\n",
    "    index=[\"Location\", \"Account\"],\n",
    "    names_to=(\"year\", \"month\", \".value\"),\n",
    "    names_pattern=r\"Y(.+):MTD:(.{3}).+(Income|Expense)\",\n",
    "    sort_by_appearance=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pivot_longer](https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.pivot_longer.html) does not solve all problems; no function does. Its aim is to make it easy to unpivot dataframes from wide to long form, while offering a lot of flexibility and power."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "98b0a9b7b4eaaa670588a142fd0a9b87eaafe866f1db4228be72b4211d12040f"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit ('base': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}