audio_introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sascha Spors,\n",
    "Professorship Signal Theory and Digital Signal Processing,\n",
    "Institute of Communications Engineering (INT),\n",
    "Faculty of Computer Science and Electrical Engineering (IEF),\n",
    "University of Rostock,\n",
    "Germany\n",
    "\n",
    "# Data Driven Audio Signal Processing - A Tutorial with Computational Examples\n",
    "\n",
    "Master Course #24512\n",
    "\n",
    "- lecture: https://github.com/spatialaudio/data-driven-audio-signal-processing-lecture\n",
    "- tutorial: https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise\n",
    "\n",
    "Feel free to contact lecturer frank.schultz@uni-rostock.de"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Digital Audio Signal Fundamentals\n",
    "\n",
    "TBD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Digital Audio Signal Formats / Parameters\n",
    "\n",
    "- for computer processing **digital** signals are required\n",
    "- analog -> analog-to-digital converter -> digital\n",
    "- typically represented as streams/files\n",
    "    - uncompressed as e.g. PCM\n",
    "    - lossless compressed as e.g. FLAC\n",
    "    - lossy compressed as e.g. AC3 / MP3 / AAC / Vorbis / G711 / G722\n",
    "- encoder\n",
    "    - typically large processing load, since mostly off-line rendering or some latency allowed\n",
    "    - if lossy compression very often a psycho-acoustical model is employed to reduce data\n",
    "- decoder\n",
    "    - typically very low processing load due to low computational capabilities and real time demand\n",
    "- most DSP algorithms work on uncompressed audio data, thus inherent parameters are:\n",
    "    - bit resolution / quantization (typically in the range of 8-24 Bit integer, 32/64 Bit floating)\n",
    "    - sampling frequency (typical: 1/2/4 x 32, 44.1, 48 kHz, sometimes also 8/12/16 kHz)\n",
    "    - number of audio channels (mono, stereo, 5.1, 7.1, 4-128 in microphone arrays, several hundreds in audio productions)\n",
    "- vector / matrix representation\n",
    "\n",
    "### Quantization\n",
    "\n",
    "- uncompressed audio typically uses [Pulse Code Modulation](https://en.wikipedia.org/wiki/Pulse-code_modulation) (PCM) to represent the data digitally\n",
    "- quantization of amplitude values is required\n",
    "- typically done with linear quantizer\n",
    "- number of bits $B$\n",
    "- then there are $2^B$ possible quantization steps, e.g. for $B=8$ this leads to $256$ quantization steps\n",
    "- when assigning **integer** numbers to sample values the convention holds (for the **midtread quantizer**)\n",
    "    - minimum integer is $-(2^{B-1})$, e.g. for $B=8$ this leads to integer -128\n",
    "    - maximum integer is $+(2^{B-1})-1$, e.g. for $B=8$ this leads to integer +127\n",
    "    - zero can be exactly represented\n",
    "    - sample values smaller than $-(2^{B-1})$ / larger than $+(2^{B-1})-1$ will be clipped to the min/max integer\n",
    "- convention for analog-to-digital converters (ADC), digital-to-analog converters (DAC) and audio files is to interpret the samples for the range -1...+1\n",
    "- unless using explicitly a fixed point DSP (still often used for embedded hardware, less power consumption, smaller chip size), nowadays (PC based) processing is performed with floating/double precision\n",
    "- thus scaling the (integer) data might be required, cf. `scale_wav()` below\n",
    "\n",
    "### Read and Plot PCM Wav File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from scipy.io import wavfile\n",
    "\n",
    "\n",
    "def scale_wav(x):\n",
    "    print(\"dtype for wav file content:\", x.dtype)\n",
    "    if x.dtype == \"float32\" or x.dtype == \"float64\":\n",
    "        return x  # already in +-1.0 double range\n",
    "    else:  # we assume integer coded wav files:\n",
    "        tmp = str(x.dtype)\n",
    "        print(\n",
    "            \"quantization bit resolution might be lower than storage bit resolution!\"\n",
    "        )\n",
    "        # normalize to bring to +-1.0 double range\n",
    "        a = 1 / (2 ** (int(tmp[3:]) - 1))\n",
    "        return a * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folder = \"audio_ex01/\"\n",
    "\n",
    "# fs, x = wavfile.read(folder+'sine1k_16Bit.wav')  # integer PCM\n",
    "fs, x = wavfile.read(folder + \"sine1k_24Bit.wav\")  # integer PCM\n",
    "# fs, x = wavfile.read(folder+'sine1k_32Bit.wav')  # float PCM\n",
    "# fs, x = wavfile.read(folder+'sine1k_64Bit.wav')  # double PCM\n",
    "\n",
    "x = scale_wav(x)\n",
    "# to work with x should have double precision unless\n",
    "# special applications require for another format\n",
    "print(\"dtype for read in signal x:\", x.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(9, 10))\n",
    "plt.subplot(3, 1, 1)\n",
    "plt.stem(x[:48], basefmt=\"C0:\", linefmt=\"C0:\", markerfmt=\"C0o\")\n",
    "plt.xlabel(\"sample index k\")\n",
    "plt.ylabel(\"x[k]\")\n",
    "\n",
    "# if required the samples can additionally be normalized\n",
    "# for example to represent analog units, such as sound\n",
    "# pressure if the audio signal was recorded by a\n",
    "# calibrated microphone\n",
    "# furthermore we can plot over the time rather than sample\n",
    "# index\n",
    "# example:\n",
    "k = np.arange(x.size)\n",
    "pressure_norm = 20  # full scale represents 20 Pascal peak\n",
    "plt.subplot(3, 1, 2)\n",
    "plt.stem(\n",
    "    k[:48] / fs * 1000,\n",
    "    x[:48] * pressure_norm,\n",
    "    basefmt=\"C0:\",\n",
    "    linefmt=\"C0:\",\n",
    "    markerfmt=\"C0o\",\n",
    ")\n",
    "plt.xlabel(\"time in ms\")\n",
    "plt.ylabel(\"sound pressure in Pascal\")\n",
    "# or plot with another appropriate labeling of x and y axis\n",
    "plt.subplot(3, 1, 3)\n",
    "plt.stem(\n",
    "    k[:48] / fs * 1000,\n",
    "    x[:48] * pressure_norm,\n",
    "    basefmt=\"C0:\",\n",
    "    linefmt=\"C0:\",\n",
    "    markerfmt=\"C0o\",\n",
    ")\n",
    "plt.xlabel(\"t / ms\")\n",
    "plt.ylabel(\"p(t) / Pa\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Copyright\n",
    "\n",
    "- the notebooks are provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources)\n",
    "- the text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)\n",
    "- the code of the IPython examples is licensed under the [MIT license](https://opensource.org/licenses/MIT)\n",
    "- feel free to use the notebooks for your own purposes\n",
    "- please attribute the work as follows: *Frank Schultz, Data Driven Audio Signal Processing - A Tutorial Featuring Computational Examples, University of Rostock* ideally with relevant file(s), github URL https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise, commit number and/or version tag, year.\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "1743232f157dd0954c61aae30535e75a2972519a625c7e796bafe0cd9a07bf7e"
  },
  "kernelspec": {
   "display_name": "myddasp",
   "language": "python",
   "name": "myddasp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}