notebooks/langchain/langchain-vector-store-using-elser.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Similarity search with Langchain and SparseVectorRetrievalStrategy(ELSER Model)\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/notebooks/langchain/langchain-vector-store-using-elser.ipynb)\n",
    "\n",
    "\n",
    "This workbook demonstrates similiarity search using [SparseVectorRetrievalStrategy](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.SparseRetrievalStrategy.html#langchain.vectorstores.elasticsearch.SparseRetrievalStrategy) (ELSER). First,  we  split the documents into chunks using `langchain` and then index into elasticsearch through [`ElasticsearchStore.from_documents`](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.ElasticsearchStore.html#langchain.vectorstores.elasticsearch.ElasticsearchStore.from_documents). \n",
    "\n",
    "The [SparseVectorRetrievalStrategy](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.SparseRetrievalStrategy.html#langchain.vectorstores.elasticsearch.SparseRetrievalStrategy) converts each document into tokens and would be stored in `vector` field with datatype `rank_features`. Hence, the inference is handled within elasticsearch.\n",
    "\n",
    "The `similarity_search` performs semantic search using `text_expansion` and sets the query.\n",
    "\n",
    "We will then see how to filter the metadata within query. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install packages and import modules\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# install packages\n",
    "!python3 -m pip install -qU langchain langchain-elasticsearch openai tiktoken\n",
    "\n",
    "# import modules\n",
    "from getpass import getpass\n",
    "from langchain_elasticsearch import ElasticsearchStore\n",
    "from urllib.request import urlopen\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to Elasticsearch\n",
    "\n",
    "ℹ️ We're using an Elastic Cloud deployment of Elasticsearch for this notebook. If you don't have an Elastic Cloud deployment, sign up [here](https://cloud.elastic.co/registration?onboarding_token=vectorsearch&utm_source=github&utm_content=elasticsearch-labs-notebook) for a free trial. \n",
    "\n",
    "We'll use the **Cloud ID** to identify our deployment, because we are using Elastic Cloud deployment. To find the Cloud ID for your deployment, go to https://cloud.elastic.co/deployments and select your deployment.\n",
    "\n",
    "\n",
    "We will use [ElasticsearchStore](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.ElasticsearchStore.html) to connect to our elastic cloud deployment, This would help create and index data easily. We will be using text embedding from ELSER model `.elser_model_2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#finding-your-cloud-id\n",
    "ELASTIC_CLOUD_ID = getpass(\"Elastic Cloud ID: \")\n",
    "\n",
    "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#creating-an-api-key\n",
    "ELASTIC_API_KEY = getpass(\"Elastic Api Key: \")\n",
    "\n",
    "vector_store = ElasticsearchStore(\n",
    "    es_cloud_id=ELASTIC_CLOUD_ID,\n",
    "    es_api_key=ELASTIC_API_KEY,\n",
    "    index_name=\"workplace_index\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the dataset \n",
    "\n",
    "Let's download the sample dataset and deserialize the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://raw.githubusercontent.com/elastic/elasticsearch-labs/main/example-apps/chatbot-rag-app/data/data.json\"\n",
    "\n",
    "response = urlopen(url)\n",
    "\n",
    "workplace_docs = json.loads(response.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split Documents into Passages\n",
    "\n",
    "\n",
    "We will chunk these documents into 500 token passages with an overlap of 0 tokens using a simple splitter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = []\n",
    "content = []\n",
    "\n",
    "for doc in workplace_docs:\n",
    "    content.append(doc[\"content\"])\n",
    "    metadata.append(\n",
    "        {\n",
    "            \"name\": doc[\"name\"],\n",
    "            \"summary\": doc[\"summary\"],\n",
    "            \"rolePermissions\": doc[\"rolePermissions\"],\n",
    "        }\n",
    "    )\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(\n",
    "    chunk_size=512, chunk_overlap=256\n",
    ")\n",
    "docs = text_splitter.create_documents(content, metadatas=metadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index data into elasticsearch\n",
    "\n",
    "Next, we will index data to elasticsearch using [ElasticsearchStore.from_documents](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.ElasticsearchStore.html#langchain.vectorstores.elasticsearch.ElasticsearchStore.from_documents). We will use Cloud ID,  Password and Index name values set in the `Create cloud deployment` step.\n",
    "\n",
    "In the instance, we will set `strategy` to [ElasticsearchStore.SparseVectorRetrievalStrategy()](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.elasticsearch.SparseRetrievalStrategy.html#langchain.vectorstores.elasticsearch.SparseRetrievalStrategy)\n",
    "\n",
    "Note: Before we begin indexing, ensure you have [downloaded and deployed ELSER model](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html#download-deploy-elser) in your deployment and is running in ml node. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "documents = vector_store.from_documents(\n",
    "    docs,\n",
    "    es_cloud_id=ELASTIC_CLOUD_ID,\n",
    "    es_api_key=ELASTIC_API_KEY,\n",
    "    index_name=\"workplace_index\",\n",
    "    strategy=ElasticsearchStore.SparseVectorRetrievalStrategy(\n",
    "        model_id=\".elser_model_2\"\n",
    "    ),\n",
    "    bulk_kwargs={\n",
    "        \"request_timeout\": 60,\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results functions\n",
    "Next, we will create a small function to show the results of our query in human-readable outputs. This function would be used in our examples to display the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def showResults(output):\n",
    "    print(\"Total results: \", len(output))\n",
    "    for index in range(len(output)):\n",
    "        print(output[index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying the dataset with similarity_search\n",
    "\n",
    "Now that we have indexed our sample data to elasticsearch, we will perform a similarity search on query - `How does the compensation work?`. By default returns top `4` documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total results:  4\n",
      "page_content='Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.\n",
      "\n",
      "Performance-Based Compensation:\n",
      "In addition to the defined compensation bands, we emphasize a performance-based compensation model. Performance evaluations will be conducted regularly, and employees exceeding performance expectations will be eligible for bonuses, incentives, and salary increases. This approach rewards high achievers and motivates employees to excel in their roles.\n",
      "\n",
      "Conclusion:\n",
      "By implementing this compensation bands strategy, our IT company aims to establish fair and competitive compensation practices that align with market standards and foster employee satisfaction. Regular evaluations and market benchmarking will enable us to adapt and refine the strategy to meet the evolving needs of our organization.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Introduction:\n",
      "This document outlines the compensation bands strategy for the various teams within our IT company. The goal is to establish a fair and competitive compensation structure that aligns with industry standards, rewards performance, and attracts top talent. By implementing this strategy, we aim to foster employee satisfaction and retention while ensuring the company's overall success.\n",
      "\n",
      "Purpose:\n",
      "The purpose of this compensation bands strategy is to:\n",
      "a. Define clear guidelines for salary ranges based on job levels and market benchmarks.\n",
      "b. Support equitable compensation practices across different teams.\n",
      "c. Encourage employee growth and performance.\n",
      "d. Enable effective budgeting and resource allocation.\n",
      "\n",
      "Job Levels:\n",
      "To establish a comprehensive compensation structure, we have defined distinct job levels within each team. These levels reflect varying degrees of skills, experience, and responsibilities. The levels include:\n",
      "a. Entry-Level: Employees with limited experience or early career professionals.\n",
      "b. Intermediate-Level: Employees with moderate experience and demonstrated competence.\n",
      "c. Senior-Level: Experienced employees with advanced skills and leadership capabilities.\n",
      "d. Leadership-Level: Managers and team leaders responsible for strategic decision-making.\n",
      "\n",
      "Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Performance Ratings\n",
      "Based on the performance evaluation, employees will receive a performance rating that reflects their overall performance during the cycle. The rating system should be clearly defined and consistently applied across the organization. Performance ratings will be used to inform decisions regarding promotions, salary increases, and other rewards or recognition.\n",
      "Promotions and Advancements\n",
      "High-performing employees who consistently demonstrate strong performance, leadership, and a commitment to the company’s values may be considered for promotions or other advancement opportunities. Promotions will be based on factors such as performance ratings, skills, experience, and the needs of the organization. Employees interested in pursuing a promotion should discuss their career goals and development plans with their supervisor.\n",
      "Performance Improvement Plans\n",
      "Employees who receive a low performance rating or are struggling to meet their performance goals may be placed on a Performance Improvement Plan (PIP). A PIP is a structured plan designed to help the employee address specific areas of concern, set achievable improvement goals, and receive additional support or resources as needed. Employees on a PIP will be closely monitored and re-evaluated at the end of the improvement period to determine if satisfactory progress has been made.\n",
      "Recognition and Rewards\n",
      "Our company believes in recognizing and rewarding employees for their hard work and dedication. In addition to promotions and salary increases, employees may be eligible for other forms of recognition or rewards based on their performance. This may include bonuses, awards, or other incentives designed to motivate and celebrate employee achievements. The specific criteria and eligibility for these rewards will be communicated by the HR department or management.' metadata={'summary': 'This Performance Management Policy outlines a consistent and transparent process for evaluating, recognizing, and rewarding employees. It includes goal setting, ongoing feedback, performance evaluations, ratings, promotions, and rewards. The policy applies to all employees and encourages open communication and professional growth.', 'rolePermissions': ['demo', 'manager'], 'name': 'Performance Management Policy'}\n",
      "page_content='Purpose\n",
      "\n",
      "The purpose of this vacation policy is to outline the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. This policy aims to promote a healthy work-life balance and encourage employees to take time to rest and recharge.\n",
      "Scope\n",
      "\n",
      "This policy applies to all full-time and part-time employees who have completed their probationary period.\n",
      "Vacation Accrual\n",
      "\n",
      "Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Part-time employees accrue vacation time on a pro-rata basis, calculated according to their scheduled work hours.\n",
      "\n",
      "Vacation time will begin to accrue from the first day of employment, but employees are eligible to take vacation time only after completing their probationary period. Unused vacation time will be carried over to the next year, up to a maximum of [Z days]. Any additional unused vacation time will be forfeited.\n",
      "Vacation Scheduling\n",
      "\n",
      "Employees are required to submit vacation requests to their supervisor at least [A weeks] in advance, specifying the start and end dates of their vacation. Supervisors will review and approve vacation requests based on business needs, ensuring adequate coverage during the employee's absence.\n",
      "\n",
      "Employees are encouraged to plan their vacations around the company's peak and non-peak periods to minimize disruptions. Vacation requests during peak periods may be subject to limitations and require additional advance notice.\n",
      "Vacation Pay\n",
      "\n",
      "Employees will receive their regular pay during their approved vacation time. Vacation pay will be calculated based on the employee's average earnings over the [B weeks] preceding their vacation.\n",
      "Unplanned Absences and Vacation Time\n",
      "\n",
      "In the event of an unplanned absence due to illness or personal emergencies, employees may use their accrued vacation time, subject to supervisor approval. Employees must inform their supervisor as soon as possible and provide any required documentation upon their return to work.\n",
      "Vacation Time and Termination of Employment\n",
      "\n",
      "If an employee's employment is terminated, they will be paid out for any unused vacation time, calculated based on their current rate of pay.\n",
      "Policy Review and Updates\n",
      "\n",
      "This vacation policy will be reviewed periodically and updated as necessary, taking into account changes in labor laws, business needs, and employee feedback.\n",
      "Questions and Concerns\n",
      "\n",
      "Employees are encouraged to direct any questions or concerns about this policy to their supervisor or the HR department.' metadata={'summary': ': This policy outlines the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Vacation requests must be submitted to supervisors at least', 'rolePermissions': ['demo', 'manager'], 'name': 'Company Vacation Policy'}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/joe/.pyenv/versions/3.10.3/lib/python3.10/site-packages/langchain_elasticsearch/vectorstores.py:764: ElasticsearchWarning: text_expansion is deprecated. Use sparse_vector instead.\n",
      "  hits = self._store.search(\n"
     ]
    }
   ],
   "source": [
    "results = documents.similarity_search(\"How does the compensation work?\")\n",
    "showResults(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying the dataset show top 10 documents\n",
    "\n",
    "Now we will set `k=10` and try same query to see top 10 documents.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total results:  10\n",
      "page_content='Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.\n",
      "\n",
      "Performance-Based Compensation:\n",
      "In addition to the defined compensation bands, we emphasize a performance-based compensation model. Performance evaluations will be conducted regularly, and employees exceeding performance expectations will be eligible for bonuses, incentives, and salary increases. This approach rewards high achievers and motivates employees to excel in their roles.\n",
      "\n",
      "Conclusion:\n",
      "By implementing this compensation bands strategy, our IT company aims to establish fair and competitive compensation practices that align with market standards and foster employee satisfaction. Regular evaluations and market benchmarking will enable us to adapt and refine the strategy to meet the evolving needs of our organization.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Introduction:\n",
      "This document outlines the compensation bands strategy for the various teams within our IT company. The goal is to establish a fair and competitive compensation structure that aligns with industry standards, rewards performance, and attracts top talent. By implementing this strategy, we aim to foster employee satisfaction and retention while ensuring the company's overall success.\n",
      "\n",
      "Purpose:\n",
      "The purpose of this compensation bands strategy is to:\n",
      "a. Define clear guidelines for salary ranges based on job levels and market benchmarks.\n",
      "b. Support equitable compensation practices across different teams.\n",
      "c. Encourage employee growth and performance.\n",
      "d. Enable effective budgeting and resource allocation.\n",
      "\n",
      "Job Levels:\n",
      "To establish a comprehensive compensation structure, we have defined distinct job levels within each team. These levels reflect varying degrees of skills, experience, and responsibilities. The levels include:\n",
      "a. Entry-Level: Employees with limited experience or early career professionals.\n",
      "b. Intermediate-Level: Employees with moderate experience and demonstrated competence.\n",
      "c. Senior-Level: Experienced employees with advanced skills and leadership capabilities.\n",
      "d. Leadership-Level: Managers and team leaders responsible for strategic decision-making.\n",
      "\n",
      "Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Performance Ratings\n",
      "Based on the performance evaluation, employees will receive a performance rating that reflects their overall performance during the cycle. The rating system should be clearly defined and consistently applied across the organization. Performance ratings will be used to inform decisions regarding promotions, salary increases, and other rewards or recognition.\n",
      "Promotions and Advancements\n",
      "High-performing employees who consistently demonstrate strong performance, leadership, and a commitment to the company’s values may be considered for promotions or other advancement opportunities. Promotions will be based on factors such as performance ratings, skills, experience, and the needs of the organization. Employees interested in pursuing a promotion should discuss their career goals and development plans with their supervisor.\n",
      "Performance Improvement Plans\n",
      "Employees who receive a low performance rating or are struggling to meet their performance goals may be placed on a Performance Improvement Plan (PIP). A PIP is a structured plan designed to help the employee address specific areas of concern, set achievable improvement goals, and receive additional support or resources as needed. Employees on a PIP will be closely monitored and re-evaluated at the end of the improvement period to determine if satisfactory progress has been made.\n",
      "Recognition and Rewards\n",
      "Our company believes in recognizing and rewarding employees for their hard work and dedication. In addition to promotions and salary increases, employees may be eligible for other forms of recognition or rewards based on their performance. This may include bonuses, awards, or other incentives designed to motivate and celebrate employee achievements. The specific criteria and eligibility for these rewards will be communicated by the HR department or management.' metadata={'summary': 'This Performance Management Policy outlines a consistent and transparent process for evaluating, recognizing, and rewarding employees. It includes goal setting, ongoing feedback, performance evaluations, ratings, promotions, and rewards. The policy applies to all employees and encourages open communication and professional growth.', 'rolePermissions': ['demo', 'manager'], 'name': 'Performance Management Policy'}\n",
      "page_content='Purpose\n",
      "\n",
      "The purpose of this vacation policy is to outline the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. This policy aims to promote a healthy work-life balance and encourage employees to take time to rest and recharge.\n",
      "Scope\n",
      "\n",
      "This policy applies to all full-time and part-time employees who have completed their probationary period.\n",
      "Vacation Accrual\n",
      "\n",
      "Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Part-time employees accrue vacation time on a pro-rata basis, calculated according to their scheduled work hours.\n",
      "\n",
      "Vacation time will begin to accrue from the first day of employment, but employees are eligible to take vacation time only after completing their probationary period. Unused vacation time will be carried over to the next year, up to a maximum of [Z days]. Any additional unused vacation time will be forfeited.\n",
      "Vacation Scheduling\n",
      "\n",
      "Employees are required to submit vacation requests to their supervisor at least [A weeks] in advance, specifying the start and end dates of their vacation. Supervisors will review and approve vacation requests based on business needs, ensuring adequate coverage during the employee's absence.\n",
      "\n",
      "Employees are encouraged to plan their vacations around the company's peak and non-peak periods to minimize disruptions. Vacation requests during peak periods may be subject to limitations and require additional advance notice.\n",
      "Vacation Pay\n",
      "\n",
      "Employees will receive their regular pay during their approved vacation time. Vacation pay will be calculated based on the employee's average earnings over the [B weeks] preceding their vacation.\n",
      "Unplanned Absences and Vacation Time\n",
      "\n",
      "In the event of an unplanned absence due to illness or personal emergencies, employees may use their accrued vacation time, subject to supervisor approval. Employees must inform their supervisor as soon as possible and provide any required documentation upon their return to work.\n",
      "Vacation Time and Termination of Employment\n",
      "\n",
      "If an employee's employment is terminated, they will be paid out for any unused vacation time, calculated based on their current rate of pay.\n",
      "Policy Review and Updates\n",
      "\n",
      "This vacation policy will be reviewed periodically and updated as necessary, taking into account changes in labor laws, business needs, and employee feedback.\n",
      "Questions and Concerns\n",
      "\n",
      "Employees are encouraged to direct any questions or concerns about this policy to their supervisor or the HR department.' metadata={'summary': ': This policy outlines the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Vacation requests must be submitted to supervisors at least', 'rolePermissions': ['demo', 'manager'], 'name': 'Company Vacation Policy'}\n",
      "page_content='Starting May 2022, the company will be implementing a two-day in-office work requirement per week for all eligible employees. Please coordinate with your supervisor and HR department to schedule your in-office workdays while continuing to follow all safety protocols.' metadata={'summary': 'Starting May 2022, employees will need to work two days a week in the office. Coordinate with your supervisor and HR department for these days while following safety protocols.', 'rolePermissions': ['demo', 'manager'], 'name': 'April Work From Home Update'}\n",
      "page_content='Effective: March 2020\n",
      "Purpose\n",
      "\n",
      "The purpose of this full-time work-from-home policy is to provide guidelines and support for employees to conduct their work remotely, ensuring the continuity and productivity of business operations during the COVID-19 pandemic and beyond.\n",
      "Scope\n",
      "\n",
      "This policy applies to all employees who are eligible for remote work as determined by their role and responsibilities. It is designed to allow employees to work from home full time while maintaining the same level of performance and collaboration as they would in the office.\n",
      "Eligibility\n",
      "\n",
      "Employees who can perform their work duties remotely and have received approval from their direct supervisor and the HR department are eligible for this work-from-home arrangement.\n",
      "Equipment and Resources\n",
      "\n",
      "The necessary equipment and resources will be provided to employees for remote work, including a company-issued laptop, software licenses, and access to secure communication tools. Employees are responsible for maintaining and protecting the company's equipment and data.\n",
      "Workspace\n",
      "\n",
      "Employees working from home are responsible for creating a comfortable and safe workspace that is conducive to productivity. This includes ensuring that their home office is ergonomically designed, well-lit, and free from distractions.\n",
      "Communication\n",
      "\n",
      "Effective communication is vital for successful remote work. Employees are expected to maintain regular communication with their supervisors, colleagues, and team members through email, phone calls, video conferences, and other approved communication tools.\n",
      "Work Hours and Availability\n",
      "\n",
      "Employees are expected to maintain their regular work hours and be available during normal business hours, unless otherwise agreed upon with their supervisor. Any changes to work hours or availability must be communicated to the employee's supervisor and the HR department.\n",
      "Performance Expectations\n",
      "\n",
      "Employees working from home are expected to maintain the same level of performance and productivity as if they were working in the office. Supervisors and team members will collaborate to establish clear expectations and goals for remote work.\n",
      "Time Tracking and Overtime\n",
      "\n",
      "Employees are required to accurately track their work hours using the company's time tracking system. Non-exempt employees must obtain approval from their supervisor before working overtime.\n",
      "Confidentiality and Data Security\n",
      "\n",
      "Employees must adhere to the company's confidentiality and data security policies while working from home. This includes safeguarding sensitive information, securing personal devices and internet connections, and reporting any security breaches to the IT department.\n",
      "Health and Well-being' metadata={'summary': 'This policy outlines the guidelines for full-time remote work, including eligibility, equipment and resources, workspace requirements, communication expectations, performance expectations, time tracking and overtime, confidentiality and data security, health and well-being, and policy reviews and updates. Employees are encouraged to direct any questions or concerns', 'rolePermissions': ['demo', 'manager'], 'name': 'Work From Home Policy'}\n",
      "page_content='Performance Management Policy\n",
      "Purpose and Scope\n",
      "The purpose of this Performance Management Policy is to establish a consistent and transparent process for evaluating, recognizing, and rewarding employee performance. This policy applies to all employees and aims to foster a culture of continuous improvement, professional growth, and open communication between employees and management.\n",
      "Performance Planning and Goal Setting\n",
      "At the beginning of each performance cycle, employees and their supervisors will collaborate to set clear, achievable, and measurable performance goals. These goals should align with the company’s strategic objectives and take into account the employee’s job responsibilities, professional development, and career aspirations.\n",
      "Ongoing Feedback and Communication\n",
      "Throughout the performance cycle, employees and supervisors are encouraged to engage in regular, constructive feedback and open communication. This includes discussing progress towards goals, addressing challenges, and identifying opportunities for improvement or additional support. Regular check-ins and updates help ensure that employees stay on track and receive the guidance they need to succeed.\n",
      "Performance Evaluation\n",
      "At the end of each performance cycle, employees will participate in a formal performance evaluation with their supervisor. This evaluation will assess the employee’s overall performance, including their achievements, areas for improvement, and progress towards goals. Both the employee and supervisor should come prepared to discuss specific examples, accomplishments, and challenges from the performance period.\n",
      "Performance Ratings\n",
      "Based on the performance evaluation, employees will receive a performance rating that reflects their overall performance during the cycle. The rating system should be clearly defined and consistently applied across the organization. Performance ratings will be used to inform decisions regarding promotions, salary increases, and other rewards or recognition.\n",
      "Promotions and Advancements\n",
      "High-performing employees who consistently demonstrate strong performance, leadership, and a commitment to the company’s values may be considered for promotions or other advancement opportunities. Promotions will be based on factors such as performance ratings, skills, experience, and the needs of the organization. Employees interested in pursuing a promotion should discuss their career goals and development plans with their supervisor.\n",
      "Performance Improvement Plans\n",
      "Employees who receive a low performance rating or are struggling to meet their performance goals may be placed on a Performance Improvement Plan (PIP). A PIP is a structured plan designed to help the employee address specific areas of concern, set achievable improvement goals, and receive additional support or resources as needed. Employees on a PIP will be closely monitored and re-evaluated at the end of the improvement period to determine if satisfactory progress has been made.\n",
      "Recognition and Rewards' metadata={'summary': 'This Performance Management Policy outlines a consistent and transparent process for evaluating, recognizing, and rewarding employees. It includes goal setting, ongoing feedback, performance evaluations, ratings, promotions, and rewards. The policy applies to all employees and encourages open communication and professional growth.', 'rolePermissions': ['demo', 'manager'], 'name': 'Performance Management Policy'}\n",
      "page_content='Review policies and procedures: Familiarize yourself with our employee handbook, which contains important information about our policies and procedures. Please read it thoroughly and adhere to the guidelines.\n",
      "Complete required training: You may be required to complete mandatory training sessions, such as safety training or anti-harassment training. Ensure that you attend and complete these sessions as soon as possible.\n",
      "Updating Tax Elections and Documents\n",
      "It is crucial to ensure your tax information is accurate and up-to-date, regardless of the country you work in. Please follow these steps to update your tax elections and documents:\n",
      "Complete tax forms: Fill out the necessary tax forms for your country or region, which determine the amount of income tax withheld from your paycheck. You should complete new tax forms if your personal or financial situation changes, such as marriage, divorce, or a change in the number of dependents.\n",
      "Submit regional tax forms: Depending on your location, you may be required to complete additional regional or local tax forms. Check with the HR department to determine which forms are necessary.\n",
      "Update your address: If you move, make sure to update your address with the HR department to ensure accurate tax reporting.\n",
      "Benefits Enrollment\n",
      "As a new employee, you are eligible for various benefits, including health insurance, retirement plans, and paid time off. You will receive detailed information about our benefits package during orientation. To enroll in the benefits, please follow these steps:\n",
      "Review benefits options: Carefully review the benefits package and choose the options that best meet your needs.\n",
      "Complete enrollment forms: Fill out the necessary forms to enroll in your chosen benefits. Submit these forms to the HR department within 30 days of your start date.\n",
      "Designate beneficiaries: If applicable, designate beneficiaries for your life insurance and retirement plans.\n",
      "Getting Settled in Your Workspace\n",
      "To help you feel comfortable and productive in your new workspace, take the following steps:\n",
      "Set up your workstation: Organize your desk, chair, and computer according to your preferences. If you require any additional equipment or accommodations, please contact the HR department.\n",
      "Obtain necessary supplies: Request any necessary office supplies, such as pens, notepads, or folders, from the designated supply area or by contacting the appropriate department.\n",
      "Familiarize yourself with office resources: Locate common areas, such as break rooms, restrooms, and meeting rooms. Familiarize yourself with office equipment, including printers, scanners, and telephones.' metadata={'summary': '\\nThis onboarding guide provides essential information to new employees on our company culture and values, key onboarding steps, tax elections and documents, benefits enrollment, and setting up their workspace.', 'rolePermissions': ['demo', 'manager'], 'name': 'New Employee Onboarding Guide'}\n",
      "page_content='Definitions\n",
      "a. Intellectual Property (IP): Refers to creations of the mind, such as inventions, literary and artistic works, designs, symbols, and images, that are protected by copyright, trademark, patent, or other forms of legal protection.\n",
      "b. Company Time: Refers to the time during which an employee is actively engaged in performing their job duties.\n",
      "c. Outside Company Time: Refers to the time during which an employee is not engaged in performing their job duties.\n",
      "\n",
      "Ownership of Intellectual Property\n",
      "a. Work Generated on Company Time\n",
      "i. Any intellectual property created, conceived, or developed by an employee during company time or using company resources, equipment, or facilities shall be considered the property of the Company.\n",
      "ii. Employees are required to promptly disclose any such intellectual property to their supervisor or the appropriate department head.\n",
      "b. Work Generated Outside Company Time\n",
      "i. Intellectual property created, conceived, or developed by an employee outside of company time and without the use of company resources, equipment, or facilities shall generally remain the property of the employee.\n",
      "ii. However, if the intellectual property is directly related to the employee's job responsibilities, or if the employee has used company resources, equipment, or facilities in its creation, it may be considered the property of the Company.\n",
      "Protection and Utilization of Intellectual Property\n",
      "a. The Company shall have the right to protect, license, and commercialize any intellectual property owned by the company as it deems appropriate.\n",
      "b. Employees are expected to cooperate with the Company in obtaining any necessary legal protection for intellectual property owned by the company, including by signing any documents or providing any necessary information or assistance.\n",
      "Confidentiality\n",
      "Employees are expected to maintain the confidentiality of any intellectual property owned by the Company and not disclose it to any third parties without the express written consent of an authorized representative of the company.\n",
      "Employee Acknowledgment\n",
      "All employees are required to sign an acknowledgment of this Intellectual Property Policy as a condition of their employment with [Company Name]. By signing the acknowledgment, employees agree to abide by the terms of this policy and understand that any violations may result in disciplinary action, up to and including termination of employment.\n",
      "Policy Review\n",
      "This Intellectual Property Policy shall be reviewed periodically and may be amended as necessary to ensure its continued effectiveness and compliance with applicable laws and regulations. Employees will be notified of any significant changes to this policy.' metadata={'summary': \"This Intellectual Property Policy outlines guidelines and procedures for the ownership, protection, and utilization of intellectual property generated by employees during their employment. It establishes the company's ownership of work generated on company time, while recognizing employee ownership of work generated outside of company time without the use of company resources. The policy\", 'rolePermissions': ['demo', 'manager'], 'name': 'Intellectual Property Policy'}\n",
      "page_content='Read the instructions carefully for each section to ensure you claim the correct amounts.\n",
      "\n",
      "Step 5: Sign and date the form\n",
      "Once you've completed the form, sign and date it at the bottom.\n",
      "\n",
      "Step 6: Submit the form to your employer\n",
      "Submit the completed and signed TD1 form to your employer. You can either scan and send it electronically, or provide a printed copy. Your employer will use the information on your TD1 form to calculate the correct amount of tax to be deducted from your pay.\n",
      "\n",
      "Step 7: Update your TD1 form as needed\n",
      "It's essential to update your TD1 form whenever your personal circumstances change, such as getting married, having a child, or becoming eligible for a new tax credit. Inform your employer of these changes and submit an updated TD1 form to ensure accurate tax deductions.\n",
      "\n",
      "Updating your tax elections forms is a crucial step in ensuring the correct tax deductions from your pay as a new employee in Canada. Follow this guide and keep your TD1 form up to date to avoid any discrepancies in your tax filings.' metadata={'summary': ': This guide gives a step-by-step explanation of how to update your TD1 Personal Tax Credits Return form. Access the form from the CRA website and choose the correct version based on your province or territory of residence. Download and open the form in Adobe Reader, fill out the form by entering', 'rolePermissions': ['demo', 'manager'], 'name': 'Updating Your Tax Elections Forms'}\n"
     ]
    }
   ],
   "source": [
    "results = documents.similarity_search(\"How does the compensation work?\", k=10)\n",
    "showResults(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Querying the dataset with filtering Metadata\n",
    "We will now add metadata filtering by Keyword at query time, to match `rolePermissions` as `manager`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total results:  4\n",
      "page_content='Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.\n",
      "\n",
      "Performance-Based Compensation:\n",
      "In addition to the defined compensation bands, we emphasize a performance-based compensation model. Performance evaluations will be conducted regularly, and employees exceeding performance expectations will be eligible for bonuses, incentives, and salary increases. This approach rewards high achievers and motivates employees to excel in their roles.\n",
      "\n",
      "Conclusion:\n",
      "By implementing this compensation bands strategy, our IT company aims to establish fair and competitive compensation practices that align with market standards and foster employee satisfaction. Regular evaluations and market benchmarking will enable us to adapt and refine the strategy to meet the evolving needs of our organization.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Introduction:\n",
      "This document outlines the compensation bands strategy for the various teams within our IT company. The goal is to establish a fair and competitive compensation structure that aligns with industry standards, rewards performance, and attracts top talent. By implementing this strategy, we aim to foster employee satisfaction and retention while ensuring the company's overall success.\n",
      "\n",
      "Purpose:\n",
      "The purpose of this compensation bands strategy is to:\n",
      "a. Define clear guidelines for salary ranges based on job levels and market benchmarks.\n",
      "b. Support equitable compensation practices across different teams.\n",
      "c. Encourage employee growth and performance.\n",
      "d. Enable effective budgeting and resource allocation.\n",
      "\n",
      "Job Levels:\n",
      "To establish a comprehensive compensation structure, we have defined distinct job levels within each team. These levels reflect varying degrees of skills, experience, and responsibilities. The levels include:\n",
      "a. Entry-Level: Employees with limited experience or early career professionals.\n",
      "b. Intermediate-Level: Employees with moderate experience and demonstrated competence.\n",
      "c. Senior-Level: Experienced employees with advanced skills and leadership capabilities.\n",
      "d. Leadership-Level: Managers and team leaders responsible for strategic decision-making.\n",
      "\n",
      "Compensation Bands:\n",
      "Based on the job levels, the following compensation bands have been established:\n",
      "a. Entry-Level Band: This band encompasses salary ranges for employees in entry-level positions. It aims to provide competitive compensation for individuals starting their careers within the company.\n",
      "\n",
      "b. Intermediate-Level Band: This band covers salary ranges for employees who have gained moderate experience and expertise in their respective roles. It rewards employees for their growing skill set and contributions.\n",
      "\n",
      "c. Senior-Level Band: The senior-level band includes salary ranges for experienced employees who have attained advanced skills and have a proven track record of delivering results. It reflects the increased responsibilities and expectations placed upon these individuals.\n",
      "\n",
      "d. Leadership-Level Band: This band comprises salary ranges for managers and team leaders responsible for guiding and overseeing their respective teams. It considers their leadership abilities, strategic thinking, and the impact they have on the company's success.\n",
      "\n",
      "Market Benchmarking:\n",
      "To ensure our compensation remains competitive, regular market benchmarking will be conducted. This involves analyzing industry salary trends, regional compensation data, and market demand for specific roles. The findings will inform periodic adjustments to our compensation bands to maintain alignment with the market.' metadata={'summary': 'This document outlines a compensation framework for IT teams. It includes job levels, compensation bands, and performance-based incentives to ensure fair and competitive wages. Regular market benchmarking will be conducted to adjust the bands according to industry trends.', 'rolePermissions': ['manager'], 'name': 'Compensation Framework For It Teams'}\n",
      "page_content='Performance Ratings\n",
      "Based on the performance evaluation, employees will receive a performance rating that reflects their overall performance during the cycle. The rating system should be clearly defined and consistently applied across the organization. Performance ratings will be used to inform decisions regarding promotions, salary increases, and other rewards or recognition.\n",
      "Promotions and Advancements\n",
      "High-performing employees who consistently demonstrate strong performance, leadership, and a commitment to the company’s values may be considered for promotions or other advancement opportunities. Promotions will be based on factors such as performance ratings, skills, experience, and the needs of the organization. Employees interested in pursuing a promotion should discuss their career goals and development plans with their supervisor.\n",
      "Performance Improvement Plans\n",
      "Employees who receive a low performance rating or are struggling to meet their performance goals may be placed on a Performance Improvement Plan (PIP). A PIP is a structured plan designed to help the employee address specific areas of concern, set achievable improvement goals, and receive additional support or resources as needed. Employees on a PIP will be closely monitored and re-evaluated at the end of the improvement period to determine if satisfactory progress has been made.\n",
      "Recognition and Rewards\n",
      "Our company believes in recognizing and rewarding employees for their hard work and dedication. In addition to promotions and salary increases, employees may be eligible for other forms of recognition or rewards based on their performance. This may include bonuses, awards, or other incentives designed to motivate and celebrate employee achievements. The specific criteria and eligibility for these rewards will be communicated by the HR department or management.' metadata={'summary': 'This Performance Management Policy outlines a consistent and transparent process for evaluating, recognizing, and rewarding employees. It includes goal setting, ongoing feedback, performance evaluations, ratings, promotions, and rewards. The policy applies to all employees and encourages open communication and professional growth.', 'rolePermissions': ['demo', 'manager'], 'name': 'Performance Management Policy'}\n",
      "page_content='Purpose\n",
      "\n",
      "The purpose of this vacation policy is to outline the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. This policy aims to promote a healthy work-life balance and encourage employees to take time to rest and recharge.\n",
      "Scope\n",
      "\n",
      "This policy applies to all full-time and part-time employees who have completed their probationary period.\n",
      "Vacation Accrual\n",
      "\n",
      "Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Part-time employees accrue vacation time on a pro-rata basis, calculated according to their scheduled work hours.\n",
      "\n",
      "Vacation time will begin to accrue from the first day of employment, but employees are eligible to take vacation time only after completing their probationary period. Unused vacation time will be carried over to the next year, up to a maximum of [Z days]. Any additional unused vacation time will be forfeited.\n",
      "Vacation Scheduling\n",
      "\n",
      "Employees are required to submit vacation requests to their supervisor at least [A weeks] in advance, specifying the start and end dates of their vacation. Supervisors will review and approve vacation requests based on business needs, ensuring adequate coverage during the employee's absence.\n",
      "\n",
      "Employees are encouraged to plan their vacations around the company's peak and non-peak periods to minimize disruptions. Vacation requests during peak periods may be subject to limitations and require additional advance notice.\n",
      "Vacation Pay\n",
      "\n",
      "Employees will receive their regular pay during their approved vacation time. Vacation pay will be calculated based on the employee's average earnings over the [B weeks] preceding their vacation.\n",
      "Unplanned Absences and Vacation Time\n",
      "\n",
      "In the event of an unplanned absence due to illness or personal emergencies, employees may use their accrued vacation time, subject to supervisor approval. Employees must inform their supervisor as soon as possible and provide any required documentation upon their return to work.\n",
      "Vacation Time and Termination of Employment\n",
      "\n",
      "If an employee's employment is terminated, they will be paid out for any unused vacation time, calculated based on their current rate of pay.\n",
      "Policy Review and Updates\n",
      "\n",
      "This vacation policy will be reviewed periodically and updated as necessary, taking into account changes in labor laws, business needs, and employee feedback.\n",
      "Questions and Concerns\n",
      "\n",
      "Employees are encouraged to direct any questions or concerns about this policy to their supervisor or the HR department.' metadata={'summary': ': This policy outlines the guidelines and procedures for requesting and taking time off from work for personal and leisure purposes. Full-time employees accrue vacation time at a rate of [X hours] per month, equivalent to [Y days] per year. Vacation requests must be submitted to supervisors at least', 'rolePermissions': ['demo', 'manager'], 'name': 'Company Vacation Policy'}\n"
     ]
    }
   ],
   "source": [
    "results = documents.similarity_search(\n",
    "    \"How does the compensation work\",\n",
    "    filter=[{\"match\": {\"metadata.rolePermissions\": \"manager\"}}],\n",
    ")\n",
    "showResults(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.3"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}