updated

Author: bansalkanav

Module 1 - Python Programming/08. Regular Expressions/.ipynb_checkpoints/reg_exp-checkpoint.ipynb

@@ -11,20 +11,158 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Regular Expressions"
+    "# Regular Expressions\n",
+    "\n",
+    "**What are Regualar Expressions?**  \n",
+    "A regular expression, often abbreviated as \"regex\" or \"regexp,\" is a powerful tool used in computer science and text processing to describe and match patterns in strings. It's a sequence of characters that defines a search pattern. These patterns can be used for tasks like searching, extracting, replacing, and validating text.\n",
+    "\n",
+    "Regular expressions are widely used in tasks like text searching and manipulation, data validation, parsing, and more. They are supported by many programming languages and text processing tools, making them a versatile and essential tool for working with text data.\n",
+    "\n",
+    "**Why Regular Expressions?**  \n",
+    "Regular expressions are used for a variety of tasks in computer science, data processing, and text analysis due to their powerful pattern matching capabilities. Here are some key reasons why regular expressions are widely used:\n",
+    "\n",
+    "1. **Pattern Matching:** Regular expressions allow you to search for specific patterns or sequences of characters within a larger body of text. This is useful for tasks like finding specific words, dates, email addresses, or other structured information.\n",
+    "\n",
+    "2. **Text Extraction:** They can be used to extract specific pieces of information from a text document, such as names, phone numbers, URLs, or any other structured data.\n",
+    "\n",
+    "3. **Data Validation:** Regular expressions are used to validate input data. For example, you can use them to check if a user-provided email address or phone number is in the correct format.\n",
+    "\n",
+    "4. **Search and Replace:** They enable you to search for specific patterns in a text and replace them with something else. This is useful for tasks like cleaning up data or making text substitutions.\n",
+    "\n",
+    "5. **Parsing and Tokenization:** Regular expressions are essential for breaking down text into smaller units or tokens. This is used in tasks like natural language processing and compiler design.\n",
+    "\n",
+    "6. **Web Scraping:** When extracting data from websites, regular expressions can be used to locate and extract specific elements or information from HTML pages.\n",
+    "\n",
+    "7. **Log File Analysis:** Regular expressions are invaluable for searching and parsing log files, allowing you to extract important information or identify patterns of interest.\n",
+    "\n",
+    "8. **Pattern Validation:** They are used to validate whether a string adheres to a specific pattern or format, such as checking if a password meets certain criteria.\n",
+    "\n",
+    "9. **Data Transformation and Cleaning:** Regular expressions can be used to clean and transform text data. For example, removing unnecessary characters or formatting.\n",
+    "\n",
+    "10. **Language Agnostic:** Regular expressions are supported by most programming languages, making them a versatile tool that can be applied in a wide range of contexts.\n",
+    "\n",
+    "11. **Efficiency:** When used correctly, regular expressions can provide very efficient search and match operations, especially for complex patterns.\n",
+    "\n",
+    "Overall, regular expressions are a fundamental tool for working with text data and are an essential skill for tasks ranging from data preprocessing to text analysis and beyond. They provide a flexible and powerful means to perform complex pattern matching operations.\n",
+    "\n",
+    "\n",
+    "**Applications of Regular Expressions**  \n",
+    "Regular expressions (regex) find applications in a wide range of real-time scenarios across various domains. Here are some of the most important applications of regex:\n",
+    "\n",
+    "1. **Data Validation and Form Input:**\n",
+    "   - Ensuring that user-provided data (like email addresses, phone numbers, passwords, etc.) adhere to specified formats before processing.\n",
+    "\n",
+    "2. **Search and Replace in Text Editors:**\n",
+    "   - Find and replace operations in text editors or IDEs, allowing for quick and precise changes in code or documents.\n",
+    "\n",
+    "3. **Log File Parsing and Analysis:**\n",
+    "   - Extracting relevant information from log files, helping to identify patterns, errors, or anomalies in system logs.\n",
+    "\n",
+    "4. **Web Scraping and Data Extraction:**\n",
+    "   - Extracting specific information from web pages, like email addresses, phone numbers, product names, etc., for further analysis.\n",
+    "\n",
+    "5. **Data Cleaning and Transformation:**\n",
+    "   - Preprocessing text data by removing unnecessary characters, fixing formatting issues, and standardizing data for analysis.\n",
+    "\n",
+    "6. **Search Engines and Information Retrieval:**\n",
+    "   - Powering search engines for matching user queries to relevant content on websites or databases.\n",
+    "\n",
+    "7. **URL Routing and Validation:**\n",
+    "   - Validating and parsing URLs to ensure they follow the correct format and extracting specific parameters from them.\n",
+    "\n",
+    "8. **Lexical Analysis in Compiler Design:**\n",
+    "   - Tokenizing source code into meaningful units for further processing by a compiler.\n",
+    "\n",
+    "9. **Natural Language Processing (NLP):**\n",
+    "   - Tokenizing sentences or words, extracting entities (like names, dates, locations), and performing advanced text processing tasks.\n",
+    "\n",
+    "10. **Network Security and Firewall Rules:**\n",
+    "    - Defining and enforcing rules for allowing or blocking specific types of traffic based on patterns in network traffic logs.\n",
+    "\n",
+    "11. **Database Querying and Validation:**\n",
+    "    - Validating and querying databases for specific patterns or formats, such as social security numbers, credit card numbers, etc.\n",
+    "\n",
+    "12. **Formal Language Theory and Automata:**\n",
+    "    - In computer science theory, regex is used in the definition of regular languages and finite automata.\n",
+    "\n",
+    "13. **Validation of Configuration Files:**\n",
+    "    - Ensuring that configuration files for software or systems follow the correct syntax and structure.\n",
+    "\n",
+    "14. **Extracting Metadata from Documents:**\n",
+    "    - Parsing documents (like PDFs, Word documents) to extract metadata such as titles, authors, dates, etc.\n",
+    "\n",
+    "15. **URL Rewriting in Web Servers:**\n",
+    "    - Modifying URLs on the fly to improve SEO or to direct traffic to specific pages.\n",
+    "\n",
+    "16. **Pattern Matching in DNA Sequences:**\n",
+    "    - Identifying specific genetic sequences or motifs in DNA for biological research.\n",
+    "\n",
+    "These are just some of the many real-time applications of regular expressions. Their versatility and powerful pattern-matching capabilities make them an invaluable tool in various fields of computer science and data processing."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "**References for Practice**  \n",
     "Try to solve all the interactive tutorial from below mentioned website:\n",
     "\n",
     "https://regexone.com/lesson/introduction_abcs\n",
     "\n",
     "https://regex101.com"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Meta Characters\n",
+    "`.`, `^`, `$`, `*`, `+`, `?`, `{`, `}`, `[`, `]`, `(`, `)`, `|`, `\\`\n",
+    "\n",
+    "### User-defined Character Classes\n",
+    "- `[abc]` - Match either a or b or c\n",
+    "- `[^abc]` - Match any character except a or b or c\n",
+    "- `[a-z]` - Match a lower case english alphabet character\n",
+    "- `[A-Z]` - Match an upper case english alphabet character\n",
+    "- `[a-zA-Z]` - Match any english alphabet character\n",
+    "- `[0-9]` - Match any digit character\n",
+    "- `[a-zA-Z0-9_]` - Match any alphanumeric character\n",
+    "- `[^a-zA-Z0-9_]` - Match any character except alphanumeric character\n",
+    "\n",
+    "### Pre-defined Character Classes\n",
+    "- `\\d` - Match a digit character i.e. `[0-9]`\n",
+    "- `\\D` - Match any character except digit character. i.e. `[^0-9]`\n",
+    "- `\\w` - Match an alpha-numeric character i.e. `[a-zA-Z0-9_]`\n",
+    "- `\\W` - Match any character except alpha-numeric character i.e. `[^a-zA-Z0-9_]`\n",
+    "- `\\s` - Match a space character.\n",
+    "- `\\S` - Match any character except space.\n",
+    "- `\\t` - Match a tab character.\n",
+    "- `\\n` - Match a next line character.\n",
+    "\n",
+    "### Quantifiers\n",
+    "- `a*` - Match zero or more number of characters\n",
+    "- `a+` - Match one or more number of characters\n",
+    "- `a?` - Match atmost 1 character i.e. 0 or 1\n",
+    "- `a{n}` - Match exactly n number of character\n",
+    "- `a{m, n}` - Match atleast m number and atmost n number of characters"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Importing the Required Module"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import re"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -53,41 +191,11 @@
    "source": [
     "# find all matches and returns a list\n",
     "\n",
-    "import re\n",
-    "\n",
     "lst = re.findall('[0-9]', '0@kM29-1')\n",
     "\n",
     "print(lst)"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 62,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "<callable_iterator object at 0x0000023C69FFF9A0>\n",
-      "0\n",
-      "2\n",
-      "9\n",
-      "1\n"
-     ]
-    }
-   ],
-   "source": [
-    "import re\n",
-    "\n",
-    "matcher = re.finditer('[0-9]', '0@kM29-1')\n",
-    "\n",
-    "print(matcher)\n",
-    "\n",
-    "for m in matcher:\n",
-    "    print(m.group())"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 67,
@@ -486,8 +594,7 @@
     "ExecuteTime": {
      "end_time": "2018-06-07T05:24:58.259291Z",
      "start_time": "2018-06-07T05:24:58.254301Z"
-    },
-    "scrolled": true
+    }
    },
    "outputs": [
     {
@@ -1262,7 +1369,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -1276,7 +1383,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.5"
+   "version": "3.9.13"
   },
   "toc": {
    "nav_menu": {},
@@ -1321,5 +1428,5 @@
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }