Access re library via CPython interop

[rebcabin]

I'm writing a type-checker, LASR, for ASR in python , and porting it to LPython.

I'd like to import re. I know it's not practical to compile re, but I would like to import it to my LPython program (see link above ... calling LPython on it produces many irrelevant errors.)

[rebcabin]

BTW, feel free to use LASR as a test case for CPython interop. It uses re, io, and pydantic. Otherwise, it's kindergarten code -- on purpose, because I want to port it to LPython.

[certik]

Once @cpythoncall works (#1704), you should be able to use it to call into re, io, pydantic and other code.

[Thirumalai-Shaktivel]

Yup! Currently, we are trying to implement @cpythoncall to work using the LLVM backend. For now, the following works using the C backend:
a.py

@pythoncall(module="b")
def get_email(text: str) -> str:
    pass

def test():
    text: str = "Hello, my email id is [email protected]. xyz"
    print(get_email(text))

test()

b.py

def get_email(text):
    import re
    # Regular expression patterns
    email_pattern = r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"

    # Matching email addresses
    email_matches = re.findall(email_pattern, text)

    # Return of List type not supported yet, so return a string
    return email_matches[0]

Output:

$ lpython expr2.py --backend c --enable-cpython
[email protected]

[rebcabin]

I'll use it as soon as I can in LLVM!

[rebcabin]

Actually, I'd like to try it on the C backend.

Would you be so kind as to reply here with the exact commands I need to issue at the terminal on a mac to use the C back end? I.e., compile python to C, compile the C, and run the a.out in the published lp conda environment ...

[Shaikh-Ubaid]

One can use it the following way (in the lp conda environment):

(lp) ubaid@ubaids-MacBook-Pro my_folder % ls
my_main.py      my_module.py
(lp) ubaid@ubaids-MacBook-Pro my_folder % cat my_main.py 
from lpython import pythoncall

@pythoncall(module = "my_module")
def get_cpython_version() -> str:
    pass

def main0():
    print("CPython version: ", get_cpython_version())

main0()
(lp) ubaid@ubaids-MacBook-Pro my_folder % cat my_module.py 
def get_cpython_version():
    import platform
    return platform.python_version()
(lp) ubaid@ubaids-MacBook-Pro my_folder % python my_main.py 
CPython version:  3.10.2
(lp) ubaid@ubaids-MacBook-Pro my_folder % lpython my_main.py --backend c --enable-cpython
CPython version:  3.10.2

[Shaikh-Ubaid]

Please note that the file my_module.py should be present in the current working directory (this would not be required once we start supporting import paths for @pythoncall decorated functions).

[Thirumalai-Shaktivel]

Would you be so kind as to reply here with the exact commands I need to issue at the terminal on a mac to use the C back end?

Sure! LPython does everything for you, you just have to specify the backend.
LPython takes care of compiling Python to C, Linking and running the a.out file.
Consider an example:

def main0():
    x: i32
    x = (2+3)*5
    print(x)

main0()

and run:

$ lpython a.py --backend c
25

Generated C file: a__tmp__generated__.c
Contents:

#include <inttypes.h>

#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include <lfortran_intrinsics.h>


struct dimension_descriptor
{
    int32_t lower_bound, length;
};

// Implementations
void main0()
{
    int32_t x;
    x = (2 + 3)*5;
    printf("%d\n", x);
}

void _lpython_main_program()
{
    main0();
}

int main(int argc, char* argv[])
{
    _lpython_set_argv(argc, argv);
    _lpython_main_program();
    return 0;
}

This is simple code, @Shaikh-Ubaid has shared a detailed explanation of the usage above.

[Thirumalai-Shaktivel]

I'll use it as soon as I can in LLVM!

Cool, I'm working on pythoncallable and lpython decorators to use the LLVM backend. Soon I will try to complete them.

[Thirumalai-Shaktivel]

Please note that the file my_module.py should be present in the current working directory

Yea, I forgot about this. @Shaikh-Ubaid, Let's add this to our TODO, that we have to specify the path to look for the module using the -I option.

[Shaikh-Ubaid]

Continued from #1919 (comment), if we are using numpy arrays, then we need to pass the option --link-numpy

(lp) ubaid@ubaids-MacBook-Pro my_folder % ls
my_main.py      my_module.py
(lp) ubaid@ubaids-MacBook-Pro my_folder % cat my_main.py 
from lpython import pythoncall, i32
from numpy import empty, int32

@pythoncall(module = "my_module")
def get_arr_sum(a: i32[:]) -> i32:
    pass

def main0():
    n: Const[i32] = 5
    a: i32[n] = empty([n], dtype=int32)
    i: i32
    for i in range(n):
        a[i] = i + 1
    print(get_arr_sum(a))

main0()
(lp) ubaid@ubaids-MacBook-Pro my_folder % cat my_module.py 
import numpy as np
def get_arr_sum(a):
    return np.sum(a)
(lp) ubaid@ubaids-MacBook-Pro my_folder % python my_main.py                              
15
(lp) ubaid@ubaids-MacBook-Pro my_folder % lpython my_main.py --backend c --link-numpy
15

numpy arrays are currently supported for passing as functions arguments. numpy arrays as return types will be supported soon.