Skip to content

SEVERE | memory leak in icu_replace.c #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
LaurentDhont opened this issue Mar 20, 2025 · 4 comments
Open

SEVERE | memory leak in icu_replace.c #10

LaurentDhont opened this issue Mar 20, 2025 · 4 comments

Comments

@LaurentDhont
Copy link

There appears to be a memory leak in icu_replace.c
It would be amazing if this could get fixed, it has been causing tremendous issues in our production server and it took about a year to find the cause. The cause is most likely related to this sqlite extension.

We tested a simple scenario with valgrind and this is the result:

valgrind --leak-check=full --track-origins=yes /usr/bin/sqlite3 test.sqlite
sqlite> create table test (id int, name text);
sqlite> insert into test values (1, "Laurent");
sqlite> insert into test values (2 ,"Sergiu");
sqlite> .load ./icu_replace                                                                                                                                                                                                                                                               
sqlite> select regex_replace("aur", name, "asdfasdfasdf") from test;                                                                                                                                                                                                                      
Lasdfasdfasdfent                                                                                                                                                                                                                                                                          
Sergiu                                                                                                                                                                                                                                                                                    
sqlite>                                                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                                          
==535548==                                                                                                                                                                                                                                                                                
==535548== HEAP SUMMARY:                                                                                                                                                                                                                                                                  
==535548==     in use at exit: 401,868 bytes in 526 blocks                                                                                                                                                                                                                                
==535548==   total heap usage: 1,105 allocs, 579 frees, 998,197 bytes allocated                                                                                                                                                                                                           
==535548==                                                                                                                                                                                                                                                                                
==535548== 1,072 (200 direct, 872 indirect) bytes in 1 blocks are definitely lost in loss record 76 of 137                                                                                                                                                                                
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)                                                                                                                                                                                           
==535548==    by 0x5EADBDC: ???                                                                                                                                                                                                                                                           
==535548==    by 0x5DEE793: ???                                                                                                                                                                                                                                                           
==535548==    by 0x5DEE994: ???                                                                                                                                                                                                                                                           
==535548==    by 0x5EA1EFE: ???                                                                                                                                                                                                                                                           
==535548==    by 0x5EA2458: ???                                                                                                                                                                                                                                                           
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==    by 0x5EAB2D9: ???                                       
==535548==    by 0x5EAB3A3: ???                                       
==535548==    by 0x5C3E681: ???                                       
==535548==                                                            
==535548== 3,492 (200 direct, 3,292 indirect) bytes in 1 blocks are definitely lost in loss record 91 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???                                       
==535548==    by 0x5DEE57B: ???                                       
==535548==    by 0x5EA1F4B: ???                                       
==535548==    by 0x5EA2287: ???                                       
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==    by 0x5EAB2D9: ???                                       
==535548==    by 0x5EAB3A3: ???                                       
==535548==    by 0x5C3E774: ???                                       
==535548==    by 0x5C3EC91: ???                                       
==535548==
==535548== 3,980 (200 direct, 3,780 indirect) bytes in 1 blocks are definitely lost in loss record 92 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???                                       
==535548==    by 0x5DEE793: ???                                       
==535548==    by 0x5DEE994: ???                                       
==535548==    by 0x5EA1EFE: ???                                       
==535548==    by 0x5EA2458: ???                                       
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==    by 0x5EAB2D9: ???                                       
==535548==    by 0x5EAB3A3: ???                                       
==535548==    by 0x5C3E6CD: ???                                       
==535548==                                                            
==535548== 6,940 (200 direct, 6,740 indirect) bytes in 1 blocks are definitely lost in loss record 116 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???                                       
==535548==    by 0x5DEE793: ???                                       
==535548==    by 0x5DEE994: ???                                       
==535548==    by 0x5EA1EFE: ???                                       
==535548==    by 0x5EA2458: ???                                       
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==    by 0x5EAB2D9: ???                                       
==535548==    by 0x5EAB3A3: ???                                       
==535548==    by 0x5C3E635: ???                                       
==535548==                                                            
==535548== 15,356 (200 direct, 15,156 indirect) bytes in 1 blocks are definitely lost in loss record 133 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???                                       
==535548==    by 0x5DEAA9B: ???                                       
==535548==    by 0x5EDA4B6: ???                                       
==535548==    by 0x5DEE46D: ???                                       
==535548==    by 0x5EA1E78: ???                                       
==535548==    by 0x5EA2231: ???                                       
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==    by 0x5EAB2D9: ???                                       
==535548==    by 0x5EAB3A3: ???                                       
==535548==                                                            
==535548== 24,292 (200 direct, 24,092 indirect) bytes in 1 blocks are definitely lost in loss record 135 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???                                       
==535548==    by 0x5DEAA9B: ???                                       
==535548==    by 0x5EDA4B6: ???                                       
==535548==    by 0x5DEE46D: ???                                       
==535548==    by 0x5DEE84A: ???                                       
==535548==    by 0x5DEE994: ???                                       
==535548==    by 0x5EA1EFE: ???                                       
==535548==    by 0x5EA2458: ???                                       
==535548==    by 0x5EA7137: ???                                       
==535548==    by 0x5EA8AC6: ???                                       
==535548==    by 0x5EAA9D2: ???                                       
==535548==
==535548== 32,624 (3,840 direct, 28,784 indirect) bytes in 1 blocks are definitely lost in loss record 136 of 137
==535548==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==535548==    by 0x5EADBDC: ???
==535548==    by 0x5C3EC7E: ???
==535548==    by 0x5C3EDA4: ???
==535548==    by 0x5C5DE49: ???
==535548==    by 0x5C5DF46: ???
==535548==    by 0x5CB6368: ???
==535548==    by 0x4862467: ???
==535548==    by 0x4971660: sqlite3VdbeExec (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==535548==    by 0x4979660: sqlite3_step (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==535548==    by 0x1292D6: ??? (in /usr/bin/sqlite3)
==535548==    by 0x12AB95: ??? (in /usr/bin/sqlite3)
==535548== 
==535548== LEAK SUMMARY:
==535548==    definitely lost: 5,040 bytes in 7 blocks
==535548==    indirectly lost: 82,716 bytes in 38 blocks
==535548==      possibly lost: 0 bytes in 0 blocks
==535548==    still reachable: 314,112 bytes in 481 blocks
==535548==         suppressed: 0 bytes in 0 blocks
==535548== Reachable blocks (those to which a pointer was found) are not shown.
==535548== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==535548== 
==535548== For lists of detected and suppressed errors, rerun with: -s
==535548== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 0 from 0
@gwenn
Copy link
Owner

gwenn commented Mar 20, 2025

Sorry but I probably won't have time to investigate.

@LaurentDhont
Copy link
Author

So sad, would you like a reimbursement to fix the issue?

@gsker
Copy link
Contributor

gsker commented Mar 21, 2025

@LaurentDhont
Perhaps you could test to see if you get the same issue with the upstream ICU extension?

I'm not sure how to test it, but it's in the debian repo as
libsqlite3-ext-icu.

Also you could try this set of extensions instead:

At least it appears to be maintained.

I loaded the binary release and ran this:

sqlite> .load '/tmp/sqlean/regexp'    
sqlite> select regexp_substr('the year is的 °21', '[a-z 的]+');    
the year is的

Just some ideas. I would not be able to help with finding the leak.

(I emailed this comment too, but it didn't show up so it might end up duplicated)

@gwenn
Copy link
Owner

gwenn commented Mar 21, 2025

main.c.txt

cc -Wall -g -o test main.c `pkg-config --libs sqlite3` `pkg-config --libs icu-i18n`
valgrind --tool=memcheck ./test
==8712== Memcheck, a memory error detector
==8712== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==8712== Using Valgrind-3.24.0 and LibVEX; rerun with -h for copyright info
==8712== Command: ./test
==8712==
Lasdfasdfasdfent
Sergiu
==8712==
==8712== HEAP SUMMARY:
==8712==     in use at exit: 89,100 bytes in 46 blocks
==8712==   total heap usage: 416 allocs, 370 frees, 653,148 bytes allocated
==8712==
==8712== LEAK SUMMARY:
==8712==    definitely lost: 0 bytes in 0 blocks
==8712==    indirectly lost: 0 bytes in 0 blocks
==8712==      possibly lost: 0 bytes in 0 blocks
==8712==    still reachable: 89,100 bytes in 46 blocks
==8712==         suppressed: 0 bytes in 0 blocks
==8712== Rerun with --leak-check=full to see details of leaked memory
==8712==
==8712== For lists of detected and suppressed errors, rerun with: -s
==8712== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gwenn @gsker @LaurentDhont