-
|
First of, hi and thank you for this project. It makes working against SQLite databases a lot easier. It seems like I've managed to run into a multi-threaded issue. I am creating the storage as per below: namespace database_manager {
// A setting as represented in the database
struct SettingHolder {
int id = 0;
std::string name;
std::string value;
};
// A room as presented in the database
struct Room {
int id = 0;
std::string friendly_name;
int display_order = 0;
};
static inline auto get_db() {
auto db = sqlite_orm::make_storage("/data/nspanelmanager_db.sqlite3",
sqlite_orm::make_table("web_settings",
sqlite_orm::make_column("id", &SettingHolder::id, sqlite_orm::primary_key()),
sqlite_orm::make_column("name", &SettingHolder::name),
sqlite_orm::make_column("value", &SettingHolder::value)),
sqlite_orm::make_table("web_room",
sqlite_orm::make_column("id", &Room::id, sqlite_orm::primary_key()),
sqlite_orm::make_column("friendly_name", &Room::friendly_name),
sqlite_orm::make_column("displayOrder", &Room::display_order)));
return db;
}
}; // namespace database_managerWhen in the main thread I can simply do something like this: auto db_room = database_manager::get_db().get<database_manager::Room>(2);
SPDLOG_DEBUG("Got room {}::{}.", db_room.id, db_room.friendly_name);
auto db_setting = database_manager::get_db().get<database_manager::SettingHolder>(2);
SPDLOG_DEBUG("Got value {}", db_setting.value);and that works absolutely fine. However, If I then do the exact same thing in another thread it will simple crash as per the backtrace: It seems that calling from a different thread causes sqlite3_mutex_enter to want to dereference a mutex that isn't set? I've looked though the issues on here that I found that talked about multi-threading and I've also tried reading up on how to do multi-threading with only SQLite3 but for the life of me I can't figure it out. I've tried both version 1.9 and version 1.8.2 from conan. Thankful for any help! |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 28 replies
-
|
Hi. Thanks for such a warm feedback! This issue looks as usual multi-threading issue. Storage doesn't handle multi-threading for, it just passed the calls to SQLite engine. Of course there is a small responsibility area of storage itself which probably has to be covered with multi-threading primitives to handle cases like this but it will reduce performance in single-threaded apps. Quick advice: try to call |
Beta Was this translation helpful? Give feedback.
-
|
What you are reporting here seems really odd. Can you check the result of SQLite3 itself supports multiple concurrency modes, however it is threadsafe by default. There is a whole range of information available to give you a complete picture of multithreading and multiple access to a single database in SQLite: Especially because you are using a fresh sqlite_orm 'storage' instance on every call there should be no problem, as I understand it. Like @fnc12 I also suggest that you create the sqlite_orm 'storage' instance only once (i.e. a singleton), and use |
Beta Was this translation helpful? Give feedback.
-
|
I've opened a thread on the SQlite forum but not heard anything back as of yet. In the mean time I've been doing some more digging, including stepping through each line of the code called and it seems that the .mutex object is simply reset to the default 0x0 which seems VERY strange to me. I also figured out the Backtrace: |
Beta Was this translation helpful? Give feedback.
-
|
I too was experiencing the same issue recently, I'm on Apple Silicon M3 Max chip, the issue was happening on sqlite v3.43.2, however when I pointed my lib to v3.48 (latest version) the error disappeared, |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your suggestion @codinghaven, unfortunetly it seems that it didn't solve my problem. It still resets the sqlite3Config.Mutex config when calling Even if I run The thing is also that earlier in the program it run The strange thing is that when it runs the first command |
Beta Was this translation helpful? Give feedback.
-
|
I've set up gdb to break in
From what I've been able to see in |
Beta Was this translation helpful? Give feedback.
-
|
Hm, it seems I've made some progress. If I have everything declare as |
Beta Was this translation helpful? Give feedback.
-
|
It seems this may be the same issue as #1267 as it works just fine when compiled as STATIC instead of SHARED in CMake, big question is why. I can switch to using STATIC compilation but that would also mean that it takes a good while to recompile the binary every time I make changes so a better solution would be nice. |
Beta Was this translation helpful? Give feedback.
-
|
Well then, it seems the issue has been that when compiling with |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the fast reply! My crash happens in the following scenario:
I also tried to reproduce the crash with raw libsqlite now: I created sqlite3* in the unittest, passed it to the same class imported from the shared library and executed a statement there. Now no crash happens. |
Beta Was this translation helpful? Give feedback.
-
|
hmm ok looks like it is |
Beta Was this translation helpful? Give feedback.
-
|
Yes, see below. sqlite_orm works fine in my production code where everything runs in my shared library by the way. Only when creating the storage in the unittest I have the crash. 0 0x0000000000000000 in ?? () |
Beta Was this translation helpful? Give feedback.
-
|
it crashes inside |
Beta Was this translation helpful? Give feedback.
-
|
I took another look at the issue and discovered the following:
Interestingly, I can avoid the crash—without statically linking SQLite—by calling sqlite3_initialize() within my shared library before invoking any methods on the storage object. This leads me to suspect that there might be two separate instances of sqlite3GlobalConfig: one in the main executable and another in the shared library. If that’s the case, maybe sqlite_orm could detect this scenario and call sqlite3_initialize() internally if necessary? But I'm not sure if this is the correct approach, as I'm not deeply familiar with the internals of either SQLite or sqlite_orm. For now, explicitly calling sqlite3_initialize() in my shared library is a good workaround allowing me to avoid static linking. |
Beta Was this translation helpful? Give feedback.
Well then, it seems the issue has been that when compiling with
add_library(.... SHARED .....in CMake it will import SQLite multiple times, changing this toadd_library(.... STATIC .....will solve this issue. It would be nice if this is solvable in SQLite ORM but I'm not sure of how as I am surely no expert in CMake. Perhaps if SQLite ORM imported SQLite as PRIVATE? I'm not sure but it seems I've solved it for my use case anyways.