How to Get Dynamic Loading of Views on macOS?

[rbugajewski]

Dynamic compilation of views works on macOS after I set the configuration variable load_dynamic_views as stated in the documentation. I can see the corresponding log output and there’s also a handle from the dlopen() call in loadLibs().

Although this mechanism works, I doesn’t look like the framework would pick up these new libraries during its runtime on macOS as I need to restart the process to see newly added CSP output.

Steps to Reproduce

1. Create a new Drogon project
2. Set load_dynamic_views to true
3. Add ../views to the dynamic_views_path configuration array
4. Set dynamic_views_output_path to ./
5. Add views/Index.csp and some example content like “Hello, world!”
6. Compile & run the application
7. Open the index page in the browser and verify that it shows “Hello, world!”
8. Edit the Index.csp file, for example by changing the string to “Hello, universe!”
9. Wait until the framework picks up the change
10. Reload the page in the browser

Expected behavior
The page should output “Hello, universe!”.

Actual results
The page still displays “Hello, world!”.

Desktop:

• OS: macOS 10.15
• Browser: Safari

[an-tao]

@rbugajewski If you run your application in macOS, dynamic views should not be compiled into the application statically. This means that if the view is statically compiled, it cannot be updated via dynamic view loading. You can create a directory outside the compilation folder and move views into it during development.
I haven't found a solution to make macos behave the same as linux on this feature. If you have any suggestions, please feel free to tell me.

[rbugajewski]

If you run your application in macOS, dynamic views should not be compiled into the application statically. This means that if the view is statically compiled, it cannot be updated via dynamic view loading.

Thanks for the clarification, I would suggest to add this one sentence to the documentation.

I particularly don’t like to move stuff around, even during development, so I currently just set an invalid views path in CMake which also works as a workaround. I just have to remember to change this before committing. I’ll think about improving the situation on macOS.

The dynamic loading works now, but I sometimes get a crash.

* thread #3, stop reason = EXC_BAD_ACCESS (code=1, address=0x10ab53488)
  * frame #0: 0x00000001067b2d02 www-cpp`std::__1::__function::__value_func<drogon::DrObjectBase* ()>::operator(this=0x00007fd2b3808600)() const at functional:1799:16
    frame #1: 0x00000001067ad515 www-cpp`std::__1::function<drogon::DrObjectBase* ()>::operator(this=0x00007fd2b3808600)() const at functional:2347:12
    frame #2: 0x00000001067ad411 www-cpp`drogon::DrClassMap::newObject(className="Index") at DrClassMap.cc:54:16
    frame #3: 0x00000001067b811d www-cpp`drogon::DrTemplateBase::newTemplate(templateName="Index") at DrTemplateBase.cc:35:9
    frame #4: 0x00000001068c1987 www-cpp`drogon::genHttpResponse(viewName=<unavailable>, data=0x0000700009a64470) at HttpResponseImpl.cc:36:18
    frame #5: 0x00000001068c1770 www-cpp`drogon::HttpResponse::newHttpViewResponse(viewName="Index", data=0x0000700009a64470) at HttpResponseImpl.cc:143:12
    frame #6: 0x0000000106777476 www-cpp`IndexController::asyncHandleHttpRequest(this=0x00007fd2b39002f0, req=std::__1::shared_ptr<drogon::HttpRequest>::element_type @ 0x00007fd2b3a0e760 strong=6 weak=1, callback=0x0000700009a65650)>&&) at IndexController.cc:6:16
    frame #7: 0x00000001069190b4 www-cpp`drogon::HttpSimpleControllersRouter::doControllerHandler(this=0x00007fd2b26036f0, ctrlBinderPtr=std::__1::shared_ptr<drogon::HttpSimpleControllersRouter::CtrlBinder>::element_type @ 0x00007fd2b26041b8 strong=1 weak=1, req=std::__1::shared_ptr<drogon::HttpRequestImpl>::element_type @ 0x00007fd2b3a0e760 strong=6 weak=1, callback=0x0000700009a6a630)>&&) at HttpSimpleControllersRouter.cc:219:21

I don’t see this as critical, because I think this is only related to dynamic loading that should only be used during development, but I wanted to let you know nonetheless.

[ihmc3jn09hk]

20200303 20:06:53.045599 UTC 7938 TRACE [operator()] new csp file:../views/Foo.csp - SharedLibManager.cc:119
20200303 20:06:53.045681 UTC 7938 TRACE [operator()] drogon_ctl create view ../views/Foo.csp -o ../views - SharedLibManager.cc:139
create view:../views/Foo.csp
create HttpView Class file by ../views/Foo.csp
className=Foo
20200303 20:06:53.155018 UTC 7938 TRACE [loadLibs] src:../views/Foo.cc - SharedLibManager.cc:184
20200303 20:06:53.155073 UTC 7938 TRACE [loadLibs] c++ ../views/Foo.cc -g -std=c++17  -I/home/drogon/lib/inc -I/home/drogon/orm_lib/inc -I/home/drogon/trantor -I/usr/include -I/home/drogon/install/include -shared -fPIC --no-gnu-unique -o ../views/Foo.so - SharedLibManager.cc:206
20200303 20:06:54.473099 UTC 7938 TRACE [loadLibs] Compiled successfully - SharedLibManager.cc:209
20200303 20:06:54.473248 UTC 7938 ERROR load ../views/Foo.so error! - SharedLibManager.cc:226
20200303 20:06:54.473262 UTC 7938 ERROR Error relocating ../views/Foo.so: _ZN7trantor6LoggerD1Ev: symbol not found - SharedLibManager.cc:227
20200303 20:06:57.980185 UTC 7938 TRACE [operator()] remove file ../views/Foo.csp.lock - SharedLibManager.cc:173

There is something wrong ?
In dynamic mode does not work and re-"make" has no problem.

Issue resolve by using drogon dynamically.
However, the view does not update even with compiled and loaded successfully for the .so
I have checked the .so generated which indeed changed.
Yet the response is still unchanged..

[an-tao]

@ihmc3jn09hk , You should configure your project with cmake .. -DCMAKE_ENABLE_EXPORTS=on for dynamical views. see this and the PR #375
And according the latest testing, I found that both on linux and on MacOS, dynamical views should not be compiled into the application statically. This means that if the view is statically compiled, it cannot be updated via dynamic view loading. @rbugajewski

[ihmc3jn09hk]

@an-tao Still no luck with the update. Do I need to new a drogon_ctl project? At my side, any view was compiled since the ${VIEWSRC} has been removed mannually for this purpose. The build directory contains no "any-views.h" or "any-views.cc". The views were built and load in runtime. However, the response was not updated even with the .so .cc .h had been updated (Checked manually). Well, for your information, I was testing with a small editing, e.g. the color of a style element in the .csp

.ele {
  color: #0F0;   //To color: #FFF;
}

I dont think it is the case because the built .cc and .so files do contain such modification. Just for your reference.

[an-tao]

Do you get "symbol not found" output when loading a view?
Please give me your OS version and compiler version, thanks.

[ihmc3jn09hk]

Do you get "symbol not found" output when loading a view?
Please give me your OS version and compiler version, thanks.

That symbol resolve error was fixed by recompile drogon with BUILD_SHARED_LIB=1

Ubuntu 18.04 with glibc. I am wondering if the problem related to musl vs glibc. I am not using musl
Reference

[an-tao]

Do you get "symbol not found" output when loading a view?
Please give me your OS version and compiler version, thanks.

That symbol resolve error was fixed by recompile drogon with BUILD_SHARED_LIB=1

So it works now?

Ubuntu 18.04 with glibc. I am wondering if the problem related to musl vs glibc. I am not using musl
Reference

I've not tested this with musl, if it can't work with musl, I think it's reasonable because musl is somehow different with glibc.

[ihmc3jn09hk]

The dynamic view is still not working on my system. Will do more testing on that part in some day. It's not fatal tho UI design in C++ is not a preferable practise.
Thanks @an-tao

[phenomicall]

I solved the issue by commenting out the dynamic generation of view files in CMakeLists.txt:

#file(GLOB_RECURSE SCP_LIST ${CMAKE_CURRENT_SOURCE_DIR}/views/*.csp)
#foreach(cspFile ${SCP_LIST})
#    message(STATUS "cspFile:" ${cspFile})
#    get_filename_component(classname ${cspFile} NAME_WE)
#    message(STATUS "view classname:" ${classname})
#    ADD_CUSTOM_COMMAND(OUTPUT ${classname}.h ${classname}.cc
#            COMMAND drogon_ctl
#            ARGS create view ${cspFile}
#            DEPENDS ${cspFile}
#            VERBATIM )
#    set(VIEWSRC ${VIEWSRC} ${classname}.cc)
#endforeach()

And uncommenting the "ENABLE_EXPORTS" line:

set_property(TARGET ${PROJECT_NAME} PROPERTY ENABLE_EXPORTS ON)

[rbugajewski]

There is an issue with dynamic view loading during development and usage of the <%layout MyLayout %> directive. The views get recompiled, but visually stay the same until full recompilation like the application wouldn’t know about the (dynamically) recompiled pages. Should this be a new issue or part of this one?

This is related to #338.

[an-tao]

1. Make sure that the .lock files are removed.

2. On MacOS, the updating of the first view can't be loaded into drogon(I can't figure out why it happens), you could make a placeholder view to solve it.

3. Make sure that the drogon_ctl installed in your system is the latest version.

[rbugajewski]

Here’s what I’ve found out for macOS. There are two cases (dynamic loading turned on is a prerequisite). The application has a Layout template that acts as a layout and a MyPage template that uses this layout.

First Case: Page Modification

Doesn’t Work

1. Launch application.
2. Modify MyPage.

In this case nothing happens as I think the first view issue applies here.

Works

1. Create dummy page called _Debug.
2. Launch application.
3. Modify _Debug.
4. Modify MyPage.

The change gets picked up, but it is required to trigger a recompile of the first dummy _Debug page by editing its contents on macOS.

Second Case: Layout Modification

Doesn’t Work

1. Create dummy page called _Debug.
2. Launch application.
3. Modify _Debug.
4. Modify Layout.
5. Reload MyPage.

In this case a dynamic recompile gets triggered, but the change doesn’t get picked up.

Doesn’t Work

1. Create dummy page called _Debug.
2. Launch application.
3. Modify _Debug.
4. Modify Layout.
5. Modify MyPage.
6. Reload MyPage.

What happens in this case is that the modifications for MyPage get picked up after dynamic recompilation, but changes in Layout stay the same until the whole application is restarted, no matter if the layout or the concrete template get modified or not.

[an-tao]

@rbugajewski Please paste the MyPage.cc generated by drogon_ctl.

[rbugajewski]

By actually creating an almost empty dummy MyPage in the project I realized that the issue is gone, and has something to do with my project setup. Sorry, didn’t want to bother you.