Dynamically Linked Library in CPP #11439
Conversation
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
@soumiiow thanks for looking into this. Out of curiosity, why doesn't this work in MacOS? |
| @@ -15,6 +15,7 @@ add_subdirectory(base) | |||
| add_subdirectory(caching) | |||
| add_subdirectory(compression) | |||
| add_subdirectory(config) | |||
| add_subdirectory(dynamicRegistry) | |||
There was a problem hiding this comment.
we use snake case for directory names "dynamic_registry"
| #include <dlfcn.h> | ||
| #include <iostream> | ||
| #include "velox/common/base/Exceptions.h" | ||
| namespace facebook::velox { |
There was a problem hiding this comment.
nit: new line before namespace definition.
| VELOX_USER_FAIL("Couldn't find Velox registry symbol: {}", error); | ||
| } | ||
| registryItem(); | ||
| std::cout << "LOADED DYLLIB 1" << std::endl; |
There was a problem hiding this comment.
for consistency, could you use LOG(INFO) and print the file name / path of the library loaded?
|
|
||
| static constexpr const char* kSymbolName = "registry"; | ||
|
|
||
| void loadDynamicLibraryFunctions(const char* fileName) { |
There was a problem hiding this comment.
we can probably omit the "Functions" from the name, and this can be used to really load anything, as long as you provide the registration functions. Let's name it loadDynamicLibrary()
| ### 1. Create a cpp file for your dynamic library | ||
| For dynamically loaded function registration, the format followed is mirrored of that of built-in function registration with some noted differences. Using [MyDynamicTestFunction.cpp](tests/MyDynamicTestFunction.cpp) as an example, the function uses the extern "C" keyword to protect against name mangling. A registry() function call is also necessary here. | ||
|
|
||
| ### 2. Register functions dynamically by creating .dylib or .so shared libraries and dropping them in a plugin directory |
There was a problem hiding this comment.
nit: the titles are too long; maybe just add the docs as a refular numbered list?
| auto signaturesBefore = getFunctionSignatures().size(); | ||
|
|
||
| // Function does not exist yet. | ||
| EXPECT_THROW(dynamicFunction(0), VeloxUserError); |
There was a problem hiding this comment.
can you use VELOX_ASSERT_THROW() instead to validate the right exception is being thrown?
| # `MyDynamicFunction.cpp` as a small .so library, and use the | ||
| # MY_DYNAMIC_FUNCTION_LIBRARY_PATH macro to locate the .so binary. | ||
| add_compile_definitions( | ||
| MY_DYNAMIC_FUNCTION_LIBRARY_PATH="${CMAKE_CURRENT_BINARY_DIR}") |
There was a problem hiding this comment.
please vendor the macro. Maybe something like VELOX_TEST_DYNAMIC_LIBRARY_PATH
| * limitations under the License. | ||
| */ | ||
|
|
||
| #include "velox/common/dynamicRegistry/DynamicLibraryLoader.h" |
There was a problem hiding this comment.
this header include is not required.
|
|
||
| // Dynamically load the library. | ||
| std::string libraryPath = MY_DYNAMIC_FUNCTION_LIBRARY_PATH; | ||
| libraryPath += "/libvelox_function_my_dynamic.so"; |
There was a problem hiding this comment.
Use CMAKE_SHARED_LIBRARY_SUFFIX and CMAKE_SHARED_LIBRARY_PREFIX to support MacOS.
https://stackoverflow.com/questions/32445070/how-does-cmake-know-which-prefixes-and-suffixes-to-add-to-shared-libraries
https://cmake.org/cmake/help/v3.0/variable/CMAKE_SHARED_LIBRARY_PREFIX.html
https://cmake.org/cmake/help/v3.0/variable/CMAKE_SHARED_LIBRARY_SUFFIX.html
There was a problem hiding this comment.
What else is an issue for MacOS?
Related to prestodb/presto#23634 in the Prestissimo space
and based off of the following PR: https://github.com/facebookincubator/velox/pull/1005/files
These changes will allow users to dynamically load functions in prestissimo using cpp. The Presto Server will use this library to dynamically load User Defined Functions (UDFs), connectors, or types.
an example of dynamically registering a function is also provided for reference, along with a unit test
Currently, this library works on linux machines but not MacOS.