April 4, 2024 by Craig Scott<\/a><\/p>\n\n\n\n CMake 3.28 was the first version to officially support C++20 modules. Tutorials and examples understandably tend to focus on the fairly simple scenario of building a basic executable, perhaps also adding in a static library. The import CMake; the Experiment is Over!<\/a> blog article from Kitware is perhaps one of the best known, and it covers exactly these things. And while these are important first steps, stopping there sells the reader short. The real fun starts when building, installing, and consuming shared libraries. This quickly exposes current limitations of toolchains and the build system, and it also highlights common misconceptions about what modules provide.<\/p>\n\n\n\n Estimated reading time: 13 minutes<\/p>\n\n\n\n NOTE: For a detailed background on linker symbol visibility and shared libraries, see my Deep CMake For Library Authors<\/a> talk from CppCon 2019. This article here assumes the reader is familiar with a number of topics from that talk.<\/p>\n\n\n\n Consider the following example:<\/p>\n\n\n\n A common misconception among developers is that C++20 modules replace the need to handle symbol visibility, but this is not the case. Modules still require symbol visibility control, just like non-module code. This likely springs from confusion between reachability<\/em>, which is a compile-time concept, and symbol visibility, which is more a link-time constraint.<\/p>\n\n\n\n A shared library might want to define and use a C++20 module in its implementation, but not expose that module as part of the shared library\u2019s public API. This is an example of where a module needs to be reachable when building the shared library, but not visible at link time. On the other hand, a library might want a C++20 module exposed in its public API, and for that it needs to export symbols to make them visible, just like non-module code.<\/p>\n\n\n\n The The above is exactly the same as how we might do it for a project without C++20 modules. Perhaps surprisingly, we add the generated header to the The C++ source code also follows a similar pattern to the non-module case. The interface source file brings in the generated header via algo-interface.cppm<\/em><\/p>\n\n\n\n algo-impl.cpp<\/em><\/p>\n\n\n\n The generated header We put the Currently, the clang and Visual Studio compilers don\u2019t require annotating the Some may ask why the The BMI (built module interface) file for the module is what drives the above need for separation. The BMI is a binary representation of the module\u2019s interface. When a compiler is compiling source code that uses the module (\u201cconsuming\u201d the module), it needs the module\u2019s BMI to know the module\u2019s interface and how to use it. This is analogous to the role of header files in the non-C++20-module world. A key difference is that BMIs capture not only the code\u2019s API (i.e. what header files traditionally do), they also capture binary aspects of how the module was built. They can embed details like the language standard used, compiler definitions, flags that affect the binary layout of symbols or how code is generated, and so on. These are all determined when the shared library is built<\/em>. But when a compiler is consuming a module, it needs a BMI that matches the settings of the consumer<\/em>. And as we will see shortly, when that module is delivered by a shared library, the BMIs are pretty much never the same between the producer and consumer of a module.<\/p>\n\n\n\n A critical problem is the compiler definition used to mark symbols for export. That definition needs to evaluate to different values depending on whether you are exporting (building) or importing (consuming) the module. For example, with the Visual Studio toolchain, our Even if CMake could provide a separate BMI for consumers, it is still unlikely that it would be useful. Currently, toolchains are typically very sensitive to any differences that affect the BMI. Even a minor or patch difference in the compiler version could be enough to render a BMI unusable by a consumer. The package vendor could never hope to provide BMIs for all the different variations of compilers, flags, etc. that consumers may use. Because of this, the consumer effectively always has to create its own BMI with the compiler settings it needs. In order for it to do that, it needs the source code that defines the module interface. So we\u2019re back to essentially having to provide some sort of equivalent to header files for our module interface definition to avoid exposing the implementation details! That sounds like modules are not delivering on their promise, but they do still provide benefits, just maybe not all the ones you thought you were getting. \ud83d\ude09<\/p>\n\n\n\n Before C++20 modules, installing an ordinary shared library target in CMake mostly meant installing the shared library binary and the headers that define the library\u2019s public API. When a shared library includes C++20 modules in its public API, we now also need a way to provide BMIs for the library\u2019s C++20 modules. As discussed in the previous section, we can\u2019t reliably provide the BMIs directly (although CMake does provide the ability to do that). Instead, we install the interface sources for the module and let CMake generate a BMI on-the-fly for the consumer.<\/p>\n\n\n\n One consequence of this is that while headers included in a module interface\u2019s global fragment are not directly part of the module interface, those headers must still be available to module consumers. Without them, the consumer cannot create their own BMI for the module. This is why the generated Continuing our earlier example, the installation commands might look something like this:<\/p>\n\n\n\n The The The relative layout of the installed files resulting from the above might look something like the following (this would be for installing a The following is a minimal project demonstrating how to consume the installed shared library with C++20 modules from the earlier examples:<\/p>\n\n\n\n The above example looks no different to any other consumer project without C++20 modules. But a shared library that provides C++20 modules as part of its public API places stronger conditions on its consumers than for the non-module case. One of the more important additional constraints is that consumers must generally use the exact same language standard version as the one the shared library was built with. The BMI that CMake generates on-the-fly for the consumer uses the same C++ language standard as the shared library, not the one used by the consumer. This is essential, because the BMI must correspond to the built module binary. If the consumer tries to build its own module-consuming C++ code with a different language standard, toolchains will usually reject the BMI provided by CMake due to the mismatch. Change the This means the consumer cannot use a newer C++ standard than the shared library. Such a restriction is not typically present for non-module cases (yes there are caveats, but in general, you\u2019ll likely get away with it for average code).<\/p>\n\n\n\n CMake also enforces another related condition on projects that install C++20 modules. For non-module code, you can control the C++ language standard used to build the module\u2019s shared library target by setting the A project will build and even install successfully with just the Perversely, the same is not true for the There is one more scenario worth highlighting, and this only appears to affect the Clang toolchain, and only Clang 18.0 or older (18.1 has a fix<\/a>). If the consumer also wants to define its own shared library target and set The fix added in Clang 18.1 removes the visibility settings from the things that must be consistent for a BMI. A potential workaround if you need to use Clang 18.0 or older is to set the Normally, a consuming project shouldn\u2019t modify the target defined by an installed dependency like this, but there doesn\u2019t seem to be an alternative workaround at this time other than use Clang 18.1 or later. You can track progress on the CMake side of this particular problem in issue 25868<\/a> in CMake\u2019s issue tracker.<\/p>\n\n\n\n The previous sections paint the picture that, provided some basic guidelines are followed, projects can install shared libraries, and they can be consumed much like any other non-module case, perhaps with one or two minor caveats or workarounds. At the time of writing, this is only true if the installing and consuming CMake projects are both using either the Over time, we would expect CMake\u2019s generators to improve their support of C++20 modules. The toolchains themselves still have limitations as well, so it will take some evolution of all the moving parts together before we finally have more or less full functionality available. See the cxxmodules manual<\/a> for the latest status of CMake\u2019s C++20 modules support.<\/p>\n","protected":false},"excerpt":{"rendered":" C++20 Modules, CMake, And Shared Libraries April 4, 2024 by Craig Scott CMake 3.28 was the first version to officially support C++20 modules. Tutorials and examples understandably tend to focus on the… Read more »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51,84,220,87,10],"tags":[341,86,342],"class_list":["post-922","post","type-post","status-publish","format-standard","hentry","category-article","category-build","category-cpp","category-coding","category-development","tag-c-modules","tag-c20","tag-shared-libraries"],"_links":{"self":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/922","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/comments?post=922"}],"version-history":[{"count":1,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/922\/revisions"}],"predecessor-version":[{"id":923,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/posts\/922\/revisions\/923"}],"wp:attachment":[{"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/media?parent=922"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/categories?post=922"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tbekk.com\/devstream\/wp-json\/wp\/v2\/tags?post=922"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Table of Contents<\/h2>\n\n\n\n
\n
Linker Symbol Visibility<\/h2>\n\n\n\n
cmake_minimum_required(VERSION 3.28.2)\nproject(cxx_modules_example LANGUAGES CXX)\n\n# These will be discussed later in the article\nset(CMAKE_CXX_EXTENSIONS FALSE)\nset(CMAKE_CXX_VISIBILITY_PRESET hidden)\nset(CMAKE_VISIBILITY_INLINES_HIDDEN TRUE)\n\nadd_library(Algo SHARED)\n\ntarget_sources(Algo\n PRIVATE\n algo-impl.cpp\n PUBLIC\n FILE_SET CXX_MODULES\n FILES\n algo-interface.cppm\n)\n\n# CMake requires the language standard to be specified as compile feature\n# when a target provides C++20 modules and the target will be installed \ntarget_compile_features(Algo PUBLIC cxx_std_20)<\/code><\/pre>\n\n\n\ngenerate_export_header()<\/code> command provided by CMake\u2019s GenerateExportHeader<\/code> module is the method of choice for managing symbol visibility in most CMake projects. This remains the case for C++20 module code as well. The CMAKE_CXX_VISIBILITY_PRESET<\/code> variable is set to hidden<\/code> and CMAKE_VISIBILITY_INLINES_HIDDEN<\/code> to TRUE<\/code> to ensure that only the symbols we want to export are made visible as part of our public API. The generate_export_header()<\/code> command then gives us a header that provides compiler definitions we can use to explicitly annotate the things we want included in our API. The following demonstrates how one might add that to the CMakeLists.txt<\/code> of the previous example:<\/p>\n\n\n\ninclude(GenerateExportHeader)\ngenerate_export_header(Algo)\ntarget_sources(Algo\n PUBLIC\n FILE_SET HEADERS\n BASE_DIRS ${CMAKE_CURRENT_BINARY_DIR}\n FILES\n ${CMAKE_CURRENT_BINARY_DIR}\/algo_export.h\n)<\/code><\/pre>\n\n\n\nPUBLIC<\/code> HEADERS<\/code> file set rather than a PRIVATE<\/code> file set, even though nothing outside the Algo<\/code> target should need to read the header. If the shared library is only ever linked to by things in the same build (i.e. not from a scenario where the library has been installed), then this could indeed be a PRIVATE<\/code> file set. Things change once the library is installed though, as will be discussed later in this article.<\/p>\n\n\n\n#include<\/code>, and we annotate the things within that interface file that we want to make part of the public API. The implementation source file doesn\u2019t need to contain any visibility-related items. Here\u2019s how these might look for the current example:<\/p>\n\n\n\nmodule;\n\n#include <algo_export.h> \/\/ <-- Generated header added to the global fragment\n\nexport module algo; \/\/ <-- Annotation not currently required, but see discussion below\n\nexport class ALGO_EXPORT Algo \/\/ <-- ALGO_EXPORT annotation added to the class definition\n{\npublic:\n void helloWorld();\n};<\/code><\/pre>\n\n\n\nmodule;\n\n#include <iostream>\n\nmodule algo;\n\nvoid Algo::helloWorld()\n{\n std::cout << \"hello world\\n\";\n}<\/code><\/pre>\n\n\n\nalgo_export.h<\/code> is included in the global fragment of the interface file (after the module<\/code> keyword given on its own at the top of the file). This makes everything the generated header defines available for use in defining the interface itself, but consumers of the module cannot directly use anything defined by the header. The global fragment is not directly part of the interface exposed by the module. But as will be seen later, everything in the global fragment may need to be available to module consumers.<\/p>\n\n\n\nALGO_EXPORT<\/code> annotation on the Algo<\/code> class, just like we would if Algo<\/code> was defined outside any C++20 module. This annotation makes the whole class visible by exporting all its symbols for the linker. The fact that the class is inside a module doesn\u2019t change how we do that, although it will likely change the mangled name of the class\u2019 symbols as seen by the linker (see this article<\/a> for a basic introduction to name mangling). For example, with clang 17 on macOS, the mangled name for Algo::helloWorld()<\/code> inside the algo<\/code> module is __ZNW4algo4Algo10helloWorldEv<\/code>, whereas if the class is defined outside any module, its mangled name is __ZN4Algo10helloWorldEv<\/code>. Note how the mangled name includes the algo<\/code> module name for the symbol defined inside the module. This follows the strong module ownership<\/a><\/em> model, where the same symbol defined in different modules declares different entities (i.e. void doSomething()<\/code> in module A<\/code> would be distinct from void doSomething()<\/code> defined in module B<\/code>). The alternative (weak module ownership) has essentially been abandoned by toolchain vendors, and it may eventually be removed from the standard due to it being problematic to implement.<\/p>\n\n\n\nexport module algo<\/code> line that declares the start of the module interface. There is some debate<\/a> over whether an annotation should be required there, or whether to work it out automatically based on whether any part of the module is made visible. Toolchains seem to be favouring working it out for themselves at the moment, but this might not remain so.<\/p>\n\n\n\nConsequences Of BMI Implementation Details<\/h2>\n\n\n\n
Algo::helloWorld()<\/code> implementation is split out to a separate file. This looks an awful lot like the existing header files pattern, but weren\u2019t modules meant to free us from all of that? Conceptually, a module could be fully defined in a single source file, and its interface extracted from that for consumers to use without needing access to the source file. Separating the interface and implementation should not be necessary. But in practice, toolchain implementation details mean if we don\u2019t separate them, we would have to install that source file along with the library (for reasons to be discussed shortly). Providing the full source files would be a non-starter for most closed-source applications, so the separation is usually desirable. When separated, only the file containing the module interface<\/em> needs to be provided along with the shared library, since that\u2019s all the toolchain needs when compiling a consumer. This also has the added advantage that the consumer doesn\u2019t have to be recompiled if the implementation of the module changes, but not its interface. This is just like the analogous case with non-module code and headers defining interfaces but not implementations.<\/p>\n\n\n\nALGO_EXPORT<\/code> annotation might need to evaluate to something like __declspec(dllexport)<\/code> when building the library, and __declspec(dllimport)<\/code> when consuming the library (issue 25539<\/a> in the CMake issue tracker goes into more detail on this specific topic). A direct consequence of this is that when packaging the shared library for distribution, the package cannot simply provide the BMI it used when building the shared library. It would have to generate a new, separate BMI for installation, and CMake doesn\u2019t currently provide such a facility.<\/p>\n\n\n\nInstalling Shared Libraries With C++20 Modules<\/h2>\n\n\n\n
algo_export.h<\/code> header was put in a PUBLIC<\/code> file set rather than a PRIVATE<\/code> one earlier in this article. This ensures the header is installed along with the target, and made available to consumers, just like any other public header that forms part of the shared library\u2019s API.<\/p>\n\n\n\ninclude(GNUInstallDirs)\n\ninstall(TARGETS Algo\n EXPORT my_package-targets\n # ... a few details omitted, see the \"Deep CMake For Library Authors\" talk\n FILE_SET CXX_MODULES\n # There's currently no convention for this location, see discussion below\n DESTINATION ${CMAKE_INSTALL_LIBDIR}\/cmake\/my_package\/src\n FILE_SET HEADERS\n DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} # Same as default, could be omitted\n INCLUDES\n DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}\n)\ninstall(EXPORT my_package-targets\n DESTINATION ${CMAKE_INSTALL_LIBDIR}\/cmake\/my_package\n CXX_MODULES_DIRECTORY .\n)\n# The following file includes the my_package-targets.cmake file\ninstall(FILES my_package-config.cmake\n DESTINATION ${CMAKE_INSTALL_LIBDIR}\/cmake\/my_package\n)<\/code><\/pre>\n\n\n\nDESTINATION<\/code> for the FILE_SET CXX_MODULES<\/code> in the install(TARGETS...)<\/code> call tells CMake where to install the module interface sources. There\u2019s currently no strong convention for where to put these in the installed layout, but putting it somewhere below where the CMake config package file is located is a reasonable choice. If the module name is part of the source file name, then it may be reasonable to install the module interface sources to a common directory, as shown in the above example. If file name clashes between modules are a potential issue, module-specific subdirectories might be necessary.<\/p>\n\n\n\nCXX_MODULES_DIRECTORY<\/code> option to install(EXPORT...)<\/code> tells CMake where it should install some additional files it generates for each module in the export set. Again, there\u2019s no strong convention yet for this location, but since the files are strongly coupled to the exported targets file (my_package-targets.cmake<\/code>), they should go in the same directory as that exported targets file, or a subdirectory below that. A relative path given to CXX_MODULES_DIRECTORY<\/code> is treated as relative to where the export file is installed to, so that\u2019s a strong hint not to install these to somewhere outside that directory. CMake 3.28.2 and later generate files with names that include the name of the EXPORT<\/code> set. This ensures names don\u2019t clash if including multiple export sets in the one CMake package. Earlier CMake versions did not include the name of the EXPORT<\/code> set in the generated file names, which is why we took the unusual step of including the CMake patch version in the minimum CMake version requirement at the very beginning of the first example (3.28.2<\/code> instead of just 3.28<\/code>):<\/p>\n\n\n\ncmake_minimum_required(VERSION 3.28.2)<\/code><\/pre>\n\n\n\nRelease<\/code> build on macOS):<\/p>\n\n\n\n<BASE>\n +-- include\n | +-- algo_export.h\n +-- lib\n +-- cmake\n | +-- my_package\n | +-- src\n | | +-- algo-interface.cppm\n | +-- cxx-modules-my_package-targets.cmake\n | +-- cxx-modules-my_package-targets-Release.cmake\n | +-- my_package-config.cmake\n | +-- my_package-targets.cmake\n | +-- my_package-targets-release.cmake\n | +-- target-Algo-Release.cmake\n +-- libAlgo.dylib<\/code><\/pre>\n\n\n\nConsuming Installed C++20 Modules<\/h2>\n\n\n\n
cmake_minimum_required(VERSION 3.28)\nproject(cxx_modules_consumer LANGUAGES CXX)\n\n# Hard-coded path just for illustration. Prefer relying on CMAKE_PREFIX_PATH instead.\nfind_package(my_package REQUIRED PATHS \/path\/to\/<BASE>)\n\n# Neither of these two are technically needed, but they make the expectation clear\nset(CMAKE_CXX_STANDARD 20)\nset(CMAKE_CXX_EXTENSIONS FALSE)\n\nadd_executable(app main.cpp)\ntarget_link_libraries(app PRIVATE Algo)<\/code><\/pre>\n\n\n\nCMAKE_CXX_STANDARD<\/code> to 23<\/code> in the above example to see this in action. With clang 17, an error like the following would result:<\/p>\n\n\n\nerror: C++23 was disabled in PCH file but is currently enabled\nerror: module file CMakeFiles\/Algo@synth_838524ceec9e.dir\/163bc42a95ce.bmi cannot be loaded due to a configuration mismatch with the current compilation [-Wmodule-file-config-mismatch]<\/code><\/pre>\n\n\n\nCXX_STANDARD<\/code> target property (this is normally set project-wide with the CMAKE_CXX_STANDARD<\/code> variable). However, this property is not sufficient for targets that will be installed. The CXX_STANDARD<\/code> target property only affects building the shared library, it does not get carried through when the target is installed. Compile features do<\/em> propagate though, and CMake relies on this for BMI creation for the consumer. CMake therefore requires installed targets to specify the language standard with a compile feature if they provide any C++20 modules. The example at the beginning of this article included the following line to achieve this:<\/p>\n\n\n\ntarget_compile_features(Algo PUBLIC cxx_std_20)<\/code><\/pre>\n\n\n\nCXX_STANDARD<\/code> property and no compile feature. But when another project tries to consume the installed module target, CMake will halt with a fatal error. The installed target lacks the information CMake needs to be able to provide a BMI for the consumer.<\/p>\n\n\n\nCXX_EXTENSIONS<\/code> target property, which does<\/em> propagate to the installed target. There is no equivalent compile feature for this property, so the target property is the only way this setting can be specified. This property setting also needs to be consistent between the installed shared library and its consumer, otherwise most toolchains will once again reject the consumer\u2019s generated BMI.<\/p>\n\n\n\nCXX_VISIBILITY_PRESET<\/code> to hidden<\/code> on that target, a problem arises with the BMI generation for the consumer. The internal Clang invocation that CMake uses to generate a BMI will have a different visibility setting than the consumer, and Clang 18.0 and older will fail with an error similar to this:<\/p>\n\n\n\nerror: default visibility for functions and variables [-fvisibility] differs in PCH file vs. current file\nerror: module file CMakeFiles\/Algo@synth_838524ceec9e.dir\/163bc42a95ce.bmi cannot be loaded due to a configuration mismatch with the current compilation [-Wmodule-file-config-mismatch]<\/code><\/pre>\n\n\n\nCXX_VISIBILITY_PRESET<\/code> property to hidden<\/code> on the installed imported target (Algo<\/code> in this example):<\/p>\n\n\n\n# Algo is an imported target that we shouldn't touch, but we need this if\n# consumers have this set on the consuming target(s)\nset_target_properties(Algo PROPERTIES CXX_VISIBILITY_PRESET hidden)<\/code><\/pre>\n\n\n\nCMake Generator Limitations<\/h2>\n\n\n\n
Ninja<\/code> or Ninja Multi-Config<\/code> generators. The Visual Studio 17 2022<\/code> generator handles part of the picture, but it cannot install a shared library that provides C++20 modules. It also cannot consume modules installed by a project built with one of the Ninja generators (the Visual Studio generator lacks support for generating the BMIs for the consumer). These are gaps in CMake\u2019s implementation, not in Visual Studio. None of the other CMake generators have any support for C++20 modules at all.<\/p>\n\n\n\n