Main . Blog

8/17/2010: Fun statically linking against the C runtime library

This week, I was requested a custom build of breezEngine to be statically linked against the C runtime library. Statically linking against the CRT is not a nice thing to do, which probably is why dynamically linking against the CRT has been set as default since MSVC 7.0.

The major trouble that comes with a static CRT is due to the fact that any self-contained module of your program (e.g. an EXE or DLL file) will instantiate its own private copy of the CRT, including its own private heap. The moment modules start sharing objects allocated on one of these heaps across module boundaries, it becomes essential that any shared object be de-allocated on the very heap that it was originally allocated on.

Things get even worse when it comes to sharing more complex objects (e.g. STL strings or containers), as these objects may very well appear to have been stack-allocated, however, memory allocation on some heap is inevitably going to occur inside any of these objects, and might do so any time you invoke a method for one of these objects.

Fortunately, I have been aware of these issues from the start, still, circumvention of the pitfalls described can be tedious from time to time, which is why I am going to share a few tricks on that matter in this entry.

The first thing to notice is that most of the problems arise from header-only / header-centric libraries, as header code is always instantiated inside the module that is actually using the library, therefore inheriting both the module's private CRT implementation and heap. Some code might even get inlined, blurring the lines between your own module and the foreign library. This is why cross-module STL container sharing performs that badly in a static CRT environment, almost inevitably leading to crashes, whenever memory that has been internally allocated by the container code instantiated in one module is freeed in another module (e.g. to make way for more memory to be allocated, see vector reallocation).

In consequence, many sources suggest that you never ever share complex types across module boundaries altogether, which would indeed be the most simple and elegant way to overcome these issues without further caution and thinking. However, this simple rule puts harsh limits to the functionality and simplicity of your library, as there no longer is a way of sharing element collections, strings and so on without the extensive use of pre-allocated buffers.

Another possible solution would be to write your own non-header-based container library, as code that is compiled into your library and subsequently only accessed via import/export linkage will always use the same heap, being the one created for the module it was compiled to. This is not really feasible for larger container libraries, yet, the observations made here are still relevant:

The third possibility to fix issues concerning the STL is to make use of allocators. STL allocators provide a not-too-complicated way of providing your own source of memory to most of the STL objects, thus it is possible to acquire (and release) memory using an internal function that has been firmly compiled and linked into one of your modules, always accessing the same private heap, regardless of the module in which the container is actually used.

Having overcome STL container problems, there are more issues to be solved, such as creation and deletion of custom-type objects scattered across several modules. One simple and elegant solution would be to make use of the abstract factory pattern, once again firmly compiling and linking both creation and deletion into one module, subsequently making calls to these functions instead of constructing or destroying objects manually. This is also the pattern that COM follows.

Another solution involves LOCALLY overloading operator new and operator delete inside often-shared classes. Overloading these will ensure that calling new and delete for the corresponding classes will acquire and release memory using the specified overloads instead of the global operators, in this case the overloads have to follow the same rules as the custom STL allocators discussed earlier. It is also possible to put these overloads into a common base class, making sharing as easy as deriving from this class.

One final remark to be made concerns virtual methods. As, at compile time, the compiler has no way of knowing which method is going to be called in the end, a virtual method called at run-time has to lead to a method instantiated in the module that the corresponding object has originally been created in, thus, code inside virtual methods will always use the same private heap that was used on construction of the object in question.

Curiously enough, this is the reason why it is possible to use STL exceptions in a static CRT DLL environment, although these exceptions are commonly implemented using a standard STL string without any additional allocation care. Due to the fact that the standard exception classes' destructors are virtual, the message string is always freed on the same heap as it has been constructed on, regardless of module boundaries crossed by exception handling.

Final thought: Avoid statically linking against the CRT when you can afford it, as the effort, complexity and implications of supporting it are high. Also, if you do, be prepared to build your own binaries of the libraries that your project depends on, as static CRT binaries have been dropped from the binary packages of many libraries (however, building your own binaries may also be necessary for many other reasons, e.g. when tuning the performance of your project via the _SECURE_SCL define).

Further reading:
[KK's Blog] Dynamically linking with MSVCRT.DLL using Visual C++ 2005 ("History of the CRT" + Loads of most valuable CRT information)
[Stackoveflow] What's the differences between VirtualAlloc and HeapAlloc?
[MSDN] Heap: Pleasures and Pains
[MSDN] Comparing Memory Allocation Methods

« Back