Vladimir Prus


vladimirprus.com

Monday, June 01, 2009

Linking 101

Recently, I see more and more people having trouble with link-time errors—as if such an error is the worst kind of luck and cannot be fixed by mere mortals. There are many possible reasons, including Java as default language in universities, and alarming spread of header-only-philia, but that's for another post. Here, I want to give a simple diagnostic procedure for link-time errors.
Let's lay some groundwork first. If your job is programming in C++, you need to know what the -I and -L options do, and how they are different. Also, given a full path to a library file (with .a or .so or .lib extension), you should be able to link to that file—in two different ways. If you don't know any of the above already, all hope is lost—you might want to consider other occupations. Otherwise, let's look at the diagnosis steps for most common error—'undefined symbol'.

First, understand where the missing symbol is supposedly defined. Educated guess is usually fine. For example, a symbol named boost::system::foobar is most likely contained in the Boost.System library (and it's surprising how many folks fail to guess so). Then, find how you are supposed to link to that logical component, using documentation for the component or the corresponding Linux package. For example, you might decide to add -lboost_filesystem to the linker command line.

Second, make sure that used physical library file is the right one, and that the linker is not picking a different version of the library from a directory you don't expect. On Linux, you can use the -t flag for the GNU linker (or use -Wl,-t on gcc command line). This will print full paths for every library used, including those specified with the -lfoo syntax. For static linking, this will also tell which object files from the static libraries were used. If you get error when running the application, you one can use the LD_DEBUG environment variable. If you set that variable to help prior to running your program, you'll get a list of possible values. The most handy value in our case is files.
On Windows, the /VERBOSE:LIB option to the Visual Studio linker will produce comparable diagnostics.

Third, if you seem to link to the right library, there are three further possibilities. First, maybe the library actually should not include the symbol. This can happen if you use wrong headers during the compilation, and can be debugged by passing the -save-temps option to gcc and checking the generated .ii file. Second, the symbol might be almost there—but slightly different—either using different calling convention (on Windows), or wchar_t mode (also on Windows) or a somewhat different types of parameters, or different namespace. In that case, you'll have to make sure the compilation options of the application match library's requirements. Finally, it could be that the library actually lacks the symbol due to library bug, and you have to complain to maintainer. To distinguish those cases, you need to manually examine the list of library symbols. With gcc, the 'nm' command will do for static libraries, while 'readelf' can be used on shared libraries (Unix only). On Windows, dumpbin.exe /symbols /out:symbols.txt somelib.lib can be used.

That's it for the common case. Below, I list some relatively common specific problems. The list does not claim to be complete, so if you know some other cases, drop me a line.

Static linking. For static linking, the order of libraries on the command line matters, so if you don't see the linking grabbing the object file with your symbol, you might want to either reorder the libraries or use the --start-group option. See ld documentation for details and note that the performance cost of the --start-group option might not be a concern these days.

References to vtable. The GNU C++ compiler sometimes reports unresolved reference to 'vtable for SomeClass'. This generally is a pure way to say that the first non-inline method of SomeClass is not defined. See GCC FAQ

Windows DLLs. On Windows, if an application wants to use a function in DLL, then both DLL and the application should record this intention, using __declspec(dllexport) and __declspec(dllimport) pair. If either party does not do so, linker complains. With mingw, a typical error is undefined reference to `_imp___WHATEVER'. It means that the library is static, whereas the applications wants to use shared library.

Windows import libraries. On Windows, it's not possible to directly link to a DLL. Instead, an import library is created and used—typically by passing /IMPLIB option to the linker. If the linker does not report any errors, but does not produce import library either, it's a sure sign that you have not exported any function from the DLL, and have to check the logic that adds __declspec(dllexport)

64-bit compilation. When building 64-bit applications with GCC, you can get an error that say something about "relocation R_X86_64_32", and suggesting the -fPIC option. The issue here is that 64-bit applications should include only code compiled with -fPIC, and if you link against any static libraries, those libraries should also be compiled with -fPIC. On Windows with Visual Studio, if you try to use 32-bit libraries when building 64-bit application, you won't see any warnings, just undefined references. If you look at symbols at the library, and see exactly the symbols that are reported as undefined, 32/64 mismatch is the most likely reason

9 comments:

Tim said...

"I don't know the best way on Windows, suggestions welcome."

I've always found Dependency Walker to be useful.

Nice post btw, although it does serve to highlight how unnecessarily complicated linking is (and hence why people hate link errors). For example why on earth does the order of static libraries matter? How is that possibly sane?

It's a shame that ELF still hasn't been fixed: http://trac.autopackage.org/wiki/LinuxProblems

xhantt said...

Depends has a really nice GUI. In the pass I've used dumpbin not as elegant but good if you like the cmd commandline.

ServAdmin said...
This comment has been removed by a blog administrator.
Pedro Alves said...

> "Windows import libraries. On Windows, it's not possible to directly link to a DLL."

Using GNU ld from binutils, the standard linker bundled with MinGW and Cygwin, it is possible. See:

http://sourceware.org/binutils/docs-2.19/ld/WIN32.html#index-direct-linking-to-a-dll-622

Moderator said...
This comment has been removed by a blog administrator.
nomad said...

it's really i needed helpful text.
thanks!

Anonymous said...

Awesome thanks!. The -Wl,-t helped me figure out that the wrong lib was used, so just had to switch the -L paths around and everything linked perfectly :)

Oliver said...

This makes sense, but what do I do when there's a complex make file that's generating this error?

I'm installing tpie and it's not clear at all how I'm supposed to configure things to recognise boost.

Vladimir Prus said...

Oliver,

if you have a complex makefile, you still have a command that fails? You can copy-paste it, and run by hand, and then add additional logging parameters, or example files used in the build, etc.