6

Background I have a project that uses other smaller projects. These projects themselves are made of other projects. A lot of this is legacy or has other managerial-fiat reasons for being arranged as is, so rolling everything into a single project is not an option. Some libraries are pre-compiled on remote shares.

I have 2 main subprojects that are giving me a headache:

  • Project Foo is an executable and library that links several static subprojects (foo_subproject_1, foo_subproject_n). These subprojects are further linked against static libraries in remote locations (some_lib, some_other_lib). Project Foo's executable compiles, links, and runs correctly

  • Project Bar is an executable that links several other projects, including libFoo. Linking fails with "undefined reference to " foo_subproject functions

As far as I can tell, the two projects are arranged similarly with their linkage instructions. Looking on SO, I discover that linking static libraries against static libraries shouldn't work, but then I'm confused as to how Project Foo is compiled successfully.

gcc and g++ 4.9.2 are the compilers (extern "C" problems of having some parts in C and some in C++ have already been checked for)

Question

I have misunderstood something about either or both how CMake add_subdirectory works, or about how the linker works. Can someone please explain how Project Foo works successfully, and Project Bar (doesn't) works as expected?

Update I looked closer at foo_lib.a and foo_runtime.

I should have determined that something was off to start with, because foo_runtime is nearly 100MB in size, and foo_lib is only 10KB.

nm reveals that foo_lib.a references a few dozen symbols, most of which are undefined. foo_runtime meanwhile references everything.

Equally confusing is that foo_subproject_1.a is similarly mostly undefined. Again, this is what I expect to see; but I don't understand how foo_runtime can be built from this?

I'm still unclear as to why some_library -> subproject -> foo_runtime is successful, but some_library -> subproject -> foo_lib -> bar isn't. At this stage of my investigations, I am expecting both commands to fail.

Project Foo is arranged (using CMake) thusly:

cmake_minimum_required(VERSION 2.6)
project(foo)

set(FOO_SRCS
    # source and headers for main foo project
)

# Project Foo libraries are subdirectories within this project
add_subdirectory(subs/foo_subproject_1)
add_subdirectory(subs/foo_subproject_2)

# Runtime executable
add_executable(foo_runtime main.c ${FOO_SRCS})
target_link_libraries(foo_runtime foo_subproject_1 foo_subproject_2)

# Library version (static library)
add_library(foo_lib STATIC ${FOO_SRCS})
target_link_libraries(foo_lib foo_subproject_1 foo_subproject_2)

Project Foo's subdirectories loosely have the following architecture:

cmake_minimum_required(VERSION 2.6)
project(foo_subproject_<n>)

set(FOO_SUBPROJECT_<N>_SRCS
    # source and headers for subproject
)

# foo_subproject's remote libraries are all static
add_library(some_lib STATIC IMPORTED)
set_target_properties(some_lib PROPERTIES IMPORTED_LOCATION /path/to/libsome_lib.a)

add_library(some_other_lib STATIC IMPORTED)
set_target_properties(some_other_lib PROPERTIES IMPORTED_LOCATION /path/to/libsome_other_lib.a)

include_directories(/paths/to/libs/include/)

# Static library for foo_subproject_N, links against static libs above
add_library(foo_subproject_<N> STATIC ${FOO_SUBPROJECT_<N>_SRCS})
target_link_libraries(foo_subproject_<N> some_library some_other_library)

Project Bar is arranged thusly:

cmake_minimum_required(VERSION 2.6)
project(bar)

set(BAR_SRCS
    # source and headers for main bar project
)

# Project Bar libraries are remote from Bar's perspective
add_library(foo_lib STATIC IMPORTED)
set_target_properties(foo_lib PROPERTIES IMPORTED_LOCATION /path/to/foo/libfoo_lib.a)

include_directories(/path/to/foo/include/)

# Runtime executable
add_executable(bar main.c ${BAR_SRCS} foo_lib)

Project Bar fails to link (compiles ok) with multiple errors of the form:

bar_frobulator.cpp:123: undefined reference to 'foo_subproject_1_init_frobulation'

where foo_subproject_1_init_frobulation lives in foo_subproject_1

KidneyChris
  • 777
  • 4
  • 12
  • Order of libraries can be significant. If library A requires items from library B then library B should be placed after library A. – john Aug 27 '18 at 11:45
  • @john I don't think that's the issue here. Foo's subprojects have their libraries in the correct order, but the subprojects aren't mutually dependent, so it doesn't matter for Foo. Bar's libraries are also independent (only foo_lib was shown here for brevity, but none of its libraries are interconnected) – KidneyChris Aug 27 '18 at 11:49
  • Isn't the answer that you need to include foo subproject libraries explicitly? If I'm reading the above correctly then you are just linking with foo project but not the sub projects. – john Aug 27 '18 at 12:10
  • @john Perhaps (but if true, the Bar stage is going to have a *lot* of things to link eventually, so I'd rather avoid that situation if I can). If true, why does the Foo Runtime work? It doesn't link against some_library (its subprojects do), but chugs along happily calling those functions. Why does this work for Foo, but not Bar? – KidneyChris Aug 27 '18 at 12:47
  • In `Foo` project, you use `add_executable` in a confusing manner: by listing `foo_subproject_1` in `add_executable` invocation, you (probably) expect *linking*, but this form implies only **file dependency**, not a linking. For link an executable, use `target_link_libraries(foo_runtime foo_subproject_1)`. The similar issue is with `add_library` invocation. In the current form, I see no relation between `Foo` and `Bar` projects, which we could compare: these are completely different projects, which links different libraries. – Tsyvarev Aug 27 '18 at 12:49
  • @Tsyvarev Whoops, that was me fat-fingering the CMake when writing the question; it does use target_link_libraries. Give me a second to correct the mistake... – KidneyChris Aug 27 '18 at 12:51
  • A static library cannot be linked against another static library indeed. A static library only serves as an input to a linker, never as an output, so it makes no sense to even speak about that. – n. 'pronouns' m. Aug 27 '18 at 13:41

2 Answers2

5

Can someone please explain how Project Foo works successfully, and Project Bar (doesn't) works as expected?

In short: Creating STATIC library doesn't involve linking step!

Details

In the Foo project you have an executable foo_runtime, which "works" because it is linked with proper libraries (e.g. with library foo_subproject_1 which defines foo_subproject_1_init_frobulation symbol).

An executable bar from Bar project doesn't perform that linking, so it fails. The line

target_link_libraries(bar foo_lib)

links with foo_lib, but this library doesn't defines the needed symbol foo_subproject_1_init_frobulation.

Note, that the line

target_link_libraries(foo_lib foo_subproject_1 foo_subproject_2)

in the Foo project doesn't perform actual linking: in general, building a static library doesn't involve linking step.

Given line just propagates include directories (and other compile-features) from foo_subproject_* libraries to the foo_lib one.

How to make it work

Because static library foo_lib doesn't track its dependency, you need to link bar with a library, which knows that. E.g., make foo_lib shared, or combine foo_subproject_* libraries into archive library, as suggested by the referenced question How to combine several C/C++ libraries into one?.

Alternatively, you may build Foo subproject within Bar one and, instead of creation of IMPORTED foo_lib target, use "normal" foo_lib target, created within Foo project. In that case, line

target_link_libraries(bar foo_lib)

would mean for CMake to (actually) link bar with foo_subproject_* libraries, because those libraries are "linked" (in CMake sense) into foo_lib. Again, the last "linking" has a meaning only for CMake: the file foo_lib.a doesn't aware about needing of foo_subproject_* libraries.

Community
  • 1
  • 1
Tsyvarev
  • 45,732
  • 15
  • 64
  • 98
  • So what's going on with foo_runtime? If `target_link_libraries(foo_subproject_1 some_library some_other_library)` doesn't do linking, then why does `target_link_libraries(foo_runtime foo_subproject_1 foo_subproject_2)` work? Or is the clue in the line `propegates include directories (and other compile-features)`? – KidneyChris Aug 27 '18 at 13:37
  • Bulding static libraries (foo_subproject_1 ) doesn't involve linking step, building executable (foo_runtime) involves linking step. – KamilCuk Aug 27 '18 at 13:40
  • @KidneyShris: `foo_runtime` is not a *static library*, it is an *executable*, so it links normally. In your question post you link it directly with sub-libraries, which define all needed symbols. But would you link it with `foo_lib`, then CMake propagation will take a place, as described at the end of my answer. – Tsyvarev Aug 27 '18 at 13:40
  • Aha! I think I understand. Looked up the CMake documentation - "Library dependencies are transitive by default with this signature. When this target is linked into another target then the libraries linked to this target will appear on the link line for the other target too. " - does this mean that the inclusion of `target_link_libraries` in my subproject merely lets the main project see that some_library and some_other_library are needed. foo_runtime does linking, and gets the dependencies. foo_lib doesn't do linking, so is mostly left undefined? – KidneyChris Aug 27 '18 at 13:42
  • 1
    Yes, "transitive library dependencies" is a correct place in CMake docs. Actually, `foo_lib` contains these dependencies.. but only as long as **CMake target** in the Foo project. The *file* `foo_lib.a`, as a *static library*, doesn't contain dependencies. – Tsyvarev Aug 27 '18 at 13:52
0

Tsyvarev's answer described well-enough what I was actually doing (rather than what I thought I was doing) to get me to look up the right things to answer the root question that I had ("Why does foo_runtime work but bar_runtime doesn't, when both link against static libraries linked against static libraries?")

From the CMake documentation for target_link_libraries:

Library dependencies are transitive by default with this signature. When this target is linked into another target then the libraries linked to this target will appear on the link line for the other target too.

target_link_libraries does not cause linking in the static libraries foo_subproject_1 and foo_subproject_2 (static libraries don't invoke the linker). What is does is makes the list of required libraries available for anything that tries to link with foo_subproject_1 or foo_subproject_2

So, effectively, my foo_runtime and foo_lib target_link_libraries command is, as far as CMake is concerned:

target_link_libraries(foo_runtime foo_subproject_1 some_library some_other_library foo_subproject_2 some_library some_other_library)
target_link_libraries(foo_lib foo_subproject_1 some_library some_other_library foo_subproject_2 some_library some_other_library)

foo_lib, being static, doesn't invoke the linker. foo_runtime, being an executable, does.

bar is a totally different project, so doesn't get to take advantage of the transitive dependencies (and linking fails, because I'm missing all the symbols from the lower-level libraries).

This behaviour of target_link_libraries wasn't expected, as such the behaviour of the project as a whole was somewhat misleading (since it looked like I was bundling a bunch of libraries at the subproject level, and then bundling those at the upper level. In reality, I had just told the upper level what all the libraries it needed were).

To summerise:

Q "Why does foo_runtime work but bar_runtime doesn't, when both link against static libraries linked against static libraries?"

A "foo_runtime is not linking against static libraries linked against static libraries. foo_runtime is linked against more static libraries than you originally thought it was linking against.

bar_runtime doesn't work because your foo_lib is basically empty"

Community
  • 1
  • 1
KidneyChris
  • 777
  • 4
  • 12