150

I have a small piece of code that depends on many static libraries (a_1-a_n). I'd like to package up that code in a static library and make it available to other people.

My static library, lets call it X, compiles fine.

I've created a simple sample program that uses a function from X, but when I try to link it to X, I get many errors about missing symbols from libraries a_1 - a_n.

Is there a way that I can create a new static library, Y that contains X and all the functionality needed by X (selected bits from a_1 - a_n), so that I can distribute just Y for people to link their programs to?


UPDATE:

I've looked at just dumping everything with ar and making one mega-lib, however, that ends up including a lot of symbols that are not needed (all the .o files are about 700 MB, however, a statically linked executable is 7 MB). Is there a nice way to include only what is actually needed?


This looks closely related to How to combine several C/C++ libraries into one?.

UpAndAdam
  • 4,155
  • 2
  • 25
  • 41
Jason Sundram
  • 10,998
  • 19
  • 67
  • 84

6 Answers6

83

Static libraries do not link with other static libraries. The only way to do this is to use your librarian/archiver tool (for example ar on Linux) to create a single new static library by concatenating the multiple libraries.

Edit: In response to your update, the only way I know to select only the symbols that are required is to manually create the library from the subset of the .o files that contain them. This is difficult, time consuming and error prone. I'm not aware of any tools to help do this (not to say they don't exist), but it would make quite an interesting project to produce one.

  • Hi Neil, I've updated the question -- do you know any way of including only the .o files that are necessary? – Jason Sundram Jan 28 '10 at 21:16
  • Update -- If you find yourself wanting to do this, take a step back, and pursue something else (unless, as John Knoeller points out, you are using Visual Studio). I sunk a lot of time into this approach and didn't get anything useful. – Jason Sundram Apr 12 '11 at 17:12
  • The GNU ld tool provides an option `-r` with that the output can be used as an input for ld. If you link once you should get a relocatable the can be surrogate your libraries. (did tried it thogh). – harper Dec 28 '14 at 11:53
50

If you are using Visual Studio then yes, you can do this.

The library builder tool that comes with Visual Studio allows you to join libraries together on the command line. I don't know of any way to do this in the visual editor though.

lib.exe /OUT:compositelib.lib  lib1.lib lib2.lib
John Knoeller
  • 31,289
  • 4
  • 56
  • 90
  • 6
    In VS2008, in the project properties of compositelib under Librarian/General, if you check [x] Link Library Dependencies, it will do this for you if lib1 and lib2 are dependencies of compositelib. It seems slightly buggy, I would set the checkbox separately in each build configuration, not once in 'All Configurations'. – Spike0xff Feb 13 '13 at 18:28
  • 2
    VS2015 IDE - don't you use "Additional Dependencies" under Librarian/General to get additional libraries linked directly into the library your project is building? – davidbak May 06 '16 at 15:51
  • 1
    @davidbak I am trying to figure this out in the last couple of days and apparently this option is obsolete and does not do anything? – Montaldo May 09 '16 at 12:03
21

On Linux or MingW, with GNU toolchain:

ar -M <<EOM
    CREATE libab.a
    ADDLIB liba.a
    ADDLIB libb.a
    SAVE
    END
EOM
ranlib libab.a

Of if you do not delete liba.a and libb.a, you can make a "thin archive":

ar crsT libab.a liba.a libb.a

On Windows, with MSVC toolchain:

lib.exe /OUT:libab.lib liba.lib libb.lib
Star Brilliant
  • 2,418
  • 19
  • 28
  • Nice to know, but doesn't really solve the OP's problem, since you're including everything from libb.a into the joint lib, which can become very large if you need only a few modules from libb. – Elmar Zander Jan 15 '15 at 23:31
  • 1
    @ElmarZander But you can use `ar crsT`, which does something like "symlinking" instead of copying, then you need only to ship one copy of data. – Star Brilliant Jan 16 '15 at 14:23
  • Thin archives are notably used on the Linux kernel v4.19: https://unix.stackexchange.com/questions/5518/what-is-the-difference-between-the-following-kernel-makefile-terms-vmlinux-vml/482978#482978 – Ciro Santilli新疆棉花TRUMP BAN BAD Nov 21 '18 at 10:57
11

A static library is just an archive of .o object files. Extract them with ar (assuming Unix) and pack them back into one big library.

Nikolai Fetissov
  • 77,392
  • 11
  • 105
  • 164
7

Alternatively to Link Library Dependencies in project properties there is another way to link libraries in Visual Studio.

  1. Open the project of the library (X) that you want to be combined with other libraries.
  2. Add the other libraries you want combined with X (Right Click, Add Existing Item...).
  3. Go to their properties and make sure Item Type is Library

This will include the other libraries in X as if you ran

lib /out:X.lib X.lib other1.lib other2.lib
evpo
  • 2,188
  • 13
  • 18
  • Hi, I'm using Intel IPP and I built my own functions which I want all to be wrapped into one (Static) lib. Yet when I create the lib and then send the project to other computer which I want to be able to compile the project only using the lib I created I get an error saying an .h file of the Intel IPP Library is required. Any idea? – Royi Feb 23 '15 at 17:53
  • lib file is usually not enough. If you want to use it, you will need to include header files too. But it's off topic here as this thread talks about linking and your problem is at the compilation stage. – evpo Feb 27 '15 at 05:22
  • ok. To help you I will need more information. Look at the build output or log and see what's complaining on the missing .h file. I think it is cl.exe. If that's the case, it will give you the name of the compiled .cpp/.cc/.c file that uses the header. What's the name of that .cpp file and which project it belongs to? – evpo Mar 04 '15 at 00:37
  • I am so confused. Before I read this, it didn't seem to have ever worked (this is how my projects are already configured). But now that I've read this and reset my projects, it is apparently working. Go figure! :( – Mordachai Mar 06 '18 at 18:33
  • 1
    @Mordachai this haiku below describes your experience perfectly: Yesterday it worked Today it is not working Windows is like that – evpo Mar 06 '18 at 23:35
7

Note before you read the rest: The shell script shown here is certainly not safe to use and well tested. Use at your own risk!

I wrote a bash script to accomplish that task. Suppose your library is lib1 and the one you need to include some symbols from is lib2. The script now runs in a loop, where it first checks which undefined symbols from lib1 can be found in lib2. It then extracts the corresponding object files from lib2 with ar, renames them a bit, and puts them into lib1. Now there may be more missing symbols, because the stuff you included from lib2 needs other stuff from lib2, which we haven't included yet, so the loop needs to run again. If after some passes of the loop there are no changes anymore, i.e. no object files from lib2 added to lib1, the loop can stop.

Note, that the included symbols are still reported as undefined by nm, so I'm keeping track of the object files, that were added to lib1, themselves, in order to determine whether the loop can be stopped.

#! /bin/bash

lib1="$1"
lib2="$2"

if [ ! -e $lib1.backup ]; then
    echo backing up
    cp $lib1 $lib1.backup
fi

remove_later=""

new_tmp_file() {
    file=$(mktemp)
    remove_later="$remove_later $file"
    eval $1=$file
}
remove_tmp_files() {
    rm $remove_later
}
trap remove_tmp_files EXIT

find_symbols() {
    nm $1 $2 | cut -c20- | sort | uniq 
}

new_tmp_file lib2symbols
new_tmp_file currsymbols

nm $lib2 -s --defined-only > $lib2symbols

prefix="xyz_import_"
pass=0
while true; do
    ((pass++))
    echo "Starting pass #$pass"
    curr=$lib1
    find_symbols $curr "--undefined-only" > $currsymbols
    changed=0
    for sym in $(cat $currsymbols); do
        for obj in $(egrep "^$sym in .*\.o" $lib2symbols | cut -d" " -f3); do
            echo "  Found $sym in $obj."
            if [ -e "$prefix$obj" ]; then continue; fi
            echo "    -> Adding $obj to $lib1"
            ar x $lib2 $obj
            mv $obj "$prefix$obj"
            ar -r -s $lib1 "$prefix$obj"
            remove_later="$remove_later $prefix$obj"
            ((changed=changed+1))
        done
    done
    echo "Found $changed changes in pass #$pass"

    if [[ $changed == 0 ]]; then break; fi
done

I named that script libcomp, so you can call it then e.g. with

./libcomp libmylib.a libwhatever.a

where libwhatever is where you want to include symbols from. However, I think it's safest to copy everything into a separate directory first. I wouldn't trust my script so much (however, it worked for me; I could include libgsl.a into my numerics library with that and leave out that -lgsl compiler switch).

Elmar Zander
  • 1,134
  • 15
  • 23
  • This is excellent. For a big project (>500k symbols in >40k objects in the original library, of which I needed ~1000 symbols), this took close to an hour, whereas gcc can do essentially the same thing with dynamic linking in a few seconds. Is there some fundamental reason why this isn't easier to do and exposed by compiler tools? – ZachB Aug 04 '20 at 22:35