182

I have two SVN projects in use from another SVN repository using svn:externals.

How can I have the same repository layout structure in Git?

Peter Mortensen
  • 28,342
  • 21
  • 95
  • 123
dsimard
  • 7,180
  • 4
  • 19
  • 24
  • 8
    Anyone have a new answer to this in the last 4 years, or is the world of git the same today? – DougW Apr 19 '13 at 23:37
  • 4
    @DougW Yes, I have a [new answer below](http://stackoverflow.com/a/18088319/6309): `git submodule` can now emulate `svn:external` (since March 2013). – VonC Aug 06 '13 at 18:58
  • You should look into [Git submodules](http://git-scm.com/docs/git-submodule). It should allow almost exactly what you're looking for. – foxxtrot Feb 20 '09 at 21:32
  • For the latest version of Git I'd suggest to read about [Git submodules](http://git-scm.com/book/en/v2/Git-Tools-Submodules) in the official Git documentation. – Bulki S Maslom Jan 07 '15 at 09:54

3 Answers3

136

Git has two approaches similar to, but not exactly equivalent to svn:externals:

  • Subtree merges insert the external project's code into a separate sub-directory within your repo. This has a detailed process to set up and then is very easy for other users, because it is automatically included when the repository is checked out or cloned. This can be a convenient way to include a dependency in your project.
    It is easy to pull changes from the other project, but complicated to submit changes back. And if the other project have to merge from your code, the project histories get merged and the two projects effectively become one.

  • Git submodules (manual) link to a particular commit in another project's repository, much like svn:externals with an -r argument. Submodules are easy to set up, but all users have to manage the submodules, which are not automatically included in checkouts (or clones).
    Although it is easy to submit changes back to the other project, doing so may cause problems if the repo has changed. Therefore it is generally not appropriate to submit changes back to a project that is under active development.

The Godfather
  • 3,138
  • 2
  • 33
  • 49
Paul
  • 15,965
  • 3
  • 31
  • 25
  • 18
    FYI, it is now possible to specify specific revisions with svn:externals now (since 1.5 or 1.6 I believe?) – Nate Parsons Sep 22 '10 at 21:14
  • 9
    FYI, git submodules can be automatically managed and commited. git creates a .gitmodules file that can/should be commited just like the .gitignore file. See [http://git-scm.com/book/en/Git-Tools-Submodules] for more information. – mikijov May 30 '12 at 14:47
  • 6
    @NateParsons It has always been possible to specify exact revision numbers with `svn:externals`. With revision 1.5, the syntax was changed to a more flexible format. What was added was relative URL addressing. – David W. Aug 06 '13 at 19:52
  • @NateParsons but is it possible to omit revisions with git submodules... >_> – Trejkaz Sep 28 '16 at 03:22
  • I think it is not possible to git submodule single files like with svn:externals – user1911091 Mar 27 '20 at 20:54
  • This sounds like something almost, but not entirely unlike, tea. – Jon Trauntvein May 15 '21 at 12:28
40

As I mention in "Git submodule new version update", you can achieve the same SVN external feature with Git 1.8.2 submodules:

git config -f .gitmodules submodule.<path>.branch <branch>

This is enough for a submodule to follow a branch (as in the LATEST commit of a remote branch of a submodule upstream repo). All you need to do is a:

git submodule update --remote

That will update the submodule.

More details are in "git submodule tracking latest".

To convert an existing submodule into one tracking a branch: see all the steps in "Git submodules: Specify a branch/tag".

Peter Mortensen
  • 28,342
  • 21
  • 95
  • 123
VonC
  • 1,042,979
  • 435
  • 3,649
  • 4,283
  • Can you do partial checkout like with `svn:externals`? – nowox Aug 02 '17 at 17:03
  • @nowox Yes, you can have sparse checkout (git 1.7+ https://stackoverflow.com/a/2372044/6309) associated to submodules (https://stackoverflow.com/a/17693008/6309) – VonC Aug 02 '17 at 17:58
  • unfortunately all the sparse checkout related answers never give any example :( I'll try to write a Gist example for this... – nowox Aug 03 '17 at 06:17
  • 1
    There is still an issue with this. You still have to get the whole history of a repository where you only need one small part. In my case is 100kB over 2GB. I can of course use `--depth` but it doesn't really address the problem. – nowox Aug 03 '17 at 06:19
  • 1
    @nowox It is best to ask a new question explaining exactly what your use case is: I have no idea if your 2GB repo is a submodule, or a main repo with submodule, and what exactly you need to extract from it. – VonC Aug 03 '17 at 06:24
  • https://stackoverflow.com/questions/45476709/how-to-deal-with-partial-svnexternals-when-migrating-to-git – nowox Aug 03 '17 at 06:42
5

I'm the author of gil (git links) tool

I have an alternative solution for the problem - gil (git links) tool

It allows to describe and manage complex git repositories dependencies.

Also it provides a solution to the git recursive submodules dependency problem.

Consider you have the following project dependencies: sample git repository dependency graph

Then you can define .gitlinks file with repositories relation description:

# Projects
CppBenchmark CppBenchmark https://github.com/chronoxor/CppBenchmark.git master
CppCommon CppCommon https://github.com/chronoxor/CppCommon.git master
CppLogging CppLogging https://github.com/chronoxor/CppLogging.git master

# Modules
Catch2 modules/Catch2 https://github.com/catchorg/Catch2.git master
cpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master
fmt modules/fmt https://github.com/fmtlib/fmt.git master
HdrHistogram modules/HdrHistogram https://github.com/HdrHistogram/HdrHistogram_c.git master
zlib modules/zlib https://github.com/madler/zlib.git master

# Scripts
build scripts/build https://github.com/chronoxor/CppBuildScripts.git master
cmake scripts/cmake https://github.com/chronoxor/CppCMakeScripts.git master

Each line describe git link in the following format:

  1. Unique name of the repository
  2. Relative path of the repository (started from the path of .gitlinks file)
  3. Git repository which will be used in git clone command Repository branch to checkout
  4. Empty line or line started with # are not parsed (treated as comment).

Finally you have to update your root sample repository:

# Clone and link all git links dependencies from .gitlinks file
gil clone
gil link

# The same result with a single command
gil update

As the result you'll clone all required projects and link them to each other in a proper way.

If you want to commit all changes in some repository with all changes in child linked repositories you can do it with a single command:

gil commit -a -m "Some big update"

Pull, push commands works in a similar way:

gil pull
gil push

Gil (git links) tool supports the following commands:

usage: gil command arguments
Supported commands:
    help - show this help
    context - command will show the current git link context of the current directory
    clone - clone all repositories that are missed in the current context
    link - link all repositories that are missed in the current context
    update - clone and link in a single operation
    pull - pull all repositories in the current directory
    push - push all repositories in the current directory
    commit - commit all repositories in the current directory

More about git recursive submodules dependency problem.

chronoxor
  • 2,053
  • 2
  • 16
  • 28