Docker follow symlink outside context

Question

Yet another Docker symlink question. I have a bunch of files that I want to copy over to all my Docker builds. My dir structure is:

parent_dir
    - common_files
        - file.txt
    - dir1
        - Dockerfile  
        - symlink -> ../common_files

In above example, I want file.txt to be copied over when I docker build inside dir1. But I don't want to maintain multiple copies of file.txt. Per this link, as of docker version 0.10, docker build must

Follow symlinks inside container's root for ADD build instructions.

But I get no such file or directory when I build with either of these lines in my Dockerfile:

ADD symlink /path/dirname or ADD symlink/file.txt /path/file.txt

mount option will NOT solve it for me (cross platform...). I tried tar -czh . | docker build -t without success.

Is there a way to make Docker follow the symlink and copy the common_files/file.txt into the built container?

score 75 · Accepted Answer · answered Aug 07 '15 at 19:25

75

That is not possible and will not be implemented. Please have a look at the discussion on github issue #1676:

We do not allow this because it's not repeatable. A symlink on your machine is the not the same as my machine and the same Dockerfile would produce two different results. Also having symlinks to /etc/paasswd would cause issues because it would link the host files and not your local files.

answered Aug 07 '15 at 19:25

0x7d7b

41,374
7
39
55

Thank you. Yes I noticed that link before but I thought that was for a much older version of docker (0.6.1). 0.10's change log sort of mentions this is possible https://github.com/docker/docker/blob/master/CHANGELOG.md#0100-2014-04-08 – Ravi Aug 07 '15 at 21:27
Also if "parent_dir" is checked out in any computer and if symlink has relative path to "common_files", it will be repeatable. – Ravi Aug 07 '15 at 21:27
2

Your quote `Follow symlinks inside container's root for ADD build instructions.` means that *inside the container* symlinks are followed. Not in the build context directory. In `ADD file.txt /dir/file.txt` the directory `dir` could be a symlink. The arguments I quoted in my answer are still valid and symlinks are still not followed in the latest version. You might run into problems (regarding repeatability) when you store symlinks in revision control systems like *git*. Therefore please refer to [this question](http://stackoverflow.com/questions/86402/how-can-i-get-git-to-follow-symlinks). – 0x7d7b Aug 07 '15 at 21:37
1

I see your point regarding symlinks in git. But symlinks don't have to go into git though. A simple setup script can prepare the local env creating symlinks. To me, the cost of keeping 'n' copies of a shared file appears too high from maintenance perspective. Maybe I'll have to serve it out of apache. Thank you. – Ravi Aug 10 '15 at 14:28
23

what a shame, while I see the point I don't follow the logic and it bites me. Git handles symblinks just perfectly, and I also expect builds to work across all machines and environments where the source repo is checked out..?! – Gregor May 29 '17 at 08:31

score 14 · Answer 2 · answered Jul 15 '20 at 13:09

14

If anyone still has this issue I found a very nice solution on superuser.com:

https://superuser.com/questions/842642/how-to-make-a-symlinked-folder-appear-as-a-normal-folder

It basically suggests using tar to dereference the symlinks and feed the result into docker build:

$ tar -czh . | docker build -

answered Jul 15 '20 at 13:09

jdabrowski

1,345
10
20

I get 2 errors: `tar: Failed to clean up compressor`, `Error response from daemon: the Dockerfile (Dockerfile) cannot be empty` – Ivan Rubinson Dec 25 '20 at 11:52
@IvanRubinson seems like a problem with the tar command, not my solution, maybe lack of privileges? – jdabrowski Feb 19 '21 at 21:27

score 9 · Answer 3 · answered Jun 24 '18 at 20:21

One possibility is to run the build in the parent directory, with:

$ docker build [tags...] -f dir1/Dockerfile .

(Or equivalently, in child directory,)

$ docker build  [tags...] -f Dockerfile ..

The Dockerfile will have to be configured to do copy/add with appropriate paths. Depending on your setup, you might want a .dockerignore in the parent to leave out things you don't want to be put into the context.

score 3 · Answer 4 · answered Jun 07 '19 at 10:44

instead of using simlinks it is possible to solve problem administratively by just moving files from sites_available to sites_enabled instead of copying or making simlinks

so your site config will be in one copy only in site_available folder if it stopped or something or in sites_enabled if it should be used

score 2 · Answer 5 · answered Jun 14 '20 at 01:18

2

I know that it breaks portability of docker build, but you can use hard links instead of symbolic:

ln /some/file ./hardlink

answered Jun 14 '20 at 01:18

Eugene

1,859
14
24

8

To be clear, this works for files, not directories. – GDorn Jun 30 '20 at 16:08
Hard link works! Thanks for the idea! However, if one have some files that are hardlink and some are not, be careful when updating the files. Updating a hardlink file will change file in other directories! And hardlink files won;t show up in different colors when doing a ls like sym link too. So use it with care! – HAltos Apr 02 '21 at 08:00

score 1 · Answer 6 · answered Jan 28 '21 at 11:43

I just had to solve this issue in the same context. My solution is to use hierarchical Docker builds. In other words:

parent_dir
  - common_files
    - Dockerfile
    - file.txt

- dir1
    - Dockerfile (FROM common_files:latest)

The disadvantage is that you have to remember to build common_files before dir1. The advantage is that if you have a number of dependant images then they are all a bit smaller due to using a common layer.

Docker follow symlink outside context

6 Answers6

Linked

Related