0

We are currently encountering a problem when we try to build a PHP project requiring internal Composer packages hosted on our private TFS on-premise repositories on a dedicated build machine through a TFS build pipeline.

In our composer.json file, external as well as internal packages are referenced. So, we have something like so:

{
    "repositories": [
    { 
      "type": "composer", 
      "url": "https://composer.typo3.org/" 
    },
    { 
        "type": "git", 
        "url": "https://servername/tfs/organisation/project/_git/repository" 
    },
    "require": {
        ...
    }
}

Because we are behind a firewall, we have to set proxy information to get external packages. We do it like so:

export http_proxy="http://proxy-url"
export https_proxy="http://proxy-url"

However, we cannot get private packages when we go through our proxy, so we created another step that is responsible for internal packages installation after proxy configuration has been reverted.

So, with our private packages, Composer will simply do...

git clone --mirror 'https://service_account:password@servername/tfs/organisation/project/_git/repository'

Unfortunately, we always get the following error:

fatal: Authentication failed for 'https://service_account:password@servername/tfs/organisation/project/_git/repository'

We tried several things:

The service account used has all permissions on the private repositories concerned and it used for the Build Agent on the machine with the required permissions. The connections between our TFS server and the build machine are open. We can do ping tfs.server or telnet tfs.server 443 without any error. The "Get sources" step also works perfectly well when the build process starts. The command git clone 'https://username:password@servername/tfs/organisation/project/_git/repository' works everywhere else.

Can someone give us a little help on this or just a little hint? It would be much appreciated. Has anyone encountered such a problem or behaviour?

Many thanks in advance.

Best regards,

EDIT 1, following PatrickLu-MSFT's answer

The following post, Azure Pipelines: I am getting fatal: could not read Username for 'https://github.com': terminal prompts disabled, led us to simply tick the "Allow scripts to access the OAuth token" option. So, we tried to do it like so

  • git clone --bare https://service_account:$(System.AccessToken)@servername/tfs/organisation/project/_git/repository
  • git clone --bare https://$(System.AccessToken)@servername/tfs/organisation/project/_git/repository
  • git clone --bare https://$env:SYSTEM_ACCESSTOKEN@servername/tfs/organisation/project/_git/repository

...but it didn't work.

We also tried the following thing:

git config http.extraheader "AUTHORIZATION: bearer $(System.AccessToken)"

or 

git config http.https://servername/tfs/organisation/project/_git/repository.extraheader "AUTHORIZATION: bearer $(System.AccessToken)"

then 

git clone https://servername/tfs/organisation/project/_git/repository

...without success. But if we simply do it from a command prompt or access those URL from the browser, we don't have any trouble.

However, if we do something like so

git config http.extraheader "AUTHORIZATION: bearer $(System.AccessToken)" clone https://servername/tfs/organisation/project/_git/repository

...we get something like "repository.git" with the content of the Git folder only, not the package itself.

EDIT 2

After many well spent hours full of joy, we finally managed to get something like so:

REPO_URL=https://servername/tfs/organisation/project/_git/repository
EXTRAHEADER="AUTHORIZATION: bearer $(System.AccessToken)"
git -c http.extraheader="$EXTRAHEADER" clone $REPO_URL

We have to do it for every private package, but it seems to do the job. But with this solution, to be clean, we must have two composer.json files: one for local development, and another one for the build pipeline which does not contain the private packages.

What it is also interesting, is that the following thing doesn't work:

git config credential.https://servername/tfs/organisation.integrated true
git config credential.helper store
git config http.$REPO_URL.extraheader "$EXTRAHEADER"

git clone https://servername/tfs/organisation/project/_git/repository
# OR
composer require vendor/package-name

It seems that extraheader information is ignored.

Lancelot
  • 563
  • 2
  • 5
  • 20
  • can you try login in as the build agent service and add an entry in the windows credential manager for the "servername" (your tfs server machine) with the proper userid and password. Git for Windows actually uses the windows credential manager for authentication. You might need to run "git config --global credential.helper manager" also to make sure that sticks. – Etienne Mar 23 '20 at 18:26
  • Hi! Thanks for your feedback. I updated the question with a few information. – Lancelot Mar 25 '20 at 14:19

1 Answers1

0

Try to change the ordering of repositories, so that the git repo is listed ahead of typo3, such as below :

{
    "repositories": [
     { 
        "type": "git", 
        "url": "https://domain/tfs/organisation/project/repository" 
    },
    { 
      "type": "composer", 
      "url": "https://composer.typo3.org/" 
    },

    "require": {
        ...
    }
}

For this kind of Authentication failed error, if you are using PAT token to handle the authentication. Since you are using PAT with on-premise TFS server, make sure you already turned off Basic Authentication on server. Otherwise you will get an returned error.

We recommend you keep IIS Basic Authentication turned off at all times when using Azure DevOps Server. Only if necessary should you enable IIS Basic Authentication. When IIS Basic Authentication is enabled on your windows machine, it prevents you from using personal access tokens (PATs) as an authentication mechanism.

Source link

And for clone url in TFS, should be kind like below:

git clone https://PAT@tfsservername:8080/tfs/Collectionname/Projectname/_git/GitReponame

If this not work, since you are behind a proxy. Try to temporarily turn off the proxy when use in your original browser.

This will help to narrow down if the issue is related to proxy side.

PatrickLu-MSFT
  • 44,639
  • 4
  • 24
  • 52
  • Thanks for your answer. I added more information in my question, following your post. – Lancelot Mar 19 '20 at 11:08
  • Hi @Lancelot, `But if we simply do it from a command prompt or access those URL from the browser, we don't have any trouble.` According to your description, seems the issue is not related to authentication any more. You are OK to use authentication to get in the repo and clone files. By any chance, could this related to composer part? Something maybe wrong with how you are cloning git repository via composer. – PatrickLu-MSFT Mar 20 '20 at 02:32
  • I‘m not family with PHP, take a look at if these two links are helpful: https://stackoverflow.com/questions/21512781/clone-git-repository-via-composer and [Using private Azure DevOps git repositories with composer during Build and Release](https://fragmentedthought.com/fragments/using-private-azure-devops-git-repositories-composer-during-build-and-release) – PatrickLu-MSFT Mar 20 '20 at 02:40
  • Thanks for your answer! I will check your links. I was also wondering if that could come from a permission problem during the build process. – Lancelot Mar 20 '20 at 06:34