How to download the latest artifact from Artifactory repository?

Question

I need the latest artifact (for example, a snapshot) from a repository in Artifactory. This artifact needs to be copied to a server (Linux) via a script.

What are my options? Something like Wget / SCP? And how do I get the path of the artifact?

I found some solutions which require Artifactory Pro. But I just have Artifactory, not Artifactory Pro.

Is it possible at all to download from Artifactory without the UI and not having the Pro-Version? What is the experience?

I'm on OpenSUSE 12.1 (x86_64) if that matters.

See Artifactory documentation on this exact topic https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-RetrieveLatestArtifact — spuder, Nov 22 '16 at 19:58
Once you have the path of the artifact, you can download it easily with WGet by adding the JFROG header with your user token. https://coding-stream-of-consciousness.com/2019/06/23/download-secured-artifactory-artifact-with-wget-and-token/. (though getting the latest artifact is another issue entirely of course). — John Humphreys - w00te, Jun 23 '19 at 19:13

btiernay · Answer 1 · 2018-07-31T22:05:42.803

71

Something like the following bash script will retrieve the lastest com.company:artifact snapshot from the snapshot repo:

# Artifactory location
server=http://artifactory.company.com/artifactory
repo=snapshot

# Maven artifact location
name=artifact
artifact=com/company/$name
path=$server/$repo/$artifact
version=$(curl -s $path/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
build=$(curl -s $path/$version/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
jar=$name-$build.jar
url=$path/$version/$jar

# Download
echo $url
wget -q -N $url

It feels a bit dirty, yes, but it gets the job done.

edited Jul 31 '18 at 22:05

answered Jul 21 '13 at 14:33

btiernay

7,430
4
39
46

I had to replace the "version=" line with version=`curl -s $path/maven-metadata.xml | grep "" | sed "s/.*\([^.*/\1/"` – Vincent Blouin Jun 14 '15 at 20:11
2

This is not related with the question but it might help someone. Use *-k* with curl if you have problems with certificates. – flapjack Jun 13 '16 at 18:35
1

for authentication I've use: 'version=`curl -u user:pwd -s $path/maven-metadata.xml | grep latest | sed "s/.*\([^.*/\1/"`' – Manuel Schmitzberger Jul 05 '16 at 06:28
3

Using the "latest" value in maven-metadata is not a good idea. It will appear to work right up until it doesn't, because it's not reliability updated after you switch to a new release version. See: http://articles.javatalks.ru/articles/32 – Scott McIntyre Nov 03 '16 at 15:50

score 35 · Accepted Answer · edited Oct 26 '17 at 18:00

Artifactory has a good extensive REST-API and almost anything that can be done in the UI (perhaps even more) can also be done using simple HTTP requests.

The feature that you mention - retrieving the latest artifact, does indeed require the Pro edition; but it can also be achieved with a bit of work on your side and a few basic scripts.

Option 1 - Search:

Perform a GAVC search on a set of group ID and artifact ID coordinates to retrieve all existing versions of that set; then you can use any version string comparison algorithm to determine the latest version.

Option 2 - the Maven way:

Artifactory generates a standard XML metadata that is to be consumed by Maven, because Maven is faced with the same problem - determining the latest version; The metadata lists all available versions of an artifact and is generated for every artifact level folder; with a simple GET request and some XML parsing, you can discover the latest version.

the GAVC search is empty. Im deploying from Jenkins. Its a Free-Sytle-Job and a "Generic-Artifactory Integration" build. Does it need to be a Maven 2/3 Job or a Maven-Artifactory Integration build, to be able to Search by Maven coordinates? (When I dont deploy with Jenkins the search works) — user1338413, Jan 02 '13 at 17:27

score 15 · Answer 3 · edited Jun 13 '18 at 12:29

15

Using shell/unix tools

curl 'http://$artiserver/artifactory/api/storage/$repokey/$path/$version/?lastModified'

The above command responds with a JSON with two elements - "uri" and "lastModified"

Fetching the link in the uri returns another JSON which has the "downloadUri" of the artifact.
Fetch the link in the "downloadUri" and you have the latest artefact.

Using Jenkins Artifactory plugin

(Requires Pro) to resolve and download latest artifact, if Jenkins Artifactory plugin was used to publish to artifactory in another job:

Select Generic Artifactory Integration
Use Resolved Artifacts as ${repokey}:**/${component}*.jar;status=${STATUS}@${PUBLISH_BUILDJOB}#LATEST=>${targetDir}

edited Jun 13 '18 at 12:29

slm

12,534
12
87
106

answered Oct 09 '15 at 03:33

Sateesh Potturu

151
1
3

2

First solution also requires Pro, I am getting ""This REST API is available only in Artifactory Pro (see: http://www.jfrog.com/addons.php)" – hgrey Oct 15 '15 at 16:07
1

I am using the pro version and it works with the build number but when I use the LATEST build number it cant find the artifact. Have you seen this problem before? – CodyK Dec 03 '15 at 21:31
@Sateesh Potturu do you have a link to artifactory documentation for the syntax of the second suggestion? – therealjumbo Jan 02 '18 at 23:02

score 6 · Answer 4 · edited Jan 15 '19 at 13:42

You could also use Artifactory Query Language to get the latest artifact.

The following shell script is just an example. It uses 'items.find()' (which is available in the non-Pro version), e.g. items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}}) that searches for files that have a repository name equal to "my-repo" and match all files that start with "my-file". Then it uses the shell JSON parser ./jq to extract the latest file by sorting by the date field 'updated'. Finally it uses wget to download the artifact.

#!/bin/bash

# Artifactory settings
host="127.0.0.1"
username="downloader"
password="my-artifactory-token"

# Use Artifactory Query Language to get the latest scraper script (https://www.jfrog.com/confluence/display/RTF/Artifactory+Query+Language)
resultAsJson=$(curl -u$username:"$password" -X POST  http://$host/artifactory/api/search/aql -H "content-type: text/plain" -d 'items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}})')

# Use ./jq to pars JSON
latestFile=$(echo $resultAsJson | jq -r '.results | sort_by(.updated) [-1].name')

# Download the latest scraper script
wget -N -P ./libs/ --user $username --password $password http://$host/artifactory/my-repo/$latestFile

Just consider that AQL was introduced in Artifactory V3.5.0 – Pedro Mar 14 '18 at 09:26 — Pedro, Mar 14 '18 at 09:26

score 5 · Answer 5 · answered Nov 22 '16 at 20:09

With recent versions of artifactory, you can query this through the api.

https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-RetrieveLatestArtifact

If you have a maven artifact with 2 snapshots

name => 'com.acme.derp'
version => 0.1.0
repo name => 'foo'
snapshot 1 => derp-0.1.0-20161121.183847-3.jar
snapshot 2 => derp-0.1.0-20161122.00000-0.jar

Then the full paths would be

https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-20161121.183847-3.jar

and

https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-20161122.00000-0.jar

You would fetch the latest like so:

curl https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-SNAPSHOT.jar

score 5 · Answer 6 · answered Aug 02 '17 at 23:00

5

You can use the REST-API's "Item last modified". From the docs, it retuns something like this:

GET /api/storage/libs-release-local/org/acme?lastModified
{
"uri": "http://localhost:8081/artifactory/api/storage/libs-release-local/org/acme/foo/1.0-SNAPSHOT/foo-1.0-SNAPSHOT.pom",
"lastModified": ISO8601
}

Example:

# Figure out the URL of the last item modified in a given folder/repo combination
url=$(curl \
    -H 'X-JFrog-Art-Api: XXXXXXXXXXXXXXXXXXXX' \
    'http://<artifactory-base-url>/api/storage/<repo>/<folder>?lastModified'  | jq -r '.uri')
# Figure out the name of the downloaded file
downloaded_filename=$(echo "${url}" | sed -e 's|[^/]*/||g')
# Download the file
curl -L -O "${url}"

answered Aug 02 '17 at 23:00

djhaskin987

8,513
1
45
81

Just consider that the use of "X-JFrog-Art-Api" for authentication is supported since version 4.4.3. It will not work with older versions. – Pedro Mar 14 '18 at 10:10
3

problem is, lastModified does not necessarily mean latest build – Florian Castellane Aug 17 '18 at 08:31
Did you find any ways to find the latest version available and update the cache? Appreciate if you can help. Thanks @FlorianCastellane – Raviteja Jan 27 '21 at 08:43

score 3 · Answer 7 · edited Jul 10 '15 at 13:57

The role of Artifactory is to provide files for Maven (as well as other build tools such as Ivy, Gradle or sbt). You can just use Maven together with the maven-dependency-plugin to copy the artifacts out. Here's a pom outline to start you off...

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>A group id</groupId>
    <artifactId>An artifact id</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.3</version>
                <executions>
                    <execution>
                        <id>copy</id>
                        <phase>package</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>The group id of your artifact</groupId>
                                    <artifactId>The artifact id</artifactId>
                                    <version>The snapshot version</version>
                                    <type>Whatever the type is, for example, JAR</type>
                                    <outputDirectory>Where you want the file to go</outputDirectory>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Just run mvn install to do the copy.

score 2 · Answer 8 · answered May 26 '14 at 08:38

You can use the wget --user=USER --password=PASSWORD .. command, but before you can do that, you must allow artifactory to force authentication, which can be done by unchecking the "Hide Existence of Unauthorized Resources" box at Security/General tab in artifactory admin panel. Otherwise artifactory sends a 404 page and wget can not authenticate to artifactory.

score 2 · Answer 9 · edited Jan 31 '19 at 13:47

2

For me the easiest way was to read the last versions of the project with a combination of curl, grep, sort and tail.

My format: service-(version: 1.9.23)-(buildnumber)156.tar.gz

versionToDownload=$(curl -u$user:$password 'https://$artifactory/artifactory/$project/' | grep -o 'service-[^"]*.tar.gz' | sort | tail -1)

edited Jan 31 '19 at 13:47

NaN

5,040
4
27
46

answered Jan 31 '19 at 13:41

Onko

21
3

Perfect! Worked for me. – asg May 02 '19 at 12:52

score 1 · Answer 10 · answered Jun 14 '18 at 17:12

1

This may be new:

https://artifactory.example.com/artifactory/repo/com/example/foo/1.0.[RELEASE]/foo-1.0.[RELEASE].tgz

For loading module foo from example.com . Keep the [RELEASE] parts verbatim. This is mentioned in the docs but it's not made abundantly clear that you can actually put [RELEASE] into the URL (as opposed to a substitution pattern for the developer).

answered Jun 14 '18 at 17:12

Jason

2,380
21
21

That doesn't seem to work in my version of Artifactory (5.3.2). – James Marble Jun 28 '18 at 18:16
2

According to [documentation](https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-RetrieveLatestArtifact) this requires Artifactory Pro. – Daniel Feb 19 '20 at 23:55
You can also use `SNAPSHOT` verbatim according to the documentation Daniel links to. But it is only for Pro :( .... so go back and grab one of the two-step script answers that you skipped over hoping there was a way to do it in one step. – Partly Cloudy Oct 23 '20 at 15:51

score 1 · Answer 11 · answered Nov 13 '19 at 12:30

With awk:

     curl  -sS http://the_repo/com/stackoverflow/the_artifact/maven-metadata.xml | grep latest | awk -F'<latest>' '{print $2}' | awk -F'</latest>' '{print $1}'

With sed:

    curl  -sS http://the_repo/com/stackoverflow/the_artifact/maven-metadata.xml | grep latest | sed 's:<latest>::' | sed 's:</latest>::'

score 0 · Answer 12 · answered May 20 '19 at 11:25

If you want to download the latest jar between 2 repositores, you can use this solution. I actually use it within my Jenkins pipeline, it works perfectly. Let's say you have a plugins-release-local and plugins-snapshot-local and you want to download the latest jar between these. Your shell script should look like this

NOTE : I use jfrog cli and it's configured with my Artifactory server.

Use case : Shell script

# your repo, you can change it then or pass an argument to the script
# repo = $1 this get the first arg passed to the script
repo=plugins-snapshot-local
# change this by your artifact path, or pass an argument $2
artifact=kshuttle/ifrs16
path=$repo/$artifact
echo $path
~/jfrog rt download --flat $path/maven-metadata.xml version/
version=$(cat version/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
echo "VERSION $version"
~/jfrog rt download --flat $path/$version/maven-metadata.xml build/
build=$(cat  build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
echo "BUILD $build"
# change this by your app name, or pass an argument $3
name=ifrs16
jar=$name-$build.jar
url=$path/$version/$jar

# Download
echo $url
~/jfrog rt download --flat $url

Use case : Jenkins Pipeline

def getLatestArtifact(repo, pkg, appName, configDir){
    sh """
        ~/jfrog rt download --flat $repo/$pkg/maven-metadata.xml $configDir/version/
        version=\$(cat $configDir/version/maven-metadata.xml | grep latest | sed "s/.*<latest>\\([^<]*\\)<\\/latest>.*/\\1/")
        echo "VERSION \$version"
        ~/jfrog rt download --flat $repo/$pkg/\$version/maven-metadata.xml $configDir/build/
        build=\$(cat  $configDir/build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\\([^<]*\\)<\\/value>.*/\\1/")
        echo "BUILD \$build"
        jar=$appName-\$build.jar
        url=$repo/$pkg/\$version/\$jar

        # Download
        echo \$url
        ~/jfrog rt download --flat \$url
    """
}

def clearDir(dir){
    sh """
        rm -rf $dir/*
    """

}

node('mynode'){
    stage('mysstage'){
        def repos =  ["plugins-snapshot-local","plugins-release-local"]

        for (String repo in repos) {
            getLatestArtifact(repo,"kshuttle/ifrs16","ifrs16","myConfigDir/")
        }
        //optional
        clearDir("myConfigDir/")
    }
}

This helps alot when you want to get the latest package between 1 or more repos. Hope it helps u too! For more Jenkins scripted pipelines info, visit Jenkins docs.

Daniel · Answer 13 · 2020-02-24T10:47:23.773

0

In case you need to download an artifact in a Dockerfile, instead of using wget or curl or the likes you can simply use the 'ADD' directive:

ADD ${ARTIFACT_URL} /opt/app/app.jar

Of course, the tricky part is determining the ARTIFACT_URL, but there's enough about that in all the other answers.

However, Docker best practises strongly discourage using ADD for this purpose and recommend using wget or curl.

edited Feb 24 '20 at 10:47

answered Feb 19 '20 at 13:43

Daniel

151
1
8

Denis Kalinin · Answer 14 · 2016-08-02T15:17:07.473

I use Nexus and this code works for me—can retrive both release and last snaphsot, depending on repository type:

server="http://example.com/nexus/content/repositories"
repo="snapshots"
name="com.exmple.server"
artifact="com/example/$name"
path=$server/$repo/$artifact
mvnMetadata=$(curl -s "$path/maven-metadata.xml")
echo "Metadata: $mvnMetadata"
jar=""
version=$( echo "$mvnMetadata" | xpath -e "//versioning/release/text()" 2> /dev/null)
if [[ $version = *[!\ ]* ]]; then
  jar=$name-$version.jar
else
  version=$(echo "$mvnMetadata" | xpath -e "//versioning/versions/version[last()]/text()")
  snapshotMetadata=$(curl -s "$path/$version/maven-metadata.xml")
  timestamp=$(echo "$snapshotMetadata" | xpath -e "//snapshot/timestamp/text()")
  buildNumber=$(echo "$snapshotMetadata" | xpath -e "//snapshot/buildNumber/text()")
  snapshotVersion=$(echo "$version" | sed 's/\(-SNAPSHOT\)*$//g')
  jar=$name-$snapshotVersion-$timestamp-$buildNumber.jar
fi
jarUrl=$path/$version/$jar
echo $jarUrl
mkdir -p /opt/server/
wget -O /opt/server/server.jar -q -N $jarUrl

@gareth_bowles what the difference between maven-metadata.xml in nexus and maven-metadata.xml in artifactory ? — Lostboy, Nov 02 '18 at 08:47

How to download the latest artifact from Artifactory repository?

14 Answers14

Using shell/unix tools

Using Jenkins Artifactory plugin

Use case : Shell script

Use case : Jenkins Pipeline