How to download artifacts from CircleCI via Powershell command Invoke-RESTMethod

Question

I'm trying to get artifacts from CircleCI in Powershell and getting back an unfamiliar data format?

Powershell likes to auto-convert your API’s JSON response to a PSCustomObject. This normally would be what I want.

Here is an example of my attempts at getting clean data.

Add the necessary .NET assembly
Add-Type -AssemblyName System.Net.Http

Create the HttpClient object
$client = New-Object -TypeName System.Net.Http.Httpclient

Get the web content.
$task = $client.GetByteArrayAsync(“https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN”)

Wait for the async call to finish
$task.wait();

results:

    (({
    :path “src./file1.txt”,
    :pretty-path “src/file1.txt”,
    :node-index 0,
    :url “https://15-198716507-gh.circle-artifacts.com/0/src/file1.txt”
    } {
    :path “src/file2.txt”,
    :pretty-path “src/file2.txt”,
    :node-index 0,
    :url “https://15-198716507-gh.circle-artifacts.com/0/src/file2.txt”
    }…continued

As you can see this is not JSON or YAML. Let’s try the built-in PowerShell tools like Invoke-RestMethod.

Invoke-RESTMethod -uri https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN -Method GET

**Output:**

    ({
    :path “src/file1.txt”,
    :pretty-path “src/file1.txt”,
    :node-index 0,
    :url “https://15-198716507-gh.circle-artifacts.com/0/src/file1.txt”
    } {
    :path “src/file2.txt”,
    :pretty-path “src/file2.txt”,
    :node-index 0,
    :url “https://15-198716507-gh.circle-artifacts.com/0/src/file2.txt”
    }…continued

Dang same output. I know from the Invoke-RestMethod documentation that PS sees JSON and auto converts it to a PS object. Maybe it's converting a data type I'm not familiar with? I found it odd that PowerShell was getting EDN type when every other attempt outside PowerShell was JSON.

Maybe they should have the API updated to reply to PS request with JSON by default.

What is wrong with PowerShell not getting JSON data?

it's to share knowledge on a new CIrcleCI feature that I spend a number of hours figuring out due to not having documentation on the issue. — Stephen Lester, Aug 30 '19 at 19:26
I rewrote this. Is this more in line with SO post standards? @Sandeep Ranjan — Stephen Lester, Oct 11 '19 at 16:29

Stephen Lester · Accepted Answer · 2019-09-13T19:40:57.350

It's EDN, didn't know this until CircleCI answered a question on this topic. So if you are using PowerShell to retrieve artifacts from CircleCI you definitely want to know this.

you need to pass a header specifying the data type returned.

(Accept: application/json)

A CircleCI Support member let me know that in PowerShell you have to specify an Accept header to receive the data in JSON. No wonder I'm getting weird output! So trying again with the new accept JSON header we have this command below.

Working command to get data in JSON and have it auto-convert to a PSObject.

Invoke-RestMethod -Uri https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN -Method GET -ContentType 'application/json' -UseBasicParsing -Header @{"Accept" = "application/json"}

OUTPUT

$response|select path,url

path
----
src.orig/file1.txt
src.orig/file2.txt

url
---
https://15-824975-gh.circle-artifacts.com/0/src.orig/file1.txt
https://15-824975-gh.circle-artifacts.com/0/src.orig/file2.txt

Using the PS commands Invoke-WebRequest/Invoke-RestMethod both will receive data in EDN format if you don't do the below. Yay now I can use the data as I see fit to download my artifacts.

Reply from CircleCI that got me the solution.

@burninmedia So what's being sent back is actually a data format called EDN. If you want to return JSON you'll need to pass a header specifying so (Accept: application/json). Thanks!

Here is a simple script I wrote to download all the artifacts. Please be sure you're setting the environment variables.

if ($USERNAME -eq $null) { write-host " please add required variable USERNAME" ;exit }
if ($VCS_TYPE -eq $null) { write-host " please add required variable VCS_TYPE" ;exit}
if ($CIRCLE_TOKEN -eq $null) { write-host " please add required variable CIRCLE_TOKEN" ;exit}
if ($BUILD_NUMBER -eq $null) { write-host " please add required variable BUILD_NUMBER" ;exit}
if ($PROJECT -eq $null) { write-host " please add required variable PROJECT" ;exit}
if ($BASEPATH -eq $null) { write-host " please add required variable BASEPATH" ;exit}

$response = Invoke-RestMethod -Uri https://circleci.com/api/v1.1/project/$VCS_TYPE/$USERNAME/$PROJECT/$BUILD_NUMBER/artifacts?circle-token=$CIRCLE_TOKEN -Method GET -ContentType 'application/json' -UseBasicParsing -Header @{"Accept" = "application/json"}

ForEach ($i in $response){
$PATH = $(Split-Path -Path "$($BASEPATH)\$($i.path)")

if (-Not ( Test-Path $PATH) ) {
    write-host "Creating folder: $($PATH)"
    New-Item -ItemType Directory -Force -Path "$($PATH)"
    }
Write-Host "Saving artifact $($i.pretty_path) to file: $($BASEPATH)\$($i.path)"
Invoke-RestMethod "$($i.url)?circle-token=$($CIRCLE_TOKEN)" -UseBasicParsing -OutFile "$($BASEPATH)\$($i.path)"
}

Bash version

export CIRCLE_TOKEN=':your_token'

echo $(https://circleci.com/api/v1.1/project/$vcs-type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN) > ./artifact_json
for ((i = 0 ; i <= $(jq -c '.[].url ' ./artifact_json|wc -l) ; i++));
do
    path=$(jq -c ".[$i].path" ./artifact_json|tr -d '"');
    url=$(jq -c ".[$i].url" ./artifact_json|tr -d '"');
    pathdir=$(dirname "$path")
    echo "URL: $url"
    echo "path: $path"
    echo "Pathdir: $pathdir"
    [ -d $pathdir ] && mkdir -p "$pathdir" #check if folder exists if not mkdir
    wget -o $path $url
done
rm ./artifact_json```

How to download artifacts from CircleCI via Powershell command Invoke-RESTMethod

1 Answers1