14

I have a webapplication that can process POSTing of a html form like this:

<form action="x" method="post" enctype="multipart/form-data">
  <input name="xfa" type="file">
  <input name="pdf" type="file">
  <input type="submit" value="Submit">
</form>

Note that there are two type="file" <input> elements.

How can I script POSTing this from a Powershell script? I plan to do that to create a simple test-framework for the service.

I found WebClient.UploadFile(), but that can only handle a single file.

Thank you for taking your time.

Marian Aldenhövel
  • 565
  • 1
  • 5
  • 21
  • Upgrade to Powershell 3 or higher and use `Invoke-WebRequest`: http://technet.microsoft.com/en-us/library/hh849901.aspx. That or use `foreach` to loop through your files. – Raf Aug 01 '14 at 08:15
  • Upgrading powershell unfortunately is not an option as we still are using Windows XP. Also I cannot find a mention of file upload on the page linked, only simple Name-Value-Pairs for forms. I do not understand the suggestion to use foreach. Are you suggesting two separate calls to WebClient.UploadFile()? That would not work, as the server-side requires the content of both files in the same request and returns the result of some processing on them as response. – Marian Aldenhövel Aug 01 '14 at 13:10
  • If you search SO for "powershell post" a plethora of alternative answers will appear. – Raf Aug 01 '14 at 14:01

4 Answers4

26

I've been crafting multipart HTTP POST with PowerShell today. I hope the code below is helpful to you.

  • PowerShell itself cannot do multipart form uploads.
  • There are not many sample about it either. I built the code based on this and this.
  • Sure, Invoke-RestMethod requires PowerShell 3.0 but the code in the latter of the above links shows how to do HTTP POST with .NET directly, allowing you to have this running in Windows XP as well.

Good luck! Please tell if you got it to work.

function Send-Results {
    param (
        [parameter(Mandatory=$True,Position=1)] [ValidateScript({ Test-Path -PathType Leaf $_ })] [String] $ResultFilePath,
        [parameter(Mandatory=$True,Position=2)] [System.URI] $ResultURL
    )
    $fileBin = [IO.File]::ReadAllBytes($ResultFilePath)
    $computer= $env:COMPUTERNAME

    # Convert byte-array to string (without changing anything)
    #
    $enc = [System.Text.Encoding]::GetEncoding("iso-8859-1")
    $fileEnc = $enc.GetString($fileBin)

    <#
    # PowerShell does not (yet) have built-in support for making 'multipart' (i.e. binary file upload compatible)
    # form uploads. So we have to craft one...
    #
    # This is doing similar to: 
    # $ curl -i -F "file=@file.any" -F "computer=MYPC" http://url
    #
    # Boundary is anything that is guaranteed not to exist in the sent data (i.e. string long enough)
    #    
    # Note: The protocol is very precise about getting the number of line feeds correct (both CRLF or LF work).
    #>
    $boundary = [System.Guid]::NewGuid().ToString()    # 

    $LF = "`n"
    $bodyLines = (
        "--$boundary",
        "Content-Disposition: form-data; name=`"file`"$LF",   # filename= is optional
        $fileEnc,
        "--$boundary",
        "Content-Disposition: form-data; name=`"computer`"$LF",
        $computer,
        "--$boundary--$LF"
        ) -join $LF

    try {
        # Returns the response gotten from the server (we pass it on).
        #
        Invoke-RestMethod -Uri $URL -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -TimeoutSec 20 -Body $bodyLines
    }
    catch [System.Net.WebException] {
        Write-Error( "FAILED to reach '$URL': $_" )
        throw $_
    }
}
Community
  • 1
  • 1
akauppi
  • 14,244
  • 12
  • 73
  • 94
  • Thank you very much. That looks a lot like what I eventually discovered myself: Handcraft the multipart/form-data body. That gives me more confidence in the result. – Marian Aldenhövel Aug 01 '14 at 16:44
  • Note that the randomness of the boundary string is really not that important, if one is shipping binary contents. The length matters. Just having a long enough garble static string is probably just as reliable as using a GUID. – akauppi May 19 '15 at 08:29
  • 1
    ISO-8859-1 is only encoding where byte value == code point value, you should use ```$enc = [System.Text.Encoding]::GetEncoding("ISO-8859-1")``` – Guilherme Torres Castro Jul 23 '15 at 18:20
  • @GuilhermeTorresCastro So ASCII means 7-bit ASCII only and ISO 8859 means 8 bit. Okay. – akauppi Jul 24 '15 at 21:37
  • My current working code is a bit different than this, anyways. Please inform if anyone has issues. – akauppi Jul 24 '15 at 21:44
  • @akauppi A had an issue uploading a binary file using ASCII enconding, for texts files, ASCII and others encondings may work as expected. – Guilherme Torres Castro Jul 27 '15 at 15:10
  • 4
    Worked for me once I switched from "backtick n" to "backtick r backtick n" for $LF – Timje Nov 05 '15 at 16:21
  • @akauppi Thank you for getting me started. I've remixed your answer into a more generic solution and presented it as a separate answer (feel free to slipstream it into your own answer, and if you do ping me so I can remove my own answer). – Jeroen Dec 27 '16 at 10:55
  • I'm getting the following error: Invoke-RestMethod : The format of value 'multipart/form-data; boundary="098c2f6b-b6b7-44f3-af8b-de33baeb3986"' is invalid. Does anyone know what could be wrong? – Suhair Zain Aug 28 '17 at 14:10
  • Very clever! =) – karliwson Jan 28 '18 at 20:52
  • I am sorry, but I get Invoke-RestMethod : You must write ContentLength bytes to the request stream before calling [Begin]GetResponse. Does anyone know why or how to fix this? – ggb667 Jun 17 '20 at 22:30
  • I had to put `"Content-Type: application/octet-stream"` above `$fileEnc` to make it work. – John Fouhy Jan 21 '21 at 23:31
5

I was bothered by this thing and haven't found a satisfactory solution. Although the gist here proposed can do the yob, it is not efficient in case of large files transmittal. I wrote a blog post proposing a solution for it, basing my cmdlet on HttpClient class present in .NET 4.5. If that is not a problem for you, you can check my solution at the following address http://blog.majcica.com/2016/01/13/powershell-tips-and-tricks-multipartform-data-requests/

EDIT:

function Invoke-MultipartFormDataUpload
{
    [CmdletBinding()]
    PARAM
    (
        [string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$InFile,
        [string]$ContentType,
        [Uri][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Uri,
        [System.Management.Automation.PSCredential]$Credential
    )
    BEGIN
    {
        if (-not (Test-Path $InFile))
        {
            $errorMessage = ("File {0} missing or unable to read." -f $InFile)
            $exception =  New-Object System.Exception $errorMessage
            $errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'MultipartFormDataUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $InFile
            $PSCmdlet.ThrowTerminatingError($errorRecord)
        }

        if (-not $ContentType)
        {
            Add-Type -AssemblyName System.Web

            $mimeType = [System.Web.MimeMapping]::GetMimeMapping($InFile)

            if ($mimeType)
            {
                $ContentType = $mimeType
            }
            else
            {
                $ContentType = "application/octet-stream"
            }
        }
    }
    PROCESS
    {
        Add-Type -AssemblyName System.Net.Http

        $httpClientHandler = New-Object System.Net.Http.HttpClientHandler

        if ($Credential)
        {
            $networkCredential = New-Object System.Net.NetworkCredential @($Credential.UserName, $Credential.Password)
            $httpClientHandler.Credentials = $networkCredential
        }

        $httpClient = New-Object System.Net.Http.Httpclient $httpClientHandler

        $packageFileStream = New-Object System.IO.FileStream @($InFile, [System.IO.FileMode]::Open)

        $contentDispositionHeaderValue = New-Object System.Net.Http.Headers.ContentDispositionHeaderValue "form-data"
        $contentDispositionHeaderValue.Name = "fileData"
        $contentDispositionHeaderValue.FileName = (Split-Path $InFile -leaf)

        $streamContent = New-Object System.Net.Http.StreamContent $packageFileStream
        $streamContent.Headers.ContentDisposition = $contentDispositionHeaderValue
        $streamContent.Headers.ContentType = New-Object System.Net.Http.Headers.MediaTypeHeaderValue $ContentType

        $content = New-Object System.Net.Http.MultipartFormDataContent
        $content.Add($streamContent)

        try
        {
            $response = $httpClient.PostAsync($Uri, $content).Result

            if (!$response.IsSuccessStatusCode)
            {
                $responseBody = $response.Content.ReadAsStringAsync().Result
                $errorMessage = "Status code {0}. Reason {1}. Server reported the following message: {2}." -f $response.StatusCode, $response.ReasonPhrase, $responseBody

                throw [System.Net.Http.HttpRequestException] $errorMessage
            }

            $responseBody = [xml]$response.Content.ReadAsStringAsync().Result

            return $responseBody
        }
        catch [Exception]
        {
            $PSCmdlet.ThrowTerminatingError($_)
        }
        finally
        {
            if($null -ne $httpClient)
            {
                $httpClient.Dispose()
            }

            if($null -ne $response)
            {
                $response.Dispose()
            }
        }
    }
    END { }
}

Cheers

MaiOM
  • 826
  • 2
  • 10
  • 27
Mario Majcica
  • 690
  • 1
  • 6
  • 18
  • 1
    It's helpful to post at least a summary of the solution outlined on your blog, in case the link breaks/changes, or your server is down in the future. – ryebread Jan 13 '16 at 16:32
  • 1
    You are completely right. I added the solution code, hope it is sufficient. – MaiOM Jan 14 '16 at 16:46
3

I've remixed @akauppi's answer into a more generic solution, a cmdlet that:

  • Can take pipeline input from Get-ChildItem for files to upload
  • Takes an URL as a positional parameter
  • Takes a dictionary as a positional parameter, which it sends as additional form data
  • Takes an (optional) -Credential parameter
  • Takes an (optional) -FilesKey parameter to specify the formdata key for the files upload part
  • Supports -WhatIf
  • Has -Verbose logging
  • Exits with an error if something goes wrong

It can be called like this:

$url ="http://localhost:12345/home/upload"
$form = @{ description = "Test 123." }
$pwd = ConvertTo-SecureString "s3cr3t" -AsPlainText -Force
$creds = New-Object System.Management.Automation.PSCredential ("john", $pwd)

Get-ChildItem *.txt | Send-MultiPartFormToApi $url $form $creds -Verbose -WhatIf

Here's the code to the full cmdlet:

function Send-MultiPartFormToApi {
    # Attribution: [@akauppi's post](https://stackoverflow.com/a/25083745/419956)
    # Remixed in: [@jeroen's post](https://stackoverflow.com/a/41343705/419956)
    [CmdletBinding(SupportsShouldProcess = $true)] 
    param (
        [Parameter(Position = 0)]
        [string]
        $Uri,

        [Parameter(Position = 1)]
        [HashTable]
        $FormEntries,

        [Parameter(Position = 2, Mandatory = $false)]
        [System.Management.Automation.Credential()]
        [System.Management.Automation.PSCredential]
        $Credential,

        [Parameter(
            ParameterSetName = "FilePath",
            Mandatory = $true,
            ValueFromPipeline = $true,
            ValueFromPipelineByPropertyName = $true
        )]
        [Alias("Path")]
        [string[]]
        $FilePath,

        [Parameter()]
        [string]
        $FilesKey = "files"
    );

    begin {
        $LF = "`n"
        $boundary = [System.Guid]::NewGuid().ToString()

        Write-Verbose "Setting up body with boundary $boundary"

        $bodyArray = @()

        foreach ($key in $FormEntries.Keys) {
            $bodyArray += "--$boundary"
            $bodyArray += "Content-Disposition: form-data; name=`"$key`""
            $bodyArray += ""
            $bodyArray += $FormEntries.Item($key)
        }

        Write-Verbose "------ Composed multipart form (excl files) -----"
        Write-Verbose ""
        foreach($x in $bodyArray) { Write-Verbose "> $x"; }
        Write-Verbose ""
        Write-Verbose "------ ------------------------------------ -----"

        $i = 0
    }

    process {
        $fileName = (Split-Path -Path $FilePath -Leaf)

        Write-Verbose "Processing $fileName"

        $fileBytes = [IO.File]::ReadAllBytes($FilePath)
        $fileDataAsString = ([System.Text.Encoding]::GetEncoding("iso-8859-1")).GetString($fileBytes)

        $bodyArray += "--$boundary"
        $bodyArray += "Content-Disposition: form-data; name=`"$FilesKey[$i]`"; filename=`"$fileName`""
        $bodyArray += "Content-Type: application/x-msdownload"
        $bodyArray += ""
        $bodyArray += $fileDataAsString

        $i += 1
    }

    end {
        Write-Verbose "Finalizing and invoking rest method after adding $i file(s)."

        if ($i -eq 0) { throw "No files were provided from pipeline." }

        $bodyArray += "--$boundary--"

        $bodyLines = $bodyArray -join $LF

        # $bodyLines | Out-File data.txt # Uncomment for extra debugging...

        try {
            if (!$WhatIfPreference) {
                Invoke-RestMethod `
                    -Uri $Uri `
                    -Method Post `
                    -ContentType "multipart/form-data; boundary=`"$boundary`"" `
                    -Credential $Credential `
                    -Body $bodyLines
            } else {
                Write-Host "WHAT IF: Would've posted to $Uri body of length " + $bodyLines.Length
            }
        } catch [Exception] {
            throw $_ # Terminate CmdLet on this situation.
        }

        Write-Verbose "Finished!"
    }
}
Community
  • 1
  • 1
Jeroen
  • 53,290
  • 30
  • 172
  • 279
2

I have found a solution to my problem after studying how multipart/form-data is built. A lot of help came in the form of http://www.paraesthesia.com/archive/2009/12/16/posting-multipartform-data-using-.net-webrequest.aspx.

The solution then is to build the body of the request up manually according to that convention. I have left of niceties like correct Content-Lengths etc.

Here is an excerpt of what I am using now:

    $path = "/Some/path/to/data/"

    $boundary_id = Get-Date -Format yyyyMMddhhmmssfffffff
    $boundary = "------------------------------" + $boundary_id

    $url = "http://..."
    [System.Net.HttpWebRequest] $req = [System.Net.WebRequest]::create($url)
    $req.Method = "POST"
    $req.ContentType = "multipart/form-data; boundary=$boundary"
    $ContentLength = 0
    $req.TimeOut = 50000

    $reqst = $req.getRequestStream()

    <#
    Any time you write a file to the request stream (for upload), you'll write:
        Two dashes.
        Your boundary.
        One CRLF (\r\n).
        A content-disposition header that tells the name of the form field corresponding to the file and the name of the file. That looks like:
        Content-Disposition: form-data; name="yourformfieldname"; filename="somefile.jpg" 
        One CRLF.
        A content-type header that says what the MIME type of the file is. That looks like:
        Content-Type: image/jpg
        Two CRLFs.
        The entire contents of the file, byte for byte. It's OK to include binary content here. Don't base-64 encode it or anything, just stream it on in.
        One CRLF.
    #>

    <# Upload #1: XFA #> 
    $xfabuffer = [System.IO.File]::ReadAllBytes("$path\P7-T.xml")

    <# part-header #>
    $header = "--$boundary`r`nContent-Disposition: form-data; name=`"xfa`"; filename=`"xfa`"`r`nContent-Type: text/xml`r`n`r`n"
    $buffer = [Text.Encoding]::ascii.getbytes($header)        
    $reqst.write($buffer, 0, $buffer.length)
    $ContentLength = $ContentLength + $buffer.length

    <# part-data #>
    $reqst.write($xfabuffer, 0, $xfabuffer.length)
    $ContentLength = $ContentLength + $xfabuffer.length

    <# part-separator "One CRLF" #>
    $terminal = "`r`n"
    $buffer = [Text.Encoding]::ascii.getbytes($terminal)        
    $reqst.write($buffer, 0, $buffer.length)
    $ContentLength = $ContentLength + $buffer.length

    <# Upload #1: PDF template #>
    $pdfbuffer = [System.IO.File]::ReadAllBytes("$path\P7-T.pdf")

    <# part-header #>
    $header = "--$boundary`r`nContent-Disposition: form-data; name=`"pdf`"; filename=`"pdf`"`r`nContent-Type: application/pdf`r`n`r`n"
    $buffer = [Text.Encoding]::ascii.getbytes($header)        
    $reqst.write($buffer, 0, $buffer.length)
    $ContentLength = $ContentLength + $buffer.length

    <# part-data #>
    $reqst.write($pdfbuffer, 0, $pdfbuffer.length)
    $ContentLength = $ContentLength + $pdfbuffer.length

    <# part-separator "One CRLF" #>
    $terminal = "`r`n"
    $buffer = [Text.Encoding]::ascii.getbytes($terminal)        
    $reqst.write($buffer, 0, $buffer.length)
    $ContentLength = $ContentLength + $buffer.length

    <#
    At the end of your request, after writing all of your fields and files to the request, you'll write:

    Two dashes.
    Your boundary.
    Two more dashes.
    #>
    $terminal = "--$boundary--"
    $buffer = [Text.Encoding]::ascii.getbytes($terminal)        
    $reqst.write($buffer, 0, $buffer.length)
    $ContentLength = $ContentLength + $buffer.length

    $reqst.flush()
    $reqst.close()

    # Dump request to console
    #$req

    [net.httpWebResponse] $res = $req.getResponse()

    # Dump result to console
    #$res

    # Dump result-body to filesystem
<#    
    $resst = $res.getResponseStream()
    $sr = New-Object IO.StreamReader($resst)
    $result = $sr.ReadToEnd()
    $res.close()
#>

    $null = New-Item -ItemType Directory -Force -Path "$path\result"
    $target = "$path\result\P7-T.pdf"

    # Create a stream to write to the file system.
    $targetfile = [System.IO.File]::Create($target)

    # Create the buffer for copying data.
    $buffer = New-Object Byte[] 1024

    # Get a reference to the response stream (System.IO.Stream).
    $resst = $res.GetResponseStream()

    # In an iteration...
    Do {
        # ...attemt to read one kilobyte of data from the web response stream.
        $read = $resst.Read($buffer, 0, $buffer.Length)

        # Write the just-read bytes to the target file.
        $targetfile.Write($buffer, 0, $read)

        # Iterate while there's still data on the web response stream.
    } While ($read -gt 0)

    # Close the stream.
    $resst.Close()
    $resst.Dispose()

    # Flush and close the writer.
    $targetfile.Flush()
    $targetfile.Close()
    $targetfile.Dispose()
HNygard
  • 3,926
  • 4
  • 28
  • 38
Marian Aldenhövel
  • 565
  • 1
  • 5
  • 21
  • You can also use just 'n instead of 'r'n. Also, the $ContentLength is not needed (or if it's useful, your code doesn't yet place it anywhere). – akauppi Aug 01 '14 at 15:46
  • @akauppi: I found that `r`n was required. If only `n was used, the API I was attempting to call would encounter an exception and return a 500 status code. While just a line feed would make it look ok visually in Powershell, both carriage return and linefeed were expected. – Ellesedil Aug 27 '14 at 19:24
  • Using `"\`r\`n"` was ultimately the solution for me For added context: I started this whole process using `Invoke-RestMethod` and trying to pass a multipart body using the `Form` parameter. That didn't work in my case as the API server kept returning `"Corrupt form data: no leading boundary"`. I also tried crafting the `Body` parameter manually using `MultipartFormDataContent` as proposed by other sources to no avail: https://github.com/PowerShell/PowerShell/issues/9241#issuecomment-477467675 https://get-powershellblog.blogspot.com/2017/09/multipartform-data-support-for-invoke.html – Esteban Oct 15 '20 at 15:27