24

First to clarify this question is aimed to HTTP(s) download .For FTP may be I'll ask (and answer) another question. Here are some similar questions - but I want to be more precise .

Besides excluding external tools I want the solution(s) to be applicable for the widest possible types of windows machines (including XP,Win2003,Vista which still have big enough share). Also as WSH is one of the possible options I prefer no using of temp files and everything to be packed in a single .bat file (which is possible with both jscript and vbscript).

What are possible approaches.

  1. "Pure" batch solution with BITSADMIN - a command line utility available on every windows machine .It's not pretty convenient but is an/the only option where no other scripting language should be used.
  2. Using WSH - Three approaches are possible - WinHTTP , MSXML2.XMLHTTP ,InternetExlorer.Application - all of them are accessible ActiveX objects in order of how I prefer them.WinHTTP and MSXML2.XMLHTTP are pretty similar in their capabilities but WinHTTP has a reputation of more stable.InternetExlorer.Application is in fact just the Internet explorer accessible through ActiveX object and some UI elements are unavoidable (are they?) so I'll skip this one.
  3. Using .NET - It's possible to create a hybrid batch file with all the three default .NET compilers (Jscript.net , VB.Net , C#) with Jscript.net there is no redundant error messages so I'll prefer it.If we ignore the fact that there's a compiled .exe all the code is in one file ,so according to me this fits in the requirements :-) .With .NET we can use System.Net.WebClient or System.Net.HttpWebRequest (the WebClient relies on it) or
    System.Web.HttpRequest , but for now I'll post only System.Net.WebClient solution.And even more same ActiveX objects accessible with WSH are available here too.So there are really many ways to dowanload a file with .Net.May be in the future I'll update my answer.Anyway only the Webclient is especially designed for download.
  4. Using powershell - has same possibilities as .NET but with less chances to be installed on all machines you can meet.So I'll skip this one too.
Community
  • 1
  • 1
npocmaka
  • 51,748
  • 17
  • 123
  • 166

2 Answers2

46

The answers.All scripts should be saved with .bat/.cmd extensions and can be used directly as batch scripts.

  1. Certutuil (for some reasons in the newest win10 builds this is recognized as trojan thread ):

    certutil.exe -urlcache -split -f "https://download.sysinternals.com/files/PSTools.zip" pstools.zip

for easiness a macro can be used:

set "download=certutil.exe -urlcache -split -f"
%download% "https://download.sysinternals.com/files/PSTools.zip" pstools.zip

CertUtil command can be abused to download a file from internet.Available by default in windows since Vista.For WinXP Server 2003 Administration Tools are needed.

  1. Bitsadmin :

simplest possible way to use it

bitsadmin /transfer myDownloadJob /download /priority normal http://downloadsrv/10mb.zip c:\10mb.zip

with macro:

set "dnld=bitsadmin /transfer myDownloadJob /download /priority normal"
%dnld% "https://download.sysinternals.com/files/PSTools.zip" %cd%\pstools.zip

or with bitsDownloader.bat

call bitsDownloader.bat "https://download.sysinternals.com/files/PSTools.zip" pstools.zip
  1. winhhtpjs.bat is a command line http client that uses WinHttpRequest .It can perform whole range of http (POST,DELETE,..) requests and can be used for downloading of files too (not too big files).Also custom headers can be added.
call winhhtpjs.bat "https://example.com/files/some.zip" -saveTo "c:\somezip.zip" 
  1. XMLHTTPDownloadJS.bat is bat file that uses MSXML2.XMLHTTP object to download a file . Does not offer so rich options as winhhtpjs.bat , though is still an option.

    call XMLHTTPDownloadJS.bat "https://download.sysinternals.com/files/PSTools.zip pst2.zip" pstools.zip

  2. WebClientDownload.bat uses the .NET System.Net.WebClient class. It creates a small exe file in order to selfcompile itself and requires an installed a .net framework. As the class is presented in the very earlier versions of .net it is backward compatible enough

call webclientDownload.bat "https://download.sysinternals.com/files/PSTools.zip" pstools.zip
  1. With the latest builds of windows 10 we have the CURL command ,though this is not so backward compatible option. Mind that only the newest versions of windows has CURL installed by default.

    curl "https://download.sysinternals.com/files/PSTools.zip" --output pstools.zip

npocmaka
  • 51,748
  • 17
  • 123
  • 166
  • 2
    Can `bitsadmin` be used to download every file on a webpage? – Argyll Apr 22 '15 at 10:54
  • @Argyll - rather you'll need a program with web page crawling capabilities like wget (far more powerful than everything here ) – npocmaka Apr 22 '15 at 11:03
  • I figured. I couldn't get Wget to do that. (It returns a bunch of random characters followed by "Permission Denied". That's why I asked. – Argyll Apr 22 '15 at 11:18
  • 2
    @The WSH JScript has multiple issues. `WScript` is case sensitive, -saveTo is used as an unnamed argument, `XMLHTTPObj` sometime mispelled `WinHTTPObj`. I also suggest you to use `WScript.ScriptName` to make simpler the direct cscript call. – antonio Aug 12 '16 at 21:50
  • 2
    Dont forget the powershell command `(New-Object Net.WebClient).DownloadFile('Source, Dest')`. Works on powershell v2 and not tested v1. It's the only script that can download files on OS's with PS's & CMD's that low. – John Kens Aug 14 '18 at 14:30
  • 2
    @npocmaka hi, so, who can I get only last-modified date from file, not download them with you answer/code? thx anyway, sorry my /en.. . – It Wasn't Me Oct 14 '19 at 02:24
  • @ItWasn'tMe - you'll first need to download information about the files modification dates. But you'll have to check if the server provides such an information. – npocmaka Oct 14 '19 at 08:34
  • 2
    I found I needed the `/dynamic` switch. For example: `bitsadmin /transfer myDownloadJob /dynamic /download /priority foreground https://github.com/microsoft/vscode-cpptools/releases/latest/download/cpptools-win32.vsix %USERPROFILE%\Downloads\cpptools-win32.vsix` – user2023370 Feb 05 '20 at 13:49
  • 1
    I note that the "Modified Date" is not the time of download; as it is for all simple browser downloads. (In case you're looking for the file you just downloaded :) – user2023370 Feb 05 '20 at 13:52
  • 1
    I tried certutil and bitsadmin, and both worked well. Certutil seemed to go faster. Thank you. – mcdon Mar 13 '20 at 14:25
  • These days `certutil` will give *Access is denied* error. For macro, use `doskey` – Zimba Apr 25 '20 at 18:09
2

There's a utility (resides with CMD) on Windows which can be run from CMD (if you have write access):

set url=https://www.nsa.org/content/hl-images/2017/02/09/NSA.jpg
set file=file.jpg
certutil -urlcache -split -f %url% %file%
:also certutil.exe -verifyctl -f -split %url% %file%

Cmdlets in Powershell:

$url = "https://www.nsa.org/content/hl-images/2017/02/09/NSA.jpg"
$file = "file.jpg"
$ProgressPreference = "SilentlyContinue";
Invoke-WebRequest -Uri $url -outfile $file

.Net under PowerShell:

$url = "https://www.nsa.org/content/hl-images/2017/02/09/NSA.jpg"
$file = "file.jpg"
# Add the necessary .NET assembly
Add-Type -AssemblyName System.Net.Http
# Create the HttpClient object
$client = New-Object -TypeName System.Net.Http.Httpclient
$task = $client.GetAsync($url)
$task.wait();
[io.file]::WriteAllBytes($file, $task.Result.Content.ReadAsByteArrayAsync().Result)

C# Command-line build with csc.exe:
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/command-line-building-with-csc-exe

using System;
using System.IO;

using System.Net.Http;
using System.Threading.Tasks;

namespace DownloadImage
{
    class Program
    {
        static async Task Main(string[] args)
        {
            using var httpClient = new HttpClient();
            var url = "https://www.nsa.org/content/hl-images/2017/02/09/NSA.jpg";
            byte[] imageBytes = await httpClient.GetByteArrayAsync(url);

            using var fs = new FileStream("file.jpg", FileMode.Create);
            fs.Write(imageBytes, 0, imageBytes.Length);

        }
    }
}

Built in Windows applications. No need for external downloads.

Tested on Win 10

Zimba
  • 1,095
  • 9
  • 13
  • thanks. The issues with certutil are something new. Now it is detected as a thread. – npocmaka Apr 26 '20 at 08:54
  • `Certutil` write access by default is now only given to *TrustedInstaller*, after viruses abused it to download exe, pif, vbs & bat files. https://www.hybrid-analysis.com/sample/87cf118bff58c38bc0d54c1d3fcc553e381df838437a99a2006dd8f726406c16?environmentId=100 – Zimba Apr 27 '20 at 18:22
  • what advantage `certutil` has compared to `cURL` and `BitsAdmin`? You can use them also to download stuff... – npocmaka Apr 28 '20 at 04:53
  • 1
    `certutil` works with certificates, pfx files, crypto keys, registry, roots, active directory, policies, caches, databases, encode files (to avoid detection by antivirus), in addition to downloading files & certificates so may have been a popular choice for virus & malware – Zimba Apr 28 '20 at 08:20
  • 1
    To avoid further detection, `certutil` can also hide executables in alternate data streams of ordinary files eg. `trustme.txt` – Zimba May 03 '20 at 16:53