Download File to server from URL

Question

Well, this one seems quite simple, and it is. All you have to do to download a file to your server is:

file_put_contents("Tmpfile.zip", file_get_contents("http://someurl/file.zip"));

Only there is one problem. What if you have a large file, like 100mb. Then, you will run out of memory, and not be able to download the file.

What I want is a way to write the file to the disk as I am downloading it. That way, I can download bigger files, without running into memory problems.

That's set in your server configuration, PHP can't really get around it as far as I know (except for a direct .ini edit) — Ben, Oct 15 '10 at 00:12

score 530 · Accepted Answer · edited May 23 '17 at 11:47

530

Since PHP 5.1.0, file_put_contents() supports writing piece-by-piece by passing a stream-handle as the $data parameter:

file_put_contents("Tmpfile.zip", fopen("http://someurl/file.zip", 'r'));

From the manual:

If data [that is the second argument] is a stream resource, the remaining buffer of that stream will be copied to the specified file. This is similar with using stream_copy_to_stream().

(Thanks Hakre.)

edited May 23 '17 at 11:47

Community

1
1

answered Oct 15 '10 at 00:15

alex

438,662
188
837
957

5

That wouldn't be my first choice. If `allow_fopen_url Off` is set in php.ini (good idea for security), your script would be broken. – PleaseStand Oct 15 '10 at 00:43
4

@idealmachine I think `file_get_contents()` would not work either if that were the case (see OP). – alex Oct 15 '10 at 00:45
10

@geoff I was specific, I mentioned the function you wanted. What you may have wanted was someone to write the code for you - but I'm sure you learned something doing it yourself. Also, if we are going to comment on each other's SO interactions - [please accept some more answers](http://blog.stackoverflow.com/2009/08/new-question-asker-features/) :) – alex Oct 15 '10 at 01:37
@alex: Please see the edit, feel free to incorporate. let me know when I can remove this comment here then. – hakre Mar 31 '13 at 14:20
@hakre Thanks, much appreciated. I'll make it flow a bit easier. :) – alex Mar 31 '13 at 23:21
4

The 'b' flag should also be used in most cases with `fopen`; prevents adverse effects to images and other non plain text files. – Wayne Weibel Sep 12 '13 at 18:48
thanks this saved my 1GB of mobile data i was planning transfering file from one server to another, i could directly moved zip file instead of downloading and uploading to new server. :) – Dheeraj Thedijje Jul 05 '16 at 09:00
Would this approach be better than using `curl`? – PlayHardGoPro Nov 05 '19 at 12:49

score 137 · Answer 2 · edited Nov 23 '15 at 17:04

137

private function downloadFile($url, $path)
{
    $newfname = $path;
    $file = fopen ($url, 'rb');
    if ($file) {
        $newf = fopen ($newfname, 'wb');
        if ($newf) {
            while(!feof($file)) {
                fwrite($newf, fread($file, 1024 * 8), 1024 * 8);
            }
        }
    }
    if ($file) {
        fclose($file);
    }
    if ($newf) {
        fclose($newf);
    }
}

edited Nov 23 '15 at 17:04

Ibrahim Azhar Armar

23,281
32
116
195

answered Oct 15 '10 at 01:31

xaav

7,348
7
27
42

1

thank's for your snippit, but would you be able to explain your code @xaav? I'm not exactly brilliant at php. What is 1024*8 for ? Thank's again. – vvMINOvv Dec 10 '11 at 14:19
@wMINOw The length of the line. – David Bélanger Jun 11 '12 at 20:47
2

Specifically, it means to read up to 8KB at a time (1024 bytes per KB * 8) since the parameter is in bytes. As long as the line is <= 8KB, it will read the entire line at once. – Doktor J Sep 27 '12 at 13:49
1

Why is not this the best answer? – GunJack Dec 17 '13 at 18:30
This should be the best answer. – Akhilesh Oct 05 '14 at 16:25
1

How do you handle errors with this approach? What if a 404 is returned or the connection is interrupted or times out? – Adam Swinden Dec 29 '14 at 11:33
That would be useful, but if 404 goes through you would be downloading anything, and you can always run a search check for that file (some logic) if (!myfile) -> break; – The Bumpaster Sep 09 '15 at 22:00
Best way to download large files – Tofeeq Aug 10 '16 at 08:52
Magnificently fast - you r true maestro! – SergeDirect Oct 05 '16 at 04:04
sometimes the `fopen failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden` show this error because of web crawling prevention how to resolve it? @ @ – NomanJaved Jan 15 '17 at 18:17
NomanJaved you need to do auth request on a login form first and reuse the same curl client (or at least it's headers and cookie) for all further requests. – Oleg Abrazhaev Jun 15 '17 at 13:12
How it downloads without CURL ? – Naveen DA Nov 24 '17 at 06:00
Downloads 0KB empty file although the file is more than 17MB Skype main.db file. On PHP 5.4 also had to remove "private" from the beginning of the function. – Kaspar L. Palgi Oct 29 '18 at 20:16
This solution was very nice for me, because I had to download a huge XML file which exceeds my PHP memory_limit if 1024M. – Marco Mar 27 '19 at 09:32
How is this different/better from the answer with `fopen`? – Pacerier Jun 15 '20 at 12:52
`Note`: The $path should include the name of the file you want to create. `Example: $path = $_SERVER['DOCUMENT_ROOT'].'/uploads/' . "test.mp3";` – NomanJaved May 14 '21 at 12:13

score 69 · Answer 3 · edited Sep 25 '14 at 18:34

69

Try using cURL

set_time_limit(0); // unlimited max execution time
$options = array(
  CURLOPT_FILE    => '/path/to/download/the/file/to.zip',
  CURLOPT_TIMEOUT =>  28800, // set this to 8 hours so we dont timeout on big files
  CURLOPT_URL     => 'http://remoteserver.com/path/to/big/file.zip',
);

$ch = curl_init();
curl_setopt_array($ch, $options);
curl_exec($ch);
curl_close($ch);

I'm not sure but I believe with the CURLOPT_FILE option it writes as it pulls the data, ie. not buffered.

edited Sep 25 '14 at 18:34

Community

1
1

answered Oct 15 '10 at 00:20

prodigitalson

58,127
8
92
110

2

Normally, this would be fine, but I have this code in a web app, so I cant be sure users will have cURL installed. However, I did give this a vote up. – xaav Oct 15 '10 at 00:29
@Geoff is it a distributed web app? Because if you control the hosting, then it doesn't matter about your users (cURL is a library on your server). – alex Oct 15 '10 at 00:47
No. I do not control hosting. It is a distributed web app that anyone could have. – xaav Oct 15 '10 at 01:26
3

Curl might be missing. But almost all shared hosting companies have CURL installed by default. I mean, I haven't seen one that doesn't. – Mangirdas Skripka Jul 14 '12 at 17:12
21

As from my tests, you can't assign to CURLOPT_FILE a file path directly. It has to be a file handler. First, open the file with `$fh = fopen('/path/to/download/the/file/to.zip', 'w');` and close with `fclose($fh);` after `curl_close($ch);`. And set `CURLOPT_FILE => $fh` – Gustavo May 07 '16 at 01:06

score 23 · Answer 4 · edited Mar 17 '20 at 07:17

prodigitalson's answer didn't work for me. I got missing fopen in CURLOPT_FILE more details.

This worked for me, including local urls:

function downloadUrlToFile($url, $outFileName)
{   
    if(is_file($url)) {
        copy($url, $outFileName); 
    } else {
        $options = array(
          CURLOPT_FILE    => fopen($outFileName, 'w'),
          CURLOPT_TIMEOUT =>  28800, // set this to 8 hours so we dont timeout on big files
          CURLOPT_URL     => $url
        );

        $ch = curl_init();
        curl_setopt_array($ch, $options);
        curl_exec($ch);
        curl_close($ch);
    }
}

score 19 · Answer 5 · edited Jun 20 '20 at 09:12

19

Create a folder called "downloads" in destination server
Save [this code] into .php file and run in destination server

Downloader :

<html>
<form method="post">
<input name="url" size="50" />
<input name="submit" type="submit" />
</form>
<?php
    // maximum execution time in seconds
    set_time_limit (24 * 60 * 60);

    if (!isset($_POST['submit'])) die();

    // folder to save downloaded files to. must end with slash
    $destination_folder = 'downloads/';

    $url = $_POST['url'];
    $newfname = $destination_folder . basename($url);

    $file = fopen ($url, "rb");
    if ($file) {
      $newf = fopen ($newfname, "wb");

      if ($newf)
      while(!feof($file)) {
        fwrite($newf, fread($file, 1024 * 8 ), 1024 * 8 );
      }
    }

    if ($file) {
      fclose($file);
    }

    if ($newf) {
      fclose($newf);
    }
?>
</html>

edited Jun 20 '20 at 09:12

Community

1
1

answered Mar 10 '12 at 13:53

stra8edge

199
1
2

This assumes the user wants a standalone script rather than a solution that will work within an existing PHP application, and I believe the latter is what the OP and most others are looking for. An explanation would also be helpful for people who want to understand the approach. – Sean the Bean May 20 '16 at 20:27
1

whenever I try this always my transferred file size is 50816 but my file size is bigger than this.. 120MB.. Any idea why is this? – Riffaz Starr May 06 '17 at 16:49
`set_time_limit (24 * 60 * 60);` has to be put inside a loop. It has no effect at the beginning of the script. – Viktor Joras Apr 23 '18 at 15:17
How to download from this url: https://filehippo.com/download_mozilla-firefox-64/post_download/ – Saurin Dashadia Jul 10 '20 at 13:38

score 17 · Answer 6 · edited Feb 06 '14 at 15:33

17

set_time_limit(0); 
$file = file_get_contents('path of your file');
file_put_contents('file.ext', $file);

edited Feb 06 '14 at 15:33

Vijaysinh Parmar

1,203
1
16
19

answered Jun 26 '13 at 12:24

Dimmy

199
1
2

your answer is very simple and good working, helped me where cURL failed to get file, this worked. Thanks :) – Tommix Feb 24 '14 at 08:39
2

You might want to explain what this actually does. – alex Dec 14 '15 at 09:05
6

This does not address the OP's problem of exceeding the PHP memory limit. – user9645 Jul 19 '16 at 19:50
This is pretty simple and straightforward. Quite useful for simpler cases where the files are small or the the environment is a local development. – Valentine Shi May 16 '19 at 13:28
any idea for .xlsx files ? It's storing empty file with 0 byte memory . – Dhruv Thakkar May 28 '20 at 16:16

score 9 · Answer 7 · answered Oct 31 '14 at 07:58

9

There are 3 ways:

file_get_contents and file_put_contents
CURL
fopen

You can find examples from here.

answered Oct 31 '14 at 07:58

Hoan Huynh

334
2
3

Pradeep Kumar Prabaharan · Answer 8 · 2019-08-07T10:13:52.017

9

Use a simple method in php copy()

copy($source_url, $local_path_with_file_name);

Note: if the destination file already exists, it will be overwritten

PHP copy() Function

Note: You need to set permission 777 for the destination folder. Use this method when you are downloading to your local machine.

Special Note: 777 is a permission in Unix based system with full read/write/execute permission to owner, group and everyone. In general we give this permission to assets which are not much needed to be hidden from public on a web server. Example: images folder.

edited Aug 07 '19 at 10:13

answered Jan 03 '17 at 09:54

Pradeep Kumar Prabaharan

3,407
2
30
34

1

I will never never never set 777 as perms on a webserver, and I will kick off any webdeveloper whom has the bad idea to do that. Every time, everywhere. Be carefull ! You can not do that ! Think about security. Following OWASP rules is not enough. Having good thinking about simple things matters. – ThierryB Jul 12 '19 at 00:09
@ThierryB. Note: I've given local path. & this can be used in internal applications. Having good reading and understanding of question and answer matters. Think different scenarios. And this is not accepted/best answer. Every question has different answers with pros & cons in it.. Example for you to understand: Even Fibonacci have multiple unique solutions where only one will be best. Others will be used in different scenarios. – Pradeep Kumar Prabaharan Jul 13 '19 at 02:08
Ok, but taking time to think about best practises and implement them inside secured places will give you a better understanding of concepts you must implement. Maybe if an intruder is inside your ($)home, doing some traps or building things the best way you can will give him some headaches ;) – ThierryB Jul 16 '19 at 18:12

score 5 · Answer 9 · answered Sep 10 '13 at 17:44

I use this to download file

function cURLcheckBasicFunctions()
{
  if( !function_exists("curl_init") &&
      !function_exists("curl_setopt") &&
      !function_exists("curl_exec") &&
      !function_exists("curl_close") ) return false;
  else return true;
}

/*
 * Returns string status information.
 * Can be changed to int or bool return types.
 */
function cURLdownload($url, $file)
{
  if( !cURLcheckBasicFunctions() ) return "UNAVAILABLE: cURL Basic Functions";
  $ch = curl_init();
  if($ch)
  {

    $fp = fopen($file, "w");
    if($fp)
    {
      if( !curl_setopt($ch, CURLOPT_URL, $url) )
      {
        fclose($fp); // to match fopen()
        curl_close($ch); // to match curl_init()
        return "FAIL: curl_setopt(CURLOPT_URL)";
      }
      if ((!ini_get('open_basedir') && !ini_get('safe_mode')) || $redirects < 1) {
        curl_setopt($ch, CURLOPT_USERAGENT, '"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        //curl_setopt($ch, CURLOPT_REFERER, 'http://domain.com/');
        if( !curl_setopt($ch, CURLOPT_HEADER, $curlopt_header)) return "FAIL: curl_setopt(CURLOPT_HEADER)";
        if( !curl_setopt($ch, CURLOPT_FOLLOWLOCATION, $redirects > 0)) return "FAIL: curl_setopt(CURLOPT_FOLLOWLOCATION)";
        if( !curl_setopt($ch, CURLOPT_FILE, $fp) ) return "FAIL: curl_setopt(CURLOPT_FILE)";
        if( !curl_setopt($ch, CURLOPT_MAXREDIRS, $redirects) ) return "FAIL: curl_setopt(CURLOPT_MAXREDIRS)";

        return curl_exec($ch);
    } else {
        curl_setopt($ch, CURLOPT_USERAGENT, '"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11');
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        //curl_setopt($ch, CURLOPT_REFERER, 'http://domain.com/');
        if( !curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false)) return "FAIL: curl_setopt(CURLOPT_FOLLOWLOCATION)";
        if( !curl_setopt($ch, CURLOPT_FILE, $fp) ) return "FAIL: curl_setopt(CURLOPT_FILE)";
        if( !curl_setopt($ch, CURLOPT_HEADER, true)) return "FAIL: curl_setopt(CURLOPT_HEADER)";
        if( !curl_setopt($ch, CURLOPT_RETURNTRANSFER, true)) return "FAIL: curl_setopt(CURLOPT_RETURNTRANSFER)";
        if( !curl_setopt($ch, CURLOPT_FORBID_REUSE, false)) return "FAIL: curl_setopt(CURLOPT_FORBID_REUSE)";
        curl_setopt($ch, CURLOPT_USERAGENT, '"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11');
    }
      // if( !curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true) ) return "FAIL: curl_setopt(CURLOPT_FOLLOWLOCATION)";
      // if( !curl_setopt($ch, CURLOPT_FILE, $fp) ) return "FAIL: curl_setopt(CURLOPT_FILE)";
      // if( !curl_setopt($ch, CURLOPT_HEADER, 0) ) return "FAIL: curl_setopt(CURLOPT_HEADER)";
      if( !curl_exec($ch) ) return "FAIL: curl_exec()";
      curl_close($ch);
      fclose($fp);
      return "SUCCESS: $file [$url]";
    }
    else return "FAIL: fopen()";
  }
  else return "FAIL: curl_init()";
}

How to downlod from this url: https://filehippo.com/download_mozilla-firefox-64/post_download/ — Saurin Dashadia, Jul 10 '20 at 13:38

score 4 · Answer 10 · answered Nov 07 '13 at 14:50

4

A PHP 4 & 5 Solution:

readfile() will not present any memory issues, even when sending large files, on its own. A URL can be used as a filename with this function if the fopen wrappers have been enabled.

http://php.net/manual/en/function.readfile.php

answered Nov 07 '13 at 14:50

Eric Leroy

1,796
1
16
29

1

This does not answer the question, because the question is about writing on disk not to the output buffer. – Lorenz Meyer Nov 23 '17 at 17:18

score 0 · Answer 11 · answered Sep 29 '20 at 16:10

0

Simple solution:

<?php 
exec('wget http://someurl/file.zip');

answered Sep 29 '20 at 16:10

webHasan

338
2
10

@Netwons make sure wget available in your server. – webHasan Nov 12 '20 at 09:05
wget available to system error ======> errorCode=1 SSL/TLS handshake failure: The TLS connection was non-properly terminated. – Netwons Nov 12 '20 at 12:39
or error ======> Connecting to www.you.com (www.you.com)|178.79.180.188|:443... connected. – Netwons Nov 12 '20 at 12:47

score 0 · Answer 12 · answered Nov 11 '20 at 08:53

0

best solution

install aria2c in system &

 echo exec("aria2c \"$url\"")

answered Nov 11 '20 at 08:53

Netwons

195
2
5

Download File to server from URL

12 Answers12

Downloader :

Linked

Related