I have a file named file.txt which is update by adding lines to it.
I am reading it by this code:
$fp = fopen("file.txt", "r");
$data = "";
while(!feof($fp))
{
$data .= fgets($fp, 4096);
}
echo $data;
and a huge number of lines appears. I just want to echo the last 5 lines of the file
How can I do that ?
The file.txt is like this:
11111111111111
22222222222
33333333333333
44444444444
55555555555555
66666666666
For a large file, reading all the lines into an array with file() is a bit wasteful. Here's how you could read the file and maintain a buffer of the last 5 lines:
$lines=array();
$fp = fopen("file.txt", "r");
while(!feof($fp))
{
$line = fgets($fp, 4096);
array_push($lines, $line);
if (count($lines)>5)
array_shift($lines);
}
fclose($fp);
You could optimize this a bit more with some heuristics about likely line length by seeking to a position, say, approx 10 lines from the end, and going further back if that doesn't yield 5 lines. Here's a simple implementation which demonstrates that:
//how many lines?
$linecount=5;
//what's a typical line length?
$length=40;
//which file?
$file="test.txt";
//we double the offset factor on each iteration
//if our first guess at the file offset doesn't
//yield $linecount lines
$offset_factor=1;
$bytes=filesize($file);
$fp = fopen($file, "r") or die("Can't open $file");
$complete=false;
while (!$complete)
{
//seek to a position close to end of file
$offset = $linecount * $length * $offset_factor;
fseek($fp, -$offset, SEEK_END);
//we might seek mid-line, so read partial line
//if our offset means we're reading the whole file,
//we don't skip...
if ($offset<$bytes)
fgets($fp);
//read all following lines, store last x
$lines=array();
while(!feof($fp))
{
$line = fgets($fp);
array_push($lines, $line);
if (count($lines)>$linecount)
{
array_shift($lines);
$complete=true;
}
}
//if we read the whole file, we're done, even if we
//don't have enough lines
if ($offset>=$bytes)
$complete=true;
else
$offset_factor*=2; //otherwise let's seek even further back
}
fclose($fp);
var_dump($lines);
Untested code, but should work:
$file = file("filename.txt");
for ($i = max(0, count($file)-6); $i < count($file); $i++) {
echo $file[$i] . "\n";
}
Calling max will handle the file being less than 6 lines.
If you're on a linux system you could do this:
$lines = `tail -5 /path/to/file.txt`;
Otherwise you'll have to count lines and take the last 5, something like:
$all_lines = file('file.txt');
$last_5 = array_slice($all_lines , -5);
function ReadFromEndByLine($filename,$lines)
{
/* freely customisable number of lines read per time*/
$bufferlength = 5000;
$handle = @fopen($filename, "r");
if (!$handle) {
echo "Error: can't find or open $filename<br/>\n";
return -1;
}
/*get the file size with a trick*/
fseek($handle, 0, SEEK_END);
$filesize = ftell($handle);
/*don't want to get past the start-of-file*/
$position= - min($bufferlength,$filesize);
while ($lines > 0) {
if ($err=fseek($handle,$position,SEEK_END)) { /* should not happen but it's better if we check it*/
echo "Error $err: something went wrong<br/>\n";
fclose($handle);
return $lines;
}
/* big read*/
$buffer = fread($handle,$bufferlength);
/* small split*/
$tmp = explode("\n",$buffer);
/*previous read could have stored a partial line in $aliq*/
if ($aliq != "") {
/*concatenate current last line with the piece left from the previous read*/
$tmp[count($tmp)-1].=$aliq;
}
/*drop first line because it may not be complete*/
$aliq = array_shift($tmp);
$read = count($tmp);
if ( $read >= $lines ) { /*have read too much!*/
$tmp2 = array_slice($tmp,$read-$n);
/* merge it with the array which will be returned by the function*/
$lines = array_merge($tmp2,$lines);
/* break the cycle*/
$lines = 0;
} elseif (-$position >= $filesize) { /* haven't read enough but arrived at the start of file*/
//get back $aliq which contains the very first line of the file
$lines = array_merge($aliq,$tmp,$lines);
//force it to stop reading
$lines = 0;
} else { /*continue reading...*/
//add the freshly grabbed lines on top of the others
$lines = array_merge($tmp,$lines);
$lines -= $read;
//next time we want to read another block
$position -= $bufferlength;
//don't want to get past the start of file
$position = max($position, -$filesize);
}
}
fclose($handle);
return $lines;
}
This will be fast for larger files but alot of code for a simple task, if there LARGE FILES, use this
ReadFromEndByLine('myFile.txt',6);
This is a common interview question. Here's what I wrote last year when I was asked this question. Remember that code you get on Stack Overflow is licensed with the Creative Commons Share-Alike with attribution required.
<?php
/**
* Demonstrate an efficient way to search the last 100 lines of a file
* containing roughly ten million lines for a sample string. This should
* function without having to process each line of the file (and without making
* use of the “tail” command or any external system commands).
* Attribution: https://stackoverflow.com/a/2961731/3389585
*/
$filename = '/opt/local/apache2/logs/karwin-access_log';
$searchString = 'index.php';
$numLines = 100;
$maxLineLength = 200;
$fp = fopen($filename, 'r');
$data = fseek($fp, -($numLines * $maxLineLength), SEEK_END);
$lines = array();
while (!feof($fp)) {
$lines[] = fgets($fp);
}
$c = count($lines);
$i = $c >= $numLines? $c-$numLines: 0;
for (; $i<$c; ++$i) {
if ($pos = strpos($lines[$i], $searchString)) {
echo $lines[$i];
}
}
This solution does make an assumption about the maximum line length. The interviewer asked me how I would solve the problem if I couldn't make that assumption, and had to accommodate lines that were potentially longer than any max length I chose.
I told him that any software project has to make certain assumptions, but I could test if $c was less than the desired number of lines, and if it isn't, fseek() back further incrementally (doubling each time) until we do get enough lines.
Opening large files with file() can generate a large array, reserving a considerable chunk of memory.
You can reduce the memory cost with SplFileObject since it iterates through each line.
Use the seek method (of seekableiterator) to fetch the last line. You should then subtract the current key value by 5.
To obtain the last line, use PHP_INT_MAX. (Yes, this is a workaround.)
$file = new SplFileObject('large_file.txt', 'r');
$file->seek(PHP_INT_MAX);
$last_line = $file->key();
$lines = new LimitIterator($file, $last_line - 5, $last_line);
print_r(iterator_to_array($lines));
The most of options here suppose to read file into the memory and then work with rows. This wouldnt be a good idea, if the file too large
I think the best way is to use some OS-utility, like 'tail' in unix.
exec('tail -3 /logs/reports/2017/02-15/173606-arachni-2415.log', $output);
echo $output;
// 2017-02-15 18:03:25 [*] Path Traversal: Analyzing response ...
// 2017-02-15 18:03:27 [*] Path Traversal: Analyzing response ...
// 2017-02-15 18:03:27 [*] Path Traversal: Analyzing response ...
Here is FAST method for LARGE files with LOW memory cost - I develop Wallace Maxters answer (if you want to upvote - do it on his answer) by wrap his code inside handy function and add reverse feature
function readLastLines($filename, $num, $reverse = false)
{
$file = new \SplFileObject($filename, 'r');
$file->seek(PHP_INT_MAX);
$last_line = $file->key();
$lines = new \LimitIterator($file, $last_line - $num, $last_line);
$arr = iterator_to_array($lines);
if($reverse) $arr = array_reverse($arr);
return implode('',$arr);
}
// use it by
$lines = readLastLines("file.txt", 5) // return string with 5 last lines
This doesn't use file() so it will be more efficient for huge files;
<?php
function read_backward_line($filename, $lines, $revers = false)
{
$offset = -1;
$c = '';
$read = '';
$i = 0;
$fp = @fopen($filename, "r");
while( $lines && fseek($fp, $offset, SEEK_END) >= 0 ) {
$c = fgetc($fp);
if($c == "\n" || $c == "\r"){
$lines--;
if( $revers ){
$read[$i] = strrev($read[$i]);
$i++;
}
}
if( $revers ) $read[$i] .= $c;
else $read .= $c;
$offset--;
}
fclose ($fp);
if( $revers ){
if($read[$i] == "\n" || $read[$i] == "\r")
array_pop($read);
else $read[$i] = strrev($read[$i]);
return implode('',$read);
}
return strrev(rtrim($read,"\n\r"));
}
//if $revers=false function return->
//line 1000: i am line of 1000
//line 1001: and i am line of 1001
//line 1002: and i am last line
//but if $revers=true function return->
//line 1002: and i am last line
//line 1001: and i am line of 1001
//line 1000: i am line of 1000
?>
This function will work for REALLY large files under 4GB. The speed comes from reading a big chunk of data instead of 1 byte at a time and counting lines.
// Will seek backwards $n lines from the current position
function seekLineBackFast($fh, $n = 1){
$pos = ftell($fh);
if ($pos == 0)
return false;
$posAtStart = $pos;
$readSize = 2048*2;
$pos = ftell($fh);
if(!$pos){
fseek($fh, 0, SEEK_SET);
return false;
}
// we want to seek 1 line before the line we want.
// so that we can start at the very beginning of the line
while ($n >= 0) {
if($pos == 0)
break;
$pos -= $readSize;
if($pos <= 0){
$pos = 0;
}
// fseek returns 0 on success and -1 on error
if(fseek($fh, $pos, SEEK_SET)==-1){
fseek($fh, 0, SEEK_SET);
break;
}
$data = fread($fh, $readSize);
$count = substr_count($data, "\n");
$n -= $count;
if($n < 0)
break;
}
fseek($fh, $pos, SEEK_SET);
// we may have seeked too far back
// so we read one line at a time forward
while($n < 0){
fgets($fh);
$n++;
}
// just in case?
$pos = ftell($fh);
if(!$pos)
fseek($fh, 0, SEEK_SET);
// check that we have indeed gone back
if ($pos >= $posAtStart)
return false;
return $pos;
}
After running above function, you can just do fgets() in a loop to read each line at a time from $fh.
PHP's file() function reads the whole file into an array. This solution requires the least amount of typing:
$data = array_slice(file('file.txt'), -5);
foreach ($data as $line) {
echo $line;
}
You can use my small helper library (2 functions)
https://github.com/jasir/file-helpers
Then just use:
//read last 5 lines
$lines = \jasir\FileHelpers\FileHelpers::readLastLines($pathToFile, 5);
Least amount of ram, and outputs well. I agree with Paul Dixon...
$lines=array();
$fp = fopen("userlog.txt", "r");
while(!feof($fp))
{
$line = fgets($fp, 4096);
array_push($lines, $line);
if (count($lines)>25)
array_shift($lines);
}
fclose($fp);
while ($a <= 10) {
$a++;
echo "<br>".$lines[$a];
}
$dosya = "../dosya.txt";
$array = explode("\n", file_get_contents($dosya));
$reversed = array_reverse($array);
for($x = 0; $x < 6; $x++)
{
echo $reversed[$x];
}
I've tested this one. It works for me.
function getlast($filename,$linenum_to_read,$linelength){
// this function takes 3 arguments;
if (!$linelength){ $linelength = 600;}
$f = fopen($filename, 'r');
$linenum = filesize($filename)/$linelength;
for ($i=1; $i<=($linenum-$linenum_to_read);$i++) {
$data = fread($f,$linelength);
}
echo "<pre>";
for ($j=1; $j<=$linenum_to_read+1;$j++) {
echo fread($f,$linelength);
}
echo "</pre><hr />The filesize is:".filesize("$filename");
}
getlast("file.txt",6,230);
?>
Here is my solution:
/**
*
* Reads N lines from a file
*
* @param type $file path
* @param type $maxLines Count of lines to read
* @param type $reverse set to true if result should be reversed.
* @return string
*/
public function readLinesFromFile($file, $maxLines, $reverse=false)
{
$lines = file($file);
if ($reverse) {
$lines = array_reverse($lines);
}
$tmpArr = array();
if ($maxLines > count($lines))
exit("\$maxLines ist größer als die Anzahl der Zeilen in der Datei.");
for ($i=0; $i < $maxLines; $i++) {
array_push($tmpArr, $lines[$i]);
}
if ($reverse) {
$tmpArr = array_reverse($tmpArr);
}
$out = "";
for ($i=0; $i < $maxLines; $i++) {
$out .= $tmpArr[$i] . "</br>";
}
return $out;
}
this is read last 10 line from text file
$data = array_slice(file('logs.txt'),10);
foreach ($data as $line)
{
echo $line."<br/>";
}
If your lines are separated by a CR or LF you would try exploding your $data variable:
$lines = explode("\n", $data);
$lines should end up being an array and you can work out the number of records using sizeof() and just get the last 5.
