DOS batch FOR loop with FIND.exe is stripping out blank lines?

Question

This DOS batch script is stripping out the blank lines and not showing the blank lines in the file even though I am using the TYPE.exe command to convert the file to make sure the file is ASCII so that the FIND command is compatible with the file. Can anyone tell me how to make this script include blank lines?

@ECHO off
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE.exe "build.properties" ^| FIND.exe /V ""`) DO (
  ECHO --%%A--
)
pause

Answer 1

That is the designed behavior of FOR /F - it never returns blank lines. The work around is to use FIND or FINDSTR to prefix the line with the line number. If you can guarantee no lines start with the line number delimiter, then you simply set the appropriate delimiter and keep tokens 1* but use only the 2nd token.

::preserve blank lines using FIND, assume no line starts with ]
::long lines are truncated
for /f "tokens=1* delims=]" %%A in ('type "file.txt" ^| find /n /v ""') do echo %%B

::preserve blank lines using FINDSTR, assume no line starts with :
::long lines > 8191 bytes are lost
for /f "tokens=1* delims=:" %%A in ('type "file.txt" ^| findstr /n "^"') do echo %%B

::FINDSTR variant that preserves long lines
type "file.txt" > "file.txt.tmp"
for /f "tokens=1* delims=:" %%A in ('findstr /n "^" "file.txt.tmp"') do echo %%B
del "file.txt.tmp"

I prefer FINDSTR - it is more reliable. For example, FIND can truncate long lines - FINDSTR does not as long as it reads directly from a file. FINDSTR does drop long lines when reading from stdin via pipe or redirection.

If the file may contain lines that start with the delimiter, then you need to preserve the entire line with the line number prefix, and then use search and replace to remove the line prefix. You probably want delayed expansion off when transferring the %%A to an environment variable, otherwise any ! will be corrupted. But later within the loop you need delayed expansion to do the search and replace.

::preserve blank lines using FIND, even if a line may start with ]
::long lines are truncated
for /f "delims=" %%A in ('type "file.txt" ^| find /n /v ""') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*]=!"
  echo(!ln!
  endlocal
)

::preserve blank lines using FINDSTR, even if a line may start with :
::long lines >8191 bytes are truncated
for /f "delims=*" %%A in ('type "file.txt" ^| findstr /n "^"') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*:=!"
  echo(!ln!
  endlocal
)

::FINDSTR variant that preserves long lines
type "file.txt" >"file.txt.tmp"
for /f "delims=*" %%A in ('findstr /n "^" "file.txt.tmp"') do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*:=!"
  echo(!ln!
  endlocal
)
del "file.txt.tmp"

If you don't need to worry about converting the file to ASCII, then it is more efficient to drop the pipe and let FIND or FINDSTR open the file specified as an argument, or via redirection.

There is another work around that completely bypasses FOR /F during the read process. It looks odd, but it is more efficient. There are no restrictions with using delayed expansion, but unfortunately it has other limitations.

1) lines must be terminated by <CR><LF> (this will not be a problem if you do the TYPE file conversion)

2) lines must be <= 1021 bytes long (disregarding the <CR><LF>)

3) any trailing control characters are stripped from each line.

4) it must read from a file - you can't use a pipe. So in your case you will need to use a temp file to do your to ASCII conversion.

setlocal enableDelayedExpansion
type "file.txt">"file.txt.tmp"
for /f %%N in ('find /c /v "" ^<"file.txt.tmp"') do set cnt=%%N
<"file.txt.tmp" (
  for /l %%N in (1 1 %cnt%) do(
    set "ln="
    set /p "ln="
    echo(!ln!
  )
)
del "file.txt.tmp"

Answer 2

I wrote a very simple program that may serve as replacement for FIND and FINDSTR commands when they are used for this purpose. My program is called PIPE.COM and it just insert a blank space in empty lines, so all the lines may be directly processed by FOR command with no further adjustments (as long as the inserted space don't cares). Here it is:

@ECHO off
if not exist pipe.com call :DefinePipe
FOR /F "USEBACKQ delims=" %%A IN (`pipe ^< "build.properties"`) DO (
  ECHO(--%%A--
)
pause
goto :EOF

:DefinePipe
setlocal DisableDelayedExpansion
set pipe=´)€ì!Í!ŠÐŠà€Ä!€ü.t2€ü+u!:æu8²A€ê!´#€ì!Í!².€ê!´#€ì!Í!²+€ê!´#€ì!Í!Šò€Æ!´,€ì!Í!"Àu°´LÍ!ëÒ
setlocal EnableDelayedExpansion
echo !pipe!>pipe.com
exit /B

EDIT: Addendum as answer to new comment

The code at :DefinePipe subroutine create a 88 bytes program called pipe.com, that basically do a process equivalent to this pseudo-Batch code:

set "space= "
set line=
:nextChar
   rem Read just ONE character
   set /PC char=
   if %char% neq %NewLine% (
      rem Join new char to current line
      set line=%line%%char%
   ) else (
      rem End of line detected
      if defined line (
         rem Show current line
         echo %line%
         set line=
      ) else (
         rem Empty line: change it by one space
         echo %space%
      )
   )
goto nextChar

This way, empty lines in the input file are changed by lines with one space, so FOR /F command not longer omit they. This works "as long as the inserted space don't cares" as I said in my answer.

Note that the pipe.com program does not work in 64-bits Windows versions.

Antonio

Answer 3

Output lines including blank lines

Here's a method I developed for my own use.

Save the code as a batch file say, SHOWALL.BAT and pass the source file as a command line parameter.

Output can be redirected or piped.

@echo off

for /f "tokens=1,* delims=]" %%a in ('find /n /v "" ^< "%~1"') do echo.%%ba

exit /b

EXAMPLES:

showall source.txt

showall source.txt >destination.txt

showall source.txt | FIND "string"

An oddity is the inclusion of the '^<' (redirection) as opposed to just doing the following:

for /f "tokens=1,* delims=]" %%a in ('find /n /v "" "%~1"') do echo.%%ba

By omitting the redirection, a leading blank line is output.

Answer 4

As mentioned in this answer to the above question, it doesn't seem that lines are skipped by default using for /f in (at least) Windows XP (Community - Please update this answer by testing the below batch commands on your version & service pack of Windows).


   EDIT: Per Jeb's comment below, it seems that the ping command, in at least Windows XP, is
   causing for /f to produce <CR>'s instead of blank lines (If someone knows specifically why, would
   appreciate it if they could update this answer or comment).

   As a workaround, it seems that the second default delimited token (<space> / %%b in the example)
   returns as blank, which worked for my situation of eliminating the blank lines by way of an "parent"
   if conditional on the second token at the start of the for /f, like this:

   for /f "tokens=1,2*" %%a in ('ping -n 1 google.com') do (
      if not "x%%b"=="x" (
         {do things with non-blank lines}
      )
   )

Using the below code:

@echo off
systeminfo | findstr /b /c:"OS Name" /c:"OS Version" 
echo.&echo.
ping -n 1 google.com
echo.&echo.
for /f %%a in ('ping -n 1 google.com') do ( echo "%%a" )
echo.&echo.&echo --------------&echo.&echo.
find /?
echo.&echo.
for /f %%a in ('find /?') do ( echo "%%a" )
echo.&echo.
pause

.... the following is what I see on Windows XP, Windows 7 and Windows 2008, being the only three versions & service packs of Windows I have ready access to:

Answer 5

Thanks to dbenham, this works, although it is slightly different than his suggestion:

::preserve blank lines using FIND, no limitations
for /f "USEBACKQ delims=" %%A in (`type "file.properties" ^| find /V /N ""`) do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  set "ln=!ln:*]=!"
  echo(!ln!
  endlocal
)