Batch to remove range of lines with start and stop delimiters from text file

Question

I know batch isn't the best vehicle for this but my requirements dictate that I keep it.

I have text that looks like the following (it also has blank lines):

Line AAA text

Line BBB text
! ***@@@ START
Body text here
! ***@@@ END
Line XXX
Line YYY
!Comment Line etc

I want to remove the ! ***@@@ START and END lines and everything in between and then save over the original file.

I found and modified the code below but it strips out my blank lines and the ! characters.

@echo off
setlocal enabledelayedexpansion
set "sourceFile=c:\temp\startfile.txt"
set "tempFile=c:\temp\tempfile.txt"
set "StartPhrase=! ***@@@ START"
set "EndPhrase=! ***@@@ END"
set /a lineNum=0

REM check file for search phrase, store line as refLine
FOR /F "delims=" %%i IN (%sourceFile%) DO (
    set /a lineNum+=1
    echo !lineNum! = "%%i"
    if "%%i" == "%StartPhrase%" (
        echo Found "%StartPhrase%" on line !lineNum!
        set /a StartrefLine=!lineNum!
    )
        if "%%i" == "%EndPhrase%" (
        echo Found "%EndPhrase%" on line !lineNum!
        set /a EndrefLine=!lineNum!
    )
)

REM make backup
copy "%sourceFile%" "%sourceFile%-%DATE:/=-% %TIME::=-%.txt"

echo. 2>%tempFile%

REM Rewrite file 
set /a lineNum=0
    set /a lowEnd=%StartrefLine%
    echo "Set low end to %lowEnd%"
    set /a highEnd=%EndrefLine%
    echo "Set high end to %highEnd%"
FOR /F "delims=" %%i IN (%sourceFile%) DO (
    set /a lineNum+=1
    if !lineNum! GTR %lowEnd% (
        if !lineNum! LSS %highEnd% (
           echo "Skipping line #!lineNum!"
        )
    )
    if !lineNum! LSS %lowEnd% (
        echo "Writing Line !lineNum! %%i to temp file..."
        echo %%i >> %tempFile%
    )

    if !lineNum! GTR %highEnd% (
        echo "Writing Line !lineNum! %%i to temp file..."
        echo %%i >> %tempFile%
    )
)

REM get target filename only 
for %%F in ("%sourceFile%") do set fname=%%~nxF
REM del original file and rename tempfile
echo "Deleting original file..."
echo Y | del "%sourceFile%"
echo "Renaming %tempFile% to %fname%"
ren "%tempFile%" "%fname%"

aschipfl · Accepted Answer · 2020-12-09T15:58:53.290

A possible and quite simple way is to let the particular line markers toggle a flag that indicates whether or not the currently iterated line is to be output:

@echo off
setlocal EnableExtensions DisableDelayedExpansion

rem // Define constants here:
set "_FILE=C:\TEMP\startfile.txt"
set "_TMPF=%TEMP%\%~n0_%RANDOM%.tmp"
set "_START=! ***@@@ START"
set "_END=! ***@@@ END"

rem // Initialise flag:
set "FLAG=#"
rem // Write to temporary file:
> "%_TMPF%" (
    rem /* Loop through lines of input file; `findstr` precedes each line with
    rem    line number plus `:`, so they do not appear empty to `for /F`: */
    for /F "delims=" %%L in ('findstr /N "^" "%_FILE%"') do (
        rem // Store current line string (with line number prefix) to variable:
        set "LINE=%%L"
        rem // Toggle delayed expansion to avoid loss of `!`:
        setlocal EnableDelayedExpansion
        rem // Remove line number prefix to retrieve original line string:
        set "LINE=!LINE:*:=!"
        rem // Check contents of current line:
        if "!LINE!"=="!_START!" (
            endlocal & set "FLAG="
        ) else if "!LINE!"=="!_END!" (
            endlocal & set "FLAG=#"
        ) else (
            rem // Check state of flag for outputting:
            if defined FLAG echo(!LINE!
            endlocal
        )
    )
) && (
    rem // Move temporary file onto target file:
    move /Y "%_TMPF%" "%_FILE%"
)

endlocal
exit /B

score 0 · Answer 2 · answered Dec 09 '20 at 15:03

This task could be done with worst script interpreter available for this task with following code:

@echo off
setlocal EnableExtensions DisableDelayedExpansion
set "SourceFile=C:\Temp\startfile.txt"
if not exist "%SourceFile%" goto :EOF
set "TempFile=%SourceFile%.tmp"
set "PrintLines=1"

(for /F "delims=" %%I in ('%SystemRoot%\System32\findstr.exe /N /R "^" "%SourceFile%"') do (
    set "Line=%%I"
    setlocal EnableDelayedExpansion
    if defined PrintLines (
        if not "!Line: ***@@@ START=!" == "!Line!" (
            endlocal
            set "PrintLines="
        ) else (
            echo(!Line:*:=!
            endlocal
        )
    ) else (
        if not "!Line: ***@@@ END=!" == "!Line!" (
            endlocal
            set "PrintLines=1"
        ) else endlocal
    )
))>"%TempFile%"

move /Y "%TempFile%" "%SourceFile%"
if exist "%TempFile%" del "%TempFile%"
endlocal

for /F results in ignoring all empty lines. For that reason command findstr is used to output all lines with line number and colon at beginning in a separate command process started with %ComSpec% /c in background. So there is no empty line anymore in captured output of findstr.

Each captured line starting with a digit in range 1 to 9 is assigned completely to the loop variable I because of using option delims= to define an empty list of string delimiters to disable the default line splitting behavior on normal spaces and horizontal tabs.

The line with line number and colon at beginning is assigned to the environment variable Line while delayed environment variable expansion is disabled. This is important as otherwise the Windows command processor cmd.exe processing the batch file would parse the command line set "Line=%%I" after having replaced %%I by current line with enabled delayed expansion and would interpret each ! as beginning/end of a delayed expanded environment variable.

Then the delayed expansion is enabled for further processing the line depending on printing lines currently enabled as by default until a line is found containing the string ***@@@ START. Next the environment variable PrintLines is undefined until a line is found containing the string ***@@@ END on which PrintLines is defined again for the following lines.

Please read second half of this answer for details about the commands SETLOCAL and ENDLOCAL.

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

del /?
echo /?
endlocal /?
findstr /?
for /?
if /?
move /?
set /?
setlocal /?

I absolutely love the introductory sentence! ;-) – aschipfl Dec 09 '20 at 15:09 — aschipfl, Dec 09 '20 at 15:09

score 0 · Answer 3 · answered Dec 09 '20 at 15:54

@ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q65218525.txt"
SET "outfile=%destdir%\outfile.txt"

SET "blockstart=! ***@@@ START"
SET "blockend=! ***@@@ END"

SET "repro=Y"

(

FOR /f "tokens=1*delims=:" %%a IN ('findstr /n /r ".*" "%filename1%"') DO (
 IF /i "%%b"=="%blockstart%" SET "repro=" 
 IF /i "%%b"=="%blockend%" (SET "repro=Y") ELSE (IF DEFINED repro IF "%%b"=="" (ECHO.) ELSE (ECHO %%b))
)
)>"%outfile%"

GOTO :EOF

You would need to change the settings of sourcedir and destdir to suit your circumstances. The listing uses a setting that suits my system.

I used a file named q65218525.txt containing your data for my testing.

Produces the file defined as %outfile%

Since your code saves the original file under a new name and creates a new file with the old name but your requirement seems to be to simply overwrite the old file, I'll leave that part as an exercise.

First, define the strings involved, and a flag (which I've called repro) which is initialised to reproduce the data.

Then use findstr with the /N option to prefix each line, including blank lines, with number: which is then parsed by the for/r to put the number in %%a and the text in %%b.

Check %%b for the target strings and frob the repro flag as appropriate to select whether to reproduce the line read or not.

I thought also about this simple solution, but it has the disadvantage that all `:` at beginning of a line in the text file are removed by __FOR__ because of `delims=:`. The example file contents in question does not contain a line with one or more `:` at beginning of the line, but that is most likely just because of questioner not knowing that this is another special use case on processing the lines of the text file. — Mofi, Dec 09 '20 at 16:04
@Mofi - yes, I should have mentioned that. Should be fine if no data lines start `:` though. — Magoo, Dec 09 '20 at 16:08

score 0 · Answer 4 · answered Dec 10 '20 at 02:14

Another way to do this would be using PowerShell. Invoke it from the command line or a .bat file script using:

powershell -NoLogo -NoProfile -File '.\Slice-File.ps1'

=== Slice-File.ps1

$SourceFile = 'C:\src\t\Slice-File.txt'
$TempFile = New-TemporaryFile

$WriteIt = $true
Get-Content -Path '.\Slice-File.txt' |
    ForEach-Object {
        if ($_ -eq '! ***@@@ START') { $WriteIt = $false }
        if ($WriteIt) { $_ }
        if ($_ -eq '! ***@@@ END') { $WriteIt = $true }
    } |
    Out-File -FilePath $TempFile -Encoding ascii

Remove-Item -Path $SourceFile
Move-Item -Path $TempFile -Destination $SourceFile

Batch to remove range of lines with start and stop delimiters from text file

4 Answers4