Batch File: Merging same-name text files from text file

Question

I've created a list of file-paths to particular text files using

dir /b /s "xinfo.txt" > index.txt

All of these text files are CSV's and I would like to merge/append all of them together to create a new text file called "sourcing_directory".

I've tried the method mentioned in http://www.sidesofmarch.com/index.php/archive/2004/03/30/using-the-for-command-to-copy-files-listed-in-a-text-file/ which uses:

for /f "delims=" %%i in (filelist.txt) do echo D|xcopy "\\server\share\folder\%%i" "c:\temp\%%i" /i /z /y

To copy all the text files into one folder so I can just use Copy to merge all of them. This however doesn't seem to work either because of the way the index.txt has been structured or the fact that all the files have the same name.

There is no need to copy the files into a temporary directory. Anyway, is there a special order how the files are to be joined? Do the files contain headers which need to be removed? Note that the `copy` command is capable of concatenating multiple files into one... — aschipfl, May 31 '18 at 21:03
The CSV's don't have headers so they don't need to be joined in any order. — Ian D, May 31 '18 at 21:13
Refer also to [this related answer](https://stackoverflow.com/a/45083935) of the question [Loop through folders in subdirectories and combine text files](https://stackoverflow.com/q/45081578)... — aschipfl, Jun 01 '18 at 13:12

aschipfl · Accepted Answer · 2018-06-01T14:45:49.077

To merge multiple files, you could do this:

rem // Prepare empty file:
> "merged.csv" rem/
rem // Merge the files:
for /F "delims=" %%F in ('dir /S /B "xinfo.txt"') do (
    copy /Y "merged.csv"+"%%F" "merged.csv" /B
)

This does not check the file contents by any means, so for your CSV input data, each input file is terminated by a line-break to get a valid output CSV file.

If this cannot be guaranteed, you need to read each file by for /F, which ignores empty lines, which and also reads the last line even if not terminated by a line-break:

rem // Write to output file:
> "merged.csv" (
    for /F "delims=" %%F in ('dir /S /B "xinfo.txt"') do (
        rem // Read input files line by line:
        for /F usebackq^ delims^=^ eol^= %%L in ("%%F") do echo(%%L
    )
)

To use a header file, try this:

rem // Copy header file:
copy /Y "header.csv" "merged.csv"
rem // Merge the files:
for /F "delims=" %%F in ('dir /S /B "xinfo.txt"') do (
    copy /Y "merged.csv"+"%%F" "merged.csv" /B
)

Or this:

rem // Copy header file:
copy /Y "header.csv" "merged.csv"
rem // Append to output file:
>> "merged.csv" (
    for /F "delims=" %%F in ('dir /S /B "xinfo.txt"') do (
        rem // Read input files line by line:
        for /F usebackq^ delims^=^ eol^= %%L in ("%%F") do echo(%%L
    )
)

This worked perfectly! If I did have one file that I wanted to be the first line (say a text document with headers), how would I modify the code to add that? — Ian D, Jun 01 '18 at 13:30

Batch File: Merging same-name text files from text file

1 Answers1