Encode a FileStream to base64 with c#

Question

I know how to encode / decode a simple string to / from base64.

But how would I do that if the data is already been written to a FileStream object. Let's say I have only access to the FileStream object not to the previously stored original data in it. How would I encode a FileStream to base64 before I flush the FileStream to a file.

Ofc I could just open my file and encode / decode it after I have written the FileStream to the file, but I would like to do this all in one single step without doing two file operations one after another. The file could be larger and it would also take double time to load, encode and save it again after it was just saved a short time before.

Maybe someone of you knows a better solution? Can I convert the FileStream to a string, encode the string and then convert the string back to a FileStream for example or what would I do and how would such a code look like?

Answer 1

An easy one as an extension method

public static class Extensions
{
    public static Stream ConvertToBase64(this Stream stream)
    {
        byte[] bytes;
        using (var memoryStream = new MemoryStream())
        {
            stream.CopyTo(memoryStream);
            bytes = memoryStream.ToArray();
        }

        string base64 = Convert.ToBase64String(bytes);
        return new MemoryStream(Encoding.UTF8.GetBytes(base64));
    }
}

Answer 2

When dealing with large streams, like a file sized over 4GB - you don't want to load the file into memory (as a Byte[]) because not only is it very slow, but also may cause a crash as even in 64-bit processes a Byte[] cannot exceed 2GB (or 4GB with gcAllowVeryLargeObjects).

Fortunately there's a neat helper in .NET called ToBase64Transform which processes a stream in chunks. For some reason Microsoft put it in System.Security.Cryptography and it implements ICryptoTransform (for use with CryptoStream), but disregard that ("a rose by any other name...") just because you aren't performing any cryprographic tasks.

You use it with CryptoStream like so:

using System.Security.Cryptography;
using System.IO;

//

using( FileStream   inputFile    = new FileStream( @"C:\VeryLargeFile.bin", FileMode.Open, FileAccess.Read, FileShare.None, bufferSize: 1024 * 1024, useAsync: true ) ) // When using `useAsync: true` you get better performance with buffers much larger than the default 4096 bytes.
using( CryptoStream base64Stream = new CryptoStream( inputFile, new ToBase64Transform(), CryptoStreamMode.Read ) )
using( FileStream   outputFile   = new FileStream( @"C:\VeryLargeBase64File.txt", FileMode.CreateNew, FileAccess.Write, FileShare.None, bufferSize: 1024 * 1024, useAsync: true ) )
{
    await base64Stream.CopyToAsync( outputFile ).ConfigureAwait(false);
}

Answer 3

You may try something like that:

    public Stream ConvertToBase64(Stream stream)
    {
        Byte[] inArray = new Byte[(int)stream.Length];
        Char[] outArray = new Char[(int)(stream.Length * 1.34)];
        stream.Read(inArray, 0, (int)stream.Length);
        Convert.ToBase64CharArray(inArray, 0, inArray.Length, outArray, 0);
        return new MemoryStream(Encoding.UTF8.GetBytes(outArray));
    }

Answer 4

You can also encode bytes to Base64. How to get this from a stream see here: How to convert an Stream into a byte[] in C#?

Or I think it should be also possible to use the .ToString() method and encode this.

Answer 5

A simple Stream extension method would do the job:

public static class StreamExtensions
{
    public static string ConvertToBase64(this Stream stream)
    {
        var bytes = new Byte[(int)stream.Length];

        stream.Seek(0, SeekOrigin.Begin);
        stream.Read(bytes, 0, (int)stream.Length);

        return Convert.ToBase64String(bytes);
    }
}

The methods for Read (and also Write) and optimized for the respective class (whether is file stream, memory stream, etc.) and will do the work for you. For simple task like this, there is no need of readers, and etc.

The only drawback is that the stream is copied into byte array, but that is how the conversion to base64 via Convert.ToBase64String works unfortunately.

Answer 6

Since the file will be larger, you don't have very much choice in how to do this. You cannot process the file in place since that will destroy the information you need to use. You have two options that I can see:

1. Read in the entire file, base64 encode, re-write the encoded data.
2. Read the file in smaller pieces, encoding as you go along. Encode to a temporary file in the same directory. When you are finished, delete the original file, and rename the temporary file.

Of course, the whole point of streams is to avoid this sort of scenario. Instead of creating the content and stuffing it into a file stream, stuff it into a memory stream. Then encode that and only then save to disk.