CODEWITHSUNDEEP
vb.netmerge
cafescott
cafescott

Reputation: 31

VB.NET merge large files

I wish to use VB.NET to replace a batch file that merged large files. This is because the batch file just hangs when running as a scheduled job. (The batch file works fine everywhere else; but not as a SQL job.)

The batch file performed these steps: copy "\server\network location\large_file1.txt" + "\server\network location\large_file2.txt" + "\server\network location\large_file3.txt"

The files are quite large and I don't wish to open any of them. I think the server running the job might run out of memory.

Can this be done in VB.NET?

thanks!

UPDATE: Solution from user's comment:

Dim myBuffer(4096) As Byte 
Dim fsdest As System.IO.FileStream 
Dim fsSecondFile As System.IO.FileStream fsdest = New System.IO.FileStream(strDestinationfile, System.IO.FileMode.Append) 

fsSecondFile = New System.IO.FileStream(strSecondFile, IO.FileMode.Open, IO.FileAccess.Read) 

Do While fsSecondFile.Read(myBuffer, 0, myBuffer.Length) > 0 
    fsdest.Write(myBuffer, 0, 4095) 
Loop 

fsdesc.close() 
fsSecondFile.close()

Upvotes: 3

Views: 2782

Answers (2)

Cris70
Cris70

Reputation: 339

I have been looking for something like this. However, the solution proposed in the original message does not work correctly (it corrupts the resulting file).

I corrected the code and I am posting it here in case somebody is in need of this. Also my code is a bit different in that it doesn't modify the first file, but instead it creates a third file.

    Public Function JoinFiles(strFile1 As String, strFile2 As String, fileOutPath As String) As Boolean
    Dim myBuffer(4096) As Byte
    Dim fsdest As System.IO.FileStream
    Dim fsFirstFile, fsSecondFile As System.IO.FileStream
    Dim BytesToRead As Integer = 0
    Dim n As Integer

    Try
        fsdest = New System.IO.FileStream(fileOutPath, System.IO.FileMode.Append)

        fsFirstFile = New System.IO.FileStream(strFile1, IO.FileMode.Open, IO.FileAccess.Read)

        BytesToRead = CType(fsFirstFile.Length, Integer)

        While (BytesToRead > 0)
            n = fsFirstFile.Read(myBuffer, 0, myBuffer.Length)
            If n = 0 Then Exit While
            fsdest.Write(myBuffer, 0, n)
            BytesToRead = BytesToRead - n
        End While
        fsFirstFile.Close()


        fsSecondFile = New System.IO.FileStream(strFile2, IO.FileMode.Open, IO.FileAccess.Read)

        BytesToRead = CType(fsSecondFile.Length, Integer)

        While (BytesToRead > 0)
            n = fsSecondFile.Read(myBuffer, 0, myBuffer.Length)
            If n = 0 Then Exit While
            fsdest.Write(myBuffer, 0, n)
            BytesToRead = BytesToRead - n
        End While
        fsSecondFile.Close()

        fsdest.Close()

        Return True
    Catch ex As Exception
        Return False
    End Try
End Function

This code could be enhanced by editing the "catch" to close the streams if they are still open. Left as exercise :-)

Upvotes: 0

Grant Thomas
Grant Thomas

Reputation: 45083

No, you can't merge files without opening any of them. You will need to 'open them' at some level in order to access the data, whether this is by using System.IO provided features of the Framework, or by reading sectors of the disc with native APIs, you're still going to need to 'open'/read the data.

The solution would be to open the files, yes, but using streams and buffers so that you're doing some memory management and not just bloating the system with a 'read all' call.

So,..

  • Define a buffer (new Byte(4096))
  • open a new, empty file (Using ... new System.IO.File.Open(...))
  • for each file to be joined...
    • open that file (Using ... new System.IO.File.Open(...))
    • loop through using the Read method of FileStream to sequentially populate the buffer until it returns <= 0
    • write the buffer contents to the new file using Filestream's Write.

Upvotes: 4

Related Questions