CODEWITHSUNDEEP
vb.netguid
Mary
Mary

Reputation: 31

Efficiently storing a GUID in text?

Our app requires us to write out a large list of key, GUID value pairs for export to a plain text file.

6004, aa20dc0b-1e10-4efa-b683-fc98820b0fba

There will be potentially millions of these GUIDs, and there may not be a lot of room on the device where the file will be written, so is there a more efficient way to print the GUID to keep the file size down?

I experimented with Hex encoding

Dim g As Guid = Guid.NewGuid
Dim sb As New Text.StringBuilder
For Each b As Byte In g.ToByteArray
  sb.Append(String.Format("{0:X2}", b))
Next
Console.WriteLine(sb.ToString)

this got it down to 32 chars, each line is a bit shorter:

9870, EBB7EF914C29A6459A34EDCB61EB8C8F

Are there any other approaches to write the GUIDs to the file that are more efficient?

Upvotes: 3

Views: 992

Answers (1)

Chicken
Chicken

Reputation: 428

I agree with previous comment, SQL would be ideal (or another DB) text files can be very unstable. Just done some quick testing iterating over millions of GUIDs and storing in text files.

Here is the outcome, basically anything more than 1 million guids would start to cause issues. You can store safely 10 million, but the file would struggle to open (on an average PC) as its > 150mb see below:

GUID Testing 100-10mill

The code I used is below for you if wanted to try out. I know its not perfect, but gives you an idea of where the time goes. Main conclusions are to append the files with a bulk append, don't try to append each GUID individually if you can help it. This saves a large amount of processing and time!

Even if you convert to other formats like base or binary, I think you will still get similar variations on the file size, don't forget, you are still just saving as a text file, binary will no doubt be a larger file size due to string length!

Hope this helps comment back if you need anything!

Chicken

Dim Report_List As List(Of String)

Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    Report_List = New List(Of String)
    Dim Zeros As Integer = 2
    Do While Zeros < 8
        Dim sw As Stopwatch = New Stopwatch
        sw.Start()
        Dim CountName As String = "1" & Microsoft.VisualBasic.Left("00000000000000000000000", Zeros)
        Dim CountNum As Integer = CInt(CountName)

        Dim Master_String As System.Text.StringBuilder = New System.Text.StringBuilder

        For i = 1 To CountNum
            Dim g As Guid = Guid.NewGuid
            'Dim sb As New System.Text.StringBuilder
            'For Each b As Byte In g.ToByteArray
            '    sb.Append(String.Format("{0:X2}", b))
            'Next
            'Master_String.AppendLine(sb.ToString)
            'Master_String.AppendLine(sb.ToString)
            Master_String.AppendLine(Convert.ToBase64String(g.ToByteArray))
            i += 1
        Next

        Using sr As StreamWriter = New StreamWriter("c:\temp\test-" & CountName & ".txt", False)
            sr.Write(Master_String.ToString)
        End Using

        sw.Stop()
        Report_List.Add(sw.Elapsed.ToString & " - " & CountName)

        Zeros += 1
    Loop

    For Each lr In Report_List
        Me.ListBox1.Items.Add(lr)
    Next
End Sub

Upvotes: 1

Related Questions