CODEWITHSUNDEEP
vb.netloopsoptimizationforeachevaluate
solracid
solracid

Reputation: 413

VB optimize loop avoid with an Evaluate()

I am running a loop that evaluates input given by the user over more than 150k objects. The user sets the info to read like "obj.Name", "obj.Path", "obj.Date"... which may contain also logic evaluations such as "IIf(obj.Params>5,1,0)". It is then provided into the programm as a string.

I used the Evaluate() functions and it does work well, just that it is slow. It takes almost 6h to go over all elements. I was thinking if there is a way in which I can take the requested info and turn it into a straight-forward executable somehow, and avoid using the Evaluate expression in the whole loop (it runs for the number of requested data by the user * 150k).

This is a schematic of the loop I am running:

For Each Object in ObjectsList
    For Each UserRequest in Requests
        ResultsMatrix(i,j) = Evaluate(Requests(i))
        j += 1
    Next
    i += 1
Next

I store then the results in a Matrix which is pasted in an Excel file at the end. Is there a way in which I can do sort of working the string to be evaluated into a function's return? I'd like to avoid usig the Eval function and parse directly the string in an executable and dont evaluate it for each object. Any tips on speeding up the loop?

Upvotes: 0

Views: 94

Answers (2)

wizzardmr42
wizzardmr42

Reputation: 1644

It might be worth considering writing the requests into a set of functions and using the .NET CodeDom compilers to build it into a DLL. You can then load the assembly, find the right functions using reflection and put them into an array, then call them using reflection - that way you'll be calling .NET code and it should be far faster. Some (incomplete) code to get you started from a project where I have done this...

        Private Function CombineCode() As String

            Dim ret As New System.Text.StringBuilder
            ret.AppendLine("Imports System")
            ret.AppendLine("Imports Microsoft.VisualBasic")
            ret.AppendLine()
            ret.AppendLine("Namespace " & MainNamespace)
            ret.AppendLine("Public Class " & MainClassName)

            For Each e In _Entries
                ret.AppendLine(e.Value.Code)
            Next

            ret.AppendLine("End Class")
            ret.AppendLine("End Namespace")

            Return ret.ToString
        End Function

        Private Function Compile(Code As String) As Assembly
            'Dim d As New Dictionary(Of String, String)
            'd.Add("langversion", "14")
            Dim VBP As New Microsoft.CodeDom.Providers.DotNetCompilerPlatform.VBCodeProvider()

            Dim PM As New System.CodeDom.Compiler.CompilerParameters
            'PM.GenerateInMemory = True
            PM.GenerateExecutable = False
            PM.OutputAssembly = IO.Path.Combine(_Path, GenerateFileName() & ".dll") ' "Generated.dll"
            PM.MainClass = MainClassName
            PM.IncludeDebugInformation = True


            Dim ASM As System.Reflection.Assembly
            For Each ASM In AppDomain.CurrentDomain.GetAssemblies()
                Try
                    If ASM.Location <> "" Then PM.ReferencedAssemblies.Add(ASM.Location)
                Catch
                End Try
            Next

            PM.ReferencedAssemblies.Add("System.Web.dll")

            'Get compilation results
            Dim Results As System.CodeDom.Compiler.CompilerResults
            Results = VBP.CompileAssemblyFromSource(PM, Code)

            'Show possible compilation errors
            Dim Err As System.CodeDom.Compiler.CompilerError
            For Each Err In Results.Errors
                Throw New SyntaxErrorException("Error N. " & Err.ErrorNumber &
            " Message: " & Err.ErrorText & " Line " & Err.Line & " in code " & vbCrLf & Code)
            Next

            Return Results.CompiledAssembly

        End Function
        Private Sub FindMethods()
            Dim dt = (From t In _LatestAssembly.GetTypes() Where t.Name = MainClassName).Single
            For Each e In _Entries.Values
                e.Method = dt.GetMethod(e.MethodName)
            Next
        End Sub


Assembly = Assembly.LoadFrom(System.IO.Path.Combine(Path, sd.LatestAssemblyFile))

Upvotes: 1

Ctznkane525
Ctznkane525

Reputation: 7465

The Evaluate function is just resources on the computer itself. It's a great candidate for using Parallel.For.

In this case, j is the implied index. 

For Each Object in ObjectsList

    Parallel.For(0, Requests.Length, New ParallelOptions(), Sub(j, loopState)
           ResultsMatrix(i,j) = Evaluate(Requests(j))
        End Sub
    )
    i += 1
Next

Note, that Requests(i) is getting called repeatedly and produces the same result, so I assume you mean Requests(j).

Upvotes: 0

Related Questions