Optimizing double for loop for execution speed

Question

I am fairly new to C# and I need some help optimizing the my code.

struct Data
{
    public static int numberOfPixels;
    public static string[] filePaths;
    public static string[] fileNames;
    public static double[] z;
    public static int numberOfFrames;
    public static byte[][] imageBuffer;
    public static int bufferSize = 1000;
    public static double[] num;
    public static double[] den;
}
public class Methods
{
    public void RetrieveFileList()
    {
        Console.WriteLine("Please enter the folder path where all measurement files are stored: ");
        Data.filePaths = Directory.GetFiles(Console.ReadLine(),"*.bin");
        Data.fileNames = new string[Data.filePaths.Length];
        Data.numberOfFrames = Data.filePaths.Length;
        Data.z = new double[Data.filePaths.Length];
        int n = 0;
        foreach(string file in Data.filePaths)
        {
            Data.fileNames[n] = Path.GetFileNameWithoutExtension(file);
            n++;
        }
    }
    public void CreatePositionArray()
    {
        Console.WriteLine("Please enter the stepsize used during the scan in nanometers: ");
        double stepsize = Convert.ToDouble(Console.ReadLine());
        int n = 0;
        foreach(string file in Data.fileNames)
        {
            Data.z[n] = Convert.ToInt32(file) * stepsize / 1000; 
            n++;
        }
    }
    public void InitializeBufferArray()
    {
        Data.imageBuffer = new byte[Data.numberOfFrames][];
    }
    public byte[] ReadBinaryFile(int index)
    {
        return File.ReadAllBytes(Data.filePaths[index]); ;
    }
    public void FillImageBuffer()
    {
        for (int i = 0; i < Data.bufferSize; i++)
        {
            Data.imageBuffer[i] = ReadBinaryFile(i);
        }
        Data.numberOfPixels = Data.imageBuffer[0].Length;
        Data.num = new double[Data.numberOfPixels];
        Data.den = new double[Data.numberOfPixels];
    }
}
class Program
{
    static void Main(string[] args)
    {
        Method.RetrieveFileList();
        Method.CreatePositionArray();
        Method.InitializeBufferArray();
        Method.FillImageBuffer();
        for(int i = 0; i < Data.numberOfFrames; i++)
        {
            for (int j = 0; j < Data.numberOfPixels; j++)
            {
                double der = Math.Pow(Data.imageBuffer[i+1][j] - Data.imageBuffer[i][j], 2);
                if (der < 1) der = 0;
                Data.num[j] = Data.num[j] + Data.z[i] * der;
                Data.den[j] = Data.den[j] + der;
            }
        } 
    }
}

Specifically the two loops in my Main method. Right now this loop processes about 1000 frames with 1210000 pixels each. One iteration of the outer loop takes about 80ms to execute. What would be the best way to here? Create multiple Threads and split my buffer into predefined junks or use the Parallel class? I would appreciate any sort of help.

Thanks.

Answer 1

I've been working with "looping too much" for some time now, and I had this same problem. I looped 91 million times, and It took me about 4-5 minutes. What I did to break this down to 20-30 seconds was profiling. You gotta try out the Visual Studio profiler. It really helps you know which line of code reduces your program's performance dramatically. I've noticed that no one mentioned GPGPU(General Programming on GPU/or something like that). If you want to execute your code each frame, GPGPU can do it! You can use OpenCL, Cudafy or DirectX/OpenGL's compute shaders. If you don't want to go for GPU programming, then using the VS Profiler is your best practice.

Answer 2

Probably, you need to reverse loop order to reduce array index references. Also, instead of Math.Pow(der, 2), better use der*der - it's a bit faster

     Method.RetrieveFileList();
     Method.CreatePositionArray();
     Method.InitializeBufferArray();
     Method.FillImageBuffer();
      for (int j = 0; j < Data.numberOfPixels; j++)  {
        double num = 0
        double den = 0;
        for(int i = 0; i < Data.numberOfFrames; i++) {
           double der = Data.imageBuffer[i+1][j] - Data.imageBuffer[i][j]     
           if ((der *= der) < 1) der = 0;
           num += Data.z[i] * der;
           den += der;
        }
        Data.num[j] = num;
        Data.den[j] = den;
   } 

Quite honestly, I don't think it will drammatically increase the performance.