C# reading gunzipped 2D array slowly while using BinaryReader

Question

I am writing a binary file that has header information, and then a 2D array:

        using (FileStream fileStream = new FileStream(filePath, FileMode.Create))
        using (GZipStream gzipStream = new GZipStream(fileStream, CompressionMode.Compress))
        using (BinaryWriter writer = new BinaryWriter(gzipStream))
        {

            writer.Write("hello");
            writer.Write(data.Size1);
            writer.Write(data.Size2);
            // Write the 2D array
            for (int i = 0; i < data.Size1; i++)
                for (int j = 0; j < data.Size2; j++)
                {
                    writer.Write(Math.Round(data.data[i, j], 1));
                }
        }

I then can read this data as the following:

            using (FileStream fileStream = new FileStream(filePath, FileMode.Open))
            using (GZipStream gzipStream = new GZipStream(fileStream, CompressionMode.Decompress))
            using (BinaryReader reader = new BinaryReader(gzipStream))
            {
                string header = reader.ReadString();
                int rows = reader.ReadInt32();
                int columns = reader.ReadInt32();

                double[,] data = new double[rows, columns];

                // Read the 2D array
                for (int i = 0; i < rows; i++)
                {
                    for (int j = 0; j < columns; j++)
                    {

                        data[i, j] = reader.ReadDouble();
                    }
                }
            }

The above works well for small arrays. However, when I start getting to larger arrays (360 x 3000), it becomes very slow and takes ~ 30 seconds to read the binary file. Is there a faster way to read the data?