How can I use hopfield network to learn more patterns?

Question

Is there any relation between number of neurons and ability of Hopfield network to recognize patterns?

I write neural network program in C# to recognize patterns with Hopfield network. My network has 64 neurons. When I train network for 2 patterns, every things work nice and easy, but when I train network for more patterns, Hopfield can't find answer!

So, according to my code, how can I use Hopfield network to learn more patterns?

Should I make changes in this code?

There is my train() function:

public void Train(bool[,] pattern)
{
    //N is number of rows in our square matrix
    //Convert input pattern to bipolar
    int[,] PatternBipolar = new int[N, N];

    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
            {
                if (pattern[i, j] == true)
                {
                    PatternBipolar[i, j] = 1;
                }
                else
                {
                    PatternBipolar[i, j] = -1;
                }
            }

    //convert to row matrix
    int count1 = 0;
    int[] RowMatrix = new int[(int)Math.Pow(N, 2)];
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            {
                RowMatrix[count1] = PatternBipolar[i, j];
                count1++;
            }

    //convert to column matrix
    int count2 = 0;
    int[] ColMatrix = new int[(int)Math.Pow(N, 2)];
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            {
                ColMatrix[count2] = PatternBipolar[i, j];
                count2++;
            }

    //multiplication
    int[,] MultipliedMatrix = new int[(int)Math.Pow(N, 2), (int)Math.Pow(N, 2)];
    for (int i = 0; i < (int)Math.Pow(N, 2); i++)
        for (int j = 0; j < (int)Math.Pow(N, 2); j++)
            {
                MultipliedMatrix[i, j] = ColMatrix[i] * RowMatrix[j];
            }

    //cells in the northwest diagonal get set to zero
    for (int i = 0; i < (int)Math.Pow(N, 2); i++)
        MultipliedMatrix[i, i] = 0;

    // WightMatrix + MultipliedMatrix

    for (int i = 0; i < (int)Math.Pow(N, 2); i++)
        for (int j = 0; j < (int)Math.Pow(N, 2); j++)
            {
                WeightMatrix[i, j] += MultipliedMatrix[i, j];
            }

And there is Present() function (this function is used to return answer for a given pattern):

public void Present(bool[,] Pattern)
{
    int[] output = new int[(int)(int)Math.Pow(N, 2)];

    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            {
                OutputShowMatrix[i, j] = 0;
            }

    //convert bool to binary
    int[] PatternBinary = new int[(int)Math.Pow(N, 2)];
    int count = 0;
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
            {
                if (Pattern[i, j] == true)
                {
                    PatternBinary[count] = 1;
                }
                else
                {
                    PatternBinary[count] = 0;
                }
                count++;
            }

    count = 0;
    int activation = 0;
    for (int j = 0; j < (int)Math.Pow(N, 2); j++)
    {
        for (int i = 0; i < (int)Math.Pow(N, 2); i++)
        {
            activation = activation + (PatternBinary[i] * WeightMatrix[i, j]);
        }

        if (activation > 0)
        {
            output[count] = 1;
        }
        else
        {
            output[count] = 0;
        }

        count++;
        activation = 0;
    }

    count = 0;
    for (int j = 0; j < N; j++)
        for (int i = 0; i < N; i++)
        {
            OutputShowMatrix[i, j] = output[count++];
        }

In below images I trained Hopfield for characters A and P and when input patterns are like A or P, network recognize them in true way

Then I train it for character C:

This is where every things go wrong!

Now if I enter pattern like C, this issue happen:

And if enter pattern like A, see what happen:

And if train more patterns, whole of grid become black!

Answer 1

I've spotted only one mistake in your code: you perform only one iteration of node value calculation, without verifying if the values have converged. I've fixed this method like this:

public bool[,] Present(bool[,] pattern)
{
    bool[,] result = new bool[N, N];
    int[] activation = new int[N * N];

    int count = 0;
    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
        {
            activation[count++] = pattern[i, j] ? 1 : 0;
        }

    bool convergence = false;
    while (!convergence)
    {
        convergence = true;
        var previousActivation = (int[])activation.Clone();
        for (int i = 0; i < N * N; i++)
        {
            activation[i] = 0;
            for (int j = 0; j < N * N; j++)
            {
                activation[i] += (previousActivation[j] * WeightMatrix[i, j]);
            }

            activation[i] = activation[i] > 0 ? 1 : 0;

            if (activation[i] != previousActivation[i])
            {
                convergence = false;
            }
        }
    }

    count = 0;
    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++)
        {
            result[i, j] = activation[count++] == 1;
        }

    return result;
}

This slightly improves the results, however probably should also be improved to calculate the values asynchronously to avoid cycles.

Unfortunately, this still introduces the behaviour you've described. This is results from the phenomena called spurious patterns. For the network to learn more than one pattern consider training it with a Hebb rule. You can read about the spurious patterns, stability and learning of the Hopfield network here and here.