CODEWITHSUNDEEP
pythonnumpysoftmax
FortranFun
FortranFun

Reputation: 156

Python: Define the softmax function

import numpy as np

def softmax(x):
    row_num = x.shape[0]
    col_num = x.shape[1]
    for m in row_num:
        for n in col_num:
            new_x[m,n] = np.exp(x[m,n])/sum(x[:,n])

    return new_x

logits = [1.0, 2.0, 3.0]
logits2 = np.array([
    [1, 2, 3, 6],
    [2, 4, 5, 6],
    [3, 8, 7, 6]])

print(softmax(logits1))
print(softmax(logits2))

Above is the function for softmax (it is used to turn logits to probabilities)

I want to obtain the solution shown as below:

[ 0.09003057  0.24472847  0.66524096]

[
    [ 0.09003057  0.00242826  0.01587624  0.33333333]
    [ 0.24472847  0.01794253  0.11731043  0.33333333]
    [ 0.66524096  0.97962921  0.86681333  0.33333333]
]

However, error was revealed that "'int' object is not iterable". In addition, I want to see a more efficient code for this function with less complexity.

Upvotes: 0

Views: 2868

Answers (4)

yeachan park
yeachan park

Reputation: 180

My implementation of softmax function in numpy module is like this:

import numpy as np
def softmax(self,x,axis=0):    
    ex = np.exp(x - np.max(x,axis=axis,keepdims=True))
    return ex / np.sum(ex,axis=axis,keepdims=True) 
np.softmax = softmax.__get__(np)

Then it is possible to use softmax function as a typical numpy built-in function. like :

output = model(x)  # output.shape : (50000,10)
softmaxed_value = np.softmax(output,axis=1)

Upvotes: 0

Nolan Conaway
Nolan Conaway

Reputation: 2767

EDIT. As of version 1.2.0, scipy includes softmax as a special function:

https://scipy.github.io/devdocs/generated/scipy.special.softmax.html

In general, it is better to use a vectorized implementation rather than rely on for loops. You can take advantage of numpy's broadcasting to do this. There are a number of other questions possessing correct implementations of such a function (for example: here, here).

For the sake of associating an answer with the question, I'll paste in my general softmax function operating over an arbitrary axis, including a tricky max subtraction bit. I also wrote a more detailed blog post about it.

def softmax(X, theta = 1.0, axis = None):
    """
    Compute the softmax of each element along an axis of X.

    Parameters
    ----------
    X: ND-Array. Probably should be floats. 
    theta (optional): float parameter, used as a multiplier
        prior to exponentiation. Default = 1.0
    axis (optional): axis to compute values along. Default is the 
        first non-singleton axis.

    Returns an array the same size as X. The result will sum to 1
    along the specified axis.
    """

    # make X at least 2d
    y = np.atleast_2d(X)

    # find axis
    if axis is None:
        axis = next(j[0] for j in enumerate(y.shape) if j[1] > 1)

    # multiply y against the theta parameter, 
    y = y * float(theta)

    # subtract the max for numerical stability
    y = y - np.expand_dims(np.max(y, axis = axis), axis)

    # exponentiate y
    y = np.exp(y)

    # take the sum along the specified axis
    ax_sum = np.expand_dims(np.sum(y, axis = axis), axis)

    # finally: divide elementwise
    p = y / ax_sum

    # flatten if X was 1D
    if len(X.shape) == 1: p = p.flatten()

    return p

Upvotes: 1

Jeremy_Tamu
Jeremy_Tamu

Reputation: 755

The most efficient code should be:

import numpy as np
def softmax(x):
    return np.exp(x) / np.sum(np.exp(x), axis=0)

Upvotes: 1

Miriam Farber
Miriam Farber

Reputation: 19634

This will do the job:

logits = np.array([1.0, 2.0, 3.0])
logits2 = np.array([
    [1, 2, 3, 6],
    [2, 4, 5, 6],
    [3, 8, 7, 6]])
def softmax(x):
    r=np.exp(x - np.max(x))
    return r/r.sum(axis=0)

You get the error

"'int' object is not iterable"

because row_num (and similarly col_num) is a number, so you cannot iterate over it. You need to add range (that is, for m in range(row_num)).

There are also other issues. For example, x.shape[1] is not necessarily defined (it is not defined for logits) so it will also throw an error. new_x is not defined as well.

Upvotes: 2

Related Questions