Hinge Loss

I worked on an assignment which asked us to make a mapping rule for a smooth Hinge Funtion which is attached at the last of the blog.

So basically a hinge loss function is a function for binary classification that aims to maximize the margin between two data clouds.

Suppose that that the two data clouds are apple and banana and we want to make a binary classifier which separates them

Me and my team compared this hinge loss function to a ReLU function( Rectified Linear Unit ), and figured out that it was a function which gave binary output but has threshold on the origin instead of what our case was ,(1,0). Anyways the function , Hinge Loss  very similar to that of the ReLU.You can see that it as follows: ReLU -> $y=max(0,x)$& Our Hinge Loss ->$y=max(0,(1-x))$

.

On googling smooth function of ReLU we figured out something called as a SoftPlus Funtion . You can simply google it for a structured definition but basically it is a function which smooth ens a ReLU function making it differentiable, i.e in this case not making the y zero at any instance. It is defined for a RELU as :

. Graph is attached at the end of the blog

This allows us to find the derivative of the Soft Plus Function:. This again has a formal function name which is Sigmoid Funtion. We will discuss about this is our next blog.

The whole point of talking about comparison with ReLu and then SoftPlus, and finally differentiating it get a sigmoid function is to make it differentiable.This is required us to work with something called a gradient descent and also to figure out the mapping rule for the smooth Hing Loss Function.