|

Implementing and Visualizing Sigmoid Functions with Python: Understanding Derivatives Down to the Finest Detail

Sigmoid Functionsis an important activation function in deep learning, often used in binary classification problems. In this post, we'll learn how to use Pythonand show you how to implement this function yourself and visualize it in a graph. Plus, Derivative of a Sigmoid Functionand visualize the results. As you follow along with the lab code, you'll gain a clearer understanding of how the Sigmoid function works.

What is a sigmoid function?

The Sigmoid function is defined by the following formula

S(x) = \frac{1}{1 + e^{-x}}

This function is a non-linear function that converts the input value to a value between 0 and 1. It is often used as an activation function in neural networks. Now, let's implement a Python visualization of this formula and plot it ourselves.

Implementing a Sigmoid Function Python Visualization

First, let's implement the sigmoid function in Python code.

import numpy as np
import matplotlib.pyplot as plt

Define the # sigmoid function
def sigmoid(x):
    return 1 / (1 + np.exp(-x))

Set the # x range
x = np.linspace(-10, 10, 100)

Apply the # sigmoid function
y = sigmoid(x)

Visualize #
plt.plot(x, y, label='Sigmoid')
plt.title('Sigmoid Function')
plt.xlabel('x')
plt.ylabel('S(x)')
plt.grid(True)
plt.legend()
plt.show()

The code above defines a Sigmoid function and visualizes it. np.linspaceto generate values from -10 to 10, and applying each value to the Sigmoid function to create the y After calculating the values, plot a graph.

Code commentary

  • import numpy as npImports the NumPy library for numerical computations.
  • import matplotlib.pyplot as plt: Imports the Matplotlib library for graph visualization.
  • def sigmoid(x): return 1 / (1 + np.exp(-x))Defines the Sigmoid function. This function converts the input value to a value between 0 and 1.
  • x = np.linspace(-10, 10, 100): Generate 100 evenly spaced points from -10 to 10 and use them as x-axis values.
  • y = sigmoid(x): Apply a Sigmoid function to the generated x values to calculate the y values.
  • plt.plot(x, y, label='Sigmoid'): Plot the graph of a Sigmoid function using x and y values.
  • plt.title('Sigmoid Function'): Set the title of the graph to "Sigmoid Function".
  • plt.xlabel('x'), plt.ylabel('S(x)'): Set the labels of the x and y axes to 'x' and 'S(x)' respectively.
  • plt.grid(True): Displays a grid in the graph.
  • plt.legend(): Adds a legend to the graph.
  • plt.show(): Displays the finished graph on the screen.
시그모이드 함수 시각화
(Sigmoid function visualized in Python)

Sigmoid Function Derivatives

The derivative of a sigmoid function is defined as follows

S'(x) = S(x) \times (1 - S(x))

Let's implement and visualize this differential function in Python as well.

# Sigmoid Function Derivative Definition
def sigmoid_derivative(x):
    return sigmoid(x) * (1 - sigmoid(x))

Apply the # sigmoid function derivative
y_deriv = sigmoid_derivative(x)

Visualize #
plt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange')
plt.title('Sigmoid Derivative Function')
plt.xlabel('x')
plt.ylabel("S'(x)")
plt.grid(True)
plt.legend()
plt.show()

This code computes the derivative of a Sigmoid function and visualizes the result. The derivative is derived from the original Sigmoid function, and you can see from the graph that the area of greatest slope is near x=0.

Code commentary

  • def sigmoid_derivative(x): return sigmoid(x) * (1 - sigmoid(x))Defines the derivative of a sigmoid function. The derivative of a sigmoid function is expressed as S(x) * (1 - S(x)).
  • y_deriv = sigmoid_derivative(x): Compute the value of y_deriv by applying the derivative of the Sigmoid function to the values of x defined earlier.
  • plt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange'): Plot the derivative graph of the Sigmoid function in orange using the values of x and y_deriv.
  • plt.title('Sigmoid Derivative Function'): Set the title of the graph to "Sigmoid Derivative Function".
  • plt.xlabel('x'), plt.ylabel("S'(x)"): Set the labels of the x and y axes to 'x' and 'S'(x)' respectively.
  • plt.grid(True): Displays a grid in the graph.
  • plt.legend(): Adds a legend to the graph.
  • plt.show(): Displays the finished graph on the screen.
시그모이드 함수 미 분 시각화
(Visualize the derivative of a sigmoid function)

Visualize Sigmoid vs. Derivative Functions

Now, let's compare the sigmoid function and its derivative on the same graph, so we can see at a glance the characteristics of the sigmoid function and the changes in its derivative.

import numpy as np
import matplotlib.pyplot as plt

Define the # Sigmoid function
def sigmoid(x):
    return 1 / (1 + np.exp(-x))

Set the # x range
x = np.linspace(-10, 10, 100)

Apply the # Sigmoid function
y = sigmoid(x)

Compute the derivative of the # Sigmoid function
y_deriv = y * (1 - y) # derivative of sigmoid

Visualize the # sigmoid and derivative functions together
plt.plot(x, y, label='Sigmoid', color='blue')
plt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange') # y_deriv is now defined
plt.title('Sigmoid Function and Its Derivative')
plt.xlabel('x')
plt.ylabel('Value')
plt.grid(True)
plt.legend()
plt.show()

The code above visualizes the sigmoid function and its derivative at the same time. The graph shows how the sigmoid function changes as a function of x and how its derivative shows a pattern.

시그모이드 함수와 미분 함수 비교 시각화
(Visualize a sigmoid function versus a derivative function)

Why use sigmoid functions?

Sigmoid functionsplays an important role in deep learning for several reasons, not the least of which is that it can be interpreted as a probability in binary classification problems by constraining the output value to be between 0 and 1. It is also differentiable, which makes it easy to compute the gradient during backpropagation. However, it has the disadvantage that it can lead to gradient vanishing problems, which is why functions like ReLU are now more commonly used.

Organize

In this post, we'll use the Implementing a Sigmoid Function with Pythonand visualizing its derivative. The sigmoid function is an important activation function in deep learning and is often used to calculate probabilities in binary classification problems. By implementing and graphing it ourselves, we were able to easily understand how the function works and how its derivative changes. We hope that by learning the concept of the sigmoid function and practicing it in Python, you will be able to better understand and utilize it in your deep learning models.

Similar Posts