{"id":2448,"date":"2024-09-14T18:29:48","date_gmt":"2024-09-14T09:29:48","guid":{"rendered":"https:\/\/secondlife.lol\/?p=2448"},"modified":"2024-11-13T22:22:17","modified_gmt":"2024-11-13T13:22:17","slug":"sigmoid-function-visualization-in-python","status":"publish","type":"post","link":"https:\/\/secondlife.lol\/en\/sigmoid-function-visualization-in-python\/","title":{"rendered":"Implementing and Visualizing Sigmoid Functions with Python: Understanding Derivatives Down to the Finest Detail"},"content":{"rendered":"<p class=\"wp-block-paragraph\"><strong><a href=\"https:\/\/ailib.secondlife.lol\/sigmoid-func-derivative-and-reason-for-use\/\">Sigmoid Functions<\/a><\/strong>is an important activation function in deep learning, often used in binary classification problems. In this post, we'll learn how to use <strong><a href=\"https:\/\/www.python.org\/\" target=\"_blank\" rel=\"noopener\">Python<\/a><\/strong>and show you how to implement this function yourself and visualize it in a graph. Plus, <strong>Derivative of a Sigmoid Function<\/strong>and visualize the results. As you follow along with the lab code, you'll gain a clearer understanding of how the Sigmoid function works.<\/p>\n\n\n<style>.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{color:var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{color:var(--global-palette2, #2B6CB0);font-size:28px;font-weight:regular;font-style:normal;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list{color:var(--global-palette1, #3182CE);line-height:2em;font-weight:regular;font-style:normal;margin-top:var(--global-kb-spacing-sm, 1.5rem);margin-right:0px;margin-bottom:0px;margin-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list .kb-table-of-contents__entry:hover{color:var(--global-palette6, #718096);}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}@media all and (max-width: 767px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}<\/style>\n\n\n<h2 class=\"wp-block-heading\">What is a sigmoid function?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The Sigmoid function is defined by the following formula<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><span class=\"wp-katex-eq\" data-display=\"false\">S(x) = \\frac{1}{1 + e^{-x}}<\/span><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This function is a non-linear function that converts the input value to a value between 0 and 1. It is often used as an activation function in neural networks. Now, let's implement a Python visualization of this formula and plot it ourselves.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Implementing a Sigmoid Function Python Visualization<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">First, let's implement the sigmoid function in Python code.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\n\nDefine the # sigmoid function\ndef sigmoid(x):\n    return 1 \/ (1 + np.exp(-x))\n\nSet the # x range\nx = np.linspace(-10, 10, 100)\n\nApply the # sigmoid function\ny = sigmoid(x)\n\nVisualize #\nplt.plot(x, y, label='Sigmoid')\nplt.title('Sigmoid Function')\nplt.xlabel('x')\nplt.ylabel('S(x)')\nplt.grid(True)\nplt.legend()\nplt.show()<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The code above defines a Sigmoid function and visualizes it. <code>np.linspace<\/code>to generate values from -10 to 10, and applying each value to the Sigmoid function to create the <code>y<\/code> After calculating the values, plot a graph.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Code commentary<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>import numpy as np<\/code>Imports the NumPy library for numerical computations.<\/li>\n\n\n\n<li><code>import matplotlib.pyplot as plt<\/code>: Imports the Matplotlib library for graph visualization.<\/li>\n\n\n\n<li><code>def sigmoid(x): return 1 \/ (1 + np.exp(-x))<\/code>Defines the Sigmoid function. This function converts the input value to a value between 0 and 1.<\/li>\n\n\n\n<li><code>x = np.linspace(-10, 10, 100)<\/code>: Generate 100 evenly spaced points from -10 to 10 and use them as x-axis values.<\/li>\n\n\n\n<li><code>y = sigmoid(x)<\/code>: Apply a Sigmoid function to the generated x values to calculate the y values.<\/li>\n\n\n\n<li><code>plt.plot(x, y, label='Sigmoid')<\/code>: Plot the graph of a Sigmoid function using x and y values.<\/li>\n\n\n\n<li><code>plt.title('Sigmoid Function')<\/code>: Set the title of the graph to \"Sigmoid Function\".<\/li>\n\n\n\n<li><code>plt.xlabel('x')<\/code>,\u00a0<code>plt.ylabel('S(x)')<\/code>: Set the labels of the x and y axes to 'x' and 'S(x)' respectively.<\/li>\n\n\n\n<li><code>plt.grid(True)<\/code>: Displays a grid in the graph.<\/li>\n\n\n\n<li><code>plt.legend()<\/code>: Adds a legend to the graph.<\/li>\n\n\n\n<li><code>plt.show()<\/code>: Displays the finished graph on the screen.<\/li>\n<\/ul>\n\n\n<style>.kb-image2448_029530-ed .kb-image-has-overlay:after{opacity:0.3;border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;}.kb-image2448_029530-ed img.kb-img, .kb-image2448_029530-ed .kb-img img{border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}<\/style>\n<div class=\"wp-block-kadence-image kb-image2448_029530-ed\"><figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"567\" height=\"455\" src=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-7.jpg\" alt=\"\uc2dc\uadf8\ubaa8\uc774\ub4dc \ud568\uc218 \uc2dc\uac01\ud654\" class=\"kb-img wp-image-2451\" srcset=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-7.jpg 567w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-7-300x241.jpg 300w\" sizes=\"(max-width: 567px) 100vw, 567px\" \/><figcaption>(Sigmoid function visualized in Python)<\/figcaption><\/figure><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Sigmoid Function Derivatives<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The derivative of a sigmoid function is defined as follows<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><span class=\"wp-katex-eq\" data-display=\"false\">S'(x) = S(x) \\times (1 - S(x))<\/span><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let's implement and visualize this differential function in Python as well.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Sigmoid Function Derivative Definition\ndef sigmoid_derivative(x):\n    return sigmoid(x) * (1 - sigmoid(x))\n\nApply the # sigmoid function derivative\ny_deriv = sigmoid_derivative(x)\n\nVisualize #\nplt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange')\nplt.title('Sigmoid Derivative Function')\nplt.xlabel('x')\nplt.ylabel(\"S'(x)\")\nplt.grid(True)\nplt.legend()\nplt.show()<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This code computes the derivative of a Sigmoid function and visualizes the result. The derivative is derived from the original Sigmoid function, and you can see from the graph that the area of greatest slope is near x=0.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Code commentary<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>def sigmoid_derivative(x): return sigmoid(x) * (1 - sigmoid(x))<\/code>Defines the derivative of a sigmoid function. The derivative of a sigmoid function is expressed as S(x) * (1 - S(x)).<\/li>\n\n\n\n<li><code>y_deriv = sigmoid_derivative(x)<\/code>: Compute the value of y_deriv by applying the derivative of the Sigmoid function to the values of x defined earlier.<\/li>\n\n\n\n<li><code>plt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange')<\/code>: Plot the derivative graph of the Sigmoid function in orange using the values of x and y_deriv.<\/li>\n\n\n\n<li><code>plt.title('Sigmoid Derivative Function')<\/code>: Set the title of the graph to \"Sigmoid Derivative Function\".<\/li>\n\n\n\n<li><code>plt.xlabel('x')<\/code>,\u00a0<code>plt.ylabel(\"S'(x)\")<\/code>: Set the labels of the x and y axes to 'x' and 'S'(x)' respectively.<\/li>\n\n\n\n<li><code>plt.grid(True)<\/code>: Displays a grid in the graph.<\/li>\n\n\n\n<li><code>plt.legend()<\/code>: Adds a legend to the graph.<\/li>\n\n\n\n<li><code>plt.show()<\/code>: Displays the finished graph on the screen.<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"576\" height=\"455\" src=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-10.jpg\" alt=\"\uc2dc\uadf8\ubaa8\uc774\ub4dc \ud568\uc218 \ubbf8 \ubd84 \uc2dc\uac01\ud654\" class=\"wp-image-2456\" srcset=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-10.jpg 576w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-10-300x237.jpg 300w\" sizes=\"(max-width: 576px) 100vw, 576px\" \/><figcaption class=\"wp-element-caption\">(Visualize the derivative of a sigmoid function)<\/figcaption><\/figure>\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">Visualize Sigmoid vs. Derivative Functions<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Now, let's compare the sigmoid function and its derivative on the same graph, so we can see at a glance the characteristics of the sigmoid function and the changes in its derivative.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\n\nDefine the # Sigmoid function\ndef sigmoid(x):\n    return 1 \/ (1 + np.exp(-x))\n\nSet the # x range\nx = np.linspace(-10, 10, 100)\n\nApply the # Sigmoid function\ny = sigmoid(x)\n\nCompute the derivative of the # Sigmoid function\ny_deriv = y * (1 - y) # derivative of sigmoid\n\nVisualize the # sigmoid and derivative functions together\nplt.plot(x, y, label='Sigmoid', color='blue')\nplt.plot(x, y_deriv, label='Sigmoid Derivative', color='orange') # y_deriv is now defined\nplt.title('Sigmoid Function and Its Derivative')\nplt.xlabel('x')\nplt.ylabel('Value')\nplt.grid(True)\nplt.legend()\nplt.show()<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The code above visualizes the sigmoid function and its derivative at the same time. The graph shows how the sigmoid function changes as a function of x and how its derivative shows a pattern.<\/p>\n\n\n<style>.kb-image2448_e9b56c-00 .kb-image-has-overlay:after{opacity:0.3;border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;}.kb-image2448_e9b56c-00 img.kb-img, .kb-image2448_e9b56c-00 .kb-img img{border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}<\/style>\n<div class=\"wp-block-kadence-image kb-image2448_e9b56c-00\"><figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"567\" height=\"455\" src=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-8.jpg\" alt=\"\uc2dc\uadf8\ubaa8\uc774\ub4dc \ud568\uc218\uc640 \ubbf8\ubd84 \ud568\uc218 \ube44\uad50 \uc2dc\uac01\ud654\" class=\"kb-img wp-image-2453\" srcset=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-8.jpg 567w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/image-8-300x241.jpg 300w\" sizes=\"(max-width: 567px) 100vw, 567px\" \/><figcaption>(Visualize a sigmoid function versus a derivative function)<\/figcaption><\/figure><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Why use sigmoid functions?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Sigmoid functions<\/strong>plays an important role in deep learning for several reasons, not the least of which is that it can be interpreted as a probability in binary classification problems by constraining the output value to be between 0 and 1. It is also differentiable, which makes it easy to compute the gradient during backpropagation. However, it has the disadvantage that it can lead to gradient vanishing problems, which is why functions like ReLU are now more commonly used.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Organize<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this post, we'll use the <strong>Implementing a Sigmoid Function with Python<\/strong>and visualizing its derivative. The sigmoid function is an important activation function in deep learning and is often used to calculate probabilities in binary classification problems. By implementing and graphing it ourselves, we were able to easily understand how the function works and how its derivative changes. We hope that by learning the concept of the sigmoid function and practicing it in Python, you will be able to better understand and utilize it in your deep learning models.<\/p>","protected":false},"excerpt":{"rendered":"<p>The sigmoid function is an important activation function in deep learning, mainly in binary classification problems...<\/p>","protected":false},"author":3,"featured_media":2458,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[7,3],"tags":[],"class_list":["post-2448","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-coding","category-python-coding"],"taxonomy_info":{"category":[{"value":7,"label":"\uc778\uacf5\uc9c0\ub2a5(AI)"},{"value":3,"label":"\ud30c\uc774\uc36c(Python)"}]},"featured_image_src_large":["https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/\uc2dc\uadf8\ubaa8\uc774\ub4dc-\ud568\uc218-\uc2dc\uac01\ud654_20240914_182340_0000-600x600.jpg",600,600,true],"author_info":{"display_name":"TERE","author_link":"https:\/\/secondlife.lol\/en\/author\/tere\/"},"comment_info":0,"category_info":[{"term_id":7,"name":"\uc778\uacf5\uc9c0\ub2a5(AI)","slug":"ai-coding","term_group":0,"term_taxonomy_id":7,"taxonomy":"category","description":"","parent":20,"count":70,"filter":"raw","cat_ID":7,"category_count":70,"category_description":"","cat_name":"\uc778\uacf5\uc9c0\ub2a5(AI)","category_nicename":"ai-coding","category_parent":20},{"term_id":3,"name":"\ud30c\uc774\uc36c(Python)","slug":"python-coding","term_group":0,"term_taxonomy_id":3,"taxonomy":"category","description":"","parent":20,"count":116,"filter":"raw","cat_ID":3,"category_count":116,"category_description":"","cat_name":"\ud30c\uc774\uc36c(Python)","category_nicename":"python-coding","category_parent":20}],"tag_info":false,"_links":{"self":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2448","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/comments?post=2448"}],"version-history":[{"count":14,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2448\/revisions"}],"predecessor-version":[{"id":3368,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2448\/revisions\/3368"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media\/2458"}],"wp:attachment":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media?parent=2448"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/categories?post=2448"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/tags?post=2448"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}