{"id":2409,"date":"2024-09-05T08:45:51","date_gmt":"2024-09-04T23:45:51","guid":{"rendered":"https:\/\/secondlife.lol\/?p=2409"},"modified":"2024-09-05T15:59:58","modified_gmt":"2024-09-05T06:59:58","slug":"python-svm-visualization","status":"publish","type":"post","link":"https:\/\/secondlife.lol\/en\/python-svm-visualization\/","title":{"rendered":"Support Vector Machine SVM Examples and Visualizations - Two-Dimensional"},"content":{"rendered":"<p class=\"wp-block-paragraph\">For classification problems in machine learning, Support Vector Machines (SVMs) are a popular and powerful algorithm. In particular, they have the advantage of being easy to understand visually by clearly drawing the decision boundary of the data. In this post, we will learn how to use the Python <code>scikit-learn<\/code> library to walk through an SVM example, and use the results in the <strong>Visualization<\/strong>so you can understand how SVMs work and see how the results are represented visually.<\/p>\n\n\n<style>.kb-image2409_1fb5e4-dd .kb-image-has-overlay:after{opacity:0.3;border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;}.kb-image2409_1fb5e4-dd img.kb-img, .kb-image2409_1fb5e4-dd .kb-img img{border-top:31px solid;border-right:31px solid;border-bottom:31px solid;border-left:31px solid;border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}@media all and (max-width: 1024px){.kb-image2409_1fb5e4-dd img.kb-img, .kb-image2409_1fb5e4-dd .kb-img img{border-top:31px solid;border-right:31px solid;border-bottom:31px solid;border-left:31px solid;}}@media all and (max-width: 767px){.kb-image2409_1fb5e4-dd img.kb-img, .kb-image2409_1fb5e4-dd .kb-img img{border-top:31px solid;border-right:31px solid;border-bottom:31px solid;border-left:31px solid;}}<\/style>\n<figure class=\"wp-block-kadence-image kb-image2409_1fb5e4-dd size-full\"><img decoding=\"async\" width=\"546\" height=\"435\" src=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2319.jpg\" alt=\"SVM \uc608\uc81c \uc378\ub124\uc77c\" class=\"kb-img wp-image-2415\" srcset=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2319.jpg 546w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2319-300x239.jpg 300w\" sizes=\"(max-width: 546px) 100vw, 546px\" \/><\/figure>\n\n\n<style>.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{color:var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{color:var(--global-palette2, #2B6CB0);font-size:28px;font-weight:regular;font-style:normal;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list{color:var(--global-palette1, #3182CE);line-height:2em;font-weight:regular;font-style:normal;margin-top:var(--global-kb-spacing-sm, 1.5rem);margin-right:0px;margin-bottom:0px;margin-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list .kb-table-of-contents__entry:hover{color:var(--global-palette6, #718096);}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}@media all and (max-width: 767px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}<\/style>\n\n\n<h2 class=\"wp-block-heading\">Understanding support vector machines (SVMs)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>SVM<\/strong>is a supervised learning algorithm that finds the best boundary (hyperplane) to separate a given set of data into two classes. The idea behind SVM is to generate an optimal classifier by maximizing the distance of the closest data points (support vectors) to the classification boundary.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">SVMs are particularly effective when operating in high dimensions, and can even classify non-linear data using a kernel function. In this example, we'll use two-dimensional data to visualize the boundaries along which an SVM classifies the data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Python code step-by-step<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. import the required libraries<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">First, load the necessary libraries to implement the SVM and visualize the results.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.svm import SVC<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>numpy<\/code>A library for arrays and math calculations.<\/li>\n\n\n\n<li><code>matplotlib.pyplot<\/code>A library for graphing and visualization.<\/li>\n\n\n\n<li><code>sklearn.datasets<\/code>: A module that provides a simple dataset, in this case the <code>make_classification<\/code> dataset that we'll use.<\/li>\n\n\n\n<li><code>train_test_split<\/code>: A function that separates data for training and testing.<\/li>\n\n\n\n<li><code>SVC<\/code>: A class that implements a support vector machine (SVM) classifier.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2. Create sample data<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create simple two-dimensional data and train it with an SVM.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Generate simple 2D data\nX, y = datasets.make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=42)\n\nSplit the # dataset into training and testing\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>n_samples=100<\/code>: Generate 100 samples.<\/li>\n\n\n\n<li><code>n_features=2<\/code>: Creates two-dimensional data with two features.<\/li>\n\n\n\n<li><code>n_informative=2<\/code>: Both features are useful information for classification.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3. Create and train an SVM model<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">After separating the generated data into training and testing, train the SVM model.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Create a # SVM model\nmodel = SVC(kernel='linear')\n\nTrain the # model\nmodel.fit(X_train, y_train)<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>kernel='linear'<\/code>: Find a straight line separating the two classes using a linear kernel.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Visualize SVM results<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Now visualize the classification boundaries of the trained SVM model and the test data. Support Vector Machine<strong> Visualization<\/strong>is very useful for understanding how SVM models classify data.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Functions to visualize # training data and classification boundaries\ndef plot_decision_boundary(X, y, model):\n    Generate a grid for visualizing # boundaries\n    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n    xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.01),\n                         np.arange(y_min, y_max, 0.01))\n\n    Calculate the predicted value at each coordinate using the # model\n    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])\n    Z = Z.reshape(xx.shape)\n\n    Visualize the # boundary\n    plt.contourf(xx, yy, Z, alpha=0.8)\n\n    Visualize # data points\n    plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', marker='o', s=50)\n    plt.xlim(xx.min(), xx.max())\n    plt.ylim(yy.min(), yy.max())\n    plt.title(\"SVM Decision Boundary\")\n    plt.show()\n\nVisualize the boundary with the # trained model\nplot_decision_boundary(X_test, y_test, model)<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>np.meshgrid<\/code>: Generates a grid to visualize the boundaries.<\/li>\n\n\n\n<li><code>model.predict<\/code>Draws a boundary based on the values predicted by the model for the generated grid.<\/li>\n\n\n\n<li><code>plt.contourf<\/code>Visualize classification boundaries based on the prediction results.<\/li>\n\n\n\n<li><code>plt.scatter<\/code>: You can visualize real data points to verify that the model has classified them correctly.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">When you run the code above, you'll see the classification boundaries of the data as trained by the SVM, and you'll see that the test data is correctly classified within those boundaries.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Support vector machine visualization exampleFacility name<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The classification boundary appears as a linear boundary separating the two classes.<\/li>\n\n\n\n<li>Each data point is displayed in a different color based on its class.<\/li>\n\n\n\n<li>The support vectors learned by the model can also be seen in the visualization as data points located close to the boundary.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Full integration code<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Below is the complete code for the support vector machine (SVM) model. We've added comments where necessary to explain the steps.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.svm import SVC\n\n# 1. Generate data\n# Create 100 samples with 2 features, creating a dataset with well-separated distributions across classes\nX, y = datasets.make_classification(n_samples=100, n_features=2, n_informative=2,\n                                    n_redundant=0, random_state=42)\n\n# 2. Data Partitioning\n# Divide the data into 70% for training and 30% for testing.\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n\n# 3. Generate SVM model\nCreate an SVM model using a # linear kernel\nmodel = SVC(kernel='linear')\n\n# 4. Train the model\nTrain the SVM model using the # training data\nmodel.fit(X_train, y_train)\n\n# 5. Define a visualization function\nFunctions to visualize the classification boundaries and data points of the # trained model\ndef plot_decision_boundary(X, y, model):\n    # 5.1 Setting up the grid for boundary visualization\n    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n    xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.01),\n                         np.arange(y_min, y_max, 0.01))\n\n    # 5.2 Compute the value predicted by the model at each point in the grid\n    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])\n    Z = Z.reshape(xx.shape)\n\n    # 5.3 Drawing classification boundaries based on predicted values\n    plt.contourf(xx, yy, Z, alpha=0.8)\n\n    # 5.4 Visualize actual data points\n    plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', marker='o', s=50)\n    plt.xlim(xx.min(), xx.max())\n    plt.ylim(yy.min(), yy.max())\n    plt.title(\"SVM Decision Boundary\")\n    plt.show()\n\n# 6. Visualize the classification boundary using test data\nplot_decision_boundary(X_test, y_test, model)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Code description<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Generate data<\/strong>: <code>datasets.make_classification<\/code> function to generate two-dimensional data so that it can be easily visualized with SVMs.<\/li>\n\n\n\n<li><strong>Partitioning data<\/strong>: <code>train_test_split<\/code>to split the data into training (70%) and testing (30%).<\/li>\n\n\n\n<li><strong>Create an SVM model<\/strong>: <code>SVC<\/code> class's <code>kernel='linear'<\/code>to generate a linear support vector machine model.<\/li>\n\n\n\n<li><strong>Model training<\/strong>: <code>fit<\/code> method to train the model based on the training data.<\/li>\n\n\n\n<li><strong>Visualization Functions<\/strong>A function for drawing the classification boundaries of the generated model, which generates a grid to visualize the classification boundaries based on the prediction results at each point.<\/li>\n\n\n\n<li><strong>Visualize results<\/strong>: Use test data to verify the classification boundaries learned by the model.<\/li>\n\n\n\n<li><\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions (FAQ)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q1. What is a support vector machine (SVM)?<\/strong><br>A1. SVM is a classification algorithm that finds the best boundary separating the given data into two classes. Support vectors are data points that play an important role in defining this boundary.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q2. <code>kernel<\/code>What is it?<\/strong><br>A2. <code>kernel<\/code>is a function that transforms data so that data with non-linear relationships can be linearly separated. There are many different types, including linear kernels, polynomial kernels, and RBF kernels.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q3. Is it possible to visualize support vector machines with only two-dimensional data?<\/strong><br>A3. While visualization is intuitively possible with two- or three-dimensional data, higher-dimensional data is harder to represent visually. However, SVMs are applicable to higher-dimensional data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q4. What are the advantages and disadvantages of SVM?<\/strong><br>A4. Advantages include good performance on high-dimensional data and resistance to overfitting. The downside is that training time can be long for large amounts of data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Clean up support vector machine visualizations<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this post, we'll take a look at Python's <code>scikit-learn<\/code>using the <strong>Support Vector Machine (SVM)<\/strong>and visualizing the results, we were able to see firsthand how SVMs set classification boundaries on data and how the results are visualized. In machine learning, visualization is a very useful tool for understanding and improving the performance of a model. Try scaling up your SVM model with more complex data and different kernels!<\/p>","protected":false},"excerpt":{"rendered":"<p>In classification problems in machine learning, Support Vector Machines (SVMs) are widely...<\/p>","protected":false},"author":3,"featured_media":2413,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[7],"tags":[208,211,210,206],"class_list":["post-2409","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-coding","tag-sklearn-","tag-svm-","tag-210","tag-206"],"taxonomy_info":{"category":[{"value":7,"label":"\uc778\uacf5\uc9c0\ub2a5(AI)"}],"post_tag":[{"value":208,"label":"Sklearn \ud29c\ud1a0\ub9ac\uc5bc"},{"value":211,"label":"SVM \uc2dc\uac01\ud654"},{"value":210,"label":"\uc11c\ud3ec\ud2b8 \ubca1\ud130 \uba38\uc2e0"},{"value":206,"label":"\ud30c\uc774\uc36c \uba38\uc2e0\ub7ec\ub2dd"}]},"featured_image_src_large":["https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/\uacfc\uc811\ud569-\ud574\uacb0\ubc29\ubc95-\ud3ec\uc2a4\ud2b8-\uc378\ub124\uc77c-600x600.jpg",600,600,true],"author_info":{"display_name":"TERE","author_link":"https:\/\/secondlife.lol\/en\/author\/tere\/"},"comment_info":0,"category_info":[{"term_id":7,"name":"\uc778\uacf5\uc9c0\ub2a5(AI)","slug":"ai-coding","term_group":0,"term_taxonomy_id":7,"taxonomy":"category","description":"","parent":20,"count":72,"filter":"raw","cat_ID":7,"category_count":72,"category_description":"","cat_name":"\uc778\uacf5\uc9c0\ub2a5(AI)","category_nicename":"ai-coding","category_parent":20}],"tag_info":[{"term_id":208,"name":"Sklearn \ud29c\ud1a0\ub9ac\uc5bc","slug":"sklearn-%ed%8a%9c%ed%86%a0%eb%a6%ac%ec%96%bc","term_group":0,"term_taxonomy_id":208,"taxonomy":"post_tag","description":"","parent":0,"count":3,"filter":"raw"},{"term_id":211,"name":"SVM \uc2dc\uac01\ud654","slug":"svm-%ec%8b%9c%ea%b0%81%ed%99%94","term_group":0,"term_taxonomy_id":211,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":210,"name":"\uc11c\ud3ec\ud2b8 \ubca1\ud130 \uba38\uc2e0","slug":"%ec%84%9c%ed%8f%ac%ed%8a%b8-%eb%b2%a1%ed%84%b0-%eb%a8%b8%ec%8b%a0","term_group":0,"term_taxonomy_id":210,"taxonomy":"post_tag","description":"","parent":0,"count":2,"filter":"raw"},{"term_id":206,"name":"\ud30c\uc774\uc36c \uba38\uc2e0\ub7ec\ub2dd","slug":"%ed%8c%8c%ec%9d%b4%ec%8d%ac-%eb%a8%b8%ec%8b%a0%eb%9f%ac%eb%8b%9d","term_group":0,"term_taxonomy_id":206,"taxonomy":"post_tag","description":"","parent":0,"count":3,"filter":"raw"}],"_links":{"self":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2409","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/comments?post=2409"}],"version-history":[{"count":6,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2409\/revisions"}],"predecessor-version":[{"id":2418,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2409\/revisions\/2418"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media\/2413"}],"wp:attachment":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media?parent=2409"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/categories?post=2409"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/tags?post=2409"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}