{"id":2420,"date":"2024-09-06T16:10:47","date_gmt":"2024-09-06T07:10:47","guid":{"rendered":"https:\/\/secondlife.lol\/?p=2420"},"modified":"2024-09-05T16:47:22","modified_gmt":"2024-09-05T07:47:22","slug":"python-3d-svm-visualization","status":"publish","type":"post","link":"https:\/\/secondlife.lol\/en\/python-3d-svm-visualization\/","title":{"rendered":"Support Vector Machine Example - Visualize in Three Dimensions"},"content":{"rendered":"<p class=\"wp-block-paragraph\">Support vector machines (SVMs), one of the most popular algorithms for data classification, are powerful on three-dimensional data as well as two-dimensional. In this Support Vector Machine examples post, we'll use the <strong>Python<\/strong>and <code>scikit-learn<\/code>, <code>matplotlib<\/code> library to use the <strong>Three-dimensional support vector machine (SVM)<\/strong>and how to visualize the results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Classification boundaries in three-dimensional space can be more complex, but visualization can make it easier to understand. Let's take a look at this example to see how SVMs handle three-dimensional data.<\/p>\n\n\n<style>.kb-image2420_676f41-75 .kb-image-has-overlay:after{opacity:0.3;border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;}.kb-image2420_676f41-75 img.kb-img, .kb-image2420_676f41-75 .kb-img img{border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}<\/style>\n<div class=\"wp-block-kadence-image kb-image2420_676f41-75\"><figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"600\" height=\"620\" src=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2326-600x620.jpeg\" alt=\"\uc11c\ud3ec\ud2b8 \ubca1\ud2b8 \uba38\uc2e0 \uc608\uc81c 3\ucc28\uc6d0 \ud3ec\uc2a4\ud2b8 \uadf8\ub9bc\" class=\"kb-img wp-image-2425\" srcset=\"https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2326-600x620.jpeg 600w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2326-300x310.jpeg 300w, https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/IMG_2326.jpeg 637w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><figcaption>(Support Vector Machine Example Run Result - 3-D )<\/figcaption><\/figure><\/div>\n\n\n<style>.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);border-top-left-radius:5px;border-top-right-radius:5px;border-bottom-right-radius:5px;border-bottom-left-radius:5px;box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title-wrap{color:var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{color:var(--global-palette2, #2B6CB0);font-size:28px;font-weight:regular;font-style:normal;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list{color:var(--global-palette1, #3182CE);line-height:2em;font-weight:regular;font-style:normal;margin-top:var(--global-kb-spacing-sm, 1.5rem);margin-right:0px;margin-bottom:0px;margin-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap .kb-table-of-content-list .kb-table-of-contents__entry:hover{color:var(--global-palette6, #718096);}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}}@media all and (max-width: 1024px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}@media all and (max-width: 767px){.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-content-wrap{border-top:3px solid var(--global-palette2, #2B6CB0);border-right:3px solid var(--global-palette2, #2B6CB0);border-bottom:3px solid var(--global-palette2, #2B6CB0);border-left:3px solid var(--global-palette2, #2B6CB0);}.kb-table-of-content-nav.kb-table-of-content-id83_5f28a6-34 .kb-table-of-contents-title{font-size:28px;}}<\/style>\n\n\n<h2 class=\"wp-block-heading\">Understanding three-dimensional support vector machines (SVMs)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>SVM<\/strong>is an algorithm that finds the optimal boundary (hyperplane) that separates data into two classes. In three dimensions, this hyperplane appears as a plane, dividing the data into different classes. In this example, we'll visualize a 3D classification boundary using data with three features.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Python code step-by-step<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. import the required libraries<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">First, we load the necessary libraries to create the three-dimensional data and implement the SVM.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\nfrom mpl_toolkits.mplot3d import Axes3D # Module for 3D visualization\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.svm import SVC<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>mpl_toolkits.mplot3d.Axes3D<\/code>Modules required to draw 3D graphs.<\/li>\n\n\n\n<li><code>SVC<\/code>: <code>scikit-learn<\/code>Support Vector Machine (SVM) classifier provided by Hadoop, Inc.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2. Create sample data<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Next, generate three-dimensional data to train the SVM.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Generate 3D data with 3 features\nX, y = datasets.make_classification(n_samples=100, n_features=3, n_informative=3, n_redundant=0, random_state=42)\n\n# Separate data for training and testing\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>n_samples=100<\/code>: Generate 100 samples.<\/li>\n\n\n\n<li><code>n_features=3<\/code>: Set each sample to have 3 features.<\/li>\n\n\n\n<li><code>n_informative=3<\/code>: All three features are important information for classification.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3. Create and train an SVM model<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Train an SVM model using the data you created.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Create an SVM model for # 3D data\nmodel = SVC(kernel='linear')\n\nTrain the # model\nmodel.fit(X_train, y_train)<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>kernel='linear'<\/code>: Find classification boundaries between data using a linear kernel.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">4. Visualize SVM 3D results<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You can now save the results of the trained SVM model to <strong>3D visualization<\/strong>In three dimensions, the taxonomy boundary appears as a plane.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Function to visualize the 3D classification boundary of a trained SVM model\ndef plot_3d_decision_boundary(X, y, model):\nfig = plt.figure(figsize=(10, 8))\nax = fig.add_subplot(111, projection='3d')\n\nPlot the # data points in 3D space\nax.scatter(X[:, 0], X[:, 1], X[:, 2], c=y, cmap='coolwarm', s=60, edgecolors='k')\n\nSet up the grid for visualizing the # boundary\nxlim = (X[:, 0].min(), X[:, 0].max())\nylim = (X[:, 1].min(), X[:, 1].max())\nzlim = (X[:, 2].min(), X[:, 2].max())\n\nxx, yy = np.meshgrid(np.linspace(xlim[0], xlim[1], 30),<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Full integration code<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Below is the complete code for a three-dimensional support vector machine (SVM) model, complete with comments.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\nfrom mpl_toolkits.mplot3d import Axes3D # Module for plotting 3D graphs\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.svm import SVC\n\n# 1. Generate 3D data\n# Generate 100 samples with 3 features.\n# n_informative=3: use only 3 significant features, n_redundant=0: no redundant features\nX, y = datasets.make_classification(n_samples=100, n_features=3, n_informative=3,\n                                    n_redundant=0, random_state=42)\n\n# 2. Separate dataset into training and testing\n# Use 701 TP3T of data for training and 301 TP3T of data for testing.\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n\n# 3. Generate SVM model\nCreate an SVM model using a # linear kernel.\nmodel = SVC(kernel='linear')\n\n# 4. Train the model\nTrain the SVM model using the # training data.\nmodel.fit(X_train, y_train)\n\n# 5. Define a 3D visualization function\ndef plot_3d_decision_boundary(X, y, model):\n    Generate a plot for a # 3D graph\n    fig = plt.figure(figsize=(10, 8))\n    ax = fig.add_subplot(111, projection='3d')\n\n    # 5.1 Visualizing data points\n    # Display the data in 3D space, color-coded according to each class.\n    ax.scatter(X[:, 0], X[:, 1], X[:, 2], c=y, cmap='coolwarm', s=60, edgecolors='k')\n\n    # 5.2 Setting up a grid for boundary visualization\n    xlim = (X[:, 0].min(), X[:, 0].max())\n    ylim = (X[:, 1].min(), X[:, 1].max())\n    zlim = (X[:, 2].min(), X[:, 2].max())\n\n    xx, yy = np.meshgrid(np.linspace(xlim[0], xlim[1], 30), np.linspace(ylim[0], ylim[1], 30))\n\n    # 5.3 Calculating the Decision Boundary Plane\n    Define the bounding plane using the weights and intercept of the # linear SVM.\n    The coef_ of the # model represents the slope of the crystal boundary plane.\n    Z = (-model.coef_[0][0] * xx - model.coef_[0][1] * yy - model.intercept_) \/ model.coef_[0][2]\n\n    # 5.4 Visualize the crystal boundary plane\n    ax.plot_surface(xx, yy, Z, color='green', alpha=0.3)\n\n    # 5.5 Set axis labels\n    ax.set_xlabel(\"Feature 1\")\n    ax.set_ylabel(\"Feature 2\")\n    ax.set_zlabel(\"Feature 3\")\n    ax.set_title(\"SVM 3D Decision Boundary\")\n\n    # 5.6 Display the graph\n    plt.show()\n\n# 6. Visualize the SVM decision boundary using test data\nplot_3d_decision_boundary(X_test, y_test, model)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Code description:<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Generate data<\/strong>: <code>make_classification<\/code> function to generate three-dimensional data so that it can be trained with an SVM. <code>n_features=3<\/code>to create three-dimensional data.<\/li>\n\n\n\n<li><strong>Partitioning data<\/strong>: <code>train_test_split<\/code>to separate data for training and testing.<\/li>\n\n\n\n<li><strong>Create and train SVM models<\/strong>: <code>SVC<\/code> class to create an SVM model with a linear kernel, and train the model on training data.<\/li>\n\n\n\n<li><strong>3D visualization<\/strong>: <code>mpl_toolkits.mplot3d.Axes3D<\/code> module to implement 3D visualization and visualize the classification boundary plane trained by the SVM. <code>ax.plot_surface<\/code> function to visualize the decision boundary plane of an SVM.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions (FAQ)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q1. What are the decision boundaries for SVMs on three-dimensional data?<\/strong><br>A1. In three-dimensional data, the decision boundary of an SVM appears as a plane. This plane separates the two classes, and the SVM learns by maximizing the distance of this plane from the nearest data points (support vectors).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q2. What are kernel functions?<\/strong><br>A2. Kernel functions are functions that help SVMs transform data into a higher-dimensional space so that they can linearly separate nonlinear data. In this example, we will use a linear kernel (<code>linear<\/code>), but there are many other kernels out there, including RBF kernels, polynomial kernels, and more.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q3. <code>coef_<\/code>and <code>intercept_<\/code>is?<\/strong><br>A3. <code>coef_<\/code>is a value that represents the slope of the decision boundary in the SVM, <code>intercept_<\/code>represents the intercept of the decision boundary. These two values allow you to define the classification boundary (plane).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q4. Can I visualize higher dimensional data besides three-dimensional data?<\/strong><br>A4. You can visualize up to three dimensions, but not more than four. However, SVMs can be powerful on high-dimensional data. When dealing with high-dimensional data, dimensionality reduction techniques (such as PCA) can be used to visualize it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q5. Can SVMs be used to classify non-linear data?<\/strong><br>A5. Yes, SVMs can be applied using nonlinear kernels, such as RBF kernels or polynomial kernels, to classify nonlinear data. Kernel methods can be used to effectively classify even complex data distributions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Finalize<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this post, we learned how to implement a three-dimensional support vector machine (SVM) using Python and visualize the results. The visualized results gave us an intuitive understanding of how SVMs set classification boundaries and learn from three-dimensional data.<br>Apply SVMs to real-world data and explore their performance further by using different kernel functions or high-dimensional data.<\/p>","protected":false},"excerpt":{"rendered":"<p>One of the most frequently used algorithms in data classification, the support vector machine (Support...<\/p>","protected":false},"author":3,"featured_media":2421,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[7],"tags":[208,212,210,206],"class_list":["post-2420","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-coding","tag-sklearn-","tag-svm-3d-","tag-210","tag-206"],"taxonomy_info":{"category":[{"value":7,"label":"\uc778\uacf5\uc9c0\ub2a5(AI)"}],"post_tag":[{"value":208,"label":"Sklearn \ud29c\ud1a0\ub9ac\uc5bc"},{"value":212,"label":"SVM 3D \uc2dc\uac01\ud654"},{"value":210,"label":"\uc11c\ud3ec\ud2b8 \ubca1\ud130 \uba38\uc2e0"},{"value":206,"label":"\ud30c\uc774\uc36c \uba38\uc2e0\ub7ec\ub2dd"}]},"featured_image_src_large":["https:\/\/secondlife.lol\/wp-content\/uploads\/2024\/09\/\uacfc\uc811\ud569-\ud574\uacb0\ubc29\ubc95-\ud3ec\uc2a4\ud2b8-\uc378\ub124\uc77c-1-600x600.jpg",600,600,true],"author_info":{"display_name":"TERE","author_link":"https:\/\/secondlife.lol\/en\/author\/tere\/"},"comment_info":0,"category_info":[{"term_id":7,"name":"\uc778\uacf5\uc9c0\ub2a5(AI)","slug":"ai-coding","term_group":0,"term_taxonomy_id":7,"taxonomy":"category","description":"","parent":20,"count":74,"filter":"raw","cat_ID":7,"category_count":74,"category_description":"","cat_name":"\uc778\uacf5\uc9c0\ub2a5(AI)","category_nicename":"ai-coding","category_parent":20}],"tag_info":[{"term_id":208,"name":"Sklearn \ud29c\ud1a0\ub9ac\uc5bc","slug":"sklearn-%ed%8a%9c%ed%86%a0%eb%a6%ac%ec%96%bc","term_group":0,"term_taxonomy_id":208,"taxonomy":"post_tag","description":"","parent":0,"count":3,"filter":"raw"},{"term_id":212,"name":"SVM 3D \uc2dc\uac01\ud654","slug":"svm-3d-%ec%8b%9c%ea%b0%81%ed%99%94","term_group":0,"term_taxonomy_id":212,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":210,"name":"\uc11c\ud3ec\ud2b8 \ubca1\ud130 \uba38\uc2e0","slug":"%ec%84%9c%ed%8f%ac%ed%8a%b8-%eb%b2%a1%ed%84%b0-%eb%a8%b8%ec%8b%a0","term_group":0,"term_taxonomy_id":210,"taxonomy":"post_tag","description":"","parent":0,"count":2,"filter":"raw"},{"term_id":206,"name":"\ud30c\uc774\uc36c \uba38\uc2e0\ub7ec\ub2dd","slug":"%ed%8c%8c%ec%9d%b4%ec%8d%ac-%eb%a8%b8%ec%8b%a0%eb%9f%ac%eb%8b%9d","term_group":0,"term_taxonomy_id":206,"taxonomy":"post_tag","description":"","parent":0,"count":3,"filter":"raw"}],"_links":{"self":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2420","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/comments?post=2420"}],"version-history":[{"count":7,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2420\/revisions"}],"predecessor-version":[{"id":2432,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/posts\/2420\/revisions\/2432"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media\/2421"}],"wp:attachment":[{"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/media?parent=2420"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/categories?post=2420"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/secondlife.lol\/en\/wp-json\/wp\/v2\/tags?post=2420"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}