2024-11-26 03:02:51 +04:00
{
"cells": [
2024-11-29 00:09:21 +04:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Цель работы\n",
"Мы будем кластеризовать автомобили, основываясь на их характеристиках, с с
]
},
2024-11-26 03:02:51 +04:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# загрузим датасет"
]
},
{
"cell_type": "code",
2024-11-28 19:55:15 +04:00
"execution_count": 2,
2024-11-26 03:02:51 +04:00
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Price</th>\n",
" <th>Levy</th>\n",
" <th>Manufacturer</th>\n",
" <th>Model</th>\n",
" <th>Prod. year</th>\n",
" <th>Category</th>\n",
" <th>Leather interior</th>\n",
" <th>Fuel type</th>\n",
" <th>Engine volume</th>\n",
" <th>Mileage</th>\n",
" <th>Cylinders</th>\n",
" <th>Gear box type</th>\n",
" <th>Drive wheels</th>\n",
" <th>Doors</th>\n",
" <th>Wheel</th>\n",
" <th>Color</th>\n",
" <th>Airbags</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>45654403</td>\n",
" <td>13328</td>\n",
" <td>1399</td>\n",
" <td>LEXUS</td>\n",
" <td>RX 450</td>\n",
" <td>2010</td>\n",
" <td>Jeep</td>\n",
" <td>Yes</td>\n",
" <td>Hybrid</td>\n",
" <td>3.5</td>\n",
" <td>186005 km</td>\n",
" <td>6.0</td>\n",
" <td>Automatic</td>\n",
" <td>4x4</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Silver</td>\n",
" <td>12</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>44731507</td>\n",
" <td>16621</td>\n",
" <td>1018</td>\n",
" <td>CHEVROLET</td>\n",
" <td>Equinox</td>\n",
" <td>2011</td>\n",
" <td>Jeep</td>\n",
" <td>No</td>\n",
" <td>Petrol</td>\n",
" <td>3</td>\n",
" <td>192000 km</td>\n",
" <td>6.0</td>\n",
" <td>Tiptronic</td>\n",
" <td>4x4</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Black</td>\n",
" <td>8</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>45774419</td>\n",
" <td>8467</td>\n",
" <td>-</td>\n",
" <td>HONDA</td>\n",
" <td>FIT</td>\n",
" <td>2006</td>\n",
" <td>Hatchback</td>\n",
" <td>No</td>\n",
" <td>Petrol</td>\n",
" <td>1.3</td>\n",
" <td>200000 km</td>\n",
" <td>4.0</td>\n",
" <td>Variator</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Right-hand drive</td>\n",
" <td>Black</td>\n",
" <td>2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>45769185</td>\n",
" <td>3607</td>\n",
" <td>862</td>\n",
" <td>FORD</td>\n",
" <td>Escape</td>\n",
" <td>2011</td>\n",
" <td>Jeep</td>\n",
" <td>Yes</td>\n",
" <td>Hybrid</td>\n",
" <td>2.5</td>\n",
" <td>168966 km</td>\n",
" <td>4.0</td>\n",
" <td>Automatic</td>\n",
" <td>4x4</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>White</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>45809263</td>\n",
" <td>11726</td>\n",
" <td>446</td>\n",
" <td>HONDA</td>\n",
" <td>FIT</td>\n",
" <td>2014</td>\n",
" <td>Hatchback</td>\n",
" <td>Yes</td>\n",
" <td>Petrol</td>\n",
" <td>1.3</td>\n",
" <td>91901 km</td>\n",
" <td>4.0</td>\n",
" <td>Automatic</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Silver</td>\n",
" <td>4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19232</th>\n",
" <td>45798355</td>\n",
" <td>8467</td>\n",
" <td>-</td>\n",
" <td>MERCEDES-BENZ</td>\n",
" <td>CLK 200</td>\n",
" <td>1999</td>\n",
" <td>Coupe</td>\n",
" <td>Yes</td>\n",
" <td>CNG</td>\n",
" <td>2.0 Turbo</td>\n",
" <td>300000 km</td>\n",
" <td>4.0</td>\n",
" <td>Manual</td>\n",
" <td>Rear</td>\n",
" <td>02-Mar</td>\n",
" <td>Left wheel</td>\n",
" <td>Silver</td>\n",
" <td>5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19233</th>\n",
" <td>45778856</td>\n",
" <td>15681</td>\n",
" <td>831</td>\n",
" <td>HYUNDAI</td>\n",
" <td>Sonata</td>\n",
" <td>2011</td>\n",
" <td>Sedan</td>\n",
" <td>Yes</td>\n",
" <td>Petrol</td>\n",
" <td>2.4</td>\n",
" <td>161600 km</td>\n",
" <td>4.0</td>\n",
" <td>Tiptronic</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Red</td>\n",
" <td>8</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19234</th>\n",
" <td>45804997</td>\n",
" <td>26108</td>\n",
" <td>836</td>\n",
" <td>HYUNDAI</td>\n",
" <td>Tucson</td>\n",
" <td>2010</td>\n",
" <td>Jeep</td>\n",
" <td>Yes</td>\n",
" <td>Diesel</td>\n",
" <td>2</td>\n",
" <td>116365 km</td>\n",
" <td>4.0</td>\n",
" <td>Automatic</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Grey</td>\n",
" <td>4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19235</th>\n",
" <td>45793526</td>\n",
" <td>5331</td>\n",
" <td>1288</td>\n",
" <td>CHEVROLET</td>\n",
" <td>Captiva</td>\n",
" <td>2007</td>\n",
" <td>Jeep</td>\n",
" <td>Yes</td>\n",
" <td>Diesel</td>\n",
" <td>2</td>\n",
" <td>51258 km</td>\n",
" <td>4.0</td>\n",
" <td>Automatic</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>Black</td>\n",
" <td>4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19236</th>\n",
" <td>45813273</td>\n",
" <td>470</td>\n",
" <td>753</td>\n",
" <td>HYUNDAI</td>\n",
" <td>Sonata</td>\n",
" <td>2012</td>\n",
" <td>Sedan</td>\n",
" <td>Yes</td>\n",
" <td>Hybrid</td>\n",
" <td>2.4</td>\n",
" <td>186923 km</td>\n",
" <td>4.0</td>\n",
" <td>Automatic</td>\n",
" <td>Front</td>\n",
" <td>04-May</td>\n",
" <td>Left wheel</td>\n",
" <td>White</td>\n",
" <td>12</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>19237 rows ×
"</div>"
],
"text/plain": [
" ID Price Levy Manufacturer Model Prod. year Category \\\n",
"0 45654403 13328 1399 LEXUS RX 450 2010 Jeep \n",
"1 44731507 16621 1018 CHEVROLET Equinox 2011 Jeep \n",
"2 45774419 8467 - HONDA FIT 2006 Hatchback \n",
"3 45769185 3607 862 FORD Escape 2011 Jeep \n",
"4 45809263 11726 446 HONDA FIT 2014 Hatchback \n",
"... ... ... ... ... ... ... ... \n",
"19232 45798355 8467 - MERCEDES-BENZ CLK 200 1999 Coupe \n",
"19233 45778856 15681 831 HYUNDAI Sonata 2011 Sedan \n",
"19234 45804997 26108 836 HYUNDAI Tucson 2010 Jeep \n",
"19235 45793526 5331 1288 CHEVROLET Captiva 2007 Jeep \n",
"19236 45813273 470 753 HYUNDAI Sonata 2012 Sedan \n",
"\n",
" Leather interior Fuel type Engine volume Mileage Cylinders \\\n",
"0 Yes Hybrid 3.5 186005 km 6.0 \n",
"1 No Petrol 3 192000 km 6.0 \n",
"2 No Petrol 1.3 200000 km 4.0 \n",
"3 Yes Hybrid 2.5 168966 km 4.0 \n",
"4 Yes Petrol 1.3 91901 km 4.0 \n",
"... ... ... ... ... ... \n",
"19232 Yes CNG 2.0 Turbo 300000 km 4.0 \n",
"19233 Yes Petrol 2.4 161600 km 4.0 \n",
"19234 Yes Diesel 2 116365 km 4.0 \n",
"19235 Yes Diesel 2 51258 km 4.0 \n",
"19236 Yes Hybrid 2.4 186923 km 4.0 \n",
"\n",
" Gear box type Drive wheels Doors Wheel Color Airbags \n",
"0 Automatic 4x4 04-May Left wheel Silver 12 \n",
"1 Tiptronic 4x4 04-May Left wheel Black 8 \n",
"2 Variator Front 04-May Right-hand drive Black 2 \n",
"3 Automatic 4x4 04-May Left wheel White 0 \n",
"4 Automatic Front 04-May Left wheel Silver 4 \n",
"... ... ... ... ... ... ... \n",
"19232 Manual Rear 02-Mar Left wheel Silver 5 \n",
"19233 Tiptronic Front 04-May Left wheel Red 8 \n",
"19234 Automatic Front 04-May Left wheel Grey 4 \n",
"19235 Automatic Front 04-May Left wheel Black 4 \n",
"19236 Automatic Front 04-May Left wheel White 12 \n",
"\n",
"[19237 rows x 18 columns]"
]
},
2024-11-28 19:55:15 +04:00
"execution_count": 2,
2024-11-26 03:02:51 +04:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"df = pd.read_csv(\"..//static//csv//car_price_prediction.csv\", sep=\",\")\n",
"df\n"
]
2024-11-28 19:55:15 +04:00
},
2024-11-29 00:09:21 +04:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Предобработка данных\n",
"Мы удалим неинформативные столбцы, такие как ID, преобразуем категориальные переменные в числовые (one-hot encoding), а
]
},
2024-11-28 19:55:15 +04:00
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
2024-11-29 00:09:21 +04:00
"outputs": [],
"source": [
"# Удаляем неинформативный столбец ID\n",
"df = df.drop(columns=[\"ID\"])\n",
"\n",
"# Преобразование категориальных данных в числовые с
"df = pd.get_dummies(df, drop_first=True)\n",
"\n",
"# Нормализация числовых данных\n",
"from sklearn.preprocessing import StandardScaler\n",
"\n",
"scaler = StandardScaler()\n",
"df_scaled = scaler.fit_transform(df)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Визуализация данных с
"Для визуализации мы применим метод PCA, который уменьшит количество измерений до двух, сохраняя при этом максимальное количество информации. \n",
"Ключевые термины:\n",
"- PCA (Principal Component Analysis) — метод снижения размерности, который находит новые оси в данных, вдоль которых разброс максимален, и проецирует данные на эти оси.\n",
"- Снижение размерности — процесс упрощения данных за счёт уменьшения числа признаков."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
2024-11-28 19:55:15 +04:00
"outputs": [
{
"data": {
2024-11-29 00:09:21 +04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAIjCAYAAADslLiSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOydZ3hU1daA35n0HpIQQkiooSNFQKWJCigoIBaKegH1qij23rte7B94rwUbTaRbsIBItYEgvUNCTUICSUhCembmfD/WngwhoQRBIK73efJAZk7ZZ58zmfess/baNsuyLBRFURRFURSlmmE/0w1QFEVRFEVRlNOBiq6iKIqiKIpSLVHRVRRFURRFUaolKrqKoiiKoihKtURFV1EURVEURamWqOgqiqIoiqIo1RIVXUVRFEVRFKVaoqKrKIqiKIqiVEtUdBVFURRFUZRqiYquoiiKoiiKUi1R0f2HMn78eGw2W9mPv78/TZo04Z577iE9Pb3C8unp6TzyyCM0a9aMwMBAgoKCaN++Pa+88grZ2dmV7uOCCy7AZrPxwQcf/OX23nzzzeXa6+3tTXx8PEOGDGHTpk1/efuKoiiKolQ/vM90A5Qzy0svvUSDBg0oKiri119/5YMPPuCHH35gw4YNBAYGArBixQquvPJK8vLy+Ne//kX79u0B+PPPP3nttdf4+eefmTdvXrntbt++nRUrVlC/fn0mT57MXXfd9Zfb6ufnxyeffAKAw+EgKSmJDz/8kLlz57Jp0yZiY2P/8j4URVEURak+qOj+w+nTpw8dOnQA4LbbbiMyMpJ33nmHb775hhtuuIHs7GyuueYavLy8WL16Nc2aNSu3/quvvsrHH39cYbuff/450dHRvP3221x//fXs2rWL+vXr/6W2ent7869//avcaxdddBF9+/bl+++/5/bbb/9L21cURVEUpXqhqQtKOS677DIAdu7cCcDYsWNJSUnhnXfeqSC5ALVq1eKZZ56p8PoXX3zB9ddfT9++fQkLC+OLL76osExBQQFbtmwhIyPjpNsbExMDiAS7cadl7Nq1q+w1l8tF69atsdlsjB8/vuz1F154gRYtWhAcHExoaCgXXXQRX3/9ddn7zz//PD4+Phw4cKDCvu+44w7Cw8MpKioC4JtvvuGqq64iNjYWPz8/GjVqxMsvv4zT6ayw7q5du8qlYhz+c+Qyh7cX4O6778Zms3HzzTeXvfbll19ywQUXEBERQUBAAM2aNeP111/HsqyyZXbv3s3IkSNp2rQpAQEBREZGMnDgwHL9dHj//fnnn+Vez8jIwGaz8cILL5R7vbLX3nzzTWw2G5dcckm513fs2MHAgQOJjY3FbreXHXOrVq0q9FFlLF68+Lj9drqO9YUXXsBms1W4Xv/8888K5+nmm28mODj4mMdy+PYLCwtp1qwZzZo1o7CwsGyZrKwsateuTefOnSu9jo48jqP9HHl+Vq9eTZ8+fQgNDSU4OJgePXqwbNmyCtvNzs7mwQcfpH79+vj5+REXF8ewYcMq9MGRqUVH2++WLVu4/vrriYiIwN/fnw4dOjB79uxj9pMbl8vFmDFjOO+88/D396dmzZr07t273Lmz2Wzcc889Fdbt27dvuRvtqny2xo4di91uZ8aMGcdcf9u2bURERHDjjTeW22Z2djYPPPAA8fHx+Pn5kZCQwOuvv47L5aqwvbfeeqtC21u1alXuc+T+DCxevLjccldddVWlfb5o0SK6detGjRo1yp2byvrpcNz7mTZtGk899RQxMTEEBQXRv39/9u7dW27ZX375hYEDB1K3bl38/PyIj4/nwQcfLHctw/H/3gJccskl2Gw2BgwYUKFNI0aMqPTvhcvlYvTo0bRs2RJ/f39q1arFiBEjOHjwYLnl6tevT9++fZk3bx5t27bF39+fFi1a8OWXX5Zbrqp/A0/F5+lYf9uO/Dy5/xYdTl5eHjExMZVeG/90NKKrlCMpKQmAyMhIAGbPnk1AQADXX3/9CW/jjz/+IDExkXHjxuHr68u1117L5MmTeeqpp8ott3z5ci699FKef/75Cn84job7C9bpdLJjxw4ef/xxIiMj6du37zHXmzRpEuvXr6/wen5+Ptdccw3169ensLCQ8ePHc91117F06VIuuOAChg4dyksvvcS0adPKfTGUlJQwc+ZMrrvuOvz9/QH54xgcHMxDDz1EcHAwCxcu5LnnniM3N5c333yz0nbdcccddOvWDRBZ/eqrr455HImJiZVG0HNzc7nwwgsZPnw4Pj4+zJ07lyeeeAJvb28efvhhQFJQfv/9d4YMGUJcXBy7du3igw8+4JJLLmHTpk1lqSp/lezsbEaNGlXhdafTSf/+/dm9ezcPPPAATZo0wWaz8eqrr1Z5H/fddx8dO3YEYOLEifz000/l3v+7jvVUERAQwIQJE+jSpQtPP/0077zzDiDilZOTw/jx4/Hy8jrudtypSG7y8vIqpA1t3LiRbt26ERoaymOPPYaPjw9jx47lkksuYcmSJVx44YVl63br1o3Nmzdz6623cv7555ORkcHs2bNJTk4mKiqq3HajoqL4v//7v7Lfhw4dWmG/Xbp0oU6dOjzxxBMEBQUxffp0BgwYwKxZs7jmmmuOeWz//ve/GT9+PH369OG2227D4XDwyy+/sGzZsrKnUn+Fo322RowYwdatWxk+fDj169cvu+4OJysri759+9K8eXPGjRtX9npBQQHdu3cnJSWFESNGULduXX7//XeefPJJ9u3bx+jRo/9yuwF+/vlnfvjhhwqv79y5k6uuuoratWvz3HPPUbNmTaDiuTkWr776Kjabjccff5z9+/czevRoevbsyZo1awgICABgxowZFBQUcNdddxEZGcny5cv573//S3JycrkbhOP9vXXj7+/P999/z/79+4mOjgbkZnDatGllf28PZ8SIEYwfP55bbrmF++67j507d/K///2P1atX89tvv+Hj41O27Pbt2xk8eDB33nknw4cPZ9y4cQwcOJC5c+fSq1evE+4XN6fq89S8eXMmTZpUtt2PPvqIzZs3l/tMtW7d+qjtePvttysdX6MAlvKPZNy4cRZgzZ8/3zpw4IC1d+9ea+rUqVZkZKQVEBBgJScnW5ZlWTVq1LDatGlTpW3fc889Vnx8vOVyuSzLsqx58+ZZgLV69epyyy1atMgCrOeff/642xw+fLgFVPipU6eOtXLlykqPbefOnZZlWVZRUZFVt25dq0+fPhZgjRs37qj72b9/vwVYb731VtlrnTp1si688MJyy3355ZcWYC1atKjstYKCggrbGzFihBUYGGgVFRWVe3379u0WYE2YMKHsteeff946/CO5c+fOCu0dNGiQ1apVKys+Pt4aPnz4UY/DsiyrRYsWVt++fY/ZvqVLl1qANXHixLLX3P23YsWKcsseOHCg0vN15GuPPfaYFR0dbbVv397q3r172etbt261AGvUqFHl1u/evbvVsmXLYx6LG/e1NHPmzLLX7r77buvIP2Wn41jd5+fAgQPlll2xYkWF8zR8+HArKCjomMdSWV8++eSTlt1ut37++WdrxowZFmCNHj36mNup6nEMGDDA8vX1tZKSkspeS01NtUJCQqyLL7647LXnnnvOAqwvv/yywv7cn203N910k9WgQYNjHl+PHj2s8847r9xnweVyWZ07d7YaN258zONbuHChBVj33XffMdsCWHfffXeFZa666iqrXr16Zb9X9bPldDqtfv36WbVr17b27t1bbv2SkhLrkksusRo0aGDt37+/3Hovv/yyFRQUZG3btq3c60888YTl5eVl7dmzp1x73nzzzQptb9myZbnPkfvv5uF/ey688MKyv2+H9/nYsWMtwFq6dGm5bR6tnw7HvZ86depYubm5Za9Pnz7dAqwxY8aUvVbZ523UqFGWzWazdu/efdR9VPb31v33oHXr1uVenzRpkhUXF2d169at3N+LX375xQKsyZMnl9v23LlzK7xer149C7BmzZpV9lpOTo5Vu3Ztq127dmWvnenPk2XJ35DDr9nDOfK7Yv/+/VZISEjZNXD4taFYlqYu/MPp2bMnNWvWLKt
2024-11-28 19:55:15 +04:00
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
2024-11-29 00:09:21 +04:00
}
],
"source": [
"# Импортируем PCA и визуализируем данные\n",
"from sklearn.decomposition import PCA\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Применяем PCA для снижения размерности до 2\n",
"pca = PCA(n_components=2)\n",
"df_pca = pca.fit_transform(df_scaled)\n",
"\n",
"# Визуализация\n",
"plt.figure(figsize=(8, 6))\n",
"plt.scatter(df_pca[:, 0], df_pca[:, 1], c='blue', edgecolor='k', alpha=0.6)\n",
"plt.title(\"PCA: Визуализация данных после снижения размерности\")\n",
"plt.xlabel(\"Главная компонента 1\")\n",
"plt.ylabel(\"Главная компонента 2\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Количество кластеров\n",
"Количество кластеров напрямую влияет на результаты кластеризации, так как оно определяет, сколько групп или сегментов будет выделено в данных. Оптимальный выбор количества кластеров важен, чтобы обеспечить баланс между точностью кластеризации и интерпретируемостью результатов. \n",
"# Зачем выбирать количество кластеров?\n",
"## Оптимальная сегментация данных\n",
"Разное количество кластеров может приводить к слишком мелкому делению (много мелких кластеров) или слишком крупному (слишком обобщённые кластеры).\n",
"-Слишком мало кластеров: важные различия в данных могут быть упущены.\n",
"-Слишком много кластеров: анализ становится сложным, и кластеры могут быть избыточно раздроблены.\n",
"## Интерпретируемость результатов\n",
"Оптимальное количество кластеров делает результаты понятными и полезными. Например, выделение 3-5 кластеров может быть удобно для анализа, тогда как 15-20 кластеров усложнят интерпретацию.\n",
"## Избежание переобучения или недообучения\n",
"Количество кластеров влияет на обобщающую способность модели. Слишком большое количество кластеров может привести к переобучению (модель подстраивается под шум), а
"## Практическая применимость\n",
"В
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Определение оптимального количества кластеров\n",
"Для выбора количества кластеров мы применим: \n",
"- Метод локтя — измеряет инерцию (размерность ошибок внутри кластеров).\n",
"- Коэффициент силуэта — показывает, насколько хорошо объекты распределены между кластерами.\n",
" \n",
"Ключевые термины: \n",
"- Инерция — сумма квадратов расстояний от точек до центроидов их кластеров. Чем меньше, тем лучше.\n",
"- Коэффициент силуэта — оценивает плотность внутри кластеров и разницу между ними (от -1 до 1)."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
2024-11-28 19:55:15 +04:00
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAIjCAYAAAD4ASZzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB+50lEQVR4nO3dd1gUV9sG8Ht2WWClSgdFQbEhKiKx16ixosaCscQWS2KJNa8aP6PGGBKTqNEYjRq7xhK7MXZjRREV7IgKIkgRkC5td74/fNnXDSCgwCxw/65rr2Rnzs48M7uLN8OZcwRRFEUQEREREVG+ZFIXQERERESk6xiaiYiIiIgKwNBMRERERFQAhmYiIiIiogIwNBMRERERFYChmYiIiIioAAzNREREREQFYGgmIiIiIiqAntQFEBHRu8vMzER8fDzUajUcHBykLoeIqNzhlWYiojLK398fgwcPhpWVFQwMDGBvb49+/fpJXRYRUbnE0EwVzsaNGyEIAgRBwIULF3KtF0URjo6OEAQBPXv2lKBCooIdOHAArVu3xt27d7Fo0SKcOHECJ06cwG+//SZ1aURE5RK7Z1CFZWhoiO3bt6N169Zay8+ePYvw8HAYGBhIVBnRm8XHx2P06NHo0qULdu/eDX19falLIiIq93ilmSqs7t27Y/fu3cjOztZavn37djRp0gR2dnYSVUb0Zhs2bEB6ejo2btzIwExEVEoYmqnCGjRoEOLi4nDixAnNsszMTPz5558YPHhwnq9Rq9VYtmwZ6tevD0NDQ9ja2mLcuHF48eKFpo2Tk5Om+0deDycnJ03b1NRUTJ8+HY6OjjAwMECdOnXw448/QhTFXPv+559/8t1mYY0YMSLP18+fP1+r3enTp9GmTRsYGRnB3NwcvXv3xr1797TazJ8/P9e+z5w5AwMDA3z66adabd70+OeffzSvX7VqFdzc3FCpUiWtNn/++Wehjq99+/aFOj5Au5vO64/27dtrtbtx4wa6du0Ka2trrXYFdd0JDQ3Vaq9QKODk5IQvvvgCmZmZuerw9/d/43G9Xtfly5fh7u6Ob7/9VvPZqVWrFr777juo1Wqt12ZnZ2PhwoWoWbMmDAwM4OTkhC+//BIZGRla7ZycnNCzZ08cP34c7u7uMDQ0hKurK/bu3avVLj4+HjNmzECDBg1gbGwMU1NTdOvWDYGBgW88Hzn+/X5kZ2eje/fusLCwwN27d4tcN5D/d+P171pOm9c/bwDQo0ePXDX9+3wD/3s/N27cqLX8/v376N+/PywsLGBoaAhPT08cPHgwV40JCQmYOnUqnJycYGBggKpVq2LYsGGIjY1943f735/hf3+nTExM0LRpU+zfv19rf+fPn8eAAQNQrVo1GBgYwNHREVOnTsXLly9z1fZvOZ/J0NBQzbI7d+6gcuXK6NmzZ64LDfl9714/V0Wp5/79+/D29oa1tTWUSiXq1KmDOXPm5Hn8Bf1MuXLlCrp27QozMzNUqlQJ7dq1w8WLF7X2l7PNnP2amprC0tISkydPRnp6ulbbonyfcuqRyWSws7PDwIEDERYWVuD5J93F7hlUYTk5OaFFixb4448/0K1bNwDA33//jcTERHz00UdYvnx5rteMGzcOGzduxMiRI/H5558jJCQEv/zyC27cuIGLFy9CoVBg2bJlSElJAQDcu3cP3377Lb788kvUq1cPAGBsbAzgVd/pXr164cyZM/jkk0/g7u6OY8eO4YsvvkBERASWLl2aZ92ff/453nvvPQDA5s2btUJ/YVhZWWlt++OPP9Zaf/LkSXTr1g01atTA/Pnz8fLlS6xYsQKtWrXC9evXtYLI6wIDA9GnTx90794dK1euBAD07dsXLi4umjZTp05FvXr1MHbsWM2ynPOyc+dOjB8/Hu3bt8ekSZNgZGSkOX9FUbVqVfj4+AAAUlJS8Nlnn72x/dKlS2FlZQUAWLRokda6xMREdOvWDaIoYtq0aXB0dNQcR2GNHTsWbdq0QUZGBo4dO4Yff/wRhoaGWLhwYVEOS0tcXBwuXLiACxcuYNSoUWjSpAlOnTqF2bNnIzQ0FKtXr9a0HT16NDZt2oT+/ftj+vTpuHLlCnx8fHDv3j3s27dPa7vBwcEYOHAgPv30UwwfPhwbNmzAgAEDcPToUXTu3BkA8PjxY+zfvx8DBgyAs7MzoqOj8dtvv6Fdu3a4e/dukUfuGD16NP755x+cOHECrq6ub1V3jte/Z2vWrCkwoJw7dw5HjhwpUr2vu3PnDlq1aoUqVapg1qxZMDIywq5du9CnTx/s2bMHH374IYBXn8M2bdrg3r17GDVqFDw8PBAbG4uDBw8iPDwc9erVw5YtWzTbXbNmDe7du6f1PW3YsKHWvnPax8bG4tdff8WAAQNw+/Zt1KlTBwCwe/dupKWl4bPPPoOlpSX8/PywYsUKhIeHY/fu3UU6zqdPn6Jr166oW7cudu3aBT293NGhbt26mmAbGxub6ztS2Hpu3ryJNm3aQKFQYOzYsXBycsKjR49w6NAhLFq0qEg/U06fPo1u3bqhSZMmmDdvHmQyGTZs2ID3338f58+fR9OmTbVq9Pb2hpOTE3x8fHD58mUsX74cL168wObNmzVtivK5bNOmDcaOHQu1Wo3bt29j2bJlePbsGc6fP1+k8086RCSqYDZs2CACEK9evSr+8ssvoomJiZiWliaKoigOGDBA7NChgyiKoli9enWxR48emtedP39eBCBu27ZNa3tHjx7Nc7koiuKZM2dEAOKZM2dyrdu/f78IQPzmm2+0lvfv318UBEF8+PCh1vLjx4+LAMQ///xTs2zChAliUb7GQ4YMEZ2dnbWWARDnzZunee7u7i7a2NiIcXFxmmWBgYGiTCYThw0bplk2b948zb5DQ0NFe3t7sXXr1uLLly/z3X/16tXF4cOH57lu0KBBorm5udbrc87f7t27C3V8LVu2FN3c3DTPnz9/nuv4cqxdu1YEID558kSzrF27dmK7du00z48dOyYCEP/4449cx/H6ZyMvISEhIgBxw4YNWssdHBzE7t27a56//nnMz7/rateunQhAnD9/vla7ESNGiADEW7duiaIoigEBASIAcfTo0VrtZsyYIQIQT58+rXVMAMQ9e/ZoliUmJor29vZi48aNNcvS09NFlUqV61gNDAzEr7/+Ot9jyPH6+zF79mxRLpeL+/fv12pTlLpFURRPnDghAhDPnj2rWTZ8+HCxevXqmud5fRebNWsmduvWLddnpEOHDmLbtm1zHeO/38+OHTuKDRo0ENPT0zXL1Gq12LJlS7FWrVqaZV999ZUIQNy7d2+u86FWq3Mt+3ftr3v9e5cj52fDrl27NMtyfqa9zsfHRxQEQeszn5ecz2RISIgYHx8vurq6inXq1BFjY2PzbN+qVSvNz01RzPtcFbaetm3biiYmJrlqzOs8iWL+P1PUarVYq1YtsUuXLlqvTUtLE52dncXOnTtrluWc0169emltY/z48SIAMTAwUBTFon+f/l3X4MGDxUqVKuV5HFQ2sHsGVWje3t54+fIlDh8+jOTkZBw+fDjfrhm7d++GmZkZOnfujNjYWM2jSZMmMDY2xpkzZ4q07yNHjkAul+Pzzz/XWj59+nSIooi///5ba3nOnwkNDQ2LtJ/XZWZmvvEGx8jISAQEBGDEiBGwsLDQLG/YsCE6d+6c51W5uLg4dOnSBSYmJjh48OBb15ecnIxKlSq90/Glp6cX+vU5XSTedD6Sk5MBAJaWlm9dU0pKCmJjYxEREYE1a9YgKioKHTt2zNUuMTERsbGxmn0WRC6X57qaN336dADAX3/9BQCa92vatGlvbJfDwcFBc3UUAExNTTFs2DDcuHEDUVFRAF6dL5ns1T8dKpUKcXFxMDY2Rp06dXD9+vVC1Q4Av/zyC3x8fLB8+XL07t1ba11R6y7Me/lve/fuxdWrV/Hdd9/lWmdjY4Pw8PA3vj4+Ph6nT5+Gt7c3kpOTNT8Pcr4PwcHBiIiIAADs2bMHjRo10jq3OYrSvep1Ofu7d+8eVq9eDSMjIzRv3lyzXqlUav4/NTUVsbGxaNmyJURRxI0bNwq
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
2024-11-29 00:09:21 +04:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAIjCAYAAADlfxjoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3AUlEQVR4nO3dd3RU1doG8GcmHdJJDymEYELoBAggRSXSm/QSqoIiTVGvFKVeRQQbwlURpYUOwgU+KaErQkILvRNJT4CQTurs74+YuQxpMyHJmfL81pqlc9o8Z2cmeTmz9z4yIYQAEREREZGek0sdgIiIiIioJrDwJSIiIiKDwMKXiIiIiAwCC18iIiIiMggsfImIiIjIILDwJSIiIiKDwMKXiIiIiAwCC18iIiIiMggsfImIiIjIILDwJSIiIiKDwMKXtN7atWshk8lw7ty5Eut+/vlnyGQy9O/fH4WFhTWSp3fv3vD29tZ4vylTpkAmk1V9ICIiIlILC1/SWbt27cKkSZPQsWNHbNmyBUZGRlJHIiIiIi3Gwpd00vHjxzF8+HAEBARg7969MDc3lzoSERERaTkWvqRzIiMj0a9fP7i6uuLgwYOwsbEpsc327dsRGBgICwsLODg4ICQkBHFxccr1cXFxGD58ONzd3WFmZgYfHx/861//QkZGRoljbdiwAR4eHrC1tcXixYuVy7du3Qo3Nzc4ODhgyZIlJfY7ePAgXnrpJVhaWmLatGkQQgAoKtrr168Pa2trzJgxQ6WLxvHjxyGTyXD8+HGVY/Xq1QsymQzz589XLps/fz5kMhkePXqksu25c+cgk8mwdu1a5bK///67xDIAmDx5MmQyGcaOHauyPDU1Fe+99x48PDxgZmYGX19fLFmyBAqFosQxly1bVuLcGzdujFdeeUXlnMp7PHtepcnJycH8+fPx0ksvwdzcHK6urhgwYADu3btXqfMDgFdeeaXULMXHmDdvHkxMTPDw4cMS+06cOBG2trbIycnBH3/8geDgYDg4OMDCwgItWrTADz/8oPx5l/dazz6KrVmzBq+99hqcnJxgZmaGgIAA/PDDD+W2z/O8vb3LPbdnFb+Pnn8832bHjh1Dx44dYWdnp7LdlClTys1S/PPfunUrZs+eDRcXF9SuXRt9+/ZFTEyMyrZ//PEHBg8eDE9PT5iZmcHDwwPvv/8+nj59WiJzQEAALC0tYW1tjbZt22L37t0lXrv4fVFee2vy3hk7dmyp3Zxe5LM5duxYWFpalt54pRz/6dOn8Pf3h7+/v0q7pKSkwNXVFe3bty+321dx17G///5buezatWuws7ND7969UVBQoLJ9RZ8TQP2fGwDcvHkTQ4YMgaOjIywsLODn54c5c+aotFt5j2d/N4aHh6N79+6wsbFBrVq10LlzZ5w6dUrl9YqPWfy61tbWqFOnDqZPn46cnByVbQsKCrBo0SLUr18fZmZm8Pb2xuzZs5Gbm6uy3bOfL7lcDhcXFwwdOhTR0dFltjtpD2OpAxBp4t69e+jevTvMzMxw8OBBuLq6lthm7dq1GDduHFq3bo3FixcjKSkJ3333HU6dOoWLFy/C1tYW9+7dQ1JSEqZOnQo7Oztcu3YNy5cvx5EjR/Dnn3/CwsICAHDq1CmMGTMG7du3x/Dhw7Fhwwbcv38fT58+xcKFCzF79mwcOnQIM2fOhKenJ4YPHw4AuH//Pvr37w9fX198/vnnOHDggLKP8uTJkzF16lRcvHgR33zzDRwdHTFr1qwyz/nkyZP4/fffq7wt7969i59//rnE8uzsbHTu3BlxcXF4++234enpib/++guzZs1CQkICvv32W41ep2HDhtiwYYPy+apVq3Djxg188803ymVNmzYtc//CwkL07t0bR44cwbBhwzB9+nRkZGQgLCwMV69eRf369TU6v2f5+/sr/+g+evQI77//vnLdqFGjsHDhQmzdulWluMvLy8OOHTswcOBAmJub46+//oKTkxM++eQTGBkZ4cSJE3j33Xdx+fJlZcE6Z84cvPXWWyqvM3HiRHTs2LFEph9++AGNGjVC3759YWxsjL179+Ldd9+FQqHA5MmTyz2fZzVv3hwffPABACAqKgpz584td/tnf0bPtkPx/r169YKrqyvmzp0LR0dHZRup67PPPoNMJsPHH3+M5ORkfPvttwgODkZkZKTy87Z9+3ZkZ2dj0qRJqFOnDiIiIvD9998jNjYW27dvVx4rKysLb7zxBry9vfH06VOsXbsWAwcOxOnTp9GmTZsSr/1sW//222/YtWtXuVnVee9IxcLCAuvWrcPLL7+MOXPm4OuvvwZQ9HslLS0Na9eu1ajbV0xMDLp37w5/f39s27YNxsYly4LyPieA+j+3y5cvo2PHjjAxMcHEiRPh7e2Ne/fuYe/evfjss88wYMAA+Pr6Krd///330bBhQ0ycOFG5rGHDhgCAo0ePokePHggMDMS8efMgl8uV/2j8448/SrwPhgwZAm9vbyxevBhnzpzB8uXL8eTJE6xfv165zVtvvYV169Zh0KBB+OCDDxAeHo7Fixfjxo0bJd4zHTt2xMSJE6FQKHD16lV8++23iI+Pxx9//KF225NEBJGWW7NmjQAg9u3bJ+rXry8AiK5du5a6bV5ennBychKNGzcWT58+VS7ft2+fACDmzp1b5uuEhYUJAGLhwoXKZX379hX16tUTOTk5QgghMjIyRL169UStWrXE/fv3hRBCKBQK8fLLL4tmzZop95s2bZqwsrISjx49EkIIkZ+fL9q2bSsAiPDwcOV2w4cPF05OTsrjHzt2TAAQx44dU24TFBQkevToIQCIefPmKZfPmzdPABAPHz5UOY+zZ88KAGLNmjXKZVFRUSWWDRkyRDRu3Fh4eHiIMWPGKJcvWrRI1K5dW9y+fVvluDNnzhRGRkYiOjpa5ZhLly4t0ZaNGjUSnTt3LrFcCCHGjBkjvLy8Sl1Xml9//VUAEF9//XWJdQqFQuPzK/byyy+LV199Vfm8tGO0a9dOBAUFqez322+/lfgZPW/OnDkCgDh58mSJdaW9zrOys7NLLOvWrZvw8fEp8/We5+bmJnr37q18Xtp74tmsMplMZZmXl5dKm/30008CgDh9+rTKdgDE5MmTy81S/J52d3cX6enpyuXbtm0TAMR3332nXFbauS9evFjIZDLx4MGDMl8jOTlZABDLli1TWX7nzh0BQKxbt065rPhzU0yT9864ceOEp6dnidd/kc/mmDFjRO3atcs8t9KOL4QQs2bNEnK5XJw8eVJs375dABDffvttuccR4n+/T6OiokRKSooICAgQfn5+yt9Vz1Pnc6Luz61Tp07CysqqxM+y+HP8vOffh89u36BBA9GtWzeVfbOzs0W9evXE66+/rlxW/LPo27evyjHeffddAUBcunRJCCFEZGSkACDeeustle0+/PBDAUAcPXq03FwjRowQtWrVKvU8SLuwqwPpjLFjxyImJgYjRozAoUOHVK4kFDt37hySk5Px7rvvqvT77dWrF/z9/fF///d/ymX5+fl49OiR8tG8eXO0atVK5bhHjhxBz549YWZmBgCwtLREQEAAHB0dUa9ePQBQzipx6dIlPH78WLlfp06dUKdOHQCAsbExAgMDAUDlSsSAAQOQnJyMq1evlnrOv/32G86ePYsvvviiUm1WlvPnz2P79u1YvHgx5HLVXwPbt29XfqX9bPsEBwejsLAQJ0+eVNk+OztbZbtHjx5V6QwbO3fuhIODA6ZOnVpiXVmzZJR3fsXy8vKUP9eyjB49GuHh4couFQCwceNGeHh4oHPnzsplz7fBhAkTYGJiUup7tCLFVz8BIC0tDY8ePULnzp1x//59pKWlqXWMnJwctfu9q9MOxV2Ait/PlTF69GhYWVkpnw8aNAiurq4q32Y8e+5ZWVl49OgR2rdvDyEELl68qHK84s/vvXv38MUXX0Aul+Pll18ucW4AKjy/Z5X33nFyckJycrLyuBVJSUlReV+U9/Mr3ub5r9/LMn/+fDRq1AhjxozBu+++i86dO2PatGlq7QsUvUf69u2Lhw8f4sCBA2X+bNV5f6jzc3v48CF
2024-11-28 19:55:15 +04:00
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Метод локтя\n",
2024-11-29 00:09:21 +04:00
"from sklearn.cluster import KMeans\n",
"\n",
2024-11-28 19:55:15 +04:00
"inertia = []\n",
"for k in range(1, 11):\n",
" kmeans = KMeans(n_clusters=k, random_state=42)\n",
" kmeans.fit(df_scaled)\n",
" inertia.append(kmeans.inertia_)\n",
"\n",
2024-11-29 00:09:21 +04:00
"# Визуализация метода локтя\n",
2024-11-28 19:55:15 +04:00
"plt.figure(figsize=(8, 6))\n",
"plt.plot(range(1, 11), inertia, marker='o')\n",
"plt.title('Метод локтя для выбора количества кластеров')\n",
"plt.xlabel('Количество кластеров')\n",
"plt.ylabel('Инерция')\n",
"plt.show()\n",
"\n",
2024-11-29 00:09:21 +04:00
"# Коэффициент силуэта\n",
"from sklearn.metrics import silhouette_score\n",
"\n",
2024-11-28 19:55:15 +04:00
"silhouette_scores = []\n",
2024-11-29 00:09:21 +04:00
"for k in range(2, 11):\n",
2024-11-28 19:55:15 +04:00
" kmeans = KMeans(n_clusters=k, random_state=42)\n",
" kmeans.fit(df_scaled)\n",
" score = silhouette_score(df_scaled, kmeans.labels_)\n",
" silhouette_scores.append(score)\n",
"\n",
2024-11-29 00:09:21 +04:00
"# Визуализация коэффициента силуэта\n",
2024-11-28 19:55:15 +04:00
"plt.figure(figsize=(8, 6))\n",
"plt.plot(range(2, 11), silhouette_scores, marker='o')\n",
2024-11-29 00:09:21 +04:00
"plt.title('Коэффициент силуэта для различных кластеров')\n",
2024-11-28 19:55:15 +04:00
"plt.xlabel('Количество кластеров')\n",
"plt.ylabel('Коэффициент силуэта')\n",
2024-11-29 00:09:21 +04:00
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2024-11-29 00:09:56 +04:00
"# Кластеризация с
"После выбора оптимального числа кластеров (например, 3), мы применим K-means для кластеризации и визуализируем результаты. \n",
"Ключевой термин:\n",
"- K-means — алгоритм кластеризации, который группирует данные вокруг центров (центроидов) кластеров."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'KMeans' is not defined",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[1;32mIn[5], line 3\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[38;5;66;03m# Кластеризация с с
"\u001b[1;31mNameError\u001b[0m: name 'KMeans' is not defined"
]
}
],
"source": [
"# Кластеризация с
"optimal_clusters = 3\n",
"kmeans = KMeans(n_clusters=optimal_clusters, random_state=42)\n",
"df['Cluster'] = kmeans.fit_predict(df_scaled)\n",
"\n",
"# Визуализация кластеров с
"plt.figure(figsize=(8, 6))\n",
"plt.scatter(df_pca[:, 0], df_pca[:, 1], c=df['Cluster'], cmap='viridis', edgecolor='k', alpha=0.6)\n",
"plt.title(\"Кластеры, определенные K-means (PCA)\")\n",
"plt.xlabel(\"Главная компонента 1\")\n",
"plt.ylabel(\"Главная компонента 2\")\n",
"plt.colorbar(label='Кластер')\n",
"plt.show()"
2024-11-28 19:55:15 +04:00
]
2024-11-26 03:02:51 +04:00
}
],
"metadata": {
"kernelspec": {
2024-11-28 19:55:15 +04:00
"display_name": ".venv",
2024-11-26 03:02:51 +04:00
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
