{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Регрессия.\n", "\n", " - Прогнозирование вероятности IT-направления:\n", " Цель: Используя такие параметры, как уровень образования, тип учебного заведения, финансовое положение, возраст и уровень гибкости, можно предсказать занятие IT-направления.\n", "\n", "Классификация.\n", "\n", " - Распределение студентов по типам учебных заведений\n", " Цель: распределить студентов по различным типам учреждений (например, государственные/частные университеты), используя данные об их образовании, возрасте, месте проживания и финансовых возможностях.\n", "\n" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelInstitution TypeGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
0University1Male23TabNoTownMidWifi4GModerate
1University1Female23MobileNoTownMidMobile Data4GModerate
2College0Female18MobileNoTownMidWifi4GModerate
3School1Female11MobileNoTownMidMobile Data4GModerate
4School1Female18MobileNoTownPoorMobile Data3GLow
....................................
1200College1Female18MobileNoTownMidWifi4GLow
1201College1Female18MobileNoRuralMidWifi4GModerate
1202School1Male11MobileNoTownMidMobile Data3GModerate
1203College1Female18MobileNoRuralMidWifi4GLow
1204School1Female11MobileNoTownPoorMobile Data3GModerate
\n", "

1205 rows × 11 columns

\n", "
" ], "text/plain": [ " Education Level Institution Type Gender Age Device IT Student \\\n", "0 University 1 Male 23 Tab No \n", "1 University 1 Female 23 Mobile No \n", "2 College 0 Female 18 Mobile No \n", "3 School 1 Female 11 Mobile No \n", "4 School 1 Female 18 Mobile No \n", "... ... ... ... ... ... ... \n", "1200 College 1 Female 18 Mobile No \n", "1201 College 1 Female 18 Mobile No \n", "1202 School 1 Male 11 Mobile No \n", "1203 College 1 Female 18 Mobile No \n", "1204 School 1 Female 11 Mobile No \n", "\n", " Location Financial Condition Internet Type Network Type Flexibility Level \n", "0 Town Mid Wifi 4G Moderate \n", "1 Town Mid Mobile Data 4G Moderate \n", "2 Town Mid Wifi 4G Moderate \n", "3 Town Mid Mobile Data 4G Moderate \n", "4 Town Poor Mobile Data 3G Low \n", "... ... ... ... ... ... \n", "1200 Town Mid Wifi 4G Low \n", "1201 Rural Mid Wifi 4G Moderate \n", "1202 Town Mid Mobile Data 3G Moderate \n", "1203 Rural Mid Wifi 4G Low \n", "1204 Town Poor Mobile Data 3G Moderate \n", "\n", "[1205 rows x 11 columns]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "from sklearn import set_config\n", "\n", "set_config(transform_output=\"pandas\")\n", "\n", "random_state=9\n", "\n", "df = pd.read_csv(\"data/students_education.csv\")\n", "def Institution_Type(value):\n", " if value == \"Private\":\n", " return 1\n", " elif value == \"Public\":\n", " return 0\n", "\n", "df['Institution Type'] = df['Institution Type'].map(Institution_Type)\n", "\n", "df" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'X_train'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
2941091012021
87611111010012
38211111010011
63420231010111
90610111010111
.................................
104400181010122
109520230112120
113011111011121
86021231010021
112621230100011
\n", "

964 rows × 10 columns

\n", "
" ], "text/plain": [ " Education Level Gender Age Device IT Student Location \\\n", "294 1 0 9 1 0 1 \n", "876 1 1 11 1 0 1 \n", "382 1 1 11 1 0 1 \n", "634 2 0 23 1 0 1 \n", "906 1 0 11 1 0 1 \n", "... ... ... ... ... ... ... \n", "1044 0 0 18 1 0 1 \n", "1095 2 0 23 0 1 1 \n", "1130 1 1 11 1 0 1 \n", "860 2 1 23 1 0 1 \n", "1126 2 1 23 0 1 0 \n", "\n", " Financial Condition Internet Type Network Type Flexibility Level \n", "294 2 0 2 1 \n", "876 0 0 1 2 \n", "382 0 0 1 1 \n", "634 0 1 1 1 \n", "906 0 1 1 1 \n", "... ... ... ... ... \n", "1044 0 1 2 2 \n", "1095 2 1 2 0 \n", "1130 1 1 2 1 \n", "860 0 0 2 1 \n", "1126 0 0 1 1 \n", "\n", "[964 rows x 10 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'y_class_train'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "294 0\n", "876 1\n", "382 1\n", "634 0\n", "906 0\n", " ..\n", "1044 1\n", "1095 1\n", "1130 1\n", "860 1\n", "1126 1\n", "Name: Institution Type, Length: 964, dtype: int64" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'X_test'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
10110110010122
94601181010122
30601182110122
10920231010110
106121230100012
.................................
90811101012122
113520180110122
89410101011011
86611111010011
100620230012122
\n", "

241 rows × 10 columns

\n", "
" ], "text/plain": [ " Education Level Gender Age Device IT Student Location \\\n", "101 1 0 11 0 0 1 \n", "946 0 1 18 1 0 1 \n", "306 0 1 18 2 1 1 \n", "109 2 0 23 1 0 1 \n", "1061 2 1 23 0 1 0 \n", "... ... ... ... ... ... ... \n", "908 1 1 10 1 0 1 \n", "1135 2 0 18 0 1 1 \n", "894 1 0 10 1 0 1 \n", "866 1 1 11 1 0 1 \n", "1006 2 0 23 0 0 1 \n", "\n", " Financial Condition Internet Type Network Type Flexibility Level \n", "101 0 1 2 2 \n", "946 0 1 2 2 \n", "306 0 1 2 2 \n", "109 0 1 1 0 \n", "1061 0 0 1 2 \n", "... ... ... ... ... \n", "908 2 1 2 2 \n", "1135 0 1 2 2 \n", "894 1 0 1 1 \n", "866 0 0 1 1 \n", "1006 2 1 2 2 \n", "\n", "[241 rows x 10 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'y_class_test'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "101 1\n", "946 1\n", "306 0\n", "109 1\n", "1061 1\n", " ..\n", "908 1\n", "1135 1\n", "894 1\n", "866 1\n", "1006 1\n", "Name: Institution Type, Length: 241, dtype: int64" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn.utils import resample\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.preprocessing import LabelEncoder\n", "from sklearn import metrics\n", "from imblearn.over_sampling import RandomOverSampler\n", "from imblearn.under_sampling import RandomUnderSampler\n", "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n", "from sklearn.metrics import ConfusionMatrixDisplay\n", "from sklearn.compose import ColumnTransformer\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.impute import SimpleImputer\n", "from sklearn.linear_model import LinearRegression, LogisticRegression\n", "from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor, RandomForestClassifier, GradientBoostingClassifier\n", "from sklearn.model_selection import train_test_split, GridSearchCV\n", "from sklearn.linear_model import SGDClassifier, SGDRegressor\n", "from sklearn.metrics import (\n", " precision_score, recall_score, accuracy_score, roc_auc_score, f1_score,\n", " matthews_corrcoef, cohen_kappa_score, confusion_matrix\n", ")\n", "from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error\n", "import numpy as np\n", "import featuretools as ft\n", "from sklearn.metrics import accuracy_score, classification_report\n", "\n", "# Определение целевых переменных\n", "X = df.drop('Institution Type', axis=1)\n", "y_class = df['Institution Type'] # Задача классификации\n", "y_reg = df['IT Student'] # Задача регрессии\n", "\n", "# Преобразование категориальных переменных\n", "categorical_features = ['Education Level', 'Gender', 'Device', 'IT Student', 'Location', 'Financial Condition',\n", " 'Internet Type', 'Network Type', 'Flexibility Level']\n", "numerical_features = ['Age']\n", "\n", "# Создание ColumnTransformer с обработкой неизвестных категорий\n", "preprocessor = ColumnTransformer(\n", " transformers=[\n", " ('num', StandardScaler(), numerical_features),\n", " ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_features)]) # Используем handle_unknown='ignore'\n", "\n", "# Разделение данных на обучающую и тестовую выборки\n", "X_train, X_test, y_class_train, y_class_test, y_reg_train, y_reg_test = train_test_split(X, y_class, y_reg, test_size=0.2, random_state=42) \n", "\n", "def estimate_bias_variance(model, X, y):\n", " predictions = np.array([model.fit(X, y).predict(X) for _ in range(1000)])\n", " bias = np.mean((y - np.mean(predictions, axis=0)) ** 2)\n", " variance = np.mean(np.var(predictions, axis=0))\n", " return bias, variance\n", "\n", "display(\"X_train\", X_train)\n", "display(\"y_class_train\", y_class_train)\n", "\n", "display(\"X_test\", X_test)\n", "display(\"y_class_test\", y_class_test)" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('cat',\n",
       "                                                                         OneHotEncoder(sparse_output=False),\n",
       "                                                                         ['Education '\n",
       "                                                                          'Level',\n",
       "                                                                          'Gender',\n",
       "                                                                          'Device',\n",
       "                                                                          'IT '\n",
       "                                                                          'Student',\n",
       "                                                                          'Location',\n",
       "                                                                          'Financial '\n",
       "                                                                          'Condition',\n",
       "                                                                          'Internet '\n",
       "                                                                          'Type',\n",
       "                                                                          'Network '\n",
       "                                                                          'Type',\n",
       "                                                                          'Flexibility '\n",
       "                                                                          'Level']),\n",
       "                                                                        ('num',\n",
       "                                                                         'passthrough',\n",
       "                                                                         ['Age'])])),\n",
       "                                       ('classifier',\n",
       "                                        SGDClassifier(loss='log_loss',\n",
       "                                                      max_iter=2000,\n",
       "                                                      random_state=42))]),\n",
       "             param_grid={'classifier__alpha': [0.0001, 0.001, 0.01],\n",
       "                         'classifier__eta0': [0.01, 0.1],\n",
       "                         'classifier__learning_rate': ['constant', 'adaptive']},\n",
       "             scoring='accuracy')
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "GridSearchCV(cv=5,\n", " estimator=Pipeline(steps=[('preprocessor',\n", " ColumnTransformer(transformers=[('cat',\n", " OneHotEncoder(sparse_output=False),\n", " ['Education '\n", " 'Level',\n", " 'Gender',\n", " 'Device',\n", " 'IT '\n", " 'Student',\n", " 'Location',\n", " 'Financial '\n", " 'Condition',\n", " 'Internet '\n", " 'Type',\n", " 'Network '\n", " 'Type',\n", " 'Flexibility '\n", " 'Level']),\n", " ('num',\n", " 'passthrough',\n", " ['Age'])])),\n", " ('classifier',\n", " SGDClassifier(loss='log_loss',\n", " max_iter=2000,\n", " random_state=42))]),\n", " param_grid={'classifier__alpha': [0.0001, 0.001, 0.01],\n", " 'classifier__eta0': [0.01, 0.1],\n", " 'classifier__learning_rate': ['constant', 'adaptive']},\n", " scoring='accuracy')" ] }, "execution_count": 92, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Задача классификации\n", "categorical_preprocessor = OneHotEncoder(sparse_output=False)\n", "\n", "# Создаем общий preprocessor\n", "preprocessor = ColumnTransformer([\n", " (\"cat\", categorical_preprocessor, categorical_features),\n", " (\"num\", \"passthrough\", numerical_features), \n", "], remainder=\"drop\") \n", "\n", "# Построение пайплайнов\n", "class_pipeline_rf = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('classifier', RandomForestClassifier(random_state=42))])\n", "\n", "class_pipeline_sgd = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('classifier', SGDClassifier(loss='log_loss', penalty='l2', random_state=42, max_iter=2000))])\n", "\n", "# Настройки гиперпараметров\n", "param_grid_class_rf = {\n", " 'classifier__n_estimators': [100, 200],\n", " 'classifier__max_depth': [None, 10, 20]}\n", "\n", "param_grid_class_sgd = {\n", " 'classifier__alpha': [0.0001, 0.001, 0.01],\n", " 'classifier__learning_rate': ['constant', 'adaptive'],\n", " 'classifier__eta0': [0.01, 0.1]}\n", "\n", "# Поиск гиперпараметров\n", "grid_search_class_rf = GridSearchCV(class_pipeline_rf, param_grid_class_rf, cv=5, scoring='accuracy')\n", "grid_search_class_rf.fit(X_train, y_class_train)\n", "\n", "grid_search_class_sgd = GridSearchCV(class_pipeline_sgd, param_grid_class_sgd, cv=5, scoring='accuracy')\n", "grid_search_class_sgd.fit(X_train, y_class_train)" ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Classification Report for Random Forest:\n", " precision recall f1-score support\n", "\n", " 0 0.87 0.86 0.86 76\n", " 1 0.93 0.94 0.94 165\n", "\n", " accuracy 0.91 241\n", " macro avg 0.90 0.90 0.90 241\n", "weighted avg 0.91 0.91 0.91 241\n", "\n", "Confusion Matrix for Random Forest:\n", "[[ 65 11]\n", " [ 10 155]]\n", "Classification Report for SGD:\n", " precision recall f1-score support\n", "\n", " 0 0.72 0.58 0.64 76\n", " 1 0.82 0.90 0.86 165\n", "\n", " accuracy 0.80 241\n", " macro avg 0.77 0.74 0.75 241\n", "weighted avg 0.79 0.80 0.79 241\n", "\n", "Confusion Matrix for SGD:\n", "[[ 44 32]\n", " [ 17 148]]\n" ] } ], "source": [ "# Оценка моделей\n", "y_class_pred_rf = grid_search_class_rf.predict(X_test)\n", "y_class_pred_sgd = grid_search_class_sgd.predict(X_test)\n", "\n", "print(\"Classification Report for Random Forest:\")\n", "print(classification_report(y_class_test, y_class_pred_rf))\n", "print(\"Confusion Matrix for Random Forest:\")\n", "print(confusion_matrix(y_class_test, y_class_pred_rf))\n", "\n", "print(\"Classification Report for SGD:\")\n", "print(classification_report(y_class_test, y_class_pred_sgd))\n", "print(\"Confusion Matrix for SGD:\")\n", "print(confusion_matrix(y_class_test, y_class_pred_sgd))" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Classification Bias (Random Forest): 0.05186721991701245\n", "Classification Variance (Random Forest): 0.0\n", "Classification Bias (SGD): 0.19398340248962656\n", "Classification Variance (SGD): 0.0\n" ] } ], "source": [ "# Оценка смещения и дисперсии\n", "bias_class_rf, variance_class_rf = estimate_bias_variance(grid_search_class_rf.best_estimator_, X_train, y_class_train)\n", "bias_class_sgd, variance_class_sgd = estimate_bias_variance(grid_search_class_sgd.best_estimator_, X_train, y_class_train)\n", "\n", "print(\"Classification Bias (Random Forest):\", bias_class_rf)\n", "print(\"Classification Variance (Random Forest):\", variance_class_rf)\n", "print(\"Classification Bias (SGD):\", bias_class_sgd)\n", "print(\"Classification Variance (SGD):\", variance_class_sgd)" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\Альфия\\PycharmProjects\\Curse\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n" ] }, { "data": { "text/html": [ "
GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('cat',\n",
       "                                                                         OneHotEncoder(sparse_output=False),\n",
       "                                                                         ['Education '\n",
       "                                                                          'Level',\n",
       "                                                                          'Gender',\n",
       "                                                                          'Device',\n",
       "                                                                          'IT '\n",
       "                                                                          'Student',\n",
       "                                                                          'Location',\n",
       "                                                                          'Financial '\n",
       "                                                                          'Condition',\n",
       "                                                                          'Internet '\n",
       "                                                                          'Type',\n",
       "                                                                          'Network '\n",
       "                                                                          'Type',\n",
       "                                                                          'Flexibility '\n",
       "                                                                          'Level']),\n",
       "                                                                        ('num',\n",
       "                                                                         'passthrough',\n",
       "                                                                         ['Age'])])),\n",
       "                                       ('regressor',\n",
       "                                        SGDRegressor(max_iter=2000,\n",
       "                                                     random_state=42))]),\n",
       "             param_grid={'regressor__alpha': [0.0001, 0.001, 0.01],\n",
       "                         'regressor__eta0': [0.01, 0.1],\n",
       "                         'regressor__learning_rate': ['constant', 'adaptive']},\n",
       "             scoring='r2')
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "GridSearchCV(cv=5,\n", " estimator=Pipeline(steps=[('preprocessor',\n", " ColumnTransformer(transformers=[('cat',\n", " OneHotEncoder(sparse_output=False),\n", " ['Education '\n", " 'Level',\n", " 'Gender',\n", " 'Device',\n", " 'IT '\n", " 'Student',\n", " 'Location',\n", " 'Financial '\n", " 'Condition',\n", " 'Internet '\n", " 'Type',\n", " 'Network '\n", " 'Type',\n", " 'Flexibility '\n", " 'Level']),\n", " ('num',\n", " 'passthrough',\n", " ['Age'])])),\n", " ('regressor',\n", " SGDRegressor(max_iter=2000,\n", " random_state=42))]),\n", " param_grid={'regressor__alpha': [0.0001, 0.001, 0.01],\n", " 'regressor__eta0': [0.01, 0.1],\n", " 'regressor__learning_rate': ['constant', 'adaptive']},\n", " scoring='r2')" ] }, "execution_count": 96, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Задача регрессии\n", "reg_pipeline_rf = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('regressor', RandomForestRegressor(random_state=42))])\n", "\n", "reg_pipeline_sgd = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('regressor', SGDRegressor(loss='squared_error', penalty='l2', random_state=42, max_iter=2000))])\n", "\n", "# Настройка гиперпараметров для регрессии\n", "param_grid_reg_rf = {\n", " 'regressor__n_estimators': [100, 200],\n", " 'regressor__max_depth': [None, 10, 20]}\n", "\n", "param_grid_reg_sgd = {\n", " 'regressor__alpha': [0.0001, 0.001, 0.01],\n", " 'regressor__learning_rate': ['constant', 'adaptive'],\n", " 'regressor__eta0': [0.01, 0.1]}\n", "\n", "# Поиск гиперпараметров\n", "grid_search_reg_rf = GridSearchCV(reg_pipeline_rf, param_grid_reg_rf, cv=5, scoring='r2')\n", "grid_search_reg_rf.fit(X_train, y_reg_train)\n", "\n", "grid_search_reg_sgd = GridSearchCV(reg_pipeline_sgd, param_grid_reg_sgd, cv=5, scoring='r2')\n", "grid_search_reg_sgd.fit(X_train, y_reg_train)" ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Regression Metrics for Random Forest:\n", "Mean Squared Error: 0.0\n", "R2 Score: 1.0\n", "Regression Metrics for SGD:\n", "Mean Squared Error: 4011897878459.718\n", "R2 Score: -20174462396433.535\n" ] } ], "source": [ "# Оценка моделей\n", "y_reg_pred_rf = grid_search_reg_rf.predict(X_test)\n", "y_reg_pred_sgd = grid_search_reg_sgd.predict(X_test)\n", "\n", "print(\"Regression Metrics for Random Forest:\")\n", "print(\"Mean Squared Error:\", mean_squared_error(y_reg_test, y_reg_pred_rf))\n", "print(\"R2 Score:\", r2_score(y_reg_test, y_reg_pred_rf))\n", "\n", "print(\"Regression Metrics for SGD:\")\n", "print(\"Mean Squared Error:\", mean_squared_error(y_reg_test, y_reg_pred_sgd))\n", "print(\"R2 Score:\", r2_score(y_reg_test, y_reg_pred_sgd))" ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Regression Bias (Random Forest): 0.0\n", "Regression Variance (Random Forest): 0.0\n", "Regression Bias (SGD): 4382665100501.0005\n", "Regression Variance (SGD): 1.1089741676829076e-15\n" ] } ], "source": [ "# Оценка смещения и дисперсии\n", "bias_reg_rf, variance_reg_rf = estimate_bias_variance(grid_search_reg_rf.best_estimator_, X_train, y_reg_train)\n", "bias_reg_sgd, variance_reg_sgd = estimate_bias_variance(grid_search_reg_sgd.best_estimator_, X_train, y_reg_train)\n", "\n", "print(\"Regression Bias (Random Forest):\", bias_reg_rf)\n", "print(\"Regression Variance (Random Forest):\", variance_reg_rf)\n", "print(\"Regression Bias (SGD):\", bias_reg_sgd)\n", "print(\"Regression Variance (SGD):\", variance_reg_sgd)" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.5" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }