{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Регрессия.\n", "\n", " - Прогнозирование вероятности IT-направления:\n", " Цель: Используя такие параметры, как уровень образования, тип учебного заведения, финансовое положение, возраст и уровень гибкости, можно предсказать занятие IT-направления.\n", "\n", "Классификация.\n", "\n", " - Распределение студентов по типам учебных заведений\n", " Цель: распределить студентов по различным типам учреждений (например, государственные/частные университеты), используя данные об их образовании, возрасте, месте проживания и финансовых возможностях.\n", "\n" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelInstitution TypeGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
0University1Male23Tab0TownMidWifi4GModerate
1University1Female23Mobile0TownMidMobile Data4GModerate
2College0Female18Mobile0TownMidWifi4GModerate
3School1Female11Mobile0TownMidMobile Data4GModerate
4School1Female18Mobile0TownPoorMobile Data3GLow
....................................
1200College1Female18Mobile0TownMidWifi4GLow
1201College1Female18Mobile0RuralMidWifi4GModerate
1202School1Male11Mobile0TownMidMobile Data3GModerate
1203College1Female18Mobile0RuralMidWifi4GLow
1204School1Female11Mobile0TownPoorMobile Data3GModerate
\n", "

1205 rows × 11 columns

\n", "
" ], "text/plain": [ " Education Level Institution Type Gender Age Device IT Student \\\n", "0 University 1 Male 23 Tab 0 \n", "1 University 1 Female 23 Mobile 0 \n", "2 College 0 Female 18 Mobile 0 \n", "3 School 1 Female 11 Mobile 0 \n", "4 School 1 Female 18 Mobile 0 \n", "... ... ... ... ... ... ... \n", "1200 College 1 Female 18 Mobile 0 \n", "1201 College 1 Female 18 Mobile 0 \n", "1202 School 1 Male 11 Mobile 0 \n", "1203 College 1 Female 18 Mobile 0 \n", "1204 School 1 Female 11 Mobile 0 \n", "\n", " Location Financial Condition Internet Type Network Type Flexibility Level \n", "0 Town Mid Wifi 4G Moderate \n", "1 Town Mid Mobile Data 4G Moderate \n", "2 Town Mid Wifi 4G Moderate \n", "3 Town Mid Mobile Data 4G Moderate \n", "4 Town Poor Mobile Data 3G Low \n", "... ... ... ... ... ... \n", "1200 Town Mid Wifi 4G Low \n", "1201 Rural Mid Wifi 4G Moderate \n", "1202 Town Mid Mobile Data 3G Moderate \n", "1203 Rural Mid Wifi 4G Low \n", "1204 Town Poor Mobile Data 3G Moderate \n", "\n", "[1205 rows x 11 columns]" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "from sklearn import set_config\n", "\n", "set_config(transform_output=\"pandas\")\n", "\n", "random_state=9\n", "\n", "df = pd.read_csv(\"students_education.csv\")\n", "def Institution_Type(value):\n", " if value == \"Private\":\n", " return 1\n", " elif value == \"Public\":\n", " return 0\n", "\n", "df['Institution Type'] = df['Institution Type'].map(Institution_Type)\n", "\n", "df" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelInstitution TypeGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
0UniversityPrivateMale23TabNoTownMidWifi4GModerate
1UniversityPrivateFemale23MobileNoTownMidMobile Data4GModerate
2CollegePublicFemale18MobileNoTownMidWifi4GModerate
3SchoolPrivateFemale11MobileNoTownMidMobile Data4GModerate
4SchoolPrivateFemale18MobileNoTownPoorMobile Data3GLow
....................................
1200CollegePrivateFemale18MobileNoTownMidWifi4GLow
1201CollegePrivateFemale18MobileNoRuralMidWifi4GModerate
1202SchoolPrivateMale11MobileNoTownMidMobile Data3GModerate
1203CollegePrivateFemale18MobileNoRuralMidWifi4GLow
1204SchoolPrivateFemale11MobileNoTownPoorMobile Data3GModerate
\n", "

1205 rows × 11 columns

\n", "
" ], "text/plain": [ " Education Level Institution Type Gender Age Device IT Student \\\n", "0 University Private Male 23 Tab No \n", "1 University Private Female 23 Mobile No \n", "2 College Public Female 18 Mobile No \n", "3 School Private Female 11 Mobile No \n", "4 School Private Female 18 Mobile No \n", "... ... ... ... ... ... ... \n", "1200 College Private Female 18 Mobile No \n", "1201 College Private Female 18 Mobile No \n", "1202 School Private Male 11 Mobile No \n", "1203 College Private Female 18 Mobile No \n", "1204 School Private Female 11 Mobile No \n", "\n", " Location Financial Condition Internet Type Network Type Flexibility Level \n", "0 Town Mid Wifi 4G Moderate \n", "1 Town Mid Mobile Data 4G Moderate \n", "2 Town Mid Wifi 4G Moderate \n", "3 Town Mid Mobile Data 4G Moderate \n", "4 Town Poor Mobile Data 3G Low \n", "... ... ... ... ... ... \n", "1200 Town Mid Wifi 4G Low \n", "1201 Rural Mid Wifi 4G Moderate \n", "1202 Town Mid Mobile Data 3G Moderate \n", "1203 Rural Mid Wifi 4G Low \n", "1204 Town Poor Mobile Data 3G Moderate \n", "\n", "[1205 rows x 11 columns]" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.read_csv(\"students_education.csv\")" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'X_train'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
294SchoolFemale9Mobile0TownRichMobile Data4GLow
876SchoolMale11Mobile0TownMidMobile Data3GModerate
382SchoolMale11Mobile0TownMidMobile Data3GLow
634UniversityFemale23Mobile0TownMidWifi3GLow
906SchoolFemale11Mobile0TownMidWifi3GLow
.................................
1044CollegeFemale18Mobile0TownMidWifi4GModerate
1095UniversityFemale23Computer1TownRichWifi4GHigh
1130SchoolMale11Mobile0TownPoorWifi4GLow
860UniversityMale23Mobile0TownMidMobile Data4GLow
1126UniversityMale23Computer1RuralMidMobile Data3GLow
\n", "

964 rows × 10 columns

\n", "
" ], "text/plain": [ " Education Level Gender Age Device IT Student Location \\\n", "294 School Female 9 Mobile 0 Town \n", "876 School Male 11 Mobile 0 Town \n", "382 School Male 11 Mobile 0 Town \n", "634 University Female 23 Mobile 0 Town \n", "906 School Female 11 Mobile 0 Town \n", "... ... ... ... ... ... ... \n", "1044 College Female 18 Mobile 0 Town \n", "1095 University Female 23 Computer 1 Town \n", "1130 School Male 11 Mobile 0 Town \n", "860 University Male 23 Mobile 0 Town \n", "1126 University Male 23 Computer 1 Rural \n", "\n", " Financial Condition Internet Type Network Type Flexibility Level \n", "294 Rich Mobile Data 4G Low \n", "876 Mid Mobile Data 3G Moderate \n", "382 Mid Mobile Data 3G Low \n", "634 Mid Wifi 3G Low \n", "906 Mid Wifi 3G Low \n", "... ... ... ... ... \n", "1044 Mid Wifi 4G Moderate \n", "1095 Rich Wifi 4G High \n", "1130 Poor Wifi 4G Low \n", "860 Mid Mobile Data 4G Low \n", "1126 Mid Mobile Data 3G Low \n", "\n", "[964 rows x 10 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'y_class_train'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "294 0\n", "876 1\n", "382 1\n", "634 0\n", "906 0\n", " ..\n", "1044 1\n", "1095 1\n", "1130 1\n", "860 1\n", "1126 1\n", "Name: Institution Type, Length: 964, dtype: int64" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'X_test'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Education LevelGenderAgeDeviceIT StudentLocationFinancial ConditionInternet TypeNetwork TypeFlexibility Level
101SchoolFemale11Computer0TownMidWifi4GModerate
946CollegeMale18Mobile0TownMidWifi4GModerate
306CollegeMale18Tab1TownMidWifi4GModerate
109UniversityFemale23Mobile0TownMidWifi3GHigh
1061UniversityMale23Computer1RuralMidMobile Data3GModerate
.................................
908SchoolMale10Mobile0TownRichWifi4GModerate
1135UniversityFemale18Computer1TownMidWifi4GModerate
894SchoolFemale10Mobile0TownPoorMobile Data3GLow
866SchoolMale11Mobile0TownMidMobile Data3GLow
1006UniversityFemale23Computer0TownRichWifi4GModerate
\n", "

241 rows × 10 columns

\n", "
" ], "text/plain": [ " Education Level Gender Age Device IT Student Location \\\n", "101 School Female 11 Computer 0 Town \n", "946 College Male 18 Mobile 0 Town \n", "306 College Male 18 Tab 1 Town \n", "109 University Female 23 Mobile 0 Town \n", "1061 University Male 23 Computer 1 Rural \n", "... ... ... ... ... ... ... \n", "908 School Male 10 Mobile 0 Town \n", "1135 University Female 18 Computer 1 Town \n", "894 School Female 10 Mobile 0 Town \n", "866 School Male 11 Mobile 0 Town \n", "1006 University Female 23 Computer 0 Town \n", "\n", " Financial Condition Internet Type Network Type Flexibility Level \n", "101 Mid Wifi 4G Moderate \n", "946 Mid Wifi 4G Moderate \n", "306 Mid Wifi 4G Moderate \n", "109 Mid Wifi 3G High \n", "1061 Mid Mobile Data 3G Moderate \n", "... ... ... ... ... \n", "908 Rich Wifi 4G Moderate \n", "1135 Mid Wifi 4G Moderate \n", "894 Poor Mobile Data 3G Low \n", "866 Mid Mobile Data 3G Low \n", "1006 Rich Wifi 4G Moderate \n", "\n", "[241 rows x 10 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "'y_class_test'" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "101 1\n", "946 1\n", "306 0\n", "109 1\n", "1061 1\n", " ..\n", "908 1\n", "1135 1\n", "894 1\n", "866 1\n", "1006 1\n", "Name: Institution Type, Length: 241, dtype: int64" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn.utils import resample\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.preprocessing import LabelEncoder\n", "from sklearn import metrics\n", "from imblearn.over_sampling import RandomOverSampler\n", "from imblearn.under_sampling import RandomUnderSampler\n", "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n", "from sklearn.metrics import ConfusionMatrixDisplay\n", "from sklearn.compose import ColumnTransformer\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.impute import SimpleImputer\n", "from sklearn.linear_model import LinearRegression, LogisticRegression\n", "from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor, RandomForestClassifier, GradientBoostingClassifier\n", "from sklearn.model_selection import train_test_split, GridSearchCV\n", "from sklearn.linear_model import SGDClassifier, SGDRegressor\n", "from sklearn.metrics import (\n", " precision_score, recall_score, accuracy_score, roc_auc_score, f1_score,\n", " matthews_corrcoef, cohen_kappa_score, confusion_matrix\n", ")\n", "from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error\n", "import numpy as np\n", "import featuretools as ft\n", "from sklearn.metrics import accuracy_score, classification_report\n", "\n", "# Определение целевых переменных\n", "X = df.drop('Institution Type', axis=1)\n", "y_class = df['Institution Type'] # Задача классификации\n", "y_reg = df['IT Student'] # Задача регрессии\n", "\n", "# Преобразование категориальных переменных\n", "categorical_features = ['Education Level', 'Gender', 'Device', 'IT Student', 'Location', 'Financial Condition',\n", " 'Internet Type', 'Network Type', 'Flexibility Level']\n", "numerical_features = ['Age']\n", "\n", "# Создание ColumnTransformer с обработкой неизвестных категорий\n", "preprocessor = ColumnTransformer(\n", " transformers=[\n", " ('num', StandardScaler(), numerical_features),\n", " ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_features)]) # Используем handle_unknown='ignore'\n", "\n", "# Разделение данных на обучающую и тестовую выборки\n", "X_train, X_test, y_class_train, y_class_test, y_reg_train, y_reg_test = train_test_split(X, y_class, y_reg, test_size=0.2, random_state=42) \n", "\n", "def estimate_bias_variance(model, X, y):\n", " predictions = np.array([model.fit(X, y).predict(X) for _ in range(1000)])\n", " bias = np.mean((y - np.mean(predictions, axis=0)) ** 2)\n", " variance = np.mean(np.var(predictions, axis=0))\n", " return bias, variance\n", "\n", "display(\"X_train\", X_train)\n", "display(\"y_class_train\", y_class_train)\n", "\n", "display(\"X_test\", X_test)\n", "display(\"y_class_test\", y_class_test)" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('cat',\n",
       "                                                                         OneHotEncoder(sparse_output=False),\n",
       "                                                                         ['Education '\n",
       "                                                                          'Level',\n",
       "                                                                          'Gender',\n",
       "                                                                          'Device',\n",
       "                                                                          'IT '\n",
       "                                                                          'Student',\n",
       "                                                                          'Location',\n",
       "                                                                          'Financial '\n",
       "                                                                          'Condition',\n",
       "                                                                          'Internet '\n",
       "                                                                          'Type',\n",
       "                                                                          'Network '\n",
       "                                                                          'Type',\n",
       "                                                                          'Flexibility '\n",
       "                                                                          'Level']),\n",
       "                                                                        ('num',\n",
       "                                                                         'passthrough',\n",
       "                                                                         ['Age'])])),\n",
       "                                       ('classifier',\n",
       "                                        SGDClassifier(loss='log_loss',\n",
       "                                                      max_iter=2000,\n",
       "                                                      random_state=42))]),\n",
       "             param_grid={'classifier__alpha': [0.0001, 0.001, 0.01],\n",
       "                         'classifier__eta0': [0.01, 0.1],\n",
       "                         'classifier__learning_rate': ['constant', 'adaptive']},\n",
       "             scoring='accuracy')
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "GridSearchCV(cv=5,\n", " estimator=Pipeline(steps=[('preprocessor',\n", " ColumnTransformer(transformers=[('cat',\n", " OneHotEncoder(sparse_output=False),\n", " ['Education '\n", " 'Level',\n", " 'Gender',\n", " 'Device',\n", " 'IT '\n", " 'Student',\n", " 'Location',\n", " 'Financial '\n", " 'Condition',\n", " 'Internet '\n", " 'Type',\n", " 'Network '\n", " 'Type',\n", " 'Flexibility '\n", " 'Level']),\n", " ('num',\n", " 'passthrough',\n", " ['Age'])])),\n", " ('classifier',\n", " SGDClassifier(loss='log_loss',\n", " max_iter=2000,\n", " random_state=42))]),\n", " param_grid={'classifier__alpha': [0.0001, 0.001, 0.01],\n", " 'classifier__eta0': [0.01, 0.1],\n", " 'classifier__learning_rate': ['constant', 'adaptive']},\n", " scoring='accuracy')" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Задача классификации\n", "categorical_preprocessor = OneHotEncoder(sparse_output=False)\n", "\n", "# Создаем общий preprocessor\n", "preprocessor = ColumnTransformer([\n", " (\"cat\", categorical_preprocessor, categorical_features),\n", " (\"num\", \"passthrough\", numerical_features), \n", "], remainder=\"drop\") \n", "\n", "# Построение пайплайнов\n", "class_pipeline_rf = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('classifier', RandomForestClassifier(random_state=42))])\n", "\n", "class_pipeline_sgd = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('classifier', SGDClassifier(loss='log_loss', penalty='l2', random_state=42, max_iter=2000))])\n", "\n", "# Настройки гиперпараметров\n", "param_grid_class_rf = {\n", " 'classifier__n_estimators': [100, 200],\n", " 'classifier__max_depth': [None, 10, 20]}\n", "\n", "param_grid_class_sgd = {\n", " 'classifier__alpha': [0.0001, 0.001, 0.01],\n", " 'classifier__learning_rate': ['constant', 'adaptive'],\n", " 'classifier__eta0': [0.01, 0.1]}\n", "\n", "# Поиск гиперпараметров\n", "grid_search_class_rf = GridSearchCV(class_pipeline_rf, param_grid_class_rf, cv=5, scoring='accuracy')\n", "grid_search_class_rf.fit(X_train, y_class_train)\n", "\n", "grid_search_class_sgd = GridSearchCV(class_pipeline_sgd, param_grid_class_sgd, cv=5, scoring='accuracy')\n", "grid_search_class_sgd.fit(X_train, y_class_train)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Classification Report for Random Forest:\n", " precision recall f1-score support\n", "\n", " 0 0.87 0.86 0.86 76\n", " 1 0.93 0.94 0.94 165\n", "\n", " accuracy 0.91 241\n", " macro avg 0.90 0.90 0.90 241\n", "weighted avg 0.91 0.91 0.91 241\n", "\n", "Confusion Matrix for Random Forest:\n", "[[ 65 11]\n", " [ 10 155]]\n", "Classification Report for SGD:\n", " precision recall f1-score support\n", "\n", " 0 0.72 0.58 0.64 76\n", " 1 0.82 0.90 0.86 165\n", "\n", " accuracy 0.80 241\n", " macro avg 0.77 0.74 0.75 241\n", "weighted avg 0.79 0.80 0.79 241\n", "\n", "Confusion Matrix for SGD:\n", "[[ 44 32]\n", " [ 17 148]]\n" ] } ], "source": [ "# Оценка моделей\n", "y_class_pred_rf = grid_search_class_rf.predict(X_test)\n", "y_class_pred_sgd = grid_search_class_sgd.predict(X_test)\n", "\n", "print(\"Classification Report for Random Forest:\")\n", "print(classification_report(y_class_test, y_class_pred_rf))\n", "print(\"Confusion Matrix for Random Forest:\")\n", "print(confusion_matrix(y_class_test, y_class_pred_rf))\n", "\n", "print(\"Classification Report for SGD:\")\n", "print(classification_report(y_class_test, y_class_pred_sgd))\n", "print(\"Confusion Matrix for SGD:\")\n", "print(confusion_matrix(y_class_test, y_class_pred_sgd))" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Classification Bias (Random Forest): 0.05186721991701245\n", "Classification Variance (Random Forest): 0.0\n", "Classification Bias (SGD): 0.19398340248962656\n", "Classification Variance (SGD): 0.0\n" ] } ], "source": [ "# Оценка смещения и дисперсии\n", "bias_class_rf, variance_class_rf = estimate_bias_variance(grid_search_class_rf.best_estimator_, X_train, y_class_train)\n", "bias_class_sgd, variance_class_sgd = estimate_bias_variance(grid_search_class_sgd.best_estimator_, X_train, y_class_train)\n", "\n", "print(\"Classification Bias (Random Forest):\", bias_class_rf)\n", "print(\"Classification Variance (Random Forest):\", variance_class_rf)\n", "print(\"Classification Bias (SGD):\", bias_class_sgd)\n", "print(\"Classification Variance (SGD):\", variance_class_sgd)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n", "c:\\Users\\novop\\Downloads\\ckmai-main\\ckmai\\.venv\\Lib\\site-packages\\sklearn\\linear_model\\_stochastic_gradient.py:1616: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.\n", " warnings.warn(\n" ] }, { "data": { "text/html": [ "
GridSearchCV(cv=5,\n",
       "             estimator=Pipeline(steps=[('preprocessor',\n",
       "                                        ColumnTransformer(transformers=[('cat',\n",
       "                                                                         OneHotEncoder(sparse_output=False),\n",
       "                                                                         ['Education '\n",
       "                                                                          'Level',\n",
       "                                                                          'Gender',\n",
       "                                                                          'Device',\n",
       "                                                                          'IT '\n",
       "                                                                          'Student',\n",
       "                                                                          'Location',\n",
       "                                                                          'Financial '\n",
       "                                                                          'Condition',\n",
       "                                                                          'Internet '\n",
       "                                                                          'Type',\n",
       "                                                                          'Network '\n",
       "                                                                          'Type',\n",
       "                                                                          'Flexibility '\n",
       "                                                                          'Level']),\n",
       "                                                                        ('num',\n",
       "                                                                         'passthrough',\n",
       "                                                                         ['Age'])])),\n",
       "                                       ('regressor',\n",
       "                                        SGDRegressor(max_iter=2000,\n",
       "                                                     random_state=42))]),\n",
       "             param_grid={'regressor__alpha': [0.0001, 0.001, 0.01],\n",
       "                         'regressor__eta0': [0.01, 0.1],\n",
       "                         'regressor__learning_rate': ['constant', 'adaptive']},\n",
       "             scoring='r2')
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" ], "text/plain": [ "GridSearchCV(cv=5,\n", " estimator=Pipeline(steps=[('preprocessor',\n", " ColumnTransformer(transformers=[('cat',\n", " OneHotEncoder(sparse_output=False),\n", " ['Education '\n", " 'Level',\n", " 'Gender',\n", " 'Device',\n", " 'IT '\n", " 'Student',\n", " 'Location',\n", " 'Financial '\n", " 'Condition',\n", " 'Internet '\n", " 'Type',\n", " 'Network '\n", " 'Type',\n", " 'Flexibility '\n", " 'Level']),\n", " ('num',\n", " 'passthrough',\n", " ['Age'])])),\n", " ('regressor',\n", " SGDRegressor(max_iter=2000,\n", " random_state=42))]),\n", " param_grid={'regressor__alpha': [0.0001, 0.001, 0.01],\n", " 'regressor__eta0': [0.01, 0.1],\n", " 'regressor__learning_rate': ['constant', 'adaptive']},\n", " scoring='r2')" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Задача регрессии\n", "reg_pipeline_rf = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('regressor', RandomForestRegressor(random_state=42))])\n", "\n", "reg_pipeline_sgd = Pipeline(steps=[\n", " ('preprocessor', preprocessor),\n", " ('regressor', SGDRegressor(loss='squared_error', penalty='l2', random_state=42, max_iter=2000))])\n", "\n", "# Настройка гиперпараметров для регрессии\n", "param_grid_reg_rf = {\n", " 'regressor__n_estimators': [100, 200],\n", " 'regressor__max_depth': [None, 10, 20]}\n", "\n", "param_grid_reg_sgd = {\n", " 'regressor__alpha': [0.0001, 0.001, 0.01],\n", " 'regressor__learning_rate': ['constant', 'adaptive'],\n", " 'regressor__eta0': [0.01, 0.1]}\n", "\n", "# Поиск гиперпараметров\n", "grid_search_reg_rf = GridSearchCV(reg_pipeline_rf, param_grid_reg_rf, cv=5, scoring='r2')\n", "grid_search_reg_rf.fit(X_train, y_reg_train)\n", "\n", "grid_search_reg_sgd = GridSearchCV(reg_pipeline_sgd, param_grid_reg_sgd, cv=5, scoring='r2')\n", "grid_search_reg_sgd.fit(X_train, y_reg_train)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Regression Metrics for Random Forest:\n", "Mean Squared Error: 0.0\n", "R2 Score: 1.0\n", "Regression Metrics for SGD:\n", "Mean Squared Error: 4011897878459.718\n", "R2 Score: -20174462396433.535\n" ] } ], "source": [ "# Оценка моделей\n", "y_reg_pred_rf = grid_search_reg_rf.predict(X_test)\n", "y_reg_pred_sgd = grid_search_reg_sgd.predict(X_test)\n", "\n", "print(\"Regression Metrics for Random Forest:\")\n", "print(\"Mean Squared Error:\", mean_squared_error(y_reg_test, y_reg_pred_rf))\n", "print(\"R2 Score:\", r2_score(y_reg_test, y_reg_pred_rf))\n", "\n", "print(\"Regression Metrics for SGD:\")\n", "print(\"Mean Squared Error:\", mean_squared_error(y_reg_test, y_reg_pred_sgd))\n", "print(\"R2 Score:\", r2_score(y_reg_test, y_reg_pred_sgd))" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Regression Bias (Random Forest): 0.0\n", "Regression Variance (Random Forest): 0.0\n", "Regression Bias (SGD): 4382665100501.0005\n", "Regression Variance (SGD): 1.1089741676829076e-15\n" ] } ], "source": [ "# Оценка смещения и дисперсии\n", "bias_reg_rf, variance_reg_rf = estimate_bias_variance(grid_search_reg_rf.best_estimator_, X_train, y_reg_train)\n", "bias_reg_sgd, variance_reg_sgd = estimate_bias_variance(grid_search_reg_sgd.best_estimator_, X_train, y_reg_train)\n", "\n", "print(\"Regression Bias (Random Forest):\", bias_reg_rf)\n", "print(\"Regression Variance (Random Forest):\", variance_reg_rf)\n", "print(\"Regression Bias (SGD):\", bias_reg_sgd)\n", "print(\"Regression Variance (SGD):\", variance_reg_sgd)" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.5" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }