968 lines
60 KiB
Plaintext
968 lines
60 KiB
Plaintext
|
{
|
|||
|
|
"cells": [
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Лабораторная работа №3\n",
|
|||
|
|
"\n",
|
|||
|
|
"## Набор данных Students Performance in Exams (Успеваемость студентов на экзаменах)\n",
|
|||
|
|
"\n",
|
|||
|
|
"Выгрузка данных из CSV файла в датафрейм"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 178,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"import pandas as pd\n",
|
|||
|
|
"# Загрузка данных\n",
|
|||
|
|
"df = pd.read_csv(\"..//..//static//csv//StudentsPerformance.csv\")\n",
|
|||
|
|
"\n",
|
|||
|
|
"random_state=9"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Описание набора \n",
|
|||
|
|
"\n",
|
|||
|
|
"Контекст\n",
|
|||
|
|
"Оценки, полученные студентами\n",
|
|||
|
|
"\n",
|
|||
|
|
"Содержание\n",
|
|||
|
|
"Этот набор данных состоит из оценок, полученных учащимися по различным предметам.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Вдохновение\n",
|
|||
|
|
"Понять влияние происхождения родителей, подготовки к тестированию и т. д. на успеваемость учащихся."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Анализ содержимого\n",
|
|||
|
|
"\n",
|
|||
|
|
"*Объекты наблюдения:* студенты, участвующие в экзаменах.\n",
|
|||
|
|
"\n",
|
|||
|
|
"*Атрибуты объектов:* \n",
|
|||
|
|
"\n",
|
|||
|
|
"1. gender — пол: определяет гендерную принадлежность студента (мужской, женский). \n",
|
|||
|
|
"2. race/ethnicity — этническая принадлежность: группа, к которой относится студент (например, различные расовые/этнические категории). \n",
|
|||
|
|
"3. parental level of education — уровень образования родителей(например, среднее образование, высшее образование и т.д.). \n",
|
|||
|
|
"4. lunch — тип обеда: информация о том, получает ли студент бесплатный или платный обед. \n",
|
|||
|
|
"5. test preparation course — курс подготовки к тесту\n",
|
|||
|
|
"6. math score — результаты экзаменов по математике.\n",
|
|||
|
|
"7. reading score — результаты экзаменов по чтению.\n",
|
|||
|
|
"8. writing score — результаты экзаменов по письму.\n",
|
|||
|
|
"\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Бизнес-цель:\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Цель**: Прогнозирование успеваемости студентов на основе различных факторов, таких как пол, раса/этническая принадлежность, уровень образования родителей, тип обеда и участие в подготовительных курсах. \n",
|
|||
|
|
"\n",
|
|||
|
|
"**Эффект**: Предсказание результатов студентов позволяет выявить тех, кто может столкнуться с трудностями в учебе. Это дает возможность образовательным учреждениям предпринимать превентивные меры: например, организовывать дополнительные занятия, персонализированные консультации, улучшать условия обучения и даже вмешиваться на более ранних стадиях, чтобы повысить общий уровень успеваемости.\n",
|
|||
|
|
"\n",
|
|||
|
|
"\n",
|
|||
|
|
"### Техническая цель\n",
|
|||
|
|
"**Цель**: Создание регрессионной модели, которая будет предсказывать общий балл студентов (или другой числовой показатель успеваемости) на основе категориальных и числовых данных. Это потребует использования методов линейной или нелинейной регрессии для определения зависимости между признаками (пол, уровень образования родителей и т.д.) и итоговым баллом. \n",
|
|||
|
|
"\n",
|
|||
|
|
"**Подход**: Для решения задачи нужно использовать числовые признаки, такие как \"math score\", \"reading score\", \"writing score\", и выполнить их агрегацию (например, суммирование или среднее), чтобы построить прогноз для общего балла. Модели, такие как линейная регрессия или регрессия на основе деревьев решений, подойдут для этого. у которых ожидаются низкие результаты на экзаменах, тем самым повышая их шансы на успешную сдачу экзаменов.\n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Конструирование признаков для решения задач\n",
|
|||
|
|
"\n",
|
|||
|
|
"Можно создать новый признак, который будет представлять общую успеваемость студента. Например, можно суммировать баллы по всем предметам и создать общий балл. \n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 179,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"df['total_score'] = df['math score'] + df['reading score'] + df['writing score']\n",
|
|||
|
|
"df = df.drop(columns=['math score', 'reading score', 'writing score'])\n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Выберем три модели для задач регрессии\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. Линейная регрессия служит базовой моделью, чтобы понять, насколько линейны зависимости между признаками и целевой переменной. Это важный шаг для проверки простых гипотез.\n",
|
|||
|
|
"\n",
|
|||
|
|
"2. Случайный лес используется для обработки данных с более сложными зависимостями, когда данные могут содержать нелинейные связи, которые линейная регрессия не может уловить.\n",
|
|||
|
|
"\n",
|
|||
|
|
"3. Градиентный бустинг — это более сложная модель, которая позволяет добиться высокой точности, особенно при наличии сложных закономерностей в данных. Он может предложить лучшее качество прогноза при оптимальной настройке гиперпараметров.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Эти три модели дадут нам хорошее сочетание простоты (линейная регрессия), гибкости (случайный лес) и мощности (градиентный бустинг), что позволит тщательно исследовать зависимости и добиться хорошего качества предсказаний."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Разделение набора данных на обучающую и тестовые выборки (80/20) для задачи регрессии и создание ориентира\n",
|
|||
|
|
"\n",
|
|||
|
|
"Целевой признак -- total_score"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 180,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.model_selection import train_test_split\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Разбиение на признаки и целевую переменную\n",
|
|||
|
|
"X = df.drop(columns=[\"total_score\"]) # Признаки\n",
|
|||
|
|
"y = df[\"total_score\"] # Целевая переменная\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Разбиение на обучающую и тестовую выборки\n",
|
|||
|
|
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=random_state)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Формирование конвейера"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 181,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.compose import ColumnTransformer\n",
|
|||
|
|
"from sklearn.discriminant_analysis import StandardScaler\n",
|
|||
|
|
"from sklearn.impute import SimpleImputer\n",
|
|||
|
|
"from sklearn.pipeline import Pipeline\n",
|
|||
|
|
"from sklearn.preprocessing import OneHotEncoder\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Пайплайн для обработки признаков\n",
|
|||
|
|
"columns_to_drop = [\"total_score\"]\n",
|
|||
|
|
"num_columns = [column for column in X.columns if X[column].dtype != \"object\"]\n",
|
|||
|
|
"cat_columns = [column for column in X.columns if X[column].dtype == \"object\"]\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Обработчики для числовых и категориальных признаков\n",
|
|||
|
|
"num_imputer = SimpleImputer(strategy=\"median\")\n",
|
|||
|
|
"num_scaler = StandardScaler()\n",
|
|||
|
|
"preprocessing_num = Pipeline([(\"imputer\", num_imputer), (\"scaler\", num_scaler)])\n",
|
|||
|
|
"\n",
|
|||
|
|
"cat_imputer = SimpleImputer(strategy=\"constant\", fill_value=\"unknown\")\n",
|
|||
|
|
"cat_encoder = OneHotEncoder(handle_unknown=\"ignore\", sparse_output=False, drop=\"first\")\n",
|
|||
|
|
"preprocessing_cat = Pipeline([(\"imputer\", cat_imputer), (\"encoder\", cat_encoder)])\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Обрабатываем признаки\n",
|
|||
|
|
"features_preprocessing = ColumnTransformer(\n",
|
|||
|
|
" transformers=[\n",
|
|||
|
|
" (\"prepocessing_num\", preprocessing_num, num_columns),\n",
|
|||
|
|
" (\"prepocessing_cat\", preprocessing_cat, cat_columns),\n",
|
|||
|
|
" ],\n",
|
|||
|
|
" remainder=\"passthrough\"\n",
|
|||
|
|
")"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Настройка гиперпараметров для каждой модели и обучение"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 182,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"text/html": [
|
|||
|
|
"<style>#sk-container-id-8 {\n",
|
|||
|
|
" /* Definition of color scheme common for light and dark mode */\n",
|
|||
|
|
" --sklearn-color-text: black;\n",
|
|||
|
|
" --sklearn-color-line: gray;\n",
|
|||
|
|
" /* Definition of color scheme for unfitted estimators */\n",
|
|||
|
|
" --sklearn-color-unfitted-level-0: #fff5e6;\n",
|
|||
|
|
" --sklearn-color-unfitted-level-1: #f6e4d2;\n",
|
|||
|
|
" --sklearn-color-unfitted-level-2: #ffe0b3;\n",
|
|||
|
|
" --sklearn-color-unfitted-level-3: chocolate;\n",
|
|||
|
|
" /* Definition of color scheme for fitted estimators */\n",
|
|||
|
|
" --sklearn-color-fitted-level-0: #f0f8ff;\n",
|
|||
|
|
" --sklearn-color-fitted-level-1: #d4ebff;\n",
|
|||
|
|
" --sklearn-color-fitted-level-2: #b3dbfd;\n",
|
|||
|
|
" --sklearn-color-fitted-level-3: cornflowerblue;\n",
|
|||
|
|
"\n",
|
|||
|
|
" /* Specific color for light theme */\n",
|
|||
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|||
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
|
|||
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|||
|
|
" --sklearn-color-icon: #696969;\n",
|
|||
|
|
"\n",
|
|||
|
|
" @media (prefers-color-scheme: dark) {\n",
|
|||
|
|
" /* Redefinition of color scheme for dark theme */\n",
|
|||
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|||
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
|
|||
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|||
|
|
" --sklearn-color-icon: #878787;\n",
|
|||
|
|
" }\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 {\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 pre {\n",
|
|||
|
|
" padding: 0;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 input.sk-hidden--visually {\n",
|
|||
|
|
" border: 0;\n",
|
|||
|
|
" clip: rect(1px 1px 1px 1px);\n",
|
|||
|
|
" clip: rect(1px, 1px, 1px, 1px);\n",
|
|||
|
|
" height: 1px;\n",
|
|||
|
|
" margin: -1px;\n",
|
|||
|
|
" overflow: hidden;\n",
|
|||
|
|
" padding: 0;\n",
|
|||
|
|
" position: absolute;\n",
|
|||
|
|
" width: 1px;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-dashed-wrapped {\n",
|
|||
|
|
" border: 1px dashed var(--sklearn-color-line);\n",
|
|||
|
|
" margin: 0 0.4em 0.5em 0.4em;\n",
|
|||
|
|
" box-sizing: border-box;\n",
|
|||
|
|
" padding-bottom: 0.4em;\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-container {\n",
|
|||
|
|
" /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
|
|||
|
|
" but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
|
|||
|
|
" so we also need the `!important` here to be able to override the\n",
|
|||
|
|
" default hidden behavior on the sphinx rendered scikit-learn.org.\n",
|
|||
|
|
" See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
|
|||
|
|
" display: inline-block !important;\n",
|
|||
|
|
" position: relative;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-text-repr-fallback {\n",
|
|||
|
|
" display: none;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"div.sk-parallel-item,\n",
|
|||
|
|
"div.sk-serial,\n",
|
|||
|
|
"div.sk-item {\n",
|
|||
|
|
" /* draw centered vertical line to link estimators */\n",
|
|||
|
|
" background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
|
|||
|
|
" background-size: 2px 100%;\n",
|
|||
|
|
" background-repeat: no-repeat;\n",
|
|||
|
|
" background-position: center center;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Parallel-specific style estimator block */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel-item::after {\n",
|
|||
|
|
" content: \"\";\n",
|
|||
|
|
" width: 100%;\n",
|
|||
|
|
" border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
|
|||
|
|
" flex-grow: 1;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel {\n",
|
|||
|
|
" display: flex;\n",
|
|||
|
|
" align-items: stretch;\n",
|
|||
|
|
" justify-content: center;\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
" position: relative;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel-item {\n",
|
|||
|
|
" display: flex;\n",
|
|||
|
|
" flex-direction: column;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel-item:first-child::after {\n",
|
|||
|
|
" align-self: flex-end;\n",
|
|||
|
|
" width: 50%;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel-item:last-child::after {\n",
|
|||
|
|
" align-self: flex-start;\n",
|
|||
|
|
" width: 50%;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-parallel-item:only-child::after {\n",
|
|||
|
|
" width: 0;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Serial-specific style estimator block */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-serial {\n",
|
|||
|
|
" display: flex;\n",
|
|||
|
|
" flex-direction: column;\n",
|
|||
|
|
" align-items: center;\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
" padding-right: 1em;\n",
|
|||
|
|
" padding-left: 1em;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
|
|||
|
|
"clickable and can be expanded/collapsed.\n",
|
|||
|
|
"- Pipeline and ColumnTransformer use this feature and define the default style\n",
|
|||
|
|
"- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
|
|||
|
|
"*/\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Pipeline and ColumnTransformer style (default) */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-toggleable {\n",
|
|||
|
|
" /* Default theme specific background. It is overwritten whether we have a\n",
|
|||
|
|
" specific estimator or a Pipeline/ColumnTransformer */\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Toggleable label */\n",
|
|||
|
|
"#sk-container-id-8 label.sk-toggleable__label {\n",
|
|||
|
|
" cursor: pointer;\n",
|
|||
|
|
" display: block;\n",
|
|||
|
|
" width: 100%;\n",
|
|||
|
|
" margin-bottom: 0;\n",
|
|||
|
|
" padding: 0.5em;\n",
|
|||
|
|
" box-sizing: border-box;\n",
|
|||
|
|
" text-align: center;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 label.sk-toggleable__label-arrow:before {\n",
|
|||
|
|
" /* Arrow on the left of the label */\n",
|
|||
|
|
" content: \"▸\";\n",
|
|||
|
|
" float: left;\n",
|
|||
|
|
" margin-right: 0.25em;\n",
|
|||
|
|
" color: var(--sklearn-color-icon);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Toggleable content - dropdown */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-toggleable__content {\n",
|
|||
|
|
" max-height: 0;\n",
|
|||
|
|
" max-width: 0;\n",
|
|||
|
|
" overflow: hidden;\n",
|
|||
|
|
" text-align: left;\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-toggleable__content.fitted {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-toggleable__content pre {\n",
|
|||
|
|
" margin: 0.2em;\n",
|
|||
|
|
" border-radius: 0.25em;\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-toggleable__content.fitted pre {\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
|
|||
|
|
" /* Expand drop-down */\n",
|
|||
|
|
" max-height: 200px;\n",
|
|||
|
|
" max-width: 100%;\n",
|
|||
|
|
" overflow: auto;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
|
|||
|
|
" content: \"▾\";\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Pipeline/ColumnTransformer-specific style */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Estimator-specific style */\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Colorize estimator box */\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label label.sk-toggleable__label,\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label label {\n",
|
|||
|
|
" /* The background is the default theme color */\n",
|
|||
|
|
" color: var(--sklearn-color-text-on-default-background);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* On hover, darken the color of the background */\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Label box, darken color on hover, fitted */\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Estimator label */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label label {\n",
|
|||
|
|
" font-family: monospace;\n",
|
|||
|
|
" font-weight: bold;\n",
|
|||
|
|
" display: inline-block;\n",
|
|||
|
|
" line-height: 1.2em;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-label-container {\n",
|
|||
|
|
" text-align: center;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Estimator-specific */\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator {\n",
|
|||
|
|
" font-family: monospace;\n",
|
|||
|
|
" border: 1px dotted var(--sklearn-color-border-box);\n",
|
|||
|
|
" border-radius: 0.25em;\n",
|
|||
|
|
" box-sizing: border-box;\n",
|
|||
|
|
" margin-bottom: 0.5em;\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator.fitted {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* on hover */\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator:hover {\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 div.sk-estimator.fitted:hover {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Common style for \"i\" and \"?\" */\n",
|
|||
|
|
"\n",
|
|||
|
|
".sk-estimator-doc-link,\n",
|
|||
|
|
"a:link.sk-estimator-doc-link,\n",
|
|||
|
|
"a:visited.sk-estimator-doc-link {\n",
|
|||
|
|
" float: right;\n",
|
|||
|
|
" font-size: smaller;\n",
|
|||
|
|
" line-height: 1em;\n",
|
|||
|
|
" font-family: monospace;\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
" border-radius: 1em;\n",
|
|||
|
|
" height: 1em;\n",
|
|||
|
|
" width: 1em;\n",
|
|||
|
|
" text-decoration: none !important;\n",
|
|||
|
|
" margin-left: 1ex;\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|||
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
".sk-estimator-doc-link.fitted,\n",
|
|||
|
|
"a:link.sk-estimator-doc-link.fitted,\n",
|
|||
|
|
"a:visited.sk-estimator-doc-link.fitted {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|||
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* On hover */\n",
|
|||
|
|
"div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
|
|||
|
|
".sk-estimator-doc-link:hover,\n",
|
|||
|
|
"div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
|
|||
|
|
".sk-estimator-doc-link:hover {\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|||
|
|
" color: var(--sklearn-color-background);\n",
|
|||
|
|
" text-decoration: none;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|||
|
|
".sk-estimator-doc-link.fitted:hover,\n",
|
|||
|
|
"div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|||
|
|
".sk-estimator-doc-link.fitted:hover {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|||
|
|
" color: var(--sklearn-color-background);\n",
|
|||
|
|
" text-decoration: none;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* Span, style for the box shown on hovering the info icon */\n",
|
|||
|
|
".sk-estimator-doc-link span {\n",
|
|||
|
|
" display: none;\n",
|
|||
|
|
" z-index: 9999;\n",
|
|||
|
|
" position: relative;\n",
|
|||
|
|
" font-weight: normal;\n",
|
|||
|
|
" right: .2ex;\n",
|
|||
|
|
" padding: .5ex;\n",
|
|||
|
|
" margin: .5ex;\n",
|
|||
|
|
" width: min-content;\n",
|
|||
|
|
" min-width: 20ex;\n",
|
|||
|
|
" max-width: 50ex;\n",
|
|||
|
|
" color: var(--sklearn-color-text);\n",
|
|||
|
|
" box-shadow: 2pt 2pt 4pt #999;\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background: var(--sklearn-color-unfitted-level-0);\n",
|
|||
|
|
" border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
".sk-estimator-doc-link.fitted span {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background: var(--sklearn-color-fitted-level-0);\n",
|
|||
|
|
" border: var(--sklearn-color-fitted-level-3);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
".sk-estimator-doc-link:hover span {\n",
|
|||
|
|
" display: block;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* \"?\"-specific style due to the `<a>` HTML tag */\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 a.estimator_doc_link {\n",
|
|||
|
|
" float: right;\n",
|
|||
|
|
" font-size: 1rem;\n",
|
|||
|
|
" line-height: 1em;\n",
|
|||
|
|
" font-family: monospace;\n",
|
|||
|
|
" background-color: var(--sklearn-color-background);\n",
|
|||
|
|
" border-radius: 1rem;\n",
|
|||
|
|
" height: 1rem;\n",
|
|||
|
|
" width: 1rem;\n",
|
|||
|
|
" text-decoration: none;\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|||
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 a.estimator_doc_link.fitted {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|||
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"/* On hover */\n",
|
|||
|
|
"#sk-container-id-8 a.estimator_doc_link:hover {\n",
|
|||
|
|
" /* unfitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|||
|
|
" color: var(--sklearn-color-background);\n",
|
|||
|
|
" text-decoration: none;\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"#sk-container-id-8 a.estimator_doc_link.fitted:hover {\n",
|
|||
|
|
" /* fitted */\n",
|
|||
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|||
|
|
"}\n",
|
|||
|
|
"</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=5,\n",
|
|||
|
|
" estimator=Pipeline(steps=[('features_preprocessing',\n",
|
|||
|
|
" ColumnTransformer(remainder='passthrough',\n",
|
|||
|
|
" transformers=[('prepocessing_num',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(strategy='median')),\n",
|
|||
|
|
" ('scaler',\n",
|
|||
|
|
" StandardScaler())]),\n",
|
|||
|
|
" []),\n",
|
|||
|
|
" ('prepocessing_cat',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(fill_value='unknown',\n",
|
|||
|
|
" strategy='constant')),\n",
|
|||
|
|
" ('enc...\n",
|
|||
|
|
" OneHotEncoder(drop='first',\n",
|
|||
|
|
" handle_unknown='ignore',\n",
|
|||
|
|
" sparse_output=False))]),\n",
|
|||
|
|
" ['gender',\n",
|
|||
|
|
" 'race/ethnicity',\n",
|
|||
|
|
" 'parental '\n",
|
|||
|
|
" 'level '\n",
|
|||
|
|
" 'of '\n",
|
|||
|
|
" 'education',\n",
|
|||
|
|
" 'lunch',\n",
|
|||
|
|
" 'test '\n",
|
|||
|
|
" 'preparation '\n",
|
|||
|
|
" 'course'])])),\n",
|
|||
|
|
" ('model',\n",
|
|||
|
|
" GradientBoostingRegressor(random_state=9))]),\n",
|
|||
|
|
" n_jobs=-1,\n",
|
|||
|
|
" param_grid={'model__learning_rate': [0.01, 0.1, 0.2],\n",
|
|||
|
|
" 'model__max_depth': [3, 5, 7],\n",
|
|||
|
|
" 'model__n_estimators': [50, 100, 200]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-85\" type=\"checkbox\" ><label for=\"sk-estimator-id-85\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> GridSearchCV<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(cv=5,\n",
|
|||
|
|
" estimator=Pipeline(steps=[('features_preprocessing',\n",
|
|||
|
|
" ColumnTransformer(remainder='passthrough',\n",
|
|||
|
|
" transformers=[('prepocessing_num',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(strategy='median')),\n",
|
|||
|
|
" ('scaler',\n",
|
|||
|
|
" StandardScaler())]),\n",
|
|||
|
|
" []),\n",
|
|||
|
|
" ('prepocessing_cat',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(fill_value='unknown',\n",
|
|||
|
|
" strategy='constant')),\n",
|
|||
|
|
" ('enc...\n",
|
|||
|
|
" OneHotEncoder(drop='first',\n",
|
|||
|
|
" handle_unknown='ignore',\n",
|
|||
|
|
" sparse_output=False))]),\n",
|
|||
|
|
" ['gender',\n",
|
|||
|
|
" 'race/ethnicity',\n",
|
|||
|
|
" 'parental '\n",
|
|||
|
|
" 'level '\n",
|
|||
|
|
" 'of '\n",
|
|||
|
|
" 'education',\n",
|
|||
|
|
" 'lunch',\n",
|
|||
|
|
" 'test '\n",
|
|||
|
|
" 'preparation '\n",
|
|||
|
|
" 'course'])])),\n",
|
|||
|
|
" ('model',\n",
|
|||
|
|
" GradientBoostingRegressor(random_state=9))]),\n",
|
|||
|
|
" n_jobs=-1,\n",
|
|||
|
|
" param_grid={'model__learning_rate': [0.01, 0.1, 0.2],\n",
|
|||
|
|
" 'model__max_depth': [3, 5, 7],\n",
|
|||
|
|
" 'model__n_estimators': [50, 100, 200]})</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-86\" type=\"checkbox\" ><label for=\"sk-estimator-id-86\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">best_estimator_: Pipeline</label><div class=\"sk-toggleable__content fitted\"><pre>Pipeline(steps=[('features_preprocessing',\n",
|
|||
|
|
" ColumnTransformer(remainder='passthrough',\n",
|
|||
|
|
" transformers=[('prepocessing_num',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(strategy='median')),\n",
|
|||
|
|
" ('scaler',\n",
|
|||
|
|
" StandardScaler())]),\n",
|
|||
|
|
" []),\n",
|
|||
|
|
" ('prepocessing_cat',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(fill_value='unknown',\n",
|
|||
|
|
" strategy='constant')),\n",
|
|||
|
|
" ('encoder',\n",
|
|||
|
|
" OneHotEncoder(drop='first',\n",
|
|||
|
|
" handle_unknown='ignore',\n",
|
|||
|
|
" sparse_output=False))]),\n",
|
|||
|
|
" ['gender', 'race/ethnicity',\n",
|
|||
|
|
" 'parental level of '\n",
|
|||
|
|
" 'education',\n",
|
|||
|
|
" 'lunch',\n",
|
|||
|
|
" 'test preparation '\n",
|
|||
|
|
" 'course'])])),\n",
|
|||
|
|
" ('model',\n",
|
|||
|
|
" GradientBoostingRegressor(learning_rate=0.01, n_estimators=200,\n",
|
|||
|
|
" random_state=9))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-87\" type=\"checkbox\" ><label for=\"sk-estimator-id-87\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> features_preprocessing: ColumnTransformer<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.compose.ColumnTransformer.html\">?<span>Documentation for features_preprocessing: ColumnTransformer</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>ColumnTransformer(remainder='passthrough',\n",
|
|||
|
|
" transformers=[('prepocessing_num',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(strategy='median')),\n",
|
|||
|
|
" ('scaler', StandardScaler())]),\n",
|
|||
|
|
" []),\n",
|
|||
|
|
" ('prepocessing_cat',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(fill_value='unknown',\n",
|
|||
|
|
" strategy='constant')),\n",
|
|||
|
|
" ('encoder',\n",
|
|||
|
|
" OneHotEncoder(drop='first',\n",
|
|||
|
|
" handle_unknown='ignore',\n",
|
|||
|
|
" sparse_output=False))]),\n",
|
|||
|
|
" ['gender', 'race/ethnicity',\n",
|
|||
|
|
" 'parental level of education', 'lunch',\n",
|
|||
|
|
" 'test preparation course'])])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-88\" type=\"checkbox\" ><label for=\"sk-estimator-id-88\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">prepocessing_num</label><div class=\"sk-toggleable__content fitted\"><pre>[]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-89\" type=\"checkbox\" ><label for=\"sk-estimator-id-89\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> SimpleImputer<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.impute.SimpleImputer.html\">?<span>Documentation for SimpleImputer</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>SimpleImputer(strategy='median')</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-90\" type=\"checkbox\" ><label for=\"sk-estimator-id-90\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> StandardScaler<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.StandardScaler.html\">?<span>Documentation for StandardScaler</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>StandardScaler()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-91\" type=\"checkbox\" ><label for=\"sk-estimator-id-91\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">prepocessing_cat</label><div class=\"sk-toggleable__content fitted\"><pre>['gender', 'race/ethnicity', 'parental level of education', 'lunch', 'test preparation course']</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-92\" type=\"checkbox\" ><label for=\"sk-estimator-id-92\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> SimpleImputer<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.impute.SimpleImputer.html\">?<span>Documentation for SimpleImputer</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>SimpleImputer(fill_value='unknown', strategy='constant')</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-93\" type=\"checkbox\" ><label for=\"sk-estimator-id-93\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\"> OneHotEncoder<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.OneHotEncoder.html\">?<span>Documentation for OneHotEncoder</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>OneHotEncoder(drop='first', handle_unknown='ignore', sparse_output=False)</pre></div> </div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class
|
|||
|
|
],
|
|||
|
|
"text/plain": [
|
|||
|
|
"GridSearchCV(cv=5,\n",
|
|||
|
|
" estimator=Pipeline(steps=[('features_preprocessing',\n",
|
|||
|
|
" ColumnTransformer(remainder='passthrough',\n",
|
|||
|
|
" transformers=[('prepocessing_num',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(strategy='median')),\n",
|
|||
|
|
" ('scaler',\n",
|
|||
|
|
" StandardScaler())]),\n",
|
|||
|
|
" []),\n",
|
|||
|
|
" ('prepocessing_cat',\n",
|
|||
|
|
" Pipeline(steps=[('imputer',\n",
|
|||
|
|
" SimpleImputer(fill_value='unknown',\n",
|
|||
|
|
" strategy='constant')),\n",
|
|||
|
|
" ('enc...\n",
|
|||
|
|
" OneHotEncoder(drop='first',\n",
|
|||
|
|
" handle_unknown='ignore',\n",
|
|||
|
|
" sparse_output=False))]),\n",
|
|||
|
|
" ['gender',\n",
|
|||
|
|
" 'race/ethnicity',\n",
|
|||
|
|
" 'parental '\n",
|
|||
|
|
" 'level '\n",
|
|||
|
|
" 'of '\n",
|
|||
|
|
" 'education',\n",
|
|||
|
|
" 'lunch',\n",
|
|||
|
|
" 'test '\n",
|
|||
|
|
" 'preparation '\n",
|
|||
|
|
" 'course'])])),\n",
|
|||
|
|
" ('model',\n",
|
|||
|
|
" GradientBoostingRegressor(random_state=9))]),\n",
|
|||
|
|
" n_jobs=-1,\n",
|
|||
|
|
" param_grid={'model__learning_rate': [0.01, 0.1, 0.2],\n",
|
|||
|
|
" 'model__max_depth': [3, 5, 7],\n",
|
|||
|
|
" 'model__n_estimators': [50, 100, 200]})"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"execution_count": 182,
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "execute_result"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.linear_model import LinearRegression\n",
|
|||
|
|
"from sklearn.ensemble import RandomForestRegressor\n",
|
|||
|
|
"from sklearn.ensemble import GradientBoostingRegressor\n",
|
|||
|
|
"from sklearn.model_selection import GridSearchCV\n",
|
|||
|
|
"from sklearn.pipeline import Pipeline\n",
|
|||
|
|
"\n",
|
|||
|
|
"# 1. Линейная регрессия\n",
|
|||
|
|
"linear_model = LinearRegression()\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Параметры для настройки\n",
|
|||
|
|
"linear_param_grid = {\n",
|
|||
|
|
" 'model__fit_intercept': [True, False], # Использовать ли свободный член\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"# 2. Случайный лес\n",
|
|||
|
|
"rf_model = RandomForestRegressor(random_state=random_state)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Параметры для настройки\n",
|
|||
|
|
"rf_param_grid = {\n",
|
|||
|
|
" 'model__n_estimators': [50, 100, 200],\n",
|
|||
|
|
" 'model__max_depth': [None, 10, 20],\n",
|
|||
|
|
" 'model__min_samples_split': [2, 5],\n",
|
|||
|
|
" 'model__min_samples_leaf': [1, 2],\n",
|
|||
|
|
" 'model__max_features': ['sqrt', 'log2'], # Заменить 'auto' на 'sqrt'\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"# 3. Градиентный бустинг\n",
|
|||
|
|
"gb_model = GradientBoostingRegressor(random_state=random_state)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Параметры для настройки\n",
|
|||
|
|
"gb_param_grid = {\n",
|
|||
|
|
" 'model__n_estimators': [50, 100, 200], # Количество деревьев\n",
|
|||
|
|
" 'model__learning_rate': [0.01, 0.1, 0.2], # Темп обучения\n",
|
|||
|
|
" 'model__max_depth': [3, 5, 7], # Максимальная глубина деревьев\n",
|
|||
|
|
"}\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Создание пайплайна для линейной регрессии\n",
|
|||
|
|
"linear_pipeline = Pipeline([\n",
|
|||
|
|
" (\"features_preprocessing\", features_preprocessing),\n",
|
|||
|
|
" (\"model\", linear_model)\n",
|
|||
|
|
"])\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Создание пайплайна для случайного леса\n",
|
|||
|
|
"rf_pipeline = Pipeline([\n",
|
|||
|
|
" (\"features_preprocessing\", features_preprocessing),\n",
|
|||
|
|
" (\"model\", rf_model)\n",
|
|||
|
|
"])\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Создание пайплайна для градиентного бустинга\n",
|
|||
|
|
"gb_pipeline = Pipeline([\n",
|
|||
|
|
" (\"features_preprocessing\", features_preprocessing),\n",
|
|||
|
|
" (\"model\", gb_model)\n",
|
|||
|
|
"])\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Настройка гиперпараметров с использованием GridSearchCV\n",
|
|||
|
|
"linear_search = GridSearchCV(linear_pipeline, linear_param_grid, cv=5, n_jobs=-1)\n",
|
|||
|
|
"rf_search = GridSearchCV(rf_pipeline, rf_param_grid, cv=5, n_jobs=-1)\n",
|
|||
|
|
"gb_search = GridSearchCV(gb_pipeline, gb_param_grid, cv=5, n_jobs=-1)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Обучение моделей\n",
|
|||
|
|
"linear_search.fit(X_train, y_train.values.ravel()) # Преобразуем y_train в одномерный массив\n",
|
|||
|
|
"rf_search.fit(X_train, y_train.values.ravel()) # Преобразуем y_train в одномерный массив\n",
|
|||
|
|
"gb_search.fit(X_train, y_train.values.ravel()) # Преобразуем y_train в одномерный массив\n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Оценка качества моделей\n",
|
|||
|
|
"\n",
|
|||
|
|
"Для оценки качества моделей будем использовать следующие метрики:\n",
|
|||
|
|
"\n",
|
|||
|
|
"R^2 — это классическая метрика для оценки качества модели в регрессии. Она покажет, насколько хорошо модель объясняет данные, и полезна, чтобы понять, имеет ли смысл использовать модель.\n",
|
|||
|
|
"\n",
|
|||
|
|
"MAE и MSE — они дают представление о точности модели и масштабах ошибок. MAE поможет понять средний уровень ошибки, а MSE и RMSE — сосредоточатся на более крупных ошибках, что может быть критично для задач, где крупные ошибки имеют большое значение.\n",
|
|||
|
|
"\n",
|
|||
|
|
"RMSE полезна в контексте конкретных приложений, когда важно понять ошибку в тех же единицах, что и целевая переменная, особенно если данные в регрессии имеют четкую физическую или экономическую интерпретацию."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 183,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"name": "stdout",
|
|||
|
|
"output_type": "stream",
|
|||
|
|
"text": [
|
|||
|
|
"Linear Regression - R^2: 0.25121834623887884, MAE: 30.509764831673372, MSE: 1399.0220694458058, RMSE: 37.403503438124694\n",
|
|||
|
|
"Random Forest - R^2: 0.19039682481837739, MAE: 31.82764043548629, MSE: 1512.6608723426755, RMSE: 38.89294116343833\n",
|
|||
|
|
"Gradient Boosting - R^2: 0.19994717344160973, MAE: 31.688226363085978, MSE: 1494.8170210307603, RMSE: 38.66286359067006\n"
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Прогнозы для каждой модели\n",
|
|||
|
|
"y_pred_linear = linear_search.predict(X_test)\n",
|
|||
|
|
"y_pred_rf = rf_search.predict(X_test)\n",
|
|||
|
|
"y_pred_gb = gb_search.predict(X_test)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# R^2\n",
|
|||
|
|
"r2_linear = r2_score(y_test, y_pred_linear)\n",
|
|||
|
|
"r2_rf = r2_score(y_test, y_pred_rf)\n",
|
|||
|
|
"r2_gb = r2_score(y_test, y_pred_gb)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# MAE\n",
|
|||
|
|
"mae_linear = mean_absolute_error(y_test, y_pred_linear)\n",
|
|||
|
|
"mae_rf = mean_absolute_error(y_test, y_pred_rf)\n",
|
|||
|
|
"mae_gb = mean_absolute_error(y_test, y_pred_gb)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# MSE\n",
|
|||
|
|
"mse_linear = mean_squared_error(y_test, y_pred_linear)\n",
|
|||
|
|
"mse_rf = mean_squared_error(y_test, y_pred_rf)\n",
|
|||
|
|
"mse_gb = mean_squared_error(y_test, y_pred_gb)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# RMSE\n",
|
|||
|
|
"rmse_linear = mse_linear ** 0.5\n",
|
|||
|
|
"rmse_rf = mse_rf ** 0.5\n",
|
|||
|
|
"rmse_gb = mse_gb ** 0.5\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Вывод результатов\n",
|
|||
|
|
"print(f\"Linear Regression - R^2: {r2_linear}, MAE: {mae_linear}, MSE: {mse_linear}, RMSE: {rmse_linear}\")\n",
|
|||
|
|
"print(f\"Random Forest - R^2: {r2_rf}, MAE: {mae_rf}, MSE: {mse_rf}, RMSE: {rmse_rf}\")\n",
|
|||
|
|
"print(f\"Gradient Boosting - R^2: {r2_gb}, MAE: {mae_gb}, MSE: {mse_gb}, RMSE: {rmse_gb}\")\n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"Линейная регрессия: По всем метрикам (R^2, MAE, MSE, RMSE) линейная регрессия показывает лучшие результаты. Она объясняет большую часть вариации в данных и имеет наименьшие ошибки как в абсолютных величинах, так и в квадратных отклонениях.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Random Forest и Gradient Boosting: Эти модели показывают худшие результаты по сравнению с линейной регрессией, как по объяснению вариации данных, так и по величине ошибок."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"#### Оценка смещения и дисперсии лучшей модели (линейная регрессия)."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 184,
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"name": "stdout",
|
|||
|
|
"output_type": "stream",
|
|||
|
|
"text": [
|
|||
|
|
"Смещение (Bias): 0.13896485197494712\n",
|
|||
|
|
"Дисперсия (Variance): 432.34643616182956\n",
|
|||
|
|
"RMSE: 37.95928016105437\n"
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"import numpy as np\n",
|
|||
|
|
"from sklearn.model_selection import cross_val_predict, KFold\n",
|
|||
|
|
"from sklearn.metrics import mean_squared_error\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Количество фолдов для кросс-валидации\n",
|
|||
|
|
"n_splits = 5\n",
|
|||
|
|
"kf = KFold(n_splits=n_splits, shuffle=True, random_state=42)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Прогнозы с использованием кросс-валидации с конвейером\n",
|
|||
|
|
"y_pred = cross_val_predict(linear_pipeline, X_train, y_train.values.ravel(), cv=kf)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Среднее значение целевой переменной\n",
|
|||
|
|
"y_true_mean = np.mean(y_train)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Смещение: разница между средним предсказанием модели и средним значением целевой переменной\n",
|
|||
|
|
"bias = np.mean(y_pred) - y_true_mean\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Дисперсия: среднее отклонение предсказаний от среднего предсказания\n",
|
|||
|
|
"variance = np.mean((y_pred - np.mean(y_pred)) ** 2)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Средняя ошибка (для контекста)\n",
|
|||
|
|
"mse = mean_squared_error(y_train, y_pred)\n",
|
|||
|
|
"rmse = np.sqrt(mse)\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Выводим результаты\n",
|
|||
|
|
"print(f\"Смещение (Bias): {bias}\")\n",
|
|||
|
|
"print(f\"Дисперсия (Variance): {variance}\")\n",
|
|||
|
|
"print(f\"RMSE: {rmse}\")\n"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"Смещение и дисперсия вместе составляют основную часть ошибки модели. Смещение (bias) относительно невелико, но модель имеет достаточно высокую дисперсию. Это означает, что модель может страдать от переобучения (overfitting), то есть она слишком хорошо подгоняется под обучающие данные и не обобщает информацию на новые данные. В таких случаях модель может показывать хорошие результаты на обучающих данных, но её способность предсказывать для новых, невиданных данных может быть ограничена.\n",
|
|||
|
|
"\n",
|
|||
|
|
"RMSE также указывает на наличие значительных ошибок в предсказаниях, и, вероятно, модель нуждается в улучшении. "
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"metadata": {
|
|||
|
|
"kernelspec": {
|
|||
|
|
"display_name": "aimvenv",
|
|||
|
|
"language": "python",
|
|||
|
|
"name": "python3"
|
|||
|
|
},
|
|||
|
|
"language_info": {
|
|||
|
|
"codemirror_mode": {
|
|||
|
|
"name": "ipython",
|
|||
|
|
"version": 3
|
|||
|
|
},
|
|||
|
|
"file_extension": ".py",
|
|||
|
|
"mimetype": "text/x-python",
|
|||
|
|
"name": "python",
|
|||
|
|
"nbconvert_exporter": "python",
|
|||
|
|
"pygments_lexer": "ipython3",
|
|||
|
|
"version": "3.12.6"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"nbformat": 4,
|
|||
|
|
"nbformat_minor": 2
|
|||
|
|
}
|