{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Column Transformer with Mixed Types\n\n.. currentmodule:: sklearn\n\nThis example illustrates how to apply different preprocessing and feature\nextraction pipelines to different subsets of features, using\n:class:`~compose.ColumnTransformer`. This is particularly handy for the\ncase of datasets that contain heterogeneous data types, since we may want to\nscale the numeric features and one-hot encode the categorical ones.\n\nIn this example, the numeric data is standard-scaled after mean-imputation. The\ncategorical data is one-hot encoded via ``OneHotEncoder``, which\ncreates a new category for missing values. We further reduce the dimensionality\nby selecting categories using a chi-squared test.\n\nIn addition, we show two different ways to dispatch the columns to the\nparticular pre-processor: by column names and by column data types.\n\nFinally, the preprocessing pipeline is integrated in a full prediction pipeline\nusing :class:`~pipeline.Pipeline`, together with a simple classification\nmodel.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Authors: The scikit-learn developers\n# SPDX-License-Identifier: BSD-3-Clause"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import numpy as np\n\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.datasets import fetch_openml\nfrom sklearn.feature_selection import SelectPercentile, chi2\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import RandomizedSearchCV, train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import OneHotEncoder, StandardScaler\n\nnp.random.seed(0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Load data from https://www.openml.org/d/40945\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "X, y = fetch_openml(\"titanic\", version=1, as_frame=True, return_X_y=True)\n\n# Alternatively X and y can be obtained directly from the frame attribute:\n# X = titanic.frame.drop('survived', axis=1)\n# y = titanic.frame['survived']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Use ``ColumnTransformer`` by selecting column by names\n\nWe will train our classifier with the following features:\n\nNumeric Features:\n\n* ``age``: float;\n* ``fare``: float.\n\nCategorical Features:\n\n* ``embarked``: categories encoded as strings ``{'C', 'S', 'Q'}``;\n* ``sex``: categories encoded as strings ``{'female', 'male'}``;\n* ``pclass``: ordinal integers ``{1, 2, 3}``.\n\nWe create the preprocessing pipelines for both numeric and categorical data.\nNote that ``pclass`` could either be treated as a categorical or numeric\nfeature.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "numeric_features = [\"age\", \"fare\"]\nnumeric_transformer = Pipeline(\n    steps=[(\"imputer\", SimpleImputer(strategy=\"median\")), (\"scaler\", StandardScaler())]\n)\n\ncategorical_features = [\"embarked\", \"sex\", \"pclass\"]\ncategorical_transformer = Pipeline(\n    steps=[\n        (\"encoder\", OneHotEncoder(handle_unknown=\"ignore\")),\n        (\"selector\", SelectPercentile(chi2, percentile=50)),\n    ]\n)\npreprocessor = ColumnTransformer(\n    transformers=[\n        (\"num\", numeric_transformer, numeric_features),\n        (\"cat\", categorical_transformer, categorical_features),\n    ]\n)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Append classifier to preprocessing pipeline.\nNow we have a full prediction pipeline.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "clf = Pipeline(\n    steps=[(\"preprocessor\", preprocessor), (\"classifier\", LogisticRegression())]\n)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n\nclf.fit(X_train, y_train)\nprint(\"model score: %.3f\" % clf.score(X_test, y_test))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "HTML representation of ``Pipeline`` (display diagram)\n\nWhen the ``Pipeline`` is printed out in a jupyter notebook an HTML\nrepresentation of the estimator is displayed:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "clf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Use ``ColumnTransformer`` by selecting column by data types\n\nWhen dealing with a cleaned dataset, the preprocessing can be automatic by\nusing the data types of the column to decide whether to treat a column as a\nnumerical or categorical feature.\n:func:`sklearn.compose.make_column_selector` gives this possibility.\nFirst, let's only select a subset of columns to simplify our\nexample.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "subset_feature = [\"embarked\", \"sex\", \"pclass\", \"age\", \"fare\"]\nX_train, X_test = X_train[subset_feature], X_test[subset_feature]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Then, we introspect the information regarding each column data type.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "X_train.info()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can observe that the `embarked` and `sex` columns were tagged as\n`category` columns when loading the data with ``fetch_openml``. Therefore, we\ncan use this information to dispatch the categorical columns to the\n``categorical_transformer`` and the remaining columns to the\n``numerical_transformer``.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-info\"><h4>Note</h4><p>In practice, you will have to handle yourself the column data type.\n   If you want some columns to be considered as `category`, you will have to\n   convert them into categorical columns. If you are using pandas, you can\n   refer to their documentation regarding [Categorical data](https://pandas.pydata.org/pandas-docs/stable/user_guide/categorical.html).</p></div>\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.compose import make_column_selector as selector\n\npreprocessor = ColumnTransformer(\n    transformers=[\n        (\"num\", numeric_transformer, selector(dtype_exclude=\"category\")),\n        (\"cat\", categorical_transformer, selector(dtype_include=\"category\")),\n    ]\n)\nclf = Pipeline(\n    steps=[(\"preprocessor\", preprocessor), (\"classifier\", LogisticRegression())]\n)\n\n\nclf.fit(X_train, y_train)\nprint(\"model score: %.3f\" % clf.score(X_test, y_test))\nclf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The resulting score is not exactly the same as the one from the previous\npipeline because the dtype-based selector treats the ``pclass`` column as\na numeric feature instead of a categorical feature as previously:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "selector(dtype_exclude=\"category\")(X_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "selector(dtype_include=\"category\")(X_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Using the prediction pipeline in a grid search\n\nGrid search can also be performed on the different preprocessing steps\ndefined in the ``ColumnTransformer`` object, together with the classifier's\nhyperparameters as part of the ``Pipeline``.\nWe will search for both the imputer strategy of the numeric preprocessing\nand the regularization parameter of the logistic regression using\n:class:`~sklearn.model_selection.RandomizedSearchCV`. This\nhyperparameter search randomly selects a fixed number of parameter\nsettings configured by `n_iter`. Alternatively, one can use\n:class:`~sklearn.model_selection.GridSearchCV` but the cartesian product of\nthe parameter space will be evaluated.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "param_grid = {\n    \"preprocessor__num__imputer__strategy\": [\"mean\", \"median\"],\n    \"preprocessor__cat__selector__percentile\": [10, 30, 50, 70],\n    \"classifier__C\": [0.1, 1.0, 10, 100],\n}\n\nsearch_cv = RandomizedSearchCV(clf, param_grid, n_iter=10, random_state=0)\nsearch_cv"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Calling 'fit' triggers the cross-validated search for the best\nhyper-parameters combination:\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "search_cv.fit(X_train, y_train)\n\nprint(\"Best params:\")\nprint(search_cv.best_params_)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The internal cross-validation scores obtained by those parameters is:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(f\"Internal CV score: {search_cv.best_score_:.3f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can also introspect the top grid search results as a pandas dataframe:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n\ncv_results = pd.DataFrame(search_cv.cv_results_)\ncv_results = cv_results.sort_values(\"mean_test_score\", ascending=False)\ncv_results[\n    [\n        \"mean_test_score\",\n        \"std_test_score\",\n        \"param_preprocessor__num__imputer__strategy\",\n        \"param_preprocessor__cat__selector__percentile\",\n        \"param_classifier__C\",\n    ]\n].head(5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The best hyper-parameters have be used to re-fit a final model on the full\ntraining set. We can evaluate that final model on held out test data that was\nnot used for hyperparameter tuning.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(\n    \"accuracy of the best model from randomized search: \"\n    f\"{search_cv.score(X_test, y_test):.3f}\"\n)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.14"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}