{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Visualization of MLP weights on MNIST\n\nSometimes looking at the learned coefficients of a neural network can provide\ninsight into the learning behavior. For example if weights look unstructured,\nmaybe some were not used at all, or if very large coefficients exist, maybe\nregularization was too low or the learning rate too high.\n\nThis example shows how to plot some of the first layer weights in a\nMLPClassifier trained on the MNIST dataset.\n\nThe input data consists of 28x28 pixel handwritten digits, leading to 784\nfeatures in the dataset. Therefore the first layer weight matrix has the shape\n(784, hidden_layer_sizes[0]).  We can therefore visualize a single column of\nthe weight matrix as a 28x28 pixel image.\n\nTo make the example run faster, we use very few hidden units, and train only\nfor a very short time. Training longer would result in weights with a much\nsmoother spatial appearance. The example will throw a warning because it\ndoesn't converge, in this case this is what we want because of resource\nusage constraints on our Continuous Integration infrastructure that is used\nto build this documentation on a regular basis.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Authors: The scikit-learn developers\n# SPDX-License-Identifier: BSD-3-Clause\n\nimport warnings\n\nimport matplotlib.pyplot as plt\n\nfrom sklearn.datasets import fetch_openml\nfrom sklearn.exceptions import ConvergenceWarning\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.neural_network import MLPClassifier\n\n# Load data from https://www.openml.org/d/554\nX, y = fetch_openml(\"mnist_784\", version=1, return_X_y=True, as_frame=False)\nX = X / 255.0\n\n# Split data into train partition and test partition\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, test_size=0.7)\n\nmlp = MLPClassifier(\n    hidden_layer_sizes=(40,),\n    max_iter=8,\n    alpha=1e-4,\n    solver=\"sgd\",\n    verbose=10,\n    random_state=1,\n    learning_rate_init=0.2,\n)\n\n# this example won't converge because of resource usage constraints on\n# our Continuous Integration infrastructure, so we catch the warning and\n# ignore it here\nwith warnings.catch_warnings():\n    warnings.filterwarnings(\"ignore\", category=ConvergenceWarning, module=\"sklearn\")\n    mlp.fit(X_train, y_train)\n\nprint(\"Training set score: %f\" % mlp.score(X_train, y_train))\nprint(\"Test set score: %f\" % mlp.score(X_test, y_test))\n\nfig, axes = plt.subplots(4, 4)\n# use global min / max to ensure all weights are shown on the same scale\nvmin, vmax = mlp.coefs_[0].min(), mlp.coefs_[0].max()\nfor coef, ax in zip(mlp.coefs_[0].T, axes.ravel()):\n    ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=0.5 * vmin, vmax=0.5 * vmax)\n    ax.set_xticks(())\n    ax.set_yticks(())\n\nplt.show()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.14"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}