.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/use_cases/plot_employee_salaries.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_use_cases_plot_employee_salaries.py: .. _example_use_case_employee_salaries: ============================================== Simplified and structured experiment reporting ============================================== This example shows how to leverage `skore` for structuring useful experiment information allowing to get insights from machine learning experiments. .. GENERATED FROM PYTHON SOURCE LINES 13-15 Loading a non-trivial dataset ============================= .. GENERATED FROM PYTHON SOURCE LINES 17-19 We use a skrub dataset that contains information about employees and their salaries. We will see that this dataset is non-trivial. .. GENERATED FROM PYTHON SOURCE LINES 21-26 .. code-block:: Python from skrub.datasets import fetch_employee_salaries datasets = fetch_employee_salaries() df, y = datasets.X, datasets.y .. GENERATED FROM PYTHON SOURCE LINES 27-29 Let's first have a condensed summary of the input data using a :class:`skrub.TableReport`. .. GENERATED FROM PYTHON SOURCE LINES 31-35 .. code-block:: Python from skrub import TableReport TableReport(df) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 36-60 From the table report, we can make the following observations: * Looking at the *Table* tab, we observe that the year related to the ``date_first_hired`` column is also present in the ``date`` column. Hence, we should beware of not creating twice the same feature during the feature engineering. * Looking at the *Stats* tab: - The type of data is heterogeneous: we mainly have categorical and date-related features. - The ``division`` and ``employee_position_title`` features contain a large number of categories. It is something that we should consider in our feature engineering. * Looking at the *Associations* tab, we observe that two features are holding the exact same information: ``department`` and ``department_name``. Hence, during our feature engineering, we could potentially drop one of them if the final predictive model is sensitive to the collinearity. In terms of target and thus the task that we want to solve, we are interested in predicting the salary of an employee given the previous features. We therefore have a regression task at end. .. GENERATED FROM PYTHON SOURCE LINES 62-64 .. code-block:: Python TableReport(y) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 65-68 Later in this example, we will show that `skore` stores similar information when a model is trained on a dataset, thus enabling us to get quick insights on the dataset used to train and test the model. .. GENERATED FROM PYTHON SOURCE LINES 70-84 Tree-based model ================ Let's start by creating a tree-based model using some out-of-the-box tools. For feature engineering, we use skrub's :class:`~skrub.TableVectorizer`. To deal with the high cardinality of the categorical features, we use a :class:`~skrub.StringEncoder` to encode the categorical features. Finally, we use a :class:`~sklearn.ensemble.HistGradientBoostingRegressor` as a base estimator, it is a rather robust model. Modelling ^^^^^^^^^ .. GENERATED FROM PYTHON SOURCE LINES 86-96 .. code-block:: Python from sklearn.ensemble import HistGradientBoostingRegressor from sklearn.pipeline import make_pipeline from skrub import StringEncoder, TableVectorizer model = make_pipeline( TableVectorizer(high_cardinality=StringEncoder()), HistGradientBoostingRegressor(), ) model .. raw:: html
Pipeline(steps=[('tablevectorizer', TableVectorizer()),
                    ('histgradientboostingregressor',
                     HistGradientBoostingRegressor())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 97-102 Evaluation ^^^^^^^^^^ Let us compute the cross-validation report for this model using a :class:`skore.CrossValidationReport`: .. GENERATED FROM PYTHON SOURCE LINES 104-111 .. code-block:: Python from skore import CrossValidationReport hgbt_model_report = CrossValidationReport( estimator=model, X=df, y=y, splitter=5, n_jobs=4 ) hgbt_model_report.help() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 112-116 A report provides a collection of useful information. For instance, it allows to compute on demand the predictions of the model and some performance metrics. Let's cache the predictions of the cross-validated models once and for all. .. GENERATED FROM PYTHON SOURCE LINES 118-120 .. code-block:: Python hgbt_model_report.cache_predictions(n_jobs=4) .. GENERATED FROM PYTHON SOURCE LINES 121-125 Now that the predictions are cached, any request to compute a metric will be performed using the cached predictions and will thus be fast. We can now have a look at the performance of the model with some standard metrics. .. GENERATED FROM PYTHON SOURCE LINES 127-129 .. code-block:: Python hgbt_model_report.metrics.summarize().frame() .. raw:: html
HistGradientBoostingRegressor
mean std
Metric
0.911027 0.016488
RMSE 8672.883305 1111.578431
Fit time (s) 2.611937 0.472511
Predict time (s) 0.172976 0.010085


.. GENERATED FROM PYTHON SOURCE LINES 130-133 Similarly to what we saw in the previous section, the :class:`skore.CrossValidationReport` also stores some information about the dataset used. .. GENERATED FROM PYTHON SOURCE LINES 135-138 .. code-block:: Python data_display = hgbt_model_report.data.analyze() data_display .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 139-144 The display obtained allows for a quick overview with the same HTML-based view as the :class:`skrub.TableReport` we have seen earlier. In addition, you can access a :meth:`skore.TableReportDisplay.plot` method to have a particular focus on one potential analysis. For instance, we can get a figure representing the correlation matrix of the dataset. .. GENERATED FROM PYTHON SOURCE LINES 146-148 .. code-block:: Python data_display.plot(kind="corr") .. image-sg:: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_001.png :alt: Cramer's V Correlation :srcset: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 149-156 We get the results from some statistical metrics aggregated over the cross-validation splits as well as some performance metrics related to the time it took to train and test the model. The :class:`skore.CrossValidationReport` also provides a way to inspect similar information at the level of each cross-validation split by accessing an :class:`skore.EstimatorReport` for each split. .. GENERATED FROM PYTHON SOURCE LINES 158-161 .. code-block:: Python hgbt_split_1 = hgbt_model_report.estimator_reports_[0] hgbt_split_1.metrics.summarize(favorability=True).frame() .. raw:: html
HistGradientBoostingRegressor Favorability
Metric
0.910637 (↗︎)
RMSE 8607.961973 (↘︎)
Fit time (s) 2.815443 (↘︎)
Predict time (s) 0.167178 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 162-164 The favorability of each metric indicates whether the metric is better when higher or lower. .. GENERATED FROM PYTHON SOURCE LINES 166-172 Linear model ============ Now that we have established a first model that serves as a baseline, we shall proceed to define a quite complex linear model: a pipeline with a complex feature engineering that uses a linear model as the base estimator. .. GENERATED FROM PYTHON SOURCE LINES 174-176 Modelling ^^^^^^^^^ .. GENERATED FROM PYTHON SOURCE LINES 178-226 .. code-block:: Python import numpy as np from sklearn.compose import make_column_transformer from sklearn.linear_model import RidgeCV from sklearn.pipeline import make_pipeline from sklearn.preprocessing import OneHotEncoder, SplineTransformer from skrub import DatetimeEncoder, DropCols, GapEncoder, ToDatetime def periodic_spline_transformer(period, n_splines=None, degree=3): if n_splines is None: n_splines = period n_knots = n_splines + 1 # periodic and include_bias is True return SplineTransformer( degree=degree, n_knots=n_knots, knots=np.linspace(0, period, n_knots).reshape(n_knots, 1), extrapolation="periodic", include_bias=True, ) one_hot_features = ["gender", "department_name", "assignment_category"] datetime_features = "date_first_hired" date_encoder = make_pipeline( ToDatetime(), DatetimeEncoder(resolution="day", add_weekday=True, add_total_seconds=False), DropCols("date_first_hired_year"), ) date_engineering = make_column_transformer( (periodic_spline_transformer(12, n_splines=6), ["date_first_hired_month"]), (periodic_spline_transformer(31, n_splines=15), ["date_first_hired_day"]), (periodic_spline_transformer(7, n_splines=3), ["date_first_hired_weekday"]), ) feature_engineering_date = make_pipeline(date_encoder, date_engineering) preprocessing = make_column_transformer( (feature_engineering_date, datetime_features), (OneHotEncoder(drop="if_binary", handle_unknown="ignore"), one_hot_features), (GapEncoder(n_components=100), "division"), (GapEncoder(n_components=100), "employee_position_title"), ) model = make_pipeline(preprocessing, RidgeCV(alphas=np.logspace(-3, 3, 100))) model .. raw:: html
Pipeline(steps=[('columntransformer',
                     ColumnTransformer(transformers=[('pipeline',
                                                      Pipeline(steps=[('pipeline',
                                                                       Pipeline(steps=[('todatetime',
                                                                                        ToDatetime()),
                                                                                       ('datetimeencoder',
                                                                                        DatetimeEncoder(add_total_seconds=False,
                                                                                                        add_weekday=True,
                                                                                                        resolution='day')),
                                                                                       ('dropcols',
                                                                                        DropCols(cols='date_first_hired_year'))])),
                                                                      ('columntransformer',
                                                                       ColumnTransformer(transfor...
           4.03701726e+01, 4.64158883e+01, 5.33669923e+01, 6.13590727e+01,
           7.05480231e+01, 8.11130831e+01, 9.32603347e+01, 1.07226722e+02,
           1.23284674e+02, 1.41747416e+02, 1.62975083e+02, 1.87381742e+02,
           2.15443469e+02, 2.47707636e+02, 2.84803587e+02, 3.27454916e+02,
           3.76493581e+02, 4.32876128e+02, 4.97702356e+02, 5.72236766e+02,
           6.57933225e+02, 7.56463328e+02, 8.69749003e+02, 1.00000000e+03])))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 227-240 In the diagram above, we can see what how we performed our feature engineering: * For categorical features, we use two approaches. If the number of categories is relatively small, we use a `OneHotEncoder`. If the number of categories is large, we use a `GapEncoder` that is designed to deal with high cardinality categorical features. * Then, we have another transformation to encode the date features. We first split the date into multiple features (day, month, and year). Then, we apply a periodic spline transformation to each of the date features in order to capture the periodicity of the data. * Finally, we fit a :class:`~sklearn.linear_model.RidgeCV` model. .. GENERATED FROM PYTHON SOURCE LINES 242-248 Evaluation ^^^^^^^^^^ Now, we want to evaluate this linear model via cross-validation (with 5 folds). For that, we use skore's :class:`~skore.CrossValidationReport` to investigate the performance of our model. .. GENERATED FROM PYTHON SOURCE LINES 250-255 .. code-block:: Python linear_model_report = CrossValidationReport( estimator=model, X=df, y=y, splitter=5, n_jobs=4 ) linear_model_report.help() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 256-264 We observe that the cross-validation report has detected that we have a regression task at hand and thus provides us with some metrics and plots that make sense with regards to our specific problem at hand. To accelerate any future computation (e.g. of a metric), we cache the predictions of our model once and for all. Note that we do not necessarily need to cache the predictions as the report will compute them on the fly (if not cached) and cache them for us. .. GENERATED FROM PYTHON SOURCE LINES 266-272 .. code-block:: Python import warnings with warnings.catch_warnings(): warnings.simplefilter(action="ignore", category=FutureWarning) linear_model_report.cache_predictions(n_jobs=4) .. GENERATED FROM PYTHON SOURCE LINES 273-274 We can now have a look at the performance of the model with some standard metrics. .. GENERATED FROM PYTHON SOURCE LINES 276-278 .. code-block:: Python linear_model_report.metrics.summarize(favorability=True).frame() .. raw:: html
RidgeCV Favorability
mean std
Metric
0.765180 0.022767 (↗︎)
RMSE 14103.629945 1172.175959 (↘︎)
Fit time (s) 10.064724 2.052521 (↘︎)
Predict time (s) 0.408238 0.008181 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 279-284 Comparing the models ==================== Now that we cross-validated our models, we can make some further comparison using the :class:`skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 286-291 .. code-block:: Python from skore import ComparisonReport comparator = ComparisonReport([hgbt_model_report, linear_model_report]) comparator.metrics.summarize(favorability=True).frame() .. raw:: html
mean std Favorability
Estimator HistGradientBoostingRegressor RidgeCV HistGradientBoostingRegressor RidgeCV
Metric
0.911027 0.765180 0.016488 0.022767 (↗︎)
RMSE 8672.883305 14103.629945 1111.578431 1172.175959 (↘︎)
Fit time (s) 2.611937 10.064724 0.472511 2.052521 (↘︎)
Predict time (s) 0.172976 0.408238 0.010085 0.008181 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 292-297 In addition, if we forgot to compute a specific metric (e.g. :func:`~sklearn.metrics.mean_absolute_error`), we can easily add it to the report, without re-training the model and even without re-computing the predictions since they are cached internally in the report. This allows us to save some potentially huge computation time. .. GENERATED FROM PYTHON SOURCE LINES 299-306 .. code-block:: Python from sklearn.metrics import get_scorer metric = {"R²": "r2", "RMSE": "rmse", "MAE": get_scorer("neg_mean_absolute_error")} metric_kwargs = {"response_method": "predict"} comparator.metrics.summarize(metric=metric, metric_kwargs=metric_kwargs).frame() .. raw:: html
mean std
Estimator HistGradientBoostingRegressor RidgeCV HistGradientBoostingRegressor RidgeCV
Metric
0.911027 0.765180 0.016488 0.022767
RMSE 8672.883305 14103.629945 1111.578431 1172.175959
MAE 4672.991598 9939.712091 176.786948 390.123295


.. GENERATED FROM PYTHON SOURCE LINES 307-310 Finally, we can even get a deeper understanding by analyzing each split in the :class:`~skore.CrossValidationReport`. Here, we plot the actual-vs-predicted values for each split. .. GENERATED FROM PYTHON SOURCE LINES 312-314 .. code-block:: Python linear_model_report.metrics.prediction_error().plot(kind="actual_vs_predicted") .. image-sg:: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_002.png :alt: Prediction Error for RidgeCV Data source: Test set :srcset: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 315-325 Conclusion ========== This example showcased `skore`'s integrated approach to machine learning workflow, from initial data exploration with `TableReport` through model development and evaluation with `CrossValidationReport`. We demonstrated how `skore` automatically captures dataset information and provides efficient caching, enabling quick insights and flexible model comparison. The workflow highlights `skore`'s ability to streamline the entire ML process while maintaining computational efficiency. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 40.977 seconds) .. _sphx_glr_download_auto_examples_use_cases_plot_employee_salaries.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_employee_salaries.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_employee_salaries.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_employee_salaries.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_