This article will talk about how to include SimpleImputer before CountVectorizer in a scikit-learn Pipeline. How to include SimpleImputer before CountVectorizer in a scikit-learn Pipeline Solution 1:
The best solution I have found is to insert a custom transformer into the <code>Pipeline</code> that reshapes the output of <code>SimpleImputer</code> from 2D to 1D before it is passed to <code>CountVectorizer</code>.
Here's the complete code:<code>import pandas as pd</code><code>import numpy as np
</code><code> </code><code>df = pd.DataFrame({'text':['abc def', 'abc ghi', np.nan]})</code><code>from sklearn.impute import SimpleImputer
</code><code> </code><code>imp = SimpleImputer(strategy='constant')</code><code> </code><code>from sklearn.feature_extraction.text import CountVectorizer
</code><code> </code><code>vect = CountVectorizer()
</code><pre class="lang-py prettyprint-override"><code> </code></pre><pre class="lang-py prettyprint-override"><code>#CREATE TRANSFORMER</code></pre><pre class="lang-py prettyprint-override"><code>from sklearn.preprocessing import FunctionTransformer
</code></pre><pre class="lang-py prettyprint-override"><code>one_dim = FunctionTransformer(np.reshape, kw_args={'newshape':-1})

# INCLUDE TRANSFORMER IN PIPELINE
</code></pre><pre class="lang-py prettyprint-override"><code>from sklearn.pipeline import make_pipeline
</code></pre><pre class="lang-py prettyprint-override"><code>pipe = make_pipeline(imp, one_dim, vect)

pipe.fit_transform(df[['text']]).toarray()
</code></pre>
It has been <a href="https://github.com/scikit-learn/scikit-learn/issues/16972" rel="noreferrer">proposed on GitHub</a> that <code>CountVectorizer</code> should allow 2D input as long as the second dimension is 1 (meaning: a single column of data). That modification to <code>CountVectorizer</code> would be a great solution to this problem!
 Solution 2:
One solution would be to create a class off SimpleImputer and override its <code>transform()</code> method:
<pre><code>import pandas as pd
</code></pre><pre><code>import numpy as np
</code></pre><pre><code>from sklearn.impute import SimpleImputer

class ModifiedSimpleImputer(SimpleImputer): </code></pre><pre><code>def transform(self, X):</code></pre><pre><code>
 return super().transform(X).flatten()


df = pd.DataFrame({'text':['abc def', 'abc ghi', np.nan]})

imp = ModifiedSimpleImputer(strategy='constant')

from sklearn.feature_extraction.text import CountVectorizer
</code></pre><pre><code>vect = CountVectorizer()

from sklearn.pipeline import make_pipeline
</code></pre><pre><code>pipe = make_pipeline(imp, vect)

pipe.fit_transform(df[['text']]).toarray()
</code></pre>
 Solution 3:
I use this one dimensional wrapper for sklearn Transformer when I have one dimensional data. I think, this wrapper can be used to wrap the simpleImputer for the one dimensional data (a pandas series with string values) in your case.
<pre><code>class OneDWrapper:
</code></pre><pre><code> """One dimensional wrapper for sklearn Transformers"""

 def __init__(self, transformer):
 </code></pre><pre><code> self.transformer = transformer

 def fit(self, X, y=None):
 </code></pre><pre><code>self.transformer.fit(np.array(X).reshape(-1, 1))
 </code></pre><pre><code> return self

 def transform(self, X, y=None):
</code></pre><pre><code> return self.transformer.transform(
</code></pre><pre><code> np.array(X).reshape(-1, 1)).ravel()

 def inverse_transform(self, X, y=None):
</code></pre><pre><code>return self.transformer.inverse_transform(
</code></pre><pre><code> np.expand_dims(X, axis=1)).ravel()
</code></pre>
Now, you don't need an additional step in the pipeline.
<pre><code>one_d_imputer = OneDWrapper(SimpleImputer(strategy='constant'))
</code></pre><pre><code>pipe = make_pipeline(one_d_imputer, vect)
</code></pre><pre><code>pipe.fit_transform(df['text']).toarray() 
</code></pre><pre><code># note we are feeding a pd.Series here!

</code></pre>
 Thank you for reading the article.

This tutorial is about how to include SimpleImputer before CountVectorizer in a scikit-learn Pipel.

How to include SimpleImputer before CountVectorizer in a scikit-learn Pipeline?

How to load a keras model saved as .pb

How to train new classes on pretrained yolov4 model in darknet