Yunfeng Xi
Reputation: 1
TfidfVectorizer does not use the whole set of words in all documents?
I am trying to build a TFIDF model with TfidfVectorizer. The feature name list namely the number of column of sparse matrix is shorter than the length of word set of documents even though I set min_df as 1. What happened?
Upvotes: 0
Views: 651
Answers (1)
jtitusj
Reputation: 3086
Did you check the stop_words and max_features? If you provide values in either of these two, it will exclude some words.
Upvotes: 1
Related Questions
- TfidfVectorizer seems to be giving incorrect results
- Python TfidfVectorizer throwing : empty vocabulary; perhaps the documents only contain stop words"
- How to get TF-IDF value of a word from all set of documents?
- Understanding TfidfVectorizer output
- TfidfVectorizer gives high weight to stop words
- scikit-learn TfidfVectorizer ignoring certain words
- Get full text from TfidfVectorizer
- tfidfvectorizer prints results based on all words
- Python tfidf returning same values regardless of idf
- TfidfVectorizer stop_words parameter does not work?