Format: application/octet-stream
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "---\n",
8 "\n",
9 "_You are currently looking at **version 1.1** of this notebook. To download notebooks and datafiles, as well as get help on Jupyter notebooks in the Coursera platform, visit the [Jupyter Notebook FAQ](https://www.coursera.org/learn/python-text-mining/resources/d9pwm) course resource._\n",
10 "\n",
11 "---"
12 ]
13 },
14 {
15 "cell_type": "markdown",
16 "metadata": {},
17 "source": [
18 "# Assignment 3\n",
19 "\n",
20 "In this assignment you will explore text message data and create models to predict if a message is spam or not. "
21 ]
22 },
23 {
24 "cell_type": "code",
25 "execution_count": 1,
26 "metadata": {},
27 "outputs": [
28 {
29 "data": {
30 "text/html": [
31 "<div>\n",
32 "<style>\n",
33 " .dataframe thead tr:only-child th {\n",
34 " text-align: right;\n",
35 " }\n",
36 "\n",
37 " .dataframe thead th {\n",
38 " text-align: left;\n",
39 " }\n",
40 "\n",
41 " .dataframe tbody tr th {\n",
42 " vertical-align: top;\n",
43 " }\n",
44 "</style>\n",
45 "<table border=\"1\" class=\"dataframe\">\n",
46 " <thead>\n",
47 " <tr style=\"text-align: right;\">\n",
48 " <th></th>\n",
49 " <th>text</th>\n",
50 " <th>target</th>\n",
51 " </tr>\n",
52 " </thead>\n",
53 " <tbody>\n",
54 " <tr>\n",
55 " <th>0</th>\n",
56 " <td>Go until jurong point, crazy.. Available only ...</td>\n",
57 " <td>0</td>\n",
58 " </tr>\n",
59 " <tr>\n",
60 " <th>1</th>\n",
61 " <td>Ok lar... Joking wif u oni...</td>\n",
62 " <td>0</td>\n",
63 " </tr>\n",
64 " <tr>\n",
65 " <th>2</th>\n",
66 " <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
67 " <td>1</td>\n",
68 " </tr>\n",
69 " <tr>\n",
70 " <th>3</th>\n",
71 " <td>U dun say so early hor... U c already then say...</td>\n",
72 " <td>0</td>\n",
73 " </tr>\n",
74 " <tr>\n",
75 " <th>4</th>\n",
76 " <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
77 " <td>0</td>\n",
78 " </tr>\n",
79 " <tr>\n",
80 " <th>5</th>\n",
81 " <td>FreeMsg Hey there darling it's been 3 week's n...</td>\n",
82 " <td>1</td>\n",
83 " </tr>\n",
84 " <tr>\n",
85 " <th>6</th>\n",
86 " <td>Even my brother is not like to speak with me. ...</td>\n",
87 " <td>0</td>\n",
88 " </tr>\n",
89 " <tr>\n",
90 " <th>7</th>\n",
91 " <td>As per your request 'Melle Melle (Oru Minnamin...</td>\n",
92 " <td>0</td>\n",
93 " </tr>\n",
94 " <tr>\n",
95 " <th>8</th>\n",
96 " <td>WINNER!! As a valued network customer you have...</td>\n",
97 " <td>1</td>\n",
98 " </tr>\n",
99 " <tr>\n",
100 " <th>9</th>\n",
101 " <td>Had your mobile 11 months or more? U R entitle...</td>\n",
102 " <td>1</td>\n",
103 " </tr>\n",
104 " </tbody>\n",
105 "</table>\n",
106 "</div>"
107 ],
108 "text/plain": [
109 " text target\n",
110 "0 Go until jurong point, crazy.. Available only ... 0\n",
111 "1 Ok lar... Joking wif u oni... 0\n",
112 "2 Free entry in 2 a wkly comp to win FA Cup fina... 1\n",
113 "3 U dun say so early hor... U c already then say... 0\n",
114 "4 Nah I don't think he goes to usf, he lives aro... 0\n",
115 "5 FreeMsg Hey there darling it's been 3 week's n... 1\n",
116 "6 Even my brother is not like to speak with me. ... 0\n",
117 "7 As per your request 'Melle Melle (Oru Minnamin... 0\n",
118 "8 WINNER!! As a valued network customer you have... 1\n",
119 "9 Had your mobile 11 months or more? U R entitle... 1"
120 ]
121 },
122 "execution_count": 1,
123 "metadata": {},
124 "output_type": "execute_result"
125 }
126 ],
127 "source": [
128 "import pandas as pd\n",
129 "import numpy as np\n",
130 "\n",
131 "spam_data = pd.read_csv('spam.csv')\n",
132 "\n",
133 "spam_data['target'] = np.where(spam_data['target']=='spam',1,0)\n",
134 "spam_data.head(10)"
135 ]
136 },
137 {
138 "cell_type": "code",
139 "execution_count": 2,
140 "metadata": {
141 "collapsed": true
142 },
143 "outputs": [],
144 "source": [
145 "from sklearn.model_selection import train_test_split\n",
146 "\n",
147 "\n",
148 "X_train, X_test, y_train, y_test = train_test_split(spam_data['text'], \n",
149 " spam_data['target'], \n",
150 " random_state=0)"
151 ]
152 },
153 {
154 "cell_type": "markdown",
155 "metadata": {},
156 "source": [
157 "### Question 1\n",
158 "What percentage of the documents in `spam_data` are spam?\n",
159 "\n",
160 "*This function should return a float, the percent value (i.e. $ratio * 100$).*"
161 ]
162 },
163 {
164 "cell_type": "code",
165 "execution_count": 3,
166 "metadata": {
167 "collapsed": true
168 },
169 "outputs": [],
170 "source": [
171 "def answer_one():\n",
172 " \n",
173 " return len(spam_data[spam_data['target']==1])/len(spam_data['target'])*100"
174 ]
175 },
176 {
177 "cell_type": "code",
178 "execution_count": 4,
179 "metadata": {},
180 "outputs": [
181 {
182 "data": {
183 "text/plain": [
184 "13.406317300789663"
185 ]
186 },
187 "execution_count": 4,
188 "metadata": {},
189 "output_type": "execute_result"
190 }
191 ],
192 "source": [
193 "answer_one()"
194 ]
195 },
196 {
197 "cell_type": "markdown",
198 "metadata": {},
199 "source": [
200 "### Question 2\n",
201 "\n",
202 "Fit the training data `X_train` using a Count Vectorizer with default parameters.\n",
203 "\n",
204 "What is the longest token in the vocabulary?\n",
205 "\n",
206 "*This function should return a string.*"
207 ]
208 },
209 {
210 "cell_type": "code",
211 "execution_count": 5,
212 "metadata": {
213 "collapsed": true
214 },
215 "outputs": [],
216 "source": [
217 "from sklearn.feature_extraction.text import CountVectorizer\n",
218 "\n",
219 "def answer_two():\n",
220 " \n",
221 " # List all tokens and their counts as a dictionary:\n",
222 " vocabulary = CountVectorizer().fit(X_train).vocabulary_\n",
223 " \n",
224 " # You want only the keys, i.e, the words:\n",
225 " vocabulary = [x for x in vocabulary.keys()]\n",
226 " \n",
227 " # Store the lengths in a separate list:\n",
228 " len_vocabulary = [len(x) for x in vocabulary]\n",
229 " \n",
230 " # Use the index of the longest token:\n",
231 " return vocabulary[np.argmax(len_vocabulary)]\n"
232 ]
233 },
234 {
235 "cell_type": "code",
236 "execution_count": 6,
237 "metadata": {},
238 "outputs": [
239 {
240 "data": {
241 "text/plain": [
242 "'com1win150ppmx3age16subscription'"
243 ]
244 },
245 "execution_count": 6,
246 "metadata": {},
247 "output_type": "execute_result"
248 }
249 ],
250 "source": [
251 "answer_two()"
252 ]
253 },
254 {
255 "cell_type": "markdown",
256 "metadata": {},
257 "source": [
258 "### Question 3\n",
259 "\n",
260 "Fit and transform the training data `X_train` using a Count Vectorizer with default parameters.\n",
261 "\n",
262 "Next, fit a fit a multinomial Naive Bayes classifier model with smoothing `alpha=0.1`. Find the area under the curve (AUC) score using the transformed test data.\n",
263 "\n",
264 "*This function should return the AUC score as a float.*"
265 ]
266 },
267 {
268 "cell_type": "code",
269 "execution_count": 7,
270 "metadata": {
271 "collapsed": true
272 },
273 "outputs": [],
274 "source": [
275 "from sklearn.naive_bayes import MultinomialNB\n",
276 "from sklearn.metrics import roc_auc_score\n",
277 "\n",
278 "def answer_three():\n",
279 " \n",
280 " cv = CountVectorizer().fit(X_train)\n",
281 " \n",
282 " # Transform both X_train and X_test with the same CV object:\n",
283 " X_train_cv = cv.transform(X_train)\n",
284 " X_test_cv = cv.transform(X_test)\n",
285 " \n",
286 " # Classifier for prediction:\n",
287 " clf = MultinomialNB(alpha=0.1)\n",
288 " clf.fit(X_train_cv, y_train)\n",
289 " preds_test = clf.predict(X_test_cv)\n",
290 " \n",
291 " return roc_auc_score(y_test, preds_test)\n"
292 ]
293 },
294 {
295 "cell_type": "code",
296 "execution_count": 8,
297 "metadata": {},
298 "outputs": [
299 {
300 "data": {
301 "text/plain": [
302 "0.97208121827411165"
303 ]
304 },
305 "execution_count": 8,
306 "metadata": {},
307 "output_type": "execute_result"
308 }
309 ],
310 "source": [
311 "answer_three()"
312 ]
313 },
314 {
315 "cell_type": "markdown",
316 "metadata": {},
317 "source": [
318 "### Question 4\n",
319 "\n",
320 "Fit and transform the training data `X_train` using a Tfidf Vectorizer with default parameters.\n",
321 "\n",
322 "What 20 features have the smallest tf-idf and what 20 have the largest tf-idf?\n",
323 "\n",
324 "Put these features in a two series where each series is sorted by tf-idf value and then alphabetically by feature name. The index of the series should be the feature name, and the data should be the tf-idf.\n",
325 "\n",
326 "The series of 20 features with smallest tf-idfs should be sorted smallest tfidf first, the list of 20 features with largest tf-idfs should be sorted largest first. \n",
327 "\n",
328 "*This function should return a tuple of two series\n",
329 "`(smallest tf-idfs series, largest tf-idfs series)`.*"
330 ]
331 },
332 {
333 "cell_type": "code",
334 "execution_count": 9,
335 "metadata": {
336 "collapsed": true
337 },
338 "outputs": [],
339 "source": [
340 "from sklearn.feature_extraction.text import TfidfVectorizer\n",
341 "\n",
342 "def answer_four():\n",
343 " \n",
344 " tfidf = TfidfVectorizer().fit(X_train)\n",
345 " feature_names = np.array(tfidf.get_feature_names())\n",
346 " \n",
347 " X_train_tf = tfidf.transform(X_train)\n",
348 " \n",
349 " max_tf_idfs = X_train_tf.max(0).toarray()[0] # Get largest tfidf values across all documents.\n",
350 " sorted_tf_idxs = max_tf_idfs.argsort() # Sorted indices\n",
351 " sorted_tf_idfs = max_tf_idfs[sorted_tf_idxs] # Sorted TFIDF values\n",
352 " \n",
353 " # feature_names doesn't need to be sorted! You just access it with a list of sorted indices!\n",
354 " smallest_tf_idfs = pd.Series(sorted_tf_idfs[:20], index=feature_names[sorted_tf_idxs[:20]]) \n",
355 " largest_tf_idfs = pd.Series(sorted_tf_idfs[-20:][::-1], index=feature_names[sorted_tf_idxs[-20:][::-1]])\n",
356 " \n",
357 " return (smallest_tf_idfs, largest_tf_idfs)"
358 ]
359 },
360 {
361 "cell_type": "code",
362 "execution_count": 10,
363 "metadata": {},
364 "outputs": [
365 {
366 "data": {
367 "text/plain": [
368 "(sympathetic 0.074475\n",
369 " healer 0.074475\n",
370 " aaniye 0.074475\n",
371 " dependable 0.074475\n",
372 " companion 0.074475\n",
373 " listener 0.074475\n",
374 " athletic 0.074475\n",
375 " exterminator 0.074475\n",
376 " psychiatrist 0.074475\n",
377 " pest 0.074475\n",
378 " determined 0.074475\n",
379 " chef 0.074475\n",
380 " courageous 0.074475\n",
381 " stylist 0.074475\n",
382 " psychologist 0.074475\n",
383 " organizer 0.074475\n",
384 " pudunga 0.074475\n",
385 " venaam 0.074475\n",
386 " diwali 0.091250\n",
387 " mornings 0.091250\n",
388 " dtype: float64, 146tf150p 1.000000\n",
389 " havent 1.000000\n",
390 " home 1.000000\n",
391 " okie 1.000000\n",
392 " thanx 1.000000\n",
393 " er 1.000000\n",
394 " anything 1.000000\n",
395 " lei 1.000000\n",
396 " nite 1.000000\n",
397 " yup 1.000000\n",
398 " thank 1.000000\n",
399 " ok 1.000000\n",
400 " where 1.000000\n",
401 " beerage 1.000000\n",
402 " anytime 1.000000\n",
403 " too 1.000000\n",
404 " done 1.000000\n",
405 " 645 1.000000\n",
406 " tick 0.980166\n",
407 " blank 0.932702\n",
408 " dtype: float64)"
409 ]
410 },
411 "execution_count": 10,
412 "metadata": {},
413 "output_type": "execute_result"
414 }
415 ],
416 "source": [
417 "answer_four()"
418 ]
419 },
420 {
421 "cell_type": "markdown",
422 "metadata": {},
423 "source": [
424 "### Question 5\n",
425 "\n",
426 "Fit and transform the training data `X_train` using a Tfidf Vectorizer ignoring terms that have a document frequency strictly lower than **3**.\n",
427 "\n",
428 "Then fit a multinomial Naive Bayes classifier model with smoothing `alpha=0.1` and compute the area under the curve (AUC) score using the transformed test data.\n",
429 "\n",
430 "*This function should return the AUC score as a float.*"
431 ]
432 },
433 {
434 "cell_type": "code",
435 "execution_count": 11,
436 "metadata": {
437 "collapsed": true
438 },
439 "outputs": [],
440 "source": [
441 "def answer_five():\n",
442 " \n",
443 " tf = TfidfVectorizer(min_df=3).fit(X_train)\n",
444 " X_train_tf = tf.transform(X_train)\n",
445 " X_test_tf = tf.transform(X_test)\n",
446 " clf = MultinomialNB(alpha=0.1)\n",
447 " clf.fit(X_train_tf, y_train)\n",
448 " pred = clf.predict(X_test_tf)\n",
449 " return roc_auc_score(y_test, pred)"
450 ]
451 },
452 {
453 "cell_type": "code",
454 "execution_count": 12,
455 "metadata": {},
456 "outputs": [
457 {
458 "data": {
459 "text/plain": [
460 "0.94162436548223349"
461 ]
462 },
463 "execution_count": 12,
464 "metadata": {},
465 "output_type": "execute_result"
466 }
467 ],
468 "source": [
469 "answer_five()"
470 ]
471 },
472 {
473 "cell_type": "markdown",
474 "metadata": {},
475 "source": [
476 "### Question 6\n",
477 "\n",
478 "What is the average length of documents (number of characters) for not spam and spam documents?\n",
479 "\n",
480 "*This function should return a tuple (average length not spam, average length spam).*"
481 ]
482 },
483 {
484 "cell_type": "code",
485 "execution_count": 13,
486 "metadata": {
487 "collapsed": true
488 },
489 "outputs": [],
490 "source": [
491 "def answer_six():\n",
492 " \n",
493 " len_spam = [len(x) for x in spam_data.loc[spam_data['target']==1, 'text']]\n",
494 " len_not_spam = [len(x) for x in spam_data.loc[spam_data['target']==0, 'text']]\n",
495 " return (np.mean(len_not_spam), np.mean(len_spam))"
496 ]
497 },
498 {
499 "cell_type": "code",
500 "execution_count": 14,
501 "metadata": {},
502 "outputs": [
503 {
504 "data": {
505 "text/plain": [
506 "(71.023626943005183, 138.8661311914324)"
507 ]
508 },
509 "execution_count": 14,
510 "metadata": {},
511 "output_type": "execute_result"
512 }
513 ],
514 "source": [
515 "answer_six()"
516 ]
517 },
518 {
519 "cell_type": "markdown",
520 "metadata": {},
521 "source": [
522 "<br>\n",
523 "<br>\n",
524 "The following function has been provided to help you combine new features into the training data:"
525 ]
526 },
527 {
528 "cell_type": "code",
529 "execution_count": 15,
530 "metadata": {
531 "collapsed": true
532 },
533 "outputs": [],
534 "source": [
535 "def add_feature(X, feature_to_add):\n",
536 " \"\"\"\n",
537 " Returns sparse feature matrix with added feature.\n",
538 " feature_to_add can also be a list of features.\n",
539 " \"\"\"\n",
540 " from scipy.sparse import csr_matrix, hstack\n",
541 " return hstack([X, csr_matrix(feature_to_add).T], 'csr')"
542 ]
543 },
544 {
545 "cell_type": "markdown",
546 "metadata": {},
547 "source": [
548 "### Question 7\n",
549 "\n",
550 "Fit and transform the training data X_train using a Tfidf Vectorizer ignoring terms that have a document frequency strictly lower than **5**.\n",
551 "\n",
552 "Using this document-term matrix and an additional feature, **the length of document (number of characters)**, fit a Support Vector Classification model with regularization `C=10000`. Then compute the area under the curve (AUC) score using the transformed test data.\n",
553 "\n",
554 "*This function should return the AUC score as a float.*"
555 ]
556 },
557 {
558 "cell_type": "code",
559 "execution_count": 16,
560 "metadata": {
561 "collapsed": true
562 },
563 "outputs": [],
564 "source": [
565 "from sklearn.svm import SVC\n",
566 "\n",
567 "def answer_seven():\n",
568 " \n",
569 " len_train = [len(x) for x in X_train]\n",
570 " len_test = [len(x) for x in X_test]\n",
571 " \n",
572 " tf = TfidfVectorizer(min_df=5).fit(X_train)\n",
573 " X_train_tf = tf.transform(X_train)\n",
574 " X_test_tf = tf.transform(X_test)\n",
575 " \n",
576 " X_train_tf = add_feature(X_train_tf, len_train)\n",
577 " X_test_tf = add_feature(X_test_tf, len_test)\n",
578 " \n",
579 " clf = SVC(C=10000)\n",
580 " clf.fit(X_train_tf, y_train)\n",
581 " pred = clf.predict(X_test_tf)\n",
582 " \n",
583 " return roc_auc_score(y_test, pred)"
584 ]
585 },
586 {
587 "cell_type": "code",
588 "execution_count": 17,
589 "metadata": {},
590 "outputs": [
591 {
592 "data": {
593 "text/plain": [
594 "0.95813668234215565"
595 ]
596 },
597 "execution_count": 17,
598 "metadata": {},
599 "output_type": "execute_result"
600 }
601 ],
602 "source": [
603 "answer_seven()"
604 ]
605 },
606 {
607 "cell_type": "markdown",
608 "metadata": {},
609 "source": [
610 "### Question 8\n",
611 "\n",
612 "What is the average number of digits per document for not spam and spam documents?\n",
613 "\n",
614 "*This function should return a tuple (average # digits not spam, average # digits spam).*"
615 ]
616 },
617 {
618 "cell_type": "code",
619 "execution_count": 18,
620 "metadata": {
621 "collapsed": true
622 },
623 "outputs": [],
624 "source": [
625 "def answer_eight():\n",
626 " \n",
627 " dig_spam = [sum(char.isnumeric() for char in x) for x in spam_data.loc[spam_data['target']==1,'text']]\n",
628 " dig_not_spam = [sum(char.isnumeric() for char in x) for x in spam_data.loc[spam_data['target']==0,'text']]\n",
629 " return (np.mean(dig_not_spam), np.mean(dig_spam))"
630 ]
631 },
632 {
633 "cell_type": "code",
634 "execution_count": 19,
635 "metadata": {},
636 "outputs": [
637 {
638 "data": {
639 "text/plain": [
640 "(0.29927461139896372, 15.76037483266399)"
641 ]
642 },
643 "execution_count": 19,
644 "metadata": {},
645 "output_type": "execute_result"
646 }
647 ],
648 "source": [
649 "answer_eight()"
650 ]
651 },
652 {
653 "cell_type": "markdown",
654 "metadata": {},
655 "source": [
656 "### Question 9\n",
657 "\n",
658 "Fit and transform the training data `X_train` using a Tfidf Vectorizer ignoring terms that have a document frequency strictly lower than **5** and using **word n-grams from n=1 to n=3** (unigrams, bigrams, and trigrams).\n",
659 "\n",
660 "Using this document-term matrix and the following additional features:\n",
661 "* the length of document (number of characters)\n",
662 "* **number of digits per document**\n",
663 "\n",
664 "fit a Logistic Regression model with regularization `C=100`. Then compute the area under the curve (AUC) score using the transformed test data.\n",
665 "\n",
666 "*This function should return the AUC score as a float.*"
667 ]
668 },
669 {
670 "cell_type": "code",
671 "execution_count": 20,
672 "metadata": {
673 "collapsed": true
674 },
675 "outputs": [],
676 "source": [
677 "from sklearn.linear_model import LogisticRegression\n",
678 "\n",
679 "def answer_nine():\n",
680 " \n",
681 " dig_train = [sum(char.isnumeric() for char in x) for x in X_train]\n",
682 " dig_test = [sum(char.isnumeric() for char in x) for x in X_test]\n",
683 " \n",
684 " tf = TfidfVectorizer(min_df = 5, ngram_range = (1,3)).fit(X_train)\n",
685 " X_train_tf = tf.transform(X_train)\n",
686 " X_test_tf = tf.transform(X_test)\n",
687 " \n",
688 " X_train_tf = add_feature(X_train_tf, dig_train)\n",
689 " X_test_tf = add_feature(X_test_tf, dig_test)\n",
690 " \n",
691 " clf = LogisticRegression(C=100).fit(X_train_tf, y_train)\n",
692 " pred = clf.predict(X_test_tf)\n",
693 " \n",
694 " return roc_auc_score(y_test, pred)"
695 ]
696 },
697 {
698 "cell_type": "code",
699 "execution_count": 21,
700 "metadata": {},
701 "outputs": [
702 {
703 "data": {
704 "text/plain": [
705 "0.96787090640544626"
706 ]
707 },
708 "execution_count": 21,
709 "metadata": {},
710 "output_type": "execute_result"
711 }
712 ],
713 "source": [
714 "answer_nine()"
715 ]
716 },
717 {
718 "cell_type": "markdown",
719 "metadata": {},
720 "source": [
721 "### Question 10\n",
722 "\n",
723 "What is the average number of non-word characters (anything other than a letter, digit or underscore) per document for not spam and spam documents?\n",
724 "\n",
725 "*Hint: Use `\\w` and `\\W` character classes*\n",
726 "\n",
727 "*This function should return a tuple (average # non-word characters not spam, average # non-word characters spam).*"
728 ]
729 },
730 {
731 "cell_type": "code",
732 "execution_count": 22,
733 "metadata": {
734 "collapsed": true
735 },
736 "outputs": [],
737 "source": [
738 "def answer_ten():\n",
739 " \n",
740 " return (np.mean(spam_data.loc[spam_data['target']==0,'text'].str.count('\\W')), \n",
741 " np.mean(spam_data.loc[spam_data['target']==1,'text'].str.count('\\W')))"
742 ]
743 },
744 {
745 "cell_type": "code",
746 "execution_count": 23,
747 "metadata": {},
748 "outputs": [
749 {
750 "data": {
751 "text/plain": [
752 "(17.291813471502589, 29.041499330655956)"
753 ]
754 },
755 "execution_count": 23,
756 "metadata": {},
757 "output_type": "execute_result"
758 }
759 ],
760 "source": [
761 "answer_ten()"
762 ]
763 },
764 {
765 "cell_type": "markdown",
766 "metadata": {},
767 "source": [
768 "### Question 11\n",
769 "\n",
770 "Fit and transform the training data X_train using a Count Vectorizer ignoring terms that have a document frequency strictly lower than **5** and using **character n-grams from n=2 to n=5.**\n",
771 "\n",
772 "To tell Count Vectorizer to use character n-grams pass in `analyzer='char_wb'` which creates character n-grams only from text inside word boundaries. This should make the model more robust to spelling mistakes.\n",
773 "\n",
774 "Using this document-term matrix and the following additional features:\n",
775 "* the length of document (number of characters)\n",
776 "* number of digits per document\n",
777 "* **number of non-word characters (anything other than a letter, digit or underscore.)**\n",
778 "\n",
779 "fit a Logistic Regression model with regularization C=100. Then compute the area under the curve (AUC) score using the transformed test data.\n",
780 "\n",
781 "Also **find the 10 smallest and 10 largest coefficients from the model** and return them along with the AUC score in a tuple.\n",
782 "\n",
783 "The list of 10 smallest coefficients should be sorted smallest first, the list of 10 largest coefficients should be sorted largest first.\n",
784 "\n",
785 "The three features that were added to the document term matrix should have the following names should they appear in the list of coefficients:\n",
786 "['length_of_doc', 'digit_count', 'non_word_char_count']\n",
787 "\n",
788 "*This function should return a tuple `(AUC score as a float, smallest coefs list, largest coefs list)`.*"
789 ]
790 },
791 {
792 "cell_type": "code",
793 "execution_count": 26,
794 "metadata": {
795 "collapsed": true
796 },
797 "outputs": [],
798 "source": [
799 "def answer_eleven():\n",
800 " \n",
801 " len_train = [len(x) for x in X_train]\n",
802 " len_test = [len(x) for x in X_test]\n",
803 " dig_train = [sum(char.isnumeric() for char in x) for x in X_train]\n",
804 " dig_test = [sum(char.isnumeric() for char in x) for x in X_test]\n",
805 " \n",
806 " # Not alpha numeric:\n",
807 " nan_train = X_train.str.count('\\W')\n",
808 " nan_test = X_test.str.count('\\W')\n",
809 " \n",
810 " cv = CountVectorizer(min_df = 5, ngram_range=(2,5), analyzer='char_wb').fit(X_train)\n",
811 " X_train_cv = cv.transform(X_train)\n",
812 " X_test_cv = cv.transform(X_test)\n",
813 " \n",
814 " X_train_cv = add_feature(X_train_cv, [len_train, dig_train, nan_train])\n",
815 " X_test_cv = add_feature(X_test_cv, [len_test, dig_test, nan_test])\n",
816 " \n",
817 " clf = LogisticRegression(C=100).fit(X_train_cv, y_train)\n",
818 " pred = clf.predict(X_test_cv)\n",
819 " \n",
820 " score = roc_auc_score(y_test, pred)\n",
821 " \n",
822 " feature_names = np.array(cv.get_feature_names() + ['length_of_doc', 'digit_count', 'non_word_char_count'])\n",
823 " sorted_coef_index = clf.coef_[0].argsort()\n",
824 " small_coeffs = list(feature_names[sorted_coef_index[:10]])\n",
825 " large_coeffs = list(feature_names[sorted_coef_index[:-11:-1]])\n",
826 " \n",
827 " return (score, small_coeffs, large_coeffs)"
828 ]
829 },
830 {
831 "cell_type": "code",
832 "execution_count": 27,
833 "metadata": {},
834 "outputs": [
835 {
836 "data": {
837 "text/plain": [
838 "(0.97885931107074342,\n",
839 " ['. ', '..', '? ', ' i', ' y', ' go', ':)', ' h', 'go', ' m'],\n",
840 " ['digit_count', 'ne', 'ia', 'co', 'xt', ' ch', 'mob', ' x', 'ww', 'ar'])"
841 ]
842 },
843 "execution_count": 27,
844 "metadata": {},
845 "output_type": "execute_result"
846 }
847 ],
848 "source": [
849 "answer_eleven()"
850 ]
851 },
852 {
853 "cell_type": "code",
854 "execution_count": null,
855 "metadata": {
856 "collapsed": true
857 },
858 "outputs": [],
859 "source": []
860 }
861 ],
862 "metadata": {
863 "coursera": {
864 "course_slug": "python-text-mining",
865 "graded_item_id": "Pn19K",
866 "launcher_item_id": "y1juS",
867 "part_id": "ctlgo"
868 },
869 "kernelspec": {
870 "display_name": "Python 3",
871 "language": "python",
872 "name": "python3"
873 },
874 "language_info": {
875 "codemirror_mode": {
876 "name": "ipython",
877 "version": 3
878 },
879 "file_extension": ".py",
880 "mimetype": "text/x-python",
881 "name": "python",
882 "nbconvert_exporter": "python",
883 "pygments_lexer": "ipython3",
884 "version": "3.6.2"
885 }
886 },
887 "nbformat": 4,
888 "nbformat_minor": 2
889 }
890