Format: application/x-python-code
1 U
2
3 ���`. � @ s� d dl Zd dlZd dlmZmZmZ d dlm Z d dl
4 Z
5 e
6 � d�Z e �
7 d�Z e �dd� e D �� e �dd d
8 d d d
9 ddddddddddddg� G dd� d�ZdS ) � N)�CountVectorizer�TfidfTransformer�TfidfVectorizer)� stopwordsZen_core_web_sm�englishc C s g | ] }|� � �qS � )�
10 capitalize)�.0�sr r �)C:\Users\PC\Pytho\Projects\NLP\Process.py�
11 <listcomp> s r �,�`�?z'sz''z``zn't�(�)�;�'�re�!z--�-�*z're�:c @ s0 e Zd Zd
12 dd�Zdddddd gefd
13 d �Zd S )�Process� c C s
14 || _ d S )N)�m)�selfr r r r �__init__ s zProcess.__init__ZNNSZNNZADJZJJZJJRZVBc s* t | �}� fdd�tjD �}d�|�} | S )Nc s g | ]}|j � kr|j�qS r )Ztag_Zlemma_)r �token�� allowed_posr r r s
15 z'Process.pre_process.<locals>.<listcomp>� )�nlpr �words�join)�textr r r# �new_textr r r � pre_process s
16 zProcess.pre_processN)r )�__name__�
17 __module__� __qualname__r r r' r r r r r s
18 r )�pandas�pd�numpy�npZ sklearn.feature_extraction.textr r r Z nltk.corpusr Zspacy�loadr" r# �extendr r r r r �<module> s
19
20 .