eaglewarrior
/ clean_text.py
Last active
March 5, 2021 15:32
Cleaning the text and make it word cloud ready
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import re | |
| import unicodedata | |
| from nltk.corpus import stopwords | |
| import nltk | |
| def removetitle(text): | |
| return re.sub(r'.*:', '', text) | |
| def removebrackets(text): | |
| return re.sub('[\(\[].*?[\)\]]', ' ', text) |
eaglewarrior
/ AI_news_article_scraping.py
Created
March 5, 2021 15:21
All together the code for scraping articles
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ### Importing all packages | |
| import requests | |
| import urllib.request | |
| import time | |
| import spacy | |
| from bs4 import BeautifulSoup | |
| from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator | |
| import matplotlib.pyplot as plt | |
| ### Making query for searching article |