Practice Project - Sentiment Analysis
Watson AI
Embedded Files
Guidelines
Guidelines
In the practice project, you will be introduced to embeddable Watson AI libraries. Then, you will create a web app integrated with Watson AI libraries to perform sentiment analysis on the provided text.
After developing the app, you will deploy the said application over the web using Flask framework. All activities that get you to deploy a functional AI-based web apps are broken down into tasks as listed below:
Task 1: Clone the project repository
Task 2: Create a sentiment analysis application using Watson NLP library
Task 3: Format the output of the application
Task 4: Package the application
Task 5: Run Unit tests on your application
Task 6: Deploy as web application using Flask
Task 7: Incorporate error handling
Task 8: Run static code analysis
Methods
Methods
Embeddable Watson AI libraries
Embeddable Watson AI libraries
In this project, you'll be using embeddable libraries to create an AI powered Python application.
Embeddable Watson AI libraries include the NLP library, the text-to-speech library and the speech-to-text library. These libraries can be embedded and distributed as part of your application. For your convenience, these libraries have been pre-installed on Skills Network Labs Cloud IDE for use in this project.
The NLP library includes functions for sentiment analysis, emotion detection, text classification, language detection, etc. among others.
The speech-to-text library contains functions that perform the transcription service and generates written text from spoken audio.
The text-to-speech library generates natural sounding audio from written text.
All available functions, in each of these libraries, calls pretrained AI models that are all available on the Cloud IDE servers, available to all users for free.
Files
Files
Task 1: Clone the project repository
Task 1: Clone the project repository
git clone https://github.com/ibm-developer-skills-network/zzrjt-practice-project-emb-ai.git practice_project
cd practice_project
Task 2: Create a sentiment analysis application using Watson NLP library
Task 2: Create a sentiment analysis application using Watson NLP library
NLP sentiment analysis is the practice of using computers to recognize sentiment or emotion expressed in a text. Through NLP, sentiment analysis categorizes words as positive, negative or neutral.
Sentiment analysis is often performed on textual data to help businesses monitor brand and product sentiment in customer feedback, and understanding customer needs. It helps attain the attitude and mood of the wider public which can then help gather insightful information about the context.
For creating the sentiment analysis application, we'll be making use of the Watson Embedded AI Libraries. Since the functions of these libraries are already deployed on the Cloud IDE server, there is no need of importing these libraries to our code. Instead, we need to send a POST request to the relevant model with the required text and the model will send the appropriate response.
import requests
def <function_name>(<input_args>):
url = '<relevant_url>'
headers = {<header_dictionary>}
myobj = {<input_dictionary_to_the_function>}
response = requests.post(url, json = myobj, headers=header)
return response.text
For this project, you'll be using the BERT based Sentiment Analysis function of the Watson NLP Library. For accessing this funciton, the URL, the headers and the input json format is as follows.
URL: 'https://sn-watson-sentiment-bert.labs.skills.network/v1/watson.runtime.nlp.v1/NlpService/SentimentPredict'
Headers: {"grpc-metadata-mm-model-id": "sentiment_aggregated-bert-workflow_lang_multi_stock"}
Input json: { "raw_document": { "text": text_to_analyse } }
In this task, you need to create a file named sentiment_analysis.py in practice_project folder. In this file, write the function for running sentiment analysis using the Watson NLP BERT Seniment Analysis function, as discussed above.
Let us call this function sentiment_analyzer. Assume that that text to be analysed is passed to the function as an argument and is stored in the variable text_to_analyse.
import requests
def sentiment_analyzer(text_to_analyse):
url = 'https://sn-watson-sentiment-bert.labs.skills.network/v1/watson.runtime.nlp.v1/NlpService/SentimentPredict'
myobj = { "raw_document": { "text": text_to_analyse } }
header = {"grpc-metadata-mm-model-id": "sentiment_aggregated-bert-workflow_lang_multi_stock"}
response = requests.post(url, json = myobj, headers=header)
return response.text
To test:
python3.11
from sentiment_analysis import sentiment_analyzer
response = sentiment_analyzer("I love this new technology")
print(response)
Task 3: Format the output of the application
Task 3: Format the output of the application
The output of the application created is in the form of a dictionary, but has been formatted as a text.
To access relevant pieces of information from this output, we need to first convert this text into a dictionary.
Since dictionaries are the default formatting system for JSON files, we make use of the in-built Python library json.
import requests
import json
def sentiment_analyzer(text_to_analyse):
url = 'https://sn-watson-sentiment-bert.labs.skills.network/v1/watson.runtime.nlp.v1/NlpService/SentimentPredict'
myobj = { "raw_document": { "text": text_to_analyse } }
header = {"grpc-metadata-mm-model-id": "sentiment_aggregated-bert-workflow_lang_multi_stock"}
response = requests.post(url, json = myobj, headers=header)
formatted_response = json.loads(response.text)
label = formatted_response['documentSentiment']['label']
score = formatted_response['documentSentiment']['score']
return {'label': label, 'score': score}
Task 4: Package the application
Task 4: Package the application
Let's keep the name of the package as SentimentAnalysis. The steps involved in packaging are:
Create a folder in the working directory, with the name as the package name.
Put (or move) the application code, also called module, in the package folder.
Create the __init__.py file, referencing the module.
The final folder structure should look as shown in the image below.
mkdir SentimentAnalysis
mv ./sentiment_analysis.py ./SentimentAnalysis
from . import sentiment_analysis
To test, run Python:
from SentimentAnalysis.sentiment_analysis import sentiment_analyzer
sentiment_analyzer("This is fun.")
Task 5: Run Unit tests on your application
Task 5: Run Unit tests on your application
For running unit tests, we need to create a new file that calls the required application function from the package and tests its for a known text and output pair.
For this, complete the following steps.
Create a new file in practice_project folder, called test_sentiment_analysis.py.
In this file, import the sentiment_analyzer function from the SentimentAnalysis package. Also import the unittest library.
from SentimentAnalysis.sentiment_analysis import sentiment_analyzer
import unittest
Create the unit test class. Let's call it TestSentimentAnalyzer. Define test_sentiment_analyzer as the function to run the unit tests.
class TestSentimentAnalyzer(unittest.TestCase):
def test_sentiment_analyzer(self):
Define 3 unit tests in the said function and check for the validity of the following statement - label pairs.
“I love working with Python”: “SENT_POSITIVE”
“I hate working with Pyhton”: “SENT_NEGATIVE”
“I am neutral on Python”: “SENT_NEUTRAL”
class TestSentimentAnalyzer(unittest.TestCase):
def test_sentiment_analyzer(self):
result_1 = sentiment_analyzer('I love working with Python')
self.assertEqual(result_1['label'], 'SENT_POSITIVE')
result_2 = sentiment_analyzer('I hate working with Python')
self.assertEqual(result_2['label'], 'SENT_NEGATIVE')
result_3 = sentiment_analyzer('I am neutral on Python')
self.assertEqual(result_3['label'], 'SENT_NEUTRAL')
Add the following line at the end of the file.
unittest.main()
Task 6: Deploy as web application using Flask
Task 6: Deploy as web application using Flask
To ease the process of deployment, you have been provided with 3 files which are going to be used for this task.
index.html in templates folder: Code for the web interface
mywebscript.js in static folder: This javascript file executes a GET request and takes the text provided by the user as input.
server.py in the practice_project folder.
''' Executing this function initiates the application of sentiment
analysis to be executed over the Flask channel and deployed on
localhost:5000.
'''
# Import Flask, render_template, request from the flask pramework package : TODO
# Import the sentiment_analyzer function from the package created: TODO
from flask import Flask, render_template, request
from SentimentAnalysis.sentiment_analysis import sentiment_analyzer
#Initiate the flask app : TODO
app = Flask("Sentiment Analyzer")
@app.route("/sentimentAnalyzer")
def sent_analyzer():
''' This code receives the text from the HTML interface and
runs sentiment analysis over it using sentiment_analysis()
function. The output returned shows the label and its confidence
score for the provided text.
'''
# TODO
text_to_analyze = request.args.get('textToAnalyze')
response = sentiment_analyzer(text_to_analyze)
label = response['label']
score = response['score']
return "The given text has been identified as {} with a score of {}.".format(label.split('_')[1], score)
@app.route("/")
def render_index_page():
''' This function initiates the rendering of the main application
page over the Flask channel
'''
#TODO
return render_template('index.html')
if __name__ == "__main__":
''' This functions executes the flask app and deploys it on localhost:5000
'''
#TODO
app.run(host = "0.0.0.0", port = 4996)
Task 7: Incorporate Error handling
Task 7: Incorporate Error handling
To incorporate error handling, we need to identify the different forms of error codes that may be received in response to the GET query initiated by the sent_analyzer function in server.py.
In the case of invalid entries, the system responds with 500 error code, indicating that there is something wrong at the server end.
Invalid entry could be anything that the model is not able to interpret. However, in the situation of this error, this application output doesn't get updated.
To fix this bug in our application, we need to study the response received from the Watson AI library function, when the server generates 500 error. To test this, we need to retrace the steps taken in Task 2, and test the Watson AI library with an invalid string input.
Open a python shell in the terminal and run the following commands to check the required output.
import requests
url = "https://sn-watson-sentiment-bert.labs.skills.network/v1/watson.runtime.nlp.v1/NlpService/SentimentPredict"
headers = {"grpc-metadata-mm-model-id": "sentiment_aggregated-bert-workflow_lang_multi_stock"}
myobj = { "raw_document": { "text": "as987da-6s2d aweadsa" } }
response = requests.post(url, json = myobj, headers=headers)
print(response.status_code)
myobj = { "raw_document": { "text": "Testing this application for error handling" } }
response = requests.post(url, json = myobj, headers=headers)
print(response.status_code)
Task 8: Run static code analysis
Task 8: Run static code analysis
Finally, in Task 8, we check the quality of your coding skills as per the PEP8 guidelines by running static code analysis.
Normally, this is done at the time of packaging and unit testing the application. However, we have kept this step at the of this project since the codes are updated in all tasks before this.
Once your files for this project are now ready, let us test them for adherence to the PEP8 guidelines.
The first step in this process is to install the PyLint library using the terminal.
python3.11 -m pip install pylint
Next, use pylint to run static code analyse server.py.
On terminal bash execute the following command.
pylint server.py
If all aspects of PEP8 guide have been incorporated in your code, then the score generated should be 10/10. In case it isn't follow the instructions given by the modify to correct the code appropriately.
Key Takeaways
Key Takeaways
References
References
Page updated
Google Sites
Report abuse