Description

Assignment-8: Neural Network
(Read all the instructions carefully & adhere to them.)

Description : Develop a neural network based model for intent classification that identifies the intention expressed in an user utterance for an Air Travel Information System. The overall task is to recognize the intent such as flight, airfare, airline etc. (total 17 categories) expressed in an utterance.
For eg.: “ i would like to find a flight from charlotte to Las Vegas that makes a stop in St. Louis ”, the intent for this particular utterance is “ flight ”.

Training dataset : ATIS.txt.
Description of the dataset can be found in https://github.com/Microsoft/CNTK/tree/master/Examples/LanguageUnderstanding/ATIS Test dataset : ATIS_RAWTEST.txt

Dataset Format :

0 |S0 178:1 |# BOS |S1 14:1 |# flight |S2 128:1 |# O
0 |S0 479:1 |# i |S2 128:1 |# O
…
0 |S0 606:1 |# morning |S2 12:1 |# B-arrive_time.period_of_day
0 |S0 179:1 |# EOS |S2 128:1 |# O

Extract only the words appearing between #BOS and #EOS tag to form an utterance. The corresponding label for that utterance is at the third column of the beginning of the utterance. So, for this particular example, the data that you need to extract is marked in bold , ignore the rest.

Task : Preprocess and extract the data from its corresponding file. Implement a neural network based classifier to identify the intent of an utterance.

Input and Outputs of the network:
i) Input : Input to the network should be the unigram vector
ii) Output layer should have as many neurons as per the no.of classes (i.e. one neuron
per category). If an utterance has multiple intents, take only one of the intent from that. Use “softmax” as the activation for the output layer. Represent output categories as:
a) 1 0 0 0…. – flight
b) 0 1 0 0…. – airfare
c) 0 0 1 0…. – airline and so on
iii) Network must have atleast one hidden layer (can be more as per your choice)

Report :
1. Accuracy of the model
2. Precision and Recall of individual classes
3. Overall F1-measure
4. Some test cases with its corresponding output from the model

Instructions :
a) For implementation, you can use any library.
b) Try with different loss functions/activation functions/optimizer (as given for assignment 7) and observe :
i) Weights
ii) Loss
iii) Accuracy