Academia.eduAcademia.edu

Natural Language Processing

Name : Christoffel Daniel Yesaya Tambunan Student ID: 9169420231 Course : Natural Language Programming Prove that the loss J (Equation 2) is the same as the cross-entropy loss between y and (note that y, are vectors and is a scalar): Answer : y is a one-hot encoded vector. For For Substitute it, The loss J is the same as cross-entropy loss. Compute the partial derivative of with respect to vc. Please write your answer in terms of y, , and U. Show your work (the whole procedure) to receive full credit. Answer : Compute the partial derivatives of with respect to each of the ‘outside’ word vectors, There will be two cases: when , the true ‘outside’ word vector, and , for all other words. Please write your answer in terms of y, , and vc. In this part, you may use specific elements within these terms as well (such as y1, y2, . . . ). Note that uw is a vector while y1, y2, . . . are scalars. Show your work (the whole procedure) to receive full credit. Answer : When w=o : When w≠o : Write down the partial derivative of with respect to U. Please break down you answer in terms of the column . No derivations are necessary, just an answer in the form of a matrix. Answer : The partial derivatives of each component of J with respect to each element of U (parameter of outside words vec). Suppose the center word is and the context window is, where is the context window size. Recall that for the skip-gram version of word2vec, the total loss for the context window is: Here, represents an arbitrary loss term for the center word and outside word is equal to Equation 2. Write down three partial derivatives: Write your answers in terms of and. This is very simple – each solution should be one line. Answer : = 0