RR17. Seq2Seq

Seq2Seq

Motivation

---

Machine Translation?

Machine translation을 위해 고안됨.

Machine Translation (MT) is the task of translating a sentence x from one language (the source language) to a sentence y in another language (the target language).

How to Learn?

Large corpus부터, token간 conditional probability modeling.
translation이기에, parallel한 데이터 필요. (pairs of human-translated French/English)

Alignment

---

Note

Alignment is the correspondence between particular words in the translated sentence pair

• Typological differences between languages lead to complicated alignments!
• Note: Some words have no counterpart.

Multi column

many to one

one to many

many to many

Seq2Seq

---

Overview of Seq2Seq

Attention

many-to-many 상황에서 context vector 압축이 굉장히 challenging.
모든 input 정보를 하나의 state(encoder의 last state)에 담는 게 어렵다.

with attention

---

Note

• attention으로 해결하려고 도전.

with RNN and Attention

---

Note

RNN, Attention을 같이 적용.

$\text{Intuition : Context vector attends to the relevant part of the input seq}$

$\text{Attention Map}$

원본 링크