It would be a bad idea to use one-hot encoding where each input word is encoded as 19,999 zero-values and 1 one-value. For these types of NLP problems you should use what is called a word embedding, ...