update

markdown/solution.md

@@ -1,6 +1,9 @@
 ### Solution
-To solve this problem, I trained a BERT model and utilize the fill-mask task to solve this problem.
+To gain the trianing data, I scraped from the trending topics from Twitter, filtered out the tweets in different languages and left with only tweets in English. There are more than 30k tweets in our training dataset.
+
+
+Then, a BERT model is trained to do the fill-mask task to solve this problem.
 Some special modifications I made to the model are as follows:
-1. Hashtags are usually consists of multiple words without space. When tokenizing these hashtags, they will be splitted into different words and therefore cannot form up an hashtag. Therefore, I added the top 1000 trending hashtags to the token dictioanry and provide special token_ids for each hastag. 
+1. Hashtags are usually consists of multiple words without space. When tokenizing these hashtags, they will be splitted into different words and therefore cannot form up an hashtag when we decode these tokens. For example, "#TheFirstLady" passed in regular tokenizer will be splitted into "#", "The", "First", "#Lady". Therefore, I added the top 1000 trending hashtags to the token dictioanry and provide special token_ids for each hastag. 
 2. When masking the tweets during training, I intentionally mask the tokens that is a hastag, so that the model will learn to predict the place of hashtags specifically.
 3. After training the model, we limit the potential candidates of [MASK] with the topics existed on Twitter, to get the relavent hashtags the user can add on.