Greyatom Hackathon Quora Question pairs NLP Machine learning problem
Problem-1: Quora Question pairs Early 2017, Quora released their first datasets [ Link ] to public. The dataset is related to the problem of identifying duplicate questions. About Quora: Quora is a place to gain and share knowledge—about anything. It’s a platform to ask questions and connect with people who contribute unique insights and quality answers. This empowers people to learn from each other and to better understand the world. Over 100 million people visit Quora every month, so it's no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question, and make writers feel they need to answer multiple versions of the same question. Quora values canonical questions because they provide a better experience to active seekers and writers, and offer more value to both of these groups in the long term. Currently, Quora uses a Random Forest model to identify duplicate questions. In this task, the Greyatom cohort were challenged to build advanced machine learning models to tackle this NLP problem. Doing so will make it easier to find high quality answers to questions resulting in an improved experience for Quora writers, seekers, and readers. Metric: log_loss, Accuracy
- optimize your models for log loss. The lower the log loss the better the model The Greyatom cohort can use any machine learning model to solve this problem. Some of the techniques which might help them in solving this problem are
- N_grams
- TF-IDF
- Word2vec
- Doc2vec The Greyatom cohort is requested to apply only machine learning models to solve this problem. Some links which might help:
- https://www.linkedin.com/pulse/duplicate-quora-question-abhishek-thakur/
- The problem is also hosted on Kaggle and you can find a lot of kernels which solved this problem. https://www.kaggle.com/c/quora-question-pairs#description