DeepPavlov

Mentors: Student: Anastasiia Sedova

<p>The main goal of the project is to provide the user of DeepPavlov Framework with an out-of-box solution for relation extraction. I would consider relation extraction as a multi-label classification task and design a pipeline that could reuse the existing DeepPavlov components (such as NER with incorporated tokenizer, already implemented MT-BERT, and others) as much as possible. I would train several relation extraction models with different parameters (for example, different training data, classifier, classifier inputs, inputs encoding, number of relations to be extracted, etc), add them to the DeepPavlov storage, and give the user an opportunity to load them (and, occasionally, additionally train with his/her own training data) and find the one that would suit his/her data the best way. The user's input would be a text and a config with RE parameters, and relational triples would be the output. Thus, the whole preprocessing, data encoding and model application would happen inside the module, while the overall relation extraction component remains clear and convenient for the user.</p>

Mentors: Student: Niklas Muennighoff

<p>The TripPy architecture brings transformer models to Goal-oriented Chatbots. While setting new SoTA results on MultiWOZ among others, TripPy also simplifies the previous standard architecture. A better + simpler solution is what we strive for in all fields. This project implements TripPy in DeepPavlov to allow anybody to build fully functional Dialog Bots on top of TripPy leveraging the DeepPavlov ecosystem. Further, the project includes a simple demo in the form of a WeatherBot as well as a more complex chatbot demo built on the new model in DeepPavlov.</p>

Mentors: Student: Anshuman Singh

<p>Multi-task learning shares information between related tasks, reducing the number of parameters required. State of the art results across natural language understanding tasks in the GLUE benchmark has been previously used transfer learning from a large task: unsupervised training with BERT, where a separate BERT model was fine-tuned for each task.</p> <p>In the current state of DeepPavlov, multi-task BERT is implemented in Tensorflow which need to be refactored such that DeepPavlov uses new frameworks such as PyTorch. The refactored code also needs to incorporate techniques such as PAL-BERT within the DeepPavlov library by matching the results of the GLUE benchmark on the respective techniques.</p>