Giter VIP home page Giter VIP logo

dfpassageretrieve's Introduction

DFPassageRetrieve

新闻文本数据的语义检索与智能问答的baseline, 2022-06-20排在第3名。

详细的思路请参考博客:https://zhuanlan.zhihu.com/p/531463300

运行方法

Step1:获取词表,运行get_vocab.py

Step2:预训练,TF环境,非常耗时,运行pretrain_roformer.py

Step3:预训练模型转为Torch,运行convert_tf_roformer_to_pt.py

Step4:微调,Torch环境,运行run_finetune.py

Step5:预测结果并生成提交文件,Torch+Faiss,运行run_predict.py

dfpassageretrieve's People

Contributors

dunzhang avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.