daviddwlee84 / deeplearningpractice Goto Github PK

• Set higher layers (inference) a higher learning rate and lower layers (BERT representation) a lower learning rate
• ...

• Inside–outside–beginning (tagging) - Wikipedia
• nlp - Difference between IOB and IOB2 format? - Data Science Stack Exchange
• KrishnanGanapathy-NamedEntityRecognition.pdf
• NLP | IOB tags - GeeksforGeeks
• Representing Text Chunks

• Few Shot Learning
• ...

• BERTology — transformers 3.0.2 documentation
• [2002.12327] A Primer in BERTology: What we know about how BERT works

• The Illustrated GPT-2 (Visualizing Transformer Language Models) – Jay Alammar – Visualizing machine learning one concept at a time.
  • 完全图解GPT-2：看完这篇就够了（一） - 知乎
  • 完全图解GPT-2：看完这篇就够了（二） - 知乎
• Fine-tune GPT2 for Text Generation Using Pytorch | Towards Data Science
  • itsuncheng/fine-tuning-GPT2: Codebase for the Medium Article on Fine-tuning GPT2 for Text Generation
• Beginner’s Guide to Retrain GPT-2 (117M) to Generate Custom Text Content | by Ng Wai Foong | AI Innovation | Medium
  • nshepperd/gpt-2: Code for the paper "Language Models are Unsupervised Multitask Learners"
• Fine-tune a non-English GPT-2 Model with Huggingface | by Philipp Schmid | Towards Data Science

Huggingface

GPT2LMHeadModel

• OpenAI GPT2 — transformers 3.4.0 documentation
• Examples — transformers 2.0.0 documentation (About fine-tune)
• Converting Tensorflow Checkpoints — transformers 3.4.0 documentation
• How to train a new language model from scratch using Transformers and Tokenizers

TODO: WWM?!

• Attacking Machine Learning with Adversarial Examples
• 6.2 Adversarial Examples | Interpretable Machine Learning

Label Studio

• Label Studio

BRAT

• nlplab/brat: brat rapid annotation tool (brat) - for all your textual annotation needs

doccano

• doccano/doccano: Open source text annotation tool for machine learning practitioner.

• 3 subword algorithms help to improve your NLP model performance | by Edward Ma | Medium

• Deep Learning for Human Language Processing (2020,Spring) - YouTube
  • [DLHLP 2020] Deep Learning for Coreference Resolution - YouTube
  • [DLHLP 2020] Overview of NLP Tasks - YouTube
  • [DLHLP 2020] BERT and its family - Introduction and Fine-tune - YouTube
  • [DLHLP 2020] BERT and its family - ELMo, BERT, GPT, XLNet, MASS, BART, UniLM, ELECTRA, and more - YouTube

• https://github.com/mtobeiyf/keras-flask-deploy-webapp

• Sentence Similarity
• CS224n?! & other courses, textbook
• NCTU homework

https://github.com/ckyken/DL/tree/master/HW1

• Move some content to PaperReading repository
• Deep Learning State of the Art (2020) | MIT Deep Learning Series - YouTube

• deepwel/Chinese-Annotator: Annotator for Chinese Text Corpus (UNDER DEVELOPMENT) 中文文本标注工具
• brat rapid annotation tool
  • nlplab/brat: brat rapid annotation tool (brat) - for all your textual annotation needs
  • brat: a Web-based Tool for NLP-Assisted Text Annotation - ACL Anthology

CS 520

• Introduction
• Slides
• Video

Paper

• [1605.07723] Data Programming: Creating Large Training Sets, Quickly
• Snorkel: rapid training data creation with weak supervision | SpringerLink
  • Snorkel AI: Programmatic Approach to Labeling Training Data | by Yitaek Hwang | Towards Data Science
  • snorkel-team/snorkel: A system for quickly generating training data with weak supervision
  • snorkel-team/snorkel-tutorials: A collection of tutorials for Snorkel
  • Get Started · Snorkel
• [1812.00417] Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
• [1909.05372] Overton: A Data System for Monitoring and Improving Machine-Learned Products
  • p33-re-cidr20.pdf

Article

• Weak Supervision for Science and Medicine: A Year in Review

Book
Deep Learning from Scratch
https://github.com/oreilly-japan/deep-learning-from-scratch-2

Framework

Package

Tool/Script e.g. text clean up

Project?!

Pre-trained model?!

Examples?!

...

• EDA
  • jasonwei20/eda_nlp: Data augmentation for NLP, presented at EMNLP 2019
  • zhanlaoban/EDA_NLP_for_Chinese: An implement of the paper of EDA for Chinese corpus.中文语料的EDA数据增强工具。NLP数据增强。论文阅读笔记。