dair-ai / nlp_paper_summaries Goto Github PK

http://www.cleverhans.io/2020/04/06/stealing-bert.html

A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
https://arxiv.org/abs/1811.03402

Potentially a video summary.

Convert to website. Get inspiration from NLP Overview and NLP Progress

@omarsar Thanks for starting this initiative.

I was wondering if we can expand this from nlp_paper_summaries to paper_summaries instead and group papers by the domain: "NLP", "Computer Vision" etc. There is no existing platform that curates such paper summaries in one place and so paper_summaries could fill that gap. Just a thought.

Deep Learning Based Text Classification: A Comprehensive Review
https://arxiv.org/pdf/2004.03705.pdf

TABERT: Pretraining for Joint Understanding of
Textual and Tabular Data

https://scontent.fyyz1-2.fna.fbcdn.net/v/t39.8562-6/106708899_597765107810230_1899215558892880563_n.pdf?_nc_cat=107&_nc_sid=ae5e01&_nc_ohc=ca-FiDMrqEsAX87i9-7&_nc_ht=scontent.fyyz1-2.fna&oh=0c3b30f0b6eef7fa5268ddb1f2b0a496&oe=5F297C05

Source: https://medium.com/dair-ai

A great summary of improvements in Transformers. It could be used to extract a great recipe around this topic.

https://lilianweng.github.io/lil-log/2020/04/07/the-transformer-family.html

Got permission from @jadevaibhav to publish the following article on dair.ai's main website.

Article:
https://github.com/jadevaibhav/Critique-of-Taylor-s-Law-for-Human-Linguistic-Sequences/blob/master/README.md

Tasks:

• Migrate the article to our main website repo: https://github.com/dair-ai/dair-ai.github.io/tree/master/_posts
• Proofread and review and make necessary changes
• Add author's information
• Publish on the main website (dair.ai)
• Add the entry to the NLP Paper Summaries repo (let's figure out under which folder we can add it)
• Share on social media and our Slack group

Author's information:

jadevaibhav:
  name: Vaibhav Jade
  github:  jadevaibhav

In the coming days, I will add a series of papers summaries and improve categorization

https://arxiv.org/pdf/2004.03720.pdf

• Kaj Bostrom and Greg Durrett

I have already submitted a draft on Medium, please take a look there.

Add an excerpt to each paper summary so as to provide readers a sort of TLDR.

Please say hi and add your name below if you wish to contribute to this project. Make sure to link your GitHub account so that I can add you to this project.

https://arxiv.org/pdf/1910.11292v3.pdf

Predicting In-game Actions from Interviews
of NBA Players

Someone on Twitter suggested adding oral presentations of those papers if they were available. Your thoughts?

Currently, we have different variants of naming hyperlinks:

The question is, which of these would make the most sense for the repository at the moment?

I would add this one arXiv:1905.05950v2 [cs.CL] 9 Aug 2019

Hi,
@omarsar as the title suggests would it be possible to include summaries of seminal DL papers like https://arxiv.org/abs/1611.03530. If yes then kindly allow me to work on it.
Thanks

This is going to require a huge effort to maintain due to the nature of the project. I have already received very positive feedback on this idea and it would be nice to get volunteers to help maintain this. If you are interested, please email me to [email protected] or DM me on Twitter.

Language Models are Few-Shot Learners

https://arxiv.org/abs/2005.14165

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.

Hi all,

I have this idea to potentially use the paper summaries here to help out students. I notice a lot of courses online and in universities recommend papers for students but this can be intimidating for some and even discouraging. What if we create "recipes" for students where we recommend them a journey on what paper summaries to look at first before jumping into the actual corresponding papers. Paper summaries are more approachable/friendly and can help guide students better before they jump into paper reading. This could be a nice addition as we keep expanding the list. Thoughts?