BERT Blog Posts

--- tags: nlp, 298, --- # BERT Blog Posts Blog posts by [Chris McCormick](http://mccormickml.com/archive/) - 29 Jun 2021 » [Combining Categorical and Numerical Features with Text in BERT](http://mccormickml.com/2021/06/29/combining-categorical-numerical-features-with-bert/) - In this tutorial we’ll look at the topic of classifying text with BERT, but where we also have additional numerical or categorical features that we want to use to improve our predictions. - 27 May 2021 » [How To Build Your Own Question Answering System](http://mccormickml.com/2021/05/27/question-answering-system-tf-idf/) - In this post, we’ll create a very simple question answering system that, given a natural language question, returns the most likely answers from a corpus of documents. - This represents an easy to follow and low-effort example of a question answering system that can still be of practical use for you own application if you choose to swap in your own dataset or build something more elaborate on top of this. Additionally, the concepts covered here will be useful in later posts when we look at variations of more complex questions answering systems. - 22 Jun 2020 » [Domain-Specific BERT Models](http://mccormickml.com/2020/06/22/domain-specific-bert-tutorial/) - If your text data is domain specific (e.g. legal, financial, academic, industry-specific) or otherwise different from the “standard” text corpus used to train BERT and other langauge models you might want to consider either continuing to train BERT with some of your text data or looking for a domain-specific language model. - 19 May 2020 » [Existing Tools for Named Entity Recognition](http://mccormickml.com/2020/05/19/existing-ner-tools/) - In conjunction with our tutorial for fine-tuning BERT on Named Entity Recognition (NER) tasks here, we wanted to provide some practical guidance and resources for building your own NER application since fine-tuning BERT may not be the best solution for every NER application. - 14 Apr 2020 » [Trivial BERsuiT - How much trivia does BERT know?](http://mccormickml.com/2020/04/14/trivial-bersuit/) [no notebook] - As I’ve been doing all of this research into BERT, I’ve been really curious–just how much trivia does BERT know? We use BERT for it’s impressive knowledge of language, but how many factoids are encoded in there along with all of the language understanding? - It turns out, kind of a lot! We’re going to look at some fun examples in this post. - 10 Mar 2020 » [Question Answering with a Fine-Tuned BERT](http://mccormickml.com/2020/03/10/question-answering-with-a-fine-tuned-BERT/) - What does it mean for BERT to achieve “human-level performance on Question Answering”? Is BERT the greatest search engine ever, able to find the answer to any question we pose it? - In Part 1 of this post / notebook, I’ll explain what it really means to apply BERT to QA, and illustrate the details. - Part 2 contains example code–we’ll be downloading a model that’s already been fine-tuned for question answering, and try it out on our own text! - 11 Nov 2019 » [BERT Research - Ep. 1 - Key Concepts & Sources](http://mccormickml.com/2019/11/11/bert-research-ep-1-key-concepts-and-sources/) [no code] - In this “research notes” blog post, and the ones that follow it, I’ll be sharing what I am learning about BERT, as well as identifying the areas where I am perhaps confused or where I still need to learn more. - I think that the NLP community is currently missing an in-depth tutorial on the BERT model which does not require extensive background knowledge in LSTMs and Attention. - 22 Jul 2019 » [BERT Fine-Tuning Tutorial with PyTorch](http://mccormickml.com/2019/07/22/BERT-fine-tuning/) - In this tutorial I’ll show you how to use BERT with the huggingface PyTorch library to quickly and efficiently fine-tune a model to get near state of the art performance in sentence classification. More broadly, I describe the practical application of transfer learning in NLP to create high performance models with minimal effort on a range of NLP tasks. - 14 May 2019 » [BERT Word Embeddings Tutorial](http://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/) - In this post, I take an in-depth look at word embeddings produced by Google’s BERT and show you how to get started with BERT by producing your own word embeddings.