How to understand AI/ML research publications
Do data scientists and machine learning professionals need to read research papers?
The short answer is “yes”. And don’t worry if you don’t have a formal education or only have an undergraduate degree in machine learning.
Reading academic papers can be daunting for the uneducated. However, a lack of academic reading experience should not prevent data scientists from leveraging a valuable source of information and knowledge for machine learning and AI development .
This article is aimed at data scientists of all skill levels and provides hands-on tutorials for reading research papers published in academic journals such as NeurIPS , JMLR and ICML .
The first phase of learning how to read research papers will guide you through relevant topics and selection of research papers before committing to reading research papers in earnest.
Step 1: Identify Topic
There are many thematic domains in the domain of machine learning and data science. However, this does not necessarily mean that tackling every single topic within machine learning is the best choice.
Introductory knowledge is recommended in general, but long-term machine learning learners often shift to specialization in terms of career prospects, machine learning practices, and industry interests. be done.
Finding and working on a niche topic can be difficult, but it’s worth it. However, a good rule of thumb is to choose an area of ML in which you are interested in pursuing a professional position, or in which you already have experience.
Deep learning is one of my areas of interest and I’m a computer vision engineer who specializes in using deep learning models in apps to solve computer vision problems. Therefore, I am interested in topics such as pose estimation, action classification, and gesture identification.
Some examples of role-based ML/DS job titles and related themes are provided below for your reference.
(*Translation Note 1) The above table is translated into Japanese below.
Map of Machine Learning Jobs and Topics
| computer vision engineer | Convolutional Neural Networks, Real-time Object Detection, Generative Adversarial Networks, Pose Estimation |
| natural language processing engineer | Transformer, recurrent neural network, LSTM, language modeling, computational linguistics, morphological analysis , text classification, word embedding |
| data engineer | Data pipelines, data modeling, security and privacy |
| data scientist | Data Pipeline Optimization, Data Augmentation, Data Visualization Approach, Pattern Recognition |
| deep learning engineer | Single-shot detection, Transformer, Feature extraction |
| robotics engineer | Inverse kinematics, automation, map planning, reinforcement learning |
| AI Ethicist | AI Ethics, Explainable AI |
In this article, I will choose the topic of “Pose Estimation” to investigate and select related research papers to study.
Step 2: Find Research Papers
PapersWithCode is one of the best tools to use when researching machine learning related research papers, datasets , code, and other related material .
Use the search engine on the PapersWithCode website to get research papers and content related to our chosen topic of “Pose Estimation.” The image below shows how to do that.
The search results page provides a short description of the topic you searched for and a table of related datasets, models, papers, and code. Without further elaboration, in our use case we are interested in “great papers with code” (related to the topic). This section contains papers related to tasks and topics. For this article I chose DensePose: Dense Human Pose Estimation In The Wild .
Step 3: Fastpass (gain context and understanding)
At this point we have already selected a paper to study and are ready to derive valuable learning and discoveries from its content.
It’s no surprise that the first thing you’re likely to do is start taking notes, read the document from beginning to end, and perhaps take a break in the middle. However, a more practical way to read is to grasp the context of the research paper’s content. In any research paper, the title, abstract, and conclusion are three important parts to gaining understanding.
Fastpass goals for the selected papers are:
- Make sure the paper is relevant (to your chosen topic).
- Understand the context of the paper by knowing the content, methods, and findings of the paper.
- Recognize the author’s goals, methodology, and achievements.
Title
The title is the first place where authors and readers share information. Therefore, the titles of research papers should be direct and unambiguous.
The title of a research paper is the most important point because it indicates the relevance of the research to the reader’s work. The importance of the title is to make the content of the paper easily recognizable.
In the case of the case taken in this article, the title is “DensePose: Estimating Raw Dense Human Pose”. This is meant to give a rough overview of the research and consider how to provide adequate pose estimation in environments with high activity levels and realistic situations.
Gist
The abstract section summarizes the content of the paper. A short section of 300 to 500 characters that concisely conveys the content of the paper. An abstract is also a short statement that outlines the content of the paper, the researcher’s aims, methods, and techniques.
When reading the abstracts of machine learning research papers, we usually come across references to datasets , methods, algorithms , and other terms. Keywords related to the content of the article provide context. It may be useful to take notes and record all keywords.
In the case of the present paper, “DensePose: Estimation of Raw Dense Human Pose”, we identified the following keywords in the abstract: pose estimation, COCO dataset, CNN, region-based model, real-time.
Conclusion
Especially for data scientists and practitioners without advanced academic experience, reading a paper from top to bottom during the first pass can often be tiring. Extracting information from the second half of a paper can seem tedious after a long study session, but conclusions are often short. It is therefore recommended to read the conclusion section on fastpass.
The conclusion section briefly summarizes the author(s) of the study and/or its contributions, achievements, promises for future development, and limitations.
Before reading the main content of the research paper, read the conclusion to see if the researcher’s contribution, problem domain, and results meet your needs.
By following this extraordinarily simple first-pass step, the scope and purpose of the research paper, as well as the context of the content, can be fully understood and overviewed. Read more carefully after this step for more detailed information.
Step 4: Second Pass (Familiarize yourself with the content)
Familiarity with content is the process associated with the first step. This process is the step involved in the introductory section and figures in the research paper.
As mentioned earlier, familiarizing yourself with the paper’s findings will allow for more detailed and comprehensive consideration in later steps, so there’s no need to jump straight to the heart of the research paper.
Introduction
An introduction to a research paper is written to outline the purpose of the research activity. The purpose of the research written in this way refers to and explains the problem area, research scope, previous research efforts, methodology, etc.
In general, the introduction finds correspondence with previous research results that have addressed similar problems using similar or different methods. Citations of other papers indicate the scope and breadth of the problem domain and broaden the reader’s search area. At this point, it should be sufficient to incorporate the procedure described in step 3.
Another advantage of the introduction is that it presents the knowledge necessary to approach and understand the content of the research paper.
Graph/Illustration/Number
Illustrative material in research papers helps readers understand the underlying elements of the problem definition and explanations of the methods presented. Papers typically use tables to provide information on the quantitative performance of new technologies compared to similar approaches.
Visual representations of data and performance in general foster an intuitive understanding of the paper’s context. In the aforementioned high-density pose paper, the authors illustrate the performance of the pose estimation approach they discuss.
In the field of deep learning, topological diagrams of the structure of artificial neural networks are often seen. This also helps the reader’s intuitive understanding. Illustrations and diagrams allow the reader to interpret the information for himself and gain a broader perspective without preconceived notions of what the results should be.
Step 5: Third Pass (Peruse)
The third pass, like the second pass, will cover more important parts of the text. The most important thing in this pass is to avoid formulating complex calculations and techniques that you find difficult. This pass also skips words and definitions that you don’t understand or are unfamiliar with. You should write down these unfamiliar terms, algorithms , and techniques for later reference.
In this path, the primary goal is to gain a broad understanding of the content of the paper. Continue reading the paper again from the abstract to the conclusion, but be sure to include breaks between sections. Additionally, I recommend getting a notepad and jotting down key insights and lessons along with unfamiliar terms and concepts.
The Pomodoro Technique is a great way to manage the time you spend reading and studying. The method is to divide the day into blocks of work followed by short breaks.
In my case, a 50/15 ratio of studying for 50 minutes and taking a break for 15 minutes works well. I do this twice in a row and then take a longer break for about 30 minutes. Before you get used to time management techniques, it’s a good idea to adopt a relatively simple division, such as 25/5, and adjust your time allocation according to your concentration and spare time.
Step 6: Fourth Pass (Final Pass)
The final pass usually requires mental and academic ability as you review terms, jargon, concepts, algorithms, etc. that you noted in the previous pass. This path focuses on using external sources to understand unknown parts of the recorded paper.
Intensive study of an unknown subject is indefinite, sometimes lasting days or weeks. A key ingredient to the success of the final pass is locating the right sources for further exploration.
Unfortunately, the Internet does not provide a comprehensive and rich source of the information you are looking for. However, there are multiple sources on the Internet, and using them appropriately can fill in knowledge gaps. Below are some of those resources.
- Machine Learning Subreddit
- Deep Learning Subreddit
- Papers With Code
- Top conferences such as NeurlPS , ICML and ICLR
- Research Gate
- Apple official machine learning site
Techniques and algorithms are mentioned in the References section of research papers. Therefore, the paper under study is inspired by, or based on, the references. As such, the bibliography section is a valuable resource for perusal sessions.
・・・
Step 7: Summary (Optional)
After nearly a decade of academic and professional involvement in technical subjects and professions, the most effective way to retain newly learned information in long-term memory is to revisit the topics explored. . Rewriting or typing out new information in my own words helped me to reinforce the ideas presented so that they were easier to understand and remember.
Image Source: NeONBRAND on Unsplash
Furthermore, by utilizing blogs and social media, it is also possible to publish learning results and notes. Attempting to explain a concept you’ve just explored to a large audience, assuming an audience unfamiliar with the topic or subject, requires a substantial and detailed understanding of the topic.
Summary
There is no doubt that reading research papers can be intimidatingly challenging for data scientists and ML novices. Even veterans have a hard time digesting research papers at a glance.
The nature of the data science profession is very practical and complex. In other words, since the field of data science is closely related to AI, which is still a developing field, practitioners are also required to use an academic mindset.
In summary, all the steps to take to read a research paper are:
- Identify a topic.
- Find relevant research papers.
- Read the title, abstract, and conclusion to gain a vague understanding of the aims and results of research activities.
- While reading the introduction, explore the illustrations and graphs introduced in the paper to familiarize yourself with the content.
- Peruse and digest the main content written throughout the paper.
- Look up unfamiliar words, technical terms, concepts, methods, etc. using external sources.
- Summarize essential points, definitions, algorithms, etc. in your own words.
Thank you for reading.
