The Importance of Literature Review
The literature review is an essential part of every research. Identifying the potential areas of development, limitations in the current research and most importantly, gaining the essential topic knowledge, are the key reasons for the high importance of literature review. While many papers are easily absorbable even by people from outside the area of expertise, others with their domain-specific jargon can prove difficult to digest, even by the people from within the field.
In my case, it is usually the ML papers that I have a hard time understanding. With their high association to the mathematic theory and complex explanation of the models applied, they usually require multiple reads to fully comprehend. Based on the information from other blogs and websites I managed to develop a system that facilitates my ML paper’s reading and understanding process.
Why ML Papers Can Be Hard to Understand?
The main reason behind the “challenging” nature of many ML papers is possibly their interdisciplinary nature. To deliver an effective ML model it is necessary to consider statistics, mathematics, programming and in some cases economics and finance theory. Given their complex nature, it is normal for them to be difficult to understand. Therefore, the main recommendation for reading ML papers is:
Do not get frustrated if you can’t grasp all of the concepts from the first read
That is the most important part of reading every paper, however, in my opinion, crucial when it comes to understanding ML papers and using my guide.
Secondly:
Take Regular breaks at least 5 minutes for 1h of work
It is crucial to give your brain rest, especially when working on a computer and looking at the screen all the time. Make yourself a coffee, meditate, sit down, everything that would make you stand up from your chair and change the environment (even for a while).
Summary for Reading ML Papers
The guide you can see below is a summary of Andrew NG lecture on how to read the ML papers with the addition of my personal tips and other information I have found online on the topic (the rest of the sources are listed in the references section). Nevertheless creating a “manual” on how to read ML papers might prove useful to any of you, hence I decided to collect all information in one place in a form of a structured guide.
First Pass- Title -> Abstract -> Graphs
Reading every ML paper should begin with identifying the Context of the research, which can be easily inferred from the paper’s title and abstract. Moreover, it is well documented that people can process images much faster than text and more importantly we are more likely to remember information stored within pictures. That said when it comes to ML papers, understanding the familiarizing with pictures can complement the information from the abstract.
Second Pass- Introduction and Conclusion (ONLY!)
The main idea behind skipping the rest of the text and focusing on the introduction and conclusion is that they are likely to contain all of the information on the author’s Motivation and Results. Understanding those two is essential to comprehend the tools and techniques used in an article. In my experience, they often made novel and complex concepts, easier to grasp providing the reader with a bigger picture regarding the whole idea of the paper. Although they usually contain less domain-specific jargon, they mention crucial to the topic concepts and terms. You can easily spot challenging expressions and learn about them before undertaking the whole document.
However, remember if you are new to a field, even Introduction and a Conclusion might require some more time to understand.
Third Pass- Read the Whole Article
Provided you have already gone through the first two steps, you should have sufficient knowledge to understand how authors have Implemented their methodology. Very often ML papers (the good ones should) will be accompanied by the mathematics equations, to explain the basis of the theory/model. In my experience, if you read the mathematics formula couple of times and you have a hard time understanding it, just skip it (for now). Do not get stuck and try to read them over and over again, in the hope of finally understanding the concept. By progressing with the text, you are more likely to understand the maths behind it, so keep reading and don’t get frustrated with complex mathematics concepts.
Fourth Pass- Read the Article Again 😉
Are you still there? Good now go over the thing again! As harsh it might sound it is often necessary to read the article couple of times to fully understand it. However, bear in mind that if there are still areas that you have a hard time understanding, it is better to skip them. It is very likely for you not to understand all of the concepts mentioned in the article. As long as you feel comfortable with what authors did and how they did it, it should be enough to understand the article. It is all part of the learning process and we are unable to comprehend everything at one go. Try exploring other authors’ work or related research to learn about what concepts are crucial and which aren’t anymore.
What about the Code?
Some ML papers include the code which authors have used. In some cases, authors put their code on Github sometimes including thorough explanations of their programming choices. If you are interested in using the authors’ approach, you can download the code from the linked repository and try to run it yourself or recreate it using the methods you know.
Conclusion
Hoepfully now, reading ML articles will be less of a struggle and more of a joy to you. Remember that ML is a vastly developing discipline, hence it can initially be confusing and hard to understand. However, I hope that this guide will make the whole process more pleasant.
References:
-
Stanford CS230: Deep Learning | Autumn 2018 | Lecture 8 – Career Advice / Reading Research Papers. Special thanks to Andrew NG who inspired me to write that guide and provided great insight into understaing ML papers.
- Rampersad, N., David R. Cheriton School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1 (Canada) nrampersad@ math. uwaterloo. ca.