How can we get an award in a major AI conference? I simply answer 'you must aim for the award at an upcoming conference!'
This blog post introduces several tips and lessons from my recent experience in one of the major computer vision conference. Our paper 'Pre-training without Natural Images' was selected as the ACCV 2020 Best Paper Honorable Mention Award*!
*However, the award is not the end of our research. This is very beginning of our journey. I’d like to improve the technology and employ on various vision-based tasks and real applications.
What we did?
We achieved a pre-training without any natural images for transfer learning. The following video contains the examples of pre-training dataset and brief description.
Though the conventional image datasets caused several problems such as privacy-violation, annotation-labor, and AI-ethics, our proposed dataset does not contain these elements in pre-training phase. In our experiment, a dataset consists of Fractal geometry recorded the best performance as formula-driven supervised learning. The natural objects/scenes are created under the law of fractal geometry. Therefore, this is a reasonable result. The detailed description is shown on the project page.
In the next step, I introduce the tips and lessons how to reach the idea and result.
1. The first idea is important, and keep polishing your idea
- Researchers are always thinking their idea at every moments. However, sometimes the idea is being an incremental one. This tends to be a boring topic. Of course, an initial idea is not always an excellent one. Therefore, we must update the research topic through the experiments from the results and considerations. A failure case may change a good inspiration when you keep thinking!
- I often conduct brain storming with my collaborators. Though I will leave the detailed procedure for another post, brain storming is undoubtedly an important tool for polishing your idea. I believe that the balance between self-improvement and interactive discussion is quite important. You can improve your idea when you are reading/writing an academic paper (self-improvement), joining a meeting, and discussing in an international conference (interactive discussion).
- However, the above-mentioned improvements cannot be happened if you don’t have a mind of self-improvement for an initial idea. You need to have a flexible perspective.
2. The research must be improved, improved and improved
- This is similar to the first tip/lesson. However, in this case, the tip/lesson indicates the contents such as specified method and experimental results. We have a lot of candidate approaches in order to achieve the decided idea. I would like to choose a better way as well as possible.
- I have kept improving my research topic for five years. This is my first experience focusing on my effort for a single project in such a long moment. Totally, we have spent over 200,000 GPU hours for the experiments.
3. Get collaborators involved in your research
- I believe that one of the biggest turning point was to get a research grant. I got group-type Japanese research grant (this is called Scientific Research type (A) from Japan Society for the Promotion of Science). I had written the application document with three collaborative researchers. Luckily, I have collaborated with their graduate students. As the result of getting our grant, we could construct a strong research team!
- As conducting a collaborative research, I have extended the research area. Now in 2021, I am focusing on several sub-tasks in computer vision not limited to image recognition. Graduate students are powerful to execute their research and have a flexible mind.
4. Use the greatest resources
- The usage of super computer accelerated our research dramatically! We have used ABCI (AI Bridging Cloud Infrastructure, by AIST) which was top-ranked in TOP500 (5th place) and Green500 (8th place). I am a researcher in AIST, therefore, it was very easy to register the super computer.
- How much is it faster to use the super computer? Our experimental speed was accelerated as follows. Before: 90 days on Deep Learning Box (4 Titan X GPUs) vs. After: 2 days on ABCI N node (4 NVIDIA V100 GPU x N nodes; N is up to 1088). For example, my three-month experiment was shortened to a weekend experiment. Finally, we have spent a lot of GPU hours in order to comprehensively investigate the characteristics of FractalDB.
5. Change your strategy as necessary
- I decided to change the submission strategy based on the reviews. My paper was rejected at several times in the premier computer vision conferences such as CVPR/ICCV. I thought that our first idea cannot be accepted in the top-tier conferences since the performance rate is not reached to the ImageNet pre-trained model. In a top-tier computer vision conference, accuracy is one of the very important elements. On behalf of that, I changed the strategy from submitting a top-tier conference to get award nominee in another major computer vision conference.
- I have also considered re-writing the section of abstract/introduction in our paper. I have improved our paper at each time based on the review comments. Before until nominated the award list, a major comment said “the paper is interesting but performance rate is bad.”
6. Your idea must be clearly presented
- At each submission to an international conference, I have revised several times. When I look back the revised paper, the version was over 20+. I thoroughly wrote my paper ver.1 one month before the deadline, then the paper was shared to be reviewed by group members and internal reviewers. The iterative review makes it clearer and stronger.
- An internal review is very effective for revising your paper. We can additionally conduct experiments, re-write the introduction, and re-make the figures after the internal review. Anyway we should write ver.1 in the early time. The document (How to write a good CVPR submission) also describes “good writing is re-writing” in the last slide.
7. Aim for it, otherwise you cannot get the award
- Through the tips and lessons, I got a much better ratings at final review results in ACCV 2020. There were three strong accepts from reviewers. Some reviewers put on their comments like “this is the best paper I have reviewed this year including top-tier conferences” and “I congratulate the authors for a very good work”. I appreciate the all co-authors, reviewers, area chairs and program committee. In this way, our paper was nominated as an award finalist.
- I have prepared presentation material in our project page (see below). I’m not sure whether the preparation is effective or not, I thoroughly created the contents such as camera-ready paper, oral/poster presentation, codes/trained weights, and YouTube video as well as possible.
Finally, our paper was selected as the Best Paper Honorable Mention Award!
