Publishing in Computing Research
Over the past decade, I have had the privilege, and sometimes the misfortune, of publishing across several different computing communities. My work is pretty interdisciplinary, spanning networking, wireless, multimedia systems, and AR/VR, which means I submit papers to conferences such as SIGCOMM, MobiCom, INFOCOM, NSDI, IMC, ACM Multimedia, MMSys, SIGGRAPH, IEEE VR, and ISMAR. Although these conferences all belong under the umbrella of computer science, they often feel like entirely different academic worlds. They ask different questions, value different things, and sometimes seem to practice entirely different definitions of research.
The first thing that struck me was the enormous difference in review culture. Computer networking conferences (e.g., SIGCOMM or MobiCom) are relatively small communities with a narrow focus. Their papers tend to emphasize the big picture, such as why the problem matters, whether it is relevant to today's industry, and whether the ideas can realistically influence systems in the very near future. A paper can sometimes succeed or fail based on how compellingly the problem is articulated, even more than the technical details themselves.
Multimedia and AR/VR conferences operate very differently. Papers are expected to get to the technical contribution quickly, often assuming that reviewers already possess the necessary domain expertise. The reviews tend to focus on the ideas, algorithms, experiments, and technical soundness rather than spending pages debating whether the problem itself is important.
One thing that has increasingly frustrated me in networking conferences is how attached communities become to familiar applications and trends. This is specific to one of my areas which is video streaming and application layer networking. There are researchers who have spent decades working on video streaming problems, and understandably so. The area has produced excellent research and tremendous real-world impact creating an entire industry ecosystem of online video. But when entirely new forms of media emerge, such as volumetric video, point clouds, or dynamic 3D scenes, there is often resistance to treating them as first-class networking problems. They forget that we are enjoying the fruits of radical video streaming innovation decades ago.
This has always puzzled me. At the application layer, data is still data. The networking questions remain largely the same e.g., how should the network transport it, allocate resources to it, or adapt itself to new demands? Yet networking researchers sometimes become attached not to the fundamental problem, but to the particular applications they have grown comfortable with.
I have seen a similar shift in attitudes toward core networking research itself. Older generations of networking papers often introduced bold and radical ideas supported by analysis or simulations. Today, simulation-only papers are frequently met with a simple question -- "Where is the real testbed?" Testbeds are valuable, but they are not always necessary for fundamental research. Sometimes the contribution is the idea itself, or the identification of a new design space that others can later build upon.
This push toward immediate validation can unintentionally bias communities toward incremental work. If every paper must demonstrate a deployable prototype, then naturally the incentives favor solving small, practical problems over exploring risky new directions. The irony is that many of the systems we now consider foundational began as speculative ideas long before the technology existed to implement them. A field that only rewards what can be built today may slowly lose its ability to imagine what should exist tomorrow.
The biggest difference I have observed, however, is review quality. Networking reviewers often review up to twenty papers for each conference. Under those conditions, even excellent reviewers simply do not have the time to deeply understand every paper, particularly interdisciplinary ones. In multimedia and AR/VR conferences, reviewers usually handle far fewer submissions, which often allows for much more thoughtful and technically detailed reviews.
I have had papers rejected in both communities. But after reading reviews from multimedia and AR/VR conferences, I am usually left thinking, "I disagree with the decision, but the reviewers understood the paper." I cannot always say the same after networking conferences reviews. A rejection based on disagreement is healthy for science; a rejection based on a very high level review without technical details is much harder to accept.
Another aspect of networking research community that I find fascinating is its tendency to respond to dissatisfaction by creating new conferences. Networking researchers became unhappy with aspects of INFOCOM and created CoNEXT. More recently, frustrations with SIGCOMM have contributed to the creation of Nines conference. New conferences are not inherently bad, and experimentation with publishing models is important. My concern is that these decisions are often driven more by frustration with the status quo than by a long-term vision for how the community should evolve. New conferences frequently emerge around existing social networks of advisors, students, collaborators, and colleagues, which can unintentionally make outsiders feel that they are joining an already established club rather than helping build an open scientific community. From the perspective of a young researcher, the constant fragmentation can feel chaotic. Where should a graduate student publish? Which community should they invest their time in? Which venue will still matter ten years from now? Senior researchers often have the reputation and network to comfortably move between conferences, but younger researchers are still searching for a home community.
Perhaps the most troubling realization for me has been that many computing communities do not particularly respect each other's research traditions. This becomes particularly problematic during funding reviews. Networking research today is a collection of loosely connected communities like SIGCOMM, INFOCOM, CoNEXT, MobiCom, GlobeCom, ICNP, etc, each with its own culture, expectations, and definition of novelty. Researchers from one venue do not always appreciate the style of work produced by another, even when they are fundamentally studying the same problems. A paper that feels perfectly natural to a SIGCOMM audience may seem uninteresting to an INFOCOM reviewer, and vice versa. The same disconnect often appears in funding panels, where proposals end up being judged not on the merits of the ideas themselves, but on whether they conform to the norms of whichever subcommunity happens to be reviewing them. For a field that prides itself on systems thinking, we have built surprisingly fragmented research communities that often struggle to understand each other.
After nearly a decade of publishing across these venues, I have come to a simple conclusion that computer science is still a remarkably young discipline. Other sciences settled long ago on journals, editors, revisions, and continuity. We chose conferences because we wanted quick feedback and faster dissemination. Ironically, many of our conferences are now reinventing the very mechanisms that journals have used for decades. We have introduced rebuttals, revisions, reviewer continuity, resubmissions, and shepherding. We simply avoid calling them major revisions, editors, and journal processes. In many ways, our communities are all trying to solve the same problems while pretending to be inventing something entirely new. Perhaps that is simply the growing pain of a young field.