Guide to Successful Papers Submission

William Newman, Kate Ehrlich, John Tang, Robin Jeffries and Jock Mackinlay

As the CHI conference has grown to become a popular and prestigious conference for presenting innovative HCI research and practice, the guidelines for good CHI submissions also have evolved. Consequently, trying to prepare a paper that speaks to CHI's values may at times feel like aiming for an elusively moving target. Coupled with the fact that competition for acceptance of CHI papers is high (typically only 15-30% of the submitted papers get accepted), authors may get the impression that the CHI review process is stacked against them.

We want to dispel this notion that the CHI review process is an adversarial one, and instead characterize the process as one where authors and reviewers work together to create the best quality conference possible. Toward that end, we want to communicate to prospective authors (and to reviewers of CHI papers) what makes a successful paper submission to CHI. By "successful", we mean a submission that presents the work with all the information needed to get a fair review. If the result of the work is a new and important contribution to HCI, then "successful" submissions should be accepted. But, if the review process finds that the work does not meet the quality threshold for accepted papers, then a "successful" submission will still generate constructive feedback to the authors.

Our observations are based on a number of studies, including a survey of CHI papers from 1989 to 1992 and a study of reviews of CHI 95 papers. We hope that making some general statements about CHI's values for papers and making authors aware of common problems will lead to a more successful submission and review process.

A useful place to start discussing successful CHI submissions is the form that reviewers use to submit their reviews. The wording of the review form has changed from year to year. The full set of questions in this year's form are included at the end of this guide. In effect, reviewers are being asked to rate your paper by addressing the following questions:

• Does the paper present a significant and original contribution, of benefit to HCI?
• Is the validity of the work adequately demonstrated?
• Is the work clearly and concisely described?

Let's look at each of these in turn.

The role of CHI conferences has always been to present new and important contributions to HCI. This year the emphasis on presenting contributions has been made more explicit: papers are expected to offer contributions that clearly advance the field of HCI in some way. Thus your paper may contribute a new interaction technique, or a more advanced theory, or a guideline for improving usability, or experience gained during the design of an interactive system. A complete list of the types of contribution your CHI paper may offer is given at the end of this Guide and in the Call for Papers. Orient your paper towards one of these types of contribution.

If you are in a position to make a contribution to HCI, there will be people who stand to benefit. For example, if you are presenting a new interaction technique for small screens, it will be of potential use to people involved in developing handheld computers. A case study describing how you developed a new interactive product will probably catch the eye of HCI educators looking for teaching materials. As you write your paper, keep in mind the kinds of people you think might benefit from reading it.

As always, CHI is looking for original contributions. But how original must your contribution be to gain acceptance? The answers have not really changed with the shift of emphasis to contributions. Your contribution doesn't need to be original just for the sake of it. You simply need to address: What is new and significant about the contribution? How is it different from previous contributions? How does it offer useful and relevant benefit to the CHI community? What can the community learn from this paper? Your contribution needs to be original enough to provide something in answer to all of these questions.

You should also put your work in the context of related previous work. Citing related work and describing how your work differs is a good start towards demonstrating what's new about your work. There's no need (or space) to overdo it, but reviewers frequently complained about the lack of connection with relevant work.

Sometimes authors bury the new and significant parts of the work among the many other issues crammed into the paper. With the limits of eight pages (and about 20 minutes of presentation) to describe your work, it's important to choose an appropriate focus for the paper. Think carefully about the issues you want to highlight. Keep in mind that you want the reader to identify your contribution and why it is important.

The validity of your paper's contribution needs to be adequately supported by appropriate arguments, analyses, evaluations or data. Otherwise it will be hard for readers to judge whether they can take up your ideas with confidence and expect to receive the benefits you are claiming to offer. Reviewers are therefore asked to look for evidence of the validity of the results you are presenting. Here are some suggestions on how to meet this criterion.

First of all, bear in mind that reviewers often mention issues with the work, of an obvious or important nature, that have not been addressed by the authors. They also often criticize papers for not providing enough evidence or sound reasoning for their claims. A similar concern is lack of justification for design choices and not explaining why certain design features have been included. In summary, you should not only explain what you did, but why you did it so that readers (including reviewers) can be convinced that you made appropriate choices. Explaining your choices can also stimulate more research by helping others see alternative approaches.

One way to support your contribution is to include some evaluation or assessment of the contribution. What reviewers are looking for here is evidence that you have done more than just think about the benefits that you hope the CHI community will gain from your work ¾ you have tried to confirm that these benefits are really there. And it is accepted that some types of contribution are easier to evaluate than others. For example, if you have developed a new type of menu, capable of reducing errors in selecting menu items, you will probably have run an experiment to measure error rates with this and other types of menu. If you have come up with an innovative system to support collaborative writing, you may have tried to evaluate it in real-world conditions, offering it to a group of co-authors for use in a joint writing task, and conducting studies to determine how the system helps them. If you have developed a new system design methodology, you may have found it hard to compare with other methodologies, but you will probably want to report on your experiences in using it. Any evaluations like these, conducted to convince yourself that you've got something of benefit to the HCI community, can be used to convince your reviewers and readers.

Finally, in those cases where you have collected data for analysis, an appropriate methodology should have been selected and correctly applied to support the work. Authors should provide sufficient data and/or well-supported arguments, explain what analyses were made and why, cite relevant work, and cover the important issues at the appropriate level of detail.

You would be surprised at the number of reviewer comments that indicate that the work was not adequately described. Usually the authors simply did not take enough care in describing what they did. Describing the work involves not only good writing, but also a solid structure that helps the reader follow the explanation. The text should be supported with figures, tables and even videos; these should be clear and easy to understand. Authors should write clearly and concisely (avoiding jargon), organize their paper to flow logically and smoothly, provide the right level of detail, and make good use of figures to support the text.

Although all presentations at CHI are made using English, CHI is a conference with an international audience (and an international panel of reviewers). Keeping a global perspective also means using the English language in a way that effectively communicates across cultural boundaries. For authors who are not native speakers of English, reviewers try to assess the quality of the work independent of language issues. However, it is in your best interest to communicate as clearly and effectively as you can in English. If a native English speaker is available, it is a good idea to ask them to proofread your paper before you submit it. For those whose first language is English, it helps to keep in mind that non-native English speakers will be reading and reviewing the paper. It is best to avoid long, complex sentences as well as regional colloquialisms, jokes or puns that are difficult for someone outside your culture to understand.

It's important to note that CHI reviews papers on an as-is basis. The review process does not include enough time for a second review after the author has made requested changes, so reviewers must make a decision whether the submission in its current form is an acceptable CHI publication (however, reviewers often do make suggestions, and authors of accepted papers are encouraged to revise their work before a final draft is required). Try to avoid describing future work, or work expected to be completed before the conference. Although these planned activities are often interesting, you cannot rely on it to gain acceptance to the conference. Reviewers often comment that a submission is premature and should be resubmitted when more of the work has been completed.

Starting with CHI 99, the same basic review criteria will apply to all papers. How significant is the paper's contribution, and what benefit does it offer to others in HCI? Are the results valid? Is the work original? Is it well presented? The aim is make sure that all papers are reviewed equally carefully, without bias towards any one kind of paper. These review criteria are explained in the Call for Papers. They hinge on the contribution that the paper offers; for CHI 99 the following kinds of contribution may be submitted:

• Designs for interactive systems
• Interaction techniques
• Tools for use in interactive system design and development
• Methodology: techniques, methods and development processes for use in the interactive systems development life cycle
• Theories and theory-based models relevant to the design and development of interactive systems
• Guidelines and design heuristics
• Empirical findings, quantitative or qualitative, relating to the design, development and use of interactive systems, or concerning the validity of related theories
• Experience gained in adapting designs and applying HCI contributions to real-world conditions, presented in the form of design briefings or case studies
• Critical reviews that focus on specific aspects of design and development of interactive systems or of other HCI work.

You will discover that each of these categories maps onto at least one type of contribution in the above list. For example, papers categorized in previous years as an "Empirical" paper probably offer a contribution that is either a guideline/heuristic, some empirical findings, or possibly an empirical validation of a theory-based model.

Given the length constraints on CHI papers, and on their presentation at the conference, you should focus primarily on one kind of contribution. Trying to juggle too many kinds of contributions in one paper often detracts from effectively communicating your focus.

With the large number of submissions that are typically received, CHI's review process is highly competitive. The intent of the review process is to bring the highest quality, most interesting, most thought-provoking papers to the conference. Doing a good job writing (and reviewing) CHI papers is a lot of work, but we hope that this work is rewarded with a high quality conference that benefits us all. We hope this document has helped give you some concrete guidelines on how to write a successful CHI submission (if not, feel free to send feedback to CHI99-PAPERS@ACM.ORG).

Best wishes, and we look forward to seeing a good collection of successful submissions this year.