Publishing in a Cybersecurity/Cryptography Conference: Which Are Best? Here’s AR, PR and CR

Publishing in a Cybersecurity/Cryptography Conference: Which Are Best? Here’s AR, PR and CR

I know it’s an extreme viewpoint, but there is a some truth in this:

While other industry may see sales targets, or customer engagement measures as PKIs; in research, we are often assessed in terms of the quality of our research outputs. This often relates to citation measures — and which can give a measure of the quality of a paper and its novelity/contribution. A poor quality paper can often lead to just a few citations, and where there is little interest in the methods proposed. A high impact paper, though, can gain 100s of citations within a relatively short time period. Overall, some of the best papers can gain 1,000s of citations, and often change the course and direction of research work — they make an impact.

And, so, one of the most confusing things for anyone new to research is where, when and what they should publish. Should they publish 10 papers in low-quality journals or one great paper in a high-quality venue? Well, a good deal of this relates to the peer review process, and the general impact of the venue. Many would say that many weak papers are now accepted in poor-quality journals, and you could basically write up work that has very little novelty, and for it to be published. So, many researchers give the advice that quality is often better than quantity — especially when we have poor quality outputs.

If you are publishing in a cybersecurity conference, this article (sorry about it not being HTTPs — not great for an article on cybersecurity) outlines the ranking of academic conferences [here]:

As expected, those conferences which have been around for a while, and which have built up a strong impact, are the top ones. Overall, IEEE S&P (Symposium on Security and Privacy) is top, followed by USENIX Security, NDSS (Network and Distributed System Security (NDSS) Symposium), ACM CCS (Conference on Computer and Communications Security) and Eurocrypt.

These are ranked for CIF (Conference Impact Factor) and three ratios: AR (No. of accepted papers / No. of submissions) PR (No. of accepted papers / No. of participants); and CR (No. of accepted papers / No. of citations). A lower value is the best for the factors.

For AR, IEEE S&P has 591 submissions and only 76.5 papers accepted (12.5%) — an acceptance rate of just 1 in 8. It must be remembered, too, that because of the prestige of the conference, many of the 591 submissions are likely to be of high quality (and which would often be accepted by many journals).

With PR, S&P has 76.5 papers accepted, but 797.7 eager participants. With many conferences, the audience is often made up of those who have published papers, and so the quality of these types of conferences means that papers can be accepted so that the organisers can cover their costs (or even make a profit). Unfortunately, there are many conferences like this (and often in sunny and warm places in the world — and in the Summertime). We can see with S&P that we have a ratio of 9.6%, and where there are more than 10 times more attendees than papers published, and where many are there because they want to hear about the best research.

With CR, we have the number of accepted papers against the number of citations. This is a key quality metric for many researchers and relates to how visible the paper is, and its general quality. A weak conference often produces very few citations, as the work is unlikely to be picked up and read, or where there are weak standards for peer review. A good quality conference values novelty and contribution highly and gets good reviews (as it is prestigious to review for a high-quality conference — and can be used as a measure of esteem). For S&P, we see a CR of 3.4%, and where the average number of citations for a paper is around 30 — which most researchers would be happy to achieve, and see their paper as having a strong impact.

And, how do we assess a journal against a conference. Well, Google Scholar can assess for the h5-index/h5-median [here]:

We see that the top two are journal, but these are followed by S&P and USENIX, and then another journal (Computer & Security), and then by a few conferences. So the thought that journals are better than conferences is not quite the case for these top tier conferences.

It should be remember, the survey papers can often pick up large amounts of citations, but provide little in the way of novelty. So, many conferences often reject survey papers, and focus purely on those that make a scientific contribution. So, a high quality conference publications is often actually a better assessment of the quality of a paper than a journal (which will often have a much higher acceptance rate and also can accept survey papers).

Conclusions

A paper stands on its own. And if it is a great paper, it is a great paper. But, with weak standards around for the peer review, the quality of the venue for the publication often has a strong influence on the quality of the work. So, for those new to research, think carefully about where you want to publish — and have a thick skin.

If you are looking to publish a paper in these top conferences, you have missed IEEE S&P and NDSS for 2023, but USENIX is open for Feb 2023 submission [here].

For the UK, we have just been through a research excellence assessment (REF 2021), and where research outputs are graded in terms in a star system such as for 4* (world-leading) and 3* (internationally-leading). If you are interested, here’s the ratings of the overall score in Scotland for Computer Science (UoA 11), and ranked for world-leading research outputs (and ordered for FTE staff for equal placing):

For many research-focused universities, the focus is increasingly on 4* outputs — and which show that they are working on ground-breaking work. A key element of this is the focus on high quality conferences and journals.