BPPF 2021 : Workshop on Benchmarking: Past, Present and Future
BPPF 2021 : Workshop on Benchmarking: Past, Present and Future

BPPF 2021 : Workshop on Benchmarking: Past, Present and Future

Event Date: August 05, 2021 - August 06, 2021
Submission Deadline: April 26, 2021
Notification of Acceptance: May 28, 2021
Camera Ready Version Due: June 07, 2021




Where have we been, and where are we going? We have an amazing collection of invited speakers that can share with us first hand knowledge of how benchmarking became important in Information Retrieval, and then in speech (starting around 1975), and then in language (in 1988). Much of this history is described in this video and two 2016 Interspeech keynotes: Makhoul describes how benchmarking overcame resistance in speech in this keynote, and Jurafsky describes how this approach moved from speech to language in this keynote.

It is easier to talk about the past than the future. These days, benchmarks evolve more bottom up (such as papers with code). There used to be more top-down leadership from government (and industry, in the case of systems, with benchmarks such as SPEC). Going forward, there may be more top-down leadership from organizations like MLPerf and/or influencers like David Ferrucci , who was responsible for IBM’s success with Jeopardy, and has recently written a paper suggesting how the community should think about benchmarking for machine comprehension (To Test Machine Comprehension, Start by Defining Comprehension). Tasks such as reading comprehension become even more interesting as we move beyond English. Multilinguality introduces many challenges, and even more opportunities.

Call for Papers

We accept three types of submissions, long papers, short papers and abstracts, all following the ACL2021 style, and the ACL submission policy:

Long papers may consist of up to eight (8) pages of content, plus unlimited references, short papers may consist of up to four (4) pages of content; final versions will be given one additional page of content so that reviewers' comments can be taken into account. Abstracts may consist of up to two (2) pages of content, plus unlimited references but will not be given any additional page upon acceptance. Submissions should be sent in electronic forms, using the Softconf START conference management system. The submission site will be announced on the workshop page once available.

We invite original research papers from a wide range of topics, including but not limited to:

  1. What important technologies and underlying sciences need to be fostered, now and in the future?
  2. In each case, are there existing tasks/benchmarks that move the field in the right direction?
  3. Where are there gaps?
  4. For the gaps, are there initial steps that are accessible, attractive, and cost effective?
  5. How large should a benchmark be?
    1. How much data do we need to measure significant differences?
    2. How much data do machines need to obtain good performance?
    3. How much data do babies need to learn language?

Best Deals


BPPF 2021 : Workshop on Benchmarking: Past, Present and Future will take place in Bangkok. It’s a 2 days event starting on Aug 05, 2021 (Thursday) and will be winded up on Aug 06, 2021 (Friday).

BPPF 2021 falls under the following areas: NLP, etc. Submissions for this Workshop can be made by Apr 26, 2021. Authors can expect the result of submission by May 28, 2021. Upon acceptance, authors should submit the final version of the manuscript on or before Jun 07, 2021 to the official website of the Workshop.

Please check the official event website for possible changes before you make any travelling arrangements. Generally, events are strict with their deadlines. It is advisable to check the official website for all the deadlines.

Other Details of the BPPF 2021

  • Short Name: BPPF 2021
  • Full Name: Workshop on Benchmarking: Past, Present and Future
  • Timing: 09:00 AM-06:00 PM (expected)
  • Fees: Check the official website of BPPF 2021
  • Event Type: Workshop
  • Website Link:
  • Location/Address: Bangkok

Credits and Sources

[1] BPPF 2021 : Workshop on Benchmarking: Past, Present and Future

Check other Conferences, Workshops, Seminars, and Events


MWE 2021: 7th Workshop on Multiword Expressions
Bangkok, Thailand
Aug 05, 2021
MLNLP 2021: 2nd International Conference on Machine Learning Techniques and NLP
Copenhagen, Denmark
Sep 18, 2021
NLPA 2021: 2nd International Conference on Natural Language Processing and Applications
Copenhagen, Denmark
Apr 24, 2021
STARSEM 2021: The 10th Joint Conference on Lexical and Computational Semantics
Bangkok, Thailand
Aug 05, 2021
JDSA-OID 2021: Special Issue on Online Information Disorder: Fake News, Bots, and Trolls