DIFM 2023 : First Workshop on Distributed infrastructures for foundation models

The DIFM workshop is co-located with ACM/IFIP Middleware 2023, which takes place from December 11-15 in Bologna, Italy.

Following the innovations in deep learning, foundation models (FM) are the next evolution in machine learning. Foundation models, including large language models, are driving the recent breakthroughs around conversational AI chatbots (such as ChatGPT), image generation (such as Stable Diffusion), and code assistants (such as GitHub CoPilot).

Foundation models across text, speech, and vision domains can be trained at scale using self-supervision techniques and applied to a broad set of downstream tasks. They address a limitation of deep learning models which typically required large task-specific labeled datasets. There is tremendous activity in this space from large enterprises, such as Google, Microsoft, Amazon, and IBM; and startups, such as OpenAI,Anthropic, Stability AI, and Cohere. As well there have been a flurry of papers in this space from academia and the open source community, resulting in models such as Alpaca, and efforts such as OpenAssistant.

Much of the attention has been on the models and datasets, with academic communities relying on the training and serving infrastructure provided by large cloud providers. Innovations on the infrastructure aspects have largely been taking place in closed enterprises and startups. This workshop will serve as a venue for academics and practitioners to share their findings, visions, and ideas around these infrastructure challenges and concerns. There are still challenging open problems that need attention. One piece of evidence of these challenges is OpenAI’s GPT-4 technical report. While this report is light on technical details, it includes an extensive Acknowledgements section that listed large dedicated teams focused on infrastructure aspects such as “Compute cluster scaling”, “Distributed training infrastructure”, “Hardware correctness”, “Training run babysitting”, “Deployment & post-training”, “Data infrastructure”, “Acceleration forecasting”, “Inference research”, “Inference infrastructure”, and “Reliability engineering”. This suggests the importance of middleware infrastructure to train and serve FMs, and the need for research in this space.

The scope of this workshop includes, but is not limited to:

Resource scheduling algorithms and optimizations for FM serving workloads, including batch, streaming, and synchronous invocation patterns.
Novel techniques to train large FMs.
Frameworks for fine-tuning FMs.
Programming model abstractions for FMs (such as LangChain).
Case studies of FM middleware.
Novel debugging and logging techniques, both in cases of black-box FMs available through an API, and locally available FMs.
Deployment of FMs in resource constrained environments (such as edge platforms, web browsers, and mobile devices).
Dates and location
Paper submissions: October 9, 2023
Notification to authors: October 20, 2023
Camera-ready copy due: October 27, 2023

Papers and Submissions
We are looking for the following types of submissions:
Research and industry papers (up to 8 pages): Reports on original results including novel techniques, significant case studies or surveys. Authors may include extra material beyond the six pages as a clearly marked appendix, which reviewers are not obliged to read but could read.
Position papers (up to 4 pages): Reports identifying unaddressed problems and research challenges.
Abstracts (up to 1 page): An extended abstract on a preliminary or ongoing work.
Papers must be written in English and submitted in PDF format. All papers should follow ACM formatting instructions, specifically the ACM SIG Proceedings Standard Style. The author kit containing the templates for the required style can be found at http://www.acm.org/publications/proceedings-template.

Submissions should not be blinded for review. Please submit your papers via the submission site: https://difm23.hotcrp.com/

All accepted papers will appear in the Middleware 2023 companion proceedings, available in the ACM Digital Library. All accepted papers will also be presented at the workshop, and at least one author of each paper must register for the workshop.

Workshop Co-chairs
Bishwaranjan Bhattacharjee, IBM Research
Vatche Isahagian, IBM Research
Vinod Muthusamy, IBM Research
Program Committee (Tentative)
Parag Chandakkar, Walmart Labs
Ian Foster, Argonne National Laboratory and the University of Chicago
Matthew Hill, Dataminr
Mayoore Jaiswal, Nvidia
Gauri Joshi, Carnegie Mellon University
Jayaram K. R., IBM Research
Ruben Mayer, Technical University of Munich
Pietro Michiardi, Eurecom
Phuong Nguyen, eBay
Peter Pietzuch, Imperial College
Chuan Wu, University of Hong Kong