|
--------------------------------------------------------------------------------------------
Call for Papers and Shared Task Participation
The 1st International Workshop of AI Werewolf and Dialog System (AIWolfDial2019)
Collocated with INLG 2019 conference, October 29, 2019, Tokyo, Japan
https://aiwolfdial.kanolab.net/
--------------------------------------------------------------------------------------------
( Workshop aims )
"Are You a Werewolf?", or "Mafia" (hereafter "werewolf game"), is a communication game conducted solely through discussion. Players must exert their cognitive faculties fully in order to win. In the game, players must hide information, in contrast to perfect information games such as chess or Reversi. Each player acquires secret information from other players’ conversations and behavior and acts by hiding information to accomplish their objectives. Players are required persuasion for earning confidence, and speculation for detecting fabrications.
We employ this werewolf game as a novel way of evaluations for dialog systems. While studies of dialog systems are very hot topics recently, they are still insufficient to make natural conversations with consistent context, or with complex sentences. One of the fundamental issues is a lack of an appropriate evaluation.
Because the werewolf game forces players to deceive, persuade, and detect lies, neither inconsistent nor vague response are evaluated as “unnatural”, losing in the game. Our werewolf game competition and evaluation could be a new interesting evaluation criteria for dialog systems, but also for imperfect information game theories.
In addition, the werewolf game allows any conversation, so the game includes both task-oriented and non-task-oriented conversations. This aspect would provide a handy intermediate goal rather than to create a general dialog system from scratch.
( Important dates )
Call for Workshop papers: June 30, 2019
Shared task run: end of August
Submissions due: Sep 8, 2019 (INLG main conference notification is Sep 1)
Notification of acceptance: October 1, 2019
Camera-ready papers due: October 15, 2019
( Submission )
Submission Type
We call for short papers in 2-4 pages excluding references, both for shared task papers and papers in general. Please use the ACL LaTeX style files or Microsoft Word template.
Topics
Paper topics include followings but not limited to:
- AI werewolf agents for natural language and/or protocols
- Natural language processing for dialog systems of conversation games
- Corpora, resources, analysis on conversation games
- Natural language processing for deeper syntactic and semantic communications
- Imperfect information game and natural language
- Game strategy and natural language processing
- Deceiving and persuasion by automatic agents
- Middle-language for task-oriented dialog systems
- Evaluation of dialog systems using games
- Instructions for authors
Submission site will open in August.
( Shared task )
Background
We have been annually holding the AI werewolf contests under the AI Werewolf project. The AI werewolf contest has two divisions, the protocol division and the natural language division. The protocol division asks participants to implement an AI werewolf player agent that communicates in a middle language called the AI werewolf protocol. The natural language division asks participants to implement an AI werewolf agent that communicates in natural language. We follow the previous configurations of the natural language division in our AIWolfDial2019 shared task.
The Werewolf Game in this Shared Task
Agents of the AIWolfDial2019 shared task will play the werewolf game of five players, including roles of a seer, a werewolf, a possessed, and two villagers. Players do not know other players' roles. All players, other than the werewolf, are humans.
A game consists of a couple of days, continuing until the human team or the werewolf team survives. A werewolf can specify and attack another player in the end of the day; the attached player will be eliminated from the game. All of players are required to vote to another player, and a player voted most will be eliminated from the game. When humans survive, the villager team wins. When a werewolf survives, the werewolf team wins. A possessed is a human but belongs to the werewolf team. A seer can specify another player in the end of the day, then either human or werewolf is notified.
Language Requirement
A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that communicate either in English or Japanese. Agents of the Japanese language are required to make an English version, at least by using machine translation internally.
AIWolf Agent APIs
A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that connects to our AIWolf server via network. Therefore, there is no limitation on the client side hardware/software other than the network I/O specifications. The AIWolf project provides APIs to implement your agent. http://aiwolf.org/en/server describes how to implement your agent in Java. Other libraries include .NET and Python. We will provide a sample agent code and a server to connect and test your agent.
Agent Specification
- Day 0 has greetings only.
- The end of Day 0 has an inspection by a seer, and the game starts from Day 1.
- After Day 1, the end of the days have votes by all players and an attack by a werewolf. Vote, attack, inspection are made via specific APIs (network communications).
- A day consists of a couple turns, where all of agents can make a talk for each turn, receiving talks of previous turns.
- An agent should make a talk within a specified periods after a talk request is sent.
- During days, Agents can communicate anything in natural language. A talk should consists of normal letters and punctuations only. An agent returns "Skip" when nothing to talk, returns "Over" if nothing to talk anymore in that day.
- Use Agent[0x] (e.g. Agent[05], x is 1-5) to mention other agents．
- An anchor e.g. "))Agent[0x]" could be inserted at the beginning of a talk to refer to another agent, to whom your agent with to talk with. That agent is assumed to respond something to your agent by using an anchor.
Testing Your Agent System Beforehand
We will provide an AIWolf server running, where participants can try connecting with dummy agents to check their system behavior. Participants are required to check their systems certainly work before the shared task run.
Shared Task Run and System Evaluation
A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that connects to our AIWolf server at the specified timing. We evaluate this play's logs by a couple of reviewers, performing subjective evaluations.
Registration
- A team should send a mail to aiwolf at kanolab.net (replace at by @) to register the shared task, describing your team name, a contact e-mail address, names and affiliations of your members (please mark a contact person when a team consists of multiple members). There is no fee required to register/participate the shared task.
- Participants should submit a system design description document to the organizers. This document and logs of the games might be used for research purpose and included and published in our overview paper without any further permission. Participants are encouraged to submit a paper to the workshop.
- Reviewers will perform subjective evaluations on the game logs. Games will be between agents, and/or human players.
( Committee )
Contact
E-mail to aiwolf at kanolab.net (replace at by @)
Organizers
Yoshinobu Kano, Shizuoka University, Japan
Michimasa Inaba, The University of Electro-Communications, Japan
Fujio Toriumi, The University of Tokyo, Japan
Hirotaka Osawa, University of Tsukuba, Japan
Daisuke Katagami, Tokyo Polytechnic University, Japan
Takashi Otsuki, Yamagata University, Japan
Program committee members
Claus Aranha, Tsukuba University, Japan
Hitoshi Matsubara, Future University Hakodate, Japan
TBA
|
