What Went Wrong and Why: Lessons from AI Research and Applications

-

AAAI-08 Workshop on

What Went Wrong and Why: Lessons from AI Research and Applications

CALL FOR PAPERS

Submissions due: April 7th, 2008

Bugs, glitches, and failures shape research and development by charting the boundaries of technology; they identify errors, reveal assumptions, and expose design flaws. When a system works we focus on its input/output behavior, but when a problem occurs, we examine the mechanisms that generated behavior to account for the flaw and hypothesize corrections. This process produces insight and forces incremental refinement. In a sense, failures are the mother of necessity, and therefore the grandmother of invention.

Unfortunately, bugs, glitches, and failures are rarely mentioned in academic discourse. Their role in informing design and development is essentially lost. The first What Went Wrong and Why workshop during the 2006 AAAI spring symposium [1,2] started to address this gap by inviting AI researchers and system developers to discuss their most revealing bugs, and relate problems to lessons learned. Revised versions of the articles and the invited talks will be published as a special issue of the AI-Magazine in Summer 2008 [3].

The first workshop clarified that WWWW experiences can be studied at three different levels of abstraction: the Strategic (AI research in general), Tactical (research area) and Execution (project or implementation) levels. An additional category turned out to be the study of how, why and when failures occur in the first place.

The second workshop will continue our analysis of failures in research. In addition to examining the links between failure and insight, we would like to determine if there is a hidden structure behind our tendency to make mistakes that can be utilized to provide guidance in research.

As such, we invite researchers to submit papers (8 pages in AAAI format) connecting problems they have encountered to lessons learned on the tactical or execution level. We would also welcome papers on the study of failures themselves. We encourage authors to elaborate on what they believe was the source cause of the failure, how the problem helped them arrive at a better solution, and to suggest a broader categorization of failures and how to utilize them. Papers should be submitted to submission@whatwentwrongandwhy.org

Important Dates

* Submissions Due: April 7, 2008

* Notifications: April 21, 2008

* Final Papers Due: May 5, 2008

* Workshop: July 13 or 14, 2008 (TBA) in Chicago at AAAI 2008


Chairs: Mehmet H. Göker and Daniel Shapiro

Mehmet H. Göker, PricewaterhouseCoopers, CAR, (mehmet.goker@us.pwc.com)

Daniel Shapiro, CSLI/Stanford University, & Applied Reactivity, Inc. (dgs@stanford.edu)

Program Committee

David Aha (Naval Research Laboratory), Ralph Bergmann (Universität Trier, Lehrstuhl für Wirtschaftsinformatik II), Carl Hewitt (MIT EECS - emeritus), Jean-Gabriel Ganascia (University Pierre et Marie Curie, LIP6), David Leake (Indiana University, Computer Science Department), Doug Lenat (Cycorp Inc.), Ramon Lopez de Mantaras (CSIC Artificial Intelligence Research Institute), Edwina Rissland (University of Massachusetts Amherst, Department of Computer Science), Ted Senator (SAIC).

References:

[1] Shapiro, D., Göker, M. (eds.), 'What Went Wrong and Why: Lessons From AI Research and Applications', Papers from the AAAI Spring Symposium, March 27-29, 2006, Stanford, CA. Technical Report SS-06-08, AAAI Press, Menlo Park, 2006.

[2] A. Abdecker, R. Alami, C Baral, T. Bickmore, E. Durfee, T. Fong, M. Göker, N. Green, M. Liberman, C. Lebiere, J. Martin, G. Mentzas, D. Musliner, N. Nicolov, I. Nourbakhsh, F. Salvetti, D. Shapiro, D. Schreckenghost, A. Sheth, L. Stojanovic, V. SunSpiral, R. Wray, "AAAI Spring Symposium Reports" , AI Magazine, VOl 27, Nr. 3, Fall 2006, pp. 107-112, American Association for Artificial Intelligence (AAAI), Menlo Park, 2006

[3] Shapiro, D. Göker, M. (eds.), 'Special Issue on What Went Wrong and Why", AI Magazine, Vol. 29, Number 2, Summer 2008 (to appear)

Comments

Post a Comment

Popular posts from this blog

(revised on Medium) On a sachet of brown sugar (series: notes to myself)

-
Image
Mid-morning, somewhere in Italy. Seating at a bar, in an elegant square, summery weather, a kind breeze, refreshing. An espresso macchiato comes with a brown sachet of sugar, to be added if you wish. I look at it. I look around. I have time to think. How do people deal with this task? Can one divide people, the sugary kind, into types according to how they open a sachet of sugar? A student, with interesting tattoos on her left shoulder, opens it by pinching the two sides and pulling them, gently ungluing the sachet. It requires her to use four fingers, each couple pinching a side of the sachet. She is attentive and careful. She better be. I know that feat is demanding. Too much force and the sachet may give in abruptly, spreading the sugar everywhere. Too little, and the little shell will hold on to its content and won’t be convinced to release its treasure. I love the cautions determination. She puts her hands together, palms touching at the bottom as if praying, to measure the effort...
Read more

Breve commento su "Non è il mio lutto" e la morte di Berlusconi.

-
Image
Ho sempre criticato Berlusconi, i suoi votanti, il suo partito, il berlusconismo, gli italoforzuti, i suoi governi, la tragedia (soprattutto morale, ma anche istituzionale, politica, culturale ed economica) che tutto ciò è stato per il paese... Berlusconi è stato un disastro per l'Italia, una sorta di Trump al potere o vicino al potere per decenni. Servirà moltissimo lavoro e tanto tempo per riparare i danni fatti, ammesso che si possano riparare.  Ma Mercoledì 14 Giugno ho spostato la lezione del corso che tengo a Bologna, affinché chi volesse potesse participare ai funerali di stato e commemorare la morte di Berlusconi, e perché l'attuale governo ha proclamato il lutto nazionale per un politico che (con mio scandalo e vergogna) è stato eletto democraticamente da milioni di italiani e italiane (che gli dei dell'Olimpo li perdonino, io non ci riesco, e posso solo rispettarne le scelte), e ha ricoperto la carica di primo ministro quattro volte.  Non è stata una forma di risp...
Read more
-
I just published "On asking the wrong kind of questions (series: notes to myself)", happy 2024. https://link.medium.com/thDMgqTvZFb
Read more

Onlife: Sulla morte di Corman McCarthy e "the best writers" della letteratura americana

-
Image
Corman McCarthy è stato descritto da alcuni come il più grande scrittore americano di [aggiungere qui un numero di secoli, oppure "la nostra epoca", oppure "una generazione" o semplicemente "sempre"]. Siccome sono insipiente, e lo avevo solo sentito nominare ma mai letto (ma i mie amici come mai non me lo hanno mai consigliato? che ci stanno a fare? O almeno Amazon, visto il numero di libri che compro, un "you may also like" lo poteva sputare no?), sono andato a vedere chi fosse. Ho letto un po' di cose su di lui, tutte molto interessanti. E poi mi sono chiesto chi rientra oggi in una lista più o meno accurata degli scrittori americani più importante degli ultimi 100 anni. Insomma, se uno dice, per esempio, "è morto il più grande scrittore americano del secolo" ti viene il dubbio di ricostruire la competizione gli ultimi cento anni. Le posizioni sono ovviamente e come sempre discutibili, ma mi è stato utile ricordare a chi si sta co...
Read more

Sulla morte come "distanza che si apre nella vita"

-
Image
“La morte è una distanza che si apre nella vita, ma non è sparizione, distruzione, putrefazione.” Ho letto questa frase (chiamiamola non-P), è parte di un articolo giornalistico (non si può imporre un rigore scientifico, non è il contesto). Ho letto il resto dell’articolo. Non migliora. Non ha importanza. Desidero soffermarmi solo su questa frase. Da tanti anni non sono più un filosofo analitico, ma mi è rimasta l’inclinazione (il vizio senza pelo del lupo?) di (cercare di) usare le parole con attenzione e cura. Cercando di ancorarle a significati che non siano “futuristici” (parole in libertà). Troppe persone confondono il fatto che una frase sia grammaticale con il fatto che abbia un reale senso. Si può anche dire: P) “La morte non è una distanza che si apre nella vita, è sparizione, distruzione, putrefazione.” P è altrettanto grammaticale e credo che abbia molto più senso di non-P, almeno biologicamente e concettualmente. I fatti puntano in questa direzione. E chi sostiene il contra...
Read more

The Loebner Prize from a judge's perspective

-
Image
This year, for the first time in its history, the Loebner Prize competition was held in England, at the University of Reading to be precise. It was organised by Kevin Warwick and Huma Shah. Independently of whether Turing might have been pleased (he was not well treated in this country, recall?), there was a satisfying sense of “coming home” of the Turing Test (henceforth TT ). Expectations were high, and they very highly advertised too. The meeting was perfectly organised. Having been invited to play the role of a judge, together with several other colleagues, including two members of the IEG , Mariarosaria Taddeo and Matteo Turilli ( here are their pictures and Rosaria's interview ) , I enjoyed the opportunity to see from close-up the machinery and the TT . It was intriguing and great fun. Because there were interviews with the BBC and other things going on, and because we were also supposed to take part in the parallel AISB Symposium on the TT , I had time to test only...
Read more

On the importance of being pedantic (series: notes to myself)

-
Image
They say that to call someone pedantic is an insult. I'm not so sure. True, someone pedantic is obsessed with minor details, small errors, or tiny imperfections. And a pedantic is also someone who cares too much about all such things not to let you know about them, advising or correcting, disagreeing or disapproving. And yes, if you are a pedant, you are more than just occasionally pedantic. This can happen to anybody, the excessive emphasis on some narrow or boring detail being a tendency we all share when it comes to matters about which we care very much. But if you are a pedant, being pedantic is your daily stance, your intrinsic nature, your way of living. You are always, consistently, reliably, systematically pedantic, from the moment you wake up, about the exact place where the slippers should be next to your bed, all the way to the moment you go to sleep, and the exact place where the phone should be placed to recharge. If you are a true pedant, no detail is too trivial, no ...
Read more

On the art of biting one's own tongue (series: notes to myself)

-
The art of biting one's own tongue consists in the ability not to engage when someone says something unpleasant, untrue, malicious, or abusive about you. Instead of answering a biased question, arguing against a ludicrous opinion, complaining about an abusive message, correcting a meaningless error, countering a fallacy, explaining a patent mistake, objecting to a groundless criticism, rectifying a willful misrepresentation, rejecting an insinuation, responding to a provocation, retorting to a nasty remark, replying to an offensive allegation, … in short, instead of engaging with your mindless interlocutors you simply ignore them and do absolutely nothing, not even acknowledging that you might have received their communication, not even sharing a “no comment”, just silence. As far as they know, you might have never got the email, read the tweet or the Facebook comment, seen the Instagram picture. If you bite your own tongue appropriately, for them their communication might have nev...
Read more

Mind the app - considerations on the ethical risks of COVID-19 apps

-
[22 April update: at the Digital Ethics Lab (OII, University of Oxford) we have elaborated a list of 16 questions to check whether an app is ethically justifiable, the full article, open access, is available here ] There is a lot of talk about apps to deal with the pandemic . Some of the best solutions use the Bluetooth connection of mobile phones to determine the contact between people and therefore the probability of contagion. In theory, it may seem simple. In practice, there are several ethical problems, not only legal and  technical ones . To understand them, it is useful to distinguish between the validation and the verification of a system.  The validation of a system answers the question: "are we building the right system?". The answer is no if the app is illegal,  for example, the use of an app in the EU must comply with the GDPR; mind that this is necessary but not sufficient to make the app also ethically acceptable, see below; is unnecessar...
Read more