What Went Wrong and Why: Lessons from AI Research and Applications

-

AAAI-08 Workshop on

What Went Wrong and Why: Lessons from AI Research and Applications

CALL FOR PAPERS

Submissions due: April 7th, 2008

Bugs, glitches, and failures shape research and development by charting the boundaries of technology; they identify errors, reveal assumptions, and expose design flaws. When a system works we focus on its input/output behavior, but when a problem occurs, we examine the mechanisms that generated behavior to account for the flaw and hypothesize corrections. This process produces insight and forces incremental refinement. In a sense, failures are the mother of necessity, and therefore the grandmother of invention.

Unfortunately, bugs, glitches, and failures are rarely mentioned in academic discourse. Their role in informing design and development is essentially lost. The first What Went Wrong and Why workshop during the 2006 AAAI spring symposium [1,2] started to address this gap by inviting AI researchers and system developers to discuss their most revealing bugs, and relate problems to lessons learned. Revised versions of the articles and the invited talks will be published as a special issue of the AI-Magazine in Summer 2008 [3].

The first workshop clarified that WWWW experiences can be studied at three different levels of abstraction: the Strategic (AI research in general), Tactical (research area) and Execution (project or implementation) levels. An additional category turned out to be the study of how, why and when failures occur in the first place.

The second workshop will continue our analysis of failures in research. In addition to examining the links between failure and insight, we would like to determine if there is a hidden structure behind our tendency to make mistakes that can be utilized to provide guidance in research.

As such, we invite researchers to submit papers (8 pages in AAAI format) connecting problems they have encountered to lessons learned on the tactical or execution level. We would also welcome papers on the study of failures themselves. We encourage authors to elaborate on what they believe was the source cause of the failure, how the problem helped them arrive at a better solution, and to suggest a broader categorization of failures and how to utilize them. Papers should be submitted to submission@whatwentwrongandwhy.org

Important Dates

* Submissions Due: April 7, 2008

* Notifications: April 21, 2008

* Final Papers Due: May 5, 2008

* Workshop: July 13 or 14, 2008 (TBA) in Chicago at AAAI 2008


Chairs: Mehmet H. Göker and Daniel Shapiro

Mehmet H. Göker, PricewaterhouseCoopers, CAR, (mehmet.goker@us.pwc.com)

Daniel Shapiro, CSLI/Stanford University, & Applied Reactivity, Inc. (dgs@stanford.edu)

Program Committee

David Aha (Naval Research Laboratory), Ralph Bergmann (Universität Trier, Lehrstuhl für Wirtschaftsinformatik II), Carl Hewitt (MIT EECS - emeritus), Jean-Gabriel Ganascia (University Pierre et Marie Curie, LIP6), David Leake (Indiana University, Computer Science Department), Doug Lenat (Cycorp Inc.), Ramon Lopez de Mantaras (CSIC Artificial Intelligence Research Institute), Edwina Rissland (University of Massachusetts Amherst, Department of Computer Science), Ted Senator (SAIC).

References:

[1] Shapiro, D., Göker, M. (eds.), 'What Went Wrong and Why: Lessons From AI Research and Applications', Papers from the AAAI Spring Symposium, March 27-29, 2006, Stanford, CA. Technical Report SS-06-08, AAAI Press, Menlo Park, 2006.

[2] A. Abdecker, R. Alami, C Baral, T. Bickmore, E. Durfee, T. Fong, M. Göker, N. Green, M. Liberman, C. Lebiere, J. Martin, G. Mentzas, D. Musliner, N. Nicolov, I. Nourbakhsh, F. Salvetti, D. Shapiro, D. Schreckenghost, A. Sheth, L. Stojanovic, V. SunSpiral, R. Wray, "AAAI Spring Symposium Reports" , AI Magazine, VOl 27, Nr. 3, Fall 2006, pp. 107-112, American Association for Artificial Intelligence (AAAI), Menlo Park, 2006

[3] Shapiro, D. Göker, M. (eds.), 'Special Issue on What Went Wrong and Why", AI Magazine, Vol. 29, Number 2, Summer 2008 (to appear)

Comments

Post a Comment

Popular posts from this blog

On the importance of being pedantic (series: notes to myself)

-
Image
They say that to call someone pedantic is an insult. I'm not so sure. True, someone pedantic is obsessed with minor details, small errors, or tiny imperfections. And a pedantic is also someone who cares too much about all such things not to let you know about them, advising or correcting, disagreeing or disapproving. And yes, if you are a pedant, you are more than just occasionally pedantic. This can happen to anybody, the excessive emphasis on some narrow or boring detail being a tendency we all share when it comes to matters about which we care very much. But if you are a pedant, being pedantic is your daily stance, your intrinsic nature, your way of living. You are always, consistently, reliably, systematically pedantic, from the moment you wake up, about the exact place where the slippers should be next to your bed, all the way to the moment you go to sleep, and the exact place where the phone should be placed to recharge. If you are a true pedant, no detail is too trivial, no
Read more

Mind the app - considerations on the ethical risks of COVID-19 apps

-
[22 April update: at the Digital Ethics Lab (OII, University of Oxford) we have elaborated a list of 16 questions to check whether an app is ethically justifiable, the full article, open access, is available here ] There is a lot of talk about apps to deal with the pandemic . Some of the best solutions use the Bluetooth connection of mobile phones to determine the contact between people and therefore the probability of contagion. In theory, it may seem simple. In practice, there are several ethical problems, not only legal and  technical ones . To understand them, it is useful to distinguish between the validation and the verification of a system.  The validation of a system answers the question: "are we building the right system?". The answer is no if the app is illegal,  for example, the use of an app in the EU must comply with the GDPR; mind that this is necessary but not sufficient to make the app also ethically acceptable, see below; is unnecessary, f
Read more

Call for expressions of interest: research position for a project on Digital Sovereignty and the Governance, Ethical, Legal, and Social Implications (GELSI) of digital innovation.

-
Image
THIS CALL IS NOW CLOSED Call for expressions of interest: research position for a project on Digital Sovereignty and the Governance, Ethical, Legal, and Social Implications (GELSI) of digital innovation. The Centre for Digital Ethics of the University of Bologna https://centri.unibo.it/digital-ethics/en invites expressions of interest (EOI) from early career researchers, with a relevant Master or PhD degree, interested in developing a research project in the field of Digital Sovereignty and the Governance, Ethical, Legal, and Social Implications (GELSI) of digital innovation. The EOI should include three documents (all in English): 1. a letter (ca. 500 words), including relevant details about personal motivation, relevant skills, experience, and a link to a Google Scholar profile; 2. a short CV (max 3 pages), including a list of publications; and 3. a short (ca. 1,000 words) outline of the proposed research project about digital sovereignty and the GELSI of digital innovat
Read more

Il sapore della felicità condivisa

-
Il 19 marzo del 1850, Charlotte Brönte scrive una lettera a William Smith Williams, l’editore presso Smith Elder che per primo aveva riconosciuto in lei una straordinaria scrittrice. Williams le ha inviato alcuni libri da leggere, e lei, nel ringraziarlo, usa una frase divenuta famosa e che oggi troviamo citata ovunque: “la felicità non condivisa può a malapena essere chiamata felicità. Non ha sapore/gusto.” (Happiness quite unshared can scarcely be called happiness—it has no taste). Per le persone misantrope e asociali, o anche semplicemente un po’ solitarie, che ritengono che la solitudine sia l’unica felicità (beata solitudo, sola beatitudo), l’affermazione sembra del tutto errata. Eppure, è difficile non riconoscere che esiste un tipo di felicità che richiede la sua condivisione per essere goduta a pieno e non restare incompleta. Non si tratta della felicità intesa come contentezza. Come si sa, questa fa riferimento allo stare bene con se stessi e con la propria vita, in pace con l
Read more

On the art of biting one's own tongue (series: notes to myself)

-
The art of biting one's own tongue consists in the ability not to engage when someone says something unpleasant, untrue, malicious, or abusive about you. Instead of answering a biased question, arguing against a ludicrous opinion, complaining about an abusive message, correcting a meaningless error, countering a fallacy, explaining a patent mistake, objecting to a groundless criticism, rectifying a willful misrepresentation, rejecting an insinuation, responding to a provocation, retorting to a nasty remark, replying to an offensive allegation, … in short, instead of engaging with your mindless interlocutors you simply ignore them and do absolutely nothing, not even acknowledging that you might have received their communication, not even sharing a “no comment”, just silence. As far as they know, you might have never got the email, read the tweet or the Facebook comment, seen the Instagram picture. If you bite your own tongue appropriately, for them their communication might have nev
Read more

Gauss Professorship

-
Image
Every year, the Göttingen Academy of Sciences (Akademie der Wissenschaften zu Göttingen) elects a Gauss Professor (Gauß-Professur der Akademie der Wissenschaften zu Göttingen) to honour the memory of the great scientist . Normally, the Professorship is awarded to an "importat scientist in one of Gauss' fields of interest". So, no philosopher has been elected so far. They just made an exception. Next year, I will be their Gauss Professor, in recognition of my work on the philosophy of information. An overwhelming honour, but very exciting. No teaching, complete freedom of research and great support. Shame on me if I do not make the most of such a privilege. I better start revising my German.
Read more

On Philosophy's envy of her four sisters (series: notes to myself)

-
Image
Philosophy has four sisters: Mathematics, Poetry, Religion, and Science. She envies all of them. She would like to be as powerfully abstract, and rigorously demonstrative, and practically useful, and timeless ... as she believes Mathematics to be. But failing, she ends up calculating the qualitative, formalising the undefinable, inferring the trivial, and missing the obvious: useless, empty, self-referential. She would like to be as delicately perceptive, and existentially fine-tuned, and meaningfully profound, and experientially rich ... as she believes Poetry to be. But failing, she ends up blathering obscure intuitions, prophesying in an esoteric language, playing with words while disrespecting facts and reasonings, obsessed with invented etymologies but oblivious of actual meanings: unhinged, nonsensical, unaccountable. She would like to be as methodologically driven, and systematically explanatory, and epistemically foundational, and progressively cumulative ... as she believes
Read more

On Pascal and the door of a Church (series: notes to myself)

-
Image
We live in a tiny village. Green fields and trees anywhere you look, apart from the local Church and its graveyard, our quiet neighbourhood.  The other day I went to visit it. I had been planning to meditate inside it, surrounded by symbols, evidence of other people's faith, of their beliefs in transcendence and supernaturalism. A step on Pascal's road, I had thought. Only, in my case, to the re-acquisition of faith, not its acquisition or protection. Keep trying and you will believe, apparently. Well, it turns out to be way more difficult in practice than when you read it in a book. It feels like wanting to believe that the world is different from what it screams to be: messy, accidental, random, chaotic, godless, so pervasively and bottomlessly historical.  I thought I could get inside, in the silent space of the navel of the Church, spend a moment reflecting, and maybe record some thoughts, like I'm doing now. While walking to the Church, I even wondered whether some loc
Read more

Rate and Rank

-
Image
Good and bad days make no news and are soon forgotten. But the best or worst moments in you life are memorable, worth mentioning and indeed ranking. The same applies to books and DVDs, restaurants and hotels, sofas and lamps, jokes and quotes. We love ranking because it’s fun and because it takes away the unpleasant doubt that accompanies every daily choice. It’s a mental-energy short-cut that can make you laugh (“what’s the most embarrassing thing Bush ever said?”) or get you through the roundabouts of life more smoothly. “This is the best fridge you can buy for that price” doesn’t get any straighter ( http://www.pricerunner.co.uk/ ). Ranking used to be done with friends in a pub or other social occasions, but the Web is clearly the perfect arena for the ranking aficionados. They can go global, harness whole databases and never miss a niche of interest. Web ranking has transformed the word of mouth to a word of mouse. And with the ease and transparency of the web, there emerges a soci
Read more

Between a rock and a hard place: Elon Musk's open letter and the Italian ban of Chat-GPT

-
Image
Interview by Adele Sarno for HuffPost, the Italian original is here The following English translation is provided by Google, apologies for any imprecision. Luciano Floridi, the digital philosopher, works between Oxford and Bologna, from next summer, he will leave Oxford to direct the Center for Digital Ethics at Yale. He has received the highest honour granted by the Italian Republic: Cavaliere di Gran Croce. According to the Elsevier Scopus database, he is the most cited living philosopher in the world. If today we talk about the "philosophy of information", it is thanks to him, who for 30 years, studied the connections between philosophy and the digital world. Professor Floridi, ChatGPT has been at the centre of the debate, especially these days. Elon Musk and a thousand other experts have written a letter asking for its development to be stopped for six months. In Italy, on the other hand, the privacy guarantor has decided to stop for 20 days until it complies with the pr
Read more