Second Workshop on Scientific Knowledge Creation, Dissemination, and Evaluation

From LiquidPubWiki

Jump to: navigation, search

Contents

Workshop Topics and Panel Organization

The workshop is organized as a set of presentation/demos and informal panels. Demos will show the results from the first two years of the LiquidPub project. Panels are meant to be very unstructured, with the idea of extracting everybody's more or less wild thoughts on the various topics. We will ask participants to help lead the discussions on each of the panels.

Photos

If you have some photos to share, please put a link below:

Presentation and demos

Here is the list of presentations done by participants:

  • Roberto Casati, Is this the scientific article of tomorrow?

People about the Second Snow Workshop

Wybo Wiersma's blog-post about the workshop is here

Anita de Waard and Cesare Pautasso twits are at [1] and [2]

Panel Notes

The panels ran in parallel by "moderators" on Friday morning, and then "reporters" reported on main issues discussed and main conclusions achieved (see the notes below). For each panel we started with 5-6 "statements" of 3-7 minutes each, done in powerpoint or, mostly, just telling what people think is important on the subject.

The purpose of the panels was to get perticipants' feedback on the LiquidPub plan of work and transform it into actionable ideas.

Panel A: Dissemination and discovery of scientific knowledge

Reporter: Nardine Osman, IIIA-CSIC

Moderator: Zhang Yi-Cheng, University of Fribourg

Statements:

"My interest in the open source movement and Web 2.0 to foster meaningful collaboration, drive innovation, and strenngthen the peer review process for public good research" by Darren Dahly, University of Leeds

"Developing measures of /impact/, /promisingness/ and /relatedness/ of scientific publications" by Peep Küngas, University of Tartu

"The role of scientific communities on the relevance of scientific contributions" by Luca Cernuzzi, Universidad Católica - Asunción

"Pros and Cons of Liquid Publications for Scientific Knowledge Dissemination, seen from an External Viewpoint" by Stefano Ceri, Politecnico di Milano

"Dialectic: Consensus and Presuppositions" Munindar P. Singh, North Carolina State University (IFAAMAS Board representative)

Participants:

Marcos Baez, University of Trento

Fabio Casati, University of Trento

Judith Simon, CNRS

Muhammad Imran, University of Trento

Volha Bryl, FBK

Cara Shank, University of North Carolina

Questions and Ideas

The way scientific knowledge is disseminated today is similar to how it was centuries ago. This is true despite a transition from an ecosystem where the scarce resource was related to the cost of dissemination (mainly printing and distribution) to one where the scarce resource is people's attention and time, buried by a dissemination overload.

Should we leverage the Web, advances in social networks, open source software, blogs, tags, and the various ways in which the web has impacted creation, collaboration, and selection/evaluation of "interesting" content among an ocean of contributions? Should this lead to a different way to disseminate knowledge?What kind of knowledge might be considered as scientific knowledge?

Is it true that early dissemination, more collaboration, and more feedback would benefit scientific production? What are the tools and methods that achieve this?

How do metrics bias the ways we disseminate scientific knowledge? Do metrics help us or do they stand in the way? Which metrics can be borrowed from the social web or open source development? How can we differentiate between mature and immature research? If anyone is able of sharing anything with everyone else, how can reputation metrics differentiate mature, "integral" research from preliminary ideas or contributions? And is such a distinction necessary?

Model for journals: will journals in the future be as they are today, which is also how they were before the web? Will we go towards a completely new notion of "journals"? Does it make sense to have the notion of "journal" and "issue" of a journal, where the issue is based on time of publication? Who will determine what goes into a journal and if a journal is good or not (and is this relevant)?

Is it possible to separate the knowledge production process (e.g., paper writing) from publication? Is the community capable of replacing peer-review? At what extent and in which scenarios would this be possible? How would the current scenario change if we assume there are no publishers?

As the Google example shows, digitalization might push towards monopoly. Ideally speaking we do not need many access points when storing, disseminating and searching for (discovering of) the scientific contributions. Is the unique access point to scientific research feasible? Who should be in charge of managing the scientific "search engine" that browses the web? Public institutions, private publishers? What's the role of workshops and conferences on the dissemination of scientific contributions?

Statements

Luca Cernuzzi. The role of scientific communities on the relevance of scientific contributions.

The ranking process is often used for evaluation and comparison purposes and thus it would be interesting to find out an accurate, reliable and widely-accepted method for it. Usually, more accepted ways of ranking are based on citations.

However, citation analysis has also its flaws [Seglen P.O. Citation rates and journal impact factors are not suitable for evaluation of research. Acta orthopaedica Scandinavica 69, 3 (1998), 224-229]. There are implicit factors (e.g, self-citation) that might influence the interpretation of the results using a citation-based metric. Citations are nevertheless indicators of scholarly impact. Indeed, the evaluation methods that are part of citation analysis have to be improved. Our statement is that scientific communities may influence the impact (and the relevance) of scientific contributions. Thus, to detect and analyze communities may facilitate the evaluation of individual productivity and impact by normalizing to the community, measuring interdisciplinary and broadness of researchers.

Moreover, considering scientific communities will provide the means for a better understanding of the social behavior in the scope of scientific research, identifying patterns in projects developments, research trends, successful research profiles, and perhaps improving the possible collaboration.

Peep Küngas. Developing measures of /impact/, /promisingness/ and /relatedness/ of scientific publications

The ongoing project has unveiled that there is a wealth of information available in social graphs that can help to identify interesting and promising publications early on during their stage. The richness of the patterns that can be identified with these data exceeds initial expectations. Due to its limited resources, the project has had to focus on measures of reputation and credit attribution based on collaborative filtering and scoring. But equally valuable, and to a large extent complementary are measures of popularity, impact, interestingness and promisingness that can be extracted by analyzing co-authorship graphs, citation graphs and historical download counts.

While parts of these data are used in traditional forms of quality assessment of publications, the way they are used does not fully exploit their full richness. For example, impact factors are generally calculated based on citation counts. This corresponds to simply counting the indegree of a publication in a citation graph (i.e. number of citations). Notions in the style of “page-rank”, which are being studied in activity WP4.2 in the existing project, provide more robust metrics of impact than pure citation count. However, they are based purely on the citation graph.

More importantly, mainstream measures of quality assessment are geared towards measuring the “impact” of publications a posteriori, i.e. once the paper has reached a certain age. We contend that in order improve the fluidity of the scientific dissemination process, researchers need to be able to identify, as early as possible, “interesting” and “promising” publications both in the traditional publications world and in the context of liquid publications. Such measures will be central in the liquid publication world, where “expert” peer-reviews will be replaced by community feedback.

In this project extension, we will focus on defining and implementing metrics that can be used to rank Scientific Knowledge Objects (SKOs) with respect to their level of “interest”. We will focus specifically on measures of /impact/, /promisingness/ and /relatedness/ while developing specific metrics for measuring these properties of scientific work.

Notes

  • On Sharing of Data: 
 - We need to make use of Web 2.0 to change the way we do science.
   Key challenge: how to collaborate with each other? I'm interested in web based tools for collaboration.
   As scientists we are lagging behind in the use of web 2.0. (Darren Dahly)
 - American universities are switching from open collaboration to being isolated entities. 
   Universities are more isolated from one another because of patent issues in medical sciences, protection issues, etc. (Darren Dahly)
 - E.G. Lots of effort is put in collecting the data, so researchers want to make sure they
   get the credit and be the first to publish research based on that data. (Darren Dahly)
 - Introducing Marcos' presentation:
   To what extend can we push the idea of sharing and putting everything out there?
   How can we assess what is scientific what is not scientific? (Zhang Yi-Cheng)
 - Get data from particular sources, limiting the scope.
   What we need to do is scoping our search. (Maurizio Marchese)
 - Can we really separate the availability from publications? (Marcos Baez)
 - One can put some content on his webpage, then wait for it to pop up on google scholar. 
   I like to put it in a particular well known open access repository. (Maurizio Marchese)
 - Google scholar clusters citations. (Marcos Baez)
 - What motivates people for sharing data? How do you motivate authors to present papers in a radical way? (Marcos Baez)
 - Credits and increasing one's reputation. Contributing to science. (Luca Cernuzzi)
 - But is it only about sharing data, or about collaboration? (Maurizio Marchese)
  • The sharing of data issue was triggered by the idea that peer review in science may be viewed as a process of falsificationism: 
 - It is hard to create real peer review, because it is hard to tell what the real quality of the science is. 
   Researchers may write a fake paper, be dishonest about results, etc. Basically, peer review doesn't really happen. 
   Because we cannot take data, replicate the analysis, and asses. (Darren Dahly)
 - In the ACM community (ACM SIGMOD), there is an effort called "replicablity". What was done is the following: 
   take data of papers accepted and replicate the experiments.  If replicating the experiment produced the same result, 
   then a stamp certifying your work is given. 70% of the papers were stamped, saying this has been replicated successfully. 
   To do this, however, authors should provide data and links to code. (Stefano Ceri)
 - However, we cannot be sure this "replicablity" will continue, because there is a lot of work and the effect is not too visible.
   Results either say nothing, or you get stamped. They do not say your software didn't run, for example. (Stefano Ceri)
 - So researchers will be forced to provide higher quality code, since they will be ashamed of providing poor code. (Maurizio Marchese)
 - In the database field, this could work, because you can generate data, but not in other fields. (Munindar P. Singh)
 - But true peer review is not about providing data sets, but arguing about the methodology...
   So you need to get authors to post their stuff & the community to respond. But the problem is that not much comments are usually posted, 
   except in social topics, general topics, climate change, linked to face book, etc.
   Furthermore, you'll be putting your own neck on the line if you're wrong. (Munindar P. Singh)
 - Why it cannot be like open source? All researchers working together on a product, but no need to take explicit credit. (Darren Dahly)
  • Other notes on peer review: 
 - Sharing & Peer review: the more reviewers, the more fair the reputation. 
   Can we actually rely on the community to review content? (Anonymous)
 - Major point: How can u tell good or interesting things? How can you certify that a job is good? 
   You need a lot of work, and every day less people are available for doing this. I understand this is a major problem. (Luca Cernuzzi)
 - Either we assess expertise to let them rate, or make sure there is enough people to rate (quality versus quantity). 
   Otherwise we're lost. (Judith Simon)
  • What is peer review? 
 - A real peer review for Darren is a process in which reviewers go into the details of the work, replicating data, etc. 
   In our opinion, an ideal peer review is to know whether there is an ideal ranking that you are aiming to? 
   We said the best thing you can have is if everyone reads everything, then you can come out with the ideal results. 
   The more people you have, the more accurate your results are. 
   But other groups said it is not really the ideal ranking since not all the reviewers are experts. 
   Ideal ranking is getting all EXPERTS to read all papers. This brings us back to the question: who are the experts? (Maurizio Marchese)
 - Peer review is not to find errors, but to identify assumptions that the authors have not said. 
   Pinpoint the hidden assumptions of the world. If there is a flaw, a technical error, 
   the role of the reviewer would still be to force the assumptions. (Munindar P. Singh)
 - The top goal of peer review is to rank a list of objects (paper, proposal, whatever). 
   The goal of peer review is not to find the top and the loser, but to assess those in the grey area (the critical papers) 
   to be able to make a decision on whether to accept or reject. Assessing those critical papers is what needs to be done well. 
   The goal is to have good reviewers for those papers. Then once the paper is accepted, it has its own life. 
   The community will decide with time. But it is this threshold for accepting that is crucial. (Stefano Ceri)
 - To understand peer review is to understand what science is: focusing on competition & decision making, ...? 
   Summary: What is peer review for, focusing on competition, or collaboration? (Peep Küngas)
 - In peer review, we mainly have two dimensions. 
   Dimension 1: selection and ranking. Dimension 2: providing feedback (Stefano Ceri & Maurizio Marchese)
 - In one of the conferences/workshops, we did separate the two things. Reviewers had to read lot of contributions. 
   And they were asked to rank, and not to rate papers. Then a reviewer is chosen later on to read the paper only to give suggestions. 
   This process happens after the selection. So first they read quickly, they select, then they provide feedback. (Maurizio Marchese)
 - You can have the "stamp of the world", saying this is scientific, etc. 
   But simply putting your stuff online for the people to judge, this is stronger science. (Darren Dahly)
 - Knowledge is created THROUGH interaction, THROUGH dissemination. Peer review is that process of knowledge creation. 
   The idea to separate dissemination from creation is old fashioned. 
   We should be more ambitious and focus on new ways of creating knowledge. (Munindar P. Singh)
 - In peer review most people are outsiders. It is not a beauty contest. 
   Amazon has a multilayer structure: books, reviews, authors. Reviews are based on sections. 
   Finding experts is not important because they are self appointed, since reviews themselves are also rated. (Zhang Yi-Cheng)
 - An expert might say a book is excellent. As a newbie to the subject, as a reader you might say it is not clear enough. 
   Many might find your comments more accurate. Authority in review is not that important. 
   Because there are different aspects and different requirements. (Munindar P. Singh)
 - Also, we often rely on people going around and making postings publicly. 
   But the real opinion is not usually online, but possibly shared with another. (Munindar P. Singh)
  • On the importance of community analysis: 
 - I'm interested in web services and service networks. Impact is not only related to citation networks but also the communities themselves. 
   You find an article because you're doing a research on a similar topic. 
   A lot of information is available in the social graphs. (Peep Küngas)
 - Discovering communities, overlapping communities, etc. is important.
   Idea of community is useful to have a better choice of reviewers.  
   Understanding communities helps understanding social behavior in scientific research. (Luca Cernuzzi)
 - Reputation is needed for smaller communities. (Zhang Yi-Cheng)
 - Small highly connected group is something that is good. (Maurizio Marchese)
 - Good for collaborative sharing and contribution. 
   But for impact, then we also need to consider the number of participants in the community,
   self citations, patterns and trends, etc. to help facilitate evaluation of individual productivity. (Luca Cernuzzi)
 - Metrics are interesting and useful but meaningless without information about the communities. 
   Number of papers, citations in a community: now we're talking about impact.  
   Impact in computer science has different indicators than that in philosophy. (Maurizio Marchese)
 - Sleeping beauties: articles that go unnoticed over a long period of time. 
   But having things on the web is shortening the time for citation. (Peep Küngas and Maurizio Marchese)
 - Peer review reflects how science is being done. How we do research. 
   E.G. sometimes we have to go back in time and start our research from an earlier point and do things all over again. (Peep Küngas)
  • What is the incentive for reviewers? 
 - [a note made about publishing reviews] (Peep Küngas)
 - As a reviewer, I do not want to go around and publish my reviews. (Munindar P. Singh)
 - But they want to gain a reputation (as a reviewer). (Peep Küngas)
 - A system where reviewers are public, they can view my data, and I will publish their review. (Anonymous)
 - In PLoS one, reviews are open. (Maurizio Marchese)
 - Reviews can be made public. You might reach glory (if you write good reviews), but might make enemies as well. 
   If the review is bas, you lose credit. But knowledge will be the outcome of these models. (Munindar P. Singh)
 - In any case, what is useful here about gaining credit. If I have a reputation as a good reviewer, so what? 
   How is this credit useful for me? (Maurizio Marchese)
 - It means more work (more reviews)! (Luca Cernuzzi)
 - We need on the side a system that collects this credit and makes use of it. (Maurizio Marchese)
 - In some systems, you monitor actions, and based on their contribution, 
   reviewers may be eventually be viewed more as "collaborators". (Peep Küngas)
  • On the notion of a Liquid Journal 
 - It was suggested yesterday by Anita to be called a "scientific social bookmarking". (Maurizio Marchese)
 - The basic idea: we are filtering the most debatable, the most interesting content. 
   It does not matter where it is published, but the trick is in being able to find it (discovering it). 
   Finding what is interesting is done by the community, decoupling publishing from dissimentation. (Maurizio Marchese)
 - Splitting between creation and dissemination. (Munindar P. Singh)
 - To have a service where anybody could go search, post, etc. This is a very good idea. 
   Setting processes for creation and sharing. Very good. 
   But a journal is to give someone recognition of the work. 
   So how should you start this process in order to be successful? You need to become a known and successful place for doing journal. 
   What pushes people to do journals: reputation. And reputation comes very slowly. 
   You have to find partners with recognized authority (conference ...).
   You want someone to say I'm ready to experiment and use this technology to push my own work. 
   We can get visibility and credibility (passing credibility to us). Our asset as a repository. 
   This is important because then the user will think that he could use it for his stuff. 
   When you think of a journal, you have to think of publisher, author, TOC, organisation, these has to be clear. 
   Like easy chair, it has to be used slowly by more people, and get other organizations interested in using it. (Stefano Ceri)
 - There is something we can do with this LJ platform. There are two main things here. 
   First: LJ is a new kind of a journal. We still need a publisher, credibility, etc. 
   Second: LJ is a social bookmarking. E.G. editor can be a PhD student who uses the infrastructure to find information. (Maurizio Marchese)
 - Seperate knowledge from publishing. (Stefano Ceri)
 - In LJ, the credibility in question is that of the compiler who selects and finds the interesting material. 
   Then there can be a moment when they go public and they say on this topic these are the most interesting contributions. 
   Then in the community people might want to know what you are reading. (Maurizio Marchese)
 - The Journal name is a bit misleading. I view it more as a twitter. (Stefano Ceri)
 - We propose a new way of doing journals. We say you define your own process and own kind of rules. 
   Different communities have different leaves. (Maurizio Marchese)
 - With the same technology (Liquid Journal), you can have traditional journals to very social scenarios. (Stefano Ceri)
 - But a journal is not just pointers. To me it is not a journal. (Stefano Ceri)
 - Results come from queries and are automatically updated. Maybe this can evolve into self adaptive systems? 
   You might end up heading in this direction. (Luca Cernuzzi)

Panel B: Collaborative creation of scientific knowledge

Reporter: Joseph Wakeling, CNRS

Moderator: Fausto Giunchiglia, University of Trento

Statements:

"LogiLogi: A minimalistic platform for philosophical discussions" by Wybo Wiersma, King's College London

"Working towards Structured Collaboration in Creative Environments" by Moshe Chai Barukh, University of New South Wales

Participants:

Maria Teresa Serafini, Author, RCS

Azzurra Ragone, University of Trento

Carles Sierra, IIIA-CSIC

Maurizio Marchese, University of Trento

Alfred Hofmann, Springer

Matus Medo, University of Fribourg

Questions and Ideas

How do books evolve? How can tools and Web 2.0 technologies support a new collaborative way of producing and maintaining books? How can books be shaped in the Web 2.0?

How to support authors in the creation, reuse and in the sharing of their material? How to handle the consequent problems of credit attribution and royalties?

How to produce books which are one and thousand at the same time, that is books tailored for the needs of the prospective readers? And which are constantly up-to-date?

What is the meaning of the world "publish" in this scenario? What roles do publishers and libraries would play?

What is the key value publishers provided in the past, what is the value they provide today, and what is the value they will provide in the future?

How would multifaceted books be disseminated? Is the written part of a book still the most important part within a digital environment? Who is in charge of preserving LB over time?

Statements

Wybo Wiersma, LogiLogi: A minimalistic platform for philosophical discussions

LogiLogi is a Web 2.0 application that wants to meet the need of philosophers, students, and others for in-depth, quick-turnaround, high quality discussions without taking the fun out of it by making things too complicated.

It tries to do this by finding an informal middle-road between good conversations and journal-papers by providing a form of quick, informal publication, peer-review, and annotation of short philosophical texts. It is intended for all those ideas that one cannot turn into a full sized paper, but that one deems too interesting to leave to the winds.

It does not make use of forum-threads (avoiding their many problems, such as things going off-topic), but of tags and links that can also be added to texts by others than the original author. And it features a rating-system modelled after Journal-based-review, in which well-rated texts earn authors more voting-power within their peergroup (of which there are multiple).

LogiLogi is Free Software, has been under development by between 2 and 10 people for 3 years, and a public beta is already online and fully functional at http://www.LogiLogi.org.

If we compare LogiLogi and LiquidPub we see that, though LogiLogi is not very successful at attracting users, it is relatively simple and straightforward (and this is a good thing). Where LogiLogi aims to be something small that is used at the side, LiquidPub seems to try to surpass existing practices and compete with/change/facilitate journals (and the organisation of conferences, prestige-metrices/indices, etc.)

The sizable ambition of the LiquidPub project might make it more vulnerable to the following four common problems facing software-projects:

  • Significant delays in, or termination of development because of unexpected/under-estimated technological complexities (fates of the Memex and Xanadu projects, and more than half of software-projects).
  • Usability-problems due to complexity and feature-creep. If software wants to do too much, it becomes very complicated to use. Deminishing or preventing uptake (Engelbarts NLS system faced this fate).
  • Critical mass problems. Most collaborative applications, including LogiLogi, need a certain number of users to be useful, and won't be used until used by others. LiquidPub could face similar problems.
  • Network-effects. Getting people to use LogiLogi is hard as long as only things published in journals are credited. Similarly, getting LiquidPub's ranking/credit system, or LiquidPub's fragmented-documents-approach to be taken up in the face of widespread MS Word and/or LateX use can be a challenge.

These are just some ideas and remarks, and should be approached as not more than tentative.

Notes

Unfortunately, the notes from this panel were lost. If you remember sth, please write below or send your notes to Azzurra Ragone (surname @ disi.unitn.it)

Panel C: Novel methods for organizing conferences

Reporter: Aliaksandr Birukou, University of Trento

Moderator: Roberto Casati, CNRS

Statements:

"In-situ reviewing vs. pre-conference reviewing" by Marc Herbstritt, Schloss Dagstuhl

"Unconferences: running conferences without chairs" by Cesare Pautasso, University of Lugano

"Economic and organizational issues for future publications; Trends in other domains" by Ethan Munson, ACM SIGWEB

"Dagstuhl seminar on organizing conferences" by Anita de Waard, Elsevier

"Sheeps and shepherds: *PLoP conferences" by Aliaksandr Birukou, University of Trento

Participants:

Jordi Sabater-Mir, IIIA-CSIC

Katsiaryna Mirylenka, University of Trento

Diego Ponte, University of Trento

Ronald Chenu, University of Trento

Questions and ideas

Is physical presence required for productive work? Are presentations essential? With the improvement in ICT, can't we just share videos and papers online?

Can/Should conference presentations take into account feedback received at the conference? Can discussion over paper(s) involve a wider community, not only conference participants?

As in many other process design issues, one may either limit oneself to just mimicking standard conference workflow, or at the opposite end, try to take advantage of the new technological possibilities for redesigning the particular instrument.

In the case of conferences, the time dimension is going to be affected.

A conference is a time-management, coordination business. Co-temporaneous presence is what defines the essence of a conference, now.

Liquidconferences should respect this but can also incorporate more extended participation patterns, coming closer to forums. What are the advantages or disadvantages of loosening the tight time-management requirements and constraints of an ordinary conference? Should they be replaced one by one, test wise, as is being done in the platform developed at CNRS-Nicod?

A web based platform is naturally conducive to the implementation of certain time constraints. This need some exploring, as there may be interesting potentialities.

We know that currently a lot of scientific contributions are closely linked to conferences. Indeed some publishers earn money by organizing conferences rather than on selling books/journals. How will digitalization affect this? Furthermore, how might conferences shape the dissemination and discovering of scientific knowledge?

Statements

Aliaksandr Birukou. Sheeps and shepherds: *PLoP conferences

Shepherding[1] is a modification of the traditional peer-review process where a shepherd (reviewer) works together with the authors (sheep) of a paper to improve the paper. This process is non-anonymous. Shepherds are usually chosen among experienced authors participated in previous conferences and their task is to discuss the submission with sheep so that they can refine the paper prior to the conference. Shepherding usually results in several rounds of providing feedback and improving the paper. This process still includes the accept/reject decisions being made right after submission and after shepherding. Post shepherding papers may be accepted directly into a conference workshop (writers' workshop), or into a writing group. Writing group papers receive additional face-to-face shepherding at the conference itself, and those papers that reach the required standard are considered for workshop review on the final day of the conference.

Writers' workshops follow a special format which has been adopted from reviewing poetry. Before the conference, everybody reads each other's papers. In the actual workshop, authors give each other feedback on their work in a peer review fashion. Each writers' workshop contains 5 to 8 papers; a session of around an hour is devoted to each paper. In such a session, the authors of the paper under discussion remain silent while the other authors have a discussion about it, and explain what additional insights and views they have. Authors (as well as non-authors who may join) stay with their workshop over the entire conference. This way authors get a lot of ideas on how they can improve their work. Authors incorporate the feedback they receive at the writers' workshop into their papers before the papers go into the final proceedings after the conference.

It is worth saying that in line with the spirit of shepherding, the *PLoP conferences are structured differently from conventional conferences. Apart from writing groups and writers' workshops there are also focus groups, which are free-format discussion groups which bring together people who are interested in a challenging topics, and Birds of a Feather (BOF) sessions, which are spontaneous events, organized on site. Contents and format of a BOF session is up to the group joining the session. In addition to these, there are also game breaks that feature non-competitive games in order to let participants know each other and activate the creative halves of their brains, and build up a community of trust. Finally, all participants stay in the same, usually remote, place and have meals altogether.

Shepherding is applied in *PLoP (Pattern Languages of Programs) series of conferences, such as PLOP, EuroPLoP, ChiliPLoP, Mensore PLoP, KoalaPLoP, VikingPLoP, and SugarLoafPLoP.

Marc Herbstritt. In-situ reviewing vs. pre-conference reviewing.

Conferences are typically setup such that the contributions are selected in advance by a reviewing process. The term "conference" stems from the latin and originally meant "to bring together, compare" and in the context of scientific conferences this is split into a pre-conference review process ("compare") and the conference meeting ("to bring together"). Furthermore, one effect of today's pre-conference reviewing processes is that physical attendance at the conference is often intertwined with the acceptance of a submitted paper due to financial support of research grants.

Is it now time to think about a merge of the review process and the conference meeting by using new technologies?

Notes

Notes in MindMap format, do not forget to include this picture.

Notes in .doc format.

Notes in MindMap
Notes in MindMap
FlashMeeting
FlashMeeting
Roberto Casati

Should look for innovating the way conferences are organized, do not replicate pre-Web practices.

Reuse of recording of meetings as tutorial afterwards.

Conference – just a way of managing time of scientific interactions, journal – another timeflow for managing time of scientific interactions.

Ethan Munson

Running conferences as business activity.

SIG WEB records videos of conferences. Cost of video production is too high. May be producing webinars would be a more viable model.

Face-to-face social aspect of conferences is important.

CS conferences are strange, in other fields they only reject non-coherent presentations, sth. Like 5% reject, so what you are presenting is not necessarily good.

CS likes conferences <1000 people, while most of other scientists in US have 1-2-3 huge meetings per year (30-40 thousand people). Probably one of reasons, CS does not do neither business nor job fair during such events. CS uses tremendous amount of volunteer labor (at most – reimbursing expenses).

Is face-to-face aspect important?

Next 20 years – people do not have to pay for coming to the conferences, time of people could be used better. May be Facebook, Twitter, etc. can be used to replace some percent of face-to-face interactions. Twitter is more and more used for discussion during conferences (WWW conference in Spain). Coffee breaks of those who are twittering. You can follow talks you are not attending. You can identify others participants by conference name put into the hash tag. Of course there is a trade-off between listening to the speaker and reading twitter messages.

Attitude to Internet connection depends on the community. In Dagstuhl author decide do they want or not

Anita de Waard

People exchange ideas, using artifacts, twitter, etc. People, artifacts, interactions.

The ideas are exchanged, people meet each other, find jobs, grant collaborators, etc.

You can have things now and then, here and there. Conferences: here and now.

So, what part of it can be not here and now.

Dagstuhl workshop on how you change communication of science.

  1. Conference platform to be set up before, during, after the conference
  2. Experiments on peer-review – how to make it more open
  3. Invited submissions – do not mentioning discussion about quality. Vote – which talks to go to?

Risks of creating smaller and smaller communities, e.g. like you only recommend sth people usually read. Diversity of recommendations.

Platform can be used by people not attending the conference.

ABCD format – A, Background, Contribution, Discussion. Core contribution sentences instead of introduction. Presentations - Twitter channel, no video of the speaker, but can download audio and slides. Speakers spend time answering questions coming from twitter.

Concept of local conferences – to not travel too far: Australia, Eastern Europe, etc. Other can join via Internet

LiquidPub paper model better works in journal, not conference model – there people are reading anyway, also previous versions, while on conferences you cannot expect everyone consumed previous papers.

De-formatting conferences and seminars (not just presentations + questions + writing notes).

Picture of flashmeeting between Roberto and Carles, reuse by Gloria. Can produce video document of an interaction.

Image:Image004.jpg

Marc Herbstritt

A key to success is collaboration – not necessarily true assumption (while, this seems to be central assumption for LiquidPub). E.g., mathematicians who do not use e-mails. This might be cultural, or personal. May be LP is good for those who want to collaborate, it gives tools that support collaborations. But we need to leave the people choice.

Time constraints

Having conference deadlines is not a good idea because research is done only till that deadline and not continued afterwards.

Acceptance of 35-40% percent – bad conference -> wrong. Especially for those who are in the middle (not strong accept and not strong reject).

Personalized views of conferences, is it possible to get program in advance, A-attending talks, B-attending talks – priority. An advanced program and people can book in advance presentations. And can see who else in the audience. Program shaped based on wills of those who care to go before the conference and form the program, so there is incentive.

Organizing science slams as poetry slams – speaking for 15 minutes about your research.

Semantic conf – 5 mins’ ad-hoc presentation slot – people just decide at the conf. what to present. Good for attention span and for young researchers.

Time management vs. intellectual brokerage aspect – the latter now does not always exist.

Poster sessions – online international poster session – two days’ period during which people can “book” you and you can explain poster to 5 to 25 people simultaneously, do not need to repeat explanation many times.

Now time constraint is the only thing to get sth done.

Credibility – impact factor, rotating program chairs, … Money issue – people running conferences get money.

Cesare Pautasso

Unconferences – do not have the program. You know the topic, you know the people. The first day you put program together. Dagstuhl runs like this. No classical figure of program chair. You think why you go there (nice people, nice talks, good location, …).

Any survey about why you go to the conference? May be ACM has for such conferences – they do post-conference surveys.

Unconferences – around 100 people. Works with geeks – basecamp, mashup camp, etc. Time and space management problem. 6 rooms and set of time slots – people step up and write a title of what they want to speak about – demo, question, presentation – stick to the board, put names of attendants. Someone frozes on the wiki. Informal facilitators. Proposers of sessions take responsibility of organizing them.

Session starts when it starts, finishes when it finishes, whatever happens – happens, who come – are the best people. The law of two feet – you can use your feet to go around.

There should be an outcome – someone reports on the session. Someone facilitates the discussion – the organizer moderates the session.

Discussing dissertation with people from outside of your research group.

Analog could be organizing a wiki in unconference format.

Key to making it worthwhile – people budgeted two days, committed, etc. – not the case if it happens online. Paying for something makes you more responsible.

Can we get research credits and spent for conference

Audience knows more than a speaker

Aliaksandr Birukou

(see the statement below).

Sounds like a school, or an ideal form of a workshop.

May be journal special issue can be derived from the conference.

We underestimate the importance of delegating research part to someone – like mentors, shepherds.

People not introduce themselves – but someone introduces everyone.

Num of people for a session – 10 is max.

Anthropology of scientific conferences
Conference style/Meeting format
  • Personalized program
  • Responding to questions for which people voted most.
  • (wrong) Lecture – someone (expert) presents to the others (novices). Pre-packaged knowledge need to be transmitted to others
Proposal for novel conference management
  • Conferences is a way of managing time of scientific interaction – journals are another one
Document format
  • ABCD format
  • Flashmeeting video
  • Recording presentations as tutorials
  • Using conference videos for webinars
Things to think about when organizing conferences
  • Get reviewers (done by PC or ?)
  • Submission system
  • Resolving the program (PC chair does that)
  • Rotating PC chairs, members, …
Functions of conferences
  • Hiring
  • Business fair (selling equipment, etc)
  • Networking
  • Dissemination

List of Participants

1. Darren Dahly, University of Leeds

2. Cara Shank, University of North Carolina

3. Luca Cernuzzi, Universidad Católica - Asunción

4. Ethan Munson, ACM SIGWEB

5. Anita de Waard, Elsevier

6. Munindar P. Singh, IFAAMAS Board representative

7. Stefano Ceri, EDBT Board representative

8. Maria Teresa Serafini

9. Peep Küngas, University of Tartu

10. Wybo Wiersma, King's College London

11. Jeff Johnson, Open University

12. Marc Herbstritt, Dagstuhl

13. Moshe Chai Barukh, University of New South Wales

14. Cesare Pautasso, University of Lugano

15. Zhang Yi-Cheng, University of Fribourg

16. Joseph Wakeling, CNRS

17. Matus Medo, University of Fribourg

18. Alfred Hofmann, Springer

19. Carles Sierra, IIIA-CSIC

20. Jordi Sabater-Mir, IIIA-CSIC

21. Nardine Osman, IIIA-CSIC

22. Judith Simon, CNRS

23. Roberto Casati, CNRS

24. Fausto Giunchiglia, University of Trento

25. Ronald Chenu, University of Trento

26. Katsiaryna Mirylenka, University of Trento

27. Maurizio Marchese, University of Trento

28. Aliaksandr Birukou, University of Trento

29. Marcos Baez, University of Trento

30. Diego Ponte, University of Trento

31. Azzurra Ragone, University of Trento

32. Muhammad Imran, University of Trento

33. Volha Bryl, FBK

34. Wang Binhong

How to get to Ovronnaz

If you are flying, then the closest airport is Geneva, Switzerland.

We suggest three options for getting to Ovronnaz from there:

1. you arrive on the 3d of February, stay a night in Fribourg and then we arrange a transfer to Ovronnaz (carpooling). The direct train to Fribourg from Geneva airport takes 1.5 hours. Then, from Fribourg Ovronnaz is 1.5 hours by car.

2. you rent a car and go directly to Ovronnaz (Google maps says it takes 1 hour 40 mins, but add at least 30 mins, because you are not using highways).

3. use train + bus connection to Ovronnaz. Please, use [3] to plan your journey. Put any european city and it will show you connections. The stop is named Ovronnaz, Centre therm.

If you come via Fribourg, consider also arriving to Bern, Basel or Zurich.

Here is the website of Ovronnaz.

Retrieved from "http://wiki.liquidpub.org/mediawiki/index.php/Second_Workshop_on_Scientific_Knowledge_Creation%2C_Dissemination%2C_and_Evaluation"
Personal tools