Snow Workshop Raw Notes

From LiquidPubWiki

Jump to: navigation, search

Contents

First combined Snow Workshop panel notes.(Feb 2009)

Panel 1 Knowledge Generation processes

Ethan Munson

  • The system we're describing has some resemblance to Nelson's Xanadu and Bush's Memex. (How is it different?_
  • Computer Science is different in that it has no "mature" journals. Example of a mature journal is Journal of Biochemistry. Published weekly?, small issues, short and concise papers.
  • ACM digial library example. Important to the ACM, provides significant income to the society.
  • Archival concerns are important. Not sure how ACM handles this. Higher failure probably for electronic storage. Centralized and de-centralized approaches (LOCKSS).

Stefano Ceri (see also Workshop notes from Stefano, including two references)

  • Review codes of behavoir.
  • Authors should be reviewers
  • Reviewers should do good reviews (don't favour friends and avoid conflicts of interes in general)
  • ... best reviewer award should be awarded
  • reviews should be published
  • authors should score reviewers
  • explanation of an ACM proposal: visibility and evaluation of the reviewers interesting paper concerning how bad are reviews and in general metrics used to evaluate researchers
  • Review control is still some low-tech, verbal token-passing system in most cases. Really should be something better.
  • Mentions a metrics paper. Always be careful of the behavior that the metrics foster. Can easily foster bad behavoir.
  • The Wisconsin Model.
    • good numbers on experiments mean publication
    • even if the overall thing is not significant, if you improve ca 30-40% with respect to your competitors, then you are done.
    • Asks why is it so hard to change systems?
  • double blind don't work, maybe it's better an open approach we try to anticipate the reviewing process

Questions and comment:

  • Fausto adds that there are restricitons on possible solutions over itme in a field.
  • Hofmann adds that averages and behavoir differ widely across communities.
  • Ethan comments about resouces and public models...
  • Andrei V says that quality of reviews is low simply because people lack time.
  • Joe add that "friendly" reviews aid in knowledge by encouraging changes and collaboration. So, peer-review works better if the atmosphere is "friendly". [Be nice.]

Manuela Veloso

  • Suggests compiling directions in the field so that people can easily declare what they are doing, what htey are working on. Rather than following the jargon of fashion of the day.
  • Gives a story of the ideal paper outline: have something to say, say it, stop after you say it, give it a good title.
  • keep learning how to write a paper
  • Veloso comments on deriviatives [rate of change] and improvement.
  • In the business of confusing others ...
  • there is no education to reviewing for our students
  • graphs, equation, formal definition - better results as the confusion grows
  • avoid changing topics just to be publishable: "Don't let your career be a random walk through publishable material."
    • One should point out, however, that there is the opposite problem of neglecting stubbornly the latest technology developments. In computer science, technology is often what makes possible qualitative changes. Disregarding technology is as wrong as just be a technology geek.
  • journal of interactive media in education -> also reward for best reviewer
  • Ecology of the publication process, multiple reviews. Re-evaluation.
  • Something about conferences provide volume...

Questions and comment:

  • Ethan comments that reviewing is a true scholarly effort.
  • Voronkov suggest publishing papers of definitions so that people can cite them and reduce the size of papers. [How to agree on definitions?] [SKOs should provide reusable definitions.]
  • Anita DW mention genre studies of intros to position papers in fields. Asks how could we write papers for machine consumption? Is your paper a "good" rhetorical text? How do we define "good"? They tried modular publishing, but it was generally very hard to read and there were few writers that could do it okay. Some work on verb tense, I've forgotten the details of, but it was facsinating.
  • Manuela suggests Grand Challenges in fields to motivate more significant work.

Paolo Bouquet

  • Paolo B says dissemination is different, some catching up to do, asks is the paper the end? What new tools would help collaboration? How do we move from readers to authors? Why have demo sessions gained so much in importnace? (Easier to understand, maybe). Are the business models enough? What other means are there to publicise ideas?
  • creation and dissemination of scientific knowledge has actually changed multimedia, web, ...enabled many more activities and changed the way of disseminating things
  • much more interaction rather than writing and presenting papers
  • before -> papers, posters, demo
  • now -> it's more interesting to partecipate to the demo sessions!
  • Also for people who actually do have visibility
  • Another point is to have the right means to disseminate, good ideas are not sufficient

Questions and comment:

  • Toby there are many kinds of readers, pull/push technologies
  • Fabio blogs, communities etc. can be used to give scientific contributions and to evaluate careers and contributions
  • Manuela objects to blogs as a review or information source. Says not good.
  • Joe constraints have been removed from dissemination since you don't really have to be published. Good rule is to "be friendly" as in opensouce projects.
  • Sihem following passion in research, should be able to answer why are you working on "that" problem?
  • Paolo blogs can provide friendly review. Paolo tends to favor blogs, because there are a lot of feedback and much more discussion on ideas

Other discussion:

  • the generation mechanism varies from one research field to another, and therefore, this should be taken into account when thinking on novel mechanisms for knowledge creation.
  • Changes in generation process should imply changes not only in technology but also in society
  • Some metrics are bad for science and introduce noise in the generation process. Is research just about fashion? (trends)
  • What about reusing? (auto-plagiarism?) This is a challenge. The generation process should allow reusing previous research. Currently is not possible and people waste a lot of time to present new results. Options: a) knowledge artifacts continuously evolving. b) Explicit reuse of knowledge artifacts. Challenges: what does it mean to write in a reusable way?. How to merge contributions? (i.e. different authors with different writing styles).
  • Modular research is a good example.
  • What is the role of incremental papers?. In some fields the progress is done by increments.


Panel 2 Review and evaluation

Aymard DeTouzlin

  • Essential question for the EC is "Where should we invest the public's money?"
  • Therre are sometimes other concerns than the number of papers produced.
  • Review and evaluation are customized for each program.
  • Asks how can we open (ID?) process more?
  • Main focus on the impact and the innovation of the project, the entity of the problems it challenges and the value-added it could bring in research.
  • Neither reputation nore publications are of some interest for the EU; they funded the project because it could bring important changes and improvements in the procedure of creating adn disseminating knowledge.
  • No target on a specific domain, we are inside ICT, but the accent is on multidisciplinarity
  • He explains the way FET-open projects are evaluated (2 stages: short proposals -> full proposals)
  • 80 - 100 reviewers, expert that, following the goal of the projects' call that evaluate the short proposals
  • competitive review during the full proposals' evaluation
  • 25 funded projects over 400 per year, quite tough (6-7% are accepted)
  • there are proposals to "open" this process, by giving some reviewing capabilities also to a selected community

Questions and comment:

  • Enrico 2008 article claiming that people who have been selected actually receive more credits afterwards. Assessment done through: number of publications, number of citations, h-index.
  • Gloria how are criteria determined? why don't you consider publications and citations very much?
  • Aymard
    • FET inspired criteria, different for differetn programs.
    • mainstream program is more industry-oriented, FET-Open tends to give the right relevance to publication, but they are not the core motivation that leads a project to be accepted
  • Paolo
    • problem is reviewers have no accountability.
    • there is a way to evaluate the evaluators, because they have some responsability on the choice of the projects that are more likely to succeed / fail. It may take many years to evaluate the impact of a project, how can you say if an evaluator is a good one or not?
  • Toby: the problems exists and actually if we compare it to the conference model, chairs evaluate (privately) reviewers as well. In his experience, conferences are also an opportunity to assess reviewers.
  • Enrico FET projects evaluation tend to favor less the cliques, not always the same important persons receive money
  • Marco R
    • need a more open process
    • he underlines the fact that ideas contained in papers should be protected against plagiarism, so the opening is important, but is relevant as well to somehow have something printed that certificates the copyright, etc.
  • Fabio:
    • the majority takes more than 2 hours to review papers, only 4 take less than one hour to review papers
    • His idea is trying to distribute papers to more reviewers, asking less time to each of them for the evaluation process.

Tony Cohen

  • Problems finding reviewers, diversity of reviewers, authors submit and then say no to reviewing because they are often overloaded.
  • He stresses the importance of the choice of the reviewers; they concern a lot about reviewers' "quality", he spends a lot of time in the process of selection of reviewers
  • Length of the paper makes a difference: the longer the paper, the more likely is for the paper to be rejected
  • Money helps, There is a journal in biology in which reviewers are paid 40-50 $ per paper (200$ if they manage to review all of them in 2 weeks)
  • Although sometimes articles published in journals are refinement of previous articles published in the proceedings of a conference, there are differences between conference papers and journal papers: journals are very concerned on paper that will have a great impact, in conference it is not always so.
  • Some journals requires reviewers to sign their reviews, but it is generally not well accepted
  • Their reviewers are asked to spend quite some time checking the references, they want to be sure that they are correct and properly used
  • Impact factors have been improved by electronic offering of papers.

Alberto Silvani

  • Evaluation - there are positive and negative conflicts.
  • Two core points:
    • how to manage conflicts in evaluation
    • how to use different metrics in evaluation
  • How do diversify and specialize tastes?
  • Dynamic situations
  • Concerning the first point, he criticizes against the consensus model, because in his opinion, it doesn't really solve the conflicts between reviewers in a positive way.
  • Peer review can be improved by specializing exercises and procedures for specific tasks, peer reviewing is too general.
  • He brings as an example the blind - double blind procedures -> it is not always effective (every research field, every type of artifacts)
  • Attribution and measuring contribution is a core aspect in the evaluation; people have to measure the entity of the efforts of the authors of a particular article during its lifecycle
  • Disagreement can be solved by the person who manages the overall process, and not by the reviewers themselves. (my comment: For instance, in the FET-Open, by the rapporteur): this can be a good solution in the sense that we have a clear decision and not an "arithmetic mean" of the single opinions, but it might also be not a good thing (discussion is required!)
  • About the metrics: try to normalize and try to apply them as often as possible

Questions and comment:

  • Fabio: in your opinion, how does the acceptance rate change according to the time that the reviewer spends in the process?
  • Stefano C says that he takes 10 minutes to take the decision and 90 minutes to motivate it.
  • Stefano C shows an e-mail that is brought as an evidence to the claim that it would be better to publish the reviews (either good or bad)
  • He? thinks that, usually, reviews' quality is very low.
  • Fausto suggests that in the LP world, reviewers have to publish their reviews (at least in the case of rejection) and to provide feedback, suggestions, etc. to give the opportunity to the authors to improve and to understand the motivations behind the rejection.
  • Joe co-authors may not always agree, tells of journal article where the authors split in two groups on conclusions, presented both sides. Suggests that the LPplatform could facilitate friendly disagreement.
  • Ethan can have have malicious (acks perhaps unfortunate, but accurate, word choice) participants who may not think htey are that. [Jim Possible they are deluded. Personal experience.] Only really persistent crazy people get into the academy. One-time crazies get filtered out. Conflicted articles on Wikipedia as an example of how to deal with this.
  • Ceri bad reviews and english errors. Shows an email of a review with many english errors that is actually complaining about english errors in the manuscript.

Farouk Toumani

  • we should exploit web 2.0 for the assessment procedure, using downloads, bookmarks, etc.
  • peer review has some problems (<quite a long list here>)

Questions and comment:

  • Stefano Mizzaro: it's true also because I don't get credits for being a good reviewer (in the typical peer-review world)
  • Joe: quality control is not ensured in peer review, we should take advantage of the new infrastructures available nowadays

Stefano Mizzaro

  • Not exploiting ratings enough. After publication there is an army of readers but no exploited, we don't get feedback from them except in the form of citations which have problems of interpretation. Perhaps CiteUlike helps. Also reviewing is not rewarded enough.
  • Stefano M published a useful paper: S. Mizzaro. Quality Control in Scholarly Publishing: A New Proposal, Journal of the American Society for Information Science and Technology, 54(11):989-1005, 2003. (Email Stefano for a copy, or download Mizzaro2003 preprint).
    • "good review(er)s are the scarce resource in the scholarly knowledge generation process" [Mizzaro2003]
    • "when reviewers do good and provide a good review, they are not rewarded - in terms of scientific reputation ** and when they do bad they are not punished. At least not systematically" [Mizzaro2003]
    • "peer review is a quality filter. It is a priori (before publication) and a posteriori (after writing). It might be replaced/complemented" [Mizzaro2003]
  • Once we agree that reviews are a scarce resource, and I assume that we do, the question is: how to solve this problem? My proposal is to simply follow what's being done in the Web 2.0, a.k.a. social web: use the *readers*. Readers read papers, and they can usually say if the paper was good or bad (besides provide commentaries, tag, collect, etc.).
  • However, simply using the readers is not enough, because there wouldn't be reviewers evaluation. Therefore we also need some reputation mechanism (like the one I proposed years ago [Mizzaro 2003], or anything similar/equivalent/better) that rewards good review(er)s, and punishes bad ones, in terms of scientific reputation.
  • This would exploit the "wisdom of crowd" of readers, who would make up a huge review force. Using and evaluating readers/reviewers would also be much more precise than bibliometric indexes (h-index, etc.), that are quite fashionable these days but largely misused.

Andrei Voronkov

  • quality of reviews is difficult


Other discussion:

  • Fausto reviews should/could be more informal, distributed, anticipated
  • Paolo scientists should move from readers to actors - be more active
  • All researchers complain about the quality of the reviews they receive. One of the reasons of this problem is that the review job is not rewarded.
  • Aymard We do not have a defined mechanism to do reviewers evaluation
  • What else?
    • How do we identify contribution which are relevant?. This is sometimes subjective, although some conferences have “recipes” (i.e, how much better should be better respect existing approaches). The problem still remain, numbers cannot capture relevance.
    • We need to disconnect ideas from people, and evaluate ideas. Most people review papers by looking at the authors’ names.
    • If you cannot convince them, confuse them!. People tend to use formalisms, making their papers complicated or not readable, making the review process complicated.
    • What is the difference between bad science and bad written papers?
    • What’s a good review?. Good feedback. Encourage authors and guide them.
    • We need a better review force. The community could be introduced in the process to assess and validate the scientific contributions. We can exploit web 2.0 (social reviewing) and other statistics.
  • So, what we can do?
    • Award for best review in a conference?
    • To publish reviews?
    • Open peer-review
  • FET: research Financing organization care about results about improvements to human knowledge and changes for the world
  • Challenge: how to select the best submissions to reach their previously stated goals?
  • Related Paper: Does the committee peer review selects the best applicants for funding?
  • Evaluating Evaluations, normally the approach is that most reviewers do not take responsibility for their reviews.
  • A posteriori analysis or reviews with the possibility of rebuttal or replies help in the previous
  • Opening review to communities may help its discussion but there is also the protection of idea concern, so some protection should be offered in this sense
  • Ensuring fair reviews and reviewer is challenge.
  • Being "overloaded" is the main reason for people for reject the opportunity to be a reviewer, meaning that people are using a lot of their time reviewed
  • Most of the papers get accepted with "strong modifications", almost none gets accepted "unconditionally"
  • Pages: their numbers are important for references, they become a limiting factor for accepting/rejecting works. However they are bound to become irrelevant when switch away from printed media happens
  • Publication Queue: putting the paper online first before the printing of the actual physical copy. Ironically, being in the publication queue can affects the impact of the paper
  • There are two parts of reviewing. One is deciding to accept, the other is how to give your opinion and help the paper. The second takes much more time
  • Publishing the review, even anonymously could provide interesting insights, specially in the quality of the reviewer

Panel 3 Tools and methodologies

Nicu Sebe

  • Revision histories are important
  • Need to move to the idea of "maintainers" rather than "authors".

Anita de Waard

  • Reassembling incremental research. Gave example of "splitting up the column" in a lab and each grad publishing on part of it. They would like to re-integrate the research for the whole column.
  • Modular publication, rhetorical structure of papers, and non-textual elements of papers.
  • Conferences models: ABCDE?

Said Hacid (Said has kindly supplied a presentation of his points.)

  • Open Access initiatives and such directly imply we need to evaluate ranking tools.
  • Moving to some kind of Publish, then Select model, reverse of current.
  • Need specialized ranking tools.

Toby Walsh

  • For collaboration, email is it! Seems foolish.
  • He brings in his experience in collaborative work
  • In his opinion, the most used technology used for collaborative work is still the e-mail
  • However, he admits that he would be glad to have some more features (he made the example of having the actual time in different countries, in order to coordinate the work with people which are from different countries in the world).
  • Unfortunately, so far he didn't find something sufficiently appealing to him, that can convince him to change his habits
  • So, from the LP perspective, he says that we have to be very careful during our requirements' analysis phase, because if people don't see a true advantage in using a different tool for creation and collaboration, it is going to be completely unused
  • Describes the effort of maintaining a handbook. It's a huge maintenance problem.

Victor Henning

  • Describes Mendeley activity and plans.
  • He is the co-founder of this firm, which developed a web application that offers some interesting features in order both to make things easier when we have to create knowledge and to collaborate
  • They also want to infer some interesting data about impact and popularity
  • These are the main features:
    • you install the software and then you have your personal bibliography in pdfs, each pdfs is parsed and then indexed to speed up search
    • you can share your bibliography with them, allowing you to have a remote site which you can refer if you have multiple laptops, etc. where different documents of you biblio have been stored
    • they can analyze their database of pdfs and tell users which are, for instance, the most frequently read pdfs
    • in this sense, they also want to build a sort of recommendation system (Amazon-like)
  • A few numbers: they have 10,000 registered users, mainly from CS and, in general, in scientific disciplines. They have 1.8M documents indexed
  • He makes a comparison with audio files and, in particular, with LastFM, which essentially is a portal through which you can listen to music. They log your activity and, by analyzing the data they have, they can report "trends" about music (most frequently listened song, etc.)

Questions and comment:

  • Maurizio: we cannot perfectly juxtapose audio files with published papers put in an electronic format, because of the fees and the access rules that exist between the user and the publisher (in other words: if I pay for free access to pdf, I just cannot share the pdf with everyone)

Anita de Waard

  • she claims that it is actually true that a lot of literature is iterative (she brought her personal experience in solid state physics, where they replay the same experiment changing only the particular element involved)
  • to overcome this issue, we need to have versioning
  • it is also interesting to identify specific patterns that characterize the structure of the papers published in the various disciplines
  • To that extent, they developed a tool called "ABCD" which essentially enables people to create papers by filling pre-defined sections (NB check this note)

Sihem Amer-Yahia

  • Other dissemination methods
  • "Get credit model".

Marco Ronchetti (Marco has kindly supplied his speaking notes to us.)

  • Advocates the need of having alternative methods, both to disseminate knowledge at a research level and in education (let's say "teaching level").
  • He thinks that, for instance, videos could be a useful mean to convey knowledge (for example Youtube online tutorials).

Questions and comment:

  • Alfred Hoffman disagrees on this last point, bringing as evidence that supplementary materials (in addition to papers and, more in general, published items) has been used very rarely when they give the chance to their authors
  • he? thinks that there also top researchers (like Tim Barners-Lee) who decided to try to convey knowledge through alternative means (but he already is a top guy, he doesn't have to get credits or career advancement, so it's easier for him)
  • Luc Schneider: a final remark concerning the delicate issue of copyright in our LP world: what is the amount of contribution that enables you to be "officially" considered as a co-author? Moreover, copyright in a collaborative world like LP, with versions and so on, has to be passed somehow between authors (similar to what happens in OS, where we have software licenses)
  • network models
  • Possible collaboration between Anita and Luc S on rhetorical structure of articles.
  • Joe need cooperation on maintenance.
  • Jim point on confluence of Nicu mention of moving from maintainers to authors, Toby description of maintenance and the analogy to expense of software maintenance, and Joe on the reputation of maintainers in opensource software projects. [Joe suggests a collab using launch pad to maintain a paper.]

Other discussion:

  • What’s a scientific contribution?. what is good for science, and what can fasten its progress.
    • Video lectures. (For example, showing the behavior of a child could be more descriptive than its statistical information)
    • Blogs
    • Process models, experiments.
    • Libraries like in software? What about reusing mathematical definitions.
  • Documents should have different views. Readers should be able to consume contributions in different ways.
  • How to structure scientific contributions?
    • Core sentences. no abstract -> core sentence
    • Understand what the smallest publishable unit is. How to compose these units.
  • Methodologies for scientific publishing
  • Real collaborative work. No virtual tokens.
  • A possible issue: contributors suggesting their own work. auto-citations->contributors will suggest their own publications.
  • Maintainers of knowledge.
  • Tools
    • Package revolutionary ideas in what people already know.
    • Sharing annotations
    • Concepts map could be used to get the big picture. For example, in forums it's hard to get what is going on.
    • Rankings could introduce noise -> people worry about who's first/second..
    • Information overloading. How we can find relevant contributions?. Indexing, not just googleing it. List of interesting things
    • Design of the interface is important. It could change a lot the behavior of users.
  • Risks in opening the system to everybody
    • Malicious participants. People who try to do research, but who are not good at it.
  • Dynamic and fluid(non static-based) evolution
  • Documents personalized (detail on demand)
  • Transition to Paper to a New media (compared from going from speech to writing)
  • The definition of Smallest publishable unit deemed initially important for this
  • "Magma model": science is a liquid object and it burst out of the solid foundation to later solidify and become the new base
  • Links as information and interpretation. Relationships also contain information about the element
  • Rhetoric itself has structure. Elements of Discourse are identifiable and probably generalized and modularized
  • BCD model (Background, Contribution, Discussion) is one attempt at this
  • Authors make claims in their documents, they eventually solidify into facts
  • Impact of ICT in sharing KW. Distribution
    • Minimizing duplicating activities
    • Faster dissemination of scientific kw
    • Multichannel publishing (into different media)
    • Communication may include not scientific recipients
  • Open access decreases the risk of duplication
  • The only ICT tool that is currently being used for collaboration is email
  • Going further than citations to give information about documents (most read, association of tastes, etc)
  • Papers not normally position themselves adequately within the community
  • Sharing annotations beyond the "forum model" in which subjects are broken down and the global vision is lost
  • Indexing from not only full-text but a more semantic one would be necessary even more when physical publishing becomes obsolete
  • A paradigm shift from "author" from "maintainers" seems to be necessary


Panel 4 Publishers

Alfred Hofmann

  • 25K papers, 60K authors per year in computer science.
  • Issue: do only journal publications count for getting credits? For instance their LNCS (mainly proceedings) are actually indexed by ISI
  • In this sense, it is fairly common to have conference papers moved to journals in order to get recognitions by the community
  • He stresses the fact that it is difficult to have overall standards and common metrics to evaluate research, because of the heterogeneity of the different communities (ex: bioinformatics <-> biology)
  • 10.8 - 11 cent per page is the cost in LNCS or in ACM publishing
  • he is skeptical about the degree of collaboration among scientists, so he thinks that the overall process should be enabled by encouraging the share of ideas, and this is one of the main goals of LP
  • he also thinks that still conferences and journals, more than blogs and wikis, are the only model of knowledge dissemination that ensures a certain quality
  • publishing is much more than storing pdfs on a server and getting visibility on Google: there is the preservation issue.

Questions and comment:

  • Alessandro Rossi: he thinks instead that blogs are good sources of scientific knowledge: he brings as an example Lawrence Lessing's blog, which he thinks is commonly referred as a source of "smart ideas"
  • Rossi uses book "The Cult of the Amateur" in a class.

Anita deWaard

  • Okkam, Science Direct, Learning Object Model, SWAN Sem Web Alzheimers Net, COHERE model?

Enrico Scalas

  • peer review in Plos is traditional in some sense because is confidential; however it is also a new approach, because they send the papers to reviewers that can disclose their names to the authors and even publish their reviews, but this final option is never used
  • he reports also an example of a model used by a small publisher in Schleswig-Holstein (Kiel), where they don't have a confidential policy with respect to the reviewers, so reviews are published and the overall process is more considered as a discussion between authors and referees*Demos are needed
  • Empirical investigations and better economic models
  • Should examine environmental impact.
  • Discusses classical rhetorical and logic errors in physics papers.
  • Abandoning the "Heroic Model" of science
  • Economics E-Journal as an example where rejected papers are still available, unlike now where they usually just disappear.

Barbara Kulemenos

  • missed...

Other discussion:

  • Rossi for publishers the major asset is the boards, others argue it is the content, back-catalog.
  • Some sort of evolution currently exists in papers. For example, conference papers who also have a "journal version"
  • Particularity from the Scientific papers: the consumers are the same people as the producers
  • Even Open Access publishing also has costs
  • Publishers are improving their services
  • Critiques like the one on "Cult of the Amateur" expose the Web2.0 community-based production
  • There is significant group of the scientific community that is unhappy with the "status quo" and wants new functionalities to improve their output and collaboration
  • Structure Digital Abstract: A graph-like structure that contains the me main idea of the document
  • E-learning: for each piece of knowledge you create you specify the audience
  • Not only papers, General practice guidelines are also important to disseminate and keep updated
  • Publishers provide stability and a set of other services that are normally underestimated by the rest of the world
  • Even if full text is available previous to publication, this doesn't affect the publisher's profit? (yes! At least in High-Energy Physics)
  • Inertia from the current system is quite significant. PlosOne publishes works and allows comments, however people just do not use it.
  • Advise: start with demos! Even if the most important features are not implemented
  • Environmental impact possiblities of Liquidpub: would paper use decrease?
  • Heroic model: it continues in science. Science depends on "exceptional individuals" that generate bursts of advances. And yet precursors and people who continue the original work also play mayor roles.
  • PlosOne-like initiative: all papers are immeditely accepted as discusion papers and then evaluated if the are "promoted" to the level of accepted papers. Even if they are rejected they remain published.
  • "No publishing process is for free"
  • The selection of paper and the quality publishers claim to offer, actually comes from (sometimes) unpaid boards of scientifics
  • Authors and Researchers are really not concerned with the survival of publishers/libraries they will be as happy if they get those services from elsewhere
  • Does the current restricted way of publishing/story actually helps or detracts from scientific progress
  • The web has mostly damaged the ACM(SIGs in general), not so much the publishers

Wrap-Up: LiquidPub Conference Planning

  • Gloria conversations
  • Enrico unconferences
  • Ethan CS conferences are small, except SIGGRAPH. Large ones have trade shows. Keynotes in small conferences need to be *interesting*, not so *important* [Jim memories of ALife 7 in 2000 keynotes and the interesting talk by the geologist from Aus.]
  • Discussion [Jim I think what we need are more good, old-fashioned arranged dinner parties.]
  • Stefano M evaluations from attendees
  • Rossi/Joe sprint sessions [probably Joe, unclear from notes]
  • Alfred asks what the students think since they will be attending most confs in the future
  • Matus formats differ across fields, as they should
  • Ethan suggest experimenting in sessions rather than whole conferences as a way of reducing risk and making less radical changes. Points out there are scaling issues with conversations, too big and discussion breaks down. [Weapons of Mass Discussion!] Discussant/Questioner format often works.
  • Carles suggests thinking of community rather than co-authorship, have a conventional conference on hwo to have a LP conference, no proceedings - reviews will be very different.
  • Maurizio we are trying to bootstrap a new conference model with a conference
  • Fausto bootstrapping needs better definition of roles especiallly for phds and students
  • Barbara asks one-community or interdiciplinary? Mentions budget concerns for attendees. Suggest phd training since they will adopt this new model. Need introductions to the network. Observes that trade shows can distract and that they definitely change the atmosphere.
  • Ethan suggest focusing on technology support.
  • Jim need good netowrk connectivity [Grappa!]
  • Fabio network conns reduce attention span, there is a cost for the connectivity, sort of an attention market in a way.
  • Fausto this is a foundational conference
  • Ethan identify where the novelty is, start a working group/session
  • Tony start with reviews
  • Stefano M look better at existing work on conference models, be very careful about organizing *another* conference on electronic publishing!
  • Ethan need a steering committee
  • Maurizio need a really good name
  • Fausto what to do? Send us comments and write-ups.
  • Fabio mentions surveys
  • Ethan joint NSF funding for development work in NAmerica
  • General Planning
    • Within a year
    • One of the main objective is to bootstrap content
    • Reasons for a conference (Listen to panels, Interact with peers, etc)
    • Default is physical presence but is not necessary
  • What is the most important parts of the a conference?
    • Interaction with another members of the community(not necessarily face to face)
    • Oficial and unoficial presentation and work and opinions
    • Discussion, agreements and disagreements between participants
  • The Unconference model
    • Interested group of people, they get in touch through the web
    • They decide a place to meet among themselves
    • No agenda is fixed, that is done on the first talk
  • Another alternative model(Economy)
    • Planary lectures (one for day)
    • Parallel lectures: all participants on the same level (same time for everyone)
    • Discussants appointed (they have to read the paper being presented so they can help the discussion going after the presentation)
  • Tiered approach to conferences
    • Highest Level participants get oral presentations
    • Most of the participants only get poster presentations
  • Computer Science events
    • Workshop:(small) there are no expectators
    • Composium:(medium)
    • Conference:(big)
  • Advise: Separate the name of the conference from the name of the project (should not be called directly the Liquidpub conference)
  • Opinion about a Liquidpub-based conference: while the structure and organization of the conference may be the same, the way of selecting the works to be presented and how these works are actually created can be the only difference
  • Communities are the base of the Conference. When making the LP conference there is a necesity to identify what community wants to be included
  • Advise: Research what else is offered in the market. People do take economics in consideration before deciding where to go.
  • One of the open problems is that the tools to enable the Liquid publications will not be 100% ready so "innovation" has be to carefully managed based on the previous and also trying not to alienate participants
  • Make it a "foundational conference" with the knowledge that all the tools will not be available but with the objective of providing the "root" or start for the evolution of knowledge of the platform
  • The novelty factor has to be significant enough so that its not just "another" research methodology conference
  • Stefano M: First, a concrete idea for a novel conference format. The conference would be divided into phases, and at each phase all participants compete to have space on stage in the next phase. To be more concrete, what I have in mind is that on the first day all participants are given a short amount of time (5m-10m) and attendees vote with a numeric score. The highest scored presentations are given a longer time slot in the next day. And this could be iterated for a third day/phase and so on. Attendees scores must be collected automatically; I'm working at a system that allows to do this either on the Web, by simply using a web browser, or via mobile phone, e.g., by means of a Bluetooth connection. The system described in [Mizzaro 2003] could be used to weight in different way the participants, on the basis of the goodness of their votes. People in the Program committee, and/or experts in the field, could be given higher weights as well. Again, similarly to my proposal of exploiting the "wisdom of the crowd/readers", I'm here suggesting to exploit the "wisdom of the crowd/attendees". Again, this is not done today, at least not in an explicit/formalized/quantitative way.
  • Stefano M: a second remark, which is more general. During the workshop, I sometimes had the feeling that what was being said was based on personal opinions, anecdotal experiences (i.e., I was once told by my supervisor that...; I go to conference just to meet people; etc.). This is useful, but it is not good science. I suggest to: (i) do some well designed poll / user study at some conferences, to have a reliable sample (people in Carezza obviously were not a reliable sample); and (ii) take into account the huge amount of literature on:
    • scholarly publishing,
    • electronic scholarly publishing,
    • peer review,
    • etc.
  • Stefano M: I'm not saying that no one was an expert; indeed some of the workshop participants did undertake literature surveys. However, some others simply stated personal opinions. At least in my opinion ;-)
Retrieved from "http://wiki.liquidpub.org/mediawiki/index.php/Snow_Workshop_Raw_Notes"
Personal tools