From owner-chemistry@ccl.net Mon Oct 17 00:06:00 2011 From: "Brian Salter-Duke brian.james.duke(-)gmail.com" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45655-111016222252-8882-LlVmcsOIAH5Q0wDNKujL+Q(a)server.ccl.net> X-Original-From: Brian Salter-Duke Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Date: Mon, 17 Oct 2011 13:22:42 +1100 MIME-Version: 1.0 Sent to CCL by: Brian Salter-Duke [brian.james.duke]|[gmail.com] I do not understand your point. The manifesto is supporting open source. Any open source code meets all the criteria of the manifesto, What is being closed? The manifesto is fully in accordance with what Google does. Brian. On Mon, Oct 17, 2011 at 4:45 AM, Sergio Manzetti sergio.manzetti++gmail.com wrote: > I agree with Sebastian, what is so "open" about closing open source into a > manifesto? I was  even more surprised that Google put their signature on > that manifesto, a company who endorses free openness in information flow. > > Sergio > > > > On Fri, Oct 14, 2011 at 11:18 PM, Brian.James.Duke{:}gmail.com > wrote: >> >> I just encountered the Science Code Manifesto, which essentially states >> that all computer code used for scientific analysis and modeling should >> be available for review. It appears to have started with the Climate >> Code Foundation. I encourage you to visit the web site and consider >> endorsing the Manifesto. >> >> http://sciencecodemanifesto.org/ >> >> Note that this is not specifying open source code, so GAMESS(US), >> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd >> MPQC satisfies the points of the manifesto. Of course some other quantum >> chemistry codes do not. I think we should be putting pressure on the >> authors of such codes to meet the criteria in this manifesto. >> >> Brian. > > -- Brian Salter-Duke (aka Brian Duke) Brian.James.Duke|-|gmail.com From owner-chemistry@ccl.net Mon Oct 17 04:46:00 2011 From: "Konrad Hinsen hinsen|*|cnrs-orleans.fr" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45656-111017032523-17308-NDaxASCS/q4W9rW4NM6U1Q]-[server.ccl.net> X-Original-From: Konrad Hinsen Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Date: Mon, 17 Oct 2011 09:25:10 +0200 Mime-Version: 1.0 (Apple Message framework v936) Sent to CCL by: Konrad Hinsen [hinsen||cnrs-orleans.fr] On 16 oct. 11, at 11:05, Sebastian Kozuch kozuchs^_^yahoo.com wrote: > Let’s say that I receive a manuscript for review that uses program > X, which I may have access to its code now (since I’m the reviewer). > Do I have to spend a week trying to understand its algorithms to > check if the frequencies are correctly calculated? For me, most (if > not all) the programs are in practice black boxes, as I’m hardly a > programmer. Therefore, open or closed software (usually) doesn’t > make me any difference, except from a philosophical perspective. It may not make a difference in the routine operations of science, but in case of exceptional importance of some work, it makes all the difference. It is then up to the community to decide in each case whether the effort of finding the right expert for such an evaluation is worth the trouble. > Now, let me consider an analogy from experimental chemistry. I have > to review a manuscript where the authors tested some compound with a > 600 MHz NMR. My lab is much more humble, and I only have access to a > 400 MHz NMR. Id est, I cannot reproduce the results of the paper > that I’m reviewing. Is here any difference compared to the > theoretical case? Yes: you can always find a 600 MHz NMR spectrometer if you think it's worth it. You can also hand back the review to the editor of the journal and say it needs to be done by someone with the right equipment. > I had in a couple of cases problems to reproduce the results of > papers, mostly because the authors didn’t provide enough > information. However, for 100% reproducibility I need 100% the same > conditions (and a lot of resources and time). This makes the > selection of the software just a small issue. It is just one issue, but it's one that we can address, so why shouldn't we? Insisting on sufficient descriptions for reproducibility is another action we can take right now, so we should as well. Personally I do in all reviews I do, and all authors have complied so far. > I would like to hear the thoughts of other people about this issue, > as I consider the peer reviewing process a very complex and far from > perfect system. I agree, but my conclusion is that we should take every possible step to improve it, even if it's only a small step. -- --------------------------------------------------------------------- Konrad Hinsen Centre de Biophysique Moléculaire, CNRS Orléans Synchrotron Soleil - Division Expériences Saint Aubin - BP 48 91192 Gif sur Yvette Cedex, France Tel. +33-1 69 35 97 15 E-Mail: research at khinsen dot fastmail dot net --------------------------------------------------------------------- From owner-chemistry@ccl.net Mon Oct 17 08:56:01 2011 From: "Sergio Manzetti sergio.manzetti(a)gmail.com" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45657-111017045933-26678-8p3/62L4ZFLtKBdTWfs0pA]_[server.ccl.net> X-Original-From: Sergio Manzetti Content-Type: multipart/alternative; boundary=20cf307d0080fe843204af7ad27c Date: Mon, 17 Oct 2011 10:59:27 +0200 MIME-Version: 1.0 Sent to CCL by: Sergio Manzetti [sergio.manzetti!=!gmail.com] --20cf307d0080fe843204af7ad27c Content-Type: text/plain; charset=ISO-8859-1 Sebastian pointed out the details. It is an exclusive method which excludes those that cannot meet the criteria of the manifesto. Therefore it is closing in its effect on the population of potential scientific results and not to mention attempts by the scientific community to meet the criteria of the manifesto. The manifesto you mention should be the sole responsibility of each journal that aims to publish such work. Cheers Sergio On Mon, Oct 17, 2011 at 4:22 AM, Brian Salter-Duke brian.james.duke(-) gmail.com wrote: > > Sent to CCL by: Brian Salter-Duke [brian.james.duke]|[gmail.com] > I do not understand your point. The manifesto is supporting open > source. Any open source code meets all the criteria of the manifesto, > What is being closed? The manifesto is fully in accordance with what > Google does. > > Brian. > > On Mon, Oct 17, 2011 at 4:45 AM, Sergio Manzetti > sergio.manzetti++gmail.com wrote: > > I agree with Sebastian, what is so "open" about closing open source into > a > > manifesto? I was even more surprised that Google put their signature on > > that manifesto, a company who endorses free openness in information flow. > > > > Sergio > > > > > > > > On Fri, Oct 14, 2011 at 11:18 PM, Brian.James.Duke{:}gmail.com > > wrote: > >> > >> I just encountered the Science Code Manifesto, which essentially states > >> that all computer code used for scientific analysis and modeling should > >> be available for review. It appears to have started with the Climate > >> Code Foundation. I encourage you to visit the web site and consider > >> endorsing the Manifesto. > >> > >> http://sciencecodemanifesto.org/ > >> > >> Note that this is not specifying open source code, so GAMESS(US), > >> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd > >> MPQC satisfies the points of the manifesto. Of course some other quantum > >> chemistry codes do not. I think we should be putting pressure on the > >> authors of such codes to meet the criteria in this manifesto. > >> > >> Brian. > > > > > > > > -- > Brian Salter-Duke (aka Brian Duke) > Brian.James.Duke#%#gmail.com> > > --20cf307d0080fe843204af7ad27c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sebastian pointed out the details. It is an exclusive method which excludes= those that cannot meet the criteria of the manifesto. Therefore it is clos= ing in its effect on the population of potential scientific results and not= to mention attempts by the scientific community to meet the criteria of th= e manifesto. The manifesto you mention should be the sole responsibility of= each journal that aims to publish such work.

Cheers

Sergio

On Mon, Oct 17, = 2011 at 4:22 AM, Brian Salter-Duke brian.james.duke(-)gmail.com <owner-chemistry(_)ccl.net> wrote:

Sent to CCL by: Brian Salter-Duke [brian.james.duke]|[gmail.com]
I do not understand your point. The manifesto is supporting open
source. Any open source code meets all the criteria of the manifesto,
What is being closed? The manifesto is fully in accordance with what
Google does.

Brian.

On Mon, Oct 17, 2011 at 4:45 AM, Sergio Manzetti
sergio.manzetti++gmail.com <owner-chemistry#%#ccl.n= et> wrote:
> I agree with Sebastian, what is so "open" about closing open= source into a
> manifesto? I was=A0 even more surprised that Google put their si= gnature on
> that manifesto, a company who endorses free openness in information fl= ow.
>
> Sergio
>
>
>
> On Fri, Oct 14, 2011 at 11:18 PM, Brian.James.Duke{:}gmail.com
> <owner-chemistry]-[ccl.net> wrote:
>>
>> I just encountered the Science Code Manifesto, which essentially s= tates
>> that all computer code used for scientific analysis and modeling s= hould
>> be available for review. It appears to have started with the Clima= te
>> Code Foundation. I encourage you to visit the web site and conside= r
>> endorsing the Manifesto.
>>
>> htt= p://sciencecodemanifesto.org/
>>
>> Note that this is not specifying open source code, so GAMESS(US),<= br> >> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3= amd
>> MPQC satisfies the points of the manifesto. Of course some other q= uantum
>> chemistry codes do not. I think we should be putting pressure on t= he
>> authors of such codes to meet the criteria in this manifesto.
>>
>> Brian.
>
>



--
Brian Salter-Duke (aka Brian Duke)
Brian.James.Duke#%#gmail.com=



-=3D This is automatically added to each message by the mailing script =3D-=
E-mail to subscribers: CHEMISTRY(_)ccl.n= et or use:
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEM= ISTRY-REQUEST(_)ccl.net or use
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

Subscribe/Unsubscribe:
=A0 =A0 =A0http://www.ccl.net/chemistry/sub_unsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs
Conferences: http://server.ccl.net/chemistry/announcements/co= nferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.shtml
=A0 =A0 =A0h= ttp://www.ccl.net/spammers.txt

RTFI: http://www.ccl.net/chemistry/aboutccl/instructions/



--20cf307d0080fe843204af7ad27c-- From owner-chemistry@ccl.net Mon Oct 17 09:31:00 2011 From: "=?UTF-8?B?Sm/Do28gQnJhbmTDo28=?= jbrandao+/-ualg.pt" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45658-111017081615-1566-q4FDFhQhG0FXpa+wXfOsug/./server.ccl.net> X-Original-From: =?UTF-8?B?Sm/Do28gQnJhbmTDo28=?= Content-Type: multipart/alternative; boundary="------------010400040809000206060404" Date: Mon, 17 Oct 2011 13:15:56 +0100 MIME-Version: 1.0 Sent to CCL by: =?UTF-8?B?Sm/Do28gQnJhbmTDo28=?= [jbrandao(_)ualg.pt] This is a multi-part message in MIME format. --------------010400040809000206060404 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Sorry, but I disagree. "but if there are real concerns about the work, it is necessary that the scientific community can look seriously at the code to see exactly what the program is doing." In my opinion: If anyone has real concerns about any work, the best way for science development is to write (or use) a different code and compare the results. João Brandão Em 16-10-2011 22:33, Brian Salter-Duke brian.james.duke.:.gmail.com escreveu: > Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] > The review of manuscripts by referees prior to publication is just a > small part of what this is all about. The main point is about proper > review by the scientific community after publication. There, while not > everyone has access to a 600MHz NMR or has paid to get Gaussian, some > people in the community will have. The guys who paid for Gaussian > however, may not have bought the code and in some cases, it is > impossible to buy the code, as they are commercial secrets. This > manifesto is saying that commercial secrets are not compatible with > good science. > > Of course the need to this kind of review is not common, but if there > are real concerns about the work, it is necessary that the scientific > community can look seriously at the code to see exactly what the > program is doing. > > Brian. > > On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com > wrote: > >> I like the idea of open software, and in principle I support the proposal of >> having access to the software code for the possibility of full review of a >> theoretical work. However, I feel that this is more wishful thinking than >> real scientific life. >> >> Let’s say that I receive a manuscript for review that uses program X, which >> I may have access to its code now (since I’m the reviewer). Do I have to >> spend a week trying to understand its algorithms to check if the frequencies >> are correctly calculated? For me, most (if not all) the programs are in >> practice black boxes, as I’m hardly a programmer. Therefore, open or closed >> software (usually) doesn’t make me any difference, except from a >> philosophical perspective. >> >> Now, let me consider an analogy from experimental chemistry. I have to >> review a manuscript where the authors tested some compound with a 600 MHz >> NMR. My lab is much more humble, and I only have access to a 400 MHz NMR. Id >> est, I cannot reproduce the results of the paper that I’m reviewing. Is here >> any difference compared to the theoretical case? >> >> I had in a couple of cases problems to reproduce the results of papers, >> mostly because the authors didn’t provide enough information. However, for >> 100% reproducibility I need 100% the same conditions (and a lot of resources >> and time). This makes the selection of the software just a small issue. >> >> I would like to hear the thoughts of other people about this issue, as I >> consider the peer reviewing process a very complex and far from perfect >> system. >> >> >> >> >> Sebastian >> >> ________________________________ >> From: Brian.James.Duke{:}gmail.com >> To: "Kozuch, Sebastian " >> Sent: Friday, October 14, 2011 11:18 PM >> Subject: CCL: Science code manifesto >> >> I just encountered the Science Code Manifesto, which essentially states >> that all computer code used for scientific analysis and modeling should >> be available for review. It appears to have started with the Climate >> Code Foundation. I encourage you to visit the web site and consider >> endorsing the Manifesto. >> >> http://sciencecodemanifesto.org/ >> >> Note that this is not specifying open source code, so GAMESS(US), >> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd >> MPQC satisfies the points of the manifesto. Of course some other quantum >> chemistry codes do not. I think we should be putting pressure on the >> authors of such codes to meet the criteria in this manifesto. >> >> Brian. >> >> >> > > > --------------010400040809000206060404 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit Sorry, but I disagree.

 
"but if there are real concerns about the work, it is necessary that the scientific
community can look seriously at the code to see exactly what the program is doing."

In my opinion:
If anyone has real concerns about any work, the best way for science development is to write (or use) a different code and compare the results.

João Brandão




Em 16-10-2011 22:33, Brian Salter-Duke brian.james.duke.:.gmail.com escreveu: 
Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com]
The review of manuscripts by referees prior to publication is just a
small part of what this is all about. The main point is about proper
review by the scientific community after publication. There, while not
everyone has access to a 600MHz NMR or has paid to get Gaussian, some
people in the community will have. The guys who paid for Gaussian
however, may not have bought the code and in some cases, it is
impossible to buy the code, as they are commercial secrets. This
manifesto is saying that commercial secrets are not compatible with
good science.

Of course the need to this kind of review is not common, but if there
are real concerns about the work, it is necessary that the scientific
community can look seriously at the code to see exactly what the
program is doing.

Brian.

On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com
<owner-chemistry:+:ccl.net> wrote:
  
I like the idea of open software, and in principle I support the proposal of
having access to the software code for the possibility of full review of a
theoretical work. However, I feel that this is more wishful thinking than
real scientific life.

Let’s say that I receive a manuscript for review that uses program X, which
I may have access to its code now (since I’m the reviewer). Do I have to
spend a week trying to understand its algorithms to check if the frequencies
are correctly calculated? For me, most (if not all) the programs are in
practice black boxes, as I’m hardly a programmer. Therefore, open or closed
software (usually) doesn’t make me any difference, except from a
philosophical perspective.

Now, let me consider an analogy from experimental chemistry. I have to
review a manuscript where the authors tested some compound with a 600 MHz
NMR. My lab is much more humble, and I only have access to a 400 MHz NMR. Id
est, I cannot reproduce the results of the paper that I’m reviewing. Is here
any difference compared to the theoretical case?

I had in a couple of cases problems to reproduce the results of papers,
mostly because the authors didn’t provide enough information. However, for
100% reproducibility I need 100% the same conditions (and a lot of resources
and time). This makes the selection of the software just a small issue.

I would like to hear the thoughts of other people about this issue, as I
consider the peer reviewing process a very complex and far from perfect
system.




Sebastian

________________________________
> From: Brian.James.Duke{:}gmail.com <owner-chemistry()ccl.net>
To: "Kozuch, Sebastian " <kozuchs()yahoo.com>
Sent: Friday, October 14, 2011 11:18 PM
Subject: CCL: Science code manifesto

I just encountered the Science Code Manifesto, which essentially states
that all computer code used for scientific analysis and modeling should
be available for review. It appears to have started with the Climate
Code Foundation. I encourage you to visit the web site and consider
endorsing the Manifesto.

http://sciencecodemanifesto.org/

Note that this is not specifying open source code, so GAMESS(US),
GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd
MPQC satisfies the points of the manifesto. Of course some other quantum
chemistry codes do not. I think we should be putting pressure on the
authors of such codes to meet the criteria in this manifesto.

Brian.

--------------010400040809000206060404-- From owner-chemistry@ccl.net Mon Oct 17 10:05:00 2011 From: "Jim Kress ccl_nospam~~kressworks.com" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45659-111017091530-9759-212JmnH0TzufYRRwBEyN/A(_)server.ccl.net> X-Original-From: "Jim Kress" Content-Language: en-us Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" Date: Mon, 17 Oct 2011 09:15:17 -0400 MIME-Version: 1.0 Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com] Although it may be the kiss of death, I agree with Brian. The fundamental requisite of scientific work is that it must be reproducible by ANY OTHER SCIENTIST skilled in the art. The lack of access to the tools necessary to establish reproducibility should obviate the publication of any scientific paper until such access and detailed examination is made available. Jim Kress > -----Original Message----- > From: owner-chemistry+ccl_nospam==kressworks.com]-[ccl.net > [mailto:owner-chemistry+ccl_nospam==kressworks.com]-[ccl.net] On Behalf > Of Brian Salter-Duke brian.james.duke.:.gmail.com > Sent: Sunday, October 16, 2011 5:33 PM > To: Kress, Jim > Subject: CCL:G: Science code manifesto > > > Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The > review of manuscripts by referees prior to publication is just a small part of > what this is all about. The main point is about proper review by the scientific > community after publication. There, while not everyone has access to a > 600MHz NMR or has paid to get Gaussian, some people in the community will > have. The guys who paid for Gaussian however, may not have bought the > code and in some cases, it is impossible to buy the code, as they are > commercial secrets. This manifesto is saying that commercial secrets are not > compatible with good science. > > Of course the need to this kind of review is not common, but if there are real > concerns about the work, it is necessary that the scientific community can > look seriously at the code to see exactly what the program is doing. > > Brian. > > On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com > wrote: > > I like the idea of open software, and in principle I support the > > proposal of having access to the software code for the possibility of > > full review of a theoretical work. However, I feel that this is more > > wishful thinking than real scientific life. > > > > Let’s say that I receive a manuscript for review that uses program X, > > which I may have access to its code now (since I’m the reviewer). Do I > > have to spend a week trying to understand its algorithms to check if > > the frequencies are correctly calculated? For me, most (if not all) > > the programs are in practice black boxes, as I’m hardly a programmer. > > Therefore, open or closed software (usually) doesn’t make me any > > difference, except from a philosophical perspective. > > > > Now, let me consider an analogy from experimental chemistry. I have to > > review a manuscript where the authors tested some compound with a 600 > > MHz NMR. My lab is much more humble, and I only have access to a 400 > > MHz NMR. Id est, I cannot reproduce the results of the paper that I’m > > reviewing. Is here any difference compared to the theoretical case? > > > > I had in a couple of cases problems to reproduce the results of > > papers, mostly because the authors didn’t provide enough information. > > However, for 100% reproducibility I need 100% the same conditions (and > > a lot of resources and time). This makes the selection of the software just a > small issue. > > > > I would like to hear the thoughts of other people about this issue, as > > I consider the peer reviewing process a very complex and far from > > perfect system. > > > > > > > > > > Sebastian > > > > ________________________________ > > From: Brian.James.Duke{:}gmail.com > > To: "Kozuch, Sebastian " > > Sent: Friday, October 14, 2011 11:18 PM > > Subject: CCL: Science code manifesto > > > > I just encountered the Science Code Manifesto, which essentially > > states that all computer code used for scientific analysis and > > modeling should be available for review. It appears to have started > > with the Climate Code Foundation. I encourage you to visit the web > > site and consider endorsing the Manifesto. > > > > http://sciencecodemanifesto.org/ > > > > Note that this is not specifying open source code, so GAMESS(US), > > GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd > > MPQC satisfies the points of the manifesto. Of course some other > > quantum chemistry codes do not. I think we should be putting pressure > > on the authors of such codes to meet the criteria in this manifesto. > > > > Brian. > > > > > > > > -- > Brian Salter-Duke (aka Brian Duke) > Brian.James.Duke:+:gmail.com> From owner-chemistry@ccl.net Mon Oct 17 10:55:00 2011 From: "Pedro Silva pedros^ufp.edu.pt" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45660-111017105126-7011-aNFWrOSxLbTqfvTpwaXihg]|[server.ccl.net> X-Original-From: Pedro Silva Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 17 Oct 2011 15:51:16 +0100 MIME-Version: 1.0 Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt] 2011/10/17 João Brandão jbrandao+/-ualg.pt : > Sorry, but I disagree. > > "but if there are real concerns about the work, it is necessary that the > scientific > community can look seriously at the code to see exactly what the program is > doing." > > In my opinion: > If anyone has real concerns about any work, the best way for science > development is to write (or use) a different code and compare the results. > The problem is: if the results do not agree with each other, how can you adjudicate between the competing claims without access to the code? From owner-chemistry@ccl.net Mon Oct 17 11:38:00 2011 From: "Herbert Fruchtl herbert.fruchtl{}st-andrews.ac.uk" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45661-111017113509-11551-nG3Pwo15+Uxzf8UEA5/YBA::server.ccl.net> X-Original-From: Herbert Fruchtl Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Date: Mon, 17 Oct 2011 16:36:02 +0100 MIME-Version: 1.0 Sent to CCL by: Herbert Fruchtl [herbert.fruchtl a st-andrews.ac.uk] Strictly speaking, it's reproducible if the method is competely published. You won't sit down and code it to check the results in one paper, but if a method is useful, sooner or later somebody else will write another program implementing it. Luckily, software patents haven't taken root in computational chemistry yet. I seem to remember that in the early days of DFT, commercial programs with closed-source functionals started appearing. I don't know if they are still in use in some industrial settings, but results obtained with them would certainly not be publishable. Herbert On 17/10/11 14:15, Jim Kress ccl_nospam~~kressworks.com wrote: > > Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com] > Although it may be the kiss of death, I agree with Brian. > > The fundamental requisite of scientific work is that it must be reproducible by ANY OTHER SCIENTIST skilled in the art. The lack of access to the tools necessary to establish reproducibility should obviate the publication of any scientific paper until such access and detailed examination is made available. > > Jim Kress > >> -----Original Message----- >> From: owner-chemistry+ccl_nospam==kressworks.com ~~ ccl.net >> [mailto:owner-chemistry+ccl_nospam==kressworks.com ~~ ccl.net] On Behalf >> Of Brian Salter-Duke brian.james.duke.:.gmail.com >> Sent: Sunday, October 16, 2011 5:33 PM >> To: Kress, Jim >> Subject: CCL:G: Science code manifesto >> >> >> Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The >> review of manuscripts by referees prior to publication is just a small part of >> what this is all about. The main point is about proper review by the scientific >> community after publication. There, while not everyone has access to a >> 600MHz NMR or has paid to get Gaussian, some people in the community will >> have. The guys who paid for Gaussian however, may not have bought the >> code and in some cases, it is impossible to buy the code, as they are >> commercial secrets. This manifesto is saying that commercial secrets are not >> compatible with good science. >> >> Of course the need to this kind of review is not common, but if there are real >> concerns about the work, it is necessary that the scientific community can >> look seriously at the code to see exactly what the program is doing. >> >> Brian. >> >> On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com >> wrote: >>> I like the idea of open software, and in principle I support the >>> proposal of having access to the software code for the possibility of >>> full review of a theoretical work. However, I feel that this is more >>> wishful thinking than real scientific life. >>> >>> Let’s say that I receive a manuscript for review that uses program X, >>> which I may have access to its code now (since I’m the reviewer). Do I >>> have to spend a week trying to understand its algorithms to check if >>> the frequencies are correctly calculated? For me, most (if not all) >>> the programs are in practice black boxes, as I’m hardly a programmer. >>> Therefore, open or closed software (usually) doesn’t make me any >>> difference, except from a philosophical perspective. >>> >>> Now, let me consider an analogy from experimental chemistry. I have to >>> review a manuscript where the authors tested some compound with a 600 >>> MHz NMR. My lab is much more humble, and I only have access to a 400 >>> MHz NMR. Id est, I cannot reproduce the results of the paper that I’m >>> reviewing. Is here any difference compared to the theoretical case? >>> >>> I had in a couple of cases problems to reproduce the results of >>> papers, mostly because the authors didn’t provide enough information. >>> However, for 100% reproducibility I need 100% the same conditions (and >>> a lot of resources and time). This makes the selection of the software just a >> small issue. >>> >>> I would like to hear the thoughts of other people about this issue, as >>> I consider the peer reviewing process a very complex and far from >>> perfect system. >>> >>> >>> >>> >>> Sebastian >>> >>> ________________________________ >>> From: Brian.James.Duke{:}gmail.com >>> To: "Kozuch, Sebastian " >>> Sent: Friday, October 14, 2011 11:18 PM >>> Subject: CCL: Science code manifesto >>> >>> I just encountered the Science Code Manifesto, which essentially >>> states that all computer code used for scientific analysis and >>> modeling should be available for review. It appears to have started >>> with the Climate Code Foundation. I encourage you to visit the web >>> site and consider endorsing the Manifesto. >>> >>> http://sciencecodemanifesto.org/ >>> >>> Note that this is not specifying open source code, so GAMESS(US), >>> GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd >>> MPQC satisfies the points of the manifesto. Of course some other >>> quantum chemistry codes do not. I think we should be putting pressure >>> on the authors of such codes to meet the criteria in this manifesto. >>> >>> Brian. >>> >>> >> >> >> >> -- >> Brian Salter-Duke (aka Brian Duke) >> Brian.James.Duke:+:gmail.com> > -- Herbert Fruchtl Senior Scientific Computing Officer School of Chemistry, School of Mathematics and Statistics University of St Andrews -- The University of St Andrews is a charity registered in Scotland: No SC013532 From owner-chemistry@ccl.net Mon Oct 17 12:37:01 2011 From: "=?ISO-8859-1?Q?Jordi_Vill=E0_i_Freixa?= jordi.villafreixa/a\gmail.com" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45662-111017115137-1836-ey6VUXEDFuv++f3z4s6diQ-,-server.ccl.net> X-Original-From: =?ISO-8859-1?Q?Jordi_Vill=E0_i_Freixa?= Content-Type: multipart/alternative; boundary=002215b02e325607ce04af8094c8 Date: Mon, 17 Oct 2011 17:51:26 +0200 MIME-Version: 1.0 Sent to CCL by: =?ISO-8859-1?Q?Jordi_Vill=E0_i_Freixa?= [jordi.villafreixa\a/gmail.com] --002215b02e325607ce04af8094c8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Allow me to be drastic here. In my opinion this is a quite absurd discussion. Having the ability to make code transparent for others to reuse and build on top of is obviously way better for science than having black boxes. It is not needed to provide the ability to the referee (who by definition is a busy guy) to rerun the calculations, but to the scientific community as a whole. It may be the tim= e already to reduce the often over-critic referee system and move to a more open science schema. Repositories for code have been extremely useful in th= e past and I think being concerned about the ability of the scientific community to improve our results is way more positive that believing that w= e and only we can run a code and understand its results. Having myself a foot in enterpreneourship in addition to academia, I think that closing/selling code is futurless in a world with tens of thousands of developers ready for better implementations for each new idea. What makes sense is understanding the possible use of each technology, not closing the= m in a useless effort of overprotection. So, the need for the manifesto is clear. Start opening our eyes to a wider conception of science, far from egoistic and closed minded uses of it. The system should follow this, one day or another. 2011/10/17 Pedro Silva pedros^ufp.edu.pt > > Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt] > 2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt : > > Sorry, but I disagree. > > > > "but if there are real concerns about the work, it is necessary that th= e > > scientific > > community can look seriously at the code to see exactly what the progra= m > is > > doing." > > > > In my opinion: > > If anyone has real concerns about any work, the best way for science > > development is to write (or use) a different code and compare the > results. > > > > The problem is: if the results do not agree with each other, how can > you adjudicate between the competing claims without access to the > code? > > > > -=3D This is automatically added to each message by the mailing script = =3D-> > > --=20 -- Jordi Vill=E0 i Freixa Computational Biochemistry and Biophysics lab Research Program on Biomedical Informatics (GRIB) - IMIM/UPF Parc de Recerca Biom=E8dica de Barcelona C/ Doctor Aiguader, 88; 08003 Barcelona (Spain) Tel: +34 93 316 0504 // Fax: +34 93 316 0550 e-mail: jvilla|*|imim.es http://cbbl.imim.es --002215b02e325607ce04af8094c8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Allow me to be drastic here.

In my opinion this is a qui= te absurd discussion. Having the ability to make code transparent for other= s to reuse and build on top of is obviously way better for science than hav= ing black boxes. It is not needed to provide the ability to the referee (wh= o by definition is a busy guy) to rerun the calculations, but to the scient= ific community as a whole. It may be the time already to reduce the often o= ver-critic referee system and move to a more open science schema. Repositor= ies for code have been extremely useful in the past and I think being conce= rned about the ability of the scientific community to improve our results i= s way more positive that believing that we and only we can run a code and u= nderstand its results.

Having myself a foot in enterpreneourship in addition to aca= demia, I think that closing/selling code is futurless in a world with tens = of thousands of developers ready for better implementations for each new id= ea. What makes sense is understanding the possible use of each technology, = not closing them in a useless effort of overprotection.

So, the need for the manifesto is clear. Start opening = our eyes to a wider conception of science, far from egoistic and closed min= ded uses of it. The system should follow this, one day or another.



2011/10/17 Pedro Sil= va pedros^ufp.edu.pt &l= t;owner-chemistry|*|ccl.net>= ;

Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt]
2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt <owner-chemistry^-^ccl.net>:
> Sorry, but I disagree.
>
> "but if there are real concerns about the work, it is necessary t= hat the
> scientific
> community can look seriously at the code to see exactly what the progr= am is
> doing."
>
> In my opinion:
> If anyone has real concerns about any work, the best way for science > development is to write (or use) a different code and compare the resu= lts.
>

The problem is: if the results do not agree with each other, how can
you adjudicate between the competing claims without access to the
code?



-=3D This is automatically added to each message by the mailing script =3D-=
E-mail to subscribers: CHEMISTRY|*|ccl.n= et or use:
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEM= ISTRY-REQUEST|*|ccl.net or use
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

Subscribe/Unsubscribe:
=A0 =A0 =A0http://www.ccl.net/chemistry/sub_unsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs
Conferences: http://server.ccl.net/chemistry/announcements/co= nferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.shtml
=A0 =A0 =A0h= ttp://www.ccl.net/spammers.txt

RTFI: http://www.ccl.net/chemistry/aboutccl/instructions/





--
--
Jordi = Vill=E0 i Freixa
Computational Biochemistry and Biophysics lab
Resear= ch Program on Biomedical Informatics (GRIB) - IMIM/UPF
Parc de Recerca B= iom=E8dica de Barcelona
C/ Doctor Aiguader, 88; 08003 Barcelona (Spain)
Tel: +34 93 316 0504 // = Fax: +34 93 316 0550
e-mail: jvilla|*|imim.es http://cbbl.imim.es
--002215b02e325607ce04af8094c8-- From owner-chemistry@ccl.net Mon Oct 17 13:12:01 2011 From: "Sergio Manzetti sergio.manzetti]~[gmail.com" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45663-111017120314-12798-lpniXhCdMNwtUyzmWHAH3g_-_server.ccl.net> X-Original-From: Sergio Manzetti Content-Type: multipart/alternative; boundary=20cf307c9b26052b2704af80bebc Date: Mon, 17 Oct 2011 18:03:05 +0200 MIME-Version: 1.0 Sent to CCL by: Sergio Manzetti [sergio.manzetti|-|gmail.com] --20cf307c9b26052b2704af80bebc Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Jim, althoughit may be a kiss of life, I agree with Sebastian. The availibility of tools does not mean that it is available to any scientist as long as a manifesto governs the work and responsibility of a journal to identify the work as correct or not. The reproducibility is enitrely in the hands of the journals, and a manifesto as the one Brian suggests does not replace the journals responsibility or change the skills of a reviewer. As any other manifesto, itattempts to trap principle in time= , but with time any principle dissolves into new hands and new generations. The ultimate responsibility is in the reviewers to request information, and if they do not, the editor should be aware of the quality of the reviewers work. Sergio On Mon, Oct 17, 2011 at 3:15 PM, Jim Kress ccl_nospam~~kressworks.com < owner-chemistry^ccl.net> wrote: > > Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com] > Although it may be the kiss of death, I agree with Brian. > > The fundamental requisite of scientific work is that it must be > reproducible by ANY OTHER SCIENTIST skilled in the art. The lack of acce= ss > to the tools necessary to establish reproducibility should obviate the > publication of any scientific paper until such access and detailed > examination is made available. > > Jim Kress > > > -----Original Message----- > > From: owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net > > [mailto:owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net] On B= ehalf > > Of Brian Salter-Duke brian.james.duke.:.gmail.com > > Sent: Sunday, October 16, 2011 5:33 PM > > To: Kress, Jim > > Subject: CCL:G: Science code manifesto > > > > > > Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The > > review of manuscripts by referees prior to publication is just a small > part of > > what this is all about. The main point is about proper review by the > scientific > > community after publication. There, while not everyone has access to a > > 600MHz NMR or has paid to get Gaussian, some people in the community wi= ll > > have. The guys who paid for Gaussian however, may not have bought the > > code and in some cases, it is impossible to buy the code, as they are > > commercial secrets. This manifesto is saying that commercial secrets ar= e > not > > compatible with good science. > > > > Of course the need to this kind of review is not common, but if there a= re > real > > concerns about the work, it is necessary that the scientific community > can > > look seriously at the code to see exactly what the program is doing. > > > > Brian. > > > > On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com > > wrote: > > > I like the idea of open software, and in principle I support the > > > proposal of having access to the software code for the possibility of > > > full review of a theoretical work. However, I feel that this is more > > > wishful thinking than real scientific life. > > > > > > Let=92s say that I receive a manuscript for review that uses program = X, > > > which I may have access to its code now (since I=92m the reviewer). D= o I > > > have to spend a week trying to understand its algorithms to check if > > > the frequencies are correctly calculated? For me, most (if not all) > > > the programs are in practice black boxes, as I=92m hardly a programme= r. > > > Therefore, open or closed software (usually) doesn=92t make me any > > > difference, except from a philosophical perspective. > > > > > > Now, let me consider an analogy from experimental chemistry. I have t= o > > > review a manuscript where the authors tested some compound with a 600 > > > MHz NMR. My lab is much more humble, and I only have access to a 400 > > > MHz NMR. Id est, I cannot reproduce the results of the paper that I= =92m > > > reviewing. Is here any difference compared to the theoretical case? > > > > > > I had in a couple of cases problems to reproduce the results of > > > papers, mostly because the authors didn=92t provide enough informatio= n. > > > However, for 100% reproducibility I need 100% the same conditions (an= d > > > a lot of resources and time). This makes the selection of the softwar= e > just a > > small issue. > > > > > > I would like to hear the thoughts of other people about this issue, a= s > > > I consider the peer reviewing process a very complex and far from > > > perfect system. > > > > > > > > > > > > > > > Sebastian > > > > > > ________________________________ > > > From: Brian.James.Duke{:}gmail.com > > > To: "Kozuch, Sebastian " > > > Sent: Friday, October 14, 2011 11:18 PM > > > Subject: CCL: Science code manifesto > > > > > > I just encountered the Science Code Manifesto, which essentially > > > states that all computer code used for scientific analysis and > > > modeling should be available for review. It appears to have started > > > with the Climate Code Foundation. I encourage you to visit the web > > > site and consider endorsing the Manifesto. > > > > > > http://sciencecodemanifesto.org/ > > > > > > Note that this is not specifying open source code, so GAMESS(US), > > > GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 am= d > > > MPQC satisfies the points of the manifesto. Of course some other > > > quantum chemistry codes do not. I think we should be putting pressure > > > on the authors of such codes to meet the criteria in this manifesto. > > > > > > Brian. > > > > > > > > > > > > > > -- > > Brian Salter-Duke (aka Brian Duke) > > Brian.James.Duke:+:gmail.com> > > > > -=3D This is automatically added to each message by the mailing script = =3D-> > > --20cf307c9b26052b2704af80bebc Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Jim, althoughit=A0 may be a kiss of life, I agree with Sebastian.

Th= e availibility of tools does not mean that it is available to any scientist= as long as a manifesto governs the work and responsibility of a journal to= identify the work as correct or not. The reproducibility is enitrely in th= e hands of the journals, and a manifesto as the one Brian suggests does not= replace the journals responsibility or change the skills of a reviewer. As= any other manifesto, itattempts to trap principle in time, but with time a= ny principle dissolves into new hands and new generations. The ultimate res= ponsibility is in the reviewers to request information, and if they do not,= the editor should be aware of the quality of the reviewers work.

Sergio

On Mon, Oct 17, 2011 at 3:15 P= M, Jim Kress ccl_nospam~~kressworks.com <owner-c= hemistry^ccl.net> wrote:

Sent to CCL by: "Jim Kress" [ccl_nospam .. kressworks.com]
Although it may be the kiss of death, I agree with Brian.

The fundamental requisite of scientific work is that it must be reproducibl= e by ANY OTHER SCIENTIST skilled in the art. =A0The lack of access to the t= ools necessary to establish reproducibility should obviate the publication = of any scientific paper until such access and detailed examination is made = available.

Jim Kress

> -----Original Message-----
> From: owner-chemistry+ccl_nospam=3D=3Dkressworks.com ~~ ccl.net
> [mailto:owner-chemistr= y+ccl_nospam=3D=3Dk= ressworks.com ~~ ccl.net] On Behalf
> Of Brian Salter-Duke brian.james.duke.:.gmail.com
> Sent: Sunday, October 16, 2011 5:33 PM
> To: Kress, Jim
> Subject: CCL:G: Science code manifesto
>
>
> Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] The
> review of manuscripts by referees prior to publication is just a small= part of
> what this is all about. The main point is about proper review by the s= cientific
> community after publication. There, while not everyone has access to a=
> 600MHz NMR or has paid to get Gaussian, some people in the community w= ill
> have. The guys who paid for Gaussian however, may not have bought the<= br> > code and in some cases, it is impossible to buy the code, as they are<= br> > commercial secrets. This manifesto is saying that commercial secrets a= re not
> compatible with good science.
>
> Of course the need to this kind of review is not common, but if there = are real
> concerns about the work, it is necessary that the scientific community= can
> look seriously at the code to see exactly what the program is doing. >
> Brian.
>
> On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com
> <owner-chemistry:+:ccl= .net> wrote:
> > I like the idea of open software, and in principle I support the<= br> > > proposal of having access to the software code for the possibilit= y of
> > full review of a theoretical work. However, I feel that this is m= ore
> > wishful thinking than real scientific life.
> >
> > Let=92s say that I receive a manuscript for review that uses prog= ram X,
> > which I may have access to its code now (since I=92m the reviewer= ). Do I
> > have to spend a week trying to understand its algorithms to check= if
> > the frequencies are correctly calculated? For me, most (if not al= l)
> > the programs are in practice black boxes, as I=92m hardly a progr= ammer.
> > Therefore, open or closed software (usually) doesn=92t make me an= y
> > difference, except from a philosophical perspective.
> >
> > Now, let me consider an analogy from experimental chemistry. I ha= ve to
> > review a manuscript where the authors tested some compound with a= 600
> > MHz NMR. My lab is much more humble, and I only have access to a = 400
> > MHz NMR. Id est, I cannot reproduce the results of the paper that= I=92m
> > reviewing. Is here any difference compared to the theoretical cas= e?
> >
> > I had in a couple of cases problems to reproduce the results of > > papers, mostly because the authors didn=92t provide enough inform= ation.
> > However, for 100% reproducibility I need 100% the same conditions= (and
> > a lot of resources and time). This makes the selection of the sof= tware just a
> small issue.
> >
> > I would like to hear the thoughts of other people about this issu= e, as
> > I consider the peer reviewing process a very complex and far from=
> > perfect system.
> >
> >
> >
> >
> > Sebastian
> >
> > ________________________________
> > From: Brian.James.Duke{:}gmail.com <owner-chemistry()ccl.net>
> > To: "Kozuch, Sebastian " <kozuchs()yahoo.com>
> > Sent: Friday, October 14, 2011 11:18 PM
> > Subject: CCL: Science code manifesto
> >
> > I just encountered the Science Code Manifesto, which essentially<= br> > > states that all computer code used for scientific analysis and > > modeling should be available for review. It appears to have start= ed
> > with the Climate Code Foundation. I encourage you to visit the we= b
> > site and consider endorsing the Manifesto.
> >
> > ht= tp://sciencecodemanifesto.org/
> >
> > Note that this is not specifying open source code, so GAMESS(US),=
> > GAMESS(UK), DALTON etc., as well as open source codes such as PSI= 3 amd
> > MPQC satisfies the points of the manifesto. Of course some other<= br> > > quantum chemistry codes do not. I think we should be putting pres= sure
> > on the authors of such codes to meet the criteria in this manifes= to.
> >
> > Brian.
> >
> >
>
>
>
> --
> Brian Salter-Duke (aka Brian Duke)
> Brian.James.Duke:+:gmai= l.com>



-=3D This is automatically added to each message by the mailing script =3D-=
E-mail to subscribers: CHEMISTRY^ccl.n= et or use:
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEM= ISTRY-REQUEST^ccl.net or use
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

Subscribe/Unsubscribe:
=A0 =A0 =A0http://www.ccl.net/chemistry/sub_unsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs
Conferences: http://server.ccl.net/chemistry/announcements/co= nferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.shtml
=A0 =A0 =A0h= ttp://www.ccl.net/spammers.txt

RTFI: http://www.ccl.net/chemistry/aboutccl/instructions/





--20cf307c9b26052b2704af80bebc-- From owner-chemistry@ccl.net Mon Oct 17 13:47:01 2011 From: "Christopher Cramer cramer**umn.edu" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45664-111017122328-29372-5s0h/w8IAthsSxOVBjp/GQ*|*server.ccl.net> X-Original-From: Christopher Cramer Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=us-ascii Date: Mon, 17 Oct 2011 18:22:30 +0200 Mime-Version: 1.0 (Apple Message framework v1084) Sent to CCL by: Christopher Cramer [cramer|umn.edu] As I grow older, I carve notches into a stick each time Fortran vs. C++ and open-source vs. purchased executables comes up on CCL. It's getting to where I'll soon need a new stick. Personally, I think there is a middle ground in this doctrinaire argument (the latter one -- I simply ignore the Fortran vs. C++ one). Let us accept that all scientific reports accepted by reputable disseminators of information should be reproducible, as a matter of principle. But, let's also be serious -- after you read a fascinating paper on some conclusions derived from a complicated femtosecond spectroscopy experiment, are you going to spend $2M to build your own seriously decked out laser table in order to verify that the critical Figure 6 of the paper is not fabricated? Are you going to insist that the author shut his or her lab down for a week while you come and demand your sacred right to reproduce the experiment on his or her existing table? At some level, if only for the sake of efficiency, one must have some trust in one's colleagues' scientific integrity -- and, in those rare instances where someone DOES fabricate results, if they are actually INTERESTING results, then some other group somewhere WILL put together the analogous laser and publish competing results and the community will sort it all out the way it did with cold fusion, polywater, plastic fantastic, etc. As for computational chemistry, my opinion would be that all theories/models MUST be reported in the literature in a sufficiently detailed manner that someone skilled in the art (to borrow Jim Kress' phrase taken from IP law) COULD implement them into code (assuming the goal is to report future results from that new theory/model). That implementation might be nowhere near as fast as a proprietary one, but it allows results to be checked. If results FAIL to agree with a proprietary code, that's certainly a legitimate thing to note in a publication and that will likely inspire a third party to try it -- or the for-profit organization to revisit its code -- and the community will sort it out. Advocacy for all code being open-source is a values-based activity -- and I would not want to imply that I denigrate the philosophy underlying such advocacy. I will say, however, that the success of the scientific endeavor does not depend so critically upon it that its value is self-evident. Chris -- Christopher J. Cramer Elmore H. Northey Professor University of Minnesota Department of Chemistry 207 Pleasant St. SE Minneapolis, MN 55455-0431 -------------------------- Phone: (612) 624-0859 || FAX: (612) 626-7541 Mobile: (952) 297-2575 email: cramer/./umn.edu jabber: cramer/./jabber.umn.edu http://pollux.chem.umn.edu (website includes information about the textbook "Essentials of Computational Chemistry: Theories and Models, 2nd Edition") From owner-chemistry@ccl.net Mon Oct 17 14:54:01 2011 From: "Sergio Manzetti sergio.manzetti||gmail.com" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45665-111017143537-6872-3s7dA6s3iGvgVW68U3wihw:+:server.ccl.net> X-Original-From: Sergio Manzetti Content-Type: multipart/alternative; boundary=bcaec548a7e70c5f1a04af82dfd0 Date: Mon, 17 Oct 2011 20:35:29 +0200 MIME-Version: 1.0 Sent to CCL by: Sergio Manzetti [sergio.manzetti|,|gmail.com] --bcaec548a7e70c5f1a04af82dfd0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Jordi, the discussion is not absurd, its a result and reflection of the community part of it. Brian asked for the discussion by proposing the manifesto on the list, and Sebastion has furthered the discussion. Having a response saying that the discussion is absurd, is absurd. Absurd or not, th= e discussion is what it is, agreed or not agreed with you. With my sincere best regards Sergio 2011/10/17 Jordi Vill=E0 i Freixa jordi.villafreixa/agmail.com < owner-chemistry[-]ccl.net> > Allow me to be drastic here. > > In my opinion this is a quite absurd discussion. Having the ability to ma= ke > code transparent for others to reuse and build on top of is obviously way > better for science than having black boxes. It is not needed to provide t= he > ability to the referee (who by definition is a busy guy) to rerun the > calculations, but to the scientific community as a whole. It may be the t= ime > already to reduce the often over-critic referee system and move to a more > open science schema. Repositories for code have been extremely useful in = the > past and I think being concerned about the ability of the scientific > community to improve our results is way more positive that believing that= we > and only we can run a code and understand its results. > > Having myself a foot in enterpreneourship in addition to academia, I thin= k > that closing/selling code is futurless in a world with tens of thousands = of > developers ready for better implementations for each new idea. What makes > sense is understanding the possible use of each technology, not closing t= hem > in a useless effort of overprotection. > > So, the need for the manifesto is clear. Start opening our eyes to a wide= r > conception of science, far from egoistic and closed minded uses of it. Th= e > system should follow this, one day or another. > > > > 2011/10/17 Pedro Silva pedros^ufp.edu.pt > >> >> Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt] >> 2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt : >> > Sorry, but I disagree. >> > >> > "but if there are real concerns about the work, it is necessary that t= he >> > scientific >> > community can look seriously at the code to see exactly what the progr= am >> is >> > doing." >> > >> > In my opinion: >> > If anyone has real concerns about any work, the best way for science >> > development is to write (or use) a different code and compare the >> results. >> > >> >> The problem is: if the results do not agree with each other, how can >> you adjudicate between the competing claims without access to the >> code? >> >> >> >> -=3D This is automatically added to each message by the mailing script = =3D-** >> E-mail to subscribers: CHEMISTRY(!)ccl.net or use:>> >> E-mail to administrators: CHEMISTRY-REQUEST(!)ccl.net or use>> **>> >> >> > > > -- > -- > Jordi Vill=E0 i Freixa > Computational Biochemistry and Biophysics lab > Research Program on Biomedical Informatics (GRIB) - IMIM/UPF > Parc de Recerca Biom=E8dica de Barcelona > C/ Doctor Aiguader, 88; 08003 Barcelona (Spain) > Tel: +34 93 316 0504 // Fax: +34 93 316 0550 > e-mail: jvilla(!)imim.es http://cbbl.imim.es > -- --bcaec548a7e70c5f1a04af82dfd0 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Jordi, the discussion is not absurd, its a result and reflection of the com= munity part of it. Brian asked for the discussion by proposing the manifest= o on the list, and Sebastion has furthered the discussion. Having a respons= e saying that the discussion is absurd, is absurd. Absurd or not, the discu= ssion is what it is, agreed or=A0 not agreed with you.

With my sincere best regards

Sergio

2011/10/17 Jordi Vill=E0 i Freixa jordi.villafreixa/agmail.com <owner-chemistry[-]ccl.net>
Allow me to be drastic here.

=
In my opinion this is a quite absurd discussion. Having the ability to= make code transparent for others to reuse and build on top of is obviously= way better for science than having black boxes. It is not needed to provid= e the ability to the referee (who by definition is a busy guy) to rerun the= calculations, but to the scientific community as a whole. It may be the ti= me already to reduce the often over-critic referee system and move to a mor= e open science schema. Repositories for code have been extremely useful in = the past and I think being concerned about the ability of the scientific co= mmunity to improve our results is way more positive that believing that we = and only we can run a code and understand its results.

Having myself a foot in enterpreneourship in addition to aca= demia, I think that closing/selling code is futurless in a world with tens = of thousands of developers ready for better implementations for each new id= ea. What makes sense is understanding the possible use of each technology, = not closing them in a useless effort of overprotection.

So, the need for the manifesto is clear. Start opening = our eyes to a wider conception of science, far from egoistic and closed min= ded uses of it. The system should follow this, one day or another.



2011/10/17 Pedro Sil= va pedros^ufp.edu.pt <owner-chemistry(!)ccl.net>

Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt]
2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt <owner-chemistry^-^ccl.net>:
> Sorry, but I disagree.
>
> "but if there are real concerns about the work, it is necessary t= hat the
> scientific
> community can look seriously at the code to see exactly what the progr= am is
> doing."
>
> In my opinion:
> If anyone has real concerns about any work, the best way for science > development is to write (or use) a different code and compare the resu= lts.
>

The problem is: if the results do not agree with each other, how can
you adjudicate between the competing claims without access to the
code?



-=3D This is automatically added to each message by the mailing script =3D-=
E-mail to subscribers: CHEMISTRY(!)ccl.net or use:
=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEMISTRY-REQUEST(!)ccl.net or use

=A0 =A0 =A0http://www.ccl.net/cgi-bin/ccl/send_ccl_message

Subscribe/Unsubscribe:
=A0 =A0 =A0http://www.ccl.net/chemistry/sub_unsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs
Conferences: http://server.ccl.net/chemistry/announcements/co= nferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.shtml
=A0 =A0 =A0h= ttp://www.ccl.net/spammers.txt

RTFI: http://www.ccl.net/chemistry/aboutccl/instructions/





--
--
Jordi Vill=E0 i Freixa
Computational Biochemistr= y and Biophysics lab
Research Program on Biomedical Informatics (GRIB) -= IMIM/UPF
Parc de Recerca Biom=E8dica de Barcelona
C/ Doctor Aiguader, 88; 08003 Barcelona (Spain)
Tel: +34 93 316 0504 // Fa= x: +34 93 316 0550
e-mail: jvilla(= !)imim.es http://cbbl= .imim.es



--
--bcaec548a7e70c5f1a04af82dfd0-- From owner-chemistry@ccl.net Mon Oct 17 15:29:00 2011 From: "Dr W.R. Pitt wrp24*_*cam.ac.uk" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45666-111017143732-8473-GHRDN5EK5qrRKAanIBUWng!^!server.ccl.net> X-Original-From: "Dr W.R. Pitt" Content-Transfer-Encoding: 8bit Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Date: 17 Oct 2011 19:37:24 +0100 Mime-Version: 1.0 Sent to CCL by: "Dr W.R. Pitt" [wrp24!=!cam.ac.uk] I have two points to add to the discussion: 1/ As far as I can tell the the Science Code Manifesto does not insist on the use of open source software. In the discussion page it states "Use of languages, libraries, systems, and tools which are widely available is strongly recommended." The data analysis script should be made available but the manifesto seems to leave room for that script to be written using a commercial toolkit. I use commercial software as well as free software. Sometimes the former is only way (within the time I have available) of doing the job I have in mind. From my point of view, it is better to do science using commercial software and publish it than not doing it all. 2/ I do agree that the analysis scripts should be made public. However, I'm not sure it is necessary for a referee to be able to re-run the analysis script and check every last detail. The wider peer community would have the opportunity to do this. A natural consequence of this would be multiple versions of results in papers, derived from bug-fixes, following reader input. Much more rigorous testing of work prior to publication should result, which can only be a good thing. Relivant articles : Peer reviewers swamped, so extras get the heave-ho http://www.timeshighereducation.co.uk/story.asp?storyCode=413390§ioncode=26 Report from the Publishing Open Data Working Group meeting http://blogs.openaccesscentral.com/blogs/bmcblog/entry/report_from_the_publishing_open On Oct 17 2011, Jordi Villà i Freixa jordi.villafreixa/agmail.com wrote: >Allow me to be drastic here. > > In my opinion this is a quite absurd discussion. Having the ability to > make code transparent for others to reuse and build on top of is > obviously way better for science than having black boxes. It is not > needed to provide the ability to the referee (who by definition is a busy > guy) to rerun the calculations, but to the scientific community as a > whole. It may be the time already to reduce the often over-critic referee > system and move to a more open science schema. Repositories for code have > been extremely useful in the past and I think being concerned about the > ability of the scientific community to improve our results is way more > positive that believing that we and only we can run a code and understand > its results. > > Having myself a foot in enterpreneourship in addition to academia, I > think that closing/selling code is futurless in a world with tens of > thousands of developers ready for better implementations for each new > idea. What makes sense is understanding the possible use of each > technology, not closing them in a useless effort of overprotection. > >So, the need for the manifesto is clear. Start opening our eyes to a wider >conception of science, far from egoistic and closed minded uses of it. The >system should follow this, one day or another. > > > >2011/10/17 Pedro Silva pedros^ufp.edu.pt > >> >> Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt] >> 2011/10/17 João Brandão jbrandao+/-ualg.pt : >> > Sorry, but I disagree. >> > >> > "but if there are real concerns about the work, it is necessary that >> > the scientific community can look seriously at the code to see exactly >> > what the program >> is >> > doing." >> > >> > In my opinion: >> > If anyone has real concerns about any work, the best way for science >> > development is to write (or use) a different code and compare the >> results. >> > >> >> The problem is: if the results do not agree with each other, how can >> you adjudicate between the competing claims without access to the >> code?> >> >> > > > From owner-chemistry@ccl.net Mon Oct 17 16:05:00 2011 From: "Andrew Dalke dalke . dalkescientific.com" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45667-111017124001-8650-GgQGGE73fv3xAgOALoDHjA{:}server.ccl.net> X-Original-From: Andrew Dalke Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=windows-1252 Date: Mon, 17 Oct 2011 18:42:14 +0200 Mime-Version: 1.0 (Apple Message framework v1084) Sent to CCL by: Andrew Dalke [dalke%dalkescientific.com] On Oct 14, 2011, at 11:18 PM, Brian.James.Duke{:}gmail.com wrote: > I just encountered the Science Code Manifesto, which essentially states > that all computer code used for scientific analysis and modeling should > be available for review. To be precise: All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper. I interpret that to mean that you don't need to ship source code for the entire operating system, analysis tools, compiler, and so on. For that matter, you can publish your Gaussian script, and not need to include the source for Gaussian itself. It does seems like it leaves a hole for people who write a general purpose program for one project, then use it (unchanged) for something which is published. Of course, the editors may object to that practice. There's also a problem with: Researchers who use or adapt science source code in their research must credit the code’s creators in resulting publications Consider the 70 authors and 15 libraries which make up the CDK chemistry toolkit. There might be 100+ creators. It's beyond reason to include everyone in the publications, and the general expectation is to reference the project, not all of the code's creators. Finally, if you are a follower of the Free Software Foundation's ideas on software freedom, then you agree that there's a freedom to sell software (http://www.gnu.org/philosophy/selling.html ) for as much money as you want. This manifesto is almost at odds with the FSF viewpoint; I see it omits any discussion of how much it should cost to access the curated software. These aren't big problems, but they do point out that this is a complex issue. Andrew Dalke dalke]~[dalkescientific.com From owner-chemistry@ccl.net Mon Oct 17 16:39:00 2011 From: "Sergio Manzetti sergio.manzetti]*[gmail.com" To: CCL Subject: CCL: Manifesto and scientific development Message-Id: <-45668-111017151959-18375-umfuewUf7Lr1WyIB6sKbWw(~)server.ccl.net> X-Original-From: Sergio Manzetti Content-Type: multipart/alternative; boundary=bcaec548a7e7d2ffb604af837d28 Date: Mon, 17 Oct 2011 21:19:53 +0200 MIME-Version: 1.0 Sent to CCL by: Sergio Manzetti [sergio.manzetti : gmail.com] --bcaec548a7e7d2ffb604af837d28 Content-Type: text/plain; charset=ISO-8859-1 Any manifesto works more like an inhibitor of productivity and creativity, exactly because it, with this manifesto in particular, imposes a force factor on the premises of computational scientific creativity. Open source philopsophy is an enormous value to scientific creativity and allows all groups to contribute to the scientific arena. This philosophy is not in tune with such this manifesto, it seems rather that the manifesto is correlated with a need to rigidify processes which have to be based on openess and sincerity among scientists and by the scrutinizing potential of reviewers. If a manifesto should replace that very profound criteria, we can as well pack our stuff and start working with something else. The nature of the manifesto is more inquisitory which is itself an original enemy of science. So why adopt such measures? If a scientific work was falsified and presented incredible findings it would be naturally exposed and investigated by the scientific arena it self, because science is itself selv-investigating. Sergio --bcaec548a7e7d2ffb604af837d28 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Any manifesto works more like an inhibitor of productivity and creativity, = exactly because it, with this manifesto in particular, imposes a force fact= or on the premises of computational scientific creativity. Open source phil= opsophy is an enormous value to scientific creativity and allows all groups= to contribute to the scientific arena. This philosophy is not in tune with= such this=A0 manifesto, it seems rather that the manifesto is correlated w= ith a need to rigidify processes which have to be based on openess and sinc= erity among scientists and by the scrutinizing potential of reviewers. If a= manifesto should replace that very profound criteria, we can as well pack = our stuff and start working with something else. The nature of the manifest= o is more inquisitory which is itself an original enemy of science. So why = adopt such measures?
If a scientific work was falsified and presented incredible findings it wou= ld be naturally exposed and investigated by the scientific arena it self, b= ecause science is itself selv-investigating.


Sergio
--bcaec548a7e7d2ffb604af837d28-- From owner-chemistry@ccl.net Mon Oct 17 17:24:00 2011 From: "James Eilers jeilers-$-siue.edu" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45669-111017162132-31236-tG+6s6i+CvhtFEqHQcwAyg[A]server.ccl.net> X-Original-From: James Eilers Content-Type: multipart/alternative; boundary=Apple-Mail-5--874230954 Date: Mon, 17 Oct 2011 15:21:08 -0500 Mime-Version: 1.0 (Apple Message framework v936) Sent to CCL by: James Eilers [jeilers-,-siue.edu] --Apple-Mail-5--874230954 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit I think you need to parse their sentence more carefully. I don't see it as applying to any of the widely used Computational Chemistry codes--- either those that are open source, or those that are licensed, patented, and/or copy write protected. "All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper." On Oct 14, 2011, at 4:18 PM, Brian.James.Duke{:}gmail.com wrote: > I just encountered the Science Code Manifesto, which essentially > states > that all computer code used for scientific analysis and modeling > should > be available for review. It appears to have started with the Climate > Code Foundation. I encourage you to visit the web site and consider > endorsing the Manifesto. > > Note that this is not specifying open source code, so GAMESS(US), > GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd > MPQC satisfies the points of the manifesto. Of course some other > quantum > chemistry codes do not. I think we should be putting pressure on the > authors of such codes to meet the criteria in this manifesto. > > Brian. In climate/weather science, where there may be a lot of "our computer models show", there may good reason for such a manifesto; but we don't need one, and I doubt that many areas of science do Jim James E. Eilers Professor Chemistry Department Southern Illinois University Edwardsville (618)650-3559 --Apple-Mail-5--874230954 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable
I think you need to parse = their sentence more carefully.  I don't see it as applying to any = of the widely used Computational Chemistry codes--- either those that = are  open source, or those that are licensed, patented, and/or = copy write protected. 

"All source = code written = specifically to process data for a published=20 paper must be available to the reviewers and readers of the = paper."
On Oct 14, 2011, at 4:18 PM, = Brian.James.Duke{:}gmail.com wrote:

I just = encountered the Science Code Manifesto, which essentially states
that all computer code = used for scientific analysis and modeling should
be available = for review. It appears to have started with the Climate
Code = Foundation. I encourage you to visit the web site and = consider
endorsing the Manifesto.

Note that this is not specifying open source code, so = GAMESS(US),
GAMESS(UK), DALTON etc., as well as open source codes = such as PSI3 amd
MPQC satisfies the points of the manifesto. Of = course some other quantum
chemistry codes do not. I think we should = be putting pressure on the
authors of such codes to meet the criteria = in this manifesto.

Brian.

In = climate/weather science, where there may be a lot of "our computer = models show", there may good reason for such a manifesto; but we don't = need one, and I doubt that many areas of science = do
Jim

James E. Eilers

Professor

Chemistry Department

Southern Illinois University Edwardsville

=

(618)650-3559   

=

= --Apple-Mail-5--874230954-- From owner-chemistry@ccl.net Mon Oct 17 17:59:01 2011 From: "Brian Salter-Duke brian.james.duke|,|gmail.com" To: CCL Subject: CCL:G: Science code manifesto Message-Id: <-45670-111017164116-18430-A56iytuT9Uxzz2A9XmVkjA||server.ccl.net> X-Original-From: Brian Salter-Duke Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Date: Tue, 18 Oct 2011 07:41:07 +1100 MIME-Version: 1.0 Sent to CCL by: Brian Salter-Duke [brian.james.duke^_^gmail.com] Just using a different code leads to confusion. Code A gives X. Code B gives Y, But why? A known example of this confusion is the differing meanings of B3LYP in different programs, but what if the reason is not clear. Also, what if it is an entirely new development that is not in any other program. Of course I assume the paper explains the method and someone else could code it up, but that is time consuming. It would be much preferable to check the code that gave the results. However, I am a total open source supporter and nothing is going to convince me that keeping scientific code closed to the users of the program is anything other than bad science and really rather offensive. Science does not progress by secrecy. Brian. 2011/10/17 João Brandão jbrandao+/-ualg.pt : > Sorry, but I disagree. > > "but if there are real concerns about the work, it is necessary that the > scientific > community can look seriously at the code to see exactly what the program is > doing." > > In my opinion: > If anyone has real concerns about any work, the best way for science > development is to write (or use) a different code and compare the results. > > João Brandão > > > > > Em 16-10-2011 22:33, Brian Salter-Duke brian.james.duke.:.gmail.com > escreveu: > > Sent to CCL by: Brian Salter-Duke [brian.james.duke(~)gmail.com] > The review of manuscripts by referees prior to publication is just a > small part of what this is all about. The main point is about proper > review by the scientific community after publication. There, while not > everyone has access to a 600MHz NMR or has paid to get Gaussian, some > people in the community will have. The guys who paid for Gaussian > however, may not have bought the code and in some cases, it is > impossible to buy the code, as they are commercial secrets. This > manifesto is saying that commercial secrets are not compatible with > good science. > > Of course the need to this kind of review is not common, but if there > are real concerns about the work, it is necessary that the scientific > community can look seriously at the code to see exactly what the > program is doing. > > Brian. > > On Sun, Oct 16, 2011 at 8:05 PM, Sebastian Kozuch kozuchs^_^yahoo.com > wrote: > > > I like the idea of open software, and in principle I support the proposal of > having access to the software code for the possibility of full review of a > theoretical work. However, I feel that this is more wishful thinking than > real scientific life. > > Let’s say that I receive a manuscript for review that uses program X, which > I may have access to its code now (since I’m the reviewer). Do I have to > spend a week trying to understand its algorithms to check if the frequencies > are correctly calculated? For me, most (if not all) the programs are in > practice black boxes, as I’m hardly a programmer. Therefore, open or closed > software (usually) doesn’t make me any difference, except from a > philosophical perspective. > > Now, let me consider an analogy from experimental chemistry. I have to > review a manuscript where the authors tested some compound with a 600 MHz > NMR. My lab is much more humble, and I only have access to a 400 MHz NMR. Id > est, I cannot reproduce the results of the paper that I’m reviewing. Is here > any difference compared to the theoretical case? > > I had in a couple of cases problems to reproduce the results of papers, > mostly because the authors didn’t provide enough information. However, for > 100% reproducibility I need 100% the same conditions (and a lot of resources > and time). This makes the selection of the software just a small issue. > > I would like to hear the thoughts of other people about this issue, as I > consider the peer reviewing process a very complex and far from perfect > system. > > > > > Sebastian > > ________________________________ >> From: Brian.James.Duke{:}gmail.com > To: "Kozuch, Sebastian " > Sent: Friday, October 14, 2011 11:18 PM > Subject: CCL: Science code manifesto > > I just encountered the Science Code Manifesto, which essentially states > that all computer code used for scientific analysis and modeling should > be available for review. It appears to have started with the Climate > Code Foundation. I encourage you to visit the web site and consider > endorsing the Manifesto. > > http://sciencecodemanifesto.org/ > > Note that this is not specifying open source code, so GAMESS(US), > GAMESS(UK), DALTON etc., as well as open source codes such as PSI3 amd > MPQC satisfies the points of the manifesto. Of course some other quantum > chemistry codes do not. I think we should be putting pressure on the > authors of such codes to meet the criteria in this manifesto. > > Brian. > > > > > > -- Brian Salter-Duke (aka Brian Duke) Brian.James.Duke^^^gmail.com From owner-chemistry@ccl.net Mon Oct 17 19:23:00 2011 From: "David A Mannock dmannock_-_ualberta.ca" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45671-111017181439-2628-AxwW0lPcefvwQOuucvsVgg^^server.ccl.net> X-Original-From: David A Mannock Content-Type: multipart/alternative; boundary=001485f44bf03cf02804af85ee7c Date: Mon, 17 Oct 2011 16:14:28 -0600 MIME-Version: 1.0 Sent to CCL by: David A Mannock [dmannock[-]ualberta.ca] --001485f44bf03cf02804af85ee7c Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Having just reviewed an area of sterol/lipid interactions in 2010, it is clear that experimental design is key in obtaining consistent results in that area. It is no different in computer chemistry. It is important to understand how programs do the calculations, so that discrepancies can be identified and explained. This should be performed by authors in their Introduction and Discussion sections of manuscripts or at least in review articles. However, in the rush to publish, given the pressure to commercialize, these details are often overlooked. Having a manifesto is a nice idea, but it might be appropriate to start by having code writers explain the calculation and code employed in a supplementary appendix available for electronic download. On Mon, Oct 17, 2011 at 12:37 PM, Dr W.R. Pitt wrp24*_*cam.ac.uk < owner-chemistry : ccl.net> wrote: > > Sent to CCL by: "Dr W.R. Pitt" [wrp24!=3D!cam.ac.uk] > I have two points to add to the discussion: > > 1/ As far as I can tell the the Science Code Manifesto does not insist on > the use of open source software. In the discussion page it states "Use of > languages, libraries, systems, and tools which are widely available is > strongly recommended." The data analysis script should be made available = but > the manifesto seems to leave room for that script to be written using a > commercial toolkit. I use commercial software as well as free software. > Sometimes the former is only way (within the time I have available) of do= ing > the job I have in mind. From my point of view, it is better to do science > using commercial software and publish it than not doing it all. > > 2/ I do agree that the analysis scripts should be made public. However, I= 'm > not sure it is necessary for a referee to be able to re-run the analysis > script and check every last detail. The wider peer community would have t= he > opportunity to do this. A natural consequence of this would be multiple > versions of results in papers, derived from bug-fixes, following reader > input. Much more rigorous testing of work prior to publication should > result, which can only be a good thing. > > Relivant articles : > Peer reviewers swamped, so extras get the heave-ho http://www.** > timeshighereducation.co.uk/**story.asp?storyCode=3D413390&**sectioncode= =3D26 > > Report from the Publishing Open Data Working Group meeting http://blogs.*= * > openaccesscentral.com/blogs/**bmcblog/entry/report_from_the_** > publishing_open > > > > > > > On Oct 17 2011, Jordi Vill=E0 i Freixa jordi.villafreixa/agmail.com wrote= : > > Allow me to be drastic here. >> >> In my opinion this is a quite absurd discussion. Having the ability to >> make code transparent for others to reuse and build on top of is obvious= ly >> way better for science than having black boxes. It is not needed to prov= ide >> the ability to the referee (who by definition is a busy guy) to rerun th= e >> calculations, but to the scientific community as a whole. It may be the = time >> already to reduce the often over-critic referee system and move to a mor= e >> open science schema. Repositories for code have been extremely useful in= the >> past and I think being concerned about the ability of the scientific >> community to improve our results is way more positive that believing tha= t we >> and only we can run a code and understand its results. >> >> Having myself a foot in enterpreneourship in addition to academia, I thi= nk >> that closing/selling code is futurless in a world with tens of thousands= of >> developers ready for better implementations for each new idea. What make= s >> sense is understanding the possible use of each technology, not closing = them >> in a useless effort of overprotection. >> >> So, the need for the manifesto is clear. Start opening our eyes to a wid= er >> conception of science, far from egoistic and closed minded uses of it. T= he >> system should follow this, one day or another. >> >> >> >> 2011/10/17 Pedro Silva pedros^ufp.edu.pt >> >> >>> Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt] >>> 2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt : >>> > Sorry, but I disagree. >>> > >>> > "but if there are real concerns about the work, it is necessary that = > >>> the scientific community can look seriously at the code to see exactly = > >>> what the program >>> is >>> > doing." >>> > >>> > In my opinion: >>> > If anyone has real concerns about any work, the best way for science >>> > development is to write (or use) a different code and compare the >>> results. >>> > >>> >>> The problem is: if the results do not agree with each other, how can >>> you adjudicate between the competing claims without access to the >>> code?> >>> >>> >>> >> >> >> > > > -=3D This is automatically added to each message by the mailing script = =3D-> http://www.ccl.net/cgi-bin/**ccl/send_ccl_message > http://www.ccl.net/cgi-bin/**ccl/send_ccl_message chemistry/announcements/**conferences/ > > Search Messages: http://www.ccl.net/chemistry/**searchccl/index.shtml http://www.ccl.net/spammers.**txt > > RTFI: http://www.ccl.net/chemistry/**aboutccl/instructions/ > > > --001485f44bf03cf02804af85ee7c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Having just reviewed an area of sterol/lipid interactions in 2010, it is cl= ear that experimental design is key in obtaining consistent results in that= area. It is no different in computer chemistry. It is important to underst= and how programs do the calculations, so that discrepancies can be identifi= ed and explained. This should be performed by authors in their Introduction= and Discussion sections of manuscripts or at least in review articles. How= ever, in the rush to publish, given the pressure to commercialize, these de= tails are often overlooked. Having a manifesto is a nice idea, but it might= be appropriate to start by having code writers explain the calculation and= code employed in a supplementary appendix available for electronic downloa= d.

On Mon, Oct 17, 2011 at 12:37 PM, Dr W.R. Pi= tt wrp24*_*cam.ac.uk <= ;owner-chemistry : ccl.net>= wrote:

Sent to CCL by: "Dr W.R. Pitt" [wrp24!=3D!cam.ac.uk]
I have two points to add to the discussion:

1/ As far as I can tell the the Science Code Manifesto does not insist on t= he use of open source software. In the discussion page it states "Use = of languages, libraries, systems, and tools which are widely available is s= trongly recommended." The data analysis script should be made availabl= e but the manifesto seems to leave room for that script to be written using= a commercial toolkit. I use commercial software as well as free software. = Sometimes the former is only way (within the time I have available) of doin= g the job I have in mind. From my point of view, it is better to do science= using commercial software and publish it than not doing it all.

2/ I do agree that the analysis scripts should be made public. However, I&#= 39;m not sure it is necessary for a referee to be able to re-run the analys= is script and check every last detail. The wider peer community would have = the opportunity to do this. A natural consequence of this would be multiple= versions of results in papers, derived from bug-fixes, following reader in= put. Much more rigorous testing of work prior to publication should result,= which can only be a good thing.

Relivant articles :
Peer reviewers swamped, so extras get the heave-ho http://www.timeshighereducation.co.uk/stor= y.asp?storyCode=3D413390&sectioncode=3D26

Report from the Publishing Open Data Working Group meeting http://blogs.openaccesscentral.com/blogs/<= u>bmcblog/entry/report_from_the_publishing_open






On Oct 17 2011, Jordi Vill=E0 i Freixa jordi.villafreixa/agmail.com wrote:

Allow me to be drastic here.

In my opinion this is a quite absurd discussion. Having the ability to make= code transparent for others to reuse and build on top of is obviously way = better for science than having black boxes. It is not needed to provide the= ability to the referee (who by definition is a busy guy) to rerun the calc= ulations, but to the scientific community as a whole. It may be the time al= ready to reduce the often over-critic referee system and move to a more ope= n science schema. Repositories for code have been extremely useful in the p= ast and I think being concerned about the ability of the scientific communi= ty to improve our results is way more positive that believing that we and o= nly we can run a code and understand its results.

Having myself a foot in enterpreneourship in addition to academia, I think = that closing/selling code is futurless in a world with tens of thousands of= developers ready for better implementations for each new idea. What makes = sense is understanding the possible use of each technology, not closing the= m in a useless effort of overprotection.

So, the need for the manifesto is clear. Start opening our eyes to a wider<= br> conception of science, far from egoistic and closed minded uses of it. The<= br> system should follow this, one day or another.



2011/10/17 Pedro Silva pedros^ufp.edu.pt <owner-chemistry(!)ccl.net>


Sent to CCL by: Pedro Silva [pedros(_)ufp.edu.pt]
2011/10/17 Jo=E3o Brand=E3o jbrandao+/-ualg.pt <owner-chemistry^-^ccl.net>:
> Sorry, but I disagree.
>
> "but if there are real concerns about the work, it is necessary t= hat > the scientific community can look seriously at the code to see exa= ctly > what the program
is
> doing."
>
> In my opinion:
> If anyone has real concerns about any work, the best way for science > development is to write (or use) a different code and compare the
results.
>

The problem is: if the results do not agree with each other, how can
you adjudicate between the competing claims without access to the
code?>








-=3D This is automatically added to each message by the mailing script =3D-=

E-mail to subscribers: CHEMISTRY : ccl.net or use:
=A0 =A0 http://www.ccl.net/cgi-bin/ccl/send_ccl_message

E-mail to administrators: CHEMISTRY-REQUEST : ccl.net or use

=A0 =A0 http://www.ccl.net/cgi-bin/ccl/send_ccl_message
http://www.ccl.net/chemistry/sub_u= nsub.shtml

Before posting, check wait time at: http://www.ccl.net

Job: http://www.ccl.n= et/jobs Conferences: http://server.ccl.net/chemist= ry/announcements/conferences/

Search Messages: http://www.ccl.net/chemistry/searchccl/index.= shtml


--001485f44bf03cf02804af85ee7c-- From owner-chemistry@ccl.net Mon Oct 17 21:13:00 2011 From: "Mark Zottola mzottola{=}gmail.com" To: CCL Subject: CCL: Science code manifesto Message-Id: <-45672-111017210917-30523-ZlLG8+3i/zkcc2q/4Rt7GQ|,|server.ccl.net> X-Original-From: Mark Zottola Content-Type: multipart/alternative; boundary=bcaec544ec58f25e6004af885e4a Date: Mon, 17 Oct 2011 21:09:10 -0400 MIME-Version: 1.0 Sent to CCL by: Mark Zottola [mzottola::gmail.com] --bcaec544ec58f25e6004af885e4a Content-Type: text/plain; charset=ISO-8859-1 For all our concerns about the impact of this manifesto on science, I think most are missing what I believe to be an important aspect. If a code is readily available for the scientific public, then it is readily available to the general public. As one area mentioned in the manifesto was climate modeling, it is clear that enterprising non-scientists or muck-raking types can access these codes. In any well-commented code, there are bound to comments like "applying the Fandoozie trick" or other such innocucous statements. In addition, in any code of over 1000 lines there are bound to be bugs. Can you imagine the public discourse when some rabel-rouser claims the software for climate modeling is flawed and he shows the offending code as "proof". Flaws and "tricks", common to all software, have the potential to be exploited for political gain under this manifesto. I'm not sure where the middle ground is when trying to make software open, accessible, and maintaining peer review without the introduction of politics. I think this needs a lot more thought and consideration. Mark Zottola --bcaec544ec58f25e6004af885e4a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
For all our concerns about the impact of this manifesto on science, I = think most are missing what I believe to be an important aspect.=A0 If a co= de is readily available for the scientific public, then it is readily avail= able to the general public.=A0 As one area mentioned in the manifesto was c= limate modeling, it is clear that enterprising non-scientists or muck-rakin= g types can access these codes.
=A0
In any well-commented code, there are bound to comments = like "applying the Fandoozie trick" or other such innocucous stat= ements.=A0 In addition, in any code of over 1000 lines there are bound to b= e bugs.=A0 Can you imagine the public discourse when some rabel-rouser clai= ms the software for climate modeling is flawed and he shows the offending c= ode as "proof".=A0 Flaws and "tricks", common to all so= ftware, have the potential to be exploited for political gain under this ma= nifesto.
=A0
I'm not sure where the middle ground is when trying = to make software open, accessible, and maintaining peer review without the = introduction of politics.=A0 I think this needs a lot more thought and cons= ideration.

Mark Zottola
--bcaec544ec58f25e6004af885e4a--