Wang_dependency-conflicts about fse18 HOT 13 CLOSED

Suggested status from the authors: reproduced.

Comments:

• If we follow the instruction to the letter, the provided code runs fine.
• The provided scripts.jar assumes that the plugins are located in D:\plugin-decca. If it is not there, the command using scripts.jar will not run.
• Location of the code and data are hardcoded and thus would require multiple manual changes if we ever want to change them.
• Copying commands from pdf is error prone. It is better if the documentation is in raw text or markdown format.
• The authors claim it is reproduced, but they did not mention who reproduced it. Perhaps they forgot or due to double blind?

I'm okay with Reusable badge but Reproduced seems to be too strong -- see https://conf.researchr.org/track/fse-2018/fse-2018-Artifacts.

from fse18.

Hi,@anonymous-reviewer18, thanks for your efforts on our artifact!

To facilitate the reviewers' evaluation, in the INSTALL document, we provide the illustrative example to explain how to install DECCA, and suggest users to keep the unzip directory to be consistent with the above example. I feel sorry that the description make you confused. It should be noted that the location of data (e.g, D:\plugin-decca) is not hardcoded, it can be replaced with user's actual unzip directory in the install commands.

To avoid the error-proness of copying commands from pdf, we provide the docx version of documents.

Before releasing DECCA, we have asked four master students with an academic software engineering background (other than the authors) to reproduce the main results, without the use of author-supplied artifacts.

Attachments:
INSTALL.docx
README.docx
Reports.docx

Thanks again ^_^!

from fse18.

Dear @wangying8052 , thanks very much for your comment (I appreciate that you care enough to watch over this thread).

Now, I ask for your help, to refrain from further comment, till our next reviewer makes their determination.

(But, of course, if the next reviewer gets stuck on technical install issues, then your comments on how to fix that problem would be absolutely most very welcome.)

Have a great day!

from fse18.

Do I need to have 64 bit JDK for this to work?

from fse18.

The commands given should be executed outside of the directory that was unzipped i.e., outside of d:\plugin\decca. If this is done inside even with full paths, the command does not work.

I would have also liked to see a plain Markdown file instead of pdf or word.

In the end, I was able to install and run Decca successfully.

I agree that the artifact is functional and if a DOI is provided could be available. However, I do not see it as being reproduced.

from fse18.

Dear @fse18-artifact-reviewer1 , thank you for your efforts to review our tool. Thanks for all the reviewers' and chairs' hard work .
We have released the source code of Decca in Github, and it is availble at https://github.com/DeccaDC/Decca.
Also, the runable bytecode, script and raw data can be downloaded at https://github.com/DeccaDC/RawData.

We have followed your suggestions and updated the INSTALL and README documents into .md format.
https://github.com/DeccaDC/Decca/blob/master/INSTALL.md
https://github.com/DeccaDC/Decca/blob/master/README.md

We will create a webpage for Decca later.

Thanks again!!

from fse18.

This artifact is now functional and available. The authors have requested the reproduced badge but this does not qualify since this paper is done by the authors themselves. If the paper and results were by someone else using this tool then it would be reproduced (as I read it from the CFP). Given the current state, we have no way of verifying validity of who reproduced this since that was not the paper accepted at FSE.

The chairs can correct this if I am incorrectly viewing the reproduced badge.

from fse18.

The authors offer no evidence that anyone else has replicated or reproduced their work in which case reusable and available would be appropriate (given the GH release).

from fse18.

Dear @timm , I misunderstood the DOI's meaning. Thanks for your explanation in #33 . I will apply a DOI for our artifact. We can make it better!

from fse18.

Dear @timm , @fse18-artifact-reviewer1 and @anonymous-reviewer18 ,
Webpage of Decca: https://deccadc.github.io/fse18/
DOI for the source code of Decca: https://zenodo.org/record/1315177#.W1ArQ6huaUk
DOI for the runnable bytecode of Decca, script and raw data: https://zenodo.org/record/1315183#.W1Arr6huaUk

Thanks a lot!

from fse18.

very good. please update your INSTALL file in our repo with these urls

from fse18.

@timm Yes, done. Thank you very much. ^_^

from fse18.

Thank you @wangying8052 for addressing the comments. I'm also okay with reusable and available badges.

from fse18.