Sung_JSdep about fse16 HOT 8 OPEN

@sch8906

Based on "3. INSTALLATION", I installed and Z3 SMT solver nodejs in Ubuntu 12.04 Desktop (64 bit) using Pararell Desktop 9, then conducted make install after Step 1 and 2 in the INSTALL file.

I obtained the following message:

Testing for software dependencies - if an error occurs, consult your local package manager for the program immeadiately checked for
which g++ > /dev/null
which qmake > /dev/null
make: *** [check] Error 1

Would you please tell me how to solve this message? At least, g++ seems to work for my environment.

kamei@ubuntu:~/Downloads/JSdep/artemis-modified/Artemis$ which g++
/usr/bin/g++

And could you please share a virtual disk image with me?

We have a virtual disk image with full configuration. However, because of the limitation of size we cannot upload it (The size is about 3.1 GB). If you want to use it, please contact to Chungha. from https://github.com/sch8906/JSdep

from fse16.

@yasutaka-kamei

Thank you for checking it. When you try to install Artemis, it checks the dependencies and when it ends after checking which qmake >/dev/null, it means qt (qmake) is not installed properly. You have to install it correctly first because Artemis needs qt4 to install it.

If you followed the instruction correctly, it will install qt. If you installed it and still have the same problem, then you have to update QTDIR and PATH as described in the instruction at README under artemis-modified directory (it is described at number 1 in Readme file)

For the virtual disk image, I am out of town for the conference, and I left my flash drive at home.
I will be back on next Saturday. If it is fine with that time I will send you it.

Please let me know whether this answers your questions and concerns.

from fse16.

@sch8906

Thank you for the quick reply. It was very helpful and I successfully installed the modified Artemis in Ubuntu 10.4. I also modified several lines to deal with new error messages. I will write how I solved in my review!

For the virtual disk image, next Saturday may not work for me due to the deadline of FSE 2016 artifacts track. So, you don't have to send it to me.

from fse16.

@yasutaka-kamei

I am glad to hear that it works.

And I am sorry for your inconvinience of the virtual disk image.

I really have it :( but I am in Canada because of CAV until Friday.

Sincerely,
Chungha

from fse16.

Very Brief Summary

The artifact is an implementation of JSdep, a research static analysis tool for Javascript.

Insightful

As with the other submissions, the insightfulness of the artifact is established in the associated research track paper.

Useful

The artifact includes the only Javascript static analysis tool that extracts the dependencies of DOM events. Given the ubiquity of these events, it is useful to be able to extract them. In addition, the artifact provides scripts for using the tool to reproduce the results that are presented in the paper.

Usable

The paper includes an example of installing JSdep on a specific version of Ubuntu. In addition, there are usage instructions on the github page that are tested with that same OS version. The website does mention a VM, but it is unfortunate the it is not uploaded anywhere. One issue with not uploading the VM is that the tool may become hard to use if it is not updated.

Overall this is a useful artifact that belongs in the track. I do suggest that the authors find some webspace for the 3.1gb VM. Also, the paper format may need to be changed depending on the final publication requirements.

from fse16.

This artifact paper provides a replication package to reproduce the results of the author's FSE'16 paper. The package includes JSdep (a static analysis tool for JavaScript), the modified Artemis (a tool for automated testing of web applications) and the used datasets in the FSE'16 paper.

• Insightful
  Future studies can compare their proposed methods with JSdep and Artemis, which are provided by this artifact paper. Such comparison will derive insightful knowledge in the domain of testing web applications.
• Useful
  We can run and compare the test coverage of various Artemis configurations using the replication package provided by this artifact paper. Therefore, I think that this artifact paper is useful.
• Usable
  • Easy to understand?
    Yes.
  • Accompanied by tutorial notes?
    Yes. However, this artifact paper lacks some of the parts that are described in its web site (https://github.com/sch8906/JSdep).
    • realpath is not explained in the paper
    • HOW TO INTERPRET dep.txt? should be described in the paper.
    • make-run-artemis-all maybe means make run-artemis-all
  • Artifacts need not be executable but if so, are they:
    Yes. However, when I conducted make run-artemis-all iter=500, I obtained many warnings similar to like this "Warning: Default listener for javascript_eval_call was invoked, args: 27". I also needed to modify (1) the URL of the qt's git repo to https://github.com/qt/qt.git and (2) the permission in /bin to create /bin/altermis/ when I conducted make fetch-qt
  • Easy to download, install, or execute?
    Yes.
  • Available in a virtual machine image?
    I would like to suggest that the authors find online space and upload a VM file there.
  • Available online?
    Yes.
  • Supported by configuration management tools to permit easy updates?
    Yes.
• Score
  I think that this artifact is insightful, useful and usable. So I would like to say "Platinum" for this paper.

from fse16.

The artefact description introduces the tool clear way. It describes the briefly JSdep and
Artemis, artefacts related to the FSE paper.

The discussion on the insightfulness is to the point, but misses the discussion why these
artefacts are timely. The discussion on usefulness is fine and makes clear why the artefacts
are useful.

The description on usable addressed in a separate section. It is unclear why the authors
mention that there are patches. The installation steps are well described.
The description is well structured, detailed and easy to understand. There are no
tutorial notes available, only a README containing the same information as the Section 2 of
the description. The dependencies are explicit described. The artefact does not provide
a virtual machine or is online available.

I would classify this artefact as: ? maybe diamond

from fse16.

Note these labels are still "under discussion" and are still subject to change prior to the final notifications Friday.

from fse16.