iterate on study protocol about codescoop HOT 3 CLOSED

Qualitative measurements: what else did people want to do that CodeScoop doesn't currently support? + Interpretation (discussion): How hard is it to enable these features?

from codescoop.

User Study
Possible design(s):
(H1) Ask them to extract requested functionality from existing code with no assistance except compile checking
(H1) Extract requested functionality from existing code with CodeScoop
Allow folks 5 min max to add explanatory comments, rename variables, reorder statements without breaking compile checking
(H3) Ask them to review the choices they made and reflect on what was hard or easy to answer
(H2) Show them version created by simulated automated method
(H2) Ask them to explain how theirs is better or worse and why

from codescoop.

H1: ‘Folks will take more time, more clicks, more deletions, more errors, more steps to create executable examples by hand’
H2: “Humans (with intent that is only minimally known to the machine, e.g., a few lines from which to start a slice) are better than machines at selecting shareable subsets of code.”
H3: “Which inclusion choices are easiest and hardest to reason about as a human.”

from codescoop.