This sample autograder builds on top of sample-cpp-diff-autograder by adding an extra layer of functionalilty: the ability to directly record the results of tests from the tddFuncs.h/tddFuncs.cpp unit testing framework into the Gradescope JSON.
The autograder can still incorporate diff-based tests before or after the unit tests are executed. The key point is that each stage in the grading pipeline adds on to the results.json of the previous stage.
The corresponding assignment can be found at github.com/ucsb-gradescope-tools/sample-cpp-tddFuncs.assignment.
Editing bash scripts (e.g. diffs.sh
) on Windows will convert the line endings to Windows style, causing the script to be unusable by the Docker container. Either edit the files in a Unix/Mac environment or use a tool to de-convert the line endings BEFORE uploading your autograder.zip.**
On MacOS, g++
automatically links to clang++
. Hence, if you are
doing anything involving clang
or clang++
, it might work on MacOS,
then fail to work once you load the script to Gradescope. Be sure in
that case that your apt-get.sh includes an install of the appropriate
clang++
toolchain.
Any extra files (e.g. a Makefile) that should be in the same directory as the student submission while the submission is being built. In this repo, the BUILD-FILES directory has three Makefiles:
Makefile
, the Makefile from the assignment repo.Makefile.check
A separate Makefile for checking syntax that is referenced indiffs.sh
; this allows the instructor to assign a certain number of points for a "clean compile" with no warnings or errors.Makefile.tdd
This is a makefile that builds versions of the executable that link with the version oftddFuncs.cpp
that generates the JSON for Gradescope.
When converting an assignment from submit.cs, any files that were "build files" on submit.cs go into this directory, along with the Makefile for the assignment.
Any extra files (e.g. data files) that should be in the same directory as the student submission while the submission is being executed. In this repo, the EXECUTION-FILES directory has input.txt
file from the provided-files directory in the assignment repo.
A reference solution which will be used to generate the expected outcome for all tests. Like student solutions, this solution will be run with all files from EXECUTION-FILES in the same directory.
A bash script that creates the expected test output using the reference solution. Leave this unchanged when creating your own autograder.
A bash script for installing any necessary dependencies for the assignment, e.g. g++.
A bash script describing the tests to be run. See this page for further documentation.
A bash script for generating the results of the student submission. At the top of this file, you must specify what student files should be copied from /autograder/submission
for testing by listing them in the definition of the environment
variable EXPECTED_FILES.
For example, the grade.sh
script in this file starts with:
EXPECTED_FILES="add.h add.cpp addCmdLine.cpp sumStdin.cpp"
You also may need to set the make targets for the Makefile
and Makefile.tdd
in the BUILD_FILES
directory
if they are something other than all
and tests
:
# Set the make target for the compilation phase
MAKE_TARGET_BUILD="all"
# Set the make target for the execution/testing phase (tddFuncs tests)
# This should be a target in the Makefile.tdd makefile
MAKE_TARGET_TESTS="tests"
- Create a new empty private repo, e.g. with the name PRIVATE-cs8-s18-labxx-gs
- Clone the empty repo with
git clone <url>
cd
into that directory:cd PRIVATE-cs16-s18-labxx-gs
- Add a remote for this sample repo via:
git remote add template [email protected]:ucsb-gradescope-tools/sample-cpp-tddFuncs-autograder.git
- Pull from this sample repo via
git pull template master
- Push to origin with
git push origin master
-
In
grade.sh
edit the environment variableEXPECTED_FILES
to list the files you expect the student to submit. Only these will be copied from the student submission into the space where the assignment is graded.If you prefer all files to be copied in, edit the
grade.sh
script to copy all files from/autograder/submission
into the target directory. -
Change the reference solution
REFERENCE-SOLUTION
, including the Makefile to be a reference solution for your assignment. In this directory, you should have aMakefile
that matches the one you'd expect the students to be testing with, and versions oftddFuncs.h
andtddFuncs.cpp
that match what you'd expect the students to use. -
Look at the three Makefiles in
BUILD-FILES
, i.e.Makefile
,Makefile.check
andMakefile.tdd
and create appropriate versions for your assignment. -
Make sure that instructor versions of files containing the instructor tests are in
BUILD-FILES
. -
If you are also using diff based grading, edit the files
diffs.sh
to contain your tests. This is where you can include a syntax check based onMakefile.check
. If you are using ONLY unit tests, this file can be an empty file. -
Make sure that the
Makefile
target specified ingrade.sh
match the ones inMakefile
andMakefile.tdd
, respectively, from theBUILD-FILES
directory.
To test your autograder locally, try putting a correct sample solution in the SAMPLE-SOLUTION-1 directory and an incorrect sample solution in the SAMPLE-SOLUTION-2 directory. (You have the option of creating additional SAMPLE-SOLUTION-nn directories, as few or as many as you see fit if you want to test a wider range of solution possibilities.)
To check what will happen, run:
MAKE-REFERENCE.sh
./grade.sh SAMPLE-SOLUTION-1
./grade.sh SAMPLE-SOLUTION-2
- etc.
(When converting assignments from UCSB's submit.cs, you might adapt a "perfect" solution and a "flawed" solution from among previous student submissions, by looking at the grade assigned by submit.cs)
In each case, look at the file results.json
to see whether it reflects what you expect the resulting grade to be.
Step 4: Create an Autograder.zip
using the link-gs-zip-with-repo tool.
- Clone the link-gs-zip-with-repo.
- In that repo, edit
env.sh
to point to your repo. You don't need to commit that change. - Run the script
./make_deploy_keys.sh
and upload your deploy key to the new repo you created for your assignment. - Run the script
./make_autograder_zip.sh
and then upload theAutograder.zip
to Gradescope.
You are now ready to test your autograded assignment.