thoward27 / grade Goto Github PK
View Code? Open in Web Editor NEWA utility for programming assignment grading.
License: GNU Affero General Public License v3.0
A utility for programming assignment grading.
License: GNU Affero General Public License v3.0
Blocked on: actions/setup-python#30
Describe the bug
The leaderboard's default score is None, it should be whatever value ensures it shows up in the last place.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Leaderboard default values should put students in last place.
Is your feature request related to a problem? Please describe.
When a test case fails currently, the reason for the assertion failure is printed in full, along with other potentially confusing information for students.
Describe the solution you'd like
It would be great to have a way to specify an error message for each assert, and have that error message be the only thing that gets output if the assert fails.
Describe alternatives you've considered
A function-wide error message decorator could work, but would lack the nuance of being able to give the student what their output was, or having different messages for asserts for different incorrect outputs.
Blocked on: readthedocs/readthedocs.org#6324
Have to skip valgrind assertions until it is resolved.
Blocked on: actions/runner-images#59
Need to update sphinx to include the new directory structure.
Describe the bug
The page for the grade pipeline documentation is empty.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The documentation for the pipeline functions should be there.
Environment (please complete the following information):
Describe the bug
When using the '@visibility' decorator for an individual test case function, nothing is actually changed. The visibility for the whole assignment is still 'visible', while the visibility value for the individual test cases is not output as a part of the JSON file. The attribute appears to be set correctly, but just not included in the JSON output.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The visibility of an individual test case should show up in the list of properties for that test case in the JSON file.
Describe the bug
When using PartialCredit on a list of pipelines and a list of values, it raises an error upon calling the 'score' function. This appears to be due to this code from partialcredit.py:
line 31: self.max_score = value
and
line 41: return min(self.max_score, round(self._score, 2))
Since value can be an int or a list inside init, max_score can be set to a list, and an error pops up when trying to compare that list to the proper score.
Is your feature request related to a problem? Please describe.
The names of each test case in grade's output is just the class name, function name, and function docstring, instead of being user-defined.
Describe the solution you'd like
Potentially a name decorator that will override the default name if one exists for a test function.
Describe alternatives you've considered
The docstring for each test function could also be used by itself, but this wouldn't allow more descriptive test docstrings, forcing them to be solely names.
Describe the bug
The json output from grade report is formatted without any newlines.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The json output has newlines to easily read the output if needed.
The documentation leaves a lot to be desired. Here are some things that need to be done.
Currently, unlike the standard Gradescope autograder, when a student's code fails to compile, nothing informative is shown (not even to the instructor). Ideally, the compilation failure (stderr) could be captured and shown (possibly even to the student) as the test output for the compilation check. This would cut down on TA support time, as often it's something as simple as the student forgot their compile script.
The Results class should be able to produce valid Markdown grade reports. Similarly to JSON, it should return a string, and be a property of the results class.
I have some work I can dredge up on this from a project a while back, I'll work to integrate it in and we can move from there.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.