apo5698 / helli-grading Goto Github PK

View Code? Open in Web Editor NEW

2.0 1.0 0.0 17.08 MB

A web-based automatic grading system (for CSC 116@NCSU).

License: MIT License

Ruby 67.36% JavaScript 7.93% HTML 14.33% SCSS 0.56% Shell 0.22% Gherkin 0.40% Slim 3.20% TypeScript 6.00%

grading-system ncsu

helli-grading's Introduction

Helli

helli-grading's People

Contributors

Stargazers

Watchers

helli-grading's Issues

Dir.chdir is not thread-safe

Dir.chdir is a process-based operation (see https://bugs.ruby-lang.org/issues/9785). If multiple Helli::Process are created concurrently, the server process may not be able to go back to the original directory after the block.

https://github.com/apo5698/ags/blob/9f96a924a0cce6505d82826cec0d15434d0a9e5b/app/lib/helli/process.rb#L20-L38

dependencies.json not generated on app load

All dependencies are not available.

2020-10-10T07:40:55.915808+00:00 app[web.1]: F, [2020-10-10T07:40:55.915678 #9] FATAL -- : [f3d29577-1f8e-45f2-8559-3737285dfe84]   
2020-10-10T07:40:55.915811+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] ActionView::Template::Error (No such file or directory @ rb_sysopen - /app/public/api/dependencies.json):
2020-10-10T07:40:55.915812+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     1: <div class="mb-2">
2020-10-10T07:40:55.915812+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     2:   <%= link_to('Reload', { controller: 'settings', action: 'reload' }, class: 'btn btn-secondary') %>
2020-10-10T07:40:55.915813+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     3:   <%= link_to('dependencies.json', DependenciesUtil.json, class: 'btn btn-secondary') %>
2020-10-10T07:40:55.915814+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     4: </div>
2020-10-10T07:40:55.915815+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     5: <table class="table table-hover">
2020-10-10T07:40:55.915815+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]     6:   <thead>
2020-10-10T07:40:55.915816+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84]   
2020-10-10T07:40:55.915816+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] app/lib/dependencies_util.rb:18:in `initialize'
2020-10-10T07:40:55.915817+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] app/lib/dependencies_util.rb:18:in `open'
2020-10-10T07:40:55.915817+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] app/lib/dependencies_util.rb:18:in `json'
2020-10-10T07:40:55.915818+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] app/views/settings/_dependencies.html.erb:3
2020-10-10T07:40:55.915818+00:00 app[web.1]: [f3d29577-1f8e-45f2-8559-3737285dfe84] app/views/settings/index.html.erb:19

Skip downloading attachment if file exists or content identical

Compare checksum

Exit status is present for inactive GradeItem

Suppress Encoding::UndefinedConversionError on attachment download

Server Log

2021-02-01T01:10:39.509693+00:00 app[web.1]: I, [2021-02-01T01:10:39.509630 #89]  INFO -- : [a84d49f3-7e68-4109-b6f1-6d24af872ea0] Completed 500 Internal Server Error in 38ms (ActiveRecord: 6.3ms | Allocations: 3519)
2021-02-01T01:10:39.510283+00:00 app[web.1]: F, [2021-02-01T01:10:39.510220 #89] FATAL -- : [a84d49f3-7e68-4109-b6f1-6d24af872ea0]   
2021-02-01T01:10:39.510287+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] Encoding::UndefinedConversionError ("\xE3" from ASCII-8BIT to UTF-8):
2021-02-01T01:10:39.510288+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0]   
2021-02-01T01:10:39.510289+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/lib/helli/attachment.rb:20:in `write'
2021-02-01T01:10:39.510289+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/lib/helli/attachment.rb:20:in `block in download'
2021-02-01T01:10:39.510290+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/lib/helli/attachment.rb:18:in `map'
2021-02-01T01:10:39.510291+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/lib/helli/attachment.rb:18:in `download'
2021-02-01T01:10:39.510291+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/lib/helli/attachment.rb:31:in `download_one'
2021-02-01T01:10:39.510292+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/models/grade_item.rb:87:in `run'
2021-02-01T01:10:39.510292+00:00 app[web.1]: [a84d49f3-7e68-4109-b6f1-6d24af872ea0] app/controllers/api/grade_items_controller.rb:23:in `update'

Add favicon

Show passed items in feedback

Heroku request timeout exceeded

According to Request Timeout:

The countdown for this 30 second timeout begins after the entire request (all request headers and, if applicable, the request body) has been sent from the router to the dyno. The request must then be processed in the dyno by your application, and a response delivered back to the router, within 30 seconds to avoid the timeout.

Although it will not affect backend tasks running, if a "large" grading is triggered (e.g. checkstyle), HTTP 503 would still be likely to happen:

rails-ujs.js:216 POST http://ags.herokuapp.com/courses/1/assignments/3/grading/6 503 (Service Unavailable)

Consider sending one request for each grade item, not the whole rubric.

Run a single-file program without compiling

See JEP 330: Launch Single-File Source-Code Programs

zyBooks feedback is not showing

Errors should be returned as the second element of the array:

# code above...
error = []
error << ["Total: #{actual} => #{point}"] if something_happen

[[point, '', 0], error]

https://github.com/apo5698/helli/blob/c291af6c22d3aeff284c3a807c010f5a7d27dabb/app/models/rubrics/item/zybooks.rb#L24-L26

Also change

https://github.com/apo5698/helli/blob/c291af6c22d3aeff284c3a807c010f5a7d27dabb/app/models/grade_item.rb#L101

self.error = capture[1]

Disable table expandable for zyBooks

Edit grade items

[HTML] Do not trim whitespaces or escape characters for errors and feedback

Better HTTP error pages

HTTP 400
HTTP 404
HTTP 500

Add Java 11 to Heroku

Run:

heroku buildpacks:add -i 2 heroku/jvm

Add system.properties file with:

java.runtime.version=11

Random null value in column "exitstatus"

2021-02-15T18:19:49.545546+00:00 app[web.1]: I, [2021-02-15T18:19:49.545479 #88]  INFO -- : [d3e81ef0-bb88-4ecd-bcb5-463d673a75df] Completed 500 Internal Server Error in 5190ms (ActiveRecord: 28.5ms | Allocations: 6058)
2021-02-15T18:19:49.547353+00:00 app[web.1]: F, [2021-02-15T18:19:49.547286 #88] FATAL -- : [d3e81ef0-bb88-4ecd-bcb5-463d673a75df]   
2021-02-15T18:19:49.547354+00:00 app[web.1]: [d3e81ef0-bb88-4ecd-bcb5-463d673a75df] ActiveRecord::NotNullViolation (PG::NotNullViolation: ERROR:  null value in column "exitstatus" violates not-null constraint
2021-02-15T18:19:49.547366+00:00 app[web.1]: DETAIL:  Failing row contains (5977, 99, 21, Error, , , null, 0.00, Program runs: Program does not run, 2021-02-15 05:09:05.287818, 2021-02-15 18:19:49.521765, {"Program exits with code ","Output does not match regular expre...).
2021-02-15T18:19:49.547367+00:00 app[web.1]: ):
2021-02-15T18:19:49.547368+00:00 app[web.1]: [d3e81ef0-bb88-4ecd-bcb5-463d673a75df]   
2021-02-15T18:19:49.547368+00:00 app[web.1]: [d3e81ef0-bb88-4ecd-bcb5-463d673a75df] app/models/grade_item.rb:123:in `run'
2021-02-15T18:19:49.547369+00:00 app[web.1]: [d3e81ef0-bb88-4ecd-bcb5-463d673a75df] app/controllers/api/grade_items_controller.rb:23:in `update'

Refactor schemas

User

AGS Users. (TAs or instructors, NOT STUDENTS)

Has many Course

first_name: string
last_name: string
email: string
password_digest: string
role: enum

Course

Course examples:

CSC 116 (004) Fall 2020
CSC 116 (006) Spring 2021

Belongs to User
Has many Assignment

name: string
section: string
semester: enum
year: integer

Assignment

Assignment examples:

Day 1 Exercise
Project 2
Lab 4

Belongs to Course
Has one Rubric
Has many Participant
Has many Grade

name: string
category: enum
description: text
programs: array

Student (currently not used)

Belongs to Assignment

full_name: string
email_address: string

Participant (replace Submission)

Participant is different than Student because they are generated from the gradesheet of each assignment.

Belongs to Assignment
Has one Grade
Has many Attachment

programming_total: decimal
zybooks_total: decimal

Grade

Belongs to Participant

identifier: integer
full_name: string
email_address: string
status: enum
grade: decimal
maximum_grade: decimal
grade_can_be_changed: boolean
last_modified_submission: timestamp
last_modified_grade: timestamp
feedback_comments: text

Attachment (to be determined)

Might be duplicated with ActiveStorageBlob

Belongs to Participant

filename: string
type: string/enum

Rubric

A generic grading rubric.

Belongs to Assignment
Has many RubricItem

RubricItem

Rubric items examples:

Write/Compile/Execute
Teaching Staff Black Box Testing
Teaching Staff Unit Testing
Student Black Box Testing
Student Unit Testing
Style

Has many Criterion
Has many Program

type: enum
description: text

Criterion

Criterions determine whether a participant should be awarded or deducted points. Examples:

point	expected behavior	actual behavior	feedback
1	file is named 'HelloWorld.java'	file is named 'Helloworld.java'	file is incorrectly named
2	program compiles	program compiles
1	program runs	program fails to run	exception thrown at line...

point: decimal
expect: enum/string
actual: enum/string
feedback: text

No permission control on Course

A user can access courses created by others, which is a big problem...

Grade zyBooks

Generate GradingItem from gradesheet

If a student submits nothing for an assignment, this student will not have a GradingItem, result in missing in the grades export. But the gradesheet contains information of all participants, including those whose status is "No submission" or "Overdue".