justmars / corpus-x Goto Github PK

Instead of limiting the same to Codification, all statutes should be copied to the database.

Note some bugs detected that should be resolved upstream:

Size of some item content exceed that set by validators, see ra9154, ra9486

Validation needed detail=PosixPath('/Users/mv/code/corpus/statutes/ra/9154/details.yaml'); e=ValidationError(model='StatuteUnit', errors=[{'loc': ('content',), 'msg': 'ensure this value has at most 100000 characters', 'type': 'value_error.any_str.max_length', 'ctx': {'limit_value': 100000}}])
Validation needed detail=PosixPath('/Users/mv/code/corpus/statutes/ra/9486/details.yaml'); e=ValidationError(model='StatuteUnit', errors=[{'loc': ('content',), 'msg': 'ensure this value has at most 100000 characters', 'type': 'value_error.any_str.max_length', 'ctx': {'limit_value': 100000}}])

1918 rules of court not accepted by validator

DB insertion: e=SyntaxWarning("idx='1918' invalid serial of StatuteSerialCategory.RulesOfCourt")

Need a more robust method to check for missing values.

For instance, this SQL query reveals no missing statute ids after the updating function is completed:

select * from lex_tbl_codification_events_statute where affector_statute_id is null -- 0 rows

In contrast, the following material paths are missing from the same table:

select * from lex_tbl_codification_events_statute where affector_material_path is null group by statute -- 80 individual rows, but when group by is utilized, reveals that it involves 38 statutes

• 00-5-03 (2 variants) not matching the proper statute id
• 19-03-24 (3 variants) not matching the proper statute id
• There are NULLs for statute category and statute serial id
• Missing sc-cir
• Missing oca-cir

Upon reset of the database, there appears to be missing yaml files. Investigate.