Giter VIP home page Giter VIP logo

standards-importer's Introduction

standards-importer

Usage:

  • clone repo
  • modify the Rakefile to use your own API key
  • modify config.yml to fit your database config
  • bundle install
  • rake

If you wish to test without downloading the full set of standards, rake import[3] will limit to 3 jurisdictions and 3 sets per jurisdiction.

standards-importer's People

Contributors

aleksclark avatar scottmessinger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

standards-importer's Issues

MySQL db:migrate does not work

When trying to db:migrate after db:create against MySQL, I get the following complaint :
Mysql2::Error: Invalid default value for 'parent_ids': CREATE TABLEstandards(idint(11) auto_increment PRIMARY KEY,jurisdiction_idint(11) NOT NULL,csp_idvarchar(255),parent_idsint(11) DEFAULT NULL NOT NULL,education_levelsvarchar(255) DEFAULT '--- []\n' NOT NULL,titlevarchar(255),subjectvarchar(255),documentvarchar(255),indexedtinyint(1) DEFAULT 0 NOT NULL,chi ld_countint(11) DEFAULT 0) ENGINE=InnoDB

After looking at file standards-importer/db/migrate/20150708101057_create_initial_tables.rb :

      t.column :parent_ids, :integer, array: true, null: false, default: []
      t.column :education_levels, :string, array: true, null: false, default: []

The susmentioned code creates arrays which are not handle gracefully in MySQL world.
Did you already run your scripts over a MySQL DB ?

Perhaps, if you give me an overview of your schema I can fork and create what's needed for it ?

Fails fetching sets

Fetching set for several ids fail. API calls to failing fetch set ids seems to work

A work around can be had by wrapping the body of the for loop in line 50 of common_standards_download.rb in a begin rescue statement:

begin puts "Fetching sets for #{jur.title}" jur.standardSets.each_with_index do |ss, i| break if @limit && i >= @limit result.push(auth_get('standard_sets/' + ss.id)) end rescue Exception => ex puts "FAILED #{jur.id}: #{ex}" end

FAILED IDS:

  • AP/College Board FAILED 0A5FD99233A74D8FA3A74F52E5F6CDEC: no implicit conversion of nil into String
  • BELLEVUE SCHOOL DISTRICT FAILED 83E62C7E58914847B9226BD7CCC83DC9: 783: unexpected token at 'An unhandled lowlevel error occured. The application logs may have details.
  • Surrey School District (No 36)
    FAILED E5C475D6241A494582FC636F11097670: 783: unexpected token at 'An unhandled lowlevel error occured. The application logs may have details.

ThreadError: can't create Thread: Not enough space

Hello! Thanks a bunch for this project. I realize the above error is most likely a problem with rake; however, since I am not very familiar with ruby, I was wondering what you guys might do in a situation like mine.

Steps to Reproduce

  1. Follow your steps listed in the Readme.md
  2. Sit back and wait a while. Jurisdictions goes by quickly. While Fetching sets for Pennsylvania, you may get the following error:

TRACE.log

DB problems

The script doesn't seem to create the DB and tables before running.

Yesterday I ran the script and after downloading a lot of stuff, I got a error saying standards DB doesn't exist. So I created it.

Today, after re-downloading all the data again I get a new error

PG::UndefinedTable: ERROR:  relation "jurisdictions" does not exist
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "jurisdictions" does not exist
LINE 5:                WHERE a.attrelid = '"jurisdictions"'::regclas...

Perhaps the README should include the CREATE TABLE statements? Or I might not be using the tool right...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.