burke-software / django-simple-import Goto Github PK

For one of the columns in an xlsx file, only one value for the whole column imported. Here's what was happening in the majority of cases:

On line 299 of simple_import/views.py, there is a check to see if the value of the cell is in the database_values list. In my case, here's what was checking as False:

# from debugger pane:
cell = u'AL'
database_values = ('AL', 'AK', 'AS', ...)
verbose_values = ('Alabama', 'Alaska', 'American Samoa', ...)

The cell value was unicode, but the database values were not, so the cell in database_values == False. Same would have been true of verbose_values. FWIW, these choices are from localflavor: https://github.com/django/django-localflavor/blob/master/localflavor/us/us_states.py

My first thought is to coerce cell, the values in database_values, and the values in verbose_values all to unicode before making the check. Any thoughts on that?

django==1.6.8
django-simple-import==1.19

For example if the user see's "Monday" but the database is "M" the import won't work.

choices  = ['M', 'Monday'] ,etc

@bufke is this error caused by a date problem? Picture below.

Original birth date date is in yyyy-mm-dd format, but reads as 36290 in the preview. My apologies if unrelated to the error.

set_method_from_cell() was moved after new_object.save() for good reason in 32a6c5d. This breaks methods like set_password() that do not save the object themselves. My original fix was to put new_object.save() inside of set_method_from_cell(), but this wastefully hits the database for each field instead of once per objects.

The best place for a second save() call after calling the methods is probably in do_import() on line 369: https://github.com/burke-software/django-simple-import/blob/master/simple_import/views.py#L369. Looking there, though, why is there a second loop of set_field_from_cell() starting at line 370? Can I remove it? I don't see what it accomplishes since new_object is never saved afterwards.

https://app.getsentry.com/burke-software-and-consulting/schooldriver/group/51617866/

1. Go to match relations.
2. Hit back. You get a 500

Internal Server Error: /simple_import/match_columns/26/
storages.backends.s3boto in _open

With s3 at least.

After a failed import (column match), subsequent attempts fail with the error "Column match with this Column name and Import setting already exists" on several fields. i'm not sure yet which fields they pertain to, as they all look like this:

<ul class="errorlist">
    <li>__all__
        <ul class="errorlist">
            <li>Column match with this Column name and Import setting already exists.</li>
        </ul>
    </li>
</ul>

Foreign keys now have options for user, user_id. To the user the options are just both User. This also breaks unit tests.

We should switch.

Basically the fault is a lack of decent python ods libraries.

Google Drive ods just doesn't work at all.
Libreoffice ods files sometimes show extra blank columns which breaks things. Could possibly work around this.

If anyone knows of a functional ods python library please let me know.

i have multiple spreadsheets to import for one model. Each spreadsheet represents a different data source, and my model has a "source" attribute (which in this case is an FK) which needs to be set to a default value for the whole import. As it is, i can't import the file as-is without getting a "Source is required but has no match" error. i can modify the spreadsheet to get around this, but it's not ideal. It would be great if i could set the default for that field, even though there's no column in the spreadsheet.

The current PyPI package has a ton of extraneous files included in the static folder that get collected when running ./manage.py collectstatic. As far as I can tell, they don't need to be included in the package at all, but especially not in the static folder. Can these be moved somewhere else?

Keep getting this issue when trying to run django-sis

from simple_import.compat import User
ImportError: No module named compat

In an xlsx file, i have a column of string values, such as "66", "783 Foo". On import into a CharField, it seems that any value that can be interpreted as a decimal is converted to one, like "66.0", while other values like "783 Foo" remain as-is. The desired result is that the data is not reformatted that way.

All cells containing 0 are ignored unless the user selects "Clear field on blank cell."
https://github.com/burke-software/django-simple-import/blob/master/simple_import/views.py#L357

Tested on django-sis applicant import. "Present School" field-

Returns "invalid literal for int() with base 10: 'Redwood JR High'"

If one reuses the same header twice in a import, unexpected things happen.

Either simple import should allow working with duplicate headers or it should show a validation error.

A base template could be included that the other templates inherit from. This way, a user could override the base template without having to necessarily override the other four templates, when integrating with their markup. i'm happy to provide a pull request for this; just let me know if there are any preferences/conventions to consider.

https://github.com/burke-software/django-sis/blob/master/ecwsp/sis/models.py#L250
Verbose name is Student Contact
Real name is EmergencyContact
https://github.com/burke-software/django-sis/blob/master/ecwsp/sis/models.py#L447
Reference from student is called Emergency Contact too

When I try an upload via the admin interface I keep receiving the error

"Column 'user_id' cannot be null"

Should this auto-populate from the user id of my login?

thanks!

Use case: attempting to update records using only username as key alerts the need to return and add fname/lname/mdlusr to your document.

KeyError: u"Migration simple_import.0001_initial dependencies reference nonexistent parent node (u'contenttypes', u'0002_remove_content_type_name')"

Migration "0002_remove_content_type_name" wasn't introduced until Django 1.8, so i believe the fix will be to depend on "0001_initial" instead.

Using Django 1.7.8.

It's a big str encoding mess right now...

Hi,

Not sure it is a bug...

When trying to update my model which have unique_together = ('filed_1', 'filed_2', 'field_3' , 'field_4', 'field_5',) I have only one field to choose which in this model is impossible.

Any thoughts ?

Cheers,

Jakub

Use case: school using django-sis requests all student passwords to be changed. This can't be done currently via simple import.

Additionally, the "Recent Actions" display does not show new users.

Hi,

Since you last update, all versions has been dropped. I use 1.9 in one of my project.

BR

Mathias

http://i.imgur.com/O269BvN.png

Why have it if there is really no option.

Could have it default to ID to save time. This would still show the user they must use the ID and not type in say a name.

Maybe as an option that is off by default

Error at /simple_import/match_columns/3/
new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

django==1.6.8
django-simple-import==1.19

PR incoming... first i'm gonna see if i can isolate the offending newline and make a test, if it doesn't take too long.

Anytime soon to support sql import? Will be cool to support sql.