when import csv file ,KeyError: u'id' occured about django-import-export HOT 36 CLOSED

I had this issue, too, the first time I tried to use the tool, because the primary key in my database is set to one of my defined fields. You simply need to say which field that import_export should use instead. Specify this in the Meta class of your Model Resources class. To use tukhfatov's example:

class PromoCodeResource(resources.ModelResource):

    class Meta:
         model = PromoCode
         exclude = ('pc_used','pc_date',)
         import_id_fields = ['pc_code']

from django-import-export.

If you override get_instance in your ModelResource like this it works (even without the 'id' column trick in the CSV):

    def get_instance(self, instance_loader, row):
        try:
            params = {}
            for key in instance_loader.resource.get_import_id_fields():
                field = instance_loader.resource.fields[key]
                params[field.attribute] = field.clean(row)
            return self.get_queryset().get(**params)
        except Exception:
            return None

I havent had time to look into the why though.

from django-import-export.

import_id_fields = ('pub_id')

Did you forget the trailing comma? ('pub_id',)

from django-import-export.

Your model does not have id field, right? Of this is case get_instance should be overridden. See doc on import workflow

from django-import-export.

I figured it out that if you don't have an 'id' field in your model one can assign
import_id_fields=['some_other_field'] in class Meta of ModelResource and upload a file without the id field. It worked quite smoothly for me.

from django-import-export.

Thanks, caljess599, for sharing your solution for this problem. I also stumbled onto this as I was using a custom primary key, and the above fixed my import problems :)

from django-import-export.

@DaveJ61 : re-reading http://django-import-export.readthedocs.io/en/stable/getting_started.html#importing-data , it seems to me that with the first csv line as you stated:

"id","barcode","partcode","manufacturer","description","quantity","reorderquantity","minquantity","maxquantity","cost","rrp","category"

and the fields in your Meta resource class as they are, you shouldn't have to declare import_id_fields = ['item_id'] as your csv file already uses the default id field.
Only if you would use item_id as column name in your csv you would have to do this.
Futhermore, I think you don't need any of the declared fields in import_id_fields.

The error means import-export is looking for an item_id column in your csv file but can't find it. And sure enough, it isn't present afaics.

from django-import-export.

Oops... I haven't been coding for a little while as I have been too busy with my job. I'll take a look at this a little later today and report back.

from django-import-export.

It looks like you are not exporting id field. django-import-export cannot know which is instance to initialize without it. Relevant docs:

https://django-import-export.readthedocs.org/en/latest/import_workflow.html#import-data-method-workflow

from django-import-export.

Got the same error.
In my case problem was in csv file that I used to import.
The original exported file is nonBOM -> that means the first bytes of csv file is actual "id,".
But my editor saved file as UTF8-BOM (http://en.wikipedia.org/wiki/Byte_order_mark) -> file starts with 3 additional bytes at the beginning: EF BB BF

from django-import-export.

It would be great to add failing test for this.

from django-import-export.

Hi, Im trying to import xls file, but had such error

Errors

Line number: 1 - u'id'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/import_export/resources.py", line 317, in import_data
instance, new = self.get_or_init_instance(instance_loader, row)
File "/usr/local/lib/python2.7/dist-packages/import_export/resources.py", line 149, in get_or_init_instance
instance = self.get_instance(instance_loader, row)
KeyError: u'id'

this is my model:
class PromoCode(models.Model):
pc_code = models.IntegerField(primary_key = True)
pc_used = models.IntegerField(default = 0)
pc_date = models.DateField(blank = True)

class Meta:
    db_table = 'promo_code'

resource file:
class PromoCodeResource(resources.ModelResource):
class Meta:
model = PromoCode
exclude = ('pc_used','pc_date',)

admin.py
class PromoCodeAdmin(ImportExportModelAdmin):
resource_class = PromoCodeResource
pass

admin.site.register(PromoCode, PromoCodeAdmin)

from django-import-export.

It doesn't works for me

class SignUpResource(resources.ModelResource):
    class Meta:
        model = SignUp
        exclude = ('id','gender','timestamp', )
        import_id_fields = ['id','gender','timestamp',]

I still have same error.

from django-import-export.

You are excluding the 'id' field as well as telling django to use it for importing objects. That can't work imho.
I think you need to at least remove the id field from exclude, and put only the id field in import_id_fields. But if id is the primary key for your object, you shouldn't have to do anything custom. It's hard to tell without more code (of your model, specifically).

from django-import-export.

Hey, check the separator in your import file. Should be a comma!
In my case the error was the csv file.
The file was also an export that i tried to import directly (for test).
My exported csv file had a semicolon as separator and not the expected comma...

from django-import-export.

Model = Testlines

That should be model

from django-import-export.

my bad. thanks for your time shaggyfrog !

from django-import-export.

I have read just about everything about this problem and I have been trying to eliminate this for nearly 2 days.
I have tried all sorts of combinations to try and get this running but have failed.

from django.contrib import admin
from import_export import resources
from import_export.widgets import ForeignKeyWidget
from import_export.admin import ImportExportModelAdmin
from models import Item

class ItemResource(resources.ModelResource):
    class Meta:
        model = Item
        import_id_fields = ['item_id', 'partcode', 'manufacturer', 'description', 'quantity', 'reorderquantity', 'minquantity', 'maxquantity', 'cost', 'rrp', 'category',]
        fields = ('barcode', 'partcode', 'manufacturer', 'description', 'quantity', 'reorderquantity', 'minquantity', 'maxquantity', 'cost', 'rrp', 'category',)
        #exclude = ('id',)

KeyError: 'item_id'
ERROR:root:'item_id'
Traceback (most recent call last):
  File "/home/david/projects/trc/venv/local/lib/python2.7/site-packages/import_export/resources.py", line 440, in import_row
    instance, new = self.get_or_init_instance(instance_loader, row)
  File "/home/david/projects/trc/venv/local/lib/python2.7/site-packages/import_export/resources.py", line 262, in get_or_init_instance
    instance = self.get_instance(instance_loader, row)
  File "/home/david/projects/trc/venv/local/lib/python2.7/site-packages/import_export/resources.py", line 256, in get_instance
    return instance_loader.get_instance(row)
  File "/home/david/projects/trc/venv/local/lib/python2.7/site-packages/import_export/instance_loaders.py", line 31, in get_instance
    field = self.resource.fields[key]

I have tried with or without an id column in both csv and xlsx files. I have tried using import_id_fields. I even created an item_id field in my database even though I shouldn't need one as another filed is the primary key. I am finding the documentation difficult to follow and the example is incomplete. It also seems there are quite a few others having this issue.
Can someone please help?

from django-import-export.

Hi @DaveJ61, can we see the first line in the csv file you are trying to import?

from django-import-export.

Hi, thanks for the reply. The first line is as follows:-

"id","barcode","partcode","manufacturer","description","quantity","reorderquantity","minquantity","maxquantity","cost","rrp","category"

from django-import-export.

Hi, thanks for the reply. I have tried every alternative and I believe I have already tried what you have suggested. However, I will try again and let you know.

from django-import-export.

Thanks,solution to my problem perfectly

from django-import-export.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from django-import-export.

Just create an empty column namely id and then import the same file via Django import_export.

from django-import-export.

Just create an empty column namely id and then import the same file via Django import_export.

thanks, this makes import records with auto-increment id

from django-import-export.

i use
import_export.version '2.0.2'
django.version '3.0.4'
import .xlsx excel data have the same problem!

#错误
行号: 1 - 'id'
vpc-erymihwm, subnet-naku2y0n, 测试环境, 192.0.0.0/24, False, False, ap-beijing-3, rtb-mhj5kobp, 2017-12-11 13:08:14, 212, None, None, False, 253, []
Traceback (most recent call last):
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 500, in import_row
instance, new = self.get_or_init_instance(instance_loader, row)
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 277, in get_or_init_instance
instance = self.get_instance(instance_loader, row)
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 266, in get_instance
self.fields[f] for f in self.get_import_id_fields()
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 266, in
self.fields[f] for f in self.get_import_id_fields()
KeyError: 'id'

行号: 2 - 'id'
vpc-erymihwm, subnet-c4diznbt, 通用环境, 192.0.1.0/24, False, False, ap-beijing-3, rtb-a4j9hs2v, 2017-12-15 19:31:05, 252, None, None, False, 253, []
Traceback (most recent call last):
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 500, in import_row
instance, new = self.get_or_init_instance(instance_loader, row)
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 277, in get_or_init_instance
instance = self.get_instance(instance_loader, row)
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 266, in get_instance
self.fields[f] for f in self.get_import_id_fields()
File "D:\Anaconda3\lib\site-packages\import_export\resources.py", line 266, in
self.fields[f] for f in self.get_import_id_fields()
KeyError: 'id'

##models.py

from django.db import models

class Subnets(models.Model):
VpcId = models.CharField(max_length=200, null=True, blank=True)
SubnetId = models.CharField(
max_length=20, primary_key=True, null=False)
SubnetName = models.CharField(max_length=200, null=True, blank=True)
CidrBlock = models.CharField(max_length=20, unique=True, null=False)
IsDefault = models.CharField(max_length=200, null=True, blank=True)
EnableBroadcast = models.CharField(max_length=200, null=True, blank=True)
Zone = models.CharField(max_length=200, null=True, blank=True)
RouteTableId = models.CharField(max_length=200, null=True, blank=True)
CreatedTime = models.CharField(max_length=200, null=True, blank=True)
AvailableIpAddressCount = models.IntegerField(null=True, blank=True)
Ipv6CidrBlock = models.CharField(max_length=200, null=True, blank=True)
NetworkAclId = models.CharField(max_length=200, null=True, blank=True)
IsRemoteVpcSnat = models.CharField(max_length=200, null=True, blank=True)
TotalIpAddressCount = models.IntegerField(null=True, blank=True)
TagSet = models.CharField(max_length=200, null=True, blank=True)

def __self__(self):
    return self.CidrBlock

##admin.py

from django.contrib import admin
from backend.models import Subnets
from import_export import resources
from import_export.admin import ImportExportModelAdmin

class SubnetsResource(resources.ModelResource):
class Meta:
model = Subnets
fields = ('VpcId', 'SubnetId', 'SubnetName', 'CidrBlock', 'IsDefault', 'EnableBroadcast', 'Zone', 'RouteTableId', 'CreatedTime',
'AvailableIpAddressCount', 'Ipv6CidrBlock', 'NetworkAclId', 'IsRemoteVpcSnat', 'TotalIpAddressCount', 'TagSet',)
import_id_fields = ('SubnetId',)
class SubnetsAdmin(ImportExportModelAdmin):
reosurce_class = SubnetsResource
admin.site.register(Subnets, SubnetsAdmin)

i set import_id_fields = ('SubnetId',) but it doesn't effect！
i try models.py delete SubnetId primary_key and add " id = models.AutoField(primary_key=True)" ! the import success! but, i don't want models structure like this!
any one can help me?

year,later i try install django-import-export 2.0.1 replace 2.0.2，the import success!

from django-import-export.

Similar problem... Need Some help with it
I got KeyError: 'u' instead of 'id'

Line number: 1 - 'u'
Traceback (most recent call last):
File "C:\Users\poke6\Desktop\Web\env\lib\site-packages\import_export\resources.py", line 639, in import_row
instance, new = self.get_or_init_instance(instance_loader, row)
File "C:\Users\poke6\Desktop\Web\env\lib\site-packages\import_export\resources.py", line 334, in get_or_init_instance
instance = self.get_instance(instance_loader, row)
File "C:\Users\poke6\Desktop\Web\env\lib\site-packages\import_export\resources.py", line 321, in get_instance
import_id_fields = [
File "C:\Users\poke6\Desktop\Web\env\lib\site-packages\import_export\resources.py", line 322, in
self.fields[f] for f in self.get_import_id_fields()
KeyError: 'u'

class Publication(models.Model):
pub_id = models.AutoField(primary_key=True)
publication_number = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
reference_name = models.CharField(max_length=40, blank=True)
first_author = models.CharField(max_length=40, blank=True)
year = models.IntegerField(blank=True, null=True)
endnote = models.CharField(max_length=255, blank=True)
title = models.CharField(max_length=255, blank=True)
publication_type = models.CharField(max_length=40, blank=True)
irrelevance_reason = models.TextField(blank=True)
disease_treated = models.CharField(max_length=40, blank=True)
total_patients = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
patients_using_device = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
event_description = models.TextField(blank=True)
country = models.CharField(max_length=40, blank=True)
objective = models.TextField(blank=True)
endpoint = models.TextField(blank=True)
conclusion = models.TextField(blank=True)
treatment_details = models.TextField(blank=True)
follow_up = models.TextField(blank=True)
technical_success = models.CharField(max_length=9, blank=True)
technical_failure = models.CharField(max_length=9, blank=True)
technical_statement = models.TextField(blank=True)
procedural_success = models.CharField(max_length=9, blank=True)
procedural_failure = models.CharField(max_length=9, blank=True)
procedural_statement = models.TextField(blank=True)
morbidity_rate = models.CharField(max_length=9, blank=True)
morbidity_statement = models.TextField(blank=True)
recommendation = models.TextField(blank=True)
evidence = models.CharField(max_length=9, blank=True)
d = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
a = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
p = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)
r = models.DecimalField(max_digits=11, decimal_places=0, null=True, blank=True)

def __str__(self):

return f'{self.publication_number}, {self.title}, {self.first_author}, {self.year}'

class PublicationResource(resources.ModelResource):
class Meta:
model = Publication
import_id_fields = ('pub_id')
fields = ('pub_id','publication_number', 'reference_name', 'first_author', 'year',
'endnote', 'title', 'publication_type','irrelevance_reason', 'disease_treated', 'total_patients',
'patients_using_device', 'event_description', 'country','objective', 'endpoint', 'conclusion',
'treatment_details', 'follow_up', 'technical_success','technical_failure', 'technical_statement', 'procedural_success',
'procedural_failure', 'procedural_statement', 'morbidity_rate','morbidity_statement', 'recommendation', 'evidence',
'd','a','p','r',)

class PublicationAdmin(ImportExportModelAdmin):
list_display = ['pub_id', 'publication_number', 'first_author', 'year',
'endnote', 'title', 'publication_type', 'disease_treated', 'total_patients',
'patients_using_device']
# form = PublicationForm
list_filter = ['year', ]
search_fields = ['first_author','year',]
resource_class = PublicationResource

admin.site.register(Publication, PublicationAdmin)

from django-import-export.

@shaggyfrog OMG Yes!!! Thank you!

from django-import-export.

from django-import-export.

@shaggyfrog , whats the version number of your package, am using this version django-import-export==2.3.0, but still getting the error.

Below is my code :

resources.py

from import_export import resources
from applicants import models


class CommonResourcesClass(resources.ModelResource):
    class Meta:
        model = models.Applicants
        fields = ('email',)
        exclude = ('pub_id', 'id',)
        import_id_fields = ('pub_id',)

models.py

class Applicants(TimeStampedModel, models.Model):
    pub_id = models.AutoField(primary_key=True)
    email = models.CharField(_('Email'), max_length=50, blank=True, null=True)
    first_name = models.CharField(_('First Name'), max_length=50, blank=True, null=True)
    last_name = models.CharField(_('Last Name'), max_length=50, blank=True, null=True)

    def __str__(self):
        return self.email

    class Meta:
        verbose_name_plural = "Applicants"

This is the error am getting :

> Line number: 1 - 'pub_id'
> 1.0, [email protected]
> Traceback (most recent call last):
> File "dashboard-backend/venv/lib/python3.8/site-packages/import_export/resources.py", line 639, in import_row
> instance, new = self.get_or_init_instance(instance_loader, row)
> File "backend/venv/lib/python3.8/site-packages/import_export/resources.py", line 334, in get_or_init_instance
> instance = self.get_instance(instance_loader, row)
> File "/dashboard-backend/venv/lib/python3.8/site-packages/import_export/resources.py", line 321, in get_instance
> import_id_fields = [
> File "/dashboard-backend/venv/lib/python3.8/site-packages/import_export/resources.py", line 322, in <listcomp>
> self.fields[f] for f in self.get_import_id_fields()
> KeyError: 'pub_id'
> 
> 
> 

from django-import-export.

I think this is occurring because you declare pub_id in exclude, but then use it in import_id_fields. Remove it from exclude and it should work

2.3.0 is an older version - there have been lots of fixes and additions since then, so I suggest to upgrade to the latest version 2.7.1

from django-import-export.

Hi @matthewhegarty , I actually did that but same error

from django-import-export.

I actually upgraded it 2.7.1 but still the same error, I also removed pub_id form exclude but still the same error

from django-import-export.

What happens if you try this:

fields = ('pub_id',)
import_id_fields = ('pub_id',)

If that still doesn't work, step through with a debugger and see where the issue is.

from django-import-export.

If I do this, it doesn't work reason in my excel file I only have one column names email , so this is the error I get :

Line number: 1 - __str__ returned non-string (type NoneType)
1.0, [email protected]
Traceback (most recent call last):
File "dashboard-backend/venv/lib/python3.8/site-packages/import_export/resources.py", line 703, in import_row
row_result.add_instance_info(instance)
File "dashboard-backend/venv/lib/python3.8/site-packages/import_export/results.py", line 43, in add_instance_info
self.object_repr = force_str(instance)
File "dashboard-backend/venv/lib/python3.8/site-packages/django/utils/encoding.py", line 64, in force_str
s = str(s)
TypeError: __str__ returned non-string (type NoneType)

@matthewhegarty

from django-import-export.

Finally it works, I added the fields like this :

class CommonResourcesClass(resources.ModelResource):
    class Meta:
        model = models.Applicants
        fields = ('pub_id', 'email',)
        import_id_fields = ('pub_id',)

from django-import-export.