simone / django-compositekey Goto Github PK

Hello,
I am trying to use django-compositekey on django 1.9 project and it is giving me this error: "ImportError: cannot import name EmptyShortCircuit". What can I do about it. Please help me.

The class "UseConcat" assumes:

concat = "||"
cq = "quote(%s%s)" # if the value is not an int sorround it with quotes

but it is not true for all database backends. In my case, SQL server, I had to use

concat = "+"
cq = "cast(%s%s as nvarchar(4000))" # never adds quotes

which also implies using direct string casting instead of "self.quote_v" in the following line (around 67), because SQL server does not add quotes when casting to string:

params = [SEPARATOR.join([self.quote_v(v) for v in val]) for val in self.values]

I wanted to use compositekey in the following model:

from compositekey import db

class Relationship(models.Model):
    class Meta:
        db_table = 'follows'

    id = db.MultiFieldPK("u", "tu")
    u = models.IntegerField(db_column='user_id')
    tu = models.IntegerField(db_column='target_user_id')
    d = models.DateTimeField(db_column='created', default=datetime.utcnow)

It's a new model and none of my other models have any fk relation to it.

However when I add this I get an exception when I try a manage.py shell etc

  File "/Users/anentropic/Documents/Dev/Work/myproj/otherapp/models.py", line 57, in <module>
    class UnrelatedModel(AnotherUnrelatedModel):
  File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/django/db/models/base.py", line 167, in __new__
    new_class.add_to_class(field.name, copy.deepcopy(field))
  File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/django/db/models/base.py", line 219, in add_to_class
    value.contribute_to_class(cls, name)
  File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/compositekey/db/models/fields/related.py", line 31, in contribute_to_class
    assert not self.fields_ext or not self._fields,\
AttributeError: 'ForeignKey' object has no attribute 'fields_ext'

As best I can tell it looks like maybe the fk monkeypatch has arrived too late to affect models which have already been imported by the time I reach my first model using a composite key?

if so it's odd that this isn't mentioned in the docs?

Hi,

How do you tell django that you have a select_related query? I am enforced to disable select_related queries with an "if False and ..." hack because django does not populate the columns of the select_related tables. The reason for select_related queries not being populated is that when django receives the list of fields that must fill the sql query, the composite foreignkey fields are not there anymore because they have no db_column so there is no field.rel.to to populate the related query.

I tried giving a pseudo "field.rel" to any of the actual fields that compose the composite key but I don't have enought django knowledge to understand what's going on there yet.

If you have any clue on where to start looking I will make further research.

Thank you!

many problems, to be resolved

• UNIQUE ("library_id", "book_author,book_name")

MultiValueDictKeyError at /admin/sample/book/Simone-LDAP server/
"Key 'chapter_set-0-id' not found

id is "virtual" but have to be present, (maybe hidden in this kind of formset)

Hi Simone,

First of all thanks a lot for the project. It has helped me a lot dealing with legacy databases.

I was curious about plans to support Django 1.6 and 1.7. Do you have any ? Do you have pointers about how this could be done ?

Trying out django-compositekey on a legacy database created with inspectdb, it appears as though an attempt is made to append to the unique_together tuple:

...
File "/xxx/.virtualenvs/xxx/lib/python2.7/site-packages/compositekey/db/models/fields/multiplekey.py", line 71, in lazy_init
cls._meta.unique_together.append(names)
AttributeError: 'tuple' object has no attribute 'append'

Wrong JOIN between a multiple column with object_id.
see test composite_generic_relations
class GenericRelationsTests(TestCase):
def test_generic_relations(self):

Hi,

when we have the following models:

class BAR(models.Model):
    bar = models.DecimalField(primary_key=True, max_digits=3, decimal_places=0, blank=False, null=False, )

class FOO(models.Model):
    id = db.MultiFieldPK("foo", "bar")
    foo = models.DecimalField(max_digits=9, decimal_places=0, blank=False, null=False, )
    bar = models.ForeignKey(BAR)

the related name on db.models.fields.wrap Line 25

setattr(obj, cache_name, assemble_pk(*[f.get_prep_value(getattr(obj, f.name)) for f in fields]))

looks for the fieldname, not for the attname like in

setattr(obj, cache_name, assemble_pk(*[f.get_prep_value(getattr(obj, f.attname)) for f in fields]))

which doesn't find the related object and raises a DoesNotExist.

I'm not sure if choosing the field instead of the attname has some purpose. Furthermore this could break dependencies.

Tested with 1.4.x Branch on Django 1.4.3.

Cheers

In postgresql some many to many auto generated try to add constraint with models not yet preparated (installed)
needs, check order of pendings constraints

Hi,
I wonder if there is a way to use django-compositekey with Contenttype to define generics objects.
Thanks

When creating the db schema, the django command tries to create a column with an invalid identifier ("ID") as below:

(django1.4)[fabio@latitude testprj]$ python manage.py syncdb
activate_fk_monkey_patch
activate_get_from_clause_monkey_patch
activate_delete_monkey_patch
activate_make_atom_monkey_patch
activate_get_fields_with_model_monkey_patch
activate_insert_query_monkey_patch
activate_add_fields_monkey_patch
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log
Creating table sample_book
Traceback (most recent call last):
File "manage.py", line 9, in
execute_from_command_line(sys.argv)
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/init.py", line 422, in execute_from_command_line
utility.execute()
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/init.py", line 361, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(_args, *_options.dict)
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, *_options)
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/env/django1.4/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 101, in handle_noargs
cursor.execute(statement)
File "/env/django1.4/lib/python2.7/site-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/env/django1.4/lib/python2.7/site-packages/django/db/backends/oracle/base.py", line 654, in execute
return self.cursor.execute(query, self._param_generator(params))
django.db.utils.DatabaseError: ORA-00904: "ID": invalid identifier

(django1.4)[fabio@latitude testprj]$

EDIT: "ID" is a valid identifier for oracle databases, seems like a column named "ID" is missing in the table being queried.

standard single primary key
SELECT * FROM AUTORI WHERE ID IN (1, 2, 3, 4, 5)

with a multiple field key (ID1 and ID2)

As has become the query with composite key?
SELECT * FROM AUTHORS WHERE ID_1 IN (1, 2, 3, 4, 5) AND IN ID_2 ("a", "b", "c", "d", "e")

:-(
This does not work, because it also select all the intersections

Obviously I need also the DELETE WHERE ***

We can also SELECT * WHERE (ID_1=1 AND ID_2="a") OR (..)
but is seem to be a bad smell

The composite primary key of an m2m join table row appears to be incorrect if it consists of two other model objects (instead of a varchar for example).

This is best demonstrated with an example

# models.py
from django.db import models
from compositekey import db


class Author(models.Model):
    name = models.CharField(max_length=100)


class Book(models.Model):
    title = models.CharField(max_length=100)


class BookAuthor(models.Model):
    id = db.MultiFieldPK('author', 'book')
    author = models.ForeignKey('Author')
    book = models.ForeignKey('Book')

# Using the django shell
>>> author = Author()
>>> author.name = "Charles Darwin"
>>> author.save()
>>> 
>>> book = Book()
>>> book.title = "Origin of the Species"
>>> book.save()
>>> 
>>> ba = BookAuthor(author=author, book=book)
>>> ba.save()
>>> 
>>> print author.pk
1
>>> print book.pk
1
>>> print ba.pk
Author object-Book object
>>> 

The issue is that the primary key of the book author (ba.pk) is not unique when
quering the python instance.

The fix for this issue is relatively simple and a patch is included.
https://gist.github.com/3957348

The only thing that needs to be modified is the function assemble_pk in utils.py
where a check has been added to test if one of the "values" is a django Model.

The patch was generated using the version of the code on PyPI (which I noticed
is slightly different to the one in github)

FIX
self.connection.ops.return_insert_id()

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 71, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 96, in __iter__
    self._fetch_all()
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 857, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python2.7/dist-packages/compositekey/db/models/query.py", line 155, in iterator
    for row in compiler.results_iter():
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 713, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 776, in execute_sql
    sql, params = self.as_sql()
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 84, in as_sql
    ordering, o_params, ordering_group_by = self.get_ordering()
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 403, in get_ordering
    self.query.get_meta(), default_order=asc):
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 437, in find_ordering_name
    field, cols, alias, joins, opts = self._setup_joins(pieces, opts, alias)
  File "/usr/local/lib/python2.7/dist-packages/compositekey/db/models/sql/compiler.py", line 90, in _setup_joins
    opts, alias, False)
ValueError: need more than 5 values to unpack

django-compositekey from pip:

Name: django-compositekey
Version: 1.5.0

Django from Debian:

Package: python-django                   
State: installed
Automatically installed: no
Version: 1.6.6-1

Support python3

There are some django aggregation tests that with compositekey id go in error.
See compositekey_aggragarion tests

Dear Team,

Hope you all doing Well!
In my project, i am using 'django-compositekey-1.5.0'.

Model.py:

class Book(models.Model):
id = db.MultiFieldPK("name", "author")
name = models.CharField(max_length=100)
author = models.CharField(max_length=100)
publisher = models.CharField(max_length=25)

class Meta:
    ordering = ('name', 'author')

def __unicode__(self):
    return u"%s %s" % (self.name, self.author)

Got Below Error:
File "C:\Users\manoj.remala\AppData\Local\Programs\Python\Python37-32\lib\site-packages\compositekey\db\models\fields\wrap.py", line 49
if len(values) <> len(fields):
^
SyntaxError: invalid syntax

(venv1) C:\Users\manoj.remala\PycharmProjects\MANOJ_IVY\IVYFSSWeb\IVYFSS>

Note: I have changed not equal symbol '<>' to '!='

Now, got a new issue
File "C:\Users\manoj.remala\AppData\Local\Programs\Python\Python37-32\lib\site-packages\compositekey\db\models\base.py", line 8, in
from itertools import izip
ImportError: cannot import name 'izip' from 'itertools' (unknown location)

Could not able to figure it out. Could you please help.

Thanks
Manoj R

Hello,

currently an update on the parent table doesn't update the children e.g. in mysql it results in "Cannot delete or update a parent row". Deletes work (warning in the admin menu doesn't include the children). Setting ON UPDATE CASCADE on the foreign key in mysql does the job, but breaks the concept of django with emulating the cascades.

Besides: Great work simone, you've got the only working compound PK solution working.

from compositekey import db
from django.db import models

class Blog(models.Model):
id = db.MultiFieldPK("title", "author")
title = models.CharField(max_length=100)
author = models.CharField(max_length=100)

class Meta:
    app_label = 'foo'

class Post(models.Model):
post = models.CharField(max_length=100)
blog = models.ForeignKey(Blog, to_field="id",
fields_ext={
"author": {"db_column" :"author"},
"title" : {"db_column" :"title"},
})
class Meta:
app_label = 'foo'

CREATE TABLE IF NOT EXISTS solar_post (
id int(11) NOT NULL AUTO_INCREMENT,
title varchar(100) NOT NULL,
author varchar(100) NOT NULL,
post varchar(100) NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY bar (title,author),
KEY solar_post_841a7e28 (title),
KEY solar_post_a5d5a658 (author)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=6 ;

OLD:
ALTER TABLE solar_post ADD CONSTRAINT solar_post_ihgdbfk_1 FOREIGN KEY (title, author) REFERENCES solar_blog (title, author)

NEW:
ALTER TABLE solar_post ADD CONSTRAINT solar_post_ihgdbfk_1 FOREIGN KEY (title, author) REFERENCES solar_blog (title, author) ON UPDATE CASCADE

This issue is a very minor issue that can be worked around relatively easily.

The issue is caused when creating a new instance of a model using composite keys without any parameters.

This is best demonstrated with an example

# models.py
from django.db import models
from compositekey import db


class Author(models.Model):
    name = models.CharField(max_length=100)


class Book(models.Model):
    title = models.CharField(max_length=100)


class BookAuthor(models.Model):
    id = db.MultiFieldPK('author', 'book')
    author = models.ForeignKey('Author')
    book = models.ForeignKey('Book')

# Using the django shell
>>> author = Author()
>>> author.name = "Charles Darwin"
>>> author.save()
>>> 
>>> book = Book()
>>> book.title = "Origin of the Species"
>>> book.save()
>>> 
>>> ba = BookAuthor()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/ssd/workspace/OpenSource/django-compositekey-fixed/VirtualEnv/local/lib/python2.7/site-packages/compositekey/db/models/base.py", line 108, in patched_model_init
    self.pk
  File "/ssd/workspace/OpenSource/django-compositekey-fixed/VirtualEnv/local/lib/python2.7/site-packages/django/db/models/base.py", line 428, in _get_pk_val
    return getattr(self, meta.pk.attname)
  File "/ssd/workspace/OpenSource/django-compositekey-fixed/VirtualEnv/local/lib/python2.7/site-packages/compositekey/db/models/fields/wrap.py", line 25, in _get
    setattr(obj, cache_name, assemble_pk(*[f.get_prep_value(getattr(obj, f.name)) for f in fields]))
  File "/ssd/workspace/OpenSource/django-compositekey-fixed/VirtualEnv/local/lib/python2.7/site-packages/django/db/models/fields/related.py", line 343, in __get__
    raise self.field.rel.to.DoesNotExist
DoesNotExist

The workaround for this issue is to simply create the BookAuthor using keyword arguments

>>> ba = BookAuthor(author=author, book=book)
>>> ba.save()
>>> 

Hi!

Using model.object.raw() I've got to the case when model instance fields don't match db tables fields.

RawQuerySet.__iter__ requires primary key fields to be in the cursor ( https://github.com/django/django/blob/stable/1.4.x/django/db/models/query.py#L1480 ), that's why I've added fake id value like

SELECT 'fake' id, * FROM ...

As a result I've got values in the model shifted. I've tried to move fake id after *, but that didn't help. Digging where it fails I've found that it happens in patched_model_init as it expects args matching _meta.db_fields not true _meta.fields.

It could be solved moving id = MultiFieldPK(...) after all the value fields. But I suspect it won't solve the problem in case of model inheritance.

I've added tests for the case. https://github.com/soulne4ny/django-compositekey/tree/master/testprj/composite_raw/.

I've tried to make a patch, but it does not work.

thank you!!!!!

I had problems with old 0.1-alpha version and oracle backend... but was solved with the last version of 1.4.x branch...

"("Trying to 'quote' a multiple key FIELD. Operation not supported yet. Please fix it..."

I'm using Django to interface with a read-only legacy database (in addition to a primary database).

One of the legacy tables has been created haphazardly without a primary key column, and is essentially a M2M table. The auto-generated model looks like so:

class LegacyTable(models.Model):
    a = models.ForeignKey(A, db_column='a')
    b = models.ForeignKey(B, db_column='b')

    class Meta:
        managed = False
        db_table = 'legacy_table'

Obviously, running queries on this model throws the usual errors since there's no id column and no primary_key defined. I tried adding the following composite key field, thinking it might make Django happy:

id = db.MultiFieldPK('a', 'b')

but when trying to run a simple query, I get:

  File "/home/user/.virtualenvs/proj/lib/python2.7/site-packages/compositekey/db/models/sql/compiler.py", line 90, in _setup_joins
    opts, alias, False)
ValueError: need more than 5 values to unpack

Is this an error on my part, or is this scenario not supported at all? I'm running Django 1.6 if it matters.

Hello,
I'm using django-compositekey (id = db.MultiFieldPK("field1", "field2", "field3")) on an oracle remote database created with inspectdb. It is successful on some tables but there is a table which returns this error : "quantize result has too many digits for current context". Note that field1 is Charfield and field2 and 3 are IntegerField. My django version is 1.5.
Can somebody help me?

This breaks compatibility with django 1.5.

In compositekey/db/models/sql/query.py:

from django.db.models.sql.constants import LHS_JOIN_COL, RHS_JOIN_COL, LOOKUP_SEP, LHS_ALIAS

In django 1.5, the constants.py file no longer has the values that we are trying to import. The constants in question are used for tuple lookups.

In the testapp example, Author.friends doesn't permit to relate an author with himself.

Classic Author (with no CFP) works.

Observation:
MayToMany in django assume that symmetrical=True,
I think there is an check in the django code for this unique constraint to evolve for the composite keys.

this is the exception:

IntegrityError at /admin/sample/author/'Simone'-'33'/
duplicate key value violates unique constraint "sample_author_friends_from_author_name_from_author_age_to_a_key"
DETAIL: Key (from_author_name, from_author_age, to_author_name, to_author_age)=(Simone, 33, Simone, 33) already exists.

The output dlelete query seem to be ok, but I think the raprpesentation string of date is different from the database quoteliteral concatenation of the datetime field.

DELETE FROM "composite_basic_article" WHERE quote_literal("headline")||'#'||quote_literal("pub_date") IN (E'''Fourth article''#''2005-07-31 00:00:00''',E'''Default headline''#''2005-07-31 00:00:00''',E'''Article 7''#''2005-07-31 12:30:00''',E'''Article 6''#''2005-07-31 00:00:00''',E'''Area man programs in Python''#''2005-07-28 00:00:00''')

infact this don't delete the right articles

Compile query from model not "composite" but with virtualrelation m2m

for example Business.objects.filter(employees__employee_code=420)

where Employee has one ManyToMany to Business.