Faster joins about alenka HOT 10 CLOSED

Good news! It sorts the data in every segment seperately, and can be used an additional to "sort -t "|" -k11 lineitem.tbl > lineitems.tbl" to sort all data by field "shipdate" which reduces time for FILTER?

And by using thrust::is_sorted() before join we can know, do we need to sort data or no, is it correct?

from alenka.

Yes.
It creates a file *.sort where it stores the indexes of sorted columns. So
when doing a join it checks if a segment is already sorted by looking into
the file.
Using thrust::is_sorted probably would be more expensive, it would have to
check all data in a column.

Anton

Подольск ?

On Thu, Jul 4, 2013 at 3:44 PM, AlexeyAB [email protected] wrote:

Good news! It sorts the data in every segment seperately, and can be used
an additional to "sort -t "|" -k11 lineitem.tbl > lineitems.tbl" to sort
all data by field "shipdate" which reduces time for FILTER?

And by using thrust::is_sorted() before join we can know, do we need to
sort data or no, is it correct?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/38#issuecomment-20475062
.

from alenka.

Stored in the "*.sort-file" indexes, which uses with "thrust::gather/scatter" to produce sorted segment?

Yes, and you, Moscow?

from alenka.

The sort file simply has one or more numbers like 0 - meaning that a segment is already presorted on first field.
03 - presorted on first and third fields.

Yes.

from alenka.

.sort is a metadata-file. Why not to use bitmask, an example 5(101) mean that on 1st and 3d fields is already sorted?

А как догадались, что с Подольска? :)

from alenka.

Yes, it is possible to use a bitmask, although the effect on performance would be nil. It is just a list of sorted fields.

Посмотрел Google analytics, там есть города посетителей. А почему Подольск ? С такими интересами в Сан Франциско было бы интереснее и лучше :-)

from alenka.

I can't read it :)

from alenka.

Updated.

from alenka.

Вариант. Но не так хорош разговорный английский, и сейчас не так далеко от Москвы, пока смотрю найдется ли здесь достойное применение. А сами не думали о Сан Франциско? :)

from alenka.

Английский - ерунда, учится быстро. А мне никак по семейным обстоятельствам :-(

from alenka.