Sometimes pick_state.top() returns None for large Planes about bevy_mod_picking HOT 11 CLOSED

It seems like the example completely breaks when the Plane size is set to 50.0.

The plane is only highlighted when the mouse goes outside.

Any ideas why this might be happening?

from bevy_mod_picking.

I've been looking into it, and it doesn't seem like the issue is with objects being partially off the screen, at least not for small ones. After moving the camera so that a cube is halfway off the screen, selecting it still works fine.

I've also tried with a simple barycentric implementation kindly stolen taken from StackExchange, but the example still breaks with a Plane of size 30.0. I think we can discard the point_in_tri function as being the issue.

Here's the code I used, it's also here:

/// Checks if a point is inside a triangle, using barycentric coordinates
fn point_in_tri(p: &Vec2, a: &Vec2, b: &Vec2, c: &Vec2) -> bool {
    // https://stackoverflow.com/questions/2049582/how-to-determine-if-a-point-is-in-a-2d-triangle
    let s = a.y() * c.x() - a.x() * c.y() + (c.y() - a.y()) * p.x() + (a.x() - c.x()) * p.y();
    let t = a.x() * b.y() - a.y() * b.x() + (a.y() - b.y()) * p.x() + (b.x() - a.x()) * p.y();

    if (s < 0.0) != (t < 0.0) {
        return false;
    }

    let area = -b.y() * c.x() + a.y() * (c.x() - b.x()) + a.x() * (b.y() - c.y()) + b.x() * c.y();

    return if area < 0.0 {
        s <= 0.0 && s + t >= area
    } else {
        s >= 0.0 && s + t <= area
    };
}

It looks like the issue might be with the transformation into ndc coordinates then, I'll try to look a bit into it.

from bevy_mod_picking.

Is it possible that the points that are behind the camera get multiplied by -1?

I'm thinking it might be happening in the /w part of the projection transformation. My initial suspicion was that it seemed related to going offscreen, but I think you're right, it's specifically cases where the vertices go behind the camera.

from bevy_mod_picking.

Yes! Adding .abs() to w fixes it!

let transformed = projection_matrix.mul_vec4(vertex_pos.extend(1.0));
let w = transformed.w().abs();
triangle[i] = Vec3::from(transformed.truncate() / w);

I should have thought about that sooner!

I'll have to check if this fixes my initial issue, but I have the feeling that it's something else, as in my game the plane is entirely in front of the camera.

from bevy_mod_picking.

It does fix it! I used to have dead spots all over the place, and it works perfectly now. Thanks for all the help!

from bevy_mod_picking.

Thanks for reporting this! I'm almost certain this is related to #5 and #6 . I'd have to do some debugging with the triangle area calculation code however. I wonder, does this happen to other objects that are partially off screen?

from bevy_mod_picking.

The vertices being used by the point_in_tri function look a bit odd. The first two numbers are tri area, and cursor tri area (hit if equal), and the three pairs that follow are the x/y coordinates of the vertices in ndc, this is with a plane size of 18:
No Hit: 493.499 743.078 0.359,0.710 70.805,139.869 2.939,-1.200
Hit: 182.905 182.905 0.359,0.710 -1.089,0.445 70.805,139.869

After plotting the points, they don't make a ton of sense compared to what I see on screen.
After dropping the plane size to 17, picking behaves as expected. Points look like this:
No Hit: 96.096 135.051 0.349,0.690 -14.860,-29.355 2.651,-1.082
Hit: 37.960 37.960 0.349,0.690 -1.047,0.427 -14.860,-29.355

It appears that one of the points, after being transformed to ndc space, is flying up to +++x and +++y.

from bevy_mod_picking.

Yup, that's what I was getting at with the highlighted coordinates; for some reason at some threshold, some of the points are shooting off into space when transformed into NDC.

from bevy_mod_picking.

Is it possible that the points that are behind the camera get multiplied by -1? I'm not too confident with projections and stuff, but I remember reading about this somewhere. The tests I have been doing seem to suggest this, but I'm not really sure.

Here is the information for when the mouse is outside of the plane, the black circle in the following screenshot:

Mouse ndc coordinates are [-0.9342407, 0.953928].

Ndc coordinates for the detected triangle are:
a: [0.4486993, 0.8863647], b: [-1.5006784, 0.61249065], c: [1.7613124, 3.4793124]

Real coordinates for the detected triangle are:
a: [15, 0, -15], b: [-15, 0, -15], c: [-15, 0, 15]

The c point in ndc coordinates checks out in magnitude, but should be negative.

I have no idea if this is what's actually happening or how to go about fixing it if it is, any ideas?

from bevy_mod_picking.

Another thing that makes me think this is the issue is that when initializing the Camera in the example with

Camera3dComponents {
    translation: Translation::new(0.0, 20.0, 0.0),
    rotation: Rotation(Quat::from_xyzw(-0.5, -0.5, -0.5, 0.5).normalize()),
    ..Default::default()
},

and setting the Plane to size 50.0, everything works correctly.

The code above makes the camera be top down with most of the plane still off screen, but all of the vertices are in front of the camera.

from bevy_mod_picking.

Looks like we had the same thought! I did (basically) the same thing in the PR, let me know if this fixes your initial issue. I played around with a few plane sizes and it seems to resolve that problem. The picking doesn't seem to be exact for the plane (there is some dead space by the edge of the plane) which concerns me. Knowing that the current point_in_tri implementation method is susceptible to floating point error, I think this is acceptable, and we can look at picking accuracy when we merge in the barycentric PR you have in draft.

from bevy_mod_picking.