iPad Pro - Camera feed is rotated 180 deg. about ofxarkit HOT 14 CLOSED

hey again @64BitsPerMinute

hmmm this is new to me, I've had other people with iPads use this addon and this is the first i've heard of it.

Just so I'm clear, cause you mention another function call that shouldn't have anything to do with the camera feed, it's the camera feed itself thats rotated incorrectly? I'll admit I generally keep the iPhone 6s I have locked to portrait but in the rare instances where I forget to lock things, I've never seen the orientation of the camera feed shown incorrectly.

In any case I will make sure to take a look. If anyone else reading this happens to have an iPad Pro, help is appreciated in testing.

Lastly, while I think i get the general gist of the issue, when you get a second, photos showing this issue would be most appreciated just to make sure I'm not mis-understanding.

(side note - ofRotate won't affect the camera feed, that's being processed by a separate shader. You can check out ARCam.h/ARCam.mm to see how the camera feed is getting processed)

from ofxarkit.

Ahhhh no need for screenshots - I see whats happening. I take back what I said, I could have swore this was working before somehow, maybe I'm just not remembering correctly or getting my thoughts jumbled.

So the issue is I'm not handling orientation events at the moment for everything - the reason for this being to make things easier on me and also to still give a good amount of flexibility in how the addon is used.

i'll be sure to add a method to allow for rotation of the camera image in this next push.

from ofxarkit.

just chiming in that I've seen this too on ipad vs iphone se -- a collaborator sees what I've made on the ipad as vertically flipped or rotated... I wasn't sure if it was differences in OF, addon version or what not but handn't gotten into debugging it deeper.

from ofxarkit.

Now that I'm a bit more awake, I see why I thought what I did, locking the orientation will cause the camera image to rotate on its own(though technically an orientation event isn't happening).

I've added a couple methods to help address actual orientation changes. The intended workflow is that you use these methods within the deviceOrientationChanged method in ofApp or wherever else you intend to handle orientation changes in your app.

I could not get my phone to recognize the upside-down portrait state and actually rotate to that state so anything involving that is technically still not solved.

• updateDeviceOrientation - this is more or less an automatic handler which should automatically rotate the camera and react to orientation changes. I would consider this to still be slightly un-reliable at the moment if you're using an iPad, since I don't have an iPad to test on.

which leads us to

• updateInterfaceOrientation - this will set the orientation prop of ARCam when the orientation of the device changes. This should ensure that ARKit's camera's projection/view matrices stay valid.

• updateRotationMatrix - this will let you set up a custom rotation value for the camera feed. From what I can see, the camera is rotated -90 degrees by default, keep that in mind when setting up a new value.

I should note that these functions are within ARCam.h - there should be corresponding functions in ARProcessor as well(possibly with slightly different names)

All in all, in my opinion, if you're going to be changing orientations a lot, it might be less of a hassle to simply lock the orientation in portrait on your device and simply call updateInterfaceOrientation for fewer headaches(this is just my opinion though).

Edit
I was wrong! There might be a slight headache since the orientation prop will never change since device is locked (I'm dumb 😞 )

from ofxarkit.

Thanks for the detailed update and response! Will try things out later tonight or tomorrow.

from ofxarkit.

Hey all, thanks for all that work!
I managed to get the addon running on my iPad Pro 10.5" and have also the camera-rotated-by-180 deg problem. also it is zoomed in way to much.
In the general tab of the project, i set the deployment info to iPad and Portrait. Also i locked the iPad to portrait.
now where it is getting strange is: i copied the example basic and changed in ARCam.mm line 54 from
rotation.makeRotationMatrix(90, ofVec3f(0,0,1));
to
rotation.makeRotationMatrix(-90, ofVec3f(0,0,1));
then the image is correct in the copied project. but not in the original project. there the line 54 has no effect at all. even if i clean build. what do i miss here? some xCode cache stuff? thanks

from ofxarkit.

Hey @HKBMedialab !

First off - for the zooming
Please file a separate issue next time. I've started one for you here
#23

it just gets too confusing otherwise 😛

Rotation
so just so I have it clear

• orientation is locked to portrait
• you're on an iPhone? You mention iPad too so I'm not sure

If the orientation is locked on your device, at the very least, on an iPhone, the image should look fine no matter how you hold the device the last time I checked.

Theoretically, if you're on an iPad,it should be the same situation, but what happens when you make your change to line 54 and also remove everything in the if statement in the ARShaders::camera_convert_vertex

from ofxarkit.

Hey @sortofsleepy, thanks and sorry, i am relatively new to git and still marveling. will do it next time.
And sorry for the phone confusion. that was a typo. i am on a iPad pro 10.5".
and yes. orientation is locked to portrait on the device. Also in the General Tab of the Project Targets i have only checked the Portrait checkbox.

I am not sure if this is related to this addon or to my XCode settings as its really strange. I try to explain it:

Project A is the example-basic from here.
Project B is a exact copy of this example in the same directory. Just renamed.

While in Project A line 54 hase absolutely no effect, it will affect the rotation in Project B.
To get the right rotation i have to set it to rotation.makeRotationMatrix(-90, ofVec3f(0,0,1));

I know, this sounds strange...

Edit
I overlooked somehow line 115
if([deviceType isEqualToString:@"iPad"]){ rotation.makeRotationMatrix(90, ofVec3f(0,0,1)); }
If I change it there to
rotation.makeRotationMatrix(90, ofVec3f(0,0,1));
too (as after the setup, apparently updateDeviceOrientation() gets called),it is working for both projects. So thats fine. Still its confusing to me that both exact same projects seem to behave different. The updateDeviceOrientation() doesn't get called in the copy of the project...

from ofxarkit.

@HKBMedialab - I'm not able to replicate this for some reason, you're right in that this is really weird.

I have a 6S to test with and it is

• locked to portrait
• example settings for supported orientations - only have portrait checked.

I'm testing using the example-basic project directly in the addon repo and the behavior is what I expect it to be without having to make any changes to anything.

Couple of follow up questions

• just to make sure, you're testing the example with the folder name of example-basic correct?all the examples likely have the same scheme name of example-basic (something I need to change at some point) so I just want to make sure I'm testing the right example.

• Are you saying that in Project A and Project B that when you change line 54 to rotation.makeRotationMatrix(-90, ofVec3f(0,0,1)); that it only works in Project B

• in line 115 are you changing it to

  • rotation.makeRotationMatrix(90, ofVec3f(0,0,1));
    or
  • rotation.makeRotationMatrix(-90, ofVec3f(0,0,1));

from ofxarkit.

ah - never mind something weird just happened .. weird!! I'll look into it

from ofxarkit.

@sortofsleepy thanks and yes, i am working with a (hopefully) unchangedexample-basic directly from the addon repo. if I change both lines to
rotation.makeRotationMatrix(-90, ofVec3f(0,0,1)); then the rotation is fine. which in a way seems consistent as its the same rotation as you add to the iphone. so we may not need the devicespecific if statements there.
to the different behavior:
Project A is firing updateDeviceOrientation() function after setup while Project B doesn't.
So there the -90 rotation got overwritten. but with the change in block 115 everything seems to be fine.

from ofxarkit.

@HKBMedialab - ok I've pushed a fix to the develop branch with your suggestion and while I was unable to consistently reproduce the issue I'm hopeful that with your suggestion + one other change that this'll fix it.

Before when ios 11 was in beta - I found I had to flip the image inversely when trying to apply the scaling correction, hence why it was 90 before and not -90, good to know things can be more consistent now!

from ofxarkit.

uh, embarassing, but i found the difference between A and B:
i didn't called processor->deviceOrientationChanged();
in
void ofApp::deviceOrientationChanged(int newOrientation){ }
in Project B. Sorry for that confusion. So its not so weird after all. Both now behave the same again. Nevertheless, -90 is for me the right rotation. Thanks for fixing!

from ofxarkit.

Closing since, in addition to user testing, self testing appears to show this issue mostly resolved(aside from UpsideDownPortrait which as far as I can tell is likely a project setup issue).

Please feel free to re-open if not.

from ofxarkit.