snap() captures whole page instead of element - can't replicate in Mac/Ubuntu about rpa-python HOT 8 OPEN

@kensoh I did some tests, using the coordinates returned in the tagui log, I used the following code to screenshot and it works fine on both Windows and Centos.

import rpa as r
import json
from websocket import create_connection
import base64
import requests

r.init()
r.url('https://baidu.com')
r.wait(2)

res = requests.get('http://localhost:9222/json').content
# print(json.loads(res.decode('utf-8'))[0]['webSocketDebuggerUrl'])


wsconn = json.loads(res.decode('utf-8'))[0]['webSocketDebuggerUrl']
ws = create_connection(wsconn,suppress_origin=True)

# r.snap('//img[ @id=\"s_lg_img\" ]','1111.png')

# centos
# [12] {"id":12,"result":{"result":{"type":"object","value":{"top":44,"left":526,"width":270,"height":129}}}}

# windows
# [8] {"id":8,"result":{"result":{"type":"object","value":{"top":44,"left":540,"width":270,"height":129}}}}

# do snap
# [9] {"id":9,"method":"Page.captureScreenshot","params":{"format":"png","quality":80,"clip":{"x":548,"y":44,"width":270,"height":129,"scale":1},"fromSurface":true}}

request = {}
request['id'] = 1
request['method'] = 'Page.captureScreenshot'
request['params'] = {"format":"png","quality":80,"clip":{"x":526,"y":44,"width":270,"height":129,"scale":1},"fromSurface":True}
ws.send(json.dumps(request))
result = ws.recv()
png = json.loads(result)['result']['data']
rpng = base64.b64decode(png)
# print(base64.b64decode(rpng))
open('testbaidu.png','wb').write(rpng)
ws.close()

r.wait(10)

How to continue troubleshooting this problem?

from rpa-python.

I tested the site "baidu.com" and reproduced the problem.

Linux system:
r.snap('//img[ @id="s_lg_img" ]','222.png')
the result is the entire web page

Windows system:
r.snap('//img[ @id="s_lg_img" ]','222.png')
it is the correct element

from rpa-python.

Hi @yucongshub thanks for raising this! The backend API only returns what Chrome returns. So my initial hunch is there might be some change or bug in Chrome in Linux that could be the root cause. And this is the first report of such issue.

I've tried replicating on Mac and it is also working fine -

RPA:Desktop kensoh$ python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpa as r
>>> r.init()
True
>>> r.url('www.baidu.com')
True
>>> r.snap('//img[ @id="s_lg_img" ]','222.png')
True

I have a Linux VPS but it is an old CentOS so I can't run Chrome there. Can you share with me the log file after running the snap() step from your Linux machine? I think you can directly attach the file here. It can be found by print(r.tagui_location()) and then go to .tagui/src/tagui_chrome.log

That is the transaction log between the TagUI engine and Chrome browser through websocket connection. So it contains the exact and entire low-level information returned from Chrome.

from rpa-python.

Adding on @yucongshub - I've tried using the Google Colab notebook example (which runs on Ubuntu) and it appears to be working correctly. The notebook example link is the following and found on README.md page of this repo -

https://colab.research.google.com/drive/1or8DtXZP8ZxJYK52me0dA6O9A1dXKKOE?usp=sharing

Using the following lines, I have downloaded the log file and attaching it below for comparison later.

log_file = r.tagui_location() + '/.tagui/src/tagui_chrome.log'
from google.colab import files; files.download(log_file)

tagui_chrome.log

from rpa-python.

Hi @kensoh , I tested again and provided the previous and this time log files. I saw that the difference between the log files you provided is that there are no pixel coordinates in Page.captureScreenshot?

tagui_chrome.log.2024.1.15.log
tagui_chrome.log.2024.1.11.log

from rpa-python.

Hi @yucongshub can you tell me more what do you mean? Can you share more on what is the difference between on your Linux and the example using Google Colab Ubuntu? Is it your Linux doesn't return coordinates and the Google Colab Ubuntu returns?

If on your Linux there is no coordinates returned, then you can't use workaround to capture the screenshot. Unless you use another system to first collect the coordinates and hard code the solution on your Linux system. Otherwise directly on Linux you don't have those coordinates to use the workaround.

What is your Linux and version? I suspect it might be some Linux/Chrome edge case issue that is hard to replicate on other Linux/Chrome. The solution might be to use the one you shared above, collect the coordinates on another machine and use them to do the screen shot on your Linux/Chrome.

from rpa-python.

Hi @kensoh , The Linux operating environment I use is centos7.9. I tested it in Windows, google colab, and centos7.9. Windows and google colab returned the correct element screenshots, but centos7.9 was wrong. Its coordinates and screenshot codes The execution order seems different from windows and google colab. How can I debug the order of obtaining element coordinates and taking screenshots? I want to compare and see how windows and centos7.9 run. I have provided the log files of the three, you can check from the logs.
tagui_chrome.log.google-colab.log
tagui_chrome.log.windows.log
tagui_chrome.log.centos7.log

from rpa-python.