Giter VIP home page Giter VIP logo

rpa-python's Introduction

RPA for Python 🐍

v1.50 • Use Cases • API Reference • About & Credits • Try on Cloud • PyCon Video • Telegram Chat • 中文 • हिन्दी • Español • Français • عربى • বাংলা • Русский • Português • Bahasa • Deutsch • More..

RPA for Python demo in Jupyter notebook

To install this Python package for RPA (robotic process automation) -

pip install rpa

To use it in Jupyter notebook, Python script or interactive shell -

import rpa as r

Notes on operating systems and optional visual automation mode -

  • 🏳️‍🌈 Windows - if visual automation is faulty, try setting your display zoom level to recommended % or 100%
  • 🍎 macOS - due to tighter security, install PHP manually and see solutions for PhantomJS and Java popups
  • 🐧 Linux - visual automation mode requires special setup on Linux, see how to install OpenCV and Tesseract
  • 🍇 Raspberry Pi - use this setup guide to run the package on Raspberry Pies (low-cost automation servers)

Use Cases

RPA for Python's simple and powerful API makes robotic process automation fun! You can use it to quickly automate away repetitive time-consuming tasks on websites, desktop applications, or the command line.

As a bonus and token of my appreciation, any new bug reported will be appreciated with a US$200 gift card from your preferred merchant. Any feature suggestion accepted will be appreciated with a US$100 gift card.

WEB AUTOMATION

r.init()
r.url('https://duckduckgo.com')
r.type('//*[@name="q"]', 'decentralisation[enter]')
r.wait() # ensure results are fully loaded
r.snap('page', 'results.png')
r.close()

VISUAL AUTOMATION

r.init(visual_automation = True)
r.dclick('outlook_icon.png')
r.click('new_mail.png')
...
r.type('message_box.png', 'Hi Gillian,[enter]This is ...')
r.click('send_button.png')
r.close()

OCR AUTOMATION

r.init(visual_automation = True, chrome_browser = False)
print(r.read('pdf_report_window.png'))
print(r.read('image_preview.png'))
r.hover('anchor_element.png')
print(r.read(r.mouse_x(), r.mouse_y(), r.mouse_x() + 400, r.mouse_y() + 200))
r.close()

KEYBOARD AUTOMATION

r.init(visual_automation = True, chrome_browser = False)
r.keyboard('[cmd][space]')
r.keyboard('safari[enter]')
r.keyboard('[cmd]t')
r.keyboard('snatcher[enter]')
r.wait(2.5)
r.snap('page.png', 'results.png')
r.close()

MOUSE AUTOMATION

r.init(visual_automation = True)
r.type(600, 300, 'neo kobe city')
r.click(900, 300)
r.snap('page.png', 'results.png')
r.hover('button_to_drag.png')
r.mouse('down')
r.hover(r.mouse_x() + 300, r.mouse_y())
r.mouse('up')
r.close()

TELEGRAM NOTIFICATION

first, look up @rpapybot on your Telegram app to approve receiving messages

r.telegram('1234567890', 'ID can be string or number, r.init() is not required')
r.telegram(1234567890, 'Hello World. Olá Mundo. नमस्ते दुनिया. 안녕하세요 세계. 世界,你好。')
r.telegram(1234567890, 'Use backslash n for new line\nThis is line 2 of the message')

SECURE TEMPORARY STORAGE

securely share files up to 100 MB on PrivateBin, which will self-destruct after 1 week

bin_url = r.bin('secret_agent_report.pdf', 'optional password')
r.telegram(1234567890, 'Access confidential report at ' + bin_url)

API Reference

Notes • Element Identifiers • Core Functions • Basic Functions • Pro Functions • Helper Functions


GENERAL NOTES

See sample Python script, the RPA Challenge solution, and RedMart groceries example. To send a Telegram app notification, simply look up @rpapybot to allow receiving messages. To automate Chrome browser invisibly, use headless mode. To run 10X faster instead of normal human speed, use turbo mode (read the caveats!). Some CAPTCHAs can be solved using services like 2Captcha or directly by replicating user actions.

Securely share files up to 100 MB with built-in temporary online storage, on a dedicated PrivateBin server. You can even run RPA on your phone browser using this Colab notebook (eg datascraping with up to 5 Colab sessions). By design this package has enterprise security and you can install, update and use it without the internet.

Fully control error handling by setting error(True) to raise Python exception on error, and manage with try-except. For fine-grained control on web browser file download location, use download_location(). For overriding default folder location to install and invoke TagUI (a forked version optimised for rpa package), use tagui_location().

If you are using non-English operating system and get "invalid continuation byte" error, you can set code page to support UTF-8 or change your Python script's encoding to your OS encoding. See this example for Chinese. Use focus() to make Windows/Mac application windows to be in focus (see here for pywin32 alternative).

ELEMENT IDENTIFIERS

An element identifier helps to tell RPA for Python exactly which element on the user interface you want to interact with. For example, //*[@id='email'] is an XPath pointing to the webpage element having the id attribute 'email'.

  • 🌐 For web automation, the web element identifier can be XPath selector, CSS selector, or the following attributes - id, name, class, title, aria-label, text(), href, in decreasing order of priority. Recommend writing XPath manually or simply using attributes. There is automatic waiting for an element to appear before timeout happens, and error is returned that the element cannot be found. To change the default timeout of 10 seconds, use timeout(). PS - if you are using a Chrome extension to read XPaths, use SelectorsHub.

  • 📸 An element identifier can also be a .png or .bmp image snapshot representing the UI element (can be on desktop applications, terminal window or web browser). If the image file specified does not exist, OCR will be used to search for that text on the screen to act on the UI element containing the text, eg r.click('Submit Form.png'). Transparency (0% opacity) is supported in .png images. x, y coordinates of elements on the screen can be used as well. Notes for visually automating 2 monitors, and macOS Retina display issue.

  • 📄 A further image identifier example is a png image of a window (PDF viewer, MS Word, textbox etc) with the center content of the image set as transparent. This allows using read() and snap() to perform OCR and save snapshots of application windows, containers, frames, textboxes with varying content. See this image example of a PDF frame with content removed to be transparent. For read() and snap(), x1, y1, x2, y2 coordinates pair can be used to define the region of interest on the screen to perform OCR or capture snapshot.

CORE FUNCTIONS

Function Parameters Purpose
init() visual_automation=False,chrome_browser=True start TagUI, auto-setup on first run
close() close TagUI, Chrome browser, SikuliX
pack() for deploying package without internet
update() for updating package without internet
error() True or False set to True to raise exception on error
debug() True or False or text_to_log print & log debug info to rpa_python.log

by default RPA for Python runs at normal human speed, to run 10X faster use init(turbo_mode = True)

BASIC FUNCTIONS

Function Parameters Purpose
url() webpage_url (no parameter to return current URL) go to web URL
click() element_identifier (or x, y using visual automation) left-click on element
rclick() element_identifier (or x, y using visual automation) right-click on element
dclick() element_identifier (or x, y using visual automation) double-click on element
hover() element_identifier (or x, y using visual automation) move mouse to element
type() element_identifier (or x, y), text ('[enter]'/'[clear]') enter text at element
select() element_identifier (or x, y), value or text (or x, y) choose dropdown option
read() element_identifier ('page' is web page) (or x1, y1, x2, y2) return element text
snap() element_identifier ('page' is web page), filename_to_save save screenshot to file
load() filename_to_load return file content
dump() text_to_dump, filename_to_save save text to file
write() text_to_write, filename_to_save append text to file
ask() text_to_prompt ask & return user input

to wait for an element to appear until timeout() value, use hover(). to drag-and-drop, do it this way

PRO FUNCTIONS

Function Parameters Purpose
telegram() telegram_id, text_to_send (first look up @rpapybot) send Telegram message
keyboard() keys_and_modifiers (using visual automation) send keystrokes to screen
mouse() 'down' or 'up' (using visual automation) send mouse event to screen
focus() app_to_focus (full name of app) make application in focus
wait() delay_in_seconds (default 5 seconds) explicitly wait for some time
table() table number or XPath, filename_to_save save webpage table to CSV
bin() file_to_bin, password (optional but recommended) secure temporary storage
upload() element_identifier (CSS), filename_to_upload upload file to web element
download() download_url, filename_to_save (optional) download from URL to file
unzip() file_to_unzip, unzip_location (optional) unzip zip file to specified location
frame() main_frame id or name, sub_frame (optional) set web frame, frame() to reset
popup() string_in_url (no parameter to reset to main page, especially important when used to control another browser tab) set context to web popup tab
run() command_to_run (use ; between commands) run OS command & return output
dom() statement_to_run (JS code to run in browser) run code in DOM & return output
vision() command_to_run (Python code for SikuliX) run custom SikuliX commands
timeout() timeout_in_seconds (blank returns current timeout) change wait timeout (default 10s)

keyboard() modifiers and special keys -

[shift] [ctrl] [alt] [win] [cmd] [clear] [space] [enter] [backspace] [tab] [esc] [up] [down] [left] [right] [pageup] [pagedown] [delete] [home] [end] [insert] [f1] .. [f15] [printscreen] [scrolllock] [pause] [capslock] [numlock]

HELPER FUNCTIONS

Function Parameters Purpose
exist() element_identifier True or False if element shows before timeout
present() element_identifier return True or False if element is present now
count() element_identifier return number of web elements as integer
clipboard() text_to_put or no parameter put text or return clipboard text as string
get_text() source_text,left,right,count=1 return text between left & right markers
del_chars() source_text,characters return text after deleting given characters
mouse_xy() return '(x,y)' coordinates of mouse as string
mouse_x() return x coordinate of mouse as integer
mouse_y() return y coordinate of mouse as integer
title() return page title of current web page as string
text() return text content of current web page as string
timer() return time elapsed in sec between calls as float

to type a large amount of text quickly, use clipboard() and keyboard() to paste instead of type()

About & Credits

TagUI is a leading open-source RPA software 🤖 with tens of thousands of users. It was created in 2016-2017 when I left DBS Bank as a test automation engineer, for a one-year sabbatical to Eastern Europe. Most of its code base was written in Novi Sad Serbia. In 2018, I joined AI Singapore to continue development of TagUI.

Over a few months in 2019, I took on a daddy role full-time, taking care of my newborn baby girl and wife 🤠🤱. In between nannying, I used my time pockets to create this Python package built on TagUI. I hope pip install rpa would make life easier for Python users from different walks of life.

I had been maintaining the package (and a forked version of TagUI optimised for it) in my personal time. But Marcelo Cecin, Luis Alejandro, Jozsef Fulop, Tolani Jaiye-Tikolo, Shyan Chua, Laurence Liew, myself, will be the new team maintaining this package. We're happy that tens of thousands of people use it 🐍

For technical info, see its intuitive architecture below and ample comments in this single-file package.

RPA for Python architecture

I would like to credit and express my appreciation to these amazing open-source contributors below ❤️

Philip's LinkedIn Post

License

RPA for Python is open-source software released under Apache 2.0 license

One Last Thing.. Mindly

I rarely make product recommendations, other than the amazing OpenRPA software, and the open-source RPA tools I personally worked on. I'd like to recommend Mindly mindmapping app available on phone and macOS.

A mindmap is an intuitive way to store, organise and retrieve info, as it mimics how the mind works - relationships between different concepts and memories. It's perfect to make productive use of time pockets on the go.

Below image is a Mindly example on benefits of coffee. I personally use it to map out my life for the next 13 years, reflect how to be a better husband, keep a list of traditional British foods, store supermarket member barcodes, as well as note-taking on the go. There's even a mindmap for my 3YO daughter to play with, she just enjoys dragging the nodes into the bin. So I created a dummy mindmap on standby that she can destroy.

Best of all, the free version should meet the needs of most users. I have not exceeded the free limit of 100-node per mindmap, but I purchased it quite early on after using it, to support the work of the team behind this app.

PS - I don't know Mindly's team, just recommending the app here because it rocks

Mindly Mindmapping App

rpa-python's People

Contributors

fi-do avatar kensoh avatar lookang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rpa-python's Issues

ask() to show question in Chrome browser if available [done]

ask() shows the question in terminal window. This is not intuitive if automation is running on the Chrome browser and a prompt appears in terminal window to ask for user input. Raising an issue to update behaviour to show prompt in Chrome browser, if it's available.

PS - similar issue exists if chrome_browser = False and visual_automation = True. The terminal may not be in the foreground during the automation flow to be visible for user inputs. However, there isn't a consistent way to show SikuliX popup with focus across platforms. Thus KIV for now.

TagUI Chrome web browser extension usable for TagUI for Python?

user query to original TagUI project


I am currently working on evaluating open source RPA tools, specifically TagUI. I have a few queries regarding the TagUI Chrome Web Browser Extension. They are as follows:

  • I learned that the exported data, i.e. that of the image above, is for TagUI. I was wondering if there is a possibility that this data extracted can be in Python language directly, so that the action can be tested and recoded in Python.
  • Hence, I was wondering if you have any plans on developing the Chrome Web Browser Extension, to include support for the TagUI-Python API.

Thank you and have a great week ahead.

Cleaner usage and execution without TagUI intermediary files [done]

During usage, the following temporary files will be generated in order to use TagUI -

tagui_python - simple flow to initiate TagUI live mode
tagui_python.raw - raw file for TagUI module expansion
tagui_python.js - generated JavaScript code for TagUI
tagui_local.js - TagUI for Python custom functions file
tagui_python.log - log file for automation execution
tagui_python.txt - to retrieve data output from TagUI

When debug mode is on using debug(True), only tagui_python.log and tagui_python.txt will be retained while the rest are deleted. That essentially means that the first 4 files above do not really add value to the typical TagUI for Python user. Having them in the script folder during execution may add clutter, clunkiness and confusion.

Furthermore, TagUI for Python has been downloaded over 5000 times since initial release, without users reporting issue that requires above 4 temporary files for debugging and troubleshooting.

Raising issue to update code to immediately delete the first 4 files upon connecting to TagUI to remove clutter. For the last 2 files, they will be retained until close() is used, where they will be deleted if debug(False) (default).

Use in air-gap environments with network restriction and no internet [done]

Now I understand that from tagui.py that it is downloading from

def setup():
    """function to setup TagUI to user home folder on Linux / macOS / Windows"""

    # get user home folder location to setup tagui
    if platform.system() == 'Windows':
        home_directory = os.environ['APPDATA']

Would it be good to change the code such that it does a check on whether the tagui folder exists and uses the existing folder rather than download a new one?

I ask this because I'm working in an environment where remote host downloading is not allowed, (yay web isolation, no curl either).
So by inspecting the code, I know that this folder is required and I've downloaded manually from https://github.com/tebelorg/Tump/releases/download/v1.0.0/TagUI_Windows.zip

Fatal: Access is denied; did you install phantomjs? - firewall / virus scanner

Tried another sample code on windows 10:

import tagui as t
t.init()
t.url('https://www.google.com')
t.type('//input[@class="gLFyf gsfi"]', 'weather singapore[enter]')
temp = t.read('wob_tm')
print("temp = " + temp)
t.close()

Error message

[TAGUI][ERROR] - following happens when starting TagUI...

Fatal: Access is denied; did you install phantomjs?

[TAGUI][ERROR] - use init() before using url()
[TAGUI][ERROR] - use init() before using type()
[TAGUI][ERROR] - use init() before using read()
temp =
[TAGUI][ERROR] - use init() before using close()

Should have phantomjs installed. I'm running TagUI on this machine.

Checklist for TagUI Python package [done]

  • python package project skeleton
  • invoking, controlling, closing tagui
  • compatibility with python 2 and 3
  • conversion to tagui python module
  • address known and potential issues
    • send single quote in text string input
    • echo with correct quotes in tagui log
    • parameter to invoke visual automation
    • accept (x,y) coordinates as parameters
    • validate sending of backslash correctly
    • validate path for filename parameters
    • auto sikulix jython 1st-run integration
  • pre-release iteration, testing and docs
    • review, iterate, refine features list
    • auto download + setup + test tagui
    • test usage with python 2 and 3 envs
    • test usage python interactive mode
    • test usage with mac, windows, linux
    • test usage with jupyter notebook
    • test, validate, tune all functions
    • github and pypi documentation
  • announce and start distribution

Can I use OCR to extract information from image files? - can but not ideal

user query


I have a question regarding OCR automation. I tried reading from a jpg image but it is unable to find the image. However, the code works if I read a png file.

image003

Appreciate if you could help with this as I am trying to automate a laborious process that involves extracting numbers from hundreds of images that are saved in jpg format.

[TAGUI][ERROR] - failed downloading from.. macOS + Python 3 issue [fixed]

I was able to install tagui using pip but is unable to start it. t.init() gives

[TAGUI][INFO] - setting up TagUI for use in your Python environment
[TAGUI][INFO] - downloading TagUI (~200MB) and unzipping to below folder...
[TAGUI][INFO] - /Users/my_id
[TAGUI][ERROR] - failed downloading from https://github.com/tebelorg/Tump/releases/download/v1.0.0/TagUI_macOS.zip
False

I am sure I am connected to the internet and I was able to manually download the file (TagUI_macOS.zip). Unzipped to the location but getting same error. Any workarounds?

Difference between TagUI vs TagUI for Python? - different target users

@kensoh, really excited by your new TagUI-Python as it opens up a whole new spectrum of possibilities in RPA applications, especially interfacing with those established machine learning and deep learning Python frameworks.

Not sure if the following have been documented somewhere,

  1. I looked at the command sets between the two.
    TagUI: https://github.com/kelaberetiv/TagUI#steps-description
    TagUI-Python: https://github.com/tebelorg/TagUI-Python#api-reference
    It seems that they are about the same. So is it correct to say that almost all the commands in TagUI are supported in TagUI-Python?

  2. Can you highlight some major differences between the two (besides one written in TagUI script, and the other in Python)?

Error using run() on Windows - for eg run("cmd /c dir") [fixed]

Hi Ken,

Tried to run a DOS command using your suggestion in: aisingapore/TagUI#473

The following works in TagUI:

run cmd /c dir
echo run_result

In TagUI-Python on windows 10:

run_result = t.run("cmd /c dir")
print("run_result)

The above will give the error message:

Traceback (most recent call last):
  File "test1.py", line 2, in <module>
    run_result = t.run("cmd /c dir")
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\site-packages\tagui.py", line 1173, in run
    shell=True))
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "C:\Users\user\AppData\Local\Programs\Python\Python37-32\lib\subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'cmd /c dir; exit 0' returned non-zero exit status 1.

MSVCR110.dll not found on Windows - see this issue for the solution to fix

I have tried to run a simple script to navigate into a website in python. Aparently tagUi is not able to initalize due to a missing MSVCR100.dll dependency.

I have tried to reinstall the package with this link: https://www.microsoft.com/en-us/download/confirmation.aspx?id=14632

After reinstallation and reboting my machine, the result is the same. I am using python with Anaconda and Windows 10, probably is because of this?

Thanks!
Captura

TagUI-Python vs TagUI - summary of differences and roadmap

Background

Have spent the last few weeks trying to convert one complicated script I’ve written using TagUI into TagUI-Python to print 3 different reports across two corporate websites. The printing of the report is not that straightforward as it involves some lookup of values before you can print the report. To minimize the duplication of codes, I stored those commonly used functions in separate files and call these functions using the tagui command. As a result, the final app contains more than 50 files!

Result

Pleased to say that I have successfully ported the entire TagUI script into TagUI-Python! The app runs perfectly without any glitch! As I can now combine multiple functions into Python modules, the number of files also drastically reduced form 50+ files to only 3 main files + 4 Python modules.

The only commands that I’ve found to be different between TagUI and TagUI-Python are the two I’ve highlighted earlier:

Other than the above, as TagUI-Python runs on top of TagUI, there’s almost no changes required. All that’s required is the translation of the syntax e.g.

  • read
    TagUI: read //path/to/selector to var_name
    TagUI-Python: var_name = t.read('//path/to/selector')
  • type
    TagUI: type //path/to/selector as string_to_type
    TagUI-Python: t.type('//path/to/selector', 'string_to_type')

Sample code to highlight major advantages of TagUI for Python

Thanks to @kensoh in highlighting the major differences between TagUI and TagUI-Python in Issue #8.

In the reply, Ken mentioned that "TagUI for Python only has a subset of TagUI's features." I have been wondering why then do people want to switch to TagUI-Python when TagUI has more features, and TagUI-Python actually runs on top of TagUI (which means that there's an additional layer of TagUI between TagUI-Python and the website).

Decided to convert some of my TagUI scripts into TagUI-Python so that I can have some real comparisons between the two.

After one week of testing TagUI-Python, I must say that I was truly surprised and amazed by TagUI-Python!

First, TagUI-Python can do almost everything that TagUI can do. Conversion is very simple and straightforward - just a change of the syntax. The flow and xpath remains exactly the same, as can be seen in the following sample code.

Second, TagUI-Python is written as a native python package and run as a pure Python script. This means that you can leverage any standard Python packages out there, such as pandas used in the sample code below to save the data as a csv file.

Third, using TagUI-Python, you can now leverage on Python's native data structure (such as lists and classes), and also modules/packages to organize libraries of functions.

So below is a sample code that reads the title and url of a list of books from goodreads.com (total 18 books in that page).

Sample Code 1 - in TagUI

//goodreads.gui
//190706

https://www.goodreads.com/list/show/17347.Books_that_ll_help_you_deliver_stunning_presentations
 
read //h1[contains(@class, "h1")] to list_title
echo "list_title = " + list_title
 
num_books = count('//td/a//span[@itemprop="name"]')
echo "num_books = " + num_books

csv_header = '"Sno", "Title", "URL"\r\n'
dump csv_header to book_list.csv
 
for (n=1; n<=num_books; n++)
{
	read (//td/a//span[@itemprop="name"])['+n+'] to book
	echo "book = " + book
 
	read (//td/a/@href)['+n+'] to url2
	echo "url2 = " + url2
 
	record = '"' + n + '","' + book + '","' + url2 + '"' 
    write record to book_list.csv
}

Sample Code 2 - in TagUI-Python

In comparison, the following is the same robot re-written using TagUI-Python

from pandas import DataFrame
import tagui as t
t.init()

t.url('https://www.goodreads.com/list/show/17347.Books_that_ll_help_you_deliver_stunning_presentations')

list_title = t.read('//h1[contains(@class, "h1")]')
print("list_title = " + list_title)

num_books = t.count('//td/a//span[@itemprop="name"]')
print("num_books = ", num_books)

books = [''] * num_books
url = [''] * num_books
sno = [''] * num_books
for n in range(1, num_books+1):
	sno[n-1] = n

	books[n-1] = t.read(f'(//td/a//span[@itemprop="name"])[{n}]/..')
	print(n, "book = "+books[n-1])

	url[n-1] = t.read(f'(//td/a/@href)[{n}]')
	print("   url = "+url[n-1])

Books_Info = {'Sno':sno, 'Title': books, 'URL': url}
df = DataFrame(Books_Info, columns=['Sno','Title', 'URL'])
export_csv = df.to_csv ('book_list1.csv', index = None)

Note the use of pandas to save data to .csv file. With TagUI-Python, you can now leverage on all the standard python packages out there, including those machine learning libraries including Numpy, Scikit-learn, TensorFlow, Keras, etc.! This will be one of the major reason for moving to TagUI-Python if you're working on machine learning or intelligent RPA project.

Visual automation attach to application window on Windows

Hi all,
i am using visual automation from tagui to scrap data from an mobile app via bluestack

currently, my solution is to use multiple windows VM on my computer.
each running an instance of tagui and bluestack.

does anyone know if it is possible to let a tagui instance to only focus on the screen of a certain bluestack instance via process id?

Setting up TagUI-Python on Windows 10 - slow downloading see #15

@kensoh, trying to set up TagUI-Python on my Windows 10 machine.

  1. Have run "pip install tagui"

  2. Running "pip list" show

Package    Version
---------- -------
tagui      1.1.0
  1. Test script as follows:
import tagui as t
t.init()
t.echo("test123")
t.close()
  1. Running the above script from command prompt, it shows the following and stops there:
[TAGUI][INFO] - setting up TagUI for use in your Python environment
[TAGUI][INFO] - downloading TagUI (~200MB) and unzipping to below folder...
[TAGUI][INFO] - C:\Users\username\AppData\Roaming
  1. If I press Ctrl-C, the following messages appear:
[TAGUI][INFO] - setting up TagUI for use in your Python environment
[TAGUI][INFO] - downloading TagUI (~200MB) and unzipping to below folder...
[TAGUI][INFO] - C:\Users\username\AppData\Roaming
Traceback (most recent call last):
  File "test2.py", line 2, in <module>
    t.init()
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\site-packages\tagui.py", line 359, in init
    if not setup():
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\site-packages\tagui.py", line 208, in setup
    if not download(tagui_zip_url, home_directory + '/' + tagui_zip_file):
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\site-packages\tagui.py", line 1092, in download
    import urllib.request; urllib.request.urlretrieve(download_url, filename_to_save)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 276, in urlretrieve
    block = fp.read(bs)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 447, in read
    n = self.readinto(b)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\http\client.py", line 491, in readinto
    n = self.fp.readinto(b)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1052, in recv_into
    return self.read(nbytes, buffer)
  File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 911, in read
    return self._sslobj.read(len, buffer)
KeyboardInterrupt
  1. Are there any other settings I need to do to get TagUI-Python running on Windows 10? Is there anything that I've missed?

Facing issues while downloading TagUI installation - network issue

Dear Ken Soh,

From 1st July I am facing issue with the python script. Whenever I'm running the python script it starts downloading the update file, but fails after some time which leads to error in the execution.
Even I downloaded the file from the link and placed it in the desired folder but even though the issue is not solved. I am getting the following error.

error

snap() not working? - captures whole page instead of web element [fixed]

Hi! I was hoping to use TagUI to automate generating of a .png image, as part of a small personal project. This is done through a website:

https://www.myfakeinfo.com/nationalidno/get-china-citizenidandphoto.

As I attempted to take a snapshot of the image generated, using t.snap('log1','front.png'), where 'log1' is the element id, ''front.png' is the intended saved image. However, this action takes a screenshot of the whole webpage, instead of just saving the image.

I was just wondering if this is your intended output, and could there be anyway I can use TagUI API to carry out saving image?

Thanks!

timeout() hangs if visual automation mode is not used [fixed]

There is a bug in the timeout step in upstream TagUI project (aisingapore/TagUI#497). When timeout() is used with visual_automation = False, it hangs indefinitely. Raising a PR for a new release that implements the bug-fix upstream. Below are technical details of the bug -


In commit aisingapore/TagUI@5cba169 from issue aisingapore/TagUI#443 and PR aisingapore/TagUI#453 the goal is to also update SikuliX integration internal timeout variable. There is a bug introduced that makes timeout step hangs if visual automation is not used.

TagUI for Python OCR Findings, Tips and References

Hi @kensoh, while doing research and testing on OCR, I found that TagUI-Python is extremely functional in this aspect. It is able to carry out OCR on webpage images (t.url -> t.read) and local images as well (using visual_automation = True, chrome_browser = False -> reading using coordinates).

I also researched and learned that there is a TagUI RPA Studio, her OCR is more user friendly as users do not need to use t.mouse_xy() to find the coordinates. Also Kantu's OCR is also able to detect images and do search and verification, which could possibly be added into TagUI-Python in the future.

How about the reverse (generate script by recording actions)?

This is not an issue per se but more of a question: Is there a way to automatically generate the commands for a series of actions as a script, out of the actions itself when performed once.

For example, a script records the actions and one can simply perform the actions once, then the corresponding commands are created.

Different behaviour in TagUI write and TagUI for Python write() - by design

Hi Ken,

Just tried some write() in TagUI-Python

Found out that

  • write in TagUI automatically includes new line at the end of the write string
  • write() in TagUI-Python does not

Can I confirm the above behaviour?

Would be great if the two can be standardized - so that it's easier for people who need to port scripts from TagUI to TagUI-Python.

Running TagUI-Python for Windows / desktop apps without Chrome

In #25, you commented that:

There are also major advantages of TagUI for Python over a number of Python packages for
digital automation. Some of them can only automate webapps, some can only automate
keyboard + mouse. But this tool can do all, plus ability to automate using computer vision and
extract info using OCR, all within one seamless API

Currently in TagUI-Python, when one issues t.init(), it will open a new empty chrome window.

For windows/desktop apps, or if the user only want to automate keyboard + mouse, the empty chrome window will "confuse" the user, or even block the windows/desktop app required for image-based recognition.

Possible to have the same behaviour as that of TagUI i.e. a new chrome window will only appear at the first initiation of http://

visible() helper function not in TagUI for Python - streamline by design

Hi Ken,

Just found out that visible() is not present in TagUI-Python!

Is there any particular reason why this is not ported over from TagUI to TagUI-Python?

I'm using present() in the meantime, though it could happen that an element is present but not visible. So being able to test if an element is visible is quite useful sometimes.

Auto-flush TagUI processes, eg Ctrl+C is used or forget to close() [done]

It is possible that TagUI processes (main TagUI, integration background processes - SikuliX, Chrome, Chrome browser) are not exited cleanly under certain conditions. For eg, if user Ctrl+C to kill a TagUI for Python script, or if user forgets to use close() to gracefully close off the running processes.

These dead processes have insignificant impact to TagUI for Python functionality. However they can cause unnecessary drain on battery and CPU cycles. Raising this issue to make a commit that runs a TagUI script that flush all TagUI-related processes.

This will be run within init() so that an accidental Ctrl+C or forgetting to close() allows immediate resumption of use by relaunching Jupyter notebook or Python interactive shell or Python script, and using init(). Placing within close() is meaningless for the scenarios where dead processes happen.

Support OCR capability and regions of interests for read() and snap() [done]

Raising an issue to officially support OCR use case in TagUI for Python.

Transparency (0% opacity) is supported by SikuliX in .png images, for eg using an image of an UI element with transparent background to enable clicking on an UI element that appears on different backgrounds on different occasions.

A further example is an image of the window or frame (PDF viewer, MS Word, textbox etc) with the center content of the image set as transparent. This allows using read() and snap() to perform OCR and save snapshots for application windows, containers, frames, textboxes with varying content.

t.read('image_preview_frame.png')
t.snap('application_window_frame.png')

Also, for read() and snap(), x1, y1, x2, y2 coordinates pair can be used to define the region of interest on the screen to perform OCR or capture snapshot. Upstream feature has been implemented in TagUI and can now also be implemented here. For example -

t.read(200, 200, 600, 400)
t.snap(200, 200, 600, 400, 'results.png')

Minor suggestion - auto-wait & timeout for UI element [already done]

This is pretty sweet stuff but I think a little more sugar can't hurt. I have found the following to be useful because such automations can be quite brittle:

def t_wait_for(item, dt = 0.1):
	while not t.present(item):
		t.wait(0.1)

Then again, it strikes me that many of the actions could benefit from having a wait keyword argument to indicate whether to have this done before the relevant action.

Furthermore, time outs can be included as yet another keyword argument. (There already is an implicit time out of 0, why not customise it?)

What do you think?

run('echo something') on Windows shows exit 0 appended

Hi Ken,

In windows 10, the following works:

run_result = t.run("echo %SYSTEMROOT%")
print(run_result)

Result is: C:\WINDOWS; exit 0

While it's not difficult in python to use some function to remove that "; exit 0" at the end, thought it might be more convenient for users if TagUI-Python does not return that in the result string.

How to get current date and write it into calendar section of a web page

Hello Ken Soh,
I am working on a TAGUI project where whenever the script for web automation runs the current date should be automatically written in the calendar on the webpage. I am attaching a picture of the calendar followed by this post. Please suggest to me how to automate the calendar.

calendar

Automation example - RedMart rescheduling or repeating groceries order

Whenever there is a need to reschedule or repeat our groceries delivery on RedMart.com, there is a need to manually add back all the previous items one by one to create a new order. There is no option to repeat an order (unbelievable).

My wife is very frustrated with that, because in between clicks there is so much waiting time, and the website cannot be ctrl-clicked to open new tabs to do this efficiently. So I wrote below automation script for her. Instead of wasting time doing mindless work every week, now she can double-click this Python script and automate the process herself.

PS - we have since left Singapore and no longer use RedMart, thus below code would likely not work with updates to the website. But it should give you an idea of what's possible with rpa package and how you can approach a scenario.

redmart_order

import rpa as r
r.init()

# get URL of old groceries order to re-add all items to cart
r.url(r.ask('Paste the URL of your groceries order and login to RedMart'))

# set a maximum 5-minute timeout for user to login to RedMart
r.timeout(300)

# use exist() function with XPath to check if logged in
if not r.exist('(//*[@class="order-item"])[1]//*[@class="item-pic"]'):
    r.dom('alert("Quitting as groceries order page is not detected after 5 min")')

# change back to default of 10 seconds to quit fast on error
r.timeout(10)

# get count of total items to place order for, using XPath identifier
total_items = r.count('//*[@class="order-item"]//*[@class="item-pic"]')

# to track total quantity of items to order
total_quantity = 0

# loop to add all the items and their quantity
for item in range(1, total_items + 1):
    # get count of quantity to order for item, before clicking into item
    item_quantity = int(r.read('(//*[@class="order-item"])['+ str(item) + ']//*[@class="item-quantity"]//*[@class="text"]'))
    r.click('(//*[@class="order-item"])[' + str(item) + ']//*[@class="item-pic"]')

    # first click ADD TO CART, then click + icon for subsequent quantity
    if r.exist('ADD TO CART') and not r.present('Out of stock'):

        # handle case where item is in cart, thus no ADD TO CART button
        if r.present('next-icon-add'):
            r.click('next-icon-add')
        else:
            r.click('ADD TO CART')

        for additional_quantity in range(item_quantity - 1):
            r.click('next-icon-add')

        # wait to ensure adding of item has been registered before going back
        r.wait(3)
        total_quantity = total_quantity + item_quantity

    # go back to the previous page with list of items
    r.dom('history.back()')

    # optional wait to slow down the pace of iteration in the automation
    r.wait(3)

# show popup with summary of the quantity of items added before exiting
r.dom('alert("A total quantity of ' + str(total_quantity) + ' items has been added to cart")')
r.close()

Using dom() with variables from Python environment

Hi Ken,

In TagUI sample folder '9_misc', you have the following sample code:

dom_json = '#header p'
dom return document.querySelector(dom_json).innerText
echo 'dom_result using 1 dom_json variable - ' dom_result

May I know what is the corresponding code in TagUI-Python?

How do we pass in that dom_json?

I suppose you will return dom_result, so we do not need this.

[TAGUI][ERROR] - missing image file, OCR and echo() - pending replication

user query


I was trying out the OCR automation using TagUI for Python. Ran through the use cases examples, but got an error as seen below:

unnamed

This is my code:

import tagui as t
t.init(visual_automation = True)
t.echo(t.read('‪C:/Users/USER/testocr.png'))
t.close()

I have tried using an online .png URL to test out this script, instead of a local .png file. Both does not seem to work.

I would also like to ask what does the echo action do. It says on Github that the purpose of echo is to print text to output, how come the output location is not specified in the script?

Streamlining API - remove show() and save(), remove visible() [done]

TagUI was initially designed for non-developer users. Thus it has provisions that make getting thing done as easy as possible for non-developer users. Steps like show and save are nothing more than read followed by printing out to screen or saving to text file.

Porting these steps over adds bloat to the Python API. Thus raising an issue to track and remove the redundant show() and save(). In place of t.show(), print(t.read()) or t.echo(t.read()) can be used. In place of t.save(), t.dump(t.read()) can be used.

Also, TagUI has a helper function visible() which checks webpage base on CSS rules whether a web element is visible on the page. An element can be present in the backend DOM but hidden from rendering. This helper function would be overly confusing and only adds a small benefit, since present() can be used and most users do not care of the fine line between being present in DOM but not visible. Also, for visual automation, present() and visible() means the same thing. Thus removing visible() improves clarity to standardise usage to present().

LIVE mode in TagUI-Python for 2FA - live mode NA in Python, use ask()

@kensoh, in the documentation it says that:

TagUI-Python uses TagUI in LIVE mode to send instructions from Python to it

Does this mean that there is now no more LIVE mode in TagUI-Python?

In TagUI, the LIVE mode is one of the best features that makes TagUI outshine many other RPA tools, even some of the leading ones. Not only does it allow you to do interactive/live testing and debugging, it also makes it possible to automate those websites that require 2FA (e.g. all the banks in Singapore and also many corporate websites).

I use the LIVE mode to pause the script so that people can enter username/password followed by 2FA before continuing.

How to achieve this in TagUI-Python (i.e. pausing the script so that people can perform 2FA)?

frame() and popup() - explore technical feasibility and hacks [done]

Hi Ken,

Have been using frame() and popup() in TagUI on some websites that use iFrame and popup windows. They worked amazingly well! And these 2 commands are another unique TagUI features that are not available in many other RPA tools.

Was wondering if these two commands have been ported to TagUI-Python?

Files missing error - change location to store extracted TagUI files [fixed]

The default unzip location is tagui subfolder under user's temporary folder.

However, after some days, operating system may delete files partially within the tagui folder, making it inaccurate to assume that if tagui executable file exists, the installation is working.

Raising issue to iterate on more persistent locations to place the unzip tagui folder. For eg ~/.tagui on Linux and macOS, %APPDATA%\tagui on Windows.

Running time doubled when in function - functions were called twice

@kensoh, while testing python functions, modules, classes, etc. as highlighted in Issue #25, there's one strange behaviour I noticed.

For the 2 scripts below that achieves the same thing:
Sample Code 1 - takes 15.86 seconds
Sample Code 2 - takes 36.92 seconds (almost doubled!)

Can you think of any reasons why this is so?

Sample Code 1 - A simple script in sequential order

from pandas import DataFrame
import tagui as t
t.init()

t.url('https://www.goodreads.com/list/show/17347.Books_that_ll_help_you_deliver_stunning_presentations')

list_title = t.read('//h1[contains(@class, "h1")]')
print("list_title = " + list_title)

num_books = t.count('//td/a//span[@itemprop="name"]')
print("num_books = ", num_books)

books = [''] * num_books
url = [''] * num_books
sno = [''] * num_books
for n in range(1, num_books+1):
	sno[n-1] = n

	books[n-1] = t.read(f'(//td/a//span[@itemprop="name"])[{n}]/..')
	print(n, "book = "+books[n-1])

	url[n-1] = t.read(f'(//td/a/@href)[{n}]')
	print("   url = "+url[n-1])

Books_Info = {'Sno':sno, 'Title': books, 'URL': url}
df = DataFrame(Books_Info, columns=['Sno','Title', 'URL'])
export_csv = df.to_csv ('book_list1.csv', index = None)

Sample Code 2 - using functions

from pandas import DataFrame
import tagui as t
t.init()

def read_list_title():
	list_title = t.read('//h1[contains(@class, "h1")]')
	print("list_title = " + list_title)

def read_element(selector):
	t.hover(selector)
	val = t.read(selector)
	return val

def read_book_list():
	num_books = t.count('//td/a//span[@itemprop="name"]')
	print("num_books = ", num_books)
	sno = [''] * num_books
	books = [''] * num_books
	url = [''] * num_books

	for n in range(1, num_books+1):
		sno[n-1] = n

		books[n-1] = read_element(f'(//td/a//span[@itemprop="name"])[{n}]/..')
		print(n, "book = "+books[n-1])

		url[n-1] = read_element('(//td/a/@href)[{}]'.format(n))
		print("   url = "+url[n-1])

	Books_Info = {'Sno':sno, 
		'Title': books, 
		'URL': url
	}
	save_to_csv(Books_Info, 'book_list2.csv')

def save_to_csv(data, csv_file):
	df = DataFrame(data, columns=['Sno','Title', 'URL'])
	export_csv = df.to_csv (csv_file, index = None)

# main program starts here
t.url('https://www.goodreads.com/list/show/17347.Books_that_ll_help_you_deliver_stunning_presentations')
read_list_title()
read_book_list()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.