PyAutoGUI

PyAutoGUI is a cross-platform Python library used for GUI (Graphical User Interface) automation. It empowers Python scripts to programmatically control the mouse and keyboard, making it an invaluable tool for automating repetitive tasks, performing automated testing, and even creating simple robotic process automation (RPA) bots. It works by simulating user interactions directly with the operating system's graphical interface, rather than interacting with application-specific APIs.

Key features and capabilities of PyAutoGUI include:
- Mouse Control: It can move the mouse cursor to specific X,Y coordinates on the screen, perform various clicks (left, right, middle), drag elements, and scroll the mouse wheel.
- Keyboard Control: The library allows scripts to type strings of text, press individual keys (like Enter, Space, Tab), and execute complex hotkey combinations (e.g., Ctrl+C, Alt+Tab, Shift+F10).
- Screenshots: PyAutoGUI can take screenshots of the entire screen or specific regions. These screenshots can be saved to files or used internally for image recognition.
- Image Recognition: A powerful feature is its ability to locate images (e.g., a button icon, a specific piece of text) on the screen. This allows for robust automation where the script can react to the visual state of the GUI.
- Message Boxes: It provides functions to display simple alert, confirm, or prompt message boxes, which can be useful for debugging or interactive scripts.
- Failsafe Mechanism: To prevent runaway scripts, PyAutoGUI includes a failsafe. If the mouse cursor is moved to the top-left corner of the screen, it will immediately stop the script, giving the user an emergency stop option.

Installation of PyAutoGUI is straightforward using pip: `pip install pyautogui`. Due to its direct interaction with the GUI, it's often recommended to include a short `time.sleep()` at the beginning of a script to give the user a few seconds to move their mouse away from the top-left corner (to avoid accidentally triggering the failsafe) and prepare the environment. PyAutoGUI is a robust solution for automating tasks that would otherwise be tedious and time-consuming for a human user, enhancing productivity and consistency.

Example Code

import pyautogui
import time

 --- Configuration and Failsafe --- 
 Give yourself a few seconds to move the mouse away from the top-left corner (0,0).
 Moving the mouse to (0,0) will trigger the failsafe and stop the script immediately.
print("PyAutoGUI script starting in 5 seconds. Please move your mouse away from the top-left corner.")
time.sleep(5)

print("Script started. Current mouse position:", pyautogui.position())

 --- Mouse Automation ---
 Move the mouse to a specific coordinate (e.g., 100, 100) over 1 second
print("Moving mouse to (100, 100)...")
pyautogui.moveTo(100, 100, duration=1)
print("Mouse moved. Current position:", pyautogui.position())

 Click the left mouse button at the current position
print("Clicking mouse...")
pyautogui.click()
print("Mouse clicked.")

 --- Keyboard Automation ---
 Type a string of text with a short interval between each character
print("Typing 'Hello PyAutoGUI!'...")
pyautogui.typewrite("Hello PyAutoGUI!\n", interval=0.1)  \n for Enter key
print("Text typed.")

 Press a hotkey combination (e.g., Ctrl+Shift+Esc to open Task Manager on Windows)
 Be cautious with hotkeys, as they can affect your system.
 print("Pressing Ctrl+Shift+Esc...")
 pyautogui.hotkey('ctrl', 'shift', 'esc')
 time.sleep(1)  Give system time to react
 print("Ctrl+Shift+Esc pressed.")

 --- Screenshot and Image Recognition ---
 Take a screenshot of the entire screen and save it to a file
screenshot_filename = "my_screenshot.png"
print(f"Taking a screenshot and saving it as {screenshot_filename}...")
pyautogui.screenshot(screenshot_filename)
print(f"Screenshot saved to {screenshot_filename}.")

 --- Example of finding an image (requires an image file to be present) ---
 You can use pyautogui.locateOnScreen('button.png') to find an image
 If found, it returns a Box object (left, top, width, height) of the first match.
 try:
      Replace 'target_image.png' with an actual image file on your screen
     button_location = pyautogui.locateOnScreen('target_image.png', confidence=0.9)
     if button_location:
         print(f"Image found at: {button_location}")
          Move to the center of the found image and click
         pyautogui.click(pyautogui.center(button_location))
         print("Clicked on the found image.")
     else:
         print("Image 'target_image.png' not found on screen.")
 except pyautogui.ImageNotFoundException:
     print("Error: 'target_image.png' not found or could not be loaded.")
 except Exception as e:
     print(f"An error occurred while trying to locate image: {e}")

print("PyAutoGUI automation task complete.")

Example Code

Related Topics