click

Click on GUI elements using image recognition, text matching, or coordinates.

Usage Examples

Click by Image

actions:
  - click:
      target:
        image: "save_button.png"
        confidence: 0.8
      description: "Click the Save button"

Click by Text

actions:
  - click:
      target:
        text: "OK"
        case_sensitive: false
      description: "Click OK dialog button"

Click at Coordinates

actions:
  - click:
      target:
        coordinates: [150, 300]
      description: "Click at specific location"

Parameters

Parameter	Type	Description
target	Target	Element to click - image, text, or coordinates (required)
type	string	Click type: left, right, or double (default: left)
description	string	Human-readable description (optional)

Target Options

• image: Path to reference image with optional confidence (0.0-1.0)

• text: Text to find with optional text_match configuration

• coordinates: [x, y] pixel coordinates

• strategy: Selection strategy when multiple matches found (SweepStrategy, BestConfidenceStrategy, etc.)

• use_cache: bool, default unset (auto-reuse) - a match cached by an earlier target resolution (e.g. a wait_until check) on an identical target (same image/text/position/strategy/offset) is reused automatically, skipping a fresh screenshot + CV pass, as long as no other target resolution has happened in between. Set explicitly to false to opt out and force a fresh detection for this target even when that condition holds. See Reusing a wait_until Match (Performance) below.

Advanced Text Matching

ADARE supports multiple text matching modes to handle OCR inaccuracies and enable flexible pattern matching.

Substring Matching (Default)

Simple case-insensitive substring matching. No configuration needed.

actions:
  - click:
      target:
        text: "Documents"
      description: "Click Documents (case-insensitive substring match)"

Regex Pattern Matching

Match text using regular expression patterns.

actions:
  - click:
      target:
        text: "File \\d+"
        text_match:
          mode: regex
          flags: [IGNORECASE]
      description: "Click text matching 'File 1', 'File 2', etc."

Fuzzy Matching - Missing Characters Mode

Tolerate missing characters that OCR often fails to detect.

Allow any character to be missing:

actions:
  - click:
      target:
        text: "More..."
        text_match:
          mode: fuzzy
          allow_missing_chars: true
          max_missing: 3
      description: "Match 'More' even if any characters are missing"

Allow only specific characters to be missing (recommended):

actions:
  - click:
      target:
        text: "More..."
        text_match:
          mode: fuzzy
          allow_missing_chars: "."
          max_missing: 3
      description: "Match 'More' when only dots are missing"

Allow multiple specific characters to be missing:

actions:
  - click:
      target:
        text: "Price: $19.99"
        text_match:
          mode: fuzzy
          allow_missing_chars: [".", "$", ":"]
          max_missing: 4
      description: "Match even if dots, dollar sign, or colon are missing"

Fuzzy Matching - Percentage Similarity Mode

Match text based on similarity threshold (0.0-1.0), useful for OCR character confusion.

actions:
  - click:
      target:
        text: "Settings"
        text_match:
          mode: fuzzy
          min_similarity: 0.85
      description: "Match 'Settinqs' (OCR confused 'g' with 'q')"

Combined Regex + Fuzzy Matching

Use regex patterns with fuzzy tolerance for maximum flexibility.

actions:
  - click:
      target:
        text: "Doc.*ents"
        text_match:
          mode: regex_fuzzy
          flags: [IGNORECASE]
          min_similarity: 0.8
      description: "Flexible pattern with error tolerance"

Case-Sensitive Fuzzy Matching

Enable case-sensitive comparison for fuzzy matching.

actions:
  - click:
      target:
        text: "LOGIN"
        text_match:
          mode: fuzzy
          min_similarity: 0.9
          case_sensitive: true
      description: "Case-sensitive fuzzy match for 'LOGIN'"

Text Match Configuration Fields

Field	Type	Description
mode	string	Matching mode: substring, regex, fuzzy, regex_fuzzy (default: substring)
flags	list	Regex flags: IGNORECASE, MULTILINE, DOTALL, VERBOSE
allow_missing_chars	bool/string/list	Allowed missing characters (fuzzy mode): true (any char), "." (only dots), [".", ","] (specific chars)
max_missing	int	Max missing chars allowed (requires allow_missing_chars)
min_similarity	float	Minimum similarity ratio 0.0-1.0 (fuzzy mode)
case_sensitive	bool	Enable case-sensitive matching (default: false)

Use Cases

• Missing dots: OCR often misses periods, ellipses, decimal points

• Version numbers: Use regex to match dynamic version strings (e.g., v\\d+\\.\\d+)

• Character confusion: OCR may confuse similar characters (O/0, l/1, rn/m)

• Partial matches: Fuzzy matching handles truncated or slightly modified text

Target Selection Strategies

When multiple matches are found for a target, you can use strategies to select which one to click.

ClosestToStrategy - Find Target Near Another Element

Click elements based on proximity to reference targets (text or images). Useful when the same icon appears multiple times but you want the one near specific text.

Find icon closest to text:

actions:
  - click:
      target:
        image: "delete_icon.png"
        strategy:
          ClosestToStrategy:
            text: "File1.txt"
      description: "Click delete icon nearest to File1.txt"

Find text closest to image:

actions:
  - click:
      target:
        text: "OK"
        strategy:
          ClosestToStrategy:
            image: "warning_icon.png"
      description: "Click OK button near warning icon"

Limit search distance (optimization):

actions:
  - click:
      target:
        image: "submit_button.png"
        strategy:
          ClosestToStrategy:
            text: "Form Title"
            max_distance: 200
      description: "Click submit button within 200 pixels of form title"

The max_distance parameter (in pixels):

• Filters out matches beyond the specified distance

• Enables performance optimization by cropping the screenshot before CV processing

• Action fails if no matches found within the distance limit

Fixed coordinates (backwards compatible):

actions:
  - click:
      target:
        image: "button.png"
        strategy:
          ClosestToStrategy:
            x: 500
            y: 300
      description: "Click button closest to coordinates (500, 300)"

Other Selection Strategies

• SweepStrategy: Select nth occurrence in reading order (top-to-bottom, left-to-right)

• BestConfidenceStrategy: Select match with highest confidence score (default for images)

• TopLeftStrategy: Select top-left match (default for text)

• TopRightStrategy, BottomLeftStrategy, BottomRightStrategy: Corner-based selection

• LargestStrategy, SmallestStrategy: Select by bounding box area

Example with SweepStrategy:

actions:
  - click:
      target:
        text: "Documents"
        strategy:
          SweepStrategy:
            index: 2
      description: "Click the second occurrence of Documents"

Reusing a wait_until Match (Performance)

CV detection (image/text matching) can take anywhere from ~200ms to over a second per attempt. If a wait_until already confirmed a target is on screen, a following click on the identical target automatically skips re-running detection - no flag needed:

actions:
  - wait_until:
      condition:
        exists:
          image: "save_button.png"
      timeout: 30.0
  - click:
      target:
        image: "save_button.png"
      description: "Click Save (automatically reuses the wait_until match)"

Boundary: the cache is only valid for the single target-resolution attempt immediately following the one that populated it - it is not a whole-run cache. A target resolution is any exists/not_exists check inside a wait_until or any target lookup performed by an action like click. Non-resolving actions in between (keyboard input, a pause, a drag by fixed coordinate) do not invalidate the cache, since they never look at the screen for a target - only another target resolution does, whether or not it targets the same image/text/position/strategy/offset.

Opt out with use_cache: false when you know the screen changed without another find running in between - for example, a keyboard shortcut that closes the dialog the cached match was found in:

actions:
  - wait_until:
      condition:
        exists:
          image: "save_button.png"
      timeout: 30.0
  - keyboard:
      hotkey: "alt+f4"
      description: "Close the dialog without clicking Save"
  - click:
      target:
        image: "save_button.png"
        use_cache: false
      description: "Force fresh detection - the dialog is gone, don't trust the stale match"

Notes

• Image files should be in the experiment directory

• Text matching uses OCR (Tesseract)

• Coordinates are relative to top-left corner

• Use wait_until before clicking if element may not be immediately visible - see Reusing a wait_until Match (Performance) above to skip the redundant re-detection

See Also

• wait_until for waiting on elements

• keyboard for keyboard input

• drag for drag-and-drop operations