Remote Window Controller

A Python application that allows you to remotely control application windows, capture screenshots, and automatically navigate through content by detecting changes between screenshots.

Features

• Window Selection: Browse and select from a list of available application windows
• Automatic Navigation: Maximize selected window and send forward arrow key presses
• Screenshot Capture: Automatically capture screenshots at configurable intervals
• Change Detection: Compare consecutive screenshots to detect when content changes
• Smart Loop Control: Automatically stop when no more changes are detected
• GUI Interface: User-friendly graphical interface with real-time status updates
• Configurable Delay: Adjustable delay between key presses and screenshot captures

Installation

Prerequisites

• Python 3.11 (managed by mise, requires Python 3.8+)
• mise (development environment manager)
• uv (ultra-fast Python package manager)

Quick Start

• Clone or download this repository to your local machine

• Install mise (if not already installed):

# Windows (PowerShell)
   powershell -c "irm https://mise.jdx.dev/install.ps1 | iex"
   
   # macOS/Linux
   curl https://mise.jdx.dev/install.sh | sh

• Install Python version and uv:

# Install Python 3.11 and uv (mise will read .mise.toml)
   mise install
   
   # Install uv if not already installed
   mise use uv@latest

• Install dependencies:

uv sync

• Run the application:

uv run python -m remote_window_controller

Alternative Installation Methods

Using the provided batch file (Windows):

install.bat

Manual installation with pip (fallback):

pip install pygetwindow pyautogui Pillow opencv-python numpy scikit-image

Development setup with linting:

# Install with development dependencies
uv sync --extra dev

# Run linting
uv run ruff check .

# Run formatting
uv run black .

# Run tests
uv run pytest

Using mise for development environment:

# Install mise and Python version
mise install

# Activate the environment
mise activate

# Install dependencies
uv sync

Usage

GUI Mode (Recommended)

• Launch the application - The GUI will open automatically
• Select Output Directory - Click "Browse" to choose where screenshots will be saved
• Set Delay - Configure the delay between key presses (default: 1.0 seconds)
• Refresh Windows - Click "Refresh Windows" to get the current list of available windows
• Select Window - Choose the window you want to control from the list
• Start Control - Click "Start Control" to begin the automation process

Command Line Mode

If the GUI fails to start, the application will automatically fall back to command line mode:

• Run uv run python -m remote_window_controller
• Follow the prompts to:

• Select a window by number

• Enter output directory path

• Set delay in seconds

How It Works

• Window Selection: The application lists all available windows with titles
• Maximization: Selected window is maximized using F11 key
• Initial Capture: First screenshot is captured and saved
• Navigation Loop:

• Send forward arrow key press

• Wait for configured delay

• Capture new screenshot

• Compare with previous screenshot

• If different: save image and continue loop

• If same: exit fullscreen and stop
• Cleanup: Window exits fullscreen mode and returns to normal view

Output Files

Screenshots are saved in the specified output directory with sequential naming:

• capture_0000.png - Initial screenshot
• capture_0001.png - After first key press
• capture_0002.png - After second key press
• And so on...

Configuration

• Delay: Adjustable from 0.1 to 10.0 seconds
• Output Directory: Any writable directory on your system
• Image Comparison: Uses structural similarity index (SSIM) with 95% threshold

Requirements

• Python 3.7 or higher
• Windows operating system (for window control functionality)
• Sufficient disk space for screenshot storage

Troubleshooting

Common Issues

• "No windows found": Make sure you have applications open with visible windows
• "Failed to select window": The window might be minimized or not accessible
• Permission errors: Ensure the output directory is writable
• GUI not starting: The application will automatically fall back to command line mode

Tips

• Make sure the target application is visible and not minimized
• Use a reasonable delay (1-2 seconds) to allow content to load
• Ensure sufficient disk space for screenshots
• Close unnecessary applications to reduce the window list clutter

Screenshot Troubleshooting

If you're getting black screenshots instead of window content:

• Run as Administrator: Right-click and "Run as administrator"
• Check Display Settings: Ensure your display scaling is set to 100%
• Disable Hardware Acceleration: Some applications block screenshots when hardware acceleration is enabled
• Try Different Applications: Some applications (like games or video players) may block screenshots
• Check Windows Privacy Settings: Go to Settings > Privacy > Camera and ensure apps can access camera (affects some screenshot methods)

The application will automatically test screenshot capability and warn you if there are issues.

Dependencies

• pygetwindow: Window enumeration and control
• pyautogui: Screenshot capture and key simulation
• Pillow: Image processing
• opencv-python: Image comparison
• scikit-image: Structural similarity calculations
• numpy: Numerical operations
• tkinter: GUI interface (included with Python)

Development Environment

This project uses mise for development environment management and uv for package management:

mise Configuration

The .mise.toml file specifies:

• Python 3.11 as the required Python version
• Environment variables for the project
• Tool management for consistent development setup

mise Commands

• Install tools: mise install
• Activate environment: mise activate
• Check current tools: mise list
• Update tools: mise upgrade

uv Package Management

• Install dependencies: uv sync
• Add new dependency: uv add package-name
• Remove dependency: uv remove package-name
• Run with dependencies: uv run python script.py
• Update dependencies: uv sync --upgrade

Benefits

mise:

• Consistent environments across team members
• Automatic tool installation and version management
• Fast switching between project environments
• Cross-platform compatibility

uv:

• 10-100x faster than pip for dependency resolution
• Reproducible builds with lock files
• Better dependency resolution with conflict detection
• Virtual environment management built-in

ruff (replaces flake8):

• 10-100x faster than flake8 for linting
• Python 3.11+ compatible with modern tooling
• Built-in import sorting (replaces isort)
• Comprehensive rule set covering style, bugs, and complexity

License

This project is open source and available under the MIT License.