pynput<\/strong><\/h2>\n\n\n\nThe first package we\u2019ll discuss is pynput<\/strong>. One of the advantages of pynput<\/strong> is that is works on both Windows and macOS. Another nice feature is that it has functionality to monitor keyboard and mouse input. Let\u2019s get started with pynput<\/strong> by installing it with pip:<\/p>\n\n\n\npip install pynput<\/code><\/pre>\n\n\n\nOnce you have it installed, you can get started by importing the Controller<\/em> and Button<\/em> classes. Then, we\u2019ll create an instance of the Controller<\/em> class, which we\u2019ll call mouse<\/em>. This will simulate your computer\u2019s mouse to allow you to programmatically click buttons and move the mouse around on the screen.<\/p>\n\n\n\nfrom pynput.mouse import Button, Controller\n \nmouse = Controller()<\/code><\/pre>\n\n\n\nNext, let\u2019s look at a couple simple commands. To right or left-click, we can use the Button class imported above.<\/p>\n\n\n\n
# left-click\nmouse.press(Button.left)\n \n# right-click\nmouse.press(Button.right)<\/code><\/pre>\n\n\n\nTo double click, you just need to add the number two as the second parameter.<\/p>\n\n\n\n
mouse.press(Button.left, 2)<\/code><\/pre>\n\n\n\nWe can also move the mouse pointer to a different position by using the move<\/em> method.<\/p>\n\n\n\nmouse.move(50, -50)\n \nmouse.move(100, -200)<\/code><\/pre>\n\n\n\npynput<\/strong> can control the keyboard, as well. To do that, we need to import the Key<\/em> class<\/p>\n\n\n\n\nfrom pynput.keyboard import Key<\/code><\/pre>\n\n\n\nTo make your keyboard type, you can use the aptly-named keyboard.type<\/em> method.<\/p>\n\n\n\n\t\nkeyboard.type(\"this is a test\")<\/code><\/pre>\n\n\n\nAs mentioned above, pynput<\/strong> can also monitor mouse movements and keyboard presses. To learn more about that functionality and pynput<\/strong>, check out this link.<\/a><\/p>\n\n\n\npyautogui<\/strong><\/h2>\n\n\n\nPerhaps the most commonly known package for simulating mouse clicks and keyboard entries is the pyautogui<\/strong> library. pyautogui<\/strong> works on Windows, Linux, and macOS. If you don\u2019t have it installed, you can get it using pip:<\/p>\n\n\n\npip install pyautogui<\/code><\/pre>\n\n\n\npyautogui<\/strong> is also straightforward to use. For example, if you want to simulate typing a string of text, just use the typewrite<\/em> method:<\/p>\n\n\n\npyautogui.typewrite(\"test pyautogui!\")<\/code><\/pre>\n\n\n\nTo left-click your mouse, you can use the click<\/em> method. To right-click, you can use the rightClick<\/em> method.<\/p>\n\n\n\n# left-click\npyautogui.click(100, 200)\n \n# right-click\npyautogui.rightClick(100, 200)<\/code><\/pre>\n\n\n\nSearching for an image on the screen<\/strong><\/h4>\n\n\n\nOne of the coolest features of pyautogui<\/strong> is that it can search for an image on the computer screen. This is really helpful if you need to find a particular button to click. You can search for an image by inputting the image file name into the locateOnScreen<\/em> method. The function returns the topleft coordinate along with the height and width of the identified image.<\/p>\n\n\n\nlocation = pyautogui.locateOnScreen(\"random_image.png\")<\/code><\/pre>\n\n\n\n![\"\"](\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\")
<\/figure>\n\n\n\nTo get the center of identified image, use the center<\/em> method. Then, you can use the click<\/em> method to click on the center of the identified image \u2013 in this case, a button on the screen.<\/p>\n\n\n\n<\/figure>\n\n\n\ncenter = pyautogui.center(location)\n \npyautogui.click(center)<\/code><\/pre>\n\n\n\nSometimes an image may not be found exactly on a screen. In this case, you can add the confidence<\/em> parameter to locateOnScreen<\/em> to give Python a confidence level of identifying the image.<\/p>\n\n\n\npyautogui.locateOnScreen(\"random_image.png\", confidence = 0.95)<\/code><\/pre>\n\n\n\nTaking a screenshot<\/strong><\/h4>\n\n\n\nYou can take a screenshot with pyautogui<\/strong> using the screenshot<\/em> method. Passing a filename will save the screenshot out to that file.<\/p>\n\n\n\n\t\ns = pyautogui.screenshot(\"sample_screenshot.png\")<\/code><\/pre>\n\n\n\nIt\u2019s also possible to take a screenshot of a specific region, rather than the full screen:<\/p>\n\n\n\n
\t\npyautogui.screenshot(region = c(0, 0, 100, 200))<\/code><\/pre>\n\n\n\npywinauto<\/strong><\/h2>\n\n\n\nOn Windows, another option we can look into is the pywinauto<\/strong> library. The main disadvantage of this library is that it does not work on macOS or Linux. However, it also offers a couple of nice advantages for Windows users. One, it\u2019s syntax is object-oriented \u2013 it\u2019s made to be more Pythonic. Secondly, because of its design, the library can make it easier to perform certain tasks, like clicking on specific buttons or finding menu items in an application.<\/p>\n\n\n\nFor example, let\u2019s start by launching Notepad, typing some text, and saving the file. We can do that using the code snippet below. Here, we start Notepad by using the Application<\/em> class. Then, we refer to the Notepad file we just opened by \u201cUnitledNotepad\u201d. We can use the Edit.type_keys<\/em> to start typing text.<\/p>\n\n\n\nfrom pywinauto.application import Application\n \napp = Application(backend=\"uia\").start(\"notepad.exe\")\napp.UntitledNotepad.Edit.type_keys(\"Starting notepad...\")\napp.UntitledNotepad.menu_select(\"File->SaveAs\")\nsub_app=app.UntitledNotepad.child_window(title_re = \"Save As\")\nsub_app.FileNameCombo.type_keys(\"test_file.txt\")\nsub_app.Save.click()<\/code><\/pre>\n\n\n\nLearn more about pywinauto<\/strong> by checking out this link<\/a>.<\/p>\n\n\n\nConclusion<\/strong><\/h2>\n\n\n\nThat\u2019s it for this post! We covered three packages for doing robotic process automation with Python. Check out my other Python posts here<\/a>.<\/p>\n\n\n\n
pip install pynput<\/code><\/pre>\n\n\n\nOnce you have it installed, you can get started by importing the Controller<\/em> and Button<\/em> classes. Then, we\u2019ll create an instance of the Controller<\/em> class, which we\u2019ll call mouse<\/em>. This will simulate your computer\u2019s mouse to allow you to programmatically click buttons and move the mouse around on the screen.<\/p>\n\n\n\nfrom pynput.mouse import Button, Controller\n \nmouse = Controller()<\/code><\/pre>\n\n\n\nNext, let\u2019s look at a couple simple commands. To right or left-click, we can use the Button class imported above.<\/p>\n\n\n\n
# left-click\nmouse.press(Button.left)\n \n# right-click\nmouse.press(Button.right)<\/code><\/pre>\n\n\n\nTo double click, you just need to add the number two as the second parameter.<\/p>\n\n\n\n
mouse.press(Button.left, 2)<\/code><\/pre>\n\n\n\nWe can also move the mouse pointer to a different position by using the move<\/em> method.<\/p>\n\n\n\nmouse.move(50, -50)\n \nmouse.move(100, -200)<\/code><\/pre>\n\n\n\npynput<\/strong> can control the keyboard, as well. To do that, we need to import the Key<\/em> class<\/p>\n\n\n\n\nfrom pynput.keyboard import Key<\/code><\/pre>\n\n\n\nTo make your keyboard type, you can use the aptly-named keyboard.type<\/em> method.<\/p>\n\n\n\n\t\nkeyboard.type(\"this is a test\")<\/code><\/pre>\n\n\n\nAs mentioned above, pynput<\/strong> can also monitor mouse movements and keyboard presses. To learn more about that functionality and pynput<\/strong>, check out this link.<\/a><\/p>\n\n\n\npyautogui<\/strong><\/h2>\n\n\n\nPerhaps the most commonly known package for simulating mouse clicks and keyboard entries is the pyautogui<\/strong> library. pyautogui<\/strong> works on Windows, Linux, and macOS. If you don\u2019t have it installed, you can get it using pip:<\/p>\n\n\n\npip install pyautogui<\/code><\/pre>\n\n\n\npyautogui<\/strong> is also straightforward to use. For example, if you want to simulate typing a string of text, just use the typewrite<\/em> method:<\/p>\n\n\n\npyautogui.typewrite(\"test pyautogui!\")<\/code><\/pre>\n\n\n\nTo left-click your mouse, you can use the click<\/em> method. To right-click, you can use the rightClick<\/em> method.<\/p>\n\n\n\n# left-click\npyautogui.click(100, 200)\n \n# right-click\npyautogui.rightClick(100, 200)<\/code><\/pre>\n\n\n\nSearching for an image on the screen<\/strong><\/h4>\n\n\n\nOne of the coolest features of pyautogui<\/strong> is that it can search for an image on the computer screen. This is really helpful if you need to find a particular button to click. You can search for an image by inputting the image file name into the locateOnScreen<\/em> method. The function returns the topleft coordinate along with the height and width of the identified image.<\/p>\n\n\n\nlocation = pyautogui.locateOnScreen(\"random_image.png\")<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nTo get the center of identified image, use the center<\/em> method. Then, you can use the click<\/em> method to click on the center of the identified image \u2013 in this case, a button on the screen.<\/p>\n\n\n\n<\/figure>\n\n\n\ncenter = pyautogui.center(location)\n \npyautogui.click(center)<\/code><\/pre>\n\n\n\nSometimes an image may not be found exactly on a screen. In this case, you can add the confidence<\/em> parameter to locateOnScreen<\/em> to give Python a confidence level of identifying the image.<\/p>\n\n\n\npyautogui.locateOnScreen(\"random_image.png\", confidence = 0.95)<\/code><\/pre>\n\n\n\nTaking a screenshot<\/strong><\/h4>\n\n\n\nYou can take a screenshot with pyautogui<\/strong> using the screenshot<\/em> method. Passing a filename will save the screenshot out to that file.<\/p>\n\n\n\n\t\ns = pyautogui.screenshot(\"sample_screenshot.png\")<\/code><\/pre>\n\n\n\nIt\u2019s also possible to take a screenshot of a specific region, rather than the full screen:<\/p>\n\n\n\n
\t\npyautogui.screenshot(region = c(0, 0, 100, 200))<\/code><\/pre>\n\n\n\npywinauto<\/strong><\/h2>\n\n\n\nOn Windows, another option we can look into is the pywinauto<\/strong> library. The main disadvantage of this library is that it does not work on macOS or Linux. However, it also offers a couple of nice advantages for Windows users. One, it\u2019s syntax is object-oriented \u2013 it\u2019s made to be more Pythonic. Secondly, because of its design, the library can make it easier to perform certain tasks, like clicking on specific buttons or finding menu items in an application.<\/p>\n\n\n\nFor example, let\u2019s start by launching Notepad, typing some text, and saving the file. We can do that using the code snippet below. Here, we start Notepad by using the Application<\/em> class. Then, we refer to the Notepad file we just opened by \u201cUnitledNotepad\u201d. We can use the Edit.type_keys<\/em> to start typing text.<\/p>\n\n\n\nfrom pywinauto.application import Application\n \napp = Application(backend=\"uia\").start(\"notepad.exe\")\napp.UntitledNotepad.Edit.type_keys(\"Starting notepad...\")\napp.UntitledNotepad.menu_select(\"File->SaveAs\")\nsub_app=app.UntitledNotepad.child_window(title_re = \"Save As\")\nsub_app.FileNameCombo.type_keys(\"test_file.txt\")\nsub_app.Save.click()<\/code><\/pre>\n\n\n\nLearn more about pywinauto<\/strong> by checking out this link<\/a>.<\/p>\n\n\n\nConclusion<\/strong><\/h2>\n\n\n\nThat\u2019s it for this post! We covered three packages for doing robotic process automation with Python. Check out my other Python posts here<\/a>.<\/p>\n\n\n\n
![\"\"](\"\/campus\/wp-content\/uploads\/sites\/2\/2022\/06\/python-pyautogui-locateonscreen-the-automatic-net.png\")
<\/figure>\n\n\n\n![\"\"](\"\/campus\/wp-content\/uploads\/sites\/2\/2022\/06\/pyautogui-center-the-automatic-net.png\")
<\/figure>\n\n\n\n