Automating forms with Selenium or React Native

While serving my conscription period, I've noticed that there are quite a number of digital forms to fill up. And these are not 1-time forms, these are forms I have to fill every day. Being a problem solver myself (and a lazy one at that), I've wanted to make these forms much easier to fill up. I wanted to

  1. Store data in a much easier manner
  2. Process it into the data expected by the form
  3. Autofill the form

After some experimentation, I've figured out a few solutions

Selenium

I've done some CTF challenges in June 2022 (HTB CTF, SEETF), some of which include the usage of this Python library called selenium as a Web Bot. I had to read code using this library before, and I figured I could use it to automate the submission of my forms.

Getting it up and running is actually quite easy. Firstly, you need to install it and its necessary dependencies. You should pip install selenium, but you also need to install a Web Browser Driver. I used ChromeDriver, but there is also GeckoDriver for Firefox.

Here is a sample piece of code. it fills up a text box which you can select by the element ID and clicks an element. With these concepts you probably can make a script to autofill your own form

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

def visit_report():
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--incognito")
    client = webdriver.Chrome(chrome_options=chrome_options,
        executable_path=r"/run/media/hacker/Windows/Users/zunmu/Documents/Stuff/Linux Tools/chromedriver")
		#executable_path=r"D:\\chromedriver.exe")
    client.set_page_load_timeout(10)
    client.set_script_timeout(10)

    client.get('https://www.w3schools.com/html/html_forms.asp')
    time.sleep(3)
    ### Fill up text fields
    client.find_element_by_id("fname").send_keys("Data")
    client.find_element_by_id("lname").click()
    ### Traversing the DOM
    button = (
        client.find_element_by_id("fname")
        .find_element(by=By.XPATH,value='..')
        .find_elements_by_xpath(".//*")[-1]
    )
    button.click()
    time.sleep(300)
    client.quit()

visit_report()

Selenium Python has some useful resources like

  1. https://selenium-python.readthedocs.io/locating-elements.html
  2. https://stackoverflow.com/questions/24795198/get-all-child-elements

One of the benefits of this is that you can use whatever libraries you want and that it's easy to code. I ended up combining Selenium and openpyxl, an Excel Sheet Python Library, to read the data from an Excel Sheet and autofill a form.

Mobile Selenium?

The top is great and all, but I wanted something mobile. I don't bring my laptop with me everywhere after all. My first approach was to just run my selenium code on mobile.

I had previously set up Kali NetHunter on my Android Phone and was thinking if it could just run my Python Script as is. Turns out, Chrome isn't supported on my Kali NetHunter installation due to some SUID misconfiguration.

I switched the driver to GeckoDriver, to take advantage of the Firefox preinstalled on nethunter. I found an ARM binary here, downloaded it, and updated my script with the new WebDriver like here.

It generally ended up working quite well, no need to recode everything, but

  1. You have to go through NetHunter, which is not a very intuitive interface.
  2. It only works on Android, and not on iOS

I wanted a better solution so that I can share the joy of auto-filling forms with others

React-Native / Expo

On a whim, I decided to search up React Native WebView, and check if it has the functionality to autofill them. I got something even better, we can inject our own custom Javascript code with the injectedJavaScript prop. Referenced from here.

<WebView 
      style={styles.container}
      source={{uri: 'https://github.com/react-native-community/react-native-webview'}}  
      javaScriptEnabled={true}
      injectedJavaScript={`document.body.style.backgroundColor = 'red';
      setTimeout(function() { window.alert('hi') }, 2000);
      true;`}    
      onMessage={(event) => {}}
    />

%[http://snack.expo.dev/@hackin7/basic-webview-usage]

You should take note that the code you want to run to autofill the document should probably be in setTimeout to run a few seconds after the document loads (probably to give it time to load).

With this, I spent a weekend working on making a Proof of Concept App to automatically fill up forms. I used document.getElementById("<id>").value = <value>; and <element>.click(); in Javascript. Making the app is as hard as making a typical mobile app (harder than selenium, but nothing too complicated or challenging).

And that's my journey towards building an Application to automatically fill up the form I need.

Overall

Generally, Selenium is good for quick prototyping, while you can use React Native/ Expo to finalise a proper application.

There are probably many other ways to do this. For example, the Android SDK's WebView probably has the functionality to interact with the elements, as shown here. Alternatively, you could just manually code a Python script, which could generate Javascript to paste in the Developer Console (though not the most elegant solution).
Image description