CODEWITHSUNDEEP
androidfirebasefirebase-test-lab
Bitwise DEVS
Bitwise DEVS

Reputation: 3449

Firebase Robo Test Script keeps failing

We are using Firebase UI with Phone Auth and it seems Robo Script does not work properly with it. Device like LG Nexus 5, API Level 23 hammerhead works fine following some part of the script but many other physical devices we tried failed following the script.

This is a success OTP with Firebase Auth with LG Nexus 5

enter image description here

This one is totally incorrect at the very beginning Samsung SM-G981U1 enter image description here

And the list of non-working physical device with Robo script goes on, not sure what is special about Nexus 5 or probably it has something to do with the screen resolution? One thing I noticed is after putting 9876543210 in EditText of Firebase UI phone Auth the test device with Robo Script always fails to click Verify Phone Number button where most likely the error occurred except for LG Nexus 5 which successfully execute this part of the script.

Another sample

enter image description here

Here it supposed to hit Verify Phone Number button after successfully putting 9876543210 in EditText which is based on our Robo Script but it didn't and the whole input was replaced with 6504992804 which we never record and not in Robo Script. So our conclusion was the error usually happens due to failure of script satisfying the click of Verify Phone Number button.

enter image description here

Robo Script sample

[
  {
    "eventType": "DELAYED_MESSAGE_POSTED",
    "timestamp": 1627584568123,
    "actionCode": -1,
    "delayTime": 7000,
    "canScrollTo": false,
    "elementDescriptors": []
  },
  {
    "eventType": "VIEW_TEXT_CHANGED",
    "timestamp": 1627584581148,
    "replacementText": "9876543210",
    "actionCode": -1,
    "delayTime": 0,
    "canScrollTo": true,
    "elementDescriptors": [
      {
        "className": "com.google.android.material.textfield.TextInputEditText",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 0,
        "resourceId": "com.devsbitwise.cryptonian:id/phone_number",
        "contentDescription": "",
        "text": ""
      },
      {
        "className": "android.widget.FrameLayout",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 0,
        "resourceId": "",
        "contentDescription": "",
        "text": ""
      },
      {
        "className": "com.google.android.material.textfield.TextInputLayout",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 1,
        "resourceId": "com.devsbitwise.cryptonian:id/phone_layout",
        "contentDescription": "",
        "text": ""
      }
    ]
  },
  {
    "eventType": "VIEW_CLICKED",
    "timestamp": 1627584599899,
    "replacementText": "Verify Phone Number",
    "actionCode": -1,
    "delayTime": 0,
    "canScrollTo": true,
    "elementDescriptors": [
      {
        "className": "com.google.android.material.button.MaterialButton",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 2,
        "resourceId": "com.devsbitwise.cryptonian:id/send_code",
        "contentDescription": "",
        "text": "Verify Phone Number"
      },
      {
        "className": "androidx.constraintlayout.widget.ConstraintLayout",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 0,
        "resourceId": "",
        "contentDescription": "",
        "text": ""
      },
      {
        "className": "android.widget.ScrollView",
        "recyclerViewChildPosition": -1,
        "adapterViewChildPosition": -1,
        "groupViewChildPosition": 1,
        "resourceId": "",
        "contentDescription": "",
        "text": ""
      }
    ]
  },
  {
    "eventType": "DELAYED_MESSAGE_POSTED",
    "timestamp": 1627584605228,
    "actionCode": -1,
    "delayTime": 500,
    "canScrollTo": false,
    "elementDescriptors": []
  },

UPDATE

Last remaining problem is probably the consistency, here I used the same Robo script in first test it succeed but in the second with no changes in the code at all it fails.

First Test (Nexus 5 Virtual SDK 21) enter image description here

Second Test (Nexus 5 Virtual SDK 21) enter image description here

First Test (Nexus 9 Virtual SDK 24) enter image description here

Second Test (Nexus 9 Virtual SDK 24) enter image description here

First Test (Nexus 7 Virtual SDK 22) enter image description here

Second Test (Nexus 7 Virtual SDK 22) enter image description here

Upvotes: 0

Views: 912

Answers (1)

Stanislav Negara
Stanislav Negara

Reputation: 662

You are right that Roboscript fails to click the Verify Phone Number button, as the result says that Roboscript failed on action 3, which is clicking this button.

Overall, I think the problem might be more related to the API level rather than a specific physical (or virtual) device and the difference in API levels between the device on which you recorded your Roboscript and the device on which you perform it. The best way to overcome any discrepancies is to simplify your Roboscript as much as possible. For example, as long as your target elements have unique identifying attributes (e.g., resource id, text, etc.), you can use just those to identify them, like this:

[
  {
    "eventType": "DELAYED_MESSAGE_POSTED",
    "delayTime": 7000
  },
  {
    "eventType": "VIEW_TEXT_CHANGED",
    "replacementText": "9876543210",
    "elementDescriptors": [
      {
        "resourceId": "com.devsbitwise.cryptonian:id/phone_number"
      }
    ]
  },
  {
    "eventType": "VIEW_CLICKED",
    "elementDescriptors": [
      {
        "resourceId": "com.devsbitwise.cryptonian:id/send_code"
      }
    ]
  },
  {
    "eventType": "DELAYED_MESSAGE_POSTED",
    "delayTime": 500
  },

Upvotes: 1

Related Questions