Merge pull request #389 from zebrunner/develop

3.0.2

README.md

@@ -43,16 +43,16 @@ Feel free to support the development with a [**donation**](https://www.paypal.co
 * update `roles/devices/vars/main.yml` file according to the obligatory/optional comments inside.
   > Register all whitelisted Android and iOS devices with their udids!
 * Run ansible-playbook script to download the required components and set up udev rules:
-  ```
-  ansible-playbook -vvv -i hosts devices.yml
+  ```bash
+  ./zebrunner.sh ansible
   ```
   > To reregister the devices list only, you can use the following command:
-  ```
-  ansible-playbook -vvv -i hosts devices.yml --tag registerDevices
+  ```bash
+  ./zebrunner.sh ansible devices
   ```
   > To provide extra arguments including sudo permissions, you can use the below command:
-  ```
-  ansible-playbook -vvv -i hosts --user=USERNAME --extra-vars "ansible_sudo_pass=PSWD" devices.yml
+  ```bash
+  ./zebrunner.sh ansible --user=USERNAME --extra-vars "ansible_sudo_pass=PSWD"
   ```
  * Devices management script is deployed to /usr/local/bin/zebrunner-farm.
  * Udev rules with whitelisted devices are in /etc/udev/rules.d/90_mcloud.rules.
@@ -67,16 +67,16 @@ Feel free to support the development with a [**donation**](https://www.paypal.co
   > Register all whitelisted iOS devices (phones, tablets or TVes) with their udids!
   > Important! Only iOS devices supported on MacOS!
 * Run ansible-playbook script to download the required components and set up udev rules:
-  ```
-  ansible-playbook -vvv -i hosts mac-devices.yml
+  ```bash
+  ./zebrunner.sh ansible
   ```
   > To reregister the devices list only, you can use the following command:
-  ```
-  ansible-playbook -vvv -i hosts mac-devices.yml --tag registerDevices
+  ```bash
+  ./zebrunner.sh ansible devices
   ```
   > To provide extra arguments including sudo permissions, you can use the below command:
-  ```
-  ansible-playbook -vvv -i hosts --user=USERNAME --extra-vars "ansible_sudo_pass=PSWD" mac-devices.yml
+  ```bash
+  ./zebrunner.sh ansible --user=USERNAME --extra-vars "ansible_sudo_pass=PSWD"
   ```
  * Devices management script is deployed to /usr/local/bin/zebrunner-farm.
  * Whitelisted devices properties are in /usr/local/bin/mcloud-devices.txt.
@@ -107,19 +107,23 @@ Feel free to support the development with a [**donation**](https://www.paypal.co
 
 You need an Apple Developer account to sign in and build **WebDriverAgent**.
 
-1. Open **WebDriverAgent.xcodeproj** in Xcode.
-2. Ensure a team is selected before building the application. To do this, go to *Targets* and select each target (one at a time). There should be a field for assigning team certificates to the target.
-3. Remove your **WebDriverAgent** folder from *DerivedData* and run *Clean build folder* (just in case).
-4. Build the application by selecting the *WebDriverAgentRunner* target and build for *Generic iOS Device*. Run *Product -> Build for testing*. This will create a *Products/Debug-iphoneos* in the specified project directory.  
+1. Clone **WebDriverAgent** from source you prefer. We recommend our [forked repository](https://github.com/zebrunner/WebDriverAgent) with performance improvements.
+    ```bash
+   git clone https://github.com/zebrunner/WebDriverAgent.git
+   ```
+2. Open **WebDriverAgent.xcodeproj** in Xcode.
+3. Ensure a team is selected before building the application. To do this, go to *Targets* and select each target (one at a time). There should be a field for assigning team certificates to the target.
+4. Remove your **WebDriverAgent** folder from *DerivedData* and run *Clean build folder* (just in case).
+5. Build the application by selecting the *WebDriverAgentRunner* target and build for *Generic iOS Device*. Run *Product -> Build for testing*. This will create a *Products/Debug-iphoneos* in the specified project directory.  
  *Example*: **/Users/$USER/Library/Developer/Xcode/DerivedData/WebDriverAgent-dzxbpamuepiwamhdbyvyfkbecyer/Build/Products/Debug-iphoneos**
-5. Go to the "Products/Debug-iphoneos" directory and run:
+6. Go to the "Products/Debug-iphoneos" directory and run:
  **mkdir Payload**
-6. Copy the WebDriverAgentRunner-Runner.app to the Payload directory:
+7. Copy the WebDriverAgentRunner-Runner.app to the Payload directory:
  **cp -r WebDriverAgentRunner-Runner.app Payload**
-7. Finally, zip up the project as an *.ipa file:
+8. Finally, zip up the project as an *.ipa file:
  **zip -r WebDriverAgent.ipa ./Payload**
    > Make sure to specify relative `./Payload` to archive only Payload folder content 
-8. Share built ipa via WDA_FILE variable in roles/devices/vars/main.yml file.
+9. Share built ipa via WDA_FILE variable in roles/devices/vars/main.yml file.
    > to override WDA_FILE artifacts per each device use `wda_file` and `wda_bundleid` iOS device properties and re-execute ansible playbook.
 
 
@@ -152,6 +156,22 @@ Follow the below algorithm to identify any configuration issues with MCloud agen
   // artifacts uploader container:
   docker logs -f device-<Name>-<udid>-uploader
   ```
+* If you have any problems running ansible:
+  * Make sure you have sudo access and try to run ansible with sudo permissions.
+  * Try to rub ansible commands manually (`<devices_file>` name is `mac-devices` on macOS  or `devices` on Linux servers)
+
+  > To download the required components and set up udev rules:
+  ```
+  ansible-playbook -vvv -i hosts <devices_file>.yml
+  ```
+  > To reregister the devices list only, you can use the following command:
+  ```
+  ansible-playbook -vvv -i hosts <devices_file>.yml --tag registerDevices
+  ```
+  > To provide extra arguments including sudo permissions, you can use the below command:
+  ```
+  ansible-playbook -vvv -i hosts --user=USERNAME --extra-vars "ansible_sudo_pass=PSWD" <devices_file>.yml
+  ```
 
 ## Documentation and free support
 * [Zebrunner PRO](https://zebrunner.com)

roles/devices/templates/mcloud-devices.txt

@@ -1,3 +1,3 @@
 {% for device in devices %}
-{{ device.name }}|{{ device.os }}|{{ device.id }}|{{ device.adb_port }}|{{ device.min_port }}|{{ device.max_port }}|{{ device.proxy_port | default('0') }}|{{ device.appium_port }}|{{  device.wireless | default('false') }}|{{ device.wda_file | default('/dev/null')}}|{{ device.wda_bundleid | default('com.facebook.WebDriverAgentRunner.xctrunner')}}|{{ device.server_proxy_port | default('0')}}
+{{ device.id }}|{{ device.os }}|{{ device.name }}|{{ device.location | default('') }}|{{ device.appium_port }}|{{ device.adb_port }}|{{ device.proxy_port | default('0') }}|{{ device.server_proxy_port | default('0') }}|{{ device.min_port }}|{{ device.max_port }}|{{ device.wireless | default('false') }}|{{ device.wda_file | default('/dev/null') }}|{{ device.wda_bundleid | default('com.facebook.WebDriverAgentRunner.xctrunner') }}
 {% endfor %}

roles/devices/templates/zebrunner-farm

@@ -48,26 +48,30 @@ function create_containers() {
   local platform_name=$(cat ${devices} | grep "$udid" | cut -d '|' -f 2)
   #echo "platform_name: $platform_name"
 
-  local device_adb_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 4)
-  #echo "device_adb_port: $device_adb_port"
+  local location=$(cat ${devices} | grep "$udid" | cut -d '|' -f 4)
+  #echo "location: $location"
 
-  local stf_min_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 5)
-  #echo "stf_min_port: $stf_min_port"
+  local appium_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 5)
+  #echo "appium_port: $appium_port"
 
-  local stf_max_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 6)
-  #echo "stf_max_port: $stf_max_port"
+  local device_adb_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 6)
+  #echo "device_adb_port: $device_adb_port"
 
   local proxy_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 7)
   #echo "proxy_port: $proxy_port"
 
-  local appium_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 8)
-  #echo "appium_port: $appium_port"
+  local server_proxy_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 8)
+  #echo "server_proxy_port: $server_proxy_port"
+
+  local stf_min_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 9)
+  #echo "stf_min_port: $stf_min_port"
 
-  local wireless=$(cat ${devices} | grep "$udid" | cut -d '|' -f 9)
+  local stf_max_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 10)
+  #echo "stf_max_port: $stf_max_port"
+
+  local wireless=$(cat ${devices} | grep "$udid" | cut -d '|' -f 11)
   #echo "wireless: $wireless"
 
-  local server_proxy_port=$(cat ${devices} | grep "$udid" | cut -d '|' -f 12)
-  #echo "server_proxy_port: $server_proxy_port"
 
   # without 'block_global=false' mitmproxy does not want to accept requests from devices
   # --set confdir allows to provide custom folder with CA Authority certificates
@@ -150,7 +154,7 @@ function create_containers() {
       -e STF_PROVIDER_DEVICE_NAME="${device_name}" \
       -e DEVICE_UDID="$STF_DEVICE_UDID" \
       -e STF_PROVIDER_HOST={{ STF_PROVIDER_HOST }} \
-      -e STF_PROVIDER_NAME="{{ STF_PROVIDER_NAME }}" \
+      -e STF_PROVIDER_NAME="{{ STF_PROVIDER_NAME }}${location:+_$location}" \
       -e STF_PROVIDER_MIN_PORT=$stf_min_port \
       -e STF_PROVIDER_MAX_PORT=$stf_max_port \
       -p $stf_min_port-$stf_max_port:$stf_min_port-$stf_max_port \
@@ -219,7 +223,7 @@ function create_containers() {
 
     #TODO: comment echoing wda related detailes before release
     local wdaIpaPath={{ WDA_FILE }}
-    local deviceWdaIpaPath=$(cat ${devices} | grep "$udid" | cut -d '|' -f 10)
+    local deviceWdaIpaPath=$(cat ${devices} | grep "$udid" | cut -d '|' -f 12)
     if [ ! -z $deviceWdaIpaPath ] && [ ! "$deviceWdaIpaPath" == "/dev/null" ]; then
       wdaIpaPath=$deviceWdaIpaPath
     fi
@@ -230,7 +234,7 @@ function create_containers() {
     echo "wdaIpaPath: $wdaIpaPath"
 
     local wdaBundleId={{ WDA_BUNDLEID }}
-    local deviceWdaBundleId=$(cat ${devices} | grep "$udid" | cut -d '|' -f 11)
+    local deviceWdaBundleId=$(cat ${devices} | grep "$udid" | cut -d '|' -f 13)
     if [ ! -z $deviceWdaBundleId ]; then
       wdaBundleId=$deviceWdaBundleId
     fi
@@ -277,7 +281,7 @@ function create_containers() {
       -e STF_PROVIDER_DEVICE_NAME="${device_name}" \
       -e DEVICE_UDID="${udid}" \
       -e STF_PROVIDER_HOST={{ STF_PROVIDER_HOST }} \
-      -e STF_PROVIDER_NAME="{{ STF_PROVIDER_NAME }}" \
+      -e STF_PROVIDER_NAME="{{ STF_PROVIDER_NAME }}${location:+_$location}" \
       -e STF_PROVIDER_MIN_PORT=$stf_min_port \
       -e STF_PROVIDER_MAX_PORT=$stf_max_port \
       -p $stf_min_port-$stf_max_port:$stf_min_port-$stf_max_port \
@@ -348,9 +352,9 @@ function stop() {
   # if $device is empty do stop for all connected devices otherwise stop only filtered
   if [ ! -z $filter_device ] && [ "$filter_device" != "ios" ] && [ "$filter_device" != "android" ]; then
     echo stopping containers for $filter_device...
-    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
+    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
     local device_name_underscored=${device_name// /_}
-    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
+    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
 
     if [ -z "$device_name" ] || [ -z $udid ]; then
       echo_warning "Unable to find device by \"$filter_device\"!"
@@ -361,13 +365,13 @@ function stop() {
   else
     # as no device filter provided continue with stopping all device containers
     while read -r line; do
-      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
+      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
       #echo "device_name: $device_name"
       local device_name_underscored=${device_name// /_}
 
-      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
+      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
       #echo "udid: $udid"
-      local wireless=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 9)
+      local wireless=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 11)
 
       if [ -L "/dev/device-${device_name}-${udid}" ] || [ "$wireless" == "True" ]; then
         stop_containers ${device_name_underscored} ${udid}
@@ -402,9 +406,9 @@ function down() {
   # if $device is empty do rm for all connected devices otherwise rm only filtered
   if [ ! -z $filter_device ] && [ "$filter_device" != "ios" ] && [ "$filter_device" != "android" ]; then
     #echo removing containers for $filter_device...
-    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
+    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
     local device_name_underscored=${device_name// /_}
-    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
+    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
 
     if [ -z "$device_name" ] || [ -z $udid ]; then
       echo_warning "Unable to find device by \"$filter_device\"!"
@@ -414,11 +418,11 @@ function down() {
     remove_containers ${device_name_underscored} ${udid}
   else
     while read -r line; do
-      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
+      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
       #echo "device_name: $device_name"
       local device_name_underscored=${device_name// /_}
 
-      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
+      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
       #echo "udid: $udid"
       remove_containers ${device_name_underscored} ${udid}
     done < ${devices}
@@ -529,7 +533,7 @@ function verify_containers() {
   local udid=$2
   #echo "udid: $udid"
 
-  local wireless=$(cat ${devices} | grep "$udid" | cut -d '|' -f 9)
+  local wireless=$(cat ${devices} | grep "$udid" | cut -d '|' -f 11)
 
   local container=device-$device-$udid
   if [ ! -L "/dev/$container" ] && [ "$wireless" != "True" ]; then
@@ -574,11 +578,11 @@ function verify_containers() {
       connectorRestarts="$(docker inspect --format='{{ '{{' }}.RestartCount{{ '}}' }}' $container-connector)"
     fi
 
-    echo -e ",Container,State,Status,Uptime,Restarts,
-,---------,---------,---------,---------,---------,
-,Connector,$connectorState,$connectorStatus,$connectorUptime,$connectorRestarts,
-,Appium,$appiumState,$appiumStatus,$appiumUptime,$appiumRestarts,
-,STF,$stfState,$stfStatus,$stfUptime,$stfRestarts," | column -t -s ',' -o ' | ' -L
+    echo -e ",| Container,| State,| Status,| Uptime,| Restarts,|
+,| ---------,| ---------,| ---------,| ---------,| ---------,|
+,| Connector,| $connectorState,| $connectorStatus,| $connectorUptime,| $connectorRestarts,|
+,| Appium,| $appiumState,| $appiumStatus,| $appiumUptime,| $appiumRestarts,|
+,| STF,| $stfState,| $stfStatus,| $stfUptime,| $stfRestarts,|" | column -t -s ','
 
   fi
   echo -e "==============================================================\n"
@@ -590,9 +594,9 @@ function start() {
 
   # if $device is empty do start for all connected devices otherwise start only filtered
   if [ ! -z $filter_device ] && [ "$filter_device" != "ios" ] && [ "$filter_device" != "android" ]; then
-    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
+    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
     local device_name_underscored=${device_name// /_}
-    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
+    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
 
     if [ -z "$device_name" ] || [ -z $udid ]; then
       echo_warning "Unable to find device by \"$filter_device\"!"
@@ -603,13 +607,13 @@ function start() {
     echo -e "\n-----------------------------------------------------------------"
   else
     while read -r line; do
-      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
+      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
       #echo "device_name: $device_name"
       local device_name_underscored=${device_name// /_}
 
-      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
+      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
       #echo "udid: $udid"
-      local wireless=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 9)
+      local wireless=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 11)
       if [ -L "/dev/device-${device_name}-${udid}" ] || [ "$wireless" == "True" ]; then
         create_containers "${device_name}" "${udid}" "${device_name_underscored}" && start_containers "${device_name}" "${udid}" "${device_name_underscored}"
         echo -e "\n-----------------------------------------------------------------"
@@ -643,18 +647,18 @@ function status() {
 
   # if $device is empty do start for all connected devices otherwise start only filtered
   if [ ! -z $filter_device ] && [ "$filter_device" != "ios" ] && [ "$filter_device" != "android" ]; then
-    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
+    local device_name=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
     #echo "device_name: $device_name"
     local device_name_underscored=${device_name// /_}
-    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 3)
+    local udid=$(cat ${devices} | grep "$filter_device" | cut -d '|' -f 1)
     #echo "udid: $udid"
     verify_containers "${device_name_underscored}" "${udid}"
   else
     while read -r line; do
-      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
+      local device_name=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
       #echo "device_name: $device_name"
       local device_name_underscored=${device_name// /_}
-      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 3)
+      local udid=$(echo ${line} | grep "$filter_device" | cut -d '|' -f 1)
       if [ -z $udid ]; then
         continue
       fi
@@ -680,15 +684,31 @@ function echo_help() {
   echo "
       Usage: zebrunner-farm [option]
       Arguments:
-          status         Status of MCloud Agent and each whitelisted device
-          start [udid]   Start devices containers or exact device by udid
-          stop [udid]    Stop and keep devices containers or exact device by udid
-          restart [udid] Restart all devices containers or exact device by udid
-          down [udid]    Stop and remove devices containers"
+          status               Status of MCloud Agent and each whitelisted device
+          start [udid]         Start devices containers or exact device by udid
+          stop [udid]          Stop and keep devices containers or exact device by udid
+          restart [udid]       Restart all devices containers or exact device by udid
+          down [udid]          Stop and remove devices containers
+          ansible ['devices']  Run ansible-playbook script with custom or predefined args"
   echo_telegram
   exit 0
 }
 
+# IMPORTANT! In case of any changes please copy them in ./zebrunner.sh !
+function ansible() {
+  if [[ "$1" == "" ]]; then
+    arg="devices.yml"
+  elif [[ "$1" == "devices" ]]; then
+    arg="devices.yml --tag registerDevices"
+  else
+    arg="$@ devices.yml"
+  fi
+
+  echo "ansible-playbook -vvv -i hosts $arg"
+  echo "*******************************************************************"
+  ansible-playbook -vvv -i hosts $arg
+}
+
 #CONTAINERS=( `docker ps --format '{{ '{{' }}.Names{{ '}}' }}' | grep device | grep -v appium | cut -d '-' -f 2-` )
 #echo CONTAINERS: $CONTAINERS
 
@@ -724,6 +744,9 @@ restart)
 down)
   down $2
   ;;
+ansible)
+  ansible "${@:2}"
+  ;;
 shutdown)
   shutdown
   ;;