Merge pull request #454 from berndfinger/issue-385-new-detection-method

Issue 385 new detection method

roles/sap_install_media_detect/README.md

@@ -16,6 +16,27 @@ RAR files. See the comments and examples for the RAR file handling in `defaults/
 when the role was run, it will remain enabled. If the EPEL repo was not present, the associated GPG key will be removed and the EPEL repo
 will be disabled as the last task.
 
+## Execution Flow
+
+- At the beginning of the execution of the role, a new tool `sapfile` is pushed to a temporary directory on the managed node.
+- Also a package which contains a command for extracting and listing content of files of type `RAR` is installed.
+- The next step is to check if source and/or target directories exist. If role parameter `sap_install_media_detect_target_directory` is defined, files will later be copied from `sap_install_media_detect_source_directory`. This is the `remote_dir` case.
+- If the system on which the `sap_install_media_detect_source_directory` is not writable, the role would normally fail because one or both of the following conditions are not met:
+  - The SAPCAR EXE file is not executable.
+  - There are one or more `ZIP` or `RAR` files without extension.
+- In this `remote_dir` case, to make sure the role does not fail, it needs to be run first on the node on which `sap_install_media_detect_source_directory` is writable, with role parameter `sap_install_media_detect_file_server_only` set to `true` so the role will not perform and further file detection activities.
+- After the SAPCAR EXE file is executable and there are no more `ZIP` or `RAR` files without extension, the role can be called on a managed node where `sap_install_media_detect_source_directory` is not writable.
+- A new list of all files with the correct final file names will then be created, and for each of the files, the SAP file types are determined using the `sapfile` tool, either using the file names or - if this information is not sufficient - from information inside the file.
+- We then assert that there is at least (or exactly, depending on the file type) one file available for each of the `sap_install_media_detect_*` parameters. For example, if `sap_install_media_detect_kernel_db` is set to `saphana`, then there must be one SAP Kernel DB dependent file for SAP HANA.
+- In case of `remote_dir`, the next step is to copy all files from `sap_install_media_detect_source_directory` to `sap_install_media_detect_target_directory`.
+- Then we extract files which are configured in `sapfile` to be extracted, and copy or move files which are configured in `sapfile` to be copied or moved. Certain files like SAPCAR*.EXE and SAP Host Agent will be copied to two different directories.
+- Once all necessary files have been extracted and all files are copied or moved to where we want them to be, we are using the Ansible find module to identify the different file types by using file or directory name patterns.
+- The last step is to fill all required `sap_swpm` parameters from the result of the previous find step, and display all the variables.
+
+## Variables and Parameters
+
+See the file `defaults/main.yml`.
+
 ## Dependencies
 
 This role does not depend on any other Ansible Role.
@@ -25,9 +46,14 @@ This role does not depend on any other Ansible Role.
 With the following tags, the role can be called to perform certain activities only:
 - tag `sap_install_media_detect_rar_handling`: Only perform the tasks for enabling the listing and extracting of files of type `RAR`. This
   includes enabling and disabling the EPEL repo for RHEL systems, if desired.
-- tag `sap_install_media_detect_add_rar_extension`: Only add `.rar` to any files in `sap_install_media_detect_source_directory` which are of type `RAR` and have no ending. Needs to be used with tag `sap_install_media_detect_create_file_list`.
+- tag `sap_install_media_detect_add_file_extension`: Add file name extensions to any files in `sap_install_media_detect_source_directory` which are of type `RAR` or `ZIP` and have no ending. Needs to be used with tag `sap_install_media_detect_create_file_list_phase_1`.
 - tag `sap_install_media_detect_check_directories`: Find out if the directory `sap_install_media_detect_target_directory` or `sap_install_media_detect_source_directory` is writable.
-- tag `sap_install_media_detect_create_file_list`: Only create a list of all files in `sap_install_media_detect_source_directory`, and create a list of any files which have no ending and are of type `RAR`.
+- tag `sap_install_media_detect_provide_sapfile_utility `: Provides the sapfile utility on the managed node. This tool is required for determining the SAP file type.
+- tag `sap_install_media_detect_create_file_list_phase_1`: Create a list of all files in `sap_install_media_detect_source_directory`, and create a list of any files which have no ending and are of type `RAR`.
+- tag `sap_install_media_detect_create_file_list_phase_2`: Create a final list of all required files in `sap_install_media_detect_source_directory` or `sap_install_media_detect_target_directory` (if that one is defined)
+- tag `sap_install_media_detect_organize_files`: Copies all required files from `sap_install_media_detect_source_directory` or `sap_install_media_detect_target_directory` (if that one is defined) and extracts all required files into the target directories if specified by the output of the sapfile command.
+- tag `sap_install_media_detect_find_files_after_extraction`: Finds all required files after they have been extracted so the final variables can be filled in the next step.
+- tag `sap_install_media_detect_set_global_vars`: Set all final variables for later use by Ansible roles or tasks.
 
 ## License
 

roles/sap_install_media_detect/defaults/main.yml

@@ -16,36 +16,68 @@ sap_install_media_detect_use_rpm_key_module_for_removing_the_key: true
 # If this role is running on a system on which the SAP software is to be installed, set the following to false.
 sap_install_media_detect_file_server_only: false
 
+# Fully qualified path to the sapcar program. If not specified, the sapcar program (or a link with the name sapcar, pointing to
+# the actual SAPCAR*EXE program) is expected to be located in one of the PATH directories.
+#sap_install_media_detect_sapcar_path: '/usr/local/bin/sapcar'
+
+# Fully qualified path to the program for listing RAR files, including the argument for listing files. If not specified,
+# the lsar program (or a link with the name lsar, pointing to the actual lsar program) is expected to be located in one of
+# the PATH directories.
+# If sap_install_media_detect_rar_package is set to 'EPEL', this variable is not used.
+#sap_install_media_detect_rar_list: '/usr/bin/lsar'
+sap_install_media_detect_rar_list: '/usr/bin/unrar lb'
+
+# Fully qualified path to the program for extracting RAR files, including the argument for extracting files. If not specified,
+# the unar program (or a link with the name unar, pointing to the actual unar program) is expected to be located in one of
+# the PATH directories.
+# If sap_install_media_detect_rar_package is set to 'EPEL', this variable is not used.
+#sap_install_media_detect_rar_extract: '/usr/bin/unar'
+sap_install_media_detect_rar_extract: '/usr/bin/unrar x'
+
+# Fully qualified path to an additional argument to the program for extracting RAR files, for specifying the directory into
+# which the archive is to be extracted. Needs to be empty or start with a space character.
+# If sap_install_media_detect_rar_package is set to 'EPEL', this variable is not used.
+#sap_install_media_detect_rar_extract_argument: ' -o'
+sap_install_media_detect_rar_extract_directory_argument: ''
+
 # Directory where the SAP software is located
 sap_install_media_detect_source_directory: /software
 
 # Directory where the SAP software is located after the role is run, if different from sap_install_media_detect_source_directory
 #sap_install_media_detect_target_directory: /software_local
 
+# Create target directory if it does not yet exist. If set to false, perform a check only
+sap_install_media_detect_create_target_directory: true
+
 # If there are two files of the same RAR or ZIP type, one with and one without suffix, the following parameter will determine what
 # the role will do for such a file: skip the file renaming, fail, or overwrite the file with the suffix by the file without suffix
 sap_install_media_detect_rename_target_file_exists: 'skip'
 #sap_install_media_detect_rename_target_file_exists: 'fail'
 #sap_install_media_detect_rename_target_file_exists: 'overwrite'
 
-# local_dir, remote_dir (e.g. NFS, s3fuse)
-#sap_install_media_detect_source: local_dir
+# If you want the role to not extract archives which have the extract flag set, set the following parameter to `false`.
+sap_install_media_detect_extract_archives: true
+
+# If you want the role to not move or copy archive files to the `target_dir` subdirectories, set the following parameter to `false`.
+sap_install_media_detect_move_or_copy_archives: true
 
-# If the following parameter is set to true and the target directory for the extracted files (= *_extracted) already exists,
-# the extraction of the corresponding file will be skipped. If set to false, target directories will be removed initially,
-# forcing a fresh extraction, including re-creation of the target directory.
-sap_install_media_detect_skip_extraction_if_target_dir_exists: false
+# By default, the presence of at least one file for each file type according to the configured role parameters is asserted. Set the
+# following parameter to 'false' to skip this step.
+#sap_install_media_detect_assert_after_sapfile: false
 
 # saphana, sapase, sapmaxdb, oracledb, ibmdb2
-sap_install_media_detect_db: "saphana"
+#sap_install_media_detect_db: 'saphana'
+sap_install_media_detect_db: ''
 
 sap_install_media_detect_swpm: true
 sap_install_media_detect_hostagent: true
 sap_install_media_detect_igs: true
 sap_install_media_detect_kernel: true
+# saphana, sapase, sapmaxdb, oracledb, ibmdb2
+#sap_install_media_detect_kernel_db: 'sapmaxdb'
 sap_install_media_detect_webdisp: false
 
-# saps4hana, sapecc, sapecc_ides, sapbw4hana, sapnwas_abap, sapnwas_java
+# saps4hana, sapbw4hana, sapecc, sapecc_ides, sapnwas_abap, sapnwas_java, sapsolman_abap, sapsolman_java
 sap_install_media_detect_export:
 
 # e.g. /db_backup