Add tree option for FileSource2 AddFile

Add a new feature of FileSource2 that can specify each individual input
file with or without tree option.

README.md

@@ -77,3 +77,4 @@ source util/clang-format-all.sh
 
 * [Static analyzer using Clang-tidy](config/clang_tidy/README.md)
 * [CMake build system for R3BRoot](doc/cmake_usage.md)
+* [How to use an unmerged pull request](doc/git_usage.md#fetch-the-update-from-an-unmerged-pull-request-pr)

doc/git_usage.md

@@ -0,0 +1,187 @@
+## Fetch the update from an unmerged pull request (PR)
+
+More often than not, people need to use new features from a pull request that is not yet merged to the dev branch. In this case, the new features can still be added to your local repository with following steps:
+
+### Step 1: Identify the remote and branch name
+
+Go to the [R3BRoot pull request](https://github.com/R3BRootGroup/R3BRoot/pulls) webpage and find the pull request that has the new features you need. For example:
+
+![git_pr](pics/git_pr_example.png)
+
+In the example above, you could see the pull request is made from `YanzhaoW:edwin_filesource_update`. Here the user who made the PR has the account name "YanzhaoW". The branch, which is used to create the PR, is called "edwin_filesource_update". His remote repository can be checked by clicking this `YanzhaoW:edwin_filesource_update` and you will be redirected to the webpage of his remote repository. As for the example above, the remote url can be check by clicking the green button and choose the `HTTPS` tab:
+
+<img src="pics/git_green_button.png" width="400">
+
+In this case, the remote url is `https://github.com/YanzhaoW/R3BRoot.git`.
+
+### Step 2: Add the remote
+
+First, go to your local repository and check all your existing avaliable remote urls:
+
+```bash
+git remote -v
+```
+
+the output could be:
+
+```text
+origin  [email protected]:YanzhaoW/R3BRoot.git (fetch)
+origin  [email protected]:YanzhaoW/R3BRoot.git (push)
+pulrich [email protected]:P-Ulrich/R3BRoot_Fork.git (fetch)
+pulrich [email protected]:P-Ulrich/R3BRoot_Fork.git (push)
+slonzen https://github.com/SimonL2718/R3BRoot.git (fetch)
+slonzen https://github.com/SimonL2718/R3BRoot.git (push)
+upstream        https://github.com/R3BRootGroup/R3BRoot.git (fetch)
+upstream        https://github.com/R3BRootGroup/R3BRoot.git (push)
+
+```
+
+Here each remote url is attached to an unique name, called "remote name". One special url, namely `https://github.com/R3BRootGroup/R3BRoot.git`, represents the git repository of R3BRoot, which everyone pushes the updates to. Thus, when you need to update your dev branch, please use the remote name corresponding to this remote url. In the case above, the remote name is "upstream".
+
+If the remote url from the step 1 already exists, go to step 3. If you can't find the remote url from the step 1, please add it with:
+
+```bash
+git remote add [remote name] [remote url]
+```
+
+For example
+
+```bash
+git remote add yanzhao https://github.com/YanzhaoW/R3BRoot.git
+```
+
+You can use a different remote name for the new remote url. But please make sure it's different from all existing remote names. Once this is done, check the remotes again with `git remote -v` to make sure the new remote is added correclty.
+
+### Step 3: Update the dev branch
+
+Once the new remote name is added, first make sure you are in a clean git status:
+
+```bash
+git status
+```
+
+you should expect an message with "nothing to commit, working tree clean". If not, either commit the changes or reset to last commit (ATTENTION: if you use reset, all the uncommitted changes would be lost):
+
+```bash
+git reset --hard HEAD
+```
+
+After git status is clean, switch to dev branch and update it from the upstream
+
+```bash
+git checkout dev
+git pull --rebase upstream dev
+```
+
+Here the name "upstream" is the remote name that points to the remote url `https://github.com/R3BRootGroup/R3BRoot`. Change "upstream" to the corresponding name if you have a different situation.
+
+### Step 4: Fetch the update from the remote
+
+After dev branch is updated, now you need to fetch the update from the pull request.
+
+```bash
+git fetch [remote name] [branch name]
+```
+
+Here the `[remote name]` is the name you used/found in step 2 and `[branch name]` is the branch name you identified in step 1.
+
+### Step 5: Checkout
+
+After the update is fetched, check the new branch with:
+
+```bash
+git checkout [banch name]
+```
+
+Here, again, `[branch name]` is the branch name you identified in step 1.
+
+### Step 6: Recompile the program
+
+Recompile the program with:
+
+```bash
+cmake --build ./build -- -j${number_of_threads}
+source build/config.sh
+```
+
+### How to apply the changes on a new branch
+
+First, make sure you are in the branch that contains the changes and git status is clean. Then to apply your changes on another branch, use:
+
+```bash
+git rebase [another branch]
+```
+
+It's likely that you encounter some "conflicts" that must be resolved before moving forward. If this happens, please refer to [Resolving merge conflicts after a Git rebase](https://docs.github.com/en/get-started/using-git/resolving-merge-conflicts-after-a-git-rebase) for the instruction.
+
+### How to pull the latest version of the pull request branch
+
+To pull the latest version, use
+
+```bash
+git pull --rebase [remote name] [branch name]
+```
+
+Note that this may fail if the person who made the PR `force push`ed the update to his remote. In such case, first go to the dev branch and delete the remote branch:
+
+```bash
+git checkout dev
+git branch -D remote_branch_name
+```
+
+then start it over from step 4.
+
+## How to make your local git repository in a clean state
+
+Whenever you need to pull the updates from the dev branch or other people's branch, your git repository must be in a **clean state**:
+
+```bash
+git status
+```
+
+if your state is clean, you should get a message "nothing to commit, working tree clean". If not, use one of the following methods.
+
+### Method 1: commit to your own branch
+
+#### step 1: Create new branch if necessary
+
+First make sure you are not in the dev or master branch (which can be checked by `git branch`). If so, create a new branch for your changes:
+
+```bash
+git checkout -b [branch name]
+```
+Here `[branch name]` is can any name. Once on the new branch, commit all the changes to the new branch:
+
+```bash
+git add -A
+git commit -m "any message you want to write for the branch"
+```
+
+After this is done, go back to the dev branch:
+
+```bash
+git checkout dev
+```
+
+### Method 2: Reset back to the latest commit
+
+If you don't need any uncommitted changes, you could reset it back to the latest head of the current branch:
+
+```bash
+git reset --hard HEAD
+```
+Be aware that this will erase any uncommitted change permanently.
+
+### Method 3: Stash the change
+
+You could first put all the changes into a temporary stash with
+
+```bash
+git stash
+```
+
+After you completed all the actions, to get your changes back:
+
+```bash
+git stash pop
+```

neuland/executables/neulandAna.cxx

@@ -1,4 +1,3 @@
-#include "FairFileSource.h"
 #include "FairParRootFileIo.h"
 #include "FairRootFileSink.h"
 #include "FairRunAna.h"
@@ -12,7 +11,6 @@
 #include "R3BNeulandDigitizer.h"
 #include "R3BNeulandHitMon.h"
 #include "R3BProgramOptions.h"
-#include "TRandom3.h"
 #include "TStopwatch.h"
 #include <TObjString.h>
 #include <boost/program_options.hpp>
@@ -42,8 +40,10 @@ auto main(int argc, const char** argv) -> int
         programOptions.Create_Option<std::string>("paddle", R"(set the paddle name. e.g. "neuland")", "neuland");
     auto channelName =
         programOptions.Create_Option<std::string>("channel", R"(set the channel name. e.g. "tamex")", "tacquila");
-    auto simuFileName =
-        programOptions.Create_Option<std::string>("simuFile", "set the filename of simulation input", "simu.root");
+    auto simuFileName = programOptions.Create_Option<std::string>(
+        "simuFile", "set the filename of simulation input generated by R3BRoot", "simu.root");
+    auto simuTreeFileName =
+        programOptions.Create_Option<std::string>("simuTreeFile", "set the filename of simulation tree-only input", "");
     auto paraFileName =
         programOptions.Create_Option<std::string>("paraFile", "set the filename of parameter sink", "para.root");
     auto paraFileName2 =
@@ -52,6 +52,7 @@ auto main(int argc, const char** argv) -> int
         programOptions.Create_Option<std::string>("digiFile", "set the filename of digitization output", "digi.root");
     auto logLevel = programOptions.Create_Option<std::string>("logLevel,v", "set log level of fairlog", "error");
     auto eventNum = programOptions.Create_Option<int>("eventNum,n", "set total event number", 0);
+    auto run_id = programOptions.Create_Option<int>("run-id", "set the run ID (Only needed if reading only trees)", -1);
     auto hitLevelPar =
         programOptions.Create_Option<std::string>("hitLevelPar", "set the name of hit level parameter if needed.", "");
 
@@ -108,7 +109,18 @@ auto main(int argc, const char** argv) -> int
     FairLogger::GetLogger()->SetLogScreenLevel(logLevel->value().c_str());
 
     auto run = std::make_unique<FairRunAna>();
-    auto filesource = std::make_unique<R3BFileSource2>(simuFileName->value().c_str());
+    auto fairroot_input_files = R3B::GetFilesFromRegex(simuFileName->value());
+    auto tree_input_files = R3B::GetFilesFromRegex(simuTreeFileName->value());
+    auto filesource = std::make_unique<R3BFileSource2>();
+    if (run_id->value() >= 0)
+    {
+        filesource->SetInitRunID(run_id->value());
+        R3BLOG(info, fmt::format("Filesource2: Set to run id {}", run_id->value()));
+    }
+
+    filesource->AddFile(std::move(tree_input_files), true);
+    filesource->AddFile(std::move(fairroot_input_files), false);
+
     auto filesink = std::make_unique<FairRootFileSink>(digiFileName->value().c_str());
     run->SetSource(filesource.release());
     run->SetSink(filesink.release());

r3bbase/CMakeLists.txt

@@ -68,7 +68,7 @@ add_library_with_dictionary(
 
 if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
     message(STATUS "Linking -lstdc++fs due to the old version of gcc")
-    target_link_libraries(R3BBase PRIVATE -lstdc++fs)
+    target_link_libraries(R3BBase PUBLIC -lstdc++fs)
 endif()
 
 if("${FairRoot_VERSION}" VERSION_LESS 19.0.0)

r3bbase/R3BFileSource2.cxx

@@ -12,6 +12,7 @@
  ******************************************************************************/
 
 #include "R3BFileSource2.h"
+
 #include "R3BException.h"
 #include "R3BLogger.h"
 #include <FairEventHeader.h>
@@ -221,16 +222,16 @@ void R3BEventProgressPrinter::Print(uint64_t event_num, double speed_per_ms)
     std::cout << std::flush;
 }
 
-auto R3BInputRootFiles::AddFileName(std::string fileName) -> std::optional<std::string>
+auto R3BInputRootFiles::AddFileName(std::string fileName, bool is_tree_file) -> std::optional<std::string>
 {
     auto const msg = fmt::format("Adding {} to file source\n", fileName);
     R3BLOG(info, msg);
     if (fileNames_.empty())
     {
-        Intitialize(fileName);
+        Intitialize(fileName, is_tree_file);
         register_branch_name();
     }
-    if (!ValidateFile(fileName))
+    if (!ValidateFile(fileName, is_tree_file))
     {
         return fileName;
     }
@@ -289,11 +290,11 @@ auto R3BInputRootFiles::ExtractMainFolder(TFile* rootFile) -> std::optional<TKey
     return GetDataFromAnyFolder(rootFile, folderNames);
 }
 
-auto R3BInputRootFiles::ValidateFile(const std::string& filename) -> bool
+auto R3BInputRootFiles::ValidateFile(const std::string& filename, bool is_tree_file) -> bool
 {
     auto rootFile = R3B::make_rootfile(filename.c_str());
 
-    if (is_tree_file_)
+    if (is_tree_file)
     {
         if (!is_friend_)
         {
@@ -334,11 +335,11 @@ auto R3BInputRootFiles::ExtractRunId(TFile* rootFile) -> std::optional<uint>
     return runID;
 }
 
-void R3BInputRootFiles::Intitialize(std::string_view filename)
+void R3BInputRootFiles::Intitialize(std::string_view filename, bool is_tree_file)
 {
     auto file = R3B::make_rootfile(filename.data());
 
-    if (is_tree_file_)
+    if (is_tree_file)
     {
         branchList_ = GetBranchListFromTree(file.get(), treeName_);
         return;
@@ -424,9 +425,9 @@ R3BFileSource2::R3BFileSource2()
 {
 }
 
-void R3BFileSource2::AddFile(std::string fileName)
+void R3BFileSource2::AddFile(std::string file_name, bool is_tree_file)
 {
-    if (auto const res = inputDataFiles_.AddFileName(std::move(fileName)); res.has_value())
+    if (auto const res = inputDataFiles_.AddFileName(std::move(file_name), is_tree_file); res.has_value())
     {
         if (not dataFileNames_.empty())
         {
@@ -438,16 +439,32 @@ void R3BFileSource2::AddFile(std::string fileName)
         }
         else
         {
-            R3BLOG(error, fmt::format("Failed to add the first root file {:?}", fileName));
+            R3BLOG(error, fmt::format("Failed to add the first root file {:?}", file_name));
         }
     }
-    dataFileNames_.emplace_back(fileName);
+    dataFileNames_.emplace_back(file_name);
+}
+
+void R3BFileSource2::AddFile(std::vector<std::string> file_names, bool is_tree_file)
+{
+    for (auto& file_name : file_names)
+    {
+        AddFile(std::move(file_name), is_tree_file);
+    }
+}
+
+void R3BFileSource2::AddFriend(std::vector<std::string> file_names, bool is_tree_file)
+{
+    for (auto& file_name : file_names)
+    {
+        AddFriend(std::move(file_name), is_tree_file);
+    }
 }
 
-void R3BFileSource2::AddFriend(std::string_view fileName)
+void R3BFileSource2::AddFriend(std::string file_name, bool is_tree_file)
 {
     //
-    auto rootfile = R3B::make_rootfile(fileName.data());
+    auto rootfile = R3B::make_rootfile(file_name.c_str());
     auto friendGroup = std::find_if(inputFriendFiles_.begin(),
                                     inputFriendFiles_.end(),
                                     [&rootfile](const auto& friends)
@@ -460,7 +477,7 @@ void R3BFileSource2::AddFriend(std::string_view fileName)
         friendGroup = --inputFriendFiles_.end();
         friendGroup->SetTitle(fmt::format("FriendTree_{}", inputFriendFiles_.size()));
     }
-    auto res = friendGroup->AddFileName(std::string{ fileName });
+    auto res = friendGroup->AddFileName(file_name, is_tree_file);
     if (res.has_value())
     {
         R3BLOG(error,
@@ -471,12 +488,17 @@ void R3BFileSource2::AddFriend(std::string_view fileName)
     else
     {
         // TODO: really need it?
-        friendFileNames_.emplace_back(fileName);
+        friendFileNames_.emplace_back(std::move(file_name));
     }
 }
 
 Bool_t R3BFileSource2::Init()
 {
+    if (inputDataFiles_.is_empty())
+    {
+        throw R3B::logic_error{ "No input file available!" };
+    }
+
     inputDataFiles_.RegisterTo(FairRootManager::Instance());
 
     for (auto& friendGroup : inputFriendFiles_)
@@ -509,7 +531,7 @@ void R3BFileSource2::FillEventHeader(FairEventHeader* evtHeader)
     if (init_runID != GetRunId())
     {
         R3BLOG(
-            error,
+            warn,
             fmt::format("runID {} being set is different from the runID {} in the data file!", GetRunId(), init_runID));
     }
     SetRunId(init_runID); // NOLINT