Strange issue with too many histories in BEAM/DOSXYZnrc?

[livingag]

Hi all, we have an automated system for calculating linac treatment plans using BEAM/DOSXYZnrc from DICOM files. We have recently run into a weird issue I am having trouble figuring out: the system scales the number of histories to be simulated based on the treatment field area, and when a larger field is being simulated (femur and whole brain have been examples), it seems like something is getting flipped or mirrored, and the resulting dose distribution is completely incorrect. When I halve the number of histories used, everything looks OK.

Everything is calculated using an Ubuntu TORQUE cluster with 43 nodes, using the Varian IAEA phase space files. Available hard disk space on the master node is maybe a little low, but does changing the number of histories have an impact on disk space usage (I thought it wouldn't)?

Any ideas or questions would be welcomed as this is really perplexing me.

[crcrewso]

This sounds like an interesting application. What are the number of histories that work and that don't? Are you using an intermediate phase space or are you using a shared library?

[livingag]

~250m histories produces the abnormal results, and then 125m histories works fine. I haven't tested to figure out what the upper limit is on histories before this happens.

I am using a vendor supplied IAEA phase space scored just above the jaws, then simulating through SYNCJAWS and SYNCVMLC compiled as a shared library using source 20 in DOSXYZnrc.

[crcrewso]

2 more questions for you

1. What version of EGSnrc are you using
2. could you try recompiling with mcmodel=large?
   Once you recompile you'll have to remake and remake the libraries

[livingag]

We are using EGSnrc 2021 at the moment.

Will try recompiling with that flag tomorrow (end of the day here) and let you know how it goes.

(Apologies for making a new comment reply instead of replying to your message before and making a bit of a mess in the discussion)

[crcrewso]

I'm thinking about it again, if you're using the Varian IAEA phsp files how did you add them up? Are you only using 1 of the 50 sub-files? If so have you tried using a different one?

[livingag]

I combined a few (can't remember how many) of the 50 sub-files using addphsp - I will try using different ones

[crcrewso]

I had a huge problem adding all of them up. It took days and a lot of troubleshooting. I seem to remember some of the files were bad and I had to watch the output very carefully before using the combined file. I do remember that if I hit the end of the file, or where the addphsp started to produce erronious results it would start to have real issues

[ojalaj]

I have never had an issue downloading and concatenating all the Varian phsp files and using the combined phsp file as a source.

[ftessier]

Yes, let us know how that works. Previous discussions on -mdmodel include #723 and #982. It also depend on your OS, notably on macOS as per #905.

However, I don't see how your issue is related to memory, since it does work with fewer histories, and memory does not normally scale with the number of histories. Does this also happen if you use a simple source instead of a phase space file? If you can reproduce the problem with a simple source and a simple DOSXYZnrc phantom, we could try to reproduce the problem at our end.

[rtownson]

I don't think I've seen a dependence on the number of histories like this before.

This is just a stab in the dark - try reducing the number of batches to 1.

[rtownson]

If you are using a large recycling number, this could cause some wild dose effects. Make sure recycling is <10, unless you are using azimuthal particle redistribution like MCTWIST (not part of BEAMnrc by default).

[ftessier]

Yes, it would be useful to know how many particles are in the phsp file to begin with, and if recycling (or restarting!) is used. Although I am not sure how that would bias the end result. In my experience, restarting leads to a severe underestimate of uncertainty, and recycling leads to the proper limit in reducing the uncertainty. But I don't recall the dose itself being completely wrong on account of either (I mean provided there is at least a representative number of particles in the phsp file, which I presume is the case here).

I am curious now: @livingag, how many particles are in the phsp file, and have you tuned recycling accordingly?

[livingag]

Hi, thanks so much to everyone for the help and apologies for not responding sooner. My learning with BEAM/DOSXYZnrc has been mostly self-taught and a lot of fumbling around until things work, so am probably doing some things non-optimally (or just plain wrong).

There are roughly 100m particles in the phsp file, and I haven't done any tuning of recycling at all (it is just set to 0 currently). I have output restart data set to never and don't do any restarting. Photon splitting is set to 32 (from the manual).

Initially I was using source 9 and have only recently transitioned to using phsp files for a Varian TrueBeam model, so wasn't sure how big a phsp file is needed and (naively?) thought DOSXYZ would handle the recycling. I have been getting great agreement with the planning system (RayStation Monte Carlo) until now, so thought I was on the right track.

[rtownson]

Are you using DOSXYZ directly with the Varian phsp source? Or DOSXYZ with a phase-space output from BEAMnrc, which itself uses the Varian phsp source? Or DOSXYZ with source 9, and BEAMnrc with the Varian phsp? Just so we're on the same page.

If you are using the Varian phsp in BEAMnrc, have BEAMnrc output a phase-space and then take a look at it in beamdp - does the particle scatter plot look correct for the collimator setup you're using in BEAMnrc?

In the DOSXYZnrc output, check for messages about recycling and restarting. How many particles does it read from the phsp and how many are you simulating, in the case where the result looks bad?

Is there a chance you've accidentally turned on any of the particle redistribution options in BEAMnrc or DOSXYZnrc? `Phase-space data redistribution' in DOSXYZnrc would likely not be valid for your beam.

Fred's comment about restarting is not related to the output restart data option. He was asking about the phase-space restarting, meaning that your simulation asked for more particles that exist in the file, and the file started reading from the beginning again (it will print warning messages during the simulation).

[livingag]

I am using source 20 in DOSXYZ, so phase-space through a shared BEAM library. When changing to using the phase-space, I did as you suggested and generated a phase space at the bottom of the MLC, and everything looked ok to me but perhaps I will try that again.

When running the case that looks bad, it seems to restart the chunk after every second batch (EDIT: just checked and it is reporting the same warning when using less histories):

****** Running       568465 histories



      This simulation uses a phase space source.
      This run will use from particle      917409 to particle      1146760
      in the source file.



 BATCH #  TIME-ELAPSED  TOTAL CPUTIME  RATIO  TIME OF DAY  RNG pointers

     1          0.0            0.0      0.00    09:05:48   ixx jxx =   97  33 
     2         37.1           37.0      1.00    09:06:25   ixx jxx =   44  77 



 ***WARNING*** USED ALL PARTICLES FROM CHUNK IN SOURCE FILE!
               RESTARTING FROM FIRSTPARTICLE IN THIS CHUNK.


 *******************************************************************************

Here are the BEAM and DOSXYZ input files for the case that looks bad (as .txt files since GitHub doesn't like the .egsinp extension):

B840711_BEAM.txt
B840711_DOSXYZ.txt

Here is an example of the problem using the input files above, with the bad result with more histories on the top left (solid line profile) and the exact same simulation with just half the histories on the bottom left (dotted line profile). I have hidden the patient CT data but it is treating the left femur, with essentially two parallel opposed beams:

[livingag]

Turns out I had ISMOOTH enabled for some reason - is this able to be toggled in the GUI? I have only really been using the GUI for making input file templates, so not sure when I would have turned this on if it is not on by default.

Anyway re-running the plan with ISMOOTH set to 0 fixes the weird results.

[rtownson]

Glad you sorted it out! ISMOOTH is controlled in the GUI by Phase-space data redistribution when you select a phase-space source, but it's not in the GUI for source 20 (but it is still available as an option, when you manually edit the input file). I can see it being easily missed if you started with a phase-space source with it turned on, then changed to source 20 while working in the GUI.

[livingag]

Just out curiosity, what is the mechanism for this error only appearing when using more histories? Was it just not doing any phase-space data redistribution when I was using the lower number of histories?

[karl63]

I wrote my own C++ program using the dcmtk toolkit from "offis" about years ago that did the trick. In the meantime there are other posibilities like Slicer-3D that can read the dose from the 3ddose files and transform it. Sent with [Proton Mail](https://proton.me/mail/home) secure email.

…

On Tuesday, October 29th, 2024 at 10:03 PM, physicsmed ***@***.***> wrote: thank u ***@***.***(https://github.com/livingag) for the answer, but how you can put the results of DOSXYZnrc into a DICOM file? — Reply to this email directly, [view it on GitHub](#995 (reply in thread)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AJK7WO3YZAOZ63DCRQ242IDZ57Z27AVCNFSM6AAAAABPNIOKXKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMBZGI2TONQ). You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[karl63]

If you are not familiar with C++ or Python programming ,Have a look at this discussion here: https://discourse.slicer.org/t/converting-dosxyz-output-3ddose-to-dicom-rt-to-be-read-in-pinncale-and-eclipse/14499 you might be able to transform the 3ddose file in dicom without having to write code.