Memory error and using SuperLU

[wavemixer]

I am trying to inject a mode into a very wide (15um) and thin waveguide (100nm) so the grid size is fairly fine (10nm). However get the mode solver crashes with frequency memory errors when SuperLU is called. Is there a solution besides using a workstation with more memory?

...
~\Anaconda3\lib\site-packages\scipy\sparse\linalg\dsolve\linsolve.py in splu(A, permc_spec, diag_pivot_thresh, relax, panel_size, options)
335 _options["SymmetricMode"] = True
336
--> 337 return _superlu.gstrf(N, A.nnz, A.data, A.indices, A.indptr,
338 csc_construct_func=csc_construct_func,
339 ilu=False, options=_options)

MemoryError:

[wavemixer]

I tried running the same script on a more powerful workstation and got validation errors. If you have seen this and there is a quick fix, I would appreciate the help.

[momchil-flex]

This error just means that you have to give a run_time to your Simulation, even though it is not used in the mode solver. We added this check so that users don't end up submitting simulations with 0 run time by mistake. You can just put 1e-12 for example.

[wavemixer]

That fixed the validation error on Simulation. But I picked up one on the mode solver.

[wavemixer]

This is only an issue for the virtual machine at a server farm. This has never popped up on my personal/work machines. The only other difference is the version of Jupyter NB.

[momchil-flex]

You are just using two different version of tidy3d, the older one being on your machine. We modified the mode API slightly, and the error basically tells you what you need to do: now you supply a whole list of freqs, and the mode_spec also goes in the initialization of the solver. I would recommend upgrading your local tidy3d installation too (pip install --upgrade tidy3d-beta). Our docs are always up-to-date with the latest version, so you can consult those too when something like this pops up.

[wavemixer]

Thanks!

[momchil-flex]

One reason why you are hitting a memory error could be that scipy is trying to multi-thread. One thing you could try is explicitly turning that off. How to do that exactly depends on your machine and the scipy installation, but you can try this, which should cover almost all cases:

import os
for env_var in [
    "OPENBLAS_NUM_THREADS",
    "MKL_NUM_THREADS",
    "OMP_NUM_THREADS",
    "NUMEXPR_NUM_THREADS",
]:
    try:
        os.environ[env_var] = "1"
    except KeyError:
        pass

You have to do this before importing tidy3d.

[wavemixer]

This seemed to have worked, but the mode solver run time shot up.

[momchil-flex]

Well, you can try setting it to 2 or 4 and find a balance of performance and memory. But fundamentally, it's a tough computation. You may want to wait a little for the nonuniform grid that I mention in my other comment.

[momchil-flex]

Note that we are also preparing a release in which you can use an automatically generated non-uniform grid. This could significantly reduce the total number of grid points in your problem, as you can have a fine grid around the waveguide and a coarse one away from it. This should be out next week.