Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to never export specific files #671

Open
nicorum opened this issue Feb 20, 2024 · 2 comments
Open

Add option to never export specific files #671

nicorum opened this issue Feb 20, 2024 · 2 comments

Comments

@nicorum
Copy link

nicorum commented Feb 20, 2024

Currently, the only option is to export either all files or none.
This can be a problem if the files are several gigabytes in size or depend on other files in the same directory (structure). This makes the export feature unusable in these cases. Also, it is completely unnecessary to export the files if you know they will never change. A possible solution would be to mark these files with a "do not export" flag in the core file.

@olofk
Copy link
Owner

olofk commented Mar 7, 2024

I have several answers to this :)

  1. Depending on other files in the same directory is a bit of a problem for reproducible builds, which is why everything is copied to a clean build tree by default.
  2. If you switch over to the new Flow API, you will have built-in caching, which will only update files that have actually changed. Not all tools are yet available with the new API though (help wanted!)
  3. There is actually a hack that might help you. Any file that is specified using an absolute path name will not be copied. This is typically intended for things like simulation files that lives in your EDA tool's installation directory, but can be (ab)used for data files as well. It has the drawback that they will need to be in a fixed location though.
  4. Adding a "do not export" flag might still be an option, but I need to think of the implications of that.

@nicorum
Copy link
Author

nicorum commented Mar 12, 2024

Since I'm using an external IP, I can't change this behavior.
As this IP is installed at an absolute path solution three seems reasonable for my problem.

But that leads me to another problem with environment variables:
The IP's installation directory is defined by an environment variable.
Without the --resolve-env-vars-early argument, fusesoc cannot know if a path starting with $PATHTOIP is relative or absolute. This leads to two problems:

  1. The export function in the core.py currently tries to copy the files, which is impossible without resolving the environment variables. Leading into an RuntimeError. This should also apply when $PATHTOIP is an relative path.
  2. The "rel_root" gets prepended to $PATHTOIP while creating the EDAM file. Prepending a relative path to an absolute path breaks the path. The os.path.join function normally prevents this, but this is not possible when the absolute path is masked behind an environment variable.

A simple workaround would be to use the --resolve-env-vars-early argument, but I don't like the idea to be forced to use this argument.

I currently see multiple possible solutions:

  1. always resolve the environment variables to only check if the path is relative or absolute (bad idea)
  2. never export files that contain an environment variable
  3. add an flag or something similar to mark the path as absolute
    But all this solution have their downsides.

I think this whole topic is somewhat related to the discussion in issue #641.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants