Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an example how to measure (code coverage) profiling data and save it on the host #211

Open
briansmith opened this issue Feb 21, 2024 · 0 comments

Comments

@briansmith
Copy link

I would like to use cargo-dinghy in GitHub actions to measure code coverage for iOS/TVOS/WatchOS builds. I use Rust's source-based code coverage measurement currently. This mechanism requires rebuilding the project with specific RUSTFLAGS, which cause the compiler to add instrumentation to the executable to measure how many times each branch is taken. When the executable terminates, it writes a file containing these counts. Then we take that file, post-process it, and send it to codecov.io or some other service. That service then aggregates the code coverage measurements with all the other ones gathered for that CI run. This way, we get a complete summary of all the code coverage across all targets.

If somebody has already hacked together an unofficial solution for this, I would appreciate a link.

Some challenges I see:

  • Similar to Benchmarking errors due to insufficient permissions for using Criterion  #81 regarding benchmarking with criterion.rs, which also writes (and reads) files, we need a way to tell the profile file writer the path to a writable (on the device) directory, so that it can successfully serialize the profiling data, and we need a way to copy that profiling data from the device back into the host, so that we can submit it to the code coverage analysis service.
  • Similar to How to build with non-default crate features? #129, we need to make sure we control all the settings of the build, in particular RUSTFLAGS (and other environment variables like CFLAGS) that dinghy will pass to cargo {build,test}.

Previously, for Android, and not using dinghy, I have done similar, with test input files, using adb to copy files to/from the device. In the long term, it would be nice if dinghy could abstract away the file I/O so that we wouldn't have to write target-OS-specific scripts. However, in the short term, demonstrating how to do it with an example that uses each individual tool (adb for Android, iOS's equivalent) would be a very useful starting point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant