Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optionally specify version in exports TASK-1244 #325

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rgraber
Copy link

@rgraber rgraber commented Dec 17, 2024

Adds an optional version_uid parameter to export methods that, if passed, will parse all submissions under the specified version. This allows the export to pick up submissions that were edited after the fact to include answers to new questions that may have been added in later versions.
Increases the minor version for form's sake, even though we don't actually have a published package.

👀 Preview steps

Testing this locally is a bit of a pain, but if you really want to you can

  1. clone this repo into your local copy of kpi
  2. update kpi's requirements.in to use -e /container/path/to/kpi/formpack instead of the git address currently there
  3. update kpi/models/import_export_task.py:872-887 with
        with self.result.storage.open(absolute_filepath, 'wb') as output_file:
            if export_type == 'csv':
                for line in export.to_csv(submission_stream, version_uid=self.asset.latest_deployed_version_uid):
                    output_file.write((line + '\r\n').encode('utf-8'))
            elif export_type == 'geojson':
                for line in export.to_geojson(
                    submission_stream, version_uid=self.asset.latest_deployed_version_uid, flatten=flatten
                ):
                    output_file.write(line.encode('utf-8'))
            elif export_type == 'xls':
                # XLSX export actually requires a filename (limitation of
                # pyexcelerate?)
                with tempfile.NamedTemporaryFile(
                        prefix='export_xlsx', mode='rb'
                ) as xlsx_output_file:
                    export.to_xlsx(xlsx_output_file.name, submission_stream, version_uid=self.asset.latest_deployed_version_uid)
  1. start up a kpi container
  2. pip-compile dependencies/pip/requirements.in && pip-compile dependencies/pip/dev_requirements.in
  3. pip install -r dependencies/pip/dev_requirements.txt
  4. Create a new survey with 1 question
  5. Deploy and add a few submissions
  6. Update the survey to add an additional question and redeploy
  7. Update some of the old submissions with an answer to the new question
  8. Export the data to csv, xlsx, or geojson
  9. 🟢 You should see all answers, including backfilled ones

@rgraber rgraber changed the title feat: include backfilled answers in exports feat: include backfilled answers in exports TASK-1244 Dec 17, 2024
@rgraber rgraber changed the title feat: include backfilled answers in exports TASK-1244 feat: optionally specify version in exports TASK-1244 Dec 17, 2024
@coveralls
Copy link

coveralls commented Dec 17, 2024

Coverage Status

coverage: 86.669% (+0.005%) from 86.664%
when pulling 7c0a6bc on rsgraber/TASK-1244-export-backfilled-questions
into 451df4c on main.

@rgraber rgraber marked this pull request as ready for review December 17, 2024 15:32
@rgraber rgraber requested a review from jnm December 17, 2024 15:33
@rgraber rgraber marked this pull request as draft December 18, 2024 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants