Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add preserve stop times flag to avoid normalizing stop_sequence #304

Merged
merged 5 commits into from
May 19, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@
<dependency>
<groupId>com.conveyal</groupId>
<artifactId>gtfs-lib</artifactId>
<version>6.0.1</version>
<version>6.0.2</version>
<!-- Exclusions added in order to silence SLF4J warnings about multiple bindings:
http://www.slf4j.org/codes.html#multiple_bindings
-->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,12 +176,13 @@ public void run () {
// because the error presumably already occurred and has a better error message.
cancel(cancelMessage);
}
// Run final steps of job pending completion or error. Note: any tasks that depend on job success should
// check job status to determine if final step should be executed (e.g., storing feed version in MongoDB).
// TODO: should we add separate hooks depending on state of job/sub-tasks (e.g., success, catch, finally)
// Complete the job (as success if no errors encountered, as failure otherwise).
if (!parentJobErrored && !subTaskErrored) status.completeSuccessfully("Job complete!");
else status.complete(true);
// Run final steps of job pending completion or error. Note: any tasks that depend on job success should
// check job status in jobFinished to determine if final step should be executed (e.g., storing feed
// version in MongoDB).
// TODO: should we add separate hooks depending on state of job/sub-tasks (e.g., success, catch, finally)
jobFinished();

// We retain finished or errored jobs on the server until they are fetched via the API, which implies they
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ public void jobLogic() {
Collection<Snapshot> existingSnapshots = feedSource.retrieveSnapshots();
int version = existingSnapshots.size();
status.update("Creating snapshot...", 20);
FeedLoadResult loadResult = makeSnapshot(namespace, DataManager.GTFS_DATA_SOURCE);
FeedLoadResult loadResult = makeSnapshot(namespace, DataManager.GTFS_DATA_SOURCE, !feedSource.preserveStopTimesSequence);
snapshot.version = version;
snapshot.namespace = loadResult.uniqueIdentifier;
snapshot.feedLoadResult = loadResult;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ public class ExportSnapshotToGTFSJob extends MonitorableJob {
private static final Logger LOG = LoggerFactory.getLogger(ExportSnapshotToGTFSJob.class);
private final Snapshot snapshot;
private final String feedVersionId;
private File tempFile;

public ExportSnapshotToGTFSJob(Auth0UserProfile owner, Snapshot snapshot, String feedVersionId) {
super(owner, "Exporting snapshot " + snapshot.name, JobType.EXPORT_SNAPSHOT_TO_GTFS);
Expand All @@ -40,7 +41,6 @@ public Snapshot getSnapshot () {

@Override
public void jobLogic() {
File tempFile;
try {
tempFile = File.createTempFile("snapshot", "zip");
} catch (IOException e) {
Expand All @@ -52,6 +52,7 @@ public void jobLogic() {
FeedLoadResult result = exporter.exportTables();
if (result.fatalException != null) {
status.fail(String.format("Error (%s) encountered while exporting database tables.", result.fatalException));
return;
landonreed marked this conversation as resolved.
Show resolved Hide resolved
}

// Override snapshot ID if exporting feed for use as new feed version.
Expand All @@ -71,12 +72,15 @@ public void jobLogic() {
status.fail(String.format("Could not store feed for snapshot %s", snapshot.id), e);
}
}
// Delete snapshot temp file.
tempFile.delete();
}

@Override
public void jobFinished () {
if (!status.error) status.completeSuccessfully("Export complete!");
// Delete snapshot temp file.
if (tempFile != null) {
LOG.info("Deleting temporary GTFS file for exported snapshot at {}", tempFile.getAbsolutePath());
tempFile.delete();
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,17 @@ public class FeedSource extends Model implements Cloneable {
/**
* The collection of which this feed is a part
*/
//@JsonView(JsonViews.DataDump.class)
public String projectId;

/**
* When snapshotting a GTFS feed for editing, gtfs-lib currently defaults to normalize stop sequence values to be
* zero-based and incrementing. This can muck with GTFS files that are linked to GTFS-rt feeds by stop_sequence, so
* this override flag currently provides a workaround for feeds that need to be edited but do not need to edit
* stop_times or individual patterns. WARNING: enabling this flag for a feed and then attempting to edit patterns in
* complicated ways (e.g., modifying the order of pattern stops) could have unexpected consequences.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it'd be good to have some kind of UI warning for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, just read the PR description. Maybe edit this comment to say that there is no UI setting for this and it is not recommended to do this unless absolutely necessary.

*/
public boolean preserveStopTimesSequence;

/**
* Get the Project of which this feed is a part
*/
Expand Down