Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API with queue history in CSV #166

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add API with queue history in CSV #166

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7a027b3
Add API with queue history in CSV
piotrwasko Oct 15, 2021
6f8da31
Add queue ID and name to events response and use it in CSV
piotrwasko Oct 17, 2021
7bf19c4
Merge branch 'master' into add-csv-history
piotrwasko Oct 17, 2021
7ca2fb3
Merge branch 'master' into add-csv-history
daltonfury42 Oct 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions simplq/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@
<artifactId>liquibase-core</artifactId>
<version>4.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
<version>1.8</version>
</dependency>

<!-- Tests -->
<dependency>
Expand Down Expand Up @@ -155,9 +160,9 @@
</configuration>
<dependencies>
<dependency>
<groupId>postgresql</groupId>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.1-901.jdbc4</version>
<version>42.2.24</version>
</dependency>
</dependencies>
</plugin>
Expand Down
12 changes: 12 additions & 0 deletions simplq/src/main/java/me/simplq/controller/QueueController.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package me.simplq.controller;

import javax.validation.Valid;

import lombok.RequiredArgsConstructor;
import me.simplq.controller.model.queue.CreateQueueRequest;
import me.simplq.controller.model.queue.CreateQueueResponse;
Expand All @@ -9,6 +10,7 @@
import me.simplq.controller.model.queue.PatchQueueResponse;
import me.simplq.controller.model.queue.PauseQueueRequest;
import me.simplq.controller.model.queue.QueueDetailsResponse;
import me.simplq.controller.model.queue.QueueEventsCsvResponse;
import me.simplq.controller.model.queue.QueueEventsResponse;
import me.simplq.controller.model.queue.QueueStatusResponse;
import me.simplq.controller.model.queue.UpdateQueueStatusResponse;
Expand All @@ -22,6 +24,7 @@
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
Expand Down Expand Up @@ -86,4 +89,13 @@ public ResponseEntity<QueueEventsResponse> getQueueEvents(
@PathVariable("queueId") String queueId) {
return ResponseEntity.ok(queueService.getQueueEvents(queueId));
}

@GetMapping(path = "/queue/{queueId}/history", produces = "text/csv")
@ResponseBody
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need @ResponseBody?

public QueueEventsCsvResponse getQueueHistoryCSV(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename function to getQueueEventsCsv and make the path "/queue/{queueId}/eventsCsv"

@PathVariable("queueId") String queueId) {
return queueService.getQueueEventsInCsv(queueId);
}


}
59 changes: 59 additions & 0 deletions simplq/src/main/java/me/simplq/controller/converters/QueueEventsCsvResponseConverter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package me.simplq.controller.converters;

import me.simplq.controller.model.queue.QueueEventsCsvResponse;
import me.simplq.controller.model.queue.QueueEventsResponse;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import org.springframework.http.ContentDisposition;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.AbstractHttpMessageConverter;
import org.springframework.http.converter.HttpMessageNotReadableException;
import org.springframework.http.converter.HttpMessageNotWritableException;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.io.OutputStreamWriter;
import java.nio.charset.StandardCharsets;

@Component
public class QueueEventsCsvResponseConverter extends AbstractHttpMessageConverter<QueueEventsCsvResponse> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's write a converter for QueueEventsResponse and remove QueueEventsCsvResponse and getQueueEventsInCsv service function.

As per this, it will use the media type to determine the converter, this should work. Can you try?

Copy link
Author

@piotrwasko piotrwasko Oct 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to use Accept header to decide on the response content, then maybe just one endpoint would be enough?

  • GET /queue/{queueId}/events with Accept: application/json would return the events in JSON,
  • GET /queue/{queueId}/events with Accept: text/csvwould return a CSV

What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QueueEventsResponse does not include queue name. I wanted to use it to make the filename more accessible. Do you think a generic name like history.csv is ok?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the Accept header if that works would be the best way, in my opinion.

I wanted to use it to make the filename more accessible.

Yeah, that would be desirable. Can you try adding the queue Id and name to QueueEventsResponse? In case that's difficult, we can fallback to a generic name.


private static final MediaType TEXT_CSV = new MediaType("text", "csv", StandardCharsets.UTF_8);

QueueEventsCsvResponseConverter() {
super(TEXT_CSV);
}

@Override
protected boolean supports(Class<?> aClass) {
return QueueEventsCsvResponse.class.equals(aClass);
}

@Override
protected QueueEventsCsvResponse readInternal(Class<? extends QueueEventsCsvResponse> aClass, HttpInputMessage httpInputMessage) throws IOException, HttpMessageNotReadableException {
throw new HttpMessageNotReadableException("Reading history from CSV not supported", httpInputMessage);
}

@Override
protected void writeInternal(QueueEventsCsvResponse response, HttpOutputMessage output) throws IOException, HttpMessageNotWritableException {
output.getHeaders().setContentType(TEXT_CSV);
output.getHeaders().setContentDisposition(ContentDisposition.builder("attachment").filename(response.getFileName()).build());
CSVPrinter csvPrinter = new CSVPrinter(new OutputStreamWriter(output.getBody()), CSVFormat.DEFAULT);
for (QueueEventsResponse.Event event : response.getEventResponse().getEvents()) {
printEvent(csvPrinter, event);
}
csvPrinter.flush();
csvPrinter.close();
}

private void printEvent(CSVPrinter csvPrinter, QueueEventsResponse.Event event) throws IOException {
csvPrinter.printRecord(
event.getTokenNumber(),
event.getTokenName(),
event.getEventType(),
event.getEventTimestamp()
);
}
}
14 changes: 14 additions & 0 deletions simplq/src/main/java/me/simplq/controller/model/queue/QueueEventsCsvResponse.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package me.simplq.controller.model.queue;

import lombok.Builder;
import lombok.Data;

import java.util.Date;
import java.util.List;

@Data
@Builder
public class QueueEventsCsvResponse {
String fileName;
QueueEventsResponse eventResponse;
}
27 changes: 21 additions & 6 deletions simplq/src/main/java/me/simplq/service/QueueEventsService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import lombok.val;
import me.simplq.controller.model.queue.QueueDetailsResponse;
import me.simplq.controller.model.queue.QueueEventsCsvResponse;
import me.simplq.controller.model.queue.QueueEventsResponse;
import me.simplq.controller.model.queue.QueueEventsResponse.Event;
import me.simplq.controller.model.queue.QueueEventsResponse.Event.EventType;
Expand All @@ -14,6 +17,23 @@
public class QueueEventsService {

public QueueEventsResponse getQueueEvents(QueueDetailsResponse queueDetails) {
return QueueEventsResponse.builder()
.events(
getTokenEventsStream(queueDetails)
.sorted(Comparator.comparing(Event::getEventTimestamp).reversed())
.collect(Collectors.toList()))
.build();
}

public QueueEventsCsvResponse getQueueEventsInCsv(QueueDetailsResponse queueDetails) {

return QueueEventsCsvResponse.builder()
.fileName(String.format("%s.history.csv", queueDetails.getQueueName()))
.eventResponse(getQueueEvents(queueDetails))
.build();
}

private Stream<Event> getTokenEventsStream(QueueDetailsResponse queueDetails) {
// Make creation events for active tokens.
var activeTokenEventStream =
queueDetails.getTokens().stream()
Expand Down Expand Up @@ -46,11 +66,6 @@ public QueueEventsResponse getQueueEvents(QueueDetailsResponse queueDetails) {
.build()))
.flatMap(List::stream);

return QueueEventsResponse.builder()
.events(
Stream.concat(activeTokenEventStream, removedTokenEventStream)
.sorted(Comparator.comparing(Event::getEventTimestamp).reversed())
.collect(Collectors.toList()))
.build();
return Stream.concat(activeTokenEventStream, removedTokenEventStream);
}
}
7 changes: 7 additions & 0 deletions simplq/src/main/java/me/simplq/service/QueueService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import me.simplq.controller.model.queue.PatchQueueResponse;
import me.simplq.controller.model.queue.PauseQueueRequest;
import me.simplq.controller.model.queue.QueueDetailsResponse;
import me.simplq.controller.model.queue.QueueEventsCsvResponse;
import me.simplq.controller.model.queue.QueueEventsResponse;
import me.simplq.controller.model.queue.QueueStatusResponse;
import me.simplq.controller.model.queue.UpdateQueueStatusResponse;
Expand All @@ -22,6 +23,7 @@
import me.simplq.exceptions.SQInvalidRequestException;
import me.simplq.utils.predicates.QueueThrowingPredicate;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;

@Service
Expand Down Expand Up @@ -149,6 +151,11 @@ public QueueEventsResponse getQueueEvents(String queueId) {
return queueEventsService.getQueueEvents(this.getQueueDetailsResponseInternal(queueId));
}

@Transactional
public QueueEventsCsvResponse getQueueEventsInCsv(String queueId) {
return queueEventsService.getQueueEventsInCsv(this.getQueueDetailsResponseInternal(queueId));
}

private QueueDetailsResponse getQueueDetailsResponseInternal(String queueId) {
return queueRepository
.findById(queueId)
Expand Down