Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge branch 'release/0.0.3' of https://github.com/29ana-notai/Team29_BE #41

Merged
merged 59 commits into from
Nov 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
f342234
Feat: POST 저장 기능, OCR 구현, 폴더 수정, 페이지별 조회 기능 구현 및 배포 (#66)
yugyeom-ghim Oct 25, 2024
d0e1294
Feat: AI서버에서 STT 결과 응답 왔을때 STT 결과 처리하는 기능 구현 (#27)
yugyeom-ghim Nov 1, 2024
4e8c2f8
Chore: 배포 자동화 및 Slack Webhook 설정 (#29)
yugyeom-ghim Nov 1, 2024
89dfb1e
Refactor: Document, Folder 이슈 해결 (#28)
yunjunghun0116 Nov 1, 2024
3aa5fe1
Chore: 0.0.3 버전 배포
yugyeom-ghim Nov 1, 2024
75553f3
Feat: 스웨거에서 multipart-formdata 보낼 수 있는 기능 구현
yugyeom-ghim Nov 3, 2024
114e23d
Chore: 서브모듈 설정파일 tesseract 경로 변경
yugyeom-ghim Nov 3, 2024
5de17fa
Fix: OCR content columnDefinition의 text를 longtext로 변경
yugyeom-ghim Nov 3, 2024
dd0dde5
Chore: 버전 0.0.3으로 변경
yugyeom-ghim Nov 3, 2024
79647f6
Fix: 오디오 파일 이름 정규표현식 제거
hynseoj Nov 4, 2024
8d55a03
fix: AI서버 stt요청 multipart formdata 오류 해결
yugyeom-ghim Nov 4, 2024
5071eb1
fix: InputStreamResource를 ByteArrayResource로 변경하여 스트림 재사용 문제 해결
yugyeom-ghim Nov 4, 2024
83bde51
Fix: AI서버에서의 결과에 토큰이 필요하지 않도록 수정
yugyeom-ghim Nov 5, 2024
3080d1d
Fix: AI서버 api 명세에 맞게 request dto 수정
yugyeom-ghim Nov 5, 2024
bf3c18c
Fix: 페이지 넘김 테이블 녹음 종료 시간 nullable 하게 수정
hynseoj Nov 5, 2024
a0a6c69
Fix: ai 기능 요청에서 페이지 번호 0 가능하도록 수정
hynseoj Nov 5, 2024
8aa7ff2
Fix: ai 기능 요청시 Summary, Problem 을 cascade 옵션 없이 각각 영속화하도록 수정
hynseoj Nov 5, 2024
16422aa
Fix: ai 기능 재요청 관련 트랜잭션 수정
hynseoj Nov 6, 2024
3e0b2e7
Chore: STT 로깅 설정
yugyeom-ghim Nov 6, 2024
8b55b75
Chore: recording 디버깅을 위해 exception message 형태 변경
yugyeom-ghim Nov 6, 2024
c6e0441
Fix: filePath 컬럼 제한 길이를 넘어 생기는 오류 수정
yugyeom-ghim Nov 6, 2024
93a3b68
Test: 로직 변경으로 인한 테스트코드 인자 수정
yugyeom-ghim Nov 6, 2024
15370f5
Fix: 페이지 넘김 이벤트의 EndTime이 null일때 예외처리 추가
yugyeom-ghim Nov 6, 2024
934196b
Fix: annotation의 pageNumber가 양수인 조건에서 0이상으로 변경
yugyeom-ghim Nov 6, 2024
4e756a0
Fix : delete 204 -> 200으로 변경
mingjuu Nov 7, 2024
fa22fc1
Merge remote-tracking branch 'origin/Weekly10' into release/0.0.3
yugyeom-ghim Nov 8, 2024
0fd2383
Refactor: 폴더 삭제 요청시 EntityManager로 인해 발생한 이슈 해결 및 폴더 조회 요청 개선 (#33)
yunjunghun0116 Nov 11, 2024
5be216f
Revert "Refactor: 폴더 삭제 요청시 EntityManager로 인해 발생한 이슈 해결 및 폴더 조회 요청 개선…
yugyeom-ghim Nov 11, 2024
7909d8c
Refactor: 폴더 삭제 요청시 EntityManager로 인해 발생한 이슈 해결 및 폴더 조회 요청 개선 (#35)
yugyeom-ghim Nov 11, 2024
b10bc40
Fix: 멤버가 영속성 컨텍스트에 존재하지 않은 상태로 조회되어 생기는 오류 해결
yugyeom-ghim Nov 11, 2024
835090f
Merge branch 'release/0.0.3' of https://github.com/29ana-notai/Team29…
yugyeom-ghim Nov 11, 2024
aa30af5
Fix: member 가 영속성 컨텍스트에 존재하지 않은 채로 조회해 생기는 오류 해결
yugyeom-ghim Nov 11, 2024
56f803f
Merge branch 'Weekly11' into release/0.0.3
yugyeom-ghim Nov 11, 2024
7f9ac48
fix: argumentResolver에서 id를 반환하도록 수정
yugyeom-ghim Nov 12, 2024
6a4d6e1
Feat: STT, OCR 결과 포함하여 AI서버에 요청 (#37)
yugyeom-ghim Nov 12, 2024
57c67e9
Feat: STT결과 반환 기능
yugyeom-ghim Nov 13, 2024
4d75d7b
Fix: 오류 발생하는 테스트 코드 수정
hynseoj Nov 13, 2024
c4e726d
feat: STT 요청상태 반환하는 기능 구현
yugyeom-ghim Nov 13, 2024
b88d368
feat: STT 결과에 소유검증 추가
yugyeom-ghim Nov 13, 2024
3cdcbff
Chore: SttTaskController base endpoint추가
yugyeom-ghim Nov 13, 2024
15577cf
Chore: 토큰에서 추출한 memberId가 null일 경우 예외
yugyeom-ghim Nov 13, 2024
eac0b88
Refactor: STT 도메인 연관관계 재구성 및 STT 페이지별 상태체크 기능 구현 (#38)
yugyeom-ghim Nov 14, 2024
94f2d79
test: SttTask 단위테스트 작성
yugyeom-ghim Nov 14, 2024
f54e50c
Update README.md
Shsin9797 Nov 14, 2024
d60707f
Update README.md
Shsin9797 Nov 14, 2024
260dd65
Update README.md
Shsin9797 Nov 14, 2024
0f103e0
Update README.md
Shsin9797 Nov 14, 2024
adc67e4
Update README.md
Shsin9797 Nov 14, 2024
4332347
Chore: 어노테이션 결과가 없을때 빈 배열이 반환되도록 변경
yugyeom-ghim Nov 15, 2024
ceaf1c3
Merge branch 'release/0.0.3' of https://github.com/29ana-notai/Team29…
yugyeom-ghim Nov 15, 2024
7354869
Fix: STT 상태 조회시 발생하는 null pointer exception 해결
yugyeom-ghim Nov 15, 2024
b4a8ea6
FIx: 어노테이션 생성날자, 수정날자 반환타입 변경 및 테스트코드 수정, 단위테스트 추가
yugyeom-ghim Nov 15, 2024
aee4d3d
Chore: 에러메시지 오타 수정
yugyeom-ghim Nov 15, 2024
3c72b16
Chore: 레코딩ID 로깅
yugyeom-ghim Nov 15, 2024
eb2bf82
Fix: 녹음 파일 문서 소유 검증 오류 수정
hynseoj Nov 15, 2024
8549e82
Merge remote-tracking branch 'origin/release/0.0.3' into release/0.0.3
hynseoj Nov 15, 2024
07a4111
Fix: STT조회시 Null Point Exception 수정
yugyeom-ghim Nov 15, 2024
f2b331e
Merge branch 'release/0.0.3' of https://github.com/29ana-notai/Team29…
yugyeom-ghim Nov 15, 2024
a88d5be
Merge branch 'Weekly11' of https://github.com/29ana-notai/Team29_BE i…
yugyeom-ghim Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
321 changes: 307 additions & 14 deletions README.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,9 @@ public List<AnnotationResponse> getAnnotationsByDocumentAndPageNumbers(
document.validateOwner(member);

List<Annotation> annotations = annotationRepository.findByDocumentIdAndPageNumberIn(documentId, pageNumbers);
if (annotations.isEmpty()) {
throw new NotFoundException(ANNOTATION_NOT_FOUND);
}

return annotations.stream()
.map(AnnotationResponse::from)
.collect(Collectors.toList());
.toList();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

import notai.annotation.domain.Annotation;

import java.time.LocalDateTime;

public record AnnotationResponse(
Long id,
Long documentId,
Expand All @@ -11,8 +13,8 @@ public record AnnotationResponse(
int width,
int height,
String content,
String createdAt,
String updatedAt
LocalDateTime createdAt,
LocalDateTime updatedAt
) {

public static AnnotationResponse from(Annotation annotation) {
Expand All @@ -25,8 +27,8 @@ public static AnnotationResponse from(Annotation annotation) {
annotation.getWidth(),
annotation.getHeight(),
annotation.getContent(),
annotation.getCreatedAt().toString(),
annotation.getUpdatedAt().toString()
annotation.getCreatedAt(),
annotation.getUpdatedAt()
);
}
}
2 changes: 1 addition & 1 deletion src/main/java/notai/common/exception/ErrorMessages.java
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public enum ErrorMessages {

// auth
INVALID_ACCESS_TOKEN("유효하지 않은 토큰입니다."),
INVALID_REFRESH_TOKEN("유요하지 않은 Refresh Token입니다."),
INVALID_REFRESH_TOKEN("유효하지 않은 Refresh Token입니다."),
EXPIRED_REFRESH_TOKEN("만료된 Refresh Token입니다."),
INVALID_LOGIN_TYPE("지원하지 않는 소셜 로그인 타입입니다."),
NOTFOUND_ACCESS_TOKEN("토큰 정보가 존재하지 않습니다."),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package notai.pageRecording.application;

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import notai.document.domain.Document;
import notai.document.domain.DocumentRepository;
import notai.member.domain.Member;
Expand All @@ -13,6 +14,7 @@
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Slf4j
@Service
@Transactional
@RequiredArgsConstructor
Expand All @@ -24,6 +26,8 @@ public class PageRecordingService {
private final MemberRepository memberRepository;

public void savePageRecording(Long memberId, PageRecordingSaveCommand command) {
log.info("recordingId: {}, documentId: {}", command.recordingId(), command.documentId());

Recording foundRecording = recordingRepository.getById(command.recordingId());
Document foundDocument = documentRepository.getById(command.documentId());
Member member = memberRepository.getById(memberId);
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/notai/recording/domain/Recording.java
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ public void updateFilePath(FilePath filePath) {
}

public void validateDocumentOwnership(Document document) {
if (this.document.getId().equals(document.getId())) {
if (!this.document.getId().equals(document.getId())) {
throw new NotFoundException(RECORDING_NOT_FOUND);
}
}
Expand Down
15 changes: 8 additions & 7 deletions src/main/java/notai/stt/application/SttService.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,21 @@ public class SttService {
* AI 서버로부터 받은 STT 결과를 처리하여 페이지별 STT 데이터를 생성하고 저장합니다.
* 1. STT 테스크와 관련 엔티티들을 조회
* 2. 음성 인식된 단어들을 페이지와 매칭
* 3. 매칭 결과를 저장하고 테스크를 완료 처리
* 3. 테스크를 완료처리하고 매칭 결과 저장
*/
public void updateSttResult(UpdateSttResultCommand command) {
SttTask sttTask = sttTaskRepository.getById(command.taskId());
Stt stt = sttTask.getStt();
Recording recording = stt.getRecording();
Recording recording = sttTask.getRecording();

List<PageRecording> pageRecordings = pageRecordingRepository.findAllByRecordingIdOrderByStartTime(recording.getId());
List<PageRecording> pageRecordings =
pageRecordingRepository.findAllByRecordingIdOrderByStartTime(recording.getId());

SttPageMatchedDto matchedResult = stt.matchWordsWithPages(command.words(), pageRecordings);
List<Stt> pageMatchedSttResults = Stt.createFromMatchedResult(recording, matchedResult);
sttRepository.saveAll(pageMatchedSttResults);
SttPageMatchedDto matchedResult = Stt.matchWordsWithPages(command.words(), pageRecordings);
List<Stt> pageMatchedSttResults = Stt.createFromMatchedResult(sttTask, matchedResult);

sttTask.complete();
sttTaskRepository.save(sttTask);

sttRepository.saveAll(pageMatchedSttResults);
}
}
10 changes: 3 additions & 7 deletions src/main/java/notai/stt/application/SttTaskService.java
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
import notai.recording.domain.Recording;
import notai.recording.domain.RecordingRepository;
import notai.stt.application.command.SttRequestCommand;
import notai.stt.domain.Stt;
import notai.stt.domain.SttRepository;
import notai.sttTask.domain.SttTask;
import notai.sttTask.domain.SttTaskRepository;
Expand All @@ -38,14 +37,14 @@ public void submitSttTask(SttRequestCommand command) {

try {
byte[] audioBytes = Files.readAllBytes(audioFile.toPath());

ByteArrayResource resource = new ByteArrayResource(audioBytes) {
@Override
public String getFilename() {
return audioFile.getName();
}
};

TaskResponse response = aiClient.submitSttTask(resource);
createAndSaveSttTask(recording, response);
} catch (IOException e) {
Expand All @@ -62,10 +61,7 @@ private File validateAudioFile(String audioFilePath) {
}

private void createAndSaveSttTask(Recording recording, TaskResponse response) {
Stt stt = new Stt(recording);
sttRepository.save(stt);

SttTask sttTask = new SttTask(response.taskId(), stt, TaskStatus.PENDING);
SttTask sttTask = new SttTask(response.taskId(), TaskStatus.PENDING, recording);
sttTaskRepository.save(sttTask);
}
}
60 changes: 34 additions & 26 deletions src/main/java/notai/stt/domain/Stt.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@
import lombok.NoArgsConstructor;
import notai.common.domain.RootEntity;
import notai.pageRecording.domain.PageRecording;
import notai.recording.domain.Recording;
import notai.stt.application.command.UpdateSttResultCommand;
import notai.stt.application.dto.SttPageMatchedDto;
import notai.sttTask.domain.SttTask;

import java.util.ArrayList;
import java.util.List;
Expand All @@ -34,8 +34,8 @@ public class Stt extends RootEntity<Long> {

@NotNull
@ManyToOne(fetch = LAZY)
@JoinColumn(name = "recording_id")
private Recording recording;
@JoinColumn(name = "stt_task_id")
private SttTask sttTask;

private Integer pageNumber;

Expand All @@ -46,12 +46,12 @@ public class Stt extends RootEntity<Long> {

private Integer endTime;

public Stt(Recording recording) {
this.recording = recording;
public Stt(SttTask sttTask) {
this.sttTask = sttTask;
}

public Stt(Recording recording, Integer pageNumber, String content, Integer startTime, Integer endTime) {
this.recording = recording;
public Stt(SttTask sttTask, Integer pageNumber, String content, Integer startTime, Integer endTime) {
this.sttTask = sttTask;
this.pageNumber = pageNumber;
this.content = content;
this.startTime = startTime;
Expand All @@ -62,20 +62,20 @@ public Stt(Recording recording, Integer pageNumber, String content, Integer star
* 페이지별 STT 결과로부터 새로운 STT 엔티티를 생성합니다.
* 시작/종료 시간은 페이지 내 첫/마지막 단어의 시간으로 설정합니다.
*/
public static Stt createFromPageContent(Recording recording, SttPageMatchedDto.PageMatchedContent content) {
public static Stt createFromPageContent(SttTask sttTask, SttPageMatchedDto.PageMatchedContent content) {
return new Stt(
recording,
content.pageNumber(),
content.content(),
content.words().get(0).startTime(),
content.words().get(content.words().size() - 1).endTime()
sttTask,
content.pageNumber(),
content.content(),
content.words().get(0).startTime(),
content.words().get(content.words().size() - 1).endTime()
);
}

/**
* 음성 인식된 단어들을 페이지 기록과 매칭하여 페이지별 STT 결과를 생성합니다.
*/
public SttPageMatchedDto matchWordsWithPages(
public static SttPageMatchedDto matchWordsWithPages(
List<UpdateSttResultCommand.Word> words,
List<PageRecording> pageRecordings
) {
Expand All @@ -85,8 +85,8 @@ public SttPageMatchedDto matchWordsWithPages(

// 페이지 번호 순으로 자동 정렬됨
Map<Integer, List<SttPageMatchedDto.PageMatchedWord>> pageWordMap = new TreeMap<>();
int wordIndex = 0;
PageRecording lastPage = pageRecordings.get(pageRecordings.size() - 1);
int wordIndex = 0;
PageRecording lastPage = pageRecordings.get(pageRecordings.size() - 1);

// 각 페이지별로 매칭되는 단어들을 찾아 처리
for (PageRecording page : pageRecordings) {
Expand All @@ -97,7 +97,7 @@ public SttPageMatchedDto matchWordsWithPages(
// 현재 페이지의 시간 범위에 속하는 단어들을 찾아 매칭
while (wordIndex < words.size()) {
UpdateSttResultCommand.Word word = words.get(wordIndex);

// 페이지 시작 시간보다 이른 단어는 건너뛰기
if (word.start() + TIME_THRESHOLD < pageStart) {
wordIndex++;
Expand All @@ -124,14 +124,22 @@ public SttPageMatchedDto matchWordsWithPages(
}

// 페이지별로 단어들을 하나의 텍스트로 합치는 과정
List<SttPageMatchedDto.PageMatchedContent> pageContents = pageWordMap.entrySet().stream()
List<SttPageMatchedDto.PageMatchedContent> pageContents = pageWordMap
.entrySet().stream()
.map(entry -> {
Integer pageNumber = entry.getKey();
List<SttPageMatchedDto.PageMatchedWord> pageWords = entry.getValue();
String combinedContent = pageWords.stream()
.map(SttPageMatchedDto.PageMatchedWord::word)
.collect(Collectors.joining(" "));
return new SttPageMatchedDto.PageMatchedContent(pageNumber, combinedContent, pageWords);
List<SttPageMatchedDto.PageMatchedWord>
pageWords = entry.getValue();
String combinedContent =
pageWords.stream()
.map(SttPageMatchedDto.PageMatchedWord::word)
.collect(Collectors.joining(
" "));
return new SttPageMatchedDto.PageMatchedContent(
pageNumber,
combinedContent,
pageWords
);
})
.toList();

Expand All @@ -141,9 +149,9 @@ public SttPageMatchedDto matchWordsWithPages(
/**
* 페이지 매칭 결과로부터 STT 엔티티들을 생성하고 저장합니다.
*/
public static List<Stt> createFromMatchedResult(Recording recording, SttPageMatchedDto matchedResult) {
public static List<Stt> createFromMatchedResult(SttTask sttTask, SttPageMatchedDto matchedResult) {
return matchedResult.pageContents().stream()
.map(content -> createFromPageContent(recording, content))
.toList();
.map(content -> createFromPageContent(sttTask, content))
.toList();
}
}
2 changes: 1 addition & 1 deletion src/main/java/notai/stt/presentation/SttController.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
@RequestMapping("/api/stt")
@RequiredArgsConstructor
public class SttController {
private final SttQueryService sttQueryService;
private final SttQueryService sttQueryService;

@GetMapping("/documents/{documentId}/pages/{pageNumber}")
public SttPageResponse getSttByPage(
Expand Down
31 changes: 16 additions & 15 deletions src/main/java/notai/stt/presentation/response/SttPageResponse.java
Original file line number Diff line number Diff line change
@@ -1,30 +1,31 @@
package notai.stt.presentation.response;

import notai.stt.domain.Stt;

import java.util.List;

public record SttPageResponse(
Integer pageNumber,
List<SttContent> contents
Integer pageNumber,
List<SttContent> contents
) {
public static SttPageResponse of(Integer pageNumber, List<Stt> sttList) {
List<SttContent> contents = sttList.stream()
.map(SttContent::from)
.toList();
return new SttPageResponse(pageNumber, contents);
}

public record SttContent(
String content,
Integer startTime,
Integer endTime
String content,
Integer startTime,
Integer endTime
) {
public static SttContent from(Stt stt) {
return new SttContent(
stt.getContent(),
stt.getStartTime(),
stt.getEndTime()
stt.getContent(),
stt.getStartTime(),
stt.getEndTime()
);
}
}

public static SttPageResponse of(Integer pageNumber, List<Stt> sttList) {
List<SttContent> contents = sttList.stream()
.map(SttContent::from)
.toList();
return new SttPageResponse(pageNumber, contents);
}
}
1 change: 1 addition & 0 deletions src/main/java/notai/stt/query/SttQueryRepository.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@

public interface SttQueryRepository {
List<Stt> findAllByDocumentIdAndPageNumber(Long documentId, Integer pageNumber);

List<Stt> findAllByDocumentId(Long documentId);
}
17 changes: 12 additions & 5 deletions src/main/java/notai/stt/query/SttQueryRepositoryImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

import com.querydsl.jpa.impl.JPAQueryFactory;
import lombok.RequiredArgsConstructor;
import notai.stt.domain.Stt;
import static notai.stt.domain.QStt.stt;
import notai.stt.domain.Stt;
import static notai.sttTask.domain.QSttTask.sttTask;

import java.util.List;

Expand All @@ -15,18 +16,24 @@ public class SttQueryRepositoryImpl implements SttQueryRepository {
public List<Stt> findAllByDocumentIdAndPageNumber(Long documentId, Integer pageNumber) {
return queryFactory
.selectFrom(stt)
.join(stt.recording).fetchJoin()
.where(stt.recording.document.id.eq(documentId)
.join(stt.sttTask, sttTask).fetchJoin()
.join(sttTask.recording).fetchJoin()
.join(sttTask.recording.document).fetchJoin()
.where(sttTask.recording.document.id.eq(documentId)
.and(stt.pageNumber.eq(pageNumber)))
.orderBy(stt.pageNumber.asc())
.fetch();
}

@Override
public List<Stt> findAllByDocumentId(Long documentId) {
return queryFactory
.selectFrom(stt)
.join(stt.recording).fetchJoin()
.where(stt.recording.document.id.eq(documentId))
.join(stt.sttTask, sttTask).fetchJoin()
.join(sttTask.recording).fetchJoin()
.join(sttTask.recording.document).fetchJoin()
.where(sttTask.recording.document.id.eq(documentId))
.orderBy(stt.pageNumber.asc())
.fetch();
}
}
Loading
Loading