Feature/pn 12317 (#525)

* Feature/785 (#509) * Scrittura variabili d'ambiente 'feature-flag' & nuova tipologia documenti 'pnPaperAttachment' * Fix Variabili Ambiente 'Feature Flag' & Tipo Documento * [FIX] microservice.yml e microservice-dev-cfg.json --------- Co-authored-by: Nico Ponziani <[email protected]> * Feature/773 (#512) * Init Test PdfRaster * Creazione classi Service per richiamare il microservizio PdfRaster * Definizione properties pdfraster * Creazione Entity nuove tabelle per la conversione e classi di Service per inserimento * Creazione Classe record per lettura properties PdfRaster * Modifica Classi di configurazione per pdfraster * Creazione Classi webClient per richiamare pdfraster * Creazione metodi di test * FIX metodo lavorazioneRichiestaKoPdfRasterNotFound -> aggiunta di .verify() * [WIP] (FIX) Test retry/lavorazioneCartaceo * [FIX] casi di test retryGestioneCartaceo * [FIX] modificati i metodi che utilizzano le exception come parametri nei casi di test di pdfraster * Refactoring of lavorazioneCartaceo, lavorationeCartaceoTest, gestioneCartaceoRetry, gestioneCartaceoRetryTest --------- Co-authored-by: Nico Ponziani <[email protected]> * Feature/781 (#511) * Creazione Entity nuove tabelle per la conversione e classi di Service per inserimento * 781: RedPhase complete * 781: RedPhase fix entity and DTO * 781: RedPhase complete * 781: GreenPhase complete * 781: Replaced RunTimeException, applied a resolving patch for entity and YAML file management. * 781: Blue phase complete * 781: Blue phase bug fix log and retryWhen in updateRequestConversion --------- Co-authored-by: Nico Ponziani <[email protected]> Co-authored-by: CristianoZingaretti <[email protected]> * feature/12317-763-bis: removed pdf raster references. (#513) Removed redundant dtos. Co-authored-by: Nico Ponziani <[email protected]> * feature/12317-806: added sha256 field to AttachmentToConvert entity. (#515) Co-authored-by: Nico Ponziani <[email protected]> * feature/12317-774: Implemented pdfRasterStep() (#514) * feature/12317-774-bis: Implemented pdfRasterStep() * feature/12317-774 fix method gestioneRetryCartaceo_Retry_PutRequestKo() in CartaceoRetryTest * feature/12317-775: Added configurations for rastering in lavorazione cartaceo. General refactoring. --------- Co-authored-by: Nico Ponziani <[email protected]> * feature/12317-789: Availability Manager (#516) Co-authored-by: Nico Ponziani <[email protected]> * PN-12317: fixed postFile method * PN-12317: fixed PnECPaperDocumentTypeForRasterized parameter * Feature/pn 12317 infra (#518) * PN-12317: added availability queue * PN-12317: fixed env param * PN-12317: added tables * PN-12317: added oputputs DLQ info * PN-12317: added anv, parameters, ... * PN-12317: removed useless stream specification on new tables * PN-12317: fixed some resource names * PN-12317: fixe some resource names * PN-12317: fixed some wrong names * PN-12317: fixed some wrong names --------- Co-authored-by: Gabriele Libianchi <gabriele.libianchi@dgsspa> Co-authored-by: mottone-dgs <[email protected]> * PN-12317: fixed some wrong names * PN-12317: fixed log trace in uploadFile * feature/12317-fix_upload_s3 modify put request in uploadFile, get con… (#517) * feature/12317-fix_upload_s3 modify put request in uploadFile, get contentType from fileDownloadResponse * feature/12317-fix_upload_s3 fix null pointer CartaceoServiceTest/CartaceoRetryTest --------- Co-authored-by: Nico Ponziani <[email protected]> Co-authored-by: mottone-dgs <[email protected]> * PN-12317: removed NetworkLoadBalancerLink and Version parameters * PN-12317: fixed PnEcEventRuleAvailabilityManager * PN-12317: fixed some references in PnEcManagedPolicy * PN-12317: fixed the name of EventBusName in PnEcEventRuleAvailabilityManager * PN-12317: added missing policy sqs:SendMessage for PnEcEventRuleAvailabilityManager * Feature/pn 12317 fix set expiration (#519) * feature/12317-fix_set_expiration set expiration PdfConversionEntity in updateRequestConversion * feature/12317-fix_set_expiration set expiration in second * PN-12317: restored reference PnEcEventBusNameTrackerExternalNotification --------- Co-authored-by: Nico Ponziani <[email protected]> Co-authored-by: mottone-dgs <[email protected]> * PN-12317: EventBusName of PnEcEventRuleAvailabilityManager fixed to Pn-Ss-Notifications-Bus * Feature/pn 12317 fix set expiration (#520) * feature/12317-fix_set_expiration set expiration PdfConversionEntity in updateRequestConversion * feature/12317-fix_set_expiration set expiration in second * feature/12317-fix_set_expiration check idempotence * feature/12317-fix_set_expiration acknowledge * feature/12317-fix_set_expiration fix objectMapper conversion -> @JsonProperty on field xPagopaExtchCxId * feature/12317-fix_set_expiration fix idempotence logic * feature/12317-fix_set_expiration if check attachment --------- Co-authored-by: Nico Ponziani <[email protected]> * feature/12317-fix_lavorazione_carteceo modify method overridePaIdIfRequired , putStepRequest (#521) Co-authored-by: Nico Ponziani <[email protected]> * PN-12317: adjusted some paramaters related to PDFRaster --------- Co-authored-by: NicoPonziani <[email protected]> Co-authored-by: Nico Ponziani <[email protected]> Co-authored-by: CristianoZingarettiDGS <[email protected]> Co-authored-by: CristianoZingaretti <[email protected]> Co-authored-by: mottone-dgs <[email protected]> Co-authored-by: Martina Mattei <[email protected]> Co-authored-by: Gabriele Libianchi <gabriele.libianchi@dgsspa> Co-authored-by: mottone-dgs <[email protected]>
pagopa · Sep 13, 2024 · 6d6b095 · 6d6b095
1 parent 24c0b49
commit 6d6b095
Show file tree

Hide file tree

Showing 50 changed files with 2,667 additions and 428 deletions.
diff --git a/docs/openapi/internal/externalchannel/gestore-repository-v1.yaml b/docs/openapi/internal/externalchannel/gestore-repository-v1.yaml
@@ -363,4 +363,10 @@ components:
             type: number
             description: >-
               come da specifiche di configurazione delle retry policy per la sola parte di interesse del canale
-            example: '"SMS":[5,10,20,40],"EMAIL":[5,10,20,40],"PEC":[5,10,20,40],"PAPER":[5,10,20,40]'
+            example: '"SMS":[5,10,20,40],"EMAIL":[5,10,20,40],"PEC":[5,10,20,40],"PAPER":[5,10,20,40]'
+
+
+    RequestConversionDto:
+      $ref: '../schemas/internal-schemas-paper-v1.yaml#/components/schemas/RequestConversionDto'
+    PdfConversionDto:
+      $ref: '../schemas/internal-schemas-paper-v1.yaml#/components/schemas/PdfConversionDto'
diff --git a/docs/openapi/internal/externalchannel/schemas/internal-schemas-paper-v1.yaml b/docs/openapi/internal/externalchannel/schemas/internal-schemas-paper-v1.yaml
@@ -367,3 +367,76 @@ components:
         anonymous:
           type: boolean
           description: default false, diventa true durante la lettura dell'aggiornamento dello stato
+
+
+    ################################################################
+    ###      PDF RASTER - DTO PER RICHIESTE CONVERSIONE PDF      ###
+    ################################################################
+
+    AttachmentToConvertDto:
+      type: object
+      properties:
+        originalFileKey:
+          type: string
+          description: Chiave del file originale
+        newFileKey:
+          type: string
+          description: Chiave del nuovo file
+        converted:
+          type: boolean
+          description: Stato della conversione
+        sha256:
+          type: string
+          description: Hash del file
+      required:
+        - originalFileKey
+        - newFileKey
+
+    RequestConversionDto:
+      type: object
+      properties:
+        requestId:
+          type: string
+          description: ID della richiesta
+        requestTimestamp:
+          type: string
+          format: date-time
+          description: Timestamp della richiesta
+        expiration:
+          type: number
+          format: float
+          description: Scadenza della entry in formato decimale
+        originalRequest:
+          $ref: '../../external/schemas-paper-v1.1.yaml#/components/schemas/PaperEngageRequest'
+        attachments:
+          type: array
+          items:
+            $ref: '#/components/schemas/AttachmentToConvertDto'
+        xPagopaExtchCxId:
+          type: string
+          description: ID del client
+      required:
+        - requestId
+        - requestTimestamp
+        - expiration
+        - originalRequest
+        - attachments
+        - xPagopaExtchCxId
+
+    PdfConversionDto:
+      type: object
+      properties:
+        fileKey:
+          type: string
+          description: Chiave del file
+        requestId:
+          type: string
+          description: ID della richiesta
+        expiration:
+          type: number
+          format: float
+          description: Scadenza della entry in formato decimale
+      required:
+        - fileKey
+        - requestId
+        - expiration
diff --git a/scripts/aws/cfn/microservice-dev-cfg.json b/scripts/aws/cfn/microservice-dev-cfg.json
@@ -42,6 +42,10 @@
     "SpringCodecMaxInMemorySize": "256KB",
     "PnEcConsAllowedFutureOffsetDuration": "1m",
     "CpuValue": "1024",
-    "MemoryAmount": "4GB"
+    "MemoryAmount": "4GB",
+    "PnECPaperDocumentTypeToRaster": "PN_NOTIFICATION_ATTACHMENTS",
+    "PnECPaperDocumentTypeForRasterized": "PN_PAPER_ATTACHMENT",
+    "PnECPaperPAIdToRaster": "ALL",
+    "PnECPaperPAIdOverride": "15376371009"
   }
 }
diff --git a/scripts/aws/cfn/microservice.yml b/scripts/aws/cfn/microservice.yml
@@ -56,11 +56,6 @@ Parameters:
     Type: String
     Description: 'Base URL of the load balancer where the service is going to be reachable'
 
-  NetworkLoadBalancerLink:
-    Type: String
-    Description: 'network load balancer link for API-GW'
-    Default: ''
-
   PnCoreTargetEventBus:
     Type: String
     Description: 'ARN of Pn Core Event Bus where to forward external notifications'
@@ -70,10 +65,6 @@ Parameters:
     Type: String
     Description: 'Secret ID for Identity Pec'
 
-  Version:
-    Type: String
-    Description: 'keep track of used projects commitIds'
-
   PnEcManagedPolicyName:
     Type: String
     Description: 'Nome della managed policy associata ad External Channel'
@@ -177,6 +168,22 @@ Parameters:
     Type: String
     Description: 'ARN della Tabella DynamoDB relativa alle richieste'
 
+  PnEcTableNameRichiesteConversione:
+    Type: String
+    Description: 'Nome della Tabella DynamoDB relativa alle richieste di conversione dei PDF'
+
+  PnEcTableArnRichiesteConversione:
+    Type: String 
+    Description: 'ARN della Tabella DynamoDB relativa alle richieste di conversione dei PDF'
+
+  PnEcTableNameConversionePdf:
+    Type: String
+    Description: 'Nome della Tabella DynamoDB di supporto per la tabella RichiesteConversione'
+
+  PnEcTableArnConversionePdf:
+    Type: String
+    Description: 'ARN della Tabella DynamoDB di upporto per la tabella RichiesteConversione'
+
   ### QUEUES  ###
   ### TRACKER : SMS  ###
 
@@ -613,6 +620,14 @@ Parameters:
     Type: String
     Description: '[Canale CARTACEO] URL della Coda Errori CARTACEO'
 
+  PnEcDLQueueNameErroriCARTACEO:
+    Type: String
+    Description: '[Canale CARTACEO] Nome della DLQ della Coda Errori afferenti al canale CARTACEO'
+
+  PnEcDLQueueArnErroriCARTACEO:
+    Type: String
+    Description: '[Canale CARTACEO] ARN della DLQ della Coda Errori afferenti al canale CARTACEO'
+
   PnEcAlarmArnQueueErroriCARTACEO:
     Type: String
     Description: "[Alarm SQS] ARN dell'allarme messaggi vecchi nella coda PnEcQueueErroriCARTACEO"
@@ -710,6 +725,19 @@ Parameters:
     Type: String
     Description: 'ARN Bucket Staging per messaggi SQS troppo grandi'
 
+### PdfRaster - Availability queue  ###
+  PnEcQueueNameAvailabilityManager:
+    Description: 'Nome della coda Disponibilità per pdf-raster'
+    Type: String
+
+  PnEcQueueArnAvailabilityManager:
+    Description: 'ARN della coda Disponibilità per pdf-raster'
+    Type: String
+
+  PnEcQueueUrlAvailabilityManager:
+    Description: 'Url della coda Disponibilità per pdf-raster'
+    Type: String
+
   #### Parametri Indipendenti ####
 
   PnEcConsolidatoreBaseURL:
@@ -833,6 +861,28 @@ Parameters:
     Description: Memory amount reserved to the task pod.
     AllowedValues: [ 2GB, 4GB, 6GB, 8GB ]
 
+  PnECPaperPAIdToRaster:
+    Type: String
+    Description: 'Feature flag per la funzionalità di conversione degli allegati cartacei.
+    Può assumere tre valori:
+    - la sola stringa “NOTHING”: significa che la nuova funzionalità è completamente disabilitata
+    - la sola stringa “ALL”: tutti gli allegati del tipo specificato verranno convertiti
+    - <elenco di ID delle PA>: se il campo requestPaId della richiesta è contenuto in questo elenco, allora verrà applicata la conversione dei PDF del tipo specificato, altrimenti si procederà come di consueto
+    '
+
+  PnECPaperDocumentTypeToRaster:
+    Type: String
+    Description: 'Tipologia per la selezione del documento da convertire'
+
+  PnECPaperDocumentTypeForRasterized:
+    Type: String
+    Description: 'Tipologia da applicare agli allegati cartacei convertiti'
+
+  PnECPaperPAIdOverride:
+    Type: String
+    Description: 'Valore del requestPaID da sovrascrivere nella richiesta cartacea. Se non valorizzato, la funzionalità è disattivata'
+
+
 Conditions:
   PnCoreTargetEventBusExists: !Not [ !Equals [!Ref PnCoreTargetEventBus, '' ] ]
   HasAlbDimensionName: !Not [ !Equals [ !Ref ApplicationLoadBalancerMetricsDimensionName, '' ] ]
@@ -921,6 +971,14 @@ Resources:
         ContainerEnvEntry68: !Sub 'PnEcNamirialServerCacheEndpoint=${PnEcNamirialServerCacheEndpoint}'
         ContainerEnvEntry69: !Sub 'SpringCodecMaxInMemorySize=${SpringCodecMaxInMemorySize}'
         ContainerEnvEntry70: !Sub 'PnEcConsAllowedFutureOffsetDuration=${PnEcConsAllowedFutureOffsetDuration}'
+        ContainerEnvEntry71: !Sub 'PnECPaperPAIdToRaster=${PnECPaperPAIdToRaster}'
+        ContainerEnvEntry72: !Sub 'PnECPaperDocumentTypeToRaster=${PnECPaperDocumentTypeToRaster}'
+        ContainerEnvEntry73: !Sub 'PnECPaperDocumentTypeForRasterized=${PnECPaperDocumentTypeForRasterized}'
+        ContainerEnvEntry74: !Sub 'PnECPaperPAIdOverride=${PnECPaperPAIdOverride}'
+        ContainerEnvEntry75: !Sub 'PnEcQueueNameAvailabilityManager=${PnEcQueueNameAvailabilityManager}'
+        ContainerEnvEntry76: !Sub 'PnEcDLQueueNameErroriCARTACEO=${PnEcDLQueueNameErroriCARTACEO}'
+        ContainerEnvEntry77: !Sub 'PnEcTableNameRichiesteConversione=${PnEcTableNameRichiesteConversione}'
+        ContainerEnvEntry78: !Sub 'PnEcTableNameConversionePdf=${PnEcTableNameConversionePdf}'
         ContainerSecret1: !Sub 'ConsolidatoreClientApiKey=arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:pn-EC-Secrets:ConsolidatoreClientApiKey:AWSCURRENT:'
         MicroServiceSecretPrefix: pn-EC-Secrets
         MappedPaths: '/external-channels/*,/external-channel/*,/consolidatore-ingress/*'
@@ -964,6 +1022,8 @@ Resources:
               - !Ref PnEcTableArnAnagrafica
               - !Ref PnEcTableArnRichieste
               - !Ref PnEcTableArnRichiesteMetadati
+              - !Ref PnEcTableArnRichiesteConversione
+              - !Ref PnEcTableArnConversionePdf
           - Effect: Allow
             Action:
               - kms:Encrypt
@@ -1001,6 +1061,8 @@ Resources:
               - !Ref PnEcQueueArnErroriCARTACEO
               - !Ref PnEcQueueArnScaricamentoEsitiPEC
               - !Ref PnEcQueueArnCancellazioneRicevutePEC
+              - !Ref PnEcQueueArnAvailabilityManager
+              - !Ref PnEcDLQueueArnErroriCARTACEO
           - Effect: Allow
             Action:
               - 'events:DescribeEventBus'
@@ -1594,3 +1656,53 @@ Resources:
               return { batchItemFailures} ;
             }))
           }
+
+
+### PdfRaster - Availability queue  ###
+
+  PnEcQueuePolicyAvailabilityManager:
+    Type: AWS::SQS::QueuePolicy
+    Properties:
+      Queues:
+        - !Ref PnEcQueueUrlAvailabilityManager
+      PolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Action:
+              - 'sqs:*'
+            Effect: 'Allow'
+            Resource: !Ref PnEcQueueArnAvailabilityManager
+            Principal:
+              AWS: !Ref AWS::AccountId
+          - Sid: 'AllowPutEventsFromEventRule'
+            Action:
+              - 'sqs:SendMessage'
+            Effect: 'Allow'
+            Resource: !Ref PnEcQueueArnAvailabilityManager
+            Principal:
+              Service:
+              - 'events.amazonaws.com'
+            Condition:
+              ArnEquals:
+                aws:SourceArn: !GetAtt PnEcEventRuleAvailabilityManager.Arn
+
+  PnEcEventRuleAvailabilityManager:
+    Type: AWS::Events::Rule
+    Properties:
+      Description: 'This Event rule is used to forward availability events for paper attachments to the queue PnEcQueueAvailabilityManagerStack'
+      EventBusName: 'Pn-Ss-Notifications-Bus'
+      EventPattern:
+        source:
+          - 'GESTORE DISPONIBILITA'
+        account:
+          - !Ref AWS::AccountId
+        region:
+          - !Ref AWS::Region
+        detail:
+          documentType:
+          - PN_PAPER_ATTACHMENT
+
+      State: 'ENABLED'
+      Targets:
+        - Arn: !Ref PnEcQueueArnAvailabilityManager
+          Id: 'PnEcQueueAvailabilityManager'