Cygnus LD Subscription stop sending data to postgres (Batch accumulation time reached)

[fadelcg]

Hello, I am trying to persist the IoT data that I have by applying Cygnus LD and connect it to postgres.

I ran the following command for the subscription:

curl -L -X POST 'http://localhost:1026/ngsi-ld/v1/subscriptions/' -H 'Content-Type: application/ld+json' -H 'NGSILD-Tenant: openiot' --data-raw '{ 
  "description": "Notify me of all changes", 
  "type": "Subscription", 
  "entities": [{"type": "Device"}], 
  "watchedAttributes": ["co2", "formaldehyd", "humidity", "pm25", "temperature", "voc"], 
  "notification": { 
    "attributes": ["co2", "formaldehyd", "humidity", "pm25", "temperature", "voc"], 
    "format": "keyValues", 
    "endpoint": { 
      "uri": "http://cygnus-ngsi-ld:5050/notify", 
      "accept": "application/json", 
      "receiverInfo": [ 
        { 
          "key": "fiware-service", 
          "value": "openiot" 
        } 
      ] 
    } 
  }, 
   "@context": [ "http://context/user-context.jsonld", "http://context/ngsi-context.jsonld" ]
}' 

The subscription is successfull and it saves the IoT data to Postgres. But it only works usually for 3 or 4 Changes / Input Data. After that, Cygnus will stop saving the data in Postgres with following logs:

When I run the subscription again, Cygnus will save the data again to postgres for a moment. After that it stops again sending the data and the above logs will show up again.

Could you please help me solving the problem that Cygnus can continuosly save the data to postgres?

Thank you.

[fgalan]

Maybe the author of Cygnus-LD (CC: @anmunoz ) or some FIWARE Foundation expert (CC: @jason-fox ) could provide support on this.

[fadelcg]

Here is another error i found when i run this command

curl --location 'http://localhost:1026/ngsi-ld/v1/subscriptions/' \
--header 'NGSILD-Tenant: openiot' \
--header 'fiware-servicepath: /'

This is the part of the response:

{"id":"urn:ngsi-ld:subscription:335fffc4-6084-11ef-9578-0242ac13000c","type":"Subscription","description":"Notify me of all changes","entities":[{"type":"https://uri.fiware.org/ns/dataModels#Device"}],"watchedAttributes":["https://w3id.org/saref#humidity","https://w3id.org/saref#temperature"],"status":"paused","isActive":false,"notification":{"attributes":["https://w3id.org/saref#humidity","https://w3id.org/saref#temperature"],"format":"keyValues","endpoint":{"uri":"http://cygnus-ngsi-ld:5050/notify","accept":"application/json","receiverInfo":[{"key":"fiware-service","value":"openiot"}]},"status":"failed","timesSent":3,"lastNotification":"2024-08-22T12:44:00.387Z","lastFailure":"2024-08-22T12:44:00.387Z","consecutiveErrors":3,"lastErrorReason":"Content-Length not found but the status code is not a 204"},"origin":"cache","jsonldContext":"http://172b20d8e896:1026/ngsi-ld/v1/jsonldContexts/335ff920-6084-11ef-9578-0242ac13000c"}]

[AlvaroVega]

Just to be sure is not related with batches you can try to reduce size and timeout about batches in sink configuration, ie:
batch_size = 1
batch_timeout = 10

@fadelcg Which version of cygnus are you using?

[fadelcg]

I edited the .conf file. The batch size is already 1 and batch timeout was 30 and I changed to 10. It still have the same problem.

I used before the version from: https://hub.docker.com/r/fiware/cygnus-ld
And now I am testing the older version from: telefonicaiot/fiware-cygnus-ld:2.7.0.

Both version has the same issue: batch accumulation timed out.

The Postgres I am using is Postgres 13.

[AlvaroVega]

I used before the version from: https://hub.docker.com/r/fiware/cygnus-ld
And now I am testing the older version from: telefonicaiot/fiware-cygnus-ld:2.7.0.

Last pushed 2 years ago by jasonmhfox