Prototype a MQTT Source #7919

ctmphuongg · 2024-05-10T23:12:42Z

Proposed Changes

(The main working files are cmd/mqttsource/main.go, config/tools/mqttsource/mqttsource.yaml, the rest is packages installed to use mqtt_paho, generated by using go get paho and go mod vendor)

Install packages for mqtt_paho
Prototype an mqtt container source

Notes: For testing

MQTT needs a source to listen from. For local testing, run this command to open the mosquitto mqtt broker for source.

docker run -it --rm --name mosquitto -p 1883:1883 eclipse-mosquitto:2.0 mosquitto -c /mosquitto-no-auth.conf

Keep the mqttsource running by go build the file and run

./main -source localhost:1883 -eventType mqtt.event -eventSource mqtt://localhost:1883 -topic "test-topic"

Send a message to the same topic, using

mosquitto_pub -t 'test-topic' -m '{"specversion" : "1.0","type" :"com.example.someevent", "id" : "1234-1234-1234","source" : "/mycontext/subcontext","data":{"msg":"hello world!"}}' -D PUBLISH user-property Content-Type application/cloudevents+json; charset=utf-8

The message will be logged and sent to sink.

knative-prow · 2024-05-10T23:12:51Z

Hi @ctmphuongg. Thanks for your PR.

I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2024-05-10T23:19:58Z

Codecov Report

Attention: Patch coverage is 0% with 58 lines in your changes are missing coverage. Please review.

Project coverage is 69.26%. Comparing base (7e1c082) to head (ba1a157).
Report is 65 commits behind head on main.

Files	Patch %	Lines
cmd/mqttsource/main.go	0.00%	58 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7919      +/-   ##
==========================================
+ Coverage   69.22%   69.26%   +0.04%     
==========================================
  Files         339      345       +6     
  Lines       19494    16052    -3442     
==========================================
- Hits        13494    11119    -2375     
+ Misses       5337     4247    -1090     
- Partials      663      686      +23

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Cali0707 · 2024-05-13T19:24:45Z

/cc @matzew

Not sure if you have any initial feedback here...

ctmphuongg · 2024-05-14T15:07:47Z

/cc @Leo6Leo

cmd/mqttsource/main.go

Leo6Leo

Great job @ctmphuongg ! The PR looks great overall, and I have tested it locally, everything works perfectly.

Currently the CI is failing due to the check failure on codegen. Please run

./hack/update-codegen.sh

to fix the problem!

cmd/mqttsource/main.go

config/tools/mqttsource/mqttsource.yaml

Leo6Leo · 2024-05-27T22:02:11Z

/ok-to-test

Leo6Leo · 2024-05-28T13:48:26Z

Thanks for fixing the review comment! @ctmphuongg
/lgtm
/cc @pierDipi @creydr @Cali0707

Cali0707

Hey @ctmphuongg awesome start! I left a few comments regarding error handling/setting CE attribute properly.

Otherwise, we generally use camel case variable names, not snake case in Knative. So, I went through and suggested a new variable name on the first occurrence of each variable I saw that was snake case. I would recommend going through and renaming those in your IDE instead of just committing my suggestions, as I didn't actually correct every instance of the variables, just the first

cmd/mqttsource/main.go

Cali0707 · 2024-05-28T15:08:54Z

cmd/mqttsource/main.go

+	newEvent.SetType(eventType)
+	newEvent.SetSource(eventSource)
+	newEvent.SetID(event.ID())
+	_ = newEvent.SetData(cloudevents.ApplicationJSON, data)


We should probably handle the error here, if we fail to set the data does it make sense to dispatch the event?

I'll probably just log the error and return so the event would not be sent?

cmd/mqttsource/main.go

Cali0707

/lgtm
/approve

Thanks for working on this @ctmphuongg !!

If you're interested, I think the next steps for this feature would be to:

Add some docs to explain how to deploy it/use it
Write a blog post about the feature

Let me know if you're interested in this/would like any help!

knative-prow · 2024-05-30T18:20:48Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, ctmphuongg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Cali0707]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ctmphuongg · 2024-07-22T02:51:15Z

Just add a blog post for this PR here! Knative Docs #6062

ctmphuongg added 2 commits May 10, 2024 19:06

imp mqtt source

f926ddb

package installs

2e36a40

knative-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 10, 2024

knative-prow bot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels May 10, 2024

knative-prow bot requested review from Cali0707 and odacremolbap May 10, 2024 23:12

format

f91dbad

knative-prow bot requested a review from matzew May 13, 2024 19:24

fix mqtt sending

86b00ed

knative-prow bot requested a review from Leo6Leo May 14, 2024 15:07

del debug stm

d9d62a3

Leo6Leo reviewed May 14, 2024

View reviewed changes

cmd/mqttsource/main.go Show resolved Hide resolved

add boiler template

cd40d41

ctmphuongg marked this pull request as ready for review May 15, 2024 15:58

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2024

ctmphuongg requested a review from Leo6Leo May 15, 2024 15:58

Merge branch 'knative:main' into mqtt

5038750

Leo6Leo requested changes May 27, 2024

View reviewed changes

cmd/mqttsource/main.go Outdated Show resolved Hide resolved

cmd/mqttsource/main.go Outdated Show resolved Hide resolved

config/tools/mqttsource/mqttsource.yaml Outdated Show resolved Hide resolved

knative-prow bot assigned Leo6Leo May 27, 2024

update codegent

d456eb6

knative-prow bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 27, 2024

knative-prow bot requested a review from creydr May 28, 2024 13:48

knative-prow bot requested a review from pierDipi May 28, 2024 13:48

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label May 28, 2024

Cali0707 requested changes May 28, 2024

View reviewed changes

knative-prow bot assigned Cali0707 May 28, 2024

knative-prow bot removed the lgtm Indicates that a PR is ready to be merged. label May 28, 2024

ctmphuongg added 2 commits May 28, 2024 20:46

rename vars, handle errors

80ed20c

format

ba1a157

ctmphuongg requested a review from Cali0707 May 29, 2024 13:58

Cali0707 approved these changes May 30, 2024

View reviewed changes

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label May 30, 2024

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 30, 2024

knative-prow bot merged commit a58c135 into knative:main May 30, 2024
35 of 36 checks passed

Cali0707 mentioned this pull request Jun 5, 2024

Add e2e test for the MQTT source #7966

Open

ctmphuongg mentioned this pull request Jul 22, 2024

Blog Post for PR: Prototype a MQTT Source #7919 knative/docs#6062

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prototype a MQTT Source #7919

Prototype a MQTT Source #7919

ctmphuongg commented May 10, 2024 •

edited

Loading

knative-prow bot commented May 10, 2024

codecov bot commented May 10, 2024 •

edited

Loading

Cali0707 commented May 13, 2024

ctmphuongg commented May 14, 2024

Leo6Leo left a comment

Leo6Leo commented May 27, 2024

Leo6Leo commented May 28, 2024 •

edited

Loading

Cali0707 left a comment

Cali0707 May 28, 2024

ctmphuongg May 29, 2024

Cali0707 left a comment

knative-prow bot commented May 30, 2024

ctmphuongg commented Jul 22, 2024

Prototype a MQTT Source #7919

Prototype a MQTT Source #7919

Conversation

ctmphuongg commented May 10, 2024 • edited Loading

Proposed Changes

Notes: For testing

knative-prow bot commented May 10, 2024

codecov bot commented May 10, 2024 • edited Loading

Codecov Report

Cali0707 commented May 13, 2024

ctmphuongg commented May 14, 2024

Leo6Leo left a comment

Choose a reason for hiding this comment

Leo6Leo commented May 27, 2024

Leo6Leo commented May 28, 2024 • edited Loading

Cali0707 left a comment

Choose a reason for hiding this comment

Cali0707 May 28, 2024

Choose a reason for hiding this comment

ctmphuongg May 29, 2024

Choose a reason for hiding this comment

Cali0707 left a comment

Choose a reason for hiding this comment

knative-prow bot commented May 30, 2024

ctmphuongg commented Jul 22, 2024

ctmphuongg commented May 10, 2024 •

edited

Loading

codecov bot commented May 10, 2024 •

edited

Loading

Leo6Leo commented May 28, 2024 •

edited

Loading