V2 init #2

Keyrxng · 2024-08-30T13:50:37Z

Resolves ubiquity-os/plugins-wishlist#25
Required by #1

Dual action/worker plugin serving as Telegram Bot kernel and an UbiquityOS plugin.

The worker instance runs with GrammyJS and Hono, supporting the Bot API methods and features.
The action instance runs with GramJS, supporting personal account methods and features.
Creating a workroom
Closing a workroom
Closing a workroom with rate limit handling
Reopening a workroom

Requirements

Config:

  - uses:
    - plugin: ubiquity-os-marketplace/telegram-bridge@development
      with:
        botId: 7487982261
  - uses:  
    - plugin: http://localhost:3000
      with:
        botId: 7487982261

Database: Run the migration inside the Supabase SQL editor. Obtain url and service key.
Repository: Make yourself an admin of the repo. Mint a classic PAT, repo scoped.
TG Bot: Contact the BotFather and create a new bot, obtain the token for it.
TG Account: Head to my.telegram and obtain the api_hash and app_id.
Smee: Create a new url at https://smee.io/new.
yarn setup-env: It'll run through each set of env vars, store and upload them. Enter 0 for BOT_ADMINS for now, don't wrap input in quotes and if you need to run it more than once and you are copy pasting from .dev.vars you should sanitize the additional \ applied to line breaks in the private key.
yarn sms-auth: It'll ask for details via the terminal, store the session and exit.
TG Webhook: smee -u <url> -P "/webhook" in terminal B
yarn worker: should be working at this point. Speak to your bot on TG and use /setcommands.

If you have any issues at this stage let me know please.

To use the workflow functions simply create a new task in a repo and assign a price label. Check the actions tab of the telegram-bot repo and wait. Close, repeat. Reopen, repeat.

Everything should be running smoothly. I E2E tested with new DBs etc a couple of times so hopefully there is no issues for anyone testing.

Features

Commands

/setcommands - private, admin
/chatid - private, groups
/botid - private
/myid - private

Workflow functions

create-chat: Creates a workroom per task once price labeled. @org/repo#1 url schema and posts an invite link on the task
close-chat: Kicks everyone once the task is closed, archives the chat and snapshots userIds.
reopen-chat: Unarchives the chat, re-invites everyone previously involved.

Environment helpers

setup-env: Uses the CLI to walk through saving env vars locally and pushing them to repo secrets.
sms-auth: Requires the personal account holder to create a session via 2FA confirmed via a Telegram client, i.e you'll get a desktop notification from TG with the code.

github-actions · 2024-08-30T13:51:50Z

Unused devDependencies (2)

Filename	devDependencies
package.json	`smee-client` `supabase`

Keyrxng · 2024-08-30T13:54:07Z

rfc @0x4007 @gentlementlegen @rndquu @whilefoo

I'll clean up and streamline things a lot if I should continue ahead with this approach

whilefoo

I don't know what's going in the code, I'm guessing it's a lot of boilerplate. But the full picture I got is that when issue is created the kernel sends event to this plugin which creates a workroom and deletes it when the issue is deleted.
I also saw there is a telegram webhook, what is the purpose of that?

0x4007 · 2024-09-02T20:39:04Z

Let's do individual chat rooms instead of a group with topics. This is to maximize signal to noise.

Also the data should not be deleted, instead, the users should be kicked when the project is closed as complete. This is to reduce clutter on one's telegram inbox and search history. If necessary they can always rejoin from the original link.

Keyrxng · 2024-09-02T20:42:29Z

@whilefoo sorry mate I had some commits I hadn't pushed but to sum things up so far:

This worker is split into two distinct parts.

A: Hono app which supports the telegram bot
B: Standard Ubiquity-OS plugin architecture

A - We are hooked into our BotFather bot and this is the "kernel" for it. So we'll create all commands, functions etc through this worker which is separate from but able to interact with the plugin side.

B - We are also hooked into the kernel as a plugin and in this PR you see the callbacks relating to the "workrooms" feature only.

I'm guessing it's a lot of boilerplate

Yeah still a little bit of boilerplate and a few helper commands I've added for setup idk if they'll be kept. I expected that if this PR progressed I'd need to build in basic commands and features so I haven't cleaned up completely.

But the full picture I got is that when issue is created the kernel sends event to this plugin which creates a workroom and deletes it when the issue is deleted

issues.labeled fires and kernel sends payload to this worker
Github path uses the tg bot api to create/close workrooms, post github comment etc
Bot path can hear the event for new chat created and we can act on forwarding that message etc.
We do not delete chats but we do update the DB with a "closed" status and close/kick everyone from it.
Still to add some safety checks like is it a price label that's been added etc.

This feature aside, it'll house all telegram bot chat commands and features etc.

The task discussion lays out a spec to enable the opening of new group chats from a personal account vs this implementation which is using a main supergroup and opening forums (topics). I spent hours trying various libs and approaches and decided to progress everything else and either it can be tasked out or if it's a blocker more time spent on it.

Here is the troubles with that so far:

The endpoint is not REST and requires special auth handling. There are libraries and frameworks built for the MTProto API however, because we are building Cloudflare workers and not hosting a node server for example packages do not work out of the box and/or they are just incompatible with that env.
Frameworks and libs are built targeting browser or node, while cloudlfare is a weird custom mix of both for using modules such as crypto which is essential in these libs for handling auth and sessions which would need patched for deployment.
We could attempt to handle all that these frameworks do re: MTProto auth, sessions etc and build with Cloudflare V8 in mind but I'm not sure that's worth the time investment.
Implementation details aside, it's not ergonomic for a number of reasons to use separate group chats for each new workroom. The idea behind them is empowering collaboration and oversight of off-site task related activity.

If they are separate chats we have no global overview.
3rd parties like investors eg cannot see activity in these without joining them directly.
They'll only have the team member(s) that join the workroom when it's created unless we automate that

Keyrxng · 2024-09-02T20:59:16Z

Let's do individual chat rooms instead of a group with topics. This is to maximize signal to noise.

Also the data should not be deleted, instead, the users should be kicked when the project is closed as complete. This is to reduce clutter on one's telegram inbox and search history. If necessary they can always rejoin from the original link.

Only saw this after posting my comment above.

Agreed, no chats are deleted. I mentioned my troubles with the MTProto API so far and tried all that I could think over the weekend but had very little luck and it involved a lot of "hacks" like patching node_modules because cloudflare wants any Node module prefixed with "node:" for a start, then the methods differ etc.

Anyway, I'll spend more time on it if need be but in regards to everything else, the way it communicates with the kernel and the way I have built callbacks and what that means for adding additional "plugin"-like features that respond to events, happy for me to proceed with this approach?

0x4007 · 2024-09-02T21:15:27Z

Why not make some sort of GitHub actions based micro service that just opens chat rooms?

It sounds to me that cloudflare is limiting this due to incompatible node modules.

Keyrxng · 2024-09-02T21:37:10Z

Why not make some sort of GitHub actions based micro service that just opens chat rooms?

It sounds to me that cloudflare is limiting this due to incompatible node modules.

I had considered that we'd likely use the workflows and the worker in unison depending on what sort of features we add but hadn't considered it for this feature but I will attempt to do that although it probably isn't pretty.

issues.labeled fires, kernel forwards to this worker.
worker needs to repository_dispatch to kick off the workflow (I think that's the best way)
this worker cannot wait until the response so it would require a dedicated workflow and ts script
The script may need packages separate from src although hopefully not.
I struggle to think rn how to facilitate this worker communicating with itself elegantly through workflows and passing data between.

I'll see how I get on with it and report back

whilefoo · 2024-09-03T08:18:01Z

2. worker needs to repository_dispatch to kick off the workflow (I think that's the best way)

Why does the worker need to start the workflow, the kernel can directly start the workflow? Actually is there any reason why even use a worker if we move everything to the workflow?

0x4007 · 2024-09-03T09:00:24Z

worker needs to repository_dispatch to kick off the workflow (I think that's the best way)

Why does the worker need to start the workflow, the kernel can directly start the workflow? Actually is there any reason why even use a worker if we move everything to the workflow?

Eventually the bot should reply to messages on Telegram, so building for speed is important.

Keyrxng · 2024-09-03T09:18:54Z

Actually is there any reason why even use a worker if we move everything to the workflow?

The workflow is a tool that the worker will use, it is not replacing the worker instance. So it would be a workflow which handles one action: create a group chat using MTProto API. The cf worker can dispatch to it's own repo to execute functions using a different env than what cf provides, which if it works could be awesome.

Why does the worker need to start the workflow, the kernel can directly start the workflow?

So this worker would repo_dispatch to it's own repo to execute the createGroupchat.yml workflow. This way it is not either-or for worker or workflow we can have both for this bot. I don't think it's possible to tell the kernel to fire off an arbitrary workflow is it as we have one entry which is compute.yml?

Eventually the bot should reply to messages on Telegram, so building for speed is important.

This is how I thought of things. If this plugin is a worker we have instant response on the telegram and github side instead of having a dedicated worker just handling telegram requests and us building plugins/features as plugins which interact with the telegram bot. While also having the benefit of dispatching workflows to it's own repo although I thought that would be much further down the line lol

Keyrxng · 2024-09-30T11:50:16Z

Adding it solved my issue, I am successfully logged in.

I don't understand how they two items are related and why it prevented you from logging in when it never stopped me before.

I will keep as a migration and properly name it.

I would also advise to add yarn add -DE smee-client to have smee up and ready for the project, and update wrangler to the latest version to avoid warnings.

Will do.

Glad to hear you are now set up.

gentlementlegen · 2024-09-30T12:00:59Z

It stopped me from logging in because the migration for init never ran so the table didn't exist so adding the entry into Supabase failed when running the script.

Keyrxng · 2024-09-30T12:10:56Z

It stopped me from logging in because the migration for init never ran so the table didn't exist so adding the entry into Supabase failed when running the script.

Ah okay so it skipped it because you ran SB locally my bad, I assumed you'd use the UI my mistake.

With the Supabase CLI I get a successful run of supabase start + supabase db start. If I run db reset which should do the same as db start afaik but resets the local instance, I get errored out regarding container health.

Since I do not ever use Supabase this way, what do you expect to see in regards to how the local SB stuff should work?

The default config.toml comes with a host of unnecessary items but I'm not 100% which affect the local instances so is the default preferred similar to the default tsconfig.json. Remove only things that are commented out or everything that's not required i.e auth email etc...?

gentlementlegen · 2024-09-30T12:22:47Z

I guess you are using windows if you get an error with the healthcheck. You have to opt out from the analytics in the toml and everything will be running.

Keyrxng · 2024-09-30T12:23:21Z

I guess you are using windows if you get an error with the healthcheck. You have to opt out from the analytics in the toml and everything will be running.

much obliged fellow hipster

Keyrxng · 2024-09-30T13:15:40Z

The error remained but I gathered it's OS or my sys related so I moved on. I was able to setup a local DB so it appears to be working

gentlementlegen · 2024-10-01T02:45:57Z

@Keyrxng I think I have the full setup, I changed my configuration file and all that the kernel does when I try to use the command /myid is No handler found for event issue_comment. What am I doing wrong?

.github/workflows/compute.yml

Keyrxng · 2024-10-01T02:51:46Z

@Keyrxng I think I have the full setup, I changed my configuration file and all that the kernel does when I try to use the command /myid is No handler found for event issue_comment. What am I doing wrong?

That's a telegram bot command, send that to your bot father bot on tg.

Only issues.labelled | closed | reopen will trigger a workflow function. The worker should respond to any bot command in the readme, you may have to use /setcommands on the tg bot too

I am not sure re: payload logging I will look into it first thing

README.md

Keyrxng · 2024-10-02T15:54:11Z

Should I make a task and create a setup tutorial, showing e2e functionality with videos, would that be beneficial?

Should I make a task and create a test suite that covers GitHub and both TG apis and start to work on that?

gentlementlegen · 2024-10-04T04:27:19Z

I think having the full list of steps numbered that ensure a successful setup would be enough. Sorry I'll test again the setup, working on multiple things.

gentlementlegen · 2024-10-04T08:39:26Z

I confirm it works. I got a room chat created on issue creation:

Meniole#4

gentlementlegen · 2024-10-04T08:41:22Z

@Keyrxng Could you add a migration script and type generation script similar to here?

Keyrxng · 2024-10-04T14:14:21Z

@Keyrxng Could you add a migration script and type generation script similar to here?

@gentlementlegen #3 removes the Supabase dependency completely and will be merged very soon, considering this should I still add your request?

gentlementlegen · 2024-10-05T07:22:45Z

@Keyrxng then that is fine.

gentlementlegen

We will need to add the manifest action later on for this to have the configuration checked by the kernel.

whilefoo · 2024-10-07T20:31:10Z

@Keyrxng I get this error when I put a label on an issue:

[2024-10-07T20:27:25.477] [INFO] - [Running gramJS version 2.24.10]
[2024-10-0[7](https://github.com/ubiquity-whilefoo/telegram-bot/actions/runs/11223403676/job/31197960956#step:6:8)T20:27:25.480] [INFO] - [Connecting to 149.154.167.91:80/TCPFull...]
[2024-10-07T20:27:25.5[8](https://github.com/ubiquity-whilefoo/telegram-bot/actions/runs/11223403676/job/31197960956#step:6:9)3] [INFO] - [Connection to 149.154.167.91:80/TCPFull complete!]
[2024-10-07T20:27:25.584] [INFO] - [Using LAYER 184 for initial connect]
	› Creating chat with name: 
	› {
	    "chatName": "@ubiquibot-whilefoo-testing/testing#1",
	    "caller": "createChat"
	  }
	›   ↳  <anonymous> (/home/runner/work/telegram-bot/telegram-bot/src/handlers/workflow-proxy.ts:28:[9](https://github.com/ubiquity-whilefoo/telegram-bot/actions/runs/11223403676/job/31197960956#step:6:10))
	⚠ Error in creating chat: 
	⚠ {
	    "er": {},
	    "caller": "createChat"
	  }
	⚠   ↳  <anonymous> (/home/runner/work/telegram-bot/telegram-bot/src/handlers/workflow-proxy.ts:28:9)

https://github.com/ubiquity-whilefoo/telegram-bot/actions/runs/11223403676/job/31197960956

chore: init

664a31c

Keyrxng added 12 commits September 1, 2024 22:23

chore: attempts at MTProto auth > failed > moving on

6023f85

chore: repo structure, remove unused

58ba05e

feat: callback proxy

0673806

chore: supabase

cd59b43

chore: use callbacks, error logging and env

0bea908

chore: getDeepVal util and close workroom on issues.closed

19fc70c

chore: supa chats and closed on issues.closed

1846d98

chore: supa chats and closed on issues.closed

61f1da8

chore: use numeric status response

bd321d6

chore: reopen chatroom on issues.reopen

e212e2b

chore: fix bot_admins schema, use workroom vs chatroom names

2b46d55

fix: proxy usage, event support for labeled/close/reopen with storage

bf8dee4

whilefoo reviewed Sep 2, 2024

View reviewed changes

Keyrxng added 6 commits September 3, 2024 18:29

chore: test

c6e7af5

chore: workflow functions

a94a2e3

chore: typeguards and error helpers

36124fa

chore: proxy types and new env vars

43e13ab

chore: repo dispatch approach

d266dfd

chore: remove need to specify workflow function

1c23980

Keyrxng added 2 commits September 30, 2024 14:09

chore: log pat error, add supabase cli, add smee-client

2d030cd

chore: knip ignore smee and supabase

3db28f6

gentlementlegen reviewed Oct 1, 2024

View reviewed changes

.github/workflows/compute.yml Show resolved Hide resolved

gentlementlegen reviewed Oct 1, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

Keyrxng added 2 commits October 2, 2024 16:47

chore: log wf payload, update readme

188cd1d

chore: remove packageManager

e52735d

Keyrxng mentioned this pull request Oct 3, 2024

DM Notifications #3

Merged

gentlementlegen approved these changes Oct 5, 2024

View reviewed changes

gentlementlegen requested review from 0x4007 and whilefoo October 6, 2024 16:39

ubiquity-os bot merged commit 77a12d5 into ubiquity-os-marketplace:development Oct 12, 2024
2 checks passed

This was referenced Oct 22, 2024

feat: task limit improvement ubiquity-os-marketplace/command-start-stop#66

Merged

Create on assignment #8

Merged

Telegram Bot V2 #1

Closed

Keyrxng mentioned this pull request Nov 28, 2024

Telegram GitHub Issue "Workroom" ubiquity-os/plugins-wishlist#25

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V2 init #2

V2 init #2

Keyrxng commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

Keyrxng commented Aug 30, 2024

whilefoo left a comment

0x4007 commented Sep 2, 2024 •

edited

Loading

Keyrxng commented Sep 2, 2024

Keyrxng commented Sep 2, 2024

0x4007 commented Sep 2, 2024

Keyrxng commented Sep 2, 2024 •

edited

Loading

whilefoo commented Sep 3, 2024

0x4007 commented Sep 3, 2024

Keyrxng commented Sep 3, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Oct 1, 2024 •

edited

Loading

Keyrxng commented Oct 1, 2024 •

edited

Loading

Keyrxng commented Oct 2, 2024

gentlementlegen commented Oct 4, 2024

gentlementlegen commented Oct 4, 2024

gentlementlegen commented Oct 4, 2024

Keyrxng commented Oct 4, 2024

gentlementlegen commented Oct 5, 2024

gentlementlegen left a comment

whilefoo commented Oct 7, 2024

V2 init #2

V2 init #2

Conversation

Keyrxng commented Aug 30, 2024 • edited Loading

Requirements

Features

Commands

Workflow functions

Environment helpers

github-actions bot commented Aug 30, 2024 • edited Loading

Unused devDependencies (2)

Keyrxng commented Aug 30, 2024

whilefoo left a comment

Choose a reason for hiding this comment

0x4007 commented Sep 2, 2024 • edited Loading

Keyrxng commented Sep 2, 2024

Keyrxng commented Sep 2, 2024

0x4007 commented Sep 2, 2024

Keyrxng commented Sep 2, 2024 • edited Loading

whilefoo commented Sep 3, 2024

0x4007 commented Sep 3, 2024

Keyrxng commented Sep 3, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

Keyrxng commented Sep 30, 2024

gentlementlegen commented Oct 1, 2024 • edited Loading

Keyrxng commented Oct 1, 2024 • edited Loading

Keyrxng commented Oct 2, 2024

gentlementlegen commented Oct 4, 2024

gentlementlegen commented Oct 4, 2024

gentlementlegen commented Oct 4, 2024

Keyrxng commented Oct 4, 2024

gentlementlegen commented Oct 5, 2024

gentlementlegen left a comment

Choose a reason for hiding this comment

whilefoo commented Oct 7, 2024

Keyrxng commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading

0x4007 commented Sep 2, 2024 •

edited

Loading

Keyrxng commented Sep 2, 2024 •

edited

Loading

gentlementlegen commented Oct 1, 2024 •

edited

Loading

Keyrxng commented Oct 1, 2024 •

edited

Loading