Documentation for Usage

[apersaud]

Hi everyone,

This module looks interesting, however I was having a hard time finding documentation on "what are the use cases this solves" or the benefits of configuring Parse Server to use this feature. Is this for background job processing, distributed query performance, or is this regarding the live query server? Any additional info would be useful.

[acinader]

hi @apersaud! Oh, you're jogging my memory here :).

I made this when @flovilmart was working on parse-server/src/ParseMessageQueue and it seemed like a perfect use case for sqs so I did this little repo as a proof of concept. It works. I've tested it, a while ago.

The use case that @flovilmart was interested in was distributing workload for push notifications across a cluster of machines which if I recall correctly, is what I tested.

I am not currently using it in production just because we don't do large pushes so I don't need it...yet.

As far as documentation goes, I suggest taking a look at: https://github.com/parse-community/parse-server-sqs-mq-adapter/blob/master/spec/SQSEventEmitterMQ.spec.js

which is the unit test. Note that one unit test is x'd out: https://github.com/parse-community/parse-server-sqs-mq-adapter/blob/master/spec/SQSEventEmitterMQ.spec.js

which shows how to use in production.

While I am not currently using, I'd love to help you use it if you think it's a fit and would be glad to do code reviews and/or try to help out if you're having an issue.

[flovilmart]

We’re using something similar now with Google Cloud Pub/Sub + Google Cloud Functions to send the pushes from the functions instead of the server to prevent impacting our front end instances

[flovilmart]

You can very well imagine the same setup with AWS lambdas (where you would run the push worker) and SQS as the message queue to communicate with the lambdas from parse server. I’ll write a blog post about it soon :)

[vincenzoarieta93]

@acinader @flovilmart how parse-server-sqs-mq-adapter can be used to remove the push notifications overhead from my core server that relies on Parse? Do I need to implement a lambda consumer too?

[acinader]

Yes, that was the idea. The lambda would then fire up a parse-server instance to handle the push.

[vincenzoarieta93]

@acinader will the aforesaid lambda function use Parse sdk to fire up push notifications?

[acinader]

You can't send pushes with the SDK. You need the parse-server in the lambda.

[vincenzoarieta93]

@acinader Yes. That's what I meant. The lambda function will use Parse sdk to send notifications. Is that right?

[acinader]

We use SDK to denote the various parse-server clients, like the JS, IOS, and PHP sdk's.

So what I mean is: an instance of parse-server will run in lambda, not the JS sdk.

Pretty sure we are understanding each other and saying the same thing, but making sure :).

[vincenzoarieta93]

@acinader you mean by importing the module parse-server in the lambda? If not, please help me out to understand :)

[acinader]

yup

[vincenzoarieta93]

@acinader what is supposed to be the payload passed by parse-server-sqs-mq-adapter to the lambda?

[acinader]

ha. I am not exactly sure, and I don't think we even really need to know.

Here you can see how the push queue is configured: https://github.com/parse-community/parse-server/blob/master/src/Push/PushQueue.js#L19

Here you can see a test that shows how the queue is used https://github.com/parse-community/parse-server-sqs-mq-adapter/blob/master/spec/SQSEventEmitterMQ.spec.js

The use case that it was designed for did not include lambda, but assumed dedicated servers that would the queue would empty into.

So I am not exactly sure how to put lambda in the middle. We'd want to add a 'lambda consumer' i think. Probably the best first step would be to get a test setup using two servers, one to create the pushes and put them in the sqs and a second one to process the queue. Once that's working, then we'd look at how to lambda consumer that could be configured into the lambda parse-server instance.

Or you could just not use lambda and have dedicated ec2 instances (or beanstalk, or ecs, or however you're doing it) for sending pushes. The labmda solution would be a nice cost effective way to handle the load without affecting realtime users of your parse-server, but dedicating instances could be a first step?