NestJs Integration

[kstan79]

So far I'd tested integrate with nestjs and using openapi. it work smoothly, below is example of controller, service class, and swagger-ui result, it work as expected!

//controller
import { Controller, Get,Post,Req,Param} from '@nestjs/common';
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  //get all workflow
  @Get('/')
  getWorkflows() {
    return this.appService.getPendingWorkflows();
  }
  //create workflow
  @Post('/:workflow')
  postStart(@Param('workflow') workflow:string) {
    return this.appService.startWorkflow(workflow);
  }
  // others required endpoint
  //get /:workflow/:id  //fetch info
  //put /:workflow/:id  //update info
  //post/:workflow/:id/:taskId  //invoke {user:xxx}
  //...
}

// service class
import { Injectable,Logger } from '@nestjs/common';
import {BPMNServer} from 'bpmn-server'
import {configuration} from './configuration'

@Injectable()
export class AppService {
  private readonly logger = new Logger();

  bpmnServer:BPMNServer
  constructor(){
    this.bpmnServer = new BPMNServer(configuration)
  }

  async startWorkflow(workflow:string){
    const result =  await this.bpmnServer.engine.start(workflow);        
    return {
      id: result.id,
      data:result.getData('id'),
      status: result.status
    }
  }
  async getPendingWorkflows(){
    const tmp:any[] = await this.bpmnServer.dataStore.findInstances({
      status:'running',
    },'summary')

    const result = tmp.map((proc)=>({
      id: proc.id,
      workflowname: proc.name,
      startedAt: proc.startedAt,
      tasks :proc.items
              .filter((task)=>task.status=='wait')
              .map((task)=>({
                id:task.id,
                type:task.type,
                assignee:task.assignee,
                candidateUsers:task.candidateUsers,
                candidateGroups:task.candidateGroups
              }))
    }))

    return result
  }
}

I'm curious so far any documentation of upload bpmn file? I don't know the process of upload file until the engine support the new definition.

[ralphhanna]

Thank you for your great contribution, this looks very good.

I am working on a new API still under development, can release soon that will help you alot

below is an example of calls:

    let api = new Request(server, { userId: 'user1', userGroups:['Owner','Others'],  });

    
    let items = await api.data.findItems({ "items.candidateGroups": "Owner", "items.candidateUsers": "User1" });
    items = await api.findItems({ "$or": [{ "items.candidateGroups": "Owner" }, { "items.candidateUsers": "User1" }]});


    console.log(api.user);
    api = new Request(server, APIUser.SystemUser());
    console.log(api.user);

    let response = await api.engine.start('Buy Used Car');

    response = await api.engine.invoke({id: response.id , "items.elementId": 'task_Buy' });
    var src = await  api.model.getModel('Buy Used Car');
    var list = await api.model.listModels({name:'deleteLater'});

    await api.model.saveModel('deleteLater', src, null);
    await api.model.deleteModel('deleteLater');
    list = await api.model.listModels({});

Your feedback on the above example is highly appreciated

BTW, in your call of getPendingWorkflows, you can use findItems instead of findInstances it uses MongoDB faster

[kstan79]

thanks reply, i mean so far any reference of upload bpmn file? Is it I put the file in defination folder it will auto generate new model, or i need to read file content then run await api.model.saveModel('deleteLater', src, null);?

I saw there is svg property too, I shall update manually in model collection?

thanks getPendingWorkflows advise!

[ralphhanna]

auto generate uses the saveModel function
But you can use saveModel for files outside of definitions folder or when you keep all your models in DB.
The last parameter is the svg

     async saveModel(name, source,svg) 

[ralphhanna]

Can you please share a sample project for nestJs, OpenAPI and swagger
I will integrate into next release
Thanks very much

[kstan79]

Will do, also will write setup documentation for you

[kstan79]

you may refer minimal example here:
https://github.com/kstan79/nestjs-bpmn-server

[kstan79]

since you will integrate with nestjs at next release, i think it is better i wait your latest release, Im worry it will have huge impact. By the way, I have more complex working example with:

1. keycloak integration
2. swagger-ui perform api-key/oauth2 autorization
3. some working middleware, decorators, practise of request scope user.context
4. api-request logging
5. mongodb transaction (if neccessary)

if you need help in nestjs kindly let me know, we can create discord channel for this project, good for public discussion. since this project community not only 1-2 person now

[ralphhanna]

As you see from sample code above, the new API requires users identification, there two types of Requests:

User Initiated Calls

NextJs need to pass the userName and userGroups as well as Tenant (if needed)

    let api = new APIClient(server, { userId: 'user1', userGroups:['Owner','Others'], tenantId: tenant });
   ap.engine.start(....);

This api should have the scope of the session; i.e. Injected into the session

System Initiated Calls

This is needed to invoke Service Calls from the application logic

    api = new APIClient(server, APIUser.SystemUser());
    api.engine.invoke(...);

This api should have the scope of the application like your example

Alternatively:
We can use one APIClient for the entire app but switch users before every call

  api.setUser({ userId: 'user1', userGroups:['Owner','Others'], tenantId: tenant });

[kstan79]

Hm your user session mean the after user authenticated session? I hope you just give example and it not exists in bpmn server, cause session in backend is bad ideal. Session is frontend scope and implement in backend not only make things complicated, also add burden on server.

Internally with my team we did discuss what is the right way of supply user/group information into bpmn server.

Due to different product having different requirement, I think it maybe good for initiate bpmn supply with callback url(or call back function). Bpmn engine can obtain callback info from source. Like obtain email, tel, supervisor, who is superadmins and etc.

With this approach you can have peace of mind, static structure, but reserve flexibility.

However, I'm feel this ideal is rough too

Bpmn just need to verify assignee/candidateUser/candidategroup from call back url (or callback function) I think problem solve

[kstan79]

i think sample code better can illustrate it:

remote bpmn client wish to execute workflow:

//since using code gen from openapi-generator, so below code is different with your documentation
// you can assume below code from frontend or another microservice wish to instantiate new bpmn workflow

import {bpmnApi} from './openapi' //code generate by openapi-generator using openapi.yaml. example typescript-axios
const token = 'xxxxxxxxx' //obtain from keycloak, github/google/microsoft or etc

const axiosconfig = {url: 'http://localhost:3000/bpmn', headers:{ Authorization: `Bearer ${token}`} }
const api = new bpmnApi(axiosconfig)
const data : any = {id:'1111', tenantId:1,  invoiceNo:'SI-0001', amount:2000, details:[...]}

const workflowSetting = {
   callback: 'http://localhost:8000/inv/callback',
   userservice:  'http://localhost:8000/userservice',
   tenantId:1,....
} 

const result = await api.startWorkflow('samplebpmn', data, workflowSetting )

for developer friendly, and easy of manage source code, we wish to

1. define assignee, candidateUsers, candidateGrouo in bpmn instead of source code
2. my opinion is avoid developer manually fetch current user/group details, no need to find who is their supervisor, and others kind of role fetching cause they may fetch wrongly. and no point since it well define in bpmn
3. as above source code, it is tedious and hard to find what kind of dynamic user/dynamic group needed. to audit the code we shall compare with bpmn file which is kind of manual job. We use bearer token, which recommended security token and it supply current user info. we can use the token info to communicate with userservice, query all the the static/dynamic user/group according definition in bpmn
4. if no userservice available, we can still fall back manually supply dynamic user properties in workflowSetting as your example. the dynamic user property can include your example plus with
   a. initiator's supervisor
   b. push notification property like mobile no, tel, pusher session, firebaseid, cause bpmn require this info to intiate reminder or messaging

in short, after think many time, dont supply user/group in source code. no define = no bug

userservice callback function and callback api can something like below:

POST http://localhost:8080/userservice/getInfo
{
  assignee: 'me', //read from assignee properties
  users: ['user1','user2'],   // read from candidateUsers properties
  groups: ['accountants','ceo']. //read from candidateGroup
}

result of user service api (yet to decide)

[{uid:'xxxx', name:'user1', email:'[email protected]', firebaseToken:'xxxxx', hp:'000000', ...},{....}]

above is rough ideal but i think can illustate well my ideal, which is don't maintain user in bpmn server

[kstan79]

below is example of some static group (ceo, branchManagers) and some dynamic user ($supervisor, $me). If we implement call back user service, we able to auto send message/email/push notification to everybody in the group/users, call for action.

<?xml version="1.0" encoding="UTF-8"?>
<bpmn:definitions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:dc="http://www.omg.org/spec/DD/20100524/DC" xmlns:di="http://www.omg.org/spec/DD/20100524/DI" xmlns:camunda="http://camunda.org/schema/1.0/bpmn" id="Definitions_1" targetNamespace="http://bpmn.io/schema/bpmn">
  <bpmn:process id="Process_1" name="approve1" isExecutable="false">
    <bpmn:startEvent id="StartEvent_1">
      <bpmn:outgoing>Flow_1a63g0z</bpmn:outgoing>
    </bpmn:startEvent>
    <bpmn:endEvent id="Event_0qpgd23">
      <bpmn:incoming>Flow_15oysi6</bpmn:incoming>
    </bpmn:endEvent>
    <bpmn:userTask id="Activity_07kqvhs" name="approve1" camunda:assignee="$supervisor" camunda:candidateGroups="branchManagers,ceo" camunda:dueDate="2000-01-01T09:10:10" camunda:followUpDate="2000-01-01T00:10:10">
      <bpmn:incoming>Flow_1a63g0z</bpmn:incoming>
      <bpmn:outgoing>Flow_1x9zeai</bpmn:outgoing>
    </bpmn:userTask>
    <bpmn:sequenceFlow id="Flow_15oysi6" sourceRef="Activity_068mwb7" targetRef="Event_0qpgd23" />
    <bpmn:sequenceFlow id="Flow_1x9zeai" sourceRef="Activity_07kqvhs" targetRef="Activity_068mwb7" />
    <bpmn:sequenceFlow id="Flow_1a63g0z" sourceRef="StartEvent_1" targetRef="Activity_07kqvhs" />
    <bpmn:userTask id="Activity_068mwb7" name="acknowledge and proceed" camunda:assignee="$me" camunda:candidateGroups="customerservice">
      <bpmn:incoming>Flow_1x9zeai</bpmn:incoming>
      <bpmn:outgoing>Flow_15oysi6</bpmn:outgoing>
    </bpmn:userTask>
  </bpmn:process>
  <bpmndi:BPMNDiagram id="BPMNDiagram_1">
    <bpmndi:BPMNPlane id="BPMNPlane_1" bpmnElement="Process_1">
      <bpmndi:BPMNShape id="_BPMNShape_StartEvent_2" bpmnElement="StartEvent_1">
        <dc:Bounds x="-18" y="92" width="36" height="36" />
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape id="Activity_0f4317o_di" bpmnElement="Activity_07kqvhs">
        <dc:Bounds x="100" y="70" width="100" height="80" />
        <bpmndi:BPMNLabel />
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape id="Activity_1kohkxu_di" bpmnElement="Activity_068mwb7">
        <dc:Bounds x="270" y="60" width="100" height="80" />
        <bpmndi:BPMNLabel />
      </bpmndi:BPMNShape>
      <bpmndi:BPMNShape id="Event_0qpgd23_di" bpmnElement="Event_0qpgd23">
        <dc:Bounds x="512" y="62" width="36" height="36" />
      </bpmndi:BPMNShape>
      <bpmndi:BPMNEdge id="Flow_15oysi6_di" bpmnElement="Flow_15oysi6">
        <di:waypoint x="370" y="80" />
        <di:waypoint x="512" y="80" />
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge id="Flow_1x9zeai_di" bpmnElement="Flow_1x9zeai">
        <di:waypoint x="200" y="110" />
        <di:waypoint x="235" y="110" />
        <di:waypoint x="235" y="100" />
        <di:waypoint x="270" y="100" />
      </bpmndi:BPMNEdge>
      <bpmndi:BPMNEdge id="Flow_1a63g0z_di" bpmnElement="Flow_1a63g0z">
        <di:waypoint x="18" y="110" />
        <di:waypoint x="100" y="110" />
      </bpmndi:BPMNEdge>
    </bpmndi:BPMNPlane>
  </bpmndi:BPMNDiagram>
</bpmn:definitions>

[ralphhanna]

I agree with above statements, here is a summary of next release design:

1. bpmnServer (npm package) is a backend only, is passed user information in the API
2. The Application (front end) is concerned with managing users authentication and user info (UserName,Groups,Tenant)
3. bpmn model will define assignee,userGroups,etc. as in example you provided
4. bpmnServer will ensure that current user (passed by 2) s part of the assigned group (defined in 3)
5. for group notification, a callback is required to the application to list users in a group , this callback implementation is left entirely to the application
6. Demo Application(Express or NestJS) , provides a complete implementation of users management using Passport and MongoDB
7. I will provide a full tutorial for the above

[kstan79]

Thanks of make it clear, keyword "demo application" scope responsibility very well, bpmn-server library doesnt store and responsible in manage user database.

[kstan79]

For (4), i think we have to further polish the ideal:

Assume below is the getPendingWorkflow :

  async getPendingWorkflows(appuser: UserContext,) {
    const tmp:any[] = await this.bpmnServer.dataStore.findItems({
      'items.status':'wait',      
    })    
 }

According observation of this project, I notice there is multiple stages of process user task:
[1. initiate user task]
it resolve static and dynamic assignee, candidate users/groups and update the instance record, below is some of my comment:
a. hard coded assignee, candidate users/groups is captured perfectly, and able to filter as expected
b. dynamic property which is inside context this is capture properly (example this.userId, this.data.preparedBy)
c. dynamic property which is not inside this not able to process, Example want to set assignee as this.userId's supervisor, and there is no this property exists in context. As we know, obtain remote data require external db/api call, we need place to pass in call back during user task begin begin. Else, $supervisor can't resolve, at the end findItem() will detect asignee=null
d. when this user task activated, we wish to push messaging. again, additional remote property is required and it not inside the context. Solution is to execute another callback/method (may define in delegates).

• i recommend define external resolver in delegates, which can use by user task if notice assignee, candidateUser, candidateGroups cannot resolve internally.
• Also create dummy method messagingService, community can override this method to use own way send notification message

[2. search task by specific user]

1. usually we need 1 simple function which allow us to search specific user task
2. in this function we only want to supply the user name and user group the user belong to, such as: const props = {userId:'kstan', group:['g1','g2']}
3. base on documentation we shall use findItems(), which I feel not developer friendly, cause developer need to find way filter 3 properties (assignee, candidate user & groups).
   ** I recommend we have dedicated method getPendingUserTasks, with fix method name , no more confusion, at what situation use what kind of filter, return what kind of property

const pendingItems = await this.bpmnServer.getPendingUserTasks(props)

[3. invoke specific user task]

1. base on documentation, there is dedicated method i think thats good:

    response = await server.engine.invoke({id: response.id , "items.elementId": 'task_Buy' },{},'user1');

[4. end of user task]

1. it maybe good we can have a way to tell user the specifc user task invoke (either accept, reject or etc). Again can define in delegate afterUserTask callback. Community can override this method in delegate as they want.

I agree with above statements, here is a summary of next release design:

1. bpmnServer (npm package) is a backend only, is passed user information in the API
2. The Application (front end) is concerned with managing users authentication and user info (UserName,Groups,Tenant)
3. bpmn model will define assignee,userGroups,etc. as in example you provided
4. bpmnServer will ensure that current user (passed by 2) s part of the assigned group (defined in 3)
5. for group notification, a callback is required to the application to list users in a group , this callback implementation is left entirely to the application
6. Demo Application(Express or NestJS) , provides a complete implementation of users management using Passport and MongoDB
7. I will provide a full tutorial for the above

[kstan79]

to make life easy. I would like to suggest simplest userResolver I can imagine:

    <bpmn:userTask id="Activity_07kqvhs" name="approve1" camunda:assignee="@(supervisor)" camunda:candidateUsers="jackie,$(this.userId)" camunda:candidateGroups="branchManagers,ceo" camunda:dueDate="2000-01-01T09:10:10" camunda:followUpDate="2000-01-01T00:10:10">

first it will resolve by bpmn-server's resolver become like below:

{ ...
assignee:'@(supervisor)',  // we can remain using '$', I just want to make it clear different
candidateUsers:['jackie', 'kstan'],
candidateGroups:['branchManagers', 'ceo'],
}

then if detect any thing unresolve then execute remote resolver:

const toResolve  = { 
assignee:'@(supervisor)',  // we can remain using '$', I just want to make it clear different
candidateUsers:['jackie', 'kstan'],
candidateGroups:['branchManagers', 'ceo'],
}

const afterResolve = await  this.delegates.userResolver(toResolve, this.data, this.uid, item,)

/*
{
assignee: 'john',
candidateUsers:['jackie', 'kstan'],
candidateGroups:['branchManagers', 'ceo'],
}
*/

[ralphhanna]

Not sure but let me try this:

1. User1 will Start Process Request Vacation
2. System will store the initiator as item.data.requester='User1'
3. Since Task Request has assignee=item.data.requester only User1 can invoke this task
4. Once Task Request is completed by User1 , Task Approve will be created.
5. In Current Release: Javascript trigger will calculate the supervisorId and assign it the task, this is done by invoke a service call from myServices class, which in turn can call DB calls.
6. Next Release; you will be able to calculate the supervisor in the assignee field like this; #(services.getSupervisor(this.data.requester))
7. To Notify the supervisor user, another JS trigger will invoke notify method (documentation to come next release)

As for findItems in next release:

    let pending = await bpmnAPI.data.findItems({ "items.status": 'wait', "items.type": 'bpmn:UserTask' }, user); 

We can easily add another method instead:

    let pending = await bpmnAPI.data.getPendingTasks({}, user); 

will return all pending tasks that the user is AUTHORIZED TO SEE
To See only Tasks Assigned to User

    let pending = await bpmnAPI.data.getPendingTasks({'items.assignee':user.userName},user); 

or you can use filter for the items to check item.assignee

[kstan79]

i have a new ideal which is create new injectable class extend BPMNServer class, maybe it is workable. I'll try up and feed back here.

[ralphhanna]

Please Note that we have to separate bpmn-server from the Workflow App.
Workflow App is a specific implementation of bpmn-server with own UI

Here is a sample folder structure under release 2.0.0 for the DEMO App

• UserAccess Folder

• Provides User Authentication using Passport
• UserService is accessible to the rest of the app and to bpmn-server
• configuration.ts/js is now in WorkflowApp folder and has an additional code:

		userService: function (server) {
			return new UserService(server);
		}

[kstan79]

yes, the implementation of workflow service consider as "product", bpmn-server at the moment play role as library

[kstan79]

I'd successfully make bpmn-server delegates integrate into nestjs as below:
make deletgate patter as nestjs style, and inject require provider:

//workflow.delegate.ts

import { CustomerService } from 'src/simpleapp/services/cust.service';
import { Injectable } from "@nestjs/common";
import { IExecution, Item, NODE_ACTION, FLOW_ACTION, IAppDelegate , IDefinition,BPMNServer} from "bpmn-server";
import { moddleOptions} from 'bpmn-server/src/elements/js-bpmn-moddle';
import { CategoryService } from 'src/simpleapp/services/cat.service';
import { InvoiceService } from 'src/simpleapp/services/inv.service';

@Injectable()
export class WorkflowDelegate implements IAppDelegate {
    server:BPMNServer;
    servicesProvider: any;
    constructor(
        //we can inject many provider here
        private custserv:CustomerService,
        private catserv:CategoryService,
        private invserv:InvoiceService,
    ){

    }

    //config will use setServer to define bpmnserver
    setServer(server:BPMNServer){
        this.server = server;
        let self = this;
        server.listener.on('all', async function ({ context, event }) {
            await self.executionEvent(context, event);
        });
    }
    
    startUp(options) {}
    sendEmail(to, msg, body) {
        throw Error("sendEmail must be implemented by AppDelegate");
    }

    get moddleOptions() {
        return moddleOptions;
    }
    async executionStarted(execution: IExecution) {}
    async executionEvent(context,event) {}
    
    async messageThrown(messageId, data, messageMatchingKey: any , item: Item) {
        const msgId = item.node.messageId;
        item.context.logger.log("Message Issued" + msgId);
        // issue it back for others to receive
        const resp = await item.context.engine.throwMessage(msgId, data, messageMatchingKey);
        if (resp && resp.instance) {
            item.context.logger.log(" invoked another process " + resp.instance.id + " for " + resp.instance.name);
        }
        else
            await this.issueMessage(messageId, data);
    }
    
    async issueMessage(messageId, data) {}
    async issueSignal(signalId, data) {}
    async signalThrown(signalId, data, messageMatchingKey: any, item: Item) {
        item.context.logger.log("Signal Issued" + signalId);

        const resp = await item.context.engine.throwSignal(signalId, data, messageMatchingKey);
        if (resp && resp.instance) {
            item.context.logger.log(" invoked another process " + resp.instance.id + " for " + resp.instance.name);
        }
        else
            await this.issueSignal(signalId, data);
    }
    async serviceCalled(serviceName, data, item: Item) {
        item.context.logger.log("Service called:"+serviceName+data);
    }



    scopeEval(scope, script) {
        let result;

        try {
            var js = `
            var item=this;
            var data=this.data;
            var input=this.input;
            var output=this.output;
            return (${script});`;
            result = Function(js).bind(scope)();
        }
        catch (exc) {
            console.log('error in script evaluation', js);
            console.log(exc);
        }
        return result;
    }
    async scopeJS(scope, script) {
        const AsyncFunction = Object.getPrototypeOf(async function () { }).constructor;
        let result;
        try {
            var js = `
            var item=this;
            var data=this.data;
            var input=this.input;
            var output=this.output;
            ${script}`;
            result = await new AsyncFunction(js).bind(scope)();
            scope.token.log("..executing js is done " + scope.id);
        }
        catch (exc) {
            scope.token.log("ERROR in executing Script " + exc.message + "\n" + script);
            console.log('error in script execution', js);
            console.log(exc);
        }
        return result;

    }
}

Prepare config class, which inject WorkflowDelegate, then obtain configuration object via getConfig():

// workflow.config.ts

import { Injectable, Logger } from '@nestjs/common';
import {
  Configuration,
  ModelsDatastore,
  ModelsDatastoreDB,
  DataStore,
  NoCacheManager,
  CacheManager,
  BPMNServer,
} from 'bpmn-server';
import { WorkflowDelegate } from './workflow.delegate';

let _this:WorkflowConfig  //we need another variable to access WorkflowConfig 
@Injectable()
export class WorkflowConfig {    
    constructor(private workflowdelegate:WorkflowDelegate){//inject delegate into config class

    }
    setServer(server:BPMNServer){
        //this become  Configuration object, use _this instead
        _this.workflowdelegate.setServer(server)
        return _this.workflowdelegate
    }
    getConfig(){        
        const templatesPath = __dirname + '/emailTemplates/';
        _this = this //store reference of current object into _this
        const configuration = new Configuration({           
            definitionsPath: process.env.BPMN_PATH,
            templatesPath: templatesPath,
            timers: {
                precision: 3000,
            },
            database: {
                MongoDB: {
                db_url: process.env.MONGO_DB_URL,
                db: process.env.MONGO_DB_NAME, //'bpmn'
                },
            },
            apiKey: process.env.API_KEY,
            /* Define Server Services */
            logger: function (server) {
                return new Logger();
            },
            definitions: function (server:BPMNServer) {
                return new ModelsDatastore(server);
            },
            appDelegate:_this.setServer,            
            dataStore: function (server:BPMNServer) {
                return new DataStore(server);
            },
            cacheManager: function (server:BPMNServer) {
                return new NoCacheManager(server);
            },
        });
        return configuration
    }
}

change service class to inject configuration class, and initialize bpmnserver with the configuration

//workflow.service.ts 
@Injectable()
export class WorkflowService {
  protected bpmnServer: BPMNServer;
  private readonly logger = new Logger();
  constructor(
    private workflowconfig:  WorkflowConfig
  ) {
    this.bpmnServer = new BPMNServer(this.workflowconfig.getConfig());
  }

since only delegate require to access others injectable resources in nestjs, so I only make config and appdelegate become injectable. others remain

[ralphhanna]

Great job
I am easy about integrating configuration and appDelegate into one class, less is better!!!
Thanks