From a29bbd017b27158bb675b12097e420def9f64aab Mon Sep 17 00:00:00 2001 From: wbc6080 Date: Wed, 6 Dec 2023 10:28:15 +0800 Subject: [PATCH] update mapper docs Signed-off-by: wbc6080 --- docs/concept/device/mapper.md | 24 +- docs/developer/mapper-framework.md | 165 +++++++++ docs/developer/mappers/bluetooth.md | 328 ------------------ docs/developer/mappers/index.md | 137 +++++++- docs/developer/mappers/modbus.md | 81 ----- .../current/concept/device/mapper.md | 20 +- .../current/developer/mapper-framework.md | 160 +++++++++ .../current/developer/mappers/bluetooth.md | 328 ------------------ .../current/developer/mappers/index.md | 141 +++++++- .../current/developer/mappers/modbus.md | 81 ----- static/img/device/dmi-datapanel.png | Bin 0 -> 625232 bytes static/img/device/mapper-framework.png | Bin 0 -> 1162282 bytes static/img/device/mapper.png | Bin 0 -> 646804 bytes 13 files changed, 619 insertions(+), 846 deletions(-) create mode 100644 docs/developer/mapper-framework.md delete mode 100644 docs/developer/mappers/bluetooth.md delete mode 100644 docs/developer/mappers/modbus.md create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/developer/mapper-framework.md delete mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/developer/mappers/bluetooth.md delete mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/developer/mappers/modbus.md create mode 100644 static/img/device/dmi-datapanel.png create mode 100644 static/img/device/mapper-framework.png create mode 100644 static/img/device/mapper.png diff --git a/docs/concept/device/mapper.md b/docs/concept/device/mapper.md index ebf4f90c03..abe1d2cba3 100644 --- a/docs/concept/device/mapper.md +++ b/docs/concept/device/mapper.md @@ -2,5 +2,27 @@ title: Mapper sidebar_position: 1 --- +## Mapper +Mapper is an interface between KubeEdge and devices. It could set/get device data, +get and report the device status. KubeEdge uses device controller, device twin and mapper to control +the devices. The device controller is on the cloud side, it uses CRD to define and control devices. +The device twin is on the edge side, it stores the value/status from the mapper and transfers the messages +with device controller and mapper. Meanwhile, DMI in the device twin is used for registing mapper and transfer +device instance and device model to user mapper. + +Currently, a mapper is responsible for managing devices that use a **certain protocol**.DMI will package the device +and model using the same protocol into lists and send them to the mapper of this protocol. In the future we will +explore the possibility of two or more mappers with same protocol running on same node. + +### Architecture +![mapper](/img/device/mapper.png) + +### Creating a Mapper +Users can easily generate their own mapper through **[mapper-framework](../../developer/mapper-framework)** + +You can follew the **[guide](../../developer/mappers#how-to-create-your-own-mappers)** to create your mapper. + + + + -tbd... \ No newline at end of file diff --git a/docs/developer/mapper-framework.md b/docs/developer/mapper-framework.md new file mode 100644 index 0000000000..822dda0fe4 --- /dev/null +++ b/docs/developer/mapper-framework.md @@ -0,0 +1,165 @@ +--- +title: Mapper-Framework +sidebar_position: 1 +--- +## Mapper-Framework +Mapper-Framework provides a new Mapper development framework, which integrates DMI device data management +(data plane) capabilities, allowing devices to process data at the edge or in the cloud, +improving the flexibility of device data management. Mapper-Framework can automatically generate users' Mapper projects, +simplify the complexity of user design and implementation of Mapper, and improve Mapper development efficiency. + +## Architecture +![mapper framework](/img/device/mapper-framework.png) + +1. Mapper-Framework provide a data push interface for pushing to user applications, the destination rule are defined through CRD. +2. Mapper-Framework provide a database interface that can save data to database, the push rule are defined through CRD. +3. Mapper-Framework provide the REST API, these APIs can access devices to obtain data. API does not support changing device property, +because this will result in Inconsistent messages between cloud and edge. +4. Mapper-Framework provide device driver interfaces to initialize the device and obtain device data. +5. Mapper-Framework provide Makefile that can generate a mapper with one command. + +DMI device management plane and device data plane can be implemented by mapper-framework,Developers only need to focus on device drivers. +The red part in generator (Driver) means that developers need to implement it. +Defined interfaces DevPanel to manage devices, new interfaces will be added when features are added. + +## Details +### Data Stream +![dmi datapanel](/img/device/dmi-datapanel.png) +#### Data Normalization + +In order to transfer data between interface modules, standardization of data is necessary. +These data should contain the necessary information generated by the data. +The standardized data definition is [DataModel](https://github.com/kubeedge/kubeedge/blob/master/staging/src/github.com/kubeedge/mapper-framework/pkg/common/datamodel.go#L4) + +#### Push +The data push module can push property values of devices to reachable user apps for consumption according to +destination rules that defined by CRD. To meet the new requirements, the current v1beta1 CRD definitions of Device +Instance add new fields [PushMethod](https://github.com/kubeedge/kubeedge/blob/master/pkg/apis/devices/v1beta1/device_instance_types.go#L116) + +An example of a configuration file that defines mapper to push data to user applications is as follows: +```yaml +apiVersion: devices.kubeedge.io/v1beta1 +kind: Device +metadata: + name: beta1-device +spec: + deviceModelRef: + name: beta1-model + nodeName: worker-node1 + properties: + - name: temp + pushMethod: + mqtt: + address: tcp://127.0.0.1:1883 + topic: temp + qos: 0 + retained: false + ... +``` + +In the current CRD definition, MQTT and HTTP protocols are supported, and the cycle(default 1s) is defined by +the `DeviceProperty.ReportCycle`. When mapper is executed, it will automatically parse the value of the `pushMethod` field +and execute the `DataPanel` interface to push data. +In the future, `DataPanel` interface will add more interfaces to ensure data security. + +#### DataBase +The database module can store device data to a local database +according to destination rules that defined by [DBMethod](https://github.com/kubeedge/kubeedge/blob/master/pkg/apis/devices/v1beta1/device_instance_types.go#L155). +An example of a configuration file that defines mapper to push data to user database is as follows: +```yaml +apiVersion: devices.kubeedge.io/v1beta1 +kind: Device +metadata: + name: beta1-device +spec: + deviceModelRef: + name: beta1-model + nodeName: worker-node1 + properties: + - name: temp + pushMethod: + dbMethod: + influxdb2: + influxdb2ClientConfig: + url: http://127.0.0.1:8086 + org: test-org + bucket: test-bucket + influxdb2DataConfig: + measurement: stat + tag: + unit: temperature + fieldKey: beta1test + ... +``` + +Now we provide Influx2,Redis,TDengine database interfaces, we will add more databases later. + +#### Pull +The HTTP server was created to provide API services. It supports directly obtaining device data from the device. +The URLs listed below are given in the form of local IP. You can use these services from any network accessible to mapper. + +Port `7777` is enabled by default. + +`deviceInstance-ID` according to your own CRD definition +`propertyName` according to your own CRD definition + +#### Ping +1. Detect whether the RESTful service starts normally + Method: **GET** + Url: https://127.0.0.1:7777/api/v1/ping + Response: + ```json + { + "apiVersion": "v1", + "statusCode": 200, + "timeStamp": "2023-08-18T09:57:29+08:00", + "Message": "This is v1 API, the server is running normally." + } + ``` +#### Device Data +1. Get device's Data + Method=**GET** + Url: https://127.0.0.1:7777/api/v1/device/deviceInstance-ID/propertyName + Response: + ```json + { + "apiVersion": "v1", + "statusCode": 200, + "timeStamp": "2023-08-18T09:57:35+08:00", + "Data": { + "DeviceName": "deviceInstance-ID", + "PropertyName": "propertyName", + "Value": "data", + "Type": "dataType", + "CollectTimeStamp": 1692323855044 + } + } + ``` +#### Device MetaData +1. Get device's Model + Method=**GET** + Url: https://127.0.0.1:7777/api/v1/meta/model/deviceInstance-ID + Response: + ```json + { + "apiVersion": "v1", + "statusCode": 200, + "timeStamp": "2023-08-18T09:57:37+08:00", + "name": "model-name", + "properties": [ + { + "name": "propertyName-1", + "dataType": "property data type", + "description": "property description", + "accessMode": "ReadWrite", + "defaultValue": 100 + }, + ... + ] + } + ``` + + + + + diff --git a/docs/developer/mappers/bluetooth.md b/docs/developer/mappers/bluetooth.md deleted file mode 100644 index e4fddb7d82..0000000000 --- a/docs/developer/mappers/bluetooth.md +++ /dev/null @@ -1,328 +0,0 @@ ---- -title: Bluetooth -sidebar_position: 1 ---- -## Introduction - -Mapper is an application that is used to connect and control devices. This is an implementation of mapper for -bluetooth protocol. The aim is to create an application through which users can easily operate devices using bluetooth protocol for communication to the KubeEdge platform. The user is required to provide the mapper with the information required to control their device through the configuration file. These can be changed at runtime by providing the input through the MQTT broker. - -## Running the mapper - - 1. Please ensure that bluetooth service of your device is ON - 2. Set 'bluetooth=true' label for the node (This label is a prerequisite for the scheduler to schedule bluetooth_mapper pod on the node) - - ```shell - kubectl label nodes bluetooth=true - ``` - - 3. Build and deploy the mapper by following the steps given below. - -### Building the bluetooth mapper - - ```shell -cd $GOPATH/src/github.com/kubeedge/kubeedge/mappers/bluetooth_mapper -make bluetooth_mapper_image -docker tag bluetooth_mapper:v1.0 /bluetooth_mapper:v1.0 -docker push /bluetooth_mapper:v1.0 - -Note: Before trying to push the docker image to the remote repository please ensure that you have signed into docker from your node, if not please type the followig command to sign in - docker login - # Please enter your username and password when prompted -``` - -### Deploying bluetooth mapper application - -```shell -cd $GOPATH/src/github.com/kubeedge/kubeedge/mappers/bluetooth_mapper - -# Please enter the following details in the deployment.yaml :- -# 1. Replace with the name of your edge node at spec.template.spec.voluems.configMap.name -# 2. Replace with your dockerhub username at spec.template.spec.containers.image - -kubectl create -f deployment.yaml -``` - -## Modules - -The bluetooth mapper consists of the following five major modules :- - - 1. Action Manager - 2. Scheduler - 3. Watcher - 4. Controller - 5. Data Converter - - -### Action Manager - - A bluetooth device can be controlled by setting a specific value in physical register(s) of a device and readings can be acquired by - getting the value from specific register(s). We can define an Action as a group of read/write operations on a device. A device may support - multiple such actions. The registers are identified by characteristic values which are exposed by the device through entities called characteristic-uuids. - Each of these actions should be supplied through config-file to action manager or at runtime through MQTT. The values specified initially through the configuration - file can be modified at runtime through MQTT. Given below is a guide to provide input to action manager through the configuration file. - - action-manager: - actions: # Multiple actions can be added - - name: - perform-immediately: - device-property-name: - - ....... - ....... - -1. Multiple actions can be added in the action manager module. Each of these actions can either be executed by the action manager of invoked by other modules of -the mapper like scheduler and watcher. - -2. Name of each action should be unique, it is using this name that the other modules like the scheduler or watcher can invoke which action to perform. - -3. Perform-immediately field of the action manager tells the action manager whether it is supposed to perform the action immediately or not, if it set to true then the action manger will -perform the event once. - -4. Each action is associated with a device-property-name, which is the property-name defined in the device CRD, which in turn contains the implementation details required by the action. - - - -### Scheduler - - Scheduler is a component which can perform an action or a set of actions at regular intervals of time. They will make use of the actions previously defined in the action manager module, - it has to be ensured that before the execution of the schedule the action should be defined, otherwise it would lead to an error. The schedule can be configured to run for a specified number of times - or run infinitely. The scheduler is an optional module and need not be specified if not required by the user. The user can provide input to the scheduler through configuration file or - through MQTT at runtime. The values specified initially by the user through the configuration file can be modified at runtime through MQTT. Given below is a guide to provide input to scheduler - through the configuration file. - - scheduler: - schedules: - - name: - interval: