Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update mapper and mapper-framework docs #475

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion docs/concept/device/mapper.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,27 @@
title: Mapper
sidebar_position: 1
---
## Mapper
Mapper is an interface between KubeEdge and devices. It could set/get device data,
get and report the device status. KubeEdge uses device controller, device twin and mapper to control
the devices. The device controller is on the cloud side, it uses CRD to define and control devices.
The device twin is on the edge side, it stores the value/status from the mapper and transfers the messages
with device controller and mapper. Meanwhile, DMI in the device twin is used for registing mapper and transfer
device instance and device model to user mapper.

Currently, a mapper is responsible for managing devices that use a **certain protocol**.DMI will package the device
and model using the same protocol into lists and send them to the mapper of this protocol. In the future we will
explore the possibility of two or more mappers with same protocol running on same node.

### Architecture
![mapper](/img/device/mapper.png)

### Creating a Mapper
Users can easily generate their own mapper through **[mapper-framework](../../developer/mapper-framework)**

You can follew the **[guide](../../developer/mappers#how-to-create-your-own-mappers)** to create your mapper.





tbd...
165 changes: 165 additions & 0 deletions docs/developer/mapper-framework.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
title: Mapper-Framework
sidebar_position: 1
---
## Mapper-Framework
Mapper-Framework provides a new Mapper development framework, which integrates DMI device data management
(data plane) capabilities, allowing devices to process data at the edge or in the cloud,
improving the flexibility of device data management. Mapper-Framework can automatically generate users' Mapper projects,
simplify the complexity of user design and implementation of Mapper, and improve Mapper development efficiency.

## Architecture
![mapper framework](/img/device/mapper-framework.png)

1. Mapper-Framework provide a data push interface for pushing to user applications, the destination rule are defined through CRD.
2. Mapper-Framework provide a database interface that can save data to database, the push rule are defined through CRD.
3. Mapper-Framework provide the REST API, these APIs can access devices to obtain data. API does not support changing device property,
because this will result in Inconsistent messages between cloud and edge.
4. Mapper-Framework provide device driver interfaces to initialize the device and obtain device data.
5. Mapper-Framework provide Makefile that can generate a mapper with one command.

DMI device management plane and device data plane can be implemented by mapper-framework,Developers only need to focus on device drivers.
The red part in generator (Driver) means that developers need to implement it.
Defined interfaces DevPanel to manage devices, new interfaces will be added when features are added.

## Details
### Data Stream
![dmi datapanel](/img/device/dmi-datapanel.png)
#### Data Normalization

In order to transfer data between interface modules, standardization of data is necessary.
These data should contain the necessary information generated by the data.
The standardized data definition is [DataModel](https://github.com/kubeedge/kubeedge/blob/master/staging/src/github.com/kubeedge/mapper-framework/pkg/common/datamodel.go#L4)

#### Push
The data push module can push property values of devices to reachable user apps for consumption according to
destination rules that defined by CRD. To meet the new requirements, the current v1beta1 CRD definitions of Device
Instance add new fields [PushMethod](https://github.com/kubeedge/kubeedge/blob/master/pkg/apis/devices/v1beta1/device_instance_types.go#L116)

An example of a configuration file that defines mapper to push data to user applications is as follows:
```yaml
apiVersion: devices.kubeedge.io/v1beta1
kind: Device
metadata:
name: beta1-device
spec:
deviceModelRef:
name: beta1-model
nodeName: worker-node1
properties:
- name: temp
pushMethod:
mqtt:
address: tcp://127.0.0.1:1883
topic: temp
qos: 0
retained: false
...
```

In the current CRD definition, MQTT and HTTP protocols are supported, and the cycle(default 1s) is defined by
the `DeviceProperty.ReportCycle`. When mapper is executed, it will automatically parse the value of the `pushMethod` field
and execute the `DataPanel` interface to push data.
In the future, `DataPanel` interface will add more interfaces to ensure data security.

#### DataBase
The database module can store device data to a local database
according to destination rules that defined by [DBMethod](https://github.com/kubeedge/kubeedge/blob/master/pkg/apis/devices/v1beta1/device_instance_types.go#L155).
An example of a configuration file that defines mapper to push data to user database is as follows:
```yaml
apiVersion: devices.kubeedge.io/v1beta1
kind: Device
metadata:
name: beta1-device
spec:
deviceModelRef:
name: beta1-model
nodeName: worker-node1
properties:
- name: temp
pushMethod:
dbMethod:
influxdb2:
influxdb2ClientConfig:
url: http://127.0.0.1:8086
org: test-org
bucket: test-bucket
influxdb2DataConfig:
measurement: stat
tag:
unit: temperature
fieldKey: beta1test
...
```

Now we provide Influx2,Redis,TDengine database interfaces, we will add more databases later.

#### Pull
The HTTP server was created to provide API services. It supports directly obtaining device data from the device.
The URLs listed below are given in the form of local IP. You can use these services from any network accessible to mapper.

Port `7777` is enabled by default.

`deviceInstance-ID` according to your own CRD definition
`propertyName` according to your own CRD definition

#### Ping
1. Detect whether the RESTful service starts normally
Method: **GET**
Url: https://127.0.0.1:7777/api/v1/ping
Response:
```json
{
"apiVersion": "v1",
"statusCode": 200,
"timeStamp": "2023-08-18T09:57:29+08:00",
"Message": "This is v1 API, the server is running normally."
}
```
#### Device Data
1. Get device's Data
Method=**GET**
Url: https://127.0.0.1:7777/api/v1/device/deviceInstance-ID/propertyName
Response:
```json
{
"apiVersion": "v1",
"statusCode": 200,
"timeStamp": "2023-08-18T09:57:35+08:00",
"Data": {
"DeviceName": "deviceInstance-ID",
"PropertyName": "propertyName",
"Value": "data",
"Type": "dataType",
"CollectTimeStamp": 1692323855044
}
}
```
#### Device MetaData
1. Get device's Model
Method=**GET**
Url: https://127.0.0.1:7777/api/v1/meta/model/deviceInstance-ID
Response:
```json
{
"apiVersion": "v1",
"statusCode": 200,
"timeStamp": "2023-08-18T09:57:37+08:00",
"name": "model-name",
"properties": [
{
"name": "propertyName-1",
"dataType": "property data type",
"description": "property description",
"accessMode": "ReadWrite",
"defaultValue": 100
},
...
]
}
```





Loading