Skip to content

Commit

Permalink
templates: Proposed VSC->IFEX changes
Browse files Browse the repository at this point in the history
These are proposed changes to the descriptions (.md) and template files
(.tpl) for DTDL, Protobuf, and SDS.

The approach is to replace VSC with IFEX when talking about the language
features and format, and to keep VSC when talking about the Vehicle
Service Catalog as the source of the example files.  In some cases
when VSC or IFEX is superfluous, the word has simply been deleted
for simplicity.

The changes are trivial, so copyright statements (2022 Robert Bosch
GmbH) do not need to be amended.

Signed-off-by: Gunnar Andersson <gunnar@[email protected]>
  • Loading branch information
Gunnar Andersson authored and Gunnar Andersson committed Aug 9, 2023
1 parent dd356ae commit 11f1b38
Show file tree
Hide file tree
Showing 6 changed files with 103 additions and 86 deletions.
2 changes: 1 addition & 1 deletion ifex/templates/dtdl/AST_dtdl.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
{%- endmacro %}

{# Define Types #}
{# Not all vsc types can be represented in dtdl #}
{# Not all ifex types can be represented in dtdl #}
{# Add all type conversion #}
{{ save_type("uint16", "integer") }}
{{ save_type("int16", "integer") }}
Expand Down
79 changes: 47 additions & 32 deletions ifex/templates/dtdl/dtdl.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@

## Introduction

The DTDL Generator converts VSC services to correct DTDL according to the [DTDL specification](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md), keeping as much as possible of the information in the VSC definition.
It consists of a [generator template](dtdl.tpl) that has been created using the template framework of the vsc-tools project.
The DTDL Generator converts VSC (IFEX format) services to correct DTDL according to the [DTDL specification](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md), keeping as much as possible of the information in the IFEX definition.
It consists of a [generator template](dtdl.tpl) that has been created using the template framework of the vsc-tools (now ifex) project.

## Known Limitations

Expand All @@ -18,28 +18,32 @@ Known limitations of the implementation:
* No support to put files under other prefix, `global:covesa` always used
* No support for namespaces of arbitrary depth. Currently everything expected to be within the same namespace
* No support for including other files or referencing types in other namespaces
* No support for including error messages defined in VSC.
* No support for including error messages defined in IFEX.


In general all data in the example VSC service can be converted, but minor details are lost:
In general all data in the example service can be converted, but minor details are lost:

- Explicit min/max values (from VSC typedefs)
- Implicit min/max values (from VSC types like `uint8` as a bigger type like `integer` must be used in DTDL)
- Explicit min/max values (from IFEX typedefs)
- Implicit min/max values (from IFEX types like `uint8` as a bigger type like `integer` must be used in DTDL)


## Conversion concept
The intention is to generate correct DTDL according to the [DTDL specification](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md), keeping as much as possible of the information from the VSC definition.
The intention is to generate correct DTDL according to the [DTDL specification](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md), keeping as much as possible of the information from the IFEX definition.
For now the focus is on concepts used in the [VSC example service](https://github.com/COVESA/vehicle_service_catalog/blob/master/comfort-service.yml).
The reason for this limitation is that the functionality of the tooling in the [COVESA vsc-tools repo](https://github.com/COVESA/vsc-tools) currently has significant limitations, concerning e.g. nested namespaces and inclusion of data from other files.
In this section mappings are described for all VSC concepts used in the example file.
The reason for this limitation is that the functionality of the tooling in the [COVESA vsc-tools repo](https://github.com/COVESA/ifex) (now IFEX) currently has significant limitations, concerning e.g. nested namespaces and inclusion of data from other files.
In this section mappings are described for all IFEX concepts used in the example file.

### VSC Namespace
Represented as [DTDL Interface](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#interface). Proposed default naming for [example VSC service](https://github.com/COVESA/vehicle_service_catalog/blob/master/comfort-service.yml) is `"dtmi:global:covesa:comfort:seats"`, to be aligned with the homepage [http://covesa.global/](http://covesa.global/) of COVESA. In more advanced tooling the prefix (`global:covesa`) could be an input parameter.
### Namespace
Represented as [DTDL Interface](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#interface). Proposed default naming for [example service](https://github.com/COVESA/vehicle_service_catalog/blob/master/comfort-service.yml) is `"dtmi:global:covesa:comfort:seats"`, to be aligned with the homepage [http://covesa.global/](http://covesa.global/) of COVESA. In more advanced tooling the prefix (`global:covesa`) could be an input parameter.

It must be noted that for now VSC has not yet aligned on a specific tree/naming structure, e.g. whether there should exist a top level called `Vehicle` similar to [VSS](https://github.com/COVESA/vehicle_signal_specification).

### VSC Primitive Types
General approach is to convert from [VSC native datatype](https://github.com/COVESA/vehicle_service_catalog#native-data-types) to [DTDL datatype](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#primitive-schemas).
### Primitive Types
General approach is to convert from [IFEX native datatype](

FIXME: LINK

https://github.com/COVESA/vehicle_service_catalog#native-data-types) to [DTDL datatype](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#primitive-schemas).

- `uint8`, `int8`, `uint16`, `int16`, `int32` -> `integer` (signed 4-byte)
- `uint32`, `uint64`, `int64` -> `long` (signed 8-byte). Some values of uint64 cannot be represented, but likely has no impact. Should possibly in long term give warning during conversion.
Expand All @@ -48,14 +52,21 @@ General approach is to convert from [VSC native datatype](https://github.com/COV
- `double` -> `double`
- `string` -> `string`

A conversion tool could possibly add a DTDL comment giving a textual description of the base type, like `"Comment":"Original VSC type: uint16"`.
A conversion tool could possibly add a DTDL comment giving a textual description of the base type, like `"Comment":"Original IFEX type: uint16"`.

### Struct

FIXME: LINK

[IFEX Structs](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-structs) can be represented as [DTDL Interface Object Schemas](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#interface-schemas).

### Typedef

### VSC Struct
[VSC Structs](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-structs) can be represented as [DTDL Interface Object Schemas](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#interface-schemas).
FIXME: LINK

### VSC Typedef
No good DTDL representation exist for [VSC Typedefs](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-typedefs). Suggested conversion is to use base-type, i.e. for the example below `integer`. A conversion tool could possibly add a DTDL comment giving a textual description of the base type, like `"Comment":"Original VSC type: movement_t (int16, min: -1000, max:1000"`.
No good DTDL representation exist for [IFEX Typedefs](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-typedefs). Suggested conversion is to use base-type, i.e. for the example below `integer`. A conversion tool could possibly add a DTDL comment giving a textual description of the base type, like `"Comment":"Original IFEX type: movement_t (int16, min: -1000, max:1000"`.

FIXME: LINK

```
typedefs:
Expand All @@ -67,29 +78,33 @@ No good DTDL representation exist for [VSC Typedefs](https://github.com/COVESA/v
The movement of a seat component
```

### VSC Enumerations
[VSC Enumerations](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-enumerations) should be straightforward to represent as [DTDL enum](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#enum).
### Enumerations
FIXME: LINK
[Enumerations](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-enumerations) should be straightforward to represent as [DTDL enum](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#enum).

### VSC Methods
[VSC Methods](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-methods) should be possible to represent as [DTDL Command](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#command)
### Methods
FIXME: LINK
[Methods](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-methods) should be possible to represent as [DTDL Command](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#command)

If the VSC method has more than 1 in-param or more than 1 out-param an inline struct needs to be used in DTDL representation. The struct must then have a name, a possible approach is to use `in` for the in-parameter and `out` for the out-parameter.
If the IFEX method has more than 1 in-param or more than 1 out-param an inline struct needs to be used in DTDL representation. The struct must then have a name, a possible approach is to use `in` for the in-parameter and `out` for the out-parameter.

### Events
Assuming the [VSC Events](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-events) are sent by the vehicle, it could be represented as [DTDL Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry).
FIXME: LINK
Assuming the [Events](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-events) are sent by the vehicle, it could be represented as [DTDL Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry).

### Properties
[VSC Properties](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-properties) is intended to be able to represent [VSS signals](https://github.com/COVESA/vehicle_signal_specification). They match [DTDL Property](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#property) quite well, but to a certain extent also [DTDL Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry).
FIXME: LINK
[IFEX Properties](https://github.com/COVESA/vehicle_service_catalog#namespace-list-object-properties) is intended to be able to represent [VSS signals](https://github.com/COVESA/vehicle_signal_specification). They match [DTDL Property](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#property) quite well, but to a certain extent also [DTDL Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry).

In the DTDL documentation examples for [Telemetry](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#telemetry-examples) and [Properties](https://github.com/Azure/opendigitaltwins-dtdl/blob/master/DTDL/v2/dtdlv2.md#property-examples) it seems that Telemetry is used for data similar to VSS sensors and Properties for items similar to VSS actuators and VSS attributes.
But in VSS an actuator can also act as a sensor, i.e. when you set it you set the "wanted" value, but when you read it you get the "actual" value rather than the "wanted" value.
An example is `Vehicle.Cabin.Seat.Row1.Pos1.Position`.

For now all VSC properties are represented as DTDL Writeable Property.
For now all IFEX properties are represented as DTDL Writeable Property.

### Handling of generic fields
- VSC description can be represented as DTDL description property.
- VSC does not have structured comments (only `#` comments that are ignored when creating internal model), not possible to convert to DTDL comment property.
- IFEX description can be represented as DTDL description property.
- IFEX does not have structured comments (only `#` comments that are ignored when creating internal model), not possible to convert to DTDL comment property.

## Using the DTDL Generator

Expand All @@ -98,11 +113,11 @@ For now all VSC properties are represented as DTDL Writeable Property.
The tool can be used like below:

```
# go to vsc-tools if not already there
cd vsc-tools
# make sure that vsc has been cloned
# go to ifex tools if not already there
cd ifex
# make sure that VSC has been cloned
git clone https://github.com/COVESA/vehicle_service_catalog/
vscgen vehicle_service_catalog/comfort-service.yml dtdl.tpl > dtdl_generated.json
ifexgen vehicle_service_catalog/comfort-service.yml dtdl.tpl > dtdl_generated.json
```

### Validating generated DTDL files
Expand Down
40 changes: 21 additions & 19 deletions ifex/templates/protobuf/AST_protobuf.tpl
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
{# Protobuf template #}
{# (C) 2022 Robert Bosch GmbH #}
// protobuf/gRPC definition generated using template: protobuf.tpl
// protobuf code generated by protobuf.tpl
{# https://developers.google.com/protocol-buffers/docs/proto#services #}
syntax = "proto3";

{# Define Types #}
{# Not all vsc types can be represented as is in protobuf #}
{% set typedefs = dict() %}
{# Add all type conversion #}
{% set _=typedefs.__setitem__("int16", "int32") %}
{% set _=typedefs.__setitem__("uint8", "uint32") %}
{% set _=typedefs.__setitem__("uint16", "uint32") %}
{% set _=typedefs.__setitem__("boolean", "bool") %}
{% set x=typedefs.__setitem__("int16", "int32") %}
{% set x=typedefs.__setitem__("uint8", "uint32") %}
{% set x=typedefs.__setitem__("uint16", "uint32") %}
{% set x=typedefs.__setitem__("boolean", "bool") %}

package swdv.{{item.name}};

Expand All @@ -21,45 +21,49 @@ message operation_result {
}

{% for n in item.namespaces %}
// IFEX: from Namespace {{n.name}}
// IFEX Namespace {{n.name}}
{# Typedefs and enum must be handled before structs, to convert types right #}
{# Limitation: for now we ignore namespaces #}
{% for t in n.typedefs %}
// IFEX Typedef {{t.name}}
{# Typedef - Just using base type, but check if that one also needs to be expanded #}
{% if t.datatype in typedefs %}
{% set type = typedefs[t.datatype] %}
{% else %}
{% set type = t.datatype %}
{% endif %}
{% set x=typedefs.__setitem__(t.name, type) %}

{% endfor %}
{% for t in n.enumerations %}
// IFEX: Enum {{t.name}}
// IFEX Enum {{t.name}}
enum {{t.name}} {
{% for x in t.options %}
{{ x.name }} = {{ loop.index - 1}};
{% endfor %}
}

{% endfor %}
{% for x in n.structs %}
// IFEX: Struct {{x.name}}
// IFEX Struct {{x.name}}
{# Cannot use dots in names #}
message {{x.name}} {
{% for m in x.members %}
{% if m.datatype in typedefs %}
{{typedefs[m.datatype] |replace(".", "_")}} {{ m.name }} = {{ loop.index }}; // IFEX original type: {{m.datatype}}
{% if m.datatype in typedefs %}
{% set type = typedefs[m.datatype] %}
{% else %}
{{m.datatype |replace(".", "_")}} {{ m.name }} = {{ loop.index }};
{% set type = m.datatype %}
{% endif %}
{{type|replace(".", "_")}} {{ m.name }} = {{ loop.index }};
{% endfor %}
}

{% endfor %}
{% for x in n.methods %}
// IFEX: Method {{x.name}}
// IFEX Method {{x.name}}
message {{ x.name }}_request {
{% for x in x.input %}
{% if x.datatype in typedefs %}
{% if x.datatype in typedefs %}
{% set type = typedefs[x.datatype] %}
{% else %}
{% set type = x.datatype %}
Expand All @@ -70,7 +74,7 @@ message {{ x.name }}_request {

message {{ x.name }}_response {
{% for x in x.output %}
{% if x.datatype in typedefs %}
{% if x.datatype in typedefs %}
{% set type = typedefs[x.datatype] %}
{% else %}
{% set type = x.datatype %}
Expand All @@ -79,20 +83,18 @@ message {{ x.name }}_response {
{% endfor %}
}

{% endfor %}
service {{ n.name }}_service {
{% for x in n.methods %}
service {{ x.name }}_service {
rpc {{ x.name }}({{ x.name }}_request) returns ({{ x.name }}_response);
{% endfor %}
}

{% endfor %}

{% for x in n.events %}
// IFEX Event {{x.name}}
{# Limitation: for now just creating a message #}
message {{ x.name }} {
{% for x in x.input %}
{% if x.datatype in typedefs %}
{% if x.datatype in typedefs %}
{% set type = typedefs[x.datatype] %}
{% else %}
{% set type = x.datatype %}
Expand Down
26 changes: 13 additions & 13 deletions ifex/templates/protobuf/protobuf.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

## Introduction

The Protobuf Generator converts VSC services to a [Google Protocol Buffer](https://developers.google.com/protocol-buffers/) definition.
The Protobuf Generator converts VSC (IFEX format) services to a [Google Protocol Buffer](https://developers.google.com/protocol-buffers/) definition.

## Known Limitations

*The Protobuf Generator has been developed as proof-of-concept and does not currently have product quality!*

The general tooling framework provided by VSC is currently not that advanced, partially depending on that VSC as language not yet is that stable.
The general tooling framework provided by IFEX is currently not that advanced, partially depending on that IFEX as language not yet is that stable.
The open source tooling perform syntactic checks, but it does not perform semantic checks, e.g. verifying that a referenced datatype actually exists.

Known limitations of the current implementation:
Expand All @@ -20,12 +20,12 @@ Known limitations of the current implementation:

## Conversion Concept

* VSC typedefs are not represented in protobuf, instead the base type is used.
* VSC Structs are represented in protobuf.
* VSC methods are represented as rpc in protobuf.
* VSC events are represented as messages in protobuf.
* VSC properties are represented as a read rpc and a write rpc in Protobuf.
* VSC Error Messages are currently not imported.
* IFEX typedefs are not represented in protobuf, instead the base type is used.
* IFEX Structs are represented in protobuf.
* IFEX methods are represented as rpc in protobuf.
* IFEX events are represented as messages in protobuf.
* IFEX properties are represented as a read rpc and a write rpc in Protobuf.
* IFEX Error Messages are currently not imported.

## Using the Protobuf Generator

Expand All @@ -43,12 +43,12 @@ apt install -y protobuf-compiler
The code below runs the prototype protobuf generator and verifies that the generated code is syntactically correct.

```
# go to vsc-tools if not already there
cd vsc-tools
# make sure that vsc has been cloned
# go to ifex tools if not already there
cd ifex
# make sure that VSC has been cloned
git clone https://github.com/COVESA/vehicle_service_catalog/
# Run the generator
vscgen vehicle_service_catalog/comfort-service.yml protobuf.tpl > vsc.proto
ifexgen vehicle_service_catalog/comfort-service.yml protobuf.tpl > comfort-service.proto
```

### Validating generated Protobuf files
Expand All @@ -57,6 +57,6 @@ Generated Protobuf files can be validated/compiled using `protoc`.

```
mkdir tmp
protoc --cpp_out=tmp vsc.proto
protoc --cpp_out=tmp comfort-service.proto
```

Loading

0 comments on commit 11f1b38

Please sign in to comment.