tensor_typedef.h File Reference

Common header file for NNStreamer, the GStreamer plugin for neural networks. More...

#include <stddef.h>
#include <stdint.h>

Include dependency graph for tensor_typedef.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
union	tensor_element
	To make the code simple with all the types. "C++ Template"-like. More...
struct	GstTensorMemory
	The unit of each data tensors. It will be used as an input/output tensor of other/tensors. More...
struct	GstTensorInfo
	Internal data structure for tensor info. More...
struct	GstTensorsInfo
	Internal meta data exchange format for a other/tensors instance. More...
struct	GstTensorsConfig
	Internal data structure for configured tensors info (for other/tensors). More...
struct	GstSparseTensorInfo
	Internal data structure for sparse tensor info. More...
struct	GstTensorMetaInfo
	Data structure to describe a tensor data. This represents the basic information of a memory block for tensor stream. More...

Macros
#define	NNS_TENSOR_RANK_LIMIT   (16)
#define	NNS_TENSOR_SIZE_LIMIT   (256)
	The number of tensors NNStreamer supports is 256. The max memories of gst-buffer is 16 (See NNS_TENSOR_MEMORY_MAX). Internally NNStreamer handles the memories as tensors. If the number of tensors is larger than 16, we modify the last memory and combine tensors into the memory. More...
#define	NNS_TENSOR_SIZE_LIMIT_STR   "256"
#define	NNS_TENSOR_MEMORY_MAX   (16)
	This value, 16, can be checked with gst_buffer_get_max_memory(), which is GST_BUFFER_MEM_MAX in gstreamer/gstbuffer.c. We redefined the value because GST_BUFFER_MEM_MAX is not exported and we need static value. To modify (increase) this value, you need to update gstreamer/gstbuffer.c as well. More...
#define	NNS_TENSOR_SIZE_EXTRA_LIMIT   (NNS_TENSOR_SIZE_LIMIT - NNS_TENSOR_MEMORY_MAX)
	Max number of extra tensors. More...
#define	NNS_MIMETYPE_TENSOR   "other/tensor"
#define	NNS_MIMETYPE_TENSORS   "other/tensors"
#define	GST_TENSOR_NUM_TENSORS_RANGE   "(int) [ 1, " NNS_TENSOR_SIZE_LIMIT_STR " ]"
#define	GST_TENSOR_RATE_RANGE   "(fraction) [ 0, max ]"
#define	GST_TENSOR_TYPE_ALL   "{ float16, float32, float64, int64, uint64, int32, uint32, int16, uint16, int8, uint8 }"
	Possible tensor element types. More...
#define	GST_TENSOR_FORMAT_ALL   "{ static, flexible, sparse }"
	Possible tensor formats. More...
#define	GST_TENSOR_CAP_DEFAULT
	Default static capability for other/tensor. More...
#define	GST_TENSORS_CAP_MAKE(fmt)
	Caps string for the caps template of tensor stream. format should be a string that describes the data format, or possible formats of incoming tensor. More...
#define	GST_TENSORS_CAP_WITH_NUM(num)
	Caps string for the caps template (other/tensors, static tensor stream with fixed number of tensors). num should be a string format that describes the number of tensors, or the range of incoming tensors. The types and dimensions of tensors should be described for caps negotiation. More...
#define	GST_TENSORS_CAP_DEFAULT   GST_TENSORS_CAP_WITH_NUM (GST_TENSOR_NUM_TENSORS_RANGE)
	Caps string for the caps template of static tensor stream. More...
#define	GST_TENSORS_FLEX_CAP_DEFAULT   GST_TENSORS_CAP_MAKE ("flexible")
	Caps string for the caps template of flexible tensors. This mimetype handles non-static, flexible tensor stream without specifying the data type and shape of the tensor. The maximum number of tensors in a buffer is 16 (NNS_TENSOR_SIZE_LIMIT). More...
#define	GST_TENSORS_SPARSE_CAP_DEFAULT   GST_TENSORS_CAP_MAKE ("sparse")
	Caps string for the caps template of sparse tensors. This mimetype handles non-static, sparse tensor stream without specifying the data type and shape of the tensor. The maximum number of tensors in a buffer is 16 (NNS_TENSOR_SIZE_LIMIT). More...

Typedefs
typedef enum _nns_tensor_type	tensor_type
	Possible data element types of other/tensor. More...
typedef enum _nns_media_type	media_type
	Float16 compiler extension support. More...
typedef enum _tensor_format	tensor_format
	Data format of tensor stream in the pipeline. More...
typedef enum _nns_tensor_layout	tensor_layout
	Tensor layout format for other/tensor. More...
typedef uint32_t	tensor_dim[NNS_TENSOR_RANK_LIMIT]

Enumerations
enum	_nns_tensor_type {   _NNS_INT32 = 0, _NNS_UINT32, _NNS_INT16, _NNS_UINT16,   _NNS_INT8, _NNS_UINT8, _NNS_FLOAT64, _NNS_FLOAT32,   _NNS_INT64, _NNS_UINT64, _NNS_FLOAT16, _NNS_END }
	Possible data element types of other/tensor. More...
enum	_nns_media_type {   _NNS_MEDIA_INVALID = -1, _NNS_VIDEO = 0, _NNS_AUDIO = 1, _NNS_TEXT = 2,   _NNS_OCTET = 3, _NNS_TENSOR = 4, _NNS_MEDIA_ANY = 0x1000 }
	Float16 compiler extension support. More...
enum	_tensor_format { _NNS_TENSOR_FORMAT_STATIC = 0, _NNS_TENSOR_FORMAT_FLEXIBLE, _NNS_TENSOR_FORMAT_SPARSE, _NNS_TENSOR_FORMAT_END }
	Data format of tensor stream in the pipeline. More...
enum	_nns_tensor_layout { _NNS_LAYOUT_ANY = 0, _NNS_LAYOUT_NHWC, _NNS_LAYOUT_NCHW, _NNS_LAYOUT_NONE }
	Tensor layout format for other/tensor. More...

Detailed Description

Common header file for NNStreamer, the GStreamer plugin for neural networks.

NNStreamer Common Header, Typedef part, for export as devel package. Copyright (C) 2018 MyungJoo Ham myung.nosp@m.joo..nosp@m.ham@s.nosp@m.amsu.nosp@m.ng.co.nosp@m.m

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Library General Public License as published by the Free Software Foundation; version 2.1 of the License.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public License for more details.

Date

01 Jun 2018

See also

http://github.com/nnstreamer/nnstreamer

Author

MyungJoo Ham myung.nosp@m.joo..nosp@m.ham@s.nosp@m.amsu.nosp@m.ng.co.nosp@m.m

Bug:

No known bugs except for NYI items

To Packagers:

This file is to be packaged as "devel" package for NN developers.

Definition in file tensor_typedef.h.

Macro Definition Documentation

GST_TENSOR_CAP_DEFAULT

#define GST_TENSOR_CAP_DEFAULT

Value:

    NNS_MIMETYPE_TENSOR ", " \
    "framerate = " GST_TENSOR_RATE_RANGE

Default static capability for other/tensor.

Definition at line 78 of file tensor_typedef.h.

GST_TENSOR_FORMAT_ALL

#define GST_TENSOR_FORMAT_ALL   "{ static, flexible, sparse }"

Possible tensor formats.

Definition at line 73 of file tensor_typedef.h.

GST_TENSOR_NUM_TENSORS_RANGE

#define GST_TENSOR_NUM_TENSORS_RANGE   "(int) [ 1, " NNS_TENSOR_SIZE_LIMIT_STR " ]"

Definition at line 62 of file tensor_typedef.h.

GST_TENSOR_RATE_RANGE

#define GST_TENSOR_RATE_RANGE   "(fraction) [ 0, max ]"

Definition at line 63 of file tensor_typedef.h.

GST_TENSOR_TYPE_ALL

#define GST_TENSOR_TYPE_ALL   "{ float16, float32, float64, int64, uint64, int32, uint32, int16, uint16, int8, uint8 }"

Possible tensor element types.

Definition at line 68 of file tensor_typedef.h.

GST_TENSORS_CAP_DEFAULT

#define GST_TENSORS_CAP_DEFAULT   GST_TENSORS_CAP_WITH_NUM (GST_TENSOR_NUM_TENSORS_RANGE)

Caps string for the caps template of static tensor stream.

Definition at line 115 of file tensor_typedef.h.

GST_TENSORS_CAP_MAKE

#define GST_TENSORS_CAP_MAKE

(

fmt

)

Value:

    NNS_MIMETYPE_TENSORS ", " \
    "format = (string) " fmt ", " \
    "framerate = " GST_TENSOR_RATE_RANGE

Caps string for the caps template of tensor stream. format should be a string that describes the data format, or possible formats of incoming tensor.

type should be one of types in GST_TENSOR_TYPE_ALL "type = (string) uint8" dimension should be a formatted string with rank NNS_TENSOR_RANK_LIMIT "dimension = (string) dim1:dim2:dim3:dim4" If the data format is static, another tensor information should be described in caps.

• num_tensors: The number of tensors in a GstBuffer.
• types: The list of data type in the tensor. type should be one of types in GST_TENSOR_TYPE_ALL. (types=(string)"type1,type2,type3")
• dimensions: The list of dimension in the tensor. dimension should be a formatted string with rank NNS_TENSOR_RANK_LIMIT. (dimensions=(string)"dim1:dim2:dim3:dim4,dim1:dim2:dim3:dim4")

Definition at line 97 of file tensor_typedef.h.

GST_TENSORS_CAP_WITH_NUM

#define GST_TENSORS_CAP_WITH_NUM

(

num

)

Value:

    NNS_MIMETYPE_TENSORS ", " \
    "format = (string) static, num_tensors = " num ", " \
    "framerate = " GST_TENSOR_RATE_RANGE

Caps string for the caps template (other/tensors, static tensor stream with fixed number of tensors). num should be a string format that describes the number of tensors, or the range of incoming tensors. The types and dimensions of tensors should be described for caps negotiation.

Definition at line 107 of file tensor_typedef.h.

GST_TENSORS_FLEX_CAP_DEFAULT

#define GST_TENSORS_FLEX_CAP_DEFAULT   GST_TENSORS_CAP_MAKE ("flexible")

Caps string for the caps template of flexible tensors. This mimetype handles non-static, flexible tensor stream without specifying the data type and shape of the tensor. The maximum number of tensors in a buffer is 16 (NNS_TENSOR_SIZE_LIMIT).

Definition at line 123 of file tensor_typedef.h.

GST_TENSORS_SPARSE_CAP_DEFAULT

#define GST_TENSORS_SPARSE_CAP_DEFAULT   GST_TENSORS_CAP_MAKE ("sparse")

Caps string for the caps template of sparse tensors. This mimetype handles non-static, sparse tensor stream without specifying the data type and shape of the tensor. The maximum number of tensors in a buffer is 16 (NNS_TENSOR_SIZE_LIMIT).

Definition at line 131 of file tensor_typedef.h.

NNS_MIMETYPE_TENSOR

#define NNS_MIMETYPE_TENSOR   "other/tensor"

Definition at line 59 of file tensor_typedef.h.

NNS_MIMETYPE_TENSORS

#define NNS_MIMETYPE_TENSORS   "other/tensors"

Definition at line 60 of file tensor_typedef.h.

NNS_TENSOR_MEMORY_MAX

#define NNS_TENSOR_MEMORY_MAX   (16)

This value, 16, can be checked with gst_buffer_get_max_memory(), which is GST_BUFFER_MEM_MAX in gstreamer/gstbuffer.c. We redefined the value because GST_BUFFER_MEM_MAX is not exported and we need static value. To modify (increase) this value, you need to update gstreamer/gstbuffer.c as well.

Definition at line 52 of file tensor_typedef.h.

NNS_TENSOR_RANK_LIMIT

#define NNS_TENSOR_RANK_LIMIT   (16)

Definition at line 34 of file tensor_typedef.h.

NNS_TENSOR_SIZE_EXTRA_LIMIT

#define NNS_TENSOR_SIZE_EXTRA_LIMIT   (NNS_TENSOR_SIZE_LIMIT - NNS_TENSOR_MEMORY_MAX)

Max number of extra tensors.

Definition at line 57 of file tensor_typedef.h.

NNS_TENSOR_SIZE_LIMIT

#define NNS_TENSOR_SIZE_LIMIT   (256)

The number of tensors NNStreamer supports is 256. The max memories of gst-buffer is 16 (See NNS_TENSOR_MEMORY_MAX). Internally NNStreamer handles the memories as tensors. If the number of tensors is larger than 16, we modify the last memory and combine tensors into the memory.

Definition at line 42 of file tensor_typedef.h.

NNS_TENSOR_SIZE_LIMIT_STR

#define NNS_TENSOR_SIZE_LIMIT_STR   "256"

Definition at line 43 of file tensor_typedef.h.

Typedef Documentation

media_type

typedef enum _nns_media_type media_type

Float16 compiler extension support.

Possible input stream types for other/tensor.

This is related with media input stream to other/tensor. There is no restrictions for the outputs.

In order to prevent enum-mix issues between device profiles, we explicitly define numbers for each enum type.

tensor_dim

typedef uint32_t tensor_dim[NNS_TENSOR_RANK_LIMIT]

Definition at line 247 of file tensor_typedef.h.

tensor_format

typedef enum _tensor_format tensor_format

Data format of tensor stream in the pipeline.

tensor_layout

typedef enum _nns_tensor_layout tensor_layout

Tensor layout format for other/tensor.

The layout is needed by some of the element to appropriately process the data based on the axis of the channel in the data. Layout information will be currently utilized by only some of the elements (SNAP, NNFW in tensor_filter, PADDING mode in tensor_transform)

Tensor layout is not part of the capabilities of the element, and does not take part in the caps negotiation.

NONE layout implies that the layout of the data is neither NHWC nor NCHW. ' However, ANY layout implies that the layout of the provided data is not relevant.

Note

Providing tensor layout can also decide acceleration to be supported as not all the accelerators might support all the layouts (NYI).

tensor_type

typedef enum _nns_tensor_type tensor_type

Possible data element types of other/tensor.

Note

When changing tensor type, you should update related type in ML-API and protobuf/flatbuf schema also.

Enumeration Type Documentation

_nns_media_type

enum _nns_media_type

Float16 compiler extension support.

Possible input stream types for other/tensor.

This is related with media input stream to other/tensor. There is no restrictions for the outputs.

In order to prevent enum-mix issues between device profiles, we explicitly define numbers for each enum type.

Enumerator
_NNS_MEDIA_INVALID	Uninitialized
_NNS_VIDEO	supposedly video/x-raw
_NNS_AUDIO	supposedly audio/x-raw
_NNS_TEXT	supposedly text/x-raw
_NNS_OCTET	supposedly application/octet-stream
_NNS_TENSOR	supposedly other/tensor(s) or flexible tensor
_NNS_MEDIA_ANY	any media type (find proper external converter in tensor-converter element)

Definition at line 179 of file tensor_typedef.h.

_nns_tensor_layout

enum _nns_tensor_layout

Tensor layout format for other/tensor.

The layout is needed by some of the element to appropriately process the data based on the axis of the channel in the data. Layout information will be currently utilized by only some of the elements (SNAP, NNFW in tensor_filter, PADDING mode in tensor_transform)

Tensor layout is not part of the capabilities of the element, and does not take part in the caps negotiation.

NONE layout implies that the layout of the data is neither NHWC nor NCHW. ' However, ANY layout implies that the layout of the provided data is not relevant.

Note

Providing tensor layout can also decide acceleration to be supported as not all the accelerators might support all the layouts (NYI).

Enumerator
_NNS_LAYOUT_ANY	does not care about the data layout
_NNS_LAYOUT_NHWC	NHWC: channel last layout
_NNS_LAYOUT_NCHW	NCHW: channel first layout
_NNS_LAYOUT_NONE	NONE: none of the above defined layouts

Definition at line 220 of file tensor_typedef.h.

_nns_tensor_type

enum _nns_tensor_type

Possible data element types of other/tensor.

Note

When changing tensor type, you should update related type in ML-API and protobuf/flatbuf schema also.

Enumerator
_NNS_INT32
_NNS_UINT32
_NNS_INT16
_NNS_UINT16
_NNS_INT8
_NNS_UINT8
_NNS_FLOAT64
_NNS_FLOAT32
_NNS_INT64
_NNS_UINT64
_NNS_FLOAT16	added with nnstreamer 2.1.1-devel. If you add any operators (e.g., tensor_transform) to float16, it will either be not supported or be too inefficient.
_NNS_END

Definition at line 138 of file tensor_typedef.h.

_tensor_format

enum _tensor_format

Data format of tensor stream in the pipeline.

Enumerator
_NNS_TENSOR_FORMAT_STATIC
_NNS_TENSOR_FORMAT_FLEXIBLE
_NNS_TENSOR_FORMAT_SPARSE
_NNS_TENSOR_FORMAT_END

Definition at line 193 of file tensor_typedef.h.