\hypertarget{classnntrainer_1_1DataProducer}{}\doxysection{nntrainer\+::Data\+Producer Class Reference}
\label{classnntrainer_1_1DataProducer}\index{nntrainer::DataProducer@{nntrainer::DataProducer}}


\mbox{\hyperlink{classnntrainer_1_1DataProducer}{Data\+Producer}} interface used to abstract data provider.  




{\ttfamily \#include $<$data\+\_\+producer.\+h$>$}



Inheritance diagram for nntrainer\+::Data\+Producer\+:
\nopagebreak
\begin{figure}[H]
\begin{center}
\leavevmode
\includegraphics[width=350pt]{classnntrainer_1_1DataProducer__inherit__graph}
\end{center}
\end{figure}
\doxysubsection*{Public Types}
\begin{DoxyCompactItemize}
\item 
using \mbox{\hyperlink{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}{Generator}} = std\+::function$<$ bool(unsigned int, std\+::vector$<$ Tensor $>$ \&, std\+::vector$<$ Tensor $>$ \&)$>$
\begin{DoxyCompactList}\small\item\em generator callable type which will fill a sample \end{DoxyCompactList}\end{DoxyCompactItemize}
\doxysubsection*{Public Member Functions}
\begin{DoxyCompactItemize}
\item 
virtual \mbox{\hyperlink{classnntrainer_1_1DataProducer_a8048724118f13181a67c80ff813f8ca3}{$\sim$\+Data\+Producer}} ()
\begin{DoxyCompactList}\small\item\em Destroy the Data Loader object. \end{DoxyCompactList}\item 
virtual const std\+::string \mbox{\hyperlink{classnntrainer_1_1DataProducer_aa3f081fc338b4dbf18c6639a1b12daeb}{get\+Type}} () const =0
\begin{DoxyCompactList}\small\item\em Get the producer type. \end{DoxyCompactList}\item 
virtual void \mbox{\hyperlink{classnntrainer_1_1DataProducer_adec84ff25c17552145cc8b3af842749d}{set\+Property}} (const std\+::vector$<$ std\+::string $>$ \&properties)
\begin{DoxyCompactList}\small\item\em Set the Property object. \end{DoxyCompactList}\item 
virtual \mbox{\hyperlink{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}{Generator}} \mbox{\hyperlink{classnntrainer_1_1DataProducer_aa6443d25036ccb3e85b284ca51022291}{finalize}} (const std\+::vector$<$ Tensor\+Dim $>$ \&input\+\_\+dims, const std\+::vector$<$ Tensor\+Dim $>$ \&label\+\_\+dims, void $\ast$user\+\_\+data=nullptr)
\begin{DoxyCompactList}\small\item\em finalize the class to return an immutable Generator. \end{DoxyCompactList}\item 
virtual unsigned int \mbox{\hyperlink{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}{size}} (const std\+::vector$<$ Tensor\+Dim $>$ \&input\+\_\+dims, const std\+::vector$<$ Tensor\+Dim $>$ \&label\+\_\+dims) const
\begin{DoxyCompactList}\small\item\em get the number of samples inside the dataset, if size cannot be determined, this function must return. Data\+Producer\+::\+S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+N\+ED. \end{DoxyCompactList}\item 
virtual void \mbox{\hyperlink{classnntrainer_1_1DataProducer_ac99063a390b2187cf8ec4c95e6f9aa3f}{export\+To}} (\mbox{\hyperlink{classnntrainer_1_1Exporter}{Exporter}} \&exporter, const ml\+::train\+::\+Export\+Methods \&method) const
\begin{DoxyCompactList}\small\item\em this function helps exporting the dataproducer in a predefined format, while workarounding issue caused by templated function type eraser \end{DoxyCompactList}\item 
virtual bool \mbox{\hyperlink{classnntrainer_1_1DataProducer_a5411b64f5ff26d50a13638a0e5f53a29}{is\+Multi\+Thread\+Safe}} () const
\begin{DoxyCompactList}\small\item\em denote if given producer is thread safe and can be parallelized. \end{DoxyCompactList}\end{DoxyCompactItemize}
\doxysubsection*{Static Public Attributes}
\begin{DoxyCompactItemize}
\item 
constexpr static unsigned int {\bfseries S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+N\+ED}
\end{DoxyCompactItemize}


\doxysubsection{Detailed Description}
\mbox{\hyperlink{classnntrainer_1_1DataProducer}{Data\+Producer}} interface used to abstract data provider. 



Definition at line 33 of file data\+\_\+producer.\+h.



\doxysubsection{Member Typedef Documentation}
\mbox{\Hypertarget{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}\label{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!Generator@{Generator}}
\index{Generator@{Generator}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{Generator}{Generator}}
{\footnotesize\ttfamily using \mbox{\hyperlink{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}{nntrainer\+::\+Data\+Producer\+::\+Generator}} =  std\+::function$<$bool(unsigned int, std\+::vector$<$Tensor$>$ \& , std\+::vector$<$Tensor$>$ \& )$>$}



generator callable type which will fill a sample 


\begin{DoxyParams}[1]{Parameters}
\mbox{\texttt{ in}}  & {\em index} & current index with range of \mbox{[}0, \mbox{\hyperlink{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}{size()}} -\/ 1\mbox{]}. If \mbox{\hyperlink{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}{size()}} == S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+N\+ED, this parameter can be ignored \\
\hline
\mbox{\texttt{ out}}  & {\em inputs} & allocate tensor before expected to be filled by this function \\
\hline
\mbox{\texttt{ out}}  & {\em labels} & allocate tensor before expected to be filled by this function function. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
bool true if this is the last sample, samples will N\+OT be ignored and should be used, or passed at will of caller 
\end{DoxyReturn}


Definition at line 49 of file data\+\_\+producer.\+h.



\doxysubsection{Constructor \& Destructor Documentation}
\mbox{\Hypertarget{classnntrainer_1_1DataProducer_a8048724118f13181a67c80ff813f8ca3}\label{classnntrainer_1_1DataProducer_a8048724118f13181a67c80ff813f8ca3}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!````~DataProducer@{$\sim$DataProducer}}
\index{````~DataProducer@{$\sim$DataProducer}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{$\sim$DataProducer()}{~DataProducer()}}
{\footnotesize\ttfamily virtual nntrainer\+::\+Data\+Producer\+::$\sim$\+Data\+Producer (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



Destroy the Data Loader object. 



Definition at line 58 of file data\+\_\+producer.\+h.



\doxysubsection{Member Function Documentation}
\mbox{\Hypertarget{classnntrainer_1_1DataProducer_ac99063a390b2187cf8ec4c95e6f9aa3f}\label{classnntrainer_1_1DataProducer_ac99063a390b2187cf8ec4c95e6f9aa3f}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!exportTo@{exportTo}}
\index{exportTo@{exportTo}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{exportTo()}{exportTo()}}
{\footnotesize\ttfamily virtual void nntrainer\+::\+Data\+Producer\+::export\+To (\begin{DoxyParamCaption}\item[{\mbox{\hyperlink{classnntrainer_1_1Exporter}{Exporter}} \&}]{exporter,  }\item[{const ml\+::train\+::\+Export\+Methods \&}]{method }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



this function helps exporting the dataproducer in a predefined format, while workarounding issue caused by templated function type eraser 


\begin{DoxyParams}{Parameters}
{\em exporter} & exporter that conatins exporting logic \\
\hline
{\em method} & enum value to identify how it should be exported to \\
\hline
\end{DoxyParams}


Reimplemented in \mbox{\hyperlink{classnntrainer_1_1RawFileDataProducer_a6675b17adf09ce54151e2259cfb7def4}{nntrainer\+::\+Raw\+File\+Data\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1FuncDataProducer_a8a816e38e324f6b09f48b9cdfad7b70c}{nntrainer\+::\+Func\+Data\+Producer}}.



Definition at line 118 of file data\+\_\+producer.\+h.

\mbox{\Hypertarget{classnntrainer_1_1DataProducer_aa6443d25036ccb3e85b284ca51022291}\label{classnntrainer_1_1DataProducer_aa6443d25036ccb3e85b284ca51022291}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!finalize@{finalize}}
\index{finalize@{finalize}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{finalize()}{finalize()}}
{\footnotesize\ttfamily virtual \mbox{\hyperlink{classnntrainer_1_1DataProducer_a68a584a2abd402afb289fb79dc79426f}{Generator}} nntrainer\+::\+Data\+Producer\+::finalize (\begin{DoxyParamCaption}\item[{const std\+::vector$<$ Tensor\+Dim $>$ \&}]{input\+\_\+dims,  }\item[{const std\+::vector$<$ Tensor\+Dim $>$ \&}]{label\+\_\+dims,  }\item[{void $\ast$}]{user\+\_\+data = {\ttfamily nullptr} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



finalize the class to return an immutable Generator. 

\begin{DoxyRemark}{Remarks}
this function must assume that the batch dimension of each tensor dimension is one. If actual dimension is not one, this function must ignore the batch dimension and assume it to be one. 
\end{DoxyRemark}

\begin{DoxyParams}{Parameters}
{\em input\+\_\+dims} & input dimensions. \\
\hline
{\em label\+\_\+dims} & label dimensions. \\
\hline
{\em user\+\_\+data} & user data to be used when finalize. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
Generator generator is a function that generates a sample upon call. 
\end{DoxyReturn}


Reimplemented in \mbox{\hyperlink{classnntrainer_1_1RawFileDataProducer_a0bd1b3313b7ed516427e48749b11dd5b}{nntrainer\+::\+Raw\+File\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1DirDataProducer_af83efcbbc49c0614714c8cb11f589bd2}{nntrainer\+::\+Dir\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1RandomDataOneHotProducer_a5bb09f729be6f57a5a37951a12156e5c}{nntrainer\+::\+Random\+Data\+One\+Hot\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1FuncDataProducer_a7237e5cde6cb712271d62204e0ce53c0}{nntrainer\+::\+Func\+Data\+Producer}}.



Definition at line 88 of file data\+\_\+producer.\+h.

\mbox{\Hypertarget{classnntrainer_1_1DataProducer_aa3f081fc338b4dbf18c6639a1b12daeb}\label{classnntrainer_1_1DataProducer_aa3f081fc338b4dbf18c6639a1b12daeb}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!getType@{getType}}
\index{getType@{getType}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{getType()}{getType()}}
{\footnotesize\ttfamily virtual const std\+::string nntrainer\+::\+Data\+Producer\+::get\+Type (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [pure virtual]}}



Get the producer type. 

\begin{DoxyReturn}{Returns}
const std\+::string type representation 
\end{DoxyReturn}


Implemented in \mbox{\hyperlink{classnntrainer_1_1RawFileDataProducer_a8bb02b68155b77ef77fe3cfb95cdc75e}{nntrainer\+::\+Raw\+File\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1DirDataProducer_ae8b1a0a62184a1bc0e531a2b269d3a3a}{nntrainer\+::\+Dir\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1FuncDataProducer_a18ce5cac94bd853f473eae10523337d1}{nntrainer\+::\+Func\+Data\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1RandomDataOneHotProducer_a5c731c79719aaf5f9caa3951e4f02eb0}{nntrainer\+::\+Random\+Data\+One\+Hot\+Producer}}.

\mbox{\Hypertarget{classnntrainer_1_1DataProducer_a5411b64f5ff26d50a13638a0e5f53a29}\label{classnntrainer_1_1DataProducer_a5411b64f5ff26d50a13638a0e5f53a29}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!isMultiThreadSafe@{isMultiThreadSafe}}
\index{isMultiThreadSafe@{isMultiThreadSafe}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{isMultiThreadSafe()}{isMultiThreadSafe()}}
{\footnotesize\ttfamily virtual bool nntrainer\+::\+Data\+Producer\+::is\+Multi\+Thread\+Safe (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



denote if given producer is thread safe and can be parallelized. 

\begin{DoxyNote}{Note}
if \mbox{\hyperlink{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}{size()}} == S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+E\+ND, thread safe shall be false
\end{DoxyNote}
\begin{DoxyReturn}{Returns}
bool true if thread safe. 
\end{DoxyReturn}


Reimplemented in \mbox{\hyperlink{classnntrainer_1_1DirDataProducer_a87527ec9a3698e3dadbb3f94b7660d91}{nntrainer\+::\+Dir\+Data\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1RandomDataOneHotProducer_a1258d8dc1bc034a127f3273b455bee98}{nntrainer\+::\+Random\+Data\+One\+Hot\+Producer}}.



Definition at line 127 of file data\+\_\+producer.\+h.

\mbox{\Hypertarget{classnntrainer_1_1DataProducer_adec84ff25c17552145cc8b3af842749d}\label{classnntrainer_1_1DataProducer_adec84ff25c17552145cc8b3af842749d}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!setProperty@{setProperty}}
\index{setProperty@{setProperty}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{setProperty()}{setProperty()}}
{\footnotesize\ttfamily virtual void nntrainer\+::\+Data\+Producer\+::set\+Property (\begin{DoxyParamCaption}\item[{const std\+::vector$<$ std\+::string $>$ \&}]{properties }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



Set the Property object. 


\begin{DoxyParams}{Parameters}
{\em properties} & properties to set \\
\hline
\end{DoxyParams}


Reimplemented in \mbox{\hyperlink{classnntrainer_1_1RawFileDataProducer_ae2f2ce3a68263d6f2f5c479c7791f478}{nntrainer\+::\+Raw\+File\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1DirDataProducer_ac8f864e742f6a447cc0455dcdfb18ac0}{nntrainer\+::\+Dir\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1RandomDataOneHotProducer_af816e4886e14207f4d4be3037b395174}{nntrainer\+::\+Random\+Data\+One\+Hot\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1FuncDataProducer_a64335f98c596f897264d5c393eefdaf1}{nntrainer\+::\+Func\+Data\+Producer}}.



Definition at line 71 of file data\+\_\+producer.\+h.

\mbox{\Hypertarget{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}\label{classnntrainer_1_1DataProducer_aeab0bfee2f7d741e7317716fcf43f00b}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!size@{size}}
\index{size@{size}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{size()}{size()}}
{\footnotesize\ttfamily virtual unsigned int nntrainer\+::\+Data\+Producer\+::size (\begin{DoxyParamCaption}\item[{const std\+::vector$<$ Tensor\+Dim $>$ \&}]{input\+\_\+dims,  }\item[{const std\+::vector$<$ Tensor\+Dim $>$ \&}]{label\+\_\+dims }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [virtual]}}



get the number of samples inside the dataset, if size cannot be determined, this function must return. Data\+Producer\+::\+S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+N\+ED. 

\begin{DoxyRemark}{Remarks}
this function must assume that the batch dimension of each tensor dimension is one. If actual dimension is not one, this function must ignore the batch dimension and assume it to be one 
\end{DoxyRemark}

\begin{DoxyParams}{Parameters}
{\em input\+\_\+dims} & input dimensions \\
\hline
{\em label\+\_\+dims} & label dimensions\\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
size calculated size 
\end{DoxyReturn}


Reimplemented in \mbox{\hyperlink{classnntrainer_1_1RawFileDataProducer_a5df3bab03882fedf588b2ca68d682ca4}{nntrainer\+::\+Raw\+File\+Data\+Producer}}, \mbox{\hyperlink{classnntrainer_1_1DirDataProducer_ade22374c3f052ee3bdf2863663949196}{nntrainer\+::\+Dir\+Data\+Producer}}, and \mbox{\hyperlink{classnntrainer_1_1RandomDataOneHotProducer_a6dc94b96565f1e7d2e429ff3305f22b1}{nntrainer\+::\+Random\+Data\+One\+Hot\+Producer}}.



Definition at line 106 of file data\+\_\+producer.\+h.



\doxysubsection{Member Data Documentation}
\mbox{\Hypertarget{classnntrainer_1_1DataProducer_a3755b7d3ac5cb630e07740e7f0438204}\label{classnntrainer_1_1DataProducer_a3755b7d3ac5cb630e07740e7f0438204}} 
\index{nntrainer::DataProducer@{nntrainer::DataProducer}!SIZE\_UNDEFINED@{SIZE\_UNDEFINED}}
\index{SIZE\_UNDEFINED@{SIZE\_UNDEFINED}!nntrainer::DataProducer@{nntrainer::DataProducer}}
\doxysubsubsection{\texorpdfstring{SIZE\_UNDEFINED}{SIZE\_UNDEFINED}}
{\footnotesize\ttfamily constexpr static unsigned int nntrainer\+::\+Data\+Producer\+::\+S\+I\+Z\+E\+\_\+\+U\+N\+D\+E\+F\+I\+N\+ED\hspace{0.3cm}{\ttfamily [inline]}, {\ttfamily [static]}, {\ttfamily [constexpr]}}

{\bfseries Initial value\+:}
\begin{DoxyCode}{0}
\DoxyCodeLine{=}
\DoxyCodeLine{    std::numeric\_limits<unsigned int>::max()}

\end{DoxyCode}


Definition at line 51 of file data\+\_\+producer.\+h.



The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize}
\item 
dataset/\mbox{\hyperlink{data__producer_8h}{data\+\_\+producer.\+h}}\end{DoxyCompactItemize}
