%%class
\documentclass{iaesarticle2}

%%required package. add for your convenient, but do not remove the initial
\usepackage{amsmath, amsfonts, amssymb, float, fancyhdr}
\usepackage[figuresright]{rotating}
\usepackage{authblk, graphicx, indentfirst, lastpage, lipsum}
\setlength{\affilsep}{0cm}
\renewcommand\Authfont{\normalsize}
\renewcommand\Affilfont{\normalfont\small}
\usepackage{subfig, caption, epstopdf}
\usepackage[left=2.5cm, right=2cm, top=1.5cm, bottom=2cm, includehead, includefoot]{geometry}
\usepackage{caption}
\captionsetup{labelsep=period}
\usepackage{titlesec}

\usepackage{enumitem}
%\usepackage[numbers]{natbib}
\usepackage{geometry}
\usepackage[numbers]{natbib}

\titleformat{\section}
  {\normalfont\normalsize\bfseries\uppercase}{\thesection}{1em}{}
\titlespacing*{\section}{0cm}{0.7cm}{0cm}

%%leave copyright info to the editor
\CopyrightLine[Copyright]{201x}{Institute of Advanced Engineering and Science.}

%%author
\author[*]{\bfseries Hanry Ham}
%%author's affiliation
\affil[*]{Computer Science Department, School of Computer Science, Bina Nusantara University, Jakarta, Indonesia 11480}

%%title and shortitle (for footer)
\title{Computer Vision Based 3D Reconstruction : A Review}
\shorttitle{Computer Vision Based 3D Reconstruction : A Review (Ham Hanry)}

%%starting
\begin{document}

%%indentation. do not change
\setlength{\parindent}{1.27cm}

%%header and footer setting. do not change
\pagestyle{fancy}
\fancyhfoffset{0cm}

%%journal info
\journalname{International Journal of Electrical and Computer Engineering (IJECE)}
\journalshortname{IJECE}
\journalhomepage{http://iaesjournal.com/online/index.php/IJECE}
\vol{x}
\no{x}
\months{May}
\years{2013}
\issn{2088-8708}

%%build title
\maketitle

%%border setting. do not change
\hrule
\vspace{.1em}
\hrule
\vspace{.5em}
\noindent
\parbox[t][][s]{0.275\textwidth}{%
\textbf{Article Info}
\vspace{.5em}
\hrule
\vspace{.5em}
\begin{history}

%%article info. editor's privilege
Received \today
\par 
Revised 
\par 
Accepted

\end{history}
\vspace{.5em}
\hrule
\vspace{.5em}
\begin{keyword}

%%write keyword here. separate by \sep
3D reconstruction \sep 3D point clouds \sep 3D alignment

\vspace{.5em}
\end{keyword}
\vspace{\fill}
}
\parbox{0.025\textwidth}{\hspace{0.5em}}
\parbox[t][][s]{0.7\textwidth}{%
\begin{abstract}
%% Text of abstract
3D reconstruction are used in many fields starts from the object reconstruction such as site, and cultural artifacts in both ground and under the sea levels. The scientist are beneficial for these task in order to learn and keep the environment into 3D data due to the extinction. In this paper explained vision setup that is commonly used such as single camera, stereo camera, Kinect / Structured Light/ Time of Flight camera and fusion approach. The prior works also explained how the 3D reconstruction perform in many fields and using various algorithms.
\end{abstract}
}
\parbox[l]{\textwidth}{%
\rule{0.275\textwidth}{0.5pt} \hspace{0.5cm} \hrulefill
\\
\emph{\textbf{Corresponding Author:}}\\
%% correspondence info. separate by \\
Hanry Ham\\
Computer Science Department\\
School of Computer Science\\
Bina Nusantara University\\
hanry.ham@binus.edu
}
\vspace{.5em}
\hrule
\vspace{.1em}
\hrule


%% main text

\section{Introduction}
\par
3D Reconstruction task is one of the interesting task that meet its maturity already. These can be seen from the commercial products such as product from Agisoft and Pix4D that are capable of produced high quality of large scale 3D models. Furthermore, the hardware such as the computer vision has been developed and improve since then. There are some setup camera introduced in the research such as stereo camera and Kinect.

Moreover, Kinect camera shows a great positive feedback from the researchers, proved by common vision setup that can be found in the literature review. Not only that, stereo camera setup can be found among the literature review. In addition to the stereo camera, custom stereo camera are quite popular among the researchers by combining two equals web camera that positioned by period of distance. The algorithm to perform 3D reconstruction between these camera are different due to the produced images are different as wel. Kinect abilities allows RGB image and depth map produced, on the other hand Stereo camera has to perform another depth map acquisition algorithm by combining 2 RGB images.

Numerous numbers of 3D reconstruction task can be found in capturing the site, cultural artifacts both in ground and under the sea levels \cite{Anwer2016}. The extinction factor is the most prominent issue in these area. Moreover, 3D imaging data also could help improve the accuracy of the anatomical features in order to observe some areas before coming to the surgery action. 

\subsection{Object reconstruction}
\par
The great numbers of the researchers along with the hardware supports allows such algorithm to do high processing calculation in order to perform reconstruction task. There are some sections mentioned in part  \ref{taxonomy}. The benefits of reconstruction are to perform 3D recording, visualization, representation and reconstruction \cite{Tsiafaki2015}. Moreover \citeauthor{Tsiafaki2015} explained that, there are 6 benefits in performing reconstruction and visualization: limiting the destructive nature of excavating, placing excavation data into the bigger picture, limiting fragmentation of archaeological remains, classifying archaeological finds, limiting subjectivity and publication delays, enriching and extending archaeological research.
\newline

\subsection{Vision Setup} \label{visionSetup}
In this paper will described the vision setup by 3 categories as follows:
    %Explantion here
    \begin{enumerate}
        \item Single Camera
        \newline
        A single camera is simple to calibrate, computationally efficient more compact. However, they are lack of the depth information. It requires prior knowledge from other sensor to determine the depth scale \cite{Santoso2015}.
        
        \item Stereo Camera
        \newline
        In stereo camera mechanism is that the images captured either using 2 equals web camera \cite{Harjoko2017} or any cameras. They are set by a defined distance. In addition to 2 images captured, an algorithm is used to generate depth map. However, stereo matching have several issue when the scene contains weekly textured areas, repetitive patterns or occlusions occur in both indoor and outdoor environments \cite{Evangelidis2015}.
        \begin{figure}[h]
            \centering
            \includegraphics[width=0.35\textwidth]{assets/StereoCam.png}
            \caption{Stereo Camera}
            \label{fig:stereoCam}
        \end{figure}
        
        \item Kinect / Structured Light / Time of Flight
        \newline
        %exp here
        Structured Light sensor is able to perform range detection, an accurate distance measurement is the output \cite{M2017}. Kinect camera is a product from Microsoft that has an RGBD camera. The product comes with native SDK that allows user to call the API to perform some vision task such as skeleton detection.
        
        \item Fusion
        %exp here
        \newline
        Some researchers also tried possibilities of using fusion approach where as combining depth map produced by Stereo and kinect camera to achieve higher accuracy in depth map precision. To such development allows to produce better 3D Reconstruction object, rich in features details.  Range cameras are low cost and ease to use to construct 3D point clouds in real time. One issue arise is that the transparent and reflective surfaces \cite{Ravanelli2016}. on the other hand, 3D model produced by stereo vision are mostly incomplete in low texture regions. The possibilities of combining both approached could lead to better depth map quality.
        \begin{figure}[h]
            \centering
            \includegraphics[width=0.35\textwidth]{assets/acquisitionNonGreen.png}
            \caption{Fusion Approach}
            \label{fig:fusionoCam}
        \end{figure}
\end{enumerate}

\pagebreak
\section{Taxonomy of 3D reconstruction}\label{taxonomy}
\newline
3D Reconstruction plays an important roles in several aspects such as medical imaging data, site and cultural artifact reconstruction.

\begin{enumerate}[label=(\alph*)]
    \item Medical Imaging Data
    \par
    Common surgery operation procedures uses X-Ray as a reference for the doctor to operate on specific section. However, some important features cannot be visualized well in 2D images \cite{Yao2003}. In addition to 2D images, the accuracy may increase depends on several aspects such as: number of 2D Views, the image noise, and the image distortion. Magnetic resonance Images also holds an important method while considering the operation process. The given output of MRI are in 2D images, however there are some literature can be found in manipulating those images into 3D space. By implementing such method, they would like to prove the more features captures, the more accurate result is.
    
    \item Site and cultural artifacts Reconstruction
    \par
    The site reconstruction has been widely an issue to the archaeology in order to capture the social, culture through the building, they do the reconstruction. Regular camera can only allow to capture in 2D space format. Not all the details from the building can be captured and closely observed. Since then, by using stereo camera or Kinect make this task possible along with the algorithm developed in the current research. The archaeological sites are not only on the ground but also under the sea. The reconstruction which performed under the sea rises another issue to the images captured such as degradation quality if underwater images, uneven illumination of light on the surface of objects. scattering and absorption effects \cite{Anwer2016}.
\end{enumerate}

%single view
\citet{Group2017} explained that a fully convolutional 3D denoising autoencoder neural network. They experimented using RGBD dataset and it is proved that the network could reconstruct a full scene from a single depth image by filling holes and hidden element. The network is capable of learn the object shape by inferring similarities in geometry. A real-word dataset of table top scenes \cite{Firman2016} was used using KinectFusion. Their steps can be mentioned as follows: acquisition RGBD image using Kinect, denoising and hole filling depth channel using \cite{Liu2016} algorithm, projection of the pixel into 3D space using preset equations, retrieve sensor pose from accelerometer and align point cloud data, voxelize the point cloud, and A predefined CNN layer was trained. In addition to that, the network is not constrained to a fixed 3D shape and it is capable successfully reconstructing  arbitrary scenes.
\
\citet{Yan} proposed a system called Perspective Transformer nets. The model was built by ignoring the color and texture factors. In addition to that, the experiments shows that excellent performance of the proposed model in reconstructing the object without ground-truth 3D volume as supervision. The input used were provided by \citet{Chang2015} works. The images input proposed is a single view 3D volume reconstruction \cite{Rezende2016} with perspective transformation \cite{Wu2016} run through defined encoder-decoder network that consists of a 2D convolutional encoder, a 3D up-convolutional decoder and a perspective transformer networks.

\citet{Burns2017} introduced a texture super resolution (TSR) method for 3D multi-view reconstruction. In addition, their work used video sequence as the input. Moreover to the proposed pipeline, a Photoscan from Agisoft is used to do multi-view stereo reconstruction and 3D mesh model. Then, optical flow algorithm is integrated in order to register each pixel of neighboring to the closest key-frame using KLT feature tracker \cite{Bouguet1981}. Afterwards, to support robustness to outliers the fundamental matrix filtering of the tracked 2D points and RANSAC filtering of the 2D/3D correspondences. Due to the piece-wise affine surface approximation constructed in 3D mesh, this may lead to pixels registration error. To overcome that issue, to locate the displacements, an optical flow estimation is used \cite{Plyer2016}. The object used is 2mx1m desk that has many textured objects on it as gray-scale images along with the subsampling applied to it. It is acquired using a camera with 5.5mm focal length at f/2.8 mounted on a Byaer 1/18" e2v detector. There are 3 experiments conducted  and it shows that the proposed methods outperforms compared to the registration with mesh and camera poses only, registration with optical flow only.

\citet{Zhang2017} developed a feature-based RGBD camera pose optimization for real-time 3D reconstruction. Their proposed work are ignoring corner-based feature detectors such as BRIEF and FAST due to acquired images contains huge noise around object contours. Subsequently, SURF detector was chosen due to the fact that its robustness, stability, scaleable and rotation invariant \cite{Juan2009}. In addition to that, SURF can be computed in parallel on the GPU \cite{Yan2013}. The miss-matched pairs in feature matching can be removed using RANSAC algorithm. The consistency of the global positions of matched features are tracked by proposed feature correspondence list and camera pose optimization both in the spatial and temporal dimension. Subsequently, in order to evaluate the method, voxel-hashing was used for each camera poses compared to the proposed method. It is proved that their proposed optimized camera poses outperforms the structure of the reconstruct model for the real scene data captured by a fast moving camera.

\citet{Fan2017} applied a region-based growing algorithm for 3D reconstruction by using brain MRI images. There are 3 steps in their proposed pipeline : First, the seed element is the initial state of the segmentation. Second, start the growing process from the seed element. There are 4 areas of the growth area. However there are some defined threshold value to meet the pattern of growth. Third, use the points which satisfy the growing requirement as seed element, and continue to grow. in addition to the result, their proposed method could achieve 90.52\% compared to \citet{Nadu2015} works.

\citet{Evangelidis2015} combined low-resolution depth data with high resolution stereo data to overcome the construction of high-resolution depth maps for the range-stereo fusion problem. The input used stereo images (high resolution) and depth data (low resolution) from the range camera. The low resolution depth data are projected into the color data and refined a high resolution sparse disparity map. Subsequently, the depth up-sampling algorithms were perform such as triangulation-based interpolation and join bilateral filter. then a region growing fusion were performed and final denser High resolution map as the result.

\citet{Kowalski2015} created an open source system for live, 3D data acquisition using multiple kinect v2 Sensors. To overcome the ability of the native Kinect V2 SDK, they made this flexible framework. There are 3 coordinates system of a markers: Kinect v2 sensor, coordinate system of a marker which is located at a center on a given marker and the world coordinate. The proposed pipeline as follows: first calibrations were done by calibrating 2 types of defined markers. Subsequently the Iterative Closest Points (ICP) algorithm \cite{Besl1992} was used to refine the initial estimation.

\citet{Jaiswal2014} used Kinect to assess 3D object modelling. The proposed pipeline are as follows: first, 3D point cloud, a green surface was placed behind and under the object to do the histogram-based segmentation out the object from the RGB images. Afterwards, RANSAC algorithm is used to perform a coarse alignment. Second, the registration using SIFT based \cite{Xie2013} to overcome the lack structural features or undergo significant changes in camera view. Third, global alignment is used ti eliminate inaccuracy at each registration that could lead to significant misalignment between the first and last frame. Fourth, 3D point cloud denoising is performed to refine the 3D object model, in this case Moving Least Square (MLS) 3D model denoising method \cite{Avron2010}. Fifth, surface reconstruction using Delaunay triangulation method \cite{Isenburg2006} to convert 3D point clouds into meshed. Afterwards, coloring task is performed to each vertex and simply interpolate the color in each triangle faces.

%\citet{Sobani2013} developed 3D object reconstruction using radon transform. The acquisition setup used is mobile phone camera. It captured 36 different angles in order to get the projection images with blue background behind and under the object.

\section{Discussion}
The oldest paper cited in this paper is \citeyear{Bouguet1981} and the research about 3D reconstruction is still going on. This proved that the maturity of the research in this area is achieved. There are numerous algorithms is described in solving numerous of problems. In addition, the commercial software such as Microsoft, Agisoft, intel real sense, asus and many others companies develop software and hardware to perform such calculation. The general pipelines found in the literature reviews are: first, image acquisition. There are some datasets available that can be used in order to evaluate the performance of the proposed algorithms. Moreover, chances to create own object using vision setup mentioned earlier in section \ref{visionSetup}. Second, Pre-processing step by allowing some filters applied to get the best images to construct. Third, 3D cloud points. The alignment algorithm plays an important role to get decent accuracy. Along with the refinement method in mismatched 3D cloud registration. Fourth, 3D reconstruction is where the texturing and meshed are applied as the final result.

\section{Conclusion}
In this paper explains several current 3D reconstruction methods from literature review. There are various algorithm in order to perform each step of general algorithm of 3D reconstruction. Each object constructed required special algorithms depends on the vision setup, the texture and size of the observed object. The improvement of the sensor could lead to the higher accuracy of creating 3D reconstruction in the future besides the efficient algorithms. Modeling using neural network shows a great advantages \cite{Group2017}, \cite{Yan}. The defined network will try to learn the shapes and will fill the occlusion region automatically.

\section*{Acknowledgement}
\label{}
The author also would like to acknowledge Bina Nusantara University for the grant research funding.

%% The Appendices part is started with the command \appendix;
%% appendix sections are then done as normal sections
%% \appendix

%% \section{}
%% \label{}

%% References
%%
%% Following citation commands can be used in the body text:
%% Usage of \cite is as follows:
%%   \cite{key}         ==>>  [#]
%%   \cite[chap. 2]{key} ==>> [#, chap. 2]
%%

%% References with BibTeX database:

\bibliographystyle{IEEEtranN}
\bibliography{literatur}

%% Authors are advised to use a BibTeX database file for their reference list.
%% The provided style IEEEtran.bst formats references is generally used.

%% For references without a BibTeX database:


\section*{BIOGRAPHY OF AUTHORS}

\begin{biography}[{\includegraphics[width=3cm,height=4cm,clip,keepaspectratio]{Author's_Photo}}]
\textbf{Hanry Ham}
%% Affiliation and educational background.
is a lecturer and research assistant at Bina Nusantara University with Master of Engineering from The Sirindhorn International Thai-German Graduate School of Engineering in Thailand and German (2016). He obtained Bachelor Degree in Computer Science from Bina Nusantara University (Indonesia) in 2014.
%% Research interests, teaching, professional experiences, etc.
His researches are in fields of image processing, computer vision and computer graphics.
%% Related activities and services.
He is affiliated with IEEE as student member. Besides, he is also involved in student associations, and committee of several competitions such as BNPCHS and ACM-ICPC Regional Asia Site.
%% Additional info.
\end{biography}


\end{document}

%%
%% End of file `iaesarticle2.tex'. 