Coloring as a Pretext Task for Semantic Segmentation in Point Cloud

robotics, autonomous vehicles, and 3D mapping. However, point cloud data is often scarce and
difficult to label, making it challenging to train supervised models. As a result, there is a need
to investigate different limited supervised methods to improve the performance of point cloud
processing tasks.
One promising approach is self-supervised learning, where supervision is generated from the
data itself. In this method, a model learns from the data without the need for explicit labels.
In my work, I selected coloring as a pretext task for my models because it has a strong relation
with semantic segmentation. By learning to color the point cloud, the model can learn features
that are useful for semantic segmentation, such as object boundaries and shape information.
This approach is more interesting in point cloud data compared to regular images because the
model has to learn the color from the geometry of the shape instead of relying on grayscale
color information as in 2D images. Overall, the use of coloring as a pretext task for semantic
segmentation in point cloud data is a promising approach that can lead to improved performance
in point cloud processing tasks. In my work, I have investigated two different datasets for the
task of point cloud coloring as a pretext task for semantic segmentation. The first dataset is
S3DIS, which is an indoor colored point cloud dataset.The second dataset is ShapeNet, which is
a 3D object dataset. However, ShapeNet does not include color information, so I developed a
method to extract color point cloud information from the mesh data. To train my model on the
coloring pretext task, I used a conditional GAN (Generative Adversarial Network) model. In this
framework, my model played the role of the generator, which generates the colored point cloud.
The GAN model allows the model to learn to generate realistic colored point clouds, which are
useful for the semantic segmentation task, as the model learns the relation of geometry ( XYZ)
and its color.