Intermediate output

Foreground objects: Person detection → Instance Segmentation ( I was able to improve the instance segmentation result by giving cropped image). Saved in .pickle file and .json file (y, x, height, width for person detection, Masking 1,0 value for instance segmentation)

1. Input raw image

   

2. Track foreground objects. (‣). Our Approach can handle multiple people's cases because I track the person's id.

   

3. Crop the bounding box. It will be an input for the instance segmentation result. I improved the performance of the instance segmentation model by giving cropped images.

   

4. Extract instance segmentation from the cropped image. (‣) The instance location result is saved as a binary_masking. (Same size as the original image. If 1, including, if 0, not including). By giving cropped images to the instance segmentation model, I can get a more correct result.

   

5. Crop the masking part(blue color above) from the original frame.

   

6. Make all the foreground frames with this method. save the location info to the .pickle file and .json file.

Application3 (Removed foreground video)

Using DeepMCBM results. Replace the Foreground parts in our data image with the DeepMCBM result.