• Data of scenes:
• We use LERF data (https://www.lerf.io), but the original data's cameras are not accurate. Therefore, we calibrated with COLMAP.
• The data is provided in COLMAP format: thunberg-ub.ict.usc.edu/data/Hanyuan/lerf/data/nerfstudio/[SCENE_NAME]. Let's try SCENE_NAME=figurines first.
• Task for comparison:
• Step 1: Remove pumpkin on the table. (text prompt="remove pumpkin on table")
• Step 2: At the location/bbox of pumpkin, insert a pineapple. (text prompt="a pineapple") | Step1:

1. train_init_nerf.py [30k iters 23min]
2. train_init_nerf.py [300k iters 11h20min]
3. train_distill.py [45k iters 4h7min]
4. train_distill.py [300k iters 19h45min] Step 2:
5. sample_view.py [10 iter 5min]
6. train_dreambooth.py [50 iter 7min + 400 iter 7min]
7. locating_editing_region [5min] Step3: train_editing.py [5000 iter 4h28min]

Total: 40.45h | | Instruct nerf2nerf | ‣ | | | | ViCA-NeRF | ‣ | | | | | ‣ | | |

2. Put a square cushion on the chair | 1. Turn the pumpkin into a walking astronaut

3. Turn the pumpkin into a pink ballon dog made of metal

4. Turn the pumpkin into a pineapple

5. Turn the red chair into a white sculpture

6. Turn the white hand into a green cartoon dinosaur | 1. Put a teddy bear on the bench | | edited | 1. /home/jyang/projects/VICA-NeRF/outputs/garden/vica/2024-03-01_164930 | 1. /home/jyang/projects/VICA-NeRF/outputs/bouquet/vica/2024-03-01_214417 | 1. /home/jyang/projects/VICA-NeRF/outputs/bonsai/vica/2024-03-01_164933 | 1. /home/jyang/projects/VICA-NeRF/outputs/poster/vica/2024-03-02_003320

7. /home/jyang/projects/VICA-NeRF/outputs/poster/vica/2024-03-02_022501 | 1. /home/jyang/projects/VICA-NeRF/outputs/figurines/vica/2024-03-01_215348

8. /home/jyang/projects/VICA-NeRF/outputs/figurines/vica/2024-03-02_004714

9. /home/jyang/projects/VICA-NeRF/outputs/figurines/vica/2024-03-02_123853

10. /home/jyang/projects/VICA-NeRF/outputs/figurines/vica/2024-03-04_145821

11. /home/jyang/projects/VICA-NeRF/outputs/figurines/vica/2024-03-04_141612 | 1. /home/jyang/projects/VICA-NeRF/outputs/bicycle/vica/2024-03-03_222939 | | status | done | done | done | done | done | done | | | | | | | | | | GaussianEditor | Mipnerf garden | lerf bouquet | mipnerf bonsai | lerf poster | lerf figurines | mipnerf bicycle | | location | vgl-gpu08 | vgl-gpu08 | vgl-gpu08 | vgl-gpu08 | vgl-gpu08 | vgl-gpu08 | | GS ckpt | - | /home/jyang/projects/gaussian-splatting/output/a8196970-e/point_cloud | - | /home/jyang/projects/gaussian-splatting/output/b1707dff-6/point_cloud | /home/jyang/projects/gaussian-splatting/output/5fb259a3-e/point_cloud | - | | GS ply 7k | /home/jyang/projects/GaussianEditor/dataset/garden/point_cloud/iteration_7000/point_cloud.ply | | /home/jyang/projects/GaussianEditor/dataset/bonsai/point_cloud/iteration_7000 | /home/jyang/projects/GaussianEditor/dataset/lerf-data/poster/point_cloud/splat.ply | | /home/jyang/projects/GaussianEditor/dataset/bicycle/point_cloud/iteration_7000/point_cloud.ply | | editing prompt | 1. Turn the vase into a buddha statue | 1. Turn the vase and flowers into a white sculpture | 1. Turn the bonsai into a green plant in a pot | 1. Put a pikachu on the chair

12. Put a square cushion on the chair | 1. Turn the pumpkin into a walking astronaut

13. Turn the pumpkin into a pink ballon dog made of metal

14. Turn the pumpkin into a pineapple

15. Turn the red chair into a white sculpture

16. Turn the white hand into a green cartoon dinosaur | 1. Put a teddy bear on the bench | | edited (instruct pix2pix) | 1. /home/jyang/projects/GaussianEditor/ui_result/garden/instruct/2024-03-02-18:53.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/bouquet/instruct/2024-03-03-00:28.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/bonsai/instruct/2024-03-02-18:29.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/poster/instruct/2024-03-03-00:26.ply

17. /home/jyang/projects/GaussianEditor/ui_result/poster/instruct/2024-03-03-01:03.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/figurines/instruct/2024-03-03-00:26.ply

18. /home/jyang/projects/GaussianEditor/ui_result/figurines/instruct/2024-03-03-01:02.ply

19. /home/jyang/projects/GaussianEditor/ui_result/figurines/instruct/2024-03-03-01:24.ply | | | edited (controlnet pix2pix) | 1. /home/jyang/projects/GaussianEditor/ui_result/garden/controlnet/2024-03-02-19:12.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/bouquet/controlnet/2024-03-03-00:46.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/bonsai/controlnet/2024-03-02-18:54.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/poster/controlnet/2024-03-03-00:45.ply

20. /home/jyang/projects/GaussianEditor/ui_result/poster/controlnet/2024-03-03-01:17.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/figurines/contrlnet/2024-03-03-00:45.ply

21. /home/jyang/projects/GaussianEditor/ui_result/figurines/contrlnet/2024-03-03-01:12.ply

22. /home/jyang/projects/GaussianEditor/ui_result/figurines/contrlnet/2024-03-03-01:33.ply | | | add | 1. | | | 1. /home/jyang/projects/GaussianEditor/ui_result/poster-add/2024-03-03-21:43.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/figurines-add/pumpkin/2024-03-04-16:40.ply (delete)

/home/jyang/projects/GaussianEditor/ui_result/figurines-add/pumpkin/2024-03-04-16:52.ply (add)

4. /home/jyang/projects/GaussianEditor/ui_result/figurines-add/redchair/2024-03-04-17:02.ply (delete)

/home/jyang/projects/GaussianEditor/ui_result/figurines-add/redchair/2024-03-04-17:19.ply (add)

/home/jyang/projects/GaussianEditor/ui_result/figurines-add/redchair/2024-03-04-21:05.ply

/home/jyang/projects/GaussianEditor/ui_result/figurines-add/redchair/2024-03-04-21:53.ply

5. /home/jyang/projects/GaussianEditor/ui_result/figurines-add/hand/2024-03-04-17:34.ply (delete)

/home/jyang/projects/GaussianEditor/ui_result/figurines-add/hand/2024-03-04-21:38.ply/home/jyang/projects/GaussianEditor/ui_result/figurines-add/redchair/2024-03-04-21:53.ply | 1. /home/jyang/projects/GaussianEditor/ui_result/bicycle-add/2024-03-03-17:04.ply (small)

/home/jyang/projects/GaussianEditor/ui_result/bicycle-add/2024-03-03-17:24.ply (large) |