Another approach would be to handle inside the water material. If you transfer the center location and radius of the sub’s glass sphere you can easily calculate the area that needs to be masked.
Pros:
- Cheap
- Works in all cases (inside, outside, etc.)
- Works with both transparent and opaque materials.
- Same method works for your particles and other stuff that should not show up inside
Cons:
- You have to know the number of objects that can cause masking ahead of time and each one will adds a few instructions to the shader (so only use it if you have a small number of “hero” objects)
- Only works with shapes that can be easily mathematically defined (spheres, capsules, boxes, etc.)