I didn't touch the spreadsheet (I did the calculation by hand).
Anyway, just to guide you on how to do this:
Let's work on the light liquid (oil) side. You got Vl,ax, which is the horizontal velocity of oil.
You also have Lset, which is the effective settling length. Divide Lset by VL,ax will give you the horizontal residence time (of oil).
When doing separator design, the goal is your horizontal residence time should be greater than your vertical residence time. Since you already have the vessel dimension, what you can do is to set both equally. In this way, you can back calculate the minimum particle diameter your existing vessel can disengage.
Going back to the analysis, what do you disengage in oil? Obviously it's water. So you have to consider the settling velocity of water particles in oil. Where does that happen? Obviously in the oil portion in the vessel.
Given the horizontal residence time and the vertical distance between the oil-water interface and the feed (because water has to come from the feed which is on the topmost portion of the vessel), you can calculate the equivalent settling velocity.
You don't need to use the equation given in DEP because you are doing vessel rating. You already have the vessel dimension at hand so what you can do is to back calculate everything to get the effective settling velocity (and ultimately the minimum particle diameter that can be possibly disengaged).
