Sometimes it is the little things that can drive you nuts. In VB6 I was used to having a ToggleButton control but I just went to use the equivalent in a .Net form for the first time and it was nowhere to be found.
Turns out that if you need this effect on a .Net winform, use the CheckBox control and set the Appearance property to "Button". The button appears "down" when Checked is true. Simple question, simple answer (once you find it).
Hope that helps.