Nowadays, I use VHT FlameProof (not engine enamel) for almost everything under the hood that will see heat. I've tried other stuff, and nothing looks or holds up as well. The VHT FlameProof also covers awesome without primer or any of that shit. For prep, just make sure the trans is super clean. Any grease or oil will make for a crappy finish, but that goes for any paint. Simple Green, a brush and tons of scrubbing should do the job. For tough areas, use Simple Green and scrub with a dremel + small wire wheel. The VHT finish is more of a matte, which looks nicer (and more stock), IMO.
I've done my last few transmissions this way, as well as the one I recently did in my brother's car:
1837/2000 newly painted Shep trans and old trans after ~4k miles. Both VHT'd, and the old one still looks pretty good. Most of the fucked up spots on the old trans came from removal.