How were dangerous/incompetent captains relieved of duty in the British and other Navies?

[SpyderEdgeForever]

I was watching a movie about the British Navy sailors back in the 19th century and one of the captains began to "go overboard" in his treatment of the crew, and it seems began to have severe mental problems and was unfit for duty. In real history, both antiquated and modern, under what grounds could such a captain of a ship be relieved of duty without it being declared a mutiny? Was that up to the chief medical officer to determine?