you didn’t pay attention to the dialog
Calvus states after you defeat him that he just wanted to be strong and revered like his grandfather and father were, and that the Order took him in at a young age and taught him how to fight. He’s not a curse holder, he’s a Savant that knows Aether Magic, and his downfall was that he tried mastering too many different powers at the same time, to the point that he was average in all of them. His strength was carried by his Aether Magic, not his actual combat skill
Its also stated by Warren that actual curse holders such as Wotan’s children could easily kill all of you, even Morden