navy! During the cutscene where mayor tilly is kidnapped, this is explained, and is also the same reason why they do not know who Morden is.
It is also HEAVILY suggested that the MC talks to everyone more than we see. Namely because they befriend Neviro despite us only having 2, admittedly cold, interactions with the dude via ingame dialogue.
In that same breath, its suggested that going from either cirrus to skycliff, or to the ruins to fight Elius, took us 1 whole week in lore. And was apparently also so loud that Neviro heard us fighting [look to Neviros diary]
In that same breath, since we spent well over a week in cirrus, its more than reasonable for a lot of the character interactions, lore, and of courrse, becoming known. I mean, if theres this random dude who pops up with really strong magic and is wiping out pirates for fun, people will certainly notice