(Tldr at bottom)
Unless the moves you get from it are not too useful and mastery tiers don’t affect magic size, you kinda gotta if you wanna have peak current efficiency similar to charging a full Impact Fist in AA/AR before a big battle.
I’d say while leveling up this system is alright in concept but it being only attacks is the main flaw, if it had multiple different scenarios that increase the mastery would help with it, like beating your rival with FS increasing exp, defeating npcs increasing it slightly, defeating boss mirages increasing it (increases more if you do it with the storyline quest), etc.
It only being slapping at ANYTHING just makes it a simulator like system, it’d be better imo if it rewarded you for actually doing stuff instead of just punching air for 30 minutes being the most efficient option.
Tldr; make it so actually doing stuff like fighting npcs or bosses be the most efficient method of gaining mastery so it encourages you to actually play instead of macroing for 30 minutes (not even a drawback for switching an entire FS) being the fastest option