The rules state that if there it not an explicit 'activate on hit' written in an ability's description then the ability activates when the die lands anywhere.
Is this really true of the 'Sworn Enemy' ability which "deals additional damage"?
Just seems a funny wording.
We played it that it is +4 damage on activate even on a miss as stated but wondering if that was as intended?
This is correct (and was unintuitive for us, as well). The updated rules on the Dungeon Fighter web site explicitly state which special abilities require hitting the target and which don't. Also, there is a general rule: If the special ability has a target symbol in it, then you must hit the target for it to activate. If there isn't a target symbol, then it activates no matter where the die ends up (when the special symbol is rolled).