A bid to turn a former bakery headquarters in Kirkcaldy into a gym has been refused.