Moray Leisure Centre is facing closure amid fears that running costs are “spiralling out of control” at a time when visitor numbers are dropping.