Exactly. Set goals with regular milestones. If a remote employee is hitting targets, accomplishing goals, with the necessary level of quality, why should a manager care where or how that work was performed?
If someone isn’t consistently meeting the agreed upon goals, then you have a performance issue to deal with. Don’t confound the issue with where the work was performed.
Being onsite doesn’t guarantee great performance any more than being remote risks poor performance. I’ve witnessed way too many onsite employees wasting time surfing from cube to cube, interrupting productive work to shoot the breeze. Too many managers dropping into meetings and ruining momentum. Too many people spending hours every day in useless activities on campus.