Well, here's what is solidly determined:
City Owners, and those who they appoint, can determine who is allowed within the borders of their city. They can ask people to leave and/or make blanket permissions statements, and if people violate them, they have various abilities to punish individuals, upto and including forcing individuals out or calling staff.
Any Server-owned city, however, especially spawn, cannot obviously have someone completely refuse them access since it would drastically hamper them OOC.
A guard or management could put out want orders, a thieves or assassin's guild could put out a hit.
Ultimately if we were to cede control of Port Silver in a further RP sense we would most certainly have to have a very solid set of rules for it and allow all participants to enter under the same rules.
This functions a bit differently than the GoC since it's terms of Faction and Group influence within the King's city.