Only if you have File Sharing running, and they know your user name and password.

Yes, I have File Sharing running, they DO NOT know my user name or password.

I only ask because I can share my home computer (including screen sharing), from work and visa-versa, just by clicking the icon in the Sidebar.
The work iMac has always shown up in my home Sidebar, even though my local network did not.

The only account "allowed" is myself.