No need to remove anything. If you are guilty of doing it then you should own up to it when asked.
You're no fun and are completely ignoring the INGO "screw the police and trespassing laws unless it's on my property" ethos.
I would not worry. If they are building they most likely have construction vehicles in and out anyway. I am assuming you were just getting a little mud on your tires and not destroying property. I would not worry a bit about it.
What is the worst that can happen with a simple trespass charge anyway???