A process server is trying to serve me, but due to my job, I know what he knows.?

Can I change the numbers on my house to deter his efforts? The postman knows us so it won't confse him if I do this, but I don't want to get in trouble