I found a bug!
So, since I have a scout, I can explore the territory where my enemy are. When their laborers are about to construct some house and connecting road to it, I do next. I kill the laborer that is waiting for Serf to bring him stone. On place where Laborer was killed, only dirt that he previously dig is left. After that, no other laborer will continue his work and so that house wont have a connection and serfs will not bring resources needed for constructing that house...
Stupid AI...