Again, that's assuming you're discussing laborers and not engineers and such, a-la Da Vinci. No mere fool could oversee the quality construction of a recruitment center or an entire city, or even a proper length of road. Very few had such technical knowledge in the medieval era, the common laborers all catering to the designs of the geniuses who drew up the plans and orchestrated the construction of castles, cities, roads, and walls.
I think engineer units would be quite useful. Anyway, labourer just sounds... Peasant.