The man did not trust the neighbor to look after the baby, so he stayed