Time is nothing else than the contrary state to stillness. Hence the owner of time is anything or anybody that  introduces a change in the state of things. What does time's down mean? As a non-English native speaker I don't think I have come across this expression before.