If food falls on the floor, you know about the 5 second rule. But people have a 5 second rule too. Are you missing it?
When people are not confident in their bargaining position, are hoping to not be discovered, lying, or any other situation which causes stress, their bodies crave to release the mounting pressure resulting from their circumstances. They will often release that stress in 0-5 seconds of the stimulus and it is seen in the body language.
You may see an uncomfortable movement such as adjusting clothing, looking away or closing eyes longer than a blink, hard swallow, lip licking, touching the face or neck, and wringing hands (note: this is not an exhaustive list) after they finish answering a tough question. This gives you an idea if you need to ask more targeted questions. But, if you are interviewing someone, you may miss this reaction if you start writing things down or ask the next question before they have released the discomfort.