Google has revealed they are looking at users behavior after clicking a link in Search Results. From this, we know Google watches to see if the user returns to the search results to find another result. If that happens, this tells Google that the question was not answered by the site they last visited. Similarly, if a user visits a site from Search Results, but spends very little time on that site then the users query was not resolved. These all serve as queues to Google that the site a user visited may be of lower quality. The same is true for page load speed.
Google is factoring this into determining what results to show a user. So those sites that fail to answer the question, have a confusing or dated user experience, or are extremely slow to load will see a very high bounce rate (rate at which users leave a website after just one viewed page). Google is taking note, and so should you.
