The art of building software solutions is comprised of many moving parts. There are project managers coordinating tasks; business analysts gathering requirements; architects establishing standards and designs; developers assembling processes; DBAs building database structures; quality assurance staff testing components for compliance to requirements; and an array of supporting roles providing for functional environments, infrastructure, security, etc. A common task that everyone must perform is estimating the effort necessary to deliver results. Certainly for simple and/or repetitive tasks there is no need for recurrent estimating, since applicable values are based on past known metrics. For example, creating the third or fourth version of the same database within the same environment should allow a DBA to incorporate costs experienced previously as a guide. And unless something unusual occurs, such estimates should be on target. However for creative tasks, such as designing new structures, or building new kinds of processes, there will be no previous documented events to refer to. Faced with these circumstances, individuals usually are not allowed to shrug and say, "it will take as long as it takes," and be left alone.
Although estimating is a skill, it is a skill that everyone must develop, and communicating estimates can be challenging and wrought with confusion. There is a difference between the effort needed to complete a task, versus the possible elapsed time involved in finishing a task. This difference can be a source of confusion. For example, tasks requiring more than 8 working hours will generally span more than one work day. Also, if 1 hour's effort necessitates another 10 hours waiting for results between efforts, then even an "8 hour" endeavor may take more than one work week to complete. Another area of confusion relates to scope. Managers often ask an individual for an estimate; and while that individual's piece is the most critical component, presuming that single critical piece is all that needs to be done leads to surprises later on. It is not unusual for developers to estimate how long it takes to code something and provide a number, helping everyone ignore that testing, rework, and eventual deployment tasks go "on top" of the numbers provided. Estimate providers and estimate receivers should be in agreement regarding assumptions on details related to the interactions of work efforts as opposed to elapsed times, along with any other caveats. Discussion of details is paramount for success in estimating efforts.
Accuracy in estimating is a function of experience. Individuals comfortable with both their duties and their environment will generally provide accurate estimates. When estimates from individuals are consistently inaccurate, something is interfering. Those who constantly overestimate tasks are generally afraid to go over an estimated amount. Estimates are viewed as the maximum amount of time needed to accomplish something. These individuals likely believe that if they happen to come in under the estimate then they have done well. Sadly, those receiving the consistently high estimates learn not to believe in any estimates provided. Folks who consistently underestimate often do so either because they are over-optimistic regarding their abilities, or they are attempting to please people by providing a number they believe the receiver wishes to hear, rather than a real value. To overcome discrepancies in estimating one needs a level of self-understanding that in some instances may be very hard to face. Project managers are generally in the middle of it all. They have the onerous task of trying to understand all these factors, and then juggle everything together to lay out a plan capable of tracking a team's progress through to completion of a project. It is a challenge, a struggle, and when done well a small act of the heroic.