Discussion: View Thread

Being Certain That Your Work Is Uncertain

  • 1.  Being Certain That Your Work Is Uncertain

    Posted 08-30-2022 04:14 PM

    In my own line of work, I sometimes teach junior staff how to delineate drainage areas (rainfall subcatchments) in predominantly urban environments. Oftentimes, new staff will get caught up trying to decide something like "does the left side of this 400 foot street drain to the intersection to the north or the south of the street?" in an area that is 100 acres in size. They are usually surprised (and even a little skeptical) if my answer is "it doesn't really matter", until they better understand the purposes of what we are modeling the area for, and better appreciate the limitations of things like the rational method and how peak runoff works.  

    This is an example of trying to be realistic about the inherent accuracy limitations of the particular form of modeling that we work on. With this comes an understanding of what applications this model will work for, and what applications it will not work for.

    I would be interested in learning from others about where in their own work they can be inexact, and where, in contrast, high degrees of precision are required.

    I'll leave this thread with a quote that a colleague uses in his email signature, and another that I remember from one of my engineering textbooks:

    "all models are wrong, so honest modelers report their uncertainty first and foremost"

    "Structural Engineering is the Art of molding materials we do not wholly understand into shapes we cannot precisely analyze, so as to withstand forces we cannot really assess, in such a way that the community at large has no reason to suspect the extent of our ignorance."



    ------------------------------
    Christopher Seigel P.E., M.ASCE
    Civil Engineer
    ------------------------------


  • 2.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-05-2022 11:21 AM
    hi Christopher, I'm a teacher at a community college, and I enjoy my conversations with the students regarding uncertainties and risk, it's when I assign work that students can take on to vary the parameters to see how much design outcomes vary, it is also the time when I cite web sites (NTSB) and other forms of technical reports so students can see how important it is to strengthen soft skills (technical writing) and witness how important it is to continue to get involved with their respective COP's to continue to learn from others, and yes that watershed delineation is interesting to perform, we varied it a couple of times (the DEM) so we can see how dam break flood extents will vary, check it out,
    https://www.sciencedirect.com/science/article/abs/pii/S0309170809000815
    and thank you for your input,
    Humberto
    Professor, East Los Angeles College
    Adjunct, Cal Baptist University
    Civil Engineering


    ------------------------------
    Humberto Gallegos Ph.D., P.E., M.ASCE
    Professor
    California Baptist University
    Riverside CA
    ------------------------------



  • 3.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-26-2022 09:49 AM
    Thanks for sharing! This reminds me a little bit of my own master's research.

    ------------------------------
    Christopher Seigel P.E., M.ASCE
    Civil Engineer
    ------------------------------



  • 4.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-06-2022 07:54 AM
    Your quotes contain some truth, however one must distinguish between uncertainties and errors.  An uncertainty or inaccuracy is engrained within the analysis.  Two different analyzers using the same method should get the same answer.  If an analyzer uses a less sophisticated method or inferior data (to save time or money), that analyzer may be committing an error.  If a negative consequence results from an uncertainty, there probably would be no liability.  If a negative consequence results from an error, there may be liability.  Generally, one covers uncertainty with a factor of safety appropriate to the magnitude of uncertainty.  Hence even uncertainty must be considered in analysis.  One minimizes errors by double checking work including input and output.

    ------------------------------
    James Ward P.E., M.ASCE
    RETIRED
    Worthington OH
    ------------------------------



  • 5.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-13-2022 02:33 AM
    James. You make a good point. I note, however, that the risk and uncertainty community includes error in measurements and models as one component of total uncertainty and separates from uncertainty "mistakes" as those calculations or measurements that were made or recorded incorrectly.

    ------------------------------
    William McAnally Ph.D., P.E., D.CE, D.NE, F.ASCE
    ENGINEER
    Columbus MS
    ------------------------------



  • 6.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-13-2022 02:33 AM
    Thanks for posting, Chris.
    In preparing for expert testimony in litigation support, I find that attorneys love/hate uncertainty. My clients don't want me to mention uncertainty, but opposing attorney's pounce when I do. For example, if I say the water depth was 3.6 ft plus or minus 0.3 ft, the opposing attorney will say, "So it could have been 3.9 ft, right?" I now have a prepared answer that explains there is only a 5% probability the water was as deep or deeper than 3.9 ft. Very few attorneys want to talk more about probabilities.

    ------------------------------
    William McAnally Ph.D., P.E., D.CE, D.NE, F.ASCE
    ENGINEER
    Columbus MS
    ------------------------------



  • 7.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-19-2022 04:50 PM
    appreciate the feedback William, yes that 95% confident interval is also used in land surveying,

    ------------------------------
    Humberto Gallegos Ph.D., P.E., M.ASCE
    Professor
    California Baptist University
    Riverside CA
    ------------------------------



  • 8.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-21-2022 04:08 PM

    Have you considered developing a tornado diagram to show the relative impact of uncertainty on model inputs as a means of training and communication? For those new to tornado diagrams  - copying from Wikipedia - Tornado diagrams are useful for deterministic sensitivity analysis – comparing the relative importance of variables. For each variable/uncertainty considered, one needs estimates for what the low, base, and high outcomes would be. The sensitive variable is modeled as having an uncertain value while all other variables are held at baseline values. This allows testing the sensitivity/risk associated with one uncertainty/variable. In this way tornado diagrams can quickly show where uncertainty is important to the final outcome and where it does not matter.

    Regarding your quote on structural engineering, I think it's flawed statement but maybe I'm reading it incorrectly. I think dealing with uncertainty in a meaningful way is a defining element of the art of structural engineering. This can be through analysis, design, conservatism or combination of all three. Structural failures do occur but are rare. It's not about ignorance but knowledge, experience, and wisdom.



    ------------------------------
    Mitch Winkler P.E., M.ASCE
    Houston, TX
    ------------------------------



  • 9.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-22-2022 02:20 PM
    Hi Mitch,

    Just YouTubed "tornado diagrams", thanks for the heads up, always learning from our Community of Practice,

    ------------------------------
    Humberto Gallegos Ph.D., P.E., M.ASCE
    Professor
    California Baptist University
    Riverside CA
    ------------------------------



  • 10.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-30-2022 02:52 PM
    Thanks Christopher!

    Q. "I would be interested in learning from others about where in their own work they can be inexact,
          and where, in contrast, high degrees of precision are required."

    A. Consider forming a new habit of not waiting to first "See how it all goes" before approaching a more
    experienced colleague and stating "Would you help me with this? Before I go further, I would value your critique."

    Such behaviors can become contagious within an org that talks "Teams" but acts in degrees of "Technical Isolation."
    Cheers,
    Bill




    ------------------------------
    William M. Hayden Jr., Ph.D., P.E., CMQ/OE, F.ASCE
    Buffalo, N.Y.

    "It is never too late to be what you might have been." -- George Eliot 1819 - 1880
    ------------------------------



  • 11.  RE: Being Certain That Your Work Is Uncertain

    Posted 09-30-2022 06:06 PM
    Another dimension to the initial question is that most people are really bad at assessing uncertainty. They suffer from either over-confidence or anchoring or combination of the two. Anchoring occurs when you're presented with a value (oft out of context) but it's something to hold onto, creating the risk of confidence where confidence does not exist. The result of  over-confidence and anchoring is an uncertainty range that is much too narrow. There are techniques for overcoming these cognitive biases including deliberately asking and answering how bad could it be - and how could this come about - and how good could it be and how could this come about.  Per Bill's post, effort should be made to talk with others as well as seek out viewpoints that do not necessarily mesh with your own.

    ------------------------------
    Mitch Winkler P.E., M.ASCE
    Houston, TX
    ------------------------------