{"nbformat":4,"nbformat_minor":0,"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.4"},"colab":{"provenance":[]}},"cells":[{"cell_type":"markdown","metadata":{"id":"PeYzCnSsD0Aw"},"source":["# Tutorial 1: Uncertainty\n","Developed by Rebeckah Fussell for Cornell Physics Labs. Adapted by Cristina Schlesier for Spring 2025."]},{"cell_type":"markdown","source":["## Introduction\n","\n","Experimental physics is all about the active creation of scientific knowledge. If you’ve studied physics before, you’ve probably mostly learned about existing scientific knowledge. You’ve studied the models and frameworks that explain the universe reasonably well, e.g. Newton’s Law of Gravitation. You’ve practiced putting these models to use with mathematics. But where do the models come from? Often we are taught about the creation of scientific knowledge as if it comes from thin air. Some genius had an idea in the bathtub and wrote it down and that's the end of that.\n","\n","These myths about the creation of knowledge obscure the fundamental essence of science as we understand it today. In science, we conspire with the physical world to create knowledge. We design experiments and make measurements that allow the physical world to communicate its true nature to us. Science is an active, iterative, never-ending process of asking questions and developing new ways of understanding the physical world. This process is mediated by experiment and measurement. At its core, science is about measurement.\n","\n","This tutorial is concerned with a crucial aspect of measurement known as **uncertainty**. (You may also see the less precise term 'error' used interchangeably [but we'd like to avoid this](https://m.xkcd.com/2440/). The word 'error' implies a mistake, something that should be fixed and avoided. Uncertainty, on the other hand, is a fundamental, necessary, and generally unavoidable component of measurement.).\n","\n","Say you'd like to measure the period of a pendulum. You use a stopwatch with high instrumental precision (many decimal places) and you are confident in your reaction time. You might get a measurement of 2.500 s. How confident are you in that measurement? Perhaps you try again just to check. This time you measure 2.721 s. What value do you quote as the best estimate for the period of the pendulum? And how confident will you be in that estimate?\n","\n","A measurement is only as good as its uncertainty. If I tell you I've measured the acceleration due to gravity to be 9.610 $m/s^2$, it is possible that I've made a world-shattering discovery. After all, the average value measured by professional physicists is 9.8067 $m/s^2$. These two values are clearly distinct. Or are they?\n","\n","Perhaps my experiment wasn't very precise and the uncertainty on my measurement is $\\pm$ 0.2 $m/s^2$, in which case these values *are* consistent. But what if my uncertainty was $\\pm$ 0.002 $m/s^2$?"],"metadata":{"id":"Ab6WWFK1H1Ih"}},{"cell_type":"markdown","source":["<a name=\"statsys\"></a>\n","### Statistical and Systematic Uncertainties\n","\n","There are two fundamentally different sorts of uncertainties associated with any measurement procedure, namely **statistical** (or random) and **systematic** uncertainties. Statistical uncertainties come from random fluctuations that occur during the measurement process. The example of the pendulum period given above illustrates the presence of statistical uncertainties in the measurement. I may measure the period many times and always get slightly different values.\n","\n","On the other hand, systematic uncertainties arise from all sorts of sources and basically entail everything that is not a random, statistical fluctutation.\n","\n","When we make a series of repeated measurements, statistical uncertainties cause us to measure a spread of values around the \"true\"[$^1$](#Footnotes) value. In contrast, systematic uncertainties may cause the measurement to be offset from the \"true\" value, even if the individual measurements are close to each other.\n","\n","While we will develop tools for understanding and mitigating statistical uncertainty in this tutorial, we will leave the discussion of systematic uncertainty for another time. In general there are no simple prescriptions for eliminating systematic uncertainties. To a large extent this task requires experience and in-depth knowledge of the experimental apparatus."],"metadata":{"id":"XCM1kiAGJLSF"}},{"cell_type":"markdown","source":["### Distributions\n","\n","Although measurements come with the possibility of statistical uncertainty, we still expect our measurements of a single quantity to cluster around a single value[$^2$](#Footnotes). The presence of random uncertainty means that any value is possible in principle, but that different values may have different  probabilities of occuring. A **Gaussian** (or normal) **distribution** illustrates this behavior perfectly. It is often safe to assume our measured values will follow a Gaussian distribution.\n","\n","A nice way of visualing this behavior is by plotting our measurements in a **histogram**. A histogram tells us the frequency of a value in our dataset. If I measure the period of the pendulum 10 times, my stopwatch may provide me with a set of values $T$ = (1.62, 1.76, 1.80, 1.53, 2.0, 1.7, 1.75, 1.81, 1.88, 1.67) in seconds. Run the code below to load the histogram of this data."],"metadata":{"id":"tWHyy-LhOMun"}},{"cell_type":"code","source":["import numpy as np\n","import matplotlib.pyplot as plt\n","\n","period = [1.62, 1.76, 1.80, 1.53, 2.0, 1.7, 1.75, 1.81, 1.88, 1.67]\n","\n","fig, ax = plt.subplots(1)\n","ax.hist(period)\n","plt.xlabel(\"Period T (in s)\")\n","plt.ylabel(\"Number of measurements\")"],"metadata":{"id":"oLtdBM9RSMNu"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["With this histogram, we get a general sense of where the average value may lie and how spread out our measurements are -- which is tied to how confident we are in our estimate of the quantity. Running the code multiple times should also drive home the point that no single measurement would be sufficient to perfectly extract the pendulum's period from our experiment."],"metadata":{"id":"WsfrG-HcTCLX"}},{"cell_type":"markdown","metadata":{"id":"Of6rUvBYD0A9"},"source":["## Determining (statistical) uncertainty\n","\n","Now that we've motivated the idea of uncertainty, we can finally begin to explore the statistical tools to help us understand repeated measurements. We will continue to use the example of measuing the period of a pendulum.\n","\n",">**Important note**: the code cells throughout this tutorial will build off of previous code cells. It is important to **run every code cell (SHIFT + ENTER) in the document up to the point where you are working** every time you open this tutorial. If you get an error message (particularly one that says that a particular variable is not defined) after attempting to run a code cell, first make sure that you have run every previous code cell.\n","If you see a warning like 'Warning: This notebook was not authored by Google', just click 'Run anyway'."]},{"cell_type":"markdown","metadata":{"id":"KQMxKH9gD0BF"},"source":["The code below generates a histogram showing the results of 25 trials measuring the period $T$ of a pendulum. The precision of the measuring instrument, also known as *instrumental precision*, was $\\pm$ 0.005 s because the instrument (a stopwatch) displayed values to the nearest 0.01 s. We will estimate instrumental precision as half of the last known digit."]},{"cell_type":"markdown","source":["### Q1:\n","\n","Run the code cell below and observe the histogram. Do not worry about the details of the code inside the <span style= 'font-family:Courier'>pendulum_hist()</span> function. Run the cell a few more times so that different histograms are generated.\n"],"metadata":{"id":"MYfhYX3aiU3p"}},{"cell_type":"markdown","source":["Hint: If you see an error that np is not defined, make sure you imported the packages above. It is good practice to make sure that you have run every code cell up to the point in the document you are working in every time you open the tutorial."],"metadata":{"id":"zhbebG4iicX6"}},{"cell_type":"code","metadata":{"id":"cjVBtoFID0BG"},"source":["def pendulum_hist(N):\n","    # Generate a normal distribution, center at x=0 and y=5\n","    mean = 1.5\n","    sd = 0.1\n","    data = np.random.normal(mean, sd, N)\n","\n","    #Create a histogram of [this] *these* data\n","    fig, ax = plt.subplots(1)\n","    ax.hist(data,label = \"hi\");\n","    plt.xlim(xmin=1.15, xmax = 1.85)\n","    plt.title(r\"Pendulum Period Measurements ($\\pm$ 0.005 s)\")\n","    ax.errorbar(x=[mean+sd, mean -sd,mean], y = [N/14, N/14, N/14], yerr = [N/200, N/200, N/200], label = r\"$\\sigma$\")\n","    ax.errorbar(x=[mean+sd/np.sqrt(N), mean -sd/np.sqrt(N),mean], y = [N/7, N/7, N/7], yerr = [N/200, N/200, N/200], label = r\"$\\delta$\")\n","    plt.xlabel(\"Period T (in s)\")\n","    plt.ylabel(\"Number of measurements\")\n","    plt.text(1.46, N/8, r\"$\\delta \\bar{x}$ (standard uncertainty of the mean)\")\n","    plt.text(1.46, N/17, r\"$\\sigma$ (standard deviation)\")\n","    plt.show()\n","    return data\n","\n","data=pendulum_hist(25);"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["**Q.** How does the histogram change each time you generate a new one? What stays the same?\n","\n","**[Your answer here -- keep your responses in boldface so we can easily distinguish your work from the main text of the tutorial]**\n","\n","---"],"metadata":{"id":"nk7VvpIztePD"}},{"cell_type":"markdown","metadata":{"id":"5mJThArFD0BH"},"source":["When we take repeated measurements, it is useful to calculate the mean $\\bar{x}$ of all of the measurements, as a measure of the center of our collection of measurements. The mean is defined as:\n","\\begin{equation}\n","\\bar{x} = \\frac{1}{N} \\sum_{i = 1}^{N} x_i\n","\\end{equation}\n","\n","where $N$ is the number of measurements and *$x_i$* is the $i^{th}$ value of the measurement $x$. In Python, we can calculate the mean using a built-in function in the *numpy* package (nicknamed *np* here). Run the code cell below to try it on the data from our histogram above."]},{"cell_type":"code","metadata":{"id":"YL1VRcgTD0BI"},"source":["# an example of calculating mean\n","m = np.mean(data)\n","print('the mean period of the pendulum is ', m, 'seconds')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"CwzpP21GD0BK"},"source":["When we take the same measurement over and over, we will often find some natural variation in the data. If the variation is a result of statistical uncertainty, we can characterize the width of statistical variation with the **standard deviation**, a statistical measure of the uncertainty in a single measurement. The standard deviation characterizes the spread in the distribution of individual values due to statistical uncertainty in the measurement process. In the histogram above, the standard deviation is labeled with $\\sigma$.\n","\n","Some of the desired properties of standard deviation are:\n","\n","1. We want to incorporate all available points into the calculation.\n","2. The standard deviation should stay approximately the the same as we add more points with the same level of variation.\n","3. Standard deviation should increase as we measure more points further away from the mean.\n","4. The units of standard deviation should be the same as the units of the measurement.\n","5. The standard deviation should treat values on either side of the mean the same.\n","\n","\n","Standard deviation is calculated with the following formula:\n","\n","$\\sigma = \\sqrt{\\frac{1}{N-1} \\sum_{i = 1}^{N} (x_i - \\bar{x})^2}$\n","\n","where $N$ is the number of measurements[$^3$](#Footnotes), *$x_i$* is the $i^{th}$ value of the measurement $x$, and $\\bar{x}$ is the mean.\n","\n"]},{"cell_type":"markdown","metadata":{"id":"SSzCGAN6D0BM"},"source":["<a name=\"Q2\"></a>\n","### Q2:\n","\n","**Q.** Explain how the formula for $\\sigma$ takes into account the five properties above."]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"NxhMfaQcDRMe"}},{"cell_type":"markdown","source":["Run the code cells below in sequence to define and use a function for standard deviation on your data above. Remember, you must define a function before you can use it."],"metadata":{"id":"LKsK6eXcDRjK"}},{"cell_type":"code","metadata":{"id":"kpOIGM8ID0BN"},"source":["#define a function that calculate the standard deviation\n","def standard_deviation(data):\n","    N = len(data)\n","    return np.sqrt(np.sum((data - np.mean(data))**2)/(N-1))"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"wcLrV0L0D0BO"},"source":["# calculate standard deviation of the data displayed in the histogram above\n","print('the standard deviation of the period of the pendulum measurement is ', standard_deviation(data), 's')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"qk2rG0NvD0BP"},"source":["### Q3:\n","\n","**Q.** Look back at the histogram and the formula for standard deviation. Why can the standard deviation $\\sigma$ also be viewed as an estimate of the uncertainty in any single measured value? *Hint: consider the averaging behavior of the equation for standard deviation.*\n","\n"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"3gc6Vbcn43mW"}},{"cell_type":"markdown","metadata":{"id":"HLSC7lycD0BQ"},"source":["### Q4:\n","\n","**Q.** If we added 10 more measurements to our data set, the standard deviation $\\sigma$ would not necessarily get larger. Why not?\n"]},{"cell_type":"markdown","source":["\n","**[Your answer here]**\n","\n","---"],"metadata":{"id":"ygNxh58FDk1T"}},{"cell_type":"markdown","metadata":{"id":"xpbhnvqQD0BQ"},"source":["Although we calculate standard deviation from our measurements, we use it as a representation of the physical phenomenon. After taking repeated measurements of the same phenomenon, we expect that around 68% of our measurements will fall within one standard deviation of the mean, if the measurements are approximately *normally* distributed (or distributed according to a *Gaussian* distribution).\n","\n","A single measurement is informative, but we gain much more information from the mean of a set of multiple measurements. The uncertainty in the mean of multiple measurements, therefore, should have a smaller uncertainty than that of a single measurement (i.e. the standard deviation). We define the uncertainty in the mean as:\n","\n","\\begin{equation}\n","\\delta \\bar{x} = \\frac{\\sigma}{\\sqrt{N}}\n","\\end{equation}\n","\n","where $\\sigma$ is the standard deviation and $N$ is the number of measurements.\n","\n","This definition serves to reward you for taking more data, so that the uncertainty in the mean of a collection of many measurements is smaller than the uncertainty in any single measurement.\n","\n","Note: Read the symbol $\\delta$ as \"uncertainty in\". As discussed above, $\\bar{x}$ is the symbol for the mean, so $\\delta \\bar{x}$ should be read as the mathematical notation for \"uncertainty in the mean\". It does *not* mean the \"uncertainty times the mean\".\n"]},{"cell_type":"markdown","metadata":{"id":"JHWCokaQD0BR"},"source":["### Q5:\n","\n","**Q.** Say you have taken *m* measurements (where *m* is some arbitrary number). If you do one hundred times as many measurements (i.e., $100m$), how much smaller is the uncertainty in the mean compared to when you had only had *m* measurements?\n","\n"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"8fZiXEtK82Rd"}},{"cell_type":"markdown","metadata":{"id":"qW9B37QJD0BR"},"source":["The standard uncertainty of the mean is shown in green in the histogram for Q1. The uncertainty itself is just an estimate, so we usually round it to one or at most two digits.\n","\n","Below we define a function for calculating the uncertainty in the mean.\n","\n","Run the code cells below in sequence to calculate the standard uncertainty of the mean of our histogram data."]},{"cell_type":"code","metadata":{"id":"7onUn9eeD0BR"},"source":["def standard_unc_of_mean(data):\n","    N = len(data)\n","    return standard_deviation(data) / np.sqrt(N)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"lexg8EVbD0BV"},"source":["# an example of calculating standard uncertainty of the mean\n","print('the standard uncertainty in the mean is',standard_unc_of_mean(data), 's')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["### Q6:\n","\n","**Q.** Is this measurement of the uncertainty larger or smaller than the standard deviation? Explain whether or not that makes sense."],"metadata":{"id":"ZVAjZXqYjdkt"}},{"cell_type":"markdown","source":["\n","**[Your answer here]**\n","\n","---"],"metadata":{"id":"DU2LoB_oEM2C"}},{"cell_type":"markdown","metadata":{"id":"FXT1WUyiD0BW"},"source":["Let's go back to generating histograms of the results of 25 pendulum measurement trials.\n","\n","### Q7:\n","Run the cell below and notice that the mean is printed out below the histogram. Copy this mean into the array <span style='font-family:Courier'>all_means</span> in the second code cell. Generate at least nine more datasets by re-running the first code cell, and store each mean as a new entry in <span style='font-family:Courier'>all_means</span>, with each mean separated by a comma.\n"]},{"cell_type":"code","metadata":{"id":"bOfNgn9xD0BW"},"source":["hist_data = pendulum_hist(25)\n","print(\"mean is \" + str(\"%.4f\" %np.mean(hist_data)) + \" s\") #for the curious,\"%.4f\" shortens the number of digits, so you don't have to copy and paste as much"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"d3pHx8Y-D0BX"},"source":["all_means = np.array([copy,your,means,here,...,...]) #at least 10 means for 10 different histograms\n","\n","print(\"standard deviation of our means is \" + str(\"%.4f\" % standard_deviation(all_means)) + \" s\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"NTlUsIT8D0BY"},"source":["### Q8:\n","\n","**Q.** How does the standard deviation of our list of means compare to the standard uncertainty of the mean calculated in Q6 (round each to 1 significant digit), and why does that make sense?\n"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"0J5eaTOH4-AF"}},{"cell_type":"markdown","metadata":{"id":"Mgv5AgwSD0BY"},"source":["Because the uncertainty of the mean $\\delta \\bar{x}$ is actually less than the uncertainty in each individual measured value $\\sigma$ by a factor $1/\\sqrt{N}$, we can reduce the uncertainty in our best estimate of a measured quantity by taking more and more repeated measurements.\n","\n","Run the code cell below, which will give you a histogram for 900 repeated trials measuring the period T of our same pendulum."]},{"cell_type":"code","metadata":{"id":"hjkz958mD0BZ"},"source":["big_data = pendulum_hist(900);\n","print(\"mean is \" + str(\"%.4f\" % np.mean(big_data)) + \" s\")\n","print(\"standard deviation is \" + str(\"%.4f\" % standard_deviation(big_data)) + \" s\")\n","print(\"standard uncertainty of the mean is \" + str(\"%.4f\" %standard_unc_of_mean(big_data)) + \" s\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["### Q9:\n","\n","**Q.** How do the mean, standard deviation, and standard uncertainty of the mean of this data set compare to the data set with 25 repeated trials? Explain in your own words why these results make sense."],"metadata":{"id":"3gpgE__djoQ1"}},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"yr9n5wgN9AIK"}},{"cell_type":"markdown","metadata":{"id":"bmBGb764D0Bb"},"source":["In comparing this histogram with the original histogram for 25 trials, three main features are apparent:\n","\n","1. The mean appears to be roughly the same as it was for 25 trials.\n","\n","2. The standard deviation $\\sigma$ appears to be roughly the same as it was for 25 trials. The additional trials have simply \"filled out\" the distribution more smoothly so that the histogram has a more consistent shape.\n","\n","3. The uncertainty of the mean $\\delta \\bar{x}$ is noticeably less than it was for 25 trials.\n","\n","Recall also that the instrumental precision of these measurements was limited to $\\pm 0.005$ s by the digital display of the stopwatch."]},{"cell_type":"markdown","metadata":{"id":"zySSjJM0D0Bc"},"source":["### Q10:\n","\n","**Q.** How does the uncertainty of the mean of our data set with 900 measurements compare with the instrumental precision of the measuring instrument?\n","\n"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"RrfM2LCp9Fvq"}},{"cell_type":"markdown","metadata":{"id":"IPnjZ95kD0Bc"},"source":["### Q11:\n","**Q.** What does this say about the possibility of \"beating instrumental precision\" by taking repeated measurements?\n","\n"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"71X5UFvS9GrC"}},{"cell_type":"markdown","metadata":{"id":"4vOzYHblD0Bd"},"source":["### Q12:\n","**Q.** Write the best estimate for the pendulum's period T from these 900 trials in the form $T = \\bar{T} \\pm \\delta$. Use the template given below.\n","\n"]},{"cell_type":"markdown","source":["*Double click this cell to edit it!*\n","\n","$T = [InsertValueHere]  \\pm [InsertValueHere]$."],"metadata":{"id":"ZsARrISMklAr"}},{"cell_type":"markdown","metadata":{"id":"-xqxMy9iD0Be"},"source":["### Q13:\n","\n","\n","**Q.** If the standard deviation for repeated measurements of the period of this pendulum is around 0.1 s, roughly how many repeated measurements of the pendulum period would be needed to reduce the uncertainty of the mean to $1/2$ of the instrumental precision of 0.005 s (i.e., as small as 0.0025 s)?"]},{"cell_type":"markdown","source":["**[Your answer here]**\n","\n","---"],"metadata":{"id":"v3tV7kEK9Jlv"}},{"cell_type":"markdown","source":["Submit a PDF of this document, with all code cells run and displaying output, to the Gradescope assignment."],"metadata":{"id":"rCRT-w8buiYM"}},{"cell_type":"markdown","source":["<a name=\"Footnotes\"></a>\n","## Footnotes\n","**1** We use the word \"true\" with some caution here. The idea that there exists a true value or that we can ever know it is debatable. As it stands, when we perform our experiment we certainly do not know the so-called true value. Otherwise we wouldn't bother with the measurement at all!\n","This means we must determine our confidence in the measurement entirely from the data and experimental conditions — not by comparison to any *seemingly-true-but-actually-not-true* value, such as a theoretical value or previous measurement. [back up](#statsys)\n","\n","**2** This is sufficiently true for most simple, classically-modeled phenomena. It may not be true for more complex systems and quantum mechanically-modeled phenomena. [back up](#statsys)\n","\n","**3** Don't worry too much about the fact that we subtract 1 from $N$ in the denominator. We can think of it as making sure we can't calculate the standard deviation of one measurement. It really comes from using one degree of freedom in our data to calculate the mean, which we need for the standard deviation. [back up](#Q2)\n","\n","## References\n","\n","A Practical Guide To Data Analysis for Physical Science Students by Louis Lyons"],"metadata":{"id":"7W4fAV9fcMo7"}}]}