{"nbformat":4,"nbformat_minor":0,"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.9"},"colab":{"provenance":[]}},"cells":[{"cell_type":"markdown","metadata":{"id":"eb3EX-bY440V"},"source":["## Tutorial 3: Distinguishability\n","Developed by Rebeckah Fussell for Cornell Physics Labs. Adapted by Cristina Schlesier for Spring 2025."]},{"cell_type":"markdown","metadata":{"id":"alXDst1Q440o"},"source":["In this tutorial we will continue to apply the concepts of mean, standard deviation, and standard uncertainty of the mean. We will also learn about how to compare the means of different data sets and test for distinguishability.\n","\n","Before we get started, let's load some packages and define the functions for standard deviation and standard uncertainty of the mean. These functions are identical to the ones in the previous tutorial.\n","\n","Make sure to run the cell below before proceeding (SHIFT + ENTER). As always, remember code cells build on eachother, so run each in order."]},{"cell_type":"code","metadata":{"id":"ZmxQFeoV440v"},"source":["import numpy as np\n","\n","%matplotlib notebook\n","%matplotlib inline\n","import matplotlib\n","#matplotlib.use('module://ipympl.backend_nbagg')\n","\n","import matplotlib.pyplot as plt\n","\n","#%matplotlib widget\n","from ipywidgets import *\n","\n","def standard_deviation(data):\n","    N = len(data)\n","    return np.sqrt(np.sum((data - np.mean(data))**2)/(N-1))\n","\n","def standard_unc_of_mean(data):\n","    N = len(data)\n","    return standard_deviation(data) / np.sqrt(N)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Jto5uB_6440w"},"source":["In the uncertainty tutorial we talked about taking repeated measurements of a phenomenon, and that we expect to see a distribution of values around the mean.\n","\n","Often we need to determine if the means of two data sets of measurements are coming from the same underlying phenomenon. It is possible that we are measuring two different phenomena.  It is also possible we are measuring the same phenomena. However, even if we are measuring the same phenomena when we take two different data sets, we would still expect to see some difference in the means of two different data sets.\n","\n","For example, if we take several measurements of the period of a pendulum when released from 10 degrees, and several more measurements of the period at 20 degrees, we need a method to know if the average period is the same (and therefore we are measuring the same physical phenomenon) or different when released from the two angles. We are going to call this **distinguishibility**.  \n","\n","This method should:\n","\n","1. Take into account the difference between the means of the two data sets, such that a larger difference indicates the values are more distinguishable.\n","2. Take into account the uncertainty of the means of the two data sets, such that the difference in the means is balanced by how well we know the measurements.\n","3. Not depend on units so we can interpret the value without context.\n","\n","\n","To make comparisons between two measurements with uncertainties $A \\pm \\delta A$ and $B \\pm \\delta B$, we use a quantity known as $t'$. This is defined as:\n","\n","\\begin{equation}\n","t' = \\frac{|A-B|}{\\sqrt{\\delta A^2 + \\delta B^2}}\n","\\end{equation}\n","\n","This ratio tells you how different two measurements are as a fraction of the uncertainty in the measurements. You may have heard particle physicists refer to 3-sigma or [5-sigma](https://home.cern/resources/faqs/five-sigma) effects: *sigma* here refers to the uncertainty, so a 3-sigma effect corresponds to a $t'=3$.\n","\n","For any experiment, $A$ and $B$ might be the means for two sets of repeated measurements (such as our means of repeated measurements of the period of pendula released from 10 and 20 degrees), or they might be individual measurements (such as the counts registered in a detector at two different elevations). $\\delta$ should be read as \"uncertainty in\", so $\\delta A$ and $\\delta B$ are the uncertainties in the measurements A and B (*not* the uncertainty times the measurements).\n"]},{"cell_type":"markdown","metadata":{"id":"oBRUIvKP440x"},"source":["**Q.** Explain how the formula for $t^{\\prime}$ takes into account the three properties above.\n"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","1.\n","\n","2.\n","\n","3.\n","\n","---"],"metadata":{"id":"2PJOar1mxPdg"}},{"cell_type":"markdown","metadata":{"id":"nGKIRE1L440y"},"source":["**Q.**  a) Say we measure A = 2 ± 1 s and B = 8 ± 1 s. Intuitively, would you say that A and B are distinguishable or indistinguishable? b) What about if the uncertainties were larger: are A= 2 ± 5 s and B= 8 ± 5 s distinguishable?\n","\n"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","a)\n","\n","b)\n","\n","---\n"],"metadata":{"id":"q1tHCNk-xgrk"}},{"cell_type":"markdown","source":["Run the code below to calculate the t' value for these two sets of measurements.  "],"metadata":{"id":"7sYJwvpOxb60"}},{"cell_type":"code","metadata":{"id":"hXnzajAX440y"},"source":["def t_prime(A, dA, B, dB):\n","    return abs(A - B) / np.sqrt(dA**2 + dB**2)\n","\n","\n","# an example of using t_prime\n","print('t-prime for a) measurements:',t_prime(2,1,8,1))\n","\n","# second example\n","print('t-prime for b) measurements:',t_prime(2,5,8,5))"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"sTPfSY6-4402"},"source":["## **Exploring t'**\n","You can think of $t^\\prime$ as telling us how far apart the means of the distributions are in \"units\" of uncertainty.  The larger the t' value, the less likely it is that the two sets of measurements came from the same distribution. Below, we have two sets of measurements (one in blue and one in orange), with properties you can control in the code cell below.  They have a relative mean (m), a number of measurements (N) and each a standard deviation.  \n","\n","Run the code below, and see what happens for different values of m, N, $\\sigma_1$, and $\\sigma_2$."]},{"cell_type":"code","metadata":{"scrolled":false,"id":"7Rv-qXHs4402"},"source":["x=np.linspace(-5, 10, 1000)\n","\n","# **** Adjust these parameters to explore how they impact t'. *********** #\n","# **** The figure has a fixed x-range of -2,2 so keep the means ********* #\n","# **** within those bounds. ********************************************* #\n","m=0.25      # Mean of the orange data relative to the mean of the blue data\n","N=20        # Number of measurements in each dataset\n","stdev1=.5   # Standard deviation of the blue dataset\n","stdev2=.5   # Standard deviation of the orange dataset\n","# *********************************************************************** #\n","\n","fig, ax=plt.subplots(1,1, figsize=(5,5))\n","ax.cla()\n","x=np.linspace(-2, 2, 1000)\n","ax.fill_between(x,np.e**(-x**2/stdev1**2),alpha=.5)\n","ax.fill_between(x,np.e**(-(x-m)**2/stdev2**2), alpha=.5)\n","ax.errorbar(0, 1, xerr=stdev1/np.sqrt(N), capsize=5)\n","ax.errorbar(m, 1, xerr=stdev2/np.sqrt(N), capsize=5)\n","ax.set_title(\"t'={}\".format(round(np.sqrt(N)*m/np.sqrt(stdev1**2+stdev2**2), 3)))\n","fig.canvas.draw_idle()\n","fig.canvas.draw_idle()\n","plt.tight_layout()\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"v9NI9EDq4403"},"source":["**Q.** a) What combination of factors causes $t^\\prime$ to increase?  b) What combination of factors causes it to decrease?  c) Does the difference in the mean have to change for $t^\\prime$ to change?  \n"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","a)\n","\n","b)\n","\n","c)\n","\n","---"],"metadata":{"id":"qM5CgSRW1oz4"}},{"cell_type":"markdown","metadata":{"id":"JVukCEy-4404"},"source":["## **Interpreting t' values**\n","\n","If we have two sets of measurements of the same physical phenomenon, we would expect to see a $t^\\prime$ value of approximately 1 on average. This is because, on average, most (68\\%) of our measurements should be within one standard deviation of the mean[$^1$](#Footnotes)<a name=\"F1\"></a> and so any two measurements should almost always be approximately one standard deviation away from each other.\n","\n","After calculating $t'$ for two measurements, you can evaluate their disimilarity (or distinguishability) through the following interpretation:\n","\n","\n","1.  $t'\\approx1$  If we have two sets of measurements and a t' value of approximately 1, then the sets are indistinguishible and they **may** represent the same physical phenomenon.   \n","\n","2.  $t'<<1$  If we have a t' value of much less than 1, then it is possible that either we overestimated our uncertainties, or that our current level of precision is not good enough for the phenomenon that we are trying to measure.  \n","\n","3.  $1\\lesssim t'<3$  This is a grey area.  It is still possible that our two sets of measurements are coming from the same phenomenon, but it is less likely than if our t' is somewhere close to 1.\n","\n","4.  $t' >3$ If our t' is greater than 3, then it is unlikely that our two sets of measurements were measuring the same phenomena.  This means that we have distinguished between two sets of physical phenomena.  \n","\n","\n","NOTE: $|t'| \\le 1$ **does not** mean that A and B are the same. It only tells us that the given data cannot allow us to distinguish between the two sets. For example, if you do a better measurement and decrease the uncertainties, you might later uncover a difference between A and B. That is, poor precision may be hiding a subtle difference!\n","\n","In all cases, your t' value will give you **evidence** of one interpretation over another, but it **does not \"prove\"** an interpretation[$^2$](#Footnotes).<a name=\"F2\"></a> Further testing is almost always needed to continue building evidence in favor of whatever conclusions you may be lead to draw."]},{"cell_type":"markdown","metadata":{"id":"OrB7gki84405"},"source":["**Q.** Based on these interpretations and your exploration with the blue and orange data above, what do you think we should do next in each of these 4 scenarios?\n","\n","1.  $t'\\approx1$  \n","\n","2.  $t'<<1$\n","\n","3.  $1\\lesssim t'<3$\n","\n","4.  $t' >3$"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","\n","\n","1.    \n","2.    \n","3.\n","4.\n","\n","---\n"],"metadata":{"id":"LquhrLbl2CsV"}},{"cell_type":"markdown","metadata":{"id":"rBNJaSRW4405"},"source":["## **Pendulum Example**\n","\n","Let's try an example problem. An experimenter measured the period of a pendulum at 10 degrees and 20 degrees using a simple stopwatch. They measured the time for the pendulumn to swing for one single period and conducted 14 trials for each angle. The data they collected is given in the first lines of code below. The code then generates a histogram so we can visualize the distribution of our data.\n","\n","Run the code cell below."]},{"cell_type":"code","metadata":{"id":"iRL_imIe4406"},"source":["#enter pendulum period data\n","plt.figure()\n","ten_degs = np.array([1.23, 1.36, 1.35, 1.36, 1.30, 1.27, 1.30, 1.32, 1.26, 1.26, 1.38, 1.29, 1.29, 1.32])\n","twenty_degs = np.array([1.37, 1.35, 1.38, 1.27, 1.33, 1.33, 1.26, 1.36, 1.36, 1.27, 1.29, 1.44, 1.29, 1.32])\n","\n","#plot distributions\n","plt.hist(ten_degs,bins = 7)\n","plt.title(\"Histogram of pendulum period at 10 degrees\")\n","plt.ylabel('Counts')\n","plt.xlabel('Period s')\n","\n","plt.figure();\n","plt.hist(twenty_degs,bins = 7)\n","plt.title(\"Histogram of pendulum period at 20 degrees\")\n","plt.ylabel('Counts')\n","plt.xlabel('Period s')\n","plt.show()"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"_6VsOtUp4407"},"source":["**Q.** Calculate standard deviation of the 10 degree data set.\n","*Hint: replace the dots with the name of the ten degree data array as it was defined in the previous cell.*"]},{"cell_type":"code","metadata":{"id":"VhvnIEt24407"},"source":["standard_deviation(...)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Yg5NWQZw4409"},"source":["**Q.** Calculate standard deviation of the 20 degree data set."]},{"cell_type":"code","metadata":{"id":"_tsFMsE1440-"},"source":["#write your function call here:"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"894xmgRY440-"},"source":["Now we will shift to asking some questions about the experimenter's data set based on what you learned in class.\n","\n","**Q.**  Based on the data, what is the uncertainty in the measurements of the period from **instrumental precision**?\n","*Hint: scroll back up to where the* ten_degs *and* twenty_degs *arrays are defined above. Notice that all the data reports the same number of digits after the decimal point. What does that say about the precision of the timer?*"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","---"],"metadata":{"id":"R9Xz7oOg3dz-"}},{"cell_type":"markdown","metadata":{"id":"7GIUTdZA440_"},"source":["**Q.**  Calculate the uncertainty in a <u>*single*</u> trial of each set of measurements for the period of this pendulum from: i) ten degrees and ii) twenty degrees.\n","*Hint: you will need to call one of the functions defined above (*<span style='font-family:Courier'>standard_deviation()</span>, <span style='font-family:Courier'>standard_unc_of_mean()</span>, or <span style='font-family:Courier'>t_prime()</span> *). Fill the parentheses with the name of the data set you want to make a calculation on.*"]},{"cell_type":"code","metadata":{"id":"TVZV_UDv441B"},"source":["#write function call here:"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","---"],"metadata":{"id":"DIxzp23A3kfS"}},{"cell_type":"markdown","metadata":{"id":"WiqOdhqs441B"},"source":["**Q.** Determine the measured value of the period from each data set (ten degrees and twenty degrees). Express your answer as (best estimate) $\\pm$ (uncertainty).\n"]},{"cell_type":"code","metadata":{"id":"jO-4XVGE441C"},"source":["print(\"i) ten degrees:\")\n","\n","#edit two lines below\n","mean_10 = np.mean(...)  # This function calculates the mean of a list of numbers\n","unc_10 = ...            # Assign this variable with the correct value for the uncertainty\n","\n","print(\"mean is \" + str(mean_10) + \" +/- \" + str(unc_10))"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"xrpUxpNy441C"},"source":["print(\"ii) twenty degrees:\")\n","\n","#edit two lines below\n","mean_20 = ...\n","unc_20 = ...\n","\n","print(\"mean is \" + str(mean_20) + \" +/- \" + str(unc_20))"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Hw4emcMd441C"},"source":["**Q.**  Quantitatively compare the measurements of the period of this pendulum when released from 10 degrees and 20 degrees. Use the t' function below and replace the ... with the relevant uncertainties."]},{"cell_type":"code","metadata":{"id":"g-4V6aw1441D"},"source":["print(t_prime(mean_10, ..., mean_20, ...))"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"3VViAhur441D"},"source":["**Q.** Interpret the results of your quantitative comparison in the previous question. Propose at least two next steps the experimenter could reasonably take based on these results.\n","\n"]},{"cell_type":"markdown","source":["**[Your response here]**\n","\n","Interpretation:\n","\n","Next step 1:\n","\n","Next step 2:"],"metadata":{"id":"ZaQDq5Ue5IC7"}},{"cell_type":"markdown","metadata":{"id":"5F3Oeeyz441F"},"source":["---\n","\n","\n","Save your notebook with all your answers to the questions, modified code cells, and output from each code cell. Submit your notebook by saving it as a PDF and uploading the PDF to the Gradescope assignment titled **Tutorial 2**.\n","\n","---\n","<a name=\"Footnotes\"></a>\n","##Footnotes\n","**1** The 68% refers to the area under a Gaussian (or Normal) distribution. For counting events, such as in particle physics, measurements are better characterized by a Poisson distribution. Fortunately, the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) any distribution converges towards a Gaussian distribution with a large enough number of measurements. [back up](#F1)\n","\n","**2** We would advise you to avoid the words *proof* or *prove* in your discussions of your experiments. In science, nothing is ever proven definitely. Any implication otherwise by textbooks or popular writing on science is sloppy at best and misleading at worst. In science, we only have increased or decreased levels of confidence regarding a claim, by way of experimental evidence, but we should not expect to arrive at an absolute truth.  [back up](#F2)"]}]}