Welcome

This course starts where «From data to agent» ended. You corrected the agent's report, and it now says that the Hornos Lumbre customer spends less on each visit. Management has replied with three questions: how much less, with what margin, and whether the same happens in the refurbished stores. In four blocks you will answer them with the same data and with a copilot that will write the code. Have the Hornos Lumbre files and your logbook to hand: here every figure is delivered with its margin and its record.

What you will find

• Block 1. The shape of the data: look before you summarise. How a column is distributed and how two move together: histogram, quartiles, tails and what a correlation really measures. The risk you learn to see is the summary that hides the shape, and that a copilot delivers without drawing it. In the case: the column of amounts and the scatter of temperature and sales.
• Block 2. Prepare: the odd, the missing and what gets noted. What to do with odd and missing values before calculating: telling an error from a finding, the three options for an outlier, why a value is missing and what each way of handling it implies. The risk you learn to see is the cleaning nobody noted down, whether done by a person or by a copilot.
• Block 3. Estimate with a margin: from the sample to the whole. A figure taken from a sample wobbles if the sample is repeated. You learn to measure that wobble, to read a confidence interval without attributing to it what it does not say, to calculate it by resampling and to check what your copilot has resampled. The risk: the figure without a margin and the narrow margin of a badly chosen sample.
• Block 4. Compare: is there a difference or is it chance?. Testing a difference by shuffling the data, separating what is significant from what is important and knowing when a comparison allows you to talk about cause. The risk: the difference found by dint of searching and the cause asserted without having assigned at random. The case closes with the answer to Elena.

How you will work

Each lesson opens with a short introduction and, below it, several pills: short pieces that explain a single idea and end with a case or something to do. Read them in order the first time; afterwards you can go back to the one you need.

Where you see an interactive practice, do it before reading on: it is there for you to decide, calculate or classify, and the explanation comes afterwards. It does not save what you write, so copy whatever you want to keep.

Each block ends with an exam of six questions. All the answers come with a comment, including the wrong ones: failing and reading why is part of the method.

At The Singular Academy you learn by solving real cases. If at the end of a lesson you do not know what you would do differently tomorrow, go back over it.