Tag Archives: probability

Things 131: Frozen is objectively great, Internet decay and hamsters, Shower danger

Data-based movie recommendation
In 2010, with the release of Disney’s The Princess and the Frog, I looked back at the historic trends to try to understand where Disney went wrong in the 00’s. The Princess and the Frog (and Bolt before it) were successful in terms of IMDB and Rotten Tomatoes ratings, but less so in terms of revenue.

I concluded that Disney had to somehow maintain this level of quality in order to build back their reputation. With Tangled, Wreck-it-Ralph, and most recently Frozen, that’s exactly what they’ve done. In fact, since 2011, they’ve consistently outperformed Pixar (despite owning them):

Frozen currently enjoys the highest IMDB rating Disney have received since The Lion King, although due to self-selection it will be somewhat overstated in these initial weeks after its release.

On a more personal note, I’ve now seen Frozen twice, and highly recommend it – do be advised that it is a full-on musical, but co-composed by one of the people behind The Book of Mormon, so there’s a lot to enjoy even if that wouldn’t usually be your cup of tea. It’s also highly notable for having two female leads with real agency (I’m looking at you, Brave, with your arbitrary plot-advancing Will O’ the Wisps).

Video –Automated Automata Architecture
Continuing the Disney-is-actually-pretty-good-now theme, here Disney research demonstrate how they can generate the gearing required to closely recreate an arbitrary cyclical movement, then 3D-print the result to make the automaton. I particularly like the cyber tiger at 3’30”:

(via The Kid Should See This)

Tumblr – Video games with modified objectives
No wrong way to play” collects examples of people playing video games in ways not intended by the designers. I approve of this.

Tim Link – Learning to Cheat, part 3
Two years ago I surprised myself by betraying someone pretty meanly in a public game. I began a series of blog posts post-rationalising the whole thing within a game-design framework, and after a guilty two-year gap I finally posted my full confession and/or excuse.

Internet decay
If you’ve ever navigated early entries of Things on the blog, you might have seen some dead links, and some links which went dead and got fixed, and some which died again, as I periodically go back and attempt to fight digital entropy.

Based on this insignificant sample, it seems like the half-life for links on the internet is 5-10 years, and considerably less for YouTube videos. This is pretty distressing as laziness/convenience drives us to rely on the internet for files we’re interested in – after all, your options are essentially a) saving a lolcat in downloads>pictures>cats, renaming the file so you can easily find it, and maintaining off-site backups of your data to hedge against hardware failure, or b) just image search “I have a cat and I’m not afraid to use it” from any device, which is a lot more appealing. (Naturally I still choose option a).

There’s a few good links on the subject here, including the compelling quote:

“People are coming to the realization that if nobody saves the Internet, their work will just be gone.” – Alexis Rossi, Internet Archive

Hamster fighting machine / response
Here’s an example of why it’s important to hold onto things on the internet. In 2005, Jarred Purrington made the Hamster Fighting Machine comic/poster (which you can see here or here but not on the original link because it’s dead)

In 2010, Dale Beran (writer of previously-Thinged webcomic/cogent nightmare “A lesson is learned but the damage is irreversible”) posted a lovely response.

Answer – 100 Chalices
Last time I asked if you should choose a chalice with 50/50 odds of being poisoned over one random chalice out of 100 which 100 fiends have each independently and randomly poisoned one of.

Restated, this is asking if you would prefer one-hundred 1-in-100 chances of death vs a single ½ chance. Richard correctly reasoned that the average amount of poison-per-chalice is double in the 100-chalice room, and some degree of bunching in the distribution (i.e. some chalices getting poisoned multiple times) didn’t seem likely to offset it, so the 50/50 chance is probably the best bet.

For any of you not familiar with the probability behind this sort of thing, here’s a quick summary. In the 100-chalice case, calculating all the ways a chalice could get poisoned is very difficult, but calculating the probability of it never getting poisoned is much easier as there’s only one way that can happen. The odds of avoiding poison any one time are 99/100, and this has to be repeated 100 times. So:

Odds of avoiding poisoning = 99/100 x 99/100 x … x 99/100 = (99/100)^100 = 37%. Clearly not as good as the 50% chance in the two-chalice room.

As a post-script, if you’re interested, the expected ‘bunching’ of poisonings would look a bit like this:

This is also a very important concept when evaluating risks in your own life for things that you repeat. For example, I noticed that I tended to step out of the shower in a needlessly risky way, with a risk of slipping (and getting seriously hurt) of perhaps 1-in-a-thousand. That seems tolerable, until you consider that if I showered once a day for 2 years, my odds of avoiding such a fate would be (999/1000)^730 = 48%, in other words I’d be more likely to have at least one such accident than not! So, watch out for that.

Answer – Kickstarter videos
I’ve spoken to a few people about the fact that Kickstarter videos always make me feel less motivated to put my money in. The underlying reason seems to be that a Kickstarter page typically does a great job of selling the product/reward, but the video often ends up being more about selling the people behind it (as being worthy, or in need of your money). Before the video I don’t even think about that; after the video, that’s just another reason to say no.

-Transmission finally ends

Things 112: Eyes, Guessing Cat, Amigara Fault

This week Things has a very slight Hallowe’en theme.

Puzzle
This is one where you should gather some people around the monitor and see who can do best: guess the cartoon (or CG) character from their eyes (mouse over the eyes to see the character outline that should tell you if you’re right).

And yes, it is pretty difficult – I only got 6, and I watch a lot of animation!

Video
Here’s a video that begs the question: is the cat playing the game, or just acting out of blind instinct?

To which the answer is to have a big argument about the definitions being used before concluding that you can’t tell.

Quote
In the wonderfully stylised animation The Secret of Kells, I heard the line “One beetle recognises another” and wondered if it was some kind of proverb. It turns out that it is, and actually – obviously – there are a whole bunch of Irish Proverbs, which in translated form become alternately profound, banal or hilarious, just as I imagine English proverbs must seem if you haven’t grown up with them. Here’s a list of them on Wikiquote, and here are a few of my favourites, for unstated reasons:

“Every beginning is weak.”

“Time is a good story teller.”

“A lamb becomes a sheep with distance…”

“The quiet are guilty”

Comic
The Enigma of Amigara Fault is a horror comic that impressed me with its unconventional approach. It’s 32 pages, and originally in Japanese so you have to read the panels right to left. But if you want a comic that will freak you out for Hallowe’en, it’s worth it. Unless you’re particularly claustrophobic, in which case you should probably steer clear of it entirely.

Answer – Malady X
In Things 111 I asked what the probability of having Malady X is if a randomly administered 99%-accurate test for it comes back positive. As Phil and Thomas noted, you can’t actually answer from this information alone: you also have to know what the probability of a random person actually having Malady X is. A lot of people don’t have an intuition for this fact. I’m going to attempt to explain ways to apprehend that hand-wavingly, mathematically, and visually.

Argument from hand waving and examples:
Imagine the probability of having Malady X is 0% – nobody has it. In this case, it’s certain that getting a positive result means you were simply in the 1% of cases where the test comes back incorrect.
Conversely if the probability of having it is 100% – everybody has it – then you must be in the 99% of cases where it is accurate. In this way, it’s clear the underlying probability influences the chances that the test is correct!

We might worry that these extremes somehow break the puzzle, so let’s imagine less extreme alternatives. Imagine 1,000 people are tested. If 50% (500) really have Malady X, on average we expect the test to come back positive for 99% of them (495) and also for 1% of the 500 that don’t have it (5). In this situation, 495 out of the 500 people for whom the test was positive actually have the disease – 99%.

Alternatively, if 1 person (or 0.1%) out of the 1,000 has the disease, they’re very likely to be correctly diagnosed, and we expect roughly 10 of the other 999 to get a positive result. In this case 1 out of 11 people with a positive result actually have Malady X – fewer than 10%. So clearly the underlying incidence level matters.

Argument from maths:
There are two probabilities at work: the chance the test is correct (99%) and the chance of anyone having Malady X (unknown – let’s call it X%). When you combine probabilities you multiply them, so for example the chance of anyone actually having Malady X AND getting a postive result is 99% times X%.

If someone gets a positive result and that’s all we know, we reason as follows:
A = Probability someone has Malady X and tests positive = X% times 99% times
B = Probability someone does not have Malady X but still tests positive = (100% – X%) times 1%
If you test positive, the chance you actually have it is C = A / (A+B). But if you haven’t studied probability carefully, I’m not sure you could infer this, which is why I like to come up with other ways of getting a feel for the correct answer.

Argument from visualisation:
Since there are two probabilities in question, and we combine probabilities by multiplying, this naturally suggests a visualisation where probability is represented by rectangular area (since area is calculated by multiplying height by breadth).

For example, if we imagine the actual incidence rate of Malady X is 50%, the picture would look like this (click for big):

If the test result is positive, you either have it and the result is correct (big yellow area) or you don’t have it but the test was incorrect (small dark blue area). The chance of you actually having Malady X is equal to the proportion of those combined areas that is yellow. In this case:
Yellow = 99% x 50% = 49.5%
Dark blue = 1% * 50% = 0.5%
Probability you have it = Proportion that is yellow = 49.5% / (49.5% + 0.5%) = 99%.

Alternatively if the incidence rate is, say, 2%, it looks like this:

Here we see the yellow and dark blue areas are very similar, so the chance of you being one or the other is much more even. In fact, it’s:
Yellow = 99% x 2% = 1.98%
Dark blue = 1% x 98% = 0.98%
Probability you have it = Proportion that is yellow = 1.98% / (1.98% + 0.98%) = 67% (ish).

As Peter Donnelly shows in this TED talk, this actually has some severe ramifications, because when the probability of the thing being tested for is extremely low, it becomes overwhelmingly likely that a positive result is false, but people intuitively feel that a 99% accurate test should be correct 99% of the time.

Thomas also noted:

If anyone is interested in playing around with the probabilities (even if you’re not familiar with the maths), I recommend GeNIe:
http://genie.sis.pitt.edu/
It lets you create networks of dependencies, set evidence and work out probabilities in problems just like these.

-Transmission finally ends

Things 111: Malady X, Stretching Cat, 3 Panels

Question
(Thanks to Simon for reminding me of this important probability lesson!)

At random, you are tested for Malady X. Alarmingly (particularly given that you don’t even know what Malady X is) the test comes back positive. But you know these tests are not always perfect – there’s a chance that it’s wrong, and you don’t really have Malady X at all. So you ask how accurate the test is. You are told that if someone really does have Malady X, there’s a 99% chance the test will come back positive; for someone that doesn’t have it, there is a 99% chance the test will come back negative.

What is the probability that you actually have Malady X?

Animated Gif
Here is the best animated gif of a cat I have seen in a long time:

Link
(via Silv3r): A huge and (I think?) growing collection of street fliers that play with the form, some okay and others quite, quite brilliant, can be found here (browse the other pages if you like what you see).

Picture
I am proud to be able to say that I know James White, the author of this perfect 3-panel comic, personally.

Answer
Last time I asked about what people really mean when they claim “change is accelerating”.

The most direct and plausible answer came from John B, who suggested that the scope of human knowledge is the thing that is really growing, and the subjective change we experience is what arises from these discoveries. While it’s only a proxy, one way to measure this is to track how many patents are granted over time, and on a logarithmic scale this does look kind of linear (indicating acceleration).

Bex has an alternative view. The perception of change seems to generally accelerate with age (which in itself is already enough to explain why people claim this all the time). The population of the UK (at least) is ageing. Therefore, the speed-of-change will be reported to be, on average, faster over time. Sneaky!

As the Wikipedia article on the subject currently notes, another confounding factor could be the growth of the human race itself. For example, if a fixed proportion of humans files patents, exponential growth in human race will directly lead to exponential growth in patents filed.

In any field, taking any trend and extrapolating it arbitrarily far into the future is generally unwise. If we don’t know exactly what we’re measuring, and we don’t understand the factors governing the change, even less so. Given the potential disruptions of the technology we’re seeing already, if anything it seems just as likely to me that sudden power imbalances become more likely, which could lead to large swathes of humanity being wiped out, or global human society turning into a dead-end all-powerful dictatorship with no desire to change the status quo.

Things 106: Best at Chess, Art of Science, Crowdfunding

Tim Link – Competitive Sandwich Making
Last week Clare and I ran a game based on tessellating pieces of cheese to make the best sandwich for the Hide&Seek Sandpit event. You can read about it and see the photos on my project blog, Tower of the Octopus.

Puzzle
In a chess tournament in which anyone can use any means available to them to come up with their moves, who would win? Some possible answers to give an idea of what kind of thing I’m talking about here:

  • A high-ranking chess Grandmaster
  • A really good chess-playing supercomputer
  • A huge team of moderately skilled players with some method of combining their ideas
  • A moderately skilled player with access to a moderately good computer that can run some basic chess calculations

(I had wondered about this in an abstract way before, but recently found out that has actually been done. I’ll relate what happened in that event next week, but you could of course try to Google as well as guess the answer if you wanted).

Video
(Via Phil): Art out of Science:
(Two views of the same thing. If your browser is up to it, you could try watching both videos simultaneously – start the bottom one 20s after the top):

Links
Kickstarter is one of my favourite things on the internet: people with an idea for something get a platform from which to shout about it, and to collect pre-orders or donations from people that like the idea. If there’s enough interest, the project can go ahead, and everybody wins.

So far I’ve helped fund two comics, the Wormworld Saga app (which saw so much success the creator, Daniel Lieske, decided he could give up his day job), and I’m currently backing The Endangered Alphabets Project, which is the kind of thing I like to imagine in a vague way is going on in the world, but I now have the opportunity to facilitate it directly (also, it’s only just on track to hit target, so do go check it out).

You can follow Kickstarter on Twitter, or go to their home page and scroll to the bottom to sign up for the weekly newsletter which highlights the most interesting projects.

IndieGoGo is similar but for reasons I can’t really pin down doesn’t work as well for me.

Crowdfunder is a UK version which I don’t tend to find as inspiring, but would probably be the best one for someone in the UK to create a project with (since Kickstarter requires a US bank account).

Quote
Overheard in the maths common room when I was studying for my PhD at Royal Holloway:

But nobody knows what probability is! Probability is defined in terms of randomness, and randomness is defined in terms of probability!

Answers to Monty Hall and the Two Envelopes
Last week I asked about the Monty Hall problem, which I should have introduced before the Two Envelopes problem I set two weeks ago.

The Monty Hall problem has a nice Wikipedia page, the most helpful part of which is probably the decision tree showing all possible outcomes.

In brief, the answer is that you should switch after Monty shows you an incorrect door, but certain misguided instincts steer most people away from that choice. The Endowment Effect and Loss Aversion mean that regardless of probability, people fear they would regret “giving up” their first choice more than sticking with it if they end up losing.

The more subtle effect is an instinctive (or partially trained?) feeling that the choices of others have no effect on the probabilities of our own choices in these kind of contexts. This is true when the other person has just as much information as you, but that is not the case here – Monty knows where the car is, and uses that information to ensure that he always opens a door with a goat behind it. So he has more information than you, and when you see his choice you gain some information.

Or to give an answer that might go with the grain of instinct for some people, consider this: there is a 2/3 chance that the car is behind one of the doors you don’t pick. Monty shows you that it definitely isn’t behind one of them. So there’s still a 2/3 chance the car is behind the other one, and a 1/3 chance it’s behind the one you first chose.

As for the Two Envelopes, it turns out this is more difficult than I originally remembered. Again, there’s a great Wikipedia page on the subject, which has quite a lot of detail.

As Thomas noted, a key phrase missing from the subtly specious argument for swapping is “Without Loss of Generality” (WLOG), which one must always be careful to check whenever substituting a variable (in this case, the amount in the envelope) with a specific figure (£10 in the example I gave).

Is it true that the reasoning I gave based on having £10 in the envelope truly retains the generality of the problem – would the reasoning also hold for any other amount? In short, no. For example, there could be 1p in the envelope, or any odd number of pence, in which case we would have to conclude that we had the lesser envelope (although this still means you should swap). More dramatically (here we imagine the envelopes contain cheques, and that these cheques are totally reliable), your envelope could theoretically contain over half of all the money in the world, in which case you can be sure the other envelope contains less. More realistically, it could contain more than 1/3 the amount of money you expect the person filling the envelopes to be willing to give away, in which case you would strongly suspect the other envelope to contain the lesser amount.

If you’re interested, do read the full Wikipedia article, and meanwhile, remember to watch out for unjustifiable WLOGs.