Text Complexity Ranker and Frequency-Ranked Vocabulary List Maker

textranker textranker
The more one learns about information, communication, and computation, the more interesting the structure of language itself becomes. The purpose of this package of tools is to 1. create a repository of sample text representative of a language e.g. Gutenberg. 2. Rank the frequency of words in the language. 2. Analyze a particular piece of text, e.g. a book or an article, and assess the text for its difficulty based on the distribution of words (e.g. figures to the left) given their relative frequencies in the repository. The idea is for it to be a kind of learning tool...one often has trouble deciding whether a piece of foreign language text is going to be painful to read with constant dictionary lookups or engaging. With this tool, it should be possible to assess that. On an interesting side note, this script reproduces the legendary Zipf's law plot (shown to the above) which reveals the stunning hierarchical nature of language. The scripts are available for download here. Another related script generates a ranking of words for vocabulary-learning purposes. This way, if you are learning a foreign language, you can study only the most frequent words earliest. It works by pasting in a large body of text representative of the language (or subfield) that you want to learn. The script is available here.

Bayesian Experimental Design Tool

Experimental design in the face of uncertainty can be tricky. At times it is easy to be biased, favoring conclusions that support what you think is "supposed" to happen. One can even shoot oneself in the foot by proceeding down a dead-end when the data suggest otherwise. This Bayesian experimental design tool is meant to help clarify one's hypotheses and distinguish which experiments are likely to be useful in distinguishing branching explanations of reality. The tool is entirely user-input driven, so it will not reveal anything that you technically do not have access to knowing already. But the process of filling in the form and the report it generates at the end may shed light on your inner thought process. The tool is not necessarily restricted to science either, rather it could be used in any situation where experimentation and observation are needed to make decisions under uncertainty. The output report is formatted in wiki markup to be recorded in a lab notebook. Download the script here.

Hikaru's 3 Hat-Color Prisoner Problem

puzzle The author of this puzzle is Hikaru Saito a physicist currently working at Kyushu University in Japan. This puzzle is a variant of the well-known red/blue hat prisoners in a row problem. The solutions are quite different though, and knowing the solution to the traditional problem may actually hinder your ability to solve this one. The puzzle was recently featured in Quanta Magazine's puzzle column alongside the 2 hat variant and a third which was unknown to me! Since the solutions are out there now, I have gone ahead and posted my solution to the puzzle here. The puzzle goes like this:
10 prisoners are lined up single-file. The warden puts colored hats that are either blue, red, or yellow on each of the prisoners. They are lined up such that each prisoner can see the colors of the hats of everyone in front of her, but does not know her own color or those of anyone behind. The warden then, starting from the back of the line, asks each prisoner to state the color of his/her hat. The prisoners are not permitted to say anything other than a color (red, blue, or yellow), and they can only speak it once. The warden will then go to the next prisoner and repeat the question and continue until all prisoners have been asked. At the end, those prisoners who answered incorrectly the color of their hat will be executed. Assuming that the prisoners are permitted to confer with each other before the line-up of doom, what plan could they conceive of which would maximize the number of lives saved? Let us assume that the warden can overhear this plan and adapt his hat-placement choices so as to counter any attempts at reducing deaths. e.g. if all the prisoners decide to announce blue as their color thinking that this could guarantee a 1/3 chance of survival, the warden could simply put red hats on everyone. Feel free to email me if you think you have an answer! (disclaimer: there is no prize for solving it)