Taken's Theorem and Dynamic Correlation

As much as I dislike regurgitating content instead of producing it, this Sugihara et al. paper is possibly the coolest thing I have ever seen, and has been for a couple years now.

Since MathJax (what we are using to format everything in $\LaTeX$) probably doesn't have BibTeX support, I'm going to go ahead and do an academic no-no by just providing the link to the paper, and no citation^[1].

The goal of this paper is to introduce a new method, convergent cross mapping (CCM), a test meant to help determine whether one event in a nonlinear system causes another. As the introduction notes, two populations interacting nonlinearly can go through phases where the behavior is similar, they behave oppositely, or there appears to be no relation. This makes applying traditional measures of correlation or causation useless in such situations.

Enter Taken's theorem: a theorem stating (in extreme layman's terms) that it is possible to 'reconstruct' a chaotic attractor using one of its components. WHICH IS SO COOL. Sugihara et al. concluded that if two components were members of the same system, they would not only be able to reconstruct the original system, but one component would predict the behavior of the other. Hence the 'nearest neighbors' to a data point in the first component should be associated timewise with the nearest neighbors to the corresponding data point in the second component, providing the systems are related, and this predictive ability should get better as more data are taken into account. This is the gist of CCM, which is explained more clearly below and in the paper.

What happens when this method is applied to real data on sardine and anchovy population? Read the paper to find out! If you're a member of the general public, it's less intense and more explanatory than math papers generally tend to go, and a great read if you're even a tiny bit into population ecology. (Not a whole lot is said on the exact implementation, but the numbers they're getting look like correlation coefficients between a variable and its nearest-neighbor estimate as more data is added. I should try to do this in MATLAB and post code.)

Did I mention the videos? George Sugihara's son made two brilliant videos to illustrate where the idea came from. (I want to be his friend.) Here's one on Taken's theorem:

There's also one demonstrating the manifold reconstruction:

Lastly, a brief description of how CCM works:

Yeah. I'm not kidding about this being the coolest thing ever!

I learned of this paper during a talk George Sugihara gave in 2013. Of course, the videos are very pretty, but the topic also illustrates something bigger: how applied mathematicians can make breakthroughs by studying "useless" theoretical topics. Some of my old professors were fond of claiming all pure math eventually becomes applied math. This is a great recent example of such creativity; who would have expected Taken's theorem to relate to causality in ecosystems?

Theory: it's what separates us from the engineers! Or just another excuse for the applied folks to read analysis textbooks.

[1]... Nope, my heart won't let me do it. Here's the citation:
Detecting Causality in Complex Ecosystems. George Sugihara, Robert May, Hao Ye, Chih-hao Hsieh, Ethan Deyle, Michael Fogarty, and Stephan Munch. Science. 26 October 2012: 338 (6106), 496-500. Published online 20 September 2012 [DOI:10.1126/science.1227079]

How I Learned to Stop Worrying and Love Fminsearch

(I'd intended to write a post on subsets of null sets that are not null sets, but some lovely person has already posted it on Wikipedia!)

My field involves a lot of fitting ODE parameters to experimental data, so, as expected, I have a long and storied relationship with distance minimization algorithms.

Particularly fminsearch, MATLAB's built-in Nelder-Mead simplex direct search function.

If a network executive decided for some reason to make a sitcom based on my life, fminsearch would be the lovable goofball character whose laziness is the basis for many a cheap joke.

"FMINSEARCH!!! Stop watching football and clean up all those Funyun wrappers from off the floor!" I'd scream. To which fminsearch would reply, "I can't see them! They're not contained in my initial simplex!" Oh, fminsearch....

Fminsearch is great for converging exactly to local minima, but suffers in a couple ways, the main problem being its inability to detect global minima outside its starting range. This problem arises because the underlying algorithm is local (operates on a closed subset of parameter space) and deterministic (will return the same best fit every time if options/initial conditions are unchanged). Of course, the easiest fix is then to pair it with a global, nondeterministic fitting algorithm such as MCMC (Markov Chain Monte Carlo methods) or a genetic algorithm. The new hybrid algorithm then at least has a chance of breaking out of local minimum wells. However, fminsearch is much better at converging exactly to local minima, so it's a good idea to run fminsearch at the end, just in case.

A similar issue occurs minimizing over several parameter values. Although it is possible to use fminsearch to optimize several parameters at once, my advisors and I have had more luck fitting one parameter at a time iteratively. Beware! The order in which the parameters are fitted has a huge effect on the outcome. Less sensitive parameters may not change much if they are fitted last, and if two parameters are related by dependence, it can be difficult to fit them separately. I've had more luck implementing MCMC with Latin Hypercube Sampling (LHS).

Lastly, it can be difficult to find a local minimum in which constraints on parameter size are satisfied (for example, if the algorithm keeps assigning a negative value to a parameter that shouldn't be negative). This is again a situation that should be passed to MCMC, because reducing the average step size in parameter space will cause parameter values to stay closer to the initial conditions. Another 'cheating' fix would be to alter your distance function to output absurdly high numbers when a parameter value enters the no-no range---this is probably the best way to go if you want to stick with fminsearch.

These are, at a broad level, the most important things I've learned in my years of practically dating fminsearch. I'm cataloging them here in case someone looking for guidance can be spared a few couple fights with my favorite MATLAB function.

If any readers (ha, ha) want me to post some iterative fminsearch or MCMC code, I would be happy to provide a watered-down version!

Funny Papers: Overly Expressive Lab Mouse

Scientists are people too, and they have a sense of humor. We don't claim to be a research humor blog (that's what Annals of Improbable Research is for.) However, sometimes we come across something that is so hilarious it has to be shared.

In this case, it's a silly figure in a medical research paper. I doubt the legality of posting an image from someone else's protected academic work, so here's the source (Figure 2) and a short description of what's happening in the figure:

• The first image in the chain is a newly infected lab mouse with a darling smile on its little face.
• Endotoxins produce IL-12, TNF and IFN-gamma in the mouse, contributing to shock. The mouse's smile has been reverted; it is now unhappy at its predicament.
• The toxic shock leads to weight loss. Here, weight loss is manifested as someone copy-pasting an image of the sad mouse and shrinking it a bit.
• The poor mouse dies. The authors have helpfully put 'x's in the eyes and put the mouse's little feet up to communicate to the reader that the mouse is, indeed, dead.

A shout-out to C.A. Biron and R.T. Gazzinelli for demonstrating that, while mathematicians have the best jokes, experimental biologists have us completely beat on black humor!