Information theory applied to physical systems makes contact with lively fields that include: atom-scale microscopy, communications, complex systems, data compression, evolution of replicable codes, genetics, memetics, molecular nanotechnology, molecular recognition in biology, nano-phase material engineering, silicon science, thermal physics, and quantum computation. Related links in these areas are provided below. Suggestions for this link list are welcome here.
[new] [puzzle1] [why i-fzx] [quotes] [books] [fun] [more puzzlers] [static/dynamic links]
New:
Google 10^100 project video on
niche-network
layer-multiplicity (NNLM). Notes on two 2007
talks, your
niche-network
spider, and some thermochapter
corollaries
for introductory physics class.
Papers for Complexity on the
thermal roots
of correlation-based complexity,
and
a simplex model
for layered niche-networks. Reflections on:
media
and community health,
assessing and
visualizing
the correlation-focus of
individuals and communities, a June 2006 Discovery
article,
and a May 2006 talk at UIUC.
Notes on: (i)
quantifying
risks,
(ii) thermal
roots from a talk in
May 2005 at UIUC,
(iii) some possibly
sharable
distinctions, (iv)
ways
to monitor
correlation-based complexity, (v) science as
concept-assisted
observation rather than consensus, (vi) world-view
syntheses
made possible by Bayesian inference, and (vii)
ideas that
serve us. Working notes on
a
course for informatics collaborators, and a note on
perspectives
that go hand in hand. Illustration of two approaches to
the ice-water
invention problem, linked to an attempted review of the physical
principles underlying correlation-based complexity. A note on how
a
wee bit of surprisal goes a long way, and
recent
submissions including a poster on correlation-based complexity
for ICCS2004.
The AAPT version of that Nov 2003
AJP article is now posted (with permission)
on our
CvInBits page.
Here are some notes on
a
talk at the summer 2003 AAPT meeting in
Madison WI on natural units for quantities like temperature and heat
capacity in the intro physics classroom. An attempt at "minimal
jargon" research
highlights from our group. Major updates circa Jan 2003 to
our
collection of notes on IFZX for introductory physics
students, and to the paper
on heat capacity in bits, along with (at bottom) some
worksheets on the max-ent best-guess machine as applied in
this context. The Fall 2002 update of a monograph
for intro-physics students on
how hot works.
Three abstracts and a crackerbarrel from summer 1999
in San Antonio, including outline of our talk on thermal
physics and sciences involving codes, a revision
of our heat capacity paper providing an information
physics view of water and other systems, plus a challenge
sheet on "science of the possible" calculations.
Program description
of the 16 Oct '98 symposium at Washington University on "The
Legacy of Edwin T. Jaynes", and a web-version
of our presentation there. A paper on
``heat capacity in bits'' along with some essential parts of a note on
``i-fzx for beginning-physics classes'' are now in the e-print
archives at Los Alamos. Browser-readable (HTML) translations are
in prep. Your comments on both the paper and note are
invited. Our earlier paper on statistical inference from images (PRL
64, 1031) has now been archived at
Los Alamos as well.
Elsewhere, check out the EECS/ME course for MIT freshmen on information and entropy, Serendip's developing Bryn Mawr exhibit on the essential link between life & thermodynamics, Tom Schneider's information theory resource list, and the quantum information physics page at University of New Mexico.
Note: This page is based mostly on material which has
already found its way into books and even senior undergraduate
physics texts, but some has not yet been through peer review
although that is the eventual intent. Hence should you echo, in
your own work as well as in print or on the web, a citation would
be cool. {Thanks :) /pf}
Puzzler: Most people agree that the absolute zero of temperature is impossible to reach. But few know that it is possible to attain minus absolute temperatures, and thus in effect to "approach absolute zero" from the negative as well as the positive direction. Moreover, you can begin to do so with a set of dominoes in the living room of your very own home. How so? (Hint: Part of the paradox here is related to the strange way, historically, that temperature was defined.) *More*
The shift to an information theory view shows that thermal physics is a kind of gambling theory which uses physical principles. Such physically-informed inference may have a wider range of application than demonstrated to date, since statistical inference by itself has a strong footing already in most natural and social sciences. The full range remains undemonstrated, partly because this paradigm shift is centered in the second half of the 20th century. It cannot yet be seen with hindsight, even though it has been working its way into physics textbooks at graduate/senior-undergrad levels for years. Moving it to introductory levels is one objective of these pages. Some key elements of the shift, from the perspective of an educated lay audience, are listed below in what is hopefully not the last Server (online March '95) dedicated to Information Physics on the Web.
Copyright (1970-95) by Phil Fraundorf,
Dept. of Physics
& Astronomy,
University of Missouri-StL, St. Louis MO 63121-4499
Phone: (314)516-5044, Fax:(314)516-6152
At UM-StLouis see also: anyspeed, cme, esti-lab, progs,
& wuzzlers.
For source, cite URL at http://newton.umsl.edu/infophys/infophys.html
Version release date: 06 Mar 2004.
Between March 1995 and July 1996, the access log for this page
shows 1051 visits.
Since 1 August 1996, you are visitor number [broken counter].
Mindquilts
site page requests est. around 2000/day hence over 500,000/year.
Page requests to a stat-counter linked subset since 4/7/2005:
.
Some outside web-links that
point
in to stuff here.
Quotes of the Moment:
``Rare earth'' (Springer-Verlag 2000) and ``The life and death of planet earth'' (Henry Holt & Co 2003, Owl Books 2004) by Peter D. Ward and Donald Brownlee. Books by a long-time friend about the way that insights combined from astrophysics, earth science, and bioscience are offering up a broad empirical view of our planets' future, as well as of our place in this part of the galaxy. A figure from the second book is discussed here.
``Dynamics of complex systems'' by Yaneer Bar-Yam (Westview Press, 1997). A survey book on tools relevant to this fairly encompassing theme, a large part of whose 848 pages is dedicated to an "intro to the math" chapter which attempts to cover quite a bit of ground.
``Rational Descriptions, Decisions, and Designs'' by Myron Tribus, (Pergamon Press, NY, 1969). A rich and detailed description of Bayesian approaches to inference and risk assessment in a variety of practical areas, which at the outset defines probability assignments as ``numerical encodings of a state of knowledge'', thus recognizing that the roots of inference lie not in stand-alone properties of the system in question, but rather in the state of correlation or mutual information between system and observer.
``The New Ambidextrous Universe - Revised Edition'' by Martin Gardner (W. H. Freeman, NY, 1990). A tale weaving together the delightful facts about asymmetry that Martin Gardner has in his files. A short quote from there, concerning one peril of taking your own ideas too seriously, can be found here.
``The Resurgence of the Real'' by Charlene Spretnak (Addison-Wesley, NY, 1997). An informed attempt at synthesis of historical and contemporary observations of ourselves and the world around, in areas of literature, religion, economics, politics, philosophy and science. Lots of references. Did you know that Mary Shelley's Frankenstein was originally subtitled The Modern Prometheus? Perhaps most importantly, Charlene speaks effectively to non-scientific readers. An excerpt relevant to this page may be found here.
``Science by degrees - Temperature from 0 to 0'' by
Castle, Emmenish, Henkes, Miller & Rayne (Walker & Co.,
Westinghouse Search Book Series, 1965). This is one of
the first books to describe for ordinary folks the importance of
what they call
or uncertainty slope 1/T=dS/dE (labeled ``coldness'' by Garrod
in the text discussed below). As a measure of the accessible
information lost about the state of a system, per unit
increase in thermal energy, this quantity provides insight into
the meaning and usefulness of temperature, especially in
non-quadratic systems (like spin systems and lasers) where the
usual equipartition rule does not hold.
Stuart Kauffman's ``At home in the universe: the search for the laws of self-organization and complexity'' (Oxford University Press, 1995) 321pp. Except for the boring title and some overly pretentious language (especially in the first 2/3 of the book), this is a place to find out why complex systems research might indeed be of wider interest than simply to mathematicians. Bravo!
Claude Garrod's ``Statistical mechanics and
thermodynamics'' (Oxford University Press, 1995). This
book has been chosen for it's depth and originality, even if (as
with most texts) use of it in teaching might mandate providing
some alternate and simpler ways to introduce some of the key
elements. Dr. Garrod is aware of contemporary developments on a
variety of fronts, and hence provides handles that are absent in
many of the texts currently available. More importantly, about
HALF of the book consists of worked exercises. It has potential
for both graduate and undergraduate applications, in which worked
exercises and instructor material might be used to augment
respectively the second and first halves of the book. A MathCad
(v3.1 or greater) worksheet, viewable with MathSoft's free browser, on
the Loschmidt Conjecture discussed at the end of Chapter 2 can be
found here.
Update: Students in our Fall '95 undergraduate course said
that they found the text part of the book difficult to
understand, at best. /pf
In thermodynamics: (88oct15/pf) Heat flows
from regions of low coldness to high coldness BY CHANCE. In fact,
we now know that coldness (another word for the reciprocal
of the absolute temperature) simply measures the bits of
uncertainty S about the state of a system gained per unit of
thermal energy E added thereto, i.e. the uncertainty slope
dS/dE. On average for most macroscopic and spin systems this
uncertainty slope decreases as one adds energy. A little math
will show that the most likely distribution
of energy (or any conserved quantity) between two systems,
if there is one, is a distribution for which the
individual system ``uncertainty slopes'' are the same. If not,
energy flow from low to high slope systems opens the door to a
wider range of random choices. A zeroth law with teeth
then follows. Two remarkable consequences of this are that
temperature has physical (as distinct from historical) units of energy
per unit information (e.g. room temperature is 1/40
[eV/nat]), while physical units for heat capacity are
dimension free, and effectively for any b in base-b
units of information uncertainty per b-fold increase in
temperature (e.g. the heat capacity of water is
approximately 9 bits per 2-fold increase in temperature, which
corresponds phenomenologically to 2*9 = 18 modes of thermal
energy storage or ``degrees freedom'' per molecule).
With respect to energy flow in biology: (91oct15/pf)
Photosynthetic plants act as heat engines, converting
solar heat radiation to available work, while we serve as information
engines by thermalizing available work toward the creation
of mutual information resources in our environment. The
renewable-resource energy-per-unit-time available to life on
earth (i.e. the life power stream,
or LPS) is probably around 10^15 watts, much larger than man's
rate of fuel/electricity energy production! It is interesting to
note that the energy in a vegetarian meal requires about a
nanosecond of our present day LPS, while the energy in a
carnivore's meal requires perhaps a hundred times as much, even
though the consumer gets the same thousand calories to work with
in the end. Lastly, it is interesting to note that information
engines like us, creating correlations between subsystems in our
environment by thermalizing available work at room temperature
with a caloric intake typical of humans, can produce no more than
I_out less than or equal to W_in/(kT_out)
or about 10^24 megabytes of such mutual (or correlation)
information per day? I suspect that my actual rate of
accomplishment, at least, is MUCH LESS than that!
Concerning a physical perspective on the phenomenon of
life: (96feb29/pf) We are steady-state
excitations, while many of the things that we serve, protect,
and hold sacred are replicable codes somehow ``informed''
to the world around. For example, heat engines
(like gas-powered motors and photosynthetic plants) and information
engines (like copy machines and ourselves) are
steady-state excitations, while genes (like the DNA that
holds your genetic heritage) and memes (like your favorite
songs, fashions, beliefs, laws, and ideas) are replicable codes.
This distinction between replicable codes and steady-state
excitations first became useful not with our discovery of the
DNA base-pair code (1950's), nor with the beginnings of
information theory (late 40's), nor for that matter with Plato's
formal recognition of ideas as separate entities (around 400 BC).
I suspect that this distinction became useful much earlier, with
the invention in practice of genetic bargains in cell
reproduction. The distinction became a hard reality with the
pre-vertebrate invention of programmed-death for multicelled
organisms (to use the words of Lynn Margulis in a recent Scientific
American essay), since before this one just split in two to
create one's offspring! After death, only the code's chance at
immortality survived. In the early days of replicable ideas,
memes (rhymes with "beams") bypassed the fate of the
excitations carrying them only by leaping from one individual to
another, almost like a virus. The idea of fire-building is
a likely example in this regard. The reproductive power of these
memetic replicators has since taken quantum leaps, with the
invention(s) of dance, music, language, song, theatre, writing,
the phonetic alphabet, printing presses, electronic media, and
now the world wide web. For more on this, see our notes on
challenges in tracking
the inheritance of the various kinds of codes we serve. This
incidentally also puts into perspective the surprisingly few ways
that we have for sharing these codes, and the deep importance
(following McLuhan) of the delivery modes available at any given
time.
In matters of evolution: (91oct15/pf)
Among multi-celled lifeforms on earth, it is the codes or
replicators and not the excitations that literally evolve
with time, even though both codes & organsims are key
physical components of the phenomenon we call life. Likewise,
even life's competition is between codes perhaps as much or more
than it is between organisms! Early evolutionary thinking, as one
might expect, was relatively "organism-centric" in this
regard, as have been most "hard-science" perspectives
on life. Perhaps they had better not be any longer. Ideas as
distinct from excitations not only live, and evolve, and deserve
recognition. They also deserve respect and caution since, from
the vantage point of a code, humans and computers as information
engines alike could be little more than disposable vehicles for
survival (to paraphrase Dawkins' in The Selfish Gene). As
we have I hope already learned by mistake, ideas can be powerful
and ignore our humanity just the same.
On memes of interest to humans. (97dec30/pf)
The most important memes for humans may be the
"WE-memes" which encode information on how people
behave in groups. WE-memes focus inward and outward from three
boundaries: the limits of our body, the limits of our gene-pool,
and the limits of our culture. The body WE-memes focus on
the individual (inward) and on pairs (outward). The
gene-pool WE-memes focus on family (inward) and hierarchy
(outward). The culture WE-memes focus on beliefs
(inward) and profession (outward). Richard Graves'
"Claudius" novels and the BBC Masterpiece Theatre
"I, Claudius" series it spawned may be seen as shocking
stories of organism behavior, OR as a portrait of WE-memes in
conflict (e.g. between a mother's allegiances to children and
politics). In fact who knows, but that as the dangers of
addiction to WE-meme abuse clarify, WE-memics Anonymous (WA) may
be established to help us through such conflicts in days ahead!
Certainly our ability to distinguish ourselves, as excitations,
from the ideas we are involved with, is probably a good thing.
In clinical psychology: (94aug09/pf)
A focus on information processing styles, independent of the
codes and values being supported by a given person, underlies
the effectiveness of one of the most widely-used indicators of
psychological type. One observation relevant here is that some
evolved (as distinct from manufactured) information engines (most
notably human beings) can predicate their actions on the basis of
either pattern recognition (a.k.a. perception), or
on the basis of memetic processing (i.e. code or meme-based decisions).
We all have ``Differing Gifts'' (cf. the book of this name by
Elizabeth Meyers Briggs) in this regard.
In human communities: (87feb20/pf) The
focus of the evolutionary dynamic on the genetic
replicator is seeing competition from a focus on the memetic
one, to an extent unlike that seen in communities of any other
terrestrial lifeform. With modern developments in the prospects
for memetic replication, this trend will likely continue. If your
ideas are on the wrong side of this trend, and for example
would focus on the genetic replicator alone, you might want to
help them nail down anchors on the other side as well.
In working together: (96feb17/pf) Since
evolved excitations each interpret code in context of their own
experience and origins, limit or boundary-setting in interactions
between individuals is best done not unilaterally, by the
action of one individual, but rather by the offering of
meme-based decisions to the other. This linking of
informed individual decisions, each within the purview of the
deciding individual's responsibility, is a common thread in
evolving codes related to many areas of human endeavor (peer
review and the vote among them). It is especially
important with children who are learning to take charge of their
own lives. Although individuals will in general still have their
own distinct "reality", this can allow the
uncharacterized riches within each to play a role in informing
collective action. Of course, the simplest reason
that codes are evolving in this way may be that code-based
behaviors serve the codes!
A tale (04jun03/pf)
of correlation-based
complexity: emergent drivers...(boundaries)...symbioses
Puzzlers: In the foregoing
context, what quantities are being plotted in the graphs
below? 
Hint: This one relates to the first puzzler on this page as well.
Hint: This one relates to a model introduced only in senior undergraduate courses.
Co-developer of Shannon-Jaynes entropy:
Complexity measurer & quantum computer:
Molecular sequence-logo inventor:
Genetic ethologist & coining memeticist:
Mathematical biologist:
and these topics...
Send comments, possible answers to problems posed, and/or complaints, to pfraundorf(AT)umsl(DOT)edu.
