This is a sandbox page for Thermochap. Thermochap ( talk) 22:30, 4 February 2021 (UTC)
Creating new pages is a more elaborate process than it used to be, but still worthwhile even it takes longer to do. Pages that might help include a page on:
Clues to emerging insights into the relationships examined in introductory physics come from the study of symmetry breaks associated with tipping points [1], like when a pencil balanced on its tip chooses a direction to fall, a steam bubble in a pot of hot water on the stove chooses where to form, or a folks on a picnic in a meadow choose a clear patch in which to lay their blanket. Some particular dichotomies underpinning the content of courses on "mechanics, oscillations & heat" might be called, in order of their emergence in the natural history of invention, something like "relational vs. standalone", "here vs. now", "past vs. future", and "inside vs. outside".
In an extension of John Archibald Wheeler's It From Bit [2], the basic insight is that in Newtonian physics the idea of objects (e.g. as stand-alone projectiles) is very useful, but modern insights into nature on both large and small size & time scales suggests that it's an approximation that works on intermediate size & time scales only. Assertions about correlations between subsystems e.g. measured in log-probability terms using information units, including the relationship between our ideas and reality, end up having traction in practice on all scales.
For more on this, see for example wikpedia's page on relational quantum theories inspired by Louis Crane & Carlo Rovelli. Carlo Rovelli's Seven brief lessons on physics [3] in its own way frames the importance of the tri-fold connection between emergent views of gravitational spacetime, quantum mechanical indeterminacy, and the statistical approach to thermal/information physics. One rosetta stone in his view, somewhat less immediate than the connections that I've been interested in, is Stephen Hawking's tale about the quantum-mechanical heat/evaporation of black holes.
The traveler-point break from 4 to (3+1) vector symmetry in the observer experience (now discussed in the background section of our traveler point dynamics page as well as in this voicethread) is what makes time separate from space, and the concept of "now" different from the concept of "here". In other words, this gives traveling observers in spacetime their very own scalar time direction e.g. with (compared to the map-based spatial axes) its own units as well as its own measuring instrument designs.
Examples in everyday life where this separation between here and now becomes blurred include GPS systems, which have to correct for the fact that your head is aging faster than your feet, far away astrophysical events which "occur for us" long after the light which brings word of their happening is emitted, and electrons around atoms whose position can only be determined at any specific time by knocking them out of the atom.
The laws of physical dynamics may distinguish (for observers locally) between here and now, but they generally do not distinguish between past and future. In other words, time running backward is possible even if the events that result appear (as in the video of a window breaking) to be very unlikely.
The role of subsystem correlations (which raises its head in both wavefunction collapse and in thermal/information physics) then gives that scalar time its direction, because correlation between isolated subsystems fades stochastically in the forward time direction only. This delocalized correlation statement of "the 2nd law of thermodynamics" is e.g. how YOU can tell often if a video is being played backwards or forward. Subsystem correlations are likely to grow unexpectedly in the reverse time direction only, e.g. when components of a broken egg "reassemble" back into an unbroken one.
Causal connections (which are otherwise merely relational [4]) are then free to inherit this asymmetry. In other words, normal causes explain changes in subsystem correlation which unfold in the forward time direction only.
Our experience (and that of other lifeforms) unfolds in the forward time direction through a series of attempts to buffer those otherwise vanishing correlations (e.g. for metazoans like the subsystem correlations that look in/out from skin, family, and culture). This is one way to see the emergence of a scalar time direction in our experience, even though our dynamical laws are reversible.
Life as we know it, in that sense, is a layered hierarchy of correlation-buffering subsystems [5] [6] [7] on our planet's surface (including ourselves), powered by thermalization of available work mostly from our sun [8] [9]. Life's adaptation to that particular planetary surface runs deep, even if it as always is having to adapt e.g. to changes in population, climate, etc. This is one reason that migration off the planet's surface may not be as easy as it sounds [10].
In statistical mechanics and information theory, multiplicity is often a count of the effective number of states or choices. It is also the exponential form of surprisal, as illustrated in the equation S = k ln W on Ludwig Boltzmann's 1906 tomb, which may be solved for multiplicity W to define information units e.g. via the expression: #choices = W = eS/k = 2#bits.
Note that in this context both multiplicity (W) and surprisal in natural units (i.e. S/k) are fundamentally dimensionless, although the constant k in the latter case can be used to give surprisal differing units for different application areas. For example, in thermal physics it is traditional to set k equal to the Boltzmann constant e.g. in [J/K], while in communications k is often set to 1/ln2 to give surprisal in bits.
The question of the link between information entropy and thermodynamic entropy is a debated topic. In this discussion of multiplicities we treat them, regardless of their differences, as branches of the more general topic of Bayesian statistical inference [12] [13] designed for taking the best guess based on limited information.
Multiplicity, as a technical type of effective count or enumeration, is easiest to describe in systems which have a separable number (rather than a continuum) of allowed states.
Multiplicities, in finite discrete systems with all choices equally likely, are simply the number of choices i.e. the number of possible states. To open the door to their more sophisticated uses in the next section, it helps to define them in terms of probability p and/or the log-probability measure surprisal [14] s = k ln[1/p] where k is used to define a variety of "dimensionless units" like bits (k=1/ln[2]).
The discrete-system multiplicity obtained by simple counting is just w = 1/p = es/k. Hence the surprisal ⇔ probability ⇔ multiplicity inter-conversion is summarized by:
where in terms of dimensionless multiplicity w, the units for surprisal are bits if k = 1/ln[2] relation. Boltzmann's entropy S = k ln W, where W is the "number of accessible states" and k is Boltzmann's constant e.g. in J. W. Gibbs' microcanonical ensemble, also assumes that all states are equally probable.
If the states instead differ in their accessibility (each e.g. with fractional probability pi for i=1 to N), average surprisals and geometric-average multiplicities are used for applications in thermodynamics [15] and communication theory [16], with the result e.g. that entropy-first approaches are now standard in senior undergraduate [17] [18] [19] [20] and some introductory [21] thermal physics treatments. These are derived in terms of simple desiderata e.g. in books by E. T. Jaynes [12] and Phil Gregory [13].
The conversion between multiplicities and surprisals for a set of N possibly unequal but normalized probabilities then becomes:
Here normalization means that Σipi = 1, the "multiplicity" Wgeo term represents a geometric (i.e. multiplicative) average, and k is used to determine the dimensionless "information units" used to measure surprisal. Note that 0 ≤ Savg ≤ k ln[N], and 1 ≤ Wgeo ≤ N, with the maximum in both cases obtained when all states are equally probable.
A use of this in communications theory involves 1951-vintage Huffman coding, which underlies a wide range of lossless compression strategies, and at least inspires the 1977-vintage Lempel-Ziv compression used in ZIP files as well as the non-lossy compression strategies used for GIF and PNG images. The basic idea is that Huffman and related coding strategies replace characters in a message or image with a binary code that requires fewer bits for the less common characters, in effect resulting in a "new language" comprised of codewords that occur with near-equal probability and something like the minimum codeword length, which is equal (e.g. in bits) to none other than the surprisal average or Shannon entropy value given above! In fact, compression algorithms like this can even be used for entropy estimations in thermodynamic systems [22].
If one considers two probability distributions, p and q instead of just p, the previous quantity Savg may be written as Sp/p more generally. The interconversion for the average surprisal, uncertainty, or entropy associated with expected probability-set q, as measured by operating probability-set p, can then be written:
Although written here for discrete probability-sets, these expressions are naturally adapted to continuous as well as to quantum mechanical (i.e. root-probability wavefunction) probability-sets [23] [24].
Note that the upper limit on Sp/p (in bits) is ln2[N]. Also the fact that Sq/p ≥ Sp/p, i.e. that measurements using the wrong model q are always likely to be "more surprised" by observational data than those using the operating-model p, underlies least-squares parameter-estimation and Bayesian model-selection as well as the positivity of the correlation and thermodynamic availability measures discussed below.
Just as average surprisal S/k here is defined as Σifiln[1/fi] where fi are a set of normalized fractional probabilities, so following the prescription S = k ln W it's natural to define the effective multiplicity of choices as W ≡ eS/k = eΣifiln[1/fi = Πi(1/fi)fi. Even multiplicity logarithms are huge when one is applying statistical inference to macroscopic physical systems with Avogadro's number of components. However in statistical problems where the total number of choices W is small, the multiplicity approach may be more useful.
The two-distribution notation mentioned above opens the door a wider range of applications, by introducing an explicit reference probability into the discussion. The result in terms of surprisals is the "total correlation" measure known as Kullback-Leibler divergence, relative entropy, or "net surprisal". In statistical inference generally, and in thermodynamics in particular, maximizing entropy more explicitly involves minimizing KL-divergence with reference to a uniform prior [13]
Specific application areas include: (i) thermodynamic availability, (ii) algorithmic model selection, and (iii) the evolution of complexity. The surprisal ⇔ probability ⇔ multiplicity interconversion for these correlation analyses may be written:
Log-probability measures are useful for tracking subsystem-correlations in digital as well in analog complex systems. In particular tools based on Kullback-Leibler divergence IKL ≥ 0 and the matchup-multiplicities M = eIKL/k = Πi(pi/qi)pi associated with reference probability-set qi have proven useful: (i) to engineers for measuring available-work or " exergy" in thermodynamic systems [25] [26], (ii) to communication scientists and geneticists for studies of: regulatory-protein binding-site structure [27], relatedness [28], network structure, & replication fidelity [16] [29] [30], and (iii) to behavioral ecologists wanting to select from a set of simplifying-models the one which is least surprised by experimental data [31] [32] from a complex-reality.
These multi-moment correlation-measures also have 2nd law teeth making them relevant to quantum computing [33], and they enable one to distinguish pair from higher-order correlations making them relevant to the exploration of order-emergence in a wide range of biological systems [34]
The first physical science applications of multiplicity, including those by Ludwig Boltzmann, were likely in classical (i.e. non-quantum) statistical mechanics. Then, and more generally when parameters like energy, as well as position, velocity, etc., are allowed to take on a continuum of values, states that are uniformly distributed may allow one to connect multiplicity to a "volume" in that parameter (or "phase") space.
State probabilities then become differential probabilities e.g. per unit length, and thus dimensioned. This is a problem for surprisals and average surprisals [13] because logarithms require dimensionless arguments. It is not a problem for multiplicities or for net surprisals. The former doesn't use logarithms, and the latter uses probability ratios in its logarithms.
In classical thermodynamics the absolute size of multiplicity was also a mystery, even though it was generally not crucial to predictions. This is not usually a problem in other application areas, and the uncertainty principle in quantum mechanics has to a large extent solved that problem in statistical mechanics as well.
For example, imagine a monatomic ideal gas of N distinguishable non-interacting atoms in a box of side L for which per atom (ΔxΔp)3 ≥ h3 (or any other constant that may also depend on N), where the position-momentum product on the left hand side is using either the Heisenberg uncertainty principle or momentum quantization in an infinite-well potential to divide the continuum of possible states into a finite number dependent on Planck's constant h. Then using the Newtonian expression for momentum p in terms of total energy E and particle mass m we can write:
where V ≡ L3 is box volume. It is then a simple matter to calculate S = k ln W and show that the definition of reciprocal temperature or coldness δS/δE ≡ 1/T implies equipartition i.e. E = (3/2)NkT, and that the definition of free expansion coefficient [18] δS/δV ≡ P/T implies the ideal gas law i.e. PV = NkT.
This is a sandbox page for Thermochap. Thermochap ( talk) 22:30, 4 February 2021 (UTC)
Creating new pages is a more elaborate process than it used to be, but still worthwhile even it takes longer to do. Pages that might help include a page on:
Clues to emerging insights into the relationships examined in introductory physics come from the study of symmetry breaks associated with tipping points [1], like when a pencil balanced on its tip chooses a direction to fall, a steam bubble in a pot of hot water on the stove chooses where to form, or a folks on a picnic in a meadow choose a clear patch in which to lay their blanket. Some particular dichotomies underpinning the content of courses on "mechanics, oscillations & heat" might be called, in order of their emergence in the natural history of invention, something like "relational vs. standalone", "here vs. now", "past vs. future", and "inside vs. outside".
In an extension of John Archibald Wheeler's It From Bit [2], the basic insight is that in Newtonian physics the idea of objects (e.g. as stand-alone projectiles) is very useful, but modern insights into nature on both large and small size & time scales suggests that it's an approximation that works on intermediate size & time scales only. Assertions about correlations between subsystems e.g. measured in log-probability terms using information units, including the relationship between our ideas and reality, end up having traction in practice on all scales.
For more on this, see for example wikpedia's page on relational quantum theories inspired by Louis Crane & Carlo Rovelli. Carlo Rovelli's Seven brief lessons on physics [3] in its own way frames the importance of the tri-fold connection between emergent views of gravitational spacetime, quantum mechanical indeterminacy, and the statistical approach to thermal/information physics. One rosetta stone in his view, somewhat less immediate than the connections that I've been interested in, is Stephen Hawking's tale about the quantum-mechanical heat/evaporation of black holes.
The traveler-point break from 4 to (3+1) vector symmetry in the observer experience (now discussed in the background section of our traveler point dynamics page as well as in this voicethread) is what makes time separate from space, and the concept of "now" different from the concept of "here". In other words, this gives traveling observers in spacetime their very own scalar time direction e.g. with (compared to the map-based spatial axes) its own units as well as its own measuring instrument designs.
Examples in everyday life where this separation between here and now becomes blurred include GPS systems, which have to correct for the fact that your head is aging faster than your feet, far away astrophysical events which "occur for us" long after the light which brings word of their happening is emitted, and electrons around atoms whose position can only be determined at any specific time by knocking them out of the atom.
The laws of physical dynamics may distinguish (for observers locally) between here and now, but they generally do not distinguish between past and future. In other words, time running backward is possible even if the events that result appear (as in the video of a window breaking) to be very unlikely.
The role of subsystem correlations (which raises its head in both wavefunction collapse and in thermal/information physics) then gives that scalar time its direction, because correlation between isolated subsystems fades stochastically in the forward time direction only. This delocalized correlation statement of "the 2nd law of thermodynamics" is e.g. how YOU can tell often if a video is being played backwards or forward. Subsystem correlations are likely to grow unexpectedly in the reverse time direction only, e.g. when components of a broken egg "reassemble" back into an unbroken one.
Causal connections (which are otherwise merely relational [4]) are then free to inherit this asymmetry. In other words, normal causes explain changes in subsystem correlation which unfold in the forward time direction only.
Our experience (and that of other lifeforms) unfolds in the forward time direction through a series of attempts to buffer those otherwise vanishing correlations (e.g. for metazoans like the subsystem correlations that look in/out from skin, family, and culture). This is one way to see the emergence of a scalar time direction in our experience, even though our dynamical laws are reversible.
Life as we know it, in that sense, is a layered hierarchy of correlation-buffering subsystems [5] [6] [7] on our planet's surface (including ourselves), powered by thermalization of available work mostly from our sun [8] [9]. Life's adaptation to that particular planetary surface runs deep, even if it as always is having to adapt e.g. to changes in population, climate, etc. This is one reason that migration off the planet's surface may not be as easy as it sounds [10].
In statistical mechanics and information theory, multiplicity is often a count of the effective number of states or choices. It is also the exponential form of surprisal, as illustrated in the equation S = k ln W on Ludwig Boltzmann's 1906 tomb, which may be solved for multiplicity W to define information units e.g. via the expression: #choices = W = eS/k = 2#bits.
Note that in this context both multiplicity (W) and surprisal in natural units (i.e. S/k) are fundamentally dimensionless, although the constant k in the latter case can be used to give surprisal differing units for different application areas. For example, in thermal physics it is traditional to set k equal to the Boltzmann constant e.g. in [J/K], while in communications k is often set to 1/ln2 to give surprisal in bits.
The question of the link between information entropy and thermodynamic entropy is a debated topic. In this discussion of multiplicities we treat them, regardless of their differences, as branches of the more general topic of Bayesian statistical inference [12] [13] designed for taking the best guess based on limited information.
Multiplicity, as a technical type of effective count or enumeration, is easiest to describe in systems which have a separable number (rather than a continuum) of allowed states.
Multiplicities, in finite discrete systems with all choices equally likely, are simply the number of choices i.e. the number of possible states. To open the door to their more sophisticated uses in the next section, it helps to define them in terms of probability p and/or the log-probability measure surprisal [14] s = k ln[1/p] where k is used to define a variety of "dimensionless units" like bits (k=1/ln[2]).
The discrete-system multiplicity obtained by simple counting is just w = 1/p = es/k. Hence the surprisal ⇔ probability ⇔ multiplicity inter-conversion is summarized by:
where in terms of dimensionless multiplicity w, the units for surprisal are bits if k = 1/ln[2] relation. Boltzmann's entropy S = k ln W, where W is the "number of accessible states" and k is Boltzmann's constant e.g. in J. W. Gibbs' microcanonical ensemble, also assumes that all states are equally probable.
If the states instead differ in their accessibility (each e.g. with fractional probability pi for i=1 to N), average surprisals and geometric-average multiplicities are used for applications in thermodynamics [15] and communication theory [16], with the result e.g. that entropy-first approaches are now standard in senior undergraduate [17] [18] [19] [20] and some introductory [21] thermal physics treatments. These are derived in terms of simple desiderata e.g. in books by E. T. Jaynes [12] and Phil Gregory [13].
The conversion between multiplicities and surprisals for a set of N possibly unequal but normalized probabilities then becomes:
Here normalization means that Σipi = 1, the "multiplicity" Wgeo term represents a geometric (i.e. multiplicative) average, and k is used to determine the dimensionless "information units" used to measure surprisal. Note that 0 ≤ Savg ≤ k ln[N], and 1 ≤ Wgeo ≤ N, with the maximum in both cases obtained when all states are equally probable.
A use of this in communications theory involves 1951-vintage Huffman coding, which underlies a wide range of lossless compression strategies, and at least inspires the 1977-vintage Lempel-Ziv compression used in ZIP files as well as the non-lossy compression strategies used for GIF and PNG images. The basic idea is that Huffman and related coding strategies replace characters in a message or image with a binary code that requires fewer bits for the less common characters, in effect resulting in a "new language" comprised of codewords that occur with near-equal probability and something like the minimum codeword length, which is equal (e.g. in bits) to none other than the surprisal average or Shannon entropy value given above! In fact, compression algorithms like this can even be used for entropy estimations in thermodynamic systems [22].
If one considers two probability distributions, p and q instead of just p, the previous quantity Savg may be written as Sp/p more generally. The interconversion for the average surprisal, uncertainty, or entropy associated with expected probability-set q, as measured by operating probability-set p, can then be written:
Although written here for discrete probability-sets, these expressions are naturally adapted to continuous as well as to quantum mechanical (i.e. root-probability wavefunction) probability-sets [23] [24].
Note that the upper limit on Sp/p (in bits) is ln2[N]. Also the fact that Sq/p ≥ Sp/p, i.e. that measurements using the wrong model q are always likely to be "more surprised" by observational data than those using the operating-model p, underlies least-squares parameter-estimation and Bayesian model-selection as well as the positivity of the correlation and thermodynamic availability measures discussed below.
Just as average surprisal S/k here is defined as Σifiln[1/fi] where fi are a set of normalized fractional probabilities, so following the prescription S = k ln W it's natural to define the effective multiplicity of choices as W ≡ eS/k = eΣifiln[1/fi = Πi(1/fi)fi. Even multiplicity logarithms are huge when one is applying statistical inference to macroscopic physical systems with Avogadro's number of components. However in statistical problems where the total number of choices W is small, the multiplicity approach may be more useful.
The two-distribution notation mentioned above opens the door a wider range of applications, by introducing an explicit reference probability into the discussion. The result in terms of surprisals is the "total correlation" measure known as Kullback-Leibler divergence, relative entropy, or "net surprisal". In statistical inference generally, and in thermodynamics in particular, maximizing entropy more explicitly involves minimizing KL-divergence with reference to a uniform prior [13]
Specific application areas include: (i) thermodynamic availability, (ii) algorithmic model selection, and (iii) the evolution of complexity. The surprisal ⇔ probability ⇔ multiplicity interconversion for these correlation analyses may be written:
Log-probability measures are useful for tracking subsystem-correlations in digital as well in analog complex systems. In particular tools based on Kullback-Leibler divergence IKL ≥ 0 and the matchup-multiplicities M = eIKL/k = Πi(pi/qi)pi associated with reference probability-set qi have proven useful: (i) to engineers for measuring available-work or " exergy" in thermodynamic systems [25] [26], (ii) to communication scientists and geneticists for studies of: regulatory-protein binding-site structure [27], relatedness [28], network structure, & replication fidelity [16] [29] [30], and (iii) to behavioral ecologists wanting to select from a set of simplifying-models the one which is least surprised by experimental data [31] [32] from a complex-reality.
These multi-moment correlation-measures also have 2nd law teeth making them relevant to quantum computing [33], and they enable one to distinguish pair from higher-order correlations making them relevant to the exploration of order-emergence in a wide range of biological systems [34]
The first physical science applications of multiplicity, including those by Ludwig Boltzmann, were likely in classical (i.e. non-quantum) statistical mechanics. Then, and more generally when parameters like energy, as well as position, velocity, etc., are allowed to take on a continuum of values, states that are uniformly distributed may allow one to connect multiplicity to a "volume" in that parameter (or "phase") space.
State probabilities then become differential probabilities e.g. per unit length, and thus dimensioned. This is a problem for surprisals and average surprisals [13] because logarithms require dimensionless arguments. It is not a problem for multiplicities or for net surprisals. The former doesn't use logarithms, and the latter uses probability ratios in its logarithms.
In classical thermodynamics the absolute size of multiplicity was also a mystery, even though it was generally not crucial to predictions. This is not usually a problem in other application areas, and the uncertainty principle in quantum mechanics has to a large extent solved that problem in statistical mechanics as well.
For example, imagine a monatomic ideal gas of N distinguishable non-interacting atoms in a box of side L for which per atom (ΔxΔp)3 ≥ h3 (or any other constant that may also depend on N), where the position-momentum product on the left hand side is using either the Heisenberg uncertainty principle or momentum quantization in an infinite-well potential to divide the continuum of possible states into a finite number dependent on Planck's constant h. Then using the Newtonian expression for momentum p in terms of total energy E and particle mass m we can write:
where V ≡ L3 is box volume. It is then a simple matter to calculate S = k ln W and show that the definition of reciprocal temperature or coldness δS/δE ≡ 1/T implies equipartition i.e. E = (3/2)NkT, and that the definition of free expansion coefficient [18] δS/δV ≡ P/T implies the ideal gas law i.e. PV = NkT.