14—
The Entropy of Interacting Populations:
With C. J. Everett (LA-4256, August 1969)
This report is a novel probabilistic approach to defining distributions of functionals of thermodynamical systems, including for instance, interactions between radiation and particles. (Author's note).
Abstract
A study is made of interacting populations of "particles" which closely parallels the Boltzmann kinetic theory and the Planck-Einstein-Tolman treatment of radiation interacting with matter. The analogy is perhaps surprising since it appears that our postulates do not embody the physics of such systems, but are nevertheless quite reasonable, and applicable to similar situations. While we have deliberately used the language of physics for its intuitive appeal, one may well consider for example the implications of replacing "particles" by "people" and "energy" by "wealth." It is especially interesting that the "reversibility paradox" is excluded by confining the discussion to a scalar "energy" rather than a vector "velocity." As Boltzmann might have said, "Go ahead, reverse the energy."1
I—
The System of "Particles"
1. The Interaction Postulates. We consider a system of "particles," of which there are N(E, t)dE possessing "energy" on (E, E+dE), 0 <E < oc, at time t > 0, and undergoing c(E1, E2)N(E, t)dE1N (E2,t)dE2 "collisions," or binary El, E2-interactions per unit time, between particles on the indicated ranges, where c(El, E2) > 0 is a function of form f(Ei + E2). We further assume that the (probable) number Q(E1, E2, E)dE of particles emerging on (E, E + dE) from such a collision has the properties
V1. V)(El,E2, E) > 0; 0 < E < E1 + E2 , 0 < (El&E2) < ox 02. O (E1,E2, E)= (E2, El, E) 33. (E1, E2, E) = ((E, E1 + E2 - E, E1) Eil+E 24. / (E1,E2, E)dE = 2. Jo From 42, 4'3 follows the symmetry 05. O(E 1, E2, E) = (E1, E2, E1 + E2,-E) of the p-distribution about its midpoint, and hence from 44 follows El +E2 ?6. fEO(E1, E2, E)dE = E1 + E2
The uniform distribution '(E 1,E2, E) = 2 /(E1 + E2) is by no means the only one having these properties. For example, 4(E1 + E2)- 1 sin2 27r (El + E2)-l (El + E), 0 < (E1&E) < (E1 + E2)/2, symmetrically extended to the full interval (0, E1 + E2), yields such a p-function (cf. Appendix I).
2. The Boltzmann Equation. The above assumptions imply the equation ON(E)/at - c(E, E 2)N(EI)N(E 2)f(E, E2 E)dEldE 2E+E2>E 2 - c(E, E2)N(E)N(E2)dE2=B- B 2B 3(1)
for the change of N(E, t) with time. (N.B. Hereafter all functions N(E, t) are written N(E) for brevity.)
For a solution of (1), we verify at once the conservation laws d 1" d 0° d- N(E)dE = 0 = -EN(E)dE dt Jo at Jo showing the values No,Eo of these integrals to be constant in time.
This rests on the fact that the properties of c and Q imply 0 B3dE = = EB3(E)dE, (2) that is to say j BidE = j B 2dE and a EBi(E)dE = EB2(E)dE. (3) For, these B1 integrals may be written in the form /oo /00 El+E2 JE1 JE20JE=0 and the relations in (3) follow from 44, and from V6.
We next throw (1) into the equivalent form oo) EE+ E2 1 ON(E)/t== E 2-c(E, E2))(E, E2, El) (4) E2=0 JE1=0 2 {N(E1)N(E + E2- El) - N(E)N(E2)}dEidE2 -B 1- B2 _B 3.
To see this, one makes the transformation E1= F1, E2= E + F 2 - F 1 in B 1, using c(E,E2) = f(E1+ E2) and 03, and uses V4 to express B2 as a double integral.
3. The Boltzmann H-function. For a solution N(E) of (1,4) we now define (ignoring the nice question of units) HN(t) = N(E) log N(E)dE, Jo
for which dUNC- HN(t) = EN(E)/Qt log N(E)dE= 1 B3 log N(E)dE
(since No is invariant). Using the form of B3 in (4), and making the change of notation (E, El) -- (E1, E), the last integral becomes roo oo rEl+E21 1=/ / / -c(E,E2,) (E, E2,E) JE1=O JE2=O JE=O 2 {N(E)N(E1+ E2- E)- N(E1)N(E2)} log N(E1)dE 2dE 1
Due to the symmetry in E1, E2, this is equal to the same integral with N(E2) replacing N(E1) in the log factor. Averaging the two results, we obtain 0fooo froo E1+E 2 1 B3 logNdE= / / -c(El,E2, )(E, E2, E) =0 JE=o JE= JE=O 4 {N(E)N(E1+ E2,-E)- N(E1)N(E2)} log N(El)N(E2)dEdE2dE.
If we now make the transformation E1 = F, E2 = F1+ F2 - F, E = F1 , use c(E1,E2) = f(E1 + E2) and 03, and change notation (F, F2 , F) -+ (E, E2, E) we obtain the same formula with the log factor replaced by - log N(E)N(E 1+ E 2- E). Averaging once more, we obtain finally
HN(t) = j aN(E)/Ot log N(E)dE = B3 log N(E)dE roo foo fEl+E2IN(E)N(E1+ O - E)- N()N(2)} (5) {N(E)N(Ei + E2-E) -N(Ei)N(E2)} (5) log N(E)N(E1 + E2 - E)dE2dE N(E1)N(E2) with equality iff (at the time t in question), {N(E)N(E 1+ E2 - E) - N(E1)N(E2)}0-0; O<E<Ei + E2<oc. (For, c > 0, 0 > 0, and x - y < (>)0 as log x/y < (0) .
Moreover, it is well known that a (continuous) function N(E) satisfies (6) iff it is of form 3e-~E (cf. Appendix II).
4. The Steady State Solution. If No(E) is a time independent function, the following conditions are now seen to be equivalent:
S1. No(E) satisfies (4) (steady state solution) S2. For N = No(E), B3 (E) _ 0 in (4) S3. For N = No(E), B3s (E) log N(E)dE = 0 in (5) S4. For N = No(E), {N(E)N(E1+ E2 - E) - N(E 1)N(E 2)}- 0 in (6) S5. No(E) = Pe-" E (with a =No/Eo, 3 = Noa if we stipulate the totals No, Eo).
It is here apparent that we do not have Boltzmann statistics, for which No(E)dE = 3E1/2 e- E, with a = (3/2) (No/Eo) and d = 2Noa3 /2 /7l1 /2 (see ref. 2).
5. Time Dependent Solutions. If N(E, t) is a solution of (4), with invariants No, EO, which is not the solution No(E) in S5 at any t, then HN(t) = fN(E) log N(E)dE Jo is strictly decreasing, since HN(t) < 0, t> 0, in (5). Moreover, we shall now prove HN(t)>HNO; t>0 (7) and hence the existence of lim HN(t) - HN> HNo. To verify (7), note first that, if M(E) is either N(E) or No(E), then J M log NodE = M (log 3 - aE)dE = No (log - 1), because both have the same totals No, Eo. Since these integrals have the same value, we have HN(t) - HNO= N log NdE- j No log NodE = {N log N/No + No - N} dE > 0. Jo
(For the integrand is of form f(x) = x log x/xo+xo-x, with f(xo) = 0, and f'(x) = log x/xo<(>) 0 as x <(>) xo. Thus HN(t)- HNO > 0, and if equality held for any to, we should have N(E, to) - No(E).)
If, in addition, N(E, t) is sufficiently well-behaved, with the limits lim HN(t) = 0, and lim N(E, t) = N*(E), we may conclude from (5) t-—+ct+00 that N*(E) = No(E), i.e., the time dependent solution N(E,t) of (4) approaches the steady state.
We do not investigate the existence of these limits; apparently they have not been established even in the simplest kinetic theory. (See however ref. 1.) It is clear that the second limit implies the first, since (5) then shows the existence of lim Hv(t) = C, and by the theorem t-—oo of the mean, HN(T) = (HN(t +A t)- HN(t))/At - (H N-Hk)/At 0= C.
II—
A Linked System of "Particles" and "Photons"
6. Interaction Assumptions. We now consider a system of "particles" and "photons," of which there are respectively N(E, t)dE and N(E, t)dE on (E, E + dE) at time t> 0, 0 < E < oo. Particles are subject to the rules of §1, while photon-particle interactions are governed by two positive functions A(E2, El), B(E2, El), 0 < E1 < E2 < oo, according to the Einstein postulates:
P1. N(E2 ,t)dE2B(E, E2)N(E - E2,t)dE gives the number, per unit time, of (E2, E2 + dE2) particles raised to (E, E + dE) by absorption of an (E - E2)-photon; 0 < E2 < E < oo.
P2. N(E2, t)dE2B(E2, E)N(E2- E, t)dE gives the corresponding number "induced" by the presence of (E2- E)-photons to drop to (E, E + dE) with creation of such a photon, and
P3. N(E2 ,t)dE2A(E2, E)dE gives the number of such particles spontaneously decaying to (E, E + dE), with creation of an (E2 - E)-photon, 0 < E < E2 < oc.
P4. The functions A, B are related by the equation A(E2, E) -= B(E2, E1)R(E2- E1) where R is the function of the energy difference. (In the Planck-Einstein case, R(F) = 87r(hc)-3 F2 , and N, N are numbers per cm3 .)
7.The "Boltzmann-Einstein Equation." The analogue of (1) in §2 is seen to be the linked system (cf. (4) for B3)
E aN(E)/Ot =B3 +N(E 2)B(E, E 2)N(E - E 2)dE 2fo + N(E2){B(E 2, E)N(E 2 - E) + A(E 2, E)}dE 2-J N(E)B(E 2, E)N(E 2 - E)dE 2E - N(E){B(E, E 2)N(E - E 2) + A(E, E 2)}dE 2Jo ON(F)/at = - N(E2)B(E 2+ F, E 2)N(F)dE 2Jo + j N(E 2){B(E 2, E 2- F)N(F) + A(E 2, E 2- F)}dE 2.
Combining integrals, bringing all lower limits to 0, making an inversion on the proper integral resulting, and using P4, one can show that aN(E)/atE B3- B(E, E - F){N(E)[N(F) + R(F)]-N(E - F)N(F)}dF Jo + B(E + F, E){N(E + F)[N(F) + R(F)] - N(E)N(F)}dF _ B3- B4 + B5 (8) aN(F)/at =J B(E+F,E){N(E+F)[N(F)+R(F)]-N(E)N(F)}dF _ B6 . We next verify, for a solution N, N of (8), the relations d /N(E)dE = 0=d { /CEN(E)dE +j FN(F)dF and hence the invariance of the particle number No, and of the total energy £o= Eo+ Eo. For the first, we have to prove B 3dE - B 4dE+ / B 5dE = 0 . But the first integral is zero by (2), and making the transformation E = E' + F', F = F' (9)
on the second shows it equal to the third. For the second relation we must show that 00oo /oo j EB3dE -EB4dE + EB 5dE + FB 6dF= 0. Here, the first is zero by (2), and the transformation (9) on the second makes the result clear.
8. The H-function. For a solution N, N of (8) we define3HN,(t)- j N(E) log N(E)dE + {N(F) log N(F) - [N(F) + R(F)] log [N(F) + R(F)]}dF with derivative d HN (t) = t ON/Ot log NdE + ON/Ot log NdF —-t / N/( t N+R = Bj B log NdE -B 4 log NdE (10) +FB5 log NdE +B6 log - dF. Jo Jo N +R
We know the first integral from (5), and making the transformation (9) on the second shows (10) to be HNV(t) = B3 log N(E)dE - J B(E + F,E){N( E + F )[ N(F) + R(F)] ( -N(E)N(F)} log N(E +F)(F)R(F) N(E)N(F) with equality iff (at the time t in question), both {N(E)N(E1+E2-E)-N(E1)N(E2)}- 0; 0 < E <E1 +E2< oo and {N(E+F)[N(F)+R(F)] -N(E)N(F)} 0; 0 < (E&F) < oo . (12)
(cf. §3.) As we have seen, the first of these implies N = 3e- ', and hence from the second follows N = R(F)(e"F - 1)-1 . Conversely, such a pair satisfy (12).
9. The Steady State. For a pair of time independent functions No, No, we now find that the following are equivalent: S'i. No, N satisfy (8) (steady state solution) S'2. For N = No(E), N = No(F), B3(E) - B4(E) + Bs(E) - 0 - B6(F) in (8) . S'3. For N = No(E), N = No(F), / { B3(E)- B4(E) + B5(E)} log N(E)dE + B6(F) log (F)) dF = 0 in (10), (11) Jo .( , N(F) + R(F) S'4. For N = No(E), N = No(F), {N(E)N(E1 + E 2- E) - N(E1)N(E 2)} 0 and {N(E + F)[N(F) + R(F)] - N(E)N(F)} 0 in (12). S'5. No(E) = 3e- E, No(F) = R(F)(eF- 1)-
Here, stipulation of the totals No, So determines ca, 3, and hence also No, Eo, Eo. If we take the special function R(F) = 8r(hc)-3F2 , we find again a = No/Eo, /3 = aNo, and No = 167r((3)(hc)-3 3-3 , Eo = 8/15 7r(hc)-3 a-4 . Note that for us, 1/a = Eo/No and not (2/3)Eo/No (=kT).
10. Time Dependent Solutions. If N(E, t), N(E, t) is a solution of (8), with invariants No, o, which is not the solution of S'5 at
any time t, then HN,N'(t) is strictly decreasing, and bounded below by HN ,, as we now show.
Note first that No(F) = R(eF - 1)-I implies No + R = NoeaF. Hence, if M, M is either the pair N, N or the pair No, No with the same two invariants, we find
J M log NodE +M log NodF -(M + R) log (No + R)dF = j M (log 3- oE)dE + M log NodF - M (log No + aF)dF - R log (No + R)dF = No log 3- Eo -R log (No + R)dF.
The integrals having this common value, we may write HN,N(t) - HN,o/N log NdE + N log NdF - (N + R) log (N + R)dF - { jNo log NodE+ No log NodF - (No + R) log (No + R)dF = log NNdE log N/NdEN log N/NodF Jo Jo -I (N + R) log RdF Jo N0+R = J{N log N/No + No - N}dE +0 {N log TN/No - (N + R) log N+R}dF> 70LNo +R >
We have seen (§5) that the first integrand is non-negative. The second, of form g(y) = y log y/yo - (y + R) log y + R yo + R has g(yo) = 0, and g'(y) = log (1 R/yo) - log (1 + R/y) <(>)0 as y< (>)yo and thus is also > 0. Hence HNy,(t) > HN N, by the argument of §5, and we have the existence of lim HN (t) H - > t- oo i, N,N No,NO
If we assume the limits lim H' -(t) = 0, lim N(E,t) = N*(E), t-+ooN,N t-ooc lim N(F,t)= N*(F), then from (11) follows t-*oo N*(E) = No(E) and N*(F) = No(F) as defined in S'5. The remark at the end of §5 applies here as well.
Appendix I
It is apparent that definition of a G-function ?((E1, E2, E) is equivalent to defining a function f(Ei, S, E) such that fl. f(El,S,E) > 0; 0 < (E1&E) < S < oo f2. f(E1,S,E) = f(S- El, S,E) f3. f(El,S,E)= f(E,S,El) f4. f(E1,S, E)dE= 2.
For, f(E1 ,S,E) =- (E1,S - E, E) and E(E1,E2, E) f(E1, E1 + E2, E) serve to define each in terms of the other.
Moreover, given an f-function, the function h(E1, S,E) f(E1, S, E) for E1, E on (0, S/2) satisfies hi. h(E, S,E) >0 h3. h(E, S,E) = h(E,S, El) s/2 h4. / h(E 1, S, E)dE = 1. Jo
Conversely, given such an h-function defined for all E1, S, E with 0 < (E1&E) < S/2 < oo, we may extend h to an f-function by the consistent definitions f(El, S, E) =h(E1, S, E), E1E(0, S/2),Ee(O, S/2) h(S - El, S, E), Eli(S/2, S), Ee(0, S/2) h(E, S, S - E), Ee(O, S/2), Ee(S/2, S) h(S- E1,S, S- E), Ele(S/2, S), Ee(S/2, S).
The function h(Ei, S, E) = 4/S sin2 27r(E1 +E)/S satisfies hi, 3,4 and therefore defines a p-function, as indicated.
Appendix II
Obviously N(E) in (7) satisfies (6). Conversely, setting E = 0 in (6), and L(E) = log (N(E)/N(O)), we must have L(E1 + E2) =L(E) + L(E2 ); El, E2 > . Then for integers m,n > 1, L(m 1) = mL(1), L(1)=L(n-) =nL(), and V n\n L () = L( m - ) = mL ) =-L(1). \n n)n n If fractions m/n -- E > 0, we have L(E) = L (im n) = lim L - = lim m L(1) = L(1)E Ln nlim) n) n = log (N(E)/N(O)). Thus N(E) = N(O) exp L(1)- E = oe-E.
References
1. M. Kac, Probability and Related Topics in Physical Sciences, (1959) Ch. III, Interscience Publishers, Ltd., London. 2. E. H. Kennard, Kinetic Theory of Gases, (1938) Ch. II, McGrawHill Book Co., Inc., New York. 3. R. C. Tolman, Statistical Mechanics, (1927) pp. 198-203, Chemical Catalogue Co., Inc., New York.