September 8 Information

If we define amount of digital information in bits 0/1, how can we define amount of analog information?

When you convert any analog signal into digital (taping a human talking or taking a picture of a landscape) can you calculate the amount of information lost? In case of audio, for example, through sampling you could get a file of n MB, but it's there a point when you can say you captured all the sound? I don't mean all meaningful sound, but all the sound? It's clear that for us humans there's a point when we can't hear any difference anymore. Bumptump ( talk) 09:03, 8 September 2022 (UTC) reply

I think what you are after is the Nyquist–Shannon sampling theorem. This tells you the sampling rate needed to capture information up to a certain frequency. For audio a sampling rate of 44,100 Hz is often considered as capturing all the information, as it allow frequencies within the human hearing rage of 20-20,000 Hz to be perfectly reconstructed, though as the article states some humans can hear sounds above that range, and reconstruction would require a perfect low-pass filter, so in theory some people (particularly children) could hear the artefacts produced by this rate of sampling. For the maximum possible frequency of sound (regardless of human hearing) you would need someone more knowledgeable than me to verify but some non-authorities sources suggest a limit of [ 3GHz for sound travelling in air], which would require a 6Ghz sampling frequency. -- Q Chris ( talk) 10:20, 8 September 2022 (UTC) reply

Is it just audiophile nonsense that because 44.1k can't accurately reproduce the timbre of the 20k then a human could tell the difference? Sagittarian Milky Way ( talk) 12:04, 8 September 2022 (UTC) reply

A great question. My understanding is that unless your ears can react to frequency changes over 20k then it will not be able to detect any differences in timbre. However some people, particularly children and younger people, can hear frequencies above 20k in ideal laboratory conditions, humans can hear sound as low as 12 Hz and as high as 28 kHz, though sounds at this level are greatly attenuated. I would say that I could definitely not hear any difference at over 60 years old, I think it possible that a young person could. That's why dvd audio and a lot of other formats support a higher bitrate. A sampling rate of over 56khz should cover everyone though. -- Q Chris ( talk) 13:31, 8 September 2022 (UTC) reply

So in the dark all cats are gray and in the highest pitch you can hear all sounds sound the same. Which would've been my guess but I wasn't sure. Sagittarian Milky Way ( talk) 16:48, 8 September 2022 (UTC) reply

Some more info here: Sample-rate conversion. 41.23.55.195 ( talk) 10:25, 8 September 2022 (UTC) reply

The information content of an analog signal is not only restricted by channel bandwidth but also by the signal to noise ratio. If you had an continuous analog signal in the range 0V to 31V changing slowly enough that bandwidth were irrelevant but with 1V of noise, you wouldn't be able (roughly speaking) to distinguish more than 32 different values corresponding to 5 bits of information.

Of course although the same is true whenever a digital signal is transmitted or stored in an analog medium (ie all media), the explicit quantization of digital signals is typically so far above the noise threshold that it becomes a relatively minor effect.

2A01:E34:EF5E:4640:64E1:E668:BA81:4F87 ( talk) 13:58, 8 September 2022 (UTC) reply

• I think the above discussion (with the exception of 2A01:...’s message) strayed far from the OP’s question (even though it is interesting on its own).

In general, a single point of analog signal is a real number, which is an infinite amount of information. Hence, there is no mathematical way to encode it digitally without loss. If you start considering specific cases (such as: a sound recording that will be replayed to human ears, or a transmission of a radio signal subject to noise during transmission), then we can discuss how much loss is acceptable. Tigraan ^{Click here for my talk page ("private" contact)} 15:20, 8 September 2022 (UTC) reply

Physical analog systems are still constrained by reality, however. A very large number is not the same as infinite, and even analog systems are constrained by the very real limits of the physical universe, i.e. quantum effects. Information theory still has to contend with the fine structure of the universe, which is not infinitely continuous. Things like the Bekenstein bound and Bremermann's limit and the like exist for a reason; even analog systems which are constrained in physical time and space cannot contain infinite information. The information in one system bound by space and time can be stored in an another system bound by space and time; though the storage of that information may be much larger than the information of the system it is storing, it can always do so. The thing that makes a Turing machine do its magic is its infinite nature. The ability to faithfully reproduce an analog signal is an engineering problem, not a physics problem. You only need to preserve information to the resolution of your ability to detect; no physical object has infinite resolution, and no analog signal is infinitely continuous (again, because quantum), so there is no theoretical problem with converting an analog signal to digital at a fundamental level, given the constraints of physics. -- Jayron 32 15:31, 8 September 2022 (UTC) reply

I endorse Jayron32's explanation in general, but I'm stopping by only to quibble with the short-hand notation "... because quantum..." I believe that Jayron is absolutely correct in spirit, but we can use slightly less informal verbiage to explain what he's conveying. Something about the way he quips, "because quantum!" ... well, that snagged my attention! Maybe it's something to do with the way that physicists try to answer "why?"... Because!

No analog signal is infinitely continuous because even the most fundamental elements of our universe are quantized. The important parameters like mass, distance/position, time, energy, and charge are quantized: there is a smallest-possible amount for these parameters. In a sense, the first quantization tells us that some stuff is quantized; the second quantization formally expresses that everything else is also quantized, too, also. And of course, this indubitably leads us straight into canonical quantization - or, worded informally: "yeah... even the other stuff also is additionally quantized, too." There does not exist anything that is not quantized. This last bit is more painful than most people really appreciate until they try to solve equations with it. And yet, ... this is the only mathematical formalism that is consistent with our experiments: if there were some other thing, as-yet-unobserved, that is not quantized, hiding information, ... we'd know about it.

And yet... this (in)formalism doesn't really explain "why": it simply describes the universe as we observe it... so, we can probably cut everyone a little slack here, and use a bit of a flippant answer: "because!"

We might say that a better answer about why an analog signal contains finite information really has more to do with our operational definition for the words: "signal" - "information" - "contain" - these words mean something specific in this context - and that is "why" the analog signal contains finite information! If we take this approach, we answer "why" with a purely a linguistic explanation, and not a scientific one!

We'll set aside any pedagogical considerations about shouting "because!" to end an otherwise-endless-stream-of-questions... Why is it okay to dodge the pedagogical considerations here today? Well, ... because! Nimur ( talk) 17:57, 8 September 2022 (UTC) reply

I wasn't trying to say "because!" meaning the answer didn't exist, or wasn't a valid topic of exploration in general. It certainly is. But the details of that answer are not strictly necessary (nor possible to go into and not lose one's audience in the weeds while trying to write a digestible and yet accurate response) to go in to. Analog signals do not contain infinite information. We even have ways of calculating exactly how many bits of information they may contain. Since that is finite, it is not merely theoretically possible, but actually, honest-to-goodness possible for another finite system to contain that information. You can perfectly encode all of the information in an analog signal in a purely digital one; though the resources to do so may be prohibitive, that's an engineering issue, not a physical one. That was my point. Your point about the details of why quantum mechanics provides these finite limits on the information carried by analog signals is a fascinating area of exploration, but as long as we presume we understand the absolute basic tenets of quantum mechanics, that all things are quantized (exist in discrete bits that can't be further subdivided) then that's enough for my point. You're not wrong that more details could have been provided, but more details can always be provided. It's a narrative choice on my part as to which details are strictly necessary. -- Jayron 32 19:08, 8 September 2022 (UTC) reply

Yeah. In any event, maybe I ought to apologize: I think on re-reading my response, I sounded more contrarian than I had intended. I was trying to convey that your explanation was good - it was pretty on-the-nose. Nimur ( talk) 20:50, 8 September 2022 (UTC) reply

In my neck of the woods, "on the nose" means stinking, disliked, unwanted. Is this what you meant? Because it would be in stark contrast with "good". -- Jack of Oz ^{[pleasantries]} 22:42, 8 September 2022 (UTC) reply

https://en.wiktionary.org/wiki/on_the_nose Modocc ( talk) 23:33, 8 September 2022 (UTC) reply

Thanks. -- Jack of Oz ^{[pleasantries]} 09:06, 9 September 2022 (UTC) reply

Eep! I apologize again! I intended the idiom to have its American connotation (exact and precise), and I had never before heard of the Australian interpretation which is entirely different! Thank you for informing me - I do try to make sure I'm aware of the various and different cultural connotations of the language and word choice that I use! Anyway... sorry! Nimur ( talk) 23:18, 10 September 2022 (UTC) reply

No apologies required. I've learned about the other meanings. For the meaning you intended, we're more likely to say "hit the nail on the head", or "on the money". Unpopular governments that are facing probable electoral extinction (and that means virtually all of them, ultimately) are routinely said to be "on the nose". -- Jack of Oz ^{[pleasantries]} 00:02, 11 September 2022 (UTC) reply

Unusually, the USA and Britain are aligned in this. Wiktionary:on the nose suggest that the smell meaning is an Australian peculiarity. Alansplodge ( talk) 10:27, 13 September 2022 (UTC) reply

Interesting. Yes, we are a peculiar people. -- Jack of Oz ^{[pleasantries]} 00:59, 14 September 2022 (UTC) reply

Did you know the national anthem of baseball has the lyrics root, root, root for the home team? I hear that means f**k in Oz but not England or the States. Or is it more like screw or bang? Sagittarian Milky Way ( talk) 01:36, 14 September 2022 (UTC) reply