Hi! I'm Chris Horvath, the author of Flicks, formerly of Facebook/Oculus, currently at Cruise Automation. I'm happy to answer any questions about my most-talked-about-single-line-of-code.
"A flick (frame-tick) is a very small unit of time. It is 1/705600000 of a second, exactly.
1 flick = 1/705600000 second
This unit of time is the smallest time unit which is LARGER than a nanosecond, and can in integer quantities exactly represent a single frame duration for 24 Hz, 25 Hz, 30 Hz, 48 Hz, 50 Hz, 60 Hz, 90 Hz, 100 Hz, 120 Hz, and also 1/1000 divisions of each, as well as a single sample duration for 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 32 kHz, 44.1 kHz, 48 kHz, 88.2 kHz, 96 kHz, and 192kHz, as well as the NTSC frame durations for 24 (1000/1001) Hz, 30 * (1000/1001) Hz, 60 * (1000/1001) Hz, and 120 * (1000/1001) Hz.*
That above was one hell of a run-on sentence, but it's strictly and completely correct in its description of the unit.
This makes flicks suitable for use via std::chrono::duration and std::ratio for doing timing work against the system high resolution clock, which is in nanoseconds, but doesn't get slightly out of sync when doing common frame rates."
The reason you don't see flicks used all that much is because media encoding is incredibly proprietary in the space where it matters most, live broadcast. Selling into the broadcast space, every broadcaster has petabytes of video in their own chosen format. You have to support EVERY format to sell broadly, and they're not going to let you transcode everything into your format. Using flicks gives you the ability to support as many combinations as possible with the least effort.
It matters very little when users are trained to tolerate slow transitions between videos, formats, etc...
It also doesn't matter a whole lot when doing offline transcoding either, as you can afford to do the more expensive calculation.
Even for live broadcast, how can a few integer instructions matter? If you have to compare timestamps every audio sample, that's a few multiplications every 22 microseconds (common denominator can be computed once). Or am I completely off here?
Trying to realtime encode 4k into AVC using pure software encoders takes about 80 cores/hyperthreads. Then you need to meet the tight 70ns? timings of the SMPTE 2022-6/7 protocols (i.e. 12gbps without using jumbo frames because reasons).
It's also necessary for clip switching. If you want frame accurate clip switching (i.e. show->ad->ad->show) you need consistent and precise pointers into your files.
In real modern video files every frame has its own timecode, and mobile phones typically record with variable framerate when these timecodes are placed highly irregularly (the reason can be a lack of performance or an attempt to sync with light source of the different frequency than camera's settings). Any new video editing software is expected to deal with such files, and flicks are useless here.
I’m not sure I’d agree here. RTPM is probably the most popular live streaming protocol, and it typically has huge variations in instantaneous frames rates. However it’s time base is hard coded to 1/1000. Which means it can’t even accurately represent 30fps. Flicks would be at least a massive improvement here. And even given variable frame rate, a time base must be chosen in advance, so flicks is at least as good as any other value.
> That is 367.5 times more audio samples than video frames.
Okay, but an audio sample is not comparable to a frame of video, which by itself means something to the viewer. (But, on the other hand which probably doesn't differ much from the previous or next one, especially at 120 Hz).
An audio sample is sort of more comparable to a pixel.
Its comparing 1 dimensional space to 2 demential space. So, any comparison is gong to be a little hand wavy. But temporally, I believe the comparison is apt.
> Its comparing 1 dimensional space to 2 demential space
Although I love the coinage "2 demential space", I think you mean "comparing one-dimensional space [audio] to three-dimensional space [video]". A two-dimensional signal might be a still image or a temporal sequence of samples from a one-dimensional array of sensors, such as those in a single slice of a CT machine or a linear MIMO antenna array. A video signal is three-dimensional, not two-dimensional, and probably not "2 demential" either.
No, wavelength is just one more dimension along which intensity may vary (in addition to X, and Y, and time), not five or six more dimensions, so a multi-band image is only three-dimensional, regardless of whether there are three wavelength bands (like RGB or YCbCr), four (like RGBA), 8 (like Landsat), or 210 (like HYDICE, AVIRIS, and other imaging spectrometers).
My point is, I am comparing temporal dimension to temporal dimension regardless of how many special dimensions there are. And I don’t understand the argument that an audio sample is more analogous to a pixel that it is a frame on a time line.
In particular, in data processing, all dimensionality is equivalent, since and infinite set S the same cardinality as S^n for any whole number n, and any finite set is smaller than the 1-dimensional set of naturals.
Yeah, at least if Hilbert spaces can fuck off, which is why we can approximate signal processing on digital computers at all. And, because of space-filling curves, in some sense ℝⁿ is equivalent to ℝ. But, to understand signal processing, a much more useful point of view is that ℝⁿ is significantly different for different values of n, but not completely unrelated; and ℤⁿ is a useful approximation of ℝⁿ, as is (ℤ/mℤ)ⁿ.
Yep. The algorithm is: there is a leap day for every year divisible by 4, unless the year is also divisible by 100. But if the year is divisible by 400, then there is a still leap day.
Yeah, you really dont want to be implementing a date/time library yourself. It is a far more complex problem than Fizzbuzz. There is so much nuance and easy to overlook details. Just determining whether or not a year is a leap year is about discuss complexity.
I wrote a friendlier abstraction over top Boost DateTime in C++ at a previous employer. Adding things like cross platform support of using the Olson database for historical timezone info, a friendlier interface (more akin to Python's datetime lib, which is probably the friendliest datetime lib Ive used across any language) and cross platform strptime. So, didn't do the heavy lifting myself, but rather supplied the timezone info.
There were so many subtle details. Like, what does it mean to add 1 day to a timestamp? Do you increment year/month/day as needed to be the next day? Or do you add 24 hours? This matters because if you increment the date only, you can end up with an invalid or ambiguous time. I also appreciate the author mentioning how many hours are there in a day. 23, 24 and 25 are most common for jurisdictions observing daylight saving time, but arbitrary shifts are possible. It depends on the jurisdiction. Take the US for example. It depends not just on the state, but sometimes even the county within a state. Some states dont observe DST (Arizona and Hawaii, maybe others). Parts of Indiana do, some don't (maybe this has changed).
Compound this with changes to when DST starts/ends. Its entirely political and can change on a whim. I know there are clocks that are still wrong 12 years later for 6 weeks of the year because of the 2007 US DST change, because of embedded rules for when to change.
Sorry for the rant. Spent over 18 months on that project, and I know there are bugs in there nobody bothered to fix after I left that company (fractional seconds handling in strptime I ported from NetBSD, I'm looking at you).
> Yeah, you really dont want to be implementing a date/time library yourself.
Someone should make a date/time "meta library" which iterates through all possible system time values while calling each extant language's date/time method at each iteration.
Then report the system time values for which there are discrepancies in the output.
> The number result is 0.008333333333 with the 3 repeating forever. For a computer to store a number that repeats forever would require an infinite amount of memory
This is very much not true, as I'm sure other HN readers will notice. The number is rational (it's equivalent to 1/120). Now, it is true that a floating point number may not be able to represent it exactly, but by no means does this number require "infinite memory." In fact I have represented the number exactly in this comment, which does not take up infinite space.
For irrational numbers, sure, they cannot be exactly represented. But there are no irrationals involved in this article.
I got hung up at this point in the article, so I haven't finished it yet, but it looks like the author goes on to argue that because numbers like the above cannot be represented in computer memory at all, errors will always accumulate in representations of audio/video. This makes me question whether the author understands the problem they are writing about.
Edit: the author does in fact state that rational numbers can be represented by a numerator and a denominator. The article is actually about errors the accumulate during floating point operations. It ends up making a decent argument despite false claims about representing numbers in memory.
I wish you had kept reading. In the end, you nitpicked about precisely the thing which the author ended up explaining.
The point about using infinite memory to store an irrational number is just a rhetorical device / presentation style to keep the novice reader engaged, and following along. And then you say, "Next we will explain how to solve this impossibility...," and such.
It's a good technique for writing, and presenting, but doesn't work if your audience already knows where you're going and gets impatient with you!
I did keep reading the article, as I say in my edit. I differ from you in my interpretation of this facet of the essay. To me, it reads as internal contradiction — the author says that it is impossible to represent 1/120 in memory and then goes on to describe a strategy for doing just that.
The author does not present this conflict as "it may seem impossible…" but rather as "is is impossible." That's an inaccuracy, not a writing technique.
Mate, I mean this with the best of intentions: please reflect on your requirement for exactness and consider whether it serves you well.
I did so with mine and I concluded it did not. Perhaps you will conclude differently. Perhaps the same. In either case, I think you will find it worthwhile.
The claim is only missing a qualification that the impossibility exists within the framework of ieee 754. It can be easily inferred from context and it is important to note because thats what the major CPU architectures use to store native floating point numbers. Therefore, the point is, we need a non-native (wrt the cpu) way to store and calculate with irrationals and some rationals if mathematical precision is required.
Thanks for the comment. For the record, I am very aware of iee 754, And I am aware that some numbers that have infinite repetition in decimal can have an exact representation. I actually thought about pointing that out. However I decided that it didn't add much to the post. It was written for more of a general audience and diving into those details, while would make the post more technically correct, would not actually add any value for the average reader.
I don't want to come off as combative, but I would argue that a correction would add quite a bit to the post. What you have is straight-up incorrect and wouldn't even take that much effort to fix. "For a computer to store the decimal representation a number that repeats forever would require an infinite amount of memory, so the number is approximated" would be correct and it's just a couple words more.
There are uncountably infinite real numbers and countably infinite rational numbers, so we cannot represent all of them using a {0, 1}^n where n is a finite number. No matter which numbers you can exactly represent, there will always be uncountably many real numbers which are not exactly representable, and, conversely, the set of real numbers exactly representable using a finite number of bits will have measure zero.
The issues of "repeating decimal" and exact representation as a floating point number are orthogonal. Because the mantissa in a floating point number (which is fixed width) is in base 2, only fractions with denominators that are powers of two can be exactly represented. This means that simple base-10 decimals like 0.1 cannot be exactly represented by a float. For this reason it's at least a bit misleading to talk about how some numbers cannot be represented with finite memory, and then go straight into a demonstration of how they can in fact be represented in finite memory (as a fraction).
As for the comment about a general audience, if (some of) your target audience is HN readers, I think it's reasonable to expect many readers to be familiar with computer science.
If this were my article, I would replace the paragraph in question with a discussion of the error introduced in floating point calculation — consider perhaps that many programming languages will tell you 0.1 + 0.2 = 0.30000000000000004 [0].
Thanks again for the refresher. But again I am keenly aware for how floating point works. As for audience, this is not a hacker new exclusive. It just happens to be here rite now. It will probably make rounds in other forums as well. If you want to write an post an article on how floating point works. I’ll be happy to read it!
I know this sort of criticism can be annoying, but despite commenting on HN I would surely qualify as a 'lay reader' in this context, and I have a strong preference for authors taking care to either tell the whole truth or flag that a potentially misleading simplification has been made -- even if the falseness seems like a technicality to the author. If I notice it then I lose faith in the author's credibility, and if I don't then now I've 'learned' something false.
Cripes, you're clearly smart enough to understand that the author meant "to [literally] store [all the digits/bits of] a number that repeats forever [in the given base] would require an infinite amount of memory."
> The number result is 0.008333333333 with the 3 repeating forever. For a computer to store a number that repeats forever would require an infinite amount of memory
This is clearly a literary technique to create interest. Like saying something like "I told you a billion times not to exaggerate". It should be clear to most readers that this is not an attempted mathematically precise comment.
