You could still save the branch-and-bound tree, the LP problems solved at the tree nodes, the derivations of the LP cutting planes, and the LP solutions that together constitute the proof. Then you could in principle create an independent verifier for the branch-and-bound tree and cutting plane derivations, which could potentially be much more straightforward and simple code than the entire Concorde TSP solver, and wouldn't have so high performance requirements.
I don't know Datashader that well, but from what I understand, it generates an image from a set of primitives (e.g. points), and then allows you to interactively inspect that image. It does not re-render the points on every frame like Fastplotlib/Pygfx does.
Commercial Linux distributions like Red Hat, Suse and Canonical stake their reputation on compiling a trustworthy collection of open source software, in exchange for money. Unfortunately they disclaim any legal responsibility, but at least they make reasonable efforts to analyze the security of the software they are distributing, in order to avoid PR disasters.
For some reason the same business model has not made many inroads for higher-level language ecosystems, although many companies are trying - for example the Python Conda distribution.
Although the "repo" is a list of manifest files that include third-party download sources. So even if there is an approval process it seems to be quite vulnerable to including malware.
The Burroughs-Wheeler transform has been described as a unique algorithm idea in that there are no non-trivial variations or related algorithms, unlike more conventional compression algorithms, which can be tweaked and improved in so many ways. There is no general compression theory in which BWT could be described as a special case.
It looks to me that the above still holds: Bzip2 and Bzip3 are simply combining more conventional compression algorithms with the BWT, which itself is still the same old transform. Bzip2 does Huffman coding after BWT, and Bzip3 does arithmetic coding.
> There is no general compression theory in which BWT could be described as a special case.
I would not really say this is true. BWT is spiritually similar to the various Prediction by Partial Matching (PPM) algorithms, except that instead of needing to decide how much context (i.e., preceding symbols) to use to model the next symbol, and carefully learning the probabilities for each unique context, it naturally sorts symbols with the same context (of _any_ length) so they appear next to each other, and relies on adaptation to update your learned statistics as you move from one context to the next, without ever explicitly tracking context boundaries [0].
It definitely is related to prediction by partial match (PPM).
BWT sorts rotated data and what is achieved is that same suffixes group together:
...
"Bzip2 and Bzip3 are simply combining more"
"Bzip3 are simply combining moreBzip2 and "
The preceding (to suffix) character goes to end and then gets outputted. This is much like PPM going backward. There is a PPM* algorithm (unbounded context length) where authors considered reconstruction of contexts from data, utilizing something like LZSS seach. Same idea is in BWT - context is reconstructed from data.
BWT also breaks near dependencies in data, this is why move-to-front with Huffman or arithmetic encoding works well there.
I don’t know enough to evaluate that, but it sounds plausible. Apparently modifying or integrating zstd into custom solutions was a common path in submissions to, at the very least, the GDCC 2021 T2 contest. This is all well outside of my learned competence so I’m just here to learn and ask naive questions.
Zstd is basically LZ77 + FSE. There are some other pieces to it, but they’re minor. FSE is a form of entropy coding that you can swap out with other entropy coding techniques, like Huffman or arithmetic coding. For most objectives, FSE is the clear winner of those three.
As people mentioned elsewhere in the thread, Burrows-Wheeler isn’t really composable with other systems. Nobody has figured out a reasonable way to combine LZ77 and Burrows-Wheeler. That’s why zstd + bzip2 does not work. But you do need an entropy coding system to go with Burrows-Wheeler… and FSE fits the bill.
Based on the presented multi-core benchmarks, the single-core performance is 40-50% of a single M2 Ultra core: 40% on the HPL Linpack benchmark, and 50% on Cinebench. That is compared to the average M2 Ultra core, counting both the fast "performance" cores and the slow "efficiency" cores. So the M2 performance cores are even faster than that, never mind the M3 or M4. Then again, you cannot buy a 128-core computer from Apple, so it's an apples to oranges comparison.
Based on a quick googling, apparently the original paper is "One kitchen is all you need" by Sister Eudocia, Isabelle DeVerneil, and Jane Hildebrandt, published in Modern Hospital vol. 79 issue 3, pages 120-122, 1952. https://pubmed.ncbi.nlm.nih.gov/12992940/
The Nature paper refers to potential applications in spintronics. I'm guessing they mean some kind of improvements to MRAM devices (https://en.wikipedia.org/wiki/Magnetoresistive_RAM), which can be faster than conventional DRAM at the expense of high power usage.
An important point here is that the integral of the likelihood function over different parameter values is not constrained to be 1. This is why a likelihood is not a probability or a probability density, but its own thing. The confusing bit is that the likelihood formula is exactly the same as the formula of the original probability density function...
Splitting hairs probably but personally I'd say it the other way around: a likelihood is not a probability or a probability density, so there's no reason to think that it would integrate to 1.
The reason it's not a probability or probability density is that it's not defined to be one (in fact its definition involves a potentially different probability density for each point in parameter space).
But I think I know what you're saying -- people need to understand that it's not a probability density in order to avoid making naive probabilistic statements about parameter estimates or confidence regions when their calculations haven't used a prior over the parameters.
This is also why it is surprisingly easy to break your dependencies when installing lots of stuff in pip: sooner or later you're going to end up with some dependency-of-dependency incompatible with a dependency. The practical solution is that you don't install everything in the same environment but set up a separate virtual environment for each project, and then install only what you need there.
C++ templates do have the advantage that they can dispatch on their argument types. Lisp macros are expanded at a purely syntactic level, well before the compiler attempts to do any type deduction, and you cannot provide different macro expansions for different argument types without heroic efforts (which will most likely be specific to a single Lisp compiler). Dispatching on argument types is very useful in expression template based metaprogramming, for example in the Eigen library for linear algebra (https://eigen.tuxfamily.org/).
reply