Real story. I worked on nudity detection for mobile realtime video chat app with on-device deep learning model. (It was successful.) Should be easy: label some data, start from a good network pretrained on ImageNet (we used SqueezeNet), fine tune. The problem was that ImageNet is photos, not video frames, and distribution of video frames from mobile video is somewhat different. Incredibly large proportion of frames are either blank (camera is blocked) or ceiling (camera is misdirected). We ended up adding lots of data for blank and ceiling explicitly labeled as blank and ceiling. It became an excellent ceiling detector, detecting ceilings it never have seen before.
Is there a mobile video frames dataset for labeling ceilings? Why not? I can't believe I am the only one who experienced this. Why is not a ceiling dataset worthy of, say, CVPR? It will improve mobile video on-device deep learning more than most of CVPR papers. This is a serious problem.
Edit: I understand why niche datasets remain in industry and not academia. Public datasets are better if they are of general interest. But ceiling dataset is of general interest to anyone who wants to process video frames originating from blocked or misdirected camera aka smartphones, and it's hard to imagine topics of more general interest once you have obvious things like face.
CreateML is getting really impressive. I just went through the process of using it to train an Object Detection model to detect sharks and fish at an aquarium (and deploying the model to an iPhone) last week.
It seems that it's more a problem with your inability to optimize correctly than anything else. But you are on the right way.
You are trying to do a multiple-objective optimization one objective at a time, and cycling through them.
You have already noticed that you can't totally forget the other objectives when working on one, otherwise you lose more over a cycle.
In fact what you seem to be aiming for is a single objective which is a combination of being good at work, soccer and relationship. At first order (i.e. when you don't aim for reaping the non-linear effects of being "the best" and ignore the non-linear effects in human physiology), the optimal optimization shouldn't exhibit such oscillations.
Quantify it, put it into a solver and do what's told. But the rule of thumb of good optimization (something something Jensen inequality) is that all your objectives must be growing all the time so you just have to be sure you are not letting something regress. It's called striking a balance. Then all objectives will grow following a S-curve simultaneously.
To avoid local minima (i.e. plateaus) which often plague single optimization problems, just increase the number of degree of freedoms in your internal representation and add some variability. Play various styles, try another way of working, spice up your relationships, do this simultaneously, memorize and imagine, and of course don't forget to try to improve (i.e. "pick the right move", as it's easy to just enjoy the fun of variability).
That's when the jack of all trade begin to reap benefit over the master of one. Some activities do synergize and help you progress deeper and faster.
Closely related is the eternal debate between satisficer vs maximizer which is just the primal-dual representation of the optimization problem.
Interesting perspective. What do you say to someone who's been using OLAP cube for their entire BI implementation? What would be the transition plan to adopting MPP databases?
QuestDB (https://www.questdb.io) might help. Import your file(s) by drag and drop directly into the web console. Once data is loaded, you can run SQL queries in the console to order, format, concatenate, join (including time-based joins), etc. Once finished, you can export your formatted results back to CSV. Also, it's pretty quick!
I recently had fun implementing [0] some of the concepts/math for ambiguous cylinders by Kokichi Sugihara. Some of his first illusions [1] came from researching computer vision, where the computer should understand how a 2d picture supposedly looked in 3d. When feeding it some optical illusions, he realised it would be possible to make these objects in real life!
Wow, I was expecting graphics in the style of Return of the Obra Dinn, but this is much more impressive.
Lucas Pope's work on bringing the 1-bit aesthetic into the modern age is fascinating in itself. One of my favorite excerpts is how Pope 'stabilized dither' in order to reduce the harsh effects that dithering moving images can create: https://forums.tigsource.com/index.php?topic=40832.msg136374...
I don't know how to organize wiki-like things for teams, but here are my two cents for the topic of personal knowledge bases.
First of all, regardless of software, I prefer less structured and less constrained approaches so that it doesn’t take too much time and effort to add things to my knowledge base. I try to minimize the total number of entities (lists, files, notes) and to avoid folders at all. Usually I achieve this by using tags.
Secondly, I am still in a search of a perfect approach, but here is one thing where I succeeded to some extent, namely collections of homogeneous items. I use AirTable for that, but one can use Notion. For example, here is a list of software, I started it several months ago: https://airtable.com/shr8Wd96FurJmiTLs/tbl0n17xFuXrs0kMS?blo... I have a similar list for scientific papers that I read. And I started two more lists for other things. It is important that 1) the number of lists is low (I have 4 now) 2) the lists have simple structure 3) I can always download them as csv and do whatever I want.
As for the rest, I use markdown notes (I edit them with https://typora.io) stored in a dropbox-like cloud for ideas-like stuff. I use Google Keep for super-simple lists and I also have there a temporary note that serves as a buffer for sudden thoughts and ideas, from where I move them later to more appropriate place. I use https://todoist.com for my tasks, they are not organized very well (because no tags in the free plan), but the app is really nice and the number of tasks is not that great, so for now this works fine :)
Pick a frequency. Let's say you picked 400 hertz. If you multiply this frequency by a simple fraction (1/2, 1/3, 2/5, etc) you get a new frequency that harmonizes with your original frequency. That means they sound nice when played together. The simpler the fraction, the better the two frequencies will sound (1/2 sounds nicer than 7/13, for example).
If you pick several of these fractions between 1 and 2 (such as 3/2, 4/3, 5/4, 2/1, etc) you create what's called a scale. All the frequencies in the scale harmonize with the starting frequency, but they don't necessarily harmonize with each other.
Musicians don't always want to play with the same starting frequency so they invented "equal temperament". The idea behind equal temperament is to create a scale using logarithms/exponents instead of fractions. Because it's logarithmic, any frequency in the scale can be used as a starting point and you'll get the same result.
This is a great suggestion - many of the big name consulting companies (Accenture, Ernst & Young, McKinsey, CGI, Capco) will happily subcontract out the work that they've sold, or bring on external consultants to fill out a team.
I was there at Function when it was shown on the big screen, we brought something for c64. At first I expected something else to show up, but then I remembered it was 64 bytes. Sixty four bytes! 256byte scene is insane. I've noticed recent 256b releases are high res. Not sure how they do it, definitely not 13h. Anyways, this 256b still takes the cake: https://youtu.be/eYNoaVERfR4
IMO this post has some good points but makes the executive sound like a passive referee, ultimately misunderstanding what High Output Management (also one of my favorite books!) is about. (Admittedly adding my own editorial here from my experience founding a startup and now running it as a ~3,000-person public company.)
The basic principle of HOM is that the fundamental job of an executive is to deliver results ("output"), and that the measure of an executive is the output of their organization. Importantly, there is no one right way to deliver results -- successful CEOs can have very different styles and techniques.
That said, for every effective way to deliver results there are vastly more that are ineffective. Complexity, ambiguity, and uncertainty are not your friends. Time is not your friend. Everything is situationally dependent. There are many skills to develop and principles that can help but there's no formula.
This also partially explains why the median CEO or exec is perceived as ineffective, often because they are. It's a hard job, otherwise everyone would do it well and there would be a surplus of good (and cheap) execs.
Contrary to what the post suggests, HOM does not say not that the job of an executive is to wave some kind of magic culture or "values" wand and rubber-stamp whatever emergent strategy and behavior results from that. CEOs and executives absolutely do (and must) make important decisions of all kinds, break ties, and set general direction. Occasionally they need to give commands but more typically you work collaboratively with and (as the post correctly suggests) empower your team and avoid doing too much as an individual contributor.
If you're curious about what execs do and how to be a good one, HOM is an incredible book. The Effective Executive by Drucker is another favorite.
I haven’t found near infrared radiation referenced in the article - I‘m using near infrared as a brain hack, by shining a cheap 850nm LED light on my forehead. This has, over the last 2 years, enabled me to code for weeks on end, for 12+ hours a day, with only minor cognitive decline. It’s not something I really want to do, but sometimes it’s useful.
Before I started the near infrared routine (~5 minutes every other day), 5-6 hours of coding per day was all I could do - eg after coding for 8h, I noticed serious cognitive and emotional decline, and might need to do less the following day. Not anymore - nowadays I can be productive whenever I’m awake, with little side effects. Near infrared radiation is safe (thousands of studies demonstrated only very mild side effects), and is even used to treat Alzheimer’s. I have no idea why its beneficial effects are not more widely known - for some people, it’s life changing.
Sidenote: 850nm light works way better for me than 830nm.
Slightly off-topic, but for anyone interested, I did write up my own battle with Lyme, which went on for many, many years, till I found a simple solution:
"How I recovered from Lyme Disease: I fasted for two weeks, no food, just water"
If anyone is looking to get into Category Theory as a programmer, I would recommend giving Bartosz Milewski's Category Theory for Programmers a shot[0]. Quite good, and you can read it as a blog post on that site or get the ebook/physical book to leaf through.
For a more theoretical/less applied approach that's still relevant to computer science/programming, check out Benjamin Pierce's Basic Category Theory for Computer Scientists[1]. It's a very good introduction to the parts of Category Theory that are useful for [theoretical] CS, and delves into semantics of programming languages, types, and concurrency a fair amount.
I can't find the exact quote, but I think it was Aristotle who said (something along the lines of) "we haven't mastered the art of teaching, except to those for whom it is superfluous". This is still true today, and, I think, true of each and every single management fad. Things like deadlines, status reports, daily standups, open offices, tickets, and retrospectives are designed around the assumption of a reluctant workforce who have to be clubbed into line. If you assume that's what you're dealing with, you're going to gravitate toward heavy-handed means of control and conversely, if you gravitate toward brutal overseer managerial tactics, you're sending the signal to the workforce that you expect them to push the envelope and accomplish as little as possible within the narrow parameters of your dictatorial cruelty. If you assume that we're all on the same team, you'll get good results as long as we are. If you assume that we're not, there's no level of threatening or punishment you can dish out that will.
This program utilizes a powerful technique called raymarching, raytracing's big brother. Instead of tracing along each ray in tiny fragments and returning intersections, the ray is 'marched' in larger steps, vastly increasing render speed (the downfall of classical raytracers).
How do we 'march' rays without missing geometry? With the aid of signed distance functions, geometric equations which take an arbitrary point in 3space and evaluate to 0 if that point is on the 'shell' of the geometry. For example, the signed distance function for a sphere:
This lets us do interesting things such as transform the point p to transform our object, take the minimum value of two spheres to produce a venn-diagram intersection shape, take the maximum of two spheres to 'cutout' a hole from one, etc. The logic is a bit backwards at first, but it clicks quite quickly and one can rapidly construct complex geometry without much optimization/shader magic. For example, this is a realtime capture of a 64k self-contained executable. No 3d models, no textures, no assets - just raymarching and math: [https://www.youtube.com/watch?v=ie4u2i_5OdE]
I learnt a lot more than I ever wanted to know about every aspect of elevators from Elevator Hacking: From the Pit to the Penthouse with Deviant Ollam (who seems to be like a professional elevator pen tester) & Howard Payne, from DEFCON 22. Highly recommended. You'll be talking elevatorese like a pro after you watch that.
Constructive mathematics is the way forward. It's connected to linear logic via Chu spaces [0] (https://arxiv.org/abs/1805.07518). This correspondence is unreal as constructive mathematics / linear logic unlocks the door to several key advantages such as being closer to computation, probability theory, type theory, Lie groups, a generalized topology, optimization, “differentiable Turing machines”, all the quantum shit, game theory etc the list goes on.
Linear logic is fundamentally about formulating computation as an adversarial relationship between two parties. It's also the foundation of e.g. the Rust ownership system (I know, I know, affine types).
It's also the foundation of the min-max algorithm which in turn is the foundation of Generative adversarial neural networks.
Ironically, this idea isn't exactly new. It's directly related to Dialectics (https://en.wikipedia.org/wiki/Dialectic) which has been present in Western philosophy since Aristotle through Kant, Hegel and Marx (it's a funny story).
In my opinion PyTorch code is easier to understand and debug for newcomers. Code is definitely lacking in documentation, but whenever there was a tradeoff between clarity and modularity in the end I've chosen modularity. Ideally I would like others to be able to take bits and pieces and incorporate into their projects to speed up time to delivery of their ideas.
Is there a mobile video frames dataset for labeling ceilings? Why not? I can't believe I am the only one who experienced this. Why is not a ceiling dataset worthy of, say, CVPR? It will improve mobile video on-device deep learning more than most of CVPR papers. This is a serious problem.
Edit: I understand why niche datasets remain in industry and not academia. Public datasets are better if they are of general interest. But ceiling dataset is of general interest to anyone who wants to process video frames originating from blocked or misdirected camera aka smartphones, and it's hard to imagine topics of more general interest once you have obvious things like face.