Conclusion: most of us don't need AI in the same way most of us don't need a chainsaw.
"We’re all mostly just regular people interacting with other regular people, trying to go about our business and get through our days. Nobody’s asking us to index and catalog entire libraries of information. Nobody really cares if we reuse a stock photo that somebody else might have used somewhere else at some point. We’re not so busy that we need everything predigested before it’s presented to us. We’re still clever enough to work our way through unfamiliar problems. Some of us, I would have to think, still possess the ability and the desire to produce some sort of output in our chosen medium without relinquishing creative control to AI."
I disagree, you're not including mental fatigue. I can say something like "I have a server OS upgrade that needs to be done by X date. The server uses XYZ technologies. Here is the list of stakeholders and their roles. Design me an assessment and implementation plan and outline the key milestones and their dates." Could I think through that myself? Yes but it may take a couple hours. I just got 85% of the planning work done in 2 minutes.
I feel like I'm talking with people who would rather ride a horse and buggy down a rutted dirt road for 2 hours both ways, checking the wooden wheels for damage. Giving the horses rest stops.
I think I'll take a 10 minute car drive to costco, thank you very much.
There is a very good essay in the first comment by "Roger" dated Jan-2023, reproduced below. Skip the primary essay and work from this:
"I really appreciated this piece, as designing good metrics is a problem I think about in my day job a lot. My approach to thinking about this is similar in a lot of ways, but my thought process for getting there is different enough that I wanted to throw it out there as food for thought.
1. Direct activities to achieve goals
2. Intervene in trends that are having negative impacts
3. Justify that a particular course of action is warranted
4. Validate that a decision that was made was warranted
My interpretation of Goodhart’s Law has always centered more around duration of metrics for these purposes. The chief warning is that regardless of the metric used, sooner or later it will become useless as a decision aid. I often work with people who think about metrics as a “do it right the first time, so you won’t have to ever worry about it again”. This is the wrong mentality, and Goodhart’s Law is a useful way to reach many folks with this mindset.
The implication is that the goal is not to find the “right” metrics, but to instead find the most useful metrics to support the decisions that are most critical at the moment. After all, once you pick a metric, 1 of 3 things will happen:
1. The metric will improve until it reaches a point where you are not improving it anymore, at which point it provides no more new information.
2. The metric doesn’t improve at all, which means you’ve picked something you aren’t capable of influencing and is therefore useless.
3. The metric gets worse, which means there is feedback that swamps whatever you are doing to improve it.
Thus, if we are using metrics to improve decision making, we’re always going to need to replace metrics with new ones relevant to our goals. If we are going to have to do that anyway, we might as well be regularly assessing our metrics for ones that serve our purposes more effectively. Thus, a regular cadence of reviewing the metrics used, deprecating ones that are no longer useful, and introducing new metrics that are relevant to the decisions now at hand, is crucial for ongoing success.
One other important point to make is that for many people, the purpose of metrics is not to make things better. It is instead to show that they are doing a good job and that to persuade others to do what they want. Metrics that show this are useful, and those that don’t are not. In this case, of course, a metric may indeed be useful “forever” if it serves these ends. The implication is that some level of psychological safety is needed for metric use to be more aligned with supporting the mission and less aligned with making people look good."
Thank-you. The next time metrics are mentioned, one can mention an expiration date. That can segue into evolving metrics, feedback control systems, and the crucial element of "psychological safety."
A jaded interpretation of data science is to find evidence to support predetermined decisions, which is unfair to all. Having the capability to always generate new internal tools for Just In Time Reporting (JITR) would be nice, even so reproducible ones.
This encourages adhoc and scrappy starts, which can be iterated on as formulas in source control. Instead of a gold standard of a handful of metrics, we are empowered to draw conclusions from all data in context.
I am not "Roger," but I can recognize someone who has long and practical experience with managing metrics and KPIs and their interaction with process improvement. Instead of an "expiration date" I would encourage you to define a "re-evaluation date" that allows enough time to judge the impact and efficacy of the metrics proposed and make course corrections as needed (each with its own review dates).
Here is a quote to give you a flavor of his philosophy:
"A business should be run like an aquarium, where everybody can see what's going on--what's going in, what's moving around, what's coming out. That's the only way to make sure people understand what you're doing, and why, and have some input into deciding where you are going. Then, when the unexpected happens, they know how to react and react quickly. "
While Goodhart’s Law often occurs because of a narrow focus on a metric without understanding its role in the larger system, the approach in Jack Stack’s The Great Game of Business is to make targets an educational tool, teaching employees how to interpret and impact those targets responsibly.
GGOB, by
1. involving employees in decision-making and teaching them about metrics,
2. giving them a line-of-sight for how their contribution impacts the overall business, and
3. providing a stake in the outcome
creates collective accountability and success, and reduces the likelihood of metric manipulation.
Bottom line: GGOB recognizes that business success takes everyone, at all levels, and values the input of each employee, right down to the part-time janitor. The metrics are used as tools, like the scoreboard in baseball, to guide decision making and establish what winning as a team looks like. It all comes down to education and getting everyone aligned and pulling in the same direction.
I should note that this essay kicks off an entire series that eventually culminates in a detailed examination of the Amazon Weekly Business Review (which takes some time to get to because of a) an NDA, and b) it took some time to test it in practice). The Goodhart’s Law essay uses publicly available information about the WBR to explain how to defeat Goodhart’s Law (since the ideas it draws from are five decades old); the WBR itself is a two decades-old mechanism on how to actually accomplish these high-falutin’ goals.
Over the past year, Roger and I have been talking about the difficulty of spreading these ideas. The WBR works, but as the essay shows, it is an interlocking set of processes that solves for a bunch of socio-technical problems. It is not easy to get companies to adopt such large changes.
As a companion to the essay, here is a sequence of cases about companies putting these ideas to practice:
The common thing in all these essays is that it doesn’t stop at high-falutin’ (or conceptual) recommendation, but actually dives into real world application and practice. Yes, it’s nice to say “let’s have a re-evaluation date.” But what does it actually look like to get folks to do that at scale?
Well, the WBR is one way that works in practice, at scale, and with some success in multiple companies. And we keep finding nuances in our own practice: https://x.com/ejames_c/status/1849648179337371816
It looks like any other decision record where you set a date to evaluate the impact of a policy or course of action and make sure it's working out the way that you had anticipated.
And how are you going to tell that when you have a) variation (that is, every metric wiggles wildly)? And also b) how are you able to tell if it has or hasn’t impacted other parts of your business if you do not have a method for uncovering the causal model of your business (like that aquarium quote you cited earlier?)
Reality has a lot of detail. It’s nice to quote books about goals. It’s a different thing entirely to achieve them in practice with a real business.
I agree that reality is complex, but I worry you are conflating the challenges of running an Amazon-scale business with running the smaller businesses that most of the entrepreneurs on HN will need to manage. I thought Roger offered a more practical approach in about 10% of the words that you took. I am sorry if I have offended you; I was trying to save the entrepreneurs on HN time.
As to Jack Stack's book, I think the genius of his approach is communicating simple decision rules to the folks on the front line instead of trying to establish a complex model at the executive level that can become more removed from day-to-day realities. In my experience, which involves working in a variety of roles in startups and multi-billion dollar businesses over the better part of five decades, simple rules updated based on your best judgment risk "extinction by instinct" but outperform the "analysis paralysis" that comes from trying to develop overly complex models.
How do you answer? Notice that this is a problem regardless of whether you are a big company or a small company.
b) 3 months later, your client comes back and asks: “we are having trouble with customer support. How do we know that it’s not related to this change we made?” With your superior experience working with hundreds of startups, you are able to tell them if it is or isn’t after some investigation. Your client asks you: “how can we do that for ourselves without calling on you every time we see something weird?”
How do you answer?
(My answers are in the WBR essay and the essay that comes immediately before that, natch)
It is a common excuse to wave away these ideas with “oh, these are big company solutions, not applicable to small businesses.” But a) I have applied these ideas to my own small business and doubled revenue; also b) in 1992 Donald Wheeler applied these methods to a small Japanese night club and then wrote a whole book about the results: https://www.amazon.sg/Spc-Esquire-Club-Donald-Wheeler/dp/094...
Wheeler wanted to prove, (and I wanted to verify), that ‘tools to understand how your business ACTUALLY works’ are uniformly applicable regardless of company size.
If anyone reading this is interested in being able to answer confidently to both questions, I recommend reading my essays to start with (there’s enough in front of the paywall to be useful) and then jump straight to Wheeler. I recommend Understanding Variation, which was originally developed as a 1993 presentation to managers at DuPont (which means it is light on statistics).
"WP Engine would want to make sure that it's customers are well taken care of and have a solid web hosting experience."
This is certainly one of their goals, if only to prevent or reduce churn.
But WP Engine is now owned by Silver Lake, a private equity firm, they no doubt have aggressive growth and profitability targets. Anything that injects confusion into their branding or increases costs is counter to their goals
The real question is the 8% "contribution" that WP is asking for cheaper than other alternatives. The lawsuits are cheaper if they win, but a dead loss if they don't.
The audience for this statement is WP Engine's significant customers. WordPress is in a position to do much more damage to WP Engine in the near term, which will reflect poorly on the IT manager for selecting them as a vendor. WordPress is not so subtly encouraging those customers to reconsider their decision and migrate off WP Engine.
If WP Engine decides to fork, it devalues the "just like WordPress but better" value proposition and increases operating expenses as they can no longer inherit improvements from WordPress. A fork may mean they don't hit the growth targets they promised Silver Lake. Selecting this attorney is putting down a marker that WordPress wants a verdict, not a settlement.
The other wild card potentially more damaging to WP Engine and Silver Lake is the discovery process inherent in any lawsuit.
I am not a lawyer, but I don't think most commenters are correctly decoding the relative bargaining power of the two sides.
There’s no world where forcing significant customers off WP Engine works out well for Automattic.
Those customers are not going to migrate their sites to the company that just gave them an operational and security headache (Automattic).
And most big customers do not give a shit about Wordpress per se. They just use it because it’s a free and convenient accelerant for the sites they want to build. If it starts becoming a hassle they will just move to a different CMS. There are plenty of options.
"In an email, Bruce Perens, one of the founders of the open source movement who drafted the original Open Source Definition, told The Register, "Let's be clear about WP Engine: It's built on WordPress. There would be no business without WordPress. And it's a large business with big revenue, operated as if it's funded by private equity."
"Private equity always demands big returns, regardless of the harm they do to the business. One of my customers has been completely destroyed by them – they are still operating but on such thin resources that they can't dedicate the time of one engineer to work with me on an open source compliance review, even if I do it for free.
"So, WP Engine is in that situation, and has to increase returns to the investors. What do they do? Cut any voluntary expense, which includes returning any value to the creators of WordPress. I'm told that WordPress asked for eight percent of revenue, which sounds fair to me considering that it's the basis of WP Engine's business.
"But because it's an open source project, WordPress can ask but can't demand that money, so they have to turn to hostile enforcement of their trademark and denying access to their updates."
By that logic, the 30% Apple tax must be "extremely fair" because all iOS apps are built on both the software and hardware built by Apple, which are arguably far more complex than a piece of PHP software. (/sarcasm)
Being "built" on something does not in itself imply value. What has to be taken into account is also "value that is added." If the underlying platform had high intrinsic value in itself, then no value would have had to be built on it to profit from it, which would mean WP Engine would not be the only entity profiting from it.
And to add to the reductio ad absurdum, are PHP engineers required to "donate part of their salaries" to the project just because "they're profiting off of that language?" (/sarcasm)
I did not say that what WordPress was doing was "fair" I was trying to explain how their actions can make sense from their side of the table.
As to 30% being "fair" it's a fee structure that is known in advance, you can plan accordingly. I try to deal with the world as it is, not how I would like it to be.
Why would they need to fork? WordPress itself is from B2, hence its licensing being GPL. WordPress cannot adopt the dual licensing of other A/GPL software companies like Cal.com because it's not even their copyright in the first place. Therefore, all WordPress changes must continue to be GPL. The only parts that might need to be replicated are things that are not necessarily part of the core codebase, such as the plugin repository, but it seems people are already starting to move towards a decentralized model for these.
The lawyer they've hired is a very serious guy who is concerned about his long term reputation. It's unlikely he would hostage his fortune to someone who was a "pissed off-preteen." Reasonable people may differ but I don't believe you have correctly assessed the players and the sum of forces at work in this situation.
I have sites on WPE and I've worked for a major WP plugin developer. I've been in and out of the community since 2010. I know the players pretty well.
Matt has sabotaged any chance of a good result for him and Automattic no matter how good this lawyer is. And even if he wins in court he's already lost in the court of public opinion. Its over.
We have been using and building WordPress sites since 2006. I have no experience with WP Engine but I bear them no ill will. I think the "court of public opinion" is very difficult to assess and dominant players can take a very very long time to die.
Matt may need to step down if your assessment is correct but that's distinct from what happens to Word Press as a platform.
My sense is that the Word Press negotiating position is stronger and WP Engine will either have to fork or make a much larger contribution. But I may be wrong. If that does not happen then I believe that the private equity players will do a lot more damage to open source communities because a "harvesting paradigm" will continue.
I generally agree except for "the court of public opinion" part. It seems abundantly clear to me that Matt has damaged his name beyond repair.
As I see it, the only way for WordPress to survive is if Matt steps down from the board, and the foundation & dot org mess is reorganized in a manner that makes it accountable to the community. Even then, a lot of damage has been done to the reputation of WordPress.
I won't be surprised if we eventually start seeing mainstream reporting on this case from the WSJ and NYT. Luckily, it's an election cycle so that might not happen for a while.
Reasonable people may differ. We have very different perspectives on the likely evolution of the current situation. I believe Matt will remain CEO and a year from now, the voluntary departure buyouts from WordPress will be viewed in the same way that 37 Signals weathered the buyout departures of some of their staff. See contemporaneous report at https://techcrunch.com/2021/04/30/basecamp-employees-quit-ce...
Firms that dominate markets enjoy tremendous inertia in customer choice and can take a long time to die--if, in fact, WordPress is dying. The longer-term perspective on what Silver Lake has been doing to profit from open source efforts by Word Press may reach a very different conclusion about Matt's actions than your assessment.
>Firms that dominate markets enjoy tremendous inertia in customer choice and can take a long time to die
That's a really good point. But I think (& hope) it's a little different in the open source world, because people have stronger moral opinions about where they hang their hat. We'll see.
I work for an agency that is part of a (very) large holding company. The IT dept for the holding company mandated that all agencies within to host their WordPress sites in WP Engine. I manage the sites for my company and two sister agencies, and just between the three companies there are 18 sites, not including the Dev/Staging variants! There are over 1500 agencies...granted not all have WP sites, but I am certain that many do. So if this reflects on the IT manager who made this call, it would be hugely magnified!
1. There are many to launch but most of the sites don’t have the community or credibility that PH seems to have lost.
2. If you want what PH was 3-4 years ago you have to go somewhere else.
3. If you want to build the new Product Hunt you can’t just create a site where people can submit their product and launch. You need affordances that cultivate community.
4. Once you get traction you should expect some members will game your system: successful systems always attract parasites.
I don’t believe in the value of the Product Hunt “community”.
The point of marketing is to get your product in front of customers, not to get it front of members of a community of deadbeat marketers who think they’re doing their job when they get their product in front of non-customers. If your product is “one weird trick” to get your product on the top of Product Hunt that’s appropriate for Product Hunt, but that’s about it.
One of the challenges in reaching customers is that someone else may have already assembled groups of them in the form of website audiences and communities. Some of those sites will accept sponsorships or ads, others--for example HN--are the result of audience members voting on your submission.
Assuming you believe that getting visibility on Product Hunt will draw the attention of prospects--I agree a potentially dubious assumption--you may need to find ways to cultivate "deadbeat marketeers." From what I understand, not everyone who submits a product the first time gains widespread visibility, so they must submit multiple times. Here might be a second reason to consider how to be a member in good standing of a community whose approval will be of benefit.
Applying the dye to lab mice made their skin temporarily transparent, allowing Stanford University researchers to observe the rodents' digestive system, muscle fibers, and blood vessels, according to a study published September 5 in Science (https://www.science.org/doi/10.1126/science.adm6869)
reply