Correct take, Infact I would argue enforcing work in the office probably accelerates the current tech oligopoly guard in their transformation into IBM like businesses
Exactly, this was the scary part for me. First time I've ever seen a public emotional frenzy escalate to a point where the majority agreed to suspend due process, for things that could have been prosecuted normally
I'd say that the suspension of due process started long before those protests began, back when the various levels of government started imposing forced lockdowns, forced masking, forced injections, travel bans, and curfews, among the other violations of the most fundamental of rights and liberties.
We should also keep in mind that the majority of citizens were not in favour of such actions. If the Canadian public had widely supported such measures, then they would have been self-imposed, and just voluntarily happened. The governments wouldn't have had to resort to mandates and other forceful methods like they did.
Seriously this is the part I dont understand about people parroting "prompt engineering" . Isn't it really just throwing random things at a non deterministic black box and hoping for the best?
I find it's more like that silly experiment where you have to make a sandwich exactly as a kid (or adult) writes the instructions. You _think_ you have a good set of instructions and then you get peanut butter on the outside. So, you revisit the instructions to be clearer about what you want done. That's how I see prompt engineering. In that case, you are simply learning how the model tends to follow instructions and crafting a prompt around that. Not so much random, more purposeful.
> That isn’t the model reasoning. That’s you figuring out exactly what parameters you need to use to make the model give the result you want.
If its to get the model to present a fixed answer, sure.
If its to get a model to do a better job at solving general classes of problems (such as when what you are optimizing is the built-in prompt in a ReAct/Reflexion implementation, not the prompt for a specific problem), that's, at a minimum, different from Clever Hans, even if its not “reasoning” (which is ill-defined).
If someone says they're fine tuning a model (which is changing which layers are activated for a given input) it's generally well tolerated.
If someone says they're tuning a prompt (which is changing which layers are activated for a given input) it's met with extreme skepticism.
At the end of the day ML is probabilistic. You're always throwing random things at a black box and hoping for the best. There are strategies and patterns that work consistently enough (like ReACT) that they carry across many tasks, and there are some that you'll find for your specific task.
And just like any piece of software you define your scope well, test for things within that scope, and monitor for poor outputs.
> If someone says they're fine tuning a model (which is changing which layers are activated for a given input) it's generally well tolerated.
> If someone says they're tuning a prompt (which is changing which layers are activated for a given input) it's met with extreme skepticism.
There are good reasons for that though. The first is the model-owner tuning so that given inputs yield better outputs (in theory for other users too). The second is relying on the user to diagnose and fix the error. That being the "fix" is a problem if the output is supposed to be useful to people who don't know the answers themselves, or if the model is being touted as "intelligence" with a natural language interface, which is where the scepticism comes in...
I mean, a bugfix, a recommendation not to use the 3rd menu option or a "fork this" button are all valid routes to change the runtime behaviour of a program!
(and yes, I get that the "tuning" might simply be creating the illusion that the model approaches wider usability, and that "fine tuning" might actually have worse side effects. So it's certainly reasonable to argue that when a company defines its models' scope as "advanced reasoning capabilities" the "tuning" might also deserve scepticism, and conversely if it defines its scope more narrowly as something like "code complete" there might be a bit more onus on the user to provide structured, valid inputs)
Neither option implies you own the model or don't: OpenAI owns the model and uses prompt tuning for their website interface, which is why it changes more often than the underlying models themselves. They also let you fine tune their older models, which you don't own.
You also seem to be missing that in this context prompt tuning and fine tuning are both about downstream tasks where the "user" is not you as an individual who's fine tuning and improve prompts, but the people (plural) who are using the now improved outputs.
These aren't the contexts that invite the scepticism though (except when the prompt is revealed after blowing up Sydney-style!)
The "NN provided incorrect answer to simple puzzle; experts defend the proposition the model has excellent high-level reasoning ability by arguing user is 'not good at prompting'" context is, which (amid more legitimate gripes about whether the right model is being used) is what is happening in this thread.
Technically I'm taking a large liberty saying you're "activating layers", all the layers are affecting the output and you don't pick and choose them
But you can imagine the model like a plinko board: just because the ball passes every peg, doesn't mean every peg changed it's trajectory.
When you fine tune a model, you're trying to change how the pegs are arranged so the ball falls through the board differently.
When you prompt tune you're changing how the ball will fall too. You don't get to change the board, but you can change where the ball starts or have the ball go through the board several more times than normal before the user sees it, etc.
You can't see the ball falling (which layers are doing what), only where it falls, but when you spend long enough building on these models, you do get an intuition for which prompts have an outsized effect on where the ball will land.
No, its not. While GPT-4 (like some but not all other LLMs) is somewhat nondeterministic (even at zero temperature), that doesn’t mean there aren’t things that have predictable effects on the distribution of behavior that can be discovered and leveraged.
There’s even a term of art for making a plan up front and then hitting it with a low-skew latent space match: “Chain of Thought”. Yeah, it’s seen numbered lists before.
And if at first you don’t succeed, anneal the temperature and re-roll until you’ve got something that looks authentic.
You got me beat: IMHO these things are plenty friggin awesome already and getting cooler all the time. I don't see why there is so much ink (and money) being spilled trying to get them to do things more easily done other ways.
Language models are really good at language tasks: summarization, sentiment analysis, borderline-creepy convincing chatbots, writing pretty good fiction at least in short form, the list goes on and on. At all of the traditional NLP stuff they are just super impressive.
They already represent an HCI revolution with significance something like the iPhone as a lower bound: it's a super big deal.
But while the details are absurdly complicated and the super modern ones represent an engineering achievement up there with anything ever done on a computer, they still fundamentally predict some probability-like metric (typically still via softmax [0]) based on some corpus of tokenized language (typically still via byte-pair [1]).
And when the corpus has a bunch of conversations in it? Great at generating conversations! And when the corpus has some explanations of logical reasoning? Often passably good at looking logical. And when the corpus has short stories, novellas, and novels featuring conversations between humans and science-fiction AIs? Well they can sample from that too.
But imitating William Gibson doesn't make GPT-4 any kind of sentient any more than it makes me a once-in-a-generation science fiction author.
“Real motive problem, with an AI. Not human, see?”
“Well, yeah, obviously.”
“Nope. I mean, it’s not human. And you can’t get a handle on it. Me, I’m not human either, but I respond like one. See?”
“Wait a sec,” Case said. “Are you sentient, or not?”
“Well, it feels like I am, kid, but I’m really just a bunch of ROM. It’s one of them, ah, philosophical questions, I guess...” The ugly laughter sensation rattled down Case’s spine. “But I ain’t likely to write you no poem, if you follow me. Your AI, it just might. But it ain’t no way human.” [2]
As a Canadian who moved to the US for tech, I strongly advocate for any young Canadian with capable means who is reading this to leave Canada.
An already broken system is being stacked against you
Housing prices are more expensive in Canada.
Health care systems are crumbling within Canada.
Wages are lower in Canada.
General cost of living is more expensive in Canada
I was frankly shocked after a lifetime of watching CBC just how much my quality of life improved by moving to the USA.
The US is by far the most favourable place to be a highly paid professional but I do worry about whether it is the best place to raise a family.
The level of violence here is just so much higher than other countries. Even living in San Francisco where the murder rate is half that of the US it is still double that of Europe and Canada. There was a gun battle at our local play park a couple of weeks back.
The house price / wage ratio in Canada is shockingly bad though.
> The level of violence here is just so much higher than other countries. Even living in San Francisco where the murder rate is half that of the US it is still double that of Europe and Canada.
The Bay Area is not representative of the rest of the US. Crime is out of control in SF proper and the Bay Area more generally due to reduced enforcement.
The Bay Area is definitely not representative of the US as it is far safer overall. According to CDC Wonder for 2021 deaths from assault per 100k population were:
San Francisco County 4.9
California 6.3
USA 7.8
Santa Clara County is safer at 2.7 though I would really miss living in a walkable neighbourhood.
That’s a bizarre statement. SF has ugly areas but still feels very safe. Would feel much more comfortable being dropped off in the worst places there vs Chicago, St Louis, LA, Miami, New Orleans, Atlanta and a bunch of other places
Downtown San Francisco can be unpleasant but you are more than 10x as likely to be killed in Orleans Parish (51.7 deaths by assault per 100k) than San Francisco County (4.9).
I'm really not trying to cherry pick stats, death by assault / homicide is a pretty good indicator of levels of violent crime. SF-proper (i.e. San Francisco County) has a huge and highly visible homelessness problem and is definitely an outlier on that dimension. But while that is unpleasant it is less worrying than actual violence.
The US limits gun control so even in states like California that try to do something about it there are many more deaths than in countries which actually do something about it.
Couldn't you just move outside of the city where is it significantly more safe? There are plenty of options between living in San Francisco and moving to Europe. I live in Seattle, and while there are _plenty_ of problems, I don't think it would be a bad place to have a family.
At least according to CDC Wonder, King County has about the same rate of death from assault at 4.8/100k vs 4.9/100k for San Francisco County.
Edit to add: It's not just the absolute levels that I worry about but the effect that ever present gun violence has on society with kids being subjected to frightening active shooter drills in schools that just aren't necessary in other developed countries.
I don't know. Obviously school shootings are bad. Bad things will always happen as long as humans exist, and these bad things have some impact on children. We'll always want to reduce the amount of harm our children are exposed to.
But, isn't being a child in the United States significantly better than being a child _anywhere else_ 100 years ago? Even with the psychological harm caused by school shootings, hasn't there been a net improvement? I would much rather be a child worrying about a school shooter (with, in reality, a minuscule chance of it physically effecting me) versus being a child during the great depression, either of the world wars, or the cold war.
It's a very logical argument and coming from someone nowhere close to starting a family though; I am sure that most parents might agree but would continue to want their kids to be safer.
I'm not trying to say that none of this matters... I guess what I'm trying to say is that, those kids ended up alright in far worse circumstances, so kids today will be alright too.
I had the exact opposite experience moving from Canada to San Francisco. While wages are much higher, the cost of living is not even comparable - anecdotally much, much higher in SF than either Toronto or Vancouver (both of which I have lived in). Healthcare has been a horrible experience here too, with Kaiser putting my through endless levels of bureaucracy in an effort to avoid paying for my medication.
But when did you leave Toronto? Because the situation is getting exponentially out of control. The housing market in southern Ontario & lower mainland BC is a pyramid scheme. When my wife & I first bought our house in Toronto in 2005 in a "bad" neighbourhood (Oakwood-Vaughan) it was a bit of a squeeze on our dual tech-worker salary, but we were able to do it. Fast forward almost 20 years, we would not be able to afford what that house goes for unless we financed to like, a 35 year mortgage, and my compensation has gone way up from back then.
Meanwhile COVID f'd up the health care system extremely badly and there's no real commitment from the province to getting the funding situation under control.
No it's not normal, they are talking a bit out of their ass. Based on their dates, I'm significantly younger than them and yet I was able to afford a house in one of the more desirable neighbourhoods with only me being the one working in tech.
I'm not targeting the person you are replying to with any malice, but since almost all of the major financial and business institutions in Canada are headquartered here there is an overabundance of people that would claim they work in "tech" when in reality they are making a respectable but decidedly non-tech salaries at places like TD Bank or Thompson Reuters as examples.
The range of possible salaries for devs in Toronto is quite large.
Also as an additional anecdote, every single one of my classmates who went to the USA and decided they would like to start a family, came back to Canada to start that family.
That is not to say it is all rosy here. There is an overabundance of poor or terrible talent that's been shipped in to cover the exodus of Canadian educated people chasing better salaries in the USA while business leaders and purse string holders are content to celebrate their mediocrity while being confused why productivity is so low.
Wow, you're classy... and yeah, I feel targeted a bit ... hah
Our old house @ Oakwood & Vaughan was bought for $285,000 in 2005. It's likely "worth" north of $1M now, not 20 years later. My senior software engineer salary in that period was between $75 and $100k CAD. Are you saying that a SWE salary in Toronto is over $300k now?
I know it isn't, though there are plenty making more than that Google Canada, that is a huge anomaly from the rest of the market.
The distortion in housing prices and the continued upward growth has a negative effect on the ability of young people to prosper. It might help me retire, sure, but it isn't going to do much good for my kids.
BTW, I worked as a SWE at Google for 10 years. And my wife was at Apple before that. Does that count as "tech?" Just checking.
I work as a software engineer at a company whose only products are software. Are you saying I shouldn't claim to work in tech because I don't make FAANG money?
Similar experience but my reason for staying in the US is purely for the weather and the money. Life in Canada is so much more pleasant than any city I've lived in in the US.
Instead of moving hours away to Toronto or Vancouver (if you don't live there), just move the same distance south and you can find cheaper housing, higher wages, more job opportunity, lower taxes and lower cost of living in general.
If you look at the historical ratio of income to the price of computers we're literally orders of magnitude wealthier. You can't just pick one data point and blast off.
You're ignoring interest rates which allow the overall price to go up while keeping the monthly payment the same and the fact that houses have gotten a lot bigger over the decades [0].
Freedom.to has been working pretty effectively for me on Android.
You guys care to comment on why your approach may be better? Definitely willing to try you guys out
We don't have an android solution yet, so stick with them for now :)
We found Freedom’s VPN solution to be great for completely blocking out distractions when you don’t need any internet and want to focus in on certain tasks. That can be helpful for mobile too, but we've found the day-in-day-out experience of injecting "intentionality" just-in-time to be more effective for us than pre-scheduled “allowed” and “not allowed” blocks.
