Very cool research! I really like these applications of transformers to domains other than text. It seems it would work well with any domains where the input is sequential and those input tokens relate to each other. I'm looking forward to more research in this space.
HN what do you think are interesting non-text domains where transformers would be well suited?
It's mentioned elsewhere in the thread, but I've had good success with WaniKani[0]. As an aside, the company behind it, Tofugu[1], also have a lot of good free resources.
The main tag line on the WaniKani website, "2000 Kanji. 6000 Vocabulary words. In just over a year." is very optimistic, I'm around level 12 (of 60) after that long. It might be possible to do it all in a year, but you need to put in a lot of work.
Shameless plug: I created a free and open source alternative to WaniKani https://shodoku.app/ using open dictionary data and the same SRS system as anki.
It actually has a very different learning philosophy from WaniKani so it is not really an alternative.
* shodoku teaches writing as well as reading, the point being that writing it helps you remember it.
* You learn components (radicals) and vocabulary at the same time as the associated kanji.
* The order doesn’t need to be by simplicity. This is deviates from both WaniKani and Remembering the kanji.
* You rate your self, just like anki.
I find it is actually more important to learn the kanji in the words you are learning, if a new kanji has three new components, it is not hard to simply learn these new components at the same time (and create a story / connection of them). And learning the reading of the kanji is easier if you learn words containing it. So what I do is I bookmark a couple of words each time I start a new kanji card, and during reading review, if I remember how these words are pronounced, I rate it as good.
Already the US can serve as a good example when discussing the need for unbreakable cryptography and e2e systems. The current decline nicely illustrates how quickly you can go from "The police have legitimate needs to break encryption to find heinous criminals" to something far more dystopian.
No amount of crypto is going to protect you from this mess. Technical safeguards work as long as it is backed by the law and the constitution. But when they are suppressed, the people in power will just find someone smarter than you and bribe, gaslight, bully, blackmail or beat them into helping them compromise such safeguards. And not to mention the fact that they love playing hideous psyops games. This is a social and political problem. You need social and political solutions. Technical solutions are just band-aids.
> No amount of crypto is going to protect you from this mess. Technical safeguards work as long as it is backed by the law and the constitution. But when they are suppressed, the people in power will just find someone smarter than you and bribe, gaslight, bully, blackmail or beat them into helping them compromise such safeguards.
I don't agree. Having unbreakable crypto is the absence of a tool. My point is that a democratic government can create the tool with good intentions, but you are only one election and a few months of backsliding away from the tool being used for nefarious purposes. You are right that technical solutions are just band-aids, but if you never create the tool it cannot be abused by a new authoritarian government.
For this contrived example it isn't better. I was very sceptical of tailwind before I tried it, but in certain scenarios it's nice. I think it shines if you are already using a component focused development process. As others have said you don't need to come up with class names for every thing you want to style. Rather than cross-reference between js/html files and css files everything (structure, style, and interactivity) is in one place. You also don't need to care about specificity.
Personally I think it gets a bit mad when you start wanting to do :hover, :active and media queries, but you can still use classes (using the @apply directive) for that if you wish.
But you can use CSS rules to override variable values for more specific selections. So you can define a global --primary-button-color at the root, apply it to every button, then override --primary-button-color in, say, `tool-container`.
It's possible to use with vanilla, but probably not a good experience. You want either components or at least a templating system where you can break things down into reusable parts.
It's somewhat similar to raw HTML vs something like PHP, you can have thousand of HTML files, but if you wanna change something common (say the header) it's not fun with thousands of individual files.
Lastly you can combine Tailwind with regular css classes
.btn-primary {
@apply bg-blue-500 px-2 py-1;
}
but at that point I'm not sure how much you are gaining by using Tailwind
Speculating about her motives isn't fruitful, because her motives don't matter particularly. It has many upvotes because the information in the book is newsworthy and relevant for a place like HN.
The book is a good read and she also testified in front of the Senate Judiciary Committee[0], repeating many of the claims from the book under oath. One of the striking things is that it's clear that Mark and several others from Facebook perjured themselves in prior hearings. I expect there will be no consequence for this.
I believe it will take at least a couple of generations after a new political ideology is cemented in the USA to change anything.
Market fundamentalism has been the game since the 80s with Reagan, it was building up to it but Reagan was the watershed moment when it really gripped. You see it everywhere now, here on HN especially, any deviation from the dogma of market fundamentalism is met with the usual retort about "innovation", "growth", and all the buzzwords implemented to make it seem to be the only alternative we have. Any discussion about regulation, breaking down behemoths wielding massive power, betterment of wealth distribution, workers' rights, etc. will attract that mass who are true believers of the dogma.
To undo this will require a whole political ideology from the ground up in the USA where the two parties are just two sides of the same coin, I really cannot see how this can realistically change without a series of major crises, bad enough that people will rise and understand who exactly is fucking them... It's sad to realise there's much more pain to happen before it might spark real change, we are kinda bound to live in the aftermath of the erosion of society brought by "shareholder value"-hegemony.
During the Great Depression the Americans did pull together and demanded from President Roosevelt a social reform. That was called the New Deal Coalition.
This time though the fight will be much harder because even the democrats are so strongly indoctrinated in the "free market" idolatry that they are much closer to the republicans than any true social democratic movement (such as labor unions) that would actually be needed in order to help the American working (and soon ex-middle) class.
I think you and the person you're replying to agree.
We won't get a New Deal Coalition Part 2 without our own Dust Bowl (climate-change/industrial-agronomy-induced disasters, and the massive disruption to peoples way of lives that accompanied it) and Great Depression to conclusively demonstrate that industrialized, financialized oligarchy "doesn't work".
The two-party system was just as much captured by "free market" idolatry pre-FDR as they are today. There was nearly three decades of socialistic organizing in response to crisis in the 1890s-1920s before we finally had those principles manifest in one of the two major political parties in the executive branch, with FDR in 1932.
We're barely into the nascency of our own century's progressive era. If history's any guide, it'll probably take decades and it will get much, much worse before it gets better. :/
I re-read Grapes of Wrath recently, and it was an uncanny feeling: like I was reading something that was both near-future Sci-Fi and a memory-holed but relatively-recent history.
These crises are occurring right now so I don’t think it will take multiple generations. The rise of neo-fascism, the climate crisis, and the escalating warmongering toward China - a nuclear power - should be seen as symptoms of a system breaking down because it prioritizes profit over people. Intensification of capitalism’s worst tendencies is the capitalist’s last stand. It’s either going to end in mass destruction or people throwing off their chains.
This is very much what professor Richard Wolff is saying.
What you're witnessing down is the systemic failure and breakdown of a system (capitalism) that is completely out of control and ultimately starts to attack the very institutions that enable it in its greedy search for "growth" (i.e. producing more wealth for the already wealthy).
I haven't listened to the talk but read Mobiot's book when it came out last year :)
On the same vein, I'd recommend "Capitalist Realism" by Mark Fisher, Naomi Klein's "Shock Doctrine", and even the original "A Neo-Liberal's Manifesto" by Charles Peters to understand how the term is slippery and diverged a lot from the original manifesto.
And I'm not an anti-market, full-blown communism person. The feeling I have is that all the aftermath from the dogmatic implementation of an unsound ideology has brought much of our contemporary malaise, the allowance of finance to take over the real economy, the productive economy, has just eroded any semblance of a good market-driven society. I'm against that, the supremacy of finance over all other economical activity, it's a cancer that festers on every single big corporation.
>And I'm not an anti-market, full-blown communism person
Also, it is interesting that you feel the need to say you aren't a communist before criticising the current system. I guess that is a sign of just how entrenched it is.
I see it more as a sign of how few mainstream alternatives have been proposed. I've been guilty of generally assuming a communist bent when I see a negatively zealous response to the "free market" ideology. I don't act on the assumption, but from my experience, it tends to be the most common result.
Our political system seems hell-bent on only ever having two solutions to a problem, though.
We seem to be stuck at a local maxima[1]. The current system works great for the 0.001% who have all the money and the power, so it isn't in their interest to change it. But there definitely seems to have been a failure of the imagination amongst the 99.999%. Too distracted by social media and our phones perhaps?
[1] There is a lot to dislike about the current system, but there have been far worse ones (feudalism, communism etc).
> Too distracted by social media and our phones perhaps?
I think this is a significant contributing factor. It's becoming increasingly difficult to have any semblance of a meaningful conversation with those around me. I don't really know how to describe it other than an apparent "dumbing down" of the average person. I despise elitists, and I hate to even act in a way that might come off as elitist, but I simply have no other explanation for what I am seeing. People just want to talk about the latest trend on TikTok and have no interest in applying anything close to intellectual thought to what's happening around them.
Weirdly, right at the moment the US economy is tanking because of severe departures from market fundamentalism. By the people who most claim to be pro capitalism.
There's no departure from market fundamentalism, the belief in shareholder value being supreme is still very much the current Zeitgeist.
As much as the USA's administration is jerking around with trade, the fundamental principle of what governs any corporation is still market fundamentalism: returning value to shareholders, nothing else.
Shareholders are pretty grumpy right now. They've lost a ton of money.
I'm sure that some economist will asset that this will produce more shareholder value in the long run. But the stock market suggests that shareholders do not currently believe that.
Capitalism is incompatible with free markets. Capitalism means all the wealth goes to those with the most capital, while free markets means the wealth flows freely in all directions.
To me that is the biggest win in public discourse from capitalists: conflating markets with capitalism, as if free markets could only exist under unbounded capitalism. Which, as you say, is incompatible. Capitalism does not want free markets, nor foster free markets, the best end result for a capitalist is the abolition of a market under the control of a monopoly.
Markets are fundamental, and a natural result of human socioeconomic order. Capitalism not at all.
> Capitalism means all the wealth goes to those with the most capital, while free markets means the wealth flows freely in all directions
I don't understand this distinction, why wouldn't capital accumulate under free markets? The freer the market the more capital accumulates.
In a freer market that today you would have to pay a massive toll every time you went to the grocery store, because the road owner has monopoly on that route, that would lead to much more wealth accumulation.
In economics perfect markets mean that your company that raises spherical cows has no moat against others doing the same. If you do something to gain profits to become rich someone else joins the market to compete those profits down to zero. This reduces inefficiency and makes everyone rich.
Deregulation is sold as getting closer to this, in reality it means the money collects wherever the market breaks down, monopolies, network effects, externalities, concentrated special interests, middlemen, oligarchies, gangsters, landlords etc.
When all of the capital ends up in a small number of hands, the market ceases.
Each capitalist tries to corner the market, but if they succeed, the resulting monopoly isn't a free market. In theory a competitor arises, but it takes only an instant to shut it down and restore the monopoly.
>I don't understand this distinction, why wouldn't capital accumulate under free markets?
It would, which is why businesses support deregulation - not because they believe in vigorous competition for the sake of consumers, but because they want as little friction and consequence standing between themselves and oligarchy as possible.
A market in which the wealth "flows freely in all directions" is socialist, not capitalist. "Fair" markets are regulated, and by definition not free.
I mean, I guess the obvious question is if one person lied under oath (her) or several (all the people that her testimony implies perjured).
The book sounds pretty outlandish. That's not to say that Zuck and co aren't just a whole gang of melodramatically evil and stupid people, but it a priori it seems just as probable to me that she's the one that is? I don't know much about her. Is she a reliable witness?
The indignation in this thread and from OP is ridiculous, especially when all the ire is aimed at Firefox, who are doing the same thing all browser makers do.
When you are changing the very fabric of the whole web, rolling things out in a gradual, controlled way is paramount. Not just because people can find and report issues before roll-out reaches 100%, but also because browsers collect telemetry on features and how they work in the wild that can be used to gauge the effect.
Steve Klabnik talked about how he and rest of the early Rust team recognised the importance of tools like this on a recent Oxide and Friends episode[0], it's well worth a listen.
I think this is a positive sign for Swift and Apple's continued push to increase it's viability as a general purposes language outside of Apple's ecosystems. It's still early, but directionally it's promising.
I know a number of people at Apple, some of them pretty high up. The long and short of it is, Apple doesn't really give a shit about the world outside its ecosystem. It will interoperate when it has to (e.g. wifi, bluetooth, web standards) but if an effort does not help with selling Apple hardware, that effort will not get headcount or funding. So whatever multiplatform efforts you are seeing happen in spite of, not thanks to, Apple's stance.
I don't doubt that, but someone is definitely trying.
For example they moved Swift out of the Apple GitHub org and the announcement blog post[0] calls out Swift's use outside Apple's ecosystems.
Their blog archives has more posts that I read as a focus on improving Swift outside the Apple ecosystem, some examples:
* Updating the Visual Studio Code extension for Swift[1]
* Introducing Oblivious HTTP support in Swift[2]
* How Swift's server support powers Things Cloud[3]
I can see arguments for how wider adoption of Swift outside of Apple, and especially in the server ecosystem, helps Apple. In particular, lots of apps are being written using React Native and Flutter and at least one hurdle there is that Swift lacks developer mindshare.
It’s nice that a Swift LSP exists, is supported by the company behind the language, and is actively being worked on, because that makes it much more feasible to do things like build alternative IDEs.
Contrast this to Kotlin, which doesn’t have an LSP due to conflicts of interest with Jetbrains’ business. You either use a Jetbrains IDE or you don’t write Kotlin.
This and the whole anti-Java attitude on Android circles is what made me disregard Kotlin, how can one be so out of touch with what makes their ecosystem possible in first place.
Well, for Android specifically I’m not sure anti-Java sentiment is necessarily entirely unjustified with how Android tends to be stuck on some ancient version, and it’s not as if the JVM ecosystem doesn’t have its warts (though those are largely shared by Kotlin).
Which is made on purpose to sell Kotlin on the platform, those having had this decision are part of the same anti-Java group I was mentioning, as per remarks on podcasts and other appearances.
It’s wholly a community effort and not supported by Jetbrains, which means it will always be at a disadvantage. Even development on it continues, it’ll always be a few steps behind the current version of Kotlin and will probably be missing a number of things compared to the autocomplete in Jetbrains IDEs.
It’s also built on the internal APIs of the Kotlin compiler, which are subject to change at any time. If Jetbrains shifts those around too dramatically it could break the LSP beyond the point that anybody is willing to repair it.
For corporate-developed languages first party LSPs are important.
My guess is that 30k LOC is the 1st party code, and there’s probably a bunch of 3rd party dependencies that are also compiled as part of that. But maybe I’m wrong
Apple so very clearly does not give a shit about anything outside of the Apple ecosystem, it’s kind of a core part of their company DNA at this point.
I can’t ever see Swift on the server or in other contexts catching on and honestly just with the way platforms as a whole are developing I can’t see Swift surviving as a mainstream language beyond the next 5 years. Its value proposition is already on shaky grounds and is actively getting worse with time the moment you look outside Apple’s walled garden. Unless they are able to keep that lock in at the same level moving forward it’s just got very little going for it.
I think something like 1/3 apps in the App Store are ALREADY written in Dart / Flutter which is probably even a nicer language from a developer experience point of view, with much much better tooling and documentation, that runs everywhere and has comparable performance.
> I can’t see Swift surviving as a mainstream language beyond the next 5 years.
I got into macOS development when Swift was first released, and used it heavily up until 2022 or so. I think I agree with you. The compiler is just too slow, and the language is too complex. And I think the issues are fundamental.
It sucks because I’m willing to develop for macOS exclusively, but the whole package is so rough and frustrating. Somehow they had every advantage (endless resources and complete platform control) and couldn’t put something together that’s better than Rust/Zig/whatever. Not to mention the inability to ship any useful AI developer tools, the dumb constraint of only shipping new major updates once a year at WWDC, the lack of ANY modern game development framework, etc..
I think I’ve finally talked myself into giving up on it.
Apple should have incrementally improved Objective-C. It was pretty nice to work with. Sure there were some warts, but it was quick to compile and easy to grok. Swift is a huge ball of slow complexity.
The decay in Apple software quality has neatly coincided with the adoption of Swift.
I can’t speak for Zig, but it seems like Rust is rather awkard with imperative UI frameworks, and instead is more inclined toward declarative/reactive frameworks. That’s one area where Swift is a bit more flexible, and one reason why I like it.
While I like to write simpler bits of UI in something like SwiftUI (think small components, recycled cells, etc) I find that declarative UI gets increasingly cumbersome as projects gain more features and become complex, and that doesn’t change much with the language it’s written in. As such my projects tend to be imperative-dominant with declarative components and maybe simpler screens sprinkled throughout. UIKit and SwiftUI work together nicely for this.
The only issue is the lack of UI frameworks for non-Apple platforms. There’s decent GTK+Adwaita bindings for Swift which is pretty solid for Linux, but to my knowledge the only thing out there for Windows at the moment are WinUI bindings written by The Browser Company for Arc, but as I understand it those are still pretty incomplete.
Compile times haven’t be a problem for me, even with complex codebases. Incremental builds are fast enough and I’m not running full builds often enough for that to impact my overall evaluation of the language.
> The compiler is just too slow, and the language is too complex.
also error messages are next to useless in many cases
this really trips up new users and (speaking from experience) unless you have someone that can mentor them they're gonna give up pretty fast
swift is evolving so quickly to fill gaps in capabilities but the tooling and ux of actually coding (speed, error messages, fixits etc) really needs heavy work badly imo
Since Apple cares about their ecosystem so much, I wish they would give a bit more of a shit about Xcode.
I wish they'd keep updating AppleScript more, or just completely replace it, but I do not blame Apple on that one. Swift could absolutely replace AppleScript in terms of functionality, but that wouldn't make Apple any money. So, we all know that will never happen.
I hadn't fired up Xcode in years. Yesterday, I compiled an open source app and was amazed how little the UI and UX has improved. I used to do quite a bit of Objective C mobile development up until 2012 or so.
im the opposite, i want them to give up and switch over to open source tooling - instead of all the bespoke apple stuff - we should be able to build apps with lsp and swift package manager imo
the xcode ui part can be open sourced so the community evolve/improve it better since apple has like max 3 people working on it apparently
The story of Swift on server is for Apple shops doing iDevices development, which server backends, remember Apple no longer sells OS X servers.
Additionally Apple isn't Google, Swift has more chances to survive as proprietary language on Apple ecosystem, than Dart as FOSS if Google ever gets bored as usual.
I kinda wish it did actually catch on outside the Apple ecosystem. I like the language quite a bit, but since I no longer program for MacOS even peripherally, I don't get to use it. But I'm afraid it's going to need a miracle at this point, Rust and Go have a massive, insurmountable lead on non-Apple platforms. It could be argued that Swift's catching on outside macOS would be very good for Apple, by expanding the pool of developers proficient in Swift, and removing the friction towards people using their Linux code on Apple systems. But Apple is blessed with an AppStore which is not a complete dumpster fire, so they probably don't feel the need to do any of that, and feel like they'd be sinking millions into an initiative that has zero chance of success.
That’s honestly depressing. I used flutter because I couldn’t figure out XCode, and am continually lamenting the poor type-checking, static analysis, and mediocre performance… how is swift worse?
I was asking the flutter app to act as a bluetooth mesh hub and it started dropping packets when the (external) simulator got to 5000 devices, when the same algorithm in C++ running on a pi 2020 (which is much weaker than the A17!) managed the full 16000 theoretically possible. The profiler (which… opened in a web browser?) tracked it back to most of the time being spent in averaging and rolling encryption, it wasn’t bluetooth-locked.
Developing was also frustrating, with Android Studio simply being unable to signal compilation errors, and many issues only showing up on one phone or another. The zeroconf libraries have different methods, but this could not be detected until runtime on either device. The device I wasn’t actively testing on frequently had some regression in sensors, bluetooth, or even UI.
It did display an animation OK, so no fault there I suppose.
That story never happened, I remember it well because everyone like you went insane about it but when the truth came out it ended up being that their headcount has literally changed by one person.
You’ve posted one dumb thing after another and pretended like you’ve made a great point after each one. That’s really a special kind of moron who does that. Have a good day.
Google has the attention span of a crack addled flea. It is crazy to base any part of your career on a technology that the company doesn’t use extensively internally.
It doesn’t worry you that Google doesn’t have enough confidence any Flutter to use it in its own native apps?
Just one app? Flutter will die like most google projects. At best it's a mediocre language that only does mobile. The kind of people who are fine with that kind fluff in their brain already picked swift and/or kotlin. They're not moving to a new language just to gain ONE single extra platform (none if they're already efficient in both)
I can’t tell if this is a parody account or you’re just completely mismatched between confidence and skill.
Your comment history seems to just be filled with you talking about topics that you clearly have zero conceptual understanding of with full confidence while you argue with people who try to explain to people how you are wrong.
Today it’s thinking flutter is a language or that it only does mobile or that nobody uses it. Like all three of those points are objectively wrong.
Yesterday it’s somehow managing to get confused between gRPC and MCP then the idea of stateless and stateful protocols and a bunch of other things.
Maybe you should spend a lot more time reading and less time confidently telling people they are wrong on topics you don’t actually understand.
You’re very much in a bubble. Most apps aren’t written using Flutter - including apps by Google.
It would be like Apple creating a new app and not using Swift.
It’s just an insane career move to be dependent on Flutter or any Google technology really instead of using Java or Kotlin or a more popular cross platform frameworks.
No confusion there, guy just asumming mcp should be higher than grpc in the thread where we are agueing where should mcp be. It's too idiotic for me to continue arguing with. Same to stateful vs stateless. Tool calling is stateless, if you wrap it up with a protocol and claim that it should be stateful, then it's reasonable to ask how exactly.
And then you "psst dart can do web and server" me, like kotlin, swift, typescript and basically everything else these days cannot do the same thing. No one takes server side swift or server side dart seriously, you don't pick a technology based on what it theoretically does, should be based on what it practically does, and that only.
More or less this, with some able to produce a few community nuggets, but it will take another vendor (and IBM tried at some point) to push Swift into actuality as a true cross platform language.
In another world, maybe instead of Java / Kotlin Google would have pivoted onto Swift as a language for mobile, closing the gap between the platforms a bit.
The complaint is that a lot of companies have a very shallow and short term sense of self-interest as if they were starved for cash and in a non-positive sum game
You don’t survive in the tech industry for 50 years and be the most valuable company for the last 15 by thinking short term.
This isn’t unique to Apple. Microsoft has been around for 50 years and been one of the five most valuable companies since 2000. Amazon has been around for 30.
whats your thoughts on uv https://github.com/astral-sh/uv I am blown away, installing python packages used to take so long now I dont even think about it, really made Python more viable.
HN what do you think are interesting non-text domains where transformers would be well suited?