tl;dr watch this fantastic intro to svelte talk by it's creator: https://www.youtube.com/watch?v=AdNJ3fydeao , it covers some of the growing pains of React that svelte addresses.
While it might look like the frontend is going around in circles, there are major & minor differences between the technologies, and they have each introduced novel (to JS at least) things... Off the top of my head (this timeline might not be right but it's how I went through it at least):
- Backbone got you away from jquery spaghetti, helped you actually manage your data & models. Often paired with Marionette to handle views (if you squint this is the precursor to components) and bring some order in that part of the app.
- Angular bundles backbone's data handling (services), marionette's orderly view separation (views, directives), but they made a fatal mistake in the apply/digest cycle and maybe encouraging a bit too much complexity. Angular is everything bundled together, with consistent usage/documentation/semantics, a familiar programming pattern (MVC), and a large corporate sponsor in Google it caught on like wildfire.
- Knockout sees Angular's MVC, laments it's complexity and focuses on MVVM -- simple binding of a model and a relatively simple reactive-where-necessary controller
- React comes along and suggests an even simpler world where the only first class citizen is components and everything else comes separate (this isn't necessarily new, there is an article out there comparing it to COM subsystems for windows). React is almost always used with react-router and some flux-pattern supporting data management lib -- these are also departures from how angular, backbone and knockout structured in-application communication (backbone was pure event bus, angular had the usual MVC methods, knockout was just callbacks -- if you have a handle to an observable when you change it things reload).
- Vue sees the complexity that React grew to (case in point: shouldComponentUpdate) and devises a much simpler subset along with a new way to handle reactivity -- using the `data` method of a vue component. Vue made a lot of decisions that helped it stay simple and small yet very productive and this is in large part thanks to React existing before hand.
- Svelte comes on the scene and realizes that truly optimal rendering could be achieved by just compiling away the excess and eschewing the virtual dom all together in most cases. No need to track all the structure if you compile the code with the updates necessary. Don't quote me on this but I think a whole host of things influenced this project -- smarter compilers, better type systems, the ideas around zero cost abstractions & doing more work at build time.
- Ember (formerly SproutCore) is actually an anomaly because it tries it's best to be both angular like (so large, and usable by large teams) and keeping up with the tech as it evolves (see: Glimmer, Octane). Ember also has some innovations, like Glimmer's VM-based approach -- turns out you can just ship binary data representing how to draw your components to the browser and skip a bunch of JS parsing, if you bring your own virtual machine that is optimized to draw the components.
As all this moves forward, there are ecosystem pieces like typescript that have gained tons of steam and changed how people are writing JS code these days.
How about this: Code can add value when it's a lot lower quality than we think it needs to be. Customers will pay for even a crappy solution if you are addressing the right problem. Don't over engineer. But also employ staff for the explicit purpose of gradually increasing the quality of codebases which need to continue to churn.
Not without a lot of messing around. With Intel you can pretty much just straight install macOS + Clover + FakeSMC and you're golden. With Ryzen you have to start messing around with custom kernels, which means updates will break things and also means a lot of programs won't work.
Last I checked a lot of progress had been made, but you're unable to run any 32-bit applications and some software such as the Adobe Creative Suite simply won't run.
I have been thinking about commenting and debate platforms lately, mainly two aspects
a) How to map comments/arguments in a way that the system by design encourages healthy debate and at the same time remain very intuitive to read (and contribute) for not so regular user
For example: I am big fan of http://en.arguman.org/ is great work in mapping argument but might not be easy regular naive internet users
b) How can platform ensure that majority which may be is in favor of certain side (of argument), it doesn't influence rational argument. I am referring to trolls.
Example: http://kialo.com/ is a good implementation but arguments can be pulled to one's side by sheer majority.
I probably knew indrora's grandfather, for though I worked for IBM, I consulted with Bell Labs and other parts of ATT for many years, in IBM and as an independent. In fact, I was twice the Bell Labs Distinguished Lecturer, touring all the labs--helping out with C and Unix.
As for math, it was one of my majors as an undergraduate, and I am a member of Pi Mu Epsilon, and have published a number of mathematical papers, though not recently.
But, I'm a hacker at heart, always have been, always will be. I use math only when it's helpful to solve problems. That's true of any tool I use. Why? Because the higher the math or esoteric the tool, the more you diminish the number of other people who can understand what you're doing.
Indeed, that's why I'm now writing fiction that helps people grasp a few mathematical and software concepts. For instance, my mystery novel, "Freshman Murders," has a team of math geniuses as the "detective." And I still write techie stuff, when I have something to say.
I use a Kärcher SC3 steam cleaner. It's cheap (~€100) and pretty good. Even the SC2 is decent. As an alternative, Polti high-end ones are excellent. They invented domestic steamers, and I've seen some used at hospitals. The have accessories for e.g. steaming your mattress to get rid of dust mites.
I use a Miele C3 Silence vacuum cleaner. It's the cheapest (~€220) Miele vacuum cleaner that has top (A class in EU) filtration. More expensive models just buy you more accessories. Sadly, Mieles are super expensive in the US, and you can't do arbitrage due to different voltage. Nilfisk made in Denmark used to be extraordinary, but now they are pretty mediocre. /u/touchmyfuckingcoffee is a very famous vacuum technician in Reddit, and he recommends Miele and Riccar. He doesn't like bagless. Neither do i. Worse filtration and a mess to dispose if you are allergic.
I use a Coway 1008 DH air filter. It's cheap (€400), efficient, and uses low energy. IQAir has great reputation, but in some tests it has shown mediocre performance [1]. Perhaps these tests are not measuring particles in the smallest range, where IQAir claims to excel. I've seen some guys from IQAir at HN, perhaps they can clarify.
For house cleaning I use stuff from Sonett. Bio-D is also good, cheaper and easier to find. I also use probiotics from Chrisal. In the US, these might not be easy to find. I'm sure there are equivalent brands there. Check their ingredient lists. Bio-D cleaning stuff is so safe their hand soap is basically the same formula as their countertop cleaner. Many items are certified by Allergy UK.
I'm happy to hear any other recommendations from fellow allergics. I also found a two-stage filter for my shower (Sprite + Vitamin C Sonaki) did wonders to avoid spraying chlorine into my airways and reduce inflammation and itchiness.
> anonymize your users IP addresses before sending the information to Google
That's a nice placebo that does almost nothing. Even if the packet body doesn't contain the IP address, it's still available in the IP header's Source Address field.
However, even if we assume Google - in a reversal of their general focus on gathering as much data as possible - doesn't recover the address from the IP header, their own documentation[1] for analytics collection URLs with the &aip=1 parameter (which should be present when 'anonymizeIp' is true) says:
"... the last octet of the user IP address
is set to zero ..."
Zeroing the least interesting 8 bits of the address doesn't make it anonymous. They still get to record the ASN, and they are recording at least 8 bit of fingerprintable data from other sources. I should be trivial to recover mostly-unique users, and calling this "anonymization" is at best naive and for Google, an obvious lie.
Their documentation even betrays their intentions:
"This feature is designed to help site owners comply
with their own privacy policies or, in some countries,
recommendations from local data protection authorities,
which may prevent the storage of full
IP address information."
Actually making the data anonymous isn't the goal. They just want a rubber-stamp feature that lets them comply with the letter of the law.
2.5 years ago, we tried pretty much all the GraphDBs out there. We decided to go with OrientDB (it was 2.0.something), but after only 3 months of POC, we switched to Neo4j. Why? Because the documentation was poor and we found some issues, so we couldn't trust OrientDB to be our primary DBMS.
With Neo4j everything was great at the beginning where we had 1 server only, but when we switched to the Enterprise Edition for HA (special price for startup, otherwise it would be unaffordable. Without the startup price it would be around $100K per year for 3 servers, crazy!), it didn't scale for our use case (primarily because of its master-slave architecture and other bugs and limitations here and there).
Our use case is 50% read and 50% writes, we have seen that Neo4j is mostly used as an Analytical database, so maybe with 100% reads it could be ok.
So on January 2017, we evaluated the new DataStax DSE, but it was really unstable (we think it's still an alpha version, but for marketing reasons, they pushed it as a stable GA). So we decided to give to OrientDB another chance. We've wasted $20 on buying the only available book for OrientDB, but it treats OrientDB as a Document database. After reading the OrientDB manual, we converted our Cypher queries to the new OrientDB SQL MATCH (that is very similar) and in less than 2 weeks our application was up & running with OrientDB and 5 servers.
We found 3 minor issues the team resolved in a few days, we have another issue that is still not fixed, but so far so good, we're happy in production with a 100% uptime in the last 4 months. I've seen OrientDB is much faster with multiple servers. We decided to not use sharding with OrientDB for now because it doesn't look mature enough, but maybe we will try it in 1y.
So my final advice is: Neo4j is very expensive, but it works very well for read-only use cases. You can find a lot of tutorials, examples, books around it. If you're a GraphDB rookie you should start with Neo4j, but pay attention to the license or you'll end up paying Neo4j more than Oracle.
OrientDB cost much less than Neo4j (the Community Edition is free for any purpose, but it doesn't have the non-stop incremental backup that we needed, so we purchased the Enterprise). It's much faster in our use cases, but they have less marketing people = less popular than Neo4j.
Only with 2.2.x, OrientDB reached a good level of stability and is perfect for both read and read/write use cases. DataStax DSE will be a good product, maybe, in 1-2 years, but for now, it's like a toy.
> Why is this better than just using any SQL database?
Lots of SQL databases don't have good constructs for dealing with ranges, which make lots of operations you'd want to do with temporal data awkward (PostgreSQL since 9.2 has range types, and lots of useful features attached to them, that address this.)
Beyond that, pretty much all the normal SQL vs. NoSQL considerations would seem to apply without much change when you address temporal data, so the usual mix of ACID vs. scalability, schema enforcement vs. schemaless flexibility, etc., considerations apply.
We have a similar setup for extending our app with custom fields / forms using ruby-json-schema and react-jsonschema-form [1].
ruby-json-schema has worked out quite well, we added a custom schema reader so that we could generate enums dynamically based on records in the database via custom refs eg. app://schemas/organizations?type_eq=education&tags=one, any of these internal refs are inlined at the API layer.
One issue we did run into with ruby-json-schema, is the schema caching is not thread safe. We opted to clear the cache after each validation, this ended up causing race-condition issues. In the end we had to use a mutex on write and cache the validation result (not ideal, but our app isn't that write heavy).
This is what I'm using for schema validation in FormAPI [1]. The best part about using a standard is that there's a lot of open source libraries you can use. It's really easy to validate a schema in Ruby [2]. I use json-schema-faker [3] to fill out PDFs with test data. I also use AlpacaJS [4] to generate HTML forms based on the JSON schema.
Anyway, I'm really glad I didn't try to roll my own thing for schema validation. Alpaca is insanely powerful, and it would have taken me weeks or months to build the online forms from scratch.
I really love Redux, and I use it for most things, but that is simply not true. Redux/MobX is a solution to the Angular 1-style issue where state is all over the place, but you sure can make full blown apps that way (it's not even a full solution, but rather a different set of tradeoffs...which is why a lot of people start using Redux then start pushing back in the other direction and you end up with a mess).
And even if you want to centralize state, with local state it's not even that hard to do yourself (put it at the top or in a few small containers, push everything down, have callbacks that bubble it back up. A bit verbose, but easy to abstract out, though you'll end up with something close to redux).
The main issue with all of this is that 95% of the stuff you read about all of it on the internet is wrong, namely because building a real app and maintaining it is very different from making a toy project or a Silicon Valley style MVP in 6 months. People optimize for the later, then when they start maintaining their app its shit. So all the info on the net helping you to reduce boilerplate, optimize for the first version, etc, is just hurting you (but its almost everything you find online)
The first thing I do when installing a fresh Windows 10 build is download Chocolatey [0] and use that to install a list [1] of programs that I find extremely useful or can't live without. After that I set up Cmder [2] as my default PowerShell and Bash terminal, install Nylas [3] as my default mail program, install Caprine [4] which I use constantly throughout the day to communicate with friends and family, and then I turn on the Windows Subsystem for Linux [5] for my native Bash terminal.
Somewhat related to this topic, I recently found this list of tips and tricks [6] for Windows 10 that I found pretty useful, even as a lifelong (by choice!) Windows user.
I can actually fill in some of the details about the history of UUID's. Paul Leach was an architect who worked at Apollo, OSF, and later Microsoft. I met Paul in the mid-90's when he was an architect at OSF, and I was the tech load for Kerberos v5 development at MIT. OSF DCE was going to use Kerberos for authentication, and was going to use Apollo RFC as its RPC layer.
It was from talking to Paul that I learned about UUID's, and I added libuuid into e2fsprogs 1.05, released September 7, 1996. UUID's were used in Linux in the ext2 superblock, and later on, GNOME picked it up and used it extensively, which meant among other things that if you wanted to run GNOME on FreeBSD or NetBSD or Solaris, you had to compile e2fsprogs to get libuuid. :-)
Later on Paul went on to Microsoft, and I'm fairly certain that it was due to Paul that Microsoft adopted the OSF DCE RPC layer for its internal use, and UUID's started being used extensively inside Microsoft. UUID's also got used in Intel's EFI specification for the GPT partition table, although somewhere along the way they got renamed "Globally Unique ID's" --- it's the same spec, though.
While Paul was at Microsoft, the specs for UUID's finally got standardized by the IETF as RFC 4122, so you no longer needed to get find dated copies of the OSF DCE specification (or download e2fsprogs since I had an early version of the UUID Internet Draft in the sources long before it finally squirted out the other end of the RFC publication pipeline).
As far as uuidd is concerned, the reason why it exists is because a certain very large Enterprise Resource Planning system was using libuuid to generate uuid's for its objects, and it needed to create them very, very quickly so they can initalize the customer's ERP database in finite time. They were also using the time-based UUID's, with the UUID stored in the database with the bytes cleverly rearranged so the time bits would be stored in the LSB, and the Ethernet MAC address would be in the MSB, so that a database using a B-tree (plus prefix key compression) for its indexing would be able to very efficiently index the UUID's. This is similar to k-ordering trick that Flake was using, but this very large enterprise planning company was doing in 2007, five years before team at Boundary came up with Flake, and they were doing it using standard UUID's, but simply storing the Time-based UUID bytes in a different order. (I believe they were also simply storing the ID in binary form, instead of base-62 encoding, since if you're going to have jillions of objects in your ERP database, you want them to be as efficient as possible.)
Anyway, a certain Linux distribution company contacted me on behalf of this very large Enterprise Resource Planning company, and we came up with a scheme where the uuidd daemon could issue blocks of time-based UUID's to clients, so we could amortize the UUID generation over blocks of 50 or 100 UUID's at a time. (This ERP was generating a huge number of UUID's.) I did it as a freebie, because I was tickled pick that libuuid was such a critical part of a large ERP system, and it wasn't that hard to implement the uuidd extension to libuuid.
> I'm struck by how much this sounds like a Linux fan ranting back in 1995, when Windows and "real" UNIX was king. The underdog rants were rampant back then (I'm sure I penned a few of them myself).
It sounds like a Linux fan ranting circa 1995 because that is precisely what it is: first came the rants. Then a small, underdog company named "redhat" started providing regular builds and support, while Linux was easily accessible, and subversively smuggled into enterprises. Almost 20 years later, Linux is now everywhere.
Where once there was Linux, there is now SmartOS; where once there was redhat, there is now Joyent. Where once one had to download and install Linux to run it, one now has but to plug in an USB stick, or boot SmartOS off of the network, without installing anything. Recognize the patterns?
One thing is different: while Linux has not matured yet, as evidenced, for example, by GNU libc, or by GNU binutils, or the startup subsystem preturbations, SmartOS is based on a 37 years old code base which has matured and reached operational stability about 15 years ago. The engineering required for running the code base in the biggest enterprises and government organizations has been conditioned by large and very large customers having problems running massive, mission critical infrastructure. That is why for instance there are extensive, comprehensive post-mortem analysis as well as debugging tools, and the mentality permeates the system design: for example, ctfconvert runs on every single binary and injects the source code and extra debugging information during the build; no performance penalty, but if you are running massive real-time trading, a database or a web cloud, when going gets tough, one appreciates having the tools and the telemetry. For Linux that level of system introspection is utter science fiction, 20 years later, in enterprise environments, in spite of attempts to the contrary. (Try Systemtap or DTrace on Linux; Try doing a post-mortem debug on the the kernel, or landing into a crashed kernel, inspecting system state, patching it on the fly, and continuing execution; go ahead. I'll wait.) All that engineering that went into Solaris and then illumos and now SmartOS has passed the worst trials by fire at biggest enterprises, and I should know, because I was there, at ground zero, and lived through it all.
All that hard, up-front engineering work that was put into it since the early '90's is now paying off, with a big fat dividend on top of the profits: it is trivial to pull down a pre-made image with imgadm(1M), feed a .JSON file to vmadm(1M), and have a fully working yet completely isolated UNIX server running at the speed of bare metal, in 25 seconds or less. Also, let us not forget almost ~14,000 software packages available, most of which are the exact same software available on Linux[1]. If writing shell code and the command line isn't your cup of tea, there is always Joyent's free, open source SmartDC web application for running the entire cloud from a GUI.
Therefore, my hope is that it will take less than 18 years that it took Linux for SmartOS to become king, especially since cloud is the new reality, and SmartOS has been designed from the ground up to power massive cloud infrastructure.
> I think the assumption you're making is that people choose Linux out of ignorance
That is not an assumption, but rather my very painful and frustrating experience for the last 20 years. Most of those would-be system administrators came from Windows and lack the mentoring and UNIX insights.
> (and, I think the ignorance goes both ways; folks using Solaris have been so accustomed to Zones, ZFS, and dtrace being the unique characteristic of Solaris for so long that they aren't aware of Linux' progress in all of those areas).
I actually did lots and lots of system engineering on Linux (RHEL and CentOS, to be precise) and I am acutely aware of the limitations when compared to what Solaris based operating systems like SmartOS can do: not even the latest and greatest CentOS nor RHEL can even guarantee me basic data integrity, let alone backwards compatibility. Were we in the '80's right now, I would be understanding, but if after 20 years a massive, massive army of would-be developers is incapable of getting the basic things like data integrity, scheduling, startup/shutdown or init subsystem working correctly, in the 21st century, I have zero understanding and zero mercy. After all, my time as a programmer and as an engineer is valuable, and there is also financial cost involved, that not being negligible either.
> Linux has a reasonable container story now; the fact that you don't like how some people are using it (I think Docker is a mess, and I assume you agree)
Yes, I agree. The way I see it, and I've deployed very large datacenters where the focus was operational stability and data correctness, Docker is a web 2.0 developer's attempt to solve those problems, and they are flapping. Dumping files into pre-made images did not compensate for lack of experience in lifecycle management, or lack of experience in process design. No technology can compensate for lack of a good process, and good process requires experience working in very large datacenters where operational stability and data integrity are primary goals. Working in the financial industry where tons of money are at stake by the second can be incredibly instructive and insightful when it comes to designing operationally correct, data-protecting, highly available and secure cloud based applications, but the other way around does not hold.
> Are you really complaining about being able to gzip and tar something in one command? Is that a thing that's actually happening in this conversation?
Let's talk system engineering:
gzip -dc archive.tar.gz | tar xf -
will work everywhere; I do not have to think whether I am on GNU/Linux, or HP-UX, or Solaris, or SmartOS, and if I have the above non-GNU invocation in my code, I can guarantee you, in writing, that it will work everywhere without modification. If on the other hand I use:
tar xzf archive.tar.gz
I cannot guarantee that it will work on every UNIX-like system, and I know from experience I would have to fix the code to use the first method. Therefore, only one of these methods is correct and portable, and the other one is a really bad idea. If I understand this, then why do I need GNU? I do not need it, nor do I want it. Except for a few very specific cases like GNU Make, GNU tools are actually a liability. This is on GNU/Linux, to wit:
% gcc -g hello.c -o hello
% gdb hello hello.c
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-45.el5)
...
...
...
Dwarf Error: wrong version in compilation unit header (is 4, should be 2) [in module /home/user/hello]
"/home/user/hello.c" is not a core dump: File format not recognized
(gdb)
Now, why did that happen? Because the debugger as delivered by the OS doesn't know what to do with it. Something like that is unimaginable on illumos, and by extension, SmartOS. illumos engineers would rather drop dead, than cause something like this to happen.
On top of that, on HP-UX and Solaris I have POSIX tools, so for example POSIX extended regular expressions are guaranteed to work, and the behavior of POSIX-compliant tools is well documented, well understood, and guaranteed. When one is engineering a system, especially a large distributed system which must provide data integrity and operational stability, such concerns become paramount, not to mention that the non-GNU approach is cheaper because no code must be fixed afterwards.
Back when I was working on a fintech product (this is around 2 years ago) what pushed us away from Yodlee was the appalling pricing.
They wanted us to pay a hefty initial sign up fee ($XXXXX), pay monthly fee per user ($0.XX) and also they had minimums. Being students at the time we were just shocked and disheartened until we saw Plaids great pricing (free whilst in development and $500pm to set the product live) and tried to convince them to come to London which they told us was only 6 months away. It's been 2 years and they're still not here lol.
But there are a few more options in London now. I bumped into Steve Graham way back at HN's London event and he was working on Teller (https://teller.io/) which is now in beta and works pretty well from my testing.
There's also Saltedge (https://www.saltedge.com/) which is used by a few London fintech apps and I believe they have a startup package that a few apps like Cleo (https://meetcleo.com/) started on.
Some new startups still choose Yodlee though like moneybox (https://www.moneyboxapp.com) but they raised significantly more that we had (we had a very very small grant from our University).
I recently wrote up my Caddy+Varnish+PHP-FPM solution for WordPress[0]. It's fairly complementary to the steps detailed in the OP. The site absolutely flies now: 50-75ms response times cf. 200-300ms per Google, A+ on SSL Labs cf. unencrypted HTTP/1.1, and the $10/mo VPS rarely cracks 1% CPU utilization despite it being a modestly well-trafficked site. Our organic search position has improved significantly.
Caddy was maybe an odd choice but it performs well and it's been a breath of fresh air after dealing with some gnarly NGINX and Apache configurations in the past. So far it's been easy to maintain and there's a clear CDN migration path (Fastly). I'm using Docker's official PHP-FPM images[1] to serve multiple sites with some isolation, sharing Caddy, Varnish, and MariaDB instances.
I'm not sure if this solution has value for anyone else but I've been plugging it here and there in case there is!
One of the approaches and the last Tyler, the speaker, introduced in the talk to improve the fat tail distribution of load balancing latency problem was Join-Idle-Queue algorithm[1] that came out of UIUC Extreme Computing Group, Microsoft Research and Azure in 2011.
And according to Tyler's data (fast.ly CDN data since he's the CTO there), it beats all the current load balancing algorithms including Join-Shortest-Queue. Six years later, I wonder:
a) was there any other novel / better algorithms tops JIQ in LB performance?
b) has Microsoft Azure LB uses JIQ internally in their LB offerings?
c) has any open source LB software implement JIQ algorithm on the horizon? (according to Tyler, he found none.)
Can someone with expertise share some lights on these? Thanks.
Ugh, dang, I think you do a superb job of moderating HN, but please have some sensibility for those of us in the UK.
This guy, Dominic Cummings, is the equivalent of one of Trump's most rabid advisors. In his previous role (advisor to Michael Gove as Secretary of State for Education) he demoralised the entire teaching profession. I know that sounds hyperbolic but it's absolutely true - check out the TES or any other specialist press.
He has now, absolutely mendaciously (again, check out the expert commentary on the £350m bus claim), swung us out of an economic union on which lots of HN readers' livelihoods depend - including my own. Already, every time we go abroad, the ATM reminds us that we are getting less for our earnings than we did a year ago. Already multinationals are closing warehouses and cancelling the contracts that keep us able to pay the mortgage at the end of the month. People reading this post are losing their jobs because of this guy's sixth-form philosophy.
It hasnt happened in the US yet. Come back to this post in a year's time and tell me everything's ok under President Trump. I honestly hope it will be. But please have some tolerance for those of us who are losing out from 2016's wave of populism.
As a programmer I deeply appreciate the hate towards IT in companies and replacing Excel with complex internal apps. Those apps pretty much never capture the actual workflow (by virtue of requirements being provided by management instead of actual users), and even if they did, the workflow constantly changes. Excel spreadsheets may be cringeworthy sometimes, but they're used by people because you don't have to be a trained expert to make them work, and you don't have to submit requests to other departments to change something simple.
In my line of work I see an organisation regularly burning $50m in IT budget for developping a software that is completely flawed, horribly designed, not fit for purpose, though take years to develop and for that amount of efforts and money, could be much more efficiently done in Excel for 1/10000th of the cost!
This week I am trying to extract data from a data cube that was created for reporting. Problem, the cube was designed so that only one node can be retrieved at a time, and so if I need a bulk retrieval, which I need, it needs to be queried a billion times, sinking the server, timing out the queries, taking hours. The IT team is trying to figure out a way to run an overnight job so that it sinks the server during quiet hours. Basically these guys designed a vault where you can only add data but never retrieve it. They should be fired.
[edit] and before someone starts to think "yeah but this is big data", the underlying populated numbers in this cube would fit in a 5MB spreadsheet.
I am not sure that the audience of hacker news realises how terrible is the reputation of IT and developers in large companies from the business side. They are seen as a world of hyper-bureaucratic, common sense-free, business sense-hostile, half-assed lemmings.
Excel is not great but one has to have a hard look at the alternative.
tl;dr: I was compelled to write this because I saw a lot of complaints which seem to come from the end of the Magento 1 era. Many of these complaints are reduced when considering the context, and they are all handled quite nicely in the Magento 2 era.
(Full disclosure: I'm Magento's chief evangelist, but I was an agency developer for a Magento shop from 2008-2014, so I'm not totally a shill. Also, I care that we are listening & getting things right.)
I spend a lot of time interacting with PHP developers around the world. Some have been / are Magento developers, some are not. By wearing Magento gear at conferences and usergroups I would hear quite a lot of opinions about Magento, both negative and positive. Here's the negative.
* No documentation
* Can't test it
* XML sucks
* EAV is the Devil's creation
* The view layer is impossible to understand
* Inconsistent implementation in core code
* Not true open source
* It's too complicated
* It's too slow
* etc.
Fact is, there's always more to the story. Take documentation: yes, it's tragic that there never was official developer documentation for M1, but it was an outcome of Magento's explosive origins leading to the company being totally constrained. (We did eventually produce some fantastic user docs though.) Magento 2 solves this by having several tech writers on staff, as well as a doc site to which the community can contribute via pull request.
In the case of testability, well, Magento 1 was born in 2007. Ideally, yes, it would have been built with test coverage, but that wasn't standard practice for PHP back then. (How many of you were writing tests for your PHP apps in 2007? Not too many aside from Sebastian Bergmann!) Eventually though some great tools came around, and there are many examples of fully covered installations as well as CI/CD implementations. Magento 2 solves this by covering much of the core with unit, integration, and functional tests.
Regarding XML, Magento 1 (and 2) largely favor configuration over convention, and the decision was made to use the most structurally descriptive markup possible. Of course, the implementation of it in M1 was full of Magentoism - building a DOM by combining DB data, merging multiple XPATHs, and filling out a bizarre hierarchy for scope. It's not intuitive without either experience or schema definitions. Magento 2 solves this by splitting up configuration into functionally-scoped files AND by providing XSD. The XSD help when editing files by hand in an IDE, and we are seeing the creation of tools to help developers avoid writing it altogether.
When questions about EAV come up, believe me, I understand. That's a tough storage pattern to get used to, but the ORM helps quite a lot. The implementation in M1 is imperfect though, as you do end up with domain objects which leak their storage pattern. EAV is invaluable for solving the complicated requirements of facilitating arbitrary entity attributes and scoping/translation of those values. We looked at removing it for M2, but could not find a better solution. And if you say "noSQL," sorry, you're wrong.
The view layer is the thing which trips most people up - even developers who grok Magento right away. Theme fallback and layout XML are the complicating factors here. Theme fallback/inheritance is not so hard to get, and it is fantastic at keeping custom themes DRY - especially when multiple sites are running off of one instance. If only one thing could have been documented from the beginning, it should have been layout XML. Layout XML - along with the view model approach which we took with M1, allowed for simplified, DRY controller actions and ultimately allowed frontend developers to build interfaces without having to need backend developers. In M2 we have refined layout XML as well as abstracted the entire rendering framework, allowing developers to adjust or replace framework details as they see fit.
For inconsistent implementation in core code (which is especially evident in adminhtml), this was the outcome of a small team producing a lot of code independently. It's unfortunate, and very confusing for developers new to the platform. I struggled with this when I was starting out. We've worked to eliminate this in M2, but there is work remaining to be done. Thankfully we have Issues and pull requests on GitHub, and at the very least we can evolve our test suite to help us (and others) to implement functionality in The One True Way®.
On the subject of GitHub, we are much more of an open source company than we were in the Magento 1 days. We did allow contributions, but the process for becoming a contributor was cumbersome, and development effort was only revealed through updates to an SVN endpoint with no discussion. That is very different from our approach with Magento 2. Starting with the beta in December of 2014, we allowed pull requests to the M2 codebase, eventually building in our contributor agreement into the CI process. Many of our core engineering team are regularly on GitHub interacting with a bevy of "how do I" requests along with legitimate issues and pull requests. We have improvements to make here which will help us to keep up with the volume of discussion, and you should see these soon.
When I hear that Magento is "too complicated," I have to ask, then why doesn't something similar take its place? Ultimately the problem of complexity rests with the domain of commerce, which is always custom and always complex. Consider just one area of a commerce application - say, price calculation - and look at the complexity: cost, retail price, adjustments at the catalog level, adjustments at the customer level, and then add in the multiple taxation schemes which exist in the world (tax on full price before discount vs after, tax on shipping, etc.). Complexity is everywhere, and if you want to make an app which works for most, there will be overhead. Ideally the app helps the user manage that complexity as best as possible.
When I hear that Magento is too slow, that is always a worthless discussion without details about traffic, order volume, amount of entity data, update frequency, etc. M2 is a much better application for performance than M1, largely because it was developed with a mind to commodity computing and reverse proxy (which were nascent in 2007). That allows us to build core architecture for performance as well as enterprise architecture for scalability. That said, performance for M2 is not a finished story; I imagine that we will continue to improve on it as time goes on.
Okay, if you made it this far, I owe you some coffee or beer or whatever when we happen to meet. I was compelled to write this because I saw a lot of complaints which seem to come from the end of the Magento 1 era. Many of these complaints are reduced when considering the context, and they are all handled quite nicely in the Magento 2 era. I'm always happy to see or hear feedback.
As have I. Magento somehow combines the worst aspects of both PHP and Java, while using EAV everywhere to ensure it also combines the worst parts of SQL and NoSQL.
While it might look like the frontend is going around in circles, there are major & minor differences between the technologies, and they have each introduced novel (to JS at least) things... Off the top of my head (this timeline might not be right but it's how I went through it at least):
- Backbone got you away from jquery spaghetti, helped you actually manage your data & models. Often paired with Marionette to handle views (if you squint this is the precursor to components) and bring some order in that part of the app.
- Angular bundles backbone's data handling (services), marionette's orderly view separation (views, directives), but they made a fatal mistake in the apply/digest cycle and maybe encouraging a bit too much complexity. Angular is everything bundled together, with consistent usage/documentation/semantics, a familiar programming pattern (MVC), and a large corporate sponsor in Google it caught on like wildfire.
- Knockout sees Angular's MVC, laments it's complexity and focuses on MVVM -- simple binding of a model and a relatively simple reactive-where-necessary controller
- React comes along and suggests an even simpler world where the only first class citizen is components and everything else comes separate (this isn't necessarily new, there is an article out there comparing it to COM subsystems for windows). React is almost always used with react-router and some flux-pattern supporting data management lib -- these are also departures from how angular, backbone and knockout structured in-application communication (backbone was pure event bus, angular had the usual MVC methods, knockout was just callbacks -- if you have a handle to an observable when you change it things reload).
- Vue sees the complexity that React grew to (case in point: shouldComponentUpdate) and devises a much simpler subset along with a new way to handle reactivity -- using the `data` method of a vue component. Vue made a lot of decisions that helped it stay simple and small yet very productive and this is in large part thanks to React existing before hand.
- Svelte comes on the scene and realizes that truly optimal rendering could be achieved by just compiling away the excess and eschewing the virtual dom all together in most cases. No need to track all the structure if you compile the code with the updates necessary. Don't quote me on this but I think a whole host of things influenced this project -- smarter compilers, better type systems, the ideas around zero cost abstractions & doing more work at build time.
- Ember (formerly SproutCore) is actually an anomaly because it tries it's best to be both angular like (so large, and usable by large teams) and keeping up with the tech as it evolves (see: Glimmer, Octane). Ember also has some innovations, like Glimmer's VM-based approach -- turns out you can just ship binary data representing how to draw your components to the browser and skip a bunch of JS parsing, if you bring your own virtual machine that is optimized to draw the components.
As all this moves forward, there are ecosystem pieces like typescript that have gained tons of steam and changed how people are writing JS code these days.