We recently tried to build a relatively simple PowerApps application - under the assumption it would save time over pumping out a React boilerplate form. Never again.
We wanted it to be able to integrate with our backend which lives outside the MS ecosystem. No problem we thought - that's what custom connectors are for. Except they don't work with any of the automated form generation functionality. So we pretty much have to build the form entirely manually.
Low code becomes high code all of a sudden - except it's worse than just coding and learning a new framework. You can't just input your code anywhere you want, or import your dependencies, etc... You have to hunt around in a zillion different (laggy as all hell) interfaces to find the right location where that code can live. Or maybe it's not code - but a toggle somewhere... OMG... kill me.
Good luck if you are trying to learn the lower level aspects... since all the tutorials, blogs, youtube videos - all assume that your data all lives in Sharepoint or some other MS resource - and that your connector actually works as advertised.
If you can code - and your stack doesn't already entirely consist of the MS ecosystem. Just roll your own.
I have seen several deployments of Databricks (including first hand experience) and... most use cases could be better served by Postgres (or Redshift, Athena, Snowflake for larger scale). It has honestly been such an overkill for so many workloads, it was quite astonishing. I've seen people move from Excel to Spark... to handle the same volume of data. That's obviously not Databricks' fault, but their PR is pretty much "please do all your data work in our product, it's well suited for it".
Yes, it's very good if you don't like setting up clusters (few do/can) and the UI is rather useful for getting up and running (not so much for writing code though). But you need to really understand the platform before adopting it. Please, please don't just adopt it because it's popular.
Clicked for me recently that working hard is like driving in first gear. Necessary initially, but if you don't shift up into higher gears, you're probably not doing it right, and not getting as far as fast as you probably should. If I'm tired, it's necessarily because I'm mismanaging my work and time, and that's a signal it's time to step back and figure out what's wrong.
Some people are really good at that initial torque boost on projects and can sustain it longer than others, but if work doesn't get easier it's because you're doing something wrong.
From a market-fit perspective, I think startups that drive around in first gear never get very far either. Things are only hard because we're being stupid. Nobody ever looks at a person driving a ferrari and thinks, "wow, they must have washed a lot of dishes to afford that car," and yet we still think "if I just wash these dishes hard enough I'll drive a ferrari one day."
I'm guilty of this as much as anyone, but burnout is my body intervening to tell me I'm being stupid.
I'm the head of engineering for a startup, was 3rd engineer at Eventbrite, built my own startup valued >$1M, turned down 2 offers at Google. Been doing startups 10+ years.
I do startups because I have a problem communicating my ideas in a way to influence larger organizations to do the things that interest me. Simple as that.
If I have a theory to move the needle, in a startup, I can do it, analyze it, and launch it with very little buy-in, although my impact to an industry may be less. In a BigCo, my impact on the industry may be more but the battle isn't about product-market fit, it's about corporate alignment and buy-in before I can even test product-market fit, which I just don't enjoy.
> Most successful businesses aren't based on revolutionary ideas, but rather improvements to the status quo. The media tends to focus on the revolutionary ideas, so it's easy to think that an idea isn't worth pursuing if it's not groundbreaking. But in my case, email marketing had been around forever when I started GMass but I found an unfulfilled niche and built a business out of it.
This is a really great paragraph. I think the part that I get hung up on is right at the end - how does one actually go about finding an unfulfilled niche? It seems like they are kind of difficult to find almost by virtue of them being unfulfilled.
Rather than thinking of Data Science and SWE as two different fields, think of Data Science on a spectrum, with "Advanced Data Analyst" on one side and "SWE/Machine Learning Algorithm Engineer" at the other.
Data Science is a weird field. A lot of the jobs descriptions have similar keywords, but there is just a huge amount of variance in what the job requires. There are definitely a large number of Data Science roles where solving a business problem requires you to write a good amount of code for integrating with other systems, data ET(maybe L), building UIs, etc that really is about making the core algorithm consumable by business owners.
When you interview ask what the day to day of somebody in that role is doing. You'll be able to figure out fairly quickly where they fall on this spectrum. Find the one that fits what you want.
IME, at smaller companies they don't have enough people to have 4 people (a Data Scientist, a Data Engineer, a SWE, and a Business Expert) just to get a data science project from conception to production. That's all done by one person with help from a business expert.
> As soft prerequisites, we assume basic comfortability with linear algebra/matrix calc [...]
>
That's a bit of an understatement. I think anyone interested in learning ML should invest the time needed to deeply understand Linear Algebra: vectors, linear transformations, representations, vector spaces, matrix methods, etc. Linear algebra knowledge and intuition is key to all things ML, probably even more important than calculus.
I've just finished reading the excellent book recentley shared here⁰. Working through it made me realized that the start-up I joined is ignoring every advise in the book. The ycombinater youtube channel there is a great Sam Altman talk on starting-up¹ which drove the point home further (that I've bet on a losing horse).
As an engineer I constantly get bitten by joining awesome sounding new tech start-ups that despite awesome tech they don't survive the second year (they find a market fit and fail when scaling bc they ignore all the human aspects of what it means to be a company). And in my case it's never the tech that was at fault but lack of creating processes under which successful patterns can be repeated. Somehow it's always the dynamic of the people that dooms those projects. Every time I end up coming to the hard realization that there is no other option for me to quit the company after I've wasted a lot of time and nerves, and it's often more ugly than it should be.
Save yourself a lot of support time/costs and remove the 'free' option. Your homepage sells the product well and shows its benefits. From the feedback you've already received it looks like you are providing more than $50 worth of value.
Because it gives more weight to one big error then to multiple small ones with the same sum.
We want the errors to be noise and not systematic. Noise usually has a gaussian distribution. And in a gaussian distribution multiple small values are more likely than one big one.
I wrote a similar post [1] in which I suggest charging at least 4x the rate you'd charge working full-time at a company.
So, if you normally make $60/hr working full-time (converts to $120k/year), then as a freelancer I'd suggest charging at least $240/hr if you plan on having anywhere near a similar lifestyle.
This is because 50% of your working time goes towards marketing, self-promotion, and other things clients don't pay you for. And 50% of your revenue goes towards taxes, office space, and buying products and services to support your business.
It's a rough calculation, but I think it explains why a lot of freelancers massively undercharge when first starting out. 4x your normal rate just seems so high until you think of the real costs.
So this is perhaps a lame post but I'm super tired, been up since 4am. I'll try and follow up with more detail in the morning.
This article, in my opinion, is way off. I base that on the fact that I've been dancing with Netflix for a month, I might end up working with them to try and make NUMA machines serve up content faster. As in I'd be working on exactly what happens when you hit play.
My take on how Netflix serves you movies is nothing like what this article says.
They have servers in every ISP, the servers send a heartbeat to a conductor in AWS, the heartbeat says "I've got this content and I am this over worked", when you hit play the app reaches out the conductor and says "I want this", the conductor looks around and finds a server close to you that is not overloaded and off you go.
6x revenue seems like a really high valuation, right? There's a physical product distributed via retail, we aren't talking tech margins. What's the value to Kellogg's?
My only guess is that Nutra Grain bars are being destroyed by Kind and their contemporaries. They need to compete in the category because they get some sort of incremental value when they own a number of different grocery departments. And their internal product development team just isn't very good.
The "scumbags" you describe are the ones that exhibit a strong "need for power" as described by McClelland. Engineers tend to have stronger "need for achievement", eschewing the power grab.
But organizations today and perhaps in the past have always rewarded those who attain the power. Perversely, if you want a place where achievement (ie doing the work, building things) is rewarded properly, you need someone with that mindset to suck it up and win the power game.
You know as a programmer myself for the last 15 years I can see how the market has changed and everything is becoming part of the global economy which means the rates are a lot lower which means Ukraine and India are playing a bigger and bigger role I could see private Equity trying to lower costs by outsourcing.
I think there's a fundamental flaw in the idea that you can just pay for cheaper labor- as Steve Jobs has said before the difference in a good programmer and an outstanding programmer can be 50 to 1 or 100 to 1 you can't capture that by trying to cut your cost from $100 an hour to $20 an hour.
Software inherently is one of the most scalable business models in the world the cost of manufacturing is almost zero all of the cost is in the design they should be able to make money. The mindset of being a consulting firm has to be changed.
I've been working on a side project for the last 3 months and it has finally gotten to the point where I am making more than $1000/month with more than 25 active customers 100% through word of mouth. I am working on a Show HN with some of my learning from the process so I won't get too deep into it here but here are a couple of highlights:
* You do have time. I work a time consuming job, have a wife and kids, and still found 1 hour per day to work on it, and that was enough.
* Automate everything that you can. Early on I automated the deployment, the creation of new accounts, the management of the sales, and soon the marketing.
* Have a plan and stick to it. I planned to use 1 hour per day and I did. I have a backlog and I work against that always.
* Pick a market you understand. I help a lot with my kids schools and this is software to help with that.
* Drop bad ideas when needed. I have started more side projects than I can think of. Sometimes in the past I have felt bad because I didn't want to give up on an idea. So I worked on a bad idea for way too long. Don't do that.
It turns out that when you have the right idea and are scratching an itch that real people have, it's not that hard to get people to pay you to solve their problem.
Actually, yes! Again, your milage may vary but I have had a few things that have been an indicator of a bad idea. Some of this has to do with process as well.
First off, if no one wants to talk to you about your idea at all, then it's probably a bad idea. I know this one is fairly obvious but I missed it a number of times. It's important to make sure you always are trying to get customers or users. If you can't get anyone at all then you're basically not making something anyone cares about. Psychologically you may say to yourself "if I only add another feature people will listen". Life is short. They probably won't. Try another idea.
Second, if people will talk to you but won't buy your product until you add a new feature, and everyone has a different new feature, you have a product no one wants. I have tried so many times to just add the next feature that will make everything sell. It has never worked for me. I should have just stopped and found a new idea.
Third, you have a product that you don't want to sell. I know, this one sounds silly but I have had amazing product ideas but I couldn't drive myself to want to talk about it with anyone. Maybe I made myself feel better by making a landing page and crying about no one buying it every though "I was doing everything I could". But, it wasn't everything I could do. This one is a funny one because I might have been able to sell the product, I just couldn't bring myself to get up and do it. If that's the case, you have the wrong product idea FOR YOU. Stop, think of another idea, and move on.
Those are my biggest three categories right now. It all comes down to selling the product. If you can't sell it, then it's not something to work on. With my current product I love talking to customers and potential customers. I am excited and the sales line up with that excitement. They are excited to use the product. They enjoy the product and while there have been feature requests they are either a) after the sale is complete and I have money in my pocket or b) the same requests from every customer so I can tell it's a market need and not just a customer want.
1) If you read or hear about someone you think are doing something great, contact them, tell them, ask them.
2) If you see someone who is doing something amazing, go over to them, tell then how great you think it is, ask them how they do it.
3) If there is something you want to understand but don't, figure out who do and ask them.
4) Get out amongst people.
5) Build, create, write, launch something put it out there for the world to see.
In other words.
Just be genuinely interested in the people you interact with and you will be building the best possible network without feeling you are being insincere. It doesn't matter if it's in person, by mail, via a tweet.
>> What are some strategies to deal with this beyond picking a hyper specialization?
Well, stop thinking about it. Not only it is depressing but it is also seriously wrong.
I've been hearing that since before I started. Not only did it never happen, but it's getting more unlikely to happen every day.
It's not possible to commoditize something that 99.xxx% of people are unwilling and/or unable to do.
Yes, most people simply don't want to sit at a computer all day and/or don't have what it takes to be a decent programmer. And everyday the barrier to entry is getting a bit harder with more tools and more technologies and more abstractions appearing, each adding a bit more complexity over the precedent.
If anything, look around you and watch the race to the top for the few who can follow. There are entire companies in our industry which exist to attract and bread the best and the brightest.
---
Now, I can understand why some people would be afraid of a commodity market.
Somewhere in the world, there will ALWAYS be a guy who only has $10 to pay for whatever thing he wants done AND there will ALWAYS be a guy that will offer to do the job for that price.
That first guy will always accept the offer because he has no seemingly better alternative and he'll just burn the money hoping to get something in return. (note: ain't gonna happen. You just don't get anything real for $10).
The point being: Just because there are some guys with no money and some guys who are screwing them. That doesn't mean that it's a "commodity market", let's just call that a "scam market".
Anyone serious who wants to get the job done or anyone who can get the job done will simply NOT participate in this scam market. When we're talking real work, there is simply no commodity market whatsoever (in our field).
It takes time to get used to, but after 6 years on and off, I am using it for the last 8 months regularly, and I already saved enough money for the next 4 months in advance.
But, of course, what these two have in common, is simply:
1. Spend much less than you earn
2. Try to just spend 30-40% of your income, but an emergency fund which stores 6 month worth of money, and then invest the saved money.
What I recommend, is a little game:
1. Don't eat out for a whole month
2. Don't do any big purchases for a whole month
You will see how much many you usually spend, and how your mind is triggering you to buy things. It will be really hard, so instead of thinking about "oh no, I shouldn't buy this", just write a note on a piece of paper everytime you want to buy something.
At the end of the month, you will see how "stupid" you were to want a certain things. If a certain things come up on this list over and over, then maybe it could be useful.
We wanted it to be able to integrate with our backend which lives outside the MS ecosystem. No problem we thought - that's what custom connectors are for. Except they don't work with any of the automated form generation functionality. So we pretty much have to build the form entirely manually.
Low code becomes high code all of a sudden - except it's worse than just coding and learning a new framework. You can't just input your code anywhere you want, or import your dependencies, etc... You have to hunt around in a zillion different (laggy as all hell) interfaces to find the right location where that code can live. Or maybe it's not code - but a toggle somewhere... OMG... kill me.
Good luck if you are trying to learn the lower level aspects... since all the tutorials, blogs, youtube videos - all assume that your data all lives in Sharepoint or some other MS resource - and that your connector actually works as advertised.
If you can code - and your stack doesn't already entirely consist of the MS ecosystem. Just roll your own.