This is true but keep in mind that you have to set up a pipeline for collecting the data into HDFS (Storm? batch loading?) & you have to pay for the machines.
So while your analysis is valid, there are more "costs" at play like developer time, cluster maintenance, hardware. I like to play with Spark's ML libraries but am wary about designing projects specifically around them because of this overhead, especially when trying to distribute some API/tech that you'd like others to use.
Not trying to be a downer, I actually wish the choice to go distributed was more of a no-brainer, hah. Would love for some APIs to emerge that could be used locally/distributed transparently without actually having to run a dummy cluster & data migration to run locally.
yeah i do Scala now but same deal. Where I think the OP comment really goes wrong is when he says something like "you're gonna need all that stuff and you're gonna end up re-implementing it yourself". I really don't find that to be the case... when you're developing with libs you are cognizant of your app endpoints and just add whats needed. It's better to consciously add a lib or turn a feature on than to just accept a big ball of yarn that some cowboy framework devs shat out. In the Rails app at my job it takes a ton of debugging just to figure out how many layers of decorators are firing off around every model.
I might even go so far as to say that a lot of where JVM has strength is that contexts are pretty measurable... Spring containers, Akka systems. You instantiate them and hold them basically in the palm of your hand, you understand your scopes. Any technology that allows application scope to slip out of your fingers and intermingle with everything else under the sun is something I'd be wary of.
My own anecdote: at a previous job, new development was done in Rails, while "legacy" Python applications were maintained by one (!) guy. The Python apps were about the exact opposite of the Ruby apps -- cherrypy was used for the HTTP interface; very low level (compared to Rails, at least). He used to shake his head at the stack traces that Rails would spit out. "Intermingle with everything else under the sun" sounds about right.
There is a huge legal difference between getting caught doing drugs vs facilitating the trafficking of mass quantities of controlled substances. I don't think casual marijuana smokers should get all up in arms like "hey man they are just going after everybody now!", do they offer the same support to heroin cartels?
Even if you're against the war on drugs I don't think you should really take it as a personal slight when someone operating on this scale gets arrested. Unless you really 100% believe that distribution of heavy narcotics doesn't damage society.
If people are going to take drugs, somebody has to produce and distribute them. It doesn't make much sense to me that possession and consumption of drugs should be legal but manufacture and distribution shouldn't. Regulation would be nice, but the state is falling down on its duty there.
Please read the parent I responded to. Especially the closing statement.
>If drugs were legal and treatment of abuse the focus instead of punishment Silk Road wouldn't have existed in the first place.
I cited Portgual, a country where drugs are legal and the focus is on treatment of users. Giving an example of where such a system exists and works. The "war on drugs" is the wrong approach. I implied, rather indirectly, that prison quotas have a lot to do with keeping drug use illegal. It makes it easier to fill quotas.
Now then - the parent and my focus were both more on the users and the wrong approach for the "war on drugs". A response to me focused more on distribution and extrapolating the legal stance of Portugal. So I cited a Wikipedia page that was more specific and re-iterated that my focus (and the parent I responded to, by extention) were more focused on drug users and treatment doing more good than going on witch hunts for distributors.
Congratulations, you caught Ulbricht. Now what about all the other dealers that people will turn to? Especially local dealers who might lace their drugs or have improperly manufactured drugs (ie. containing arsenic) that may lead to more deaths of users?
I have no idea how the Silk Road worked, but I imagine dealers had accounts and received feedback. This meant there was some level of Social Quality Control over the drugs. Anyone selling faulty/laced drugs would be quickly rooted from the market. Providing a 'safer' place to buy, even if still illegal, does more good for the users than having to trust shady dealers.
This is very, very related to the war on drugs. It's a criticism of the policy of it all.
I'll quote myself from another post I made:
"Mistaking some delusional world of zero crime doesn't do any good for people living in reality."
Maybe the best long-term policy is drug decriminalization and treatment (etc), and maybe were that already the case, Ulbricht would not have been tempted into doing anything bad.
Nonetheless, that has no bearing on this case. Once somebody is dabbling in murder, they need to go down, because they are clearly not somebody we want in society. That they were "tempted" into it by potential profits enabled by misguided prohibitions is irrelevant.
So yeah, decriminalize drugs, focus on treatment, etc. Maybe that will make the future of our society brighter. But Ulbricht belongs behind bars.
I don't disagree with your points. I don't like the War on Drugs either. I think it is destructive.
Ulbricht was a legitimate kingpin who at least tried to have people killed. I don't have much sympathy for him and I'd rather distance his actions from real issues with the War on Drugs.
I still disagree that Portugal's policies regarding users is that relevant here. You are right in what you are saying, though.
I feel like this is already indirectly possible with cache systems... yes there is an initial load from DB or whatever, but with Ehcache for example I think(?) all those objects just sit in the JVM, and therefore should be in RAM. If you wrote some app startup batch process to stick every object possible into the cache proactively, I think you'd have essentially what you're asking for.
I'd like to at least offer some support to this -- I'm not sure I agree with all of your conclusions but since switching to Scrum mentality I definitely notice that the goals of our team have become much less ambitious... Rather than insulating for development cycles that may not yield much for a few of months but in the end equals proper enterprise architecture, the business/devs have found it much easier to focus on small iterative tasks that are much more superficial in the grand scheme of things.
I think engineering processes benefit when they are insulated from direct scrutiny of timelines. Resorting to tackling smaller, neater issues in Agile seems to be a sign that there is a lack of technical leadership/vision. This is not necessarily the fault of Agile (I don't mind Agile, I think projects can succeed with it if approached properly), I simply agree that increasing process rigor seems to be a symptom of an engineering/mgmt team that is floundering.
In short, if your team's problem is that things are getting "messy" and "hectic" because of too much crappy copy/paste code, I have seen how Agile can just deepen that wound while masking it in a veil of "tangible progress" seen in burndown charts. Then, again, if you don't have the right people you don't have the right people. What else can you say?
my company is "totally agile" (they wish) and it gives the management layer something to do cuz they are analyzing JIRA all day coming up with horse-shit charts, so of course they keep pushing all of it like its solving high-level organizational issues in ways we will experience tangibly sometime by 2018...
if you got rid of all that process infrastructure and just had a few programmers discussing proper design patterns now & then our company would be lightyears ahead of where it is now. i think everything becomes cargo cult when you refocus technical people onto processes that discourage technicality to make management feel fuzzy about their stats. people aren't more productive now, they are just gaming the agile system
process planning has its place i just feel like its becoming the focal point of discussion rather than a means to it. bureaucracies will continue to grow, what can you do other than look for a job with a more technical group that is focused on code?
i'm a former academic, i was astronomically more productive in that setting and there were no real process restrictions imposed at all. people need to be cultured into code & self-education, not process. optimize process once there is a good technical core in place, not in lieu of one
(i've also noticed that fake agile just promotes a lack of requirements docs from business... in that case i'd be better off doing damn waterfall with a half-decent reqs doc anyway...)
Urgh Jira. Jira symbolizes the entire problem. Issue and task tracking where the actual issue and tasks are 3 layers deep from the main interface, and graphs and burn down charts are front and centre. It's all carefully architected to appeal and sell to the managers who buy it, rather then the dev's who use it.
Contrast to something like GitHub, where code is front and center, and issues are a flat list and which has no other BS cluttering it up. Why? Because GitHub isn't selling a product to managers first - it's selling to to dev's first (via getting them in on open source).
A doctor would probably classify this as some level of abuse, since medically accepted drinking levels are low (few drinks a week, I think?). Even if its just "1 per day", chemical dependency in order to function is usually seen as an issue. Sure, small doses of coffee & alcohol are well... small, but there are tons of people out there who rely on neither at all, where as the absence of these "small" doses would greatly impact your life. EDIT: I should mention though that I am not really against small-dose self-medication, as the alternative of getting medicated through the "proper system" is probably even more frightening. I do wish I could kick all small-dose habits though, rather than dealing with relapses (boring work without coffee is haaaard) & other compensatory habits.
As for your 10+ beers followed by welcome hangover comment, I believe Adorno addresses this in Minima Moralia and some of his other texts. His basic belief is that people are so alienated from themselves that they treat the weekend (their only time to really be themselves completely) as a respite that will ready them for the onslaught of the next week's work. Basically, the work week and attitudes that have arisen to rationalize it are a system of total self-alienation, with all traces of your "in an ideal world" personality smothered out of you.
Anyway, sorry if this came off sounding very dramatic. I really am not like deeply worried about you, hah, I'm just saying from a theoretical standpoint what you're describing does not prove your point at all. It's more like you're just using black humor to perpetuate bread&circus mentality.
yep there are also a lot of situations in enterprise where you don't exactly know how your code will be used in near future (in <5 years it could grow to exponentially more customers using it compared to soft releases to select test customers).
Because of that I'd say.... it's not like I see a lot of people get raked over the coals for it, but there is definitely some organizational disappointment when a silly scaling bug crops up in production[1]. I think as a mature dev it is proper to internalize as many performance-related techniques as possible so that you can write more performant code with each subsequent project. Sometimes it doesn't even affect the amount of initial coding hours, it is just one of the freebie gains that comes with experience.
[1] Many more such bugs would probably occur if the company didn't occasionally get around to load-testing, so that should be part of CI or dev cycle if possible.
Honestly, I'd rather they make it a paid service than a dystopian clusterfuck of advertising espionage.
I'm more likely to quit the wonnnnderful "free" service you describe and defend because it "costs millions of dollars a month to run". Those millions of dollars are just R&D on how to get advertisers more intimate data. If they didn't pay for all that gross research, the infrastructure wouldn't cost nearly as much.
So while your analysis is valid, there are more "costs" at play like developer time, cluster maintenance, hardware. I like to play with Spark's ML libraries but am wary about designing projects specifically around them because of this overhead, especially when trying to distribute some API/tech that you'd like others to use.
Not trying to be a downer, I actually wish the choice to go distributed was more of a no-brainer, hah. Would love for some APIs to emerge that could be used locally/distributed transparently without actually having to run a dummy cluster & data migration to run locally.