Any Data Analysts?

[alex123711]

"Follow your passion" is standard career advice in Denmark (and I presume the rest of Scandinavia). Self-delusion becomes much easier if everybody else is deluded too. I always suspected that the advice was because it's possible to graduate more students if they study something they're actually interested in. In practice "following your passion" does lead into non-monetizable traps but the welfare-state catches people from falling and let people retrain. Those who didn't find their passion the first time get a second degree, and maybe a third ... and so on. Always in pursuit of their passion.

Passion may also have some "rockstar"-payoffs in that when/while it's good, it's really good; thus creating a small group of actually passionate people that continuously hype the idea.

I'm conflicted on this point, it seems some careers 'require passion' while others don't. e.g a lot of people say you need to have a passion to become a programmer, maybe due to the required study outside of work etc? Charlie Munger also said something about how can you compete against someone who has an intense interest in the area, they will likely devote a lot more time towards something they are passionate about. But yeah a lot of people followed their passion and it led them nowhere or underpaid/ underemployed.

Here is a good video from the host of "Dirty Jobs" he talks about becoming good at something is whats rewarding - gives the example of the sewage pump business owner who really enjoys his job.

https://www.youtube.com/watch?v=CVEuPmVAb8o

[jacob]

I'm conflicted on this point, it seems some careers 'require passion' while others don't. ...

If the results and payoff are a linear function of the effort, you don't need passion. Especially when it comes to work that demands mental creativity, the payout is an exponential function. There are some possibly apocryphal research that the most productive programmers are up 100x as productive as the worst. This ratio does not exist when it comes to digging trenches, say.

PS: The effort may be a combination of total cumulative effort (experience) and rate of effort, e.g. the difference between concentrating on a problem for 100 hours per week (obsessed) and 20 hours (slacker).

[7Wannabe5]

most productive programmers are up 100x as productive as the worst.

Is this due to effort or talent? The majority of humans are not comfortable with tasks such as manipulating negative numbers, so I wonder whether there isn't a certain percentage of programmers who are completely reliant on doing it by rote with scaffolding? It's relatively easy to teach kids when to make use of various mathematical algorithms, but it is more difficult to instill number sense. I've encountered dirty little disadvantaged kids who are bouncing off the walls with hyperactivity who have number sense, and very hard-working studious affluent students who don't have it.

[jacob]

Is this due to effort or talent?

When talking to wannabePhysicists during career day, we told the prospects that success basically came down to effort limited by talent and not to bother if their IQ was not at least +1.5sd because there simply wasn't enough hours in the day to even manage to graduate.

If becomes much worse if one has to supply one's own scaffolding, that is, be constructively creative. The ability to simultaneously keep multiple balls (figments of ideas) in the air determines what kind of complexity one can form. For modern programming jobs, it's my understanding that a lot of it is now glue&stitch based on libraries and not reinventing Kernigan & Ritchie. I don't know if a data analyst is primarily a "technician", a "mechanic", or a "designer" as per http://earlyretirementextreme.com/start ... sting.html

[7Wannabe5]

When talking to wannabePhysicists during career day, we told the prospects that success basically came down to effort limited by talent and not to bother if their IQ was not at least +1.5sd because there simply wasn't enough hours in the day to even manage to graduate.

From my secret file of the odd number of things I have in common with Ted Kaczynski: When I was booted out of the Physics program at a prestigious university, and appealed my case, I was informed that my SAT scores were too high to warrant being given a second chance on the basis of "also working a 30 hr/week job to support myself."

I don't know if a data analyst is primarily a "technician", a "mechanic", or a "designer"

I think at the designer level would be more likely to be referred to as "data scientist." My very rough scan of jobs with "data analyst" in title for which I might be already qualified leads me to believe that some level of comprehension of "linear regression" might actually be enough for lowest entry level positions.

[zbigi]

powerbi has way more market share than SAS
https://trustradiusnew.wpengine.com/wp- ... x675-2.png

Hah PowerBI I remember it took me a good three months in one of my former jobs to create a simple dashboard, connected to a Hadoop cluster. The combination of bugs in PowerBI and various security restrictions imposed by my organization meant that achieving anything was near impossible. It always amazes me that this is how multibillion organizations (Microsoft, author of all those bugs in PowerBI, and the large bank I contracting for at the time) work internally in practice.

[alex123711]

If the results and payoff are a linear function of the effort, you don't need passion. Especially when it comes to work that demands mental creativity, the payout is an exponential function. There are some possibly apocryphal research that the most productive programmers are up 100x as productive as the worst. This ratio does not exist when it comes to digging trenches, say.

PS: The effort may be a combination of total cumulative effort (experience) and rate of effort, e.g. the difference between concentrating on a problem for 100 hours per week (obsessed) and 20 hours (slacker).

Not sure what you mean "If the results and payoff are a linear function of the effort, you don't need passion"?

[zbigi]

Not sure what you mean "If the results and payoff are a linear function of the effort, you don't need passion"?

In most normal, "boring" professions, being 20% better or working 20% harder will yield you roughly 20% more rewards - the payoff is more or less linear. Whereas, in professions with non-linear rewards, such as say professional sports, being 20% better can mean the difference between being in top 10 of international tennis, and making millions, and being in top 1000, and barely making rent.

[guitarplayer]

Hah PowerBI I remember it took me a good three months in one of my former jobs to create a simple dashboard, connected to a Hadoop cluster. The combination of bugs in PowerBI and various security restrictions imposed by my organization meant that achieving anything was near impossible. It always amazes me that this is how multibillion organizations (Microsoft, author of all those bugs in PowerBI, and the large bank I contracting for at the time) work internally in practice.

Then PowerBI might be a low hanging fruit for the persistent ones. One fellow at work who studied software engineering and AI works with it mostly, maybe this is the reason why?

Reading the earlier entries, I come to the conclusion I am glad for never adopting fully the high achiever persona and stopping at ‘intensely curious about some stuff’. In result I never got to a job that gave me the feeling you got @zbigi.

Currently intensely curious about the data stuff.

Also planning for the middle age crisis kicking in about a decade from now. I do love the forum for all the insights and heads up.

[zbigi]

Then PowerBI might be a low hanging fruit for the persistent ones. One fellow at work who studied software engineering and AI works with it mostly, maybe this is the reason why?

PowerBI, as far as I know, is supposed to have to have two main selling points:
- easy connectivity to everything from a local excel spreadsheet to a Hadoop cluster or esoteric NoSQL databases.
- easy way to publish interactive dashboards on company's Intranet, so that can then be used throughout organization.

I was always under impression that it's mostly meant for stuff like that - data gathering, presentation, drill-down analysis etc. - stuff that traditionally falls under the "BI" umbrella. I don't know how good it is at statistical analysis (ML/"AI") vs say Python or R.

Reading the earlier entries, I come to the conclusion I am glad for never adopting fully the high achiever persona and stopping at ‘intensely curious about some stuff’. In result I never got to a job that gave me the feeling you got @zbigi.

Currently intensely curious about the data stuff.

Also planning for the middle age crisis kicking in about a decade from now. I do love the forum for all the insights and heads up.

I wonder to what degree we are, for all practical purposes, predestined - or at least determined by things which were beoynd our control at the time. I for one was always impressed by people who go deep into a given subject and achieve mastery - so I tried to imitate them in how I approach things. I don't know why I was impressed by this particular style in the first place, it might as well have been male juvenile fascination of Kung Fu movies or something equally shallow [1]. Whereas, other people might start their live with completely different, but similarly highly determining grand idea, such as "searching for a true love" or "gonna make those monies so that I can show them who's boss" etc.

[1] There's this interesting game developer called Jonathan Blow. He's highly acomplished, currently in his 50ties, still working like a dog. When I listened to him talk (he's streaming a lot of his work, and people clip interesting pieces of out those streams and post them on YouTube), it's clear that he's still gripped with the idea of this Kung-Fu style personal mastery (as opposed to broader development within a social context), which, in my eyes, is somehow not fitting to someone his age. But, on the other hand, it allows him to make great things that many people enjoy, so society probably needs a few people like that.

EDITED: some minor grammar mishaps.

[7Wannabe5]

it's clear that he's still gripped with the idea of this Kung-Fu style personal mastery (as opposed to broader development within a social context), which, in my eyes, is somehow not fitting to someone at his age.

This. I've been recently thinking that being in grad school at my age (58) is kind of like reverting back to age 29, which is actually less fun than reverting back to age 19 or age 9, because you know that what comes next is being in your 30s. Early middle age is kind of like the boring conformist Second Grade phase of adult development, so not a phase I'm super interested in repeating.

[arbrk]

If the results and payoff are a linear function of the effort, you don't need passion. Especially when it comes to work that demands mental creativity, the payout is an exponential function. There are some possibly apocryphal research that the most productive programmers are up 100x as productive as the worst. This ratio does not exist when it comes to digging trenches, say.

PS: The effort may be a combination of total cumulative effort (experience) and rate of effort, e.g. the difference between concentrating on a problem for 100 hours per week (obsessed) and 20 hours (slacker).

I do think the most productive programmers are 100x more productive as the worst. But how much they make doesn't scale like this. I went to school with so many people who were so passionate and much better than me technically. But a lot of them are making less than me now because they chose to work on something they are passionate about and I chose to work on what makes me the most money.

They are definitely better, and I am definitely mediocre, but I still make more.

[zbigi]

I do think the most productive programmers are 100x more productive as the worst. But how much they make doesn't scale like this. I went to school with so many people who were so passionate and much better than me technically. But a lot of them are making less than me now because they chose to work on something they are passionate about and I chose to work on what makes me the most money.

They are definitely better, and I am definitely mediocre, but I still make more.

The best programmers can start their own software companies and potentially outearn everyone who's salaried - or, at least, earn nice money and have a lot of freedom and self-determination in their worklife.

[guitarplayer]

Passion is like a passion fruit - mostly sour like nothing else and you're better of having it as juice with plain water. Sometimes you get that super sweet one though I had both not long ago.

[ducknald_don]

The best programmers can start their own software companies and potentially outearn everyone who's salaried - or, at least, earn nice money and have a lot of freedom and self-determination in their worklife.

Starting your own software company is a lot different from being a good programmer. You will need a lot of skills that are completely orthogonal to writing software.

[ducknald_don]

I do think the most productive programmers are 100x more productive as the worst. But how much they make doesn't scale like this.

The worst programmers can't do anything or even have a negative impact on projects. You might think there aren't many of them but whilst I was consulting I saw them on nearly every team I worked with.

[guitarplayer]

@ducknald_don, this reminds me of the pretty well known on the forum Cipolla

1. Always and inevitably, each of us underestimates the number of stupid individuals in the world

I sometimes wonder how to bulletproof myself from falling into this group yet suspect that inevitably I end up being in it on occasion.

[ertyu]

What I usually suspect is, I am underestimating their numbers by one (me)

[zbigi]

Starting your own software company is a lot different from being a good programmer. You will need a lot of skills that are completely orthogonal to writing software.

Depends on the type of company. If you want to be an "ISV", so basically a one-man shop making and selling some piece of software, then, at least in many markets, quality of software is the determining factor. See programs like Sublime Text Editor, Total Commander etc. If you're lucky enough to understand well some other niche besides software, then you can make your program for that niche, and enjoy much smaller competition. Also, quite often, just copying and improving on an existing program can work. Of course, it's a ton of work, and that's why you need to be a very good programmer - to do it relatively quickly (say within a couple of years vs a decade it would take someone less skilled) and to have the end result be of good quality.

[ducknald_don]

That's exactly what I've been doing for the last twenty two years and in my experience having a good product isn't enough. You have to figure out how to get it in front of people. There is a lot of software out there, both commercial and open that goes nowhere. You have to find a way to rise above the crowd.