Any Data Analysts?

Anything to do with the traditional world of get a degree, get a job as well as its alternatives
ertyu
Posts: 2893
Joined: Sun Nov 13, 2016 2:31 am

Re: Any Data Analysts?

Post by ertyu »

jacob wrote:
Sun Nov 27, 2022 6:46 am
doesn't anyone over the age of 12 know how spreadsheets work these days?!
You would be surprised. Kids grew up on their iphones; friends my age have told stories of 20-somethings from ood educational backgrounds not having an intuitive understanding of directory structure for instance because you just tap the app or you tag your document on your macbook and dump it somewhere then search by keyword if you need it. Also, kids flabbergasted by microsoft word because they only ever used google docs to write their essays.

That said, thank you for the excellent advice Jacob and Lemur -- R/the google cert it is. If it turns out I like data analysis and stick with it, i'll also look into Python.

jacob
Site Admin
Posts: 15912
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Any Data Analysts?

Post by jacob »

ertyu wrote:
Sun Nov 27, 2022 7:26 am
You would be surprised. Kids grew up on their iphones; friends my age have told stories of 20-somethings from ood educational backgrounds not having an intuitive understanding of directory structure for instance because you just tap the app or you tag your document on your macbook and dump it somewhere then search by keyword if you need it. Also, kids flabbergasted by microsoft word because they only ever used google docs to write their essays.
I can totally see that. As a recent newcomer to smartphones/pads my mental framework of directory structures (based on it being explained as filing cabinets back in the 80s) has been totally useless. In fact, I'm still not sure where my files actually exist. Also implies if I ever forget what to search for they will be forever lost.

Interestingly when I worked in finance, I was one of the few guys using linux. Initially they gave me a windows machine. I spent a couple of days in frustration before giving up: "How do I do anything with this?!?" My framework was using tools to pipe around files. This is not really how windozers do it.

mathiverse
Posts: 788
Joined: Fri Feb 01, 2019 8:40 pm

Re: Any Data Analysts?

Post by mathiverse »

ertyu wrote:
Sun Nov 27, 2022 7:26 am
... not having an intuitive understanding of directory structure for instance because you just tap the app or you tag your document on your macbook and dump it somewhere then search by keyword if you need it.
It's wild. I've seen young people take an entire file path and search for it instead of just going directly to the listed folder and these are CS students.

zbigi
Posts: 979
Joined: Fri Oct 30, 2020 2:04 pm

Re: Any Data Analysts?

Post by zbigi »

jacob wrote:
Sun Nov 27, 2022 8:52 am
As a recent newcomer to smartphones/pads my mental framework of directory structures (based on it being explained as filing cabinets back in the 80s) has been totally useless. In fact, I'm still not sure where my files actually exist. Also implies if I ever forget what to search for they will be forever lost.
Apparently, Apple's research shown that the concept of a file system is one of the most confusing things in computers for the non-technical people. Hence Jobs personally pushed during iPhone development to get rid of it.

ertyu
Posts: 2893
Joined: Sun Nov 13, 2016 2:31 am

Re: Any Data Analysts?

Post by ertyu »

A bit off-topic but, to maintain fairness, gen z are digital natives to promoting themselves online. When it comes to many cash-generating opportunities, for me the limiting factor is that they require one to maintain an online presence of tiktoks/videos and the like and the idea of filming myself for a tiktok makes me quite uncomfortable. i end up looking stiff, stilted, and unengaging. editing said video/tiktok is another arcane art that seems like it requires dark magic. whereas gen z are natives to generating and editing promotional content and intuiting how to "game" various algos

macg
Posts: 174
Joined: Tue Mar 31, 2020 1:48 pm
Location: USA-FL

Re: Any Data Analysts?

Post by macg »

zbigi wrote:
Sun Nov 27, 2022 10:34 am
Apparently, Apple's research shown that the concept of a file system is one of the most confusing things in computers for the non-technical people. Hence Jobs personally pushed during iPhone development to get rid of it.
This file directory mindset discussion is interesting to me. I grew up with it while learning computers and it's ingrained in me. I have never been able to grasp how people don't understand it, frankly... I remember in one of the early Anne Rice Vampire Chronicles books, one of the characters was considered to be a computer security genius simply because she used tons of nested directories in her DOS system. I couldn't understand that (how just that was considered secure), and always presumed that the author just didn't know anything about computers ... But perhaps to others it's more difficult of a concept than I know...

zbigi
Posts: 979
Joined: Fri Oct 30, 2020 2:04 pm

Re: Any Data Analysts?

Post by zbigi »

Example thread on Hacker News about pros/cons of data science career (such threads pop up there periodically):
https://news.ycombinator.com/item?id=33766099

7Wannabe5
Posts: 9378
Joined: Fri Oct 18, 2013 9:03 am

Re: Any Data Analysts?

Post by 7Wannabe5 »

I'm currently working towards M.S. in IT with concentration in Data Analytics. Weird problem I had recently was that my prior programming experience is mostly in ancient history functional languages, and I am currently learning Java. I had an assignment that required writing some not terribly complex pseudocode, and my brain totally locked up between trying to call up the ancient and almost forgotten vs the brand new and not yet assimilated.

Anyways, I'm finding the courses I am taking fairly interesting, but I am suffering from my lifelong problem with Fear of Success, in this case taking the form that my likely "reward" for achievement will be a full-time-job-working-for-other. However, there still exists the possibility that in the medium-run I will need to pay somebody $100,000 to surgically remove my colon and replace it with a plastic bag, so I persevere.

WFJ
Posts: 416
Joined: Sat Apr 24, 2021 11:32 am

Re: Any Data Analysts?

Post by WFJ »

My experience... I am NOT a good programmer but have had to learn Fortran, SAS, Eviews, SPSS, LaTex and minimal amounts of a few others depending on data format or who was receiving the analysis. If I work again, will probably have to learn Python as this is the main language used for new hires in data analyst positions. In general, as soon as you master one system/language, there will be a new system/language that one will have to learn to remain employable/effective.

Education, hard to make any general statements as some "Data Analysts" programs might be highly theoretical while some "Stats" majors are highly applied. Much of this is only learned after enrollment. By far, the best applied stats/data analyst course I've taken was from a heavy theoretical Econ prof, so hard to generalize anything as far as training. As with almost all graduate degrees, find someone who has a job you would like and ask them how to get there, might be MA, might be PhD or just build a portfolio from experience.

My first opportunities in data analyst came in large institutions where few people in the organization were qualified for any analyst jobs. I had 0 formal stats training when first started to estimate wait time/staffing needs in a large operations center. Would be in a strait jacket if I had to do that job for more than a few months but was the first step.

zbigi
Posts: 979
Joined: Fri Oct 30, 2020 2:04 pm

Re: Any Data Analysts?

Post by zbigi »

Another day, another post on Hacker News about disullusionment with modern data analyst (aka "data scientist") career: https://ryxcommar.com/2022/11/27/goodbye-data-science/

The guy switched to software engineering (into the "data engineering" subspecialty) and finds it way better.

I've come to similar conclusions around 10 years ago, when I had to choose to specialize in one of those paths. The DS always seemed too political and close to business to me. I'd hate to have to produce results which agree with some manager's agenda. Also, making sense out of super-gnarly data sets feels more frustrating than going through old codebases. Not to mention, half of the data scientists I've worked with had a PhD from freaking Harward, Columbia or Oxford, while software engineers sometimes don't even have university degree at all. Shows which field is more saturated and harder to get into.

jacob
Site Admin
Posts: 15912
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Any Data Analysts?

Post by jacob »

zbigi wrote:
Tue Nov 29, 2022 9:41 am
Another day, another post on Hacker News about disullusionment with modern data analyst (aka "data scientist") career: https://ryxcommar.com/2022/11/27/goodbye-data-science/

The guy switched to software engineering (into the "data engineering" subspecialty) and finds it way better.
Ha! I think this describes the polarity, at least by someone who has been exhausted having found himself on the wrong side, between "science" and "engineering" fairly well. Science cares about theory. Engineering cares about building in practice. A scientists will typically have the bare minimum understanding of the building process ("Hey, as long as the program runs. If it crashes, I'll just kludge it and restart."). This is understandable because "implementation" only has to be good enough to work once and get a result. Whereas an engineering will typically take the theory for granted ("The answer is given by this equation in this parameter space and this shall never be questioned").

It was a pretty big switch for me when I moved from "research-grade" code in physics to "enterprise-grade" code in finance.

I suspect the real problem based on this article is that data science was attempted as some kind of magic bullet by people (managers) who didn't/don't really understand it.

guitarplayer
Posts: 1304
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Any Data Analysts?

Post by guitarplayer »

@zbigi, thanks for the article, maybe it'll remain at the back of my head when I navigate the next 3-5 years.

WFJ
Posts: 416
Joined: Sat Apr 24, 2021 11:32 am

Re: Any Data Analysts?

Post by WFJ »

zbigi wrote:
Tue Nov 29, 2022 9:41 am
Another day, another post on Hacker News about disullusionment with modern data analyst (aka "data scientist") career: https://ryxcommar.com/2022/11/27/goodbye-data-science/
Very good article. The only perspective that might be lacking is a layering issue, which came first, the data or the understanding of the field. One who learns data science/coding first may tend to place too much validity on the outcome of an analysis if it is complex and robust theoretically. While one who first learns the application/field may have a better understanding of the flaws of applying complex tools and the messy nature of the data in the specific application.

My experience has been that those with Expensive University PhDs/MAs are hired for PR and to follow rules "Nobody gets fired for going with IBM" while those with sporadic or odd backgrounds who work in data tend to understand data science in the specific field at a geometrically higher level.

My personal opinion is that most of the people who write these blogs are from the "learned data science first" and communicate their frustrations, able to switch roles/fields easily as the author appears to have done. Those who understand a process/field who then learn the tools of data analysis are valued in that field, but unlikely to be employable in other fields or transition to other roles as easily. For those looking for 2nd+ careers, I'd keep this in mind as there appears to be little to no cross-over between these different groups within companies, a lot of clustering.

IlliniDave
Posts: 3845
Joined: Wed Apr 02, 2014 7:46 pm

Re: Any Data Analysts?

Post by IlliniDave »

jacob wrote:
Tue Nov 22, 2022 11:13 am
You can either [make your life's focus about] being someone, doing something, or having something/someone. Which one are you?
That's interesting. Clearly from 2010 up until July 2021 I had a "having something/someone" focus, though "gaining something" is arguably more accurate. Might go back farther than that. Having/gaining something coalesced into ER. I wouldn't call my career easy or intellectually unchallenging, but mostly due to the way I managed my career path, it allowed me to devote a lot of mental energy to the something.

Now I don't know though. I have the something. Anyone who peeks at my journal can probably suss out that keeping that something is something part of me feels like I should be doing, but it's too easy to make it my life's focus. Beyond that "doing something" appeals at first blush, but it is probably overstating my eclectic set of leisure pursuits to assert they rise to the level of "something". None of them have particularly noble or lofty goals associated.

Something good to think about as the weather turns cold.
Last edited by IlliniDave on Wed Nov 30, 2022 3:49 pm, edited 1 time in total.

guitarplayer
Posts: 1304
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Any Data Analysts?

Post by guitarplayer »

@WFJ, so do I get it right from what you write that the folk hired for PR would be the 'learned data science first' ones?

The metaphor of going up the stream of data resonates with me, find this appealing. I slowly realise that I am rather low down the stream of data handlers now! There is an upside to that, heaps of room for improvement / movement. Have been chatting with team members today about version control, they generally seem not to know what a git repository is, some of them have PhDs from fancy schools.

Attended a talk where operational researcher was talking about the field and how there was a 400-500% increase in operational research vacancies in my org the past few years. Sounded really good, along the lines of some things I now study for maths and stats. But I had the 'data stream' at the back of my head listening to him. Perhaps might be a good stepping stone.

A hedge for clustering at career would be to do the next career move diagonally, then clustering should matter very little.

I have been a carer for a few years
->
doing some stats and analyzing data now
->
in a few years might become a busker or a teacher, or a stay-at-home dad.

For this, money probably should not matter anymore though. But hey, really, there is not even time for those deliberations with ERE waiting at the end of the first 5 year plan already.

jacob
Site Admin
Posts: 15912
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Any Data Analysts?

Post by jacob »

guitarplayer wrote:
Wed Nov 30, 2022 3:47 pm
Have been chatting with team members today about version control, they generally seem not to know what a git repository is, some of them have PhDs from fancy schools.
Ha! That'd be me. (I do know what github is, but I've never used it. I did use another form of version control.)

But overall, yes, the tendency going towards the science side would be to write one's own algos from scratch(*). It actually makes sense because research is not really an incremental process of writing N lines per code per day. It's more an iterative process of writing and rewriting for maximum brain damage. Being forced to use version control (sure whatever), I still kept a separate system of particularly useful versions to cut and paste in. Reversion would never happen because newer versions might contain useful notes.

(*) Wait what? This goes against every principle of engineering! Because you never know what kind of shortcuts the people selling the product might have taken. Even if you can see the code, it might not work as you expected. Unknown-knowns and all that.

Before dunking too much on the science side ... one advantage is that they can explain "why" something is the way it is and where it might break. On the engineering or often the non-phd side, the explanation would be focused on "what" was done, e.g. "After spending all yesterday afternoon reading about Hoffman analysis on the wiki, I downloaded XYZ library and analyzed the data series using the Adler equation seen here."

One of my particularly cruel interview techniques was to ask people to write up the Black-Scholes equation, which of course they had memorized as well as its derivation. But then I'd ask what a particular term meant and what would happen if it was removed. Deer in headlight. The only guy who didn't fail that one was another phd. There's more depth ... but depth also comes at the cost of speed.

guitarplayer
Posts: 1304
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Any Data Analysts?

Post by guitarplayer »

Yeah building from scratch is also lots of fun! I am doing it now with one piece of code and data set. Probably mostly just because I am allowed to (as in, nobody dumps lots of other small tasks on me and I just let myself get focused on it). I think that it is useful for personal development, to let oneself find out why things happen the way they happen.

As in, from the trilemma of deep-broad-novel, choosing the first two*. Nobody is going to discover anything new here, but a good understanding of many well established elements.

*My long term hope is that once I tick off many deep-broad fields, something novel might appear. My expectation is it will not.

I am not an expert or extensive user of GitHub yet, but getting there. The idea of clicking 'push' and having a version recorded with 5W (when-where-what-who-why(comments)) of all the changes is just much more appealing than saving a file as a v1.1 with the W5 kept in the obscurity of one's mind.

WFJ
Posts: 416
Joined: Sat Apr 24, 2021 11:32 am

Re: Any Data Analysts?

Post by WFJ »

guitarplayer wrote:
Wed Nov 30, 2022 3:47 pm
@WFJ, so do I get it right from what you write that the folk hired for PR would be the 'learned data science first' ones?

The metaphor of going up the stream of data resonates with me, find this appealing. I slowly realise that I am rather low down the stream of data handlers now! There is an upside to that, heaps of room for improvement / movement. Have been chatting with team members today about version control, they generally seem not to know what a git repository is, some of them have PhDs from fancy schools.

Attended a talk where operational researcher was talking about the field and how there was a 400-500% increase in operational research vacancies in my org the past few years. Sounded really good, along the lines of some things I now study for maths and stats. But I had the 'data stream' at the back of my head listening to him. Perhaps might be a good stepping stone.
In the business world, few of the "data analyst" position require much more than content covered in "Stats for Dummies I". Designing data systems is more of an engineering position but also requires data skills and more advanced/difficult skillset.

The FTX implosion demonstrates the idiocy of several elite University grads, probably good at memorizing facts, but absolute idiots in reality. MIT Physics/Math grad with two Stanford Prof parents and Stanford Math grad with MIT prof parents don't understand margin, is just another example I've seen where MIT/Stanford grads are shockingly stupid (both kids and parents are morons). Sadly, this is not an isolated incident where individuals with amazing credentials are total morons.

Beginning about 20 years ago, large corporations started giving aptitude tests at the beginning of the recruitment process to filter Stanford/MIT morons and compass point state intelligence as they have been burned by the former too many times. IMHO, this is why so many Ivy league grads are working in the government now. Academia and government don't give aptitude test as part of the recruitment process :(.

guitarplayer
Posts: 1304
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Any Data Analysts?

Post by guitarplayer »

I don't know about the aptitude tests, I tend of agree with Cipolla that no matter what one does there will always be more than one thinks of people that one would prefer to have less of.

Lower bar at work means more slack to discover higher order logic of the workplace, this is good for people who like exploring their environment their way.

Daniel1979
Posts: 2
Joined: Fri Nov 27, 2020 11:28 pm

Re: Any Data Analysts?

Post by Daniel1979 »

Chipping in a bit late here: If you want to learn R for data analysis, there is one outstanding book (free) published by O'Reilly and written by Hadley Wickham, who is probably the most important R programmer in the area of data science: https://r4ds.had.co.nz/

12 years after my first degree (MA Sociology) I took the MSc Analytics with Georgia Tech (fully online). I liked the coursework! Took me 5 years though; full-time would be 1-2 years.

Post Reply