Is Software Development/ Programming Oversaturated?

Anything to do with the traditional world of get a degree, get a job as well as its alternatives
zbigi
Posts: 978
Joined: Fri Oct 30, 2020 2:04 pm

Re: Is Software Development/ Programming Oversaturated?

Post by zbigi »

white belt wrote:
Sun Sep 26, 2021 9:17 am
How long can exponential growth continue in the software field? Software may be eating the world, but you can’t eat software. Software runs on hardware that has real physical limitations and requirements as we’ve all been reminded of due to the chip shortage and various supply disruptions.
Currently, a lot of people write software as if it was meant to run on some abstract Turing machine. Most software engineers who didn't graduate from some kind of CS program have rather dim concept of how computers actually work, what are computational costs and consequences of various designs etc. That is mostly fine because servers are that fast and that cheap.

BUT, if physical hardware limitations were ever to become a thing again (in the server world - they absolutely are in the mobile world), it would mean... even more jobs! Writing efficient software takes more much time than just gluing libraries together without any regard for efficiency - so companies will need more people to do it.

ducknald_don
Posts: 321
Joined: Thu Dec 17, 2020 12:31 pm
Location: Oxford, UK

Re: Is Software Development/ Programming Oversaturated?

Post by ducknald_don »

zbigi wrote:
Mon Sep 27, 2021 1:06 pm
Most software engineers who didn't graduate from some kind of CS program have rather dim concept of how computers actually work, what are computational costs and consequences of various designs etc.
Quite a lot with a CS degree seem to have the same problem. This is why interviewees have to jump through so many hoops, employers can't trust the credentials.

SavingWithBabies
Posts: 882
Joined: Mon Aug 31, 2015 2:50 pm
Location: Midwest, USA

Re: Is Software Development/ Programming Oversaturated?

Post by SavingWithBabies »

@zbigi Open source is highly variable. I have lead the open source project for a programming module that has about 2M downloads so it's fairly popular (but people use it in builds so CI things are probably increasing that number). I didn't get a whole lot of professional benefit out of that work. I did it because the startup I was at at the time needed it. And now it's basically outdated tech so it's in maintenance mode.

I think for learning what you suggest is good. For actually getting hired, the hustle stuff is better. This is based on the discussions I've been present at in the hiring meetings. Things other people said not me.

But I do agree, it is a good way to get experience and learn.

MeloTheMelon
Posts: 81
Joined: Thu Apr 22, 2021 2:18 am
Location: Germany

Re: Is Software Development/ Programming Oversaturated?

Post by MeloTheMelon »

I think there is a huge competition for the "flashy" jobs in tech, like working for FAANG or similar companies.
Everything not so sexy is still in need of loads of people.
I'm in a public institution, working as a software dev / data scientist, pay is good (especially for someone with no prior work experience) and because it is sponsored by the state, it is also pretty relax, almost no extra-hours etc.

We are a small 15 people team and are drowning in projects, we have the money to hire about 10 more devs but just can't find any.
I heard similar stories from other companies here, so it seems like there is a shortage, at least for Germany and Austria.

On the other hand, programming itself is an incredible skill, it simply allows you to do so many projects and earn some money on the side, or automate the boring stuff in your day to day life.

zbigi
Posts: 978
Joined: Fri Oct 30, 2020 2:04 pm

Re: Is Software Development/ Programming Oversaturated?

Post by zbigi »

SavingWithBabies wrote:
Thu Sep 30, 2021 5:08 pm
@zbigi Open source is highly variable. I have lead the open source project for a programming module that has about 2M downloads so it's fairly popular (but people use it in builds so CI things are probably increasing that number). I didn't get a whole lot of professional benefit out of that work. I did it because the startup I was at at the time needed it. And now it's basically outdated tech so it's in maintenance mode.
I was thinking - do a semi-significant contribution to Apache Spark or Tensorflow or PostgreSQL or something of that caliber. This is likely to raise some eyebrows for sure. Might be too hard for someone who's just getting into coding though (but should at least be a great learning experience!).

Also, in terms of job hunting, you basically only need the OS contributions to get your foot in the door - after that, actual job experience should be enough carry you forward. I don't even list my contributions in my CV any more.

On the subject of whether companies value OS contribs or "hustle" more, I suspect it's highly variable. I work in a big boring company, and I suspect we actually don't like the hustle types - they would get bored/restless in here and likely leave. Energetic people with too much initiative are not the best fit for mired bureucracies that most F500 companies are - for such jobs, you need someone who can tolerate the boredom and can stoically endure the crappiness of the working conditions (red tape everywhere, projects failing due to widespread incompetence etc.).

SavingWithBabies
Posts: 882
Joined: Mon Aug 31, 2015 2:50 pm
Location: Midwest, USA

Re: Is Software Development/ Programming Oversaturated?

Post by SavingWithBabies »

@zbigi Ah, yeah, I was thinking more of the startup industry with the hustle stuff where I think that is more desired. I personally don't like the hustle stuff too much just because it ends up being tiring if you have to work with people who hustled in but aren't actually qualified -- I'm fine with doing that and helping them if they don't have a huge attitude/ego but sometimes they do have that and it gets in the way of actually delivering value.

User avatar
Sclass
Posts: 2791
Joined: Tue Jul 10, 2012 5:15 pm
Location: Orange County, CA

Re: Is Software Development/ Programming Oversaturated?

Post by Sclass »

It’s really great hearing some of the ideas from a generation of engineers younger than me. I’ve been out of the game so long I have forgotten some of the constraints.

I meant to comment earlier on the topic of most CS grads coding like they’re ignorant of how he machine works. I’ve seen this a lot but the fact it gets mentioned shows it is self correcting. As an embedded systems developer I’ve had to “break in” sw guys who didn’t know how to code small systems - small program memory, slow clocks, fixed point. After the initial naive complaints they’d realize why the constraints were there (cost, size, power consumption, speed) and they’d start coding. Some got really good at it. The university trains on particular platforms. My school had one semester of computer organization which students feared because you had to code close to the hardware. In retrospect it was a great requirement and may have been a student’s only exposure to low level programming over an entire career. In short, this stuff can be picked up. I don’t see it as some kind of tragedy that a developer doesn’t understand the inner workings of a hardware multiplier when he first sits down.

The prevalence of prepackaged code has always been an influence on developer value. I suspect like any profession it pays to be a little ahead of the crowd. If there are tons of linkable libraries with everything done for you and code generation sw to patch it all together the value of the customized goes up. It will go up as more and more people fall back on the code configurator sw and cannot go that last 1/4 mile in the race. I know a lot of friends in physics who learn just enough Python to get good salaries as data scientists but they’ll never cut out a lucrative niche unless they leverage their overdeveloped math skills or become more sophisticated machine customizers. Not sure I’m being totally clear here but that is my observation lately. You want to be the person who can take things to the next level. I never had enough friends to get hired for just doing the basics…there was always that little bit more that was needed that nobody in house had.

I think that goes for all professions beyond the digital ecosystem.

As for hardware limits, the chip shortage etc. yep, the shortage is a problem and I’m living right in the middle of it (my side hustle uses some 40nm tech). But people have been warning that Moore’s law would hit a wall (Tjunction=temp of sun) and it would stop twenty years ago. The ecosystem’s reaction? Multi core distributed processing and flattening clock rates. Yes there are limits but like a n insurmountable wall getting covered by kudzu the machine seems to find its way around them and continue its spread. At some point we will run out of sand and the ability to melt it to wafers but on the scale of the engineering career half life it’s irrelevant. We can sit around worrying about it or we can make money now. That has some pretty serious sustainability and selfishness problems but seriously I have to survive in a world I cannot easily change alone.

A lot of the semi shortage is climate driven. That’s a sad reality finally coming home to roost. I suspect the industry will find its way out like it always has and continue to grow.

It’s exciting to see the growth. I remember sitting with a boomer (a real one :lol: ) in 1999 and he said it’s an exciting time to be working in tech. It still is and more so than back then.

zbigi
Posts: 978
Joined: Fri Oct 30, 2020 2:04 pm

Re: Is Software Development/ Programming Oversaturated?

Post by zbigi »

Sclass wrote:
Sat Oct 02, 2021 10:41 am
The prevalence of prepackaged code has always been an influence on developer value. I suspect like any profession it pays to be a little ahead of the crowd. If there are tons of linkable libraries with everything done for you and code generation sw to patch it all together the value of the customized goes up. It will go up as more and more people fall back on the code configurator sw and cannot go that last 1/4 mile in the race. I know a lot of friends in physics who learn just enough Python to get good salaries as data scientists but they’ll never cut out a lucrative niche unless they leverage their overdeveloped math skills or become more sophisticated machine customizers. Not sure I’m being totally clear here but that is my observation lately. You want to be the person who can take things to the next level. I never had enough friends to get hired for just doing the basics…there was always that little bit more that was needed that nobody in house had.
In the web/server-side software, the situation is a bit different. The preexisting software you link with aren't mere "libraries", but rather multimillion lines of code behemoths (examples: Hadoop, Spark, React, Kubernetes) which do a lot for you, but are extremely complex and, realistically, take years to master. Consequently, companies don't just look for say a JavaScript developer, they look for a React developer. The good money is in being highly specialized in technologies that are currently in vogue. This is the ERE book's specialization problem taken to the extreme - once React gets superseded by something else (and, in the front-end world, these coups happen every 2-3 years), your value goes from "hot shit" to "the guy who knows a legacy tech". Luckily, there's still a lot of work in maintaining legacy systems written using legacy tech (all the way back to COBOL systems from the 70ties - we still have them in my bank), but it pays less than being the hot shit. Also, having to run on the tech treadmill just to maintain a high salary is not the most fun way to live.

BTW none of this applies to FAANGs, they have completely different culture (basically, they don't follow the tech/library trends, they make them and everyone else follows).

User avatar
Sclass
Posts: 2791
Joined: Tue Jul 10, 2012 5:15 pm
Location: Orange County, CA

Re: Is Software Development/ Programming Oversaturated?

Post by Sclass »

Sounds like a good reason to aim for early retirement. Makes me tired just reading about it.

WFJ
Posts: 416
Joined: Sat Apr 24, 2021 11:32 am

Re: Is Software Development/ Programming Oversaturated?

Post by WFJ »

Short and long answer is "No". I do not program but use systems and interact with all kinds of programmers and there is a bottleneck in most projects due to supply of capable programmers. Good programmers make many aspects of my job easy, bad programmers are luckily, easy to spot. The programming languages and underlying systems constantly change and force programmers to always improve their skills, so it's not for everyone.

alex123711
Posts: 58
Joined: Fri May 15, 2020 8:33 pm

Re: Is Software Development/ Programming Oversaturated?

Post by alex123711 »

What about learning programming with the end goal of having your own business, how achievable is that? Or being a freelancer if you aren't able to get an entry level job or 'foot in the door'? There seems to be quite a lot of people even cs grads that struggle to find that first job.

User avatar
Sclass
Posts: 2791
Joined: Tue Jul 10, 2012 5:15 pm
Location: Orange County, CA

Re: Is Software Development/ Programming Oversaturated?

Post by Sclass »

The answer to that one depends on what you’re trying to do. I guess it depends on who you are too. Kind of a sliding scale between with several knobs. Some businesses can be really improved with some simple sw development in house. Others may require a team of seasoned programmers to make a polished product worthy of mass consumption.

It’s a useful tool for solving a lot of problems fast and cheap. Those are things important to entrepreneurs.

It’s a machine. If you can use it at just about any level it’ll make your work easier. Kind of like a caveman realizing his whacking stick can be repurposed into a lever. Or learning how to drive a passenger car, F1 car, tractor or postie bike. I think it has always been advantageous to discover and harness the potential of a new machine before everyone else.

I’ve known several small business people who morphed into sw vendors because their home brewed business tools were so effective they figured out they could sell the sw to their industry rather than do the industry. For those friends it was VB because it was easy and just good enough to make marketable products in 1998. The examples were wide spread from auto repair shops, school attendance sw, engineering design tools, to inventory management. Now Python looks like a good thing to learn for its wide scope.

I started a side hustle during the pandemic that has a sw component. I sell a digital filter. My competition is pure hw but I figured I’d be more competitive with a digital solution. Less parts. Smaller. Harder to copy. Coding instead of wiring. Precision from math not calibrated precision components. Harder to understand in general and perceived as modern by the customers. I’ve swallowed up 75% of this niche business in six months with my board. I think I was influenced by the movie Sneakers as a kid where the killer sw app is hidden in a piece of hardware. It was a good idea up till I couldn’t get the chips. Anyway, couldn’t do it without some basic C++ and assembly language skills. The competition is too stubborn to learn a line of C and too cheap to hire someone who does. Puts me in a good spot…now when are those chips arriving?

I worked in a startup before retirement doing manufacturing systems for the product I designed. The product design took a couple of years but most of the time at the end was designing machines to make the machine. A lot of sw development in Python and LabView to automate our lines and plug into our inventory control sw package. Basic stuff that allowed us to operate more efficiently.

This is why I recommend some coding rather than sw engineer level proficiency. It’s a sharp stick that can improve performance over a wide field of disciplines.

The beauty of this folds back into keeping up. It is kind of scary hearing about what it takes to stay current in sw engineering in this thread. Hellish frankly as the pace of evolution is stepped up. A product may have a slightly longer lifetime because it can be produced cheaply with legacy tools unbeknownst to the user.

ducknald_don
Posts: 321
Joined: Thu Dec 17, 2020 12:31 pm
Location: Oxford, UK

Re: Is Software Development/ Programming Oversaturated?

Post by ducknald_don »

That reminds me of my plumber who has written some amazing Excel/VBA tools to automate large parts of his business. He uses it to get a competitive edge when quoting customers.

ducknald_don
Posts: 321
Joined: Thu Dec 17, 2020 12:31 pm
Location: Oxford, UK

Re: Is Software Development/ Programming Oversaturated?

Post by ducknald_don »

WFJ wrote:
Sun Oct 03, 2021 9:07 pm
Good programmers make many aspects of my job easy, bad programmers are luckily, easy to spot.
I'm not so sure about that. It's hard to pin productivity improvements on a single programmer, especially when you work in a team. Manual work is easy to measure and incentivise but cognitive work is very subjective. This is why office environments devolve into politics in a way that you don't see on a building site or in a factory. It's easy to see how fast a brick layer or joiner is working and even whether the quality is there, try measuring the output of a programmer and you will end up in a cycle of gamed metrics.

zbigi
Posts: 978
Joined: Fri Oct 30, 2020 2:04 pm

Re: Is Software Development/ Programming Oversaturated?

Post by zbigi »

Sclass wrote:
Sat Oct 02, 2021 5:22 pm
Sounds like a good reason to aim for early retirement. Makes me tired just reading about it.
My thoughts exactly. Being in the middle of it is much more tiring than just reading about it :) Programming nowadays is an annoying, but a very high paying career with low entry requirements, that is conducive to a quick FIRE. The alternative route is to ignore the tech treadmill altogether, and just work on that legacy Java 8 system forever in some bank or insurance company. It's much more peaceful (until they hire some young guns out of college, who'll start screaming "why are we using this old shit? let's migrate everything to the latest and greatest!" - happened to me already), but it will probably double years till FIRE.

Luckily, I'm basically FI already, so I don't really have to care about learning the next great thing - if/when my knowledge expires in the next couple of years, I think I'll just retire rather than trying to catch up.

guitarplayer
Posts: 1300
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Is Software Development/ Programming Oversaturated?

Post by guitarplayer »

I must say I appreciate this thread and many different perspectives it presents on the industry. Interesting, informative and sobering.

User avatar
giskard
Posts: 320
Joined: Sat Apr 30, 2016 12:07 pm

Re: Is Software Development/ Programming Oversaturated?

Post by giskard »

I wanted to add my 2 cents to this thread:

I've been programming about 10 years and here are the changes I've noticed. It used to be easier to get into the industry as an entry level, but it was also less lucrative. It seemed like it was easier to find senior level people even 5 or so years ago.

Now, it's nearly impossible to find senior levels. The pay has gotten so high and there are just very few people to hire from. A senior level engineer that is good is not-comparable to a new grad in terms of productivity, not even in the same ballpark. A new grad (or green junior) will actually slow an experienced software team down, and every project is always behind schedule as a rule.

And another interesting thing is that the attrition rate of good programmers on the market is a little bit high. They end up in several paths:

- working for FAANG or similar at impossibly high salaries so you can't poach them anymore. These places also give golden handcuff options grants + RSU vesting schedules that almost always are on a 4 year schedule, so you are on a treadmill of wealth that you can't get off.
- In management so their skills atrophy.
- Running a company / CTO / startup etc.
- Retired early / switched careers / (stopped programming)

That said, in a few months hopefully I will be in a position (running a startup, if we can raise more money) to hire and build a small team. My plan is to interview only entry levels and just hire 2 of the smartest ones I can find and mentor them (regardless of existing skill, just looking for smart and hard working). I will pair them together and everyday pair program with them a little bit. I'm going to go very slowly and be patient with them, have them to read relevant books and practice and not just try to get them to dump out code like most places do. In about 6 months I will get a cheap and loyal workforce at the cost of my time but in the long run it should be worth it. I'll give them more options grants and raises if they stay. Then in another 6 months I will do it with another pair of entry levels, and try to keep building a pipeline of talent that way.

User avatar
fiby41
Posts: 1611
Joined: Tue Jan 13, 2015 8:09 am
Location: India
Contact:

Re: Is Software Development/ Programming Oversaturated?

Post by fiby41 »

Image

User avatar
Mister Imperceptible
Posts: 1669
Joined: Fri Nov 10, 2017 4:18 pm

Re: Is Software Development/ Programming Oversaturated?

Post by Mister Imperceptible »

white belt wrote:
Sun Sep 26, 2021 9:17 am
How long can exponential growth continue in the software field? Software may be eating the world, but you can’t eat software. Software runs on hardware that has real physical limitations and requirements as we’ve all been reminded of due to the chip shortage and various supply disruptions.
“this dovetails in sinister fashion with the basic idea that any sufficiently advanced technology cannot be distinguished from magic. highly evolved capitalism becomes such a technology and the largess and plenty it produces gets mistaken for a property of the universe rather than a made thing, a thing that must be created rather than simply reaped.”

https://doomberg.substack.com/p/where-stuff-comes-from

guitarplayer
Posts: 1300
Joined: Thu Feb 27, 2020 6:43 pm
Location: Scotland

Re: Is Software Development/ Programming Oversaturated?

Post by guitarplayer »

@mi thanks for the piece.

Post Reply