Early Retirement Extreme Forums

http://idlewords.com/talks/website_obesity.htm

Advances from hardware engineers have allowed software engineers to become supremely lazy! The productivity or information content per GHz or byte is trending ever lower. I'm puzzled as to why this is? Are engineers just lazy? Does the public consumer have an insatiable appetite for fluff-bytes?

Has all this software abstraction become a goal in itself making "Hello World" turn into a 100kb program instead of the 50 byte program it used to be in 1985?

Am I just getting old? (I first got on the net in 1989. More than 25 years ago. It was very different back then.)

Also see http://www.lowtechmagazine.com/2015/10/ ... nergy.html

PS: I fear what size the blog's homepage is? I remember that my first "homepage" (It wasn't called that at the time) loaded in <4 seconds on a 2400 baud modem following much optimization. Meaning it was about 1k ... or ~0.001MB. I'm currently accepting the status quo because it's so easy. I'm probably part of the problem.

Adding to this ... I recently procured a new computer and when I first tried browsing from it, I thought the wifi was broken seeing how sloooow everything run. It was only when I switched on my regular ad and tracker blockers (ABP and Ghostery) that things returned to normal. So do yourself a favour ...

From the first link:

These comically huge homepages for projects designed to make the web faster are the equivalent of watching a fitness video where the presenter is just standing there, eating pizza and cookies.

and

In conversations with web performance advocates, I sometimes feel like a hippie talking to SUV owners about fuel economy.

They have all kinds of weirdly specific tricks to improve mileage. Deflate the front left tire a little bit. Put a magnet on the gas cap. Fold in the side mirrors.

Most of the talk about web performance is similarly technical, involving compression, asynchronous loading, sequencing assets, batching HTTP requests, pipelining, and minification.


All of it obscures a simpler solution.

If you're only going to the corner store, ride a bicycle.

If you're only displaying five sentences of text, use vanilla HTML. Hell, serve a textfile! Then you won't need compression hacks, integral signs, or elaborate Gantt charts of what assets load in what order.

Browsers are really, really good at rendering vanilla HTML.

We have the technology.

Maybe it's not that we're getting old, and it's just the consumer mindset manifested in web development? We'll use your MB because we can and we want to!

jacob wrote:I'm puzzled as to why this is? Are engineers just lazy?

Uh... yes! Yes, we are! What is engineering? Building things that make life easier. Why make life easier? Because we're lazy.

That said, regarding the web site itself, I assume this particular type of laziness is the result of people chasing click-driven advertising dollars in an increasingly automated fashion.

Also, re "Hello World..." coming from an MS environment, software development tools have become more bloated for ease of use, which in turn produces bloated web pages and executables that need multiple supporting files in order to load/run. This was not the case in the 1989. Then you actually had to (gasp!) write code to get "Hello World" to appear, whereas today you're clicking and dragging objects around the screen, and the only things you have type are the words "Hello World" and the executable name. You're not coding at all. The software environment (IDE) that makes that possible then pumps out bigger EXEs when it compiles to support all the laziness bloat that was built for the ease of the coders.

The funny thing is, if you open up the IDE and look at the actual (automatically generated) code, a lot of it is superfluous code that's generated for every app, just in case. It's formatted according to the latest standards, and is very pretty, and includes pre-formatted comment blocks and everything. And 90+% of it you can delete and your app will be smaller and faster. But no one does this because PCs are so fast now. Also, a lot of younger developers likely don't know they can get rid of that stuff... that they don't need class-based architecture and comment blocks for "Hello World," LOL.

Having just heard of it for the first time I started trying out Ghostery this evening. Not sure that it affects speed much, at least I don't notice much, but it is mildly interesting to see all the things it purports to block. Been using ABP for a while and that made a substantial difference in general browsing speed/ease.

Another hypothesis might be the scarcity/value of developer time. For example, my job sometimes involves directing a team of developers to build an application for our business. They often ask if they should prioritize the performance of the app, or finish implementing all of the features. (I am not a dev, but a manager whose team will be the end users of their app.) My answer: as long as it's usable, just get it done! This is related to GandK's point. An IDE is useful not because it results in more optimized code, but because it results in more optimized use of dev time.

And another possibility: devs are getting worse on the margin. For example, I could imagine that back in 1989, it was only the nerds with the most programming aptitude who were programming, so the average program was written by someone who was really smart and/or knew their stuff. Now that programmers are in such high demand, people with less aptitude for programming get into the game.

Brilliant article.

I'm not in SW and know next to nothing about it, but I have been in product development (ME) for many years. Many of the same patterns outlined also apply to other engineering fields.

Basically, a combination of persistent feature creep on the definition side and schedule compression on the production side makes it darn near impossible for many engineers to truly design anything anymore. That's how you end up with so many Frankenstein products that check all the marketing boxes but basically suck from a user's perspective.

On a macro scale, the quick-buck startup mentality largely reinforces this phenomenon. Who cares if the product is designed well so long as it launches on time and is properly monetized to flip?

But I do agree that individual engineers bear some of the responsibility. I think many people don't realize that the subset of engineers who are also creative problem solvers is actually quite small. Far more simply regurgitate the process they were taught than step back and look for better ways to reach the same goal.

As a designer of small footprint embedded computers and RTOSs, I can say this is driven by the cost of development time. Call it lazy engineer, but I'd rather call it expensive engineer who can use the off the shelf power to make something work fast.

Electrons are cheap. A kid who worked for me told me "storage is cheap SClass, cycles are cheap". His time was expensive though so I encouraged him to find a solution that would consume less of his cycles and more of the machine's.

Tyler9000 said it better than me. Same idea.

The direction of progress in tech isn't always "right". For a long drawn out discussion of how technological advancement can lead to horrible wrong constructs I like Jerome Lanier's book.

http://www.amazon.com/You-Are-Not-Gadge ... ome+Lanier

I think that the principle at work here is that with more efficient tools you can (and often will) work less efficient with the same result. Example: commute times remained constant even though average velocity per person has increased. People just moved farther away from work.

I think in this specific case, there are two subcases:

- Good engineers being able to create equal up to somewhat better programs in less time
- Bad (low-skilled) engineers who can previously couldn't create anything due too lack of skills. However, now that there are 15 bootstrap-NodeJS-Angular frameworks allowing you to create websites (in the cloud no less!!), they can also create a hello world/website now (but ten times less efficient) in the same time that a good engineer used to take. Note that these low skilled engineers can function only thanks to the frameworks. Therefore, you cannot expect them to make a better solution, since they simply don't have the required skills.

Meanwhile, the programmers of the olden days are now required to pay for getting their fix (http://store.steampowered.com/app/370360/) rather than getting payed themselves.

P.S. This thread reminded me of this one: viewtopic.php?f=21&t=6399 and I chuckled

Related: Mark Zuckerberg to build AI to help at home and work

Quadalupe wrote:...
Meanwhile, the programmers of the olden days are now required to pay for getting their fix (http://store.steampowered.com/app/370360/) rather than getting payed themselves.
...

Zachtronics. I hope that guy is SWIMMING in money and happiness. SpaceChem is a truly outstanding game.
His games should be school curriculum, they're that good.

jacob wrote:I'm puzzled as to why this is? Are engineers just lazy? Does the public consumer have an insatiable appetite for fluff-bytes?

I think it has more to do with product manager and CEOs having an insatiable need for a) fluff-bytes, which is really an extension of make the logo bigger: https://www.youtube.com/watch?v=5AxwaszFbDw b) tracking everything that's happening with 20 different analytics packages.

Although, in terms of pure bandwidth (which really only correlates to speed), images are the biggest problem. Especially those huge gratuitous images at the top of pretty much every news article I read.

Also if anyone wants to see the actual talk: https://vimeo.com/147806338

I'm too lazy to read the article, but like a lot of things, it's a balance. In the 80s assembly style gurus used to stand tall but they grew less relevant as technology meant you (and people who succeeded you) could get more done making your code readable and modular (even object orientated).

Prizes for the famous "you are not expected to understand this" comment are no longer the rage.

Ultimately though the product has to be useable or otherwise what's the point?

I'm reminded of the likely apocryphal tale of the programmer who profiled his OS code, found where 90% of its time was being spent, and hand optimized that in assembler, only to find to his dismay nothing got faster, as he had optimized the idle loop.

@jacob
Marketing and popularity are the biggest determiner of which type of software we use. For example you mentioned how adblock is speeding up website load time, but did you know that someone has painstakingly created an adblocker that is more efficient, has more features, and is less monetized? It's called ublock and unfortunately most people don't know this becasue the engineers obsessed with efficiency tend to lack the marketing skills to push their software.

Quadalupe wrote:I think that the principle at work here is that with more efficient tools you can (and often will) work less efficient with the same result. Example: commute times remained constant even though average velocity per person has increased. People just moved farther away from work.

I think in this specific case, there are two subcases:

- Good engineers being able to create equal up to somewhat better programs in less time
- Bad (low-skilled) engineers who can previously couldn't create anything due too lack of skills. However, now that there are 15 bootstrap-NodeJS-Angular frameworks allowing you to create websites (in the cloud no less!!), they can also create a hello world/website now (but ten times less efficient) in the same time that a good engineer used to take. Note that these low skilled engineers can function only thanks to the frameworks. Therefore, you cannot expect them to make a better solution, since they simply don't have the required skills.

Meanwhile, the programmers of the olden days are now required to pay for getting their fix (http://store.steampowered.com/app/370360/) rather than getting payed themselves.

P.S. This thread reminded me of this one: viewtopic.php?f=21&t=6399 and I chuckled

This argument is flawed. What does your first point even mean? It doesn't make sense. On your second point, I don't think you realize that web used to be HTML and CSS. Yes the programmers of old used to sit there and hack it out but that was basically just simple web pages with links, lol. Now it is considerably more complicated. I don't know anyone that sits there in notepad typing out code from scratch when there are templates available to get them a 70% solution that they modify to their needs. That would be like re-inventing the wheel every time you need to build a car, or every time I want to build a house, I go cut the trees down and run them through the saw to make my own boards instead of buying a framed section to stand up a wall from a supply company. Engineering is note manually typing out things from scratch when you can use pieces from another project, etc.

Interestingly this hasn't been the case for operating systems, the system requirements for Windows 10 aren't much different than those for Windows Vista (now nearly ten years old). The reason for this is the move to laptops and tablets where battery life is crucial has driven the development of more efficient software.

SimpleLife wrote:...
This argument is flawed. What does your first point even mean? It doesn't make sense...

I believe it refers to the idea that the time taken to commute to work has been almost constant since before cars were invented. As cars got faster, people moved further away from work. When washing machines were invented, housewives started doing significantly more laundry. Etc. etc.

We're not using technology to develop more efficient habits, but to patch up newly acquired -bad ones- instead.

As others have said, this is a consequence of the economics in play.

Writing efficient software is a highly technical discipline that requires a substantial amount of both practical experience and theoretical understanding, including "scary math." Consequently, the supply of this labor is small. At the same time, the global demand for software (including websites and apps) is huge and growing. There are three ways out of this:

1) Wait for qualified developers to create high-priority software, and content ourselves with the fact that a lot of things we want are low priorities and will never be implemented. Essentially, the Supreme Court approach: a lot of questions never get answered. (Yeah, right.)

2) Use tools (e.g. metaprogramming, libraries, high learning curve languages and editors) that increase the productivity of the limited highly-qualified developers. This is essentially the industrial revolution applied to software. Rather than 1000 craftsmen making one bespoke car each by hand, 10 engineers create a single one-size-fits-all design and one robot assembly line. The downside is that the product, while cheap and well-designed, is not optimal for each consumer's needs. This is Google's approach: hire a small number of ace developers, invest heavily in tools, and scale up.

3) Create infrastructure that allows less-qualified developers to produce code of acceptable quality, thereby increasing the labor supply. This is essentially race-to-the-bottom outsourcing and globalization applied to software. Rather than paying an electronics technician to visit your house and repair your TV every time it breaks, you buy a flimsy new one made and shipped from the other side of the world. The product is mediocre, there is a lot of waste, and some human toll, but those costs are either small or externalized, so the masses go along with it.

The website bloat issue is a consequence of (3).

It must be said that highly competent software engineers do exist, and continue to create tight high-performance software. For example, see stuff like the talks at LambdaConf (http://www.degoesconsulting.com/lambdaconf-2015/) or CppCon (http://cppcon.org/). But they're a small minority of all developers and produce a small minority of all software. And, by the nature of software, most consumers can't recognize quality when they see it. I don't think many people spend time appreciating the Linux kernel's scheduler or GCC's peephole optimizer. This stuff is expensive, though, so the market has figured out to reserve it for situations where it's really critical.

Great post KevinW, I guess this explains why wages in the industry are falling, despite the growth in jobs relative to other industries.

Here is a post I just happened across about the future of the internet (post cheap oil). Slight OT, but interesting nevertheless - warning: doom and gloom afoot.
http://thearchdruidreport.blogspot.co.u ... ortem.html

As an aging engineer I've watched more and more processing power go into the user interface. For the web, more and more is going into advertising.