I'm becoming a much better programmer, but maybe not a faster one

After ~5 years of real-world experience I've become so much better than I used to be at software design. Interfaces, dependency injection, small classes, composability, factory methods, generics, micro-optimial algorithms, etc. However I'm not sure I have become faster at delivering work items. It used to be that someone gives me a business problem and I could whip out a solution that works, although my code was "bad." Now it takes me the same amount of time, but my code has 100 layers of abstraction over the concrete business problem. This is all very "good" code as it can be justified with "We can easily swap out some dependency and it will still work," but I question whether this future-proofing pays off on average.
 
10,444
3,966
To really master it, you must now abstract only as necessary to support future features. Don’t overdo it.

I’ve seen horrible code abstracted to the nth degree that was too fragile to extend with new features. The interfaces locked down what you could do too tightly.

I worked on one super abstracted system called Taligent. The basic app template was a GuiCompoundDocument that you would subclass from. The problem was this class was subclassed to at least 10 levels and startup was slow, GUI responses were slow and it was very difficult to know what methods to call. It was an era before the IDE tools.
 

FactChecker

Science Advisor
Gold Member
2018 Award
4,805
1,645
Abstraction is valuable when you are writing code that will have a great variety of applications which share some abstract properties, or when you are applying well-known abstract properties to a particular application. Otherwise, it obscures the details of an application that might be simple. In my experience, the vast majority of code is best as simple and direct rather than abstract. Apply the KISS method (Keep It Simple Stupid). I have seen code by people who think that they are developing an entirely new programming language (as though they are making the next Python language) for a simple problem. I personally find that annoying.
 

berkeman

Mentor
54,889
5,134
It used to be that someone gives me a business problem and I could whip out a solution that works, although my code was "bad."
We work on moderately complex software projects here at my work -- they span from small embedded devices all the way up to Enterprise and Cloud-based systems. (Yes, think IoT) We have found that writing "good" code that does not require lots of detailed (schedule killing) debugging is much more important than whipping out something fast to get to market. So to the extent that abstraction helps to write complex applications in a moderate-size development group (spanning many timezones), and to the extent that it helps to maintain and extend the code over the lifespan of the product line, that is a good thing. :smile:
 

PeroK

Science Advisor
Homework Helper
Insights Author
Gold Member
2018 Award
9,351
3,398
"The myth of future proofing"!

but I question whether this future-proofing pays off on average.
This is a good question.

I worked mostly in Business Systems. The problem I always saw was that changes when they come are often outside the scope of what was future proofed.

I sat in many client presentations where they would ask "is the system future proofed". And, of course, the answer had to be yes.

But future proofed against what scope of change?
And this was against a background of IT systems fundamentally changing architecturally every 5 years or so.

Then, when the client submitted an innocuous looking change request, the costs were enormous. For example, a requirement for full scale system, integration, performance and user acceptance testing would massively outweigh the raw development effort.

The code changes were only a minor part of the overall implementation costs.
 

FactChecker

Science Advisor
Gold Member
2018 Award
4,805
1,645
Then, when the client submitted an innocuous looking change request, the costs were enormous. For example, a requirement for full scale system, integration, performance and user acceptance testing would massively outweigh the raw development effort.

The code changes were only a minor part of the overall implementation costs.
This speaks to the wisdom of "code for testability". Where I worked, people based their entire career on developing a test system which could inject and monitor variable values in a system that was running in real time. Static variables with fixed addresses at test points were desirable.
 
1,231
600
I have seen code by people who think that they are developing an entirely new programming language ... for a simple problem. I personally find that annoying.
Every problem has its own language - but most of the times it's good enough to put that into comment and do not cement it into the structure of the product :thumbup:

However I'm not sure I have become faster at delivering work items. ... Now it takes me the same amount of time, but my code has 100 layers of abstraction
You have cleared a level but not the whole labyrinth. Just keep gathering XP and it'll get better eventually.
 
229
57
The advantages of fully applying formal design methodologies doesn't really kick in until 7-800ish lines, but they pull away rather sharply from the "organic"(to be polite) method after that.

Reason, far as I can figger, is that development overhead increases linearly with program size/complexity, but debugging is exponential.
 
Reason, far as I can figger, is that development overhead increases linearly with program size/complexity, but debugging is exponential.
I'm not so sure about that. When layers upon layers are programmed to abstractions, it can be very difficult to figure out from reading the code what it actually does at runtime in the context of the application that I'm trying to debug.
 
229
57
I'm not so sure about that. When layers upon layers are programmed to abstractions, it can be very difficult to figure out from reading the code what it actually does at runtime in the context of the application that I'm trying to debug.
Much, much easier if you know what the "abstractions" used were, and if they were applied consistently. Your shop might have standards, for such.
 

QuantumQuest

Science Advisor
Insights Author
Gold Member
838
431
However I'm not sure I have become faster at delivering work items.
I won't say you should necessarily, as long as the work items are quite or vastly different from what you have already developed and for the time span you talk about. Software development skills need a lot of time to mature but the whole thing is also very dependent on specific factors, which would make for a very long discussion.

It used to be that someone gives me a business problem and I could whip out a solution that works, although my code was "bad." Now it takes me the same amount of time, but my code has 100 layers of abstraction over the concrete business problem. This is all very "good" code as it can be justified with "We can easily swap out some dependency and it will still work," but I question whether this future-proofing pays off on average.
A solution that works but written using "bad" code - I assume this to mean not thoroughly thought /designed and / or tested and / or documented, won't give a brilliant future to your application and I think it's needless to say why as it is very obvious. On the other hand, giving an unnecessary lot of layers of abstraction without the appropriate design work beforehand, is also a call for trouble. Abstraction is neither for free nor of low price, generally speaking.

It definitely helps towards having code that can be modified / adapted without lots of effort and in a number of other things as well, but overdoing it will have performance costs - to say the least, and finally will lead to a complex piece of code for which it is many times very questionable if it is reasonably inside the demands and the constraints of the solution needed. Unfortunately, it is very evident that software development has followed this trend in many kinds of applications. Leaving aside the professional reasons that justify this, including protection of intellectual property, I think that the whole thing goes out of hand in a vast number of cases.

Now, for the question of future-proofing, I would say that in most cases it pays off on average, as long as the client remains inside the boundaries of what he / she has initially asked for. As @PeroK says, don't be surprised if the client asks for something that will essentially cancel future proofing. So, I think that keeping reasonable measures is the best thing to do.
 

rcgldr

Homework Helper
8,573
472
embedded devices - future proofing
Consider the case of hard drives. 32 bit sector addressing was good enough until hard drives exceeded 2 TB in size. The host to drive interface had already been changed to allow for 48 bit sector addressing, but it was a significant change to the firmware for the drives. Another example was the addition of the SSE family of instructions to X86 processors, where in the case of most programming languages, new code had to be written to take advantage of the xmm registers and their parallelization of operations.

abstraction versus performance
In the case of hard drives, other embedded devices, and some applications, performance is a key factor, and abstraction beyond a certain point affects performance. Compile time abstraction, such as C++ templates, allows for abstraction that generally doesn't impact performance.

encapsulation - get - set
In some cases the usage of get and set becomes excessive (this is somewhat opinion based). It's rare that a significant change to a class member isn't going to affect the code that does the get, modify, and set for that class member.

faster programmer
Usually a programmer does get faster at both design and implementation, unless a project is unique compared to prior projects, or requires the development of a new algorithm.
 
10,444
3,966
Get and set are good future proofing schemes as they allow you to add in validation during a set operation or changing an instance attribute to a computed value.

Kotlin, for example, provides getter setter methods if needed but always makes them appear as direct access. In contrast, Java is more explicit and insists on getter/setters for access to instance attributes in their java bean scheme.

Interfaces are also a good future proofing scheme allowing you to define a protocol for classes and allowing you to change out one class for another. As an example, a tax application might have a calculator interface with an agreed upon list of methods.

Calculator classes for each tax year can be written supporting these methods but doing slightly different calculations for each tax year. The tax program can maintain the same GUI but with changing tax year computations and the interface provides a clean separation.

There's also the notion that a few good interfaces can make understanding a program's flow easier although you give up the easy ability to track through from class to class through an interface as there may be several possible classes to choose from. Instead, you'll need to use a debugger to see what class is actually used on the other side of the interface.
 
Last edited:

Dr Transport

Science Advisor
Gold Member
2,199
364
Over the years, I found that I was able to produce functioning codes faster because I had a toolbox of my prior codes to draw from, i.e., like a toolkit. As you start writing a code from scratch, you find you need an algorithm that you did a couple of years ago and you adapt it into the current project. You might improve it some but the basic flow is there. The code group I worked in had developed some of these for geometry and other basic operations so that you could get a code running quickly and be inline with the groups coding standards.
 
155
61
However I'm not sure I have become faster at delivering work items.
This is the motivation for low-code platforms such as OutSystems. These platforms are establishing industrial process into software development, and reusable abstraction is the core of that. You can deliver apps about four times faster once you get up to speed, which is a huge productivity boon. They also slash the ongoing maintenance effort, which combats technical debt. As has been noted, common abstractions for aspects such interfaces, connectors, and user elements are helpful, but that's the tip of the iceberg when a drag/drop automatically creates the entire CRUD UI for a class of database items!
 
1,495
604
My recommendation is to practice coding quickly. Coding quickly and well are two different skills and a good programmer should be able to do both. You'll likely have situations where you'll need to professionally.

I find that future proofing does not pay off. I rarely write code that I would call future proof. I just write it in such a way that IF I need to change it later, I can without too much trouble. For example, any time I'm dealing with a database, I will put all of my queries in one place. However, unless I specifically have specs that say I need to make it replaceable, inside the db class, there might be a mess.

Think about it like this. If a customer says "mysql is nice, but I really need to be able to use Oracle" there are three possible responses:

1) No problem, I just have to write the DAL for oracle because everything is already using dependency injection
2) Okay, I need to modify the database layer, then write the oracle
3) Seriously? I have queries and sql dependencies everywhere, it'll take a week to refactor all of that.

Sounds like you are usually going to give response 1. It's okay to give response 2. Just don't write it so poorly that you become case 3.
 

symbolipoint

Homework Helper
Education Advisor
Gold Member
5,527
871
My recommendation is to practice coding quickly. Coding quickly and well are two different skills and a good programmer should be able to do both. You'll likely have situations where you'll need to professionally.
My beginner-level skills and only some hobbyist experience tells me that Coding Quickly means Making Mistakes That Might Be Difficult to Find and Fix. This destroys the goal of Coding Quickly.
 

FactChecker

Science Advisor
Gold Member
2018 Award
4,805
1,645
My beginner-level skills and only some hobbyist experience tells me that Coding Quickly means Making Mistakes That Might Be Difficult to Find and Fix. This destroys the goal of Coding Quickly.
On any program, there are time and budget constraints. So speed and efficiency become important.
 

.Scott

Homework Helper
2,234
726
A few points, many that have already been addressed:
1) Future-proofing: If you know of specific changes that are already planned, then coding with those features in mind makes sense. Otherwise, I have found that attempting to predict what changes or what kind of changes will be made in the future is a loosing game.
2) Instead of future-proofing, think maintainability. Work to make your code easy to understand. Document what is not obvious - including the reason that the code exists at all.
3) Fast coding: There are a few things that go on during the "coding" process. First, all the most detailed design work happens at coding time - everything that precedes that was either less detailed or only a guess. Second, is the coding itself - everything related to the syntax and form of the source code. Finally there is the typing. Certainly the coding and the typing accelerate as you gain more experience. But those final detailed design decisions are key. As far as the design is concerned, give it as much time as it needs. By scrimping on the process of understanding the requirements and other design steps, you can get very fast coding - but you risk running into dead ends or maintenance issues that will sink the schedule.
 
600
240
A few points, many that have already been addressed:
1) Future-proofing: If you know of specific changes that are already planned, then coding with those features in mind makes sense. Otherwise, I have found that attempting to predict what changes or what kind of changes will be made in the future is a loosing game.
2) Instead of future-proofing, think maintainability. Work to make your code easy to understand. Document what is not obvious - including the reason that the code exists at all.
3) Fast coding: There are a few things that go on during the "coding" process. First, all the most detailed design work happens at coding time - everything that precedes that was either less detailed or only a guess. Second, is the coding itself - everything related to the syntax and form of the source code. Finally there is the typing. Certainly the coding and the typing accelerate as you gain more experience. But those final detailed design decisions are key. As far as the design is concerned, give it as much time as it needs. By scrimping on the process of understanding the requirements and other design steps, you can get very fast coding - but you risk running into dead ends or maintenance issues that will sink the schedule.
I like IBM's model of backward compatibility. Code that was written for machines of decades ago can run unchanged on the latest systems without a hiccup. It can't use the more recent features, but it can still do now what it did then; if that model is kept in effect, code written yesterday and today will still work tomorrow, just as yesterday's code still runs today.
 
155
61
I like IBM's model of backward compatibility. Code that was written for machines of decades ago can run unchanged on the latest systems without a hiccup.
Is this their mainframe hardware? And what OS? Irrespective, old code is hard to maintain and esp. hard to extend. I worked on an insurance system that is a couple of decades old. We hit a speed hump, and one of the older devs literally pointed across the office and said "Speak to Paul, he wrote that code in the first place." Needless to say, that helped, but even Paul struggled to figure out what his code was doing more than ten years after he wrote it. Without Paul? We probably would have just hooked in a newly coded extension, that's way cheaper than decoding what dinosaur devs were thinking :biggrin:
 
600
240
Is this their mainframe hardware? And what OS? Irrespective, old code is hard to maintain and esp. hard to extend. I worked on an insurance system that is a couple of decades old. We hit a speed hump, and one of the older devs literally pointed across the office and said "Speak to Paul, he wrote that code in the first place." Needless to say, that helped, but even Paul struggled to figure out what his code was doing more than ten years after he wrote it. Without Paul? We probably would have just hooked in a newly coded extension, that's way cheaper than decoding what dinosaur devs were thinking :biggrin:
Yes, I was referring to IBM mainframe hardware. In reference to today's machines, the 'mainframe' term is retained primarily to distinguish the direct successor machines, which run a superset of the instruction set of the predecessor machines, from the other systems available. In terms of OS, I'm thinking of the whole IBM mainframe OS family, all of which operating systems observe backward compatibility for application code, as the mainframes on which they run do for any code. Old code is hard to maintain? Well, if it was poorly written in the first place, maybe it is. To gain perspective, please go and write some machine language fixes and mods with only core dump printouts to work with, you big crybaby. :cry: :rolleyes: :wink:
 

rcgldr

Homework Helper
8,573
472
Is this their mainframe hardware? And what OS?
Typically Z/OS. Think of it as hardware and an OS that can run multiple virtual machines, each with its own virtual hardware and virtual OS (tri-modal addressing), but at full speed and in parallel.

 
155
61
Ah yes, the platform that IBM spruiked as their "Highlander" - there need only be one. Pretty much jumped the shark when you could install X86 blades and run Windows apps.

"Wow, a mainframe that we can put a PC in," said nobody ever!
 

PeroK

Science Advisor
Homework Helper
Insights Author
Gold Member
2018 Award
9,351
3,398
I like IBM's model of backward compatibility. Code that was written for machines of decades ago can run unchanged on the latest systems without a hiccup. It can't use the more recent features, but it can still do now what it did then; if that model is kept in effect, code written yesterday and today will still work tomorrow, just as yesterday's code still runs today.
There's a big difference between writing a mainframe O/S (or any O/S) and a "business application", which is what I took @.Scott's advice to apply to.
 

Want to reply to this thread?

"I'm becoming a much better programmer, but maybe not a faster one" You must log in or register to reply here.

Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving
Top