MHB Data Science and Systems Integration Blog Post

Click For Summary
The discussion emphasizes the importance of data handling and the challenges faced by experts in maintaining a fresh perspective when analyzing data. It highlights the difficulty of empathizing with those less knowledgeable due to the internal assumptions formed through experience. The conversation advocates for a bottom-up approach in system design, arguing that while some guidance is beneficial, excessive direction can lead to user frustration. There is a debate about the merits of top-down versus bottom-up design, with one participant questioning whether a top-down approach can be valid if it involves using the same data for both hypothesis generation and testing. The need for a balance in documentation that is both concise and adequate is also raised, underscoring the complexities of effective communication in data-driven environments.
Ackbach
Gold Member
MHB
Messages
4,148
Reaction score
94
Really excited about this blog post by yours truly. Wineman Technology is going places!
 
Technology news on Phys.org
Interesting article!

I work with large amounts of data every day and would consider myself an expert in the field when considering the whole continuum of "Can't use a computer" to "Coding at Google". I grab data, clean it, join it, model it, and present it.

That is important to your point, because what I lose in my years of training is the ability to see data and a problem with new eyes. Once you know something, you can try to empathize with the mind of someone who doesn't know it but it is quite difficult. Even worse, you can be very unaware of the facts that you internally assume everyone must know yet they don't.

So I fully agree with you that systems benefit from building bottom-up, not top-down. Some guidance is helpful but too much will leave the end user clueless and frustrated.
 
Jameson said:
Interesting article!

I work with large amounts of data every day and would consider myself an expert in the field when considering the whole continuum of "Can't use a computer" to "Coding at Google". I grab data, clean it, join it, model it, and present it.

That is important to your point, because what I lose in my years of training is the ability to see data and a problem with new eyes. Once you know something, you can try to empathize with the mind of someone who doesn't know it but it is quite difficult.

Yeah, yeah. Total nerd. I get it. ;)

Jameson said:
Even worse, you can be very unaware of the facts that you internally assume everyone must know yet they don't.

So I fully agree with you that systems benefit from building bottom-up, not top-down.

I'm a hair puzzled. I thought I was advocating for a top-down design approach, which is simply what a typical statistics textbook will tell you. If you have an hypothesis at hand (big question, in the parlance of my blog post), then you can design the system that generates the data that, once analyzed, answers the hypothesis or question. Isn't that a top-down design approach? It is considered "cheating", is it not, to use the same data to generate an hypothesis as well as to test it?

Jameson said:
Some guidance is helpful but too much will leave the end user clueless and frustrated.

I'm very curious what you mean here. Are you talking about achieving that difficult-to-achieve balance in documentation between conciseness and adequacy?

Cheers!

By the way, I LOVE THE QUOTE-BREAKER!
 
Learn If you want to write code for Python Machine learning, AI Statistics/data analysis Scientific research Web application servers Some microcontrollers JavaScript/Node JS/TypeScript Web sites Web application servers C# Games (Unity) Consumer applications (Windows) Business applications C++ Games (Unreal Engine) Operating systems, device drivers Microcontrollers/embedded systems Consumer applications (Linux) Some more tips: Do not learn C++ (or any other dialect of C) as a...

Similar threads

  • · Replies 10 ·
Replies
10
Views
2K
  • · Replies 8 ·
Replies
8
Views
1K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 6 ·
Replies
6
Views
2K
Replies
10
Views
5K
Replies
5
Views
2K
Replies
4
Views
3K
  • · Replies 29 ·
Replies
29
Views
5K
  • · Replies 2 ·
Replies
2
Views
3K