What is your experience with estimating student workload?

[bigfooted]

I recently was asked to create some new exercises for a course that are more suited for online learning. It is a grad-level course and involves working in an online environment (jupyter) and doing some exercises on specific physics concepts using python. They were given working examples that they then had to modify for the assignment. Knowing python was not a prerequisite so the exercises did not rely on fancy programming techniques. Usually it was a matter of changing the numbers, change some input file, plot the same stuff for more conditions, and thinking about what kind of information to put in the figures.
The prof/main lecturer for the course did not have python experience and he made the exercises in 2 hours. We thought it would take students approximately 10 hours. They spent over 20 hours on it.

What is your experience with estimating the time it takes student to make an assignment? We thought that a factor of 5 compared to us making the assignment was sufficient, but apparently they needed much more time.
I think in retrospect we underestimated the pre-existing knowledge of students, but also their inability to efficiently google things they do not know. Another aspect is that even though there was a chat in which they could always ask questions, this was hardly used and nobody indicated that they were spending too much time on the exercises until it was time to hand everything in and they were not finished.

 

[.Scott]

I have made similar mistakes is estimating how long it will take someone to program. Someone attacking any type of computer programming cold turkey has several conceptual hurdles that are easy to take as granted. It isn't enough for them to know what the code does, they need to know how it does it. And if they have very little programming experience, they need to also understand how that code fits into the development environment. They can't just see something happen and attribute it to "magic". So there is a learning curve.

As an example of how deeply they need to understand whaat is going on, consider diagnostic messages. If they forget a comma, they will probably get some message other than "you forgot a comma". So they need to start looking at the syntax in the same way that the computer does. The basics are not more complicated than Math of Physics, but just having a Math of Physics background isn't at all sufficient.

 

[kuruman]

I would think a factor of 5 is quite generous. In my opinion it's not the time limit but the attitude that makes the difference. When I estimated the time required to finish an in-class exam, I used a factor that varied from 3 in introductory courses to 2 in intermediate level courses. These were in-class proctored exams. Students were not allowed to talk to each other, answer their phone, take a break, etc. The on-line homework that I assigned was treated pretty much the same way as in your case. Although students has a full week to complete the assignments, and were given ample opportunities to come to office hours with questions before the deadline, only a few usual suspects took advantage of the offer. The rest left it for the last moment. Here is a cut and pasted segment from an earlier post on this forum. My completion deadline was at 8:00 a.m. to give me a chance to review the answers and talk about them in the lecture.

A great number of students were "negative timers"; they estimated about how long they need to complete the homework and counted backwards from the 8:00 am deadline to figure out when to start. That meant 3:00-5:00 am which they found unacceptable and complained. So I asked for a show of hands, "how many of you would like to change the deadlines from 8:00 am to midnight?" About half the class raised their hands. I picked up a pencil from its tip, waved it around Harry Potter style and said, "with this magic wand I have changed the deadline from 8:00 am to midnight." After a few seconds of quizzical stares, I added "And everybody gets an automatic extension from midnight to 8:00 am." That stopped the complaints and I hope taught them something about time-management.

Thus, my opinion is that if you allow your students 20 hours for the task, nothing will change. The ones who cannot complete the task in 10 hours need to change their attitude and learn how to stay focused on their work ignoring distractions. I bet you that the prof/lecturer who completed the task in 2 hours did just that. Teaching good habits to students is part of the deal.

I also with @.Scott 's post which offers an orthogonal dimension to the above. Knowing what to fix and how to go about fixing it when code doesn't work takes some experience. However that could be remedied if students started the task immediately and give themselves extra time to ask questions before the deadline.

 

[Vanadium 50]

It's true that nobody ever said "Wow...he has the time management skills of a college freshman. I wish I could be like that". But I think this case is likely better explained by a lack of facility with programming, and worse, a lack of understanding that there is a lack of facility.

You can see examples of that here. People struggle, in part because they did something they didn't understand which happened to work by accident, and now they want to do it every single time.

 

[.Scott]

It's true that nobody ever said "Wow...he has the time management skills of a college freshman. I wish I could be like that". But I think this case is likely better explained by a lack of facility with programming, and worse, a lack of understanding that there is a lack of facility.

When it comes to time management, computer programming is especially challenging. If you tell them to read a chapter, they can can quickly determine the number of pages and how long it will take them to do the reading. With programming, everything you can think of at first will only take minutes to complete. What takes time is everything you don't think of. The solution is to get into the riskiest areas ASAP - as well before the deadlines as possible.
In the case of these Python assignments, you might suggest that your students complete one exercise with the Python code immediately - just so they can discover what they are getting into and just so they can schedule things out for themselves.

The other issue with this kind of assignment is that college students often look to others (most likely other students) to assist them. But the kind of assistance they need may be hard to come by especially under Covid rules. Of course, they can try PF.

 

[Andy Resnick]

What is your experience with estimating the time it takes student to make an assignment? We thought that a factor of 5 compared to us making the assignment was sufficient, but apparently they needed much more time.

This is a common problem without a clear solution. My approach is essentially 'partial credit'- in your case, are there intermediate steps or checkpoints within the overall problem solution that students can earn some credit for?

 

[Office_Shredder]

I agree that the first programming problems will be very tough for someone who can't program. The first time my wife had to do a similar problem in python, with no instructions on how to install python, went something like

What's python?
Ok, well we can install basic python just to get you started tonight.
Whoops, we installed 3.1.6 but you need at least 3.2 to get this function to work.
Ok, we need to install numpy.
Ok, we need to install scikit.
Ok, we need this plot thing.
Ok, we need... You know what, anaconda has all this stuff to start with. Let's install that.
Ok, now we've got anaconda. Wait, how do you even open the ide on this thing.
Great, we got it open, and got the program they gave you to run. Good luck with the homework.

 

[Vanadium 50]

Whoops, we installed 3.1.6 but you need at least 3.2 to get this function to work.

Could have been worse. Could have been "Whoops, we installed 3.1.6 but you need 2.x to get this to work"

 

[bigfooted]

This is a common problem without a clear solution. My approach is essentially 'partial credit'- in your case, are there intermediate steps or checkpoints within the overall problem solution that students can earn some credit for?

I created 6 notebooks with examples and questions for the assignment, and they were given credits for each of these sub-assignments. We expected some distribution, where some students were very fast and finished everything in time and some would struggle to finish everything. I also think that everybody working individually at home and out of sight was a factor. Then you do not see that most of them are struggling with basic things like dealing with error messages or finding out how to modify a plot command.

@Office_Shredder, we circumvented installation problems by working with jupyter online through binder, so they didn't have to install anything.

 

[bigfooted]

In the case of these Python assignments, you might suggest that your students complete one exercise with the Python code immediately - just so they can discover what they are getting into and just so they can schedule things out for themselves.

This is a very good idea. I was thinking about adding a 1-hour python tutorial that will cover all the python commands that will be needed and let them go through that first. Concluding with an exercise that needs to be finished and handed in immediately after the introduction would have given us an estimate of their capabilities.