It is really hard to write a fair and useful exam. A basic topic like 1st year grad algebra is taught over 9 months and covers possibly hundreds of pages of difficult stuff from an advanced book. How do you choose a few questions, answered in 3 hours, that measure mastery of all of that stuff? (Often people telegraph what the questions will be, but necessarily only to ones own students, hence penalizing others taking the test.)
Also there is an audience for a given test. A PhD prelim test is written for grad students who have recently taken a detailed course in a given topic, and is often written by someone who taught that course. A typical faculty member is someone who may have not thought about some of the topics for a long time. So although that faculty member may have done the hard work to be sure of passing the prelim at the time they took it, they may not still know all that stuff 5-10 years later.
I myself, when I was a faculty member, could hardly pass any prelim outside my own area without preparing for it. Thus you have to ask yourself what is being tested? Is it the knowledge of material years later, or is it the ability to knuckle down and do what is asked at the time? I.e. are you measuring hard work or knowledge?
When I myself wrote the algebra prelims, I was not appreciated, because I wrote questions that tested exactly what was required. I.e. we had a syllabus that stated exactly what one should know, and I tested exactly that. Why was this despised? Because the students did not read and study according to the required syllabus, rather they just looked at the old exams and practiced answering the same questions that had been asked in the past. Those questions focused on only a small fraction of the syllabus. Since I had to work hard to cook up questions that actually measured knowledge of the majority of the syllabus, the students who took my tests were surprized at what was asked.
E.g. instead of making them to show a group of a given order was not simple, I might ask them to answer a true or false question that tested whether they did or did not know the statement of some theorem that they had been told to know. I.e. the syllabus said to know the sylow theorems, but the old tests only asked specific problems using some of those theorems. I would ask questions testing their knowledge of the statement of those theorems. I would ask questions that could be answered if one only knew the statement of the basic spectral theorems, exactly what was required on the syllabus, but my students were stuck, only having practiced diagonalizing a given matrix.
I did not purposely sabotage my students, rather I meant to help them by asking them to answer exactly what they had been told to know on the syllabus, but they assumed I would just ask the same questions that old exams had asked. Apparently I was the only person who ever naively took the requirements at face value, and the department never asked me to write the algebra exams again.
I have told the following story before but it may still be amusing, about my own experience with prelims. I was at two grad schools, Brandeis and Utah, and in between I just walked in and sat for the prelims at Univ of Washington. So I had passed them at both Brandeis and UW, when I arrived at Utah, where I asked on day one, how I should sign up for the prelims. This question stunned the staff there who said:
"What! You want to take the prelims? no one has ever asked that! Our students usually want to avoid them as long as possible."
I said: "Well, if I can't pass, maybe I should be seeking a different career, so I want to find out right away." They answered: "wait a minute, have you ever taken them somewhere else?'
I admitted, "well yes, I have passed them twice already, at Brandeis and UW."
the answer: "oh, then you don't need to take them. you pass."
Of course as you have guessed, when i saw the prelims that year, there were plenty of difficult group theory questions I could not answer. (They had a strong specialized group theory section there, who tested their own favorite topics.)
So apparently in those days at Utah, the prelims partly measured fear of them.