The issue here is probably one of attention span. Reading comprehension is a skill, it can be improved, degenerate and is very unique to each person. Everyone has their own areas with difficulties and strengths. I myself, have a horrible habit of trying to speed read and speed write everything, which often does affect my comprehension and writing. I find myself wrong afterwards if I get too excited about it and can't slow my pace. I always have something to write with when reading books and some online texts. You don't have to take notes on the whole book, but before even starting, do some reflecting on what you need from the book and what information you should prioritize before tackling it. Building up a strategy before you begin will help guide you through the book.
It will also be your guide to writing notes on the book, because you can't take notes on the whole thing. Just try to summarize the main ideas as you go, and write complete copies of the text of short sentences/problems that you want to remember; also writing your own opinions, reminders, and questions to yourself as you go can help with remembering that information. The greatest benefit in note-taking is it slows down our thought process long enough to reach deep connections that are already in place in long-term memory, we usually read so quickly that we don't have enough time to make more associations before moving on to the next idea in the book. That's very important and note-taking can help engross a person in the material. But, not all people are same, and others use different methods in reading. Much of it does depend on what you already know about the material. There are many methods you can use as tools to improve.
I wouldn't feel so bad about believing that you cannot do it. When I first began reading material in physics many months ago, it had been very intimidating, as my prior major after high school was in education and now biology, and neither had been no where near as difficult.to understand. The literature is full of terminology and ideas that I have never encountered before, and it was just very confusing for me. I also have had little exposure to higher mathematics, and have forgotten so much of what I thought I learned in and after high school- it's not like I can re-take courses that I passed already! I do, however, consider my naivety in the subject a blessing and am trying to be cautious with how I process this information.
Although, things can be uncomfortable and frustrating for us at times, we have to remember to continue pushing through. Eventually, it will click. I do feel a little more comfortable now and understand that I have to be patient in learning this. I'm excited about learning more in physics and mathematics, but know that I have to be more patient with myself and set realistic goals. I expect it will take me many, many years to fully understand the body of knowledge in physics. Wading is confusing and uncomfortable for everyone, it can be painful at time, but we have to keep going and not give up!
As far as making lists of the books that you want to read, I think that doing that may not be a good idea at the time, at least not a very long one. Not all material can be read quickly, but most can- there is just too much information and knowledge to process and too little time to do it. You have to learn how to judge what needs a scan-through and what you need to read more carefully. Oftentimes, a good book will reference you to other good books (same goes with papers), paying attention to that will help you decide what to read next. And if it constantly being referenced and is heavily related to the topic of your interest, then you really want to put it on a list. I will say though, that I usually try to stay away from popular science books, there are so many textbooks and papers to read! I do think they have value as a secondary supplement, but I haven't really learned so much about anything from reading only that kind of text. It sometimes confuses me even more.
Tell you what, I was going to my campus library today anyway, I've actually been missing it, usually, I avoid being there unless I have to be. They are boring and I don't get along with everyone. I will find a popular science book and read it over the next few days. I will use my notes to write a little review and post it on here, you can do the same. I will post it by Friday.
