In the comments on yesterday's post, Sara asked the following:
This fall I am starting the first year of a PhD program, with the eventual goal of heading an academic lab in a faculty position. The impression I have gotten so far from completing an undergraduate thesis, working in labs as a technician, and reading various science blogs was that two people could be equally productive in their research and clever in their networking, but still one may go on to have a successful career and one may not.
Your post may be read to suggest that hard work will prepare a budding scientist for opportunities, and if you do the legwork, are productive in your research, and stay connected to potential opportunities, then you'll likely be successful. Do you think science careers are "fair" in this manner, or do you think it is sort of a crap shoot? What happens when someone picks what seems to be a promising research topic but finds out later that the line of research was a dead end?
There's a lot going on in your question and everyone you ask will have a different opinion. This may also be somewhat field-specific, so I can only tell you what I have seen in my field. That said, the number of trainees produced in any one field is proportional to the amount of money (and PI positions) in that field, so it may just be a question of scale.
In a lot of ways, applying for a TT job is like applying to college (which, ironically, you are). By the time you are applying, you should have a good feel for the schools that are reaches, those on par with your abilities and your "safety" schools. To me, the "crap shoot" argument is the same one people use for not getting into a top-ranked university. "There are so many good applicants, I'm as good as any of them but didn't get into (insert Ivy) because it's all just a crap shoot!" Yes, not everyone gets their dream position, but dreams and reality are often discongruent. If your CV is on par with one level of institution, but you are convinced you deserve something more competitive, you will wind up bitter and find all sorts of perceived slights and roadblocks to your career. I have never personally witnessed an instance where someone whose CV is appropriately competitive for the institutions they are applying to, and is willing and excited to work at that level, has ended up with no job in the end, though I'm sure it happens. Have some people settled for less-than-ideal situations? Absolutely. Have others decided to go in a different direction after some initial frustration? Yup. Might it take 30, 40 or more applications before you find something that works? Hell yes. I think I sent out somewhere in the neighborhood of 35 applications out over two years.
I'm sure you have heard or read that only a small proportion of people who get PhDs actually end up in faculty positions. While on the surface, that's some scary data, you have to realize that A) Not everyone who gets a PhD wants an academic position, B) Not everyone with a PhD is cut out for an academic position (whether or not they know it) C) Some people will just never interview well, which is a big problem unless you are so brilliant that the committee doesn't care that you mumble 90% of the time, and finally D) Some qualified people won't get the position they think they deserve and give up. I would argue that those who work hard to fill their CV and who are realistic in their job search, have a better than average chance of landing a faculty position. What I would not say is that the process is "fair", in the sense that you are guaranteed a job if you do the right things, but no less so than any other competitive position. There will be those who have the skills and never make it, but IMO they are a small percentage.
As far as a research project being a dead end and a student getting screwed because of it, this shouldn't happen for two reasons. It's the PIs job to make sure that if a student goes far enough down a path of research, they'll have something to show for it in the end. I realize that some PIs suck at mentoring, but it is still in their best interest to get a publication out the student's work. Of course, there's the possibility of being scooped, but even still, there should be some angle you can exploit to get a publication from the data. On top of that, it's the student's job to make sure that they have more than one iron in the fire, so that if one project sputters out, it's not terminal. The student who does not branch out from their initial project is probably not going to make it as a PI anyway. It's important to following interesting data and see where it leads, while keeping the initial project running at full steam. Sometimes you may wish you had a third hand, but the ability to multi-task and have a couple of different project running at the same time is what makes people successful in this job.
Finally, do your work with eye to the next step. You don't need to have a PI-ready CV as a grad student, but it is important to be competitive for a solid postdoc and find a lab that will increase your skill set and make you more competitive in the TT hiring process than those who were happy to stay the course in their training. Again, that's my opinion, but I have found that broad experience has allowed me to ask different questions and apply techniques in new ways.
There is nothing inherently "fair" about science jobs, but I don't know what competitive jobs out there have a fair and predictable path these days. However, you can put yourself in the position to succeed by working hard, reading and getting to know as many people in your field as you can. If your science is good and you show that you can think independently, you will be sought after at each stage of the game.