That is the $64,000 question :)
The flagship commercial library providers are Sound Ideas, Hollywood Edge, Soundstorm, and Blastwave FX. Sound Idea's primary library is the 6000 Series, and for Hollywood Edge it's the Premiere Edition. These are well-rounded, diverse libraries which can be a great place to start. From there you have a variety of boutique libraries put out by individuals such as Tim prebble, Frank Bry, Chuck Russom, Colin Hart, and many more. These are more geared toward specific topics and scenarios, so they won't have widely-diverse categories in each licensed set.
As far as affordability goes, that's the tricky part. The 6000 Series discs 1-40 costs about $1,200 and the Premiere Edition about $800 I believe from the first 20 discs. Soundstorm is $6000. What may be advisable then, given your budget, is to grab a decent recorder like the H4n and do your own recordings - this of course is considering that you wouldn't need to use the FX immediately and have time to build up the ranks so-to-speak with sounds that you want. If carefully watch the Sound Ideas deals, you may get lucky and be able to grab the 6000 discs 1-40 for about half price and that alone would be a good jump-start to a workable library. My feeling on this is many of these libraries are expensive, but a necessary 'evil' in the end.
Regarding the book, I think it's good but I have a few reservations about it. Overall he makes some good, valid points about the entire process of sound effects recording and preparation and he does know what he's doing. I respect his take on the process, and I personally choose to take and adapt to only the parts I like about what he offers and leave the rest.
This is all of course just my personal opinion