Pretty much what everyone else said. I live in the LA area, I do a lot of recording outside. Sometimes you'll find that different urban locations are quieter than others. I recorded the cinderblocks in my Rocks library in my mom's backyard. She is only 5 miles away from me, also in the middle of a city, but her yard is quieter than mine. Otherwise, you just have to find the quietest times.
The desert, mountains, hills, etc, are also good places to try to record in. As most know, I recorded most of the rocks library in the desert outside of LA. The problem with LA, is you run into 2 military bases in the desert here and even the hills and mountains are not far enough away from civilization. People live everywhere around here.
Mic selection and placement can help a great deal as well. I really try to get as close as possible to the source. Shotgun mics offer decent rejection and that can help. I really only try to record loud stuff in the city. Anything quiet just gets killed by the urban noise.
You just can't escape noise in the big city! I have to think that LA is better than NY. I can't imagine trying to record anything in and around NYC...