Often (but not always) "gain" on a line-in is actually attenuation. "Line-In" can refer to a number of different signal levels, ranging from -10dbV to +6 dbu (this part you may already know). A "line-in" jack can be set to take say -10dbV, at 0 attenuation, and then attenuate the signal accordingly as the signal get's hotter. However "gain" is more consistent terminology with how we audio folks work, so it's often expressed in this fashion. So at -10dbV for example is set as a gain of "10" (using a 1-10 scale) which is NO attenuation, +4dbu at "4" and say +6dbu at a gain of "0" which is actually MAX attenuation.
In some cases however, the "line-in" is set at a nominal level (let's say +4dbu) and then an amplifier circuit will provide gain for lower level signals. The difference in this amplifier circuit is a question of gain- hence the term "pre-amp" - something that processes the signal "pre" or before another amplifier. The amplifier at line level wants to see a considerably louder signal then at mic level and provides less gain, making it a much simpler circuit.
Some decent Wikipedia (i.e referenced) articles can be found here and here