Many do exactly that, especially the ones that like to go to the range and try to end up with just one ragged hole in the target. People looking for "minute of deer" aren't going to go to that extent. It all depends on what a person is looking to get out of their shooting. Fun, competition, bragging rights, enjoyment of doing it traditionally, etc.
What you need to watch out for is making sure you don't exceed the manufacturer's recommended max load, which they state in volume. If I were going to weigh charges, I would set my volume measure to the recommended max charge and then weigh a couple of charges on the scale. Then I'd know what weight I shouldn't exceed...with that brand and granulation of powder anyway.
Many argue that a couple grains of powder effect groups little and it is a waste of time. With patched roundball I've found that playing around with patch material and thickness, and lube and amount of lube effects my groups far more than a grain of powder does...besides my own shaking of course.