Depends on how dirty the floor is. With the usual state of our floors, I'd probably just brush it onto the floor, since there's usually lots of crumbs there to begin with.
Our floor always has tons of crumbs (not too surprising with the kids around and no dog to clean up after them), but I still find myself wiping the table and high chair into my hand all the time...and I think it would be less gross to just wipe them onto the floor. I'm certainly not saving myself any time since sweeping the floor is next. (Though I get Kansas Dad to do that as often as possible. I don't mind washing the dishes, most of the time, but I really hate sweeping. I can't explain it.)
I just brush them onto the floor. We have so many crumbs I can't catch them all in my hand anyway!
Comments make me happy; thanks for speaking up!