Racers upshift that way. It is faster and won't hurt a thing. As far as downshifting goes, you'll smooth the transition out by blipping the throttle while the clutch is pulled in. Helps to match engine speed to drive speed in new gear. Racers do that too.