First, it's important to understand that speed and flexibility go hand in hand. Cold muscles simply can't move fast. Now that stretching has been glossed over, it's time to look at the difference in the kinds of muscle fibers in the human body.
Cardiac and smooth muscles are those of which the organs are composed. Striated, or skeletal, muscles are under voluntary control. This means that they function when needed, and these are the muscles that athletes try to tone, grow, strengthen, etc.
Of the striated muscles, there are mainly two kinds. These are slow-twitch muscles, and fast-twitch muscles. Slow-twitch muscles are designed to be fatigue resistant whereas the fast-twitch muscles fatigue quickly, but have powerful and quick contractions.
Generally, everyone is born with a set percentage of these kinds of muscles (this is why some people are born athletes), but everyone can alter this percentage in some form through exercise. Let it be known that studies indicate muscle fibers cannot be created. Instead, what happens as a result of exercise is that existing fibers grow in size, density, etc. This is the hypertrophy versus hyperplasia theory and can be viewed here.
So, back to speed training. Simply work areas containing muscles laden with fast-twitch fibers like the thighs, arms, shoulders, and obliques by utilizing the very thing you're building on; speed.
By either avoiding the use of weights during speed training, or using very little weight, you reduce the risk of injury, which can be prevalent in exercises utilizing rapid motion. For instance, a sprinter practices pumping the legs as quickly as possible, and doing this everyday forces the fast-twitch fibers of the muscles to adapt. A martial artist throws a rapid succession of punches, and with each punch he tries to pull the fist back into a starting position faster, and faster. Using weights in these situations can cause undue stress on tendons and ligaments, so be careful. It may seem complicated, but here are simple speed building drills.
Watch this video on increasing your sprinting ability.
Watch this video on increasing your hand speed.
Watch this video on explosive speed.
Youtube fitness channels are a great place to find fun drills to try, but always consult your physician before starting a new work out regimen.