Start by cutting some straws into quarters. I had fancy paper straws I'd bought on clearance; plastic works just as well.
Cut a 3 (or so) inch length of pipe cleaner and bend over one end. Bend the remaining length in half, as shown.
Except, hmmm... Missing something. I had a spare tassel (made in the security envelope ornament post) so I tied it to the bottom tip.