I used to do that, but someone swiped my rolling machine. Nitrated cig papers work well. And if those aren't available, nitrated onion or rice paper.
But, I do police my straws, even reuse them. They don't burn as much as you would think. Just the cellophane tape is consumed, which is not plastic. To make cellophane, cellulose from wood, cotton, hemp or other sources are dissolved in alkali and carbon disulfide to make a solution called viscose. The viscose is, then, extruded through a slit into a bath of dilute sulfuric acid and sodium sulfate to reconvert the viscose into cellulose. The film is, now, passed through several more baths, to remove sulfur and to bleach the film. A final bath adds glycerin to keep the film from becoming brittle. Not much different from the making of paper from raw materials. And you can smell a paper mill from miles away. Phew!
CP