The short answer
Yes, in small amounts — oranges are safe but acidic, so moderation matters.
Why
Oranges are high in vitamin C, but they're also acidic and sugary. Too much citrus can upset a budgie's crop and digestion. Remove the peel and any seeds.
How to feed it
Peel it, remove the seeds, and offer a small segment (you can break the membrane so it's easier to eat).
How much & how often
A small piece about once a week.