Special feeders are sold which allow you to spear an orange half on a holder, to watch the orioles come to enjoy. You can just as easily drive a nail into your favorite tree and spear an orange half (juice outward) onto it.
The females are lighter in color, while the males show the distinctive orange-breast black-wing-and-head design. They have very clear, whistling notes. They also enjoy wooded areas and insects.
Note that the Baltimore Oriole is one of two birds that used to both be called the Northern Oriole. The Baltimore Oriole was so named in 1731 because its colors matched those of the English Baltimore family which ruled Maryland. It's found in the eastern US. The Bullock's Oriole was named after the owner of the London Bullock's Museum - William Bullock found this bird near Mexico. The Bullock's Oriole has more black on its throat and back, and is found in the western US.
There are also other orioles out there - a smaller Orchard Oriole, brighter Hooded Oriole, yellower Scott's Oriole. The orioles sometimes breed with each other, so the best you might be able to do is a rough guess!
More Baltimore Oriole Photos
Recipe for Oriole Nectar
The Birding Encyclopedia
State Bird Listing
Print Available from Art.com