Here is a description of plants that have either fragrant leaves or flowers.
Little Gem Marigold (Tagetes sp). Annual. I love marigolds because they are so bright and cheerful, come in all sizes, and the smaller ones are great for the front of the border. Little Gems also are very fragrant. The entire plant smells of citrus. Delightful bright, sunny colors and this delightful fragrance. It grows no more than a foot and has very interesting foliage as well. Needs full sun, water when dry and well drained soil. I have not seen any pest damage on these so think they may be resistant to earwigs and slugs.
Catnip (Nepeta cataria) Perennial usda zone 4-9. I'm including this as a fragrant plant because I understand some people like the smell of this. I don't. This is the plant of cat intoxication fame. Curiously, not all cats like it. We had a cat named Lester who didn't care for it. Our other cat Elmer vigorously defended her stash. She would eat and roll in it, be incredibly wild, and then sleep for the rest of the day. It has an interesting history. It has been used to relieve colic in babies and the root of it was brewed and served to executioners by their wives before a job. It was said to put them in the proper mood for hanging or beheading and one can only wonder what the proper mood might have been. It grows to 3 feet in fertile, well drained soil. It needs full sun, but will do ok in part sun. Will seed and spread.
Purple coneflower ( Echinacea purpurea) Perennial. Zones 2-10. This is a very pretty medicinal plant. It's purple/pink blooms appear in July-August. It needs full sun, well drained soil, and water when dry. It will grow in part sun but will be spindly and have a tendancy to flop over and have fewer blooms. It grows to 3', sometimes 4' in good soil or with added organic matter. Will bloom the second year from seed. This is the most widely used Echinacea for boosting the immune system. All parts of the plant are considered medicinal. I have read that people who have over active immune system disorders should not take this as well as it should not be taken more than 6 weeks at a time. Will multiply by clumping and seed (if the goldfinches don't eat all the seed). Attractive to butterflies, has a pleasant scent, and will attract goldfinches to your garden.
Lavender (Lavendula angustifolia). Perennial, Zones 5-8. It is said that the scent of lavender is the most popular scent in the world. It is one of the few scents that the majority of men like and it is found in a lot of shaving and soap products. As far as growing it goes, it helps to remember it is a Mediterranean plant. Hot dry summers, cool but not too wet winters. In fact, dry is a key to growing this plant. It must be in well drained soil. If you can not provide that, it needs to be grown in a pot. Another requirement for winter hardiness is that it be protected from cold wind. This sounds difficult if you are from some place where winter is on the vicious side, but micro-climates can be found or created. I had a lot of success growing it on the east side of the house, next to the foundation, and it was under the peak of the roof so no rain run off. Snow piled on it also helps. Still expect some winter kill on the branches, but the plant usually survives. The last requirement for growing lavender successfully is to make sure the soil is alkaline. Add lime. Planting it next to the house foundation if it is cement also helps. Lime leaches from the foundation. OK, now that I've mentioned all that, on to the plant itself. It prefers full sun, well drained soil, water when dry, but it can stand some drought. Clip out the winter killed branches in the spring after it has started to leaf out so you know which ones are dead and thoroughly enjoy the flowers when they come. Pick when a few flowers have opened on the stalk, tie into bunches and hang in an airy warm place out of the sun to dry. Will grow to 3' and tend to sprawl (from the weight of snow). Becomes woody with age.
Hyssop (Hyssopus officinalis). Perennial. Zone 3-9. This is a very decorative small shrub. It grows to 4' if very happy in full sun, well drained fertile soil, and water when dry. Mostly it stays around 3' but can be pruned to keep it smaller. It can tolerate some dryness, but grows better if it gets water. It has an aromatic scent that is pleasant though somewhat medicinal. A customer of mine who suffered from asthma told me she made tea from the leaves and used it to help her ailment to the point where she didn't need to use her inhaler as much. Very dark leaves and dark blue flowers make this an attractive mid-position border plant. This is the hyssop mentioned in the Bible.
Soapwort (Saponaria officinalis). Perennial. Zones 3-9. This plant was introduced by early settlers to provide a source of soap to wash cloth, particularly wool. You can almost always find it growing by creeks especially those that have had woolen mills on them. When the whole plant and the roots are soaked in water and then the water agitated, a sudsy solution is made that can be used to wash skin, hair, or fine washables. Don't get it in the eyes....it burns like a fury.I've used it on my hair and followed it with a diluted vinegar rinse. My hair shone and was very soft. It's past uses included taking it internally, but that is not recommended any longer as the saponins can cause quite the digestive upset. The plant itself grows to be about 3' tall. It prefers full sun and moist soil. I've seen it growing happily by the side of the road so it can tolerate poor soil. It spreads by root and seed. The flowers are light pink, sometimes doubled, and are evening fragrant. The scent is that of cloves. It blooms in mid-summer, around July.
Sweet Annie (Artemesia annua). Annual. This is another one with great scent which is very strong. It can grow quite tall in full sun, good soil, and adequate water. It can profusely seed itself. It is light green and ferny looking so makes a statement at the back of the border because it can grow to 5' and the light green foliage provides an interesting contrast to the usual darker green foliage types. It has tiny yellow flowers. The entire plant is fragrant. It is good to make scented herbal wreaths and swags with this because the scent stays well when dried. Any time you want to smell it's sweetness, all you need to do is brush against it. Medicinally it is being investigated for treating drug resistant strains of malaria.
Valerian (Valerian officinalis) Perennial. This is the garden valerian that has sweetly scented flowers....and the God awfullest smelling roots. It grows to 4', prefers full sun but will grow in part sun, likes lots of moisture and will grow in damp areas of the garden. It has white flowers and blooms in early to mid summer. It will seed itself and multiply by it's crown. It's roots are used medicinally...if you can stand the smell, something like rotten gym socks that have stepped in dog doo. It is said to promote calming of the nerves, anxiety, and to allow one to fall asleep. A former specific name of this plant was V. phu (Pronounce p-u, I imagine because of the smell). The roots are dug in the 2nd or preferably the third year, early in the spring just when the frost has left the ground. Zones 3-9.
Sweet Violet (Viola odorata). Perennial. Zone 6-9. Full sun to shade. This precious violet comes in blue, pink, and a sort of mauve which I think is a cross between the other two. It grows to 4" tall and bears remarkable flowers. Not only are they pretty, they are intensely fragrant. I have seen lawns dotted with these violets that you can smell yards away. They are listed as perennial in zones 6-9, but they routinely come through our Zone 5 winters. They spread endlessly, much to my delight. They will send out offsets like strawberries do and they reproduce by seed. They like moist soil, but not wet conditions. This is one of the traditional perfume scents and the flowers are candied to be an edible decoration on cakes.