I was walking by my swamp milkweed plants the other day and noticed them covered in something yellow. Upon closer inspection I saw the yellow "dots" moving, and realized that I had milkweed aphids.
Rather than use chemicals to get rid of them, I prefer to spray them with the water hose. It knocks them off the plant, but keeps any monarch eggs safe. I have also used a cotton swab to remove them from the plants. If the aphids keep returning you can also take a cotton swab dipped in rubbing alcohol to them. It is tedious but kills them on contact.