In a medium bowl, beat together the cream cheese and brown sugar until combined. Mix in mashed sweet potatoes, vanilla, cinnamon, and nutmeg until creamy. If dip is not as smooth as desired, use an immersion blender to reach the desired consistency.
For the pecan topping:
Mix all ingredients together and spread on top of the dip. (there may be some topping leftover depending on thick you want the topping to be).
Bake for 15-20 minutes or until dip is heated through. Serve with graham crackers or ginger snaps.
Notes
I boiled my sweet potatoes for about 12 minutes (until they were tender) and then mashed. You may bake or use your preferred method for preparing your sweet potatoes for the dip. If topping with marshmallows, carefully watch the dip while warming as marshmallows can go from toasted to burned very quickly.