Sprinkle the olives on a serving plate. Crumble the goat cheese over the olives and set aside.
In a bowl, whisk together the remaining ingredients to make the marinade. Pour the marinade over the the goat cheese and olives and let marinate at room temperature for at least an hour to allow the flavors to blend. Sprinkle pepper on top. Serve with baguette and/or crackers.