This surely appears to be ivory. The stock has a similar patina to ones that can be seen at the Rijksmuseum in Amsterdam: Wapens en scheepsmodellen - Kunstwerken - Rijksstudio - Rijksmuseum
I still have one of those plastic pistols. At one time, I had the entire set of the models made by that plastic kit company. The pistol below was made by me using the plastic "Dueling Pistol" kit from them as a guide. The only difference is the plastic kit was a flintlock and my gun shoots .40 caliber balls.When I was a young kid I had a plastic model of that exact pistol. Was hard as hell to line up both sides of the stock while the model glue set.