Combine yeast and warm water in a small bowl, stir gently to dissolve yeast. Let stand until yeast begins to foam, about 5-10 minutes.
Combine yeast mixture with buttermilk and oil in a large mixing bowl. Stir in whole wheat flour and 2 1/2 cups of white flour, honey, potato flour and salt. Add the additional white flour if needed, or more water if dough is too dry. The dough should come away from the bowl and be firm enough to handle with your hands.
Turn dough onto a lightly floured surface, knead 10-15 minutes or until dough is smooth and soft. Place the dough in a lightly greased bowl and cover with a damp cloth. Let the dough rise until doubled, about 1 hour.
When risen, shape dough into 18 rolls and place on a parchment lined baking tray. Let rise again until doubled, about 1 hour. When risen, carefully score the rolls in an "x" shape with a sharp knife. Sprinkle lightly with potato flour and bake at 375°F for 15-18 minutes, rotating halfway through the baking time, until the rolls are lightly browned.